[2023-10-08 07:50:10,972][52710] Saving configuration to ./train_atari/atari_asterix_APPO/config.json... [2023-10-08 07:50:11,289][52710] Rollout worker 0 uses device cpu [2023-10-08 07:50:11,290][52710] Rollout worker 1 uses device cpu [2023-10-08 07:50:11,291][52710] Rollout worker 2 uses device cpu [2023-10-08 07:50:11,291][52710] Rollout worker 3 uses device cpu [2023-10-08 07:50:11,292][52710] Rollout worker 4 uses device cpu [2023-10-08 07:50:11,293][52710] Rollout worker 5 uses device cpu [2023-10-08 07:50:11,293][52710] Rollout worker 6 uses device cpu [2023-10-08 07:50:11,293][52710] Rollout worker 7 uses device cpu [2023-10-08 07:50:11,294][52710] Rollout worker 8 uses device cpu [2023-10-08 07:50:11,294][52710] Rollout worker 9 uses device cpu [2023-10-08 07:50:11,295][52710] Rollout worker 10 uses device cpu [2023-10-08 07:50:11,295][52710] Rollout worker 11 uses device cpu [2023-10-08 07:50:11,295][52710] Rollout worker 12 uses device cpu [2023-10-08 07:50:11,296][52710] Rollout worker 13 uses device cpu [2023-10-08 07:50:11,296][52710] Rollout worker 14 uses device cpu [2023-10-08 07:50:11,296][52710] Rollout worker 15 uses device cpu [2023-10-08 07:50:11,582][52710] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 07:50:11,583][52710] InferenceWorker_p0-w0: min num requests: 2 [2023-10-08 07:50:11,587][52710] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-08 07:50:11,587][52710] InferenceWorker_p1-w0: min num requests: 2 [2023-10-08 07:50:11,633][52710] Starting all processes... [2023-10-08 07:50:11,634][52710] Starting process learner_proc0 [2023-10-08 07:50:13,304][52710] Starting process learner_proc1 [2023-10-08 07:50:13,307][53500] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 07:50:13,307][53500] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-10-08 07:50:13,326][53500] Num visible devices: 1 [2023-10-08 07:50:13,343][53500] Setting fixed seed 1234 [2023-10-08 07:50:13,344][53500] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 07:50:13,344][53500] Initializing actor-critic model on device cuda:0 [2023-10-08 07:50:13,344][53500] RunningMeanStd input shape: (4, 84, 84) [2023-10-08 07:50:13,345][53500] RunningMeanStd input shape: (1,) [2023-10-08 07:50:13,356][53500] ConvEncoder: input_channels=4 [2023-10-08 07:50:13,536][53500] Conv encoder output size: 512 [2023-10-08 07:50:13,538][53500] Created Actor Critic model with architecture: [2023-10-08 07:50:13,538][53500] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=9, bias=True) ) ) [2023-10-08 07:50:14,109][53500] Using optimizer [2023-10-08 07:50:14,109][53500] No checkpoints found [2023-10-08 07:50:14,110][53500] Did not load from checkpoint, starting from scratch! [2023-10-08 07:50:14,110][53500] Initialized policy 0 weights for model version 0 [2023-10-08 07:50:14,111][53500] LearnerWorker_p0 finished initialization! [2023-10-08 07:50:14,112][53500] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 07:50:15,093][52710] Starting all processes... [2023-10-08 07:50:15,097][53594] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-08 07:50:15,097][53594] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 [2023-10-08 07:50:15,103][52710] Starting process inference_proc0-0 [2023-10-08 07:50:15,103][52710] Starting process inference_proc1-0 [2023-10-08 07:50:15,115][53594] Num visible devices: 1 [2023-10-08 07:50:15,104][52710] Starting process rollout_proc0 [2023-10-08 07:50:15,104][52710] Starting process rollout_proc1 [2023-10-08 07:50:15,104][52710] Starting process rollout_proc2 [2023-10-08 07:50:15,137][53594] Setting fixed seed 1234 [2023-10-08 07:50:15,139][53594] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-08 07:50:15,139][53594] Initializing actor-critic model on device cuda:0 [2023-10-08 07:50:15,139][53594] RunningMeanStd input shape: (4, 84, 84) [2023-10-08 07:50:15,140][53594] RunningMeanStd input shape: (1,) [2023-10-08 07:50:15,105][52710] Starting process rollout_proc3 [2023-10-08 07:50:15,105][52710] Starting process rollout_proc4 [2023-10-08 07:50:15,107][52710] Starting process rollout_proc5 [2023-10-08 07:50:15,113][52710] Starting process rollout_proc6 [2023-10-08 07:50:15,128][52710] Starting process rollout_proc7 [2023-10-08 07:50:15,152][53594] ConvEncoder: input_channels=4 [2023-10-08 07:50:15,128][52710] Starting process rollout_proc8 [2023-10-08 07:50:15,128][52710] Starting process rollout_proc9 [2023-10-08 07:50:15,129][52710] Starting process rollout_proc10 [2023-10-08 07:50:15,129][52710] Starting process rollout_proc11 [2023-10-08 07:50:15,129][52710] Starting process rollout_proc12 [2023-10-08 07:50:15,129][52710] Starting process rollout_proc13 [2023-10-08 07:50:15,568][53594] Conv encoder output size: 512 [2023-10-08 07:50:15,570][53594] Created Actor Critic model with architecture: [2023-10-08 07:50:15,571][53594] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): MultiInputEncoder( (encoders): ModuleDict( (obs): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ReLU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ReLU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ReLU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ReLU) ) ) ) ) ) (core): ModelCoreIdentity() (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=9, bias=True) ) ) [2023-10-08 07:50:16,261][53594] Using optimizer [2023-10-08 07:50:16,262][53594] No checkpoints found [2023-10-08 07:50:16,262][53594] Did not load from checkpoint, starting from scratch! [2023-10-08 07:50:16,262][53594] Initialized policy 1 weights for model version 0 [2023-10-08 07:50:16,263][53594] LearnerWorker_p1 finished initialization! [2023-10-08 07:50:16,264][53594] Using GPUs [0] for process 1 (actually maps to GPUs [1]) [2023-10-08 07:50:17,264][52710] Starting process rollout_proc14 [2023-10-08 07:50:17,268][52710] Starting process rollout_proc15 [2023-10-08 07:50:17,271][53897] Worker 9 uses CPU cores [18, 19] [2023-10-08 07:50:17,271][53889] Worker 1 uses CPU cores [2, 3] [2023-10-08 07:50:17,308][53888] Worker 2 uses CPU cores [4, 5] [2023-10-08 07:50:17,373][53886] Worker 0 uses CPU cores [0, 1] [2023-10-08 07:50:17,496][53899] Worker 10 uses CPU cores [20, 21] [2023-10-08 07:50:17,500][53896] Worker 8 uses CPU cores [16, 17] [2023-10-08 07:50:17,572][53893] Worker 5 uses CPU cores [10, 11] [2023-10-08 07:50:17,605][53898] Worker 11 uses CPU cores [22, 23] [2023-10-08 07:50:17,651][53895] Worker 7 uses CPU cores [14, 15] [2023-10-08 07:50:17,667][53885] Using GPUs [1] for process 1 (actually maps to GPUs [1]) [2023-10-08 07:50:17,667][53885] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 [2023-10-08 07:50:17,672][53852] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-10-08 07:50:17,672][53852] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-10-08 07:50:17,686][53885] Num visible devices: 1 [2023-10-08 07:50:17,690][53852] Num visible devices: 1 [2023-10-08 07:50:17,694][53900] Worker 12 uses CPU cores [24, 25] [2023-10-08 07:50:17,736][53891] Worker 4 uses CPU cores [8, 9] [2023-10-08 07:50:17,909][53890] Worker 3 uses CPU cores [6, 7] [2023-10-08 07:50:17,953][53901] Worker 13 uses CPU cores [26, 27] [2023-10-08 07:50:18,108][53894] Worker 6 uses CPU cores [12, 13] [2023-10-08 07:50:18,361][53852] RunningMeanStd input shape: (4, 84, 84) [2023-10-08 07:50:18,362][53852] RunningMeanStd input shape: (1,) [2023-10-08 07:50:18,373][53852] ConvEncoder: input_channels=4 [2023-10-08 07:50:18,400][53885] RunningMeanStd input shape: (4, 84, 84) [2023-10-08 07:50:18,401][53885] RunningMeanStd input shape: (1,) [2023-10-08 07:50:18,412][53885] ConvEncoder: input_channels=4 [2023-10-08 07:50:18,476][53852] Conv encoder output size: 512 [2023-10-08 07:50:18,512][53885] Conv encoder output size: 512 [2023-10-08 07:50:19,139][54536] Worker 14 uses CPU cores [28, 29] [2023-10-08 07:50:19,140][52710] Inference worker 0-0 is ready! [2023-10-08 07:50:19,141][52710] Inference worker 1-0 is ready! [2023-10-08 07:50:19,141][54537] Worker 15 uses CPU cores [30, 31] [2023-10-08 07:50:19,141][52710] All inference workers are ready! Signal rollout workers to start! [2023-10-08 07:50:19,143][53900] EnvRunner 12-0 uses policy 0 [2023-10-08 07:50:19,143][53888] EnvRunner 2-0 uses policy 0 [2023-10-08 07:50:19,143][53890] EnvRunner 3-0 uses policy 1 [2023-10-08 07:50:19,143][53889] EnvRunner 1-0 uses policy 1 [2023-10-08 07:50:19,143][53898] EnvRunner 11-0 uses policy 1 [2023-10-08 07:50:19,143][53896] EnvRunner 8-0 uses policy 0 [2023-10-08 07:50:19,143][53899] EnvRunner 10-0 uses policy 0 [2023-10-08 07:50:19,143][53901] EnvRunner 13-0 uses policy 1 [2023-10-08 07:50:19,143][53886] EnvRunner 0-0 uses policy 0 [2023-10-08 07:50:19,144][53894] EnvRunner 6-0 uses policy 0 [2023-10-08 07:50:19,144][53893] EnvRunner 5-0 uses policy 1 [2023-10-08 07:50:19,144][53895] EnvRunner 7-0 uses policy 1 [2023-10-08 07:50:19,144][53897] EnvRunner 9-0 uses policy 1 [2023-10-08 07:50:19,144][53891] EnvRunner 4-0 uses policy 0 [2023-10-08 07:50:19,144][52710] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-08 07:50:19,325][54537] EnvRunner 15-0 uses policy 1 [2023-10-08 07:50:19,352][54536] EnvRunner 14-0 uses policy 0 [2023-10-08 07:50:21,569][52710] Heartbeat connected on Batcher_0 [2023-10-08 07:50:21,572][52710] Heartbeat connected on LearnerWorker_p0 [2023-10-08 07:50:21,576][52710] Heartbeat connected on Batcher_1 [2023-10-08 07:50:21,578][52710] Heartbeat connected on LearnerWorker_p1 [2023-10-08 07:50:21,585][52710] Heartbeat connected on InferenceWorker_p0-w0 [2023-10-08 07:50:21,592][52710] Heartbeat connected on InferenceWorker_p1-w0 [2023-10-08 07:50:21,592][52710] Heartbeat connected on RolloutWorker_w0 [2023-10-08 07:50:21,594][52710] Heartbeat connected on RolloutWorker_w1 [2023-10-08 07:50:21,599][52710] Heartbeat connected on RolloutWorker_w2 [2023-10-08 07:50:21,602][52710] Heartbeat connected on RolloutWorker_w3 [2023-10-08 07:50:21,604][52710] Heartbeat connected on RolloutWorker_w4 [2023-10-08 07:50:21,604][52710] Heartbeat connected on RolloutWorker_w5 [2023-10-08 07:50:21,607][52710] Heartbeat connected on RolloutWorker_w6 [2023-10-08 07:50:21,610][52710] Heartbeat connected on RolloutWorker_w7 [2023-10-08 07:50:21,615][52710] Heartbeat connected on RolloutWorker_w8 [2023-10-08 07:50:21,615][52710] Heartbeat connected on RolloutWorker_w9 [2023-10-08 07:50:21,620][52710] Heartbeat connected on RolloutWorker_w10 [2023-10-08 07:50:21,621][52710] Heartbeat connected on RolloutWorker_w11 [2023-10-08 07:50:21,624][52710] Heartbeat connected on RolloutWorker_w12 [2023-10-08 07:50:21,626][52710] Heartbeat connected on RolloutWorker_w13 [2023-10-08 07:50:21,632][52710] Heartbeat connected on RolloutWorker_w14 [2023-10-08 07:50:21,634][52710] Heartbeat connected on RolloutWorker_w15 [2023-10-08 07:50:22,015][52710] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 656.8, 1: 628.3. Samples: 3690. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-08 07:50:22,015][52710] Avg episode reward: [(0, '2.071'), (1, '1.800')] [2023-10-08 07:50:27,015][52710] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 1023.2, 1: 1024.5. Samples: 16118. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-10-08 07:50:27,016][52710] Avg episode reward: [(0, '1.620'), (1, '1.710')] [2023-10-08 07:50:29,046][53885] Updated weights for policy 1, policy_version 10 (0.0008) [2023-10-08 07:50:29,068][53852] Updated weights for policy 0, policy_version 10 (0.0008) [2023-10-08 07:50:29,403][53885] Updated weights for policy 1, policy_version 20 (0.0007) [2023-10-08 07:50:29,439][53852] Updated weights for policy 0, policy_version 20 (0.0010) [2023-10-08 07:50:29,764][53885] Updated weights for policy 1, policy_version 30 (0.0007) [2023-10-08 07:50:29,807][53852] Updated weights for policy 0, policy_version 30 (0.0008) [2023-10-08 07:50:32,007][53885] Updated weights for policy 1, policy_version 40 (0.0007) [2023-10-08 07:50:32,015][52710] Fps is (10 sec: 6553.6, 60 sec: 5091.6, 300 sec: 5091.6). Total num frames: 65536. Throughput: 0: 1288.3, 1: 1295.0. Samples: 33250. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 07:50:32,015][52710] Avg episode reward: [(0, '1.250'), (1, '1.480')] [2023-10-08 07:50:32,067][53852] Updated weights for policy 0, policy_version 40 (0.0007) [2023-10-08 07:50:32,384][53885] Updated weights for policy 1, policy_version 50 (0.0009) [2023-10-08 07:50:32,433][53852] Updated weights for policy 0, policy_version 50 (0.0007) [2023-10-08 07:50:32,744][53885] Updated weights for policy 1, policy_version 60 (0.0007) [2023-10-08 07:50:32,810][53852] Updated weights for policy 0, policy_version 60 (0.0007) [2023-10-08 07:50:35,925][53885] Updated weights for policy 1, policy_version 70 (0.0009) [2023-10-08 07:50:36,167][53852] Updated weights for policy 0, policy_version 70 (0.0008) [2023-10-08 07:50:36,290][53885] Updated weights for policy 1, policy_version 80 (0.0007) [2023-10-08 07:50:36,534][53852] Updated weights for policy 0, policy_version 80 (0.0007) [2023-10-08 07:50:36,650][53885] Updated weights for policy 1, policy_version 90 (0.0007) [2023-10-08 07:50:36,904][53852] Updated weights for policy 0, policy_version 90 (0.0008) [2023-10-08 07:50:37,015][52710] Fps is (10 sec: 16384.1, 60 sec: 9167.7, 300 sec: 9167.7). Total num frames: 163840. Throughput: 0: 1515.9, 1: 1498.2. Samples: 53866. Policy #0 lag: (min: 33.0, avg: 33.0, max: 33.0) [2023-10-08 07:50:37,015][52710] Avg episode reward: [(0, '1.580'), (1, '1.560')] [2023-10-08 07:50:40,054][53885] Updated weights for policy 1, policy_version 100 (0.0008) [2023-10-08 07:50:40,374][53852] Updated weights for policy 0, policy_version 100 (0.0007) [2023-10-08 07:50:40,427][53885] Updated weights for policy 1, policy_version 110 (0.0008) [2023-10-08 07:50:40,737][53852] Updated weights for policy 0, policy_version 110 (0.0009) [2023-10-08 07:50:40,791][53885] Updated weights for policy 1, policy_version 120 (0.0009) [2023-10-08 07:50:41,103][53852] Updated weights for policy 0, policy_version 120 (0.0008) [2023-10-08 07:50:42,015][52710] Fps is (10 sec: 19660.7, 60 sec: 11461.7, 300 sec: 11461.7). Total num frames: 262144. Throughput: 0: 1426.5, 1: 1427.7. Samples: 65280. Policy #0 lag: (min: 22.0, avg: 30.0, max: 54.0) [2023-10-08 07:50:42,015][52710] Avg episode reward: [(0, '1.300'), (1, '1.580')] [2023-10-08 07:50:42,016][53500] Saving new best policy, reward=1.300! [2023-10-08 07:50:42,016][53594] Saving new best policy, reward=1.580! [2023-10-08 07:50:44,591][53885] Updated weights for policy 1, policy_version 130 (0.0007) [2023-10-08 07:50:44,887][53852] Updated weights for policy 0, policy_version 130 (0.0008) [2023-10-08 07:50:44,950][53885] Updated weights for policy 1, policy_version 140 (0.0009) [2023-10-08 07:50:45,256][53852] Updated weights for policy 0, policy_version 140 (0.0008) [2023-10-08 07:50:45,314][53885] Updated weights for policy 1, policy_version 150 (0.0008) [2023-10-08 07:50:45,620][53852] Updated weights for policy 0, policy_version 150 (0.0008) [2023-10-08 07:50:45,682][53885] Updated weights for policy 1, policy_version 160 (0.0008) [2023-10-08 07:50:45,978][53852] Updated weights for policy 0, policy_version 160 (0.0009) [2023-10-08 07:50:47,015][52710] Fps is (10 sec: 16383.8, 60 sec: 11756.8, 300 sec: 11756.8). Total num frames: 327680. Throughput: 0: 1541.4, 1: 1535.9. Samples: 85768. Policy #0 lag: (min: 31.0, avg: 34.7, max: 63.0) [2023-10-08 07:50:47,016][52710] Avg episode reward: [(0, '1.290'), (1, '1.310')] [2023-10-08 07:50:49,449][53885] Updated weights for policy 1, policy_version 170 (0.0007) [2023-10-08 07:50:49,786][53852] Updated weights for policy 0, policy_version 170 (0.0008) [2023-10-08 07:50:49,813][53885] Updated weights for policy 1, policy_version 180 (0.0007) [2023-10-08 07:50:50,159][53852] Updated weights for policy 0, policy_version 180 (0.0009) [2023-10-08 07:50:50,185][53885] Updated weights for policy 1, policy_version 190 (0.0007) [2023-10-08 07:50:50,532][53852] Updated weights for policy 0, policy_version 190 (0.0010) [2023-10-08 07:50:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 11962.2, 300 sec: 11962.2). Total num frames: 393216. Throughput: 0: 1631.9, 1: 1640.6. Samples: 107574. Policy #0 lag: (min: 31.0, avg: 43.7, max: 63.0) [2023-10-08 07:50:52,016][52710] Avg episode reward: [(0, '1.310'), (1, '1.470')] [2023-10-08 07:50:52,023][53500] Saving new best policy, reward=1.310! [2023-10-08 07:50:54,030][53885] Updated weights for policy 1, policy_version 200 (0.0007) [2023-10-08 07:50:54,169][53852] Updated weights for policy 0, policy_version 200 (0.0008) [2023-10-08 07:50:54,395][53885] Updated weights for policy 1, policy_version 210 (0.0007) [2023-10-08 07:50:54,534][53852] Updated weights for policy 0, policy_version 210 (0.0008) [2023-10-08 07:50:54,756][53885] Updated weights for policy 1, policy_version 220 (0.0008) [2023-10-08 07:50:54,900][53852] Updated weights for policy 0, policy_version 220 (0.0007) [2023-10-08 07:50:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 12113.4, 300 sec: 12113.4). Total num frames: 458752. Throughput: 0: 1564.8, 1: 1566.3. Samples: 118582. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 07:50:57,016][52710] Avg episode reward: [(0, '1.680'), (1, '1.760')] [2023-10-08 07:50:57,018][53594] Saving new best policy, reward=1.760! [2023-10-08 07:50:57,018][53500] Saving new best policy, reward=1.680! [2023-10-08 07:50:58,502][53885] Updated weights for policy 1, policy_version 230 (0.0008) [2023-10-08 07:50:58,611][53852] Updated weights for policy 0, policy_version 230 (0.0007) [2023-10-08 07:50:58,876][53885] Updated weights for policy 1, policy_version 240 (0.0008) [2023-10-08 07:50:58,982][53852] Updated weights for policy 0, policy_version 240 (0.0008) [2023-10-08 07:50:59,232][53885] Updated weights for policy 1, policy_version 250 (0.0009) [2023-10-08 07:50:59,356][53852] Updated weights for policy 0, policy_version 250 (0.0008) [2023-10-08 07:51:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 12229.3, 300 sec: 12229.3). Total num frames: 524288. Throughput: 0: 1632.4, 1: 1634.9. Samples: 140072. Policy #0 lag: (min: 4.0, avg: 11.2, max: 36.0) [2023-10-08 07:51:02,016][52710] Avg episode reward: [(0, '1.510'), (1, '1.810')] [2023-10-08 07:51:02,017][53594] Saving new best policy, reward=1.810! [2023-10-08 07:51:02,959][53885] Updated weights for policy 1, policy_version 260 (0.0007) [2023-10-08 07:51:03,002][53852] Updated weights for policy 0, policy_version 260 (0.0008) [2023-10-08 07:51:03,316][53885] Updated weights for policy 1, policy_version 270 (0.0008) [2023-10-08 07:51:03,375][53852] Updated weights for policy 0, policy_version 270 (0.0008) [2023-10-08 07:51:03,686][53885] Updated weights for policy 1, policy_version 280 (0.0007) [2023-10-08 07:51:03,750][53852] Updated weights for policy 0, policy_version 280 (0.0007) [2023-10-08 07:51:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 12321.0, 300 sec: 12321.0). Total num frames: 589824. Throughput: 0: 1763.6, 1: 1767.6. Samples: 162590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:51:07,016][52710] Avg episode reward: [(0, '2.070'), (1, '1.460')] [2023-10-08 07:51:07,024][53500] Saving new best policy, reward=2.070! [2023-10-08 07:51:07,560][53852] Updated weights for policy 0, policy_version 290 (0.0007) [2023-10-08 07:51:07,624][53885] Updated weights for policy 1, policy_version 290 (0.0009) [2023-10-08 07:51:07,929][53852] Updated weights for policy 0, policy_version 300 (0.0009) [2023-10-08 07:51:07,976][53885] Updated weights for policy 1, policy_version 300 (0.0007) [2023-10-08 07:51:08,297][53852] Updated weights for policy 0, policy_version 310 (0.0008) [2023-10-08 07:51:08,340][53885] Updated weights for policy 1, policy_version 310 (0.0008) [2023-10-08 07:51:08,678][53852] Updated weights for policy 0, policy_version 320 (0.0009) [2023-10-08 07:51:08,707][53885] Updated weights for policy 1, policy_version 320 (0.0008) [2023-10-08 07:51:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 12395.3, 300 sec: 12395.3). Total num frames: 655360. Throughput: 0: 1736.5, 1: 1737.1. Samples: 172428. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 07:51:12,016][52710] Avg episode reward: [(0, '1.460'), (1, '1.590')] [2023-10-08 07:51:12,366][53852] Updated weights for policy 0, policy_version 330 (0.0008) [2023-10-08 07:51:12,654][53885] Updated weights for policy 1, policy_version 330 (0.0008) [2023-10-08 07:51:12,740][53852] Updated weights for policy 0, policy_version 340 (0.0008) [2023-10-08 07:51:13,012][53885] Updated weights for policy 1, policy_version 340 (0.0007) [2023-10-08 07:51:13,099][53852] Updated weights for policy 0, policy_version 350 (0.0008) [2023-10-08 07:51:13,375][53885] Updated weights for policy 1, policy_version 350 (0.0009) [2023-10-08 07:51:16,718][53852] Updated weights for policy 0, policy_version 360 (0.0009) [2023-10-08 07:51:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 12456.9, 300 sec: 12456.9). Total num frames: 720896. Throughput: 0: 1798.6, 1: 1784.8. Samples: 194502. Policy #0 lag: (min: 26.0, avg: 28.6, max: 58.0) [2023-10-08 07:51:17,016][52710] Avg episode reward: [(0, '1.440'), (1, '1.600')] [2023-10-08 07:51:17,077][53852] Updated weights for policy 0, policy_version 370 (0.0007) [2023-10-08 07:51:17,123][53885] Updated weights for policy 1, policy_version 360 (0.0008) [2023-10-08 07:51:17,441][53852] Updated weights for policy 0, policy_version 380 (0.0007) [2023-10-08 07:51:17,494][53885] Updated weights for policy 1, policy_version 370 (0.0008) [2023-10-08 07:51:17,860][53885] Updated weights for policy 1, policy_version 380 (0.0008) [2023-10-08 07:51:21,247][53852] Updated weights for policy 0, policy_version 390 (0.0008) [2023-10-08 07:51:21,510][53885] Updated weights for policy 1, policy_version 390 (0.0009) [2023-10-08 07:51:21,619][53852] Updated weights for policy 0, policy_version 400 (0.0010) [2023-10-08 07:51:21,876][53885] Updated weights for policy 1, policy_version 400 (0.0007) [2023-10-08 07:51:21,998][53852] Updated weights for policy 0, policy_version 410 (0.0007) [2023-10-08 07:51:22,015][52710] Fps is (10 sec: 13107.4, 60 sec: 13107.2, 300 sec: 12508.6). Total num frames: 786432. Throughput: 0: 1797.6, 1: 1804.8. Samples: 215972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:51:22,016][52710] Avg episode reward: [(0, '1.460'), (1, '1.480')] [2023-10-08 07:51:22,237][53885] Updated weights for policy 1, policy_version 410 (0.0007) [2023-10-08 07:51:25,780][53852] Updated weights for policy 0, policy_version 420 (0.0008) [2023-10-08 07:51:26,145][53885] Updated weights for policy 1, policy_version 420 (0.0008) [2023-10-08 07:51:26,158][53852] Updated weights for policy 0, policy_version 430 (0.0009) [2023-10-08 07:51:26,517][53885] Updated weights for policy 1, policy_version 430 (0.0009) [2023-10-08 07:51:26,532][53852] Updated weights for policy 0, policy_version 440 (0.0009) [2023-10-08 07:51:26,883][53885] Updated weights for policy 1, policy_version 440 (0.0009) [2023-10-08 07:51:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 13035.5). Total num frames: 884736. Throughput: 0: 1798.2, 1: 1789.3. Samples: 226718. Policy #0 lag: (min: 31.0, avg: 32.2, max: 55.0) [2023-10-08 07:51:27,016][52710] Avg episode reward: [(0, '1.570'), (1, '1.900')] [2023-10-08 07:51:27,181][53594] Saving new best policy, reward=1.900! [2023-10-08 07:51:30,396][53852] Updated weights for policy 0, policy_version 450 (0.0008) [2023-10-08 07:51:30,627][53885] Updated weights for policy 1, policy_version 450 (0.0007) [2023-10-08 07:51:30,758][53852] Updated weights for policy 0, policy_version 460 (0.0008) [2023-10-08 07:51:30,990][53885] Updated weights for policy 1, policy_version 460 (0.0009) [2023-10-08 07:51:31,126][53852] Updated weights for policy 0, policy_version 470 (0.0008) [2023-10-08 07:51:31,360][53885] Updated weights for policy 1, policy_version 470 (0.0008) [2023-10-08 07:51:31,492][53852] Updated weights for policy 0, policy_version 480 (0.0008) [2023-10-08 07:51:31,715][53885] Updated weights for policy 1, policy_version 480 (0.0009) [2023-10-08 07:51:32,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 13490.1). Total num frames: 983040. Throughput: 0: 1802.2, 1: 1805.9. Samples: 248134. Policy #0 lag: (min: 1.0, avg: 9.9, max: 33.0) [2023-10-08 07:51:32,016][52710] Avg episode reward: [(0, '1.820'), (1, '1.520')] [2023-10-08 07:51:35,158][53852] Updated weights for policy 0, policy_version 490 (0.0008) [2023-10-08 07:51:35,406][53885] Updated weights for policy 1, policy_version 490 (0.0008) [2023-10-08 07:51:35,532][53852] Updated weights for policy 0, policy_version 500 (0.0008) [2023-10-08 07:51:35,766][53885] Updated weights for policy 1, policy_version 500 (0.0007) [2023-10-08 07:51:35,899][53852] Updated weights for policy 0, policy_version 510 (0.0008) [2023-10-08 07:51:36,141][53885] Updated weights for policy 1, policy_version 510 (0.0009) [2023-10-08 07:51:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 13465.5). Total num frames: 1048576. Throughput: 0: 1790.4, 1: 1775.3. Samples: 268030. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 07:51:37,016][52710] Avg episode reward: [(0, '1.680'), (1, '1.430')] [2023-10-08 07:51:39,715][53852] Updated weights for policy 0, policy_version 520 (0.0008) [2023-10-08 07:51:39,941][53885] Updated weights for policy 1, policy_version 520 (0.0008) [2023-10-08 07:51:40,075][53852] Updated weights for policy 0, policy_version 530 (0.0007) [2023-10-08 07:51:40,303][53885] Updated weights for policy 1, policy_version 530 (0.0007) [2023-10-08 07:51:40,449][53852] Updated weights for policy 0, policy_version 540 (0.0008) [2023-10-08 07:51:40,666][53885] Updated weights for policy 1, policy_version 540 (0.0008) [2023-10-08 07:51:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13443.9). Total num frames: 1114112. Throughput: 0: 1800.3, 1: 1795.6. Samples: 280396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:51:42,016][52710] Avg episode reward: [(0, '1.930'), (1, '1.570')] [2023-10-08 07:51:44,211][53852] Updated weights for policy 0, policy_version 550 (0.0009) [2023-10-08 07:51:44,404][53885] Updated weights for policy 1, policy_version 550 (0.0008) [2023-10-08 07:51:44,576][53852] Updated weights for policy 0, policy_version 560 (0.0008) [2023-10-08 07:51:44,766][53885] Updated weights for policy 1, policy_version 560 (0.0007) [2023-10-08 07:51:44,954][53852] Updated weights for policy 0, policy_version 570 (0.0009) [2023-10-08 07:51:45,137][53885] Updated weights for policy 1, policy_version 570 (0.0007) [2023-10-08 07:51:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 13424.7). Total num frames: 1179648. Throughput: 0: 1783.7, 1: 1771.3. Samples: 300048. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 07:51:47,016][52710] Avg episode reward: [(0, '1.720'), (1, '1.800')] [2023-10-08 07:51:48,532][53852] Updated weights for policy 0, policy_version 580 (0.0008) [2023-10-08 07:51:48,902][53852] Updated weights for policy 0, policy_version 590 (0.0008) [2023-10-08 07:51:49,097][53885] Updated weights for policy 1, policy_version 580 (0.0009) [2023-10-08 07:51:49,269][53852] Updated weights for policy 0, policy_version 600 (0.0007) [2023-10-08 07:51:49,465][53885] Updated weights for policy 1, policy_version 590 (0.0008) [2023-10-08 07:51:49,825][53885] Updated weights for policy 1, policy_version 600 (0.0009) [2023-10-08 07:51:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13407.6). Total num frames: 1245184. Throughput: 0: 1785.7, 1: 1766.6. Samples: 322442. Policy #0 lag: (min: 17.0, avg: 21.4, max: 49.0) [2023-10-08 07:51:52,015][52710] Avg episode reward: [(0, '1.790'), (1, '1.790')] [2023-10-08 07:51:53,025][53852] Updated weights for policy 0, policy_version 610 (0.0008) [2023-10-08 07:51:53,411][53852] Updated weights for policy 0, policy_version 620 (0.0009) [2023-10-08 07:51:53,665][53885] Updated weights for policy 1, policy_version 610 (0.0009) [2023-10-08 07:51:53,786][53852] Updated weights for policy 0, policy_version 630 (0.0010) [2023-10-08 07:51:54,034][53885] Updated weights for policy 1, policy_version 620 (0.0008) [2023-10-08 07:51:54,147][53852] Updated weights for policy 0, policy_version 640 (0.0008) [2023-10-08 07:51:54,394][53885] Updated weights for policy 1, policy_version 630 (0.0008) [2023-10-08 07:51:54,761][53885] Updated weights for policy 1, policy_version 640 (0.0007) [2023-10-08 07:51:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 13392.3). Total num frames: 1310720. Throughput: 0: 1786.0, 1: 1773.1. Samples: 332586. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 07:51:57,015][52710] Avg episode reward: [(0, '1.870'), (1, '1.710')] [2023-10-08 07:51:57,987][53852] Updated weights for policy 0, policy_version 650 (0.0009) [2023-10-08 07:51:58,357][53852] Updated weights for policy 0, policy_version 660 (0.0008) [2023-10-08 07:51:58,426][53885] Updated weights for policy 1, policy_version 650 (0.0009) [2023-10-08 07:51:58,734][53852] Updated weights for policy 0, policy_version 670 (0.0008) [2023-10-08 07:51:58,798][53885] Updated weights for policy 1, policy_version 660 (0.0008) [2023-10-08 07:51:59,162][53885] Updated weights for policy 1, policy_version 670 (0.0010) [2023-10-08 07:52:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 13378.4). Total num frames: 1376256. Throughput: 0: 1778.8, 1: 1770.4. Samples: 354214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:52:02,016][52710] Avg episode reward: [(0, '1.860'), (1, '1.880')] [2023-10-08 07:52:02,490][53852] Updated weights for policy 0, policy_version 680 (0.0007) [2023-10-08 07:52:02,856][53852] Updated weights for policy 0, policy_version 690 (0.0007) [2023-10-08 07:52:02,925][53885] Updated weights for policy 1, policy_version 680 (0.0010) [2023-10-08 07:52:03,219][53852] Updated weights for policy 0, policy_version 700 (0.0007) [2023-10-08 07:52:03,300][53885] Updated weights for policy 1, policy_version 690 (0.0008) [2023-10-08 07:52:03,671][53885] Updated weights for policy 1, policy_version 700 (0.0010) [2023-10-08 07:52:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13365.8). Total num frames: 1441792. Throughput: 0: 1798.2, 1: 1772.3. Samples: 376646. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:52:07,016][52710] Avg episode reward: [(0, '2.060'), (1, '1.680')] [2023-10-08 07:52:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000000704_720896.pth... [2023-10-08 07:52:07,053][53852] Updated weights for policy 0, policy_version 710 (0.0008) [2023-10-08 07:52:07,429][53852] Updated weights for policy 0, policy_version 720 (0.0007) [2023-10-08 07:52:07,495][53885] Updated weights for policy 1, policy_version 710 (0.0010) [2023-10-08 07:52:07,804][53852] Updated weights for policy 0, policy_version 730 (0.0008) [2023-10-08 07:52:07,863][53885] Updated weights for policy 1, policy_version 720 (0.0008) [2023-10-08 07:52:08,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000000736_753664.pth... [2023-10-08 07:52:08,226][53885] Updated weights for policy 1, policy_version 730 (0.0009) [2023-10-08 07:52:11,600][53852] Updated weights for policy 0, policy_version 740 (0.0008) [2023-10-08 07:52:11,970][53852] Updated weights for policy 0, policy_version 750 (0.0010) [2023-10-08 07:52:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13354.4). Total num frames: 1507328. Throughput: 0: 1781.6, 1: 1763.8. Samples: 386260. Policy #0 lag: (min: 4.0, avg: 5.6, max: 32.0) [2023-10-08 07:52:12,015][52710] Avg episode reward: [(0, '2.120'), (1, '1.810')] [2023-10-08 07:52:12,118][53885] Updated weights for policy 1, policy_version 740 (0.0007) [2023-10-08 07:52:12,343][53852] Updated weights for policy 0, policy_version 760 (0.0009) [2023-10-08 07:52:12,486][53885] Updated weights for policy 1, policy_version 750 (0.0007) [2023-10-08 07:52:12,636][53500] Saving new best policy, reward=2.120! [2023-10-08 07:52:12,846][53885] Updated weights for policy 1, policy_version 760 (0.0008) [2023-10-08 07:52:16,080][53852] Updated weights for policy 0, policy_version 770 (0.0007) [2023-10-08 07:52:16,453][53852] Updated weights for policy 0, policy_version 780 (0.0009) [2023-10-08 07:52:16,733][53885] Updated weights for policy 1, policy_version 770 (0.0007) [2023-10-08 07:52:16,828][53852] Updated weights for policy 0, policy_version 790 (0.0008) [2023-10-08 07:52:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13343.9). Total num frames: 1572864. Throughput: 0: 1797.4, 1: 1771.0. Samples: 408712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:52:17,016][52710] Avg episode reward: [(0, '1.720'), (1, '2.230')] [2023-10-08 07:52:17,106][53885] Updated weights for policy 1, policy_version 780 (0.0007) [2023-10-08 07:52:17,183][53852] Updated weights for policy 0, policy_version 800 (0.0007) [2023-10-08 07:52:17,487][53885] Updated weights for policy 1, policy_version 790 (0.0007) [2023-10-08 07:52:17,843][53594] Saving new best policy, reward=2.230! [2023-10-08 07:52:17,846][53885] Updated weights for policy 1, policy_version 800 (0.0008) [2023-10-08 07:52:20,879][53852] Updated weights for policy 0, policy_version 810 (0.0008) [2023-10-08 07:52:21,248][53852] Updated weights for policy 0, policy_version 820 (0.0009) [2023-10-08 07:52:21,506][53885] Updated weights for policy 1, policy_version 810 (0.0009) [2023-10-08 07:52:21,626][53852] Updated weights for policy 0, policy_version 830 (0.0009) [2023-10-08 07:52:21,875][53885] Updated weights for policy 1, policy_version 820 (0.0007) [2023-10-08 07:52:22,015][52710] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 13600.9). Total num frames: 1671168. Throughput: 0: 1795.9, 1: 1793.1. Samples: 429538. Policy #0 lag: (min: 12.0, avg: 17.4, max: 44.0) [2023-10-08 07:52:22,016][52710] Avg episode reward: [(0, '2.200'), (1, '2.030')] [2023-10-08 07:52:22,026][53500] Saving new best policy, reward=2.200! [2023-10-08 07:52:22,244][53885] Updated weights for policy 1, policy_version 830 (0.0007) [2023-10-08 07:52:25,308][53852] Updated weights for policy 0, policy_version 840 (0.0008) [2023-10-08 07:52:25,676][53852] Updated weights for policy 0, policy_version 850 (0.0008) [2023-10-08 07:52:26,041][53852] Updated weights for policy 0, policy_version 860 (0.0007) [2023-10-08 07:52:26,046][53885] Updated weights for policy 1, policy_version 840 (0.0007) [2023-10-08 07:52:26,420][53885] Updated weights for policy 1, policy_version 850 (0.0007) [2023-10-08 07:52:26,781][53885] Updated weights for policy 1, policy_version 860 (0.0010) [2023-10-08 07:52:27,015][52710] Fps is (10 sec: 19660.5, 60 sec: 14745.6, 300 sec: 13837.9). Total num frames: 1769472. Throughput: 0: 1798.7, 1: 1772.7. Samples: 441106. Policy #0 lag: (min: 8.0, avg: 31.1, max: 40.0) [2023-10-08 07:52:27,016][52710] Avg episode reward: [(0, '1.880'), (1, '1.720')] [2023-10-08 07:52:29,639][53852] Updated weights for policy 0, policy_version 870 (0.0008) [2023-10-08 07:52:30,012][53852] Updated weights for policy 0, policy_version 880 (0.0007) [2023-10-08 07:52:30,376][53852] Updated weights for policy 0, policy_version 890 (0.0009) [2023-10-08 07:52:30,722][53885] Updated weights for policy 1, policy_version 870 (0.0009) [2023-10-08 07:52:31,093][53885] Updated weights for policy 1, policy_version 880 (0.0008) [2023-10-08 07:52:31,456][53885] Updated weights for policy 1, policy_version 890 (0.0007) [2023-10-08 07:52:32,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14199.5, 300 sec: 13810.4). Total num frames: 1835008. Throughput: 0: 1796.5, 1: 1799.6. Samples: 461870. Policy #0 lag: (min: 23.0, avg: 45.2, max: 48.0) [2023-10-08 07:52:32,016][52710] Avg episode reward: [(0, '1.710'), (1, '2.280')] [2023-10-08 07:52:32,017][53594] Saving new best policy, reward=2.280! [2023-10-08 07:52:34,208][53852] Updated weights for policy 0, policy_version 900 (0.0008) [2023-10-08 07:52:34,572][53852] Updated weights for policy 0, policy_version 910 (0.0007) [2023-10-08 07:52:34,951][53852] Updated weights for policy 0, policy_version 920 (0.0007) [2023-10-08 07:52:35,205][53885] Updated weights for policy 1, policy_version 900 (0.0009) [2023-10-08 07:52:35,583][53885] Updated weights for policy 1, policy_version 910 (0.0009) [2023-10-08 07:52:35,955][53885] Updated weights for policy 1, policy_version 920 (0.0010) [2023-10-08 07:52:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13784.9). Total num frames: 1900544. Throughput: 0: 1793.5, 1: 1773.1. Samples: 482942. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 07:52:37,016][52710] Avg episode reward: [(0, '2.140'), (1, '2.420')] [2023-10-08 07:52:37,029][53594] Saving new best policy, reward=2.420! [2023-10-08 07:52:38,715][53852] Updated weights for policy 0, policy_version 930 (0.0007) [2023-10-08 07:52:39,084][53852] Updated weights for policy 0, policy_version 940 (0.0009) [2023-10-08 07:52:39,449][53852] Updated weights for policy 0, policy_version 950 (0.0007) [2023-10-08 07:52:39,717][53885] Updated weights for policy 1, policy_version 930 (0.0007) [2023-10-08 07:52:39,821][53852] Updated weights for policy 0, policy_version 960 (0.0007) [2023-10-08 07:52:40,093][53885] Updated weights for policy 1, policy_version 940 (0.0008) [2023-10-08 07:52:40,465][53885] Updated weights for policy 1, policy_version 950 (0.0010) [2023-10-08 07:52:40,826][53885] Updated weights for policy 1, policy_version 960 (0.0009) [2023-10-08 07:52:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13761.2). Total num frames: 1966080. Throughput: 0: 1800.9, 1: 1800.7. Samples: 494660. Policy #0 lag: (min: 31.0, avg: 34.9, max: 63.0) [2023-10-08 07:52:42,016][52710] Avg episode reward: [(0, '2.180'), (1, '2.030')] [2023-10-08 07:52:43,470][53852] Updated weights for policy 0, policy_version 970 (0.0007) [2023-10-08 07:52:43,831][53852] Updated weights for policy 0, policy_version 980 (0.0008) [2023-10-08 07:52:44,205][53852] Updated weights for policy 0, policy_version 990 (0.0008) [2023-10-08 07:52:44,505][53885] Updated weights for policy 1, policy_version 970 (0.0008) [2023-10-08 07:52:44,869][53885] Updated weights for policy 1, policy_version 980 (0.0010) [2023-10-08 07:52:45,236][53885] Updated weights for policy 1, policy_version 990 (0.0010) [2023-10-08 07:52:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13739.1). Total num frames: 2031616. Throughput: 0: 1803.0, 1: 1783.9. Samples: 515624. Policy #0 lag: (min: 21.0, avg: 21.1, max: 28.0) [2023-10-08 07:52:47,016][52710] Avg episode reward: [(0, '2.260'), (1, '2.080')] [2023-10-08 07:52:47,017][53500] Saving new best policy, reward=2.260! [2023-10-08 07:52:47,960][53852] Updated weights for policy 0, policy_version 1000 (0.0008) [2023-10-08 07:52:48,346][53852] Updated weights for policy 0, policy_version 1010 (0.0009) [2023-10-08 07:52:48,713][53852] Updated weights for policy 0, policy_version 1020 (0.0007) [2023-10-08 07:52:49,107][53885] Updated weights for policy 1, policy_version 1000 (0.0008) [2023-10-08 07:52:49,481][53885] Updated weights for policy 1, policy_version 1010 (0.0008) [2023-10-08 07:52:49,854][53885] Updated weights for policy 1, policy_version 1020 (0.0008) [2023-10-08 07:52:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13718.4). Total num frames: 2097152. Throughput: 0: 1802.4, 1: 1782.2. Samples: 537952. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 07:52:52,016][52710] Avg episode reward: [(0, '1.960'), (1, '2.310')] [2023-10-08 07:52:52,347][53852] Updated weights for policy 0, policy_version 1030 (0.0008) [2023-10-08 07:52:52,714][53852] Updated weights for policy 0, policy_version 1040 (0.0008) [2023-10-08 07:52:53,083][53852] Updated weights for policy 0, policy_version 1050 (0.0008) [2023-10-08 07:52:53,460][53885] Updated weights for policy 1, policy_version 1030 (0.0008) [2023-10-08 07:52:53,830][53885] Updated weights for policy 1, policy_version 1040 (0.0009) [2023-10-08 07:52:54,191][53885] Updated weights for policy 1, policy_version 1050 (0.0009) [2023-10-08 07:52:56,824][53852] Updated weights for policy 0, policy_version 1060 (0.0008) [2023-10-08 07:52:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13699.0). Total num frames: 2162688. Throughput: 0: 1802.7, 1: 1785.3. Samples: 547722. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 07:52:57,016][52710] Avg episode reward: [(0, '2.120'), (1, '1.870')] [2023-10-08 07:52:57,195][53852] Updated weights for policy 0, policy_version 1070 (0.0007) [2023-10-08 07:52:57,565][53852] Updated weights for policy 0, policy_version 1080 (0.0007) [2023-10-08 07:52:58,167][53885] Updated weights for policy 1, policy_version 1060 (0.0010) [2023-10-08 07:52:58,538][53885] Updated weights for policy 1, policy_version 1070 (0.0010) [2023-10-08 07:52:58,902][53885] Updated weights for policy 1, policy_version 1080 (0.0009) [2023-10-08 07:53:01,234][53852] Updated weights for policy 0, policy_version 1090 (0.0007) [2023-10-08 07:53:01,596][53852] Updated weights for policy 0, policy_version 1100 (0.0009) [2023-10-08 07:53:01,974][53852] Updated weights for policy 0, policy_version 1110 (0.0010) [2023-10-08 07:53:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13680.9). Total num frames: 2228224. Throughput: 0: 1808.6, 1: 1776.1. Samples: 570024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:53:02,016][52710] Avg episode reward: [(0, '2.170'), (1, '1.810')] [2023-10-08 07:53:02,348][53852] Updated weights for policy 0, policy_version 1120 (0.0009) [2023-10-08 07:53:02,786][53885] Updated weights for policy 1, policy_version 1090 (0.0009) [2023-10-08 07:53:03,153][53885] Updated weights for policy 1, policy_version 1100 (0.0009) [2023-10-08 07:53:03,518][53885] Updated weights for policy 1, policy_version 1110 (0.0008) [2023-10-08 07:53:03,886][53885] Updated weights for policy 1, policy_version 1120 (0.0009) [2023-10-08 07:53:06,065][53852] Updated weights for policy 0, policy_version 1130 (0.0010) [2023-10-08 07:53:06,432][53852] Updated weights for policy 0, policy_version 1140 (0.0009) [2023-10-08 07:53:06,801][53852] Updated weights for policy 0, policy_version 1150 (0.0009) [2023-10-08 07:53:07,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 13859.0). Total num frames: 2326528. Throughput: 0: 1811.7, 1: 1786.9. Samples: 591470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:53:07,016][52710] Avg episode reward: [(0, '2.140'), (1, '1.990')] [2023-10-08 07:53:07,531][53885] Updated weights for policy 1, policy_version 1130 (0.0009) [2023-10-08 07:53:07,899][53885] Updated weights for policy 1, policy_version 1140 (0.0008) [2023-10-08 07:53:08,269][53885] Updated weights for policy 1, policy_version 1150 (0.0009) [2023-10-08 07:53:10,534][53852] Updated weights for policy 0, policy_version 1160 (0.0007) [2023-10-08 07:53:10,911][53852] Updated weights for policy 0, policy_version 1170 (0.0010) [2023-10-08 07:53:11,281][53852] Updated weights for policy 0, policy_version 1180 (0.0008) [2023-10-08 07:53:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 13837.2). Total num frames: 2392064. Throughput: 0: 1807.8, 1: 1775.3. Samples: 602346. Policy #0 lag: (min: 31.0, avg: 31.7, max: 44.0) [2023-10-08 07:53:12,016][52710] Avg episode reward: [(0, '2.520'), (1, '2.100')] [2023-10-08 07:53:12,018][53500] Saving new best policy, reward=2.520! [2023-10-08 07:53:12,027][53885] Updated weights for policy 1, policy_version 1160 (0.0009) [2023-10-08 07:53:12,390][53885] Updated weights for policy 1, policy_version 1170 (0.0009) [2023-10-08 07:53:12,766][53885] Updated weights for policy 1, policy_version 1180 (0.0008) [2023-10-08 07:53:15,091][53852] Updated weights for policy 0, policy_version 1190 (0.0008) [2023-10-08 07:53:15,456][53852] Updated weights for policy 0, policy_version 1200 (0.0009) [2023-10-08 07:53:15,825][53852] Updated weights for policy 0, policy_version 1210 (0.0008) [2023-10-08 07:53:16,518][53885] Updated weights for policy 1, policy_version 1190 (0.0008) [2023-10-08 07:53:16,888][53885] Updated weights for policy 1, policy_version 1200 (0.0008) [2023-10-08 07:53:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 13816.7). Total num frames: 2457600. Throughput: 0: 1818.7, 1: 1783.4. Samples: 623966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:53:17,016][52710] Avg episode reward: [(0, '1.960'), (1, '1.920')] [2023-10-08 07:53:17,251][53885] Updated weights for policy 1, policy_version 1210 (0.0007) [2023-10-08 07:53:19,278][53852] Updated weights for policy 0, policy_version 1220 (0.0009) [2023-10-08 07:53:19,633][53852] Updated weights for policy 0, policy_version 1230 (0.0007) [2023-10-08 07:53:20,003][53852] Updated weights for policy 0, policy_version 1240 (0.0007) [2023-10-08 07:53:21,003][53885] Updated weights for policy 1, policy_version 1220 (0.0009) [2023-10-08 07:53:21,373][53885] Updated weights for policy 1, policy_version 1230 (0.0008) [2023-10-08 07:53:21,734][53885] Updated weights for policy 1, policy_version 1240 (0.0008) [2023-10-08 07:53:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13797.3). Total num frames: 2523136. Throughput: 0: 1813.3, 1: 1792.7. Samples: 645214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:53:22,016][52710] Avg episode reward: [(0, '2.100'), (1, '2.260')] [2023-10-08 07:53:23,800][53852] Updated weights for policy 0, policy_version 1250 (0.0008) [2023-10-08 07:53:24,169][53852] Updated weights for policy 0, policy_version 1260 (0.0007) [2023-10-08 07:53:24,543][53852] Updated weights for policy 0, policy_version 1270 (0.0008) [2023-10-08 07:53:24,910][53852] Updated weights for policy 0, policy_version 1280 (0.0008) [2023-10-08 07:53:25,481][53885] Updated weights for policy 1, policy_version 1250 (0.0008) [2023-10-08 07:53:25,849][53885] Updated weights for policy 1, policy_version 1260 (0.0008) [2023-10-08 07:53:26,224][53885] Updated weights for policy 1, policy_version 1270 (0.0008) [2023-10-08 07:53:26,596][53885] Updated weights for policy 1, policy_version 1280 (0.0009) [2023-10-08 07:53:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 13953.4). Total num frames: 2621440. Throughput: 0: 1815.0, 1: 1778.3. Samples: 656358. Policy #0 lag: (min: 31.0, avg: 44.5, max: 63.0) [2023-10-08 07:53:27,016][52710] Avg episode reward: [(0, '2.260'), (1, '2.130')] [2023-10-08 07:53:28,731][53852] Updated weights for policy 0, policy_version 1290 (0.0008) [2023-10-08 07:53:29,102][53852] Updated weights for policy 0, policy_version 1300 (0.0009) [2023-10-08 07:53:29,464][53852] Updated weights for policy 0, policy_version 1310 (0.0008) [2023-10-08 07:53:30,239][53885] Updated weights for policy 1, policy_version 1290 (0.0010) [2023-10-08 07:53:30,604][53885] Updated weights for policy 1, policy_version 1300 (0.0009) [2023-10-08 07:53:30,975][53885] Updated weights for policy 1, policy_version 1310 (0.0008) [2023-10-08 07:53:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14199.4, 300 sec: 13931.4). Total num frames: 2686976. Throughput: 0: 1805.7, 1: 1792.3. Samples: 677532. Policy #0 lag: (min: 26.0, avg: 26.7, max: 43.0) [2023-10-08 07:53:32,016][52710] Avg episode reward: [(0, '2.320'), (1, '2.580')] [2023-10-08 07:53:32,018][53594] Saving new best policy, reward=2.580! [2023-10-08 07:53:33,318][53852] Updated weights for policy 0, policy_version 1320 (0.0009) [2023-10-08 07:53:33,698][53852] Updated weights for policy 0, policy_version 1330 (0.0009) [2023-10-08 07:53:34,066][53852] Updated weights for policy 0, policy_version 1340 (0.0010) [2023-10-08 07:53:34,640][53885] Updated weights for policy 1, policy_version 1320 (0.0009) [2023-10-08 07:53:35,006][53885] Updated weights for policy 1, policy_version 1330 (0.0010) [2023-10-08 07:53:35,365][53885] Updated weights for policy 1, policy_version 1340 (0.0009) [2023-10-08 07:53:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13910.6). Total num frames: 2752512. Throughput: 0: 1801.3, 1: 1783.8. Samples: 699280. Policy #0 lag: (min: 28.0, avg: 40.4, max: 60.0) [2023-10-08 07:53:37,016][52710] Avg episode reward: [(0, '2.320'), (1, '2.450')] [2023-10-08 07:53:37,723][53852] Updated weights for policy 0, policy_version 1350 (0.0009) [2023-10-08 07:53:38,104][53852] Updated weights for policy 0, policy_version 1360 (0.0008) [2023-10-08 07:53:38,468][53852] Updated weights for policy 0, policy_version 1370 (0.0007) [2023-10-08 07:53:39,050][53885] Updated weights for policy 1, policy_version 1350 (0.0011) [2023-10-08 07:53:39,425][53885] Updated weights for policy 1, policy_version 1360 (0.0009) [2023-10-08 07:53:39,790][53885] Updated weights for policy 1, policy_version 1370 (0.0010) [2023-10-08 07:53:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13890.8). Total num frames: 2818048. Throughput: 0: 1802.1, 1: 1795.8. Samples: 709628. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 07:53:42,016][52710] Avg episode reward: [(0, '2.490'), (1, '2.300')] [2023-10-08 07:53:42,299][53852] Updated weights for policy 0, policy_version 1380 (0.0009) [2023-10-08 07:53:42,668][53852] Updated weights for policy 0, policy_version 1390 (0.0010) [2023-10-08 07:53:43,043][53852] Updated weights for policy 0, policy_version 1400 (0.0009) [2023-10-08 07:53:43,481][53885] Updated weights for policy 1, policy_version 1380 (0.0009) [2023-10-08 07:53:43,851][53885] Updated weights for policy 1, policy_version 1390 (0.0007) [2023-10-08 07:53:44,213][53885] Updated weights for policy 1, policy_version 1400 (0.0007) [2023-10-08 07:53:46,872][53852] Updated weights for policy 0, policy_version 1410 (0.0008) [2023-10-08 07:53:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13872.0). Total num frames: 2883584. Throughput: 0: 1793.2, 1: 1798.4. Samples: 731642. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 07:53:47,016][52710] Avg episode reward: [(0, '2.650'), (1, '2.230')] [2023-10-08 07:53:47,244][53852] Updated weights for policy 0, policy_version 1420 (0.0009) [2023-10-08 07:53:47,628][53852] Updated weights for policy 0, policy_version 1430 (0.0008) [2023-10-08 07:53:47,992][53885] Updated weights for policy 1, policy_version 1410 (0.0008) [2023-10-08 07:53:47,993][53500] Saving new best policy, reward=2.650! [2023-10-08 07:53:47,997][53852] Updated weights for policy 0, policy_version 1440 (0.0007) [2023-10-08 07:53:48,349][53885] Updated weights for policy 1, policy_version 1420 (0.0008) [2023-10-08 07:53:48,711][53885] Updated weights for policy 1, policy_version 1430 (0.0008) [2023-10-08 07:53:49,076][53885] Updated weights for policy 1, policy_version 1440 (0.0010) [2023-10-08 07:53:51,786][53852] Updated weights for policy 0, policy_version 1450 (0.0007) [2023-10-08 07:53:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13854.0). Total num frames: 2949120. Throughput: 0: 1812.7, 1: 1801.1. Samples: 754090. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:53:52,016][52710] Avg episode reward: [(0, '2.590'), (1, '2.490')] [2023-10-08 07:53:52,146][53852] Updated weights for policy 0, policy_version 1460 (0.0007) [2023-10-08 07:53:52,511][53852] Updated weights for policy 0, policy_version 1470 (0.0008) [2023-10-08 07:53:52,747][53885] Updated weights for policy 1, policy_version 1450 (0.0008) [2023-10-08 07:53:53,111][53885] Updated weights for policy 1, policy_version 1460 (0.0011) [2023-10-08 07:53:53,475][53885] Updated weights for policy 1, policy_version 1470 (0.0007) [2023-10-08 07:53:56,148][53852] Updated weights for policy 0, policy_version 1480 (0.0007) [2023-10-08 07:53:56,519][53852] Updated weights for policy 0, policy_version 1490 (0.0007) [2023-10-08 07:53:56,897][53852] Updated weights for policy 0, policy_version 1500 (0.0008) [2023-10-08 07:53:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13836.9). Total num frames: 3014656. Throughput: 0: 1794.7, 1: 1802.1. Samples: 764204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:53:57,016][52710] Avg episode reward: [(0, '3.010'), (1, '2.530')] [2023-10-08 07:53:57,037][53500] Saving new best policy, reward=3.010! [2023-10-08 07:53:57,146][53885] Updated weights for policy 1, policy_version 1480 (0.0008) [2023-10-08 07:53:57,507][53885] Updated weights for policy 1, policy_version 1490 (0.0007) [2023-10-08 07:53:57,880][53885] Updated weights for policy 1, policy_version 1500 (0.0007) [2023-10-08 07:54:00,504][53852] Updated weights for policy 0, policy_version 1510 (0.0009) [2023-10-08 07:54:00,862][53852] Updated weights for policy 0, policy_version 1520 (0.0009) [2023-10-08 07:54:01,239][53852] Updated weights for policy 0, policy_version 1530 (0.0009) [2023-10-08 07:54:01,666][53885] Updated weights for policy 1, policy_version 1510 (0.0009) [2023-10-08 07:54:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 13967.5). Total num frames: 3112960. Throughput: 0: 1812.1, 1: 1805.9. Samples: 786778. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 07:54:02,016][52710] Avg episode reward: [(0, '2.800'), (1, '2.680')] [2023-10-08 07:54:02,034][53885] Updated weights for policy 1, policy_version 1520 (0.0008) [2023-10-08 07:54:02,401][53885] Updated weights for policy 1, policy_version 1530 (0.0010) [2023-10-08 07:54:02,619][53594] Saving new best policy, reward=2.680! [2023-10-08 07:54:05,020][53852] Updated weights for policy 0, policy_version 1540 (0.0008) [2023-10-08 07:54:05,386][53852] Updated weights for policy 0, policy_version 1550 (0.0007) [2023-10-08 07:54:05,753][53852] Updated weights for policy 0, policy_version 1560 (0.0009) [2023-10-08 07:54:06,088][53885] Updated weights for policy 1, policy_version 1540 (0.0009) [2023-10-08 07:54:06,452][53885] Updated weights for policy 1, policy_version 1550 (0.0009) [2023-10-08 07:54:06,833][53885] Updated weights for policy 1, policy_version 1560 (0.0008) [2023-10-08 07:54:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14199.4, 300 sec: 13948.6). Total num frames: 3178496. Throughput: 0: 1793.3, 1: 1815.5. Samples: 807612. Policy #0 lag: (min: 17.0, avg: 31.3, max: 49.0) [2023-10-08 07:54:07,016][52710] Avg episode reward: [(0, '2.960'), (1, '2.450')] [2023-10-08 07:54:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth... [2023-10-08 07:54:07,120][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000001568_1605632.pth... [2023-10-08 07:54:09,448][53852] Updated weights for policy 0, policy_version 1570 (0.0008) [2023-10-08 07:54:09,823][53852] Updated weights for policy 0, policy_version 1580 (0.0008) [2023-10-08 07:54:10,196][53852] Updated weights for policy 0, policy_version 1590 (0.0010) [2023-10-08 07:54:10,572][53852] Updated weights for policy 0, policy_version 1600 (0.0008) [2023-10-08 07:54:10,633][53885] Updated weights for policy 1, policy_version 1570 (0.0008) [2023-10-08 07:54:11,001][53885] Updated weights for policy 1, policy_version 1580 (0.0011) [2023-10-08 07:54:11,374][53885] Updated weights for policy 1, policy_version 1590 (0.0009) [2023-10-08 07:54:11,749][53885] Updated weights for policy 1, policy_version 1600 (0.0008) [2023-10-08 07:54:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14071.3). Total num frames: 3276800. Throughput: 0: 1817.1, 1: 1811.6. Samples: 819648. Policy #0 lag: (min: 30.0, avg: 36.7, max: 62.0) [2023-10-08 07:54:12,015][52710] Avg episode reward: [(0, '2.840'), (1, '2.700')] [2023-10-08 07:54:12,016][53594] Saving new best policy, reward=2.700! [2023-10-08 07:54:14,239][53852] Updated weights for policy 0, policy_version 1610 (0.0009) [2023-10-08 07:54:14,607][53852] Updated weights for policy 0, policy_version 1620 (0.0008) [2023-10-08 07:54:14,970][53852] Updated weights for policy 0, policy_version 1630 (0.0008) [2023-10-08 07:54:15,428][53885] Updated weights for policy 1, policy_version 1610 (0.0008) [2023-10-08 07:54:15,797][53885] Updated weights for policy 1, policy_version 1620 (0.0008) [2023-10-08 07:54:16,173][53885] Updated weights for policy 1, policy_version 1630 (0.0007) [2023-10-08 07:54:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14051.0). Total num frames: 3342336. Throughput: 0: 1802.8, 1: 1816.1. Samples: 840382. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 07:54:17,016][52710] Avg episode reward: [(0, '2.540'), (1, '2.400')] [2023-10-08 07:54:18,737][53852] Updated weights for policy 0, policy_version 1640 (0.0007) [2023-10-08 07:54:19,110][53852] Updated weights for policy 0, policy_version 1650 (0.0008) [2023-10-08 07:54:19,482][53852] Updated weights for policy 0, policy_version 1660 (0.0008) [2023-10-08 07:54:19,969][53885] Updated weights for policy 1, policy_version 1640 (0.0010) [2023-10-08 07:54:20,338][53885] Updated weights for policy 1, policy_version 1650 (0.0010) [2023-10-08 07:54:20,703][53885] Updated weights for policy 1, policy_version 1660 (0.0008) [2023-10-08 07:54:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14031.6). Total num frames: 3407872. Throughput: 0: 1811.7, 1: 1804.0. Samples: 861990. Policy #0 lag: (min: 4.0, avg: 18.9, max: 36.0) [2023-10-08 07:54:22,016][52710] Avg episode reward: [(0, '2.920'), (1, '2.770')] [2023-10-08 07:54:22,025][53594] Saving new best policy, reward=2.770! [2023-10-08 07:54:23,060][53852] Updated weights for policy 0, policy_version 1670 (0.0007) [2023-10-08 07:54:23,428][53852] Updated weights for policy 0, policy_version 1680 (0.0008) [2023-10-08 07:54:23,804][53852] Updated weights for policy 0, policy_version 1690 (0.0010) [2023-10-08 07:54:24,533][53885] Updated weights for policy 1, policy_version 1670 (0.0008) [2023-10-08 07:54:24,895][53885] Updated weights for policy 1, policy_version 1680 (0.0007) [2023-10-08 07:54:25,257][53885] Updated weights for policy 1, policy_version 1690 (0.0009) [2023-10-08 07:54:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14012.9). Total num frames: 3473408. Throughput: 0: 1810.1, 1: 1816.7. Samples: 872834. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 07:54:27,016][52710] Avg episode reward: [(0, '2.850'), (1, '2.840')] [2023-10-08 07:54:27,018][53594] Saving new best policy, reward=2.840! [2023-10-08 07:54:27,331][53852] Updated weights for policy 0, policy_version 1700 (0.0010) [2023-10-08 07:54:27,690][53852] Updated weights for policy 0, policy_version 1710 (0.0009) [2023-10-08 07:54:28,068][53852] Updated weights for policy 0, policy_version 1720 (0.0009) [2023-10-08 07:54:28,937][53885] Updated weights for policy 1, policy_version 1700 (0.0010) [2023-10-08 07:54:29,305][53885] Updated weights for policy 1, policy_version 1710 (0.0010) [2023-10-08 07:54:29,673][53885] Updated weights for policy 1, policy_version 1720 (0.0009) [2023-10-08 07:54:31,796][53852] Updated weights for policy 0, policy_version 1730 (0.0007) [2023-10-08 07:54:32,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13995.0). Total num frames: 3538944. Throughput: 0: 1819.1, 1: 1800.2. Samples: 894512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:54:32,015][52710] Avg episode reward: [(0, '3.140'), (1, '2.860')] [2023-10-08 07:54:32,016][53594] Saving new best policy, reward=2.860! [2023-10-08 07:54:32,166][53852] Updated weights for policy 0, policy_version 1740 (0.0008) [2023-10-08 07:54:32,530][53852] Updated weights for policy 0, policy_version 1750 (0.0007) [2023-10-08 07:54:32,894][53500] Saving new best policy, reward=3.140! [2023-10-08 07:54:32,895][53852] Updated weights for policy 0, policy_version 1760 (0.0007) [2023-10-08 07:54:33,495][53885] Updated weights for policy 1, policy_version 1730 (0.0008) [2023-10-08 07:54:33,863][53885] Updated weights for policy 1, policy_version 1740 (0.0011) [2023-10-08 07:54:34,227][53885] Updated weights for policy 1, policy_version 1750 (0.0011) [2023-10-08 07:54:34,595][53885] Updated weights for policy 1, policy_version 1760 (0.0010) [2023-10-08 07:54:36,644][53852] Updated weights for policy 0, policy_version 1770 (0.0007) [2023-10-08 07:54:37,011][53852] Updated weights for policy 0, policy_version 1780 (0.0007) [2023-10-08 07:54:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13977.8). Total num frames: 3604480. Throughput: 0: 1818.8, 1: 1798.1. Samples: 916852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:54:37,015][52710] Avg episode reward: [(0, '2.610'), (1, '3.000')] [2023-10-08 07:54:37,022][53594] Saving new best policy, reward=3.000! [2023-10-08 07:54:37,384][53852] Updated weights for policy 0, policy_version 1790 (0.0008) [2023-10-08 07:54:38,290][53885] Updated weights for policy 1, policy_version 1770 (0.0008) [2023-10-08 07:54:38,654][53885] Updated weights for policy 1, policy_version 1780 (0.0009) [2023-10-08 07:54:39,025][53885] Updated weights for policy 1, policy_version 1790 (0.0010) [2023-10-08 07:54:41,109][53852] Updated weights for policy 0, policy_version 1800 (0.0009) [2023-10-08 07:54:41,489][53852] Updated weights for policy 0, policy_version 1810 (0.0011) [2023-10-08 07:54:41,861][53852] Updated weights for policy 0, policy_version 1820 (0.0007) [2023-10-08 07:54:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13961.3). Total num frames: 3670016. Throughput: 0: 1821.7, 1: 1798.4. Samples: 927110. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 07:54:42,016][52710] Avg episode reward: [(0, '2.980'), (1, '3.250')] [2023-10-08 07:54:42,017][53594] Saving new best policy, reward=3.250! [2023-10-08 07:54:42,774][53885] Updated weights for policy 1, policy_version 1800 (0.0009) [2023-10-08 07:54:43,133][53885] Updated weights for policy 1, policy_version 1810 (0.0007) [2023-10-08 07:54:43,510][53885] Updated weights for policy 1, policy_version 1820 (0.0007) [2023-10-08 07:54:45,499][53852] Updated weights for policy 0, policy_version 1830 (0.0008) [2023-10-08 07:54:45,870][53852] Updated weights for policy 0, policy_version 1840 (0.0010) [2023-10-08 07:54:46,238][53852] Updated weights for policy 0, policy_version 1850 (0.0009) [2023-10-08 07:54:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14067.6). Total num frames: 3768320. Throughput: 0: 1821.6, 1: 1791.2. Samples: 949356. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 07:54:47,016][52710] Avg episode reward: [(0, '3.320'), (1, '2.930')] [2023-10-08 07:54:47,016][53500] Saving new best policy, reward=3.320! [2023-10-08 07:54:47,298][53885] Updated weights for policy 1, policy_version 1830 (0.0009) [2023-10-08 07:54:47,663][53885] Updated weights for policy 1, policy_version 1840 (0.0008) [2023-10-08 07:54:48,028][53885] Updated weights for policy 1, policy_version 1850 (0.0007) [2023-10-08 07:54:50,006][53852] Updated weights for policy 0, policy_version 1860 (0.0009) [2023-10-08 07:54:50,375][53852] Updated weights for policy 0, policy_version 1870 (0.0009) [2023-10-08 07:54:50,753][53852] Updated weights for policy 0, policy_version 1880 (0.0008) [2023-10-08 07:54:51,773][53885] Updated weights for policy 1, policy_version 1860 (0.0008) [2023-10-08 07:54:52,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14050.0). Total num frames: 3833856. Throughput: 0: 1819.4, 1: 1801.5. Samples: 970552. Policy #0 lag: (min: 26.0, avg: 33.4, max: 58.0) [2023-10-08 07:54:52,016][52710] Avg episode reward: [(0, '3.160'), (1, '3.640')] [2023-10-08 07:54:52,145][53885] Updated weights for policy 1, policy_version 1870 (0.0009) [2023-10-08 07:54:52,507][53885] Updated weights for policy 1, policy_version 1880 (0.0009) [2023-10-08 07:54:52,799][53594] Saving new best policy, reward=3.640! [2023-10-08 07:54:54,434][53852] Updated weights for policy 0, policy_version 1890 (0.0008) [2023-10-08 07:54:54,803][53852] Updated weights for policy 0, policy_version 1900 (0.0007) [2023-10-08 07:54:55,178][53852] Updated weights for policy 0, policy_version 1910 (0.0009) [2023-10-08 07:54:55,546][53852] Updated weights for policy 0, policy_version 1920 (0.0008) [2023-10-08 07:54:56,057][53885] Updated weights for policy 1, policy_version 1890 (0.0008) [2023-10-08 07:54:56,418][53885] Updated weights for policy 1, policy_version 1900 (0.0010) [2023-10-08 07:54:56,782][53885] Updated weights for policy 1, policy_version 1910 (0.0009) [2023-10-08 07:54:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14033.1). Total num frames: 3899392. Throughput: 0: 1814.3, 1: 1787.2. Samples: 981714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:54:57,016][52710] Avg episode reward: [(0, '3.150'), (1, '3.420')] [2023-10-08 07:54:57,155][53885] Updated weights for policy 1, policy_version 1920 (0.0009) [2023-10-08 07:54:59,242][53852] Updated weights for policy 0, policy_version 1930 (0.0008) [2023-10-08 07:54:59,614][53852] Updated weights for policy 0, policy_version 1940 (0.0008) [2023-10-08 07:54:59,977][53852] Updated weights for policy 0, policy_version 1950 (0.0008) [2023-10-08 07:55:00,905][53885] Updated weights for policy 1, policy_version 1930 (0.0011) [2023-10-08 07:55:01,282][53885] Updated weights for policy 1, policy_version 1940 (0.0010) [2023-10-08 07:55:01,645][53885] Updated weights for policy 1, policy_version 1950 (0.0008) [2023-10-08 07:55:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14132.5). Total num frames: 3997696. Throughput: 0: 1815.2, 1: 1804.8. Samples: 1003286. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-08 07:55:02,016][52710] Avg episode reward: [(0, '3.330'), (1, '4.060')] [2023-10-08 07:55:02,017][53500] Saving new best policy, reward=3.330! [2023-10-08 07:55:02,018][53594] Saving new best policy, reward=4.060! [2023-10-08 07:55:03,761][53852] Updated weights for policy 0, policy_version 1960 (0.0010) [2023-10-08 07:55:04,135][53852] Updated weights for policy 0, policy_version 1970 (0.0007) [2023-10-08 07:55:04,503][53852] Updated weights for policy 0, policy_version 1980 (0.0007) [2023-10-08 07:55:05,500][53885] Updated weights for policy 1, policy_version 1960 (0.0010) [2023-10-08 07:55:05,889][53885] Updated weights for policy 1, policy_version 1970 (0.0009) [2023-10-08 07:55:06,266][53885] Updated weights for policy 1, policy_version 1980 (0.0009) [2023-10-08 07:55:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14114.7). Total num frames: 4063232. Throughput: 0: 1806.5, 1: 1797.0. Samples: 1024144. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 07:55:07,016][52710] Avg episode reward: [(0, '3.290'), (1, '3.600')] [2023-10-08 07:55:08,213][53852] Updated weights for policy 0, policy_version 1990 (0.0008) [2023-10-08 07:55:08,575][53852] Updated weights for policy 0, policy_version 2000 (0.0007) [2023-10-08 07:55:08,947][53852] Updated weights for policy 0, policy_version 2010 (0.0007) [2023-10-08 07:55:09,923][53885] Updated weights for policy 1, policy_version 1990 (0.0008) [2023-10-08 07:55:10,302][53885] Updated weights for policy 1, policy_version 2000 (0.0008) [2023-10-08 07:55:10,669][53885] Updated weights for policy 1, policy_version 2010 (0.0007) [2023-10-08 07:55:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14097.5). Total num frames: 4128768. Throughput: 0: 1811.3, 1: 1808.7. Samples: 1035734. Policy #0 lag: (min: 1.0, avg: 2.7, max: 29.0) [2023-10-08 07:55:12,016][52710] Avg episode reward: [(0, '3.440'), (1, '3.480')] [2023-10-08 07:55:12,017][53500] Saving new best policy, reward=3.440! [2023-10-08 07:55:12,747][53852] Updated weights for policy 0, policy_version 2020 (0.0008) [2023-10-08 07:55:13,111][53852] Updated weights for policy 0, policy_version 2030 (0.0008) [2023-10-08 07:55:13,478][53852] Updated weights for policy 0, policy_version 2040 (0.0009) [2023-10-08 07:55:14,530][53885] Updated weights for policy 1, policy_version 2020 (0.0009) [2023-10-08 07:55:14,898][53885] Updated weights for policy 1, policy_version 2030 (0.0008) [2023-10-08 07:55:15,261][53885] Updated weights for policy 1, policy_version 2040 (0.0010) [2023-10-08 07:55:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 4194304. Throughput: 0: 1805.5, 1: 1791.7. Samples: 1056388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:55:17,016][52710] Avg episode reward: [(0, '3.360'), (1, '4.060')] [2023-10-08 07:55:17,195][53852] Updated weights for policy 0, policy_version 2050 (0.0008) [2023-10-08 07:55:17,568][53852] Updated weights for policy 0, policy_version 2060 (0.0007) [2023-10-08 07:55:17,947][53852] Updated weights for policy 0, policy_version 2070 (0.0007) [2023-10-08 07:55:18,320][53852] Updated weights for policy 0, policy_version 2080 (0.0011) [2023-10-08 07:55:19,154][53885] Updated weights for policy 1, policy_version 2050 (0.0008) [2023-10-08 07:55:19,524][53885] Updated weights for policy 1, policy_version 2060 (0.0008) [2023-10-08 07:55:19,884][53885] Updated weights for policy 1, policy_version 2070 (0.0007) [2023-10-08 07:55:20,251][53885] Updated weights for policy 1, policy_version 2080 (0.0008) [2023-10-08 07:55:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4259840. Throughput: 0: 1811.1, 1: 1786.4. Samples: 1078738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:55:22,016][52710] Avg episode reward: [(0, '3.290'), (1, '4.060')] [2023-10-08 07:55:22,048][53852] Updated weights for policy 0, policy_version 2090 (0.0009) [2023-10-08 07:55:22,419][53852] Updated weights for policy 0, policy_version 2100 (0.0009) [2023-10-08 07:55:22,788][53852] Updated weights for policy 0, policy_version 2110 (0.0008) [2023-10-08 07:55:23,991][53885] Updated weights for policy 1, policy_version 2090 (0.0008) [2023-10-08 07:55:24,365][53885] Updated weights for policy 1, policy_version 2100 (0.0008) [2023-10-08 07:55:24,737][53885] Updated weights for policy 1, policy_version 2110 (0.0008) [2023-10-08 07:55:26,404][53852] Updated weights for policy 0, policy_version 2120 (0.0007) [2023-10-08 07:55:26,780][53852] Updated weights for policy 0, policy_version 2130 (0.0008) [2023-10-08 07:55:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4325376. Throughput: 0: 1806.0, 1: 1793.7. Samples: 1089094. Policy #0 lag: (min: 26.0, avg: 28.1, max: 56.0) [2023-10-08 07:55:27,015][52710] Avg episode reward: [(0, '4.080'), (1, '3.860')] [2023-10-08 07:55:27,150][53852] Updated weights for policy 0, policy_version 2140 (0.0007) [2023-10-08 07:55:27,296][53500] Saving new best policy, reward=4.080! [2023-10-08 07:55:28,509][53885] Updated weights for policy 1, policy_version 2120 (0.0009) [2023-10-08 07:55:28,883][53885] Updated weights for policy 1, policy_version 2130 (0.0010) [2023-10-08 07:55:29,247][53885] Updated weights for policy 1, policy_version 2140 (0.0007) [2023-10-08 07:55:30,902][53852] Updated weights for policy 0, policy_version 2150 (0.0009) [2023-10-08 07:55:31,268][53852] Updated weights for policy 0, policy_version 2160 (0.0010) [2023-10-08 07:55:31,646][53852] Updated weights for policy 0, policy_version 2170 (0.0009) [2023-10-08 07:55:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 4423680. Throughput: 0: 1812.2, 1: 1794.3. Samples: 1111650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:55:32,016][52710] Avg episode reward: [(0, '3.700'), (1, '3.600')] [2023-10-08 07:55:32,808][53885] Updated weights for policy 1, policy_version 2150 (0.0009) [2023-10-08 07:55:33,178][53885] Updated weights for policy 1, policy_version 2160 (0.0008) [2023-10-08 07:55:33,542][53885] Updated weights for policy 1, policy_version 2170 (0.0009) [2023-10-08 07:55:35,374][53852] Updated weights for policy 0, policy_version 2180 (0.0009) [2023-10-08 07:55:35,752][53852] Updated weights for policy 0, policy_version 2190 (0.0011) [2023-10-08 07:55:36,126][53852] Updated weights for policy 0, policy_version 2200 (0.0008) [2023-10-08 07:55:37,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14329.0). Total num frames: 4489216. Throughput: 0: 1805.0, 1: 1803.6. Samples: 1132942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:55:37,016][52710] Avg episode reward: [(0, '3.650'), (1, '4.130')] [2023-10-08 07:55:37,164][53885] Updated weights for policy 1, policy_version 2180 (0.0008) [2023-10-08 07:55:37,538][53885] Updated weights for policy 1, policy_version 2190 (0.0007) [2023-10-08 07:55:37,908][53885] Updated weights for policy 1, policy_version 2200 (0.0009) [2023-10-08 07:55:38,206][53594] Saving new best policy, reward=4.130! [2023-10-08 07:55:39,919][53852] Updated weights for policy 0, policy_version 2210 (0.0007) [2023-10-08 07:55:40,290][53852] Updated weights for policy 0, policy_version 2220 (0.0010) [2023-10-08 07:55:40,665][53852] Updated weights for policy 0, policy_version 2230 (0.0010) [2023-10-08 07:55:41,038][53852] Updated weights for policy 0, policy_version 2240 (0.0010) [2023-10-08 07:55:41,841][53885] Updated weights for policy 1, policy_version 2210 (0.0008) [2023-10-08 07:55:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 4554752. Throughput: 0: 1809.5, 1: 1808.3. Samples: 1144512. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) [2023-10-08 07:55:42,016][52710] Avg episode reward: [(0, '4.200'), (1, '4.680')] [2023-10-08 07:55:42,017][53500] Saving new best policy, reward=4.200! [2023-10-08 07:55:42,209][53885] Updated weights for policy 1, policy_version 2220 (0.0008) [2023-10-08 07:55:42,580][53885] Updated weights for policy 1, policy_version 2230 (0.0008) [2023-10-08 07:55:42,948][53594] Saving new best policy, reward=4.680! [2023-10-08 07:55:42,950][53885] Updated weights for policy 1, policy_version 2240 (0.0007) [2023-10-08 07:55:44,709][53852] Updated weights for policy 0, policy_version 2250 (0.0007) [2023-10-08 07:55:45,087][53852] Updated weights for policy 0, policy_version 2260 (0.0008) [2023-10-08 07:55:45,460][53852] Updated weights for policy 0, policy_version 2270 (0.0007) [2023-10-08 07:55:46,593][53885] Updated weights for policy 1, policy_version 2250 (0.0007) [2023-10-08 07:55:46,965][53885] Updated weights for policy 1, policy_version 2260 (0.0008) [2023-10-08 07:55:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 4620288. Throughput: 0: 1803.2, 1: 1800.4. Samples: 1165444. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 07:55:47,016][52710] Avg episode reward: [(0, '3.950'), (1, '4.250')] [2023-10-08 07:55:47,340][53885] Updated weights for policy 1, policy_version 2270 (0.0009) [2023-10-08 07:55:49,293][53852] Updated weights for policy 0, policy_version 2280 (0.0010) [2023-10-08 07:55:49,678][53852] Updated weights for policy 0, policy_version 2290 (0.0011) [2023-10-08 07:55:50,051][53852] Updated weights for policy 0, policy_version 2300 (0.0009) [2023-10-08 07:55:51,065][53885] Updated weights for policy 1, policy_version 2280 (0.0009) [2023-10-08 07:55:51,428][53885] Updated weights for policy 1, policy_version 2290 (0.0008) [2023-10-08 07:55:51,791][53885] Updated weights for policy 1, policy_version 2300 (0.0010) [2023-10-08 07:55:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 4718592. Throughput: 0: 1801.2, 1: 1814.5. Samples: 1186852. Policy #0 lag: (min: 9.0, avg: 19.6, max: 41.0) [2023-10-08 07:55:52,016][52710] Avg episode reward: [(0, '4.790'), (1, '4.450')] [2023-10-08 07:55:52,029][53500] Saving new best policy, reward=4.790! [2023-10-08 07:55:53,707][53852] Updated weights for policy 0, policy_version 2310 (0.0010) [2023-10-08 07:55:54,076][53852] Updated weights for policy 0, policy_version 2320 (0.0010) [2023-10-08 07:55:54,450][53852] Updated weights for policy 0, policy_version 2330 (0.0007) [2023-10-08 07:55:55,600][53885] Updated weights for policy 1, policy_version 2310 (0.0009) [2023-10-08 07:55:55,972][53885] Updated weights for policy 1, policy_version 2320 (0.0008) [2023-10-08 07:55:56,342][53885] Updated weights for policy 1, policy_version 2330 (0.0007) [2023-10-08 07:55:57,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 4784128. Throughput: 0: 1805.6, 1: 1797.6. Samples: 1197876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:55:57,016][52710] Avg episode reward: [(0, '4.360'), (1, '4.820')] [2023-10-08 07:55:57,017][53594] Saving new best policy, reward=4.820! [2023-10-08 07:55:58,031][53852] Updated weights for policy 0, policy_version 2340 (0.0009) [2023-10-08 07:55:58,393][53852] Updated weights for policy 0, policy_version 2350 (0.0008) [2023-10-08 07:55:58,760][53852] Updated weights for policy 0, policy_version 2360 (0.0010) [2023-10-08 07:55:59,950][53885] Updated weights for policy 1, policy_version 2340 (0.0007) [2023-10-08 07:56:00,318][53885] Updated weights for policy 1, policy_version 2350 (0.0007) [2023-10-08 07:56:00,686][53885] Updated weights for policy 1, policy_version 2360 (0.0008) [2023-10-08 07:56:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4849664. Throughput: 0: 1804.0, 1: 1820.2. Samples: 1219474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:56:02,016][52710] Avg episode reward: [(0, '3.850'), (1, '5.450')] [2023-10-08 07:56:02,018][53594] Saving new best policy, reward=5.450! [2023-10-08 07:56:02,509][53852] Updated weights for policy 0, policy_version 2370 (0.0009) [2023-10-08 07:56:02,880][53852] Updated weights for policy 0, policy_version 2380 (0.0008) [2023-10-08 07:56:03,252][53852] Updated weights for policy 0, policy_version 2390 (0.0007) [2023-10-08 07:56:03,621][53852] Updated weights for policy 0, policy_version 2400 (0.0009) [2023-10-08 07:56:04,502][53885] Updated weights for policy 1, policy_version 2370 (0.0009) [2023-10-08 07:56:04,877][53885] Updated weights for policy 1, policy_version 2380 (0.0007) [2023-10-08 07:56:05,246][53885] Updated weights for policy 1, policy_version 2390 (0.0008) [2023-10-08 07:56:05,606][53885] Updated weights for policy 1, policy_version 2400 (0.0008) [2023-10-08 07:56:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 4915200. Throughput: 0: 1806.7, 1: 1812.4. Samples: 1241602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:56:07,016][52710] Avg episode reward: [(0, '4.190'), (1, '5.540')] [2023-10-08 07:56:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000002400_2457600.pth... [2023-10-08 07:56:07,060][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000000704_720896.pth [2023-10-08 07:56:07,063][53594] Saving new best policy, reward=5.540! [2023-10-08 07:56:07,305][53852] Updated weights for policy 0, policy_version 2410 (0.0007) [2023-10-08 07:56:07,675][53852] Updated weights for policy 0, policy_version 2420 (0.0008) [2023-10-08 07:56:08,045][53852] Updated weights for policy 0, policy_version 2430 (0.0010) [2023-10-08 07:56:08,114][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth... [2023-10-08 07:56:08,156][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000000736_753664.pth [2023-10-08 07:56:09,336][53885] Updated weights for policy 1, policy_version 2410 (0.0009) [2023-10-08 07:56:09,709][53885] Updated weights for policy 1, policy_version 2420 (0.0007) [2023-10-08 07:56:10,072][53885] Updated weights for policy 1, policy_version 2430 (0.0008) [2023-10-08 07:56:11,759][53852] Updated weights for policy 0, policy_version 2440 (0.0009) [2023-10-08 07:56:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4980736. Throughput: 0: 1803.5, 1: 1820.4. Samples: 1252170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:56:12,016][52710] Avg episode reward: [(0, '4.610'), (1, '5.510')] [2023-10-08 07:56:12,122][53852] Updated weights for policy 0, policy_version 2450 (0.0007) [2023-10-08 07:56:12,490][53852] Updated weights for policy 0, policy_version 2460 (0.0007) [2023-10-08 07:56:13,720][53885] Updated weights for policy 1, policy_version 2440 (0.0011) [2023-10-08 07:56:14,089][53885] Updated weights for policy 1, policy_version 2450 (0.0009) [2023-10-08 07:56:14,463][53885] Updated weights for policy 1, policy_version 2460 (0.0009) [2023-10-08 07:56:16,108][53852] Updated weights for policy 0, policy_version 2470 (0.0010) [2023-10-08 07:56:16,483][53852] Updated weights for policy 0, policy_version 2480 (0.0008) [2023-10-08 07:56:16,860][53852] Updated weights for policy 0, policy_version 2490 (0.0008) [2023-10-08 07:56:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5046272. Throughput: 0: 1803.0, 1: 1811.2. Samples: 1274290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:56:17,016][52710] Avg episode reward: [(0, '4.300'), (1, '5.680')] [2023-10-08 07:56:17,017][53594] Saving new best policy, reward=5.680! [2023-10-08 07:56:18,180][53885] Updated weights for policy 1, policy_version 2470 (0.0011) [2023-10-08 07:56:18,544][53885] Updated weights for policy 1, policy_version 2480 (0.0010) [2023-10-08 07:56:18,923][53885] Updated weights for policy 1, policy_version 2490 (0.0011) [2023-10-08 07:56:20,416][53852] Updated weights for policy 0, policy_version 2500 (0.0009) [2023-10-08 07:56:20,785][53852] Updated weights for policy 0, policy_version 2510 (0.0009) [2023-10-08 07:56:21,154][53852] Updated weights for policy 0, policy_version 2520 (0.0008) [2023-10-08 07:56:22,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 5144576. Throughput: 0: 1806.1, 1: 1808.3. Samples: 1295588. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-08 07:56:22,016][52710] Avg episode reward: [(0, '4.000'), (1, '5.200')] [2023-10-08 07:56:22,610][53885] Updated weights for policy 1, policy_version 2500 (0.0010) [2023-10-08 07:56:22,986][53885] Updated weights for policy 1, policy_version 2510 (0.0007) [2023-10-08 07:56:23,358][53885] Updated weights for policy 1, policy_version 2520 (0.0008) [2023-10-08 07:56:24,941][53852] Updated weights for policy 0, policy_version 2530 (0.0009) [2023-10-08 07:56:25,310][53852] Updated weights for policy 0, policy_version 2540 (0.0007) [2023-10-08 07:56:25,676][53852] Updated weights for policy 0, policy_version 2550 (0.0010) [2023-10-08 07:56:26,049][53852] Updated weights for policy 0, policy_version 2560 (0.0008) [2023-10-08 07:56:27,002][53885] Updated weights for policy 1, policy_version 2530 (0.0008) [2023-10-08 07:56:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 5210112. Throughput: 0: 1809.3, 1: 1801.0. Samples: 1306978. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-08 07:56:27,016][52710] Avg episode reward: [(0, '5.220'), (1, '5.450')] [2023-10-08 07:56:27,016][53500] Saving new best policy, reward=5.220! [2023-10-08 07:56:27,373][53885] Updated weights for policy 1, policy_version 2540 (0.0007) [2023-10-08 07:56:27,746][53885] Updated weights for policy 1, policy_version 2550 (0.0008) [2023-10-08 07:56:28,114][53885] Updated weights for policy 1, policy_version 2560 (0.0009) [2023-10-08 07:56:29,609][53852] Updated weights for policy 0, policy_version 2570 (0.0007) [2023-10-08 07:56:29,989][53852] Updated weights for policy 0, policy_version 2580 (0.0010) [2023-10-08 07:56:30,368][53852] Updated weights for policy 0, policy_version 2590 (0.0009) [2023-10-08 07:56:31,745][53885] Updated weights for policy 1, policy_version 2570 (0.0009) [2023-10-08 07:56:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 5275648. Throughput: 0: 1816.7, 1: 1808.9. Samples: 1328596. Policy #0 lag: (min: 31.0, avg: 40.3, max: 63.0) [2023-10-08 07:56:32,016][52710] Avg episode reward: [(0, '5.050'), (1, '5.240')] [2023-10-08 07:56:32,113][53885] Updated weights for policy 1, policy_version 2580 (0.0008) [2023-10-08 07:56:32,466][53885] Updated weights for policy 1, policy_version 2590 (0.0007) [2023-10-08 07:56:34,193][53852] Updated weights for policy 0, policy_version 2600 (0.0011) [2023-10-08 07:56:34,567][53852] Updated weights for policy 0, policy_version 2610 (0.0010) [2023-10-08 07:56:34,939][53852] Updated weights for policy 0, policy_version 2620 (0.0009) [2023-10-08 07:56:36,396][53885] Updated weights for policy 1, policy_version 2600 (0.0007) [2023-10-08 07:56:36,772][53885] Updated weights for policy 1, policy_version 2610 (0.0010) [2023-10-08 07:56:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 5341184. Throughput: 0: 1819.1, 1: 1811.3. Samples: 1350220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:56:37,016][52710] Avg episode reward: [(0, '5.280'), (1, '5.620')] [2023-10-08 07:56:37,022][53500] Saving new best policy, reward=5.280! [2023-10-08 07:56:37,143][53885] Updated weights for policy 1, policy_version 2620 (0.0009) [2023-10-08 07:56:38,798][53852] Updated weights for policy 0, policy_version 2630 (0.0008) [2023-10-08 07:56:39,163][53852] Updated weights for policy 0, policy_version 2640 (0.0009) [2023-10-08 07:56:39,539][53852] Updated weights for policy 0, policy_version 2650 (0.0008) [2023-10-08 07:56:40,717][53885] Updated weights for policy 1, policy_version 2630 (0.0011) [2023-10-08 07:56:41,086][53885] Updated weights for policy 1, policy_version 2640 (0.0008) [2023-10-08 07:56:41,456][53885] Updated weights for policy 1, policy_version 2650 (0.0007) [2023-10-08 07:56:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 5439488. Throughput: 0: 1819.8, 1: 1809.6. Samples: 1361200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:56:42,016][52710] Avg episode reward: [(0, '5.610'), (1, '5.340')] [2023-10-08 07:56:42,017][53500] Saving new best policy, reward=5.610! [2023-10-08 07:56:43,150][53852] Updated weights for policy 0, policy_version 2660 (0.0007) [2023-10-08 07:56:43,523][53852] Updated weights for policy 0, policy_version 2670 (0.0007) [2023-10-08 07:56:43,894][53852] Updated weights for policy 0, policy_version 2680 (0.0008) [2023-10-08 07:56:45,140][53885] Updated weights for policy 1, policy_version 2660 (0.0009) [2023-10-08 07:56:45,517][53885] Updated weights for policy 1, policy_version 2670 (0.0008) [2023-10-08 07:56:45,882][53885] Updated weights for policy 1, policy_version 2680 (0.0009) [2023-10-08 07:56:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 5505024. Throughput: 0: 1822.9, 1: 1814.3. Samples: 1383148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:56:47,015][52710] Avg episode reward: [(0, '5.250'), (1, '5.480')] [2023-10-08 07:56:47,567][53852] Updated weights for policy 0, policy_version 2690 (0.0007) [2023-10-08 07:56:47,934][53852] Updated weights for policy 0, policy_version 2700 (0.0008) [2023-10-08 07:56:48,307][53852] Updated weights for policy 0, policy_version 2710 (0.0007) [2023-10-08 07:56:48,673][53852] Updated weights for policy 0, policy_version 2720 (0.0007) [2023-10-08 07:56:49,567][53885] Updated weights for policy 1, policy_version 2690 (0.0009) [2023-10-08 07:56:49,940][53885] Updated weights for policy 1, policy_version 2700 (0.0008) [2023-10-08 07:56:50,306][53885] Updated weights for policy 1, policy_version 2710 (0.0010) [2023-10-08 07:56:50,676][53885] Updated weights for policy 1, policy_version 2720 (0.0010) [2023-10-08 07:56:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5570560. Throughput: 0: 1829.8, 1: 1808.4. Samples: 1405320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:56:52,015][52710] Avg episode reward: [(0, '5.160'), (1, '5.720')] [2023-10-08 07:56:52,022][53594] Saving new best policy, reward=5.720! [2023-10-08 07:56:52,209][53852] Updated weights for policy 0, policy_version 2730 (0.0007) [2023-10-08 07:56:52,584][53852] Updated weights for policy 0, policy_version 2740 (0.0007) [2023-10-08 07:56:52,955][53852] Updated weights for policy 0, policy_version 2750 (0.0007) [2023-10-08 07:56:54,409][53885] Updated weights for policy 1, policy_version 2730 (0.0007) [2023-10-08 07:56:54,778][53885] Updated weights for policy 1, policy_version 2740 (0.0008) [2023-10-08 07:56:55,134][53885] Updated weights for policy 1, policy_version 2750 (0.0007) [2023-10-08 07:56:56,486][53852] Updated weights for policy 0, policy_version 2760 (0.0009) [2023-10-08 07:56:56,864][53852] Updated weights for policy 0, policy_version 2770 (0.0009) [2023-10-08 07:56:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5636096. Throughput: 0: 1833.5, 1: 1813.6. Samples: 1416290. Policy #0 lag: (min: 15.0, avg: 31.0, max: 47.0) [2023-10-08 07:56:57,016][52710] Avg episode reward: [(0, '6.070'), (1, '6.020')] [2023-10-08 07:56:57,016][53594] Saving new best policy, reward=6.020! [2023-10-08 07:56:57,245][53852] Updated weights for policy 0, policy_version 2780 (0.0008) [2023-10-08 07:56:57,391][53500] Saving new best policy, reward=6.070! [2023-10-08 07:56:58,910][53885] Updated weights for policy 1, policy_version 2760 (0.0011) [2023-10-08 07:56:59,274][53885] Updated weights for policy 1, policy_version 2770 (0.0010) [2023-10-08 07:56:59,647][53885] Updated weights for policy 1, policy_version 2780 (0.0007) [2023-10-08 07:57:00,865][53852] Updated weights for policy 0, policy_version 2790 (0.0008) [2023-10-08 07:57:01,243][53852] Updated weights for policy 0, policy_version 2800 (0.0009) [2023-10-08 07:57:01,621][53852] Updated weights for policy 0, policy_version 2810 (0.0009) [2023-10-08 07:57:02,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5734400. Throughput: 0: 1838.9, 1: 1806.7. Samples: 1438344. Policy #0 lag: (min: 25.0, avg: 34.8, max: 57.0) [2023-10-08 07:57:02,016][52710] Avg episode reward: [(0, '5.970'), (1, '5.960')] [2023-10-08 07:57:03,402][53885] Updated weights for policy 1, policy_version 2790 (0.0009) [2023-10-08 07:57:03,777][53885] Updated weights for policy 1, policy_version 2800 (0.0007) [2023-10-08 07:57:04,157][53885] Updated weights for policy 1, policy_version 2810 (0.0008) [2023-10-08 07:57:05,227][53852] Updated weights for policy 0, policy_version 2820 (0.0010) [2023-10-08 07:57:05,597][53852] Updated weights for policy 0, policy_version 2830 (0.0009) [2023-10-08 07:57:05,966][53852] Updated weights for policy 0, policy_version 2840 (0.0011) [2023-10-08 07:57:07,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5799936. Throughput: 0: 1840.8, 1: 1805.7. Samples: 1459682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:57:07,016][52710] Avg episode reward: [(0, '5.460'), (1, '6.190')] [2023-10-08 07:57:07,027][53594] Saving new best policy, reward=6.190! [2023-10-08 07:57:07,840][53885] Updated weights for policy 1, policy_version 2820 (0.0008) [2023-10-08 07:57:08,197][53885] Updated weights for policy 1, policy_version 2830 (0.0007) [2023-10-08 07:57:08,568][53885] Updated weights for policy 1, policy_version 2840 (0.0008) [2023-10-08 07:57:09,526][53852] Updated weights for policy 0, policy_version 2850 (0.0009) [2023-10-08 07:57:09,894][53852] Updated weights for policy 0, policy_version 2860 (0.0008) [2023-10-08 07:57:10,272][53852] Updated weights for policy 0, policy_version 2870 (0.0007) [2023-10-08 07:57:10,636][53852] Updated weights for policy 0, policy_version 2880 (0.0010) [2023-10-08 07:57:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 5865472. Throughput: 0: 1842.0, 1: 1805.9. Samples: 1471130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:57:12,016][52710] Avg episode reward: [(0, '5.650'), (1, '5.430')] [2023-10-08 07:57:12,283][53885] Updated weights for policy 1, policy_version 2850 (0.0009) [2023-10-08 07:57:12,652][53885] Updated weights for policy 1, policy_version 2860 (0.0008) [2023-10-08 07:57:13,020][53885] Updated weights for policy 1, policy_version 2870 (0.0007) [2023-10-08 07:57:13,387][53885] Updated weights for policy 1, policy_version 2880 (0.0009) [2023-10-08 07:57:14,368][53852] Updated weights for policy 0, policy_version 2890 (0.0008) [2023-10-08 07:57:14,747][53852] Updated weights for policy 0, policy_version 2900 (0.0008) [2023-10-08 07:57:15,117][53852] Updated weights for policy 0, policy_version 2910 (0.0008) [2023-10-08 07:57:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 5931008. Throughput: 0: 1838.2, 1: 1805.9. Samples: 1492578. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 07:57:17,016][52710] Avg episode reward: [(0, '6.370'), (1, '6.250')] [2023-10-08 07:57:17,018][53500] Saving new best policy, reward=6.370! [2023-10-08 07:57:17,159][53885] Updated weights for policy 1, policy_version 2890 (0.0010) [2023-10-08 07:57:17,517][53885] Updated weights for policy 1, policy_version 2900 (0.0007) [2023-10-08 07:57:17,887][53885] Updated weights for policy 1, policy_version 2910 (0.0009) [2023-10-08 07:57:17,953][53594] Saving new best policy, reward=6.250! [2023-10-08 07:57:18,709][53852] Updated weights for policy 0, policy_version 2920 (0.0008) [2023-10-08 07:57:19,093][53852] Updated weights for policy 0, policy_version 2930 (0.0009) [2023-10-08 07:57:19,476][53852] Updated weights for policy 0, policy_version 2940 (0.0010) [2023-10-08 07:57:21,520][53885] Updated weights for policy 1, policy_version 2920 (0.0008) [2023-10-08 07:57:21,892][53885] Updated weights for policy 1, policy_version 2930 (0.0009) [2023-10-08 07:57:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 5996544. Throughput: 0: 1850.9, 1: 1812.9. Samples: 1515092. Policy #0 lag: (min: 3.0, avg: 11.9, max: 35.0) [2023-10-08 07:57:22,016][52710] Avg episode reward: [(0, '6.160'), (1, '6.230')] [2023-10-08 07:57:22,276][53885] Updated weights for policy 1, policy_version 2940 (0.0009) [2023-10-08 07:57:23,103][53852] Updated weights for policy 0, policy_version 2950 (0.0010) [2023-10-08 07:57:23,482][53852] Updated weights for policy 0, policy_version 2960 (0.0010) [2023-10-08 07:57:23,857][53852] Updated weights for policy 0, policy_version 2970 (0.0009) [2023-10-08 07:57:25,909][53885] Updated weights for policy 1, policy_version 2950 (0.0010) [2023-10-08 07:57:26,283][53885] Updated weights for policy 1, policy_version 2960 (0.0008) [2023-10-08 07:57:26,652][53885] Updated weights for policy 1, policy_version 2970 (0.0008) [2023-10-08 07:57:27,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 6094848. Throughput: 0: 1844.5, 1: 1808.5. Samples: 1525582. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 07:57:27,016][52710] Avg episode reward: [(0, '6.250'), (1, '5.660')] [2023-10-08 07:57:27,304][53852] Updated weights for policy 0, policy_version 2980 (0.0009) [2023-10-08 07:57:27,680][53852] Updated weights for policy 0, policy_version 2990 (0.0008) [2023-10-08 07:57:28,055][53852] Updated weights for policy 0, policy_version 3000 (0.0008) [2023-10-08 07:57:30,377][53885] Updated weights for policy 1, policy_version 2980 (0.0008) [2023-10-08 07:57:30,752][53885] Updated weights for policy 1, policy_version 2990 (0.0009) [2023-10-08 07:57:31,121][53885] Updated weights for policy 1, policy_version 3000 (0.0007) [2023-10-08 07:57:31,677][53852] Updated weights for policy 0, policy_version 3010 (0.0009) [2023-10-08 07:57:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 6160384. Throughput: 0: 1852.8, 1: 1817.0. Samples: 1548288. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) [2023-10-08 07:57:32,016][52710] Avg episode reward: [(0, '6.530'), (1, '6.210')] [2023-10-08 07:57:32,044][53852] Updated weights for policy 0, policy_version 3020 (0.0008) [2023-10-08 07:57:32,419][53852] Updated weights for policy 0, policy_version 3030 (0.0009) [2023-10-08 07:57:32,793][53500] Saving new best policy, reward=6.530! [2023-10-08 07:57:32,793][53852] Updated weights for policy 0, policy_version 3040 (0.0007) [2023-10-08 07:57:34,751][53885] Updated weights for policy 1, policy_version 3010 (0.0009) [2023-10-08 07:57:35,121][53885] Updated weights for policy 1, policy_version 3020 (0.0007) [2023-10-08 07:57:35,480][53885] Updated weights for policy 1, policy_version 3030 (0.0010) [2023-10-08 07:57:35,847][53885] Updated weights for policy 1, policy_version 3040 (0.0009) [2023-10-08 07:57:36,348][53852] Updated weights for policy 0, policy_version 3050 (0.0009) [2023-10-08 07:57:36,720][53852] Updated weights for policy 0, policy_version 3060 (0.0007) [2023-10-08 07:57:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 6225920. Throughput: 0: 1836.7, 1: 1817.8. Samples: 1569774. Policy #0 lag: (min: 31.0, avg: 31.2, max: 39.0) [2023-10-08 07:57:37,016][52710] Avg episode reward: [(0, '7.190'), (1, '6.510')] [2023-10-08 07:57:37,023][53594] Saving new best policy, reward=6.510! [2023-10-08 07:57:37,101][53852] Updated weights for policy 0, policy_version 3070 (0.0007) [2023-10-08 07:57:37,168][53500] Saving new best policy, reward=7.190! [2023-10-08 07:57:39,520][53885] Updated weights for policy 1, policy_version 3050 (0.0009) [2023-10-08 07:57:39,890][53885] Updated weights for policy 1, policy_version 3060 (0.0010) [2023-10-08 07:57:40,262][53885] Updated weights for policy 1, policy_version 3070 (0.0010) [2023-10-08 07:57:40,658][53852] Updated weights for policy 0, policy_version 3080 (0.0008) [2023-10-08 07:57:41,032][53852] Updated weights for policy 0, policy_version 3090 (0.0008) [2023-10-08 07:57:41,403][53852] Updated weights for policy 0, policy_version 3100 (0.0010) [2023-10-08 07:57:42,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6324224. Throughput: 0: 1850.2, 1: 1820.9. Samples: 1581490. Policy #0 lag: (min: 3.0, avg: 8.0, max: 35.0) [2023-10-08 07:57:42,016][52710] Avg episode reward: [(0, '7.030'), (1, '6.180')] [2023-10-08 07:57:43,948][53885] Updated weights for policy 1, policy_version 3080 (0.0009) [2023-10-08 07:57:44,318][53885] Updated weights for policy 1, policy_version 3090 (0.0011) [2023-10-08 07:57:44,693][53885] Updated weights for policy 1, policy_version 3100 (0.0007) [2023-10-08 07:57:45,082][53852] Updated weights for policy 0, policy_version 3110 (0.0007) [2023-10-08 07:57:45,454][53852] Updated weights for policy 0, policy_version 3120 (0.0008) [2023-10-08 07:57:45,822][53852] Updated weights for policy 0, policy_version 3130 (0.0010) [2023-10-08 07:57:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6389760. Throughput: 0: 1829.4, 1: 1822.9. Samples: 1602698. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 07:57:47,016][52710] Avg episode reward: [(0, '7.060'), (1, '6.070')] [2023-10-08 07:57:48,212][53885] Updated weights for policy 1, policy_version 3110 (0.0008) [2023-10-08 07:57:48,591][53885] Updated weights for policy 1, policy_version 3120 (0.0010) [2023-10-08 07:57:48,956][53885] Updated weights for policy 1, policy_version 3130 (0.0010) [2023-10-08 07:57:49,648][53852] Updated weights for policy 0, policy_version 3140 (0.0010) [2023-10-08 07:57:50,012][53852] Updated weights for policy 0, policy_version 3150 (0.0007) [2023-10-08 07:57:50,378][53852] Updated weights for policy 0, policy_version 3160 (0.0009) [2023-10-08 07:57:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6455296. Throughput: 0: 1841.8, 1: 1827.1. Samples: 1624782. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 07:57:52,016][52710] Avg episode reward: [(0, '8.290'), (1, '6.310')] [2023-10-08 07:57:52,028][53500] Saving new best policy, reward=8.290! [2023-10-08 07:57:52,883][53885] Updated weights for policy 1, policy_version 3140 (0.0009) [2023-10-08 07:57:53,244][53885] Updated weights for policy 1, policy_version 3150 (0.0008) [2023-10-08 07:57:53,618][53885] Updated weights for policy 1, policy_version 3160 (0.0008) [2023-10-08 07:57:54,143][53852] Updated weights for policy 0, policy_version 3170 (0.0007) [2023-10-08 07:57:54,515][53852] Updated weights for policy 0, policy_version 3180 (0.0008) [2023-10-08 07:57:54,891][53852] Updated weights for policy 0, policy_version 3190 (0.0007) [2023-10-08 07:57:55,259][53852] Updated weights for policy 0, policy_version 3200 (0.0007) [2023-10-08 07:57:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6520832. Throughput: 0: 1824.8, 1: 1827.5. Samples: 1635486. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-08 07:57:57,016][52710] Avg episode reward: [(0, '7.270'), (1, '6.560')] [2023-10-08 07:57:57,018][53594] Saving new best policy, reward=6.560! [2023-10-08 07:57:57,357][53885] Updated weights for policy 1, policy_version 3170 (0.0007) [2023-10-08 07:57:57,726][53885] Updated weights for policy 1, policy_version 3180 (0.0007) [2023-10-08 07:57:58,102][53885] Updated weights for policy 1, policy_version 3190 (0.0007) [2023-10-08 07:57:58,468][53885] Updated weights for policy 1, policy_version 3200 (0.0007) [2023-10-08 07:57:59,024][53852] Updated weights for policy 0, policy_version 3210 (0.0009) [2023-10-08 07:57:59,390][53852] Updated weights for policy 0, policy_version 3220 (0.0008) [2023-10-08 07:57:59,757][53852] Updated weights for policy 0, policy_version 3230 (0.0008) [2023-10-08 07:58:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6586368. Throughput: 0: 1837.9, 1: 1822.8. Samples: 1657306. Policy #0 lag: (min: 29.0, avg: 31.9, max: 61.0) [2023-10-08 07:58:02,016][52710] Avg episode reward: [(0, '7.010'), (1, '6.890')] [2023-10-08 07:58:02,036][53885] Updated weights for policy 1, policy_version 3210 (0.0007) [2023-10-08 07:58:02,408][53885] Updated weights for policy 1, policy_version 3220 (0.0009) [2023-10-08 07:58:02,775][53885] Updated weights for policy 1, policy_version 3230 (0.0007) [2023-10-08 07:58:02,840][53594] Saving new best policy, reward=6.890! [2023-10-08 07:58:03,405][53852] Updated weights for policy 0, policy_version 3240 (0.0007) [2023-10-08 07:58:03,781][53852] Updated weights for policy 0, policy_version 3250 (0.0007) [2023-10-08 07:58:04,152][53852] Updated weights for policy 0, policy_version 3260 (0.0008) [2023-10-08 07:58:06,562][53885] Updated weights for policy 1, policy_version 3240 (0.0008) [2023-10-08 07:58:06,938][53885] Updated weights for policy 1, policy_version 3250 (0.0011) [2023-10-08 07:58:07,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6651904. Throughput: 0: 1829.6, 1: 1824.4. Samples: 1679518. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:58:07,016][52710] Avg episode reward: [(0, '7.390'), (1, '7.010')] [2023-10-08 07:58:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000003264_3342336.pth... [2023-10-08 07:58:07,057][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth [2023-10-08 07:58:07,300][53885] Updated weights for policy 1, policy_version 3260 (0.0009) [2023-10-08 07:58:07,445][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000003264_3342336.pth... [2023-10-08 07:58:07,473][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000001568_1605632.pth [2023-10-08 07:58:07,476][53594] Saving new best policy, reward=7.010! [2023-10-08 07:58:07,735][53852] Updated weights for policy 0, policy_version 3270 (0.0008) [2023-10-08 07:58:08,117][53852] Updated weights for policy 0, policy_version 3280 (0.0009) [2023-10-08 07:58:08,487][53852] Updated weights for policy 0, policy_version 3290 (0.0007) [2023-10-08 07:58:11,035][53885] Updated weights for policy 1, policy_version 3270 (0.0008) [2023-10-08 07:58:11,405][53885] Updated weights for policy 1, policy_version 3280 (0.0008) [2023-10-08 07:58:11,777][53885] Updated weights for policy 1, policy_version 3290 (0.0009) [2023-10-08 07:58:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6750208. Throughput: 0: 1827.1, 1: 1821.3. Samples: 1689760. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 07:58:12,016][52710] Avg episode reward: [(0, '6.680'), (1, '7.180')] [2023-10-08 07:58:12,016][53594] Saving new best policy, reward=7.180! [2023-10-08 07:58:12,248][53852] Updated weights for policy 0, policy_version 3300 (0.0009) [2023-10-08 07:58:12,618][53852] Updated weights for policy 0, policy_version 3310 (0.0008) [2023-10-08 07:58:12,983][53852] Updated weights for policy 0, policy_version 3320 (0.0007) [2023-10-08 07:58:15,480][53885] Updated weights for policy 1, policy_version 3300 (0.0008) [2023-10-08 07:58:15,851][53885] Updated weights for policy 1, policy_version 3310 (0.0008) [2023-10-08 07:58:16,211][53885] Updated weights for policy 1, policy_version 3320 (0.0007) [2023-10-08 07:58:16,629][53852] Updated weights for policy 0, policy_version 3330 (0.0010) [2023-10-08 07:58:17,009][53852] Updated weights for policy 0, policy_version 3340 (0.0009) [2023-10-08 07:58:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6815744. Throughput: 0: 1820.2, 1: 1816.6. Samples: 1711942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:58:17,016][52710] Avg episode reward: [(0, '6.690'), (1, '6.560')] [2023-10-08 07:58:17,384][53852] Updated weights for policy 0, policy_version 3350 (0.0009) [2023-10-08 07:58:17,757][53852] Updated weights for policy 0, policy_version 3360 (0.0008) [2023-10-08 07:58:20,031][53885] Updated weights for policy 1, policy_version 3330 (0.0007) [2023-10-08 07:58:20,402][53885] Updated weights for policy 1, policy_version 3340 (0.0008) [2023-10-08 07:58:20,761][53885] Updated weights for policy 1, policy_version 3350 (0.0007) [2023-10-08 07:58:21,133][53885] Updated weights for policy 1, policy_version 3360 (0.0007) [2023-10-08 07:58:21,453][53852] Updated weights for policy 0, policy_version 3370 (0.0008) [2023-10-08 07:58:21,822][53852] Updated weights for policy 0, policy_version 3380 (0.0008) [2023-10-08 07:58:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 6881280. Throughput: 0: 1821.9, 1: 1808.6. Samples: 1733146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:58:22,015][52710] Avg episode reward: [(0, '7.080'), (1, '6.660')] [2023-10-08 07:58:22,192][53852] Updated weights for policy 0, policy_version 3390 (0.0009) [2023-10-08 07:58:24,830][53885] Updated weights for policy 1, policy_version 3370 (0.0009) [2023-10-08 07:58:25,201][53885] Updated weights for policy 1, policy_version 3380 (0.0007) [2023-10-08 07:58:25,559][53885] Updated weights for policy 1, policy_version 3390 (0.0009) [2023-10-08 07:58:25,846][53852] Updated weights for policy 0, policy_version 3400 (0.0010) [2023-10-08 07:58:26,218][53852] Updated weights for policy 0, policy_version 3410 (0.0010) [2023-10-08 07:58:26,595][53852] Updated weights for policy 0, policy_version 3420 (0.0010) [2023-10-08 07:58:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 6979584. Throughput: 0: 1817.7, 1: 1811.2. Samples: 1744792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:58:27,016][52710] Avg episode reward: [(0, '7.720'), (1, '7.010')] [2023-10-08 07:58:29,387][53885] Updated weights for policy 1, policy_version 3400 (0.0008) [2023-10-08 07:58:29,766][53885] Updated weights for policy 1, policy_version 3410 (0.0010) [2023-10-08 07:58:30,125][53852] Updated weights for policy 0, policy_version 3430 (0.0008) [2023-10-08 07:58:30,139][53885] Updated weights for policy 1, policy_version 3420 (0.0008) [2023-10-08 07:58:30,496][53852] Updated weights for policy 0, policy_version 3440 (0.0008) [2023-10-08 07:58:30,872][53852] Updated weights for policy 0, policy_version 3450 (0.0010) [2023-10-08 07:58:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7045120. Throughput: 0: 1820.0, 1: 1800.2. Samples: 1765606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:58:32,016][52710] Avg episode reward: [(0, '7.470'), (1, '7.720')] [2023-10-08 07:58:32,017][53594] Saving new best policy, reward=7.720! [2023-10-08 07:58:33,784][53885] Updated weights for policy 1, policy_version 3430 (0.0008) [2023-10-08 07:58:34,157][53885] Updated weights for policy 1, policy_version 3440 (0.0008) [2023-10-08 07:58:34,531][53885] Updated weights for policy 1, policy_version 3450 (0.0007) [2023-10-08 07:58:34,710][53852] Updated weights for policy 0, policy_version 3460 (0.0009) [2023-10-08 07:58:35,090][53852] Updated weights for policy 0, policy_version 3470 (0.0008) [2023-10-08 07:58:35,461][53852] Updated weights for policy 0, policy_version 3480 (0.0010) [2023-10-08 07:58:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7110656. Throughput: 0: 1819.6, 1: 1797.2. Samples: 1787538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:58:37,016][52710] Avg episode reward: [(0, '7.660'), (1, '7.680')] [2023-10-08 07:58:38,158][53885] Updated weights for policy 1, policy_version 3460 (0.0008) [2023-10-08 07:58:38,537][53885] Updated weights for policy 1, policy_version 3470 (0.0009) [2023-10-08 07:58:38,901][53885] Updated weights for policy 1, policy_version 3480 (0.0011) [2023-10-08 07:58:39,158][53852] Updated weights for policy 0, policy_version 3490 (0.0009) [2023-10-08 07:58:39,524][53852] Updated weights for policy 0, policy_version 3500 (0.0007) [2023-10-08 07:58:39,891][53852] Updated weights for policy 0, policy_version 3510 (0.0009) [2023-10-08 07:58:40,271][53852] Updated weights for policy 0, policy_version 3520 (0.0010) [2023-10-08 07:58:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7176192. Throughput: 0: 1821.2, 1: 1797.7. Samples: 1798334. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-08 07:58:42,016][52710] Avg episode reward: [(0, '7.820'), (1, '7.390')] [2023-10-08 07:58:42,569][53885] Updated weights for policy 1, policy_version 3490 (0.0008) [2023-10-08 07:58:42,932][53885] Updated weights for policy 1, policy_version 3500 (0.0008) [2023-10-08 07:58:43,309][53885] Updated weights for policy 1, policy_version 3510 (0.0009) [2023-10-08 07:58:43,669][53885] Updated weights for policy 1, policy_version 3520 (0.0008) [2023-10-08 07:58:44,089][53852] Updated weights for policy 0, policy_version 3530 (0.0009) [2023-10-08 07:58:44,444][53852] Updated weights for policy 0, policy_version 3540 (0.0007) [2023-10-08 07:58:44,818][53852] Updated weights for policy 0, policy_version 3550 (0.0008) [2023-10-08 07:58:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7241728. Throughput: 0: 1813.9, 1: 1798.9. Samples: 1819882. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-08 07:58:47,016][52710] Avg episode reward: [(0, '8.390'), (1, '7.230')] [2023-10-08 07:58:47,017][53500] Saving new best policy, reward=8.390! [2023-10-08 07:58:47,424][53885] Updated weights for policy 1, policy_version 3530 (0.0007) [2023-10-08 07:58:47,795][53885] Updated weights for policy 1, policy_version 3540 (0.0007) [2023-10-08 07:58:48,160][53885] Updated weights for policy 1, policy_version 3550 (0.0008) [2023-10-08 07:58:48,473][53852] Updated weights for policy 0, policy_version 3560 (0.0009) [2023-10-08 07:58:48,850][53852] Updated weights for policy 0, policy_version 3570 (0.0009) [2023-10-08 07:58:49,219][53852] Updated weights for policy 0, policy_version 3580 (0.0007) [2023-10-08 07:58:51,891][53885] Updated weights for policy 1, policy_version 3560 (0.0009) [2023-10-08 07:58:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7307264. Throughput: 0: 1816.8, 1: 1812.4. Samples: 1842830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:58:52,015][52710] Avg episode reward: [(0, '8.330'), (1, '6.970')] [2023-10-08 07:58:52,258][53885] Updated weights for policy 1, policy_version 3570 (0.0008) [2023-10-08 07:58:52,622][53885] Updated weights for policy 1, policy_version 3580 (0.0008) [2023-10-08 07:58:52,911][53852] Updated weights for policy 0, policy_version 3590 (0.0008) [2023-10-08 07:58:53,274][53852] Updated weights for policy 0, policy_version 3600 (0.0007) [2023-10-08 07:58:53,656][53852] Updated weights for policy 0, policy_version 3610 (0.0007) [2023-10-08 07:58:56,400][53885] Updated weights for policy 1, policy_version 3590 (0.0007) [2023-10-08 07:58:56,770][53885] Updated weights for policy 1, policy_version 3600 (0.0008) [2023-10-08 07:58:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7372800. Throughput: 0: 1820.5, 1: 1801.8. Samples: 1852766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:58:57,016][52710] Avg episode reward: [(0, '8.330'), (1, '7.590')] [2023-10-08 07:58:57,137][53885] Updated weights for policy 1, policy_version 3610 (0.0008) [2023-10-08 07:58:57,347][53852] Updated weights for policy 0, policy_version 3620 (0.0008) [2023-10-08 07:58:57,726][53852] Updated weights for policy 0, policy_version 3630 (0.0008) [2023-10-08 07:58:58,092][53852] Updated weights for policy 0, policy_version 3640 (0.0008) [2023-10-08 07:59:00,779][53885] Updated weights for policy 1, policy_version 3620 (0.0009) [2023-10-08 07:59:01,149][53885] Updated weights for policy 1, policy_version 3630 (0.0010) [2023-10-08 07:59:01,522][53885] Updated weights for policy 1, policy_version 3640 (0.0010) [2023-10-08 07:59:01,708][53852] Updated weights for policy 0, policy_version 3650 (0.0008) [2023-10-08 07:59:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7471104. Throughput: 0: 1824.3, 1: 1812.0. Samples: 1875576. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 07:59:02,016][52710] Avg episode reward: [(0, '7.890'), (1, '7.500')] [2023-10-08 07:59:02,080][53852] Updated weights for policy 0, policy_version 3660 (0.0009) [2023-10-08 07:59:02,455][53852] Updated weights for policy 0, policy_version 3670 (0.0009) [2023-10-08 07:59:02,830][53852] Updated weights for policy 0, policy_version 3680 (0.0009) [2023-10-08 07:59:05,235][53885] Updated weights for policy 1, policy_version 3650 (0.0008) [2023-10-08 07:59:05,597][53885] Updated weights for policy 1, policy_version 3660 (0.0011) [2023-10-08 07:59:05,976][53885] Updated weights for policy 1, policy_version 3670 (0.0011) [2023-10-08 07:59:06,345][53885] Updated weights for policy 1, policy_version 3680 (0.0009) [2023-10-08 07:59:06,664][53852] Updated weights for policy 0, policy_version 3690 (0.0010) [2023-10-08 07:59:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 7536640. Throughput: 0: 1824.5, 1: 1806.3. Samples: 1896530. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) [2023-10-08 07:59:07,016][52710] Avg episode reward: [(0, '8.310'), (1, '8.160')] [2023-10-08 07:59:07,025][53852] Updated weights for policy 0, policy_version 3700 (0.0009) [2023-10-08 07:59:07,029][53594] Saving new best policy, reward=8.160! [2023-10-08 07:59:07,407][53852] Updated weights for policy 0, policy_version 3710 (0.0007) [2023-10-08 07:59:10,083][53885] Updated weights for policy 1, policy_version 3690 (0.0008) [2023-10-08 07:59:10,450][53885] Updated weights for policy 1, policy_version 3700 (0.0008) [2023-10-08 07:59:10,820][53885] Updated weights for policy 1, policy_version 3710 (0.0010) [2023-10-08 07:59:11,117][53852] Updated weights for policy 0, policy_version 3720 (0.0009) [2023-10-08 07:59:11,484][53852] Updated weights for policy 0, policy_version 3730 (0.0008) [2023-10-08 07:59:11,862][53852] Updated weights for policy 0, policy_version 3740 (0.0007) [2023-10-08 07:59:12,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 7634944. Throughput: 0: 1815.7, 1: 1817.5. Samples: 1908286. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 07:59:12,016][52710] Avg episode reward: [(0, '8.240'), (1, '7.820')] [2023-10-08 07:59:14,590][53885] Updated weights for policy 1, policy_version 3720 (0.0009) [2023-10-08 07:59:14,963][53885] Updated weights for policy 1, policy_version 3730 (0.0009) [2023-10-08 07:59:15,331][53885] Updated weights for policy 1, policy_version 3740 (0.0008) [2023-10-08 07:59:15,489][53852] Updated weights for policy 0, policy_version 3750 (0.0008) [2023-10-08 07:59:15,865][53852] Updated weights for policy 0, policy_version 3760 (0.0008) [2023-10-08 07:59:16,251][53852] Updated weights for policy 0, policy_version 3770 (0.0007) [2023-10-08 07:59:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7700480. Throughput: 0: 1824.7, 1: 1810.4. Samples: 1929186. Policy #0 lag: (min: 37.0, avg: 47.5, max: 48.0) [2023-10-08 07:59:17,016][52710] Avg episode reward: [(0, '8.160'), (1, '8.660')] [2023-10-08 07:59:17,018][53594] Saving new best policy, reward=8.660! [2023-10-08 07:59:18,983][53885] Updated weights for policy 1, policy_version 3750 (0.0009) [2023-10-08 07:59:19,351][53885] Updated weights for policy 1, policy_version 3760 (0.0008) [2023-10-08 07:59:19,728][53885] Updated weights for policy 1, policy_version 3770 (0.0008) [2023-10-08 07:59:19,886][53852] Updated weights for policy 0, policy_version 3780 (0.0008) [2023-10-08 07:59:20,253][53852] Updated weights for policy 0, policy_version 3790 (0.0007) [2023-10-08 07:59:20,624][53852] Updated weights for policy 0, policy_version 3800 (0.0009) [2023-10-08 07:59:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 7766016. Throughput: 0: 1824.9, 1: 1807.4. Samples: 1950992. Policy #0 lag: (min: 37.0, avg: 47.5, max: 48.0) [2023-10-08 07:59:22,016][52710] Avg episode reward: [(0, '8.120'), (1, '9.060')] [2023-10-08 07:59:22,028][53594] Saving new best policy, reward=9.060! [2023-10-08 07:59:23,416][53885] Updated weights for policy 1, policy_version 3780 (0.0008) [2023-10-08 07:59:23,784][53885] Updated weights for policy 1, policy_version 3790 (0.0009) [2023-10-08 07:59:24,158][53885] Updated weights for policy 1, policy_version 3800 (0.0010) [2023-10-08 07:59:24,322][53852] Updated weights for policy 0, policy_version 3810 (0.0009) [2023-10-08 07:59:24,688][53852] Updated weights for policy 0, policy_version 3820 (0.0008) [2023-10-08 07:59:25,066][53852] Updated weights for policy 0, policy_version 3830 (0.0008) [2023-10-08 07:59:25,439][53852] Updated weights for policy 0, policy_version 3840 (0.0008) [2023-10-08 07:59:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7831552. Throughput: 0: 1828.5, 1: 1808.0. Samples: 1961976. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:59:27,016][52710] Avg episode reward: [(0, '8.290'), (1, '8.290')] [2023-10-08 07:59:27,645][53885] Updated weights for policy 1, policy_version 3810 (0.0007) [2023-10-08 07:59:28,018][53885] Updated weights for policy 1, policy_version 3820 (0.0007) [2023-10-08 07:59:28,381][53885] Updated weights for policy 1, policy_version 3830 (0.0008) [2023-10-08 07:59:28,753][53885] Updated weights for policy 1, policy_version 3840 (0.0008) [2023-10-08 07:59:29,170][53852] Updated weights for policy 0, policy_version 3850 (0.0008) [2023-10-08 07:59:29,541][53852] Updated weights for policy 0, policy_version 3860 (0.0007) [2023-10-08 07:59:29,915][53852] Updated weights for policy 0, policy_version 3870 (0.0007) [2023-10-08 07:59:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 7897088. Throughput: 0: 1824.4, 1: 1815.3. Samples: 1983668. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 07:59:32,016][52710] Avg episode reward: [(0, '8.160'), (1, '8.290')] [2023-10-08 07:59:32,605][53885] Updated weights for policy 1, policy_version 3850 (0.0010) [2023-10-08 07:59:32,983][53885] Updated weights for policy 1, policy_version 3860 (0.0008) [2023-10-08 07:59:33,347][53885] Updated weights for policy 1, policy_version 3870 (0.0008) [2023-10-08 07:59:33,639][53852] Updated weights for policy 0, policy_version 3880 (0.0008) [2023-10-08 07:59:34,007][53852] Updated weights for policy 0, policy_version 3890 (0.0009) [2023-10-08 07:59:34,374][53852] Updated weights for policy 0, policy_version 3900 (0.0008) [2023-10-08 07:59:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 7962624. Throughput: 0: 1819.8, 1: 1810.9. Samples: 2006212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:59:37,015][52710] Avg episode reward: [(0, '8.290'), (1, '8.870')] [2023-10-08 07:59:37,110][53885] Updated weights for policy 1, policy_version 3880 (0.0008) [2023-10-08 07:59:37,481][53885] Updated weights for policy 1, policy_version 3890 (0.0008) [2023-10-08 07:59:37,853][53885] Updated weights for policy 1, policy_version 3900 (0.0009) [2023-10-08 07:59:38,077][53852] Updated weights for policy 0, policy_version 3910 (0.0008) [2023-10-08 07:59:38,449][53852] Updated weights for policy 0, policy_version 3920 (0.0010) [2023-10-08 07:59:38,817][53852] Updated weights for policy 0, policy_version 3930 (0.0007) [2023-10-08 07:59:41,471][53885] Updated weights for policy 1, policy_version 3910 (0.0008) [2023-10-08 07:59:41,844][53885] Updated weights for policy 1, policy_version 3920 (0.0008) [2023-10-08 07:59:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8028160. Throughput: 0: 1819.7, 1: 1807.2. Samples: 2015976. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 07:59:42,015][52710] Avg episode reward: [(0, '7.990'), (1, '8.970')] [2023-10-08 07:59:42,210][53885] Updated weights for policy 1, policy_version 3930 (0.0008) [2023-10-08 07:59:42,519][53852] Updated weights for policy 0, policy_version 3940 (0.0007) [2023-10-08 07:59:42,897][53852] Updated weights for policy 0, policy_version 3950 (0.0009) [2023-10-08 07:59:43,265][53852] Updated weights for policy 0, policy_version 3960 (0.0008) [2023-10-08 07:59:45,863][53885] Updated weights for policy 1, policy_version 3940 (0.0010) [2023-10-08 07:59:46,234][53885] Updated weights for policy 1, policy_version 3950 (0.0007) [2023-10-08 07:59:46,603][53885] Updated weights for policy 1, policy_version 3960 (0.0008) [2023-10-08 07:59:46,905][53852] Updated weights for policy 0, policy_version 3970 (0.0009) [2023-10-08 07:59:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8126464. Throughput: 0: 1814.4, 1: 1814.0. Samples: 2038858. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-08 07:59:47,015][52710] Avg episode reward: [(0, '8.180'), (1, '8.310')] [2023-10-08 07:59:47,267][53852] Updated weights for policy 0, policy_version 3980 (0.0007) [2023-10-08 07:59:47,639][53852] Updated weights for policy 0, policy_version 3990 (0.0008) [2023-10-08 07:59:48,014][53852] Updated weights for policy 0, policy_version 4000 (0.0009) [2023-10-08 07:59:50,490][53885] Updated weights for policy 1, policy_version 3970 (0.0007) [2023-10-08 07:59:50,872][53885] Updated weights for policy 1, policy_version 3980 (0.0009) [2023-10-08 07:59:51,239][53885] Updated weights for policy 1, policy_version 3990 (0.0009) [2023-10-08 07:59:51,563][53852] Updated weights for policy 0, policy_version 4010 (0.0007) [2023-10-08 07:59:51,608][53885] Updated weights for policy 1, policy_version 4000 (0.0008) [2023-10-08 07:59:51,933][53852] Updated weights for policy 0, policy_version 4020 (0.0007) [2023-10-08 07:59:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8192000. Throughput: 0: 1812.7, 1: 1811.7. Samples: 2059630. Policy #0 lag: (min: 31.0, avg: 39.7, max: 63.0) [2023-10-08 07:59:52,015][52710] Avg episode reward: [(0, '8.870'), (1, '8.290')] [2023-10-08 07:59:52,310][53852] Updated weights for policy 0, policy_version 4030 (0.0007) [2023-10-08 07:59:52,379][53500] Saving new best policy, reward=8.870! [2023-10-08 07:59:55,247][53885] Updated weights for policy 1, policy_version 4010 (0.0009) [2023-10-08 07:59:55,615][53885] Updated weights for policy 1, policy_version 4020 (0.0009) [2023-10-08 07:59:55,895][53852] Updated weights for policy 0, policy_version 4040 (0.0008) [2023-10-08 07:59:55,986][53885] Updated weights for policy 1, policy_version 4030 (0.0008) [2023-10-08 07:59:56,257][53852] Updated weights for policy 0, policy_version 4050 (0.0009) [2023-10-08 07:59:56,623][53852] Updated weights for policy 0, policy_version 4060 (0.0010) [2023-10-08 07:59:57,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 8290304. Throughput: 0: 1816.8, 1: 1806.1. Samples: 2071318. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) [2023-10-08 07:59:57,016][52710] Avg episode reward: [(0, '9.170'), (1, '8.490')] [2023-10-08 07:59:57,018][53500] Saving new best policy, reward=9.170! [2023-10-08 07:59:59,701][53885] Updated weights for policy 1, policy_version 4040 (0.0010) [2023-10-08 08:00:00,069][53885] Updated weights for policy 1, policy_version 4050 (0.0010) [2023-10-08 08:00:00,439][53852] Updated weights for policy 0, policy_version 4070 (0.0010) [2023-10-08 08:00:00,440][53885] Updated weights for policy 1, policy_version 4060 (0.0009) [2023-10-08 08:00:00,804][53852] Updated weights for policy 0, policy_version 4080 (0.0007) [2023-10-08 08:00:01,177][53852] Updated weights for policy 0, policy_version 4090 (0.0007) [2023-10-08 08:00:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8355840. Throughput: 0: 1809.7, 1: 1808.7. Samples: 2092012. Policy #0 lag: (min: 2.0, avg: 8.8, max: 34.0) [2023-10-08 08:00:02,016][52710] Avg episode reward: [(0, '8.340'), (1, '8.210')] [2023-10-08 08:00:04,132][53885] Updated weights for policy 1, policy_version 4070 (0.0007) [2023-10-08 08:00:04,507][53885] Updated weights for policy 1, policy_version 4080 (0.0007) [2023-10-08 08:00:04,867][53885] Updated weights for policy 1, policy_version 4090 (0.0007) [2023-10-08 08:00:04,908][53852] Updated weights for policy 0, policy_version 4100 (0.0007) [2023-10-08 08:00:05,289][53852] Updated weights for policy 0, policy_version 4110 (0.0007) [2023-10-08 08:00:05,663][53852] Updated weights for policy 0, policy_version 4120 (0.0008) [2023-10-08 08:00:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8421376. Throughput: 0: 1807.4, 1: 1812.3. Samples: 2113878. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 08:00:07,016][52710] Avg episode reward: [(0, '8.580'), (1, '8.330')] [2023-10-08 08:00:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth... [2023-10-08 08:00:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000004096_4194304.pth... [2023-10-08 08:00:07,067][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000002432_2490368.pth [2023-10-08 08:00:07,068][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000002400_2457600.pth [2023-10-08 08:00:08,554][53885] Updated weights for policy 1, policy_version 4100 (0.0008) [2023-10-08 08:00:08,919][53885] Updated weights for policy 1, policy_version 4110 (0.0009) [2023-10-08 08:00:09,292][53885] Updated weights for policy 1, policy_version 4120 (0.0009) [2023-10-08 08:00:09,320][53852] Updated weights for policy 0, policy_version 4130 (0.0008) [2023-10-08 08:00:09,695][53852] Updated weights for policy 0, policy_version 4140 (0.0008) [2023-10-08 08:00:10,059][53852] Updated weights for policy 0, policy_version 4150 (0.0008) [2023-10-08 08:00:10,428][53852] Updated weights for policy 0, policy_version 4160 (0.0007) [2023-10-08 08:00:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8486912. Throughput: 0: 1807.7, 1: 1813.7. Samples: 2124940. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 08:00:12,016][52710] Avg episode reward: [(0, '9.040'), (1, '8.860')] [2023-10-08 08:00:12,978][53885] Updated weights for policy 1, policy_version 4130 (0.0009) [2023-10-08 08:00:13,345][53885] Updated weights for policy 1, policy_version 4140 (0.0007) [2023-10-08 08:00:13,717][53885] Updated weights for policy 1, policy_version 4150 (0.0007) [2023-10-08 08:00:14,082][53885] Updated weights for policy 1, policy_version 4160 (0.0007) [2023-10-08 08:00:14,217][53852] Updated weights for policy 0, policy_version 4170 (0.0008) [2023-10-08 08:00:14,593][53852] Updated weights for policy 0, policy_version 4180 (0.0008) [2023-10-08 08:00:14,962][53852] Updated weights for policy 0, policy_version 4190 (0.0007) [2023-10-08 08:00:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8552448. Throughput: 0: 1812.4, 1: 1802.9. Samples: 2146356. Policy #0 lag: (min: 21.0, avg: 23.9, max: 53.0) [2023-10-08 08:00:17,016][52710] Avg episode reward: [(0, '8.760'), (1, '8.390')] [2023-10-08 08:00:17,795][53885] Updated weights for policy 1, policy_version 4170 (0.0007) [2023-10-08 08:00:18,161][53885] Updated weights for policy 1, policy_version 4180 (0.0009) [2023-10-08 08:00:18,532][53885] Updated weights for policy 1, policy_version 4190 (0.0007) [2023-10-08 08:00:18,600][53852] Updated weights for policy 0, policy_version 4200 (0.0009) [2023-10-08 08:00:18,973][53852] Updated weights for policy 0, policy_version 4210 (0.0010) [2023-10-08 08:00:19,339][53852] Updated weights for policy 0, policy_version 4220 (0.0009) [2023-10-08 08:00:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 8617984. Throughput: 0: 1814.8, 1: 1807.6. Samples: 2169220. Policy #0 lag: (min: 21.0, avg: 23.9, max: 53.0) [2023-10-08 08:00:22,016][52710] Avg episode reward: [(0, '9.440'), (1, '8.480')] [2023-10-08 08:00:22,024][53500] Saving new best policy, reward=9.440! [2023-10-08 08:00:22,391][53885] Updated weights for policy 1, policy_version 4200 (0.0008) [2023-10-08 08:00:22,767][53885] Updated weights for policy 1, policy_version 4210 (0.0007) [2023-10-08 08:00:22,999][53852] Updated weights for policy 0, policy_version 4230 (0.0007) [2023-10-08 08:00:23,133][53885] Updated weights for policy 1, policy_version 4220 (0.0007) [2023-10-08 08:00:23,380][53852] Updated weights for policy 0, policy_version 4240 (0.0008) [2023-10-08 08:00:23,750][53852] Updated weights for policy 0, policy_version 4250 (0.0007) [2023-10-08 08:00:26,915][53885] Updated weights for policy 1, policy_version 4230 (0.0007) [2023-10-08 08:00:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8683520. Throughput: 0: 1814.5, 1: 1806.7. Samples: 2178928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:00:27,015][52710] Avg episode reward: [(0, '8.680'), (1, '9.220')] [2023-10-08 08:00:27,293][53885] Updated weights for policy 1, policy_version 4240 (0.0008) [2023-10-08 08:00:27,532][53852] Updated weights for policy 0, policy_version 4260 (0.0008) [2023-10-08 08:00:27,654][53885] Updated weights for policy 1, policy_version 4250 (0.0009) [2023-10-08 08:00:27,867][53594] Saving new best policy, reward=9.220! [2023-10-08 08:00:27,898][53852] Updated weights for policy 0, policy_version 4270 (0.0007) [2023-10-08 08:00:28,266][53852] Updated weights for policy 0, policy_version 4280 (0.0008) [2023-10-08 08:00:31,465][53885] Updated weights for policy 1, policy_version 4260 (0.0007) [2023-10-08 08:00:31,838][53885] Updated weights for policy 1, policy_version 4270 (0.0009) [2023-10-08 08:00:31,976][53852] Updated weights for policy 0, policy_version 4290 (0.0007) [2023-10-08 08:00:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8749056. Throughput: 0: 1815.7, 1: 1798.3. Samples: 2201486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:00:32,016][52710] Avg episode reward: [(0, '8.650'), (1, '9.590')] [2023-10-08 08:00:32,202][53885] Updated weights for policy 1, policy_version 4280 (0.0009) [2023-10-08 08:00:32,340][53852] Updated weights for policy 0, policy_version 4300 (0.0007) [2023-10-08 08:00:32,490][53594] Saving new best policy, reward=9.590! [2023-10-08 08:00:32,716][53852] Updated weights for policy 0, policy_version 4310 (0.0009) [2023-10-08 08:00:33,091][53852] Updated weights for policy 0, policy_version 4320 (0.0009) [2023-10-08 08:00:35,919][53885] Updated weights for policy 1, policy_version 4290 (0.0007) [2023-10-08 08:00:36,293][53885] Updated weights for policy 1, policy_version 4300 (0.0009) [2023-10-08 08:00:36,665][53885] Updated weights for policy 1, policy_version 4310 (0.0007) [2023-10-08 08:00:36,747][53852] Updated weights for policy 0, policy_version 4330 (0.0007) [2023-10-08 08:00:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 8814592. Throughput: 0: 1818.1, 1: 1813.9. Samples: 2223070. Policy #0 lag: (min: 0.0, avg: 22.7, max: 32.0) [2023-10-08 08:00:37,016][52710] Avg episode reward: [(0, '8.520'), (1, '9.410')] [2023-10-08 08:00:37,023][53885] Updated weights for policy 1, policy_version 4320 (0.0007) [2023-10-08 08:00:37,121][53852] Updated weights for policy 0, policy_version 4340 (0.0009) [2023-10-08 08:00:37,494][53852] Updated weights for policy 0, policy_version 4350 (0.0007) [2023-10-08 08:00:40,633][53885] Updated weights for policy 1, policy_version 4330 (0.0008) [2023-10-08 08:00:41,007][53885] Updated weights for policy 1, policy_version 4340 (0.0010) [2023-10-08 08:00:41,137][53852] Updated weights for policy 0, policy_version 4360 (0.0008) [2023-10-08 08:00:41,379][53885] Updated weights for policy 1, policy_version 4350 (0.0009) [2023-10-08 08:00:41,520][53852] Updated weights for policy 0, policy_version 4370 (0.0008) [2023-10-08 08:00:41,888][53852] Updated weights for policy 0, policy_version 4380 (0.0009) [2023-10-08 08:00:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 8912896. Throughput: 0: 1817.9, 1: 1805.6. Samples: 2234374. Policy #0 lag: (min: 0.0, avg: 22.7, max: 32.0) [2023-10-08 08:00:42,016][52710] Avg episode reward: [(0, '8.510'), (1, '9.150')] [2023-10-08 08:00:45,108][53885] Updated weights for policy 1, policy_version 4360 (0.0009) [2023-10-08 08:00:45,481][53885] Updated weights for policy 1, policy_version 4370 (0.0009) [2023-10-08 08:00:45,531][53852] Updated weights for policy 0, policy_version 4390 (0.0009) [2023-10-08 08:00:45,844][53885] Updated weights for policy 1, policy_version 4380 (0.0007) [2023-10-08 08:00:45,894][53852] Updated weights for policy 0, policy_version 4400 (0.0009) [2023-10-08 08:00:46,272][53852] Updated weights for policy 0, policy_version 4410 (0.0008) [2023-10-08 08:00:47,015][52710] Fps is (10 sec: 19660.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 9011200. Throughput: 0: 1822.7, 1: 1816.4. Samples: 2255772. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) [2023-10-08 08:00:47,016][52710] Avg episode reward: [(0, '8.960'), (1, '9.240')] [2023-10-08 08:00:49,670][53885] Updated weights for policy 1, policy_version 4390 (0.0009) [2023-10-08 08:00:50,015][53852] Updated weights for policy 0, policy_version 4420 (0.0009) [2023-10-08 08:00:50,037][53885] Updated weights for policy 1, policy_version 4400 (0.0009) [2023-10-08 08:00:50,399][53852] Updated weights for policy 0, policy_version 4430 (0.0007) [2023-10-08 08:00:50,404][53885] Updated weights for policy 1, policy_version 4410 (0.0009) [2023-10-08 08:00:50,769][53852] Updated weights for policy 0, policy_version 4440 (0.0008) [2023-10-08 08:00:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 9076736. Throughput: 0: 1816.2, 1: 1800.6. Samples: 2276632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:00:52,016][52710] Avg episode reward: [(0, '9.780'), (1, '8.990')] [2023-10-08 08:00:52,026][53500] Saving new best policy, reward=9.780! [2023-10-08 08:00:54,188][53885] Updated weights for policy 1, policy_version 4420 (0.0009) [2023-10-08 08:00:54,433][53852] Updated weights for policy 0, policy_version 4450 (0.0009) [2023-10-08 08:00:54,558][53885] Updated weights for policy 1, policy_version 4430 (0.0008) [2023-10-08 08:00:54,798][53852] Updated weights for policy 0, policy_version 4460 (0.0009) [2023-10-08 08:00:54,932][53885] Updated weights for policy 1, policy_version 4440 (0.0007) [2023-10-08 08:00:55,166][53852] Updated weights for policy 0, policy_version 4470 (0.0008) [2023-10-08 08:00:55,532][53852] Updated weights for policy 0, policy_version 4480 (0.0009) [2023-10-08 08:00:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9142272. Throughput: 0: 1819.3, 1: 1815.8. Samples: 2288520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:00:57,016][52710] Avg episode reward: [(0, '8.560'), (1, '9.390')] [2023-10-08 08:00:58,558][53885] Updated weights for policy 1, policy_version 4450 (0.0007) [2023-10-08 08:00:58,930][53885] Updated weights for policy 1, policy_version 4460 (0.0007) [2023-10-08 08:00:59,252][53852] Updated weights for policy 0, policy_version 4490 (0.0008) [2023-10-08 08:00:59,302][53885] Updated weights for policy 1, policy_version 4470 (0.0007) [2023-10-08 08:00:59,620][53852] Updated weights for policy 0, policy_version 4500 (0.0007) [2023-10-08 08:00:59,663][53885] Updated weights for policy 1, policy_version 4480 (0.0008) [2023-10-08 08:00:59,992][53852] Updated weights for policy 0, policy_version 4510 (0.0008) [2023-10-08 08:01:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 9207808. Throughput: 0: 1815.2, 1: 1801.5. Samples: 2309106. Policy #0 lag: (min: 31.0, avg: 31.3, max: 41.0) [2023-10-08 08:01:02,016][52710] Avg episode reward: [(0, '8.570'), (1, '9.790')] [2023-10-08 08:01:02,018][53594] Saving new best policy, reward=9.790! [2023-10-08 08:01:03,302][53885] Updated weights for policy 1, policy_version 4490 (0.0010) [2023-10-08 08:01:03,671][53885] Updated weights for policy 1, policy_version 4500 (0.0007) [2023-10-08 08:01:03,804][53852] Updated weights for policy 0, policy_version 4520 (0.0009) [2023-10-08 08:01:04,034][53885] Updated weights for policy 1, policy_version 4510 (0.0009) [2023-10-08 08:01:04,178][53852] Updated weights for policy 0, policy_version 4530 (0.0010) [2023-10-08 08:01:04,553][53852] Updated weights for policy 0, policy_version 4540 (0.0008) [2023-10-08 08:01:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9273344. Throughput: 0: 1813.0, 1: 1803.0. Samples: 2331940. Policy #0 lag: (min: 31.0, avg: 31.3, max: 41.0) [2023-10-08 08:01:07,016][52710] Avg episode reward: [(0, '9.290'), (1, '10.210')] [2023-10-08 08:01:07,024][53594] Saving new best policy, reward=10.210! [2023-10-08 08:01:07,762][53885] Updated weights for policy 1, policy_version 4520 (0.0008) [2023-10-08 08:01:08,135][53885] Updated weights for policy 1, policy_version 4530 (0.0010) [2023-10-08 08:01:08,272][53852] Updated weights for policy 0, policy_version 4550 (0.0010) [2023-10-08 08:01:08,496][53885] Updated weights for policy 1, policy_version 4540 (0.0009) [2023-10-08 08:01:08,644][53852] Updated weights for policy 0, policy_version 4560 (0.0008) [2023-10-08 08:01:09,013][53852] Updated weights for policy 0, policy_version 4570 (0.0010) [2023-10-08 08:01:12,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9338880. Throughput: 0: 1809.6, 1: 1806.6. Samples: 2341656. Policy #0 lag: (min: 20.0, avg: 20.9, max: 41.0) [2023-10-08 08:01:12,016][52710] Avg episode reward: [(0, '9.150'), (1, '10.170')] [2023-10-08 08:01:12,199][53885] Updated weights for policy 1, policy_version 4550 (0.0008) [2023-10-08 08:01:12,573][53885] Updated weights for policy 1, policy_version 4560 (0.0008) [2023-10-08 08:01:12,770][53852] Updated weights for policy 0, policy_version 4580 (0.0007) [2023-10-08 08:01:12,940][53885] Updated weights for policy 1, policy_version 4570 (0.0007) [2023-10-08 08:01:13,141][53852] Updated weights for policy 0, policy_version 4590 (0.0009) [2023-10-08 08:01:13,512][53852] Updated weights for policy 0, policy_version 4600 (0.0009) [2023-10-08 08:01:16,419][53885] Updated weights for policy 1, policy_version 4580 (0.0007) [2023-10-08 08:01:16,783][53885] Updated weights for policy 1, policy_version 4590 (0.0007) [2023-10-08 08:01:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9404416. Throughput: 0: 1808.2, 1: 1816.8. Samples: 2364612. Policy #0 lag: (min: 20.0, avg: 20.9, max: 41.0) [2023-10-08 08:01:17,016][52710] Avg episode reward: [(0, '8.900'), (1, '9.570')] [2023-10-08 08:01:17,129][53852] Updated weights for policy 0, policy_version 4610 (0.0011) [2023-10-08 08:01:17,146][53885] Updated weights for policy 1, policy_version 4600 (0.0007) [2023-10-08 08:01:17,495][53852] Updated weights for policy 0, policy_version 4620 (0.0008) [2023-10-08 08:01:17,871][53852] Updated weights for policy 0, policy_version 4630 (0.0008) [2023-10-08 08:01:18,243][53852] Updated weights for policy 0, policy_version 4640 (0.0009) [2023-10-08 08:01:20,714][53885] Updated weights for policy 1, policy_version 4610 (0.0008) [2023-10-08 08:01:21,080][53885] Updated weights for policy 1, policy_version 4620 (0.0009) [2023-10-08 08:01:21,447][53885] Updated weights for policy 1, policy_version 4630 (0.0008) [2023-10-08 08:01:21,817][53885] Updated weights for policy 1, policy_version 4640 (0.0007) [2023-10-08 08:01:21,935][53852] Updated weights for policy 0, policy_version 4650 (0.0008) [2023-10-08 08:01:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9502720. Throughput: 0: 1811.6, 1: 1815.0. Samples: 2386266. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 08:01:22,015][52710] Avg episode reward: [(0, '9.260'), (1, '9.190')] [2023-10-08 08:01:22,315][53852] Updated weights for policy 0, policy_version 4660 (0.0008) [2023-10-08 08:01:22,681][53852] Updated weights for policy 0, policy_version 4670 (0.0009) [2023-10-08 08:01:25,482][53885] Updated weights for policy 1, policy_version 4650 (0.0008) [2023-10-08 08:01:25,861][53885] Updated weights for policy 1, policy_version 4660 (0.0010) [2023-10-08 08:01:26,230][53885] Updated weights for policy 1, policy_version 4670 (0.0007) [2023-10-08 08:01:26,380][53852] Updated weights for policy 0, policy_version 4680 (0.0008) [2023-10-08 08:01:26,753][53852] Updated weights for policy 0, policy_version 4690 (0.0007) [2023-10-08 08:01:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9568256. Throughput: 0: 1804.0, 1: 1819.1. Samples: 2397410. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) [2023-10-08 08:01:27,016][52710] Avg episode reward: [(0, '9.890'), (1, '9.910')] [2023-10-08 08:01:27,129][53852] Updated weights for policy 0, policy_version 4700 (0.0009) [2023-10-08 08:01:27,272][53500] Saving new best policy, reward=9.890! [2023-10-08 08:01:29,793][53885] Updated weights for policy 1, policy_version 4680 (0.0007) [2023-10-08 08:01:30,164][53885] Updated weights for policy 1, policy_version 4690 (0.0007) [2023-10-08 08:01:30,535][53885] Updated weights for policy 1, policy_version 4700 (0.0010) [2023-10-08 08:01:30,869][53852] Updated weights for policy 0, policy_version 4710 (0.0009) [2023-10-08 08:01:31,247][53852] Updated weights for policy 0, policy_version 4720 (0.0008) [2023-10-08 08:01:31,617][53852] Updated weights for policy 0, policy_version 4730 (0.0008) [2023-10-08 08:01:32,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 9666560. Throughput: 0: 1810.1, 1: 1818.8. Samples: 2419076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:01:32,016][52710] Avg episode reward: [(0, '9.880'), (1, '9.640')] [2023-10-08 08:01:34,338][53885] Updated weights for policy 1, policy_version 4710 (0.0009) [2023-10-08 08:01:34,704][53885] Updated weights for policy 1, policy_version 4720 (0.0008) [2023-10-08 08:01:35,078][53885] Updated weights for policy 1, policy_version 4730 (0.0008) [2023-10-08 08:01:35,170][53852] Updated weights for policy 0, policy_version 4740 (0.0008) [2023-10-08 08:01:35,541][53852] Updated weights for policy 0, policy_version 4750 (0.0008) [2023-10-08 08:01:35,920][53852] Updated weights for policy 0, policy_version 4760 (0.0009) [2023-10-08 08:01:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 9732096. Throughput: 0: 1809.1, 1: 1834.2. Samples: 2440580. Policy #0 lag: (min: 0.0, avg: 22.4, max: 32.0) [2023-10-08 08:01:37,016][52710] Avg episode reward: [(0, '8.710'), (1, '10.010')] [2023-10-08 08:01:38,744][53885] Updated weights for policy 1, policy_version 4740 (0.0007) [2023-10-08 08:01:39,118][53885] Updated weights for policy 1, policy_version 4750 (0.0008) [2023-10-08 08:01:39,474][53885] Updated weights for policy 1, policy_version 4760 (0.0007) [2023-10-08 08:01:39,593][53852] Updated weights for policy 0, policy_version 4770 (0.0008) [2023-10-08 08:01:39,958][53852] Updated weights for policy 0, policy_version 4780 (0.0007) [2023-10-08 08:01:40,332][53852] Updated weights for policy 0, policy_version 4790 (0.0008) [2023-10-08 08:01:40,707][53852] Updated weights for policy 0, policy_version 4800 (0.0007) [2023-10-08 08:01:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 9797632. Throughput: 0: 1815.8, 1: 1826.5. Samples: 2452422. Policy #0 lag: (min: 0.0, avg: 22.4, max: 32.0) [2023-10-08 08:01:42,016][52710] Avg episode reward: [(0, '8.910'), (1, '10.790')] [2023-10-08 08:01:42,016][53594] Saving new best policy, reward=10.790! [2023-10-08 08:01:43,102][53885] Updated weights for policy 1, policy_version 4770 (0.0007) [2023-10-08 08:01:43,478][53885] Updated weights for policy 1, policy_version 4780 (0.0008) [2023-10-08 08:01:43,848][53885] Updated weights for policy 1, policy_version 4790 (0.0009) [2023-10-08 08:01:44,213][53885] Updated weights for policy 1, policy_version 4800 (0.0008) [2023-10-08 08:01:44,274][53852] Updated weights for policy 0, policy_version 4810 (0.0008) [2023-10-08 08:01:44,650][53852] Updated weights for policy 0, policy_version 4820 (0.0009) [2023-10-08 08:01:45,022][53852] Updated weights for policy 0, policy_version 4830 (0.0007) [2023-10-08 08:01:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9863168. Throughput: 0: 1815.0, 1: 1837.2. Samples: 2473456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:01:47,015][52710] Avg episode reward: [(0, '9.300'), (1, '9.860')] [2023-10-08 08:01:47,948][53885] Updated weights for policy 1, policy_version 4810 (0.0010) [2023-10-08 08:01:48,322][53885] Updated weights for policy 1, policy_version 4820 (0.0010) [2023-10-08 08:01:48,681][53885] Updated weights for policy 1, policy_version 4830 (0.0007) [2023-10-08 08:01:48,801][53852] Updated weights for policy 0, policy_version 4840 (0.0009) [2023-10-08 08:01:49,167][53852] Updated weights for policy 0, policy_version 4850 (0.0009) [2023-10-08 08:01:49,541][53852] Updated weights for policy 0, policy_version 4860 (0.0008) [2023-10-08 08:01:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 9928704. Throughput: 0: 1819.7, 1: 1837.0. Samples: 2496492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:01:52,016][52710] Avg episode reward: [(0, '9.530'), (1, '11.680')] [2023-10-08 08:01:52,436][53885] Updated weights for policy 1, policy_version 4840 (0.0009) [2023-10-08 08:01:52,815][53885] Updated weights for policy 1, policy_version 4850 (0.0010) [2023-10-08 08:01:53,157][53852] Updated weights for policy 0, policy_version 4870 (0.0008) [2023-10-08 08:01:53,186][53885] Updated weights for policy 1, policy_version 4860 (0.0009) [2023-10-08 08:01:53,323][53594] Saving new best policy, reward=11.680! [2023-10-08 08:01:53,519][53852] Updated weights for policy 0, policy_version 4880 (0.0008) [2023-10-08 08:01:53,887][53852] Updated weights for policy 0, policy_version 4890 (0.0008) [2023-10-08 08:01:56,819][53885] Updated weights for policy 1, policy_version 4870 (0.0008) [2023-10-08 08:01:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9994240. Throughput: 0: 1822.9, 1: 1838.5. Samples: 2506420. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-08 08:01:57,015][52710] Avg episode reward: [(0, '9.500'), (1, '11.180')] [2023-10-08 08:01:57,184][53885] Updated weights for policy 1, policy_version 4880 (0.0007) [2023-10-08 08:01:57,556][53885] Updated weights for policy 1, policy_version 4890 (0.0008) [2023-10-08 08:01:57,587][53852] Updated weights for policy 0, policy_version 4900 (0.0009) [2023-10-08 08:01:57,954][53852] Updated weights for policy 0, policy_version 4910 (0.0009) [2023-10-08 08:01:58,328][53852] Updated weights for policy 0, policy_version 4920 (0.0008) [2023-10-08 08:02:01,365][53885] Updated weights for policy 1, policy_version 4900 (0.0009) [2023-10-08 08:02:01,726][53885] Updated weights for policy 1, policy_version 4910 (0.0007) [2023-10-08 08:02:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10059776. Throughput: 0: 1829.7, 1: 1830.7. Samples: 2529332. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) [2023-10-08 08:02:02,016][52710] Avg episode reward: [(0, '10.060'), (1, '10.620')] [2023-10-08 08:02:02,090][53852] Updated weights for policy 0, policy_version 4930 (0.0009) [2023-10-08 08:02:02,097][53885] Updated weights for policy 1, policy_version 4920 (0.0007) [2023-10-08 08:02:02,455][53852] Updated weights for policy 0, policy_version 4940 (0.0007) [2023-10-08 08:02:02,830][53852] Updated weights for policy 0, policy_version 4950 (0.0007) [2023-10-08 08:02:03,199][53852] Updated weights for policy 0, policy_version 4960 (0.0010) [2023-10-08 08:02:03,200][53500] Saving new best policy, reward=10.060! [2023-10-08 08:02:05,570][53885] Updated weights for policy 1, policy_version 4930 (0.0007) [2023-10-08 08:02:05,940][53885] Updated weights for policy 1, policy_version 4940 (0.0008) [2023-10-08 08:02:06,309][53885] Updated weights for policy 1, policy_version 4950 (0.0007) [2023-10-08 08:02:06,675][53885] Updated weights for policy 1, policy_version 4960 (0.0007) [2023-10-08 08:02:06,839][53852] Updated weights for policy 0, policy_version 4970 (0.0008) [2023-10-08 08:02:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10158080. Throughput: 0: 1826.5, 1: 1831.9. Samples: 2550894. Policy #0 lag: (min: 5.0, avg: 10.4, max: 37.0) [2023-10-08 08:02:07,015][52710] Avg episode reward: [(0, '10.020'), (1, '10.790')] [2023-10-08 08:02:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000004960_5079040.pth... [2023-10-08 08:02:07,058][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000003264_3342336.pth [2023-10-08 08:02:07,215][53852] Updated weights for policy 0, policy_version 4980 (0.0009) [2023-10-08 08:02:07,580][53852] Updated weights for policy 0, policy_version 4990 (0.0007) [2023-10-08 08:02:07,655][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000004992_5111808.pth... [2023-10-08 08:02:07,695][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000003264_3342336.pth [2023-10-08 08:02:10,322][53885] Updated weights for policy 1, policy_version 4970 (0.0009) [2023-10-08 08:02:10,693][53885] Updated weights for policy 1, policy_version 4980 (0.0010) [2023-10-08 08:02:11,059][53885] Updated weights for policy 1, policy_version 4990 (0.0009) [2023-10-08 08:02:11,327][53852] Updated weights for policy 0, policy_version 5000 (0.0007) [2023-10-08 08:02:11,701][53852] Updated weights for policy 0, policy_version 5010 (0.0010) [2023-10-08 08:02:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10223616. Throughput: 0: 1828.2, 1: 1837.1. Samples: 2562348. Policy #0 lag: (min: 5.0, avg: 10.4, max: 37.0) [2023-10-08 08:02:12,016][52710] Avg episode reward: [(0, '8.810'), (1, '11.810')] [2023-10-08 08:02:12,018][53594] Saving new best policy, reward=11.810! [2023-10-08 08:02:12,074][53852] Updated weights for policy 0, policy_version 5020 (0.0007) [2023-10-08 08:02:14,857][53885] Updated weights for policy 1, policy_version 5000 (0.0009) [2023-10-08 08:02:15,228][53885] Updated weights for policy 1, policy_version 5010 (0.0007) [2023-10-08 08:02:15,600][53885] Updated weights for policy 1, policy_version 5020 (0.0011) [2023-10-08 08:02:15,907][53852] Updated weights for policy 0, policy_version 5030 (0.0008) [2023-10-08 08:02:16,265][53852] Updated weights for policy 0, policy_version 5040 (0.0007) [2023-10-08 08:02:16,649][53852] Updated weights for policy 0, policy_version 5050 (0.0007) [2023-10-08 08:02:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 10321920. Throughput: 0: 1826.2, 1: 1833.3. Samples: 2583754. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 08:02:17,016][52710] Avg episode reward: [(0, '9.210'), (1, '11.010')] [2023-10-08 08:02:19,392][53885] Updated weights for policy 1, policy_version 5030 (0.0008) [2023-10-08 08:02:19,765][53885] Updated weights for policy 1, policy_version 5040 (0.0008) [2023-10-08 08:02:20,137][53885] Updated weights for policy 1, policy_version 5050 (0.0007) [2023-10-08 08:02:20,248][53852] Updated weights for policy 0, policy_version 5060 (0.0008) [2023-10-08 08:02:20,617][53852] Updated weights for policy 0, policy_version 5070 (0.0007) [2023-10-08 08:02:20,993][53852] Updated weights for policy 0, policy_version 5080 (0.0008) [2023-10-08 08:02:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10387456. Throughput: 0: 1822.1, 1: 1821.5. Samples: 2604544. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 08:02:22,016][52710] Avg episode reward: [(0, '10.070'), (1, '11.220')] [2023-10-08 08:02:22,025][53500] Saving new best policy, reward=10.070! [2023-10-08 08:02:23,894][53885] Updated weights for policy 1, policy_version 5060 (0.0008) [2023-10-08 08:02:24,263][53885] Updated weights for policy 1, policy_version 5070 (0.0010) [2023-10-08 08:02:24,636][53885] Updated weights for policy 1, policy_version 5080 (0.0008) [2023-10-08 08:02:24,648][53852] Updated weights for policy 0, policy_version 5090 (0.0008) [2023-10-08 08:02:25,010][53852] Updated weights for policy 0, policy_version 5100 (0.0008) [2023-10-08 08:02:25,394][53852] Updated weights for policy 0, policy_version 5110 (0.0007) [2023-10-08 08:02:25,772][53852] Updated weights for policy 0, policy_version 5120 (0.0008) [2023-10-08 08:02:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10452992. Throughput: 0: 1825.1, 1: 1821.7. Samples: 2616530. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) [2023-10-08 08:02:27,016][52710] Avg episode reward: [(0, '9.760'), (1, '10.480')] [2023-10-08 08:02:28,345][53885] Updated weights for policy 1, policy_version 5090 (0.0009) [2023-10-08 08:02:28,709][53885] Updated weights for policy 1, policy_version 5100 (0.0010) [2023-10-08 08:02:29,075][53885] Updated weights for policy 1, policy_version 5110 (0.0009) [2023-10-08 08:02:29,442][53885] Updated weights for policy 1, policy_version 5120 (0.0008) [2023-10-08 08:02:29,512][53852] Updated weights for policy 0, policy_version 5130 (0.0007) [2023-10-08 08:02:29,893][53852] Updated weights for policy 0, policy_version 5140 (0.0007) [2023-10-08 08:02:30,272][53852] Updated weights for policy 0, policy_version 5150 (0.0008) [2023-10-08 08:02:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 10518528. Throughput: 0: 1817.2, 1: 1815.8. Samples: 2636940. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-10-08 08:02:32,016][52710] Avg episode reward: [(0, '10.110'), (1, '12.330')] [2023-10-08 08:02:32,017][53500] Saving new best policy, reward=10.110! [2023-10-08 08:02:32,017][53594] Saving new best policy, reward=12.330! [2023-10-08 08:02:33,163][53885] Updated weights for policy 1, policy_version 5130 (0.0008) [2023-10-08 08:02:33,534][53885] Updated weights for policy 1, policy_version 5140 (0.0008) [2023-10-08 08:02:33,912][53885] Updated weights for policy 1, policy_version 5150 (0.0009) [2023-10-08 08:02:34,175][53852] Updated weights for policy 0, policy_version 5160 (0.0008) [2023-10-08 08:02:34,547][53852] Updated weights for policy 0, policy_version 5170 (0.0007) [2023-10-08 08:02:34,914][53852] Updated weights for policy 0, policy_version 5180 (0.0008) [2023-10-08 08:02:37,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 10584064. Throughput: 0: 1811.6, 1: 1815.8. Samples: 2659726. Policy #0 lag: (min: 0.0, avg: 25.7, max: 32.0) [2023-10-08 08:02:37,016][52710] Avg episode reward: [(0, '10.140'), (1, '11.360')] [2023-10-08 08:02:37,028][53500] Saving new best policy, reward=10.140! [2023-10-08 08:02:37,570][53885] Updated weights for policy 1, policy_version 5160 (0.0009) [2023-10-08 08:02:37,937][53885] Updated weights for policy 1, policy_version 5170 (0.0007) [2023-10-08 08:02:38,303][53885] Updated weights for policy 1, policy_version 5180 (0.0009) [2023-10-08 08:02:38,730][53852] Updated weights for policy 0, policy_version 5190 (0.0008) [2023-10-08 08:02:39,105][53852] Updated weights for policy 0, policy_version 5200 (0.0008) [2023-10-08 08:02:39,488][53852] Updated weights for policy 0, policy_version 5210 (0.0008) [2023-10-08 08:02:41,989][53885] Updated weights for policy 1, policy_version 5190 (0.0008) [2023-10-08 08:02:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10649600. Throughput: 0: 1815.2, 1: 1817.1. Samples: 2669876. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:02:42,016][52710] Avg episode reward: [(0, '9.500'), (1, '13.090')] [2023-10-08 08:02:42,360][53885] Updated weights for policy 1, policy_version 5200 (0.0009) [2023-10-08 08:02:42,735][53885] Updated weights for policy 1, policy_version 5210 (0.0010) [2023-10-08 08:02:42,960][53594] Saving new best policy, reward=13.090! [2023-10-08 08:02:43,083][53852] Updated weights for policy 0, policy_version 5220 (0.0008) [2023-10-08 08:02:43,459][53852] Updated weights for policy 0, policy_version 5230 (0.0010) [2023-10-08 08:02:43,835][53852] Updated weights for policy 0, policy_version 5240 (0.0009) [2023-10-08 08:02:46,450][53885] Updated weights for policy 1, policy_version 5220 (0.0009) [2023-10-08 08:02:46,824][53885] Updated weights for policy 1, policy_version 5230 (0.0010) [2023-10-08 08:02:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 10715136. Throughput: 0: 1806.4, 1: 1817.5. Samples: 2692408. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:02:47,016][52710] Avg episode reward: [(0, '9.180'), (1, '12.690')] [2023-10-08 08:02:47,198][53885] Updated weights for policy 1, policy_version 5240 (0.0011) [2023-10-08 08:02:47,412][53852] Updated weights for policy 0, policy_version 5250 (0.0007) [2023-10-08 08:02:47,788][53852] Updated weights for policy 0, policy_version 5260 (0.0007) [2023-10-08 08:02:48,159][53852] Updated weights for policy 0, policy_version 5270 (0.0009) [2023-10-08 08:02:48,532][53852] Updated weights for policy 0, policy_version 5280 (0.0010) [2023-10-08 08:02:51,065][53885] Updated weights for policy 1, policy_version 5250 (0.0010) [2023-10-08 08:02:51,436][53885] Updated weights for policy 1, policy_version 5260 (0.0010) [2023-10-08 08:02:51,805][53885] Updated weights for policy 1, policy_version 5270 (0.0009) [2023-10-08 08:02:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 10780672. Throughput: 0: 1809.8, 1: 1814.2. Samples: 2713974. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 08:02:52,016][52710] Avg episode reward: [(0, '9.440'), (1, '12.790')] [2023-10-08 08:02:52,172][53885] Updated weights for policy 1, policy_version 5280 (0.0009) [2023-10-08 08:02:52,402][53852] Updated weights for policy 0, policy_version 5290 (0.0007) [2023-10-08 08:02:52,784][53852] Updated weights for policy 0, policy_version 5300 (0.0010) [2023-10-08 08:02:53,156][53852] Updated weights for policy 0, policy_version 5310 (0.0010) [2023-10-08 08:02:55,918][53885] Updated weights for policy 1, policy_version 5290 (0.0009) [2023-10-08 08:02:56,284][53885] Updated weights for policy 1, policy_version 5300 (0.0008) [2023-10-08 08:02:56,653][53885] Updated weights for policy 1, policy_version 5310 (0.0008) [2023-10-08 08:02:56,788][53852] Updated weights for policy 0, policy_version 5320 (0.0008) [2023-10-08 08:02:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 10878976. Throughput: 0: 1805.2, 1: 1794.7. Samples: 2724342. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 08:02:57,016][52710] Avg episode reward: [(0, '9.670'), (1, '12.260')] [2023-10-08 08:02:57,159][53852] Updated weights for policy 0, policy_version 5330 (0.0009) [2023-10-08 08:02:57,537][53852] Updated weights for policy 0, policy_version 5340 (0.0009) [2023-10-08 08:03:00,353][53885] Updated weights for policy 1, policy_version 5320 (0.0009) [2023-10-08 08:03:00,728][53885] Updated weights for policy 1, policy_version 5330 (0.0010) [2023-10-08 08:03:01,100][53885] Updated weights for policy 1, policy_version 5340 (0.0009) [2023-10-08 08:03:01,235][53852] Updated weights for policy 0, policy_version 5350 (0.0008) [2023-10-08 08:03:01,610][53852] Updated weights for policy 0, policy_version 5360 (0.0007) [2023-10-08 08:03:01,979][53852] Updated weights for policy 0, policy_version 5370 (0.0008) [2023-10-08 08:03:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10944512. Throughput: 0: 1810.0, 1: 1805.7. Samples: 2746462. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 08:03:02,016][52710] Avg episode reward: [(0, '9.780'), (1, '12.600')] [2023-10-08 08:03:04,830][53885] Updated weights for policy 1, policy_version 5350 (0.0010) [2023-10-08 08:03:05,200][53885] Updated weights for policy 1, policy_version 5360 (0.0009) [2023-10-08 08:03:05,566][53885] Updated weights for policy 1, policy_version 5370 (0.0008) [2023-10-08 08:03:05,624][53852] Updated weights for policy 0, policy_version 5380 (0.0007) [2023-10-08 08:03:05,983][53852] Updated weights for policy 0, policy_version 5390 (0.0009) [2023-10-08 08:03:06,349][53852] Updated weights for policy 0, policy_version 5400 (0.0013) [2023-10-08 08:03:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 11042816. Throughput: 0: 1816.6, 1: 1792.2. Samples: 2766938. Policy #0 lag: (min: 3.0, avg: 12.7, max: 35.0) [2023-10-08 08:03:07,016][52710] Avg episode reward: [(0, '10.320'), (1, '13.080')] [2023-10-08 08:03:07,027][53500] Saving new best policy, reward=10.320! [2023-10-08 08:03:09,423][53885] Updated weights for policy 1, policy_version 5380 (0.0007) [2023-10-08 08:03:09,788][53885] Updated weights for policy 1, policy_version 5390 (0.0008) [2023-10-08 08:03:09,951][53852] Updated weights for policy 0, policy_version 5410 (0.0009) [2023-10-08 08:03:10,164][53885] Updated weights for policy 1, policy_version 5400 (0.0007) [2023-10-08 08:03:10,319][53852] Updated weights for policy 0, policy_version 5420 (0.0007) [2023-10-08 08:03:10,693][53852] Updated weights for policy 0, policy_version 5430 (0.0007) [2023-10-08 08:03:11,054][53852] Updated weights for policy 0, policy_version 5440 (0.0007) [2023-10-08 08:03:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11108352. Throughput: 0: 1810.7, 1: 1805.9. Samples: 2779280. Policy #0 lag: (min: 3.0, avg: 12.7, max: 35.0) [2023-10-08 08:03:12,016][52710] Avg episode reward: [(0, '9.970'), (1, '11.890')] [2023-10-08 08:03:13,844][53885] Updated weights for policy 1, policy_version 5410 (0.0007) [2023-10-08 08:03:14,214][53885] Updated weights for policy 1, policy_version 5420 (0.0007) [2023-10-08 08:03:14,531][53852] Updated weights for policy 0, policy_version 5450 (0.0009) [2023-10-08 08:03:14,578][53885] Updated weights for policy 1, policy_version 5430 (0.0007) [2023-10-08 08:03:14,903][53852] Updated weights for policy 0, policy_version 5460 (0.0008) [2023-10-08 08:03:14,944][53885] Updated weights for policy 1, policy_version 5440 (0.0007) [2023-10-08 08:03:15,278][53852] Updated weights for policy 0, policy_version 5470 (0.0009) [2023-10-08 08:03:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 11173888. Throughput: 0: 1816.6, 1: 1800.2. Samples: 2799696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:03:17,015][52710] Avg episode reward: [(0, '10.140'), (1, '12.400')] [2023-10-08 08:03:18,609][53885] Updated weights for policy 1, policy_version 5450 (0.0011) [2023-10-08 08:03:18,961][53885] Updated weights for policy 1, policy_version 5460 (0.0008) [2023-10-08 08:03:19,222][53852] Updated weights for policy 0, policy_version 5480 (0.0009) [2023-10-08 08:03:19,329][53885] Updated weights for policy 1, policy_version 5470 (0.0008) [2023-10-08 08:03:19,590][53852] Updated weights for policy 0, policy_version 5490 (0.0007) [2023-10-08 08:03:19,968][53852] Updated weights for policy 0, policy_version 5500 (0.0008) [2023-10-08 08:03:22,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11239424. Throughput: 0: 1824.1, 1: 1794.5. Samples: 2822562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:03:22,016][52710] Avg episode reward: [(0, '10.880'), (1, '12.390')] [2023-10-08 08:03:22,026][53500] Saving new best policy, reward=10.880! [2023-10-08 08:03:23,110][53885] Updated weights for policy 1, policy_version 5480 (0.0007) [2023-10-08 08:03:23,474][53885] Updated weights for policy 1, policy_version 5490 (0.0008) [2023-10-08 08:03:23,526][53852] Updated weights for policy 0, policy_version 5510 (0.0007) [2023-10-08 08:03:23,842][53885] Updated weights for policy 1, policy_version 5500 (0.0009) [2023-10-08 08:03:23,885][53852] Updated weights for policy 0, policy_version 5520 (0.0007) [2023-10-08 08:03:24,263][53852] Updated weights for policy 0, policy_version 5530 (0.0010) [2023-10-08 08:03:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11304960. Throughput: 0: 1822.4, 1: 1789.0. Samples: 2832390. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:03:27,015][52710] Avg episode reward: [(0, '10.830'), (1, '13.030')] [2023-10-08 08:03:27,569][53885] Updated weights for policy 1, policy_version 5510 (0.0007) [2023-10-08 08:03:27,792][53852] Updated weights for policy 0, policy_version 5540 (0.0007) [2023-10-08 08:03:27,936][53885] Updated weights for policy 1, policy_version 5520 (0.0007) [2023-10-08 08:03:28,166][53852] Updated weights for policy 0, policy_version 5550 (0.0008) [2023-10-08 08:03:28,310][53885] Updated weights for policy 1, policy_version 5530 (0.0007) [2023-10-08 08:03:28,542][53852] Updated weights for policy 0, policy_version 5560 (0.0009) [2023-10-08 08:03:31,955][53885] Updated weights for policy 1, policy_version 5540 (0.0007) [2023-10-08 08:03:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11370496. Throughput: 0: 1833.9, 1: 1792.3. Samples: 2855586. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:03:32,015][52710] Avg episode reward: [(0, '10.700'), (1, '12.210')] [2023-10-08 08:03:32,179][53852] Updated weights for policy 0, policy_version 5570 (0.0009) [2023-10-08 08:03:32,328][53885] Updated weights for policy 1, policy_version 5550 (0.0008) [2023-10-08 08:03:32,558][53852] Updated weights for policy 0, policy_version 5580 (0.0007) [2023-10-08 08:03:32,697][53885] Updated weights for policy 1, policy_version 5560 (0.0009) [2023-10-08 08:03:32,931][53852] Updated weights for policy 0, policy_version 5590 (0.0007) [2023-10-08 08:03:33,309][53852] Updated weights for policy 0, policy_version 5600 (0.0008) [2023-10-08 08:03:36,423][53885] Updated weights for policy 1, policy_version 5570 (0.0009) [2023-10-08 08:03:36,791][53885] Updated weights for policy 1, policy_version 5580 (0.0008) [2023-10-08 08:03:37,008][53852] Updated weights for policy 0, policy_version 5610 (0.0007) [2023-10-08 08:03:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11436032. Throughput: 0: 1826.8, 1: 1812.4. Samples: 2877736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:03:37,015][52710] Avg episode reward: [(0, '11.200'), (1, '13.890')] [2023-10-08 08:03:37,168][53885] Updated weights for policy 1, policy_version 5590 (0.0007) [2023-10-08 08:03:37,383][53852] Updated weights for policy 0, policy_version 5620 (0.0007) [2023-10-08 08:03:37,530][53594] Saving new best policy, reward=13.890! [2023-10-08 08:03:37,531][53885] Updated weights for policy 1, policy_version 5600 (0.0009) [2023-10-08 08:03:37,752][53852] Updated weights for policy 0, policy_version 5630 (0.0010) [2023-10-08 08:03:37,823][53500] Saving new best policy, reward=11.200! [2023-10-08 08:03:41,200][53885] Updated weights for policy 1, policy_version 5610 (0.0009) [2023-10-08 08:03:41,393][53852] Updated weights for policy 0, policy_version 5640 (0.0009) [2023-10-08 08:03:41,572][53885] Updated weights for policy 1, policy_version 5620 (0.0009) [2023-10-08 08:03:41,761][53852] Updated weights for policy 0, policy_version 5650 (0.0007) [2023-10-08 08:03:41,941][53885] Updated weights for policy 1, policy_version 5630 (0.0008) [2023-10-08 08:03:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11534336. Throughput: 0: 1832.9, 1: 1804.8. Samples: 2888036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:03:42,016][52710] Avg episode reward: [(0, '10.640'), (1, '12.630')] [2023-10-08 08:03:42,141][53852] Updated weights for policy 0, policy_version 5660 (0.0010) [2023-10-08 08:03:45,737][53885] Updated weights for policy 1, policy_version 5640 (0.0008) [2023-10-08 08:03:45,819][53852] Updated weights for policy 0, policy_version 5670 (0.0010) [2023-10-08 08:03:46,093][53885] Updated weights for policy 1, policy_version 5650 (0.0008) [2023-10-08 08:03:46,187][53852] Updated weights for policy 0, policy_version 5680 (0.0007) [2023-10-08 08:03:46,460][53885] Updated weights for policy 1, policy_version 5660 (0.0007) [2023-10-08 08:03:46,559][53852] Updated weights for policy 0, policy_version 5690 (0.0007) [2023-10-08 08:03:47,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 11632640. Throughput: 0: 1831.1, 1: 1816.3. Samples: 2910594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:03:47,016][52710] Avg episode reward: [(0, '10.880'), (1, '12.420')] [2023-10-08 08:03:50,202][53852] Updated weights for policy 0, policy_version 5700 (0.0009) [2023-10-08 08:03:50,266][53885] Updated weights for policy 1, policy_version 5670 (0.0009) [2023-10-08 08:03:50,568][53852] Updated weights for policy 0, policy_version 5710 (0.0008) [2023-10-08 08:03:50,635][53885] Updated weights for policy 1, policy_version 5680 (0.0009) [2023-10-08 08:03:50,945][53852] Updated weights for policy 0, policy_version 5720 (0.0010) [2023-10-08 08:03:51,007][53885] Updated weights for policy 1, policy_version 5690 (0.0008) [2023-10-08 08:03:52,015][52710] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 11698176. Throughput: 0: 1821.2, 1: 1810.6. Samples: 2930372. Policy #0 lag: (min: 1.0, avg: 3.0, max: 26.0) [2023-10-08 08:03:52,017][52710] Avg episode reward: [(0, '11.660'), (1, '11.820')] [2023-10-08 08:03:52,028][53500] Saving new best policy, reward=11.660! [2023-10-08 08:03:54,609][53885] Updated weights for policy 1, policy_version 5700 (0.0007) [2023-10-08 08:03:54,692][53852] Updated weights for policy 0, policy_version 5730 (0.0009) [2023-10-08 08:03:54,985][53885] Updated weights for policy 1, policy_version 5710 (0.0007) [2023-10-08 08:03:55,063][53852] Updated weights for policy 0, policy_version 5740 (0.0007) [2023-10-08 08:03:55,352][53885] Updated weights for policy 1, policy_version 5720 (0.0010) [2023-10-08 08:03:55,435][53852] Updated weights for policy 0, policy_version 5750 (0.0008) [2023-10-08 08:03:55,805][53852] Updated weights for policy 0, policy_version 5760 (0.0008) [2023-10-08 08:03:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11763712. Throughput: 0: 1827.1, 1: 1820.5. Samples: 2943420. Policy #0 lag: (min: 1.0, avg: 3.0, max: 26.0) [2023-10-08 08:03:57,016][52710] Avg episode reward: [(0, '11.850'), (1, '12.400')] [2023-10-08 08:03:57,017][53500] Saving new best policy, reward=11.850! [2023-10-08 08:03:58,963][53885] Updated weights for policy 1, policy_version 5730 (0.0008) [2023-10-08 08:03:59,335][53885] Updated weights for policy 1, policy_version 5740 (0.0007) [2023-10-08 08:03:59,457][53852] Updated weights for policy 0, policy_version 5770 (0.0007) [2023-10-08 08:03:59,700][53885] Updated weights for policy 1, policy_version 5750 (0.0007) [2023-10-08 08:03:59,822][53852] Updated weights for policy 0, policy_version 5780 (0.0007) [2023-10-08 08:04:00,053][53885] Updated weights for policy 1, policy_version 5760 (0.0007) [2023-10-08 08:04:00,197][53852] Updated weights for policy 0, policy_version 5790 (0.0008) [2023-10-08 08:04:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11829248. Throughput: 0: 1829.6, 1: 1808.6. Samples: 2963418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:04:02,016][52710] Avg episode reward: [(0, '12.080'), (1, '13.910')] [2023-10-08 08:04:02,018][53594] Saving new best policy, reward=13.910! [2023-10-08 08:04:02,018][53500] Saving new best policy, reward=12.080! [2023-10-08 08:04:03,772][53885] Updated weights for policy 1, policy_version 5770 (0.0008) [2023-10-08 08:04:03,780][53852] Updated weights for policy 0, policy_version 5800 (0.0008) [2023-10-08 08:04:04,134][53885] Updated weights for policy 1, policy_version 5780 (0.0007) [2023-10-08 08:04:04,157][53852] Updated weights for policy 0, policy_version 5810 (0.0007) [2023-10-08 08:04:04,500][53885] Updated weights for policy 1, policy_version 5790 (0.0007) [2023-10-08 08:04:04,515][53852] Updated weights for policy 0, policy_version 5820 (0.0007) [2023-10-08 08:04:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11894784. Throughput: 0: 1832.7, 1: 1809.4. Samples: 2986454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:04:07,015][52710] Avg episode reward: [(0, '11.560'), (1, '13.390')] [2023-10-08 08:04:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000005824_5963776.pth... [2023-10-08 08:04:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000005792_5931008.pth... [2023-10-08 08:04:07,064][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000004128_4227072.pth [2023-10-08 08:04:07,066][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000004096_4194304.pth [2023-10-08 08:04:07,982][53852] Updated weights for policy 0, policy_version 5830 (0.0007) [2023-10-08 08:04:08,237][53885] Updated weights for policy 1, policy_version 5800 (0.0009) [2023-10-08 08:04:08,351][53852] Updated weights for policy 0, policy_version 5840 (0.0007) [2023-10-08 08:04:08,609][53885] Updated weights for policy 1, policy_version 5810 (0.0009) [2023-10-08 08:04:08,728][53852] Updated weights for policy 0, policy_version 5850 (0.0007) [2023-10-08 08:04:08,974][53885] Updated weights for policy 1, policy_version 5820 (0.0007) [2023-10-08 08:04:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11960320. Throughput: 0: 1832.7, 1: 1811.1. Samples: 2996360. Policy #0 lag: (min: 17.0, avg: 32.9, max: 49.0) [2023-10-08 08:04:12,016][52710] Avg episode reward: [(0, '12.650'), (1, '13.880')] [2023-10-08 08:04:12,017][53500] Saving new best policy, reward=12.650! [2023-10-08 08:04:12,534][53852] Updated weights for policy 0, policy_version 5860 (0.0009) [2023-10-08 08:04:12,753][53885] Updated weights for policy 1, policy_version 5830 (0.0009) [2023-10-08 08:04:12,905][53852] Updated weights for policy 0, policy_version 5870 (0.0010) [2023-10-08 08:04:13,121][53885] Updated weights for policy 1, policy_version 5840 (0.0009) [2023-10-08 08:04:13,278][53852] Updated weights for policy 0, policy_version 5880 (0.0008) [2023-10-08 08:04:13,492][53885] Updated weights for policy 1, policy_version 5850 (0.0008) [2023-10-08 08:04:16,874][53852] Updated weights for policy 0, policy_version 5890 (0.0008) [2023-10-08 08:04:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12025856. Throughput: 0: 1829.0, 1: 1806.0. Samples: 3019162. Policy #0 lag: (min: 17.0, avg: 32.9, max: 49.0) [2023-10-08 08:04:17,015][52710] Avg episode reward: [(0, '11.130'), (1, '14.120')] [2023-10-08 08:04:17,016][53594] Saving new best policy, reward=14.120! [2023-10-08 08:04:17,250][53852] Updated weights for policy 0, policy_version 5900 (0.0007) [2023-10-08 08:04:17,290][53885] Updated weights for policy 1, policy_version 5860 (0.0009) [2023-10-08 08:04:17,618][53852] Updated weights for policy 0, policy_version 5910 (0.0008) [2023-10-08 08:04:17,656][53885] Updated weights for policy 1, policy_version 5870 (0.0007) [2023-10-08 08:04:17,981][53852] Updated weights for policy 0, policy_version 5920 (0.0008) [2023-10-08 08:04:18,022][53885] Updated weights for policy 1, policy_version 5880 (0.0007) [2023-10-08 08:04:21,718][53852] Updated weights for policy 0, policy_version 5930 (0.0008) [2023-10-08 08:04:21,774][53885] Updated weights for policy 1, policy_version 5890 (0.0008) [2023-10-08 08:04:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12091392. Throughput: 0: 1831.6, 1: 1812.5. Samples: 3041724. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 08:04:22,016][52710] Avg episode reward: [(0, '11.840'), (1, '14.550')] [2023-10-08 08:04:22,089][53852] Updated weights for policy 0, policy_version 5940 (0.0008) [2023-10-08 08:04:22,145][53885] Updated weights for policy 1, policy_version 5900 (0.0008) [2023-10-08 08:04:22,457][53852] Updated weights for policy 0, policy_version 5950 (0.0007) [2023-10-08 08:04:22,509][53885] Updated weights for policy 1, policy_version 5910 (0.0007) [2023-10-08 08:04:22,875][53594] Saving new best policy, reward=14.550! [2023-10-08 08:04:22,877][53885] Updated weights for policy 1, policy_version 5920 (0.0008) [2023-10-08 08:04:26,180][53852] Updated weights for policy 0, policy_version 5960 (0.0008) [2023-10-08 08:04:26,549][53852] Updated weights for policy 0, policy_version 5970 (0.0007) [2023-10-08 08:04:26,683][53885] Updated weights for policy 1, policy_version 5930 (0.0008) [2023-10-08 08:04:26,922][53852] Updated weights for policy 0, policy_version 5980 (0.0007) [2023-10-08 08:04:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 12156928. Throughput: 0: 1832.1, 1: 1807.2. Samples: 3051804. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 08:04:27,016][52710] Avg episode reward: [(0, '11.880'), (1, '14.490')] [2023-10-08 08:04:27,040][53885] Updated weights for policy 1, policy_version 5940 (0.0007) [2023-10-08 08:04:27,403][53885] Updated weights for policy 1, policy_version 5950 (0.0007) [2023-10-08 08:04:30,626][53852] Updated weights for policy 0, policy_version 5990 (0.0009) [2023-10-08 08:04:30,956][53885] Updated weights for policy 1, policy_version 5960 (0.0007) [2023-10-08 08:04:30,993][53852] Updated weights for policy 0, policy_version 6000 (0.0009) [2023-10-08 08:04:31,331][53885] Updated weights for policy 1, policy_version 5970 (0.0009) [2023-10-08 08:04:31,366][53852] Updated weights for policy 0, policy_version 6010 (0.0008) [2023-10-08 08:04:31,703][53885] Updated weights for policy 1, policy_version 5980 (0.0009) [2023-10-08 08:04:32,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 12288000. Throughput: 0: 1826.5, 1: 1811.6. Samples: 3074310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:04:32,016][52710] Avg episode reward: [(0, '13.190'), (1, '14.160')] [2023-10-08 08:04:32,018][53500] Saving new best policy, reward=13.190! [2023-10-08 08:04:35,190][53852] Updated weights for policy 0, policy_version 6020 (0.0008) [2023-10-08 08:04:35,464][53885] Updated weights for policy 1, policy_version 5990 (0.0009) [2023-10-08 08:04:35,556][53852] Updated weights for policy 0, policy_version 6030 (0.0007) [2023-10-08 08:04:35,829][53885] Updated weights for policy 1, policy_version 6000 (0.0011) [2023-10-08 08:04:35,929][53852] Updated weights for policy 0, policy_version 6040 (0.0008) [2023-10-08 08:04:36,196][53885] Updated weights for policy 1, policy_version 6010 (0.0008) [2023-10-08 08:04:37,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 12353536. Throughput: 0: 1832.5, 1: 1804.9. Samples: 3094056. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-08 08:04:37,016][52710] Avg episode reward: [(0, '12.280'), (1, '12.300')] [2023-10-08 08:04:39,646][53852] Updated weights for policy 0, policy_version 6050 (0.0007) [2023-10-08 08:04:39,811][53885] Updated weights for policy 1, policy_version 6020 (0.0007) [2023-10-08 08:04:40,022][53852] Updated weights for policy 0, policy_version 6060 (0.0008) [2023-10-08 08:04:40,172][53885] Updated weights for policy 1, policy_version 6030 (0.0008) [2023-10-08 08:04:40,392][53852] Updated weights for policy 0, policy_version 6070 (0.0008) [2023-10-08 08:04:40,534][53885] Updated weights for policy 1, policy_version 6040 (0.0007) [2023-10-08 08:04:40,774][53852] Updated weights for policy 0, policy_version 6080 (0.0007) [2023-10-08 08:04:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 12419072. Throughput: 0: 1827.2, 1: 1807.5. Samples: 3106984. Policy #0 lag: (min: 31.0, avg: 31.9, max: 51.0) [2023-10-08 08:04:42,016][52710] Avg episode reward: [(0, '12.140'), (1, '13.640')] [2023-10-08 08:04:44,251][53885] Updated weights for policy 1, policy_version 6050 (0.0009) [2023-10-08 08:04:44,453][53852] Updated weights for policy 0, policy_version 6090 (0.0007) [2023-10-08 08:04:44,624][53885] Updated weights for policy 1, policy_version 6060 (0.0009) [2023-10-08 08:04:44,821][53852] Updated weights for policy 0, policy_version 6100 (0.0007) [2023-10-08 08:04:44,981][53885] Updated weights for policy 1, policy_version 6070 (0.0009) [2023-10-08 08:04:45,206][53852] Updated weights for policy 0, policy_version 6110 (0.0008) [2023-10-08 08:04:45,353][53885] Updated weights for policy 1, policy_version 6080 (0.0009) [2023-10-08 08:04:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 12484608. Throughput: 0: 1821.5, 1: 1803.6. Samples: 3126546. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-08 08:04:47,016][52710] Avg episode reward: [(0, '12.520'), (1, '12.290')] [2023-10-08 08:04:48,930][53852] Updated weights for policy 0, policy_version 6120 (0.0008) [2023-10-08 08:04:48,992][53885] Updated weights for policy 1, policy_version 6090 (0.0007) [2023-10-08 08:04:49,309][53852] Updated weights for policy 0, policy_version 6130 (0.0008) [2023-10-08 08:04:49,359][53885] Updated weights for policy 1, policy_version 6100 (0.0008) [2023-10-08 08:04:49,689][53852] Updated weights for policy 0, policy_version 6140 (0.0010) [2023-10-08 08:04:49,725][53885] Updated weights for policy 1, policy_version 6110 (0.0009) [2023-10-08 08:04:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12550144. Throughput: 0: 1814.7, 1: 1805.3. Samples: 3149354. Policy #0 lag: (min: 40.0, avg: 55.0, max: 56.0) [2023-10-08 08:04:52,016][52710] Avg episode reward: [(0, '11.830'), (1, '12.820')] [2023-10-08 08:04:53,339][53852] Updated weights for policy 0, policy_version 6150 (0.0007) [2023-10-08 08:04:53,542][53885] Updated weights for policy 1, policy_version 6120 (0.0007) [2023-10-08 08:04:53,705][53852] Updated weights for policy 0, policy_version 6160 (0.0008) [2023-10-08 08:04:53,916][53885] Updated weights for policy 1, policy_version 6130 (0.0009) [2023-10-08 08:04:54,078][53852] Updated weights for policy 0, policy_version 6170 (0.0009) [2023-10-08 08:04:54,283][53885] Updated weights for policy 1, policy_version 6140 (0.0008) [2023-10-08 08:04:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12615680. Throughput: 0: 1810.7, 1: 1804.5. Samples: 3159042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:04:57,016][52710] Avg episode reward: [(0, '12.730'), (1, '14.040')] [2023-10-08 08:04:57,820][53852] Updated weights for policy 0, policy_version 6180 (0.0008) [2023-10-08 08:04:57,865][53885] Updated weights for policy 1, policy_version 6150 (0.0008) [2023-10-08 08:04:58,188][53852] Updated weights for policy 0, policy_version 6190 (0.0008) [2023-10-08 08:04:58,237][53885] Updated weights for policy 1, policy_version 6160 (0.0008) [2023-10-08 08:04:58,556][53852] Updated weights for policy 0, policy_version 6200 (0.0009) [2023-10-08 08:04:58,598][53885] Updated weights for policy 1, policy_version 6170 (0.0007) [2023-10-08 08:05:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12681216. Throughput: 0: 1802.4, 1: 1807.8. Samples: 3181622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:05:02,016][52710] Avg episode reward: [(0, '12.740'), (1, '13.900')] [2023-10-08 08:05:02,325][53852] Updated weights for policy 0, policy_version 6210 (0.0008) [2023-10-08 08:05:02,338][53885] Updated weights for policy 1, policy_version 6180 (0.0008) [2023-10-08 08:05:02,703][53852] Updated weights for policy 0, policy_version 6220 (0.0007) [2023-10-08 08:05:02,704][53885] Updated weights for policy 1, policy_version 6190 (0.0007) [2023-10-08 08:05:03,064][53852] Updated weights for policy 0, policy_version 6230 (0.0007) [2023-10-08 08:05:03,067][53885] Updated weights for policy 1, policy_version 6200 (0.0008) [2023-10-08 08:05:03,433][53852] Updated weights for policy 0, policy_version 6240 (0.0008) [2023-10-08 08:05:06,863][53885] Updated weights for policy 1, policy_version 6210 (0.0008) [2023-10-08 08:05:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12746752. Throughput: 0: 1809.8, 1: 1808.8. Samples: 3204562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:05:07,015][52710] Avg episode reward: [(0, '12.390'), (1, '14.870')] [2023-10-08 08:05:07,197][53852] Updated weights for policy 0, policy_version 6250 (0.0008) [2023-10-08 08:05:07,231][53885] Updated weights for policy 1, policy_version 6220 (0.0007) [2023-10-08 08:05:07,571][53852] Updated weights for policy 0, policy_version 6260 (0.0008) [2023-10-08 08:05:07,599][53885] Updated weights for policy 1, policy_version 6230 (0.0007) [2023-10-08 08:05:07,947][53852] Updated weights for policy 0, policy_version 6270 (0.0009) [2023-10-08 08:05:07,964][53594] Saving new best policy, reward=14.870! [2023-10-08 08:05:07,965][53885] Updated weights for policy 1, policy_version 6240 (0.0007) [2023-10-08 08:05:11,518][53852] Updated weights for policy 0, policy_version 6280 (0.0008) [2023-10-08 08:05:11,628][53885] Updated weights for policy 1, policy_version 6250 (0.0008) [2023-10-08 08:05:11,898][53852] Updated weights for policy 0, policy_version 6290 (0.0010) [2023-10-08 08:05:11,997][53885] Updated weights for policy 1, policy_version 6260 (0.0009) [2023-10-08 08:05:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 12812288. Throughput: 0: 1804.5, 1: 1809.5. Samples: 3214432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:05:12,016][52710] Avg episode reward: [(0, '13.110'), (1, '14.750')] [2023-10-08 08:05:12,266][53852] Updated weights for policy 0, policy_version 6300 (0.0008) [2023-10-08 08:05:12,370][53885] Updated weights for policy 1, policy_version 6270 (0.0007) [2023-10-08 08:05:15,775][53852] Updated weights for policy 0, policy_version 6310 (0.0008) [2023-10-08 08:05:16,047][53885] Updated weights for policy 1, policy_version 6280 (0.0007) [2023-10-08 08:05:16,151][53852] Updated weights for policy 0, policy_version 6320 (0.0007) [2023-10-08 08:05:16,413][53885] Updated weights for policy 1, policy_version 6290 (0.0009) [2023-10-08 08:05:16,518][53852] Updated weights for policy 0, policy_version 6330 (0.0007) [2023-10-08 08:05:16,777][53885] Updated weights for policy 1, policy_version 6300 (0.0007) [2023-10-08 08:05:17,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 12943360. Throughput: 0: 1817.7, 1: 1810.7. Samples: 3237584. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 08:05:17,016][52710] Avg episode reward: [(0, '12.660'), (1, '14.760')] [2023-10-08 08:05:20,259][53852] Updated weights for policy 0, policy_version 6340 (0.0008) [2023-10-08 08:05:20,416][53885] Updated weights for policy 1, policy_version 6310 (0.0009) [2023-10-08 08:05:20,630][53852] Updated weights for policy 0, policy_version 6350 (0.0009) [2023-10-08 08:05:20,782][53885] Updated weights for policy 1, policy_version 6320 (0.0007) [2023-10-08 08:05:21,006][53852] Updated weights for policy 0, policy_version 6360 (0.0009) [2023-10-08 08:05:21,148][53885] Updated weights for policy 1, policy_version 6330 (0.0008) [2023-10-08 08:05:22,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 13008896. Throughput: 0: 1817.8, 1: 1813.9. Samples: 3257484. Policy #0 lag: (min: 6.0, avg: 8.2, max: 38.0) [2023-10-08 08:05:22,016][52710] Avg episode reward: [(0, '14.380'), (1, '14.890')] [2023-10-08 08:05:22,025][53500] Saving new best policy, reward=14.380! [2023-10-08 08:05:22,025][53594] Saving new best policy, reward=14.890! [2023-10-08 08:05:24,666][53852] Updated weights for policy 0, policy_version 6370 (0.0007) [2023-10-08 08:05:24,809][53885] Updated weights for policy 1, policy_version 6340 (0.0009) [2023-10-08 08:05:25,040][53852] Updated weights for policy 0, policy_version 6380 (0.0008) [2023-10-08 08:05:25,184][53885] Updated weights for policy 1, policy_version 6350 (0.0009) [2023-10-08 08:05:25,404][53852] Updated weights for policy 0, policy_version 6390 (0.0007) [2023-10-08 08:05:25,550][53885] Updated weights for policy 1, policy_version 6360 (0.0009) [2023-10-08 08:05:25,773][53852] Updated weights for policy 0, policy_version 6400 (0.0007) [2023-10-08 08:05:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 13074432. Throughput: 0: 1821.6, 1: 1814.0. Samples: 3270582. Policy #0 lag: (min: 6.0, avg: 8.2, max: 38.0) [2023-10-08 08:05:27,016][52710] Avg episode reward: [(0, '12.780'), (1, '15.840')] [2023-10-08 08:05:27,017][53594] Saving new best policy, reward=15.840! [2023-10-08 08:05:29,297][53885] Updated weights for policy 1, policy_version 6370 (0.0007) [2023-10-08 08:05:29,517][53852] Updated weights for policy 0, policy_version 6410 (0.0008) [2023-10-08 08:05:29,655][53885] Updated weights for policy 1, policy_version 6380 (0.0009) [2023-10-08 08:05:29,895][53852] Updated weights for policy 0, policy_version 6420 (0.0008) [2023-10-08 08:05:30,020][53885] Updated weights for policy 1, policy_version 6390 (0.0008) [2023-10-08 08:05:30,259][53852] Updated weights for policy 0, policy_version 6430 (0.0008) [2023-10-08 08:05:30,383][53885] Updated weights for policy 1, policy_version 6400 (0.0010) [2023-10-08 08:05:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 13139968. Throughput: 0: 1824.1, 1: 1817.9. Samples: 3290438. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:05:32,016][52710] Avg episode reward: [(0, '13.090'), (1, '15.160')] [2023-10-08 08:05:33,843][53852] Updated weights for policy 0, policy_version 6440 (0.0009) [2023-10-08 08:05:34,215][53852] Updated weights for policy 0, policy_version 6450 (0.0008) [2023-10-08 08:05:34,297][53885] Updated weights for policy 1, policy_version 6410 (0.0008) [2023-10-08 08:05:34,596][53852] Updated weights for policy 0, policy_version 6460 (0.0009) [2023-10-08 08:05:34,669][53885] Updated weights for policy 1, policy_version 6420 (0.0008) [2023-10-08 08:05:35,041][53885] Updated weights for policy 1, policy_version 6430 (0.0010) [2023-10-08 08:05:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 13205504. Throughput: 0: 1832.1, 1: 1812.1. Samples: 3313342. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:05:37,015][52710] Avg episode reward: [(0, '14.340'), (1, '14.830')] [2023-10-08 08:05:38,140][53852] Updated weights for policy 0, policy_version 6470 (0.0008) [2023-10-08 08:05:38,508][53852] Updated weights for policy 0, policy_version 6480 (0.0007) [2023-10-08 08:05:38,698][53885] Updated weights for policy 1, policy_version 6440 (0.0009) [2023-10-08 08:05:38,884][53852] Updated weights for policy 0, policy_version 6490 (0.0009) [2023-10-08 08:05:39,080][53885] Updated weights for policy 1, policy_version 6450 (0.0008) [2023-10-08 08:05:39,446][53885] Updated weights for policy 1, policy_version 6460 (0.0009) [2023-10-08 08:05:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13271040. Throughput: 0: 1833.4, 1: 1815.9. Samples: 3323262. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-10-08 08:05:42,016][52710] Avg episode reward: [(0, '13.610'), (1, '13.980')] [2023-10-08 08:05:42,672][53852] Updated weights for policy 0, policy_version 6500 (0.0008) [2023-10-08 08:05:43,040][53852] Updated weights for policy 0, policy_version 6510 (0.0007) [2023-10-08 08:05:43,044][53885] Updated weights for policy 1, policy_version 6470 (0.0008) [2023-10-08 08:05:43,411][53852] Updated weights for policy 0, policy_version 6520 (0.0008) [2023-10-08 08:05:43,413][53885] Updated weights for policy 1, policy_version 6480 (0.0009) [2023-10-08 08:05:43,792][53885] Updated weights for policy 1, policy_version 6490 (0.0010) [2023-10-08 08:05:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13336576. Throughput: 0: 1834.0, 1: 1816.7. Samples: 3345902. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-10-08 08:05:47,016][52710] Avg episode reward: [(0, '14.050'), (1, '14.640')] [2023-10-08 08:05:47,061][53852] Updated weights for policy 0, policy_version 6530 (0.0010) [2023-10-08 08:05:47,432][53852] Updated weights for policy 0, policy_version 6540 (0.0010) [2023-10-08 08:05:47,679][53885] Updated weights for policy 1, policy_version 6500 (0.0010) [2023-10-08 08:05:47,801][53852] Updated weights for policy 0, policy_version 6550 (0.0010) [2023-10-08 08:05:48,046][53885] Updated weights for policy 1, policy_version 6510 (0.0009) [2023-10-08 08:05:48,174][53852] Updated weights for policy 0, policy_version 6560 (0.0008) [2023-10-08 08:05:48,421][53885] Updated weights for policy 1, policy_version 6520 (0.0008) [2023-10-08 08:05:51,851][53852] Updated weights for policy 0, policy_version 6570 (0.0008) [2023-10-08 08:05:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13402112. Throughput: 0: 1832.3, 1: 1810.4. Samples: 3368484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:05:52,016][52710] Avg episode reward: [(0, '13.160'), (1, '15.220')] [2023-10-08 08:05:52,067][53885] Updated weights for policy 1, policy_version 6530 (0.0009) [2023-10-08 08:05:52,229][53852] Updated weights for policy 0, policy_version 6580 (0.0008) [2023-10-08 08:05:52,442][53885] Updated weights for policy 1, policy_version 6540 (0.0007) [2023-10-08 08:05:52,596][53852] Updated weights for policy 0, policy_version 6590 (0.0008) [2023-10-08 08:05:52,807][53885] Updated weights for policy 1, policy_version 6550 (0.0008) [2023-10-08 08:05:53,170][53885] Updated weights for policy 1, policy_version 6560 (0.0010) [2023-10-08 08:05:56,281][53852] Updated weights for policy 0, policy_version 6600 (0.0009) [2023-10-08 08:05:56,659][53852] Updated weights for policy 0, policy_version 6610 (0.0007) [2023-10-08 08:05:56,899][53885] Updated weights for policy 1, policy_version 6570 (0.0009) [2023-10-08 08:05:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13467648. Throughput: 0: 1834.8, 1: 1808.7. Samples: 3378390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:05:57,016][52710] Avg episode reward: [(0, '14.300'), (1, '16.240')] [2023-10-08 08:05:57,020][53852] Updated weights for policy 0, policy_version 6620 (0.0008) [2023-10-08 08:05:57,260][53885] Updated weights for policy 1, policy_version 6580 (0.0008) [2023-10-08 08:05:57,622][53885] Updated weights for policy 1, policy_version 6590 (0.0008) [2023-10-08 08:05:57,695][53594] Saving new best policy, reward=16.240! [2023-10-08 08:06:00,703][53852] Updated weights for policy 0, policy_version 6630 (0.0009) [2023-10-08 08:06:01,061][53852] Updated weights for policy 0, policy_version 6640 (0.0007) [2023-10-08 08:06:01,308][53885] Updated weights for policy 1, policy_version 6600 (0.0008) [2023-10-08 08:06:01,429][53852] Updated weights for policy 0, policy_version 6650 (0.0008) [2023-10-08 08:06:01,669][53885] Updated weights for policy 1, policy_version 6610 (0.0007) [2023-10-08 08:06:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13565952. Throughput: 0: 1830.4, 1: 1810.3. Samples: 3401414. Policy #0 lag: (min: 15.0, avg: 17.1, max: 46.0) [2023-10-08 08:06:02,016][52710] Avg episode reward: [(0, '15.120'), (1, '15.360')] [2023-10-08 08:06:02,017][53500] Saving new best policy, reward=15.120! [2023-10-08 08:06:02,039][53885] Updated weights for policy 1, policy_version 6620 (0.0009) [2023-10-08 08:06:05,096][53852] Updated weights for policy 0, policy_version 6660 (0.0008) [2023-10-08 08:06:05,473][53852] Updated weights for policy 0, policy_version 6670 (0.0007) [2023-10-08 08:06:05,654][53885] Updated weights for policy 1, policy_version 6630 (0.0009) [2023-10-08 08:06:05,840][53852] Updated weights for policy 0, policy_version 6680 (0.0009) [2023-10-08 08:06:06,016][53885] Updated weights for policy 1, policy_version 6640 (0.0008) [2023-10-08 08:06:06,394][53885] Updated weights for policy 1, policy_version 6650 (0.0008) [2023-10-08 08:06:07,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 13664256. Throughput: 0: 1829.7, 1: 1814.1. Samples: 3421452. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:06:07,015][52710] Avg episode reward: [(0, '14.140'), (1, '14.640')] [2023-10-08 08:06:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000006656_6815744.pth... [2023-10-08 08:06:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000006688_6848512.pth... [2023-10-08 08:06:07,060][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000004992_5111808.pth [2023-10-08 08:06:07,063][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000004960_5079040.pth [2023-10-08 08:06:09,454][53852] Updated weights for policy 0, policy_version 6690 (0.0007) [2023-10-08 08:06:09,829][53852] Updated weights for policy 0, policy_version 6700 (0.0008) [2023-10-08 08:06:10,107][53885] Updated weights for policy 1, policy_version 6660 (0.0007) [2023-10-08 08:06:10,194][53852] Updated weights for policy 0, policy_version 6710 (0.0008) [2023-10-08 08:06:10,473][53885] Updated weights for policy 1, policy_version 6670 (0.0007) [2023-10-08 08:06:10,563][53852] Updated weights for policy 0, policy_version 6720 (0.0007) [2023-10-08 08:06:10,844][53885] Updated weights for policy 1, policy_version 6680 (0.0008) [2023-10-08 08:06:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 13729792. Throughput: 0: 1824.2, 1: 1809.6. Samples: 3434106. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:06:12,016][52710] Avg episode reward: [(0, '14.260'), (1, '14.530')] [2023-10-08 08:06:14,088][53852] Updated weights for policy 0, policy_version 6730 (0.0008) [2023-10-08 08:06:14,456][53852] Updated weights for policy 0, policy_version 6740 (0.0009) [2023-10-08 08:06:14,673][53885] Updated weights for policy 1, policy_version 6690 (0.0008) [2023-10-08 08:06:14,826][53852] Updated weights for policy 0, policy_version 6750 (0.0007) [2023-10-08 08:06:15,037][53885] Updated weights for policy 1, policy_version 6700 (0.0008) [2023-10-08 08:06:15,404][53885] Updated weights for policy 1, policy_version 6710 (0.0008) [2023-10-08 08:06:15,771][53885] Updated weights for policy 1, policy_version 6720 (0.0011) [2023-10-08 08:06:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 13795328. Throughput: 0: 1839.0, 1: 1810.5. Samples: 3454666. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:06:17,016][52710] Avg episode reward: [(0, '15.320'), (1, '14.600')] [2023-10-08 08:06:17,018][53500] Saving new best policy, reward=15.320! [2023-10-08 08:06:18,268][53852] Updated weights for policy 0, policy_version 6760 (0.0008) [2023-10-08 08:06:18,636][53852] Updated weights for policy 0, policy_version 6770 (0.0008) [2023-10-08 08:06:19,013][53852] Updated weights for policy 0, policy_version 6780 (0.0008) [2023-10-08 08:06:19,439][53885] Updated weights for policy 1, policy_version 6730 (0.0008) [2023-10-08 08:06:19,809][53885] Updated weights for policy 1, policy_version 6740 (0.0007) [2023-10-08 08:06:20,181][53885] Updated weights for policy 1, policy_version 6750 (0.0008) [2023-10-08 08:06:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 13860864. Throughput: 0: 1835.1, 1: 1812.1. Samples: 3477464. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:06:22,015][52710] Avg episode reward: [(0, '14.290'), (1, '15.860')] [2023-10-08 08:06:22,565][53852] Updated weights for policy 0, policy_version 6790 (0.0008) [2023-10-08 08:06:22,943][53852] Updated weights for policy 0, policy_version 6800 (0.0009) [2023-10-08 08:06:23,312][53852] Updated weights for policy 0, policy_version 6810 (0.0009) [2023-10-08 08:06:23,851][53885] Updated weights for policy 1, policy_version 6760 (0.0010) [2023-10-08 08:06:24,220][53885] Updated weights for policy 1, policy_version 6770 (0.0010) [2023-10-08 08:06:24,587][53885] Updated weights for policy 1, policy_version 6780 (0.0011) [2023-10-08 08:06:26,861][53852] Updated weights for policy 0, policy_version 6820 (0.0007) [2023-10-08 08:06:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13926400. Throughput: 0: 1837.4, 1: 1817.4. Samples: 3487728. Policy #0 lag: (min: 31.0, avg: 41.1, max: 63.0) [2023-10-08 08:06:27,016][52710] Avg episode reward: [(0, '13.220'), (1, '16.520')] [2023-10-08 08:06:27,016][53594] Saving new best policy, reward=16.520! [2023-10-08 08:06:27,226][53852] Updated weights for policy 0, policy_version 6830 (0.0008) [2023-10-08 08:06:27,607][53852] Updated weights for policy 0, policy_version 6840 (0.0008) [2023-10-08 08:06:28,288][53885] Updated weights for policy 1, policy_version 6790 (0.0007) [2023-10-08 08:06:28,664][53885] Updated weights for policy 1, policy_version 6800 (0.0008) [2023-10-08 08:06:29,044][53885] Updated weights for policy 1, policy_version 6810 (0.0009) [2023-10-08 08:06:31,262][53852] Updated weights for policy 0, policy_version 6850 (0.0008) [2023-10-08 08:06:31,634][53852] Updated weights for policy 0, policy_version 6860 (0.0007) [2023-10-08 08:06:32,004][53852] Updated weights for policy 0, policy_version 6870 (0.0007) [2023-10-08 08:06:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 13991936. Throughput: 0: 1847.3, 1: 1808.6. Samples: 3510418. Policy #0 lag: (min: 31.0, avg: 41.1, max: 63.0) [2023-10-08 08:06:32,016][52710] Avg episode reward: [(0, '14.400'), (1, '15.850')] [2023-10-08 08:06:32,368][53852] Updated weights for policy 0, policy_version 6880 (0.0007) [2023-10-08 08:06:32,564][53885] Updated weights for policy 1, policy_version 6820 (0.0008) [2023-10-08 08:06:32,925][53885] Updated weights for policy 1, policy_version 6830 (0.0007) [2023-10-08 08:06:33,289][53885] Updated weights for policy 1, policy_version 6840 (0.0008) [2023-10-08 08:06:35,998][53852] Updated weights for policy 0, policy_version 6890 (0.0008) [2023-10-08 08:06:36,362][53852] Updated weights for policy 0, policy_version 6900 (0.0007) [2023-10-08 08:06:36,733][53852] Updated weights for policy 0, policy_version 6910 (0.0007) [2023-10-08 08:06:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 14090240. Throughput: 0: 1823.5, 1: 1814.2. Samples: 3532180. Policy #0 lag: (min: 8.0, avg: 32.5, max: 40.0) [2023-10-08 08:06:37,016][52710] Avg episode reward: [(0, '15.350'), (1, '15.270')] [2023-10-08 08:06:37,027][53500] Saving new best policy, reward=15.350! [2023-10-08 08:06:37,149][53885] Updated weights for policy 1, policy_version 6850 (0.0008) [2023-10-08 08:06:37,524][53885] Updated weights for policy 1, policy_version 6860 (0.0008) [2023-10-08 08:06:37,887][53885] Updated weights for policy 1, policy_version 6870 (0.0009) [2023-10-08 08:06:38,251][53885] Updated weights for policy 1, policy_version 6880 (0.0009) [2023-10-08 08:06:40,394][53852] Updated weights for policy 0, policy_version 6920 (0.0009) [2023-10-08 08:06:40,770][53852] Updated weights for policy 0, policy_version 6930 (0.0007) [2023-10-08 08:06:41,142][53852] Updated weights for policy 0, policy_version 6940 (0.0007) [2023-10-08 08:06:41,834][53885] Updated weights for policy 1, policy_version 6890 (0.0007) [2023-10-08 08:06:42,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14155776. Throughput: 0: 1851.6, 1: 1817.9. Samples: 3543518. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 08:06:42,016][52710] Avg episode reward: [(0, '15.230'), (1, '14.510')] [2023-10-08 08:06:42,209][53885] Updated weights for policy 1, policy_version 6900 (0.0009) [2023-10-08 08:06:42,581][53885] Updated weights for policy 1, policy_version 6910 (0.0007) [2023-10-08 08:06:44,722][53852] Updated weights for policy 0, policy_version 6950 (0.0008) [2023-10-08 08:06:45,100][53852] Updated weights for policy 0, policy_version 6960 (0.0011) [2023-10-08 08:06:45,466][53852] Updated weights for policy 0, policy_version 6970 (0.0008) [2023-10-08 08:06:46,384][53885] Updated weights for policy 1, policy_version 6920 (0.0008) [2023-10-08 08:06:46,750][53885] Updated weights for policy 1, policy_version 6930 (0.0007) [2023-10-08 08:06:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14221312. Throughput: 0: 1825.2, 1: 1814.9. Samples: 3565218. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 08:06:47,016][52710] Avg episode reward: [(0, '14.820'), (1, '15.010')] [2023-10-08 08:06:47,117][53885] Updated weights for policy 1, policy_version 6940 (0.0007) [2023-10-08 08:06:49,133][53852] Updated weights for policy 0, policy_version 6980 (0.0008) [2023-10-08 08:06:49,503][53852] Updated weights for policy 0, policy_version 6990 (0.0007) [2023-10-08 08:06:49,880][53852] Updated weights for policy 0, policy_version 7000 (0.0007) [2023-10-08 08:06:50,730][53885] Updated weights for policy 1, policy_version 6950 (0.0009) [2023-10-08 08:06:51,091][53885] Updated weights for policy 1, policy_version 6960 (0.0008) [2023-10-08 08:06:51,469][53885] Updated weights for policy 1, policy_version 6970 (0.0009) [2023-10-08 08:06:52,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 14319616. Throughput: 0: 1856.6, 1: 1815.5. Samples: 3586694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:06:52,016][52710] Avg episode reward: [(0, '15.390'), (1, '16.040')] [2023-10-08 08:06:52,023][53500] Saving new best policy, reward=15.390! [2023-10-08 08:06:53,519][53852] Updated weights for policy 0, policy_version 7010 (0.0008) [2023-10-08 08:06:53,887][53852] Updated weights for policy 0, policy_version 7020 (0.0009) [2023-10-08 08:06:54,250][53852] Updated weights for policy 0, policy_version 7030 (0.0010) [2023-10-08 08:06:54,618][53852] Updated weights for policy 0, policy_version 7040 (0.0007) [2023-10-08 08:06:55,235][53885] Updated weights for policy 1, policy_version 6980 (0.0007) [2023-10-08 08:06:55,601][53885] Updated weights for policy 1, policy_version 6990 (0.0010) [2023-10-08 08:06:55,963][53885] Updated weights for policy 1, policy_version 7000 (0.0010) [2023-10-08 08:06:57,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 14385152. Throughput: 0: 1828.7, 1: 1815.3. Samples: 3598088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:06:57,016][52710] Avg episode reward: [(0, '15.140'), (1, '15.780')] [2023-10-08 08:06:58,387][53852] Updated weights for policy 0, policy_version 7050 (0.0009) [2023-10-08 08:06:58,766][53852] Updated weights for policy 0, policy_version 7060 (0.0009) [2023-10-08 08:06:59,132][53852] Updated weights for policy 0, policy_version 7070 (0.0010) [2023-10-08 08:06:59,634][53885] Updated weights for policy 1, policy_version 7010 (0.0010) [2023-10-08 08:06:59,999][53885] Updated weights for policy 1, policy_version 7020 (0.0008) [2023-10-08 08:07:00,371][53885] Updated weights for policy 1, policy_version 7030 (0.0007) [2023-10-08 08:07:00,739][53885] Updated weights for policy 1, policy_version 7040 (0.0007) [2023-10-08 08:07:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14450688. Throughput: 0: 1841.8, 1: 1818.8. Samples: 3619392. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 08:07:02,015][52710] Avg episode reward: [(0, '14.980'), (1, '15.210')] [2023-10-08 08:07:02,813][53852] Updated weights for policy 0, policy_version 7080 (0.0008) [2023-10-08 08:07:03,181][53852] Updated weights for policy 0, policy_version 7090 (0.0008) [2023-10-08 08:07:03,556][53852] Updated weights for policy 0, policy_version 7100 (0.0009) [2023-10-08 08:07:04,493][53885] Updated weights for policy 1, policy_version 7050 (0.0009) [2023-10-08 08:07:04,859][53885] Updated weights for policy 1, policy_version 7060 (0.0008) [2023-10-08 08:07:05,238][53885] Updated weights for policy 1, policy_version 7070 (0.0009) [2023-10-08 08:07:07,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 14516224. Throughput: 0: 1841.5, 1: 1817.2. Samples: 3642104. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) [2023-10-08 08:07:07,016][52710] Avg episode reward: [(0, '15.080'), (1, '14.840')] [2023-10-08 08:07:07,222][53852] Updated weights for policy 0, policy_version 7110 (0.0008) [2023-10-08 08:07:07,602][53852] Updated weights for policy 0, policy_version 7120 (0.0007) [2023-10-08 08:07:07,978][53852] Updated weights for policy 0, policy_version 7130 (0.0009) [2023-10-08 08:07:08,963][53885] Updated weights for policy 1, policy_version 7080 (0.0009) [2023-10-08 08:07:09,346][53885] Updated weights for policy 1, policy_version 7090 (0.0008) [2023-10-08 08:07:09,706][53885] Updated weights for policy 1, policy_version 7100 (0.0007) [2023-10-08 08:07:11,530][53852] Updated weights for policy 0, policy_version 7140 (0.0007) [2023-10-08 08:07:11,903][53852] Updated weights for policy 0, policy_version 7150 (0.0010) [2023-10-08 08:07:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14581760. Throughput: 0: 1839.7, 1: 1819.3. Samples: 3652384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:07:12,016][52710] Avg episode reward: [(0, '15.840'), (1, '15.230')] [2023-10-08 08:07:12,281][53852] Updated weights for policy 0, policy_version 7160 (0.0009) [2023-10-08 08:07:12,576][53500] Saving new best policy, reward=15.840! [2023-10-08 08:07:13,442][53885] Updated weights for policy 1, policy_version 7110 (0.0007) [2023-10-08 08:07:13,812][53885] Updated weights for policy 1, policy_version 7120 (0.0009) [2023-10-08 08:07:14,184][53885] Updated weights for policy 1, policy_version 7130 (0.0009) [2023-10-08 08:07:15,981][53852] Updated weights for policy 0, policy_version 7170 (0.0007) [2023-10-08 08:07:16,346][53852] Updated weights for policy 0, policy_version 7180 (0.0008) [2023-10-08 08:07:16,722][53852] Updated weights for policy 0, policy_version 7190 (0.0007) [2023-10-08 08:07:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14647296. Throughput: 0: 1838.7, 1: 1817.2. Samples: 3674932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:07:17,015][52710] Avg episode reward: [(0, '14.940'), (1, '16.020')] [2023-10-08 08:07:17,097][53852] Updated weights for policy 0, policy_version 7200 (0.0007) [2023-10-08 08:07:17,687][53885] Updated weights for policy 1, policy_version 7140 (0.0010) [2023-10-08 08:07:18,045][53885] Updated weights for policy 1, policy_version 7150 (0.0009) [2023-10-08 08:07:18,418][53885] Updated weights for policy 1, policy_version 7160 (0.0009) [2023-10-08 08:07:20,770][53852] Updated weights for policy 0, policy_version 7210 (0.0009) [2023-10-08 08:07:21,146][53852] Updated weights for policy 0, policy_version 7220 (0.0009) [2023-10-08 08:07:21,516][53852] Updated weights for policy 0, policy_version 7230 (0.0011) [2023-10-08 08:07:22,003][53885] Updated weights for policy 1, policy_version 7170 (0.0010) [2023-10-08 08:07:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14745600. Throughput: 0: 1833.5, 1: 1822.4. Samples: 3696692. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:07:22,015][52710] Avg episode reward: [(0, '16.060'), (1, '15.720')] [2023-10-08 08:07:22,024][53500] Saving new best policy, reward=16.060! [2023-10-08 08:07:22,372][53885] Updated weights for policy 1, policy_version 7180 (0.0007) [2023-10-08 08:07:22,738][53885] Updated weights for policy 1, policy_version 7190 (0.0007) [2023-10-08 08:07:23,108][53885] Updated weights for policy 1, policy_version 7200 (0.0007) [2023-10-08 08:07:25,259][53852] Updated weights for policy 0, policy_version 7240 (0.0011) [2023-10-08 08:07:25,637][53852] Updated weights for policy 0, policy_version 7250 (0.0008) [2023-10-08 08:07:26,002][53852] Updated weights for policy 0, policy_version 7260 (0.0009) [2023-10-08 08:07:26,823][53885] Updated weights for policy 1, policy_version 7210 (0.0007) [2023-10-08 08:07:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14811136. Throughput: 0: 1832.5, 1: 1818.4. Samples: 3707808. Policy #0 lag: (min: 9.0, avg: 17.4, max: 41.0) [2023-10-08 08:07:27,015][52710] Avg episode reward: [(0, '15.240'), (1, '16.020')] [2023-10-08 08:07:27,192][53885] Updated weights for policy 1, policy_version 7220 (0.0007) [2023-10-08 08:07:27,552][53885] Updated weights for policy 1, policy_version 7230 (0.0008) [2023-10-08 08:07:29,752][53852] Updated weights for policy 0, policy_version 7270 (0.0008) [2023-10-08 08:07:30,129][53852] Updated weights for policy 0, policy_version 7280 (0.0008) [2023-10-08 08:07:30,511][53852] Updated weights for policy 0, policy_version 7290 (0.0009) [2023-10-08 08:07:31,276][53885] Updated weights for policy 1, policy_version 7240 (0.0008) [2023-10-08 08:07:31,646][53885] Updated weights for policy 1, policy_version 7250 (0.0009) [2023-10-08 08:07:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14876672. Throughput: 0: 1828.2, 1: 1827.7. Samples: 3729734. Policy #0 lag: (min: 9.0, avg: 17.4, max: 41.0) [2023-10-08 08:07:32,016][52710] Avg episode reward: [(0, '15.090'), (1, '17.250')] [2023-10-08 08:07:32,019][53885] Updated weights for policy 1, policy_version 7260 (0.0007) [2023-10-08 08:07:32,158][53594] Saving new best policy, reward=17.250! [2023-10-08 08:07:34,085][53852] Updated weights for policy 0, policy_version 7300 (0.0008) [2023-10-08 08:07:34,451][53852] Updated weights for policy 0, policy_version 7310 (0.0008) [2023-10-08 08:07:34,822][53852] Updated weights for policy 0, policy_version 7320 (0.0007) [2023-10-08 08:07:35,745][53885] Updated weights for policy 1, policy_version 7270 (0.0009) [2023-10-08 08:07:36,123][53885] Updated weights for policy 1, policy_version 7280 (0.0009) [2023-10-08 08:07:36,491][53885] Updated weights for policy 1, policy_version 7290 (0.0008) [2023-10-08 08:07:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14974976. Throughput: 0: 1827.1, 1: 1825.2. Samples: 3751044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:07:37,016][52710] Avg episode reward: [(0, '15.130'), (1, '16.750')] [2023-10-08 08:07:38,408][53852] Updated weights for policy 0, policy_version 7330 (0.0009) [2023-10-08 08:07:38,793][53852] Updated weights for policy 0, policy_version 7340 (0.0007) [2023-10-08 08:07:39,168][53852] Updated weights for policy 0, policy_version 7350 (0.0008) [2023-10-08 08:07:39,543][53852] Updated weights for policy 0, policy_version 7360 (0.0008) [2023-10-08 08:07:40,413][53885] Updated weights for policy 1, policy_version 7300 (0.0007) [2023-10-08 08:07:40,785][53885] Updated weights for policy 1, policy_version 7310 (0.0008) [2023-10-08 08:07:41,152][53885] Updated weights for policy 1, policy_version 7320 (0.0008) [2023-10-08 08:07:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15040512. Throughput: 0: 1826.3, 1: 1824.3. Samples: 3762362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:07:42,016][52710] Avg episode reward: [(0, '15.650'), (1, '17.220')] [2023-10-08 08:07:43,267][53852] Updated weights for policy 0, policy_version 7370 (0.0007) [2023-10-08 08:07:43,648][53852] Updated weights for policy 0, policy_version 7380 (0.0007) [2023-10-08 08:07:44,009][53852] Updated weights for policy 0, policy_version 7390 (0.0008) [2023-10-08 08:07:44,761][53885] Updated weights for policy 1, policy_version 7330 (0.0008) [2023-10-08 08:07:45,131][53885] Updated weights for policy 1, policy_version 7340 (0.0008) [2023-10-08 08:07:45,491][53885] Updated weights for policy 1, policy_version 7350 (0.0008) [2023-10-08 08:07:45,866][53885] Updated weights for policy 1, policy_version 7360 (0.0008) [2023-10-08 08:07:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15106048. Throughput: 0: 1835.7, 1: 1824.4. Samples: 3784098. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 08:07:47,015][52710] Avg episode reward: [(0, '16.810'), (1, '17.820')] [2023-10-08 08:07:47,016][53594] Saving new best policy, reward=17.820! [2023-10-08 08:07:47,016][53500] Saving new best policy, reward=16.810! [2023-10-08 08:07:47,627][53852] Updated weights for policy 0, policy_version 7400 (0.0007) [2023-10-08 08:07:48,000][53852] Updated weights for policy 0, policy_version 7410 (0.0009) [2023-10-08 08:07:48,374][53852] Updated weights for policy 0, policy_version 7420 (0.0007) [2023-10-08 08:07:49,474][53885] Updated weights for policy 1, policy_version 7370 (0.0009) [2023-10-08 08:07:49,836][53885] Updated weights for policy 1, policy_version 7380 (0.0008) [2023-10-08 08:07:50,210][53885] Updated weights for policy 1, policy_version 7390 (0.0009) [2023-10-08 08:07:51,929][53852] Updated weights for policy 0, policy_version 7430 (0.0007) [2023-10-08 08:07:52,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 15171584. Throughput: 0: 1839.5, 1: 1827.5. Samples: 3807118. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 08:07:52,016][52710] Avg episode reward: [(0, '16.120'), (1, '16.880')] [2023-10-08 08:07:52,310][53852] Updated weights for policy 0, policy_version 7440 (0.0007) [2023-10-08 08:07:52,683][53852] Updated weights for policy 0, policy_version 7450 (0.0008) [2023-10-08 08:07:54,003][53885] Updated weights for policy 1, policy_version 7400 (0.0009) [2023-10-08 08:07:54,384][53885] Updated weights for policy 1, policy_version 7410 (0.0009) [2023-10-08 08:07:54,756][53885] Updated weights for policy 1, policy_version 7420 (0.0007) [2023-10-08 08:07:56,372][53852] Updated weights for policy 0, policy_version 7460 (0.0009) [2023-10-08 08:07:56,759][53852] Updated weights for policy 0, policy_version 7470 (0.0010) [2023-10-08 08:07:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15237120. Throughput: 0: 1836.7, 1: 1829.1. Samples: 3817344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:07:57,016][52710] Avg episode reward: [(0, '15.840'), (1, '16.360')] [2023-10-08 08:07:57,129][53852] Updated weights for policy 0, policy_version 7480 (0.0009) [2023-10-08 08:07:58,163][53885] Updated weights for policy 1, policy_version 7430 (0.0011) [2023-10-08 08:07:58,529][53885] Updated weights for policy 1, policy_version 7440 (0.0008) [2023-10-08 08:07:58,901][53885] Updated weights for policy 1, policy_version 7450 (0.0008) [2023-10-08 08:08:00,854][53852] Updated weights for policy 0, policy_version 7490 (0.0009) [2023-10-08 08:08:01,219][53852] Updated weights for policy 0, policy_version 7500 (0.0009) [2023-10-08 08:08:01,592][53852] Updated weights for policy 0, policy_version 7510 (0.0007) [2023-10-08 08:08:01,967][53852] Updated weights for policy 0, policy_version 7520 (0.0010) [2023-10-08 08:08:02,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 15335424. Throughput: 0: 1832.8, 1: 1833.1. Samples: 3839898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:08:02,016][52710] Avg episode reward: [(0, '15.650'), (1, '14.290')] [2023-10-08 08:08:02,443][53885] Updated weights for policy 1, policy_version 7460 (0.0010) [2023-10-08 08:08:02,803][53885] Updated weights for policy 1, policy_version 7470 (0.0007) [2023-10-08 08:08:03,183][53885] Updated weights for policy 1, policy_version 7480 (0.0008) [2023-10-08 08:08:05,741][53852] Updated weights for policy 0, policy_version 7530 (0.0007) [2023-10-08 08:08:06,112][53852] Updated weights for policy 0, policy_version 7540 (0.0009) [2023-10-08 08:08:06,484][53852] Updated weights for policy 0, policy_version 7550 (0.0007) [2023-10-08 08:08:06,773][53885] Updated weights for policy 1, policy_version 7490 (0.0008) [2023-10-08 08:08:07,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15400960. Throughput: 0: 1832.6, 1: 1829.7. Samples: 3861496. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:08:07,015][52710] Avg episode reward: [(0, '16.690'), (1, '16.060')] [2023-10-08 08:08:07,022][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000007552_7733248.pth... [2023-10-08 08:08:07,053][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000005824_5963776.pth [2023-10-08 08:08:07,146][53885] Updated weights for policy 1, policy_version 7500 (0.0008) [2023-10-08 08:08:07,514][53885] Updated weights for policy 1, policy_version 7510 (0.0007) [2023-10-08 08:08:07,882][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000007520_7700480.pth... [2023-10-08 08:08:07,884][53885] Updated weights for policy 1, policy_version 7520 (0.0007) [2023-10-08 08:08:07,910][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000005792_5931008.pth [2023-10-08 08:08:10,174][53852] Updated weights for policy 0, policy_version 7560 (0.0008) [2023-10-08 08:08:10,541][53852] Updated weights for policy 0, policy_version 7570 (0.0008) [2023-10-08 08:08:10,920][53852] Updated weights for policy 0, policy_version 7580 (0.0008) [2023-10-08 08:08:11,669][53885] Updated weights for policy 1, policy_version 7530 (0.0009) [2023-10-08 08:08:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15466496. Throughput: 0: 1835.0, 1: 1830.8. Samples: 3872770. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-08 08:08:12,015][52710] Avg episode reward: [(0, '16.170'), (1, '16.700')] [2023-10-08 08:08:12,036][53885] Updated weights for policy 1, policy_version 7540 (0.0009) [2023-10-08 08:08:12,403][53885] Updated weights for policy 1, policy_version 7550 (0.0007) [2023-10-08 08:08:14,265][53852] Updated weights for policy 0, policy_version 7590 (0.0008) [2023-10-08 08:08:14,640][53852] Updated weights for policy 0, policy_version 7600 (0.0007) [2023-10-08 08:08:14,996][53852] Updated weights for policy 0, policy_version 7610 (0.0010) [2023-10-08 08:08:16,095][53885] Updated weights for policy 1, policy_version 7560 (0.0007) [2023-10-08 08:08:16,465][53885] Updated weights for policy 1, policy_version 7570 (0.0007) [2023-10-08 08:08:16,830][53885] Updated weights for policy 1, policy_version 7580 (0.0007) [2023-10-08 08:08:17,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 15564800. Throughput: 0: 1832.8, 1: 1829.5. Samples: 3894536. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) [2023-10-08 08:08:17,016][52710] Avg episode reward: [(0, '16.340'), (1, '16.300')] [2023-10-08 08:08:18,668][53852] Updated weights for policy 0, policy_version 7620 (0.0008) [2023-10-08 08:08:19,046][53852] Updated weights for policy 0, policy_version 7630 (0.0007) [2023-10-08 08:08:19,413][53852] Updated weights for policy 0, policy_version 7640 (0.0010) [2023-10-08 08:08:20,346][53885] Updated weights for policy 1, policy_version 7590 (0.0009) [2023-10-08 08:08:20,707][53885] Updated weights for policy 1, policy_version 7600 (0.0010) [2023-10-08 08:08:21,070][53885] Updated weights for policy 1, policy_version 7610 (0.0008) [2023-10-08 08:08:22,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15630336. Throughput: 0: 1836.3, 1: 1834.2. Samples: 3916216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:08:22,016][52710] Avg episode reward: [(0, '16.860'), (1, '16.160')] [2023-10-08 08:08:22,025][53500] Saving new best policy, reward=16.860! [2023-10-08 08:08:23,061][53852] Updated weights for policy 0, policy_version 7650 (0.0009) [2023-10-08 08:08:23,443][53852] Updated weights for policy 0, policy_version 7660 (0.0008) [2023-10-08 08:08:23,807][53852] Updated weights for policy 0, policy_version 7670 (0.0008) [2023-10-08 08:08:24,185][53852] Updated weights for policy 0, policy_version 7680 (0.0008) [2023-10-08 08:08:24,686][53885] Updated weights for policy 1, policy_version 7620 (0.0007) [2023-10-08 08:08:25,057][53885] Updated weights for policy 1, policy_version 7630 (0.0010) [2023-10-08 08:08:25,436][53885] Updated weights for policy 1, policy_version 7640 (0.0009) [2023-10-08 08:08:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15695872. Throughput: 0: 1833.0, 1: 1839.2. Samples: 3927610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:08:27,016][52710] Avg episode reward: [(0, '16.480'), (1, '15.960')] [2023-10-08 08:08:27,849][53852] Updated weights for policy 0, policy_version 7690 (0.0008) [2023-10-08 08:08:28,215][53852] Updated weights for policy 0, policy_version 7700 (0.0008) [2023-10-08 08:08:28,590][53852] Updated weights for policy 0, policy_version 7710 (0.0009) [2023-10-08 08:08:29,083][53885] Updated weights for policy 1, policy_version 7650 (0.0010) [2023-10-08 08:08:29,453][53885] Updated weights for policy 1, policy_version 7660 (0.0011) [2023-10-08 08:08:29,819][53885] Updated weights for policy 1, policy_version 7670 (0.0008) [2023-10-08 08:08:30,194][53885] Updated weights for policy 1, policy_version 7680 (0.0010) [2023-10-08 08:08:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 15761408. Throughput: 0: 1829.8, 1: 1837.6. Samples: 3949132. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 08:08:32,016][52710] Avg episode reward: [(0, '17.380'), (1, '16.430')] [2023-10-08 08:08:32,340][53852] Updated weights for policy 0, policy_version 7720 (0.0009) [2023-10-08 08:08:32,700][53852] Updated weights for policy 0, policy_version 7730 (0.0008) [2023-10-08 08:08:33,076][53852] Updated weights for policy 0, policy_version 7740 (0.0010) [2023-10-08 08:08:33,225][53500] Saving new best policy, reward=17.380! [2023-10-08 08:08:33,806][53885] Updated weights for policy 1, policy_version 7690 (0.0007) [2023-10-08 08:08:34,177][53885] Updated weights for policy 1, policy_version 7700 (0.0008) [2023-10-08 08:08:34,547][53885] Updated weights for policy 1, policy_version 7710 (0.0007) [2023-10-08 08:08:36,733][53852] Updated weights for policy 0, policy_version 7750 (0.0007) [2023-10-08 08:08:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15826944. Throughput: 0: 1825.9, 1: 1847.7. Samples: 3972430. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 08:08:37,016][52710] Avg episode reward: [(0, '16.580'), (1, '16.130')] [2023-10-08 08:08:37,107][53852] Updated weights for policy 0, policy_version 7760 (0.0007) [2023-10-08 08:08:37,489][53852] Updated weights for policy 0, policy_version 7770 (0.0009) [2023-10-08 08:08:38,139][53885] Updated weights for policy 1, policy_version 7720 (0.0008) [2023-10-08 08:08:38,509][53885] Updated weights for policy 1, policy_version 7730 (0.0008) [2023-10-08 08:08:38,870][53885] Updated weights for policy 1, policy_version 7740 (0.0008) [2023-10-08 08:08:41,126][53852] Updated weights for policy 0, policy_version 7780 (0.0008) [2023-10-08 08:08:41,499][53852] Updated weights for policy 0, policy_version 7790 (0.0007) [2023-10-08 08:08:41,872][53852] Updated weights for policy 0, policy_version 7800 (0.0009) [2023-10-08 08:08:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 15892480. Throughput: 0: 1830.3, 1: 1842.9. Samples: 3982638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:08:42,016][52710] Avg episode reward: [(0, '16.350'), (1, '15.680')] [2023-10-08 08:08:42,561][53885] Updated weights for policy 1, policy_version 7750 (0.0009) [2023-10-08 08:08:42,933][53885] Updated weights for policy 1, policy_version 7760 (0.0008) [2023-10-08 08:08:43,296][53885] Updated weights for policy 1, policy_version 7770 (0.0008) [2023-10-08 08:08:45,571][53852] Updated weights for policy 0, policy_version 7810 (0.0008) [2023-10-08 08:08:45,950][53852] Updated weights for policy 0, policy_version 7820 (0.0008) [2023-10-08 08:08:46,321][53852] Updated weights for policy 0, policy_version 7830 (0.0008) [2023-10-08 08:08:46,690][53852] Updated weights for policy 0, policy_version 7840 (0.0009) [2023-10-08 08:08:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 15990784. Throughput: 0: 1829.5, 1: 1847.3. Samples: 4005352. Policy #0 lag: (min: 23.0, avg: 24.0, max: 44.0) [2023-10-08 08:08:47,016][52710] Avg episode reward: [(0, '16.380'), (1, '17.850')] [2023-10-08 08:08:47,118][53885] Updated weights for policy 1, policy_version 7780 (0.0010) [2023-10-08 08:08:47,509][53885] Updated weights for policy 1, policy_version 7790 (0.0007) [2023-10-08 08:08:47,884][53885] Updated weights for policy 1, policy_version 7800 (0.0008) [2023-10-08 08:08:48,167][53594] Saving new best policy, reward=17.850! [2023-10-08 08:08:50,163][53852] Updated weights for policy 0, policy_version 7850 (0.0007) [2023-10-08 08:08:50,544][53852] Updated weights for policy 0, policy_version 7860 (0.0007) [2023-10-08 08:08:50,909][53852] Updated weights for policy 0, policy_version 7870 (0.0008) [2023-10-08 08:08:51,523][53885] Updated weights for policy 1, policy_version 7810 (0.0009) [2023-10-08 08:08:51,897][53885] Updated weights for policy 1, policy_version 7820 (0.0007) [2023-10-08 08:08:52,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 16056320. Throughput: 0: 1830.5, 1: 1836.7. Samples: 4026520. Policy #0 lag: (min: 23.0, avg: 24.0, max: 44.0) [2023-10-08 08:08:52,016][52710] Avg episode reward: [(0, '14.600'), (1, '15.510')] [2023-10-08 08:08:52,258][53885] Updated weights for policy 1, policy_version 7830 (0.0008) [2023-10-08 08:08:52,631][53885] Updated weights for policy 1, policy_version 7840 (0.0007) [2023-10-08 08:08:54,617][53852] Updated weights for policy 0, policy_version 7880 (0.0007) [2023-10-08 08:08:54,985][53852] Updated weights for policy 0, policy_version 7890 (0.0008) [2023-10-08 08:08:55,363][53852] Updated weights for policy 0, policy_version 7900 (0.0010) [2023-10-08 08:08:56,366][53885] Updated weights for policy 1, policy_version 7850 (0.0007) [2023-10-08 08:08:56,730][53885] Updated weights for policy 1, policy_version 7860 (0.0007) [2023-10-08 08:08:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16121856. Throughput: 0: 1830.6, 1: 1843.4. Samples: 4038100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:08:57,015][52710] Avg episode reward: [(0, '15.450'), (1, '16.690')] [2023-10-08 08:08:57,106][53885] Updated weights for policy 1, policy_version 7870 (0.0008) [2023-10-08 08:08:58,927][53852] Updated weights for policy 0, policy_version 7910 (0.0010) [2023-10-08 08:08:59,289][53852] Updated weights for policy 0, policy_version 7920 (0.0007) [2023-10-08 08:08:59,658][53852] Updated weights for policy 0, policy_version 7930 (0.0007) [2023-10-08 08:09:00,792][53885] Updated weights for policy 1, policy_version 7880 (0.0009) [2023-10-08 08:09:01,167][53885] Updated weights for policy 1, policy_version 7890 (0.0009) [2023-10-08 08:09:01,537][53885] Updated weights for policy 1, policy_version 7900 (0.0010) [2023-10-08 08:09:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16220160. Throughput: 0: 1834.7, 1: 1838.0. Samples: 4059806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:09:02,016][52710] Avg episode reward: [(0, '14.160'), (1, '16.550')] [2023-10-08 08:09:03,425][53852] Updated weights for policy 0, policy_version 7940 (0.0008) [2023-10-08 08:09:03,792][53852] Updated weights for policy 0, policy_version 7950 (0.0009) [2023-10-08 08:09:04,165][53852] Updated weights for policy 0, policy_version 7960 (0.0008) [2023-10-08 08:09:05,289][53885] Updated weights for policy 1, policy_version 7910 (0.0009) [2023-10-08 08:09:05,653][53885] Updated weights for policy 1, policy_version 7920 (0.0009) [2023-10-08 08:09:06,018][53885] Updated weights for policy 1, policy_version 7930 (0.0008) [2023-10-08 08:09:07,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16285696. Throughput: 0: 1840.5, 1: 1830.8. Samples: 4081424. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 08:09:07,016][52710] Avg episode reward: [(0, '15.700'), (1, '17.930')] [2023-10-08 08:09:07,028][53594] Saving new best policy, reward=17.930! [2023-10-08 08:09:07,742][53852] Updated weights for policy 0, policy_version 7970 (0.0007) [2023-10-08 08:09:08,113][53852] Updated weights for policy 0, policy_version 7980 (0.0009) [2023-10-08 08:09:08,483][53852] Updated weights for policy 0, policy_version 7990 (0.0009) [2023-10-08 08:09:08,854][53852] Updated weights for policy 0, policy_version 8000 (0.0009) [2023-10-08 08:09:09,529][53885] Updated weights for policy 1, policy_version 7940 (0.0009) [2023-10-08 08:09:09,894][53885] Updated weights for policy 1, policy_version 7950 (0.0009) [2023-10-08 08:09:10,257][53885] Updated weights for policy 1, policy_version 7960 (0.0010) [2023-10-08 08:09:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16351232. Throughput: 0: 1838.7, 1: 1830.4. Samples: 4092720. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 08:09:12,015][52710] Avg episode reward: [(0, '15.270'), (1, '16.870')] [2023-10-08 08:09:12,548][53852] Updated weights for policy 0, policy_version 8010 (0.0010) [2023-10-08 08:09:12,909][53852] Updated weights for policy 0, policy_version 8020 (0.0008) [2023-10-08 08:09:13,279][53852] Updated weights for policy 0, policy_version 8030 (0.0007) [2023-10-08 08:09:14,027][53885] Updated weights for policy 1, policy_version 7970 (0.0011) [2023-10-08 08:09:14,402][53885] Updated weights for policy 1, policy_version 7980 (0.0009) [2023-10-08 08:09:14,777][53885] Updated weights for policy 1, policy_version 7990 (0.0007) [2023-10-08 08:09:15,139][53885] Updated weights for policy 1, policy_version 8000 (0.0009) [2023-10-08 08:09:16,999][53852] Updated weights for policy 0, policy_version 8040 (0.0009) [2023-10-08 08:09:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 16416768. Throughput: 0: 1840.1, 1: 1828.5. Samples: 4114218. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 08:09:17,016][52710] Avg episode reward: [(0, '15.470'), (1, '15.830')] [2023-10-08 08:09:17,362][53852] Updated weights for policy 0, policy_version 8050 (0.0007) [2023-10-08 08:09:17,740][53852] Updated weights for policy 0, policy_version 8060 (0.0007) [2023-10-08 08:09:18,724][53885] Updated weights for policy 1, policy_version 8010 (0.0010) [2023-10-08 08:09:19,098][53885] Updated weights for policy 1, policy_version 8020 (0.0009) [2023-10-08 08:09:19,455][53885] Updated weights for policy 1, policy_version 8030 (0.0009) [2023-10-08 08:09:21,406][53852] Updated weights for policy 0, policy_version 8070 (0.0008) [2023-10-08 08:09:21,770][53852] Updated weights for policy 0, policy_version 8080 (0.0011) [2023-10-08 08:09:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 16482304. Throughput: 0: 1824.5, 1: 1827.4. Samples: 4136764. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 08:09:22,016][52710] Avg episode reward: [(0, '16.300'), (1, '17.070')] [2023-10-08 08:09:22,144][53852] Updated weights for policy 0, policy_version 8090 (0.0007) [2023-10-08 08:09:23,139][53885] Updated weights for policy 1, policy_version 8040 (0.0008) [2023-10-08 08:09:23,503][53885] Updated weights for policy 1, policy_version 8050 (0.0007) [2023-10-08 08:09:23,885][53885] Updated weights for policy 1, policy_version 8060 (0.0009) [2023-10-08 08:09:25,661][53852] Updated weights for policy 0, policy_version 8100 (0.0007) [2023-10-08 08:09:26,034][53852] Updated weights for policy 0, policy_version 8110 (0.0009) [2023-10-08 08:09:26,423][53852] Updated weights for policy 0, policy_version 8120 (0.0010) [2023-10-08 08:09:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16580608. Throughput: 0: 1837.7, 1: 1821.5. Samples: 4147302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:09:27,016][52710] Avg episode reward: [(0, '16.570'), (1, '16.370')] [2023-10-08 08:09:27,535][53885] Updated weights for policy 1, policy_version 8070 (0.0010) [2023-10-08 08:09:27,907][53885] Updated weights for policy 1, policy_version 8080 (0.0009) [2023-10-08 08:09:28,286][53885] Updated weights for policy 1, policy_version 8090 (0.0009) [2023-10-08 08:09:30,197][53852] Updated weights for policy 0, policy_version 8130 (0.0008) [2023-10-08 08:09:30,602][53852] Updated weights for policy 0, policy_version 8140 (0.0010) [2023-10-08 08:09:30,970][53852] Updated weights for policy 0, policy_version 8150 (0.0008) [2023-10-08 08:09:31,335][53852] Updated weights for policy 0, policy_version 8160 (0.0010) [2023-10-08 08:09:31,893][53885] Updated weights for policy 1, policy_version 8100 (0.0008) [2023-10-08 08:09:32,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16646144. Throughput: 0: 1824.7, 1: 1827.4. Samples: 4169696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:09:32,016][52710] Avg episode reward: [(0, '16.820'), (1, '17.490')] [2023-10-08 08:09:32,266][53885] Updated weights for policy 1, policy_version 8110 (0.0008) [2023-10-08 08:09:32,641][53885] Updated weights for policy 1, policy_version 8120 (0.0010) [2023-10-08 08:09:35,084][53852] Updated weights for policy 0, policy_version 8170 (0.0008) [2023-10-08 08:09:35,457][53852] Updated weights for policy 0, policy_version 8180 (0.0008) [2023-10-08 08:09:35,838][53852] Updated weights for policy 0, policy_version 8190 (0.0010) [2023-10-08 08:09:36,351][53885] Updated weights for policy 1, policy_version 8130 (0.0008) [2023-10-08 08:09:36,717][53885] Updated weights for policy 1, policy_version 8140 (0.0007) [2023-10-08 08:09:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 16711680. Throughput: 0: 1827.7, 1: 1830.7. Samples: 4191148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:09:37,015][52710] Avg episode reward: [(0, '17.620'), (1, '17.310')] [2023-10-08 08:09:37,023][53500] Saving new best policy, reward=17.620! [2023-10-08 08:09:37,082][53885] Updated weights for policy 1, policy_version 8150 (0.0008) [2023-10-08 08:09:37,452][53885] Updated weights for policy 1, policy_version 8160 (0.0007) [2023-10-08 08:09:39,395][53852] Updated weights for policy 0, policy_version 8200 (0.0008) [2023-10-08 08:09:39,763][53852] Updated weights for policy 0, policy_version 8210 (0.0007) [2023-10-08 08:09:40,121][53852] Updated weights for policy 0, policy_version 8220 (0.0008) [2023-10-08 08:09:41,068][53885] Updated weights for policy 1, policy_version 8170 (0.0007) [2023-10-08 08:09:41,441][53885] Updated weights for policy 1, policy_version 8180 (0.0008) [2023-10-08 08:09:41,813][53885] Updated weights for policy 1, policy_version 8190 (0.0009) [2023-10-08 08:09:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 16809984. Throughput: 0: 1824.6, 1: 1833.9. Samples: 4202732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:09:42,016][52710] Avg episode reward: [(0, '17.120'), (1, '17.760')] [2023-10-08 08:09:43,845][53852] Updated weights for policy 0, policy_version 8230 (0.0009) [2023-10-08 08:09:44,201][53852] Updated weights for policy 0, policy_version 8240 (0.0007) [2023-10-08 08:09:44,576][53852] Updated weights for policy 0, policy_version 8250 (0.0008) [2023-10-08 08:09:45,429][53885] Updated weights for policy 1, policy_version 8200 (0.0008) [2023-10-08 08:09:45,796][53885] Updated weights for policy 1, policy_version 8210 (0.0007) [2023-10-08 08:09:46,158][53885] Updated weights for policy 1, policy_version 8220 (0.0007) [2023-10-08 08:09:47,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16875520. Throughput: 0: 1829.3, 1: 1823.8. Samples: 4224194. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:09:47,016][52710] Avg episode reward: [(0, '16.630'), (1, '16.630')] [2023-10-08 08:09:48,301][53852] Updated weights for policy 0, policy_version 8260 (0.0007) [2023-10-08 08:09:48,676][53852] Updated weights for policy 0, policy_version 8270 (0.0007) [2023-10-08 08:09:49,036][53852] Updated weights for policy 0, policy_version 8280 (0.0008) [2023-10-08 08:09:49,806][53885] Updated weights for policy 1, policy_version 8230 (0.0007) [2023-10-08 08:09:50,173][53885] Updated weights for policy 1, policy_version 8240 (0.0007) [2023-10-08 08:09:50,540][53885] Updated weights for policy 1, policy_version 8250 (0.0008) [2023-10-08 08:09:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 16941056. Throughput: 0: 1821.3, 1: 1838.4. Samples: 4246110. Policy #0 lag: (min: 17.0, avg: 38.1, max: 40.0) [2023-10-08 08:09:52,016][52710] Avg episode reward: [(0, '17.060'), (1, '16.700')] [2023-10-08 08:09:52,625][53852] Updated weights for policy 0, policy_version 8290 (0.0007) [2023-10-08 08:09:52,989][53852] Updated weights for policy 0, policy_version 8300 (0.0007) [2023-10-08 08:09:53,367][53852] Updated weights for policy 0, policy_version 8310 (0.0007) [2023-10-08 08:09:53,736][53852] Updated weights for policy 0, policy_version 8320 (0.0008) [2023-10-08 08:09:54,104][53885] Updated weights for policy 1, policy_version 8260 (0.0010) [2023-10-08 08:09:54,464][53885] Updated weights for policy 1, policy_version 8270 (0.0008) [2023-10-08 08:09:54,832][53885] Updated weights for policy 1, policy_version 8280 (0.0008) [2023-10-08 08:09:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17006592. Throughput: 0: 1826.2, 1: 1819.5. Samples: 4256778. Policy #0 lag: (min: 17.0, avg: 38.1, max: 40.0) [2023-10-08 08:09:57,016][52710] Avg episode reward: [(0, '16.470'), (1, '16.090')] [2023-10-08 08:09:57,304][53852] Updated weights for policy 0, policy_version 8330 (0.0007) [2023-10-08 08:09:57,679][53852] Updated weights for policy 0, policy_version 8340 (0.0008) [2023-10-08 08:09:58,048][53852] Updated weights for policy 0, policy_version 8350 (0.0008) [2023-10-08 08:09:58,590][53885] Updated weights for policy 1, policy_version 8290 (0.0011) [2023-10-08 08:09:58,954][53885] Updated weights for policy 1, policy_version 8300 (0.0009) [2023-10-08 08:09:59,320][53885] Updated weights for policy 1, policy_version 8310 (0.0007) [2023-10-08 08:09:59,694][53885] Updated weights for policy 1, policy_version 8320 (0.0007) [2023-10-08 08:10:01,646][53852] Updated weights for policy 0, policy_version 8360 (0.0009) [2023-10-08 08:10:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 17072128. Throughput: 0: 1827.7, 1: 1834.2. Samples: 4279006. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:10:02,015][52710] Avg episode reward: [(0, '16.240'), (1, '16.950')] [2023-10-08 08:10:02,029][53852] Updated weights for policy 0, policy_version 8370 (0.0008) [2023-10-08 08:10:02,392][53852] Updated weights for policy 0, policy_version 8380 (0.0009) [2023-10-08 08:10:03,529][53885] Updated weights for policy 1, policy_version 8330 (0.0008) [2023-10-08 08:10:03,897][53885] Updated weights for policy 1, policy_version 8340 (0.0008) [2023-10-08 08:10:04,260][53885] Updated weights for policy 1, policy_version 8350 (0.0007) [2023-10-08 08:10:06,024][53852] Updated weights for policy 0, policy_version 8390 (0.0009) [2023-10-08 08:10:06,387][53852] Updated weights for policy 0, policy_version 8400 (0.0007) [2023-10-08 08:10:06,768][53852] Updated weights for policy 0, policy_version 8410 (0.0008) [2023-10-08 08:10:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 17170432. Throughput: 0: 1820.6, 1: 1826.3. Samples: 4300878. Policy #0 lag: (min: 0.0, avg: 20.6, max: 32.0) [2023-10-08 08:10:07,016][52710] Avg episode reward: [(0, '16.290'), (1, '16.410')] [2023-10-08 08:10:07,026][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000008352_8552448.pth... [2023-10-08 08:10:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000008416_8617984.pth... [2023-10-08 08:10:07,056][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000006688_6848512.pth [2023-10-08 08:10:07,057][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000006656_6815744.pth [2023-10-08 08:10:07,060][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000008416_8617984.pth [2023-10-08 08:10:07,061][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000008352_8552448.pth [2023-10-08 08:10:07,886][53885] Updated weights for policy 1, policy_version 8360 (0.0009) [2023-10-08 08:10:08,252][53885] Updated weights for policy 1, policy_version 8370 (0.0009) [2023-10-08 08:10:08,631][53885] Updated weights for policy 1, policy_version 8380 (0.0007) [2023-10-08 08:10:10,494][53852] Updated weights for policy 0, policy_version 8420 (0.0008) [2023-10-08 08:10:10,863][53852] Updated weights for policy 0, policy_version 8430 (0.0010) [2023-10-08 08:10:11,231][53852] Updated weights for policy 0, policy_version 8440 (0.0008) [2023-10-08 08:10:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17235968. Throughput: 0: 1827.5, 1: 1825.6. Samples: 4311690. Policy #0 lag: (min: 0.0, avg: 20.6, max: 32.0) [2023-10-08 08:10:12,015][52710] Avg episode reward: [(0, '16.080'), (1, '17.260')] [2023-10-08 08:10:12,395][53885] Updated weights for policy 1, policy_version 8390 (0.0010) [2023-10-08 08:10:12,760][53885] Updated weights for policy 1, policy_version 8400 (0.0009) [2023-10-08 08:10:13,136][53885] Updated weights for policy 1, policy_version 8410 (0.0008) [2023-10-08 08:10:14,948][53852] Updated weights for policy 0, policy_version 8450 (0.0007) [2023-10-08 08:10:15,313][53852] Updated weights for policy 0, policy_version 8460 (0.0011) [2023-10-08 08:10:15,692][53852] Updated weights for policy 0, policy_version 8470 (0.0009) [2023-10-08 08:10:16,061][53852] Updated weights for policy 0, policy_version 8480 (0.0008) [2023-10-08 08:10:16,839][53885] Updated weights for policy 1, policy_version 8420 (0.0009) [2023-10-08 08:10:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17301504. Throughput: 0: 1822.6, 1: 1820.1. Samples: 4333618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:10:17,016][52710] Avg episode reward: [(0, '17.090'), (1, '16.390')] [2023-10-08 08:10:17,232][53885] Updated weights for policy 1, policy_version 8430 (0.0010) [2023-10-08 08:10:17,607][53885] Updated weights for policy 1, policy_version 8440 (0.0010) [2023-10-08 08:10:19,788][53852] Updated weights for policy 0, policy_version 8490 (0.0009) [2023-10-08 08:10:20,169][53852] Updated weights for policy 0, policy_version 8500 (0.0009) [2023-10-08 08:10:20,544][53852] Updated weights for policy 0, policy_version 8510 (0.0008) [2023-10-08 08:10:21,352][53885] Updated weights for policy 1, policy_version 8450 (0.0008) [2023-10-08 08:10:21,728][53885] Updated weights for policy 1, policy_version 8460 (0.0010) [2023-10-08 08:10:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17367040. Throughput: 0: 1832.0, 1: 1813.2. Samples: 4355180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:10:22,016][52710] Avg episode reward: [(0, '17.380'), (1, '17.140')] [2023-10-08 08:10:22,088][53885] Updated weights for policy 1, policy_version 8470 (0.0008) [2023-10-08 08:10:22,459][53885] Updated weights for policy 1, policy_version 8480 (0.0009) [2023-10-08 08:10:24,035][53852] Updated weights for policy 0, policy_version 8520 (0.0008) [2023-10-08 08:10:24,405][53852] Updated weights for policy 0, policy_version 8530 (0.0008) [2023-10-08 08:10:24,765][53852] Updated weights for policy 0, policy_version 8540 (0.0007) [2023-10-08 08:10:26,200][53885] Updated weights for policy 1, policy_version 8490 (0.0009) [2023-10-08 08:10:26,569][53885] Updated weights for policy 1, policy_version 8500 (0.0010) [2023-10-08 08:10:26,934][53885] Updated weights for policy 1, policy_version 8510 (0.0007) [2023-10-08 08:10:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17465344. Throughput: 0: 1824.3, 1: 1812.2. Samples: 4366372. Policy #0 lag: (min: 6.0, avg: 6.2, max: 14.0) [2023-10-08 08:10:27,016][52710] Avg episode reward: [(0, '17.490'), (1, '17.350')] [2023-10-08 08:10:28,517][53852] Updated weights for policy 0, policy_version 8550 (0.0007) [2023-10-08 08:10:28,887][53852] Updated weights for policy 0, policy_version 8560 (0.0009) [2023-10-08 08:10:29,262][53852] Updated weights for policy 0, policy_version 8570 (0.0007) [2023-10-08 08:10:30,682][53885] Updated weights for policy 1, policy_version 8520 (0.0008) [2023-10-08 08:10:31,061][53885] Updated weights for policy 1, policy_version 8530 (0.0009) [2023-10-08 08:10:31,429][53885] Updated weights for policy 1, policy_version 8540 (0.0008) [2023-10-08 08:10:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17530880. Throughput: 0: 1829.5, 1: 1816.3. Samples: 4388256. Policy #0 lag: (min: 6.0, avg: 6.2, max: 14.0) [2023-10-08 08:10:32,016][52710] Avg episode reward: [(0, '17.220'), (1, '16.900')] [2023-10-08 08:10:33,002][53852] Updated weights for policy 0, policy_version 8580 (0.0008) [2023-10-08 08:10:33,368][53852] Updated weights for policy 0, policy_version 8590 (0.0007) [2023-10-08 08:10:33,742][53852] Updated weights for policy 0, policy_version 8600 (0.0009) [2023-10-08 08:10:34,898][53885] Updated weights for policy 1, policy_version 8550 (0.0010) [2023-10-08 08:10:35,271][53885] Updated weights for policy 1, policy_version 8560 (0.0010) [2023-10-08 08:10:35,639][53885] Updated weights for policy 1, policy_version 8570 (0.0011) [2023-10-08 08:10:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17596416. Throughput: 0: 1821.8, 1: 1808.1. Samples: 4409458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:10:37,015][52710] Avg episode reward: [(0, '17.500'), (1, '17.170')] [2023-10-08 08:10:37,603][53852] Updated weights for policy 0, policy_version 8610 (0.0008) [2023-10-08 08:10:37,980][53852] Updated weights for policy 0, policy_version 8620 (0.0010) [2023-10-08 08:10:38,351][53852] Updated weights for policy 0, policy_version 8630 (0.0011) [2023-10-08 08:10:38,718][53852] Updated weights for policy 0, policy_version 8640 (0.0011) [2023-10-08 08:10:39,442][53885] Updated weights for policy 1, policy_version 8580 (0.0009) [2023-10-08 08:10:39,804][53885] Updated weights for policy 1, policy_version 8590 (0.0009) [2023-10-08 08:10:40,174][53885] Updated weights for policy 1, policy_version 8600 (0.0011) [2023-10-08 08:10:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 17661952. Throughput: 0: 1818.8, 1: 1821.9. Samples: 4420610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:10:42,016][52710] Avg episode reward: [(0, '16.480'), (1, '17.990')] [2023-10-08 08:10:42,018][53594] Saving new best policy, reward=17.990! [2023-10-08 08:10:42,472][53852] Updated weights for policy 0, policy_version 8650 (0.0007) [2023-10-08 08:10:42,846][53852] Updated weights for policy 0, policy_version 8660 (0.0009) [2023-10-08 08:10:43,213][53852] Updated weights for policy 0, policy_version 8670 (0.0007) [2023-10-08 08:10:43,748][53885] Updated weights for policy 1, policy_version 8610 (0.0008) [2023-10-08 08:10:44,126][53885] Updated weights for policy 1, policy_version 8620 (0.0009) [2023-10-08 08:10:44,504][53885] Updated weights for policy 1, policy_version 8630 (0.0011) [2023-10-08 08:10:44,871][53885] Updated weights for policy 1, policy_version 8640 (0.0011) [2023-10-08 08:10:47,006][53852] Updated weights for policy 0, policy_version 8680 (0.0008) [2023-10-08 08:10:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 17727488. Throughput: 0: 1811.1, 1: 1811.3. Samples: 4442014. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 08:10:47,015][52710] Avg episode reward: [(0, '16.390'), (1, '16.720')] [2023-10-08 08:10:47,383][53852] Updated weights for policy 0, policy_version 8690 (0.0009) [2023-10-08 08:10:47,738][53852] Updated weights for policy 0, policy_version 8700 (0.0009) [2023-10-08 08:10:48,556][53885] Updated weights for policy 1, policy_version 8650 (0.0010) [2023-10-08 08:10:48,931][53885] Updated weights for policy 1, policy_version 8660 (0.0007) [2023-10-08 08:10:49,299][53885] Updated weights for policy 1, policy_version 8670 (0.0007) [2023-10-08 08:10:51,475][53852] Updated weights for policy 0, policy_version 8710 (0.0011) [2023-10-08 08:10:51,844][53852] Updated weights for policy 0, policy_version 8720 (0.0010) [2023-10-08 08:10:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 17793024. Throughput: 0: 1820.8, 1: 1814.8. Samples: 4464480. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 08:10:52,016][52710] Avg episode reward: [(0, '16.340'), (1, '17.110')] [2023-10-08 08:10:52,213][53852] Updated weights for policy 0, policy_version 8730 (0.0008) [2023-10-08 08:10:52,857][53885] Updated weights for policy 1, policy_version 8680 (0.0008) [2023-10-08 08:10:53,223][53885] Updated weights for policy 1, policy_version 8690 (0.0007) [2023-10-08 08:10:53,588][53885] Updated weights for policy 1, policy_version 8700 (0.0007) [2023-10-08 08:10:55,826][53852] Updated weights for policy 0, policy_version 8740 (0.0007) [2023-10-08 08:10:56,197][53852] Updated weights for policy 0, policy_version 8750 (0.0009) [2023-10-08 08:10:56,567][53852] Updated weights for policy 0, policy_version 8760 (0.0010) [2023-10-08 08:10:57,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 17891328. Throughput: 0: 1812.2, 1: 1817.2. Samples: 4475014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 08:10:57,016][52710] Avg episode reward: [(0, '16.430'), (1, '18.400')] [2023-10-08 08:10:57,252][53885] Updated weights for policy 1, policy_version 8710 (0.0008) [2023-10-08 08:10:57,621][53885] Updated weights for policy 1, policy_version 8720 (0.0008) [2023-10-08 08:10:57,989][53885] Updated weights for policy 1, policy_version 8730 (0.0008) [2023-10-08 08:10:58,217][53594] Saving new best policy, reward=18.400! [2023-10-08 08:11:00,315][53852] Updated weights for policy 0, policy_version 8770 (0.0008) [2023-10-08 08:11:00,682][53852] Updated weights for policy 0, policy_version 8780 (0.0009) [2023-10-08 08:11:01,053][53852] Updated weights for policy 0, policy_version 8790 (0.0008) [2023-10-08 08:11:01,434][53852] Updated weights for policy 0, policy_version 8800 (0.0008) [2023-10-08 08:11:01,684][53885] Updated weights for policy 1, policy_version 8740 (0.0010) [2023-10-08 08:11:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17956864. Throughput: 0: 1819.1, 1: 1816.0. Samples: 4497196. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 08:11:02,016][52710] Avg episode reward: [(0, '15.890'), (1, '16.330')] [2023-10-08 08:11:02,066][53885] Updated weights for policy 1, policy_version 8750 (0.0008) [2023-10-08 08:11:02,429][53885] Updated weights for policy 1, policy_version 8760 (0.0007) [2023-10-08 08:11:05,120][53852] Updated weights for policy 0, policy_version 8810 (0.0009) [2023-10-08 08:11:05,497][53852] Updated weights for policy 0, policy_version 8820 (0.0008) [2023-10-08 08:11:05,865][53852] Updated weights for policy 0, policy_version 8830 (0.0009) [2023-10-08 08:11:06,177][53885] Updated weights for policy 1, policy_version 8770 (0.0010) [2023-10-08 08:11:06,587][53885] Updated weights for policy 1, policy_version 8780 (0.0009) [2023-10-08 08:11:06,949][53885] Updated weights for policy 1, policy_version 8790 (0.0008) [2023-10-08 08:11:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 18022400. Throughput: 0: 1810.6, 1: 1817.2. Samples: 4518434. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 08:11:07,015][52710] Avg episode reward: [(0, '16.430'), (1, '16.650')] [2023-10-08 08:11:07,319][53885] Updated weights for policy 1, policy_version 8800 (0.0008) [2023-10-08 08:11:09,505][53852] Updated weights for policy 0, policy_version 8840 (0.0008) [2023-10-08 08:11:09,869][53852] Updated weights for policy 0, policy_version 8850 (0.0008) [2023-10-08 08:11:10,244][53852] Updated weights for policy 0, policy_version 8860 (0.0008) [2023-10-08 08:11:10,847][53885] Updated weights for policy 1, policy_version 8810 (0.0007) [2023-10-08 08:11:11,216][53885] Updated weights for policy 1, policy_version 8820 (0.0008) [2023-10-08 08:11:11,577][53885] Updated weights for policy 1, policy_version 8830 (0.0009) [2023-10-08 08:11:12,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18120704. Throughput: 0: 1816.7, 1: 1826.4. Samples: 4530310. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 08:11:12,016][52710] Avg episode reward: [(0, '16.470'), (1, '17.470')] [2023-10-08 08:11:13,834][53852] Updated weights for policy 0, policy_version 8870 (0.0008) [2023-10-08 08:11:14,197][53852] Updated weights for policy 0, policy_version 8880 (0.0008) [2023-10-08 08:11:14,570][53852] Updated weights for policy 0, policy_version 8890 (0.0008) [2023-10-08 08:11:15,307][53885] Updated weights for policy 1, policy_version 8840 (0.0009) [2023-10-08 08:11:15,679][53885] Updated weights for policy 1, policy_version 8850 (0.0009) [2023-10-08 08:11:16,046][53885] Updated weights for policy 1, policy_version 8860 (0.0007) [2023-10-08 08:11:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18186240. Throughput: 0: 1808.8, 1: 1818.9. Samples: 4551502. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 08:11:17,015][52710] Avg episode reward: [(0, '16.410'), (1, '16.780')] [2023-10-08 08:11:18,237][53852] Updated weights for policy 0, policy_version 8900 (0.0009) [2023-10-08 08:11:18,605][53852] Updated weights for policy 0, policy_version 8910 (0.0009) [2023-10-08 08:11:18,977][53852] Updated weights for policy 0, policy_version 8920 (0.0010) [2023-10-08 08:11:19,686][53885] Updated weights for policy 1, policy_version 8870 (0.0010) [2023-10-08 08:11:20,056][53885] Updated weights for policy 1, policy_version 8880 (0.0008) [2023-10-08 08:11:20,426][53885] Updated weights for policy 1, policy_version 8890 (0.0010) [2023-10-08 08:11:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18251776. Throughput: 0: 1820.3, 1: 1831.0. Samples: 4573764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:11:22,016][52710] Avg episode reward: [(0, '16.450'), (1, '16.730')] [2023-10-08 08:11:22,679][53852] Updated weights for policy 0, policy_version 8930 (0.0009) [2023-10-08 08:11:23,048][53852] Updated weights for policy 0, policy_version 8940 (0.0008) [2023-10-08 08:11:23,419][53852] Updated weights for policy 0, policy_version 8950 (0.0007) [2023-10-08 08:11:23,786][53852] Updated weights for policy 0, policy_version 8960 (0.0007) [2023-10-08 08:11:24,238][53885] Updated weights for policy 1, policy_version 8900 (0.0011) [2023-10-08 08:11:24,614][53885] Updated weights for policy 1, policy_version 8910 (0.0007) [2023-10-08 08:11:24,979][53885] Updated weights for policy 1, policy_version 8920 (0.0008) [2023-10-08 08:11:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 18317312. Throughput: 0: 1819.1, 1: 1822.5. Samples: 4584484. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:11:27,016][52710] Avg episode reward: [(0, '17.320'), (1, '17.270')] [2023-10-08 08:11:27,506][53852] Updated weights for policy 0, policy_version 8970 (0.0008) [2023-10-08 08:11:27,877][53852] Updated weights for policy 0, policy_version 8980 (0.0008) [2023-10-08 08:11:28,255][53852] Updated weights for policy 0, policy_version 8990 (0.0008) [2023-10-08 08:11:28,398][53885] Updated weights for policy 1, policy_version 8930 (0.0009) [2023-10-08 08:11:28,770][53885] Updated weights for policy 1, policy_version 8940 (0.0008) [2023-10-08 08:11:29,133][53885] Updated weights for policy 1, policy_version 8950 (0.0008) [2023-10-08 08:11:29,496][53885] Updated weights for policy 1, policy_version 8960 (0.0007) [2023-10-08 08:11:31,896][53852] Updated weights for policy 0, policy_version 9000 (0.0010) [2023-10-08 08:11:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 18382848. Throughput: 0: 1826.4, 1: 1836.8. Samples: 4606856. Policy #0 lag: (min: 12.0, avg: 25.5, max: 44.0) [2023-10-08 08:11:32,016][52710] Avg episode reward: [(0, '16.890'), (1, '18.090')] [2023-10-08 08:11:32,265][53852] Updated weights for policy 0, policy_version 9010 (0.0010) [2023-10-08 08:11:32,638][53852] Updated weights for policy 0, policy_version 9020 (0.0010) [2023-10-08 08:11:33,226][53885] Updated weights for policy 1, policy_version 8970 (0.0009) [2023-10-08 08:11:33,592][53885] Updated weights for policy 1, policy_version 8980 (0.0008) [2023-10-08 08:11:33,963][53885] Updated weights for policy 1, policy_version 8990 (0.0009) [2023-10-08 08:11:36,149][53852] Updated weights for policy 0, policy_version 9030 (0.0009) [2023-10-08 08:11:36,515][53852] Updated weights for policy 0, policy_version 9040 (0.0007) [2023-10-08 08:11:36,886][53852] Updated weights for policy 0, policy_version 9050 (0.0008) [2023-10-08 08:11:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 18448384. Throughput: 0: 1821.6, 1: 1842.0. Samples: 4629338. Policy #0 lag: (min: 12.0, avg: 25.5, max: 44.0) [2023-10-08 08:11:37,016][52710] Avg episode reward: [(0, '17.520'), (1, '17.070')] [2023-10-08 08:11:37,561][53885] Updated weights for policy 1, policy_version 9000 (0.0010) [2023-10-08 08:11:37,927][53885] Updated weights for policy 1, policy_version 9010 (0.0011) [2023-10-08 08:11:38,289][53885] Updated weights for policy 1, policy_version 9020 (0.0009) [2023-10-08 08:11:40,456][53852] Updated weights for policy 0, policy_version 9060 (0.0007) [2023-10-08 08:11:40,818][53852] Updated weights for policy 0, policy_version 9070 (0.0008) [2023-10-08 08:11:41,183][53852] Updated weights for policy 0, policy_version 9080 (0.0010) [2023-10-08 08:11:41,994][53885] Updated weights for policy 1, policy_version 9030 (0.0010) [2023-10-08 08:11:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 18546688. Throughput: 0: 1828.5, 1: 1839.5. Samples: 4640076. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 08:11:42,015][52710] Avg episode reward: [(0, '17.740'), (1, '17.390')] [2023-10-08 08:11:42,016][53500] Saving new best policy, reward=17.740! [2023-10-08 08:11:42,355][53885] Updated weights for policy 1, policy_version 9040 (0.0010) [2023-10-08 08:11:42,728][53885] Updated weights for policy 1, policy_version 9050 (0.0010) [2023-10-08 08:11:44,757][53852] Updated weights for policy 0, policy_version 9090 (0.0010) [2023-10-08 08:11:45,133][53852] Updated weights for policy 0, policy_version 9100 (0.0007) [2023-10-08 08:11:45,502][53852] Updated weights for policy 0, policy_version 9110 (0.0008) [2023-10-08 08:11:45,867][53852] Updated weights for policy 0, policy_version 9120 (0.0007) [2023-10-08 08:11:46,442][53885] Updated weights for policy 1, policy_version 9060 (0.0010) [2023-10-08 08:11:46,808][53885] Updated weights for policy 1, policy_version 9070 (0.0008) [2023-10-08 08:11:47,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 18612224. Throughput: 0: 1824.3, 1: 1841.2. Samples: 4662146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:11:47,016][52710] Avg episode reward: [(0, '17.780'), (1, '16.380')] [2023-10-08 08:11:47,018][53500] Saving new best policy, reward=17.780! [2023-10-08 08:11:47,171][53885] Updated weights for policy 1, policy_version 9080 (0.0009) [2023-10-08 08:11:49,672][53852] Updated weights for policy 0, policy_version 9130 (0.0009) [2023-10-08 08:11:50,046][53852] Updated weights for policy 0, policy_version 9140 (0.0008) [2023-10-08 08:11:50,414][53852] Updated weights for policy 0, policy_version 9150 (0.0008) [2023-10-08 08:11:51,047][53885] Updated weights for policy 1, policy_version 9090 (0.0009) [2023-10-08 08:11:51,457][53885] Updated weights for policy 1, policy_version 9100 (0.0007) [2023-10-08 08:11:51,824][53885] Updated weights for policy 1, policy_version 9110 (0.0008) [2023-10-08 08:11:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18677760. Throughput: 0: 1834.8, 1: 1831.9. Samples: 4683436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:11:52,016][52710] Avg episode reward: [(0, '17.110'), (1, '16.280')] [2023-10-08 08:11:52,209][53885] Updated weights for policy 1, policy_version 9120 (0.0008) [2023-10-08 08:11:53,948][53852] Updated weights for policy 0, policy_version 9160 (0.0009) [2023-10-08 08:11:54,322][53852] Updated weights for policy 0, policy_version 9170 (0.0008) [2023-10-08 08:11:54,703][53852] Updated weights for policy 0, policy_version 9180 (0.0007) [2023-10-08 08:11:55,909][53885] Updated weights for policy 1, policy_version 9130 (0.0008) [2023-10-08 08:11:56,268][53885] Updated weights for policy 1, policy_version 9140 (0.0009) [2023-10-08 08:11:56,640][53885] Updated weights for policy 1, policy_version 9150 (0.0008) [2023-10-08 08:11:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18776064. Throughput: 0: 1821.3, 1: 1827.7. Samples: 4694516. Policy #0 lag: (min: 17.0, avg: 17.0, max: 21.0) [2023-10-08 08:11:57,016][52710] Avg episode reward: [(0, '17.620'), (1, '17.010')] [2023-10-08 08:11:58,261][53852] Updated weights for policy 0, policy_version 9190 (0.0009) [2023-10-08 08:11:58,635][53852] Updated weights for policy 0, policy_version 9200 (0.0010) [2023-10-08 08:11:59,011][53852] Updated weights for policy 0, policy_version 9210 (0.0010) [2023-10-08 08:12:00,328][53885] Updated weights for policy 1, policy_version 9160 (0.0007) [2023-10-08 08:12:00,703][53885] Updated weights for policy 1, policy_version 9170 (0.0009) [2023-10-08 08:12:01,077][53885] Updated weights for policy 1, policy_version 9180 (0.0008) [2023-10-08 08:12:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 18841600. Throughput: 0: 1839.8, 1: 1825.1. Samples: 4716424. Policy #0 lag: (min: 17.0, avg: 17.0, max: 21.0) [2023-10-08 08:12:02,016][52710] Avg episode reward: [(0, '16.880'), (1, '17.260')] [2023-10-08 08:12:02,815][53852] Updated weights for policy 0, policy_version 9220 (0.0010) [2023-10-08 08:12:03,173][53852] Updated weights for policy 0, policy_version 9230 (0.0010) [2023-10-08 08:12:03,541][53852] Updated weights for policy 0, policy_version 9240 (0.0009) [2023-10-08 08:12:04,727][53885] Updated weights for policy 1, policy_version 9190 (0.0010) [2023-10-08 08:12:05,103][53885] Updated weights for policy 1, policy_version 9200 (0.0010) [2023-10-08 08:12:05,468][53885] Updated weights for policy 1, policy_version 9210 (0.0010) [2023-10-08 08:12:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 18907136. Throughput: 0: 1838.7, 1: 1825.4. Samples: 4738648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:12:07,016][52710] Avg episode reward: [(0, '17.930'), (1, '17.600')] [2023-10-08 08:12:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000009216_9437184.pth... [2023-10-08 08:12:07,063][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000007520_7700480.pth [2023-10-08 08:12:07,158][53852] Updated weights for policy 0, policy_version 9250 (0.0010) [2023-10-08 08:12:07,524][53852] Updated weights for policy 0, policy_version 9260 (0.0007) [2023-10-08 08:12:07,897][53852] Updated weights for policy 0, policy_version 9270 (0.0007) [2023-10-08 08:12:08,260][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000009280_9502720.pth... [2023-10-08 08:12:08,262][53852] Updated weights for policy 0, policy_version 9280 (0.0008) [2023-10-08 08:12:08,297][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000007552_7733248.pth [2023-10-08 08:12:08,302][53500] Saving new best policy, reward=17.930! [2023-10-08 08:12:09,028][53885] Updated weights for policy 1, policy_version 9220 (0.0010) [2023-10-08 08:12:09,402][53885] Updated weights for policy 1, policy_version 9230 (0.0008) [2023-10-08 08:12:09,771][53885] Updated weights for policy 1, policy_version 9240 (0.0010) [2023-10-08 08:12:11,905][53852] Updated weights for policy 0, policy_version 9290 (0.0008) [2023-10-08 08:12:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 18972672. Throughput: 0: 1842.1, 1: 1825.7. Samples: 4749536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:12:12,016][52710] Avg episode reward: [(0, '16.650'), (1, '18.270')] [2023-10-08 08:12:12,292][53852] Updated weights for policy 0, policy_version 9300 (0.0010) [2023-10-08 08:12:12,658][53852] Updated weights for policy 0, policy_version 9310 (0.0008) [2023-10-08 08:12:13,550][53885] Updated weights for policy 1, policy_version 9250 (0.0010) [2023-10-08 08:12:13,917][53885] Updated weights for policy 1, policy_version 9260 (0.0007) [2023-10-08 08:12:14,275][53885] Updated weights for policy 1, policy_version 9270 (0.0009) [2023-10-08 08:12:14,641][53885] Updated weights for policy 1, policy_version 9280 (0.0009) [2023-10-08 08:12:16,396][53852] Updated weights for policy 0, policy_version 9320 (0.0007) [2023-10-08 08:12:16,768][53852] Updated weights for policy 0, policy_version 9330 (0.0007) [2023-10-08 08:12:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 19038208. Throughput: 0: 1840.7, 1: 1820.2. Samples: 4771598. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 08:12:17,016][52710] Avg episode reward: [(0, '17.390'), (1, '18.520')] [2023-10-08 08:12:17,018][53594] Saving new best policy, reward=18.520! [2023-10-08 08:12:17,138][53852] Updated weights for policy 0, policy_version 9340 (0.0008) [2023-10-08 08:12:18,339][53885] Updated weights for policy 1, policy_version 9290 (0.0009) [2023-10-08 08:12:18,715][53885] Updated weights for policy 1, policy_version 9300 (0.0009) [2023-10-08 08:12:19,084][53885] Updated weights for policy 1, policy_version 9310 (0.0007) [2023-10-08 08:12:20,770][53852] Updated weights for policy 0, policy_version 9350 (0.0009) [2023-10-08 08:12:21,135][53852] Updated weights for policy 0, policy_version 9360 (0.0007) [2023-10-08 08:12:21,506][53852] Updated weights for policy 0, policy_version 9370 (0.0010) [2023-10-08 08:12:22,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 19136512. Throughput: 0: 1833.4, 1: 1814.6. Samples: 4793500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:12:22,015][52710] Avg episode reward: [(0, '17.950'), (1, '17.980')] [2023-10-08 08:12:22,027][53500] Saving new best policy, reward=17.950! [2023-10-08 08:12:22,752][53885] Updated weights for policy 1, policy_version 9320 (0.0008) [2023-10-08 08:12:23,110][53885] Updated weights for policy 1, policy_version 9330 (0.0008) [2023-10-08 08:12:23,484][53885] Updated weights for policy 1, policy_version 9340 (0.0007) [2023-10-08 08:12:25,223][53852] Updated weights for policy 0, policy_version 9380 (0.0008) [2023-10-08 08:12:25,587][53852] Updated weights for policy 0, policy_version 9390 (0.0011) [2023-10-08 08:12:25,959][53852] Updated weights for policy 0, policy_version 9400 (0.0012) [2023-10-08 08:12:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19202048. Throughput: 0: 1842.4, 1: 1816.0. Samples: 4804708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:12:27,016][52710] Avg episode reward: [(0, '17.340'), (1, '17.410')] [2023-10-08 08:12:27,086][53885] Updated weights for policy 1, policy_version 9350 (0.0008) [2023-10-08 08:12:27,453][53885] Updated weights for policy 1, policy_version 9360 (0.0009) [2023-10-08 08:12:27,824][53885] Updated weights for policy 1, policy_version 9370 (0.0008) [2023-10-08 08:12:29,507][53852] Updated weights for policy 0, policy_version 9410 (0.0010) [2023-10-08 08:12:29,877][53852] Updated weights for policy 0, policy_version 9420 (0.0008) [2023-10-08 08:12:30,251][53852] Updated weights for policy 0, policy_version 9430 (0.0007) [2023-10-08 08:12:30,626][53852] Updated weights for policy 0, policy_version 9440 (0.0008) [2023-10-08 08:12:31,514][53885] Updated weights for policy 1, policy_version 9380 (0.0007) [2023-10-08 08:12:31,879][53885] Updated weights for policy 1, policy_version 9390 (0.0009) [2023-10-08 08:12:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19267584. Throughput: 0: 1831.9, 1: 1820.6. Samples: 4826510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:12:32,016][52710] Avg episode reward: [(0, '17.510'), (1, '18.740')] [2023-10-08 08:12:32,252][53885] Updated weights for policy 1, policy_version 9400 (0.0009) [2023-10-08 08:12:32,540][53594] Saving new best policy, reward=18.740! [2023-10-08 08:12:34,364][53852] Updated weights for policy 0, policy_version 9450 (0.0009) [2023-10-08 08:12:34,731][53852] Updated weights for policy 0, policy_version 9460 (0.0007) [2023-10-08 08:12:35,110][53852] Updated weights for policy 0, policy_version 9470 (0.0008) [2023-10-08 08:12:35,790][53885] Updated weights for policy 1, policy_version 9410 (0.0008) [2023-10-08 08:12:36,187][53885] Updated weights for policy 1, policy_version 9420 (0.0010) [2023-10-08 08:12:36,566][53885] Updated weights for policy 1, policy_version 9430 (0.0008) [2023-10-08 08:12:36,935][53885] Updated weights for policy 1, policy_version 9440 (0.0009) [2023-10-08 08:12:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 19365888. Throughput: 0: 1848.9, 1: 1819.5. Samples: 4848514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:12:37,016][52710] Avg episode reward: [(0, '18.830'), (1, '18.520')] [2023-10-08 08:12:37,025][53500] Saving new best policy, reward=18.830! [2023-10-08 08:12:38,857][53852] Updated weights for policy 0, policy_version 9480 (0.0009) [2023-10-08 08:12:39,229][53852] Updated weights for policy 0, policy_version 9490 (0.0008) [2023-10-08 08:12:39,606][53852] Updated weights for policy 0, policy_version 9500 (0.0007) [2023-10-08 08:12:40,585][53885] Updated weights for policy 1, policy_version 9450 (0.0009) [2023-10-08 08:12:40,954][53885] Updated weights for policy 1, policy_version 9460 (0.0010) [2023-10-08 08:12:41,326][53885] Updated weights for policy 1, policy_version 9470 (0.0007) [2023-10-08 08:12:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 19431424. Throughput: 0: 1839.2, 1: 1834.4. Samples: 4859824. Policy #0 lag: (min: 21.0, avg: 21.5, max: 36.0) [2023-10-08 08:12:42,016][52710] Avg episode reward: [(0, '19.060'), (1, '19.060')] [2023-10-08 08:12:42,018][53594] Saving new best policy, reward=19.060! [2023-10-08 08:12:42,018][53500] Saving new best policy, reward=19.060! [2023-10-08 08:12:43,427][53852] Updated weights for policy 0, policy_version 9510 (0.0007) [2023-10-08 08:12:43,801][53852] Updated weights for policy 0, policy_version 9520 (0.0009) [2023-10-08 08:12:44,173][53852] Updated weights for policy 0, policy_version 9530 (0.0008) [2023-10-08 08:12:45,120][53885] Updated weights for policy 1, policy_version 9480 (0.0008) [2023-10-08 08:12:45,491][53885] Updated weights for policy 1, policy_version 9490 (0.0009) [2023-10-08 08:12:45,857][53885] Updated weights for policy 1, policy_version 9500 (0.0010) [2023-10-08 08:12:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19496960. Throughput: 0: 1836.0, 1: 1828.8. Samples: 4881340. Policy #0 lag: (min: 21.0, avg: 21.5, max: 36.0) [2023-10-08 08:12:47,016][52710] Avg episode reward: [(0, '18.170'), (1, '18.410')] [2023-10-08 08:12:47,695][53852] Updated weights for policy 0, policy_version 9540 (0.0008) [2023-10-08 08:12:48,062][53852] Updated weights for policy 0, policy_version 9550 (0.0009) [2023-10-08 08:12:48,439][53852] Updated weights for policy 0, policy_version 9560 (0.0009) [2023-10-08 08:12:49,596][53885] Updated weights for policy 1, policy_version 9510 (0.0008) [2023-10-08 08:12:49,961][53885] Updated weights for policy 1, policy_version 9520 (0.0007) [2023-10-08 08:12:50,330][53885] Updated weights for policy 1, policy_version 9530 (0.0007) [2023-10-08 08:12:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19562496. Throughput: 0: 1842.0, 1: 1832.2. Samples: 4903990. Policy #0 lag: (min: 18.0, avg: 20.5, max: 44.0) [2023-10-08 08:12:52,016][52710] Avg episode reward: [(0, '17.980'), (1, '18.050')] [2023-10-08 08:12:52,045][53852] Updated weights for policy 0, policy_version 9570 (0.0010) [2023-10-08 08:12:52,410][53852] Updated weights for policy 0, policy_version 9580 (0.0009) [2023-10-08 08:12:52,782][53852] Updated weights for policy 0, policy_version 9590 (0.0008) [2023-10-08 08:12:53,156][53852] Updated weights for policy 0, policy_version 9600 (0.0008) [2023-10-08 08:12:53,868][53885] Updated weights for policy 1, policy_version 9540 (0.0008) [2023-10-08 08:12:54,246][53885] Updated weights for policy 1, policy_version 9550 (0.0009) [2023-10-08 08:12:54,626][53885] Updated weights for policy 1, policy_version 9560 (0.0008) [2023-10-08 08:12:56,727][53852] Updated weights for policy 0, policy_version 9610 (0.0008) [2023-10-08 08:12:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 19628032. Throughput: 0: 1839.6, 1: 1824.3. Samples: 4914414. Policy #0 lag: (min: 18.0, avg: 20.5, max: 44.0) [2023-10-08 08:12:57,016][52710] Avg episode reward: [(0, '16.650'), (1, '17.020')] [2023-10-08 08:12:57,097][53852] Updated weights for policy 0, policy_version 9620 (0.0007) [2023-10-08 08:12:57,469][53852] Updated weights for policy 0, policy_version 9630 (0.0008) [2023-10-08 08:12:58,252][53885] Updated weights for policy 1, policy_version 9570 (0.0007) [2023-10-08 08:12:58,622][53885] Updated weights for policy 1, policy_version 9580 (0.0008) [2023-10-08 08:12:58,989][53885] Updated weights for policy 1, policy_version 9590 (0.0007) [2023-10-08 08:12:59,361][53885] Updated weights for policy 1, policy_version 9600 (0.0007) [2023-10-08 08:13:01,100][53852] Updated weights for policy 0, policy_version 9640 (0.0008) [2023-10-08 08:13:01,464][53852] Updated weights for policy 0, policy_version 9650 (0.0009) [2023-10-08 08:13:01,841][53852] Updated weights for policy 0, policy_version 9660 (0.0008) [2023-10-08 08:13:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19726336. Throughput: 0: 1842.4, 1: 1832.1. Samples: 4936954. Policy #0 lag: (min: 14.0, avg: 21.7, max: 46.0) [2023-10-08 08:13:02,016][52710] Avg episode reward: [(0, '17.130'), (1, '19.210')] [2023-10-08 08:13:02,018][53594] Saving new best policy, reward=19.210! [2023-10-08 08:13:03,050][53885] Updated weights for policy 1, policy_version 9610 (0.0008) [2023-10-08 08:13:03,425][53885] Updated weights for policy 1, policy_version 9620 (0.0007) [2023-10-08 08:13:03,790][53885] Updated weights for policy 1, policy_version 9630 (0.0009) [2023-10-08 08:13:05,373][53852] Updated weights for policy 0, policy_version 9670 (0.0008) [2023-10-08 08:13:05,741][53852] Updated weights for policy 0, policy_version 9680 (0.0007) [2023-10-08 08:13:06,113][53852] Updated weights for policy 0, policy_version 9690 (0.0008) [2023-10-08 08:13:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 19791872. Throughput: 0: 1832.6, 1: 1832.5. Samples: 4958430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:13:07,016][52710] Avg episode reward: [(0, '18.050'), (1, '17.340')] [2023-10-08 08:13:07,574][53885] Updated weights for policy 1, policy_version 9640 (0.0009) [2023-10-08 08:13:07,942][53885] Updated weights for policy 1, policy_version 9650 (0.0008) [2023-10-08 08:13:08,311][53885] Updated weights for policy 1, policy_version 9660 (0.0008) [2023-10-08 08:13:09,724][53852] Updated weights for policy 0, policy_version 9700 (0.0007) [2023-10-08 08:13:10,104][53852] Updated weights for policy 0, policy_version 9710 (0.0007) [2023-10-08 08:13:10,472][53852] Updated weights for policy 0, policy_version 9720 (0.0010) [2023-10-08 08:13:11,970][53885] Updated weights for policy 1, policy_version 9670 (0.0007) [2023-10-08 08:13:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19857408. Throughput: 0: 1838.0, 1: 1834.8. Samples: 4969982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:13:12,016][52710] Avg episode reward: [(0, '17.810'), (1, '17.690')] [2023-10-08 08:13:12,336][53885] Updated weights for policy 1, policy_version 9680 (0.0007) [2023-10-08 08:13:12,701][53885] Updated weights for policy 1, policy_version 9690 (0.0007) [2023-10-08 08:13:14,136][53852] Updated weights for policy 0, policy_version 9730 (0.0008) [2023-10-08 08:13:14,497][53852] Updated weights for policy 0, policy_version 9740 (0.0008) [2023-10-08 08:13:14,874][53852] Updated weights for policy 0, policy_version 9750 (0.0008) [2023-10-08 08:13:15,245][53852] Updated weights for policy 0, policy_version 9760 (0.0007) [2023-10-08 08:13:16,262][53885] Updated weights for policy 1, policy_version 9700 (0.0008) [2023-10-08 08:13:16,630][53885] Updated weights for policy 1, policy_version 9710 (0.0009) [2023-10-08 08:13:16,999][53885] Updated weights for policy 1, policy_version 9720 (0.0010) [2023-10-08 08:13:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 19922944. Throughput: 0: 1828.7, 1: 1835.4. Samples: 4991392. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 08:13:17,015][52710] Avg episode reward: [(0, '18.550'), (1, '18.330')] [2023-10-08 08:13:18,882][53852] Updated weights for policy 0, policy_version 9770 (0.0007) [2023-10-08 08:13:19,257][53852] Updated weights for policy 0, policy_version 9780 (0.0009) [2023-10-08 08:13:19,634][53852] Updated weights for policy 0, policy_version 9790 (0.0008) [2023-10-08 08:13:20,766][53885] Updated weights for policy 1, policy_version 9730 (0.0007) [2023-10-08 08:13:21,173][53885] Updated weights for policy 1, policy_version 9740 (0.0008) [2023-10-08 08:13:21,549][53885] Updated weights for policy 1, policy_version 9750 (0.0008) [2023-10-08 08:13:21,907][53885] Updated weights for policy 1, policy_version 9760 (0.0011) [2023-10-08 08:13:22,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20021248. Throughput: 0: 1832.7, 1: 1829.5. Samples: 5013314. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 08:13:22,015][52710] Avg episode reward: [(0, '16.820'), (1, '18.820')] [2023-10-08 08:13:23,421][53852] Updated weights for policy 0, policy_version 9800 (0.0009) [2023-10-08 08:13:23,789][53852] Updated weights for policy 0, policy_version 9810 (0.0010) [2023-10-08 08:13:24,166][53852] Updated weights for policy 0, policy_version 9820 (0.0009) [2023-10-08 08:13:25,685][53885] Updated weights for policy 1, policy_version 9770 (0.0010) [2023-10-08 08:13:26,058][53885] Updated weights for policy 1, policy_version 9780 (0.0008) [2023-10-08 08:13:26,428][53885] Updated weights for policy 1, policy_version 9790 (0.0007) [2023-10-08 08:13:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20086784. Throughput: 0: 1827.7, 1: 1826.9. Samples: 5024280. Policy #0 lag: (min: 27.0, avg: 36.6, max: 59.0) [2023-10-08 08:13:27,016][52710] Avg episode reward: [(0, '17.140'), (1, '17.800')] [2023-10-08 08:13:27,803][53852] Updated weights for policy 0, policy_version 9830 (0.0007) [2023-10-08 08:13:28,175][53852] Updated weights for policy 0, policy_version 9840 (0.0008) [2023-10-08 08:13:28,553][53852] Updated weights for policy 0, policy_version 9850 (0.0010) [2023-10-08 08:13:29,978][53885] Updated weights for policy 1, policy_version 9800 (0.0009) [2023-10-08 08:13:30,345][53885] Updated weights for policy 1, policy_version 9810 (0.0010) [2023-10-08 08:13:30,720][53885] Updated weights for policy 1, policy_version 9820 (0.0009) [2023-10-08 08:13:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20152320. Throughput: 0: 1840.2, 1: 1827.4. Samples: 5046384. Policy #0 lag: (min: 27.0, avg: 36.6, max: 59.0) [2023-10-08 08:13:32,015][52710] Avg episode reward: [(0, '17.230'), (1, '17.770')] [2023-10-08 08:13:32,084][53852] Updated weights for policy 0, policy_version 9860 (0.0011) [2023-10-08 08:13:32,467][53852] Updated weights for policy 0, policy_version 9870 (0.0009) [2023-10-08 08:13:32,835][53852] Updated weights for policy 0, policy_version 9880 (0.0010) [2023-10-08 08:13:34,294][53885] Updated weights for policy 1, policy_version 9830 (0.0008) [2023-10-08 08:13:34,670][53885] Updated weights for policy 1, policy_version 9840 (0.0007) [2023-10-08 08:13:35,037][53885] Updated weights for policy 1, policy_version 9850 (0.0008) [2023-10-08 08:13:36,365][53852] Updated weights for policy 0, policy_version 9890 (0.0007) [2023-10-08 08:13:36,743][53852] Updated weights for policy 0, policy_version 9900 (0.0009) [2023-10-08 08:13:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 20217856. Throughput: 0: 1831.7, 1: 1833.8. Samples: 5068938. Policy #0 lag: (min: 26.0, avg: 26.9, max: 46.0) [2023-10-08 08:13:37,015][52710] Avg episode reward: [(0, '17.570'), (1, '18.150')] [2023-10-08 08:13:37,117][53852] Updated weights for policy 0, policy_version 9910 (0.0009) [2023-10-08 08:13:37,477][53852] Updated weights for policy 0, policy_version 9920 (0.0007) [2023-10-08 08:13:38,717][53885] Updated weights for policy 1, policy_version 9860 (0.0011) [2023-10-08 08:13:39,095][53885] Updated weights for policy 1, policy_version 9870 (0.0008) [2023-10-08 08:13:39,469][53885] Updated weights for policy 1, policy_version 9880 (0.0007) [2023-10-08 08:13:41,060][53852] Updated weights for policy 0, policy_version 9930 (0.0007) [2023-10-08 08:13:41,436][53852] Updated weights for policy 0, policy_version 9940 (0.0009) [2023-10-08 08:13:41,808][53852] Updated weights for policy 0, policy_version 9950 (0.0010) [2023-10-08 08:13:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 20316160. Throughput: 0: 1844.1, 1: 1829.0. Samples: 5079706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:13:42,016][52710] Avg episode reward: [(0, '17.170'), (1, '17.830')] [2023-10-08 08:13:43,164][53885] Updated weights for policy 1, policy_version 9890 (0.0009) [2023-10-08 08:13:43,547][53885] Updated weights for policy 1, policy_version 9900 (0.0011) [2023-10-08 08:13:43,914][53885] Updated weights for policy 1, policy_version 9910 (0.0008) [2023-10-08 08:13:44,285][53885] Updated weights for policy 1, policy_version 9920 (0.0007) [2023-10-08 08:13:45,480][53852] Updated weights for policy 0, policy_version 9960 (0.0010) [2023-10-08 08:13:45,854][53852] Updated weights for policy 0, policy_version 9970 (0.0011) [2023-10-08 08:13:46,221][53852] Updated weights for policy 0, policy_version 9980 (0.0010) [2023-10-08 08:13:47,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 20381696. Throughput: 0: 1833.0, 1: 1829.3. Samples: 5101760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:13:47,016][52710] Avg episode reward: [(0, '16.530'), (1, '18.920')] [2023-10-08 08:13:48,009][53885] Updated weights for policy 1, policy_version 9930 (0.0009) [2023-10-08 08:13:48,383][53885] Updated weights for policy 1, policy_version 9940 (0.0010) [2023-10-08 08:13:48,750][53885] Updated weights for policy 1, policy_version 9950 (0.0008) [2023-10-08 08:13:49,866][53852] Updated weights for policy 0, policy_version 9990 (0.0007) [2023-10-08 08:13:50,239][53852] Updated weights for policy 0, policy_version 10000 (0.0007) [2023-10-08 08:13:50,621][53852] Updated weights for policy 0, policy_version 10010 (0.0007) [2023-10-08 08:13:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20447232. Throughput: 0: 1841.9, 1: 1828.3. Samples: 5123586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:13:52,016][52710] Avg episode reward: [(0, '17.010'), (1, '17.890')] [2023-10-08 08:13:52,442][53885] Updated weights for policy 1, policy_version 9960 (0.0011) [2023-10-08 08:13:52,803][53885] Updated weights for policy 1, policy_version 9970 (0.0010) [2023-10-08 08:13:53,171][53885] Updated weights for policy 1, policy_version 9980 (0.0011) [2023-10-08 08:13:54,243][53852] Updated weights for policy 0, policy_version 10020 (0.0008) [2023-10-08 08:13:54,618][53852] Updated weights for policy 0, policy_version 10030 (0.0009) [2023-10-08 08:13:54,993][53852] Updated weights for policy 0, policy_version 10040 (0.0009) [2023-10-08 08:13:56,899][53885] Updated weights for policy 1, policy_version 9990 (0.0008) [2023-10-08 08:13:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20512768. Throughput: 0: 1833.1, 1: 1825.7. Samples: 5134626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:13:57,016][52710] Avg episode reward: [(0, '17.170'), (1, '18.220')] [2023-10-08 08:13:57,265][53885] Updated weights for policy 1, policy_version 10000 (0.0008) [2023-10-08 08:13:57,629][53885] Updated weights for policy 1, policy_version 10010 (0.0008) [2023-10-08 08:13:58,712][53852] Updated weights for policy 0, policy_version 10050 (0.0010) [2023-10-08 08:13:59,080][53852] Updated weights for policy 0, policy_version 10060 (0.0008) [2023-10-08 08:13:59,448][53852] Updated weights for policy 0, policy_version 10070 (0.0007) [2023-10-08 08:13:59,826][53852] Updated weights for policy 0, policy_version 10080 (0.0007) [2023-10-08 08:14:01,231][53885] Updated weights for policy 1, policy_version 10020 (0.0007) [2023-10-08 08:14:01,604][53885] Updated weights for policy 1, policy_version 10030 (0.0007) [2023-10-08 08:14:01,971][53885] Updated weights for policy 1, policy_version 10040 (0.0009) [2023-10-08 08:14:02,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 20578304. Throughput: 0: 1848.2, 1: 1825.2. Samples: 5156696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:14:02,017][52710] Avg episode reward: [(0, '18.040'), (1, '19.210')] [2023-10-08 08:14:03,373][53852] Updated weights for policy 0, policy_version 10090 (0.0008) [2023-10-08 08:14:03,752][53852] Updated weights for policy 0, policy_version 10100 (0.0007) [2023-10-08 08:14:04,114][53852] Updated weights for policy 0, policy_version 10110 (0.0011) [2023-10-08 08:14:05,513][53885] Updated weights for policy 1, policy_version 10050 (0.0007) [2023-10-08 08:14:05,921][53885] Updated weights for policy 1, policy_version 10060 (0.0007) [2023-10-08 08:14:06,292][53885] Updated weights for policy 1, policy_version 10070 (0.0009) [2023-10-08 08:14:06,676][53885] Updated weights for policy 1, policy_version 10080 (0.0009) [2023-10-08 08:14:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20676608. Throughput: 0: 1844.3, 1: 1820.6. Samples: 5178238. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:14:07,016][52710] Avg episode reward: [(0, '16.700'), (1, '18.420')] [2023-10-08 08:14:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000010112_10354688.pth... [2023-10-08 08:14:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000010080_10321920.pth... [2023-10-08 08:14:07,061][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000008416_8617984.pth [2023-10-08 08:14:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000008352_8552448.pth [2023-10-08 08:14:07,774][53852] Updated weights for policy 0, policy_version 10120 (0.0009) [2023-10-08 08:14:08,142][53852] Updated weights for policy 0, policy_version 10130 (0.0011) [2023-10-08 08:14:08,519][53852] Updated weights for policy 0, policy_version 10140 (0.0009) [2023-10-08 08:14:10,446][53885] Updated weights for policy 1, policy_version 10090 (0.0008) [2023-10-08 08:14:10,823][53885] Updated weights for policy 1, policy_version 10100 (0.0008) [2023-10-08 08:14:11,185][53885] Updated weights for policy 1, policy_version 10110 (0.0010) [2023-10-08 08:14:12,015][52710] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20742144. Throughput: 0: 1847.2, 1: 1826.4. Samples: 5189592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:14:12,015][52710] Avg episode reward: [(0, '16.380'), (1, '18.470')] [2023-10-08 08:14:12,343][53852] Updated weights for policy 0, policy_version 10150 (0.0007) [2023-10-08 08:14:12,714][53852] Updated weights for policy 0, policy_version 10160 (0.0008) [2023-10-08 08:14:13,096][53852] Updated weights for policy 0, policy_version 10170 (0.0008) [2023-10-08 08:14:15,008][53885] Updated weights for policy 1, policy_version 10120 (0.0008) [2023-10-08 08:14:15,384][53885] Updated weights for policy 1, policy_version 10130 (0.0007) [2023-10-08 08:14:15,761][53885] Updated weights for policy 1, policy_version 10140 (0.0008) [2023-10-08 08:14:16,599][53852] Updated weights for policy 0, policy_version 10180 (0.0008) [2023-10-08 08:14:16,962][53852] Updated weights for policy 0, policy_version 10190 (0.0008) [2023-10-08 08:14:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20807680. Throughput: 0: 1841.3, 1: 1818.4. Samples: 5211072. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:14:17,015][52710] Avg episode reward: [(0, '16.580'), (1, '18.530')] [2023-10-08 08:14:17,329][53852] Updated weights for policy 0, policy_version 10200 (0.0009) [2023-10-08 08:14:19,447][53885] Updated weights for policy 1, policy_version 10150 (0.0008) [2023-10-08 08:14:19,819][53885] Updated weights for policy 1, policy_version 10160 (0.0010) [2023-10-08 08:14:20,188][53885] Updated weights for policy 1, policy_version 10170 (0.0008) [2023-10-08 08:14:21,065][53852] Updated weights for policy 0, policy_version 10210 (0.0007) [2023-10-08 08:14:21,444][53852] Updated weights for policy 0, policy_version 10220 (0.0009) [2023-10-08 08:14:21,815][53852] Updated weights for policy 0, policy_version 10230 (0.0008) [2023-10-08 08:14:22,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 20873216. Throughput: 0: 1824.7, 1: 1813.4. Samples: 5232650. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:14:22,016][52710] Avg episode reward: [(0, '16.610'), (1, '18.770')] [2023-10-08 08:14:22,185][53852] Updated weights for policy 0, policy_version 10240 (0.0010) [2023-10-08 08:14:23,875][53885] Updated weights for policy 1, policy_version 10180 (0.0008) [2023-10-08 08:14:24,239][53885] Updated weights for policy 1, policy_version 10190 (0.0007) [2023-10-08 08:14:24,607][53885] Updated weights for policy 1, policy_version 10200 (0.0007) [2023-10-08 08:14:25,783][53852] Updated weights for policy 0, policy_version 10250 (0.0010) [2023-10-08 08:14:26,161][53852] Updated weights for policy 0, policy_version 10260 (0.0008) [2023-10-08 08:14:26,525][53852] Updated weights for policy 0, policy_version 10270 (0.0010) [2023-10-08 08:14:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 20971520. Throughput: 0: 1830.9, 1: 1816.5. Samples: 5243840. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) [2023-10-08 08:14:27,015][52710] Avg episode reward: [(0, '17.440'), (1, '18.670')] [2023-10-08 08:14:28,233][53885] Updated weights for policy 1, policy_version 10210 (0.0009) [2023-10-08 08:14:28,593][53885] Updated weights for policy 1, policy_version 10220 (0.0010) [2023-10-08 08:14:28,962][53885] Updated weights for policy 1, policy_version 10230 (0.0010) [2023-10-08 08:14:29,334][53885] Updated weights for policy 1, policy_version 10240 (0.0010) [2023-10-08 08:14:30,256][53852] Updated weights for policy 0, policy_version 10280 (0.0010) [2023-10-08 08:14:30,631][53852] Updated weights for policy 0, policy_version 10290 (0.0010) [2023-10-08 08:14:31,005][53852] Updated weights for policy 0, policy_version 10300 (0.0009) [2023-10-08 08:14:32,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21037056. Throughput: 0: 1821.7, 1: 1815.3. Samples: 5265424. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) [2023-10-08 08:14:32,016][52710] Avg episode reward: [(0, '16.900'), (1, '18.020')] [2023-10-08 08:14:33,104][53885] Updated weights for policy 1, policy_version 10250 (0.0009) [2023-10-08 08:14:33,483][53885] Updated weights for policy 1, policy_version 10260 (0.0008) [2023-10-08 08:14:33,843][53885] Updated weights for policy 1, policy_version 10270 (0.0010) [2023-10-08 08:14:34,614][53852] Updated weights for policy 0, policy_version 10310 (0.0007) [2023-10-08 08:14:34,979][53852] Updated weights for policy 0, policy_version 10320 (0.0009) [2023-10-08 08:14:35,348][53852] Updated weights for policy 0, policy_version 10330 (0.0007) [2023-10-08 08:14:37,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 21102592. Throughput: 0: 1832.4, 1: 1818.9. Samples: 5287896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:14:37,016][52710] Avg episode reward: [(0, '18.100'), (1, '18.040')] [2023-10-08 08:14:37,539][53885] Updated weights for policy 1, policy_version 10280 (0.0009) [2023-10-08 08:14:37,895][53885] Updated weights for policy 1, policy_version 10290 (0.0008) [2023-10-08 08:14:38,271][53885] Updated weights for policy 1, policy_version 10300 (0.0008) [2023-10-08 08:14:39,028][53852] Updated weights for policy 0, policy_version 10340 (0.0010) [2023-10-08 08:14:39,396][53852] Updated weights for policy 0, policy_version 10350 (0.0010) [2023-10-08 08:14:39,771][53852] Updated weights for policy 0, policy_version 10360 (0.0007) [2023-10-08 08:14:41,883][53885] Updated weights for policy 1, policy_version 10310 (0.0009) [2023-10-08 08:14:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21168128. Throughput: 0: 1824.1, 1: 1818.5. Samples: 5298544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:14:42,015][52710] Avg episode reward: [(0, '17.470'), (1, '18.680')] [2023-10-08 08:14:42,251][53885] Updated weights for policy 1, policy_version 10320 (0.0009) [2023-10-08 08:14:42,625][53885] Updated weights for policy 1, policy_version 10330 (0.0007) [2023-10-08 08:14:43,476][53852] Updated weights for policy 0, policy_version 10370 (0.0007) [2023-10-08 08:14:43,845][53852] Updated weights for policy 0, policy_version 10380 (0.0007) [2023-10-08 08:14:44,213][53852] Updated weights for policy 0, policy_version 10390 (0.0009) [2023-10-08 08:14:44,593][53852] Updated weights for policy 0, policy_version 10400 (0.0007) [2023-10-08 08:14:46,354][53885] Updated weights for policy 1, policy_version 10340 (0.0008) [2023-10-08 08:14:46,719][53885] Updated weights for policy 1, policy_version 10350 (0.0010) [2023-10-08 08:14:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21233664. Throughput: 0: 1830.1, 1: 1815.2. Samples: 5320730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:14:47,016][52710] Avg episode reward: [(0, '18.020'), (1, '18.980')] [2023-10-08 08:14:47,091][53885] Updated weights for policy 1, policy_version 10360 (0.0010) [2023-10-08 08:14:48,165][53852] Updated weights for policy 0, policy_version 10410 (0.0007) [2023-10-08 08:14:48,537][53852] Updated weights for policy 0, policy_version 10420 (0.0008) [2023-10-08 08:14:48,910][53852] Updated weights for policy 0, policy_version 10430 (0.0010) [2023-10-08 08:14:50,812][53885] Updated weights for policy 1, policy_version 10370 (0.0010) [2023-10-08 08:14:51,220][53885] Updated weights for policy 1, policy_version 10380 (0.0007) [2023-10-08 08:14:51,599][53885] Updated weights for policy 1, policy_version 10390 (0.0007) [2023-10-08 08:14:51,964][53885] Updated weights for policy 1, policy_version 10400 (0.0010) [2023-10-08 08:14:52,015][52710] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21331968. Throughput: 0: 1833.3, 1: 1820.9. Samples: 5342680. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:14:52,017][52710] Avg episode reward: [(0, '17.500'), (1, '17.020')] [2023-10-08 08:14:52,547][53852] Updated weights for policy 0, policy_version 10440 (0.0008) [2023-10-08 08:14:52,921][53852] Updated weights for policy 0, policy_version 10450 (0.0007) [2023-10-08 08:14:53,292][53852] Updated weights for policy 0, policy_version 10460 (0.0010) [2023-10-08 08:14:55,415][53885] Updated weights for policy 1, policy_version 10410 (0.0007) [2023-10-08 08:14:55,789][53885] Updated weights for policy 1, policy_version 10420 (0.0008) [2023-10-08 08:14:56,154][53885] Updated weights for policy 1, policy_version 10430 (0.0007) [2023-10-08 08:14:57,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21397504. Throughput: 0: 1834.0, 1: 1818.3. Samples: 5353946. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:14:57,018][52710] Avg episode reward: [(0, '18.350'), (1, '19.000')] [2023-10-08 08:14:57,179][53852] Updated weights for policy 0, policy_version 10470 (0.0009) [2023-10-08 08:14:57,561][53852] Updated weights for policy 0, policy_version 10480 (0.0008) [2023-10-08 08:14:57,934][53852] Updated weights for policy 0, policy_version 10490 (0.0008) [2023-10-08 08:14:59,901][53885] Updated weights for policy 1, policy_version 10440 (0.0010) [2023-10-08 08:15:00,262][53885] Updated weights for policy 1, policy_version 10450 (0.0011) [2023-10-08 08:15:00,630][53885] Updated weights for policy 1, policy_version 10460 (0.0011) [2023-10-08 08:15:01,590][53852] Updated weights for policy 0, policy_version 10500 (0.0009) [2023-10-08 08:15:01,960][53852] Updated weights for policy 0, policy_version 10510 (0.0011) [2023-10-08 08:15:02,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 21463040. Throughput: 0: 1834.2, 1: 1815.9. Samples: 5375326. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:15:02,016][52710] Avg episode reward: [(0, '17.080'), (1, '18.140')] [2023-10-08 08:15:02,337][53852] Updated weights for policy 0, policy_version 10520 (0.0011) [2023-10-08 08:15:04,183][53885] Updated weights for policy 1, policy_version 10470 (0.0009) [2023-10-08 08:15:04,549][53885] Updated weights for policy 1, policy_version 10480 (0.0008) [2023-10-08 08:15:04,920][53885] Updated weights for policy 1, policy_version 10490 (0.0008) [2023-10-08 08:15:06,068][53852] Updated weights for policy 0, policy_version 10530 (0.0009) [2023-10-08 08:15:06,427][53852] Updated weights for policy 0, policy_version 10540 (0.0007) [2023-10-08 08:15:06,800][53852] Updated weights for policy 0, policy_version 10550 (0.0007) [2023-10-08 08:15:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21528576. Throughput: 0: 1832.2, 1: 1823.8. Samples: 5397168. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 08:15:07,016][52710] Avg episode reward: [(0, '18.220'), (1, '17.640')] [2023-10-08 08:15:07,170][53852] Updated weights for policy 0, policy_version 10560 (0.0008) [2023-10-08 08:15:08,612][53885] Updated weights for policy 1, policy_version 10500 (0.0007) [2023-10-08 08:15:08,980][53885] Updated weights for policy 1, policy_version 10510 (0.0008) [2023-10-08 08:15:09,348][53885] Updated weights for policy 1, policy_version 10520 (0.0007) [2023-10-08 08:15:10,760][53852] Updated weights for policy 0, policy_version 10570 (0.0008) [2023-10-08 08:15:11,139][53852] Updated weights for policy 0, policy_version 10580 (0.0009) [2023-10-08 08:15:11,511][53852] Updated weights for policy 0, policy_version 10590 (0.0007) [2023-10-08 08:15:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 21626880. Throughput: 0: 1831.8, 1: 1816.8. Samples: 5408028. Policy #0 lag: (min: 25.0, avg: 37.1, max: 57.0) [2023-10-08 08:15:12,016][52710] Avg episode reward: [(0, '19.100'), (1, '18.740')] [2023-10-08 08:15:12,016][53500] Saving new best policy, reward=19.100! [2023-10-08 08:15:13,076][53885] Updated weights for policy 1, policy_version 10530 (0.0008) [2023-10-08 08:15:13,442][53885] Updated weights for policy 1, policy_version 10540 (0.0007) [2023-10-08 08:15:13,814][53885] Updated weights for policy 1, policy_version 10550 (0.0009) [2023-10-08 08:15:14,177][53885] Updated weights for policy 1, policy_version 10560 (0.0010) [2023-10-08 08:15:15,180][53852] Updated weights for policy 0, policy_version 10600 (0.0007) [2023-10-08 08:15:15,547][53852] Updated weights for policy 0, policy_version 10610 (0.0011) [2023-10-08 08:15:15,921][53852] Updated weights for policy 0, policy_version 10620 (0.0008) [2023-10-08 08:15:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 21692416. Throughput: 0: 1832.6, 1: 1821.6. Samples: 5429866. Policy #0 lag: (min: 25.0, avg: 37.1, max: 57.0) [2023-10-08 08:15:17,016][52710] Avg episode reward: [(0, '17.010'), (1, '18.030')] [2023-10-08 08:15:18,029][53885] Updated weights for policy 1, policy_version 10570 (0.0008) [2023-10-08 08:15:18,401][53885] Updated weights for policy 1, policy_version 10580 (0.0008) [2023-10-08 08:15:18,768][53885] Updated weights for policy 1, policy_version 10590 (0.0008) [2023-10-08 08:15:19,412][53852] Updated weights for policy 0, policy_version 10630 (0.0008) [2023-10-08 08:15:19,781][53852] Updated weights for policy 0, policy_version 10640 (0.0007) [2023-10-08 08:15:20,159][53852] Updated weights for policy 0, policy_version 10650 (0.0009) [2023-10-08 08:15:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 21757952. Throughput: 0: 1839.7, 1: 1815.7. Samples: 5452392. Policy #0 lag: (min: 25.0, avg: 37.1, max: 57.0) [2023-10-08 08:15:22,016][52710] Avg episode reward: [(0, '17.900'), (1, '18.540')] [2023-10-08 08:15:22,495][53885] Updated weights for policy 1, policy_version 10600 (0.0008) [2023-10-08 08:15:22,870][53885] Updated weights for policy 1, policy_version 10610 (0.0009) [2023-10-08 08:15:23,241][53885] Updated weights for policy 1, policy_version 10620 (0.0008) [2023-10-08 08:15:23,658][53852] Updated weights for policy 0, policy_version 10660 (0.0007) [2023-10-08 08:15:24,031][53852] Updated weights for policy 0, policy_version 10670 (0.0008) [2023-10-08 08:15:24,409][53852] Updated weights for policy 0, policy_version 10680 (0.0011) [2023-10-08 08:15:26,905][53885] Updated weights for policy 1, policy_version 10630 (0.0008) [2023-10-08 08:15:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 21823488. Throughput: 0: 1831.5, 1: 1819.4. Samples: 5462832. Policy #0 lag: (min: 27.0, avg: 30.1, max: 59.0) [2023-10-08 08:15:27,016][52710] Avg episode reward: [(0, '18.050'), (1, '18.730')] [2023-10-08 08:15:27,276][53885] Updated weights for policy 1, policy_version 10640 (0.0008) [2023-10-08 08:15:27,642][53885] Updated weights for policy 1, policy_version 10650 (0.0010) [2023-10-08 08:15:28,147][53852] Updated weights for policy 0, policy_version 10690 (0.0010) [2023-10-08 08:15:28,508][53852] Updated weights for policy 0, policy_version 10700 (0.0007) [2023-10-08 08:15:28,885][53852] Updated weights for policy 0, policy_version 10710 (0.0007) [2023-10-08 08:15:29,251][53852] Updated weights for policy 0, policy_version 10720 (0.0009) [2023-10-08 08:15:31,412][53885] Updated weights for policy 1, policy_version 10660 (0.0011) [2023-10-08 08:15:31,782][53885] Updated weights for policy 1, policy_version 10670 (0.0010) [2023-10-08 08:15:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 21889024. Throughput: 0: 1845.5, 1: 1816.4. Samples: 5485514. Policy #0 lag: (min: 27.0, avg: 30.1, max: 59.0) [2023-10-08 08:15:32,016][52710] Avg episode reward: [(0, '17.420'), (1, '18.690')] [2023-10-08 08:15:32,146][53885] Updated weights for policy 1, policy_version 10680 (0.0012) [2023-10-08 08:15:32,865][53852] Updated weights for policy 0, policy_version 10730 (0.0007) [2023-10-08 08:15:33,240][53852] Updated weights for policy 0, policy_version 10740 (0.0007) [2023-10-08 08:15:33,613][53852] Updated weights for policy 0, policy_version 10750 (0.0007) [2023-10-08 08:15:35,962][53885] Updated weights for policy 1, policy_version 10690 (0.0010) [2023-10-08 08:15:36,358][53885] Updated weights for policy 1, policy_version 10700 (0.0008) [2023-10-08 08:15:36,715][53885] Updated weights for policy 1, policy_version 10710 (0.0012) [2023-10-08 08:15:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 21954560. Throughput: 0: 1843.5, 1: 1815.4. Samples: 5507330. Policy #0 lag: (min: 27.0, avg: 30.1, max: 59.0) [2023-10-08 08:15:37,016][52710] Avg episode reward: [(0, '17.420'), (1, '18.460')] [2023-10-08 08:15:37,081][53885] Updated weights for policy 1, policy_version 10720 (0.0007) [2023-10-08 08:15:37,141][53852] Updated weights for policy 0, policy_version 10760 (0.0008) [2023-10-08 08:15:37,512][53852] Updated weights for policy 0, policy_version 10770 (0.0009) [2023-10-08 08:15:37,874][53852] Updated weights for policy 0, policy_version 10780 (0.0007) [2023-10-08 08:15:40,631][53885] Updated weights for policy 1, policy_version 10730 (0.0008) [2023-10-08 08:15:41,001][53885] Updated weights for policy 1, policy_version 10740 (0.0008) [2023-10-08 08:15:41,355][53885] Updated weights for policy 1, policy_version 10750 (0.0008) [2023-10-08 08:15:41,612][53852] Updated weights for policy 0, policy_version 10790 (0.0009) [2023-10-08 08:15:41,990][53852] Updated weights for policy 0, policy_version 10800 (0.0008) [2023-10-08 08:15:42,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22052864. Throughput: 0: 1841.3, 1: 1810.3. Samples: 5518266. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 08:15:42,016][52710] Avg episode reward: [(0, '18.160'), (1, '17.790')] [2023-10-08 08:15:42,359][53852] Updated weights for policy 0, policy_version 10810 (0.0008) [2023-10-08 08:15:45,079][53885] Updated weights for policy 1, policy_version 10760 (0.0008) [2023-10-08 08:15:45,447][53885] Updated weights for policy 1, policy_version 10770 (0.0008) [2023-10-08 08:15:45,813][53885] Updated weights for policy 1, policy_version 10780 (0.0008) [2023-10-08 08:15:46,136][53852] Updated weights for policy 0, policy_version 10820 (0.0009) [2023-10-08 08:15:46,499][53852] Updated weights for policy 0, policy_version 10830 (0.0007) [2023-10-08 08:15:46,866][53852] Updated weights for policy 0, policy_version 10840 (0.0007) [2023-10-08 08:15:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22118400. Throughput: 0: 1843.7, 1: 1816.5. Samples: 5540036. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 08:15:47,016][52710] Avg episode reward: [(0, '18.800'), (1, '18.430')] [2023-10-08 08:15:49,497][53885] Updated weights for policy 1, policy_version 10790 (0.0008) [2023-10-08 08:15:49,861][53885] Updated weights for policy 1, policy_version 10800 (0.0008) [2023-10-08 08:15:50,238][53885] Updated weights for policy 1, policy_version 10810 (0.0009) [2023-10-08 08:15:50,443][53852] Updated weights for policy 0, policy_version 10850 (0.0007) [2023-10-08 08:15:50,812][53852] Updated weights for policy 0, policy_version 10860 (0.0010) [2023-10-08 08:15:51,191][53852] Updated weights for policy 0, policy_version 10870 (0.0008) [2023-10-08 08:15:51,566][53852] Updated weights for policy 0, policy_version 10880 (0.0007) [2023-10-08 08:15:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22216704. Throughput: 0: 1833.1, 1: 1809.8. Samples: 5561100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 08:15:52,016][52710] Avg episode reward: [(0, '18.900'), (1, '18.320')] [2023-10-08 08:15:53,852][53885] Updated weights for policy 1, policy_version 10820 (0.0009) [2023-10-08 08:15:54,219][53885] Updated weights for policy 1, policy_version 10830 (0.0008) [2023-10-08 08:15:54,583][53885] Updated weights for policy 1, policy_version 10840 (0.0009) [2023-10-08 08:15:55,061][53852] Updated weights for policy 0, policy_version 10890 (0.0008) [2023-10-08 08:15:55,435][53852] Updated weights for policy 0, policy_version 10900 (0.0011) [2023-10-08 08:15:55,798][53852] Updated weights for policy 0, policy_version 10910 (0.0009) [2023-10-08 08:15:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 22282240. Throughput: 0: 1856.8, 1: 1815.0. Samples: 5573260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 08:15:57,016][52710] Avg episode reward: [(0, '17.140'), (1, '18.630')] [2023-10-08 08:15:58,332][53885] Updated weights for policy 1, policy_version 10850 (0.0007) [2023-10-08 08:15:58,705][53885] Updated weights for policy 1, policy_version 10860 (0.0009) [2023-10-08 08:15:59,076][53885] Updated weights for policy 1, policy_version 10870 (0.0011) [2023-10-08 08:15:59,440][53885] Updated weights for policy 1, policy_version 10880 (0.0007) [2023-10-08 08:15:59,442][53852] Updated weights for policy 0, policy_version 10920 (0.0008) [2023-10-08 08:15:59,814][53852] Updated weights for policy 0, policy_version 10930 (0.0008) [2023-10-08 08:16:00,176][53852] Updated weights for policy 0, policy_version 10940 (0.0010) [2023-10-08 08:16:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22347776. Throughput: 0: 1845.8, 1: 1807.7. Samples: 5594274. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:16:02,016][52710] Avg episode reward: [(0, '17.370'), (1, '17.990')] [2023-10-08 08:16:03,174][53885] Updated weights for policy 1, policy_version 10890 (0.0008) [2023-10-08 08:16:03,541][53885] Updated weights for policy 1, policy_version 10900 (0.0007) [2023-10-08 08:16:03,688][53852] Updated weights for policy 0, policy_version 10950 (0.0009) [2023-10-08 08:16:03,910][53885] Updated weights for policy 1, policy_version 10910 (0.0007) [2023-10-08 08:16:04,057][53852] Updated weights for policy 0, policy_version 10960 (0.0008) [2023-10-08 08:16:04,423][53852] Updated weights for policy 0, policy_version 10970 (0.0009) [2023-10-08 08:16:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 22413312. Throughput: 0: 1849.3, 1: 1812.7. Samples: 5617180. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:16:07,016][52710] Avg episode reward: [(0, '17.310'), (1, '17.080')] [2023-10-08 08:16:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000010976_11239424.pth... [2023-10-08 08:16:07,030][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000010912_11173888.pth... [2023-10-08 08:16:07,058][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000009280_9502720.pth [2023-10-08 08:16:07,073][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000009216_9437184.pth [2023-10-08 08:16:07,582][53885] Updated weights for policy 1, policy_version 10920 (0.0008) [2023-10-08 08:16:07,954][53885] Updated weights for policy 1, policy_version 10930 (0.0010) [2023-10-08 08:16:08,313][53852] Updated weights for policy 0, policy_version 10980 (0.0010) [2023-10-08 08:16:08,314][53885] Updated weights for policy 1, policy_version 10940 (0.0010) [2023-10-08 08:16:08,677][53852] Updated weights for policy 0, policy_version 10990 (0.0010) [2023-10-08 08:16:09,057][53852] Updated weights for policy 0, policy_version 11000 (0.0010) [2023-10-08 08:16:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22478848. Throughput: 0: 1839.4, 1: 1810.8. Samples: 5627094. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:16:12,016][52710] Avg episode reward: [(0, '16.810'), (1, '17.580')] [2023-10-08 08:16:12,146][53885] Updated weights for policy 1, policy_version 10950 (0.0009) [2023-10-08 08:16:12,513][53885] Updated weights for policy 1, policy_version 10960 (0.0009) [2023-10-08 08:16:12,661][53852] Updated weights for policy 0, policy_version 11010 (0.0007) [2023-10-08 08:16:12,883][53885] Updated weights for policy 1, policy_version 10970 (0.0007) [2023-10-08 08:16:13,028][53852] Updated weights for policy 0, policy_version 11020 (0.0007) [2023-10-08 08:16:13,393][53852] Updated weights for policy 0, policy_version 11030 (0.0007) [2023-10-08 08:16:13,768][53852] Updated weights for policy 0, policy_version 11040 (0.0008) [2023-10-08 08:16:16,658][53885] Updated weights for policy 1, policy_version 10980 (0.0007) [2023-10-08 08:16:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22544384. Throughput: 0: 1836.8, 1: 1813.9. Samples: 5649794. Policy #0 lag: (min: 26.0, avg: 35.5, max: 58.0) [2023-10-08 08:16:17,016][52710] Avg episode reward: [(0, '17.290'), (1, '16.850')] [2023-10-08 08:16:17,022][53885] Updated weights for policy 1, policy_version 10990 (0.0007) [2023-10-08 08:16:17,395][53885] Updated weights for policy 1, policy_version 11000 (0.0007) [2023-10-08 08:16:17,478][53852] Updated weights for policy 0, policy_version 11050 (0.0008) [2023-10-08 08:16:17,843][53852] Updated weights for policy 0, policy_version 11060 (0.0008) [2023-10-08 08:16:18,216][53852] Updated weights for policy 0, policy_version 11070 (0.0009) [2023-10-08 08:16:20,979][53885] Updated weights for policy 1, policy_version 11010 (0.0007) [2023-10-08 08:16:21,380][53885] Updated weights for policy 1, policy_version 11020 (0.0007) [2023-10-08 08:16:21,747][53885] Updated weights for policy 1, policy_version 11030 (0.0007) [2023-10-08 08:16:21,995][53852] Updated weights for policy 0, policy_version 11080 (0.0010) [2023-10-08 08:16:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 22609920. Throughput: 0: 1829.0, 1: 1828.8. Samples: 5671930. Policy #0 lag: (min: 26.0, avg: 35.5, max: 58.0) [2023-10-08 08:16:22,016][52710] Avg episode reward: [(0, '18.380'), (1, '18.450')] [2023-10-08 08:16:22,105][53885] Updated weights for policy 1, policy_version 11040 (0.0008) [2023-10-08 08:16:22,363][53852] Updated weights for policy 0, policy_version 11090 (0.0008) [2023-10-08 08:16:22,744][53852] Updated weights for policy 0, policy_version 11100 (0.0008) [2023-10-08 08:16:25,735][53885] Updated weights for policy 1, policy_version 11050 (0.0008) [2023-10-08 08:16:26,108][53885] Updated weights for policy 1, policy_version 11060 (0.0008) [2023-10-08 08:16:26,414][53852] Updated weights for policy 0, policy_version 11110 (0.0007) [2023-10-08 08:16:26,468][53885] Updated weights for policy 1, policy_version 11070 (0.0007) [2023-10-08 08:16:26,792][53852] Updated weights for policy 0, policy_version 11120 (0.0007) [2023-10-08 08:16:27,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22708224. Throughput: 0: 1831.4, 1: 1825.3. Samples: 5682818. Policy #0 lag: (min: 26.0, avg: 35.5, max: 58.0) [2023-10-08 08:16:27,015][52710] Avg episode reward: [(0, '18.690'), (1, '19.140')] [2023-10-08 08:16:27,162][53852] Updated weights for policy 0, policy_version 11130 (0.0008) [2023-10-08 08:16:30,096][53885] Updated weights for policy 1, policy_version 11080 (0.0010) [2023-10-08 08:16:30,458][53885] Updated weights for policy 1, policy_version 11090 (0.0009) [2023-10-08 08:16:30,827][53885] Updated weights for policy 1, policy_version 11100 (0.0008) [2023-10-08 08:16:30,831][53852] Updated weights for policy 0, policy_version 11140 (0.0008) [2023-10-08 08:16:31,198][53852] Updated weights for policy 0, policy_version 11150 (0.0009) [2023-10-08 08:16:31,570][53852] Updated weights for policy 0, policy_version 11160 (0.0012) [2023-10-08 08:16:32,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 22806528. Throughput: 0: 1831.5, 1: 1830.6. Samples: 5704830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:16:32,016][52710] Avg episode reward: [(0, '18.950'), (1, '19.080')] [2023-10-08 08:16:34,432][53885] Updated weights for policy 1, policy_version 11110 (0.0011) [2023-10-08 08:16:34,807][53885] Updated weights for policy 1, policy_version 11120 (0.0008) [2023-10-08 08:16:35,082][53852] Updated weights for policy 0, policy_version 11170 (0.0010) [2023-10-08 08:16:35,174][53885] Updated weights for policy 1, policy_version 11130 (0.0007) [2023-10-08 08:16:35,452][53852] Updated weights for policy 0, policy_version 11180 (0.0008) [2023-10-08 08:16:35,811][53852] Updated weights for policy 0, policy_version 11190 (0.0011) [2023-10-08 08:16:36,182][53852] Updated weights for policy 0, policy_version 11200 (0.0009) [2023-10-08 08:16:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 22872064. Throughput: 0: 1832.2, 1: 1830.8. Samples: 5725936. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:16:37,016][52710] Avg episode reward: [(0, '18.510'), (1, '19.060')] [2023-10-08 08:16:38,863][53885] Updated weights for policy 1, policy_version 11140 (0.0008) [2023-10-08 08:16:39,243][53885] Updated weights for policy 1, policy_version 11150 (0.0010) [2023-10-08 08:16:39,613][53885] Updated weights for policy 1, policy_version 11160 (0.0009) [2023-10-08 08:16:39,834][53852] Updated weights for policy 0, policy_version 11210 (0.0007) [2023-10-08 08:16:40,196][53852] Updated weights for policy 0, policy_version 11220 (0.0009) [2023-10-08 08:16:40,565][53852] Updated weights for policy 0, policy_version 11230 (0.0008) [2023-10-08 08:16:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 22937600. Throughput: 0: 1822.3, 1: 1830.2. Samples: 5737624. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-08 08:16:42,016][52710] Avg episode reward: [(0, '18.660'), (1, '17.950')] [2023-10-08 08:16:43,262][53885] Updated weights for policy 1, policy_version 11170 (0.0007) [2023-10-08 08:16:43,636][53885] Updated weights for policy 1, policy_version 11180 (0.0009) [2023-10-08 08:16:44,004][53885] Updated weights for policy 1, policy_version 11190 (0.0007) [2023-10-08 08:16:44,197][53852] Updated weights for policy 0, policy_version 11240 (0.0008) [2023-10-08 08:16:44,371][53885] Updated weights for policy 1, policy_version 11200 (0.0007) [2023-10-08 08:16:44,561][53852] Updated weights for policy 0, policy_version 11250 (0.0008) [2023-10-08 08:16:44,926][53852] Updated weights for policy 0, policy_version 11260 (0.0011) [2023-10-08 08:16:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 23003136. Throughput: 0: 1822.6, 1: 1833.6. Samples: 5758804. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-08 08:16:47,016][52710] Avg episode reward: [(0, '18.050'), (1, '18.570')] [2023-10-08 08:16:48,032][53885] Updated weights for policy 1, policy_version 11210 (0.0009) [2023-10-08 08:16:48,399][53885] Updated weights for policy 1, policy_version 11220 (0.0008) [2023-10-08 08:16:48,665][53852] Updated weights for policy 0, policy_version 11270 (0.0009) [2023-10-08 08:16:48,773][53885] Updated weights for policy 1, policy_version 11230 (0.0009) [2023-10-08 08:16:49,044][53852] Updated weights for policy 0, policy_version 11280 (0.0008) [2023-10-08 08:16:49,410][53852] Updated weights for policy 0, policy_version 11290 (0.0009) [2023-10-08 08:16:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23068672. Throughput: 0: 1821.4, 1: 1833.8. Samples: 5781666. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) [2023-10-08 08:16:52,016][52710] Avg episode reward: [(0, '17.360'), (1, '17.570')] [2023-10-08 08:16:52,371][53885] Updated weights for policy 1, policy_version 11240 (0.0007) [2023-10-08 08:16:52,736][53885] Updated weights for policy 1, policy_version 11250 (0.0008) [2023-10-08 08:16:53,005][53852] Updated weights for policy 0, policy_version 11300 (0.0008) [2023-10-08 08:16:53,107][53885] Updated weights for policy 1, policy_version 11260 (0.0008) [2023-10-08 08:16:53,380][53852] Updated weights for policy 0, policy_version 11310 (0.0009) [2023-10-08 08:16:53,747][53852] Updated weights for policy 0, policy_version 11320 (0.0007) [2023-10-08 08:16:56,797][53885] Updated weights for policy 1, policy_version 11270 (0.0008) [2023-10-08 08:16:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23134208. Throughput: 0: 1825.4, 1: 1834.4. Samples: 5791782. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-08 08:16:57,016][52710] Avg episode reward: [(0, '17.930'), (1, '17.990')] [2023-10-08 08:16:57,156][53885] Updated weights for policy 1, policy_version 11280 (0.0008) [2023-10-08 08:16:57,521][53852] Updated weights for policy 0, policy_version 11330 (0.0008) [2023-10-08 08:16:57,522][53885] Updated weights for policy 1, policy_version 11290 (0.0009) [2023-10-08 08:16:57,894][53852] Updated weights for policy 0, policy_version 11340 (0.0007) [2023-10-08 08:16:58,254][53852] Updated weights for policy 0, policy_version 11350 (0.0011) [2023-10-08 08:16:58,621][53852] Updated weights for policy 0, policy_version 11360 (0.0011) [2023-10-08 08:17:01,144][53885] Updated weights for policy 1, policy_version 11300 (0.0008) [2023-10-08 08:17:01,520][53885] Updated weights for policy 1, policy_version 11310 (0.0009) [2023-10-08 08:17:01,886][53885] Updated weights for policy 1, policy_version 11320 (0.0009) [2023-10-08 08:17:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23199744. Throughput: 0: 1825.1, 1: 1834.6. Samples: 5814482. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-08 08:17:02,016][52710] Avg episode reward: [(0, '16.110'), (1, '18.450')] [2023-10-08 08:17:02,397][53852] Updated weights for policy 0, policy_version 11370 (0.0007) [2023-10-08 08:17:02,766][53852] Updated weights for policy 0, policy_version 11380 (0.0007) [2023-10-08 08:17:03,135][53852] Updated weights for policy 0, policy_version 11390 (0.0007) [2023-10-08 08:17:05,575][53885] Updated weights for policy 1, policy_version 11330 (0.0008) [2023-10-08 08:17:05,941][53885] Updated weights for policy 1, policy_version 11340 (0.0008) [2023-10-08 08:17:06,315][53885] Updated weights for policy 1, policy_version 11350 (0.0009) [2023-10-08 08:17:06,634][53852] Updated weights for policy 0, policy_version 11400 (0.0008) [2023-10-08 08:17:06,683][53885] Updated weights for policy 1, policy_version 11360 (0.0009) [2023-10-08 08:17:07,003][53852] Updated weights for policy 0, policy_version 11410 (0.0008) [2023-10-08 08:17:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 23298048. Throughput: 0: 1832.6, 1: 1819.0. Samples: 5836252. Policy #0 lag: (min: 31.0, avg: 40.9, max: 63.0) [2023-10-08 08:17:07,015][52710] Avg episode reward: [(0, '19.530'), (1, '19.440')] [2023-10-08 08:17:07,026][53594] Saving new best policy, reward=19.440! [2023-10-08 08:17:07,383][53852] Updated weights for policy 0, policy_version 11420 (0.0008) [2023-10-08 08:17:07,527][53500] Saving new best policy, reward=19.530! [2023-10-08 08:17:10,456][53885] Updated weights for policy 1, policy_version 11370 (0.0007) [2023-10-08 08:17:10,819][53885] Updated weights for policy 1, policy_version 11380 (0.0009) [2023-10-08 08:17:10,964][53852] Updated weights for policy 0, policy_version 11430 (0.0007) [2023-10-08 08:17:11,193][53885] Updated weights for policy 1, policy_version 11390 (0.0008) [2023-10-08 08:17:11,338][53852] Updated weights for policy 0, policy_version 11440 (0.0008) [2023-10-08 08:17:11,703][53852] Updated weights for policy 0, policy_version 11450 (0.0010) [2023-10-08 08:17:12,015][52710] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 23396352. Throughput: 0: 1837.9, 1: 1826.3. Samples: 5847710. Policy #0 lag: (min: 31.0, avg: 52.6, max: 56.0) [2023-10-08 08:17:12,016][52710] Avg episode reward: [(0, '19.620'), (1, '18.490')] [2023-10-08 08:17:12,018][53500] Saving new best policy, reward=19.620! [2023-10-08 08:17:14,736][53885] Updated weights for policy 1, policy_version 11400 (0.0007) [2023-10-08 08:17:15,097][53885] Updated weights for policy 1, policy_version 11410 (0.0008) [2023-10-08 08:17:15,461][53885] Updated weights for policy 1, policy_version 11420 (0.0009) [2023-10-08 08:17:15,476][53852] Updated weights for policy 0, policy_version 11460 (0.0010) [2023-10-08 08:17:15,858][53852] Updated weights for policy 0, policy_version 11470 (0.0007) [2023-10-08 08:17:16,235][53852] Updated weights for policy 0, policy_version 11480 (0.0008) [2023-10-08 08:17:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 23461888. Throughput: 0: 1827.7, 1: 1818.0. Samples: 5868890. Policy #0 lag: (min: 31.0, avg: 52.6, max: 56.0) [2023-10-08 08:17:17,016][52710] Avg episode reward: [(0, '19.110'), (1, '19.180')] [2023-10-08 08:17:19,061][53885] Updated weights for policy 1, policy_version 11430 (0.0008) [2023-10-08 08:17:19,432][53885] Updated weights for policy 1, policy_version 11440 (0.0009) [2023-10-08 08:17:19,795][53885] Updated weights for policy 1, policy_version 11450 (0.0009) [2023-10-08 08:17:19,812][53852] Updated weights for policy 0, policy_version 11490 (0.0009) [2023-10-08 08:17:20,195][53852] Updated weights for policy 0, policy_version 11500 (0.0009) [2023-10-08 08:17:20,559][53852] Updated weights for policy 0, policy_version 11510 (0.0009) [2023-10-08 08:17:20,930][53852] Updated weights for policy 0, policy_version 11520 (0.0009) [2023-10-08 08:17:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 23527424. Throughput: 0: 1832.4, 1: 1827.5. Samples: 5890632. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 08:17:22,016][52710] Avg episode reward: [(0, '19.120'), (1, '19.450')] [2023-10-08 08:17:22,028][53594] Saving new best policy, reward=19.450! [2023-10-08 08:17:23,586][53885] Updated weights for policy 1, policy_version 11460 (0.0010) [2023-10-08 08:17:23,955][53885] Updated weights for policy 1, policy_version 11470 (0.0011) [2023-10-08 08:17:24,326][53885] Updated weights for policy 1, policy_version 11480 (0.0010) [2023-10-08 08:17:24,745][53852] Updated weights for policy 0, policy_version 11530 (0.0007) [2023-10-08 08:17:25,106][53852] Updated weights for policy 0, policy_version 11540 (0.0007) [2023-10-08 08:17:25,484][53852] Updated weights for policy 0, policy_version 11550 (0.0009) [2023-10-08 08:17:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 23592960. Throughput: 0: 1828.4, 1: 1822.2. Samples: 5901902. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 08:17:27,016][52710] Avg episode reward: [(0, '18.680'), (1, '20.280')] [2023-10-08 08:17:27,016][53594] Saving new best policy, reward=20.280! [2023-10-08 08:17:27,891][53885] Updated weights for policy 1, policy_version 11490 (0.0009) [2023-10-08 08:17:28,254][53885] Updated weights for policy 1, policy_version 11500 (0.0009) [2023-10-08 08:17:28,624][53885] Updated weights for policy 1, policy_version 11510 (0.0009) [2023-10-08 08:17:29,000][53885] Updated weights for policy 1, policy_version 11520 (0.0010) [2023-10-08 08:17:29,331][53852] Updated weights for policy 0, policy_version 11560 (0.0008) [2023-10-08 08:17:29,709][53852] Updated weights for policy 0, policy_version 11570 (0.0010) [2023-10-08 08:17:30,070][53852] Updated weights for policy 0, policy_version 11580 (0.0010) [2023-10-08 08:17:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 23658496. Throughput: 0: 1817.8, 1: 1833.9. Samples: 5923130. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) [2023-10-08 08:17:32,016][52710] Avg episode reward: [(0, '17.860'), (1, '20.360')] [2023-10-08 08:17:32,018][53594] Saving new best policy, reward=20.360! [2023-10-08 08:17:32,735][53885] Updated weights for policy 1, policy_version 11530 (0.0007) [2023-10-08 08:17:33,103][53885] Updated weights for policy 1, policy_version 11540 (0.0010) [2023-10-08 08:17:33,481][53885] Updated weights for policy 1, policy_version 11550 (0.0008) [2023-10-08 08:17:33,774][53852] Updated weights for policy 0, policy_version 11590 (0.0009) [2023-10-08 08:17:34,136][53852] Updated weights for policy 0, policy_version 11600 (0.0007) [2023-10-08 08:17:34,511][53852] Updated weights for policy 0, policy_version 11610 (0.0011) [2023-10-08 08:17:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23724032. Throughput: 0: 1827.1, 1: 1824.1. Samples: 5945972. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) [2023-10-08 08:17:37,016][52710] Avg episode reward: [(0, '18.040'), (1, '19.890')] [2023-10-08 08:17:37,247][53885] Updated weights for policy 1, policy_version 11560 (0.0007) [2023-10-08 08:17:37,625][53885] Updated weights for policy 1, policy_version 11570 (0.0008) [2023-10-08 08:17:37,988][53885] Updated weights for policy 1, policy_version 11580 (0.0009) [2023-10-08 08:17:38,045][53852] Updated weights for policy 0, policy_version 11620 (0.0009) [2023-10-08 08:17:38,419][53852] Updated weights for policy 0, policy_version 11630 (0.0007) [2023-10-08 08:17:38,785][53852] Updated weights for policy 0, policy_version 11640 (0.0008) [2023-10-08 08:17:41,656][53885] Updated weights for policy 1, policy_version 11590 (0.0010) [2023-10-08 08:17:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23789568. Throughput: 0: 1821.3, 1: 1822.2. Samples: 5955738. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) [2023-10-08 08:17:42,016][52710] Avg episode reward: [(0, '18.080'), (1, '19.930')] [2023-10-08 08:17:42,025][53885] Updated weights for policy 1, policy_version 11600 (0.0009) [2023-10-08 08:17:42,395][53885] Updated weights for policy 1, policy_version 11610 (0.0007) [2023-10-08 08:17:42,454][53852] Updated weights for policy 0, policy_version 11650 (0.0009) [2023-10-08 08:17:42,822][53852] Updated weights for policy 0, policy_version 11660 (0.0007) [2023-10-08 08:17:43,188][53852] Updated weights for policy 0, policy_version 11670 (0.0007) [2023-10-08 08:17:43,564][53852] Updated weights for policy 0, policy_version 11680 (0.0007) [2023-10-08 08:17:46,123][53885] Updated weights for policy 1, policy_version 11620 (0.0008) [2023-10-08 08:17:46,488][53885] Updated weights for policy 1, policy_version 11630 (0.0009) [2023-10-08 08:17:46,851][53885] Updated weights for policy 1, policy_version 11640 (0.0009) [2023-10-08 08:17:47,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 23855104. Throughput: 0: 1830.9, 1: 1823.9. Samples: 5978948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:17:47,016][52710] Avg episode reward: [(0, '18.870'), (1, '19.560')] [2023-10-08 08:17:47,037][53852] Updated weights for policy 0, policy_version 11690 (0.0009) [2023-10-08 08:17:47,405][53852] Updated weights for policy 0, policy_version 11700 (0.0010) [2023-10-08 08:17:47,777][53852] Updated weights for policy 0, policy_version 11710 (0.0010) [2023-10-08 08:17:50,434][53885] Updated weights for policy 1, policy_version 11650 (0.0007) [2023-10-08 08:17:50,796][53885] Updated weights for policy 1, policy_version 11660 (0.0007) [2023-10-08 08:17:51,174][53885] Updated weights for policy 1, policy_version 11670 (0.0008) [2023-10-08 08:17:51,493][53852] Updated weights for policy 0, policy_version 11720 (0.0007) [2023-10-08 08:17:51,541][53885] Updated weights for policy 1, policy_version 11680 (0.0009) [2023-10-08 08:17:51,863][53852] Updated weights for policy 0, policy_version 11730 (0.0008) [2023-10-08 08:17:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 23953408. Throughput: 0: 1819.1, 1: 1820.1. Samples: 6000016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:17:52,016][52710] Avg episode reward: [(0, '18.150'), (1, '20.430')] [2023-10-08 08:17:52,026][53594] Saving new best policy, reward=20.430! [2023-10-08 08:17:52,239][53852] Updated weights for policy 0, policy_version 11740 (0.0008) [2023-10-08 08:17:55,165][53885] Updated weights for policy 1, policy_version 11690 (0.0007) [2023-10-08 08:17:55,525][53885] Updated weights for policy 1, policy_version 11700 (0.0008) [2023-10-08 08:17:55,900][53885] Updated weights for policy 1, policy_version 11710 (0.0009) [2023-10-08 08:17:55,915][53852] Updated weights for policy 0, policy_version 11750 (0.0008) [2023-10-08 08:17:56,285][53852] Updated weights for policy 0, policy_version 11760 (0.0008) [2023-10-08 08:17:56,665][53852] Updated weights for policy 0, policy_version 11770 (0.0008) [2023-10-08 08:17:57,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 24051712. Throughput: 0: 1820.2, 1: 1829.4. Samples: 6011942. Policy #0 lag: (min: 9.0, avg: 20.5, max: 41.0) [2023-10-08 08:17:57,015][52710] Avg episode reward: [(0, '17.630'), (1, '20.020')] [2023-10-08 08:17:59,612][53885] Updated weights for policy 1, policy_version 11720 (0.0007) [2023-10-08 08:17:59,973][53885] Updated weights for policy 1, policy_version 11730 (0.0007) [2023-10-08 08:18:00,283][53852] Updated weights for policy 0, policy_version 11780 (0.0008) [2023-10-08 08:18:00,339][53885] Updated weights for policy 1, policy_version 11740 (0.0007) [2023-10-08 08:18:00,680][53852] Updated weights for policy 0, policy_version 11790 (0.0007) [2023-10-08 08:18:01,057][53852] Updated weights for policy 0, policy_version 11800 (0.0008) [2023-10-08 08:18:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 24117248. Throughput: 0: 1816.9, 1: 1817.5. Samples: 6032440. Policy #0 lag: (min: 9.0, avg: 20.5, max: 41.0) [2023-10-08 08:18:02,016][52710] Avg episode reward: [(0, '17.040'), (1, '19.950')] [2023-10-08 08:18:03,888][53885] Updated weights for policy 1, policy_version 11750 (0.0009) [2023-10-08 08:18:04,257][53885] Updated weights for policy 1, policy_version 11760 (0.0008) [2023-10-08 08:18:04,622][53852] Updated weights for policy 0, policy_version 11810 (0.0008) [2023-10-08 08:18:04,634][53885] Updated weights for policy 1, policy_version 11770 (0.0009) [2023-10-08 08:18:05,005][53852] Updated weights for policy 0, policy_version 11820 (0.0010) [2023-10-08 08:18:05,365][53852] Updated weights for policy 0, policy_version 11830 (0.0010) [2023-10-08 08:18:05,735][53852] Updated weights for policy 0, policy_version 11840 (0.0009) [2023-10-08 08:18:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24182784. Throughput: 0: 1822.9, 1: 1828.4. Samples: 6054940. Policy #0 lag: (min: 9.0, avg: 20.5, max: 41.0) [2023-10-08 08:18:07,015][52710] Avg episode reward: [(0, '18.310'), (1, '19.410')] [2023-10-08 08:18:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000011776_12058624.pth... [2023-10-08 08:18:07,023][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000011840_12124160.pth... [2023-10-08 08:18:07,059][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000010080_10321920.pth [2023-10-08 08:18:07,066][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000010112_10354688.pth [2023-10-08 08:18:08,322][53885] Updated weights for policy 1, policy_version 11780 (0.0008) [2023-10-08 08:18:08,685][53885] Updated weights for policy 1, policy_version 11790 (0.0008) [2023-10-08 08:18:09,050][53885] Updated weights for policy 1, policy_version 11800 (0.0010) [2023-10-08 08:18:09,557][53852] Updated weights for policy 0, policy_version 11850 (0.0008) [2023-10-08 08:18:09,935][53852] Updated weights for policy 0, policy_version 11860 (0.0009) [2023-10-08 08:18:10,301][53852] Updated weights for policy 0, policy_version 11870 (0.0008) [2023-10-08 08:18:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 24248320. Throughput: 0: 1817.2, 1: 1824.8. Samples: 6065790. Policy #0 lag: (min: 26.0, avg: 31.2, max: 58.0) [2023-10-08 08:18:12,016][52710] Avg episode reward: [(0, '17.660'), (1, '20.250')] [2023-10-08 08:18:12,845][53885] Updated weights for policy 1, policy_version 11810 (0.0008) [2023-10-08 08:18:13,222][53885] Updated weights for policy 1, policy_version 11820 (0.0010) [2023-10-08 08:18:13,593][53885] Updated weights for policy 1, policy_version 11830 (0.0010) [2023-10-08 08:18:13,952][53885] Updated weights for policy 1, policy_version 11840 (0.0007) [2023-10-08 08:18:13,980][53852] Updated weights for policy 0, policy_version 11880 (0.0009) [2023-10-08 08:18:14,339][53852] Updated weights for policy 0, policy_version 11890 (0.0009) [2023-10-08 08:18:14,717][53852] Updated weights for policy 0, policy_version 11900 (0.0009) [2023-10-08 08:18:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 24313856. Throughput: 0: 1832.4, 1: 1820.2. Samples: 6087496. Policy #0 lag: (min: 26.0, avg: 31.2, max: 58.0) [2023-10-08 08:18:17,016][52710] Avg episode reward: [(0, '18.960'), (1, '18.980')] [2023-10-08 08:18:17,719][53885] Updated weights for policy 1, policy_version 11850 (0.0008) [2023-10-08 08:18:18,091][53885] Updated weights for policy 1, policy_version 11860 (0.0008) [2023-10-08 08:18:18,317][53852] Updated weights for policy 0, policy_version 11910 (0.0008) [2023-10-08 08:18:18,456][53885] Updated weights for policy 1, policy_version 11870 (0.0009) [2023-10-08 08:18:18,691][53852] Updated weights for policy 0, policy_version 11920 (0.0007) [2023-10-08 08:18:19,053][53852] Updated weights for policy 0, policy_version 11930 (0.0007) [2023-10-08 08:18:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 24379392. Throughput: 0: 1828.0, 1: 1825.9. Samples: 6110394. Policy #0 lag: (min: 26.0, avg: 31.2, max: 58.0) [2023-10-08 08:18:22,015][52710] Avg episode reward: [(0, '18.400'), (1, '18.240')] [2023-10-08 08:18:22,144][53885] Updated weights for policy 1, policy_version 11880 (0.0010) [2023-10-08 08:18:22,506][53885] Updated weights for policy 1, policy_version 11890 (0.0007) [2023-10-08 08:18:22,740][53852] Updated weights for policy 0, policy_version 11940 (0.0009) [2023-10-08 08:18:22,877][53885] Updated weights for policy 1, policy_version 11900 (0.0008) [2023-10-08 08:18:23,112][53852] Updated weights for policy 0, policy_version 11950 (0.0007) [2023-10-08 08:18:23,489][53852] Updated weights for policy 0, policy_version 11960 (0.0008) [2023-10-08 08:18:26,474][53885] Updated weights for policy 1, policy_version 11910 (0.0008) [2023-10-08 08:18:26,850][53885] Updated weights for policy 1, policy_version 11920 (0.0007) [2023-10-08 08:18:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 24444928. Throughput: 0: 1830.1, 1: 1829.7. Samples: 6120430. Policy #0 lag: (min: 27.0, avg: 32.8, max: 59.0) [2023-10-08 08:18:27,015][52710] Avg episode reward: [(0, '20.220'), (1, '17.590')] [2023-10-08 08:18:27,016][53500] Saving new best policy, reward=20.220! [2023-10-08 08:18:27,216][53885] Updated weights for policy 1, policy_version 11930 (0.0007) [2023-10-08 08:18:27,253][53852] Updated weights for policy 0, policy_version 11970 (0.0009) [2023-10-08 08:18:27,624][53852] Updated weights for policy 0, policy_version 11980 (0.0008) [2023-10-08 08:18:27,998][53852] Updated weights for policy 0, policy_version 11990 (0.0007) [2023-10-08 08:18:28,365][53852] Updated weights for policy 0, policy_version 12000 (0.0008) [2023-10-08 08:18:30,800][53885] Updated weights for policy 1, policy_version 11940 (0.0008) [2023-10-08 08:18:31,166][53885] Updated weights for policy 1, policy_version 11950 (0.0008) [2023-10-08 08:18:31,539][53885] Updated weights for policy 1, policy_version 11960 (0.0008) [2023-10-08 08:18:32,015][52710] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 24543232. Throughput: 0: 1822.5, 1: 1834.4. Samples: 6143508. Policy #0 lag: (min: 27.0, avg: 32.8, max: 59.0) [2023-10-08 08:18:32,017][52710] Avg episode reward: [(0, '19.980'), (1, '18.430')] [2023-10-08 08:18:32,038][53852] Updated weights for policy 0, policy_version 12010 (0.0009) [2023-10-08 08:18:32,413][53852] Updated weights for policy 0, policy_version 12020 (0.0010) [2023-10-08 08:18:32,789][53852] Updated weights for policy 0, policy_version 12030 (0.0008) [2023-10-08 08:18:34,993][53885] Updated weights for policy 1, policy_version 11970 (0.0009) [2023-10-08 08:18:35,372][53885] Updated weights for policy 1, policy_version 11980 (0.0010) [2023-10-08 08:18:35,736][53885] Updated weights for policy 1, policy_version 11990 (0.0009) [2023-10-08 08:18:36,100][53885] Updated weights for policy 1, policy_version 12000 (0.0010) [2023-10-08 08:18:36,452][53852] Updated weights for policy 0, policy_version 12040 (0.0008) [2023-10-08 08:18:36,837][53852] Updated weights for policy 0, policy_version 12050 (0.0007) [2023-10-08 08:18:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 24608768. Throughput: 0: 1824.6, 1: 1837.4. Samples: 6164804. Policy #0 lag: (min: 27.0, avg: 32.8, max: 59.0) [2023-10-08 08:18:37,016][52710] Avg episode reward: [(0, '19.610'), (1, '18.760')] [2023-10-08 08:18:37,208][53852] Updated weights for policy 0, policy_version 12060 (0.0008) [2023-10-08 08:18:39,753][53885] Updated weights for policy 1, policy_version 12010 (0.0010) [2023-10-08 08:18:40,118][53885] Updated weights for policy 1, policy_version 12020 (0.0009) [2023-10-08 08:18:40,482][53885] Updated weights for policy 1, policy_version 12030 (0.0007) [2023-10-08 08:18:40,785][53852] Updated weights for policy 0, policy_version 12070 (0.0008) [2023-10-08 08:18:41,163][53852] Updated weights for policy 0, policy_version 12080 (0.0010) [2023-10-08 08:18:41,533][53852] Updated weights for policy 0, policy_version 12090 (0.0008) [2023-10-08 08:18:42,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 24707072. Throughput: 0: 1828.9, 1: 1829.1. Samples: 6176550. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:18:42,016][52710] Avg episode reward: [(0, '18.520'), (1, '20.170')] [2023-10-08 08:18:44,083][53885] Updated weights for policy 1, policy_version 12040 (0.0008) [2023-10-08 08:18:44,452][53885] Updated weights for policy 1, policy_version 12050 (0.0009) [2023-10-08 08:18:44,820][53885] Updated weights for policy 1, policy_version 12060 (0.0008) [2023-10-08 08:18:45,288][53852] Updated weights for policy 0, policy_version 12100 (0.0007) [2023-10-08 08:18:45,660][53852] Updated weights for policy 0, policy_version 12110 (0.0009) [2023-10-08 08:18:46,030][53852] Updated weights for policy 0, policy_version 12120 (0.0008) [2023-10-08 08:18:47,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 24772608. Throughput: 0: 1830.5, 1: 1848.6. Samples: 6198002. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:18:47,016][52710] Avg episode reward: [(0, '18.830'), (1, '20.250')] [2023-10-08 08:18:48,595][53885] Updated weights for policy 1, policy_version 12070 (0.0009) [2023-10-08 08:18:48,965][53885] Updated weights for policy 1, policy_version 12080 (0.0007) [2023-10-08 08:18:49,329][53885] Updated weights for policy 1, policy_version 12090 (0.0009) [2023-10-08 08:18:49,631][53852] Updated weights for policy 0, policy_version 12130 (0.0008) [2023-10-08 08:18:50,019][53852] Updated weights for policy 0, policy_version 12140 (0.0007) [2023-10-08 08:18:50,387][53852] Updated weights for policy 0, policy_version 12150 (0.0010) [2023-10-08 08:18:50,760][53852] Updated weights for policy 0, policy_version 12160 (0.0009) [2023-10-08 08:18:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 24838144. Throughput: 0: 1828.4, 1: 1839.1. Samples: 6219976. Policy #0 lag: (min: 22.0, avg: 23.5, max: 48.0) [2023-10-08 08:18:52,016][52710] Avg episode reward: [(0, '18.150'), (1, '20.510')] [2023-10-08 08:18:52,024][53594] Saving new best policy, reward=20.510! [2023-10-08 08:18:52,850][53885] Updated weights for policy 1, policy_version 12100 (0.0008) [2023-10-08 08:18:53,215][53885] Updated weights for policy 1, policy_version 12110 (0.0008) [2023-10-08 08:18:53,585][53885] Updated weights for policy 1, policy_version 12120 (0.0008) [2023-10-08 08:18:54,387][53852] Updated weights for policy 0, policy_version 12170 (0.0007) [2023-10-08 08:18:54,768][53852] Updated weights for policy 0, policy_version 12180 (0.0007) [2023-10-08 08:18:55,147][53852] Updated weights for policy 0, policy_version 12190 (0.0007) [2023-10-08 08:18:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 24903680. Throughput: 0: 1827.9, 1: 1838.8. Samples: 6230792. Policy #0 lag: (min: 22.0, avg: 23.5, max: 48.0) [2023-10-08 08:18:57,016][52710] Avg episode reward: [(0, '18.820'), (1, '20.410')] [2023-10-08 08:18:57,208][53885] Updated weights for policy 1, policy_version 12130 (0.0007) [2023-10-08 08:18:57,582][53885] Updated weights for policy 1, policy_version 12140 (0.0008) [2023-10-08 08:18:57,940][53885] Updated weights for policy 1, policy_version 12150 (0.0009) [2023-10-08 08:18:58,310][53885] Updated weights for policy 1, policy_version 12160 (0.0007) [2023-10-08 08:18:58,751][53852] Updated weights for policy 0, policy_version 12200 (0.0009) [2023-10-08 08:18:59,135][53852] Updated weights for policy 0, policy_version 12210 (0.0009) [2023-10-08 08:18:59,505][53852] Updated weights for policy 0, policy_version 12220 (0.0011) [2023-10-08 08:19:01,939][53885] Updated weights for policy 1, policy_version 12170 (0.0008) [2023-10-08 08:19:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 24969216. Throughput: 0: 1834.3, 1: 1845.0. Samples: 6253064. Policy #0 lag: (min: 22.0, avg: 23.5, max: 48.0) [2023-10-08 08:19:02,016][52710] Avg episode reward: [(0, '19.290'), (1, '20.830')] [2023-10-08 08:19:02,305][53885] Updated weights for policy 1, policy_version 12180 (0.0007) [2023-10-08 08:19:02,671][53885] Updated weights for policy 1, policy_version 12190 (0.0009) [2023-10-08 08:19:02,742][53594] Saving new best policy, reward=20.830! [2023-10-08 08:19:03,141][53852] Updated weights for policy 0, policy_version 12230 (0.0008) [2023-10-08 08:19:03,523][53852] Updated weights for policy 0, policy_version 12240 (0.0007) [2023-10-08 08:19:03,890][53852] Updated weights for policy 0, policy_version 12250 (0.0009) [2023-10-08 08:19:06,335][53885] Updated weights for policy 1, policy_version 12200 (0.0008) [2023-10-08 08:19:06,710][53885] Updated weights for policy 1, policy_version 12210 (0.0007) [2023-10-08 08:19:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 25034752. Throughput: 0: 1838.4, 1: 1836.6. Samples: 6275766. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 08:19:07,015][52710] Avg episode reward: [(0, '20.530'), (1, '19.500')] [2023-10-08 08:19:07,022][53500] Saving new best policy, reward=20.530! [2023-10-08 08:19:07,080][53885] Updated weights for policy 1, policy_version 12220 (0.0007) [2023-10-08 08:19:07,350][53852] Updated weights for policy 0, policy_version 12260 (0.0009) [2023-10-08 08:19:07,718][53852] Updated weights for policy 0, policy_version 12270 (0.0008) [2023-10-08 08:19:08,089][53852] Updated weights for policy 0, policy_version 12280 (0.0008) [2023-10-08 08:19:10,705][53885] Updated weights for policy 1, policy_version 12230 (0.0009) [2023-10-08 08:19:11,082][53885] Updated weights for policy 1, policy_version 12240 (0.0010) [2023-10-08 08:19:11,442][53885] Updated weights for policy 1, policy_version 12250 (0.0008) [2023-10-08 08:19:11,916][53852] Updated weights for policy 0, policy_version 12290 (0.0007) [2023-10-08 08:19:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25133056. Throughput: 0: 1841.7, 1: 1852.6. Samples: 6286672. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 08:19:12,015][52710] Avg episode reward: [(0, '19.330'), (1, '20.740')] [2023-10-08 08:19:12,282][53852] Updated weights for policy 0, policy_version 12300 (0.0009) [2023-10-08 08:19:12,658][53852] Updated weights for policy 0, policy_version 12310 (0.0007) [2023-10-08 08:19:13,034][53852] Updated weights for policy 0, policy_version 12320 (0.0007) [2023-10-08 08:19:15,057][53885] Updated weights for policy 1, policy_version 12260 (0.0008) [2023-10-08 08:19:15,425][53885] Updated weights for policy 1, policy_version 12270 (0.0009) [2023-10-08 08:19:15,796][53885] Updated weights for policy 1, policy_version 12280 (0.0009) [2023-10-08 08:19:16,696][53852] Updated weights for policy 0, policy_version 12330 (0.0009) [2023-10-08 08:19:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25198592. Throughput: 0: 1840.5, 1: 1829.8. Samples: 6308666. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 08:19:17,015][52710] Avg episode reward: [(0, '19.120'), (1, '19.150')] [2023-10-08 08:19:17,066][53852] Updated weights for policy 0, policy_version 12340 (0.0010) [2023-10-08 08:19:17,444][53852] Updated weights for policy 0, policy_version 12350 (0.0009) [2023-10-08 08:19:19,647][53885] Updated weights for policy 1, policy_version 12290 (0.0007) [2023-10-08 08:19:20,016][53885] Updated weights for policy 1, policy_version 12300 (0.0007) [2023-10-08 08:19:20,383][53885] Updated weights for policy 1, policy_version 12310 (0.0011) [2023-10-08 08:19:20,743][53885] Updated weights for policy 1, policy_version 12320 (0.0009) [2023-10-08 08:19:20,993][53852] Updated weights for policy 0, policy_version 12360 (0.0009) [2023-10-08 08:19:21,364][53852] Updated weights for policy 0, policy_version 12370 (0.0008) [2023-10-08 08:19:21,736][53852] Updated weights for policy 0, policy_version 12380 (0.0012) [2023-10-08 08:19:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 25296896. Throughput: 0: 1825.2, 1: 1842.2. Samples: 6329840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:19:22,015][52710] Avg episode reward: [(0, '18.810'), (1, '19.850')] [2023-10-08 08:19:24,357][53885] Updated weights for policy 1, policy_version 12330 (0.0009) [2023-10-08 08:19:24,729][53885] Updated weights for policy 1, policy_version 12340 (0.0007) [2023-10-08 08:19:25,094][53885] Updated weights for policy 1, policy_version 12350 (0.0007) [2023-10-08 08:19:25,423][53852] Updated weights for policy 0, policy_version 12390 (0.0009) [2023-10-08 08:19:25,802][53852] Updated weights for policy 0, policy_version 12400 (0.0009) [2023-10-08 08:19:26,174][53852] Updated weights for policy 0, policy_version 12410 (0.0007) [2023-10-08 08:19:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 25362432. Throughput: 0: 1837.7, 1: 1831.8. Samples: 6341678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:19:27,016][52710] Avg episode reward: [(0, '18.630'), (1, '20.310')] [2023-10-08 08:19:28,739][53885] Updated weights for policy 1, policy_version 12360 (0.0009) [2023-10-08 08:19:29,106][53885] Updated weights for policy 1, policy_version 12370 (0.0008) [2023-10-08 08:19:29,483][53885] Updated weights for policy 1, policy_version 12380 (0.0008) [2023-10-08 08:19:29,827][53852] Updated weights for policy 0, policy_version 12420 (0.0008) [2023-10-08 08:19:30,199][53852] Updated weights for policy 0, policy_version 12430 (0.0009) [2023-10-08 08:19:30,568][53852] Updated weights for policy 0, policy_version 12440 (0.0009) [2023-10-08 08:19:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 25427968. Throughput: 0: 1827.7, 1: 1839.7. Samples: 6363036. Policy #0 lag: (min: 8.0, avg: 30.9, max: 40.0) [2023-10-08 08:19:32,016][52710] Avg episode reward: [(0, '18.040'), (1, '20.450')] [2023-10-08 08:19:33,187][53885] Updated weights for policy 1, policy_version 12390 (0.0007) [2023-10-08 08:19:33,559][53885] Updated weights for policy 1, policy_version 12400 (0.0007) [2023-10-08 08:19:33,923][53885] Updated weights for policy 1, policy_version 12410 (0.0008) [2023-10-08 08:19:34,162][53852] Updated weights for policy 0, policy_version 12450 (0.0009) [2023-10-08 08:19:34,558][53852] Updated weights for policy 0, policy_version 12460 (0.0007) [2023-10-08 08:19:34,931][53852] Updated weights for policy 0, policy_version 12470 (0.0009) [2023-10-08 08:19:35,299][53852] Updated weights for policy 0, policy_version 12480 (0.0008) [2023-10-08 08:19:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25493504. Throughput: 0: 1844.1, 1: 1840.8. Samples: 6385796. Policy #0 lag: (min: 8.0, avg: 30.9, max: 40.0) [2023-10-08 08:19:37,016][52710] Avg episode reward: [(0, '17.780'), (1, '20.050')] [2023-10-08 08:19:37,516][53885] Updated weights for policy 1, policy_version 12420 (0.0010) [2023-10-08 08:19:37,887][53885] Updated weights for policy 1, policy_version 12430 (0.0008) [2023-10-08 08:19:38,254][53885] Updated weights for policy 1, policy_version 12440 (0.0009) [2023-10-08 08:19:38,796][53852] Updated weights for policy 0, policy_version 12490 (0.0007) [2023-10-08 08:19:39,173][53852] Updated weights for policy 0, policy_version 12500 (0.0007) [2023-10-08 08:19:39,531][53852] Updated weights for policy 0, policy_version 12510 (0.0008) [2023-10-08 08:19:41,932][53885] Updated weights for policy 1, policy_version 12450 (0.0008) [2023-10-08 08:19:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 25559040. Throughput: 0: 1830.4, 1: 1846.9. Samples: 6396270. Policy #0 lag: (min: 8.0, avg: 30.9, max: 40.0) [2023-10-08 08:19:42,016][52710] Avg episode reward: [(0, '19.300'), (1, '20.460')] [2023-10-08 08:19:42,300][53885] Updated weights for policy 1, policy_version 12460 (0.0008) [2023-10-08 08:19:42,664][53885] Updated weights for policy 1, policy_version 12470 (0.0009) [2023-10-08 08:19:43,033][53885] Updated weights for policy 1, policy_version 12480 (0.0007) [2023-10-08 08:19:43,252][53852] Updated weights for policy 0, policy_version 12520 (0.0008) [2023-10-08 08:19:43,618][53852] Updated weights for policy 0, policy_version 12530 (0.0010) [2023-10-08 08:19:43,984][53852] Updated weights for policy 0, policy_version 12540 (0.0009) [2023-10-08 08:19:46,846][53885] Updated weights for policy 1, policy_version 12490 (0.0009) [2023-10-08 08:19:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 25624576. Throughput: 0: 1842.1, 1: 1835.3. Samples: 6418548. Policy #0 lag: (min: 17.0, avg: 23.7, max: 49.0) [2023-10-08 08:19:47,016][52710] Avg episode reward: [(0, '18.830'), (1, '20.220')] [2023-10-08 08:19:47,203][53885] Updated weights for policy 1, policy_version 12500 (0.0010) [2023-10-08 08:19:47,578][53885] Updated weights for policy 1, policy_version 12510 (0.0007) [2023-10-08 08:19:47,707][53852] Updated weights for policy 0, policy_version 12550 (0.0010) [2023-10-08 08:19:48,077][53852] Updated weights for policy 0, policy_version 12560 (0.0010) [2023-10-08 08:19:48,439][53852] Updated weights for policy 0, policy_version 12570 (0.0008) [2023-10-08 08:19:51,252][53885] Updated weights for policy 1, policy_version 12520 (0.0009) [2023-10-08 08:19:51,624][53885] Updated weights for policy 1, policy_version 12530 (0.0008) [2023-10-08 08:19:51,989][53885] Updated weights for policy 1, policy_version 12540 (0.0007) [2023-10-08 08:19:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 25690112. Throughput: 0: 1835.9, 1: 1827.5. Samples: 6440616. Policy #0 lag: (min: 17.0, avg: 23.7, max: 49.0) [2023-10-08 08:19:52,016][52710] Avg episode reward: [(0, '20.430'), (1, '19.420')] [2023-10-08 08:19:52,167][53852] Updated weights for policy 0, policy_version 12580 (0.0008) [2023-10-08 08:19:52,544][53852] Updated weights for policy 0, policy_version 12590 (0.0009) [2023-10-08 08:19:52,922][53852] Updated weights for policy 0, policy_version 12600 (0.0009) [2023-10-08 08:19:55,578][53885] Updated weights for policy 1, policy_version 12550 (0.0009) [2023-10-08 08:19:55,941][53885] Updated weights for policy 1, policy_version 12560 (0.0009) [2023-10-08 08:19:56,308][53885] Updated weights for policy 1, policy_version 12570 (0.0009) [2023-10-08 08:19:56,647][53852] Updated weights for policy 0, policy_version 12610 (0.0009) [2023-10-08 08:19:57,010][53852] Updated weights for policy 0, policy_version 12620 (0.0010) [2023-10-08 08:19:57,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25788416. Throughput: 0: 1830.7, 1: 1834.1. Samples: 6451588. Policy #0 lag: (min: 17.0, avg: 23.7, max: 49.0) [2023-10-08 08:19:57,016][52710] Avg episode reward: [(0, '19.810'), (1, '19.330')] [2023-10-08 08:19:57,383][53852] Updated weights for policy 0, policy_version 12630 (0.0008) [2023-10-08 08:19:57,762][53852] Updated weights for policy 0, policy_version 12640 (0.0009) [2023-10-08 08:19:59,983][53885] Updated weights for policy 1, policy_version 12580 (0.0009) [2023-10-08 08:20:00,351][53885] Updated weights for policy 1, policy_version 12590 (0.0008) [2023-10-08 08:20:00,723][53885] Updated weights for policy 1, policy_version 12600 (0.0009) [2023-10-08 08:20:01,345][53852] Updated weights for policy 0, policy_version 12650 (0.0007) [2023-10-08 08:20:01,715][53852] Updated weights for policy 0, policy_version 12660 (0.0007) [2023-10-08 08:20:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 25853952. Throughput: 0: 1835.9, 1: 1831.7. Samples: 6473710. Policy #0 lag: (min: 19.0, avg: 21.4, max: 51.0) [2023-10-08 08:20:02,016][52710] Avg episode reward: [(0, '18.470'), (1, '19.950')] [2023-10-08 08:20:02,089][53852] Updated weights for policy 0, policy_version 12670 (0.0008) [2023-10-08 08:20:04,232][53885] Updated weights for policy 1, policy_version 12610 (0.0008) [2023-10-08 08:20:04,602][53885] Updated weights for policy 1, policy_version 12620 (0.0007) [2023-10-08 08:20:04,965][53885] Updated weights for policy 1, policy_version 12630 (0.0007) [2023-10-08 08:20:05,336][53885] Updated weights for policy 1, policy_version 12640 (0.0009) [2023-10-08 08:20:05,729][53852] Updated weights for policy 0, policy_version 12680 (0.0009) [2023-10-08 08:20:06,100][53852] Updated weights for policy 0, policy_version 12690 (0.0007) [2023-10-08 08:20:06,464][53852] Updated weights for policy 0, policy_version 12700 (0.0007) [2023-10-08 08:20:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 25952256. Throughput: 0: 1827.3, 1: 1838.3. Samples: 6494792. Policy #0 lag: (min: 19.0, avg: 21.4, max: 51.0) [2023-10-08 08:20:07,016][52710] Avg episode reward: [(0, '18.900'), (1, '19.830')] [2023-10-08 08:20:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000012640_12943360.pth... [2023-10-08 08:20:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000012704_13008896.pth... [2023-10-08 08:20:07,059][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000010912_11173888.pth [2023-10-08 08:20:07,062][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000010976_11239424.pth [2023-10-08 08:20:08,879][53885] Updated weights for policy 1, policy_version 12650 (0.0009) [2023-10-08 08:20:09,237][53885] Updated weights for policy 1, policy_version 12660 (0.0008) [2023-10-08 08:20:09,604][53885] Updated weights for policy 1, policy_version 12670 (0.0007) [2023-10-08 08:20:10,044][53852] Updated weights for policy 0, policy_version 12710 (0.0009) [2023-10-08 08:20:10,412][53852] Updated weights for policy 0, policy_version 12720 (0.0009) [2023-10-08 08:20:10,786][53852] Updated weights for policy 0, policy_version 12730 (0.0009) [2023-10-08 08:20:12,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26017792. Throughput: 0: 1834.4, 1: 1824.9. Samples: 6506348. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:20:12,016][52710] Avg episode reward: [(0, '19.030'), (1, '20.690')] [2023-10-08 08:20:13,308][53885] Updated weights for policy 1, policy_version 12680 (0.0007) [2023-10-08 08:20:13,671][53885] Updated weights for policy 1, policy_version 12690 (0.0008) [2023-10-08 08:20:14,049][53885] Updated weights for policy 1, policy_version 12700 (0.0009) [2023-10-08 08:20:14,339][53852] Updated weights for policy 0, policy_version 12740 (0.0009) [2023-10-08 08:20:14,709][53852] Updated weights for policy 0, policy_version 12750 (0.0008) [2023-10-08 08:20:15,085][53852] Updated weights for policy 0, policy_version 12760 (0.0007) [2023-10-08 08:20:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26083328. Throughput: 0: 1822.8, 1: 1837.9. Samples: 6527766. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:20:17,016][52710] Avg episode reward: [(0, '18.960'), (1, '21.350')] [2023-10-08 08:20:17,017][53594] Saving new best policy, reward=21.350! [2023-10-08 08:20:17,748][53885] Updated weights for policy 1, policy_version 12710 (0.0010) [2023-10-08 08:20:18,110][53885] Updated weights for policy 1, policy_version 12720 (0.0010) [2023-10-08 08:20:18,474][53885] Updated weights for policy 1, policy_version 12730 (0.0009) [2023-10-08 08:20:18,760][53852] Updated weights for policy 0, policy_version 12770 (0.0008) [2023-10-08 08:20:19,129][53852] Updated weights for policy 0, policy_version 12780 (0.0010) [2023-10-08 08:20:19,497][53852] Updated weights for policy 0, policy_version 12790 (0.0008) [2023-10-08 08:20:19,869][53852] Updated weights for policy 0, policy_version 12800 (0.0007) [2023-10-08 08:20:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 26148864. Throughput: 0: 1832.5, 1: 1835.2. Samples: 6550844. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:20:22,016][52710] Avg episode reward: [(0, '18.340'), (1, '21.190')] [2023-10-08 08:20:22,175][53885] Updated weights for policy 1, policy_version 12740 (0.0008) [2023-10-08 08:20:22,564][53885] Updated weights for policy 1, policy_version 12750 (0.0009) [2023-10-08 08:20:22,941][53885] Updated weights for policy 1, policy_version 12760 (0.0008) [2023-10-08 08:20:23,557][53852] Updated weights for policy 0, policy_version 12810 (0.0008) [2023-10-08 08:20:23,945][53852] Updated weights for policy 0, policy_version 12820 (0.0007) [2023-10-08 08:20:24,311][53852] Updated weights for policy 0, policy_version 12830 (0.0008) [2023-10-08 08:20:26,639][53885] Updated weights for policy 1, policy_version 12770 (0.0009) [2023-10-08 08:20:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 26214400. Throughput: 0: 1823.9, 1: 1826.7. Samples: 6560544. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 08:20:27,016][52710] Avg episode reward: [(0, '18.970'), (1, '21.520')] [2023-10-08 08:20:27,017][53885] Updated weights for policy 1, policy_version 12780 (0.0008) [2023-10-08 08:20:27,386][53885] Updated weights for policy 1, policy_version 12790 (0.0007) [2023-10-08 08:20:27,754][53594] Saving new best policy, reward=21.520! [2023-10-08 08:20:27,757][53885] Updated weights for policy 1, policy_version 12800 (0.0007) [2023-10-08 08:20:27,850][53852] Updated weights for policy 0, policy_version 12840 (0.0009) [2023-10-08 08:20:28,219][53852] Updated weights for policy 0, policy_version 12850 (0.0007) [2023-10-08 08:20:28,594][53852] Updated weights for policy 0, policy_version 12860 (0.0008) [2023-10-08 08:20:31,434][53885] Updated weights for policy 1, policy_version 12810 (0.0008) [2023-10-08 08:20:31,801][53885] Updated weights for policy 1, policy_version 12820 (0.0009) [2023-10-08 08:20:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 26279936. Throughput: 0: 1834.3, 1: 1837.1. Samples: 6583760. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 08:20:32,016][52710] Avg episode reward: [(0, '18.210'), (1, '22.170')] [2023-10-08 08:20:32,179][53885] Updated weights for policy 1, policy_version 12830 (0.0008) [2023-10-08 08:20:32,249][53594] Saving new best policy, reward=22.170! [2023-10-08 08:20:32,251][53852] Updated weights for policy 0, policy_version 12870 (0.0008) [2023-10-08 08:20:32,630][53852] Updated weights for policy 0, policy_version 12880 (0.0008) [2023-10-08 08:20:33,013][53852] Updated weights for policy 0, policy_version 12890 (0.0007) [2023-10-08 08:20:35,952][53885] Updated weights for policy 1, policy_version 12840 (0.0009) [2023-10-08 08:20:36,315][53885] Updated weights for policy 1, policy_version 12850 (0.0009) [2023-10-08 08:20:36,565][53852] Updated weights for policy 0, policy_version 12900 (0.0008) [2023-10-08 08:20:36,687][53885] Updated weights for policy 1, policy_version 12860 (0.0008) [2023-10-08 08:20:36,939][53852] Updated weights for policy 0, policy_version 12910 (0.0008) [2023-10-08 08:20:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26378240. Throughput: 0: 1838.4, 1: 1824.9. Samples: 6605464. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 08:20:37,016][52710] Avg episode reward: [(0, '19.600'), (1, '20.380')] [2023-10-08 08:20:37,315][53852] Updated weights for policy 0, policy_version 12920 (0.0011) [2023-10-08 08:20:40,523][53885] Updated weights for policy 1, policy_version 12870 (0.0009) [2023-10-08 08:20:40,885][53885] Updated weights for policy 1, policy_version 12880 (0.0007) [2023-10-08 08:20:40,947][53852] Updated weights for policy 0, policy_version 12930 (0.0008) [2023-10-08 08:20:41,251][53885] Updated weights for policy 1, policy_version 12890 (0.0008) [2023-10-08 08:20:41,311][53852] Updated weights for policy 0, policy_version 12940 (0.0008) [2023-10-08 08:20:41,682][53852] Updated weights for policy 0, policy_version 12950 (0.0008) [2023-10-08 08:20:42,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26443776. Throughput: 0: 1843.2, 1: 1824.8. Samples: 6616652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:20:42,015][52710] Avg episode reward: [(0, '19.150'), (1, '22.210')] [2023-10-08 08:20:42,016][53594] Saving new best policy, reward=22.210! [2023-10-08 08:20:42,051][53852] Updated weights for policy 0, policy_version 12960 (0.0011) [2023-10-08 08:20:45,043][53885] Updated weights for policy 1, policy_version 12900 (0.0008) [2023-10-08 08:20:45,416][53885] Updated weights for policy 1, policy_version 12910 (0.0009) [2023-10-08 08:20:45,779][53885] Updated weights for policy 1, policy_version 12920 (0.0007) [2023-10-08 08:20:46,001][53852] Updated weights for policy 0, policy_version 12970 (0.0009) [2023-10-08 08:20:46,374][53852] Updated weights for policy 0, policy_version 12980 (0.0010) [2023-10-08 08:20:46,725][53852] Updated weights for policy 0, policy_version 12990 (0.0011) [2023-10-08 08:20:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 26542080. Throughput: 0: 1832.9, 1: 1821.1. Samples: 6638138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:20:47,016][52710] Avg episode reward: [(0, '20.270'), (1, '21.650')] [2023-10-08 08:20:49,299][53885] Updated weights for policy 1, policy_version 12930 (0.0007) [2023-10-08 08:20:49,674][53885] Updated weights for policy 1, policy_version 12940 (0.0008) [2023-10-08 08:20:50,041][53885] Updated weights for policy 1, policy_version 12950 (0.0008) [2023-10-08 08:20:50,396][53852] Updated weights for policy 0, policy_version 13000 (0.0007) [2023-10-08 08:20:50,406][53885] Updated weights for policy 1, policy_version 12960 (0.0008) [2023-10-08 08:20:50,770][53852] Updated weights for policy 0, policy_version 13010 (0.0008) [2023-10-08 08:20:51,139][53852] Updated weights for policy 0, policy_version 13020 (0.0008) [2023-10-08 08:20:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 26607616. Throughput: 0: 1827.3, 1: 1820.8. Samples: 6658958. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 08:20:52,016][52710] Avg episode reward: [(0, '19.470'), (1, '23.920')] [2023-10-08 08:20:52,029][53594] Saving new best policy, reward=23.920! [2023-10-08 08:20:54,095][53885] Updated weights for policy 1, policy_version 12970 (0.0009) [2023-10-08 08:20:54,475][53885] Updated weights for policy 1, policy_version 12980 (0.0008) [2023-10-08 08:20:54,840][53885] Updated weights for policy 1, policy_version 12990 (0.0008) [2023-10-08 08:20:54,867][53852] Updated weights for policy 0, policy_version 13030 (0.0008) [2023-10-08 08:20:55,241][53852] Updated weights for policy 0, policy_version 13040 (0.0008) [2023-10-08 08:20:55,611][53852] Updated weights for policy 0, policy_version 13050 (0.0008) [2023-10-08 08:20:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26673152. Throughput: 0: 1832.2, 1: 1827.2. Samples: 6671022. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 08:20:57,015][52710] Avg episode reward: [(0, '19.360'), (1, '20.790')] [2023-10-08 08:20:58,292][53885] Updated weights for policy 1, policy_version 13000 (0.0009) [2023-10-08 08:20:58,670][53885] Updated weights for policy 1, policy_version 13010 (0.0009) [2023-10-08 08:20:59,032][53885] Updated weights for policy 1, policy_version 13020 (0.0007) [2023-10-08 08:20:59,306][53852] Updated weights for policy 0, policy_version 13060 (0.0008) [2023-10-08 08:20:59,683][53852] Updated weights for policy 0, policy_version 13070 (0.0008) [2023-10-08 08:21:00,053][53852] Updated weights for policy 0, policy_version 13080 (0.0008) [2023-10-08 08:21:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26738688. Throughput: 0: 1827.1, 1: 1823.7. Samples: 6692052. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 08:21:02,016][52710] Avg episode reward: [(0, '19.420'), (1, '19.460')] [2023-10-08 08:21:02,715][53885] Updated weights for policy 1, policy_version 13030 (0.0007) [2023-10-08 08:21:03,080][53885] Updated weights for policy 1, policy_version 13040 (0.0007) [2023-10-08 08:21:03,454][53885] Updated weights for policy 1, policy_version 13050 (0.0008) [2023-10-08 08:21:03,769][53852] Updated weights for policy 0, policy_version 13090 (0.0008) [2023-10-08 08:21:04,145][53852] Updated weights for policy 0, policy_version 13100 (0.0007) [2023-10-08 08:21:04,523][53852] Updated weights for policy 0, policy_version 13110 (0.0008) [2023-10-08 08:21:04,893][53852] Updated weights for policy 0, policy_version 13120 (0.0007) [2023-10-08 08:21:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 26804224. Throughput: 0: 1822.0, 1: 1825.5. Samples: 6714982. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 08:21:07,016][52710] Avg episode reward: [(0, '20.620'), (1, '19.020')] [2023-10-08 08:21:07,023][53500] Saving new best policy, reward=20.620! [2023-10-08 08:21:07,068][53885] Updated weights for policy 1, policy_version 13060 (0.0009) [2023-10-08 08:21:07,439][53885] Updated weights for policy 1, policy_version 13070 (0.0009) [2023-10-08 08:21:07,799][53885] Updated weights for policy 1, policy_version 13080 (0.0011) [2023-10-08 08:21:08,603][53852] Updated weights for policy 0, policy_version 13130 (0.0007) [2023-10-08 08:21:08,972][53852] Updated weights for policy 0, policy_version 13140 (0.0009) [2023-10-08 08:21:09,355][53852] Updated weights for policy 0, policy_version 13150 (0.0008) [2023-10-08 08:21:11,433][53885] Updated weights for policy 1, policy_version 13090 (0.0010) [2023-10-08 08:21:11,801][53885] Updated weights for policy 1, policy_version 13100 (0.0008) [2023-10-08 08:21:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 26869760. Throughput: 0: 1823.7, 1: 1828.6. Samples: 6724898. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 08:21:12,016][52710] Avg episode reward: [(0, '19.070'), (1, '19.400')] [2023-10-08 08:21:12,169][53885] Updated weights for policy 1, policy_version 13110 (0.0007) [2023-10-08 08:21:12,548][53885] Updated weights for policy 1, policy_version 13120 (0.0010) [2023-10-08 08:21:13,027][53852] Updated weights for policy 0, policy_version 13160 (0.0007) [2023-10-08 08:21:13,392][53852] Updated weights for policy 0, policy_version 13170 (0.0008) [2023-10-08 08:21:13,764][53852] Updated weights for policy 0, policy_version 13180 (0.0009) [2023-10-08 08:21:16,169][53885] Updated weights for policy 1, policy_version 13130 (0.0008) [2023-10-08 08:21:16,541][53885] Updated weights for policy 1, policy_version 13140 (0.0007) [2023-10-08 08:21:16,910][53885] Updated weights for policy 1, policy_version 13150 (0.0007) [2023-10-08 08:21:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 26968064. Throughput: 0: 1815.2, 1: 1833.0. Samples: 6747930. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 08:21:17,017][52710] Avg episode reward: [(0, '19.430'), (1, '20.310')] [2023-10-08 08:21:17,567][53852] Updated weights for policy 0, policy_version 13190 (0.0009) [2023-10-08 08:21:17,927][53852] Updated weights for policy 0, policy_version 13200 (0.0007) [2023-10-08 08:21:18,303][53852] Updated weights for policy 0, policy_version 13210 (0.0007) [2023-10-08 08:21:20,426][53885] Updated weights for policy 1, policy_version 13160 (0.0009) [2023-10-08 08:21:20,787][53885] Updated weights for policy 1, policy_version 13170 (0.0011) [2023-10-08 08:21:21,169][53885] Updated weights for policy 1, policy_version 13180 (0.0010) [2023-10-08 08:21:22,005][53852] Updated weights for policy 0, policy_version 13220 (0.0008) [2023-10-08 08:21:22,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 27033600. Throughput: 0: 1813.5, 1: 1835.5. Samples: 6769668. Policy #0 lag: (min: 11.0, avg: 16.9, max: 43.0) [2023-10-08 08:21:22,017][52710] Avg episode reward: [(0, '19.090'), (1, '23.950')] [2023-10-08 08:21:22,028][53594] Saving new best policy, reward=23.950! [2023-10-08 08:21:22,385][53852] Updated weights for policy 0, policy_version 13230 (0.0010) [2023-10-08 08:21:22,756][53852] Updated weights for policy 0, policy_version 13240 (0.0007) [2023-10-08 08:21:24,904][53885] Updated weights for policy 1, policy_version 13190 (0.0007) [2023-10-08 08:21:25,276][53885] Updated weights for policy 1, policy_version 13200 (0.0008) [2023-10-08 08:21:25,649][53885] Updated weights for policy 1, policy_version 13210 (0.0008) [2023-10-08 08:21:26,341][53852] Updated weights for policy 0, policy_version 13250 (0.0008) [2023-10-08 08:21:26,717][53852] Updated weights for policy 0, policy_version 13260 (0.0008) [2023-10-08 08:21:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27099136. Throughput: 0: 1809.6, 1: 1843.2. Samples: 6781028. Policy #0 lag: (min: 11.0, avg: 16.9, max: 43.0) [2023-10-08 08:21:27,016][52710] Avg episode reward: [(0, '20.140'), (1, '21.220')] [2023-10-08 08:21:27,095][53852] Updated weights for policy 0, policy_version 13270 (0.0009) [2023-10-08 08:21:27,458][53852] Updated weights for policy 0, policy_version 13280 (0.0008) [2023-10-08 08:21:29,379][53885] Updated weights for policy 1, policy_version 13220 (0.0009) [2023-10-08 08:21:29,746][53885] Updated weights for policy 1, policy_version 13230 (0.0011) [2023-10-08 08:21:30,113][53885] Updated weights for policy 1, policy_version 13240 (0.0010) [2023-10-08 08:21:31,223][53852] Updated weights for policy 0, policy_version 13290 (0.0007) [2023-10-08 08:21:31,592][53852] Updated weights for policy 0, policy_version 13300 (0.0007) [2023-10-08 08:21:31,965][53852] Updated weights for policy 0, policy_version 13310 (0.0008) [2023-10-08 08:21:32,015][52710] Fps is (10 sec: 13107.8, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 27164672. Throughput: 0: 1822.8, 1: 1827.7. Samples: 6802414. Policy #0 lag: (min: 11.0, avg: 16.9, max: 43.0) [2023-10-08 08:21:32,015][52710] Avg episode reward: [(0, '19.430'), (1, '19.630')] [2023-10-08 08:21:33,829][53885] Updated weights for policy 1, policy_version 13250 (0.0011) [2023-10-08 08:21:34,197][53885] Updated weights for policy 1, policy_version 13260 (0.0010) [2023-10-08 08:21:34,572][53885] Updated weights for policy 1, policy_version 13270 (0.0011) [2023-10-08 08:21:34,945][53885] Updated weights for policy 1, policy_version 13280 (0.0010) [2023-10-08 08:21:35,521][53852] Updated weights for policy 0, policy_version 13320 (0.0010) [2023-10-08 08:21:35,884][53852] Updated weights for policy 0, policy_version 13330 (0.0009) [2023-10-08 08:21:36,254][53852] Updated weights for policy 0, policy_version 13340 (0.0009) [2023-10-08 08:21:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27262976. Throughput: 0: 1823.4, 1: 1843.0. Samples: 6823944. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-08 08:21:37,016][52710] Avg episode reward: [(0, '18.940'), (1, '17.930')] [2023-10-08 08:21:38,521][53885] Updated weights for policy 1, policy_version 13290 (0.0007) [2023-10-08 08:21:38,894][53885] Updated weights for policy 1, policy_version 13300 (0.0008) [2023-10-08 08:21:39,259][53885] Updated weights for policy 1, policy_version 13310 (0.0009) [2023-10-08 08:21:39,958][53852] Updated weights for policy 0, policy_version 13350 (0.0008) [2023-10-08 08:21:40,328][53852] Updated weights for policy 0, policy_version 13360 (0.0010) [2023-10-08 08:21:40,700][53852] Updated weights for policy 0, policy_version 13370 (0.0007) [2023-10-08 08:21:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 27328512. Throughput: 0: 1825.4, 1: 1832.3. Samples: 6835618. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) [2023-10-08 08:21:42,016][52710] Avg episode reward: [(0, '19.060'), (1, '19.630')] [2023-10-08 08:21:42,780][53885] Updated weights for policy 1, policy_version 13320 (0.0007) [2023-10-08 08:21:43,151][53885] Updated weights for policy 1, policy_version 13330 (0.0009) [2023-10-08 08:21:43,515][53885] Updated weights for policy 1, policy_version 13340 (0.0007) [2023-10-08 08:21:44,375][53852] Updated weights for policy 0, policy_version 13380 (0.0008) [2023-10-08 08:21:44,753][53852] Updated weights for policy 0, policy_version 13390 (0.0008) [2023-10-08 08:21:45,120][53852] Updated weights for policy 0, policy_version 13400 (0.0007) [2023-10-08 08:21:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 27394048. Throughput: 0: 1831.3, 1: 1839.8. Samples: 6857252. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 08:21:47,015][52710] Avg episode reward: [(0, '18.590'), (1, '19.280')] [2023-10-08 08:21:47,206][53885] Updated weights for policy 1, policy_version 13350 (0.0009) [2023-10-08 08:21:47,571][53885] Updated weights for policy 1, policy_version 13360 (0.0010) [2023-10-08 08:21:47,945][53885] Updated weights for policy 1, policy_version 13370 (0.0011) [2023-10-08 08:21:48,654][53852] Updated weights for policy 0, policy_version 13410 (0.0008) [2023-10-08 08:21:49,033][53852] Updated weights for policy 0, policy_version 13420 (0.0011) [2023-10-08 08:21:49,394][53852] Updated weights for policy 0, policy_version 13430 (0.0008) [2023-10-08 08:21:49,766][53852] Updated weights for policy 0, policy_version 13440 (0.0008) [2023-10-08 08:21:51,656][53885] Updated weights for policy 1, policy_version 13380 (0.0010) [2023-10-08 08:21:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 27459584. Throughput: 0: 1829.4, 1: 1834.6. Samples: 6879862. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 08:21:52,016][52710] Avg episode reward: [(0, '19.720'), (1, '21.580')] [2023-10-08 08:21:52,028][53885] Updated weights for policy 1, policy_version 13390 (0.0012) [2023-10-08 08:21:52,402][53885] Updated weights for policy 1, policy_version 13400 (0.0011) [2023-10-08 08:21:53,675][53852] Updated weights for policy 0, policy_version 13450 (0.0007) [2023-10-08 08:21:54,041][53852] Updated weights for policy 0, policy_version 13460 (0.0009) [2023-10-08 08:21:54,409][53852] Updated weights for policy 0, policy_version 13470 (0.0007) [2023-10-08 08:21:56,146][53885] Updated weights for policy 1, policy_version 13410 (0.0012) [2023-10-08 08:21:56,556][53885] Updated weights for policy 1, policy_version 13420 (0.0009) [2023-10-08 08:21:56,919][53885] Updated weights for policy 1, policy_version 13430 (0.0007) [2023-10-08 08:21:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 27525120. Throughput: 0: 1825.0, 1: 1835.7. Samples: 6889630. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 08:21:57,016][52710] Avg episode reward: [(0, '18.230'), (1, '20.700')] [2023-10-08 08:21:57,295][53885] Updated weights for policy 1, policy_version 13440 (0.0008) [2023-10-08 08:21:58,054][53852] Updated weights for policy 0, policy_version 13480 (0.0007) [2023-10-08 08:21:58,424][53852] Updated weights for policy 0, policy_version 13490 (0.0009) [2023-10-08 08:21:58,796][53852] Updated weights for policy 0, policy_version 13500 (0.0009) [2023-10-08 08:22:01,017][53885] Updated weights for policy 1, policy_version 13450 (0.0009) [2023-10-08 08:22:01,377][53885] Updated weights for policy 1, policy_version 13460 (0.0007) [2023-10-08 08:22:01,746][53885] Updated weights for policy 1, policy_version 13470 (0.0009) [2023-10-08 08:22:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27623424. Throughput: 0: 1831.6, 1: 1825.5. Samples: 6912500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:22:02,016][52710] Avg episode reward: [(0, '20.330'), (1, '20.860')] [2023-10-08 08:22:02,262][53852] Updated weights for policy 0, policy_version 13510 (0.0008) [2023-10-08 08:22:02,624][53852] Updated weights for policy 0, policy_version 13520 (0.0008) [2023-10-08 08:22:03,002][53852] Updated weights for policy 0, policy_version 13530 (0.0007) [2023-10-08 08:22:05,504][53885] Updated weights for policy 1, policy_version 13480 (0.0010) [2023-10-08 08:22:05,874][53885] Updated weights for policy 1, policy_version 13490 (0.0009) [2023-10-08 08:22:06,246][53885] Updated weights for policy 1, policy_version 13500 (0.0008) [2023-10-08 08:22:06,693][53852] Updated weights for policy 0, policy_version 13540 (0.0008) [2023-10-08 08:22:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27688960. Throughput: 0: 1836.6, 1: 1819.5. Samples: 6934190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:22:07,016][52710] Avg episode reward: [(0, '19.360'), (1, '21.440')] [2023-10-08 08:22:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000013504_13828096.pth... [2023-10-08 08:22:07,054][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000011776_12058624.pth [2023-10-08 08:22:07,059][53852] Updated weights for policy 0, policy_version 13550 (0.0007) [2023-10-08 08:22:07,442][53852] Updated weights for policy 0, policy_version 13560 (0.0007) [2023-10-08 08:22:07,728][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000013568_13893632.pth... [2023-10-08 08:22:07,756][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000011840_12124160.pth [2023-10-08 08:22:10,001][53885] Updated weights for policy 1, policy_version 13510 (0.0009) [2023-10-08 08:22:10,367][53885] Updated weights for policy 1, policy_version 13520 (0.0010) [2023-10-08 08:22:10,726][53885] Updated weights for policy 1, policy_version 13530 (0.0008) [2023-10-08 08:22:11,089][53852] Updated weights for policy 0, policy_version 13570 (0.0007) [2023-10-08 08:22:11,452][53852] Updated weights for policy 0, policy_version 13580 (0.0007) [2023-10-08 08:22:11,840][53852] Updated weights for policy 0, policy_version 13590 (0.0008) [2023-10-08 08:22:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27754496. Throughput: 0: 1835.9, 1: 1818.3. Samples: 6945466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:22:12,016][52710] Avg episode reward: [(0, '19.320'), (1, '22.080')] [2023-10-08 08:22:12,210][53852] Updated weights for policy 0, policy_version 13600 (0.0012) [2023-10-08 08:22:14,318][53885] Updated weights for policy 1, policy_version 13540 (0.0010) [2023-10-08 08:22:14,696][53885] Updated weights for policy 1, policy_version 13550 (0.0010) [2023-10-08 08:22:15,065][53885] Updated weights for policy 1, policy_version 13560 (0.0008) [2023-10-08 08:22:15,938][53852] Updated weights for policy 0, policy_version 13610 (0.0008) [2023-10-08 08:22:16,308][53852] Updated weights for policy 0, policy_version 13620 (0.0008) [2023-10-08 08:22:16,675][53852] Updated weights for policy 0, policy_version 13630 (0.0009) [2023-10-08 08:22:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27852800. Throughput: 0: 1831.0, 1: 1822.6. Samples: 6966828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:22:17,016][52710] Avg episode reward: [(0, '19.700'), (1, '21.770')] [2023-10-08 08:22:18,784][53885] Updated weights for policy 1, policy_version 13570 (0.0010) [2023-10-08 08:22:19,158][53885] Updated weights for policy 1, policy_version 13580 (0.0011) [2023-10-08 08:22:19,531][53885] Updated weights for policy 1, policy_version 13590 (0.0007) [2023-10-08 08:22:19,903][53885] Updated weights for policy 1, policy_version 13600 (0.0009) [2023-10-08 08:22:20,357][53852] Updated weights for policy 0, policy_version 13640 (0.0010) [2023-10-08 08:22:20,733][53852] Updated weights for policy 0, policy_version 13650 (0.0011) [2023-10-08 08:22:21,095][53852] Updated weights for policy 0, policy_version 13660 (0.0007) [2023-10-08 08:22:22,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27918336. Throughput: 0: 1828.1, 1: 1821.1. Samples: 6988156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:22:22,016][52710] Avg episode reward: [(0, '19.240'), (1, '22.400')] [2023-10-08 08:22:23,771][53885] Updated weights for policy 1, policy_version 13610 (0.0008) [2023-10-08 08:22:24,144][53885] Updated weights for policy 1, policy_version 13620 (0.0007) [2023-10-08 08:22:24,508][53885] Updated weights for policy 1, policy_version 13630 (0.0007) [2023-10-08 08:22:24,794][53852] Updated weights for policy 0, policy_version 13670 (0.0007) [2023-10-08 08:22:25,158][53852] Updated weights for policy 0, policy_version 13680 (0.0007) [2023-10-08 08:22:25,529][53852] Updated weights for policy 0, policy_version 13690 (0.0010) [2023-10-08 08:22:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 27983872. Throughput: 0: 1822.3, 1: 1822.6. Samples: 6999640. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:22:27,016][52710] Avg episode reward: [(0, '20.430'), (1, '22.040')] [2023-10-08 08:22:27,990][53885] Updated weights for policy 1, policy_version 13640 (0.0008) [2023-10-08 08:22:28,359][53885] Updated weights for policy 1, policy_version 13650 (0.0010) [2023-10-08 08:22:28,734][53885] Updated weights for policy 1, policy_version 13660 (0.0010) [2023-10-08 08:22:29,233][53852] Updated weights for policy 0, policy_version 13700 (0.0008) [2023-10-08 08:22:29,607][53852] Updated weights for policy 0, policy_version 13710 (0.0008) [2023-10-08 08:22:29,990][53852] Updated weights for policy 0, policy_version 13720 (0.0010) [2023-10-08 08:22:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 28049408. Throughput: 0: 1826.6, 1: 1829.0. Samples: 7021754. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:22:32,016][52710] Avg episode reward: [(0, '20.230'), (1, '24.160')] [2023-10-08 08:22:32,018][53594] Saving new best policy, reward=24.160! [2023-10-08 08:22:32,417][53885] Updated weights for policy 1, policy_version 13670 (0.0007) [2023-10-08 08:22:32,778][53885] Updated weights for policy 1, policy_version 13680 (0.0007) [2023-10-08 08:22:33,150][53885] Updated weights for policy 1, policy_version 13690 (0.0007) [2023-10-08 08:22:33,502][53852] Updated weights for policy 0, policy_version 13730 (0.0009) [2023-10-08 08:22:33,864][53852] Updated weights for policy 0, policy_version 13740 (0.0008) [2023-10-08 08:22:34,244][53852] Updated weights for policy 0, policy_version 13750 (0.0007) [2023-10-08 08:22:34,608][53852] Updated weights for policy 0, policy_version 13760 (0.0008) [2023-10-08 08:22:36,728][53885] Updated weights for policy 1, policy_version 13700 (0.0008) [2023-10-08 08:22:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 28114944. Throughput: 0: 1830.1, 1: 1831.4. Samples: 7044626. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:22:37,015][52710] Avg episode reward: [(0, '19.730'), (1, '24.430')] [2023-10-08 08:22:37,100][53885] Updated weights for policy 1, policy_version 13710 (0.0008) [2023-10-08 08:22:37,466][53885] Updated weights for policy 1, policy_version 13720 (0.0009) [2023-10-08 08:22:37,750][53594] Saving new best policy, reward=24.430! [2023-10-08 08:22:38,167][53852] Updated weights for policy 0, policy_version 13770 (0.0010) [2023-10-08 08:22:38,544][53852] Updated weights for policy 0, policy_version 13780 (0.0011) [2023-10-08 08:22:38,912][53852] Updated weights for policy 0, policy_version 13790 (0.0008) [2023-10-08 08:22:40,971][53885] Updated weights for policy 1, policy_version 13730 (0.0007) [2023-10-08 08:22:41,381][53885] Updated weights for policy 1, policy_version 13740 (0.0008) [2023-10-08 08:22:41,755][53885] Updated weights for policy 1, policy_version 13750 (0.0008) [2023-10-08 08:22:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 28180480. Throughput: 0: 1838.3, 1: 1834.7. Samples: 7054912. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-08 08:22:42,016][52710] Avg episode reward: [(0, '20.640'), (1, '22.780')] [2023-10-08 08:22:42,017][53500] Saving new best policy, reward=20.640! [2023-10-08 08:22:42,112][53885] Updated weights for policy 1, policy_version 13760 (0.0007) [2023-10-08 08:22:42,551][53852] Updated weights for policy 0, policy_version 13800 (0.0008) [2023-10-08 08:22:42,921][53852] Updated weights for policy 0, policy_version 13810 (0.0008) [2023-10-08 08:22:43,297][53852] Updated weights for policy 0, policy_version 13820 (0.0009) [2023-10-08 08:22:45,811][53885] Updated weights for policy 1, policy_version 13770 (0.0009) [2023-10-08 08:22:46,186][53885] Updated weights for policy 1, policy_version 13780 (0.0008) [2023-10-08 08:22:46,557][53885] Updated weights for policy 1, policy_version 13790 (0.0008) [2023-10-08 08:22:46,949][53852] Updated weights for policy 0, policy_version 13830 (0.0010) [2023-10-08 08:22:47,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28278784. Throughput: 0: 1836.3, 1: 1832.6. Samples: 7077600. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-08 08:22:47,016][52710] Avg episode reward: [(0, '19.450'), (1, '22.710')] [2023-10-08 08:22:47,329][53852] Updated weights for policy 0, policy_version 13840 (0.0010) [2023-10-08 08:22:47,700][53852] Updated weights for policy 0, policy_version 13850 (0.0008) [2023-10-08 08:22:50,360][53885] Updated weights for policy 1, policy_version 13800 (0.0010) [2023-10-08 08:22:50,718][53885] Updated weights for policy 1, policy_version 13810 (0.0009) [2023-10-08 08:22:51,093][53885] Updated weights for policy 1, policy_version 13820 (0.0009) [2023-10-08 08:22:51,282][53852] Updated weights for policy 0, policy_version 13860 (0.0008) [2023-10-08 08:22:51,655][53852] Updated weights for policy 0, policy_version 13870 (0.0009) [2023-10-08 08:22:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28344320. Throughput: 0: 1818.5, 1: 1833.8. Samples: 7098546. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) [2023-10-08 08:22:52,016][52710] Avg episode reward: [(0, '21.520'), (1, '22.640')] [2023-10-08 08:22:52,023][53852] Updated weights for policy 0, policy_version 13880 (0.0007) [2023-10-08 08:22:52,325][53500] Saving new best policy, reward=21.520! [2023-10-08 08:22:54,717][53885] Updated weights for policy 1, policy_version 13830 (0.0007) [2023-10-08 08:22:55,092][53885] Updated weights for policy 1, policy_version 13840 (0.0007) [2023-10-08 08:22:55,457][53885] Updated weights for policy 1, policy_version 13850 (0.0009) [2023-10-08 08:22:55,791][53852] Updated weights for policy 0, policy_version 13890 (0.0008) [2023-10-08 08:22:56,167][53852] Updated weights for policy 0, policy_version 13900 (0.0007) [2023-10-08 08:22:56,537][53852] Updated weights for policy 0, policy_version 13910 (0.0007) [2023-10-08 08:22:56,906][53852] Updated weights for policy 0, policy_version 13920 (0.0007) [2023-10-08 08:22:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 28442624. Throughput: 0: 1829.9, 1: 1834.0. Samples: 7110342. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-08 08:22:57,016][52710] Avg episode reward: [(0, '20.910'), (1, '22.910')] [2023-10-08 08:22:59,126][53885] Updated weights for policy 1, policy_version 13860 (0.0008) [2023-10-08 08:22:59,495][53885] Updated weights for policy 1, policy_version 13870 (0.0007) [2023-10-08 08:22:59,868][53885] Updated weights for policy 1, policy_version 13880 (0.0007) [2023-10-08 08:23:00,548][53852] Updated weights for policy 0, policy_version 13930 (0.0010) [2023-10-08 08:23:00,924][53852] Updated weights for policy 0, policy_version 13940 (0.0010) [2023-10-08 08:23:01,292][53852] Updated weights for policy 0, policy_version 13950 (0.0009) [2023-10-08 08:23:02,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 28508160. Throughput: 0: 1820.6, 1: 1838.6. Samples: 7131494. Policy #0 lag: (min: 18.0, avg: 18.0, max: 18.0) [2023-10-08 08:23:02,017][52710] Avg episode reward: [(0, '21.430'), (1, '21.730')] [2023-10-08 08:23:03,538][53885] Updated weights for policy 1, policy_version 13890 (0.0008) [2023-10-08 08:23:03,904][53885] Updated weights for policy 1, policy_version 13900 (0.0010) [2023-10-08 08:23:04,280][53885] Updated weights for policy 1, policy_version 13910 (0.0011) [2023-10-08 08:23:04,653][53885] Updated weights for policy 1, policy_version 13920 (0.0008) [2023-10-08 08:23:05,064][53852] Updated weights for policy 0, policy_version 13960 (0.0008) [2023-10-08 08:23:05,438][53852] Updated weights for policy 0, policy_version 13970 (0.0010) [2023-10-08 08:23:05,813][53852] Updated weights for policy 0, policy_version 13980 (0.0009) [2023-10-08 08:23:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28573696. Throughput: 0: 1831.3, 1: 1835.9. Samples: 7153178. Policy #0 lag: (min: 24.0, avg: 51.3, max: 56.0) [2023-10-08 08:23:07,016][52710] Avg episode reward: [(0, '20.390'), (1, '21.990')] [2023-10-08 08:23:08,276][53885] Updated weights for policy 1, policy_version 13930 (0.0010) [2023-10-08 08:23:08,656][53885] Updated weights for policy 1, policy_version 13940 (0.0010) [2023-10-08 08:23:09,036][53885] Updated weights for policy 1, policy_version 13950 (0.0009) [2023-10-08 08:23:09,613][53852] Updated weights for policy 0, policy_version 13990 (0.0010) [2023-10-08 08:23:09,984][53852] Updated weights for policy 0, policy_version 14000 (0.0008) [2023-10-08 08:23:10,354][53852] Updated weights for policy 0, policy_version 14010 (0.0009) [2023-10-08 08:23:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28639232. Throughput: 0: 1823.0, 1: 1835.1. Samples: 7164256. Policy #0 lag: (min: 24.0, avg: 51.3, max: 56.0) [2023-10-08 08:23:12,016][52710] Avg episode reward: [(0, '20.330'), (1, '21.670')] [2023-10-08 08:23:12,691][53885] Updated weights for policy 1, policy_version 13960 (0.0010) [2023-10-08 08:23:13,054][53885] Updated weights for policy 1, policy_version 13970 (0.0010) [2023-10-08 08:23:13,421][53885] Updated weights for policy 1, policy_version 13980 (0.0010) [2023-10-08 08:23:13,886][53852] Updated weights for policy 0, policy_version 14020 (0.0008) [2023-10-08 08:23:14,260][53852] Updated weights for policy 0, policy_version 14030 (0.0009) [2023-10-08 08:23:14,637][53852] Updated weights for policy 0, policy_version 14040 (0.0007) [2023-10-08 08:23:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 28704768. Throughput: 0: 1827.0, 1: 1820.1. Samples: 7185872. Policy #0 lag: (min: 24.0, avg: 51.3, max: 56.0) [2023-10-08 08:23:17,015][52710] Avg episode reward: [(0, '20.410'), (1, '22.790')] [2023-10-08 08:23:17,153][53885] Updated weights for policy 1, policy_version 13990 (0.0009) [2023-10-08 08:23:17,530][53885] Updated weights for policy 1, policy_version 14000 (0.0008) [2023-10-08 08:23:17,898][53885] Updated weights for policy 1, policy_version 14010 (0.0009) [2023-10-08 08:23:18,310][53852] Updated weights for policy 0, policy_version 14050 (0.0007) [2023-10-08 08:23:18,683][53852] Updated weights for policy 0, policy_version 14060 (0.0008) [2023-10-08 08:23:19,054][53852] Updated weights for policy 0, policy_version 14070 (0.0007) [2023-10-08 08:23:19,430][53852] Updated weights for policy 0, policy_version 14080 (0.0009) [2023-10-08 08:23:21,557][53885] Updated weights for policy 1, policy_version 14020 (0.0009) [2023-10-08 08:23:21,924][53885] Updated weights for policy 1, policy_version 14030 (0.0007) [2023-10-08 08:23:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 28770304. Throughput: 0: 1821.4, 1: 1816.1. Samples: 7208316. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 08:23:22,016][52710] Avg episode reward: [(0, '20.020'), (1, '23.450')] [2023-10-08 08:23:22,299][53885] Updated weights for policy 1, policy_version 14040 (0.0010) [2023-10-08 08:23:23,195][53852] Updated weights for policy 0, policy_version 14090 (0.0007) [2023-10-08 08:23:23,568][53852] Updated weights for policy 0, policy_version 14100 (0.0007) [2023-10-08 08:23:23,943][53852] Updated weights for policy 0, policy_version 14110 (0.0009) [2023-10-08 08:23:26,008][53885] Updated weights for policy 1, policy_version 14050 (0.0010) [2023-10-08 08:23:26,432][53885] Updated weights for policy 1, policy_version 14060 (0.0007) [2023-10-08 08:23:26,795][53885] Updated weights for policy 1, policy_version 14070 (0.0008) [2023-10-08 08:23:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 28835840. Throughput: 0: 1818.3, 1: 1820.5. Samples: 7218660. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 08:23:27,015][52710] Avg episode reward: [(0, '21.090'), (1, '24.090')] [2023-10-08 08:23:27,160][53885] Updated weights for policy 1, policy_version 14080 (0.0008) [2023-10-08 08:23:27,536][53852] Updated weights for policy 0, policy_version 14120 (0.0008) [2023-10-08 08:23:27,912][53852] Updated weights for policy 0, policy_version 14130 (0.0010) [2023-10-08 08:23:28,276][53852] Updated weights for policy 0, policy_version 14140 (0.0011) [2023-10-08 08:23:30,747][53885] Updated weights for policy 1, policy_version 14090 (0.0007) [2023-10-08 08:23:31,118][53885] Updated weights for policy 1, policy_version 14100 (0.0010) [2023-10-08 08:23:31,489][53885] Updated weights for policy 1, policy_version 14110 (0.0010) [2023-10-08 08:23:31,786][53852] Updated weights for policy 0, policy_version 14150 (0.0009) [2023-10-08 08:23:32,015][52710] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 28934144. Throughput: 0: 1818.4, 1: 1820.1. Samples: 7241332. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 08:23:32,016][52710] Avg episode reward: [(0, '20.510'), (1, '23.570')] [2023-10-08 08:23:32,161][53852] Updated weights for policy 0, policy_version 14160 (0.0008) [2023-10-08 08:23:32,520][53852] Updated weights for policy 0, policy_version 14170 (0.0007) [2023-10-08 08:23:35,213][53885] Updated weights for policy 1, policy_version 14120 (0.0009) [2023-10-08 08:23:35,586][53885] Updated weights for policy 1, policy_version 14130 (0.0009) [2023-10-08 08:23:35,949][53885] Updated weights for policy 1, policy_version 14140 (0.0008) [2023-10-08 08:23:36,134][53852] Updated weights for policy 0, policy_version 14180 (0.0009) [2023-10-08 08:23:36,509][53852] Updated weights for policy 0, policy_version 14190 (0.0010) [2023-10-08 08:23:36,877][53852] Updated weights for policy 0, policy_version 14200 (0.0010) [2023-10-08 08:23:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 28999680. Throughput: 0: 1824.4, 1: 1820.8. Samples: 7262582. Policy #0 lag: (min: 30.0, avg: 35.5, max: 62.0) [2023-10-08 08:23:37,016][52710] Avg episode reward: [(0, '18.750'), (1, '23.720')] [2023-10-08 08:23:39,680][53885] Updated weights for policy 1, policy_version 14150 (0.0009) [2023-10-08 08:23:40,051][53885] Updated weights for policy 1, policy_version 14160 (0.0008) [2023-10-08 08:23:40,410][53885] Updated weights for policy 1, policy_version 14170 (0.0007) [2023-10-08 08:23:40,583][53852] Updated weights for policy 0, policy_version 14210 (0.0010) [2023-10-08 08:23:40,954][53852] Updated weights for policy 0, policy_version 14220 (0.0008) [2023-10-08 08:23:41,330][53852] Updated weights for policy 0, policy_version 14230 (0.0009) [2023-10-08 08:23:41,693][53852] Updated weights for policy 0, policy_version 14240 (0.0009) [2023-10-08 08:23:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 29097984. Throughput: 0: 1829.9, 1: 1817.4. Samples: 7274470. Policy #0 lag: (min: 30.0, avg: 35.5, max: 62.0) [2023-10-08 08:23:42,015][52710] Avg episode reward: [(0, '21.050'), (1, '23.380')] [2023-10-08 08:23:44,099][53885] Updated weights for policy 1, policy_version 14180 (0.0009) [2023-10-08 08:23:44,466][53885] Updated weights for policy 1, policy_version 14190 (0.0009) [2023-10-08 08:23:44,846][53885] Updated weights for policy 1, policy_version 14200 (0.0010) [2023-10-08 08:23:45,354][53852] Updated weights for policy 0, policy_version 14250 (0.0008) [2023-10-08 08:23:45,726][53852] Updated weights for policy 0, policy_version 14260 (0.0008) [2023-10-08 08:23:46,102][53852] Updated weights for policy 0, policy_version 14270 (0.0008) [2023-10-08 08:23:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29163520. Throughput: 0: 1824.8, 1: 1814.8. Samples: 7295274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:23:47,016][52710] Avg episode reward: [(0, '20.400'), (1, '24.110')] [2023-10-08 08:23:48,561][53885] Updated weights for policy 1, policy_version 14210 (0.0008) [2023-10-08 08:23:48,928][53885] Updated weights for policy 1, policy_version 14220 (0.0007) [2023-10-08 08:23:49,294][53885] Updated weights for policy 1, policy_version 14230 (0.0011) [2023-10-08 08:23:49,662][53885] Updated weights for policy 1, policy_version 14240 (0.0009) [2023-10-08 08:23:49,893][53852] Updated weights for policy 0, policy_version 14280 (0.0008) [2023-10-08 08:23:50,260][53852] Updated weights for policy 0, policy_version 14290 (0.0009) [2023-10-08 08:23:50,642][53852] Updated weights for policy 0, policy_version 14300 (0.0010) [2023-10-08 08:23:52,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29229056. Throughput: 0: 1831.5, 1: 1815.3. Samples: 7317288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:23:52,016][52710] Avg episode reward: [(0, '19.150'), (1, '26.130')] [2023-10-08 08:23:52,030][53594] Saving new best policy, reward=26.130! [2023-10-08 08:23:53,286][53885] Updated weights for policy 1, policy_version 14250 (0.0007) [2023-10-08 08:23:53,649][53885] Updated weights for policy 1, policy_version 14260 (0.0007) [2023-10-08 08:23:54,011][53885] Updated weights for policy 1, policy_version 14270 (0.0007) [2023-10-08 08:23:54,318][53852] Updated weights for policy 0, policy_version 14310 (0.0007) [2023-10-08 08:23:54,688][53852] Updated weights for policy 0, policy_version 14320 (0.0007) [2023-10-08 08:23:55,067][53852] Updated weights for policy 0, policy_version 14330 (0.0008) [2023-10-08 08:23:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 29294592. Throughput: 0: 1833.7, 1: 1816.5. Samples: 7328518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:23:57,016][52710] Avg episode reward: [(0, '19.380'), (1, '23.160')] [2023-10-08 08:23:57,727][53885] Updated weights for policy 1, policy_version 14280 (0.0007) [2023-10-08 08:23:58,089][53885] Updated weights for policy 1, policy_version 14290 (0.0007) [2023-10-08 08:23:58,454][53885] Updated weights for policy 1, policy_version 14300 (0.0010) [2023-10-08 08:23:58,563][53852] Updated weights for policy 0, policy_version 14340 (0.0008) [2023-10-08 08:23:58,930][53852] Updated weights for policy 0, policy_version 14350 (0.0008) [2023-10-08 08:23:59,300][53852] Updated weights for policy 0, policy_version 14360 (0.0007) [2023-10-08 08:24:02,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 29360128. Throughput: 0: 1838.9, 1: 1819.5. Samples: 7350498. Policy #0 lag: (min: 27.0, avg: 27.6, max: 44.0) [2023-10-08 08:24:02,016][52710] Avg episode reward: [(0, '18.360'), (1, '25.010')] [2023-10-08 08:24:02,238][53885] Updated weights for policy 1, policy_version 14310 (0.0007) [2023-10-08 08:24:02,599][53885] Updated weights for policy 1, policy_version 14320 (0.0007) [2023-10-08 08:24:02,930][53852] Updated weights for policy 0, policy_version 14370 (0.0008) [2023-10-08 08:24:02,971][53885] Updated weights for policy 1, policy_version 14330 (0.0007) [2023-10-08 08:24:03,306][53852] Updated weights for policy 0, policy_version 14380 (0.0008) [2023-10-08 08:24:03,669][53852] Updated weights for policy 0, policy_version 14390 (0.0007) [2023-10-08 08:24:04,041][53852] Updated weights for policy 0, policy_version 14400 (0.0008) [2023-10-08 08:24:06,626][53885] Updated weights for policy 1, policy_version 14340 (0.0008) [2023-10-08 08:24:06,998][53885] Updated weights for policy 1, policy_version 14350 (0.0010) [2023-10-08 08:24:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29425664. Throughput: 0: 1849.7, 1: 1818.3. Samples: 7373374. Policy #0 lag: (min: 27.0, avg: 27.6, max: 44.0) [2023-10-08 08:24:07,016][52710] Avg episode reward: [(0, '19.740'), (1, '23.160')] [2023-10-08 08:24:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000014400_14745600.pth... [2023-10-08 08:24:07,062][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000012704_13008896.pth [2023-10-08 08:24:07,357][53885] Updated weights for policy 1, policy_version 14360 (0.0009) [2023-10-08 08:24:07,563][53852] Updated weights for policy 0, policy_version 14410 (0.0008) [2023-10-08 08:24:07,656][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000014368_14712832.pth... [2023-10-08 08:24:07,684][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000012640_12943360.pth [2023-10-08 08:24:07,940][53852] Updated weights for policy 0, policy_version 14420 (0.0011) [2023-10-08 08:24:08,300][53852] Updated weights for policy 0, policy_version 14430 (0.0007) [2023-10-08 08:24:11,209][53885] Updated weights for policy 1, policy_version 14370 (0.0007) [2023-10-08 08:24:11,621][53885] Updated weights for policy 1, policy_version 14380 (0.0010) [2023-10-08 08:24:11,897][53852] Updated weights for policy 0, policy_version 14440 (0.0008) [2023-10-08 08:24:11,985][53885] Updated weights for policy 1, policy_version 14390 (0.0008) [2023-10-08 08:24:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29491200. Throughput: 0: 1851.4, 1: 1815.9. Samples: 7383690. Policy #0 lag: (min: 27.0, avg: 27.6, max: 44.0) [2023-10-08 08:24:12,016][52710] Avg episode reward: [(0, '18.590'), (1, '23.710')] [2023-10-08 08:24:12,278][53852] Updated weights for policy 0, policy_version 14450 (0.0008) [2023-10-08 08:24:12,351][53885] Updated weights for policy 1, policy_version 14400 (0.0008) [2023-10-08 08:24:12,657][53852] Updated weights for policy 0, policy_version 14460 (0.0009) [2023-10-08 08:24:16,067][53885] Updated weights for policy 1, policy_version 14410 (0.0010) [2023-10-08 08:24:16,380][53852] Updated weights for policy 0, policy_version 14470 (0.0009) [2023-10-08 08:24:16,432][53885] Updated weights for policy 1, policy_version 14420 (0.0008) [2023-10-08 08:24:16,749][53852] Updated weights for policy 0, policy_version 14480 (0.0007) [2023-10-08 08:24:16,801][53885] Updated weights for policy 1, policy_version 14430 (0.0009) [2023-10-08 08:24:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 29589504. Throughput: 0: 1849.2, 1: 1812.6. Samples: 7406116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:24:17,016][52710] Avg episode reward: [(0, '19.940'), (1, '22.820')] [2023-10-08 08:24:17,123][53852] Updated weights for policy 0, policy_version 14490 (0.0008) [2023-10-08 08:24:20,575][53885] Updated weights for policy 1, policy_version 14440 (0.0007) [2023-10-08 08:24:20,837][53852] Updated weights for policy 0, policy_version 14500 (0.0009) [2023-10-08 08:24:20,932][53885] Updated weights for policy 1, policy_version 14450 (0.0007) [2023-10-08 08:24:21,208][53852] Updated weights for policy 0, policy_version 14510 (0.0007) [2023-10-08 08:24:21,298][53885] Updated weights for policy 1, policy_version 14460 (0.0007) [2023-10-08 08:24:21,584][53852] Updated weights for policy 0, policy_version 14520 (0.0008) [2023-10-08 08:24:22,015][52710] Fps is (10 sec: 19660.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 29687808. Throughput: 0: 1828.9, 1: 1811.4. Samples: 7426396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:24:22,016][52710] Avg episode reward: [(0, '19.900'), (1, '25.030')] [2023-10-08 08:24:25,035][53852] Updated weights for policy 0, policy_version 14530 (0.0008) [2023-10-08 08:24:25,118][53885] Updated weights for policy 1, policy_version 14470 (0.0008) [2023-10-08 08:24:25,403][53852] Updated weights for policy 0, policy_version 14540 (0.0008) [2023-10-08 08:24:25,485][53885] Updated weights for policy 1, policy_version 14480 (0.0007) [2023-10-08 08:24:25,779][53852] Updated weights for policy 0, policy_version 14550 (0.0007) [2023-10-08 08:24:25,849][53885] Updated weights for policy 1, policy_version 14490 (0.0007) [2023-10-08 08:24:26,143][53852] Updated weights for policy 0, policy_version 14560 (0.0008) [2023-10-08 08:24:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 29753344. Throughput: 0: 1840.5, 1: 1815.5. Samples: 7438988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:24:27,016][52710] Avg episode reward: [(0, '20.150'), (1, '25.320')] [2023-10-08 08:24:29,628][53885] Updated weights for policy 1, policy_version 14500 (0.0008) [2023-10-08 08:24:29,769][53852] Updated weights for policy 0, policy_version 14570 (0.0007) [2023-10-08 08:24:29,991][53885] Updated weights for policy 1, policy_version 14510 (0.0007) [2023-10-08 08:24:30,143][53852] Updated weights for policy 0, policy_version 14580 (0.0007) [2023-10-08 08:24:30,358][53885] Updated weights for policy 1, policy_version 14520 (0.0007) [2023-10-08 08:24:30,519][53852] Updated weights for policy 0, policy_version 14590 (0.0007) [2023-10-08 08:24:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29818880. Throughput: 0: 1827.7, 1: 1812.0. Samples: 7459064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:24:32,016][52710] Avg episode reward: [(0, '19.190'), (1, '21.830')] [2023-10-08 08:24:33,974][53885] Updated weights for policy 1, policy_version 14530 (0.0009) [2023-10-08 08:24:34,251][53852] Updated weights for policy 0, policy_version 14600 (0.0009) [2023-10-08 08:24:34,339][53885] Updated weights for policy 1, policy_version 14540 (0.0007) [2023-10-08 08:24:34,628][53852] Updated weights for policy 0, policy_version 14610 (0.0007) [2023-10-08 08:24:34,696][53885] Updated weights for policy 1, policy_version 14550 (0.0008) [2023-10-08 08:24:35,008][53852] Updated weights for policy 0, policy_version 14620 (0.0008) [2023-10-08 08:24:35,067][53885] Updated weights for policy 1, policy_version 14560 (0.0008) [2023-10-08 08:24:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 29884416. Throughput: 0: 1845.4, 1: 1800.3. Samples: 7481344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:24:37,016][52710] Avg episode reward: [(0, '20.740'), (1, '23.750')] [2023-10-08 08:24:38,788][53885] Updated weights for policy 1, policy_version 14570 (0.0009) [2023-10-08 08:24:38,905][53852] Updated weights for policy 0, policy_version 14630 (0.0008) [2023-10-08 08:24:39,148][53885] Updated weights for policy 1, policy_version 14580 (0.0009) [2023-10-08 08:24:39,275][53852] Updated weights for policy 0, policy_version 14640 (0.0007) [2023-10-08 08:24:39,512][53885] Updated weights for policy 1, policy_version 14590 (0.0008) [2023-10-08 08:24:39,653][53852] Updated weights for policy 0, policy_version 14650 (0.0007) [2023-10-08 08:24:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 29949952. Throughput: 0: 1827.1, 1: 1797.7. Samples: 7491632. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 08:24:42,016][52710] Avg episode reward: [(0, '20.470'), (1, '24.580')] [2023-10-08 08:24:43,213][53885] Updated weights for policy 1, policy_version 14600 (0.0009) [2023-10-08 08:24:43,314][53852] Updated weights for policy 0, policy_version 14660 (0.0007) [2023-10-08 08:24:43,574][53885] Updated weights for policy 1, policy_version 14610 (0.0007) [2023-10-08 08:24:43,676][53852] Updated weights for policy 0, policy_version 14670 (0.0007) [2023-10-08 08:24:43,943][53885] Updated weights for policy 1, policy_version 14620 (0.0008) [2023-10-08 08:24:44,050][53852] Updated weights for policy 0, policy_version 14680 (0.0008) [2023-10-08 08:24:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 30015488. Throughput: 0: 1837.0, 1: 1798.0. Samples: 7514076. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 08:24:47,016][52710] Avg episode reward: [(0, '21.200'), (1, '24.380')] [2023-10-08 08:24:47,682][53885] Updated weights for policy 1, policy_version 14630 (0.0008) [2023-10-08 08:24:47,721][53852] Updated weights for policy 0, policy_version 14690 (0.0008) [2023-10-08 08:24:48,049][53885] Updated weights for policy 1, policy_version 14640 (0.0010) [2023-10-08 08:24:48,084][53852] Updated weights for policy 0, policy_version 14700 (0.0009) [2023-10-08 08:24:48,424][53885] Updated weights for policy 1, policy_version 14650 (0.0008) [2023-10-08 08:24:48,457][53852] Updated weights for policy 0, policy_version 14710 (0.0007) [2023-10-08 08:24:48,824][53852] Updated weights for policy 0, policy_version 14720 (0.0009) [2023-10-08 08:24:52,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30081024. Throughput: 0: 1825.9, 1: 1797.5. Samples: 7536428. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) [2023-10-08 08:24:52,017][52710] Avg episode reward: [(0, '21.960'), (1, '25.100')] [2023-10-08 08:24:52,028][53500] Saving new best policy, reward=21.960! [2023-10-08 08:24:52,182][53885] Updated weights for policy 1, policy_version 14660 (0.0008) [2023-10-08 08:24:52,551][53885] Updated weights for policy 1, policy_version 14670 (0.0010) [2023-10-08 08:24:52,586][53852] Updated weights for policy 0, policy_version 14730 (0.0008) [2023-10-08 08:24:52,920][53885] Updated weights for policy 1, policy_version 14680 (0.0009) [2023-10-08 08:24:52,964][53852] Updated weights for policy 0, policy_version 14740 (0.0008) [2023-10-08 08:24:53,340][53852] Updated weights for policy 0, policy_version 14750 (0.0007) [2023-10-08 08:24:56,689][53885] Updated weights for policy 1, policy_version 14690 (0.0009) [2023-10-08 08:24:56,958][53852] Updated weights for policy 0, policy_version 14760 (0.0007) [2023-10-08 08:24:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30146560. Throughput: 0: 1824.6, 1: 1788.4. Samples: 7546276. Policy #0 lag: (min: 5.0, avg: 7.7, max: 37.0) [2023-10-08 08:24:57,015][52710] Avg episode reward: [(0, '19.780'), (1, '24.230')] [2023-10-08 08:24:57,092][53885] Updated weights for policy 1, policy_version 14700 (0.0009) [2023-10-08 08:24:57,327][53852] Updated weights for policy 0, policy_version 14770 (0.0008) [2023-10-08 08:24:57,459][53885] Updated weights for policy 1, policy_version 14710 (0.0007) [2023-10-08 08:24:57,695][53852] Updated weights for policy 0, policy_version 14780 (0.0009) [2023-10-08 08:24:57,820][53885] Updated weights for policy 1, policy_version 14720 (0.0008) [2023-10-08 08:25:01,536][53852] Updated weights for policy 0, policy_version 14790 (0.0008) [2023-10-08 08:25:01,605][53885] Updated weights for policy 1, policy_version 14730 (0.0009) [2023-10-08 08:25:01,911][53852] Updated weights for policy 0, policy_version 14800 (0.0007) [2023-10-08 08:25:01,973][53885] Updated weights for policy 1, policy_version 14740 (0.0007) [2023-10-08 08:25:02,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30212096. Throughput: 0: 1821.1, 1: 1796.1. Samples: 7568892. Policy #0 lag: (min: 5.0, avg: 7.7, max: 37.0) [2023-10-08 08:25:02,016][52710] Avg episode reward: [(0, '19.230'), (1, '26.040')] [2023-10-08 08:25:02,284][53852] Updated weights for policy 0, policy_version 14810 (0.0008) [2023-10-08 08:25:02,345][53885] Updated weights for policy 1, policy_version 14750 (0.0008) [2023-10-08 08:25:05,970][53852] Updated weights for policy 0, policy_version 14820 (0.0007) [2023-10-08 08:25:06,088][53885] Updated weights for policy 1, policy_version 14760 (0.0010) [2023-10-08 08:25:06,342][53852] Updated weights for policy 0, policy_version 14830 (0.0007) [2023-10-08 08:25:06,450][53885] Updated weights for policy 1, policy_version 14770 (0.0007) [2023-10-08 08:25:06,705][53852] Updated weights for policy 0, policy_version 14840 (0.0007) [2023-10-08 08:25:06,818][53885] Updated weights for policy 1, policy_version 14780 (0.0008) [2023-10-08 08:25:07,016][52710] Fps is (10 sec: 19659.6, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 30343168. Throughput: 0: 1825.2, 1: 1804.9. Samples: 7589752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:25:07,017][52710] Avg episode reward: [(0, '19.500'), (1, '24.430')] [2023-10-08 08:25:10,354][53852] Updated weights for policy 0, policy_version 14850 (0.0007) [2023-10-08 08:25:10,518][53885] Updated weights for policy 1, policy_version 14790 (0.0008) [2023-10-08 08:25:10,720][53852] Updated weights for policy 0, policy_version 14860 (0.0008) [2023-10-08 08:25:10,876][53885] Updated weights for policy 1, policy_version 14800 (0.0009) [2023-10-08 08:25:11,093][53852] Updated weights for policy 0, policy_version 14870 (0.0008) [2023-10-08 08:25:11,252][53885] Updated weights for policy 1, policy_version 14810 (0.0007) [2023-10-08 08:25:11,468][53852] Updated weights for policy 0, policy_version 14880 (0.0007) [2023-10-08 08:25:12,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 30408704. Throughput: 0: 1818.7, 1: 1795.7. Samples: 7601634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:25:12,015][52710] Avg episode reward: [(0, '19.580'), (1, '24.710')] [2023-10-08 08:25:14,890][53885] Updated weights for policy 1, policy_version 14820 (0.0008) [2023-10-08 08:25:15,022][53852] Updated weights for policy 0, policy_version 14890 (0.0008) [2023-10-08 08:25:15,262][53885] Updated weights for policy 1, policy_version 14830 (0.0008) [2023-10-08 08:25:15,382][53852] Updated weights for policy 0, policy_version 14900 (0.0008) [2023-10-08 08:25:15,616][53885] Updated weights for policy 1, policy_version 14840 (0.0007) [2023-10-08 08:25:15,753][53852] Updated weights for policy 0, policy_version 14910 (0.0008) [2023-10-08 08:25:17,015][52710] Fps is (10 sec: 13108.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 30474240. Throughput: 0: 1820.3, 1: 1811.4. Samples: 7622490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:25:17,016][52710] Avg episode reward: [(0, '19.030'), (1, '24.130')] [2023-10-08 08:25:19,308][53852] Updated weights for policy 0, policy_version 14920 (0.0008) [2023-10-08 08:25:19,446][53885] Updated weights for policy 1, policy_version 14850 (0.0008) [2023-10-08 08:25:19,678][53852] Updated weights for policy 0, policy_version 14930 (0.0008) [2023-10-08 08:25:19,812][53885] Updated weights for policy 1, policy_version 14860 (0.0009) [2023-10-08 08:25:20,040][53852] Updated weights for policy 0, policy_version 14940 (0.0008) [2023-10-08 08:25:20,189][53885] Updated weights for policy 1, policy_version 14870 (0.0010) [2023-10-08 08:25:20,552][53885] Updated weights for policy 1, policy_version 14880 (0.0007) [2023-10-08 08:25:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 30539776. Throughput: 0: 1813.6, 1: 1806.1. Samples: 7644228. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-08 08:25:22,016][52710] Avg episode reward: [(0, '19.350'), (1, '23.480')] [2023-10-08 08:25:23,782][53852] Updated weights for policy 0, policy_version 14950 (0.0008) [2023-10-08 08:25:24,151][53852] Updated weights for policy 0, policy_version 14960 (0.0009) [2023-10-08 08:25:24,160][53885] Updated weights for policy 1, policy_version 14890 (0.0007) [2023-10-08 08:25:24,515][53852] Updated weights for policy 0, policy_version 14970 (0.0009) [2023-10-08 08:25:24,528][53885] Updated weights for policy 1, policy_version 14900 (0.0008) [2023-10-08 08:25:24,894][53885] Updated weights for policy 1, policy_version 14910 (0.0007) [2023-10-08 08:25:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 30605312. Throughput: 0: 1811.3, 1: 1822.1. Samples: 7655138. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-08 08:25:27,016][52710] Avg episode reward: [(0, '19.490'), (1, '23.340')] [2023-10-08 08:25:28,106][53852] Updated weights for policy 0, policy_version 14980 (0.0009) [2023-10-08 08:25:28,473][53852] Updated weights for policy 0, policy_version 14990 (0.0007) [2023-10-08 08:25:28,609][53885] Updated weights for policy 1, policy_version 14920 (0.0007) [2023-10-08 08:25:28,848][53852] Updated weights for policy 0, policy_version 15000 (0.0007) [2023-10-08 08:25:28,965][53885] Updated weights for policy 1, policy_version 14930 (0.0008) [2023-10-08 08:25:29,338][53885] Updated weights for policy 1, policy_version 14940 (0.0010) [2023-10-08 08:25:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 30670848. Throughput: 0: 1812.6, 1: 1809.6. Samples: 7677076. Policy #0 lag: (min: 5.0, avg: 13.0, max: 37.0) [2023-10-08 08:25:32,016][52710] Avg episode reward: [(0, '19.880'), (1, '22.340')] [2023-10-08 08:25:32,563][53852] Updated weights for policy 0, policy_version 15010 (0.0008) [2023-10-08 08:25:32,930][53852] Updated weights for policy 0, policy_version 15020 (0.0008) [2023-10-08 08:25:32,937][53885] Updated weights for policy 1, policy_version 14950 (0.0008) [2023-10-08 08:25:33,298][53852] Updated weights for policy 0, policy_version 15030 (0.0007) [2023-10-08 08:25:33,303][53885] Updated weights for policy 1, policy_version 14960 (0.0007) [2023-10-08 08:25:33,668][53852] Updated weights for policy 0, policy_version 15040 (0.0007) [2023-10-08 08:25:33,671][53885] Updated weights for policy 1, policy_version 14970 (0.0007) [2023-10-08 08:25:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 30736384. Throughput: 0: 1819.3, 1: 1820.5. Samples: 7700222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:25:37,016][52710] Avg episode reward: [(0, '20.950'), (1, '23.830')] [2023-10-08 08:25:37,252][53885] Updated weights for policy 1, policy_version 14980 (0.0008) [2023-10-08 08:25:37,409][53852] Updated weights for policy 0, policy_version 15050 (0.0008) [2023-10-08 08:25:37,619][53885] Updated weights for policy 1, policy_version 14990 (0.0009) [2023-10-08 08:25:37,775][53852] Updated weights for policy 0, policy_version 15060 (0.0007) [2023-10-08 08:25:37,985][53885] Updated weights for policy 1, policy_version 15000 (0.0009) [2023-10-08 08:25:38,143][53852] Updated weights for policy 0, policy_version 15070 (0.0009) [2023-10-08 08:25:41,698][53852] Updated weights for policy 0, policy_version 15080 (0.0008) [2023-10-08 08:25:41,722][53885] Updated weights for policy 1, policy_version 15010 (0.0007) [2023-10-08 08:25:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 30801920. Throughput: 0: 1819.8, 1: 1823.1. Samples: 7710204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:25:42,016][52710] Avg episode reward: [(0, '20.790'), (1, '23.080')] [2023-10-08 08:25:42,066][53852] Updated weights for policy 0, policy_version 15090 (0.0009) [2023-10-08 08:25:42,100][53885] Updated weights for policy 1, policy_version 15020 (0.0009) [2023-10-08 08:25:42,434][53852] Updated weights for policy 0, policy_version 15100 (0.0009) [2023-10-08 08:25:42,470][53885] Updated weights for policy 1, policy_version 15030 (0.0008) [2023-10-08 08:25:42,841][53885] Updated weights for policy 1, policy_version 15040 (0.0007) [2023-10-08 08:25:46,166][53852] Updated weights for policy 0, policy_version 15110 (0.0009) [2023-10-08 08:25:46,543][53852] Updated weights for policy 0, policy_version 15120 (0.0007) [2023-10-08 08:25:46,591][53885] Updated weights for policy 1, policy_version 15050 (0.0008) [2023-10-08 08:25:46,915][53852] Updated weights for policy 0, policy_version 15130 (0.0007) [2023-10-08 08:25:46,962][53885] Updated weights for policy 1, policy_version 15060 (0.0007) [2023-10-08 08:25:47,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30867456. Throughput: 0: 1823.9, 1: 1819.5. Samples: 7732844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:25:47,016][52710] Avg episode reward: [(0, '19.010'), (1, '20.700')] [2023-10-08 08:25:47,333][53885] Updated weights for policy 1, policy_version 15070 (0.0008) [2023-10-08 08:25:50,818][53852] Updated weights for policy 0, policy_version 15140 (0.0008) [2023-10-08 08:25:50,998][53885] Updated weights for policy 1, policy_version 15080 (0.0008) [2023-10-08 08:25:51,201][53852] Updated weights for policy 0, policy_version 15150 (0.0007) [2023-10-08 08:25:51,359][53885] Updated weights for policy 1, policy_version 15090 (0.0007) [2023-10-08 08:25:51,573][53852] Updated weights for policy 0, policy_version 15160 (0.0008) [2023-10-08 08:25:51,725][53885] Updated weights for policy 1, policy_version 15100 (0.0008) [2023-10-08 08:25:52,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 30998528. Throughput: 0: 1812.5, 1: 1818.2. Samples: 7753130. Policy #0 lag: (min: 24.0, avg: 49.0, max: 56.0) [2023-10-08 08:25:52,016][52710] Avg episode reward: [(0, '20.350'), (1, '21.720')] [2023-10-08 08:25:55,229][53852] Updated weights for policy 0, policy_version 15170 (0.0008) [2023-10-08 08:25:55,362][53885] Updated weights for policy 1, policy_version 15110 (0.0007) [2023-10-08 08:25:55,588][53852] Updated weights for policy 0, policy_version 15180 (0.0008) [2023-10-08 08:25:55,727][53885] Updated weights for policy 1, policy_version 15120 (0.0008) [2023-10-08 08:25:55,960][53852] Updated weights for policy 0, policy_version 15190 (0.0007) [2023-10-08 08:25:56,097][53885] Updated weights for policy 1, policy_version 15130 (0.0007) [2023-10-08 08:25:56,328][53852] Updated weights for policy 0, policy_version 15200 (0.0008) [2023-10-08 08:25:57,015][52710] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 31064064. Throughput: 0: 1815.2, 1: 1819.6. Samples: 7765202. Policy #0 lag: (min: 24.0, avg: 49.0, max: 56.0) [2023-10-08 08:25:57,016][52710] Avg episode reward: [(0, '20.640'), (1, '21.670')] [2023-10-08 08:25:59,721][53885] Updated weights for policy 1, policy_version 15140 (0.0007) [2023-10-08 08:26:00,092][53885] Updated weights for policy 1, policy_version 15150 (0.0007) [2023-10-08 08:26:00,095][53852] Updated weights for policy 0, policy_version 15210 (0.0009) [2023-10-08 08:26:00,450][53885] Updated weights for policy 1, policy_version 15160 (0.0008) [2023-10-08 08:26:00,464][53852] Updated weights for policy 0, policy_version 15220 (0.0007) [2023-10-08 08:26:00,832][53852] Updated weights for policy 0, policy_version 15230 (0.0008) [2023-10-08 08:26:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 31129600. Throughput: 0: 1817.7, 1: 1813.4. Samples: 7785890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:02,015][52710] Avg episode reward: [(0, '19.820'), (1, '23.260')] [2023-10-08 08:26:04,206][53885] Updated weights for policy 1, policy_version 15170 (0.0009) [2023-10-08 08:26:04,569][53852] Updated weights for policy 0, policy_version 15240 (0.0008) [2023-10-08 08:26:04,571][53885] Updated weights for policy 1, policy_version 15180 (0.0010) [2023-10-08 08:26:04,931][53885] Updated weights for policy 1, policy_version 15190 (0.0009) [2023-10-08 08:26:04,945][53852] Updated weights for policy 0, policy_version 15250 (0.0009) [2023-10-08 08:26:05,302][53885] Updated weights for policy 1, policy_version 15200 (0.0009) [2023-10-08 08:26:05,306][53852] Updated weights for policy 0, policy_version 15260 (0.0008) [2023-10-08 08:26:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 31195136. Throughput: 0: 1814.2, 1: 1824.8. Samples: 7807986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:07,016][52710] Avg episode reward: [(0, '20.080'), (1, '23.170')] [2023-10-08 08:26:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000015264_15630336.pth... [2023-10-08 08:26:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000015200_15564800.pth... [2023-10-08 08:26:07,056][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000013568_13893632.pth [2023-10-08 08:26:07,062][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000013504_13828096.pth [2023-10-08 08:26:08,978][53885] Updated weights for policy 1, policy_version 15210 (0.0007) [2023-10-08 08:26:09,049][53852] Updated weights for policy 0, policy_version 15270 (0.0007) [2023-10-08 08:26:09,346][53885] Updated weights for policy 1, policy_version 15220 (0.0008) [2023-10-08 08:26:09,425][53852] Updated weights for policy 0, policy_version 15280 (0.0007) [2023-10-08 08:26:09,713][53885] Updated weights for policy 1, policy_version 15230 (0.0007) [2023-10-08 08:26:09,798][53852] Updated weights for policy 0, policy_version 15290 (0.0009) [2023-10-08 08:26:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31260672. Throughput: 0: 1820.5, 1: 1815.3. Samples: 7818750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:12,015][52710] Avg episode reward: [(0, '21.110'), (1, '23.510')] [2023-10-08 08:26:13,466][53885] Updated weights for policy 1, policy_version 15240 (0.0008) [2023-10-08 08:26:13,483][53852] Updated weights for policy 0, policy_version 15300 (0.0009) [2023-10-08 08:26:13,832][53885] Updated weights for policy 1, policy_version 15250 (0.0007) [2023-10-08 08:26:13,851][53852] Updated weights for policy 0, policy_version 15310 (0.0007) [2023-10-08 08:26:14,204][53885] Updated weights for policy 1, policy_version 15260 (0.0007) [2023-10-08 08:26:14,219][53852] Updated weights for policy 0, policy_version 15320 (0.0008) [2023-10-08 08:26:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31326208. Throughput: 0: 1813.7, 1: 1818.7. Samples: 7840530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:17,015][52710] Avg episode reward: [(0, '20.790'), (1, '23.120')] [2023-10-08 08:26:17,902][53852] Updated weights for policy 0, policy_version 15330 (0.0008) [2023-10-08 08:26:17,971][53885] Updated weights for policy 1, policy_version 15270 (0.0007) [2023-10-08 08:26:18,271][53852] Updated weights for policy 0, policy_version 15340 (0.0008) [2023-10-08 08:26:18,334][53885] Updated weights for policy 1, policy_version 15280 (0.0007) [2023-10-08 08:26:18,646][53852] Updated weights for policy 0, policy_version 15350 (0.0009) [2023-10-08 08:26:18,710][53885] Updated weights for policy 1, policy_version 15290 (0.0007) [2023-10-08 08:26:19,014][53852] Updated weights for policy 0, policy_version 15360 (0.0009) [2023-10-08 08:26:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31391744. Throughput: 0: 1807.2, 1: 1811.9. Samples: 7863080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:22,016][52710] Avg episode reward: [(0, '20.010'), (1, '22.110')] [2023-10-08 08:26:22,559][53885] Updated weights for policy 1, policy_version 15300 (0.0008) [2023-10-08 08:26:22,864][53852] Updated weights for policy 0, policy_version 15370 (0.0008) [2023-10-08 08:26:22,929][53885] Updated weights for policy 1, policy_version 15310 (0.0008) [2023-10-08 08:26:23,237][53852] Updated weights for policy 0, policy_version 15380 (0.0008) [2023-10-08 08:26:23,299][53885] Updated weights for policy 1, policy_version 15320 (0.0009) [2023-10-08 08:26:23,615][53852] Updated weights for policy 0, policy_version 15390 (0.0008) [2023-10-08 08:26:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31457280. Throughput: 0: 1803.6, 1: 1810.5. Samples: 7872836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:27,015][52710] Avg episode reward: [(0, '19.140'), (1, '23.150')] [2023-10-08 08:26:27,069][53885] Updated weights for policy 1, policy_version 15330 (0.0007) [2023-10-08 08:26:27,212][53852] Updated weights for policy 0, policy_version 15400 (0.0007) [2023-10-08 08:26:27,476][53885] Updated weights for policy 1, policy_version 15340 (0.0007) [2023-10-08 08:26:27,580][53852] Updated weights for policy 0, policy_version 15410 (0.0007) [2023-10-08 08:26:27,847][53885] Updated weights for policy 1, policy_version 15350 (0.0007) [2023-10-08 08:26:27,950][53852] Updated weights for policy 0, policy_version 15420 (0.0007) [2023-10-08 08:26:28,216][53885] Updated weights for policy 1, policy_version 15360 (0.0010) [2023-10-08 08:26:31,503][53852] Updated weights for policy 0, policy_version 15430 (0.0007) [2023-10-08 08:26:31,877][53852] Updated weights for policy 0, policy_version 15440 (0.0009) [2023-10-08 08:26:31,939][53885] Updated weights for policy 1, policy_version 15370 (0.0009) [2023-10-08 08:26:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31522816. Throughput: 0: 1806.6, 1: 1811.7. Samples: 7895666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:32,015][52710] Avg episode reward: [(0, '19.480'), (1, '24.370')] [2023-10-08 08:26:32,239][53852] Updated weights for policy 0, policy_version 15450 (0.0009) [2023-10-08 08:26:32,303][53885] Updated weights for policy 1, policy_version 15380 (0.0008) [2023-10-08 08:26:32,677][53885] Updated weights for policy 1, policy_version 15390 (0.0008) [2023-10-08 08:26:36,018][53852] Updated weights for policy 0, policy_version 15460 (0.0009) [2023-10-08 08:26:36,401][53852] Updated weights for policy 0, policy_version 15470 (0.0008) [2023-10-08 08:26:36,511][53885] Updated weights for policy 1, policy_version 15400 (0.0008) [2023-10-08 08:26:36,772][53852] Updated weights for policy 0, policy_version 15480 (0.0009) [2023-10-08 08:26:36,877][53885] Updated weights for policy 1, policy_version 15410 (0.0009) [2023-10-08 08:26:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 31588352. Throughput: 0: 1817.1, 1: 1818.7. Samples: 7916742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:37,016][52710] Avg episode reward: [(0, '19.260'), (1, '22.690')] [2023-10-08 08:26:37,243][53885] Updated weights for policy 1, policy_version 15420 (0.0008) [2023-10-08 08:26:40,522][53852] Updated weights for policy 0, policy_version 15490 (0.0009) [2023-10-08 08:26:40,890][53852] Updated weights for policy 0, policy_version 15500 (0.0011) [2023-10-08 08:26:40,951][53885] Updated weights for policy 1, policy_version 15430 (0.0009) [2023-10-08 08:26:41,258][53852] Updated weights for policy 0, policy_version 15510 (0.0007) [2023-10-08 08:26:41,328][53885] Updated weights for policy 1, policy_version 15440 (0.0007) [2023-10-08 08:26:41,628][53852] Updated weights for policy 0, policy_version 15520 (0.0008) [2023-10-08 08:26:41,698][53885] Updated weights for policy 1, policy_version 15450 (0.0009) [2023-10-08 08:26:42,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 31719424. Throughput: 0: 1809.8, 1: 1804.4. Samples: 7927844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:42,016][52710] Avg episode reward: [(0, '20.070'), (1, '25.420')] [2023-10-08 08:26:45,311][53885] Updated weights for policy 1, policy_version 15460 (0.0008) [2023-10-08 08:26:45,328][53852] Updated weights for policy 0, policy_version 15530 (0.0009) [2023-10-08 08:26:45,676][53885] Updated weights for policy 1, policy_version 15470 (0.0008) [2023-10-08 08:26:45,698][53852] Updated weights for policy 0, policy_version 15540 (0.0008) [2023-10-08 08:26:46,043][53885] Updated weights for policy 1, policy_version 15480 (0.0007) [2023-10-08 08:26:46,071][53852] Updated weights for policy 0, policy_version 15550 (0.0007) [2023-10-08 08:26:47,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 31784960. Throughput: 0: 1812.8, 1: 1817.9. Samples: 7949272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:47,015][52710] Avg episode reward: [(0, '19.800'), (1, '24.360')] [2023-10-08 08:26:49,737][53885] Updated weights for policy 1, policy_version 15490 (0.0007) [2023-10-08 08:26:49,858][53852] Updated weights for policy 0, policy_version 15560 (0.0007) [2023-10-08 08:26:50,109][53885] Updated weights for policy 1, policy_version 15500 (0.0007) [2023-10-08 08:26:50,229][53852] Updated weights for policy 0, policy_version 15570 (0.0008) [2023-10-08 08:26:50,476][53885] Updated weights for policy 1, policy_version 15510 (0.0007) [2023-10-08 08:26:50,587][53852] Updated weights for policy 0, policy_version 15580 (0.0008) [2023-10-08 08:26:50,836][53885] Updated weights for policy 1, policy_version 15520 (0.0008) [2023-10-08 08:26:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 31850496. Throughput: 0: 1807.8, 1: 1802.0. Samples: 7970426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:26:52,016][52710] Avg episode reward: [(0, '18.570'), (1, '24.120')] [2023-10-08 08:26:54,218][53852] Updated weights for policy 0, policy_version 15590 (0.0007) [2023-10-08 08:26:54,463][53885] Updated weights for policy 1, policy_version 15530 (0.0008) [2023-10-08 08:26:54,598][53852] Updated weights for policy 0, policy_version 15600 (0.0007) [2023-10-08 08:26:54,838][53885] Updated weights for policy 1, policy_version 15540 (0.0007) [2023-10-08 08:26:54,967][53852] Updated weights for policy 0, policy_version 15610 (0.0008) [2023-10-08 08:26:55,206][53885] Updated weights for policy 1, policy_version 15550 (0.0008) [2023-10-08 08:26:57,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 31916032. Throughput: 0: 1814.1, 1: 1815.7. Samples: 7982090. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:26:57,016][52710] Avg episode reward: [(0, '18.360'), (1, '24.840')] [2023-10-08 08:26:58,673][53852] Updated weights for policy 0, policy_version 15620 (0.0007) [2023-10-08 08:26:58,941][53885] Updated weights for policy 1, policy_version 15560 (0.0008) [2023-10-08 08:26:59,046][53852] Updated weights for policy 0, policy_version 15630 (0.0009) [2023-10-08 08:26:59,303][53885] Updated weights for policy 1, policy_version 15570 (0.0007) [2023-10-08 08:26:59,414][53852] Updated weights for policy 0, policy_version 15640 (0.0008) [2023-10-08 08:26:59,675][53885] Updated weights for policy 1, policy_version 15580 (0.0007) [2023-10-08 08:27:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 31981568. Throughput: 0: 1807.2, 1: 1800.8. Samples: 8002890. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:27:02,016][52710] Avg episode reward: [(0, '18.700'), (1, '24.360')] [2023-10-08 08:27:03,228][53852] Updated weights for policy 0, policy_version 15650 (0.0007) [2023-10-08 08:27:03,534][53885] Updated weights for policy 1, policy_version 15590 (0.0008) [2023-10-08 08:27:03,601][53852] Updated weights for policy 0, policy_version 15660 (0.0008) [2023-10-08 08:27:03,905][53885] Updated weights for policy 1, policy_version 15600 (0.0008) [2023-10-08 08:27:03,973][53852] Updated weights for policy 0, policy_version 15670 (0.0009) [2023-10-08 08:27:04,266][53885] Updated weights for policy 1, policy_version 15610 (0.0008) [2023-10-08 08:27:04,340][53852] Updated weights for policy 0, policy_version 15680 (0.0008) [2023-10-08 08:27:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 32047104. Throughput: 0: 1807.5, 1: 1800.3. Samples: 8025430. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:27:07,016][52710] Avg episode reward: [(0, '18.880'), (1, '27.290')] [2023-10-08 08:27:07,031][53594] Saving new best policy, reward=27.290! [2023-10-08 08:27:07,977][53885] Updated weights for policy 1, policy_version 15620 (0.0008) [2023-10-08 08:27:08,110][53852] Updated weights for policy 0, policy_version 15690 (0.0009) [2023-10-08 08:27:08,332][53885] Updated weights for policy 1, policy_version 15630 (0.0007) [2023-10-08 08:27:08,487][53852] Updated weights for policy 0, policy_version 15700 (0.0008) [2023-10-08 08:27:08,704][53885] Updated weights for policy 1, policy_version 15640 (0.0009) [2023-10-08 08:27:08,858][53852] Updated weights for policy 0, policy_version 15710 (0.0010) [2023-10-08 08:27:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 32112640. Throughput: 0: 1806.3, 1: 1801.9. Samples: 8035206. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-10-08 08:27:12,016][52710] Avg episode reward: [(0, '19.100'), (1, '25.730')] [2023-10-08 08:27:12,430][53852] Updated weights for policy 0, policy_version 15720 (0.0007) [2023-10-08 08:27:12,499][53885] Updated weights for policy 1, policy_version 15650 (0.0008) [2023-10-08 08:27:12,800][53852] Updated weights for policy 0, policy_version 15730 (0.0007) [2023-10-08 08:27:12,862][53885] Updated weights for policy 1, policy_version 15660 (0.0007) [2023-10-08 08:27:13,167][53852] Updated weights for policy 0, policy_version 15740 (0.0007) [2023-10-08 08:27:13,232][53885] Updated weights for policy 1, policy_version 15670 (0.0008) [2023-10-08 08:27:13,594][53885] Updated weights for policy 1, policy_version 15680 (0.0009) [2023-10-08 08:27:16,792][53852] Updated weights for policy 0, policy_version 15750 (0.0007) [2023-10-08 08:27:17,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 32178176. Throughput: 0: 1808.1, 1: 1798.4. Samples: 8057958. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-10-08 08:27:17,015][52710] Avg episode reward: [(0, '19.440'), (1, '25.500')] [2023-10-08 08:27:17,167][53852] Updated weights for policy 0, policy_version 15760 (0.0008) [2023-10-08 08:27:17,355][53885] Updated weights for policy 1, policy_version 15690 (0.0008) [2023-10-08 08:27:17,543][53852] Updated weights for policy 0, policy_version 15770 (0.0008) [2023-10-08 08:27:17,722][53885] Updated weights for policy 1, policy_version 15700 (0.0007) [2023-10-08 08:27:18,095][53885] Updated weights for policy 1, policy_version 15710 (0.0010) [2023-10-08 08:27:21,330][53852] Updated weights for policy 0, policy_version 15780 (0.0007) [2023-10-08 08:27:21,722][53852] Updated weights for policy 0, policy_version 15790 (0.0007) [2023-10-08 08:27:21,837][53885] Updated weights for policy 1, policy_version 15720 (0.0008) [2023-10-08 08:27:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 32243712. Throughput: 0: 1819.0, 1: 1808.2. Samples: 8079966. Policy #0 lag: (min: 1.0, avg: 15.6, max: 33.0) [2023-10-08 08:27:22,016][52710] Avg episode reward: [(0, '18.740'), (1, '25.900')] [2023-10-08 08:27:22,101][53852] Updated weights for policy 0, policy_version 15800 (0.0010) [2023-10-08 08:27:22,212][53885] Updated weights for policy 1, policy_version 15730 (0.0009) [2023-10-08 08:27:22,576][53885] Updated weights for policy 1, policy_version 15740 (0.0009) [2023-10-08 08:27:25,649][53852] Updated weights for policy 0, policy_version 15810 (0.0007) [2023-10-08 08:27:26,012][53852] Updated weights for policy 0, policy_version 15820 (0.0007) [2023-10-08 08:27:26,158][53885] Updated weights for policy 1, policy_version 15750 (0.0009) [2023-10-08 08:27:26,378][53852] Updated weights for policy 0, policy_version 15830 (0.0008) [2023-10-08 08:27:26,526][53885] Updated weights for policy 1, policy_version 15760 (0.0008) [2023-10-08 08:27:26,754][53852] Updated weights for policy 0, policy_version 15840 (0.0008) [2023-10-08 08:27:26,891][53885] Updated weights for policy 1, policy_version 15770 (0.0008) [2023-10-08 08:27:27,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 32342016. Throughput: 0: 1813.2, 1: 1801.4. Samples: 8090500. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 08:27:27,016][52710] Avg episode reward: [(0, '18.910'), (1, '25.310')] [2023-10-08 08:27:30,444][53852] Updated weights for policy 0, policy_version 15850 (0.0008) [2023-10-08 08:27:30,703][53885] Updated weights for policy 1, policy_version 15780 (0.0008) [2023-10-08 08:27:30,801][53852] Updated weights for policy 0, policy_version 15860 (0.0007) [2023-10-08 08:27:31,075][53885] Updated weights for policy 1, policy_version 15790 (0.0008) [2023-10-08 08:27:31,177][53852] Updated weights for policy 0, policy_version 15870 (0.0010) [2023-10-08 08:27:31,445][53885] Updated weights for policy 1, policy_version 15800 (0.0009) [2023-10-08 08:27:32,015][52710] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 32440320. Throughput: 0: 1818.8, 1: 1807.2. Samples: 8112444. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 08:27:32,016][52710] Avg episode reward: [(0, '20.330'), (1, '24.560')] [2023-10-08 08:27:34,933][53852] Updated weights for policy 0, policy_version 15880 (0.0010) [2023-10-08 08:27:35,137][53885] Updated weights for policy 1, policy_version 15810 (0.0007) [2023-10-08 08:27:35,302][53852] Updated weights for policy 0, policy_version 15890 (0.0009) [2023-10-08 08:27:35,513][53885] Updated weights for policy 1, policy_version 15820 (0.0008) [2023-10-08 08:27:35,670][53852] Updated weights for policy 0, policy_version 15900 (0.0008) [2023-10-08 08:27:35,873][53885] Updated weights for policy 1, policy_version 15830 (0.0007) [2023-10-08 08:27:36,246][53885] Updated weights for policy 1, policy_version 15840 (0.0007) [2023-10-08 08:27:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 32505856. Throughput: 0: 1816.3, 1: 1791.5. Samples: 8132776. Policy #0 lag: (min: 0.0, avg: 27.8, max: 32.0) [2023-10-08 08:27:37,016][52710] Avg episode reward: [(0, '19.500'), (1, '24.150')] [2023-10-08 08:27:39,250][53852] Updated weights for policy 0, policy_version 15910 (0.0008) [2023-10-08 08:27:39,625][53852] Updated weights for policy 0, policy_version 15920 (0.0008) [2023-10-08 08:27:39,896][53885] Updated weights for policy 1, policy_version 15850 (0.0008) [2023-10-08 08:27:39,992][53852] Updated weights for policy 0, policy_version 15930 (0.0007) [2023-10-08 08:27:40,264][53885] Updated weights for policy 1, policy_version 15860 (0.0007) [2023-10-08 08:27:40,640][53885] Updated weights for policy 1, policy_version 15870 (0.0007) [2023-10-08 08:27:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 32571392. Throughput: 0: 1819.7, 1: 1806.6. Samples: 8145272. Policy #0 lag: (min: 0.0, avg: 27.8, max: 32.0) [2023-10-08 08:27:42,016][52710] Avg episode reward: [(0, '20.730'), (1, '24.460')] [2023-10-08 08:27:43,719][53852] Updated weights for policy 0, policy_version 15940 (0.0008) [2023-10-08 08:27:44,091][53852] Updated weights for policy 0, policy_version 15950 (0.0009) [2023-10-08 08:27:44,418][53885] Updated weights for policy 1, policy_version 15880 (0.0008) [2023-10-08 08:27:44,456][53852] Updated weights for policy 0, policy_version 15960 (0.0007) [2023-10-08 08:27:44,793][53885] Updated weights for policy 1, policy_version 15890 (0.0009) [2023-10-08 08:27:45,158][53885] Updated weights for policy 1, policy_version 15900 (0.0008) [2023-10-08 08:27:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 32636928. Throughput: 0: 1820.0, 1: 1792.4. Samples: 8165448. Policy #0 lag: (min: 0.0, avg: 27.8, max: 32.0) [2023-10-08 08:27:47,016][52710] Avg episode reward: [(0, '20.320'), (1, '28.100')] [2023-10-08 08:27:47,017][53594] Saving new best policy, reward=28.100! [2023-10-08 08:27:48,037][53852] Updated weights for policy 0, policy_version 15970 (0.0007) [2023-10-08 08:27:48,413][53852] Updated weights for policy 0, policy_version 15980 (0.0008) [2023-10-08 08:27:48,784][53852] Updated weights for policy 0, policy_version 15990 (0.0010) [2023-10-08 08:27:48,841][53885] Updated weights for policy 1, policy_version 15910 (0.0008) [2023-10-08 08:27:49,148][53852] Updated weights for policy 0, policy_version 16000 (0.0007) [2023-10-08 08:27:49,210][53885] Updated weights for policy 1, policy_version 15920 (0.0008) [2023-10-08 08:27:49,576][53885] Updated weights for policy 1, policy_version 15930 (0.0011) [2023-10-08 08:27:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 32702464. Throughput: 0: 1827.5, 1: 1794.8. Samples: 8188432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:27:52,016][52710] Avg episode reward: [(0, '20.530'), (1, '27.360')] [2023-10-08 08:27:52,998][53852] Updated weights for policy 0, policy_version 16010 (0.0010) [2023-10-08 08:27:53,338][53885] Updated weights for policy 1, policy_version 15940 (0.0009) [2023-10-08 08:27:53,362][53852] Updated weights for policy 0, policy_version 16020 (0.0007) [2023-10-08 08:27:53,717][53885] Updated weights for policy 1, policy_version 15950 (0.0007) [2023-10-08 08:27:53,719][53852] Updated weights for policy 0, policy_version 16030 (0.0007) [2023-10-08 08:27:54,073][53885] Updated weights for policy 1, policy_version 15960 (0.0010) [2023-10-08 08:27:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 32768000. Throughput: 0: 1829.8, 1: 1797.6. Samples: 8198438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:27:57,016][52710] Avg episode reward: [(0, '20.500'), (1, '25.630')] [2023-10-08 08:27:57,540][53852] Updated weights for policy 0, policy_version 16040 (0.0008) [2023-10-08 08:27:57,740][53885] Updated weights for policy 1, policy_version 15970 (0.0007) [2023-10-08 08:27:57,918][53852] Updated weights for policy 0, policy_version 16050 (0.0009) [2023-10-08 08:27:58,114][53885] Updated weights for policy 1, policy_version 15980 (0.0009) [2023-10-08 08:27:58,283][53852] Updated weights for policy 0, policy_version 16060 (0.0008) [2023-10-08 08:27:58,472][53885] Updated weights for policy 1, policy_version 15990 (0.0009) [2023-10-08 08:27:58,844][53885] Updated weights for policy 1, policy_version 16000 (0.0007) [2023-10-08 08:28:02,015][53852] Updated weights for policy 0, policy_version 16070 (0.0008) [2023-10-08 08:28:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 32833536. Throughput: 0: 1819.1, 1: 1809.8. Samples: 8221262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:28:02,016][52710] Avg episode reward: [(0, '20.340'), (1, '28.280')] [2023-10-08 08:28:02,390][53852] Updated weights for policy 0, policy_version 16080 (0.0008) [2023-10-08 08:28:02,493][53885] Updated weights for policy 1, policy_version 16010 (0.0009) [2023-10-08 08:28:02,757][53852] Updated weights for policy 0, policy_version 16090 (0.0008) [2023-10-08 08:28:02,871][53885] Updated weights for policy 1, policy_version 16020 (0.0009) [2023-10-08 08:28:03,235][53885] Updated weights for policy 1, policy_version 16030 (0.0009) [2023-10-08 08:28:03,304][53594] Saving new best policy, reward=28.280! [2023-10-08 08:28:06,512][53852] Updated weights for policy 0, policy_version 16100 (0.0007) [2023-10-08 08:28:06,893][53852] Updated weights for policy 0, policy_version 16110 (0.0008) [2023-10-08 08:28:06,994][53885] Updated weights for policy 1, policy_version 16040 (0.0008) [2023-10-08 08:28:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32899072. Throughput: 0: 1821.2, 1: 1813.1. Samples: 8243510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:28:07,015][52710] Avg episode reward: [(0, '20.820'), (1, '25.520')] [2023-10-08 08:28:07,261][53852] Updated weights for policy 0, policy_version 16120 (0.0008) [2023-10-08 08:28:07,356][53885] Updated weights for policy 1, policy_version 16050 (0.0007) [2023-10-08 08:28:07,552][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000016128_16515072.pth... [2023-10-08 08:28:07,590][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000014400_14745600.pth [2023-10-08 08:28:07,722][53885] Updated weights for policy 1, policy_version 16060 (0.0007) [2023-10-08 08:28:07,859][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000016064_16449536.pth... [2023-10-08 08:28:07,898][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000014368_14712832.pth [2023-10-08 08:28:10,971][53852] Updated weights for policy 0, policy_version 16130 (0.0009) [2023-10-08 08:28:11,345][53852] Updated weights for policy 0, policy_version 16140 (0.0008) [2023-10-08 08:28:11,377][53885] Updated weights for policy 1, policy_version 16070 (0.0008) [2023-10-08 08:28:11,720][53852] Updated weights for policy 0, policy_version 16150 (0.0008) [2023-10-08 08:28:11,742][53885] Updated weights for policy 1, policy_version 16080 (0.0007) [2023-10-08 08:28:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32964608. Throughput: 0: 1814.3, 1: 1809.5. Samples: 8253570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:28:12,016][52710] Avg episode reward: [(0, '20.140'), (1, '23.740')] [2023-10-08 08:28:12,082][53852] Updated weights for policy 0, policy_version 16160 (0.0008) [2023-10-08 08:28:12,105][53885] Updated weights for policy 1, policy_version 16090 (0.0007) [2023-10-08 08:28:15,614][53852] Updated weights for policy 0, policy_version 16170 (0.0009) [2023-10-08 08:28:15,827][53885] Updated weights for policy 1, policy_version 16100 (0.0009) [2023-10-08 08:28:15,984][53852] Updated weights for policy 0, policy_version 16180 (0.0009) [2023-10-08 08:28:16,201][53885] Updated weights for policy 1, policy_version 16110 (0.0008) [2023-10-08 08:28:16,353][53852] Updated weights for policy 0, policy_version 16190 (0.0007) [2023-10-08 08:28:16,559][53885] Updated weights for policy 1, policy_version 16120 (0.0009) [2023-10-08 08:28:17,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 33095680. Throughput: 0: 1816.4, 1: 1818.8. Samples: 8276028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:28:17,016][52710] Avg episode reward: [(0, '20.890'), (1, '27.360')] [2023-10-08 08:28:19,993][53852] Updated weights for policy 0, policy_version 16200 (0.0009) [2023-10-08 08:28:20,219][53885] Updated weights for policy 1, policy_version 16130 (0.0010) [2023-10-08 08:28:20,358][53852] Updated weights for policy 0, policy_version 16210 (0.0007) [2023-10-08 08:28:20,589][53885] Updated weights for policy 1, policy_version 16140 (0.0007) [2023-10-08 08:28:20,732][53852] Updated weights for policy 0, policy_version 16220 (0.0008) [2023-10-08 08:28:20,953][53885] Updated weights for policy 1, policy_version 16150 (0.0007) [2023-10-08 08:28:21,322][53885] Updated weights for policy 1, policy_version 16160 (0.0008) [2023-10-08 08:28:22,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 33161216. Throughput: 0: 1813.8, 1: 1817.7. Samples: 8296192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:28:22,016][52710] Avg episode reward: [(0, '19.950'), (1, '28.590')] [2023-10-08 08:28:22,024][53594] Saving new best policy, reward=28.590! [2023-10-08 08:28:24,523][53852] Updated weights for policy 0, policy_version 16230 (0.0009) [2023-10-08 08:28:24,890][53852] Updated weights for policy 0, policy_version 16240 (0.0009) [2023-10-08 08:28:25,125][53885] Updated weights for policy 1, policy_version 16170 (0.0009) [2023-10-08 08:28:25,270][53852] Updated weights for policy 0, policy_version 16250 (0.0007) [2023-10-08 08:28:25,485][53885] Updated weights for policy 1, policy_version 16180 (0.0007) [2023-10-08 08:28:25,845][53885] Updated weights for policy 1, policy_version 16190 (0.0008) [2023-10-08 08:28:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33226752. Throughput: 0: 1814.7, 1: 1817.3. Samples: 8308714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:28:27,016][52710] Avg episode reward: [(0, '21.030'), (1, '27.200')] [2023-10-08 08:28:29,086][53852] Updated weights for policy 0, policy_version 16260 (0.0008) [2023-10-08 08:28:29,452][53885] Updated weights for policy 1, policy_version 16200 (0.0007) [2023-10-08 08:28:29,455][53852] Updated weights for policy 0, policy_version 16270 (0.0008) [2023-10-08 08:28:29,816][53885] Updated weights for policy 1, policy_version 16210 (0.0008) [2023-10-08 08:28:29,821][53852] Updated weights for policy 0, policy_version 16280 (0.0007) [2023-10-08 08:28:30,187][53885] Updated weights for policy 1, policy_version 16220 (0.0010) [2023-10-08 08:28:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 33292288. Throughput: 0: 1810.8, 1: 1823.6. Samples: 8328998. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 08:28:32,016][52710] Avg episode reward: [(0, '19.950'), (1, '28.190')] [2023-10-08 08:28:33,334][53852] Updated weights for policy 0, policy_version 16290 (0.0008) [2023-10-08 08:28:33,702][53852] Updated weights for policy 0, policy_version 16300 (0.0007) [2023-10-08 08:28:33,945][53885] Updated weights for policy 1, policy_version 16230 (0.0009) [2023-10-08 08:28:34,074][53852] Updated weights for policy 0, policy_version 16310 (0.0008) [2023-10-08 08:28:34,312][53885] Updated weights for policy 1, policy_version 16240 (0.0007) [2023-10-08 08:28:34,434][53852] Updated weights for policy 0, policy_version 16320 (0.0009) [2023-10-08 08:28:34,682][53885] Updated weights for policy 1, policy_version 16250 (0.0009) [2023-10-08 08:28:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 33357824. Throughput: 0: 1808.8, 1: 1825.1. Samples: 8351958. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 08:28:37,016][52710] Avg episode reward: [(0, '18.970'), (1, '26.480')] [2023-10-08 08:28:38,049][53852] Updated weights for policy 0, policy_version 16330 (0.0010) [2023-10-08 08:28:38,332][53885] Updated weights for policy 1, policy_version 16260 (0.0008) [2023-10-08 08:28:38,426][53852] Updated weights for policy 0, policy_version 16340 (0.0007) [2023-10-08 08:28:38,695][53885] Updated weights for policy 1, policy_version 16270 (0.0007) [2023-10-08 08:28:38,786][53852] Updated weights for policy 0, policy_version 16350 (0.0008) [2023-10-08 08:28:39,067][53885] Updated weights for policy 1, policy_version 16280 (0.0009) [2023-10-08 08:28:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 33423360. Throughput: 0: 1811.2, 1: 1820.2. Samples: 8361854. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 08:28:42,016][52710] Avg episode reward: [(0, '20.160'), (1, '24.190')] [2023-10-08 08:28:42,521][53852] Updated weights for policy 0, policy_version 16360 (0.0009) [2023-10-08 08:28:42,764][53885] Updated weights for policy 1, policy_version 16290 (0.0010) [2023-10-08 08:28:42,891][53852] Updated weights for policy 0, policy_version 16370 (0.0008) [2023-10-08 08:28:43,132][53885] Updated weights for policy 1, policy_version 16300 (0.0008) [2023-10-08 08:28:43,250][53852] Updated weights for policy 0, policy_version 16380 (0.0008) [2023-10-08 08:28:43,504][53885] Updated weights for policy 1, policy_version 16310 (0.0011) [2023-10-08 08:28:43,869][53885] Updated weights for policy 1, policy_version 16320 (0.0010) [2023-10-08 08:28:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 33488896. Throughput: 0: 1815.4, 1: 1814.4. Samples: 8384600. Policy #0 lag: (min: 15.0, avg: 36.3, max: 40.0) [2023-10-08 08:28:47,016][52710] Avg episode reward: [(0, '21.060'), (1, '22.970')] [2023-10-08 08:28:47,033][53852] Updated weights for policy 0, policy_version 16390 (0.0008) [2023-10-08 08:28:47,402][53852] Updated weights for policy 0, policy_version 16400 (0.0009) [2023-10-08 08:28:47,550][53885] Updated weights for policy 1, policy_version 16330 (0.0010) [2023-10-08 08:28:47,770][53852] Updated weights for policy 0, policy_version 16410 (0.0009) [2023-10-08 08:28:47,921][53885] Updated weights for policy 1, policy_version 16340 (0.0008) [2023-10-08 08:28:48,290][53885] Updated weights for policy 1, policy_version 16350 (0.0008) [2023-10-08 08:28:51,545][53852] Updated weights for policy 0, policy_version 16420 (0.0007) [2023-10-08 08:28:51,926][53852] Updated weights for policy 0, policy_version 16430 (0.0008) [2023-10-08 08:28:52,011][53885] Updated weights for policy 1, policy_version 16360 (0.0008) [2023-10-08 08:28:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 33554432. Throughput: 0: 1814.3, 1: 1817.7. Samples: 8406948. Policy #0 lag: (min: 15.0, avg: 36.3, max: 40.0) [2023-10-08 08:28:52,016][52710] Avg episode reward: [(0, '20.320'), (1, '25.190')] [2023-10-08 08:28:52,297][53852] Updated weights for policy 0, policy_version 16440 (0.0007) [2023-10-08 08:28:52,378][53885] Updated weights for policy 1, policy_version 16370 (0.0009) [2023-10-08 08:28:52,745][53885] Updated weights for policy 1, policy_version 16380 (0.0010) [2023-10-08 08:28:55,839][53852] Updated weights for policy 0, policy_version 16450 (0.0007) [2023-10-08 08:28:56,212][53852] Updated weights for policy 0, policy_version 16460 (0.0007) [2023-10-08 08:28:56,339][53885] Updated weights for policy 1, policy_version 16390 (0.0008) [2023-10-08 08:28:56,585][53852] Updated weights for policy 0, policy_version 16470 (0.0008) [2023-10-08 08:28:56,703][53885] Updated weights for policy 1, policy_version 16400 (0.0008) [2023-10-08 08:28:56,959][53852] Updated weights for policy 0, policy_version 16480 (0.0008) [2023-10-08 08:28:57,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33652736. Throughput: 0: 1821.0, 1: 1816.8. Samples: 8417270. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:28:57,016][52710] Avg episode reward: [(0, '21.660'), (1, '24.240')] [2023-10-08 08:28:57,072][53885] Updated weights for policy 1, policy_version 16410 (0.0007) [2023-10-08 08:29:00,552][53852] Updated weights for policy 0, policy_version 16490 (0.0011) [2023-10-08 08:29:00,733][53885] Updated weights for policy 1, policy_version 16420 (0.0008) [2023-10-08 08:29:00,912][53852] Updated weights for policy 0, policy_version 16500 (0.0008) [2023-10-08 08:29:01,098][53885] Updated weights for policy 1, policy_version 16430 (0.0008) [2023-10-08 08:29:01,288][53852] Updated weights for policy 0, policy_version 16510 (0.0008) [2023-10-08 08:29:01,461][53885] Updated weights for policy 1, policy_version 16440 (0.0008) [2023-10-08 08:29:02,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 33751040. Throughput: 0: 1821.8, 1: 1814.1. Samples: 8439644. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:29:02,016][52710] Avg episode reward: [(0, '22.610'), (1, '27.040')] [2023-10-08 08:29:02,018][53500] Saving new best policy, reward=22.610! [2023-10-08 08:29:04,910][53852] Updated weights for policy 0, policy_version 16520 (0.0008) [2023-10-08 08:29:05,169][53885] Updated weights for policy 1, policy_version 16450 (0.0008) [2023-10-08 08:29:05,280][53852] Updated weights for policy 0, policy_version 16530 (0.0009) [2023-10-08 08:29:05,536][53885] Updated weights for policy 1, policy_version 16460 (0.0009) [2023-10-08 08:29:05,648][53852] Updated weights for policy 0, policy_version 16540 (0.0009) [2023-10-08 08:29:05,907][53885] Updated weights for policy 1, policy_version 16470 (0.0009) [2023-10-08 08:29:06,269][53885] Updated weights for policy 1, policy_version 16480 (0.0008) [2023-10-08 08:29:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 33816576. Throughput: 0: 1819.2, 1: 1814.1. Samples: 8459692. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:29:07,016][52710] Avg episode reward: [(0, '20.330'), (1, '25.290')] [2023-10-08 08:29:09,361][53852] Updated weights for policy 0, policy_version 16550 (0.0007) [2023-10-08 08:29:09,741][53852] Updated weights for policy 0, policy_version 16560 (0.0008) [2023-10-08 08:29:10,054][53885] Updated weights for policy 1, policy_version 16490 (0.0008) [2023-10-08 08:29:10,111][53852] Updated weights for policy 0, policy_version 16570 (0.0008) [2023-10-08 08:29:10,425][53885] Updated weights for policy 1, policy_version 16500 (0.0009) [2023-10-08 08:29:10,794][53885] Updated weights for policy 1, policy_version 16510 (0.0009) [2023-10-08 08:29:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 33882112. Throughput: 0: 1821.1, 1: 1812.9. Samples: 8472248. Policy #0 lag: (min: 9.0, avg: 17.2, max: 41.0) [2023-10-08 08:29:12,016][52710] Avg episode reward: [(0, '21.140'), (1, '26.940')] [2023-10-08 08:29:13,792][53852] Updated weights for policy 0, policy_version 16580 (0.0007) [2023-10-08 08:29:14,154][53852] Updated weights for policy 0, policy_version 16590 (0.0008) [2023-10-08 08:29:14,515][53852] Updated weights for policy 0, policy_version 16600 (0.0008) [2023-10-08 08:29:14,705][53885] Updated weights for policy 1, policy_version 16520 (0.0008) [2023-10-08 08:29:15,076][53885] Updated weights for policy 1, policy_version 16530 (0.0009) [2023-10-08 08:29:15,451][53885] Updated weights for policy 1, policy_version 16540 (0.0009) [2023-10-08 08:29:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 33947648. Throughput: 0: 1822.3, 1: 1809.4. Samples: 8492426. Policy #0 lag: (min: 9.0, avg: 17.2, max: 41.0) [2023-10-08 08:29:17,015][52710] Avg episode reward: [(0, '19.840'), (1, '24.270')] [2023-10-08 08:29:18,070][53852] Updated weights for policy 0, policy_version 16610 (0.0008) [2023-10-08 08:29:18,439][53852] Updated weights for policy 0, policy_version 16620 (0.0008) [2023-10-08 08:29:18,812][53852] Updated weights for policy 0, policy_version 16630 (0.0007) [2023-10-08 08:29:19,042][53885] Updated weights for policy 1, policy_version 16550 (0.0008) [2023-10-08 08:29:19,176][53852] Updated weights for policy 0, policy_version 16640 (0.0007) [2023-10-08 08:29:19,407][53885] Updated weights for policy 1, policy_version 16560 (0.0008) [2023-10-08 08:29:19,785][53885] Updated weights for policy 1, policy_version 16570 (0.0008) [2023-10-08 08:29:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 34013184. Throughput: 0: 1827.5, 1: 1810.0. Samples: 8515646. Policy #0 lag: (min: 9.0, avg: 17.2, max: 41.0) [2023-10-08 08:29:22,016][52710] Avg episode reward: [(0, '20.060'), (1, '26.500')] [2023-10-08 08:29:22,911][53852] Updated weights for policy 0, policy_version 16650 (0.0008) [2023-10-08 08:29:23,277][53852] Updated weights for policy 0, policy_version 16660 (0.0007) [2023-10-08 08:29:23,455][53885] Updated weights for policy 1, policy_version 16580 (0.0008) [2023-10-08 08:29:23,648][53852] Updated weights for policy 0, policy_version 16670 (0.0008) [2023-10-08 08:29:23,813][53885] Updated weights for policy 1, policy_version 16590 (0.0007) [2023-10-08 08:29:24,186][53885] Updated weights for policy 1, policy_version 16600 (0.0007) [2023-10-08 08:29:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34078720. Throughput: 0: 1824.3, 1: 1812.4. Samples: 8525506. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 08:29:27,016][52710] Avg episode reward: [(0, '20.320'), (1, '26.640')] [2023-10-08 08:29:27,296][53852] Updated weights for policy 0, policy_version 16680 (0.0007) [2023-10-08 08:29:27,662][53852] Updated weights for policy 0, policy_version 16690 (0.0008) [2023-10-08 08:29:27,949][53885] Updated weights for policy 1, policy_version 16610 (0.0008) [2023-10-08 08:29:28,038][53852] Updated weights for policy 0, policy_version 16700 (0.0007) [2023-10-08 08:29:28,312][53885] Updated weights for policy 1, policy_version 16620 (0.0009) [2023-10-08 08:29:28,691][53885] Updated weights for policy 1, policy_version 16630 (0.0010) [2023-10-08 08:29:29,051][53885] Updated weights for policy 1, policy_version 16640 (0.0007) [2023-10-08 08:29:31,746][53852] Updated weights for policy 0, policy_version 16710 (0.0008) [2023-10-08 08:29:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34144256. Throughput: 0: 1828.3, 1: 1816.9. Samples: 8548634. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 08:29:32,016][52710] Avg episode reward: [(0, '20.800'), (1, '24.410')] [2023-10-08 08:29:32,125][53852] Updated weights for policy 0, policy_version 16720 (0.0007) [2023-10-08 08:29:32,500][53852] Updated weights for policy 0, policy_version 16730 (0.0008) [2023-10-08 08:29:32,661][53885] Updated weights for policy 1, policy_version 16650 (0.0007) [2023-10-08 08:29:33,034][53885] Updated weights for policy 1, policy_version 16660 (0.0007) [2023-10-08 08:29:33,398][53885] Updated weights for policy 1, policy_version 16670 (0.0008) [2023-10-08 08:29:36,123][53852] Updated weights for policy 0, policy_version 16740 (0.0007) [2023-10-08 08:29:36,505][53852] Updated weights for policy 0, policy_version 16750 (0.0009) [2023-10-08 08:29:36,864][53852] Updated weights for policy 0, policy_version 16760 (0.0008) [2023-10-08 08:29:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34209792. Throughput: 0: 1824.0, 1: 1819.5. Samples: 8570908. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 08:29:37,016][52710] Avg episode reward: [(0, '20.760'), (1, '26.290')] [2023-10-08 08:29:37,035][53885] Updated weights for policy 1, policy_version 16680 (0.0007) [2023-10-08 08:29:37,402][53885] Updated weights for policy 1, policy_version 16690 (0.0008) [2023-10-08 08:29:37,779][53885] Updated weights for policy 1, policy_version 16700 (0.0009) [2023-10-08 08:29:40,447][53852] Updated weights for policy 0, policy_version 16770 (0.0007) [2023-10-08 08:29:40,836][53852] Updated weights for policy 0, policy_version 16780 (0.0007) [2023-10-08 08:29:41,200][53852] Updated weights for policy 0, policy_version 16790 (0.0008) [2023-10-08 08:29:41,359][53885] Updated weights for policy 1, policy_version 16710 (0.0008) [2023-10-08 08:29:41,568][53852] Updated weights for policy 0, policy_version 16800 (0.0009) [2023-10-08 08:29:41,721][53885] Updated weights for policy 1, policy_version 16720 (0.0009) [2023-10-08 08:29:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34308096. Throughput: 0: 1834.9, 1: 1822.0. Samples: 8581830. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 08:29:42,016][52710] Avg episode reward: [(0, '20.120'), (1, '26.860')] [2023-10-08 08:29:42,091][53885] Updated weights for policy 1, policy_version 16730 (0.0010) [2023-10-08 08:29:45,295][53852] Updated weights for policy 0, policy_version 16810 (0.0007) [2023-10-08 08:29:45,660][53852] Updated weights for policy 0, policy_version 16820 (0.0011) [2023-10-08 08:29:45,721][53885] Updated weights for policy 1, policy_version 16740 (0.0009) [2023-10-08 08:29:46,023][53852] Updated weights for policy 0, policy_version 16830 (0.0007) [2023-10-08 08:29:46,087][53885] Updated weights for policy 1, policy_version 16750 (0.0007) [2023-10-08 08:29:46,457][53885] Updated weights for policy 1, policy_version 16760 (0.0010) [2023-10-08 08:29:47,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 34406400. Throughput: 0: 1829.3, 1: 1826.7. Samples: 8604164. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) [2023-10-08 08:29:47,016][52710] Avg episode reward: [(0, '21.820'), (1, '24.210')] [2023-10-08 08:29:49,718][53852] Updated weights for policy 0, policy_version 16840 (0.0008) [2023-10-08 08:29:50,101][53852] Updated weights for policy 0, policy_version 16850 (0.0009) [2023-10-08 08:29:50,257][53885] Updated weights for policy 1, policy_version 16770 (0.0010) [2023-10-08 08:29:50,473][53852] Updated weights for policy 0, policy_version 16860 (0.0009) [2023-10-08 08:29:50,631][53885] Updated weights for policy 1, policy_version 16780 (0.0009) [2023-10-08 08:29:50,991][53885] Updated weights for policy 1, policy_version 16790 (0.0008) [2023-10-08 08:29:51,364][53885] Updated weights for policy 1, policy_version 16800 (0.0007) [2023-10-08 08:29:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 34471936. Throughput: 0: 1835.8, 1: 1827.7. Samples: 8624550. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 08:29:52,016][52710] Avg episode reward: [(0, '21.280'), (1, '26.610')] [2023-10-08 08:29:54,162][53852] Updated weights for policy 0, policy_version 16870 (0.0008) [2023-10-08 08:29:54,530][53852] Updated weights for policy 0, policy_version 16880 (0.0009) [2023-10-08 08:29:54,901][53852] Updated weights for policy 0, policy_version 16890 (0.0008) [2023-10-08 08:29:55,144][53885] Updated weights for policy 1, policy_version 16810 (0.0007) [2023-10-08 08:29:55,513][53885] Updated weights for policy 1, policy_version 16820 (0.0009) [2023-10-08 08:29:55,882][53885] Updated weights for policy 1, policy_version 16830 (0.0009) [2023-10-08 08:29:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 34537472. Throughput: 0: 1829.8, 1: 1828.4. Samples: 8636868. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 08:29:57,015][52710] Avg episode reward: [(0, '21.420'), (1, '24.020')] [2023-10-08 08:29:58,676][53852] Updated weights for policy 0, policy_version 16900 (0.0008) [2023-10-08 08:29:59,045][53852] Updated weights for policy 0, policy_version 16910 (0.0010) [2023-10-08 08:29:59,416][53852] Updated weights for policy 0, policy_version 16920 (0.0007) [2023-10-08 08:29:59,648][53885] Updated weights for policy 1, policy_version 16840 (0.0007) [2023-10-08 08:30:00,013][53885] Updated weights for policy 1, policy_version 16850 (0.0007) [2023-10-08 08:30:00,384][53885] Updated weights for policy 1, policy_version 16860 (0.0008) [2023-10-08 08:30:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 34603008. Throughput: 0: 1832.9, 1: 1828.2. Samples: 8657176. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 08:30:02,015][52710] Avg episode reward: [(0, '21.060'), (1, '26.020')] [2023-10-08 08:30:03,184][53852] Updated weights for policy 0, policy_version 16930 (0.0007) [2023-10-08 08:30:03,558][53852] Updated weights for policy 0, policy_version 16940 (0.0007) [2023-10-08 08:30:03,939][53852] Updated weights for policy 0, policy_version 16950 (0.0007) [2023-10-08 08:30:03,990][53885] Updated weights for policy 1, policy_version 16870 (0.0009) [2023-10-08 08:30:04,303][53852] Updated weights for policy 0, policy_version 16960 (0.0008) [2023-10-08 08:30:04,356][53885] Updated weights for policy 1, policy_version 16880 (0.0010) [2023-10-08 08:30:04,730][53885] Updated weights for policy 1, policy_version 16890 (0.0010) [2023-10-08 08:30:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34668544. Throughput: 0: 1825.8, 1: 1825.4. Samples: 8679950. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 08:30:07,016][52710] Avg episode reward: [(0, '20.730'), (1, '24.890')] [2023-10-08 08:30:07,023][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000016960_17367040.pth... [2023-10-08 08:30:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000016896_17301504.pth... [2023-10-08 08:30:07,053][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000015200_15564800.pth [2023-10-08 08:30:07,057][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000016896_17301504.pth [2023-10-08 08:30:07,064][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000015264_15630336.pth [2023-10-08 08:30:07,069][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000016960_17367040.pth [2023-10-08 08:30:07,899][53852] Updated weights for policy 0, policy_version 16970 (0.0009) [2023-10-08 08:30:08,274][53852] Updated weights for policy 0, policy_version 16980 (0.0009) [2023-10-08 08:30:08,320][53885] Updated weights for policy 1, policy_version 16900 (0.0009) [2023-10-08 08:30:08,660][53852] Updated weights for policy 0, policy_version 16990 (0.0009) [2023-10-08 08:30:08,693][53885] Updated weights for policy 1, policy_version 16910 (0.0008) [2023-10-08 08:30:09,057][53885] Updated weights for policy 1, policy_version 16920 (0.0007) [2023-10-08 08:30:12,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34734080. Throughput: 0: 1826.6, 1: 1831.1. Samples: 8690104. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 08:30:12,016][52710] Avg episode reward: [(0, '21.310'), (1, '26.850')] [2023-10-08 08:30:12,444][53852] Updated weights for policy 0, policy_version 17000 (0.0007) [2023-10-08 08:30:12,810][53852] Updated weights for policy 0, policy_version 17010 (0.0007) [2023-10-08 08:30:12,826][53885] Updated weights for policy 1, policy_version 16930 (0.0007) [2023-10-08 08:30:13,171][53852] Updated weights for policy 0, policy_version 17020 (0.0009) [2023-10-08 08:30:13,189][53885] Updated weights for policy 1, policy_version 16940 (0.0007) [2023-10-08 08:30:13,556][53885] Updated weights for policy 1, policy_version 16950 (0.0007) [2023-10-08 08:30:13,923][53885] Updated weights for policy 1, policy_version 16960 (0.0007) [2023-10-08 08:30:16,813][53852] Updated weights for policy 0, policy_version 17030 (0.0008) [2023-10-08 08:30:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34799616. Throughput: 0: 1819.9, 1: 1825.2. Samples: 8712662. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) [2023-10-08 08:30:17,015][52710] Avg episode reward: [(0, '21.620'), (1, '27.270')] [2023-10-08 08:30:17,183][53852] Updated weights for policy 0, policy_version 17040 (0.0007) [2023-10-08 08:30:17,553][53852] Updated weights for policy 0, policy_version 17050 (0.0007) [2023-10-08 08:30:17,660][53885] Updated weights for policy 1, policy_version 16970 (0.0007) [2023-10-08 08:30:18,023][53885] Updated weights for policy 1, policy_version 16980 (0.0008) [2023-10-08 08:30:18,395][53885] Updated weights for policy 1, policy_version 16990 (0.0007) [2023-10-08 08:30:21,230][53852] Updated weights for policy 0, policy_version 17060 (0.0009) [2023-10-08 08:30:21,603][53852] Updated weights for policy 0, policy_version 17070 (0.0008) [2023-10-08 08:30:21,970][53852] Updated weights for policy 0, policy_version 17080 (0.0007) [2023-10-08 08:30:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34865152. Throughput: 0: 1819.6, 1: 1816.5. Samples: 8734534. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 08:30:22,016][52710] Avg episode reward: [(0, '21.840'), (1, '26.110')] [2023-10-08 08:30:22,038][53885] Updated weights for policy 1, policy_version 17000 (0.0008) [2023-10-08 08:30:22,402][53885] Updated weights for policy 1, policy_version 17010 (0.0007) [2023-10-08 08:30:22,767][53885] Updated weights for policy 1, policy_version 17020 (0.0007) [2023-10-08 08:30:25,746][53852] Updated weights for policy 0, policy_version 17090 (0.0007) [2023-10-08 08:30:26,146][53852] Updated weights for policy 0, policy_version 17100 (0.0009) [2023-10-08 08:30:26,294][53885] Updated weights for policy 1, policy_version 17030 (0.0008) [2023-10-08 08:30:26,524][53852] Updated weights for policy 0, policy_version 17110 (0.0007) [2023-10-08 08:30:26,655][53885] Updated weights for policy 1, policy_version 17040 (0.0008) [2023-10-08 08:30:26,891][53852] Updated weights for policy 0, policy_version 17120 (0.0007) [2023-10-08 08:30:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34963456. Throughput: 0: 1812.4, 1: 1818.8. Samples: 8745236. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 08:30:27,016][52710] Avg episode reward: [(0, '20.910'), (1, '28.450')] [2023-10-08 08:30:27,019][53885] Updated weights for policy 1, policy_version 17050 (0.0009) [2023-10-08 08:30:30,583][53852] Updated weights for policy 0, policy_version 17130 (0.0010) [2023-10-08 08:30:30,698][53885] Updated weights for policy 1, policy_version 17060 (0.0008) [2023-10-08 08:30:30,949][53852] Updated weights for policy 0, policy_version 17140 (0.0008) [2023-10-08 08:30:31,067][53885] Updated weights for policy 1, policy_version 17070 (0.0008) [2023-10-08 08:30:31,317][53852] Updated weights for policy 0, policy_version 17150 (0.0008) [2023-10-08 08:30:31,438][53885] Updated weights for policy 1, policy_version 17080 (0.0007) [2023-10-08 08:30:32,015][52710] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 35061760. Throughput: 0: 1817.3, 1: 1817.8. Samples: 8767742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-08 08:30:32,016][52710] Avg episode reward: [(0, '21.070'), (1, '27.040')] [2023-10-08 08:30:34,867][53852] Updated weights for policy 0, policy_version 17160 (0.0007) [2023-10-08 08:30:35,153][53885] Updated weights for policy 1, policy_version 17090 (0.0009) [2023-10-08 08:30:35,236][53852] Updated weights for policy 0, policy_version 17170 (0.0009) [2023-10-08 08:30:35,514][53885] Updated weights for policy 1, policy_version 17100 (0.0007) [2023-10-08 08:30:35,607][53852] Updated weights for policy 0, policy_version 17180 (0.0008) [2023-10-08 08:30:35,889][53885] Updated weights for policy 1, policy_version 17110 (0.0007) [2023-10-08 08:30:36,255][53885] Updated weights for policy 1, policy_version 17120 (0.0009) [2023-10-08 08:30:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 35127296. Throughput: 0: 1820.4, 1: 1824.3. Samples: 8788564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-08 08:30:37,016][52710] Avg episode reward: [(0, '20.870'), (1, '26.320')] [2023-10-08 08:30:39,177][53852] Updated weights for policy 0, policy_version 17190 (0.0008) [2023-10-08 08:30:39,546][53852] Updated weights for policy 0, policy_version 17200 (0.0008) [2023-10-08 08:30:39,916][53852] Updated weights for policy 0, policy_version 17210 (0.0007) [2023-10-08 08:30:39,995][53885] Updated weights for policy 1, policy_version 17130 (0.0008) [2023-10-08 08:30:40,367][53885] Updated weights for policy 1, policy_version 17140 (0.0009) [2023-10-08 08:30:40,729][53885] Updated weights for policy 1, policy_version 17150 (0.0010) [2023-10-08 08:30:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 35192832. Throughput: 0: 1822.4, 1: 1819.9. Samples: 8800774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) [2023-10-08 08:30:42,016][52710] Avg episode reward: [(0, '20.680'), (1, '26.960')] [2023-10-08 08:30:43,548][53852] Updated weights for policy 0, policy_version 17220 (0.0008) [2023-10-08 08:30:43,917][53852] Updated weights for policy 0, policy_version 17230 (0.0011) [2023-10-08 08:30:44,287][53852] Updated weights for policy 0, policy_version 17240 (0.0008) [2023-10-08 08:30:44,418][53885] Updated weights for policy 1, policy_version 17160 (0.0009) [2023-10-08 08:30:44,790][53885] Updated weights for policy 1, policy_version 17170 (0.0010) [2023-10-08 08:30:45,154][53885] Updated weights for policy 1, policy_version 17180 (0.0011) [2023-10-08 08:30:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35258368. Throughput: 0: 1827.9, 1: 1824.2. Samples: 8821522. Policy #0 lag: (min: 26.0, avg: 26.6, max: 42.0) [2023-10-08 08:30:47,016][52710] Avg episode reward: [(0, '19.060'), (1, '24.590')] [2023-10-08 08:30:47,973][53852] Updated weights for policy 0, policy_version 17250 (0.0007) [2023-10-08 08:30:48,347][53852] Updated weights for policy 0, policy_version 17260 (0.0010) [2023-10-08 08:30:48,684][53885] Updated weights for policy 1, policy_version 17190 (0.0009) [2023-10-08 08:30:48,718][53852] Updated weights for policy 0, policy_version 17270 (0.0007) [2023-10-08 08:30:49,050][53885] Updated weights for policy 1, policy_version 17200 (0.0007) [2023-10-08 08:30:49,087][53852] Updated weights for policy 0, policy_version 17280 (0.0010) [2023-10-08 08:30:49,423][53885] Updated weights for policy 1, policy_version 17210 (0.0010) [2023-10-08 08:30:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 35323904. Throughput: 0: 1833.5, 1: 1829.8. Samples: 8844796. Policy #0 lag: (min: 26.0, avg: 26.6, max: 42.0) [2023-10-08 08:30:52,016][52710] Avg episode reward: [(0, '20.650'), (1, '25.900')] [2023-10-08 08:30:52,828][53852] Updated weights for policy 0, policy_version 17290 (0.0007) [2023-10-08 08:30:53,105][53885] Updated weights for policy 1, policy_version 17220 (0.0007) [2023-10-08 08:30:53,196][53852] Updated weights for policy 0, policy_version 17300 (0.0007) [2023-10-08 08:30:53,484][53885] Updated weights for policy 1, policy_version 17230 (0.0009) [2023-10-08 08:30:53,559][53852] Updated weights for policy 0, policy_version 17310 (0.0008) [2023-10-08 08:30:53,855][53885] Updated weights for policy 1, policy_version 17240 (0.0008) [2023-10-08 08:30:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35389440. Throughput: 0: 1835.8, 1: 1826.6. Samples: 8854912. Policy #0 lag: (min: 26.0, avg: 26.6, max: 42.0) [2023-10-08 08:30:57,016][52710] Avg episode reward: [(0, '20.560'), (1, '25.910')] [2023-10-08 08:30:57,174][53852] Updated weights for policy 0, policy_version 17320 (0.0007) [2023-10-08 08:30:57,523][53885] Updated weights for policy 1, policy_version 17250 (0.0008) [2023-10-08 08:30:57,544][53852] Updated weights for policy 0, policy_version 17330 (0.0007) [2023-10-08 08:30:57,897][53885] Updated weights for policy 1, policy_version 17260 (0.0008) [2023-10-08 08:30:57,903][53852] Updated weights for policy 0, policy_version 17340 (0.0007) [2023-10-08 08:30:58,270][53885] Updated weights for policy 1, policy_version 17270 (0.0009) [2023-10-08 08:30:58,633][53885] Updated weights for policy 1, policy_version 17280 (0.0008) [2023-10-08 08:31:01,719][53852] Updated weights for policy 0, policy_version 17350 (0.0009) [2023-10-08 08:31:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 35454976. Throughput: 0: 1841.2, 1: 1827.9. Samples: 8877774. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 08:31:02,016][52710] Avg episode reward: [(0, '19.410'), (1, '25.640')] [2023-10-08 08:31:02,085][53852] Updated weights for policy 0, policy_version 17360 (0.0007) [2023-10-08 08:31:02,410][53885] Updated weights for policy 1, policy_version 17290 (0.0009) [2023-10-08 08:31:02,452][53852] Updated weights for policy 0, policy_version 17370 (0.0008) [2023-10-08 08:31:02,781][53885] Updated weights for policy 1, policy_version 17300 (0.0007) [2023-10-08 08:31:03,149][53885] Updated weights for policy 1, policy_version 17310 (0.0010) [2023-10-08 08:31:06,003][53852] Updated weights for policy 0, policy_version 17380 (0.0007) [2023-10-08 08:31:06,380][53852] Updated weights for policy 0, policy_version 17390 (0.0007) [2023-10-08 08:31:06,751][53852] Updated weights for policy 0, policy_version 17400 (0.0007) [2023-10-08 08:31:06,879][53885] Updated weights for policy 1, policy_version 17320 (0.0008) [2023-10-08 08:31:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35520512. Throughput: 0: 1836.0, 1: 1830.5. Samples: 8899530. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 08:31:07,016][52710] Avg episode reward: [(0, '20.630'), (1, '27.920')] [2023-10-08 08:31:07,245][53885] Updated weights for policy 1, policy_version 17330 (0.0007) [2023-10-08 08:31:07,618][53885] Updated weights for policy 1, policy_version 17340 (0.0008) [2023-10-08 08:31:10,522][53852] Updated weights for policy 0, policy_version 17410 (0.0008) [2023-10-08 08:31:10,918][53852] Updated weights for policy 0, policy_version 17420 (0.0010) [2023-10-08 08:31:11,293][53852] Updated weights for policy 0, policy_version 17430 (0.0010) [2023-10-08 08:31:11,409][53885] Updated weights for policy 1, policy_version 17350 (0.0008) [2023-10-08 08:31:11,659][53852] Updated weights for policy 0, policy_version 17440 (0.0007) [2023-10-08 08:31:11,773][53885] Updated weights for policy 1, policy_version 17360 (0.0009) [2023-10-08 08:31:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35618816. Throughput: 0: 1843.3, 1: 1831.2. Samples: 8910588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:31:12,016][52710] Avg episode reward: [(0, '19.770'), (1, '27.760')] [2023-10-08 08:31:12,151][53885] Updated weights for policy 1, policy_version 17370 (0.0011) [2023-10-08 08:31:15,267][53852] Updated weights for policy 0, policy_version 17450 (0.0011) [2023-10-08 08:31:15,641][53852] Updated weights for policy 0, policy_version 17460 (0.0010) [2023-10-08 08:31:15,835][53885] Updated weights for policy 1, policy_version 17380 (0.0008) [2023-10-08 08:31:16,014][53852] Updated weights for policy 0, policy_version 17470 (0.0008) [2023-10-08 08:31:16,204][53885] Updated weights for policy 1, policy_version 17390 (0.0009) [2023-10-08 08:31:16,575][53885] Updated weights for policy 1, policy_version 17400 (0.0007) [2023-10-08 08:31:17,015][52710] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 35717120. Throughput: 0: 1835.5, 1: 1826.5. Samples: 8932532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:31:17,015][52710] Avg episode reward: [(0, '20.660'), (1, '27.570')] [2023-10-08 08:31:19,562][53852] Updated weights for policy 0, policy_version 17480 (0.0009) [2023-10-08 08:31:19,928][53852] Updated weights for policy 0, policy_version 17490 (0.0008) [2023-10-08 08:31:20,253][53885] Updated weights for policy 1, policy_version 17410 (0.0007) [2023-10-08 08:31:20,299][53852] Updated weights for policy 0, policy_version 17500 (0.0008) [2023-10-08 08:31:20,622][53885] Updated weights for policy 1, policy_version 17420 (0.0008) [2023-10-08 08:31:20,986][53885] Updated weights for policy 1, policy_version 17430 (0.0008) [2023-10-08 08:31:21,350][53885] Updated weights for policy 1, policy_version 17440 (0.0009) [2023-10-08 08:31:22,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 35782656. Throughput: 0: 1835.0, 1: 1823.6. Samples: 8953200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:31:22,016][52710] Avg episode reward: [(0, '21.220'), (1, '25.780')] [2023-10-08 08:31:24,065][53852] Updated weights for policy 0, policy_version 17510 (0.0008) [2023-10-08 08:31:24,435][53852] Updated weights for policy 0, policy_version 17520 (0.0009) [2023-10-08 08:31:24,811][53852] Updated weights for policy 0, policy_version 17530 (0.0008) [2023-10-08 08:31:25,057][53885] Updated weights for policy 1, policy_version 17450 (0.0008) [2023-10-08 08:31:25,423][53885] Updated weights for policy 1, policy_version 17460 (0.0010) [2023-10-08 08:31:25,800][53885] Updated weights for policy 1, policy_version 17470 (0.0010) [2023-10-08 08:31:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 35848192. Throughput: 0: 1824.8, 1: 1831.2. Samples: 8965296. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 08:31:27,016][52710] Avg episode reward: [(0, '21.360'), (1, '27.600')] [2023-10-08 08:31:28,424][53852] Updated weights for policy 0, policy_version 17540 (0.0009) [2023-10-08 08:31:28,789][53852] Updated weights for policy 0, policy_version 17550 (0.0010) [2023-10-08 08:31:29,165][53852] Updated weights for policy 0, policy_version 17560 (0.0010) [2023-10-08 08:31:29,448][53885] Updated weights for policy 1, policy_version 17480 (0.0008) [2023-10-08 08:31:29,814][53885] Updated weights for policy 1, policy_version 17490 (0.0007) [2023-10-08 08:31:30,175][53885] Updated weights for policy 1, policy_version 17500 (0.0009) [2023-10-08 08:31:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 35913728. Throughput: 0: 1831.9, 1: 1827.5. Samples: 8986196. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 08:31:32,016][52710] Avg episode reward: [(0, '22.880'), (1, '26.600')] [2023-10-08 08:31:32,017][53500] Saving new best policy, reward=22.880! [2023-10-08 08:31:32,956][53852] Updated weights for policy 0, policy_version 17570 (0.0010) [2023-10-08 08:31:33,334][53852] Updated weights for policy 0, policy_version 17580 (0.0011) [2023-10-08 08:31:33,704][53852] Updated weights for policy 0, policy_version 17590 (0.0007) [2023-10-08 08:31:33,796][53885] Updated weights for policy 1, policy_version 17510 (0.0007) [2023-10-08 08:31:34,066][53852] Updated weights for policy 0, policy_version 17600 (0.0007) [2023-10-08 08:31:34,162][53885] Updated weights for policy 1, policy_version 17520 (0.0007) [2023-10-08 08:31:34,530][53885] Updated weights for policy 1, policy_version 17530 (0.0007) [2023-10-08 08:31:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35979264. Throughput: 0: 1826.4, 1: 1822.5. Samples: 9008996. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 08:31:37,016][52710] Avg episode reward: [(0, '21.370'), (1, '27.330')] [2023-10-08 08:31:37,619][53852] Updated weights for policy 0, policy_version 17610 (0.0008) [2023-10-08 08:31:38,000][53852] Updated weights for policy 0, policy_version 17620 (0.0007) [2023-10-08 08:31:38,162][53885] Updated weights for policy 1, policy_version 17540 (0.0008) [2023-10-08 08:31:38,374][53852] Updated weights for policy 0, policy_version 17630 (0.0008) [2023-10-08 08:31:38,534][53885] Updated weights for policy 1, policy_version 17550 (0.0008) [2023-10-08 08:31:38,895][53885] Updated weights for policy 1, policy_version 17560 (0.0009) [2023-10-08 08:31:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36044800. Throughput: 0: 1826.8, 1: 1822.3. Samples: 9019122. Policy #0 lag: (min: 52.0, avg: 56.0, max: 56.0) [2023-10-08 08:31:42,016][52710] Avg episode reward: [(0, '20.930'), (1, '27.620')] [2023-10-08 08:31:42,035][53852] Updated weights for policy 0, policy_version 17640 (0.0009) [2023-10-08 08:31:42,408][53852] Updated weights for policy 0, policy_version 17650 (0.0009) [2023-10-08 08:31:42,579][53885] Updated weights for policy 1, policy_version 17570 (0.0008) [2023-10-08 08:31:42,779][53852] Updated weights for policy 0, policy_version 17660 (0.0007) [2023-10-08 08:31:42,939][53885] Updated weights for policy 1, policy_version 17580 (0.0008) [2023-10-08 08:31:43,312][53885] Updated weights for policy 1, policy_version 17590 (0.0010) [2023-10-08 08:31:43,674][53885] Updated weights for policy 1, policy_version 17600 (0.0009) [2023-10-08 08:31:46,369][53852] Updated weights for policy 0, policy_version 17670 (0.0007) [2023-10-08 08:31:46,745][53852] Updated weights for policy 0, policy_version 17680 (0.0008) [2023-10-08 08:31:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36110336. Throughput: 0: 1824.0, 1: 1820.0. Samples: 9041752. Policy #0 lag: (min: 52.0, avg: 56.0, max: 56.0) [2023-10-08 08:31:47,015][52710] Avg episode reward: [(0, '20.760'), (1, '26.370')] [2023-10-08 08:31:47,106][53852] Updated weights for policy 0, policy_version 17690 (0.0007) [2023-10-08 08:31:47,552][53885] Updated weights for policy 1, policy_version 17610 (0.0008) [2023-10-08 08:31:47,931][53885] Updated weights for policy 1, policy_version 17620 (0.0008) [2023-10-08 08:31:48,296][53885] Updated weights for policy 1, policy_version 17630 (0.0008) [2023-10-08 08:31:50,621][53852] Updated weights for policy 0, policy_version 17700 (0.0008) [2023-10-08 08:31:50,982][53852] Updated weights for policy 0, policy_version 17710 (0.0007) [2023-10-08 08:31:51,348][53852] Updated weights for policy 0, policy_version 17720 (0.0008) [2023-10-08 08:31:51,871][53885] Updated weights for policy 1, policy_version 17640 (0.0010) [2023-10-08 08:31:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36208640. Throughput: 0: 1818.6, 1: 1821.3. Samples: 9063324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:31:52,016][52710] Avg episode reward: [(0, '20.670'), (1, '25.390')] [2023-10-08 08:31:52,236][53885] Updated weights for policy 1, policy_version 17650 (0.0011) [2023-10-08 08:31:52,610][53885] Updated weights for policy 1, policy_version 17660 (0.0008) [2023-10-08 08:31:54,920][53852] Updated weights for policy 0, policy_version 17730 (0.0010) [2023-10-08 08:31:55,291][53852] Updated weights for policy 0, policy_version 17740 (0.0011) [2023-10-08 08:31:55,663][53852] Updated weights for policy 0, policy_version 17750 (0.0008) [2023-10-08 08:31:56,031][53852] Updated weights for policy 0, policy_version 17760 (0.0009) [2023-10-08 08:31:56,369][53885] Updated weights for policy 1, policy_version 17670 (0.0010) [2023-10-08 08:31:56,734][53885] Updated weights for policy 1, policy_version 17680 (0.0010) [2023-10-08 08:31:57,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 36274176. Throughput: 0: 1829.7, 1: 1815.5. Samples: 9074620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:31:57,016][52710] Avg episode reward: [(0, '20.470'), (1, '26.160')] [2023-10-08 08:31:57,104][53885] Updated weights for policy 1, policy_version 17690 (0.0010) [2023-10-08 08:31:59,816][53852] Updated weights for policy 0, policy_version 17770 (0.0008) [2023-10-08 08:32:00,193][53852] Updated weights for policy 0, policy_version 17780 (0.0010) [2023-10-08 08:32:00,553][53852] Updated weights for policy 0, policy_version 17790 (0.0008) [2023-10-08 08:32:00,612][53885] Updated weights for policy 1, policy_version 17700 (0.0009) [2023-10-08 08:32:00,983][53885] Updated weights for policy 1, policy_version 17710 (0.0008) [2023-10-08 08:32:01,358][53885] Updated weights for policy 1, policy_version 17720 (0.0008) [2023-10-08 08:32:02,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 36372480. Throughput: 0: 1816.5, 1: 1816.0. Samples: 9095994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:32:02,016][52710] Avg episode reward: [(0, '21.140'), (1, '26.870')] [2023-10-08 08:32:04,188][53852] Updated weights for policy 0, policy_version 17800 (0.0008) [2023-10-08 08:32:04,565][53852] Updated weights for policy 0, policy_version 17810 (0.0009) [2023-10-08 08:32:04,926][53852] Updated weights for policy 0, policy_version 17820 (0.0008) [2023-10-08 08:32:04,948][53885] Updated weights for policy 1, policy_version 17730 (0.0009) [2023-10-08 08:32:05,312][53885] Updated weights for policy 1, policy_version 17740 (0.0009) [2023-10-08 08:32:05,676][53885] Updated weights for policy 1, policy_version 17750 (0.0012) [2023-10-08 08:32:06,044][53885] Updated weights for policy 1, policy_version 17760 (0.0010) [2023-10-08 08:32:07,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 36438016. Throughput: 0: 1833.8, 1: 1820.8. Samples: 9117660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:32:07,015][52710] Avg episode reward: [(0, '18.900'), (1, '26.360')] [2023-10-08 08:32:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000017760_18186240.pth... [2023-10-08 08:32:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000017824_18251776.pth... [2023-10-08 08:32:07,061][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000016064_16449536.pth [2023-10-08 08:32:07,062][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000016128_16515072.pth [2023-10-08 08:32:08,472][53852] Updated weights for policy 0, policy_version 17830 (0.0009) [2023-10-08 08:32:08,837][53852] Updated weights for policy 0, policy_version 17840 (0.0008) [2023-10-08 08:32:09,200][53852] Updated weights for policy 0, policy_version 17850 (0.0009) [2023-10-08 08:32:09,755][53885] Updated weights for policy 1, policy_version 17770 (0.0007) [2023-10-08 08:32:10,124][53885] Updated weights for policy 1, policy_version 17780 (0.0008) [2023-10-08 08:32:10,495][53885] Updated weights for policy 1, policy_version 17790 (0.0008) [2023-10-08 08:32:12,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 36503552. Throughput: 0: 1823.2, 1: 1813.6. Samples: 9128952. Policy #0 lag: (min: 1.0, avg: 5.4, max: 33.0) [2023-10-08 08:32:12,016][52710] Avg episode reward: [(0, '20.520'), (1, '25.000')] [2023-10-08 08:32:13,015][53852] Updated weights for policy 0, policy_version 17860 (0.0009) [2023-10-08 08:32:13,387][53852] Updated weights for policy 0, policy_version 17870 (0.0007) [2023-10-08 08:32:13,761][53852] Updated weights for policy 0, policy_version 17880 (0.0010) [2023-10-08 08:32:14,166][53885] Updated weights for policy 1, policy_version 17800 (0.0009) [2023-10-08 08:32:14,533][53885] Updated weights for policy 1, policy_version 17810 (0.0009) [2023-10-08 08:32:14,898][53885] Updated weights for policy 1, policy_version 17820 (0.0007) [2023-10-08 08:32:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 36569088. Throughput: 0: 1831.0, 1: 1821.6. Samples: 9150564. Policy #0 lag: (min: 1.0, avg: 5.4, max: 33.0) [2023-10-08 08:32:17,016][52710] Avg episode reward: [(0, '19.770'), (1, '23.830')] [2023-10-08 08:32:17,387][53852] Updated weights for policy 0, policy_version 17890 (0.0007) [2023-10-08 08:32:17,750][53852] Updated weights for policy 0, policy_version 17900 (0.0007) [2023-10-08 08:32:18,129][53852] Updated weights for policy 0, policy_version 17910 (0.0008) [2023-10-08 08:32:18,369][53885] Updated weights for policy 1, policy_version 17830 (0.0007) [2023-10-08 08:32:18,490][53852] Updated weights for policy 0, policy_version 17920 (0.0008) [2023-10-08 08:32:18,739][53885] Updated weights for policy 1, policy_version 17840 (0.0007) [2023-10-08 08:32:19,102][53885] Updated weights for policy 1, policy_version 17850 (0.0007) [2023-10-08 08:32:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36634624. Throughput: 0: 1831.6, 1: 1828.3. Samples: 9173692. Policy #0 lag: (min: 1.0, avg: 5.4, max: 33.0) [2023-10-08 08:32:22,016][52710] Avg episode reward: [(0, '20.000'), (1, '23.000')] [2023-10-08 08:32:22,270][53852] Updated weights for policy 0, policy_version 17930 (0.0010) [2023-10-08 08:32:22,641][53852] Updated weights for policy 0, policy_version 17940 (0.0007) [2023-10-08 08:32:22,802][53885] Updated weights for policy 1, policy_version 17860 (0.0007) [2023-10-08 08:32:23,026][53852] Updated weights for policy 0, policy_version 17950 (0.0008) [2023-10-08 08:32:23,174][53885] Updated weights for policy 1, policy_version 17870 (0.0009) [2023-10-08 08:32:23,543][53885] Updated weights for policy 1, policy_version 17880 (0.0009) [2023-10-08 08:32:26,661][53852] Updated weights for policy 0, policy_version 17960 (0.0007) [2023-10-08 08:32:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36700160. Throughput: 0: 1828.9, 1: 1827.9. Samples: 9183676. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 08:32:27,016][52710] Avg episode reward: [(0, '20.360'), (1, '26.740')] [2023-10-08 08:32:27,030][53852] Updated weights for policy 0, policy_version 17970 (0.0007) [2023-10-08 08:32:27,086][53885] Updated weights for policy 1, policy_version 17890 (0.0008) [2023-10-08 08:32:27,405][53852] Updated weights for policy 0, policy_version 17980 (0.0008) [2023-10-08 08:32:27,451][53885] Updated weights for policy 1, policy_version 17900 (0.0009) [2023-10-08 08:32:27,817][53885] Updated weights for policy 1, policy_version 17910 (0.0007) [2023-10-08 08:32:28,189][53885] Updated weights for policy 1, policy_version 17920 (0.0008) [2023-10-08 08:32:31,089][53852] Updated weights for policy 0, policy_version 17990 (0.0008) [2023-10-08 08:32:31,464][53852] Updated weights for policy 0, policy_version 18000 (0.0007) [2023-10-08 08:32:31,836][53852] Updated weights for policy 0, policy_version 18010 (0.0007) [2023-10-08 08:32:31,947][53885] Updated weights for policy 1, policy_version 17930 (0.0009) [2023-10-08 08:32:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36765696. Throughput: 0: 1830.3, 1: 1836.1. Samples: 9206740. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 08:32:32,016][52710] Avg episode reward: [(0, '22.350'), (1, '26.800')] [2023-10-08 08:32:32,317][53885] Updated weights for policy 1, policy_version 17940 (0.0009) [2023-10-08 08:32:32,677][53885] Updated weights for policy 1, policy_version 17950 (0.0008) [2023-10-08 08:32:35,302][53852] Updated weights for policy 0, policy_version 18020 (0.0008) [2023-10-08 08:32:35,677][53852] Updated weights for policy 0, policy_version 18030 (0.0009) [2023-10-08 08:32:36,038][53852] Updated weights for policy 0, policy_version 18040 (0.0008) [2023-10-08 08:32:36,519][53885] Updated weights for policy 1, policy_version 17960 (0.0007) [2023-10-08 08:32:36,890][53885] Updated weights for policy 1, policy_version 17970 (0.0008) [2023-10-08 08:32:37,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36864000. Throughput: 0: 1829.1, 1: 1824.8. Samples: 9227748. Policy #0 lag: (min: 9.0, avg: 12.7, max: 41.0) [2023-10-08 08:32:37,016][52710] Avg episode reward: [(0, '21.720'), (1, '29.020')] [2023-10-08 08:32:37,272][53885] Updated weights for policy 1, policy_version 17980 (0.0009) [2023-10-08 08:32:37,414][53594] Saving new best policy, reward=29.020! [2023-10-08 08:32:39,654][53852] Updated weights for policy 0, policy_version 18050 (0.0009) [2023-10-08 08:32:40,023][53852] Updated weights for policy 0, policy_version 18060 (0.0011) [2023-10-08 08:32:40,401][53852] Updated weights for policy 0, policy_version 18070 (0.0009) [2023-10-08 08:32:40,760][53852] Updated weights for policy 0, policy_version 18080 (0.0008) [2023-10-08 08:32:41,007][53885] Updated weights for policy 1, policy_version 17990 (0.0011) [2023-10-08 08:32:41,378][53885] Updated weights for policy 1, policy_version 18000 (0.0009) [2023-10-08 08:32:41,742][53885] Updated weights for policy 1, policy_version 18010 (0.0009) [2023-10-08 08:32:42,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 36962304. Throughput: 0: 1833.1, 1: 1833.2. Samples: 9239602. Policy #0 lag: (min: 9.0, avg: 12.7, max: 41.0) [2023-10-08 08:32:42,016][52710] Avg episode reward: [(0, '21.820'), (1, '29.420')] [2023-10-08 08:32:42,018][53594] Saving new best policy, reward=29.420! [2023-10-08 08:32:44,490][53852] Updated weights for policy 0, policy_version 18090 (0.0008) [2023-10-08 08:32:44,855][53852] Updated weights for policy 0, policy_version 18100 (0.0008) [2023-10-08 08:32:45,231][53852] Updated weights for policy 0, policy_version 18110 (0.0009) [2023-10-08 08:32:45,424][53885] Updated weights for policy 1, policy_version 18020 (0.0010) [2023-10-08 08:32:45,797][53885] Updated weights for policy 1, policy_version 18030 (0.0009) [2023-10-08 08:32:46,161][53885] Updated weights for policy 1, policy_version 18040 (0.0009) [2023-10-08 08:32:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 37027840. Throughput: 0: 1832.7, 1: 1831.9. Samples: 9260902. Policy #0 lag: (min: 9.0, avg: 12.7, max: 41.0) [2023-10-08 08:32:47,016][52710] Avg episode reward: [(0, '22.920'), (1, '28.210')] [2023-10-08 08:32:47,018][53500] Saving new best policy, reward=22.920! [2023-10-08 08:32:48,885][53852] Updated weights for policy 0, policy_version 18120 (0.0010) [2023-10-08 08:32:49,257][53852] Updated weights for policy 0, policy_version 18130 (0.0008) [2023-10-08 08:32:49,623][53852] Updated weights for policy 0, policy_version 18140 (0.0007) [2023-10-08 08:32:49,908][53885] Updated weights for policy 1, policy_version 18050 (0.0007) [2023-10-08 08:32:50,277][53885] Updated weights for policy 1, policy_version 18060 (0.0007) [2023-10-08 08:32:50,641][53885] Updated weights for policy 1, policy_version 18070 (0.0007) [2023-10-08 08:32:51,002][53885] Updated weights for policy 1, policy_version 18080 (0.0008) [2023-10-08 08:32:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 37093376. Throughput: 0: 1837.6, 1: 1834.1. Samples: 9282890. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-08 08:32:52,016][52710] Avg episode reward: [(0, '22.250'), (1, '26.630')] [2023-10-08 08:32:53,269][53852] Updated weights for policy 0, policy_version 18150 (0.0009) [2023-10-08 08:32:53,636][53852] Updated weights for policy 0, policy_version 18160 (0.0007) [2023-10-08 08:32:53,999][53852] Updated weights for policy 0, policy_version 18170 (0.0009) [2023-10-08 08:32:54,561][53885] Updated weights for policy 1, policy_version 18090 (0.0010) [2023-10-08 08:32:54,934][53885] Updated weights for policy 1, policy_version 18100 (0.0010) [2023-10-08 08:32:55,307][53885] Updated weights for policy 1, policy_version 18110 (0.0009) [2023-10-08 08:32:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 37158912. Throughput: 0: 1835.6, 1: 1828.8. Samples: 9293850. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-08 08:32:57,016][52710] Avg episode reward: [(0, '23.730'), (1, '26.350')] [2023-10-08 08:32:57,018][53500] Saving new best policy, reward=23.730! [2023-10-08 08:32:57,631][53852] Updated weights for policy 0, policy_version 18180 (0.0007) [2023-10-08 08:32:58,009][53852] Updated weights for policy 0, policy_version 18190 (0.0008) [2023-10-08 08:32:58,374][53852] Updated weights for policy 0, policy_version 18200 (0.0008) [2023-10-08 08:32:58,941][53885] Updated weights for policy 1, policy_version 18120 (0.0007) [2023-10-08 08:32:59,311][53885] Updated weights for policy 1, policy_version 18130 (0.0007) [2023-10-08 08:32:59,678][53885] Updated weights for policy 1, policy_version 18140 (0.0008) [2023-10-08 08:33:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 37224448. Throughput: 0: 1840.4, 1: 1836.3. Samples: 9316012. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-08 08:33:02,016][52710] Avg episode reward: [(0, '22.290'), (1, '27.960')] [2023-10-08 08:33:02,059][53852] Updated weights for policy 0, policy_version 18210 (0.0008) [2023-10-08 08:33:02,418][53852] Updated weights for policy 0, policy_version 18220 (0.0009) [2023-10-08 08:33:02,788][53852] Updated weights for policy 0, policy_version 18230 (0.0009) [2023-10-08 08:33:03,155][53852] Updated weights for policy 0, policy_version 18240 (0.0008) [2023-10-08 08:33:03,431][53885] Updated weights for policy 1, policy_version 18150 (0.0008) [2023-10-08 08:33:03,800][53885] Updated weights for policy 1, policy_version 18160 (0.0008) [2023-10-08 08:33:04,173][53885] Updated weights for policy 1, policy_version 18170 (0.0008) [2023-10-08 08:33:06,674][53852] Updated weights for policy 0, policy_version 18250 (0.0008) [2023-10-08 08:33:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 37289984. Throughput: 0: 1837.6, 1: 1826.9. Samples: 9338596. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) [2023-10-08 08:33:07,016][52710] Avg episode reward: [(0, '24.900'), (1, '25.850')] [2023-10-08 08:33:07,051][53852] Updated weights for policy 0, policy_version 18260 (0.0011) [2023-10-08 08:33:07,420][53852] Updated weights for policy 0, policy_version 18270 (0.0009) [2023-10-08 08:33:07,489][53500] Saving new best policy, reward=24.900! [2023-10-08 08:33:07,812][53885] Updated weights for policy 1, policy_version 18180 (0.0008) [2023-10-08 08:33:08,181][53885] Updated weights for policy 1, policy_version 18190 (0.0009) [2023-10-08 08:33:08,547][53885] Updated weights for policy 1, policy_version 18200 (0.0008) [2023-10-08 08:33:11,058][53852] Updated weights for policy 0, policy_version 18280 (0.0010) [2023-10-08 08:33:11,438][53852] Updated weights for policy 0, policy_version 18290 (0.0010) [2023-10-08 08:33:11,796][53852] Updated weights for policy 0, policy_version 18300 (0.0007) [2023-10-08 08:33:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 37388288. Throughput: 0: 1843.2, 1: 1827.3. Samples: 9348844. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-08 08:33:12,015][52710] Avg episode reward: [(0, '22.400'), (1, '26.210')] [2023-10-08 08:33:12,077][53885] Updated weights for policy 1, policy_version 18210 (0.0008) [2023-10-08 08:33:12,451][53885] Updated weights for policy 1, policy_version 18220 (0.0009) [2023-10-08 08:33:12,810][53885] Updated weights for policy 1, policy_version 18230 (0.0007) [2023-10-08 08:33:13,178][53885] Updated weights for policy 1, policy_version 18240 (0.0007) [2023-10-08 08:33:15,501][53852] Updated weights for policy 0, policy_version 18310 (0.0008) [2023-10-08 08:33:15,888][53852] Updated weights for policy 0, policy_version 18320 (0.0011) [2023-10-08 08:33:16,249][53852] Updated weights for policy 0, policy_version 18330 (0.0008) [2023-10-08 08:33:16,819][53885] Updated weights for policy 1, policy_version 18250 (0.0008) [2023-10-08 08:33:17,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37453824. Throughput: 0: 1842.3, 1: 1822.7. Samples: 9371664. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-08 08:33:17,016][52710] Avg episode reward: [(0, '22.300'), (1, '27.380')] [2023-10-08 08:33:17,194][53885] Updated weights for policy 1, policy_version 18260 (0.0007) [2023-10-08 08:33:17,570][53885] Updated weights for policy 1, policy_version 18270 (0.0008) [2023-10-08 08:33:19,938][53852] Updated weights for policy 0, policy_version 18340 (0.0007) [2023-10-08 08:33:20,314][53852] Updated weights for policy 0, policy_version 18350 (0.0008) [2023-10-08 08:33:20,685][53852] Updated weights for policy 0, policy_version 18360 (0.0007) [2023-10-08 08:33:21,237][53885] Updated weights for policy 1, policy_version 18280 (0.0009) [2023-10-08 08:33:21,617][53885] Updated weights for policy 1, policy_version 18290 (0.0009) [2023-10-08 08:33:21,977][53885] Updated weights for policy 1, policy_version 18300 (0.0010) [2023-10-08 08:33:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37519360. Throughput: 0: 1843.3, 1: 1820.9. Samples: 9392638. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) [2023-10-08 08:33:22,016][52710] Avg episode reward: [(0, '21.920'), (1, '28.010')] [2023-10-08 08:33:24,386][53852] Updated weights for policy 0, policy_version 18370 (0.0008) [2023-10-08 08:33:24,768][53852] Updated weights for policy 0, policy_version 18380 (0.0009) [2023-10-08 08:33:25,138][53852] Updated weights for policy 0, policy_version 18390 (0.0008) [2023-10-08 08:33:25,516][53852] Updated weights for policy 0, policy_version 18400 (0.0009) [2023-10-08 08:33:25,684][53885] Updated weights for policy 1, policy_version 18310 (0.0008) [2023-10-08 08:33:26,050][53885] Updated weights for policy 1, policy_version 18320 (0.0007) [2023-10-08 08:33:26,415][53885] Updated weights for policy 1, policy_version 18330 (0.0009) [2023-10-08 08:33:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 37617664. Throughput: 0: 1836.6, 1: 1832.0. Samples: 9404686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:33:27,016][52710] Avg episode reward: [(0, '20.570'), (1, '27.170')] [2023-10-08 08:33:28,948][53852] Updated weights for policy 0, policy_version 18410 (0.0009) [2023-10-08 08:33:29,324][53852] Updated weights for policy 0, policy_version 18420 (0.0008) [2023-10-08 08:33:29,688][53852] Updated weights for policy 0, policy_version 18430 (0.0010) [2023-10-08 08:33:30,232][53885] Updated weights for policy 1, policy_version 18340 (0.0008) [2023-10-08 08:33:30,599][53885] Updated weights for policy 1, policy_version 18350 (0.0007) [2023-10-08 08:33:30,963][53885] Updated weights for policy 1, policy_version 18360 (0.0007) [2023-10-08 08:33:32,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 37683200. Throughput: 0: 1844.8, 1: 1819.1. Samples: 9425778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:33:32,016][52710] Avg episode reward: [(0, '19.980'), (1, '26.510')] [2023-10-08 08:33:33,378][53852] Updated weights for policy 0, policy_version 18440 (0.0009) [2023-10-08 08:33:33,750][53852] Updated weights for policy 0, policy_version 18450 (0.0007) [2023-10-08 08:33:34,121][53852] Updated weights for policy 0, policy_version 18460 (0.0009) [2023-10-08 08:33:34,642][53885] Updated weights for policy 1, policy_version 18370 (0.0007) [2023-10-08 08:33:35,012][53885] Updated weights for policy 1, policy_version 18380 (0.0008) [2023-10-08 08:33:35,378][53885] Updated weights for policy 1, policy_version 18390 (0.0008) [2023-10-08 08:33:35,754][53885] Updated weights for policy 1, policy_version 18400 (0.0009) [2023-10-08 08:33:37,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 37748736. Throughput: 0: 1847.2, 1: 1822.5. Samples: 9448028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:33:37,015][52710] Avg episode reward: [(0, '20.830'), (1, '26.080')] [2023-10-08 08:33:37,760][53852] Updated weights for policy 0, policy_version 18470 (0.0008) [2023-10-08 08:33:38,145][53852] Updated weights for policy 0, policy_version 18480 (0.0008) [2023-10-08 08:33:38,520][53852] Updated weights for policy 0, policy_version 18490 (0.0007) [2023-10-08 08:33:39,595][53885] Updated weights for policy 1, policy_version 18410 (0.0008) [2023-10-08 08:33:39,958][53885] Updated weights for policy 1, policy_version 18420 (0.0007) [2023-10-08 08:33:40,328][53885] Updated weights for policy 1, policy_version 18430 (0.0008) [2023-10-08 08:33:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 37814272. Throughput: 0: 1848.4, 1: 1822.1. Samples: 9459020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:33:42,016][52710] Avg episode reward: [(0, '20.840'), (1, '27.280')] [2023-10-08 08:33:42,131][53852] Updated weights for policy 0, policy_version 18500 (0.0007) [2023-10-08 08:33:42,508][53852] Updated weights for policy 0, policy_version 18510 (0.0010) [2023-10-08 08:33:42,877][53852] Updated weights for policy 0, policy_version 18520 (0.0009) [2023-10-08 08:33:44,051][53885] Updated weights for policy 1, policy_version 18440 (0.0008) [2023-10-08 08:33:44,414][53885] Updated weights for policy 1, policy_version 18450 (0.0008) [2023-10-08 08:33:44,787][53885] Updated weights for policy 1, policy_version 18460 (0.0009) [2023-10-08 08:33:46,425][53852] Updated weights for policy 0, policy_version 18530 (0.0007) [2023-10-08 08:33:46,795][53852] Updated weights for policy 0, policy_version 18540 (0.0007) [2023-10-08 08:33:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 37879808. Throughput: 0: 1847.6, 1: 1816.9. Samples: 9480916. Policy #0 lag: (min: 24.0, avg: 48.0, max: 56.0) [2023-10-08 08:33:47,016][52710] Avg episode reward: [(0, '23.150'), (1, '26.900')] [2023-10-08 08:33:47,175][53852] Updated weights for policy 0, policy_version 18550 (0.0008) [2023-10-08 08:33:47,535][53852] Updated weights for policy 0, policy_version 18560 (0.0009) [2023-10-08 08:33:48,379][53885] Updated weights for policy 1, policy_version 18470 (0.0009) [2023-10-08 08:33:48,752][53885] Updated weights for policy 1, policy_version 18480 (0.0008) [2023-10-08 08:33:49,112][53885] Updated weights for policy 1, policy_version 18490 (0.0008) [2023-10-08 08:33:51,035][53852] Updated weights for policy 0, policy_version 18570 (0.0009) [2023-10-08 08:33:51,403][53852] Updated weights for policy 0, policy_version 18580 (0.0009) [2023-10-08 08:33:51,779][53852] Updated weights for policy 0, policy_version 18590 (0.0007) [2023-10-08 08:33:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 37978112. Throughput: 0: 1831.9, 1: 1821.9. Samples: 9503018. Policy #0 lag: (min: 24.0, avg: 48.0, max: 56.0) [2023-10-08 08:33:52,016][52710] Avg episode reward: [(0, '21.310'), (1, '26.730')] [2023-10-08 08:33:53,008][53885] Updated weights for policy 1, policy_version 18500 (0.0008) [2023-10-08 08:33:53,384][53885] Updated weights for policy 1, policy_version 18510 (0.0007) [2023-10-08 08:33:53,753][53885] Updated weights for policy 1, policy_version 18520 (0.0007) [2023-10-08 08:33:55,481][53852] Updated weights for policy 0, policy_version 18600 (0.0010) [2023-10-08 08:33:55,842][53852] Updated weights for policy 0, policy_version 18610 (0.0008) [2023-10-08 08:33:56,220][53852] Updated weights for policy 0, policy_version 18620 (0.0007) [2023-10-08 08:33:57,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 38043648. Throughput: 0: 1855.4, 1: 1818.1. Samples: 9514152. Policy #0 lag: (min: 24.0, avg: 48.0, max: 56.0) [2023-10-08 08:33:57,015][52710] Avg episode reward: [(0, '21.240'), (1, '27.290')] [2023-10-08 08:33:57,351][53885] Updated weights for policy 1, policy_version 18530 (0.0007) [2023-10-08 08:33:57,725][53885] Updated weights for policy 1, policy_version 18540 (0.0008) [2023-10-08 08:33:58,097][53885] Updated weights for policy 1, policy_version 18550 (0.0007) [2023-10-08 08:33:58,460][53885] Updated weights for policy 1, policy_version 18560 (0.0008) [2023-10-08 08:33:59,883][53852] Updated weights for policy 0, policy_version 18630 (0.0008) [2023-10-08 08:34:00,253][53852] Updated weights for policy 0, policy_version 18640 (0.0009) [2023-10-08 08:34:00,628][53852] Updated weights for policy 0, policy_version 18650 (0.0007) [2023-10-08 08:34:01,973][53885] Updated weights for policy 1, policy_version 18570 (0.0008) [2023-10-08 08:34:02,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38109184. Throughput: 0: 1835.1, 1: 1822.8. Samples: 9536270. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-08 08:34:02,015][52710] Avg episode reward: [(0, '19.430'), (1, '28.500')] [2023-10-08 08:34:02,347][53885] Updated weights for policy 1, policy_version 18580 (0.0007) [2023-10-08 08:34:02,718][53885] Updated weights for policy 1, policy_version 18590 (0.0008) [2023-10-08 08:34:04,214][53852] Updated weights for policy 0, policy_version 18660 (0.0008) [2023-10-08 08:34:04,580][53852] Updated weights for policy 0, policy_version 18670 (0.0007) [2023-10-08 08:34:04,948][53852] Updated weights for policy 0, policy_version 18680 (0.0009) [2023-10-08 08:34:06,570][53885] Updated weights for policy 1, policy_version 18600 (0.0007) [2023-10-08 08:34:06,940][53885] Updated weights for policy 1, policy_version 18610 (0.0010) [2023-10-08 08:34:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 38174720. Throughput: 0: 1854.3, 1: 1829.9. Samples: 9558426. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-08 08:34:07,016][52710] Avg episode reward: [(0, '20.600'), (1, '29.210')] [2023-10-08 08:34:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000018688_19136512.pth... [2023-10-08 08:34:07,059][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000016960_17367040.pth [2023-10-08 08:34:07,309][53885] Updated weights for policy 1, policy_version 18620 (0.0008) [2023-10-08 08:34:07,451][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000018624_19070976.pth... [2023-10-08 08:34:07,489][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000016896_17301504.pth [2023-10-08 08:34:08,549][53852] Updated weights for policy 0, policy_version 18690 (0.0010) [2023-10-08 08:34:08,920][53852] Updated weights for policy 0, policy_version 18700 (0.0010) [2023-10-08 08:34:09,293][53852] Updated weights for policy 0, policy_version 18710 (0.0008) [2023-10-08 08:34:09,662][53852] Updated weights for policy 0, policy_version 18720 (0.0008) [2023-10-08 08:34:10,847][53885] Updated weights for policy 1, policy_version 18630 (0.0008) [2023-10-08 08:34:11,205][53885] Updated weights for policy 1, policy_version 18640 (0.0009) [2023-10-08 08:34:11,573][53885] Updated weights for policy 1, policy_version 18650 (0.0008) [2023-10-08 08:34:12,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 38273024. Throughput: 0: 1833.6, 1: 1823.6. Samples: 9569262. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-08 08:34:12,016][52710] Avg episode reward: [(0, '20.300'), (1, '28.960')] [2023-10-08 08:34:13,248][53852] Updated weights for policy 0, policy_version 18730 (0.0009) [2023-10-08 08:34:13,618][53852] Updated weights for policy 0, policy_version 18740 (0.0009) [2023-10-08 08:34:13,986][53852] Updated weights for policy 0, policy_version 18750 (0.0009) [2023-10-08 08:34:14,956][53885] Updated weights for policy 1, policy_version 18660 (0.0010) [2023-10-08 08:34:15,321][53885] Updated weights for policy 1, policy_version 18670 (0.0009) [2023-10-08 08:34:15,687][53885] Updated weights for policy 1, policy_version 18680 (0.0010) [2023-10-08 08:34:17,015][52710] Fps is (10 sec: 16384.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 38338560. Throughput: 0: 1857.2, 1: 1823.6. Samples: 9591412. Policy #0 lag: (min: 31.0, avg: 43.9, max: 63.0) [2023-10-08 08:34:17,015][52710] Avg episode reward: [(0, '21.020'), (1, '29.250')] [2023-10-08 08:34:17,539][53852] Updated weights for policy 0, policy_version 18760 (0.0008) [2023-10-08 08:34:17,920][53852] Updated weights for policy 0, policy_version 18770 (0.0008) [2023-10-08 08:34:18,285][53852] Updated weights for policy 0, policy_version 18780 (0.0007) [2023-10-08 08:34:19,314][53885] Updated weights for policy 1, policy_version 18690 (0.0010) [2023-10-08 08:34:19,682][53885] Updated weights for policy 1, policy_version 18700 (0.0010) [2023-10-08 08:34:20,047][53885] Updated weights for policy 1, policy_version 18710 (0.0010) [2023-10-08 08:34:20,413][53885] Updated weights for policy 1, policy_version 18720 (0.0008) [2023-10-08 08:34:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38404096. Throughput: 0: 1855.8, 1: 1839.2. Samples: 9614306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:34:22,016][52710] Avg episode reward: [(0, '20.310'), (1, '29.270')] [2023-10-08 08:34:22,063][53852] Updated weights for policy 0, policy_version 18790 (0.0008) [2023-10-08 08:34:22,426][53852] Updated weights for policy 0, policy_version 18800 (0.0007) [2023-10-08 08:34:22,802][53852] Updated weights for policy 0, policy_version 18810 (0.0007) [2023-10-08 08:34:24,006][53885] Updated weights for policy 1, policy_version 18730 (0.0010) [2023-10-08 08:34:24,373][53885] Updated weights for policy 1, policy_version 18740 (0.0011) [2023-10-08 08:34:24,739][53885] Updated weights for policy 1, policy_version 18750 (0.0010) [2023-10-08 08:34:26,477][53852] Updated weights for policy 0, policy_version 18820 (0.0008) [2023-10-08 08:34:26,868][53852] Updated weights for policy 0, policy_version 18830 (0.0008) [2023-10-08 08:34:27,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 38469632. Throughput: 0: 1858.4, 1: 1826.8. Samples: 9624854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:34:27,017][52710] Avg episode reward: [(0, '20.410'), (1, '27.420')] [2023-10-08 08:34:27,239][53852] Updated weights for policy 0, policy_version 18840 (0.0007) [2023-10-08 08:34:28,358][53885] Updated weights for policy 1, policy_version 18760 (0.0008) [2023-10-08 08:34:28,716][53885] Updated weights for policy 1, policy_version 18770 (0.0007) [2023-10-08 08:34:29,083][53885] Updated weights for policy 1, policy_version 18780 (0.0008) [2023-10-08 08:34:30,745][53852] Updated weights for policy 0, policy_version 18850 (0.0008) [2023-10-08 08:34:31,117][53852] Updated weights for policy 0, policy_version 18860 (0.0010) [2023-10-08 08:34:31,487][53852] Updated weights for policy 0, policy_version 18870 (0.0007) [2023-10-08 08:34:31,854][53852] Updated weights for policy 0, policy_version 18880 (0.0010) [2023-10-08 08:34:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 38567936. Throughput: 0: 1856.3, 1: 1849.5. Samples: 9647676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:34:32,016][52710] Avg episode reward: [(0, '20.610'), (1, '28.410')] [2023-10-08 08:34:32,779][53885] Updated weights for policy 1, policy_version 18790 (0.0007) [2023-10-08 08:34:33,143][53885] Updated weights for policy 1, policy_version 18800 (0.0007) [2023-10-08 08:34:33,509][53885] Updated weights for policy 1, policy_version 18810 (0.0007) [2023-10-08 08:34:35,436][53852] Updated weights for policy 0, policy_version 18890 (0.0010) [2023-10-08 08:34:35,802][53852] Updated weights for policy 0, policy_version 18900 (0.0008) [2023-10-08 08:34:36,161][53852] Updated weights for policy 0, policy_version 18910 (0.0008) [2023-10-08 08:34:37,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38633472. Throughput: 0: 1842.1, 1: 1856.3. Samples: 9669448. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 08:34:37,016][52710] Avg episode reward: [(0, '18.390'), (1, '28.970')] [2023-10-08 08:34:37,131][53885] Updated weights for policy 1, policy_version 18820 (0.0008) [2023-10-08 08:34:37,508][53885] Updated weights for policy 1, policy_version 18830 (0.0009) [2023-10-08 08:34:37,868][53885] Updated weights for policy 1, policy_version 18840 (0.0009) [2023-10-08 08:34:39,748][53852] Updated weights for policy 0, policy_version 18920 (0.0007) [2023-10-08 08:34:40,116][53852] Updated weights for policy 0, policy_version 18930 (0.0008) [2023-10-08 08:34:40,494][53852] Updated weights for policy 0, policy_version 18940 (0.0007) [2023-10-08 08:34:41,588][53885] Updated weights for policy 1, policy_version 18850 (0.0009) [2023-10-08 08:34:41,955][53885] Updated weights for policy 1, policy_version 18860 (0.0010) [2023-10-08 08:34:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38699008. Throughput: 0: 1854.6, 1: 1855.7. Samples: 9681114. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 08:34:42,016][52710] Avg episode reward: [(0, '20.170'), (1, '26.100')] [2023-10-08 08:34:42,320][53885] Updated weights for policy 1, policy_version 18870 (0.0008) [2023-10-08 08:34:42,688][53885] Updated weights for policy 1, policy_version 18880 (0.0008) [2023-10-08 08:34:44,177][53852] Updated weights for policy 0, policy_version 18950 (0.0007) [2023-10-08 08:34:44,550][53852] Updated weights for policy 0, policy_version 18960 (0.0008) [2023-10-08 08:34:44,915][53852] Updated weights for policy 0, policy_version 18970 (0.0008) [2023-10-08 08:34:46,277][53885] Updated weights for policy 1, policy_version 18890 (0.0007) [2023-10-08 08:34:46,644][53885] Updated weights for policy 1, policy_version 18900 (0.0008) [2023-10-08 08:34:47,006][53885] Updated weights for policy 1, policy_version 18910 (0.0008) [2023-10-08 08:34:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38764544. Throughput: 0: 1846.8, 1: 1854.7. Samples: 9702838. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 08:34:47,015][52710] Avg episode reward: [(0, '17.630'), (1, '26.820')] [2023-10-08 08:34:48,543][53852] Updated weights for policy 0, policy_version 18980 (0.0009) [2023-10-08 08:34:48,922][53852] Updated weights for policy 0, policy_version 18990 (0.0008) [2023-10-08 08:34:49,289][53852] Updated weights for policy 0, policy_version 19000 (0.0007) [2023-10-08 08:34:50,729][53885] Updated weights for policy 1, policy_version 18920 (0.0009) [2023-10-08 08:34:51,098][53885] Updated weights for policy 1, policy_version 18930 (0.0011) [2023-10-08 08:34:51,473][53885] Updated weights for policy 1, policy_version 18940 (0.0011) [2023-10-08 08:34:52,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38862848. Throughput: 0: 1858.7, 1: 1832.1. Samples: 9724514. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) [2023-10-08 08:34:52,016][52710] Avg episode reward: [(0, '20.180'), (1, '25.770')] [2023-10-08 08:34:52,853][53852] Updated weights for policy 0, policy_version 19010 (0.0007) [2023-10-08 08:34:53,215][53852] Updated weights for policy 0, policy_version 19020 (0.0011) [2023-10-08 08:34:53,588][53852] Updated weights for policy 0, policy_version 19030 (0.0011) [2023-10-08 08:34:53,955][53852] Updated weights for policy 0, policy_version 19040 (0.0008) [2023-10-08 08:34:55,291][53885] Updated weights for policy 1, policy_version 18950 (0.0009) [2023-10-08 08:34:55,677][53885] Updated weights for policy 1, policy_version 18960 (0.0010) [2023-10-08 08:34:56,049][53885] Updated weights for policy 1, policy_version 18970 (0.0009) [2023-10-08 08:34:57,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38928384. Throughput: 0: 1849.7, 1: 1853.3. Samples: 9735894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:34:57,016][52710] Avg episode reward: [(0, '21.710'), (1, '25.980')] [2023-10-08 08:34:57,685][53852] Updated weights for policy 0, policy_version 19050 (0.0007) [2023-10-08 08:34:58,044][53852] Updated weights for policy 0, policy_version 19060 (0.0007) [2023-10-08 08:34:58,419][53852] Updated weights for policy 0, policy_version 19070 (0.0007) [2023-10-08 08:34:59,694][53885] Updated weights for policy 1, policy_version 18980 (0.0008) [2023-10-08 08:35:00,059][53885] Updated weights for policy 1, policy_version 18990 (0.0010) [2023-10-08 08:35:00,414][53885] Updated weights for policy 1, policy_version 19000 (0.0010) [2023-10-08 08:35:01,902][53852] Updated weights for policy 0, policy_version 19080 (0.0009) [2023-10-08 08:35:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 38993920. Throughput: 0: 1855.6, 1: 1835.5. Samples: 9757512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:35:02,016][52710] Avg episode reward: [(0, '19.540'), (1, '28.230')] [2023-10-08 08:35:02,284][53852] Updated weights for policy 0, policy_version 19090 (0.0008) [2023-10-08 08:35:02,646][53852] Updated weights for policy 0, policy_version 19100 (0.0009) [2023-10-08 08:35:04,038][53885] Updated weights for policy 1, policy_version 19010 (0.0009) [2023-10-08 08:35:04,405][53885] Updated weights for policy 1, policy_version 19020 (0.0007) [2023-10-08 08:35:04,771][53885] Updated weights for policy 1, policy_version 19030 (0.0009) [2023-10-08 08:35:05,138][53885] Updated weights for policy 1, policy_version 19040 (0.0007) [2023-10-08 08:35:06,407][53852] Updated weights for policy 0, policy_version 19110 (0.0010) [2023-10-08 08:35:06,778][53852] Updated weights for policy 0, policy_version 19120 (0.0009) [2023-10-08 08:35:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 39059456. Throughput: 0: 1840.0, 1: 1835.0. Samples: 9779680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:35:07,015][52710] Avg episode reward: [(0, '21.840'), (1, '28.590')] [2023-10-08 08:35:07,145][53852] Updated weights for policy 0, policy_version 19130 (0.0008) [2023-10-08 08:35:08,915][53885] Updated weights for policy 1, policy_version 19050 (0.0010) [2023-10-08 08:35:09,284][53885] Updated weights for policy 1, policy_version 19060 (0.0008) [2023-10-08 08:35:09,653][53885] Updated weights for policy 1, policy_version 19070 (0.0008) [2023-10-08 08:35:10,679][53852] Updated weights for policy 0, policy_version 19140 (0.0009) [2023-10-08 08:35:11,056][53852] Updated weights for policy 0, policy_version 19150 (0.0008) [2023-10-08 08:35:11,424][53852] Updated weights for policy 0, policy_version 19160 (0.0007) [2023-10-08 08:35:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 39157760. Throughput: 0: 1850.1, 1: 1825.3. Samples: 9790250. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-10-08 08:35:12,016][52710] Avg episode reward: [(0, '22.180'), (1, '30.960')] [2023-10-08 08:35:12,017][53594] Saving new best policy, reward=30.960! [2023-10-08 08:35:13,546][53885] Updated weights for policy 1, policy_version 19080 (0.0008) [2023-10-08 08:35:13,913][53885] Updated weights for policy 1, policy_version 19090 (0.0008) [2023-10-08 08:35:14,281][53885] Updated weights for policy 1, policy_version 19100 (0.0010) [2023-10-08 08:35:15,083][53852] Updated weights for policy 0, policy_version 19170 (0.0009) [2023-10-08 08:35:15,490][53852] Updated weights for policy 0, policy_version 19180 (0.0010) [2023-10-08 08:35:15,861][53852] Updated weights for policy 0, policy_version 19190 (0.0007) [2023-10-08 08:35:16,232][53852] Updated weights for policy 0, policy_version 19200 (0.0008) [2023-10-08 08:35:17,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 39223296. Throughput: 0: 1836.2, 1: 1818.0. Samples: 9812112. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-10-08 08:35:17,016][52710] Avg episode reward: [(0, '21.170'), (1, '29.630')] [2023-10-08 08:35:17,807][53885] Updated weights for policy 1, policy_version 19110 (0.0009) [2023-10-08 08:35:18,173][53885] Updated weights for policy 1, policy_version 19120 (0.0008) [2023-10-08 08:35:18,545][53885] Updated weights for policy 1, policy_version 19130 (0.0008) [2023-10-08 08:35:19,672][53852] Updated weights for policy 0, policy_version 19210 (0.0007) [2023-10-08 08:35:20,054][53852] Updated weights for policy 0, policy_version 19220 (0.0008) [2023-10-08 08:35:20,424][53852] Updated weights for policy 0, policy_version 19230 (0.0008) [2023-10-08 08:35:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39288832. Throughput: 0: 1854.5, 1: 1807.9. Samples: 9834256. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-10-08 08:35:22,016][52710] Avg episode reward: [(0, '21.640'), (1, '27.660')] [2023-10-08 08:35:22,283][53885] Updated weights for policy 1, policy_version 19140 (0.0008) [2023-10-08 08:35:22,650][53885] Updated weights for policy 1, policy_version 19150 (0.0009) [2023-10-08 08:35:23,023][53885] Updated weights for policy 1, policy_version 19160 (0.0009) [2023-10-08 08:35:24,179][53852] Updated weights for policy 0, policy_version 19240 (0.0008) [2023-10-08 08:35:24,559][53852] Updated weights for policy 0, policy_version 19250 (0.0009) [2023-10-08 08:35:24,924][53852] Updated weights for policy 0, policy_version 19260 (0.0007) [2023-10-08 08:35:26,789][53885] Updated weights for policy 1, policy_version 19170 (0.0009) [2023-10-08 08:35:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39354368. Throughput: 0: 1831.4, 1: 1811.3. Samples: 9845036. Policy #0 lag: (min: 31.0, avg: 43.2, max: 63.0) [2023-10-08 08:35:27,016][52710] Avg episode reward: [(0, '22.650'), (1, '31.430')] [2023-10-08 08:35:27,157][53885] Updated weights for policy 1, policy_version 19180 (0.0009) [2023-10-08 08:35:27,519][53885] Updated weights for policy 1, policy_version 19190 (0.0007) [2023-10-08 08:35:27,887][53594] Saving new best policy, reward=31.430! [2023-10-08 08:35:27,890][53885] Updated weights for policy 1, policy_version 19200 (0.0008) [2023-10-08 08:35:28,708][53852] Updated weights for policy 0, policy_version 19270 (0.0007) [2023-10-08 08:35:29,079][53852] Updated weights for policy 0, policy_version 19280 (0.0008) [2023-10-08 08:35:29,443][53852] Updated weights for policy 0, policy_version 19290 (0.0009) [2023-10-08 08:35:31,556][53885] Updated weights for policy 1, policy_version 19210 (0.0008) [2023-10-08 08:35:31,924][53885] Updated weights for policy 1, policy_version 19220 (0.0007) [2023-10-08 08:35:32,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39419904. Throughput: 0: 1846.0, 1: 1805.2. Samples: 9867144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:35:32,015][52710] Avg episode reward: [(0, '21.880'), (1, '30.430')] [2023-10-08 08:35:32,292][53885] Updated weights for policy 1, policy_version 19230 (0.0011) [2023-10-08 08:35:33,059][53852] Updated weights for policy 0, policy_version 19300 (0.0009) [2023-10-08 08:35:33,418][53852] Updated weights for policy 0, policy_version 19310 (0.0007) [2023-10-08 08:35:33,793][53852] Updated weights for policy 0, policy_version 19320 (0.0008) [2023-10-08 08:35:35,962][53885] Updated weights for policy 1, policy_version 19240 (0.0009) [2023-10-08 08:35:36,328][53885] Updated weights for policy 1, policy_version 19250 (0.0008) [2023-10-08 08:35:36,696][53885] Updated weights for policy 1, policy_version 19260 (0.0008) [2023-10-08 08:35:37,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39518208. Throughput: 0: 1843.2, 1: 1811.4. Samples: 9888972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:35:37,016][52710] Avg episode reward: [(0, '21.940'), (1, '29.780')] [2023-10-08 08:35:37,410][53852] Updated weights for policy 0, policy_version 19330 (0.0008) [2023-10-08 08:35:37,781][53852] Updated weights for policy 0, policy_version 19340 (0.0010) [2023-10-08 08:35:38,147][53852] Updated weights for policy 0, policy_version 19350 (0.0008) [2023-10-08 08:35:38,518][53852] Updated weights for policy 0, policy_version 19360 (0.0008) [2023-10-08 08:35:40,361][53885] Updated weights for policy 1, policy_version 19270 (0.0009) [2023-10-08 08:35:40,729][53885] Updated weights for policy 1, policy_version 19280 (0.0009) [2023-10-08 08:35:41,106][53885] Updated weights for policy 1, policy_version 19290 (0.0010) [2023-10-08 08:35:42,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39583744. Throughput: 0: 1843.8, 1: 1803.8. Samples: 9900038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:35:42,016][52710] Avg episode reward: [(0, '23.480'), (1, '30.460')] [2023-10-08 08:35:42,188][53852] Updated weights for policy 0, policy_version 19370 (0.0009) [2023-10-08 08:35:42,549][53852] Updated weights for policy 0, policy_version 19380 (0.0007) [2023-10-08 08:35:42,915][53852] Updated weights for policy 0, policy_version 19390 (0.0007) [2023-10-08 08:35:44,944][53885] Updated weights for policy 1, policy_version 19300 (0.0009) [2023-10-08 08:35:45,313][53885] Updated weights for policy 1, policy_version 19310 (0.0007) [2023-10-08 08:35:45,686][53885] Updated weights for policy 1, policy_version 19320 (0.0008) [2023-10-08 08:35:46,505][53852] Updated weights for policy 0, policy_version 19400 (0.0007) [2023-10-08 08:35:46,872][53852] Updated weights for policy 0, policy_version 19410 (0.0007) [2023-10-08 08:35:47,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 39649280. Throughput: 0: 1844.5, 1: 1811.2. Samples: 9922020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:35:47,017][52710] Avg episode reward: [(0, '22.220'), (1, '30.930')] [2023-10-08 08:35:47,242][53852] Updated weights for policy 0, policy_version 19420 (0.0007) [2023-10-08 08:35:49,292][53885] Updated weights for policy 1, policy_version 19330 (0.0009) [2023-10-08 08:35:49,669][53885] Updated weights for policy 1, policy_version 19340 (0.0009) [2023-10-08 08:35:50,044][53885] Updated weights for policy 1, policy_version 19350 (0.0009) [2023-10-08 08:35:50,406][53885] Updated weights for policy 1, policy_version 19360 (0.0008) [2023-10-08 08:35:50,983][53852] Updated weights for policy 0, policy_version 19430 (0.0009) [2023-10-08 08:35:51,358][53852] Updated weights for policy 0, policy_version 19440 (0.0008) [2023-10-08 08:35:51,742][53852] Updated weights for policy 0, policy_version 19450 (0.0008) [2023-10-08 08:35:52,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 39747584. Throughput: 0: 1831.3, 1: 1811.8. Samples: 9943622. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:35:52,016][52710] Avg episode reward: [(0, '22.640'), (1, '28.440')] [2023-10-08 08:35:54,200][53885] Updated weights for policy 1, policy_version 19370 (0.0012) [2023-10-08 08:35:54,569][53885] Updated weights for policy 1, policy_version 19380 (0.0007) [2023-10-08 08:35:54,933][53885] Updated weights for policy 1, policy_version 19390 (0.0009) [2023-10-08 08:35:55,170][53852] Updated weights for policy 0, policy_version 19460 (0.0008) [2023-10-08 08:35:55,543][53852] Updated weights for policy 0, policy_version 19470 (0.0008) [2023-10-08 08:35:55,906][53852] Updated weights for policy 0, policy_version 19480 (0.0009) [2023-10-08 08:35:57,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 39813120. Throughput: 0: 1846.8, 1: 1821.0. Samples: 9955300. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:35:57,016][52710] Avg episode reward: [(0, '22.510'), (1, '28.010')] [2023-10-08 08:35:58,388][53885] Updated weights for policy 1, policy_version 19400 (0.0008) [2023-10-08 08:35:58,769][53885] Updated weights for policy 1, policy_version 19410 (0.0008) [2023-10-08 08:35:59,134][53885] Updated weights for policy 1, policy_version 19420 (0.0011) [2023-10-08 08:35:59,604][53852] Updated weights for policy 0, policy_version 19490 (0.0011) [2023-10-08 08:36:00,006][53852] Updated weights for policy 0, policy_version 19500 (0.0007) [2023-10-08 08:36:00,383][53852] Updated weights for policy 0, policy_version 19510 (0.0008) [2023-10-08 08:36:00,745][53852] Updated weights for policy 0, policy_version 19520 (0.0009) [2023-10-08 08:36:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 39878656. Throughput: 0: 1831.2, 1: 1826.4. Samples: 9976702. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:36:02,016][52710] Avg episode reward: [(0, '23.220'), (1, '27.530')] [2023-10-08 08:36:02,839][53885] Updated weights for policy 1, policy_version 19430 (0.0009) [2023-10-08 08:36:03,210][53885] Updated weights for policy 1, policy_version 19440 (0.0008) [2023-10-08 08:36:03,582][53885] Updated weights for policy 1, policy_version 19450 (0.0009) [2023-10-08 08:36:04,286][53852] Updated weights for policy 0, policy_version 19530 (0.0009) [2023-10-08 08:36:04,655][53852] Updated weights for policy 0, policy_version 19540 (0.0007) [2023-10-08 08:36:05,026][53852] Updated weights for policy 0, policy_version 19550 (0.0007) [2023-10-08 08:36:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 39944192. Throughput: 0: 1845.3, 1: 1825.5. Samples: 9999444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:36:07,016][52710] Avg episode reward: [(0, '21.180'), (1, '26.680')] [2023-10-08 08:36:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000019552_20021248.pth... [2023-10-08 08:36:07,059][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000017824_18251776.pth [2023-10-08 08:36:07,218][53885] Updated weights for policy 1, policy_version 19460 (0.0009) [2023-10-08 08:36:07,583][53885] Updated weights for policy 1, policy_version 19470 (0.0008) [2023-10-08 08:36:07,961][53885] Updated weights for policy 1, policy_version 19480 (0.0008) [2023-10-08 08:36:08,254][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000019488_19955712.pth... [2023-10-08 08:36:08,283][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000017760_18186240.pth [2023-10-08 08:36:08,775][53852] Updated weights for policy 0, policy_version 19560 (0.0010) [2023-10-08 08:36:09,154][53852] Updated weights for policy 0, policy_version 19570 (0.0010) [2023-10-08 08:36:09,530][53852] Updated weights for policy 0, policy_version 19580 (0.0010) [2023-10-08 08:36:11,853][53885] Updated weights for policy 1, policy_version 19490 (0.0009) [2023-10-08 08:36:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 40009728. Throughput: 0: 1833.1, 1: 1825.7. Samples: 10009682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:36:12,016][52710] Avg episode reward: [(0, '22.510'), (1, '27.300')] [2023-10-08 08:36:12,216][53885] Updated weights for policy 1, policy_version 19500 (0.0008) [2023-10-08 08:36:12,571][53885] Updated weights for policy 1, policy_version 19510 (0.0007) [2023-10-08 08:36:12,944][53885] Updated weights for policy 1, policy_version 19520 (0.0008) [2023-10-08 08:36:13,242][53852] Updated weights for policy 0, policy_version 19590 (0.0009) [2023-10-08 08:36:13,605][53852] Updated weights for policy 0, policy_version 19600 (0.0007) [2023-10-08 08:36:13,983][53852] Updated weights for policy 0, policy_version 19610 (0.0007) [2023-10-08 08:36:16,497][53885] Updated weights for policy 1, policy_version 19530 (0.0009) [2023-10-08 08:36:16,857][53885] Updated weights for policy 1, policy_version 19540 (0.0007) [2023-10-08 08:36:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40075264. Throughput: 0: 1844.8, 1: 1826.8. Samples: 10032366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:36:17,016][52710] Avg episode reward: [(0, '22.810'), (1, '29.210')] [2023-10-08 08:36:17,223][53885] Updated weights for policy 1, policy_version 19550 (0.0008) [2023-10-08 08:36:17,541][53852] Updated weights for policy 0, policy_version 19620 (0.0008) [2023-10-08 08:36:17,914][53852] Updated weights for policy 0, policy_version 19630 (0.0008) [2023-10-08 08:36:18,282][53852] Updated weights for policy 0, policy_version 19640 (0.0009) [2023-10-08 08:36:20,919][53885] Updated weights for policy 1, policy_version 19560 (0.0007) [2023-10-08 08:36:21,288][53885] Updated weights for policy 1, policy_version 19570 (0.0007) [2023-10-08 08:36:21,652][53885] Updated weights for policy 1, policy_version 19580 (0.0008) [2023-10-08 08:36:21,705][53852] Updated weights for policy 0, policy_version 19650 (0.0007) [2023-10-08 08:36:22,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 40173568. Throughput: 0: 1852.3, 1: 1826.9. Samples: 10054534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:36:22,016][52710] Avg episode reward: [(0, '21.250'), (1, '30.200')] [2023-10-08 08:36:22,076][53852] Updated weights for policy 0, policy_version 19660 (0.0010) [2023-10-08 08:36:22,435][53852] Updated weights for policy 0, policy_version 19670 (0.0011) [2023-10-08 08:36:22,809][53852] Updated weights for policy 0, policy_version 19680 (0.0009) [2023-10-08 08:36:25,394][53885] Updated weights for policy 1, policy_version 19590 (0.0008) [2023-10-08 08:36:25,765][53885] Updated weights for policy 1, policy_version 19600 (0.0011) [2023-10-08 08:36:26,130][53885] Updated weights for policy 1, policy_version 19610 (0.0011) [2023-10-08 08:36:26,593][53852] Updated weights for policy 0, policy_version 19690 (0.0008) [2023-10-08 08:36:26,967][53852] Updated weights for policy 0, policy_version 19700 (0.0007) [2023-10-08 08:36:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 40239104. Throughput: 0: 1851.4, 1: 1827.9. Samples: 10065604. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-08 08:36:27,016][52710] Avg episode reward: [(0, '21.520'), (1, '30.420')] [2023-10-08 08:36:27,334][53852] Updated weights for policy 0, policy_version 19710 (0.0007) [2023-10-08 08:36:29,794][53885] Updated weights for policy 1, policy_version 19620 (0.0009) [2023-10-08 08:36:30,166][53885] Updated weights for policy 1, policy_version 19630 (0.0011) [2023-10-08 08:36:30,539][53885] Updated weights for policy 1, policy_version 19640 (0.0009) [2023-10-08 08:36:30,961][53852] Updated weights for policy 0, policy_version 19720 (0.0009) [2023-10-08 08:36:31,327][53852] Updated weights for policy 0, policy_version 19730 (0.0009) [2023-10-08 08:36:31,706][53852] Updated weights for policy 0, policy_version 19740 (0.0007) [2023-10-08 08:36:32,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 40337408. Throughput: 0: 1847.1, 1: 1823.5. Samples: 10087194. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-08 08:36:32,016][52710] Avg episode reward: [(0, '21.070'), (1, '30.700')] [2023-10-08 08:36:34,272][53885] Updated weights for policy 1, policy_version 19650 (0.0009) [2023-10-08 08:36:34,637][53885] Updated weights for policy 1, policy_version 19660 (0.0007) [2023-10-08 08:36:35,012][53885] Updated weights for policy 1, policy_version 19670 (0.0010) [2023-10-08 08:36:35,235][53852] Updated weights for policy 0, policy_version 19750 (0.0007) [2023-10-08 08:36:35,371][53885] Updated weights for policy 1, policy_version 19680 (0.0007) [2023-10-08 08:36:35,604][53852] Updated weights for policy 0, policy_version 19760 (0.0009) [2023-10-08 08:36:35,978][53852] Updated weights for policy 0, policy_version 19770 (0.0009) [2023-10-08 08:36:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 40402944. Throughput: 0: 1839.8, 1: 1821.9. Samples: 10108402. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) [2023-10-08 08:36:37,016][52710] Avg episode reward: [(0, '20.350'), (1, '31.410')] [2023-10-08 08:36:39,068][53885] Updated weights for policy 1, policy_version 19690 (0.0009) [2023-10-08 08:36:39,435][53885] Updated weights for policy 1, policy_version 19700 (0.0009) [2023-10-08 08:36:39,613][53852] Updated weights for policy 0, policy_version 19780 (0.0008) [2023-10-08 08:36:39,794][53885] Updated weights for policy 1, policy_version 19710 (0.0007) [2023-10-08 08:36:39,981][53852] Updated weights for policy 0, policy_version 19790 (0.0009) [2023-10-08 08:36:40,345][53852] Updated weights for policy 0, policy_version 19800 (0.0008) [2023-10-08 08:36:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 40468480. Throughput: 0: 1850.1, 1: 1820.5. Samples: 10120476. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:36:42,016][52710] Avg episode reward: [(0, '21.810'), (1, '28.140')] [2023-10-08 08:36:43,411][53885] Updated weights for policy 1, policy_version 19720 (0.0007) [2023-10-08 08:36:43,781][53885] Updated weights for policy 1, policy_version 19730 (0.0009) [2023-10-08 08:36:44,012][53852] Updated weights for policy 0, policy_version 19810 (0.0010) [2023-10-08 08:36:44,146][53885] Updated weights for policy 1, policy_version 19740 (0.0009) [2023-10-08 08:36:44,372][53852] Updated weights for policy 0, policy_version 19820 (0.0008) [2023-10-08 08:36:44,743][53852] Updated weights for policy 0, policy_version 19830 (0.0009) [2023-10-08 08:36:45,120][53852] Updated weights for policy 0, policy_version 19840 (0.0009) [2023-10-08 08:36:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 40534016. Throughput: 0: 1845.6, 1: 1819.3. Samples: 10141626. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:36:47,016][52710] Avg episode reward: [(0, '23.080'), (1, '28.660')] [2023-10-08 08:36:47,837][53885] Updated weights for policy 1, policy_version 19750 (0.0008) [2023-10-08 08:36:48,208][53885] Updated weights for policy 1, policy_version 19760 (0.0008) [2023-10-08 08:36:48,570][53885] Updated weights for policy 1, policy_version 19770 (0.0010) [2023-10-08 08:36:48,890][53852] Updated weights for policy 0, policy_version 19850 (0.0007) [2023-10-08 08:36:49,250][53852] Updated weights for policy 0, policy_version 19860 (0.0010) [2023-10-08 08:36:49,618][53852] Updated weights for policy 0, policy_version 19870 (0.0010) [2023-10-08 08:36:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 40599552. Throughput: 0: 1843.6, 1: 1826.7. Samples: 10164608. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:36:52,016][52710] Avg episode reward: [(0, '20.920'), (1, '28.510')] [2023-10-08 08:36:52,240][53885] Updated weights for policy 1, policy_version 19780 (0.0007) [2023-10-08 08:36:52,601][53885] Updated weights for policy 1, policy_version 19790 (0.0008) [2023-10-08 08:36:52,970][53885] Updated weights for policy 1, policy_version 19800 (0.0007) [2023-10-08 08:36:53,358][53852] Updated weights for policy 0, policy_version 19880 (0.0007) [2023-10-08 08:36:53,727][53852] Updated weights for policy 0, policy_version 19890 (0.0007) [2023-10-08 08:36:54,102][53852] Updated weights for policy 0, policy_version 19900 (0.0007) [2023-10-08 08:36:56,734][53885] Updated weights for policy 1, policy_version 19810 (0.0009) [2023-10-08 08:36:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40665088. Throughput: 0: 1837.3, 1: 1823.6. Samples: 10174422. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:36:57,016][52710] Avg episode reward: [(0, '21.150'), (1, '28.700')] [2023-10-08 08:36:57,100][53885] Updated weights for policy 1, policy_version 19820 (0.0008) [2023-10-08 08:36:57,471][53885] Updated weights for policy 1, policy_version 19830 (0.0008) [2023-10-08 08:36:57,736][53852] Updated weights for policy 0, policy_version 19910 (0.0008) [2023-10-08 08:36:57,836][53885] Updated weights for policy 1, policy_version 19840 (0.0008) [2023-10-08 08:36:58,114][53852] Updated weights for policy 0, policy_version 19920 (0.0007) [2023-10-08 08:36:58,491][53852] Updated weights for policy 0, policy_version 19930 (0.0008) [2023-10-08 08:37:01,661][53885] Updated weights for policy 1, policy_version 19850 (0.0010) [2023-10-08 08:37:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40730624. Throughput: 0: 1842.7, 1: 1818.0. Samples: 10197094. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-08 08:37:02,016][52710] Avg episode reward: [(0, '21.420'), (1, '27.440')] [2023-10-08 08:37:02,027][53885] Updated weights for policy 1, policy_version 19860 (0.0009) [2023-10-08 08:37:02,078][53852] Updated weights for policy 0, policy_version 19940 (0.0009) [2023-10-08 08:37:02,393][53885] Updated weights for policy 1, policy_version 19870 (0.0007) [2023-10-08 08:37:02,454][53852] Updated weights for policy 0, policy_version 19950 (0.0007) [2023-10-08 08:37:02,820][53852] Updated weights for policy 0, policy_version 19960 (0.0007) [2023-10-08 08:37:06,023][53885] Updated weights for policy 1, policy_version 19880 (0.0008) [2023-10-08 08:37:06,393][53885] Updated weights for policy 1, policy_version 19890 (0.0010) [2023-10-08 08:37:06,426][53852] Updated weights for policy 0, policy_version 19970 (0.0008) [2023-10-08 08:37:06,767][53885] Updated weights for policy 1, policy_version 19900 (0.0009) [2023-10-08 08:37:06,798][53852] Updated weights for policy 0, policy_version 19980 (0.0008) [2023-10-08 08:37:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 40828928. Throughput: 0: 1828.8, 1: 1819.2. Samples: 10218698. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-08 08:37:07,016][52710] Avg episode reward: [(0, '21.890'), (1, '27.640')] [2023-10-08 08:37:07,167][53852] Updated weights for policy 0, policy_version 19990 (0.0007) [2023-10-08 08:37:07,534][53852] Updated weights for policy 0, policy_version 20000 (0.0008) [2023-10-08 08:37:10,499][53885] Updated weights for policy 1, policy_version 19910 (0.0009) [2023-10-08 08:37:10,887][53885] Updated weights for policy 1, policy_version 19920 (0.0008) [2023-10-08 08:37:11,154][53852] Updated weights for policy 0, policy_version 20010 (0.0008) [2023-10-08 08:37:11,254][53885] Updated weights for policy 1, policy_version 19930 (0.0009) [2023-10-08 08:37:11,518][53852] Updated weights for policy 0, policy_version 20020 (0.0008) [2023-10-08 08:37:11,887][53852] Updated weights for policy 0, policy_version 20030 (0.0009) [2023-10-08 08:37:12,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 40927232. Throughput: 0: 1835.8, 1: 1816.7. Samples: 10229966. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) [2023-10-08 08:37:12,015][52710] Avg episode reward: [(0, '23.480'), (1, '26.640')] [2023-10-08 08:37:15,027][53885] Updated weights for policy 1, policy_version 19940 (0.0007) [2023-10-08 08:37:15,387][53885] Updated weights for policy 1, policy_version 19950 (0.0009) [2023-10-08 08:37:15,450][53852] Updated weights for policy 0, policy_version 20040 (0.0008) [2023-10-08 08:37:15,766][53885] Updated weights for policy 1, policy_version 19960 (0.0008) [2023-10-08 08:37:15,822][53852] Updated weights for policy 0, policy_version 20050 (0.0010) [2023-10-08 08:37:16,196][53852] Updated weights for policy 0, policy_version 20060 (0.0008) [2023-10-08 08:37:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 40992768. Throughput: 0: 1831.2, 1: 1821.0. Samples: 10251544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:37:17,016][52710] Avg episode reward: [(0, '20.370'), (1, '25.610')] [2023-10-08 08:37:19,308][53885] Updated weights for policy 1, policy_version 19970 (0.0008) [2023-10-08 08:37:19,691][53885] Updated weights for policy 1, policy_version 19980 (0.0009) [2023-10-08 08:37:19,696][53852] Updated weights for policy 0, policy_version 20070 (0.0007) [2023-10-08 08:37:20,050][53885] Updated weights for policy 1, policy_version 19990 (0.0007) [2023-10-08 08:37:20,071][53852] Updated weights for policy 0, policy_version 20080 (0.0008) [2023-10-08 08:37:20,419][53885] Updated weights for policy 1, policy_version 20000 (0.0007) [2023-10-08 08:37:20,433][53852] Updated weights for policy 0, policy_version 20090 (0.0008) [2023-10-08 08:37:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 41058304. Throughput: 0: 1841.2, 1: 1817.0. Samples: 10273022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:37:22,016][52710] Avg episode reward: [(0, '21.130'), (1, '25.560')] [2023-10-08 08:37:24,162][53885] Updated weights for policy 1, policy_version 20010 (0.0007) [2023-10-08 08:37:24,291][53852] Updated weights for policy 0, policy_version 20100 (0.0008) [2023-10-08 08:37:24,521][53885] Updated weights for policy 1, policy_version 20020 (0.0007) [2023-10-08 08:37:24,664][53852] Updated weights for policy 0, policy_version 20110 (0.0007) [2023-10-08 08:37:24,898][53885] Updated weights for policy 1, policy_version 20030 (0.0009) [2023-10-08 08:37:25,037][53852] Updated weights for policy 0, policy_version 20120 (0.0007) [2023-10-08 08:37:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 41123840. Throughput: 0: 1824.1, 1: 1817.7. Samples: 10284354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:37:27,016][52710] Avg episode reward: [(0, '21.320'), (1, '25.220')] [2023-10-08 08:37:28,427][53885] Updated weights for policy 1, policy_version 20040 (0.0009) [2023-10-08 08:37:28,715][53852] Updated weights for policy 0, policy_version 20130 (0.0008) [2023-10-08 08:37:28,798][53885] Updated weights for policy 1, policy_version 20050 (0.0010) [2023-10-08 08:37:29,077][53852] Updated weights for policy 0, policy_version 20140 (0.0009) [2023-10-08 08:37:29,161][53885] Updated weights for policy 1, policy_version 20060 (0.0007) [2023-10-08 08:37:29,450][53852] Updated weights for policy 0, policy_version 20150 (0.0008) [2023-10-08 08:37:29,824][53852] Updated weights for policy 0, policy_version 20160 (0.0007) [2023-10-08 08:37:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 41189376. Throughput: 0: 1829.6, 1: 1816.7. Samples: 10305706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:37:32,016][52710] Avg episode reward: [(0, '21.860'), (1, '25.780')] [2023-10-08 08:37:32,999][53885] Updated weights for policy 1, policy_version 20070 (0.0007) [2023-10-08 08:37:33,375][53885] Updated weights for policy 1, policy_version 20080 (0.0008) [2023-10-08 08:37:33,587][53852] Updated weights for policy 0, policy_version 20170 (0.0008) [2023-10-08 08:37:33,739][53885] Updated weights for policy 1, policy_version 20090 (0.0007) [2023-10-08 08:37:33,958][53852] Updated weights for policy 0, policy_version 20180 (0.0008) [2023-10-08 08:37:34,331][53852] Updated weights for policy 0, policy_version 20190 (0.0009) [2023-10-08 08:37:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41254912. Throughput: 0: 1836.5, 1: 1807.2. Samples: 10328576. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 08:37:37,016][52710] Avg episode reward: [(0, '22.900'), (1, '23.760')] [2023-10-08 08:37:37,469][53885] Updated weights for policy 1, policy_version 20100 (0.0007) [2023-10-08 08:37:37,828][53885] Updated weights for policy 1, policy_version 20110 (0.0009) [2023-10-08 08:37:37,889][53852] Updated weights for policy 0, policy_version 20200 (0.0008) [2023-10-08 08:37:38,193][53885] Updated weights for policy 1, policy_version 20120 (0.0009) [2023-10-08 08:37:38,265][53852] Updated weights for policy 0, policy_version 20210 (0.0009) [2023-10-08 08:37:38,638][53852] Updated weights for policy 0, policy_version 20220 (0.0008) [2023-10-08 08:37:41,801][53885] Updated weights for policy 1, policy_version 20130 (0.0008) [2023-10-08 08:37:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41320448. Throughput: 0: 1837.9, 1: 1808.3. Samples: 10338498. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 08:37:42,015][52710] Avg episode reward: [(0, '21.870'), (1, '22.090')] [2023-10-08 08:37:42,172][53885] Updated weights for policy 1, policy_version 20140 (0.0011) [2023-10-08 08:37:42,499][53852] Updated weights for policy 0, policy_version 20230 (0.0008) [2023-10-08 08:37:42,553][53885] Updated weights for policy 1, policy_version 20150 (0.0007) [2023-10-08 08:37:42,871][53852] Updated weights for policy 0, policy_version 20240 (0.0008) [2023-10-08 08:37:42,924][53885] Updated weights for policy 1, policy_version 20160 (0.0008) [2023-10-08 08:37:43,238][53852] Updated weights for policy 0, policy_version 20250 (0.0008) [2023-10-08 08:37:46,665][53885] Updated weights for policy 1, policy_version 20170 (0.0007) [2023-10-08 08:37:46,860][53852] Updated weights for policy 0, policy_version 20260 (0.0010) [2023-10-08 08:37:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41385984. Throughput: 0: 1828.4, 1: 1815.9. Samples: 10361090. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 08:37:47,016][52710] Avg episode reward: [(0, '21.390'), (1, '23.000')] [2023-10-08 08:37:47,031][53885] Updated weights for policy 1, policy_version 20180 (0.0007) [2023-10-08 08:37:47,225][53852] Updated weights for policy 0, policy_version 20270 (0.0007) [2023-10-08 08:37:47,400][53885] Updated weights for policy 1, policy_version 20190 (0.0008) [2023-10-08 08:37:47,602][53852] Updated weights for policy 0, policy_version 20280 (0.0007) [2023-10-08 08:37:51,090][53852] Updated weights for policy 0, policy_version 20290 (0.0009) [2023-10-08 08:37:51,112][53885] Updated weights for policy 1, policy_version 20200 (0.0007) [2023-10-08 08:37:51,460][53852] Updated weights for policy 0, policy_version 20300 (0.0007) [2023-10-08 08:37:51,476][53885] Updated weights for policy 1, policy_version 20210 (0.0008) [2023-10-08 08:37:51,839][53852] Updated weights for policy 0, policy_version 20310 (0.0007) [2023-10-08 08:37:51,843][53885] Updated weights for policy 1, policy_version 20220 (0.0009) [2023-10-08 08:37:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 41484288. Throughput: 0: 1825.5, 1: 1815.6. Samples: 10382548. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 08:37:52,016][52710] Avg episode reward: [(0, '22.440'), (1, '21.190')] [2023-10-08 08:37:52,206][53852] Updated weights for policy 0, policy_version 20320 (0.0007) [2023-10-08 08:37:55,523][53885] Updated weights for policy 1, policy_version 20230 (0.0009) [2023-10-08 08:37:55,892][53885] Updated weights for policy 1, policy_version 20240 (0.0009) [2023-10-08 08:37:55,932][53852] Updated weights for policy 0, policy_version 20330 (0.0007) [2023-10-08 08:37:56,258][53885] Updated weights for policy 1, policy_version 20250 (0.0007) [2023-10-08 08:37:56,300][53852] Updated weights for policy 0, policy_version 20340 (0.0007) [2023-10-08 08:37:56,681][53852] Updated weights for policy 0, policy_version 20350 (0.0007) [2023-10-08 08:37:57,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 41582592. Throughput: 0: 1832.2, 1: 1813.4. Samples: 10394018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:37:57,016][52710] Avg episode reward: [(0, '21.680'), (1, '21.790')] [2023-10-08 08:37:59,862][53885] Updated weights for policy 1, policy_version 20260 (0.0009) [2023-10-08 08:38:00,241][53885] Updated weights for policy 1, policy_version 20270 (0.0007) [2023-10-08 08:38:00,272][53852] Updated weights for policy 0, policy_version 20360 (0.0009) [2023-10-08 08:38:00,609][53885] Updated weights for policy 1, policy_version 20280 (0.0007) [2023-10-08 08:38:00,647][53852] Updated weights for policy 0, policy_version 20370 (0.0009) [2023-10-08 08:38:01,004][53852] Updated weights for policy 0, policy_version 20380 (0.0010) [2023-10-08 08:38:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 41648128. Throughput: 0: 1827.6, 1: 1818.0. Samples: 10415596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:38:02,016][52710] Avg episode reward: [(0, '20.060'), (1, '21.380')] [2023-10-08 08:38:04,246][53885] Updated weights for policy 1, policy_version 20290 (0.0010) [2023-10-08 08:38:04,615][53885] Updated weights for policy 1, policy_version 20300 (0.0009) [2023-10-08 08:38:04,664][53852] Updated weights for policy 0, policy_version 20390 (0.0008) [2023-10-08 08:38:04,988][53885] Updated weights for policy 1, policy_version 20310 (0.0008) [2023-10-08 08:38:05,030][53852] Updated weights for policy 0, policy_version 20400 (0.0008) [2023-10-08 08:38:05,359][53885] Updated weights for policy 1, policy_version 20320 (0.0009) [2023-10-08 08:38:05,403][53852] Updated weights for policy 0, policy_version 20410 (0.0009) [2023-10-08 08:38:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 41713664. Throughput: 0: 1827.0, 1: 1818.6. Samples: 10437074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:38:07,016][52710] Avg episode reward: [(0, '21.450'), (1, '23.360')] [2023-10-08 08:38:07,030][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000020416_20905984.pth... [2023-10-08 08:38:07,030][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000020320_20807680.pth... [2023-10-08 08:38:07,066][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000018624_19070976.pth [2023-10-08 08:38:07,066][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000018688_19136512.pth [2023-10-08 08:38:09,020][53852] Updated weights for policy 0, policy_version 20420 (0.0010) [2023-10-08 08:38:09,021][53885] Updated weights for policy 1, policy_version 20330 (0.0009) [2023-10-08 08:38:09,384][53885] Updated weights for policy 1, policy_version 20340 (0.0009) [2023-10-08 08:38:09,391][53852] Updated weights for policy 0, policy_version 20430 (0.0008) [2023-10-08 08:38:09,756][53885] Updated weights for policy 1, policy_version 20350 (0.0008) [2023-10-08 08:38:09,758][53852] Updated weights for policy 0, policy_version 20440 (0.0008) [2023-10-08 08:38:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 41779200. Throughput: 0: 1821.8, 1: 1817.0. Samples: 10448100. Policy #0 lag: (min: 11.0, avg: 30.1, max: 32.0) [2023-10-08 08:38:12,016][52710] Avg episode reward: [(0, '18.740'), (1, '25.950')] [2023-10-08 08:38:13,452][53885] Updated weights for policy 1, policy_version 20360 (0.0007) [2023-10-08 08:38:13,503][53852] Updated weights for policy 0, policy_version 20450 (0.0008) [2023-10-08 08:38:13,819][53885] Updated weights for policy 1, policy_version 20370 (0.0007) [2023-10-08 08:38:13,867][53852] Updated weights for policy 0, policy_version 20460 (0.0008) [2023-10-08 08:38:14,185][53885] Updated weights for policy 1, policy_version 20380 (0.0007) [2023-10-08 08:38:14,230][53852] Updated weights for policy 0, policy_version 20470 (0.0007) [2023-10-08 08:38:14,603][53852] Updated weights for policy 0, policy_version 20480 (0.0007) [2023-10-08 08:38:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 41844736. Throughput: 0: 1837.9, 1: 1815.6. Samples: 10470114. Policy #0 lag: (min: 11.0, avg: 30.1, max: 32.0) [2023-10-08 08:38:17,016][52710] Avg episode reward: [(0, '21.270'), (1, '24.780')] [2023-10-08 08:38:17,862][53885] Updated weights for policy 1, policy_version 20390 (0.0009) [2023-10-08 08:38:18,183][53852] Updated weights for policy 0, policy_version 20490 (0.0009) [2023-10-08 08:38:18,229][53885] Updated weights for policy 1, policy_version 20400 (0.0007) [2023-10-08 08:38:18,554][53852] Updated weights for policy 0, policy_version 20500 (0.0008) [2023-10-08 08:38:18,594][53885] Updated weights for policy 1, policy_version 20410 (0.0007) [2023-10-08 08:38:18,922][53852] Updated weights for policy 0, policy_version 20510 (0.0007) [2023-10-08 08:38:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41910272. Throughput: 0: 1838.6, 1: 1821.6. Samples: 10493282. Policy #0 lag: (min: 11.0, avg: 30.1, max: 32.0) [2023-10-08 08:38:22,016][52710] Avg episode reward: [(0, '22.100'), (1, '26.980')] [2023-10-08 08:38:22,425][53885] Updated weights for policy 1, policy_version 20420 (0.0008) [2023-10-08 08:38:22,652][53852] Updated weights for policy 0, policy_version 20520 (0.0008) [2023-10-08 08:38:22,794][53885] Updated weights for policy 1, policy_version 20430 (0.0008) [2023-10-08 08:38:23,023][53852] Updated weights for policy 0, policy_version 20530 (0.0008) [2023-10-08 08:38:23,172][53885] Updated weights for policy 1, policy_version 20440 (0.0010) [2023-10-08 08:38:23,403][53852] Updated weights for policy 0, policy_version 20540 (0.0008) [2023-10-08 08:38:26,873][53885] Updated weights for policy 1, policy_version 20450 (0.0009) [2023-10-08 08:38:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41975808. Throughput: 0: 1832.3, 1: 1820.1. Samples: 10502854. Policy #0 lag: (min: 11.0, avg: 30.1, max: 32.0) [2023-10-08 08:38:27,016][52710] Avg episode reward: [(0, '22.080'), (1, '25.780')] [2023-10-08 08:38:27,046][53852] Updated weights for policy 0, policy_version 20550 (0.0009) [2023-10-08 08:38:27,236][53885] Updated weights for policy 1, policy_version 20460 (0.0007) [2023-10-08 08:38:27,416][53852] Updated weights for policy 0, policy_version 20560 (0.0007) [2023-10-08 08:38:27,608][53885] Updated weights for policy 1, policy_version 20470 (0.0008) [2023-10-08 08:38:27,790][53852] Updated weights for policy 0, policy_version 20570 (0.0007) [2023-10-08 08:38:27,963][53885] Updated weights for policy 1, policy_version 20480 (0.0009) [2023-10-08 08:38:31,422][53852] Updated weights for policy 0, policy_version 20580 (0.0009) [2023-10-08 08:38:31,693][53885] Updated weights for policy 1, policy_version 20490 (0.0009) [2023-10-08 08:38:31,799][53852] Updated weights for policy 0, policy_version 20590 (0.0008) [2023-10-08 08:38:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42041344. Throughput: 0: 1840.3, 1: 1814.3. Samples: 10525546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:38:32,015][52710] Avg episode reward: [(0, '21.630'), (1, '27.890')] [2023-10-08 08:38:32,068][53885] Updated weights for policy 1, policy_version 20500 (0.0007) [2023-10-08 08:38:32,164][53852] Updated weights for policy 0, policy_version 20600 (0.0008) [2023-10-08 08:38:32,427][53885] Updated weights for policy 1, policy_version 20510 (0.0008) [2023-10-08 08:38:35,812][53852] Updated weights for policy 0, policy_version 20610 (0.0007) [2023-10-08 08:38:36,147][53885] Updated weights for policy 1, policy_version 20520 (0.0008) [2023-10-08 08:38:36,176][53852] Updated weights for policy 0, policy_version 20620 (0.0007) [2023-10-08 08:38:36,516][53885] Updated weights for policy 1, policy_version 20530 (0.0008) [2023-10-08 08:38:36,545][53852] Updated weights for policy 0, policy_version 20630 (0.0009) [2023-10-08 08:38:36,877][53885] Updated weights for policy 1, policy_version 20540 (0.0009) [2023-10-08 08:38:36,903][53852] Updated weights for policy 0, policy_version 20640 (0.0008) [2023-10-08 08:38:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42139648. Throughput: 0: 1824.8, 1: 1818.3. Samples: 10546486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:38:37,016][52710] Avg episode reward: [(0, '23.130'), (1, '26.610')] [2023-10-08 08:38:40,603][53852] Updated weights for policy 0, policy_version 20650 (0.0008) [2023-10-08 08:38:40,622][53885] Updated weights for policy 1, policy_version 20550 (0.0007) [2023-10-08 08:38:40,972][53852] Updated weights for policy 0, policy_version 20660 (0.0008) [2023-10-08 08:38:41,003][53885] Updated weights for policy 1, policy_version 20560 (0.0010) [2023-10-08 08:38:41,332][53852] Updated weights for policy 0, policy_version 20670 (0.0008) [2023-10-08 08:38:41,376][53885] Updated weights for policy 1, policy_version 20570 (0.0007) [2023-10-08 08:38:42,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 42237952. Throughput: 0: 1834.8, 1: 1822.0. Samples: 10558572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:38:42,016][52710] Avg episode reward: [(0, '22.460'), (1, '28.780')] [2023-10-08 08:38:44,981][53885] Updated weights for policy 1, policy_version 20580 (0.0007) [2023-10-08 08:38:45,043][53852] Updated weights for policy 0, policy_version 20680 (0.0009) [2023-10-08 08:38:45,342][53885] Updated weights for policy 1, policy_version 20590 (0.0008) [2023-10-08 08:38:45,403][53852] Updated weights for policy 0, policy_version 20690 (0.0009) [2023-10-08 08:38:45,713][53885] Updated weights for policy 1, policy_version 20600 (0.0008) [2023-10-08 08:38:45,773][53852] Updated weights for policy 0, policy_version 20700 (0.0010) [2023-10-08 08:38:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 42303488. Throughput: 0: 1825.1, 1: 1814.2. Samples: 10579364. Policy #0 lag: (min: 19.0, avg: 23.5, max: 51.0) [2023-10-08 08:38:47,016][52710] Avg episode reward: [(0, '22.570'), (1, '27.490')] [2023-10-08 08:38:49,458][53852] Updated weights for policy 0, policy_version 20710 (0.0009) [2023-10-08 08:38:49,509][53885] Updated weights for policy 1, policy_version 20610 (0.0008) [2023-10-08 08:38:49,829][53852] Updated weights for policy 0, policy_version 20720 (0.0008) [2023-10-08 08:38:49,880][53885] Updated weights for policy 1, policy_version 20620 (0.0009) [2023-10-08 08:38:50,201][53852] Updated weights for policy 0, policy_version 20730 (0.0008) [2023-10-08 08:38:50,247][53885] Updated weights for policy 1, policy_version 20630 (0.0008) [2023-10-08 08:38:50,620][53885] Updated weights for policy 1, policy_version 20640 (0.0008) [2023-10-08 08:38:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42369024. Throughput: 0: 1834.3, 1: 1808.3. Samples: 10600990. Policy #0 lag: (min: 19.0, avg: 23.5, max: 51.0) [2023-10-08 08:38:52,016][52710] Avg episode reward: [(0, '22.750'), (1, '27.280')] [2023-10-08 08:38:53,847][53852] Updated weights for policy 0, policy_version 20740 (0.0009) [2023-10-08 08:38:54,205][53852] Updated weights for policy 0, policy_version 20750 (0.0008) [2023-10-08 08:38:54,342][53885] Updated weights for policy 1, policy_version 20650 (0.0008) [2023-10-08 08:38:54,571][53852] Updated weights for policy 0, policy_version 20760 (0.0007) [2023-10-08 08:38:54,707][53885] Updated weights for policy 1, policy_version 20660 (0.0008) [2023-10-08 08:38:55,069][53885] Updated weights for policy 1, policy_version 20670 (0.0008) [2023-10-08 08:38:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 42434560. Throughput: 0: 1829.7, 1: 1813.8. Samples: 10612058. Policy #0 lag: (min: 19.0, avg: 23.5, max: 51.0) [2023-10-08 08:38:57,016][52710] Avg episode reward: [(0, '24.810'), (1, '29.320')] [2023-10-08 08:38:58,335][53852] Updated weights for policy 0, policy_version 20770 (0.0008) [2023-10-08 08:38:58,702][53852] Updated weights for policy 0, policy_version 20780 (0.0007) [2023-10-08 08:38:58,837][53885] Updated weights for policy 1, policy_version 20680 (0.0008) [2023-10-08 08:38:59,070][53852] Updated weights for policy 0, policy_version 20790 (0.0007) [2023-10-08 08:38:59,211][53885] Updated weights for policy 1, policy_version 20690 (0.0007) [2023-10-08 08:38:59,436][53852] Updated weights for policy 0, policy_version 20800 (0.0007) [2023-10-08 08:38:59,578][53885] Updated weights for policy 1, policy_version 20700 (0.0008) [2023-10-08 08:39:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 42500096. Throughput: 0: 1832.0, 1: 1804.9. Samples: 10633772. Policy #0 lag: (min: 19.0, avg: 23.5, max: 51.0) [2023-10-08 08:39:02,016][52710] Avg episode reward: [(0, '23.170'), (1, '24.460')] [2023-10-08 08:39:02,991][53852] Updated weights for policy 0, policy_version 20810 (0.0007) [2023-10-08 08:39:03,208][53885] Updated weights for policy 1, policy_version 20710 (0.0009) [2023-10-08 08:39:03,353][53852] Updated weights for policy 0, policy_version 20820 (0.0007) [2023-10-08 08:39:03,580][53885] Updated weights for policy 1, policy_version 20720 (0.0008) [2023-10-08 08:39:03,727][53852] Updated weights for policy 0, policy_version 20830 (0.0009) [2023-10-08 08:39:03,941][53885] Updated weights for policy 1, policy_version 20730 (0.0009) [2023-10-08 08:39:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42565632. Throughput: 0: 1829.4, 1: 1802.3. Samples: 10656708. Policy #0 lag: (min: 20.0, avg: 23.1, max: 52.0) [2023-10-08 08:39:07,016][52710] Avg episode reward: [(0, '23.950'), (1, '26.930')] [2023-10-08 08:39:07,382][53852] Updated weights for policy 0, policy_version 20840 (0.0013) [2023-10-08 08:39:07,749][53885] Updated weights for policy 1, policy_version 20740 (0.0008) [2023-10-08 08:39:07,750][53852] Updated weights for policy 0, policy_version 20850 (0.0007) [2023-10-08 08:39:08,117][53885] Updated weights for policy 1, policy_version 20750 (0.0009) [2023-10-08 08:39:08,120][53852] Updated weights for policy 0, policy_version 20860 (0.0007) [2023-10-08 08:39:08,493][53885] Updated weights for policy 1, policy_version 20760 (0.0007) [2023-10-08 08:39:11,863][53852] Updated weights for policy 0, policy_version 20870 (0.0007) [2023-10-08 08:39:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42631168. Throughput: 0: 1834.6, 1: 1802.3. Samples: 10666512. Policy #0 lag: (min: 20.0, avg: 23.1, max: 52.0) [2023-10-08 08:39:12,015][52710] Avg episode reward: [(0, '23.220'), (1, '25.070')] [2023-10-08 08:39:12,173][53885] Updated weights for policy 1, policy_version 20770 (0.0008) [2023-10-08 08:39:12,248][53852] Updated weights for policy 0, policy_version 20880 (0.0008) [2023-10-08 08:39:12,544][53885] Updated weights for policy 1, policy_version 20780 (0.0007) [2023-10-08 08:39:12,620][53852] Updated weights for policy 0, policy_version 20890 (0.0007) [2023-10-08 08:39:12,925][53885] Updated weights for policy 1, policy_version 20790 (0.0009) [2023-10-08 08:39:13,284][53885] Updated weights for policy 1, policy_version 20800 (0.0008) [2023-10-08 08:39:16,236][53852] Updated weights for policy 0, policy_version 20900 (0.0007) [2023-10-08 08:39:16,611][53852] Updated weights for policy 0, policy_version 20910 (0.0008) [2023-10-08 08:39:16,982][53852] Updated weights for policy 0, policy_version 20920 (0.0009) [2023-10-08 08:39:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42696704. Throughput: 0: 1834.1, 1: 1803.8. Samples: 10689250. Policy #0 lag: (min: 20.0, avg: 23.1, max: 52.0) [2023-10-08 08:39:17,016][52710] Avg episode reward: [(0, '22.460'), (1, '27.120')] [2023-10-08 08:39:17,098][53885] Updated weights for policy 1, policy_version 20810 (0.0009) [2023-10-08 08:39:17,468][53885] Updated weights for policy 1, policy_version 20820 (0.0008) [2023-10-08 08:39:17,842][53885] Updated weights for policy 1, policy_version 20830 (0.0009) [2023-10-08 08:39:20,683][53852] Updated weights for policy 0, policy_version 20930 (0.0007) [2023-10-08 08:39:21,058][53852] Updated weights for policy 0, policy_version 20940 (0.0008) [2023-10-08 08:39:21,427][53852] Updated weights for policy 0, policy_version 20950 (0.0008) [2023-10-08 08:39:21,536][53885] Updated weights for policy 1, policy_version 20840 (0.0009) [2023-10-08 08:39:21,798][53852] Updated weights for policy 0, policy_version 20960 (0.0007) [2023-10-08 08:39:21,909][53885] Updated weights for policy 1, policy_version 20850 (0.0008) [2023-10-08 08:39:22,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 42795008. Throughput: 0: 1829.6, 1: 1815.6. Samples: 10710518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:39:22,016][52710] Avg episode reward: [(0, '22.840'), (1, '25.770')] [2023-10-08 08:39:22,282][53885] Updated weights for policy 1, policy_version 20860 (0.0008) [2023-10-08 08:39:25,486][53852] Updated weights for policy 0, policy_version 20970 (0.0008) [2023-10-08 08:39:25,851][53852] Updated weights for policy 0, policy_version 20980 (0.0009) [2023-10-08 08:39:26,098][53885] Updated weights for policy 1, policy_version 20870 (0.0007) [2023-10-08 08:39:26,216][53852] Updated weights for policy 0, policy_version 20990 (0.0008) [2023-10-08 08:39:26,496][53885] Updated weights for policy 1, policy_version 20880 (0.0007) [2023-10-08 08:39:26,860][53885] Updated weights for policy 1, policy_version 20890 (0.0007) [2023-10-08 08:39:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42860544. Throughput: 0: 1828.9, 1: 1798.0. Samples: 10721780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:39:27,016][52710] Avg episode reward: [(0, '22.090'), (1, '25.530')] [2023-10-08 08:39:30,015][53852] Updated weights for policy 0, policy_version 21000 (0.0008) [2023-10-08 08:39:30,386][53852] Updated weights for policy 0, policy_version 21010 (0.0009) [2023-10-08 08:39:30,503][53885] Updated weights for policy 1, policy_version 20900 (0.0008) [2023-10-08 08:39:30,750][53852] Updated weights for policy 0, policy_version 21020 (0.0007) [2023-10-08 08:39:30,866][53885] Updated weights for policy 1, policy_version 20910 (0.0008) [2023-10-08 08:39:31,240][53885] Updated weights for policy 1, policy_version 20920 (0.0007) [2023-10-08 08:39:32,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 42958848. Throughput: 0: 1822.6, 1: 1812.7. Samples: 10742952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:39:32,016][52710] Avg episode reward: [(0, '22.760'), (1, '25.420')] [2023-10-08 08:39:34,374][53852] Updated weights for policy 0, policy_version 21030 (0.0009) [2023-10-08 08:39:34,738][53852] Updated weights for policy 0, policy_version 21040 (0.0011) [2023-10-08 08:39:35,106][53852] Updated weights for policy 0, policy_version 21050 (0.0009) [2023-10-08 08:39:35,111][53885] Updated weights for policy 1, policy_version 20930 (0.0007) [2023-10-08 08:39:35,473][53885] Updated weights for policy 1, policy_version 20940 (0.0008) [2023-10-08 08:39:35,843][53885] Updated weights for policy 1, policy_version 20950 (0.0011) [2023-10-08 08:39:36,218][53885] Updated weights for policy 1, policy_version 20960 (0.0011) [2023-10-08 08:39:37,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43024384. Throughput: 0: 1829.1, 1: 1795.2. Samples: 10764086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:39:37,016][52710] Avg episode reward: [(0, '23.020'), (1, '24.180')] [2023-10-08 08:39:38,689][53852] Updated weights for policy 0, policy_version 21060 (0.0008) [2023-10-08 08:39:39,049][53852] Updated weights for policy 0, policy_version 21070 (0.0009) [2023-10-08 08:39:39,424][53852] Updated weights for policy 0, policy_version 21080 (0.0008) [2023-10-08 08:39:39,829][53885] Updated weights for policy 1, policy_version 20970 (0.0008) [2023-10-08 08:39:40,192][53885] Updated weights for policy 1, policy_version 20980 (0.0007) [2023-10-08 08:39:40,569][53885] Updated weights for policy 1, policy_version 20990 (0.0010) [2023-10-08 08:39:42,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 43089920. Throughput: 0: 1824.7, 1: 1814.3. Samples: 10775810. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 08:39:42,015][52710] Avg episode reward: [(0, '23.670'), (1, '25.470')] [2023-10-08 08:39:43,152][53852] Updated weights for policy 0, policy_version 21090 (0.0008) [2023-10-08 08:39:43,515][53852] Updated weights for policy 0, policy_version 21100 (0.0010) [2023-10-08 08:39:43,887][53852] Updated weights for policy 0, policy_version 21110 (0.0010) [2023-10-08 08:39:44,173][53885] Updated weights for policy 1, policy_version 21000 (0.0009) [2023-10-08 08:39:44,251][53852] Updated weights for policy 0, policy_version 21120 (0.0007) [2023-10-08 08:39:44,530][53885] Updated weights for policy 1, policy_version 21010 (0.0010) [2023-10-08 08:39:44,896][53885] Updated weights for policy 1, policy_version 21020 (0.0010) [2023-10-08 08:39:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43155456. Throughput: 0: 1826.8, 1: 1800.8. Samples: 10797016. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 08:39:47,016][52710] Avg episode reward: [(0, '22.750'), (1, '23.100')] [2023-10-08 08:39:47,910][53852] Updated weights for policy 0, policy_version 21130 (0.0008) [2023-10-08 08:39:48,271][53852] Updated weights for policy 0, policy_version 21140 (0.0008) [2023-10-08 08:39:48,542][53885] Updated weights for policy 1, policy_version 21030 (0.0009) [2023-10-08 08:39:48,647][53852] Updated weights for policy 0, policy_version 21150 (0.0007) [2023-10-08 08:39:48,915][53885] Updated weights for policy 1, policy_version 21040 (0.0009) [2023-10-08 08:39:49,275][53885] Updated weights for policy 1, policy_version 21050 (0.0010) [2023-10-08 08:39:52,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 43220992. Throughput: 0: 1834.8, 1: 1805.9. Samples: 10820540. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 08:39:52,016][52710] Avg episode reward: [(0, '22.210'), (1, '26.550')] [2023-10-08 08:39:52,128][53852] Updated weights for policy 0, policy_version 21160 (0.0010) [2023-10-08 08:39:52,496][53852] Updated weights for policy 0, policy_version 21170 (0.0008) [2023-10-08 08:39:52,877][53852] Updated weights for policy 0, policy_version 21180 (0.0008) [2023-10-08 08:39:52,992][53885] Updated weights for policy 1, policy_version 21060 (0.0008) [2023-10-08 08:39:53,355][53885] Updated weights for policy 1, policy_version 21070 (0.0009) [2023-10-08 08:39:53,719][53885] Updated weights for policy 1, policy_version 21080 (0.0010) [2023-10-08 08:39:56,693][53852] Updated weights for policy 0, policy_version 21190 (0.0009) [2023-10-08 08:39:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43286528. Throughput: 0: 1837.2, 1: 1806.7. Samples: 10830486. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 08:39:57,016][52710] Avg episode reward: [(0, '23.480'), (1, '29.200')] [2023-10-08 08:39:57,070][53852] Updated weights for policy 0, policy_version 21200 (0.0007) [2023-10-08 08:39:57,390][53885] Updated weights for policy 1, policy_version 21090 (0.0009) [2023-10-08 08:39:57,451][53852] Updated weights for policy 0, policy_version 21210 (0.0007) [2023-10-08 08:39:57,753][53885] Updated weights for policy 1, policy_version 21100 (0.0008) [2023-10-08 08:39:58,125][53885] Updated weights for policy 1, policy_version 21110 (0.0008) [2023-10-08 08:39:58,496][53885] Updated weights for policy 1, policy_version 21120 (0.0008) [2023-10-08 08:40:01,052][53852] Updated weights for policy 0, policy_version 21220 (0.0008) [2023-10-08 08:40:01,431][53852] Updated weights for policy 0, policy_version 21230 (0.0009) [2023-10-08 08:40:01,798][53852] Updated weights for policy 0, policy_version 21240 (0.0008) [2023-10-08 08:40:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43352064. Throughput: 0: 1831.0, 1: 1811.6. Samples: 10853166. Policy #0 lag: (min: 11.0, avg: 12.3, max: 36.0) [2023-10-08 08:40:02,016][52710] Avg episode reward: [(0, '22.950'), (1, '25.610')] [2023-10-08 08:40:02,227][53885] Updated weights for policy 1, policy_version 21130 (0.0007) [2023-10-08 08:40:02,611][53885] Updated weights for policy 1, policy_version 21140 (0.0008) [2023-10-08 08:40:02,984][53885] Updated weights for policy 1, policy_version 21150 (0.0010) [2023-10-08 08:40:05,475][53852] Updated weights for policy 0, policy_version 21250 (0.0008) [2023-10-08 08:40:05,839][53852] Updated weights for policy 0, policy_version 21260 (0.0008) [2023-10-08 08:40:06,212][53852] Updated weights for policy 0, policy_version 21270 (0.0007) [2023-10-08 08:40:06,577][53852] Updated weights for policy 0, policy_version 21280 (0.0007) [2023-10-08 08:40:06,674][53885] Updated weights for policy 1, policy_version 21160 (0.0007) [2023-10-08 08:40:07,015][52710] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 43450368. Throughput: 0: 1826.0, 1: 1812.5. Samples: 10874250. Policy #0 lag: (min: 11.0, avg: 12.3, max: 36.0) [2023-10-08 08:40:07,016][52710] Avg episode reward: [(0, '23.370'), (1, '27.170')] [2023-10-08 08:40:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000021280_21790720.pth... [2023-10-08 08:40:07,040][53885] Updated weights for policy 1, policy_version 21170 (0.0010) [2023-10-08 08:40:07,059][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000019552_20021248.pth [2023-10-08 08:40:07,411][53885] Updated weights for policy 1, policy_version 21180 (0.0008) [2023-10-08 08:40:07,555][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000021184_21692416.pth... [2023-10-08 08:40:07,594][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000019488_19955712.pth [2023-10-08 08:40:10,273][53852] Updated weights for policy 0, policy_version 21290 (0.0010) [2023-10-08 08:40:10,646][53852] Updated weights for policy 0, policy_version 21300 (0.0009) [2023-10-08 08:40:11,019][53852] Updated weights for policy 0, policy_version 21310 (0.0008) [2023-10-08 08:40:11,047][53885] Updated weights for policy 1, policy_version 21190 (0.0009) [2023-10-08 08:40:11,429][53885] Updated weights for policy 1, policy_version 21200 (0.0009) [2023-10-08 08:40:11,810][53885] Updated weights for policy 1, policy_version 21210 (0.0011) [2023-10-08 08:40:12,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 43515904. Throughput: 0: 1836.5, 1: 1809.9. Samples: 10885866. Policy #0 lag: (min: 11.0, avg: 12.3, max: 36.0) [2023-10-08 08:40:12,016][52710] Avg episode reward: [(0, '22.910'), (1, '29.160')] [2023-10-08 08:40:14,519][53852] Updated weights for policy 0, policy_version 21320 (0.0008) [2023-10-08 08:40:14,897][53852] Updated weights for policy 0, policy_version 21330 (0.0010) [2023-10-08 08:40:15,262][53852] Updated weights for policy 0, policy_version 21340 (0.0008) [2023-10-08 08:40:15,451][53885] Updated weights for policy 1, policy_version 21220 (0.0011) [2023-10-08 08:40:15,818][53885] Updated weights for policy 1, policy_version 21230 (0.0009) [2023-10-08 08:40:16,185][53885] Updated weights for policy 1, policy_version 21240 (0.0011) [2023-10-08 08:40:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 43614208. Throughput: 0: 1831.1, 1: 1816.8. Samples: 10907104. Policy #0 lag: (min: 27.0, avg: 27.2, max: 35.0) [2023-10-08 08:40:17,016][52710] Avg episode reward: [(0, '22.560'), (1, '25.530')] [2023-10-08 08:40:18,945][53852] Updated weights for policy 0, policy_version 21350 (0.0007) [2023-10-08 08:40:19,314][53852] Updated weights for policy 0, policy_version 21360 (0.0007) [2023-10-08 08:40:19,677][53852] Updated weights for policy 0, policy_version 21370 (0.0008) [2023-10-08 08:40:19,926][53885] Updated weights for policy 1, policy_version 21250 (0.0010) [2023-10-08 08:40:20,293][53885] Updated weights for policy 1, policy_version 21260 (0.0009) [2023-10-08 08:40:20,659][53885] Updated weights for policy 1, policy_version 21270 (0.0009) [2023-10-08 08:40:21,025][53885] Updated weights for policy 1, policy_version 21280 (0.0010) [2023-10-08 08:40:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43679744. Throughput: 0: 1844.3, 1: 1827.6. Samples: 10929322. Policy #0 lag: (min: 27.0, avg: 27.2, max: 35.0) [2023-10-08 08:40:22,016][52710] Avg episode reward: [(0, '20.850'), (1, '27.280')] [2023-10-08 08:40:23,179][53852] Updated weights for policy 0, policy_version 21380 (0.0007) [2023-10-08 08:40:23,545][53852] Updated weights for policy 0, policy_version 21390 (0.0009) [2023-10-08 08:40:23,914][53852] Updated weights for policy 0, policy_version 21400 (0.0008) [2023-10-08 08:40:24,733][53885] Updated weights for policy 1, policy_version 21290 (0.0008) [2023-10-08 08:40:25,111][53885] Updated weights for policy 1, policy_version 21300 (0.0008) [2023-10-08 08:40:25,476][53885] Updated weights for policy 1, policy_version 21310 (0.0008) [2023-10-08 08:40:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 43745280. Throughput: 0: 1833.8, 1: 1823.3. Samples: 10940380. Policy #0 lag: (min: 27.0, avg: 27.2, max: 35.0) [2023-10-08 08:40:27,016][52710] Avg episode reward: [(0, '21.710'), (1, '28.290')] [2023-10-08 08:40:27,521][53852] Updated weights for policy 0, policy_version 21410 (0.0008) [2023-10-08 08:40:27,900][53852] Updated weights for policy 0, policy_version 21420 (0.0008) [2023-10-08 08:40:28,262][53852] Updated weights for policy 0, policy_version 21430 (0.0009) [2023-10-08 08:40:28,639][53852] Updated weights for policy 0, policy_version 21440 (0.0008) [2023-10-08 08:40:29,077][53885] Updated weights for policy 1, policy_version 21320 (0.0009) [2023-10-08 08:40:29,447][53885] Updated weights for policy 1, policy_version 21330 (0.0010) [2023-10-08 08:40:29,823][53885] Updated weights for policy 1, policy_version 21340 (0.0008) [2023-10-08 08:40:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43810816. Throughput: 0: 1848.1, 1: 1826.2. Samples: 10962360. Policy #0 lag: (min: 27.0, avg: 27.2, max: 35.0) [2023-10-08 08:40:32,016][52710] Avg episode reward: [(0, '21.320'), (1, '24.610')] [2023-10-08 08:40:32,396][53852] Updated weights for policy 0, policy_version 21450 (0.0009) [2023-10-08 08:40:32,779][53852] Updated weights for policy 0, policy_version 21460 (0.0009) [2023-10-08 08:40:33,137][53852] Updated weights for policy 0, policy_version 21470 (0.0007) [2023-10-08 08:40:33,476][53885] Updated weights for policy 1, policy_version 21350 (0.0009) [2023-10-08 08:40:33,845][53885] Updated weights for policy 1, policy_version 21360 (0.0008) [2023-10-08 08:40:34,219][53885] Updated weights for policy 1, policy_version 21370 (0.0010) [2023-10-08 08:40:36,911][53852] Updated weights for policy 0, policy_version 21480 (0.0008) [2023-10-08 08:40:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 43876352. Throughput: 0: 1836.9, 1: 1821.1. Samples: 10985150. Policy #0 lag: (min: 17.0, avg: 34.0, max: 49.0) [2023-10-08 08:40:37,015][52710] Avg episode reward: [(0, '22.600'), (1, '25.850')] [2023-10-08 08:40:37,271][53852] Updated weights for policy 0, policy_version 21490 (0.0007) [2023-10-08 08:40:37,644][53852] Updated weights for policy 0, policy_version 21500 (0.0009) [2023-10-08 08:40:38,088][53885] Updated weights for policy 1, policy_version 21380 (0.0008) [2023-10-08 08:40:38,451][53885] Updated weights for policy 1, policy_version 21390 (0.0008) [2023-10-08 08:40:38,824][53885] Updated weights for policy 1, policy_version 21400 (0.0008) [2023-10-08 08:40:41,248][53852] Updated weights for policy 0, policy_version 21510 (0.0007) [2023-10-08 08:40:41,614][53852] Updated weights for policy 0, policy_version 21520 (0.0007) [2023-10-08 08:40:41,985][53852] Updated weights for policy 0, policy_version 21530 (0.0007) [2023-10-08 08:40:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 43941888. Throughput: 0: 1836.6, 1: 1824.3. Samples: 10995228. Policy #0 lag: (min: 17.0, avg: 34.0, max: 49.0) [2023-10-08 08:40:42,016][52710] Avg episode reward: [(0, '22.570'), (1, '28.640')] [2023-10-08 08:40:42,542][53885] Updated weights for policy 1, policy_version 21410 (0.0008) [2023-10-08 08:40:42,916][53885] Updated weights for policy 1, policy_version 21420 (0.0008) [2023-10-08 08:40:43,284][53885] Updated weights for policy 1, policy_version 21430 (0.0009) [2023-10-08 08:40:43,650][53885] Updated weights for policy 1, policy_version 21440 (0.0008) [2023-10-08 08:40:45,731][53852] Updated weights for policy 0, policy_version 21540 (0.0009) [2023-10-08 08:40:46,114][53852] Updated weights for policy 0, policy_version 21550 (0.0007) [2023-10-08 08:40:46,483][53852] Updated weights for policy 0, policy_version 21560 (0.0007) [2023-10-08 08:40:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44040192. Throughput: 0: 1839.9, 1: 1820.1. Samples: 11017868. Policy #0 lag: (min: 17.0, avg: 34.0, max: 49.0) [2023-10-08 08:40:47,015][52710] Avg episode reward: [(0, '22.360'), (1, '26.830')] [2023-10-08 08:40:47,271][53885] Updated weights for policy 1, policy_version 21450 (0.0009) [2023-10-08 08:40:47,639][53885] Updated weights for policy 1, policy_version 21460 (0.0010) [2023-10-08 08:40:48,018][53885] Updated weights for policy 1, policy_version 21470 (0.0008) [2023-10-08 08:40:50,042][53852] Updated weights for policy 0, policy_version 21570 (0.0007) [2023-10-08 08:40:50,413][53852] Updated weights for policy 0, policy_version 21580 (0.0009) [2023-10-08 08:40:50,791][53852] Updated weights for policy 0, policy_version 21590 (0.0008) [2023-10-08 08:40:51,156][53852] Updated weights for policy 0, policy_version 21600 (0.0011) [2023-10-08 08:40:51,637][53885] Updated weights for policy 1, policy_version 21480 (0.0010) [2023-10-08 08:40:52,003][53885] Updated weights for policy 1, policy_version 21490 (0.0009) [2023-10-08 08:40:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44105728. Throughput: 0: 1835.0, 1: 1823.2. Samples: 11038868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:40:52,017][52710] Avg episode reward: [(0, '23.470'), (1, '28.450')] [2023-10-08 08:40:52,378][53885] Updated weights for policy 1, policy_version 21500 (0.0008) [2023-10-08 08:40:54,889][53852] Updated weights for policy 0, policy_version 21610 (0.0008) [2023-10-08 08:40:55,263][53852] Updated weights for policy 0, policy_version 21620 (0.0009) [2023-10-08 08:40:55,635][53852] Updated weights for policy 0, policy_version 21630 (0.0009) [2023-10-08 08:40:56,093][53885] Updated weights for policy 1, policy_version 21510 (0.0007) [2023-10-08 08:40:56,467][53885] Updated weights for policy 1, policy_version 21520 (0.0007) [2023-10-08 08:40:56,833][53885] Updated weights for policy 1, policy_version 21530 (0.0007) [2023-10-08 08:40:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44171264. Throughput: 0: 1837.6, 1: 1824.3. Samples: 11050650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:40:57,016][52710] Avg episode reward: [(0, '23.620'), (1, '27.860')] [2023-10-08 08:40:59,188][53852] Updated weights for policy 0, policy_version 21640 (0.0009) [2023-10-08 08:40:59,560][53852] Updated weights for policy 0, policy_version 21650 (0.0008) [2023-10-08 08:40:59,923][53852] Updated weights for policy 0, policy_version 21660 (0.0008) [2023-10-08 08:41:00,415][53885] Updated weights for policy 1, policy_version 21540 (0.0008) [2023-10-08 08:41:00,787][53885] Updated weights for policy 1, policy_version 21550 (0.0008) [2023-10-08 08:41:01,229][53885] Updated weights for policy 1, policy_version 21562 (0.0009) [2023-10-08 08:41:02,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 44269568. Throughput: 0: 1838.0, 1: 1824.8. Samples: 11071926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:41:02,016][52710] Avg episode reward: [(0, '24.270'), (1, '27.130')] [2023-10-08 08:41:03,552][53852] Updated weights for policy 0, policy_version 21670 (0.0008) [2023-10-08 08:41:03,927][53852] Updated weights for policy 0, policy_version 21680 (0.0010) [2023-10-08 08:41:04,299][53852] Updated weights for policy 0, policy_version 21690 (0.0008) [2023-10-08 08:41:04,927][53885] Updated weights for policy 1, policy_version 21572 (0.0008) [2023-10-08 08:41:05,291][53885] Updated weights for policy 1, policy_version 21582 (0.0007) [2023-10-08 08:41:05,667][53885] Updated weights for policy 1, policy_version 21592 (0.0008) [2023-10-08 08:41:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44335104. Throughput: 0: 1834.4, 1: 1820.4. Samples: 11093792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:41:07,016][52710] Avg episode reward: [(0, '24.250'), (1, '26.930')] [2023-10-08 08:41:07,911][53852] Updated weights for policy 0, policy_version 21700 (0.0008) [2023-10-08 08:41:08,280][53852] Updated weights for policy 0, policy_version 21710 (0.0007) [2023-10-08 08:41:08,641][53852] Updated weights for policy 0, policy_version 21720 (0.0007) [2023-10-08 08:41:09,404][53885] Updated weights for policy 1, policy_version 21602 (0.0010) [2023-10-08 08:41:09,779][53885] Updated weights for policy 1, policy_version 21612 (0.0008) [2023-10-08 08:41:10,150][53885] Updated weights for policy 1, policy_version 21622 (0.0008) [2023-10-08 08:41:10,511][53885] Updated weights for policy 1, policy_version 21632 (0.0009) [2023-10-08 08:41:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 44400640. Throughput: 0: 1838.5, 1: 1819.9. Samples: 11105008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:41:12,016][52710] Avg episode reward: [(0, '24.830'), (1, '26.520')] [2023-10-08 08:41:12,272][53852] Updated weights for policy 0, policy_version 21730 (0.0011) [2023-10-08 08:41:12,650][53852] Updated weights for policy 0, policy_version 21740 (0.0007) [2023-10-08 08:41:13,022][53852] Updated weights for policy 0, policy_version 21750 (0.0010) [2023-10-08 08:41:13,385][53852] Updated weights for policy 0, policy_version 21760 (0.0008) [2023-10-08 08:41:13,992][53885] Updated weights for policy 1, policy_version 21642 (0.0008) [2023-10-08 08:41:14,360][53885] Updated weights for policy 1, policy_version 21652 (0.0008) [2023-10-08 08:41:14,721][53885] Updated weights for policy 1, policy_version 21662 (0.0008) [2023-10-08 08:41:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 44466176. Throughput: 0: 1833.2, 1: 1826.1. Samples: 11127026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:41:17,015][52710] Avg episode reward: [(0, '23.850'), (1, '25.920')] [2023-10-08 08:41:17,140][53852] Updated weights for policy 0, policy_version 21770 (0.0009) [2023-10-08 08:41:17,512][53852] Updated weights for policy 0, policy_version 21780 (0.0008) [2023-10-08 08:41:17,882][53852] Updated weights for policy 0, policy_version 21790 (0.0007) [2023-10-08 08:41:18,478][53885] Updated weights for policy 1, policy_version 21672 (0.0007) [2023-10-08 08:41:18,850][53885] Updated weights for policy 1, policy_version 21682 (0.0009) [2023-10-08 08:41:19,224][53885] Updated weights for policy 1, policy_version 21692 (0.0009) [2023-10-08 08:41:21,347][53852] Updated weights for policy 0, policy_version 21800 (0.0009) [2023-10-08 08:41:21,713][53852] Updated weights for policy 0, policy_version 21810 (0.0011) [2023-10-08 08:41:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 44531712. Throughput: 0: 1828.8, 1: 1833.3. Samples: 11149944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:41:22,015][52710] Avg episode reward: [(0, '24.880'), (1, '29.130')] [2023-10-08 08:41:22,090][53852] Updated weights for policy 0, policy_version 21820 (0.0011) [2023-10-08 08:41:22,950][53885] Updated weights for policy 1, policy_version 21702 (0.0007) [2023-10-08 08:41:23,323][53885] Updated weights for policy 1, policy_version 21712 (0.0008) [2023-10-08 08:41:23,689][53885] Updated weights for policy 1, policy_version 21722 (0.0010) [2023-10-08 08:41:25,629][53852] Updated weights for policy 0, policy_version 21830 (0.0008) [2023-10-08 08:41:26,005][53852] Updated weights for policy 0, policy_version 21840 (0.0007) [2023-10-08 08:41:26,369][53852] Updated weights for policy 0, policy_version 21850 (0.0007) [2023-10-08 08:41:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44630016. Throughput: 0: 1845.6, 1: 1834.5. Samples: 11160832. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:41:27,016][52710] Avg episode reward: [(0, '25.140'), (1, '27.540')] [2023-10-08 08:41:27,017][53500] Saving new best policy, reward=25.140! [2023-10-08 08:41:27,290][53885] Updated weights for policy 1, policy_version 21732 (0.0010) [2023-10-08 08:41:27,661][53885] Updated weights for policy 1, policy_version 21742 (0.0007) [2023-10-08 08:41:28,032][53885] Updated weights for policy 1, policy_version 21752 (0.0007) [2023-10-08 08:41:29,923][53852] Updated weights for policy 0, policy_version 21860 (0.0007) [2023-10-08 08:41:30,285][53852] Updated weights for policy 0, policy_version 21870 (0.0008) [2023-10-08 08:41:30,653][53852] Updated weights for policy 0, policy_version 21880 (0.0008) [2023-10-08 08:41:31,735][53885] Updated weights for policy 1, policy_version 21762 (0.0008) [2023-10-08 08:41:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44695552. Throughput: 0: 1829.3, 1: 1838.4. Samples: 11182912. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:41:32,016][52710] Avg episode reward: [(0, '24.360'), (1, '26.710')] [2023-10-08 08:41:32,094][53885] Updated weights for policy 1, policy_version 21772 (0.0010) [2023-10-08 08:41:32,463][53885] Updated weights for policy 1, policy_version 21782 (0.0007) [2023-10-08 08:41:32,842][53885] Updated weights for policy 1, policy_version 21792 (0.0009) [2023-10-08 08:41:34,326][53852] Updated weights for policy 0, policy_version 21890 (0.0010) [2023-10-08 08:41:34,725][53852] Updated weights for policy 0, policy_version 21900 (0.0008) [2023-10-08 08:41:35,082][53852] Updated weights for policy 0, policy_version 21910 (0.0008) [2023-10-08 08:41:35,456][53852] Updated weights for policy 0, policy_version 21920 (0.0009) [2023-10-08 08:41:36,454][53885] Updated weights for policy 1, policy_version 21802 (0.0007) [2023-10-08 08:41:36,813][53885] Updated weights for policy 1, policy_version 21812 (0.0008) [2023-10-08 08:41:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 44761088. Throughput: 0: 1855.3, 1: 1831.4. Samples: 11204770. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:41:37,016][52710] Avg episode reward: [(0, '25.410'), (1, '29.300')] [2023-10-08 08:41:37,027][53500] Saving new best policy, reward=25.410! [2023-10-08 08:41:37,187][53885] Updated weights for policy 1, policy_version 21822 (0.0007) [2023-10-08 08:41:39,119][53852] Updated weights for policy 0, policy_version 21930 (0.0010) [2023-10-08 08:41:39,483][53852] Updated weights for policy 0, policy_version 21940 (0.0007) [2023-10-08 08:41:39,856][53852] Updated weights for policy 0, policy_version 21950 (0.0008) [2023-10-08 08:41:40,873][53885] Updated weights for policy 1, policy_version 21832 (0.0008) [2023-10-08 08:41:41,248][53885] Updated weights for policy 1, policy_version 21842 (0.0007) [2023-10-08 08:41:41,616][53885] Updated weights for policy 1, policy_version 21852 (0.0007) [2023-10-08 08:41:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 44859392. Throughput: 0: 1830.5, 1: 1837.7. Samples: 11215720. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:41:42,016][52710] Avg episode reward: [(0, '23.980'), (1, '29.440')] [2023-10-08 08:41:43,463][53852] Updated weights for policy 0, policy_version 21960 (0.0007) [2023-10-08 08:41:43,830][53852] Updated weights for policy 0, policy_version 21970 (0.0009) [2023-10-08 08:41:44,212][53852] Updated weights for policy 0, policy_version 21980 (0.0008) [2023-10-08 08:41:45,279][53885] Updated weights for policy 1, policy_version 21862 (0.0009) [2023-10-08 08:41:45,648][53885] Updated weights for policy 1, policy_version 21872 (0.0011) [2023-10-08 08:41:46,010][53885] Updated weights for policy 1, policy_version 21882 (0.0007) [2023-10-08 08:41:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 44924928. Throughput: 0: 1850.0, 1: 1835.0. Samples: 11237750. Policy #0 lag: (min: 18.0, avg: 20.5, max: 44.0) [2023-10-08 08:41:47,016][52710] Avg episode reward: [(0, '24.050'), (1, '26.560')] [2023-10-08 08:41:47,782][53852] Updated weights for policy 0, policy_version 21990 (0.0008) [2023-10-08 08:41:48,158][53852] Updated weights for policy 0, policy_version 22000 (0.0007) [2023-10-08 08:41:48,530][53852] Updated weights for policy 0, policy_version 22010 (0.0007) [2023-10-08 08:41:49,745][53885] Updated weights for policy 1, policy_version 21892 (0.0009) [2023-10-08 08:41:50,108][53885] Updated weights for policy 1, policy_version 21902 (0.0008) [2023-10-08 08:41:50,472][53885] Updated weights for policy 1, policy_version 21912 (0.0009) [2023-10-08 08:41:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 44990464. Throughput: 0: 1850.7, 1: 1839.6. Samples: 11259854. Policy #0 lag: (min: 18.0, avg: 20.5, max: 44.0) [2023-10-08 08:41:52,016][52710] Avg episode reward: [(0, '21.700'), (1, '26.510')] [2023-10-08 08:41:52,234][53852] Updated weights for policy 0, policy_version 22020 (0.0009) [2023-10-08 08:41:52,592][53852] Updated weights for policy 0, policy_version 22030 (0.0009) [2023-10-08 08:41:52,962][53852] Updated weights for policy 0, policy_version 22040 (0.0007) [2023-10-08 08:41:54,180][53885] Updated weights for policy 1, policy_version 21922 (0.0008) [2023-10-08 08:41:54,551][53885] Updated weights for policy 1, policy_version 21932 (0.0007) [2023-10-08 08:41:54,918][53885] Updated weights for policy 1, policy_version 21942 (0.0008) [2023-10-08 08:41:55,285][53885] Updated weights for policy 1, policy_version 21952 (0.0011) [2023-10-08 08:41:56,592][53852] Updated weights for policy 0, policy_version 22050 (0.0008) [2023-10-08 08:41:56,953][53852] Updated weights for policy 0, policy_version 22060 (0.0009) [2023-10-08 08:41:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45056000. Throughput: 0: 1851.3, 1: 1830.1. Samples: 11270672. Policy #0 lag: (min: 18.0, avg: 20.5, max: 44.0) [2023-10-08 08:41:57,016][52710] Avg episode reward: [(0, '21.680'), (1, '24.700')] [2023-10-08 08:41:57,320][53852] Updated weights for policy 0, policy_version 22070 (0.0007) [2023-10-08 08:41:57,689][53852] Updated weights for policy 0, policy_version 22080 (0.0007) [2023-10-08 08:41:58,945][53885] Updated weights for policy 1, policy_version 21962 (0.0009) [2023-10-08 08:41:59,318][53885] Updated weights for policy 1, policy_version 21972 (0.0008) [2023-10-08 08:41:59,687][53885] Updated weights for policy 1, policy_version 21982 (0.0007) [2023-10-08 08:42:01,322][53852] Updated weights for policy 0, policy_version 22090 (0.0009) [2023-10-08 08:42:01,691][53852] Updated weights for policy 0, policy_version 22100 (0.0009) [2023-10-08 08:42:02,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 45121536. Throughput: 0: 1852.7, 1: 1832.7. Samples: 11292868. Policy #0 lag: (min: 18.0, avg: 20.5, max: 44.0) [2023-10-08 08:42:02,016][52710] Avg episode reward: [(0, '21.970'), (1, '23.820')] [2023-10-08 08:42:02,055][53852] Updated weights for policy 0, policy_version 22110 (0.0007) [2023-10-08 08:42:03,331][53885] Updated weights for policy 1, policy_version 21992 (0.0010) [2023-10-08 08:42:03,697][53885] Updated weights for policy 1, policy_version 22002 (0.0008) [2023-10-08 08:42:04,068][53885] Updated weights for policy 1, policy_version 22012 (0.0010) [2023-10-08 08:42:05,787][53852] Updated weights for policy 0, policy_version 22120 (0.0010) [2023-10-08 08:42:06,166][53852] Updated weights for policy 0, policy_version 22130 (0.0008) [2023-10-08 08:42:06,541][53852] Updated weights for policy 0, policy_version 22140 (0.0008) [2023-10-08 08:42:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45219840. Throughput: 0: 1831.2, 1: 1826.6. Samples: 11314546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:42:07,016][52710] Avg episode reward: [(0, '21.200'), (1, '26.980')] [2023-10-08 08:42:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000022144_22675456.pth... [2023-10-08 08:42:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000022016_22544384.pth... [2023-10-08 08:42:07,059][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000020416_20905984.pth [2023-10-08 08:42:07,067][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000020320_20807680.pth [2023-10-08 08:42:07,639][53885] Updated weights for policy 1, policy_version 22022 (0.0009) [2023-10-08 08:42:08,012][53885] Updated weights for policy 1, policy_version 22032 (0.0007) [2023-10-08 08:42:08,373][53885] Updated weights for policy 1, policy_version 22042 (0.0007) [2023-10-08 08:42:10,227][53852] Updated weights for policy 0, policy_version 22150 (0.0007) [2023-10-08 08:42:10,594][53852] Updated weights for policy 0, policy_version 22160 (0.0007) [2023-10-08 08:42:10,971][53852] Updated weights for policy 0, policy_version 22170 (0.0008) [2023-10-08 08:42:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45285376. Throughput: 0: 1842.6, 1: 1824.6. Samples: 11325856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:42:12,016][52710] Avg episode reward: [(0, '22.930'), (1, '27.830')] [2023-10-08 08:42:12,176][53885] Updated weights for policy 1, policy_version 22052 (0.0007) [2023-10-08 08:42:12,539][53885] Updated weights for policy 1, policy_version 22062 (0.0009) [2023-10-08 08:42:12,906][53885] Updated weights for policy 1, policy_version 22072 (0.0009) [2023-10-08 08:42:14,575][53852] Updated weights for policy 0, policy_version 22180 (0.0007) [2023-10-08 08:42:14,936][53852] Updated weights for policy 0, policy_version 22190 (0.0008) [2023-10-08 08:42:15,311][53852] Updated weights for policy 0, policy_version 22200 (0.0008) [2023-10-08 08:42:16,559][53885] Updated weights for policy 1, policy_version 22082 (0.0007) [2023-10-08 08:42:16,916][53885] Updated weights for policy 1, policy_version 22092 (0.0008) [2023-10-08 08:42:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45350912. Throughput: 0: 1833.7, 1: 1826.0. Samples: 11347600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:42:17,016][52710] Avg episode reward: [(0, '24.490'), (1, '27.440')] [2023-10-08 08:42:17,279][53885] Updated weights for policy 1, policy_version 22102 (0.0007) [2023-10-08 08:42:17,650][53885] Updated weights for policy 1, policy_version 22112 (0.0008) [2023-10-08 08:42:18,768][53852] Updated weights for policy 0, policy_version 22210 (0.0009) [2023-10-08 08:42:19,128][53852] Updated weights for policy 0, policy_version 22220 (0.0010) [2023-10-08 08:42:19,496][53852] Updated weights for policy 0, policy_version 22230 (0.0010) [2023-10-08 08:42:19,871][53852] Updated weights for policy 0, policy_version 22240 (0.0008) [2023-10-08 08:42:21,402][53885] Updated weights for policy 1, policy_version 22122 (0.0010) [2023-10-08 08:42:21,773][53885] Updated weights for policy 1, policy_version 22132 (0.0009) [2023-10-08 08:42:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45416448. Throughput: 0: 1845.6, 1: 1820.3. Samples: 11369734. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-10-08 08:42:22,015][52710] Avg episode reward: [(0, '22.970'), (1, '29.690')] [2023-10-08 08:42:22,136][53885] Updated weights for policy 1, policy_version 22142 (0.0007) [2023-10-08 08:42:23,661][53852] Updated weights for policy 0, policy_version 22250 (0.0008) [2023-10-08 08:42:24,033][53852] Updated weights for policy 0, policy_version 22260 (0.0008) [2023-10-08 08:42:24,396][53852] Updated weights for policy 0, policy_version 22270 (0.0008) [2023-10-08 08:42:25,898][53885] Updated weights for policy 1, policy_version 22152 (0.0008) [2023-10-08 08:42:26,259][53885] Updated weights for policy 1, policy_version 22162 (0.0011) [2023-10-08 08:42:26,625][53885] Updated weights for policy 1, policy_version 22172 (0.0010) [2023-10-08 08:42:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45514752. Throughput: 0: 1837.1, 1: 1823.0. Samples: 11380422. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-10-08 08:42:27,016][52710] Avg episode reward: [(0, '22.460'), (1, '30.700')] [2023-10-08 08:42:27,997][53852] Updated weights for policy 0, policy_version 22280 (0.0008) [2023-10-08 08:42:28,378][53852] Updated weights for policy 0, policy_version 22290 (0.0008) [2023-10-08 08:42:28,753][53852] Updated weights for policy 0, policy_version 22300 (0.0008) [2023-10-08 08:42:30,380][53885] Updated weights for policy 1, policy_version 22182 (0.0009) [2023-10-08 08:42:30,750][53885] Updated weights for policy 1, policy_version 22192 (0.0011) [2023-10-08 08:42:31,118][53885] Updated weights for policy 1, policy_version 22202 (0.0010) [2023-10-08 08:42:32,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45580288. Throughput: 0: 1854.8, 1: 1814.5. Samples: 11402870. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-10-08 08:42:32,016][52710] Avg episode reward: [(0, '23.770'), (1, '27.060')] [2023-10-08 08:42:32,084][53852] Updated weights for policy 0, policy_version 22310 (0.0009) [2023-10-08 08:42:32,449][53852] Updated weights for policy 0, policy_version 22320 (0.0009) [2023-10-08 08:42:32,820][53852] Updated weights for policy 0, policy_version 22330 (0.0007) [2023-10-08 08:42:34,711][53885] Updated weights for policy 1, policy_version 22212 (0.0009) [2023-10-08 08:42:35,075][53885] Updated weights for policy 1, policy_version 22222 (0.0008) [2023-10-08 08:42:35,443][53885] Updated weights for policy 1, policy_version 22232 (0.0008) [2023-10-08 08:42:36,507][53852] Updated weights for policy 0, policy_version 22340 (0.0007) [2023-10-08 08:42:36,889][53852] Updated weights for policy 0, policy_version 22350 (0.0007) [2023-10-08 08:42:37,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45645824. Throughput: 0: 1847.1, 1: 1816.1. Samples: 11424698. Policy #0 lag: (min: 31.0, avg: 39.5, max: 63.0) [2023-10-08 08:42:37,017][52710] Avg episode reward: [(0, '22.070'), (1, '29.790')] [2023-10-08 08:42:37,264][53852] Updated weights for policy 0, policy_version 22360 (0.0007) [2023-10-08 08:42:39,214][53885] Updated weights for policy 1, policy_version 22242 (0.0011) [2023-10-08 08:42:39,584][53885] Updated weights for policy 1, policy_version 22252 (0.0009) [2023-10-08 08:42:39,953][53885] Updated weights for policy 1, policy_version 22262 (0.0008) [2023-10-08 08:42:40,318][53885] Updated weights for policy 1, policy_version 22272 (0.0008) [2023-10-08 08:42:40,927][53852] Updated weights for policy 0, policy_version 22370 (0.0007) [2023-10-08 08:42:41,299][53852] Updated weights for policy 0, policy_version 22380 (0.0009) [2023-10-08 08:42:41,666][53852] Updated weights for policy 0, policy_version 22390 (0.0007) [2023-10-08 08:42:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 45711360. Throughput: 0: 1853.1, 1: 1817.2. Samples: 11435840. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 08:42:42,017][52710] Avg episode reward: [(0, '24.150'), (1, '28.050')] [2023-10-08 08:42:42,045][53852] Updated weights for policy 0, policy_version 22400 (0.0010) [2023-10-08 08:42:43,870][53885] Updated weights for policy 1, policy_version 22282 (0.0010) [2023-10-08 08:42:44,243][53885] Updated weights for policy 1, policy_version 22292 (0.0010) [2023-10-08 08:42:44,605][53885] Updated weights for policy 1, policy_version 22302 (0.0009) [2023-10-08 08:42:45,682][53852] Updated weights for policy 0, policy_version 22410 (0.0007) [2023-10-08 08:42:46,058][53852] Updated weights for policy 0, policy_version 22420 (0.0008) [2023-10-08 08:42:46,418][53852] Updated weights for policy 0, policy_version 22430 (0.0010) [2023-10-08 08:42:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 45809664. Throughput: 0: 1842.1, 1: 1811.2. Samples: 11457270. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 08:42:47,016][52710] Avg episode reward: [(0, '22.700'), (1, '28.400')] [2023-10-08 08:42:48,304][53885] Updated weights for policy 1, policy_version 22312 (0.0009) [2023-10-08 08:42:48,684][53885] Updated weights for policy 1, policy_version 22322 (0.0009) [2023-10-08 08:42:49,047][53885] Updated weights for policy 1, policy_version 22332 (0.0008) [2023-10-08 08:42:50,116][53852] Updated weights for policy 0, policy_version 22440 (0.0009) [2023-10-08 08:42:50,494][53852] Updated weights for policy 0, policy_version 22450 (0.0009) [2023-10-08 08:42:50,865][53852] Updated weights for policy 0, policy_version 22460 (0.0009) [2023-10-08 08:42:52,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45875200. Throughput: 0: 1844.7, 1: 1813.0. Samples: 11479140. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 08:42:52,016][52710] Avg episode reward: [(0, '24.240'), (1, '28.770')] [2023-10-08 08:42:52,596][53885] Updated weights for policy 1, policy_version 22342 (0.0009) [2023-10-08 08:42:52,972][53885] Updated weights for policy 1, policy_version 22352 (0.0008) [2023-10-08 08:42:53,333][53885] Updated weights for policy 1, policy_version 22362 (0.0009) [2023-10-08 08:42:54,460][53852] Updated weights for policy 0, policy_version 22470 (0.0008) [2023-10-08 08:42:54,836][53852] Updated weights for policy 0, policy_version 22480 (0.0007) [2023-10-08 08:42:55,190][53852] Updated weights for policy 0, policy_version 22490 (0.0007) [2023-10-08 08:42:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45940736. Throughput: 0: 1845.6, 1: 1810.4. Samples: 11490372. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:42:57,016][52710] Avg episode reward: [(0, '23.700'), (1, '31.730')] [2023-10-08 08:42:57,042][53885] Updated weights for policy 1, policy_version 22372 (0.0009) [2023-10-08 08:42:57,398][53885] Updated weights for policy 1, policy_version 22382 (0.0009) [2023-10-08 08:42:57,771][53885] Updated weights for policy 1, policy_version 22392 (0.0007) [2023-10-08 08:42:58,060][53594] Saving new best policy, reward=31.730! [2023-10-08 08:42:58,908][53852] Updated weights for policy 0, policy_version 22500 (0.0009) [2023-10-08 08:42:59,281][53852] Updated weights for policy 0, policy_version 22510 (0.0008) [2023-10-08 08:42:59,656][53852] Updated weights for policy 0, policy_version 22520 (0.0007) [2023-10-08 08:43:01,483][53885] Updated weights for policy 1, policy_version 22402 (0.0009) [2023-10-08 08:43:01,857][53885] Updated weights for policy 1, policy_version 22412 (0.0010) [2023-10-08 08:43:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46006272. Throughput: 0: 1843.6, 1: 1810.6. Samples: 11512038. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:43:02,016][52710] Avg episode reward: [(0, '23.110'), (1, '28.570')] [2023-10-08 08:43:02,224][53885] Updated weights for policy 1, policy_version 22422 (0.0009) [2023-10-08 08:43:02,595][53885] Updated weights for policy 1, policy_version 22432 (0.0009) [2023-10-08 08:43:03,332][53852] Updated weights for policy 0, policy_version 22530 (0.0007) [2023-10-08 08:43:03,708][53852] Updated weights for policy 0, policy_version 22540 (0.0009) [2023-10-08 08:43:04,074][53852] Updated weights for policy 0, policy_version 22550 (0.0008) [2023-10-08 08:43:04,445][53852] Updated weights for policy 0, policy_version 22560 (0.0008) [2023-10-08 08:43:06,286][53885] Updated weights for policy 1, policy_version 22442 (0.0008) [2023-10-08 08:43:06,654][53885] Updated weights for policy 1, policy_version 22452 (0.0007) [2023-10-08 08:43:07,015][53885] Updated weights for policy 1, policy_version 22462 (0.0008) [2023-10-08 08:43:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 46071808. Throughput: 0: 1845.5, 1: 1807.6. Samples: 11534120. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:43:07,016][52710] Avg episode reward: [(0, '22.610'), (1, '25.030')] [2023-10-08 08:43:08,115][53852] Updated weights for policy 0, policy_version 22570 (0.0009) [2023-10-08 08:43:08,478][53852] Updated weights for policy 0, policy_version 22580 (0.0009) [2023-10-08 08:43:08,852][53852] Updated weights for policy 0, policy_version 22590 (0.0009) [2023-10-08 08:43:10,845][53885] Updated weights for policy 1, policy_version 22472 (0.0007) [2023-10-08 08:43:11,215][53885] Updated weights for policy 1, policy_version 22482 (0.0007) [2023-10-08 08:43:11,579][53885] Updated weights for policy 1, policy_version 22492 (0.0009) [2023-10-08 08:43:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46170112. Throughput: 0: 1843.2, 1: 1812.0. Samples: 11544906. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:43:12,016][52710] Avg episode reward: [(0, '25.610'), (1, '27.280')] [2023-10-08 08:43:12,319][53852] Updated weights for policy 0, policy_version 22600 (0.0008) [2023-10-08 08:43:12,686][53852] Updated weights for policy 0, policy_version 22610 (0.0008) [2023-10-08 08:43:13,057][53852] Updated weights for policy 0, policy_version 22620 (0.0008) [2023-10-08 08:43:13,209][53500] Saving new best policy, reward=25.610! [2023-10-08 08:43:15,149][53885] Updated weights for policy 1, policy_version 22502 (0.0009) [2023-10-08 08:43:15,519][53885] Updated weights for policy 1, policy_version 22512 (0.0011) [2023-10-08 08:43:15,894][53885] Updated weights for policy 1, policy_version 22522 (0.0008) [2023-10-08 08:43:16,624][53852] Updated weights for policy 0, policy_version 22630 (0.0007) [2023-10-08 08:43:16,988][53852] Updated weights for policy 0, policy_version 22640 (0.0010) [2023-10-08 08:43:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46235648. Throughput: 0: 1843.3, 1: 1812.3. Samples: 11567372. Policy #0 lag: (min: 31.0, avg: 42.3, max: 63.0) [2023-10-08 08:43:17,015][52710] Avg episode reward: [(0, '24.530'), (1, '27.950')] [2023-10-08 08:43:17,366][53852] Updated weights for policy 0, policy_version 22650 (0.0009) [2023-10-08 08:43:19,633][53885] Updated weights for policy 1, policy_version 22532 (0.0008) [2023-10-08 08:43:20,002][53885] Updated weights for policy 1, policy_version 22542 (0.0008) [2023-10-08 08:43:20,377][53885] Updated weights for policy 1, policy_version 22552 (0.0008) [2023-10-08 08:43:21,127][53852] Updated weights for policy 0, policy_version 22660 (0.0008) [2023-10-08 08:43:21,501][53852] Updated weights for policy 0, policy_version 22670 (0.0008) [2023-10-08 08:43:21,873][53852] Updated weights for policy 0, policy_version 22680 (0.0007) [2023-10-08 08:43:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46301184. Throughput: 0: 1831.1, 1: 1817.9. Samples: 11588904. Policy #0 lag: (min: 31.0, avg: 42.3, max: 63.0) [2023-10-08 08:43:22,016][52710] Avg episode reward: [(0, '24.260'), (1, '25.850')] [2023-10-08 08:43:23,998][53885] Updated weights for policy 1, policy_version 22562 (0.0009) [2023-10-08 08:43:24,372][53885] Updated weights for policy 1, policy_version 22572 (0.0010) [2023-10-08 08:43:24,740][53885] Updated weights for policy 1, policy_version 22582 (0.0009) [2023-10-08 08:43:25,103][53885] Updated weights for policy 1, policy_version 22592 (0.0010) [2023-10-08 08:43:25,367][53852] Updated weights for policy 0, policy_version 22690 (0.0008) [2023-10-08 08:43:25,741][53852] Updated weights for policy 0, policy_version 22700 (0.0010) [2023-10-08 08:43:26,113][53852] Updated weights for policy 0, policy_version 22710 (0.0010) [2023-10-08 08:43:26,487][53852] Updated weights for policy 0, policy_version 22720 (0.0008) [2023-10-08 08:43:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 46399488. Throughput: 0: 1844.2, 1: 1815.0. Samples: 11600506. Policy #0 lag: (min: 31.0, avg: 42.3, max: 63.0) [2023-10-08 08:43:27,016][52710] Avg episode reward: [(0, '24.870'), (1, '26.230')] [2023-10-08 08:43:28,838][53885] Updated weights for policy 1, policy_version 22602 (0.0007) [2023-10-08 08:43:29,204][53885] Updated weights for policy 1, policy_version 22612 (0.0009) [2023-10-08 08:43:29,578][53885] Updated weights for policy 1, policy_version 22622 (0.0008) [2023-10-08 08:43:30,015][53852] Updated weights for policy 0, policy_version 22730 (0.0007) [2023-10-08 08:43:30,381][53852] Updated weights for policy 0, policy_version 22740 (0.0009) [2023-10-08 08:43:30,753][53852] Updated weights for policy 0, policy_version 22750 (0.0009) [2023-10-08 08:43:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46465024. Throughput: 0: 1831.9, 1: 1829.8. Samples: 11622046. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-08 08:43:32,016][52710] Avg episode reward: [(0, '23.820'), (1, '30.770')] [2023-10-08 08:43:33,291][53885] Updated weights for policy 1, policy_version 22632 (0.0008) [2023-10-08 08:43:33,655][53885] Updated weights for policy 1, policy_version 22642 (0.0007) [2023-10-08 08:43:34,018][53885] Updated weights for policy 1, policy_version 22652 (0.0009) [2023-10-08 08:43:34,327][53852] Updated weights for policy 0, policy_version 22760 (0.0011) [2023-10-08 08:43:34,706][53852] Updated weights for policy 0, policy_version 22770 (0.0010) [2023-10-08 08:43:35,071][53852] Updated weights for policy 0, policy_version 22780 (0.0009) [2023-10-08 08:43:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 46530560. Throughput: 0: 1849.7, 1: 1833.6. Samples: 11644888. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-08 08:43:37,016][52710] Avg episode reward: [(0, '23.550'), (1, '26.240')] [2023-10-08 08:43:37,667][53885] Updated weights for policy 1, policy_version 22662 (0.0007) [2023-10-08 08:43:38,033][53885] Updated weights for policy 1, policy_version 22672 (0.0007) [2023-10-08 08:43:38,397][53885] Updated weights for policy 1, policy_version 22682 (0.0008) [2023-10-08 08:43:38,770][53852] Updated weights for policy 0, policy_version 22790 (0.0008) [2023-10-08 08:43:39,140][53852] Updated weights for policy 0, policy_version 22800 (0.0009) [2023-10-08 08:43:39,509][53852] Updated weights for policy 0, policy_version 22810 (0.0009) [2023-10-08 08:43:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 46596096. Throughput: 0: 1829.2, 1: 1836.0. Samples: 11655308. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-08 08:43:42,016][52710] Avg episode reward: [(0, '23.950'), (1, '25.970')] [2023-10-08 08:43:42,078][53885] Updated weights for policy 1, policy_version 22692 (0.0007) [2023-10-08 08:43:42,444][53885] Updated weights for policy 1, policy_version 22702 (0.0009) [2023-10-08 08:43:42,811][53885] Updated weights for policy 1, policy_version 22712 (0.0010) [2023-10-08 08:43:43,148][53852] Updated weights for policy 0, policy_version 22820 (0.0008) [2023-10-08 08:43:43,517][53852] Updated weights for policy 0, policy_version 22830 (0.0008) [2023-10-08 08:43:43,889][53852] Updated weights for policy 0, policy_version 22840 (0.0008) [2023-10-08 08:43:46,529][53885] Updated weights for policy 1, policy_version 22722 (0.0007) [2023-10-08 08:43:46,901][53885] Updated weights for policy 1, policy_version 22732 (0.0007) [2023-10-08 08:43:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 46661632. Throughput: 0: 1851.9, 1: 1835.2. Samples: 11677954. Policy #0 lag: (min: 31.0, avg: 40.2, max: 63.0) [2023-10-08 08:43:47,016][52710] Avg episode reward: [(0, '25.400'), (1, '25.580')] [2023-10-08 08:43:47,273][53885] Updated weights for policy 1, policy_version 22742 (0.0007) [2023-10-08 08:43:47,555][53852] Updated weights for policy 0, policy_version 22850 (0.0007) [2023-10-08 08:43:47,640][53885] Updated weights for policy 1, policy_version 22752 (0.0009) [2023-10-08 08:43:47,923][53852] Updated weights for policy 0, policy_version 22860 (0.0010) [2023-10-08 08:43:48,305][53852] Updated weights for policy 0, policy_version 22870 (0.0009) [2023-10-08 08:43:48,667][53852] Updated weights for policy 0, policy_version 22880 (0.0008) [2023-10-08 08:43:51,350][53885] Updated weights for policy 1, policy_version 22762 (0.0007) [2023-10-08 08:43:51,712][53885] Updated weights for policy 1, policy_version 22772 (0.0007) [2023-10-08 08:43:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 46727168. Throughput: 0: 1851.3, 1: 1841.6. Samples: 11700304. Policy #0 lag: (min: 16.0, avg: 37.6, max: 48.0) [2023-10-08 08:43:52,016][52710] Avg episode reward: [(0, '25.590'), (1, '23.730')] [2023-10-08 08:43:52,081][53885] Updated weights for policy 1, policy_version 22782 (0.0009) [2023-10-08 08:43:52,414][53852] Updated weights for policy 0, policy_version 22890 (0.0007) [2023-10-08 08:43:52,786][53852] Updated weights for policy 0, policy_version 22900 (0.0007) [2023-10-08 08:43:53,152][53852] Updated weights for policy 0, policy_version 22910 (0.0007) [2023-10-08 08:43:55,647][53885] Updated weights for policy 1, policy_version 22792 (0.0011) [2023-10-08 08:43:56,014][53885] Updated weights for policy 1, policy_version 22802 (0.0010) [2023-10-08 08:43:56,382][53885] Updated weights for policy 1, policy_version 22812 (0.0008) [2023-10-08 08:43:56,875][53852] Updated weights for policy 0, policy_version 22920 (0.0008) [2023-10-08 08:43:57,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46825472. Throughput: 0: 1852.8, 1: 1839.6. Samples: 11711066. Policy #0 lag: (min: 16.0, avg: 37.6, max: 48.0) [2023-10-08 08:43:57,015][52710] Avg episode reward: [(0, '27.020'), (1, '25.140')] [2023-10-08 08:43:57,243][53852] Updated weights for policy 0, policy_version 22930 (0.0009) [2023-10-08 08:43:57,611][53852] Updated weights for policy 0, policy_version 22940 (0.0009) [2023-10-08 08:43:57,755][53500] Saving new best policy, reward=27.020! [2023-10-08 08:43:59,921][53885] Updated weights for policy 1, policy_version 22822 (0.0008) [2023-10-08 08:44:00,309][53885] Updated weights for policy 1, policy_version 22832 (0.0010) [2023-10-08 08:44:00,677][53885] Updated weights for policy 1, policy_version 22842 (0.0007) [2023-10-08 08:44:01,212][53852] Updated weights for policy 0, policy_version 22950 (0.0008) [2023-10-08 08:44:01,574][53852] Updated weights for policy 0, policy_version 22960 (0.0008) [2023-10-08 08:44:01,951][53852] Updated weights for policy 0, policy_version 22970 (0.0010) [2023-10-08 08:44:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46891008. Throughput: 0: 1844.8, 1: 1835.5. Samples: 11732984. Policy #0 lag: (min: 16.0, avg: 37.6, max: 48.0) [2023-10-08 08:44:02,016][52710] Avg episode reward: [(0, '24.680'), (1, '25.450')] [2023-10-08 08:44:04,433][53885] Updated weights for policy 1, policy_version 22852 (0.0009) [2023-10-08 08:44:04,798][53885] Updated weights for policy 1, policy_version 22862 (0.0008) [2023-10-08 08:44:05,174][53885] Updated weights for policy 1, policy_version 22872 (0.0007) [2023-10-08 08:44:05,720][53852] Updated weights for policy 0, policy_version 22980 (0.0008) [2023-10-08 08:44:06,088][53852] Updated weights for policy 0, policy_version 22990 (0.0007) [2023-10-08 08:44:06,461][53852] Updated weights for policy 0, policy_version 23000 (0.0010) [2023-10-08 08:44:07,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 46989312. Throughput: 0: 1829.5, 1: 1839.6. Samples: 11754014. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:44:07,016][52710] Avg episode reward: [(0, '23.510'), (1, '27.540')] [2023-10-08 08:44:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000022880_23429120.pth... [2023-10-08 08:44:07,023][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000023008_23560192.pth... [2023-10-08 08:44:07,054][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000021184_21692416.pth [2023-10-08 08:44:07,054][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000021280_21790720.pth [2023-10-08 08:44:08,952][53885] Updated weights for policy 1, policy_version 22882 (0.0008) [2023-10-08 08:44:09,328][53885] Updated weights for policy 1, policy_version 22892 (0.0008) [2023-10-08 08:44:09,709][53885] Updated weights for policy 1, policy_version 22902 (0.0008) [2023-10-08 08:44:09,925][53852] Updated weights for policy 0, policy_version 23010 (0.0008) [2023-10-08 08:44:10,072][53885] Updated weights for policy 1, policy_version 22912 (0.0009) [2023-10-08 08:44:10,290][53852] Updated weights for policy 0, policy_version 23020 (0.0010) [2023-10-08 08:44:10,665][53852] Updated weights for policy 0, policy_version 23030 (0.0010) [2023-10-08 08:44:11,029][53852] Updated weights for policy 0, policy_version 23040 (0.0009) [2023-10-08 08:44:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 47054848. Throughput: 0: 1842.0, 1: 1838.0. Samples: 11766104. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:44:12,016][52710] Avg episode reward: [(0, '22.970'), (1, '27.930')] [2023-10-08 08:44:13,648][53885] Updated weights for policy 1, policy_version 22922 (0.0009) [2023-10-08 08:44:14,011][53885] Updated weights for policy 1, policy_version 22932 (0.0011) [2023-10-08 08:44:14,375][53885] Updated weights for policy 1, policy_version 22942 (0.0009) [2023-10-08 08:44:14,706][53852] Updated weights for policy 0, policy_version 23050 (0.0007) [2023-10-08 08:44:15,086][53852] Updated weights for policy 0, policy_version 23060 (0.0007) [2023-10-08 08:44:15,444][53852] Updated weights for policy 0, policy_version 23070 (0.0009) [2023-10-08 08:44:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 47120384. Throughput: 0: 1827.3, 1: 1831.9. Samples: 11786712. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:44:17,016][52710] Avg episode reward: [(0, '21.110'), (1, '27.430')] [2023-10-08 08:44:18,022][53885] Updated weights for policy 1, policy_version 22952 (0.0008) [2023-10-08 08:44:18,393][53885] Updated weights for policy 1, policy_version 22962 (0.0009) [2023-10-08 08:44:18,757][53885] Updated weights for policy 1, policy_version 22972 (0.0008) [2023-10-08 08:44:19,008][53852] Updated weights for policy 0, policy_version 23080 (0.0009) [2023-10-08 08:44:19,378][53852] Updated weights for policy 0, policy_version 23090 (0.0010) [2023-10-08 08:44:19,744][53852] Updated weights for policy 0, policy_version 23100 (0.0007) [2023-10-08 08:44:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47185920. Throughput: 0: 1838.8, 1: 1825.0. Samples: 11809760. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) [2023-10-08 08:44:22,016][52710] Avg episode reward: [(0, '24.440'), (1, '26.950')] [2023-10-08 08:44:22,453][53885] Updated weights for policy 1, policy_version 22982 (0.0009) [2023-10-08 08:44:22,817][53885] Updated weights for policy 1, policy_version 22992 (0.0007) [2023-10-08 08:44:23,197][53885] Updated weights for policy 1, policy_version 23002 (0.0008) [2023-10-08 08:44:23,427][53852] Updated weights for policy 0, policy_version 23110 (0.0009) [2023-10-08 08:44:23,785][53852] Updated weights for policy 0, policy_version 23120 (0.0008) [2023-10-08 08:44:24,156][53852] Updated weights for policy 0, policy_version 23130 (0.0010) [2023-10-08 08:44:26,884][53885] Updated weights for policy 1, policy_version 23012 (0.0007) [2023-10-08 08:44:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47251456. Throughput: 0: 1826.9, 1: 1826.2. Samples: 11819698. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:44:27,016][52710] Avg episode reward: [(0, '23.160'), (1, '24.260')] [2023-10-08 08:44:27,255][53885] Updated weights for policy 1, policy_version 23022 (0.0008) [2023-10-08 08:44:27,618][53885] Updated weights for policy 1, policy_version 23032 (0.0009) [2023-10-08 08:44:27,767][53852] Updated weights for policy 0, policy_version 23140 (0.0009) [2023-10-08 08:44:28,132][53852] Updated weights for policy 0, policy_version 23150 (0.0008) [2023-10-08 08:44:28,502][53852] Updated weights for policy 0, policy_version 23160 (0.0008) [2023-10-08 08:44:31,434][53885] Updated weights for policy 1, policy_version 23042 (0.0007) [2023-10-08 08:44:31,794][53885] Updated weights for policy 1, policy_version 23052 (0.0009) [2023-10-08 08:44:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 47316992. Throughput: 0: 1837.6, 1: 1820.7. Samples: 11842578. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:44:32,016][52710] Avg episode reward: [(0, '24.490'), (1, '25.900')] [2023-10-08 08:44:32,139][53852] Updated weights for policy 0, policy_version 23170 (0.0007) [2023-10-08 08:44:32,159][53885] Updated weights for policy 1, policy_version 23062 (0.0008) [2023-10-08 08:44:32,499][53852] Updated weights for policy 0, policy_version 23180 (0.0010) [2023-10-08 08:44:32,527][53885] Updated weights for policy 1, policy_version 23072 (0.0007) [2023-10-08 08:44:32,879][53852] Updated weights for policy 0, policy_version 23190 (0.0009) [2023-10-08 08:44:33,244][53852] Updated weights for policy 0, policy_version 23200 (0.0009) [2023-10-08 08:44:36,189][53885] Updated weights for policy 1, policy_version 23082 (0.0011) [2023-10-08 08:44:36,558][53885] Updated weights for policy 1, policy_version 23092 (0.0007) [2023-10-08 08:44:36,916][53852] Updated weights for policy 0, policy_version 23210 (0.0008) [2023-10-08 08:44:36,922][53885] Updated weights for policy 1, policy_version 23102 (0.0008) [2023-10-08 08:44:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47415296. Throughput: 0: 1834.5, 1: 1814.1. Samples: 11864488. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:44:37,016][52710] Avg episode reward: [(0, '22.680'), (1, '26.070')] [2023-10-08 08:44:37,286][53852] Updated weights for policy 0, policy_version 23220 (0.0007) [2023-10-08 08:44:37,661][53852] Updated weights for policy 0, policy_version 23230 (0.0008) [2023-10-08 08:44:40,609][53885] Updated weights for policy 1, policy_version 23112 (0.0007) [2023-10-08 08:44:40,982][53885] Updated weights for policy 1, policy_version 23122 (0.0008) [2023-10-08 08:44:41,347][53852] Updated weights for policy 0, policy_version 23240 (0.0008) [2023-10-08 08:44:41,349][53885] Updated weights for policy 1, policy_version 23132 (0.0008) [2023-10-08 08:44:41,709][53852] Updated weights for policy 0, policy_version 23250 (0.0009) [2023-10-08 08:44:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47480832. Throughput: 0: 1834.1, 1: 1820.7. Samples: 11875532. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:44:42,016][52710] Avg episode reward: [(0, '22.250'), (1, '23.860')] [2023-10-08 08:44:42,082][53852] Updated weights for policy 0, policy_version 23260 (0.0008) [2023-10-08 08:44:45,005][53885] Updated weights for policy 1, policy_version 23142 (0.0007) [2023-10-08 08:44:45,374][53885] Updated weights for policy 1, policy_version 23152 (0.0008) [2023-10-08 08:44:45,748][53885] Updated weights for policy 1, policy_version 23162 (0.0008) [2023-10-08 08:44:45,859][53852] Updated weights for policy 0, policy_version 23270 (0.0009) [2023-10-08 08:44:46,223][53852] Updated weights for policy 0, policy_version 23280 (0.0008) [2023-10-08 08:44:46,594][53852] Updated weights for policy 0, policy_version 23290 (0.0008) [2023-10-08 08:44:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 47579136. Throughput: 0: 1833.3, 1: 1819.7. Samples: 11897366. Policy #0 lag: (min: 0.0, avg: 16.5, max: 32.0) [2023-10-08 08:44:47,016][52710] Avg episode reward: [(0, '23.140'), (1, '26.160')] [2023-10-08 08:44:49,408][53885] Updated weights for policy 1, policy_version 23172 (0.0009) [2023-10-08 08:44:49,774][53885] Updated weights for policy 1, policy_version 23182 (0.0008) [2023-10-08 08:44:50,137][53885] Updated weights for policy 1, policy_version 23192 (0.0008) [2023-10-08 08:44:50,191][53852] Updated weights for policy 0, policy_version 23300 (0.0007) [2023-10-08 08:44:50,565][53852] Updated weights for policy 0, policy_version 23310 (0.0007) [2023-10-08 08:44:50,937][53852] Updated weights for policy 0, policy_version 23320 (0.0009) [2023-10-08 08:44:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 47644672. Throughput: 0: 1830.2, 1: 1818.6. Samples: 11918212. Policy #0 lag: (min: 0.0, avg: 16.5, max: 32.0) [2023-10-08 08:44:52,016][52710] Avg episode reward: [(0, '21.590'), (1, '28.850')] [2023-10-08 08:44:53,891][53885] Updated weights for policy 1, policy_version 23202 (0.0008) [2023-10-08 08:44:54,263][53885] Updated weights for policy 1, policy_version 23212 (0.0007) [2023-10-08 08:44:54,638][53885] Updated weights for policy 1, policy_version 23222 (0.0010) [2023-10-08 08:44:54,692][53852] Updated weights for policy 0, policy_version 23330 (0.0007) [2023-10-08 08:44:55,015][53885] Updated weights for policy 1, policy_version 23232 (0.0009) [2023-10-08 08:44:55,067][53852] Updated weights for policy 0, policy_version 23340 (0.0009) [2023-10-08 08:44:55,436][53852] Updated weights for policy 0, policy_version 23350 (0.0009) [2023-10-08 08:44:55,817][53852] Updated weights for policy 0, policy_version 23360 (0.0007) [2023-10-08 08:44:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 47710208. Throughput: 0: 1833.0, 1: 1814.1. Samples: 11930226. Policy #0 lag: (min: 0.0, avg: 16.5, max: 32.0) [2023-10-08 08:44:57,016][52710] Avg episode reward: [(0, '21.730'), (1, '27.500')] [2023-10-08 08:44:58,798][53885] Updated weights for policy 1, policy_version 23242 (0.0008) [2023-10-08 08:44:59,167][53885] Updated weights for policy 1, policy_version 23252 (0.0009) [2023-10-08 08:44:59,407][53852] Updated weights for policy 0, policy_version 23370 (0.0007) [2023-10-08 08:44:59,537][53885] Updated weights for policy 1, policy_version 23262 (0.0009) [2023-10-08 08:44:59,766][53852] Updated weights for policy 0, policy_version 23380 (0.0008) [2023-10-08 08:45:00,138][53852] Updated weights for policy 0, policy_version 23390 (0.0009) [2023-10-08 08:45:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 47775744. Throughput: 0: 1839.0, 1: 1814.4. Samples: 11951112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:45:02,015][52710] Avg episode reward: [(0, '22.910'), (1, '29.400')] [2023-10-08 08:45:03,101][53885] Updated weights for policy 1, policy_version 23272 (0.0011) [2023-10-08 08:45:03,460][53885] Updated weights for policy 1, policy_version 23282 (0.0011) [2023-10-08 08:45:03,828][53885] Updated weights for policy 1, policy_version 23292 (0.0008) [2023-10-08 08:45:03,854][53852] Updated weights for policy 0, policy_version 23400 (0.0010) [2023-10-08 08:45:04,220][53852] Updated weights for policy 0, policy_version 23410 (0.0010) [2023-10-08 08:45:04,589][53852] Updated weights for policy 0, policy_version 23420 (0.0010) [2023-10-08 08:45:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 47841280. Throughput: 0: 1831.3, 1: 1815.9. Samples: 11973882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:45:07,016][52710] Avg episode reward: [(0, '23.780'), (1, '30.850')] [2023-10-08 08:45:07,624][53885] Updated weights for policy 1, policy_version 23302 (0.0007) [2023-10-08 08:45:07,981][53885] Updated weights for policy 1, policy_version 23312 (0.0007) [2023-10-08 08:45:08,139][53852] Updated weights for policy 0, policy_version 23430 (0.0009) [2023-10-08 08:45:08,344][53885] Updated weights for policy 1, policy_version 23322 (0.0008) [2023-10-08 08:45:08,513][53852] Updated weights for policy 0, policy_version 23440 (0.0008) [2023-10-08 08:45:08,884][53852] Updated weights for policy 0, policy_version 23450 (0.0008) [2023-10-08 08:45:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47906816. Throughput: 0: 1836.8, 1: 1814.6. Samples: 11984010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:45:12,016][52710] Avg episode reward: [(0, '24.250'), (1, '29.260')] [2023-10-08 08:45:12,043][53885] Updated weights for policy 1, policy_version 23332 (0.0008) [2023-10-08 08:45:12,390][53852] Updated weights for policy 0, policy_version 23460 (0.0008) [2023-10-08 08:45:12,417][53885] Updated weights for policy 1, policy_version 23342 (0.0009) [2023-10-08 08:45:12,763][53852] Updated weights for policy 0, policy_version 23470 (0.0007) [2023-10-08 08:45:12,786][53885] Updated weights for policy 1, policy_version 23352 (0.0009) [2023-10-08 08:45:13,134][53852] Updated weights for policy 0, policy_version 23480 (0.0007) [2023-10-08 08:45:16,520][53885] Updated weights for policy 1, policy_version 23362 (0.0007) [2023-10-08 08:45:16,766][53852] Updated weights for policy 0, policy_version 23490 (0.0010) [2023-10-08 08:45:16,878][53885] Updated weights for policy 1, policy_version 23372 (0.0009) [2023-10-08 08:45:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47972352. Throughput: 0: 1840.1, 1: 1817.2. Samples: 12007156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:45:17,016][52710] Avg episode reward: [(0, '24.160'), (1, '29.560')] [2023-10-08 08:45:17,136][53852] Updated weights for policy 0, policy_version 23500 (0.0008) [2023-10-08 08:45:17,246][53885] Updated weights for policy 1, policy_version 23382 (0.0008) [2023-10-08 08:45:17,506][53852] Updated weights for policy 0, policy_version 23510 (0.0008) [2023-10-08 08:45:17,611][53885] Updated weights for policy 1, policy_version 23392 (0.0009) [2023-10-08 08:45:17,882][53852] Updated weights for policy 0, policy_version 23520 (0.0008) [2023-10-08 08:45:21,336][53885] Updated weights for policy 1, policy_version 23402 (0.0007) [2023-10-08 08:45:21,584][53852] Updated weights for policy 0, policy_version 23530 (0.0009) [2023-10-08 08:45:21,703][53885] Updated weights for policy 1, policy_version 23412 (0.0007) [2023-10-08 08:45:21,951][53852] Updated weights for policy 0, policy_version 23540 (0.0008) [2023-10-08 08:45:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48037888. Throughput: 0: 1829.1, 1: 1822.6. Samples: 12028814. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-08 08:45:22,015][52710] Avg episode reward: [(0, '23.270'), (1, '30.640')] [2023-10-08 08:45:22,062][53885] Updated weights for policy 1, policy_version 23422 (0.0007) [2023-10-08 08:45:22,329][53852] Updated weights for policy 0, policy_version 23550 (0.0008) [2023-10-08 08:45:25,592][53885] Updated weights for policy 1, policy_version 23432 (0.0010) [2023-10-08 08:45:25,956][53885] Updated weights for policy 1, policy_version 23442 (0.0009) [2023-10-08 08:45:26,099][53852] Updated weights for policy 0, policy_version 23560 (0.0009) [2023-10-08 08:45:26,325][53885] Updated weights for policy 1, policy_version 23452 (0.0008) [2023-10-08 08:45:26,473][53852] Updated weights for policy 0, policy_version 23570 (0.0008) [2023-10-08 08:45:26,844][53852] Updated weights for policy 0, policy_version 23580 (0.0009) [2023-10-08 08:45:27,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 48168960. Throughput: 0: 1836.3, 1: 1819.6. Samples: 12040046. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-08 08:45:27,016][52710] Avg episode reward: [(0, '23.810'), (1, '29.060')] [2023-10-08 08:45:29,954][53885] Updated weights for policy 1, policy_version 23462 (0.0009) [2023-10-08 08:45:30,336][53885] Updated weights for policy 1, policy_version 23472 (0.0010) [2023-10-08 08:45:30,602][53852] Updated weights for policy 0, policy_version 23590 (0.0007) [2023-10-08 08:45:30,698][53885] Updated weights for policy 1, policy_version 23482 (0.0007) [2023-10-08 08:45:30,959][53852] Updated weights for policy 0, policy_version 23600 (0.0009) [2023-10-08 08:45:31,335][53852] Updated weights for policy 0, policy_version 23610 (0.0009) [2023-10-08 08:45:32,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 48234496. Throughput: 0: 1828.4, 1: 1814.7. Samples: 12061302. Policy #0 lag: (min: 8.0, avg: 33.1, max: 40.0) [2023-10-08 08:45:32,016][52710] Avg episode reward: [(0, '24.020'), (1, '29.180')] [2023-10-08 08:45:34,270][53885] Updated weights for policy 1, policy_version 23492 (0.0008) [2023-10-08 08:45:34,635][53885] Updated weights for policy 1, policy_version 23502 (0.0010) [2023-10-08 08:45:35,007][53885] Updated weights for policy 1, policy_version 23512 (0.0010) [2023-10-08 08:45:35,143][53852] Updated weights for policy 0, policy_version 23620 (0.0008) [2023-10-08 08:45:35,535][53852] Updated weights for policy 0, policy_version 23630 (0.0009) [2023-10-08 08:45:35,905][53852] Updated weights for policy 0, policy_version 23640 (0.0008) [2023-10-08 08:45:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 48300032. Throughput: 0: 1829.3, 1: 1818.8. Samples: 12082374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:45:37,016][52710] Avg episode reward: [(0, '24.600'), (1, '28.840')] [2023-10-08 08:45:38,736][53885] Updated weights for policy 1, policy_version 23522 (0.0008) [2023-10-08 08:45:39,110][53885] Updated weights for policy 1, policy_version 23532 (0.0011) [2023-10-08 08:45:39,486][53885] Updated weights for policy 1, policy_version 23542 (0.0010) [2023-10-08 08:45:39,534][53852] Updated weights for policy 0, policy_version 23650 (0.0008) [2023-10-08 08:45:39,852][53885] Updated weights for policy 1, policy_version 23552 (0.0008) [2023-10-08 08:45:39,908][53852] Updated weights for policy 0, policy_version 23660 (0.0007) [2023-10-08 08:45:40,282][53852] Updated weights for policy 0, policy_version 23670 (0.0009) [2023-10-08 08:45:40,647][53852] Updated weights for policy 0, policy_version 23680 (0.0010) [2023-10-08 08:45:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48365568. Throughput: 0: 1827.3, 1: 1815.7. Samples: 12094162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:45:42,016][52710] Avg episode reward: [(0, '24.980'), (1, '27.300')] [2023-10-08 08:45:43,641][53885] Updated weights for policy 1, policy_version 23562 (0.0008) [2023-10-08 08:45:44,012][53885] Updated weights for policy 1, policy_version 23572 (0.0010) [2023-10-08 08:45:44,300][53852] Updated weights for policy 0, policy_version 23690 (0.0009) [2023-10-08 08:45:44,388][53885] Updated weights for policy 1, policy_version 23582 (0.0009) [2023-10-08 08:45:44,670][53852] Updated weights for policy 0, policy_version 23700 (0.0009) [2023-10-08 08:45:45,031][53852] Updated weights for policy 0, policy_version 23710 (0.0009) [2023-10-08 08:45:47,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 48431104. Throughput: 0: 1825.3, 1: 1822.6. Samples: 12115266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:45:47,016][52710] Avg episode reward: [(0, '24.360'), (1, '28.540')] [2023-10-08 08:45:48,204][53885] Updated weights for policy 1, policy_version 23592 (0.0009) [2023-10-08 08:45:48,571][53885] Updated weights for policy 1, policy_version 23602 (0.0008) [2023-10-08 08:45:48,812][53852] Updated weights for policy 0, policy_version 23720 (0.0007) [2023-10-08 08:45:48,953][53885] Updated weights for policy 1, policy_version 23612 (0.0007) [2023-10-08 08:45:49,191][53852] Updated weights for policy 0, policy_version 23730 (0.0008) [2023-10-08 08:45:49,562][53852] Updated weights for policy 0, policy_version 23740 (0.0007) [2023-10-08 08:45:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 48496640. Throughput: 0: 1822.3, 1: 1827.7. Samples: 12138130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:45:52,016][52710] Avg episode reward: [(0, '24.490'), (1, '29.310')] [2023-10-08 08:45:52,527][53885] Updated weights for policy 1, policy_version 23622 (0.0008) [2023-10-08 08:45:52,893][53885] Updated weights for policy 1, policy_version 23632 (0.0010) [2023-10-08 08:45:53,258][53885] Updated weights for policy 1, policy_version 23642 (0.0008) [2023-10-08 08:45:53,258][53852] Updated weights for policy 0, policy_version 23750 (0.0007) [2023-10-08 08:45:53,618][53852] Updated weights for policy 0, policy_version 23760 (0.0008) [2023-10-08 08:45:53,987][53852] Updated weights for policy 0, policy_version 23770 (0.0007) [2023-10-08 08:45:56,854][53885] Updated weights for policy 1, policy_version 23652 (0.0008) [2023-10-08 08:45:57,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 48562176. Throughput: 0: 1818.1, 1: 1827.1. Samples: 12148048. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-08 08:45:57,016][52710] Avg episode reward: [(0, '25.490'), (1, '28.750')] [2023-10-08 08:45:57,235][53885] Updated weights for policy 1, policy_version 23662 (0.0010) [2023-10-08 08:45:57,602][53885] Updated weights for policy 1, policy_version 23672 (0.0008) [2023-10-08 08:45:57,730][53852] Updated weights for policy 0, policy_version 23780 (0.0009) [2023-10-08 08:45:58,091][53852] Updated weights for policy 0, policy_version 23790 (0.0008) [2023-10-08 08:45:58,456][53852] Updated weights for policy 0, policy_version 23800 (0.0011) [2023-10-08 08:46:01,290][53885] Updated weights for policy 1, policy_version 23682 (0.0007) [2023-10-08 08:46:01,665][53885] Updated weights for policy 1, policy_version 23692 (0.0007) [2023-10-08 08:46:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 48627712. Throughput: 0: 1811.2, 1: 1829.1. Samples: 12170966. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-08 08:46:02,016][52710] Avg episode reward: [(0, '24.090'), (1, '29.430')] [2023-10-08 08:46:02,030][53885] Updated weights for policy 1, policy_version 23702 (0.0008) [2023-10-08 08:46:02,120][53852] Updated weights for policy 0, policy_version 23810 (0.0010) [2023-10-08 08:46:02,400][53885] Updated weights for policy 1, policy_version 23712 (0.0009) [2023-10-08 08:46:02,486][53852] Updated weights for policy 0, policy_version 23820 (0.0007) [2023-10-08 08:46:02,859][53852] Updated weights for policy 0, policy_version 23830 (0.0008) [2023-10-08 08:46:03,231][53852] Updated weights for policy 0, policy_version 23840 (0.0010) [2023-10-08 08:46:06,101][53885] Updated weights for policy 1, policy_version 23722 (0.0011) [2023-10-08 08:46:06,464][53885] Updated weights for policy 1, policy_version 23732 (0.0008) [2023-10-08 08:46:06,799][53852] Updated weights for policy 0, policy_version 23850 (0.0008) [2023-10-08 08:46:06,838][53885] Updated weights for policy 1, policy_version 23742 (0.0009) [2023-10-08 08:46:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48726016. Throughput: 0: 1821.0, 1: 1819.1. Samples: 12192622. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-08 08:46:07,016][52710] Avg episode reward: [(0, '25.750'), (1, '30.520')] [2023-10-08 08:46:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000023744_24313856.pth... [2023-10-08 08:46:07,057][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000022016_22544384.pth [2023-10-08 08:46:07,163][53852] Updated weights for policy 0, policy_version 23860 (0.0010) [2023-10-08 08:46:07,538][53852] Updated weights for policy 0, policy_version 23870 (0.0010) [2023-10-08 08:46:07,606][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000023872_24444928.pth... [2023-10-08 08:46:07,635][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000022144_22675456.pth [2023-10-08 08:46:10,635][53885] Updated weights for policy 1, policy_version 23752 (0.0009) [2023-10-08 08:46:11,009][53885] Updated weights for policy 1, policy_version 23762 (0.0008) [2023-10-08 08:46:11,144][53852] Updated weights for policy 0, policy_version 23880 (0.0008) [2023-10-08 08:46:11,371][53885] Updated weights for policy 1, policy_version 23772 (0.0007) [2023-10-08 08:46:11,510][53852] Updated weights for policy 0, policy_version 23890 (0.0008) [2023-10-08 08:46:11,889][53852] Updated weights for policy 0, policy_version 23900 (0.0010) [2023-10-08 08:46:12,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48791552. Throughput: 0: 1816.5, 1: 1821.8. Samples: 12203770. Policy #0 lag: (min: 31.0, avg: 42.0, max: 63.0) [2023-10-08 08:46:12,016][52710] Avg episode reward: [(0, '26.720'), (1, '29.270')] [2023-10-08 08:46:15,319][53885] Updated weights for policy 1, policy_version 23782 (0.0008) [2023-10-08 08:46:15,607][53852] Updated weights for policy 0, policy_version 23910 (0.0008) [2023-10-08 08:46:15,707][53885] Updated weights for policy 1, policy_version 23792 (0.0008) [2023-10-08 08:46:15,974][53852] Updated weights for policy 0, policy_version 23920 (0.0008) [2023-10-08 08:46:16,079][53885] Updated weights for policy 1, policy_version 23802 (0.0008) [2023-10-08 08:46:16,350][53852] Updated weights for policy 0, policy_version 23930 (0.0008) [2023-10-08 08:46:17,015][52710] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 48889856. Throughput: 0: 1815.2, 1: 1828.4. Samples: 12225264. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:46:17,016][52710] Avg episode reward: [(0, '25.140'), (1, '28.170')] [2023-10-08 08:46:19,701][53885] Updated weights for policy 1, policy_version 23812 (0.0010) [2023-10-08 08:46:19,936][53852] Updated weights for policy 0, policy_version 23940 (0.0009) [2023-10-08 08:46:20,066][53885] Updated weights for policy 1, policy_version 23822 (0.0008) [2023-10-08 08:46:20,313][53852] Updated weights for policy 0, policy_version 23950 (0.0008) [2023-10-08 08:46:20,432][53885] Updated weights for policy 1, policy_version 23832 (0.0009) [2023-10-08 08:46:20,688][53852] Updated weights for policy 0, policy_version 23960 (0.0008) [2023-10-08 08:46:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 48955392. Throughput: 0: 1821.1, 1: 1814.0. Samples: 12245950. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:46:22,016][52710] Avg episode reward: [(0, '25.660'), (1, '26.930')] [2023-10-08 08:46:24,093][53885] Updated weights for policy 1, policy_version 23842 (0.0009) [2023-10-08 08:46:24,462][53885] Updated weights for policy 1, policy_version 23852 (0.0008) [2023-10-08 08:46:24,520][53852] Updated weights for policy 0, policy_version 23970 (0.0008) [2023-10-08 08:46:24,836][53885] Updated weights for policy 1, policy_version 23862 (0.0007) [2023-10-08 08:46:24,885][53852] Updated weights for policy 0, policy_version 23980 (0.0007) [2023-10-08 08:46:25,204][53885] Updated weights for policy 1, policy_version 23872 (0.0008) [2023-10-08 08:46:25,253][53852] Updated weights for policy 0, policy_version 23990 (0.0007) [2023-10-08 08:46:25,622][53852] Updated weights for policy 0, policy_version 24000 (0.0009) [2023-10-08 08:46:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 49020928. Throughput: 0: 1818.2, 1: 1821.7. Samples: 12257956. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 08:46:27,015][52710] Avg episode reward: [(0, '24.650'), (1, '28.810')] [2023-10-08 08:46:28,880][53885] Updated weights for policy 1, policy_version 23882 (0.0008) [2023-10-08 08:46:29,245][53885] Updated weights for policy 1, policy_version 23892 (0.0009) [2023-10-08 08:46:29,301][53852] Updated weights for policy 0, policy_version 24010 (0.0008) [2023-10-08 08:46:29,614][53885] Updated weights for policy 1, policy_version 23902 (0.0008) [2023-10-08 08:46:29,671][53852] Updated weights for policy 0, policy_version 24020 (0.0008) [2023-10-08 08:46:30,033][53852] Updated weights for policy 0, policy_version 24030 (0.0009) [2023-10-08 08:46:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 49086464. Throughput: 0: 1815.5, 1: 1813.9. Samples: 12278590. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:46:32,016][52710] Avg episode reward: [(0, '26.080'), (1, '29.110')] [2023-10-08 08:46:33,283][53885] Updated weights for policy 1, policy_version 23912 (0.0008) [2023-10-08 08:46:33,647][53885] Updated weights for policy 1, policy_version 23922 (0.0008) [2023-10-08 08:46:33,746][53852] Updated weights for policy 0, policy_version 24040 (0.0008) [2023-10-08 08:46:34,008][53885] Updated weights for policy 1, policy_version 23932 (0.0008) [2023-10-08 08:46:34,120][53852] Updated weights for policy 0, policy_version 24050 (0.0007) [2023-10-08 08:46:34,489][53852] Updated weights for policy 0, policy_version 24060 (0.0007) [2023-10-08 08:46:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49152000. Throughput: 0: 1817.9, 1: 1807.1. Samples: 12301252. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:46:37,016][52710] Avg episode reward: [(0, '25.350'), (1, '30.570')] [2023-10-08 08:46:37,723][53885] Updated weights for policy 1, policy_version 23942 (0.0008) [2023-10-08 08:46:38,052][53852] Updated weights for policy 0, policy_version 24070 (0.0009) [2023-10-08 08:46:38,088][53885] Updated weights for policy 1, policy_version 23952 (0.0007) [2023-10-08 08:46:38,423][53852] Updated weights for policy 0, policy_version 24080 (0.0010) [2023-10-08 08:46:38,455][53885] Updated weights for policy 1, policy_version 23962 (0.0009) [2023-10-08 08:46:38,790][53852] Updated weights for policy 0, policy_version 24090 (0.0009) [2023-10-08 08:46:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49217536. Throughput: 0: 1822.1, 1: 1804.4. Samples: 12311242. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:46:42,016][52710] Avg episode reward: [(0, '25.890'), (1, '29.880')] [2023-10-08 08:46:42,299][53885] Updated weights for policy 1, policy_version 23972 (0.0008) [2023-10-08 08:46:42,553][53852] Updated weights for policy 0, policy_version 24100 (0.0007) [2023-10-08 08:46:42,676][53885] Updated weights for policy 1, policy_version 23982 (0.0008) [2023-10-08 08:46:42,926][53852] Updated weights for policy 0, policy_version 24110 (0.0008) [2023-10-08 08:46:43,034][53885] Updated weights for policy 1, policy_version 23992 (0.0007) [2023-10-08 08:46:43,287][53852] Updated weights for policy 0, policy_version 24120 (0.0008) [2023-10-08 08:46:46,694][53885] Updated weights for policy 1, policy_version 24002 (0.0007) [2023-10-08 08:46:46,803][53852] Updated weights for policy 0, policy_version 24130 (0.0008) [2023-10-08 08:46:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 49283072. Throughput: 0: 1827.7, 1: 1797.3. Samples: 12334090. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) [2023-10-08 08:46:47,016][52710] Avg episode reward: [(0, '25.030'), (1, '28.620')] [2023-10-08 08:46:47,057][53885] Updated weights for policy 1, policy_version 24012 (0.0009) [2023-10-08 08:46:47,167][53852] Updated weights for policy 0, policy_version 24140 (0.0008) [2023-10-08 08:46:47,435][53885] Updated weights for policy 1, policy_version 24022 (0.0008) [2023-10-08 08:46:47,543][53852] Updated weights for policy 0, policy_version 24150 (0.0008) [2023-10-08 08:46:47,797][53885] Updated weights for policy 1, policy_version 24032 (0.0008) [2023-10-08 08:46:47,913][53852] Updated weights for policy 0, policy_version 24160 (0.0008) [2023-10-08 08:46:51,530][53885] Updated weights for policy 1, policy_version 24042 (0.0008) [2023-10-08 08:46:51,773][53852] Updated weights for policy 0, policy_version 24170 (0.0007) [2023-10-08 08:46:51,900][53885] Updated weights for policy 1, policy_version 24052 (0.0009) [2023-10-08 08:46:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 49348608. Throughput: 0: 1822.3, 1: 1808.8. Samples: 12356018. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 08:46:52,016][52710] Avg episode reward: [(0, '26.350'), (1, '27.250')] [2023-10-08 08:46:52,150][53852] Updated weights for policy 0, policy_version 24180 (0.0009) [2023-10-08 08:46:52,265][53885] Updated weights for policy 1, policy_version 24062 (0.0009) [2023-10-08 08:46:52,515][53852] Updated weights for policy 0, policy_version 24190 (0.0007) [2023-10-08 08:46:55,932][53885] Updated weights for policy 1, policy_version 24072 (0.0008) [2023-10-08 08:46:56,204][53852] Updated weights for policy 0, policy_version 24200 (0.0008) [2023-10-08 08:46:56,288][53885] Updated weights for policy 1, policy_version 24082 (0.0008) [2023-10-08 08:46:56,567][53852] Updated weights for policy 0, policy_version 24210 (0.0007) [2023-10-08 08:46:56,663][53885] Updated weights for policy 1, policy_version 24092 (0.0009) [2023-10-08 08:46:56,929][53852] Updated weights for policy 0, policy_version 24220 (0.0008) [2023-10-08 08:46:57,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 49446912. Throughput: 0: 1822.0, 1: 1802.2. Samples: 12366860. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 08:46:57,016][52710] Avg episode reward: [(0, '27.060'), (1, '29.920')] [2023-10-08 08:46:57,072][53500] Saving new best policy, reward=27.060! [2023-10-08 08:47:00,356][53885] Updated weights for policy 1, policy_version 24102 (0.0009) [2023-10-08 08:47:00,644][53852] Updated weights for policy 0, policy_version 24230 (0.0008) [2023-10-08 08:47:00,713][53885] Updated weights for policy 1, policy_version 24112 (0.0009) [2023-10-08 08:47:01,007][53852] Updated weights for policy 0, policy_version 24240 (0.0008) [2023-10-08 08:47:01,084][53885] Updated weights for policy 1, policy_version 24122 (0.0008) [2023-10-08 08:47:01,371][53852] Updated weights for policy 0, policy_version 24250 (0.0008) [2023-10-08 08:47:02,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 49545216. Throughput: 0: 1825.9, 1: 1805.2. Samples: 12388664. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 08:47:02,016][52710] Avg episode reward: [(0, '27.530'), (1, '28.950')] [2023-10-08 08:47:02,017][53500] Saving new best policy, reward=27.530! [2023-10-08 08:47:04,963][53885] Updated weights for policy 1, policy_version 24132 (0.0008) [2023-10-08 08:47:05,088][53852] Updated weights for policy 0, policy_version 24260 (0.0008) [2023-10-08 08:47:05,328][53885] Updated weights for policy 1, policy_version 24142 (0.0008) [2023-10-08 08:47:05,449][53852] Updated weights for policy 0, policy_version 24270 (0.0009) [2023-10-08 08:47:05,694][53885] Updated weights for policy 1, policy_version 24152 (0.0009) [2023-10-08 08:47:05,818][53852] Updated weights for policy 0, policy_version 24280 (0.0007) [2023-10-08 08:47:07,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 49610752. Throughput: 0: 1826.0, 1: 1802.2. Samples: 12409220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:47:07,016][52710] Avg episode reward: [(0, '24.570'), (1, '28.630')] [2023-10-08 08:47:09,380][53885] Updated weights for policy 1, policy_version 24162 (0.0007) [2023-10-08 08:47:09,531][53852] Updated weights for policy 0, policy_version 24290 (0.0008) [2023-10-08 08:47:09,749][53885] Updated weights for policy 1, policy_version 24172 (0.0009) [2023-10-08 08:47:09,898][53852] Updated weights for policy 0, policy_version 24300 (0.0009) [2023-10-08 08:47:10,123][53885] Updated weights for policy 1, policy_version 24182 (0.0008) [2023-10-08 08:47:10,267][53852] Updated weights for policy 0, policy_version 24310 (0.0008) [2023-10-08 08:47:10,487][53885] Updated weights for policy 1, policy_version 24192 (0.0008) [2023-10-08 08:47:10,638][53852] Updated weights for policy 0, policy_version 24320 (0.0009) [2023-10-08 08:47:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 49676288. Throughput: 0: 1824.4, 1: 1812.9. Samples: 12421636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:47:12,016][52710] Avg episode reward: [(0, '23.970'), (1, '29.080')] [2023-10-08 08:47:14,143][53885] Updated weights for policy 1, policy_version 24202 (0.0007) [2023-10-08 08:47:14,366][53852] Updated weights for policy 0, policy_version 24330 (0.0008) [2023-10-08 08:47:14,515][53885] Updated weights for policy 1, policy_version 24212 (0.0009) [2023-10-08 08:47:14,727][53852] Updated weights for policy 0, policy_version 24340 (0.0009) [2023-10-08 08:47:14,872][53885] Updated weights for policy 1, policy_version 24222 (0.0008) [2023-10-08 08:47:15,097][53852] Updated weights for policy 0, policy_version 24350 (0.0007) [2023-10-08 08:47:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 49741824. Throughput: 0: 1822.3, 1: 1802.4. Samples: 12441698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:47:17,016][52710] Avg episode reward: [(0, '23.580'), (1, '29.730')] [2023-10-08 08:47:18,507][53885] Updated weights for policy 1, policy_version 24232 (0.0007) [2023-10-08 08:47:18,754][53852] Updated weights for policy 0, policy_version 24360 (0.0008) [2023-10-08 08:47:18,876][53885] Updated weights for policy 1, policy_version 24242 (0.0007) [2023-10-08 08:47:19,119][53852] Updated weights for policy 0, policy_version 24370 (0.0009) [2023-10-08 08:47:19,236][53885] Updated weights for policy 1, policy_version 24252 (0.0007) [2023-10-08 08:47:19,495][53852] Updated weights for policy 0, policy_version 24380 (0.0008) [2023-10-08 08:47:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 49807360. Throughput: 0: 1828.6, 1: 1805.2. Samples: 12464772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:47:22,016][52710] Avg episode reward: [(0, '22.970'), (1, '29.390')] [2023-10-08 08:47:23,019][53885] Updated weights for policy 1, policy_version 24262 (0.0007) [2023-10-08 08:47:23,067][53852] Updated weights for policy 0, policy_version 24390 (0.0007) [2023-10-08 08:47:23,391][53885] Updated weights for policy 1, policy_version 24272 (0.0008) [2023-10-08 08:47:23,445][53852] Updated weights for policy 0, policy_version 24400 (0.0009) [2023-10-08 08:47:23,762][53885] Updated weights for policy 1, policy_version 24282 (0.0007) [2023-10-08 08:47:23,815][53852] Updated weights for policy 0, policy_version 24410 (0.0008) [2023-10-08 08:47:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49872896. Throughput: 0: 1825.3, 1: 1805.7. Samples: 12474636. Policy #0 lag: (min: 16.0, avg: 37.4, max: 48.0) [2023-10-08 08:47:27,016][52710] Avg episode reward: [(0, '24.370'), (1, '30.300')] [2023-10-08 08:47:27,425][53885] Updated weights for policy 1, policy_version 24292 (0.0007) [2023-10-08 08:47:27,592][53852] Updated weights for policy 0, policy_version 24420 (0.0008) [2023-10-08 08:47:27,789][53885] Updated weights for policy 1, policy_version 24302 (0.0008) [2023-10-08 08:47:27,955][53852] Updated weights for policy 0, policy_version 24430 (0.0007) [2023-10-08 08:47:28,164][53885] Updated weights for policy 1, policy_version 24312 (0.0007) [2023-10-08 08:47:28,328][53852] Updated weights for policy 0, policy_version 24440 (0.0007) [2023-10-08 08:47:31,817][53885] Updated weights for policy 1, policy_version 24322 (0.0008) [2023-10-08 08:47:32,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49938432. Throughput: 0: 1817.8, 1: 1814.5. Samples: 12497544. Policy #0 lag: (min: 16.0, avg: 37.4, max: 48.0) [2023-10-08 08:47:32,016][52710] Avg episode reward: [(0, '25.070'), (1, '29.090')] [2023-10-08 08:47:32,071][53852] Updated weights for policy 0, policy_version 24450 (0.0009) [2023-10-08 08:47:32,198][53885] Updated weights for policy 1, policy_version 24332 (0.0008) [2023-10-08 08:47:32,440][53852] Updated weights for policy 0, policy_version 24460 (0.0007) [2023-10-08 08:47:32,561][53885] Updated weights for policy 1, policy_version 24342 (0.0008) [2023-10-08 08:47:32,806][53852] Updated weights for policy 0, policy_version 24470 (0.0007) [2023-10-08 08:47:32,930][53885] Updated weights for policy 1, policy_version 24352 (0.0008) [2023-10-08 08:47:33,179][53852] Updated weights for policy 0, policy_version 24480 (0.0008) [2023-10-08 08:47:36,669][53885] Updated weights for policy 1, policy_version 24362 (0.0009) [2023-10-08 08:47:36,834][53852] Updated weights for policy 0, policy_version 24490 (0.0007) [2023-10-08 08:47:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50003968. Throughput: 0: 1823.5, 1: 1818.0. Samples: 12519886. Policy #0 lag: (min: 16.0, avg: 37.4, max: 48.0) [2023-10-08 08:47:37,016][52710] Avg episode reward: [(0, '25.680'), (1, '29.590')] [2023-10-08 08:47:37,036][53885] Updated weights for policy 1, policy_version 24372 (0.0009) [2023-10-08 08:47:37,199][53852] Updated weights for policy 0, policy_version 24500 (0.0008) [2023-10-08 08:47:37,399][53885] Updated weights for policy 1, policy_version 24382 (0.0008) [2023-10-08 08:47:37,578][53852] Updated weights for policy 0, policy_version 24510 (0.0008) [2023-10-08 08:47:40,887][53885] Updated weights for policy 1, policy_version 24392 (0.0008) [2023-10-08 08:47:41,245][53852] Updated weights for policy 0, policy_version 24520 (0.0007) [2023-10-08 08:47:41,256][53885] Updated weights for policy 1, policy_version 24402 (0.0008) [2023-10-08 08:47:41,616][53885] Updated weights for policy 1, policy_version 24412 (0.0007) [2023-10-08 08:47:41,618][53852] Updated weights for policy 0, policy_version 24530 (0.0008) [2023-10-08 08:47:41,979][53852] Updated weights for policy 0, policy_version 24540 (0.0007) [2023-10-08 08:47:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 50102272. Throughput: 0: 1825.8, 1: 1812.7. Samples: 12530592. Policy #0 lag: (min: 16.0, avg: 37.4, max: 48.0) [2023-10-08 08:47:42,016][52710] Avg episode reward: [(0, '27.770'), (1, '28.490')] [2023-10-08 08:47:42,124][53500] Saving new best policy, reward=27.770! [2023-10-08 08:47:45,548][53885] Updated weights for policy 1, policy_version 24422 (0.0007) [2023-10-08 08:47:45,608][53852] Updated weights for policy 0, policy_version 24550 (0.0010) [2023-10-08 08:47:45,940][53885] Updated weights for policy 1, policy_version 24432 (0.0007) [2023-10-08 08:47:45,982][53852] Updated weights for policy 0, policy_version 24560 (0.0007) [2023-10-08 08:47:46,300][53885] Updated weights for policy 1, policy_version 24442 (0.0007) [2023-10-08 08:47:46,348][53852] Updated weights for policy 0, policy_version 24570 (0.0007) [2023-10-08 08:47:47,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 50200576. Throughput: 0: 1827.9, 1: 1818.4. Samples: 12552750. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 08:47:47,016][52710] Avg episode reward: [(0, '26.650'), (1, '31.190')] [2023-10-08 08:47:49,937][53852] Updated weights for policy 0, policy_version 24580 (0.0007) [2023-10-08 08:47:49,996][53885] Updated weights for policy 1, policy_version 24452 (0.0007) [2023-10-08 08:47:50,330][53852] Updated weights for policy 0, policy_version 24590 (0.0007) [2023-10-08 08:47:50,366][53885] Updated weights for policy 1, policy_version 24462 (0.0009) [2023-10-08 08:47:50,698][53852] Updated weights for policy 0, policy_version 24600 (0.0009) [2023-10-08 08:47:50,722][53885] Updated weights for policy 1, policy_version 24472 (0.0008) [2023-10-08 08:47:52,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 50266112. Throughput: 0: 1822.8, 1: 1813.7. Samples: 12572860. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 08:47:52,016][52710] Avg episode reward: [(0, '25.090'), (1, '26.940')] [2023-10-08 08:47:54,426][53852] Updated weights for policy 0, policy_version 24610 (0.0008) [2023-10-08 08:47:54,455][53885] Updated weights for policy 1, policy_version 24482 (0.0008) [2023-10-08 08:47:54,801][53852] Updated weights for policy 0, policy_version 24620 (0.0007) [2023-10-08 08:47:54,829][53885] Updated weights for policy 1, policy_version 24492 (0.0007) [2023-10-08 08:47:55,178][53852] Updated weights for policy 0, policy_version 24630 (0.0008) [2023-10-08 08:47:55,199][53885] Updated weights for policy 1, policy_version 24502 (0.0008) [2023-10-08 08:47:55,543][53852] Updated weights for policy 0, policy_version 24640 (0.0009) [2023-10-08 08:47:55,564][53885] Updated weights for policy 1, policy_version 24512 (0.0007) [2023-10-08 08:47:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 50331648. Throughput: 0: 1824.2, 1: 1814.9. Samples: 12585394. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 08:47:57,015][52710] Avg episode reward: [(0, '24.970'), (1, '29.480')] [2023-10-08 08:47:59,295][53885] Updated weights for policy 1, policy_version 24522 (0.0008) [2023-10-08 08:47:59,335][53852] Updated weights for policy 0, policy_version 24650 (0.0008) [2023-10-08 08:47:59,658][53885] Updated weights for policy 1, policy_version 24532 (0.0007) [2023-10-08 08:47:59,708][53852] Updated weights for policy 0, policy_version 24660 (0.0007) [2023-10-08 08:48:00,032][53885] Updated weights for policy 1, policy_version 24542 (0.0007) [2023-10-08 08:48:00,070][53852] Updated weights for policy 0, policy_version 24670 (0.0007) [2023-10-08 08:48:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 50397184. Throughput: 0: 1825.0, 1: 1817.3. Samples: 12605602. Policy #0 lag: (min: 4.0, avg: 12.2, max: 36.0) [2023-10-08 08:48:02,016][52710] Avg episode reward: [(0, '23.680'), (1, '28.610')] [2023-10-08 08:48:03,648][53852] Updated weights for policy 0, policy_version 24680 (0.0008) [2023-10-08 08:48:03,771][53885] Updated weights for policy 1, policy_version 24552 (0.0009) [2023-10-08 08:48:04,025][53852] Updated weights for policy 0, policy_version 24690 (0.0008) [2023-10-08 08:48:04,129][53885] Updated weights for policy 1, policy_version 24562 (0.0007) [2023-10-08 08:48:04,400][53852] Updated weights for policy 0, policy_version 24700 (0.0008) [2023-10-08 08:48:04,504][53885] Updated weights for policy 1, policy_version 24572 (0.0009) [2023-10-08 08:48:07,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 50462720. Throughput: 0: 1822.2, 1: 1813.1. Samples: 12628360. Policy #0 lag: (min: 4.0, avg: 12.2, max: 36.0) [2023-10-08 08:48:07,017][52710] Avg episode reward: [(0, '22.840'), (1, '26.940')] [2023-10-08 08:48:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000024704_25296896.pth... [2023-10-08 08:48:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000024576_25165824.pth... [2023-10-08 08:48:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000022880_23429120.pth [2023-10-08 08:48:07,071][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000023008_23560192.pth [2023-10-08 08:48:07,972][53852] Updated weights for policy 0, policy_version 24710 (0.0008) [2023-10-08 08:48:08,115][53885] Updated weights for policy 1, policy_version 24582 (0.0009) [2023-10-08 08:48:08,330][53852] Updated weights for policy 0, policy_version 24720 (0.0009) [2023-10-08 08:48:08,477][53885] Updated weights for policy 1, policy_version 24592 (0.0007) [2023-10-08 08:48:08,700][53852] Updated weights for policy 0, policy_version 24730 (0.0008) [2023-10-08 08:48:08,843][53885] Updated weights for policy 1, policy_version 24602 (0.0008) [2023-10-08 08:48:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50528256. Throughput: 0: 1825.8, 1: 1819.1. Samples: 12638656. Policy #0 lag: (min: 4.0, avg: 12.2, max: 36.0) [2023-10-08 08:48:12,016][52710] Avg episode reward: [(0, '23.850'), (1, '28.350')] [2023-10-08 08:48:12,435][53852] Updated weights for policy 0, policy_version 24740 (0.0009) [2023-10-08 08:48:12,447][53885] Updated weights for policy 1, policy_version 24612 (0.0010) [2023-10-08 08:48:12,805][53852] Updated weights for policy 0, policy_version 24750 (0.0007) [2023-10-08 08:48:12,820][53885] Updated weights for policy 1, policy_version 24622 (0.0008) [2023-10-08 08:48:13,173][53852] Updated weights for policy 0, policy_version 24760 (0.0007) [2023-10-08 08:48:13,183][53885] Updated weights for policy 1, policy_version 24632 (0.0009) [2023-10-08 08:48:16,864][53852] Updated weights for policy 0, policy_version 24770 (0.0008) [2023-10-08 08:48:16,986][53885] Updated weights for policy 1, policy_version 24642 (0.0007) [2023-10-08 08:48:17,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50593792. Throughput: 0: 1826.5, 1: 1816.2. Samples: 12661468. Policy #0 lag: (min: 4.0, avg: 12.2, max: 36.0) [2023-10-08 08:48:17,015][52710] Avg episode reward: [(0, '25.360'), (1, '26.810')] [2023-10-08 08:48:17,240][53852] Updated weights for policy 0, policy_version 24780 (0.0008) [2023-10-08 08:48:17,356][53885] Updated weights for policy 1, policy_version 24652 (0.0009) [2023-10-08 08:48:17,604][53852] Updated weights for policy 0, policy_version 24790 (0.0007) [2023-10-08 08:48:17,720][53885] Updated weights for policy 1, policy_version 24662 (0.0007) [2023-10-08 08:48:17,974][53852] Updated weights for policy 0, policy_version 24800 (0.0007) [2023-10-08 08:48:18,084][53885] Updated weights for policy 1, policy_version 24672 (0.0009) [2023-10-08 08:48:21,547][53852] Updated weights for policy 0, policy_version 24810 (0.0007) [2023-10-08 08:48:21,827][53885] Updated weights for policy 1, policy_version 24682 (0.0008) [2023-10-08 08:48:21,917][53852] Updated weights for policy 0, policy_version 24820 (0.0007) [2023-10-08 08:48:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50659328. Throughput: 0: 1821.3, 1: 1827.4. Samples: 12684076. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 08:48:22,016][52710] Avg episode reward: [(0, '25.660'), (1, '28.750')] [2023-10-08 08:48:22,193][53885] Updated weights for policy 1, policy_version 24692 (0.0008) [2023-10-08 08:48:22,292][53852] Updated weights for policy 0, policy_version 24830 (0.0007) [2023-10-08 08:48:22,559][53885] Updated weights for policy 1, policy_version 24702 (0.0007) [2023-10-08 08:48:25,903][53852] Updated weights for policy 0, policy_version 24840 (0.0007) [2023-10-08 08:48:26,273][53852] Updated weights for policy 0, policy_version 24850 (0.0008) [2023-10-08 08:48:26,352][53885] Updated weights for policy 1, policy_version 24712 (0.0008) [2023-10-08 08:48:26,646][53852] Updated weights for policy 0, policy_version 24860 (0.0008) [2023-10-08 08:48:26,718][53885] Updated weights for policy 1, policy_version 24722 (0.0008) [2023-10-08 08:48:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50757632. Throughput: 0: 1829.5, 1: 1818.9. Samples: 12694770. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 08:48:27,016][52710] Avg episode reward: [(0, '25.660'), (1, '28.440')] [2023-10-08 08:48:27,082][53885] Updated weights for policy 1, policy_version 24732 (0.0007) [2023-10-08 08:48:30,270][53852] Updated weights for policy 0, policy_version 24870 (0.0010) [2023-10-08 08:48:30,637][53852] Updated weights for policy 0, policy_version 24880 (0.0009) [2023-10-08 08:48:30,753][53885] Updated weights for policy 1, policy_version 24742 (0.0009) [2023-10-08 08:48:31,013][53852] Updated weights for policy 0, policy_version 24890 (0.0008) [2023-10-08 08:48:31,139][53885] Updated weights for policy 1, policy_version 24752 (0.0008) [2023-10-08 08:48:31,507][53885] Updated weights for policy 1, policy_version 24762 (0.0010) [2023-10-08 08:48:32,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 50855936. Throughput: 0: 1820.9, 1: 1827.5. Samples: 12716926. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 08:48:32,016][52710] Avg episode reward: [(0, '25.110'), (1, '26.220')] [2023-10-08 08:48:34,728][53852] Updated weights for policy 0, policy_version 24900 (0.0008) [2023-10-08 08:48:35,109][53852] Updated weights for policy 0, policy_version 24910 (0.0007) [2023-10-08 08:48:35,236][53885] Updated weights for policy 1, policy_version 24772 (0.0009) [2023-10-08 08:48:35,480][53852] Updated weights for policy 0, policy_version 24920 (0.0008) [2023-10-08 08:48:35,606][53885] Updated weights for policy 1, policy_version 24782 (0.0008) [2023-10-08 08:48:35,971][53885] Updated weights for policy 1, policy_version 24792 (0.0012) [2023-10-08 08:48:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 50921472. Throughput: 0: 1839.6, 1: 1824.2. Samples: 12737728. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:48:37,016][52710] Avg episode reward: [(0, '26.900'), (1, '28.380')] [2023-10-08 08:48:38,895][53852] Updated weights for policy 0, policy_version 24930 (0.0008) [2023-10-08 08:48:39,267][53852] Updated weights for policy 0, policy_version 24940 (0.0010) [2023-10-08 08:48:39,502][53885] Updated weights for policy 1, policy_version 24802 (0.0008) [2023-10-08 08:48:39,643][53852] Updated weights for policy 0, policy_version 24950 (0.0008) [2023-10-08 08:48:39,871][53885] Updated weights for policy 1, policy_version 24812 (0.0007) [2023-10-08 08:48:40,017][53852] Updated weights for policy 0, policy_version 24960 (0.0008) [2023-10-08 08:48:40,242][53885] Updated weights for policy 1, policy_version 24822 (0.0007) [2023-10-08 08:48:40,604][53885] Updated weights for policy 1, policy_version 24832 (0.0008) [2023-10-08 08:48:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 50987008. Throughput: 0: 1826.3, 1: 1826.7. Samples: 12749778. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:48:42,016][52710] Avg episode reward: [(0, '25.270'), (1, '29.390')] [2023-10-08 08:48:43,716][53852] Updated weights for policy 0, policy_version 24970 (0.0010) [2023-10-08 08:48:44,082][53852] Updated weights for policy 0, policy_version 24980 (0.0008) [2023-10-08 08:48:44,084][53885] Updated weights for policy 1, policy_version 24842 (0.0007) [2023-10-08 08:48:44,451][53852] Updated weights for policy 0, policy_version 24990 (0.0009) [2023-10-08 08:48:44,451][53885] Updated weights for policy 1, policy_version 24852 (0.0010) [2023-10-08 08:48:44,807][53885] Updated weights for policy 1, policy_version 24862 (0.0008) [2023-10-08 08:48:47,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 51052544. Throughput: 0: 1850.9, 1: 1824.2. Samples: 12770984. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:48:47,015][52710] Avg episode reward: [(0, '24.700'), (1, '27.920')] [2023-10-08 08:48:48,057][53852] Updated weights for policy 0, policy_version 25000 (0.0008) [2023-10-08 08:48:48,418][53852] Updated weights for policy 0, policy_version 25010 (0.0011) [2023-10-08 08:48:48,571][53885] Updated weights for policy 1, policy_version 24872 (0.0007) [2023-10-08 08:48:48,796][53852] Updated weights for policy 0, policy_version 25020 (0.0009) [2023-10-08 08:48:48,940][53885] Updated weights for policy 1, policy_version 24882 (0.0008) [2023-10-08 08:48:49,316][53885] Updated weights for policy 1, policy_version 24892 (0.0008) [2023-10-08 08:48:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51118080. Throughput: 0: 1849.9, 1: 1830.5. Samples: 12793978. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 08:48:52,016][52710] Avg episode reward: [(0, '26.760'), (1, '29.460')] [2023-10-08 08:48:52,546][53852] Updated weights for policy 0, policy_version 25030 (0.0008) [2023-10-08 08:48:52,922][53852] Updated weights for policy 0, policy_version 25040 (0.0007) [2023-10-08 08:48:52,986][53885] Updated weights for policy 1, policy_version 24902 (0.0007) [2023-10-08 08:48:53,276][53852] Updated weights for policy 0, policy_version 25050 (0.0008) [2023-10-08 08:48:53,345][53885] Updated weights for policy 1, policy_version 24912 (0.0007) [2023-10-08 08:48:53,719][53885] Updated weights for policy 1, policy_version 24922 (0.0008) [2023-10-08 08:48:56,913][53852] Updated weights for policy 0, policy_version 25060 (0.0008) [2023-10-08 08:48:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 51183616. Throughput: 0: 1850.0, 1: 1827.6. Samples: 12804146. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-08 08:48:57,016][52710] Avg episode reward: [(0, '25.610'), (1, '28.650')] [2023-10-08 08:48:57,290][53852] Updated weights for policy 0, policy_version 25070 (0.0008) [2023-10-08 08:48:57,467][53885] Updated weights for policy 1, policy_version 24932 (0.0009) [2023-10-08 08:48:57,660][53852] Updated weights for policy 0, policy_version 25080 (0.0008) [2023-10-08 08:48:57,836][53885] Updated weights for policy 1, policy_version 24942 (0.0007) [2023-10-08 08:48:58,192][53885] Updated weights for policy 1, policy_version 24952 (0.0008) [2023-10-08 08:49:01,320][53852] Updated weights for policy 0, policy_version 25090 (0.0007) [2023-10-08 08:49:01,683][53852] Updated weights for policy 0, policy_version 25100 (0.0011) [2023-10-08 08:49:01,823][53885] Updated weights for policy 1, policy_version 24962 (0.0007) [2023-10-08 08:49:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 51249152. Throughput: 0: 1850.4, 1: 1831.3. Samples: 12827146. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-08 08:49:02,015][52710] Avg episode reward: [(0, '27.040'), (1, '28.460')] [2023-10-08 08:49:02,052][53852] Updated weights for policy 0, policy_version 25110 (0.0009) [2023-10-08 08:49:02,189][53885] Updated weights for policy 1, policy_version 24972 (0.0007) [2023-10-08 08:49:02,419][53852] Updated weights for policy 0, policy_version 25120 (0.0008) [2023-10-08 08:49:02,554][53885] Updated weights for policy 1, policy_version 24982 (0.0007) [2023-10-08 08:49:02,915][53885] Updated weights for policy 1, policy_version 24992 (0.0008) [2023-10-08 08:49:06,000][53852] Updated weights for policy 0, policy_version 25130 (0.0009) [2023-10-08 08:49:06,374][53852] Updated weights for policy 0, policy_version 25140 (0.0007) [2023-10-08 08:49:06,568][53885] Updated weights for policy 1, policy_version 25002 (0.0008) [2023-10-08 08:49:06,745][53852] Updated weights for policy 0, policy_version 25150 (0.0008) [2023-10-08 08:49:06,934][53885] Updated weights for policy 1, policy_version 25012 (0.0009) [2023-10-08 08:49:07,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 51347456. Throughput: 0: 1834.4, 1: 1824.7. Samples: 12848734. Policy #0 lag: (min: 1.0, avg: 9.0, max: 33.0) [2023-10-08 08:49:07,015][52710] Avg episode reward: [(0, '27.190'), (1, '28.600')] [2023-10-08 08:49:07,307][53885] Updated weights for policy 1, policy_version 25022 (0.0010) [2023-10-08 08:49:10,298][53852] Updated weights for policy 0, policy_version 25160 (0.0008) [2023-10-08 08:49:10,672][53852] Updated weights for policy 0, policy_version 25170 (0.0008) [2023-10-08 08:49:10,995][53885] Updated weights for policy 1, policy_version 25032 (0.0009) [2023-10-08 08:49:11,035][53852] Updated weights for policy 0, policy_version 25180 (0.0009) [2023-10-08 08:49:11,363][53885] Updated weights for policy 1, policy_version 25042 (0.0008) [2023-10-08 08:49:11,731][53885] Updated weights for policy 1, policy_version 25052 (0.0008) [2023-10-08 08:49:12,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 51445760. Throughput: 0: 1848.5, 1: 1830.5. Samples: 12860328. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 08:49:12,015][52710] Avg episode reward: [(0, '24.750'), (1, '29.880')] [2023-10-08 08:49:14,657][53852] Updated weights for policy 0, policy_version 25190 (0.0007) [2023-10-08 08:49:15,025][53852] Updated weights for policy 0, policy_version 25200 (0.0008) [2023-10-08 08:49:15,396][53852] Updated weights for policy 0, policy_version 25210 (0.0010) [2023-10-08 08:49:15,584][53885] Updated weights for policy 1, policy_version 25062 (0.0010) [2023-10-08 08:49:15,969][53885] Updated weights for policy 1, policy_version 25072 (0.0007) [2023-10-08 08:49:16,339][53885] Updated weights for policy 1, policy_version 25082 (0.0007) [2023-10-08 08:49:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 51511296. Throughput: 0: 1834.8, 1: 1820.8. Samples: 12881424. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 08:49:17,015][52710] Avg episode reward: [(0, '27.860'), (1, '28.800')] [2023-10-08 08:49:17,016][53500] Saving new best policy, reward=27.860! [2023-10-08 08:49:19,118][53852] Updated weights for policy 0, policy_version 25220 (0.0009) [2023-10-08 08:49:19,498][53852] Updated weights for policy 0, policy_version 25230 (0.0010) [2023-10-08 08:49:19,875][53852] Updated weights for policy 0, policy_version 25240 (0.0009) [2023-10-08 08:49:19,904][53885] Updated weights for policy 1, policy_version 25092 (0.0008) [2023-10-08 08:49:20,268][53885] Updated weights for policy 1, policy_version 25102 (0.0008) [2023-10-08 08:49:20,638][53885] Updated weights for policy 1, policy_version 25112 (0.0009) [2023-10-08 08:49:22,015][52710] Fps is (10 sec: 13106.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 51576832. Throughput: 0: 1847.0, 1: 1832.4. Samples: 12903300. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 08:49:22,016][52710] Avg episode reward: [(0, '25.100'), (1, '28.670')] [2023-10-08 08:49:23,549][53852] Updated weights for policy 0, policy_version 25250 (0.0011) [2023-10-08 08:49:23,934][53852] Updated weights for policy 0, policy_version 25260 (0.0011) [2023-10-08 08:49:24,291][53852] Updated weights for policy 0, policy_version 25270 (0.0009) [2023-10-08 08:49:24,391][53885] Updated weights for policy 1, policy_version 25122 (0.0009) [2023-10-08 08:49:24,662][53852] Updated weights for policy 0, policy_version 25280 (0.0008) [2023-10-08 08:49:24,763][53885] Updated weights for policy 1, policy_version 25132 (0.0009) [2023-10-08 08:49:25,128][53885] Updated weights for policy 1, policy_version 25142 (0.0008) [2023-10-08 08:49:25,496][53885] Updated weights for policy 1, policy_version 25152 (0.0010) [2023-10-08 08:49:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 51642368. Throughput: 0: 1830.3, 1: 1824.3. Samples: 12914232. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 08:49:27,016][52710] Avg episode reward: [(0, '26.420'), (1, '29.180')] [2023-10-08 08:49:28,228][53852] Updated weights for policy 0, policy_version 25290 (0.0011) [2023-10-08 08:49:28,593][53852] Updated weights for policy 0, policy_version 25300 (0.0010) [2023-10-08 08:49:28,955][53852] Updated weights for policy 0, policy_version 25310 (0.0008) [2023-10-08 08:49:29,175][53885] Updated weights for policy 1, policy_version 25162 (0.0009) [2023-10-08 08:49:29,550][53885] Updated weights for policy 1, policy_version 25172 (0.0008) [2023-10-08 08:49:29,923][53885] Updated weights for policy 1, policy_version 25182 (0.0008) [2023-10-08 08:49:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51707904. Throughput: 0: 1837.1, 1: 1820.1. Samples: 12935558. Policy #0 lag: (min: 3.0, avg: 3.9, max: 24.0) [2023-10-08 08:49:32,016][52710] Avg episode reward: [(0, '25.810'), (1, '30.250')] [2023-10-08 08:49:32,696][53852] Updated weights for policy 0, policy_version 25320 (0.0008) [2023-10-08 08:49:33,064][53852] Updated weights for policy 0, policy_version 25330 (0.0007) [2023-10-08 08:49:33,433][53852] Updated weights for policy 0, policy_version 25340 (0.0008) [2023-10-08 08:49:33,678][53885] Updated weights for policy 1, policy_version 25192 (0.0009) [2023-10-08 08:49:34,043][53885] Updated weights for policy 1, policy_version 25202 (0.0007) [2023-10-08 08:49:34,413][53885] Updated weights for policy 1, policy_version 25212 (0.0007) [2023-10-08 08:49:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51773440. Throughput: 0: 1839.0, 1: 1818.4. Samples: 12958560. Policy #0 lag: (min: 3.0, avg: 3.9, max: 24.0) [2023-10-08 08:49:37,016][52710] Avg episode reward: [(0, '26.120'), (1, '28.600')] [2023-10-08 08:49:37,142][53852] Updated weights for policy 0, policy_version 25350 (0.0009) [2023-10-08 08:49:37,512][53852] Updated weights for policy 0, policy_version 25360 (0.0009) [2023-10-08 08:49:37,877][53852] Updated weights for policy 0, policy_version 25370 (0.0010) [2023-10-08 08:49:37,940][53885] Updated weights for policy 1, policy_version 25222 (0.0008) [2023-10-08 08:49:38,301][53885] Updated weights for policy 1, policy_version 25232 (0.0008) [2023-10-08 08:49:38,678][53885] Updated weights for policy 1, policy_version 25242 (0.0007) [2023-10-08 08:49:41,649][53852] Updated weights for policy 0, policy_version 25380 (0.0008) [2023-10-08 08:49:42,011][53852] Updated weights for policy 0, policy_version 25390 (0.0008) [2023-10-08 08:49:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 51838976. Throughput: 0: 1833.0, 1: 1819.8. Samples: 12968522. Policy #0 lag: (min: 3.0, avg: 3.9, max: 24.0) [2023-10-08 08:49:42,015][52710] Avg episode reward: [(0, '26.490'), (1, '30.160')] [2023-10-08 08:49:42,232][53885] Updated weights for policy 1, policy_version 25252 (0.0008) [2023-10-08 08:49:42,388][53852] Updated weights for policy 0, policy_version 25400 (0.0008) [2023-10-08 08:49:42,593][53885] Updated weights for policy 1, policy_version 25262 (0.0008) [2023-10-08 08:49:42,965][53885] Updated weights for policy 1, policy_version 25272 (0.0009) [2023-10-08 08:49:45,962][53852] Updated weights for policy 0, policy_version 25410 (0.0007) [2023-10-08 08:49:46,334][53852] Updated weights for policy 0, policy_version 25420 (0.0008) [2023-10-08 08:49:46,685][53885] Updated weights for policy 1, policy_version 25282 (0.0009) [2023-10-08 08:49:46,707][53852] Updated weights for policy 0, policy_version 25430 (0.0007) [2023-10-08 08:49:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 51904512. Throughput: 0: 1836.9, 1: 1822.7. Samples: 12991832. Policy #0 lag: (min: 3.0, avg: 3.9, max: 24.0) [2023-10-08 08:49:47,016][52710] Avg episode reward: [(0, '28.290'), (1, '31.100')] [2023-10-08 08:49:47,048][53885] Updated weights for policy 1, policy_version 25292 (0.0009) [2023-10-08 08:49:47,078][53500] Saving new best policy, reward=28.290! [2023-10-08 08:49:47,080][53852] Updated weights for policy 0, policy_version 25440 (0.0007) [2023-10-08 08:49:47,419][53885] Updated weights for policy 1, policy_version 25302 (0.0008) [2023-10-08 08:49:47,785][53885] Updated weights for policy 1, policy_version 25312 (0.0009) [2023-10-08 08:49:50,795][53852] Updated weights for policy 0, policy_version 25450 (0.0008) [2023-10-08 08:49:51,159][53852] Updated weights for policy 0, policy_version 25460 (0.0008) [2023-10-08 08:49:51,528][53885] Updated weights for policy 1, policy_version 25322 (0.0008) [2023-10-08 08:49:51,537][53852] Updated weights for policy 0, policy_version 25470 (0.0008) [2023-10-08 08:49:51,889][53885] Updated weights for policy 1, policy_version 25332 (0.0007) [2023-10-08 08:49:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52002816. Throughput: 0: 1828.1, 1: 1818.0. Samples: 13012810. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:49:52,016][52710] Avg episode reward: [(0, '26.010'), (1, '32.330')] [2023-10-08 08:49:52,267][53885] Updated weights for policy 1, policy_version 25342 (0.0007) [2023-10-08 08:49:52,337][53594] Saving new best policy, reward=32.330! [2023-10-08 08:49:55,143][53852] Updated weights for policy 0, policy_version 25480 (0.0009) [2023-10-08 08:49:55,509][53852] Updated weights for policy 0, policy_version 25490 (0.0008) [2023-10-08 08:49:55,878][53852] Updated weights for policy 0, policy_version 25500 (0.0008) [2023-10-08 08:49:56,006][53885] Updated weights for policy 1, policy_version 25352 (0.0008) [2023-10-08 08:49:56,369][53885] Updated weights for policy 1, policy_version 25362 (0.0011) [2023-10-08 08:49:56,734][53885] Updated weights for policy 1, policy_version 25372 (0.0008) [2023-10-08 08:49:57,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 52101120. Throughput: 0: 1831.0, 1: 1819.9. Samples: 13024618. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:49:57,016][52710] Avg episode reward: [(0, '26.390'), (1, '29.760')] [2023-10-08 08:49:59,454][53852] Updated weights for policy 0, policy_version 25510 (0.0011) [2023-10-08 08:49:59,821][53852] Updated weights for policy 0, policy_version 25520 (0.0010) [2023-10-08 08:50:00,192][53852] Updated weights for policy 0, policy_version 25530 (0.0009) [2023-10-08 08:50:00,502][53885] Updated weights for policy 1, policy_version 25382 (0.0008) [2023-10-08 08:50:00,889][53885] Updated weights for policy 1, policy_version 25392 (0.0009) [2023-10-08 08:50:01,269][53885] Updated weights for policy 1, policy_version 25402 (0.0011) [2023-10-08 08:50:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 52166656. Throughput: 0: 1827.5, 1: 1820.8. Samples: 13045602. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:50:02,016][52710] Avg episode reward: [(0, '27.130'), (1, '29.800')] [2023-10-08 08:50:03,963][53852] Updated weights for policy 0, policy_version 25540 (0.0007) [2023-10-08 08:50:04,329][53852] Updated weights for policy 0, policy_version 25550 (0.0008) [2023-10-08 08:50:04,705][53852] Updated weights for policy 0, policy_version 25560 (0.0009) [2023-10-08 08:50:04,901][53885] Updated weights for policy 1, policy_version 25412 (0.0008) [2023-10-08 08:50:05,263][53885] Updated weights for policy 1, policy_version 25422 (0.0007) [2023-10-08 08:50:05,631][53885] Updated weights for policy 1, policy_version 25432 (0.0009) [2023-10-08 08:50:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52232192. Throughput: 0: 1825.7, 1: 1814.3. Samples: 13067102. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:50:07,016][52710] Avg episode reward: [(0, '24.860'), (1, '29.480')] [2023-10-08 08:50:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000025568_26181632.pth... [2023-10-08 08:50:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000025440_26050560.pth... [2023-10-08 08:50:07,057][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000023872_24444928.pth [2023-10-08 08:50:07,061][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000025568_26181632.pth [2023-10-08 08:50:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000023744_24313856.pth [2023-10-08 08:50:07,070][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000025440_26050560.pth [2023-10-08 08:50:08,424][53852] Updated weights for policy 0, policy_version 25570 (0.0009) [2023-10-08 08:50:08,808][53852] Updated weights for policy 0, policy_version 25580 (0.0008) [2023-10-08 08:50:09,186][53852] Updated weights for policy 0, policy_version 25590 (0.0007) [2023-10-08 08:50:09,361][53885] Updated weights for policy 1, policy_version 25442 (0.0008) [2023-10-08 08:50:09,546][53852] Updated weights for policy 0, policy_version 25600 (0.0007) [2023-10-08 08:50:09,727][53885] Updated weights for policy 1, policy_version 25452 (0.0007) [2023-10-08 08:50:10,091][53885] Updated weights for policy 1, policy_version 25462 (0.0008) [2023-10-08 08:50:10,465][53885] Updated weights for policy 1, policy_version 25472 (0.0009) [2023-10-08 08:50:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 52297728. Throughput: 0: 1826.2, 1: 1817.9. Samples: 13078216. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 08:50:12,016][52710] Avg episode reward: [(0, '24.650'), (1, '28.620')] [2023-10-08 08:50:13,071][53852] Updated weights for policy 0, policy_version 25610 (0.0008) [2023-10-08 08:50:13,429][53852] Updated weights for policy 0, policy_version 25620 (0.0008) [2023-10-08 08:50:13,802][53852] Updated weights for policy 0, policy_version 25630 (0.0008) [2023-10-08 08:50:14,095][53885] Updated weights for policy 1, policy_version 25482 (0.0010) [2023-10-08 08:50:14,462][53885] Updated weights for policy 1, policy_version 25492 (0.0010) [2023-10-08 08:50:14,835][53885] Updated weights for policy 1, policy_version 25502 (0.0009) [2023-10-08 08:50:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 52363264. Throughput: 0: 1828.4, 1: 1822.4. Samples: 13099844. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 08:50:17,016][52710] Avg episode reward: [(0, '26.170'), (1, '27.940')] [2023-10-08 08:50:17,550][53852] Updated weights for policy 0, policy_version 25640 (0.0010) [2023-10-08 08:50:17,907][53852] Updated weights for policy 0, policy_version 25650 (0.0008) [2023-10-08 08:50:18,282][53852] Updated weights for policy 0, policy_version 25660 (0.0008) [2023-10-08 08:50:18,557][53885] Updated weights for policy 1, policy_version 25512 (0.0007) [2023-10-08 08:50:18,920][53885] Updated weights for policy 1, policy_version 25522 (0.0008) [2023-10-08 08:50:19,295][53885] Updated weights for policy 1, policy_version 25532 (0.0008) [2023-10-08 08:50:21,817][53852] Updated weights for policy 0, policy_version 25670 (0.0009) [2023-10-08 08:50:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52428800. Throughput: 0: 1832.7, 1: 1820.3. Samples: 13122944. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 08:50:22,016][52710] Avg episode reward: [(0, '26.710'), (1, '32.580')] [2023-10-08 08:50:22,027][53594] Saving new best policy, reward=32.580! [2023-10-08 08:50:22,181][53852] Updated weights for policy 0, policy_version 25680 (0.0008) [2023-10-08 08:50:22,550][53852] Updated weights for policy 0, policy_version 25690 (0.0007) [2023-10-08 08:50:22,896][53885] Updated weights for policy 1, policy_version 25542 (0.0008) [2023-10-08 08:50:23,263][53885] Updated weights for policy 1, policy_version 25552 (0.0007) [2023-10-08 08:50:23,628][53885] Updated weights for policy 1, policy_version 25562 (0.0008) [2023-10-08 08:50:26,245][53852] Updated weights for policy 0, policy_version 25700 (0.0009) [2023-10-08 08:50:26,609][53852] Updated weights for policy 0, policy_version 25710 (0.0009) [2023-10-08 08:50:26,976][53852] Updated weights for policy 0, policy_version 25720 (0.0008) [2023-10-08 08:50:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 52494336. Throughput: 0: 1838.3, 1: 1822.5. Samples: 13133258. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 08:50:27,015][52710] Avg episode reward: [(0, '26.150'), (1, '28.020')] [2023-10-08 08:50:27,348][53885] Updated weights for policy 1, policy_version 25572 (0.0008) [2023-10-08 08:50:27,709][53885] Updated weights for policy 1, policy_version 25582 (0.0008) [2023-10-08 08:50:28,076][53885] Updated weights for policy 1, policy_version 25592 (0.0011) [2023-10-08 08:50:30,543][53852] Updated weights for policy 0, policy_version 25730 (0.0008) [2023-10-08 08:50:30,911][53852] Updated weights for policy 0, policy_version 25740 (0.0009) [2023-10-08 08:50:31,283][53852] Updated weights for policy 0, policy_version 25750 (0.0010) [2023-10-08 08:50:31,656][53852] Updated weights for policy 0, policy_version 25760 (0.0010) [2023-10-08 08:50:31,820][53885] Updated weights for policy 1, policy_version 25602 (0.0011) [2023-10-08 08:50:32,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 52592640. Throughput: 0: 1837.6, 1: 1818.1. Samples: 13156334. Policy #0 lag: (min: 17.0, avg: 32.4, max: 49.0) [2023-10-08 08:50:32,015][52710] Avg episode reward: [(0, '28.490'), (1, '28.840')] [2023-10-08 08:50:32,016][53500] Saving new best policy, reward=28.490! [2023-10-08 08:50:32,189][53885] Updated weights for policy 1, policy_version 25612 (0.0009) [2023-10-08 08:50:32,551][53885] Updated weights for policy 1, policy_version 25622 (0.0008) [2023-10-08 08:50:32,910][53885] Updated weights for policy 1, policy_version 25632 (0.0008) [2023-10-08 08:50:35,236][53852] Updated weights for policy 0, policy_version 25770 (0.0007) [2023-10-08 08:50:35,613][53852] Updated weights for policy 0, policy_version 25780 (0.0008) [2023-10-08 08:50:35,978][53852] Updated weights for policy 0, policy_version 25790 (0.0007) [2023-10-08 08:50:36,583][53885] Updated weights for policy 1, policy_version 25642 (0.0010) [2023-10-08 08:50:36,960][53885] Updated weights for policy 1, policy_version 25652 (0.0009) [2023-10-08 08:50:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52658176. Throughput: 0: 1842.9, 1: 1817.2. Samples: 13177514. Policy #0 lag: (min: 17.0, avg: 32.4, max: 49.0) [2023-10-08 08:50:37,016][52710] Avg episode reward: [(0, '27.080'), (1, '28.610')] [2023-10-08 08:50:37,323][53885] Updated weights for policy 1, policy_version 25662 (0.0009) [2023-10-08 08:50:39,378][53852] Updated weights for policy 0, policy_version 25800 (0.0008) [2023-10-08 08:50:39,757][53852] Updated weights for policy 0, policy_version 25810 (0.0007) [2023-10-08 08:50:40,131][53852] Updated weights for policy 0, policy_version 25820 (0.0007) [2023-10-08 08:50:41,038][53885] Updated weights for policy 1, policy_version 25672 (0.0008) [2023-10-08 08:50:41,399][53885] Updated weights for policy 1, policy_version 25682 (0.0007) [2023-10-08 08:50:41,769][53885] Updated weights for policy 1, policy_version 25692 (0.0007) [2023-10-08 08:50:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 52756480. Throughput: 0: 1839.3, 1: 1814.4. Samples: 13189032. Policy #0 lag: (min: 17.0, avg: 32.4, max: 49.0) [2023-10-08 08:50:42,015][52710] Avg episode reward: [(0, '25.050'), (1, '27.760')] [2023-10-08 08:50:43,891][53852] Updated weights for policy 0, policy_version 25830 (0.0007) [2023-10-08 08:50:44,265][53852] Updated weights for policy 0, policy_version 25840 (0.0007) [2023-10-08 08:50:44,633][53852] Updated weights for policy 0, policy_version 25850 (0.0007) [2023-10-08 08:50:45,438][53885] Updated weights for policy 1, policy_version 25702 (0.0009) [2023-10-08 08:50:45,811][53885] Updated weights for policy 1, policy_version 25712 (0.0010) [2023-10-08 08:50:46,173][53885] Updated weights for policy 1, policy_version 25722 (0.0010) [2023-10-08 08:50:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 52822016. Throughput: 0: 1845.3, 1: 1819.0. Samples: 13210494. Policy #0 lag: (min: 17.0, avg: 32.4, max: 49.0) [2023-10-08 08:50:47,016][52710] Avg episode reward: [(0, '26.340'), (1, '26.840')] [2023-10-08 08:50:48,270][53852] Updated weights for policy 0, policy_version 25860 (0.0008) [2023-10-08 08:50:48,638][53852] Updated weights for policy 0, policy_version 25870 (0.0007) [2023-10-08 08:50:49,006][53852] Updated weights for policy 0, policy_version 25880 (0.0007) [2023-10-08 08:50:49,860][53885] Updated weights for policy 1, policy_version 25732 (0.0007) [2023-10-08 08:50:50,227][53885] Updated weights for policy 1, policy_version 25742 (0.0010) [2023-10-08 08:50:50,592][53885] Updated weights for policy 1, policy_version 25752 (0.0008) [2023-10-08 08:50:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 52887552. Throughput: 0: 1855.9, 1: 1821.1. Samples: 13232566. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 08:50:52,015][52710] Avg episode reward: [(0, '24.630'), (1, '26.850')] [2023-10-08 08:50:52,526][53852] Updated weights for policy 0, policy_version 25890 (0.0009) [2023-10-08 08:50:52,892][53852] Updated weights for policy 0, policy_version 25900 (0.0007) [2023-10-08 08:50:53,251][53852] Updated weights for policy 0, policy_version 25910 (0.0008) [2023-10-08 08:50:53,623][53852] Updated weights for policy 0, policy_version 25920 (0.0008) [2023-10-08 08:50:54,317][53885] Updated weights for policy 1, policy_version 25762 (0.0009) [2023-10-08 08:50:54,693][53885] Updated weights for policy 1, policy_version 25772 (0.0008) [2023-10-08 08:50:55,070][53885] Updated weights for policy 1, policy_version 25782 (0.0009) [2023-10-08 08:50:55,435][53885] Updated weights for policy 1, policy_version 25792 (0.0008) [2023-10-08 08:50:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 52953088. Throughput: 0: 1854.8, 1: 1816.6. Samples: 13243430. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 08:50:57,016][52710] Avg episode reward: [(0, '24.050'), (1, '28.410')] [2023-10-08 08:50:57,269][53852] Updated weights for policy 0, policy_version 25930 (0.0007) [2023-10-08 08:50:57,643][53852] Updated weights for policy 0, policy_version 25940 (0.0009) [2023-10-08 08:50:58,005][53852] Updated weights for policy 0, policy_version 25950 (0.0008) [2023-10-08 08:50:59,110][53885] Updated weights for policy 1, policy_version 25802 (0.0010) [2023-10-08 08:50:59,475][53885] Updated weights for policy 1, policy_version 25812 (0.0008) [2023-10-08 08:50:59,837][53885] Updated weights for policy 1, policy_version 25822 (0.0008) [2023-10-08 08:51:01,610][53852] Updated weights for policy 0, policy_version 25960 (0.0008) [2023-10-08 08:51:01,981][53852] Updated weights for policy 0, policy_version 25970 (0.0008) [2023-10-08 08:51:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53018624. Throughput: 0: 1854.6, 1: 1817.3. Samples: 13265082. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 08:51:02,015][52710] Avg episode reward: [(0, '25.560'), (1, '27.600')] [2023-10-08 08:51:02,348][53852] Updated weights for policy 0, policy_version 25980 (0.0008) [2023-10-08 08:51:03,528][53885] Updated weights for policy 1, policy_version 25832 (0.0007) [2023-10-08 08:51:03,893][53885] Updated weights for policy 1, policy_version 25842 (0.0009) [2023-10-08 08:51:04,267][53885] Updated weights for policy 1, policy_version 25852 (0.0007) [2023-10-08 08:51:05,992][53852] Updated weights for policy 0, policy_version 25990 (0.0008) [2023-10-08 08:51:06,357][53852] Updated weights for policy 0, policy_version 26000 (0.0008) [2023-10-08 08:51:06,724][53852] Updated weights for policy 0, policy_version 26010 (0.0008) [2023-10-08 08:51:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53116928. Throughput: 0: 1828.1, 1: 1824.8. Samples: 13287324. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) [2023-10-08 08:51:07,016][52710] Avg episode reward: [(0, '25.980'), (1, '29.600')] [2023-10-08 08:51:07,857][53885] Updated weights for policy 1, policy_version 25862 (0.0008) [2023-10-08 08:51:08,231][53885] Updated weights for policy 1, policy_version 25872 (0.0008) [2023-10-08 08:51:08,588][53885] Updated weights for policy 1, policy_version 25882 (0.0008) [2023-10-08 08:51:10,290][53852] Updated weights for policy 0, policy_version 26020 (0.0009) [2023-10-08 08:51:10,658][53852] Updated weights for policy 0, policy_version 26030 (0.0008) [2023-10-08 08:51:11,023][53852] Updated weights for policy 0, policy_version 26040 (0.0007) [2023-10-08 08:51:12,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53182464. Throughput: 0: 1847.9, 1: 1818.7. Samples: 13298258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:51:12,016][52710] Avg episode reward: [(0, '25.710'), (1, '30.870')] [2023-10-08 08:51:12,372][53885] Updated weights for policy 1, policy_version 25892 (0.0008) [2023-10-08 08:51:12,741][53885] Updated weights for policy 1, policy_version 25902 (0.0008) [2023-10-08 08:51:13,109][53885] Updated weights for policy 1, policy_version 25912 (0.0008) [2023-10-08 08:51:14,693][53852] Updated weights for policy 0, policy_version 26050 (0.0008) [2023-10-08 08:51:15,067][53852] Updated weights for policy 0, policy_version 26060 (0.0010) [2023-10-08 08:51:15,448][53852] Updated weights for policy 0, policy_version 26070 (0.0009) [2023-10-08 08:51:15,824][53852] Updated weights for policy 0, policy_version 26080 (0.0008) [2023-10-08 08:51:16,750][53885] Updated weights for policy 1, policy_version 25922 (0.0008) [2023-10-08 08:51:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53248000. Throughput: 0: 1819.4, 1: 1820.5. Samples: 13320132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:51:17,016][52710] Avg episode reward: [(0, '27.310'), (1, '28.500')] [2023-10-08 08:51:17,120][53885] Updated weights for policy 1, policy_version 25932 (0.0008) [2023-10-08 08:51:17,494][53885] Updated weights for policy 1, policy_version 25942 (0.0007) [2023-10-08 08:51:17,865][53885] Updated weights for policy 1, policy_version 25952 (0.0008) [2023-10-08 08:51:19,500][53852] Updated weights for policy 0, policy_version 26090 (0.0009) [2023-10-08 08:51:19,870][53852] Updated weights for policy 0, policy_version 26100 (0.0007) [2023-10-08 08:51:20,241][53852] Updated weights for policy 0, policy_version 26110 (0.0008) [2023-10-08 08:51:21,534][53885] Updated weights for policy 1, policy_version 25962 (0.0008) [2023-10-08 08:51:21,902][53885] Updated weights for policy 1, policy_version 25972 (0.0008) [2023-10-08 08:51:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 53313536. Throughput: 0: 1841.6, 1: 1820.3. Samples: 13342302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:51:22,016][52710] Avg episode reward: [(0, '27.900'), (1, '30.630')] [2023-10-08 08:51:22,274][53885] Updated weights for policy 1, policy_version 25982 (0.0009) [2023-10-08 08:51:23,952][53852] Updated weights for policy 0, policy_version 26120 (0.0009) [2023-10-08 08:51:24,330][53852] Updated weights for policy 0, policy_version 26130 (0.0007) [2023-10-08 08:51:24,701][53852] Updated weights for policy 0, policy_version 26140 (0.0007) [2023-10-08 08:51:25,973][53885] Updated weights for policy 1, policy_version 25992 (0.0007) [2023-10-08 08:51:26,342][53885] Updated weights for policy 1, policy_version 26002 (0.0008) [2023-10-08 08:51:26,703][53885] Updated weights for policy 1, policy_version 26012 (0.0008) [2023-10-08 08:51:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 53411840. Throughput: 0: 1822.9, 1: 1824.6. Samples: 13353168. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:51:27,016][52710] Avg episode reward: [(0, '28.570'), (1, '26.950')] [2023-10-08 08:51:27,017][53500] Saving new best policy, reward=28.570! [2023-10-08 08:51:28,257][53852] Updated weights for policy 0, policy_version 26150 (0.0008) [2023-10-08 08:51:28,631][53852] Updated weights for policy 0, policy_version 26160 (0.0011) [2023-10-08 08:51:29,004][53852] Updated weights for policy 0, policy_version 26170 (0.0009) [2023-10-08 08:51:30,384][53885] Updated weights for policy 1, policy_version 26022 (0.0008) [2023-10-08 08:51:30,750][53885] Updated weights for policy 1, policy_version 26032 (0.0009) [2023-10-08 08:51:31,116][53885] Updated weights for policy 1, policy_version 26042 (0.0010) [2023-10-08 08:51:32,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 53477376. Throughput: 0: 1838.4, 1: 1818.9. Samples: 13375074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:51:32,016][52710] Avg episode reward: [(0, '26.600'), (1, '28.040')] [2023-10-08 08:51:32,823][53852] Updated weights for policy 0, policy_version 26180 (0.0008) [2023-10-08 08:51:33,194][53852] Updated weights for policy 0, policy_version 26190 (0.0007) [2023-10-08 08:51:33,567][53852] Updated weights for policy 0, policy_version 26200 (0.0007) [2023-10-08 08:51:34,629][53885] Updated weights for policy 1, policy_version 26052 (0.0009) [2023-10-08 08:51:34,996][53885] Updated weights for policy 1, policy_version 26062 (0.0008) [2023-10-08 08:51:35,365][53885] Updated weights for policy 1, policy_version 26072 (0.0008) [2023-10-08 08:51:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53542912. Throughput: 0: 1828.4, 1: 1831.0. Samples: 13397240. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 08:51:37,016][52710] Avg episode reward: [(0, '26.960'), (1, '26.530')] [2023-10-08 08:51:37,280][53852] Updated weights for policy 0, policy_version 26210 (0.0008) [2023-10-08 08:51:37,654][53852] Updated weights for policy 0, policy_version 26220 (0.0010) [2023-10-08 08:51:38,024][53852] Updated weights for policy 0, policy_version 26230 (0.0009) [2023-10-08 08:51:38,395][53852] Updated weights for policy 0, policy_version 26240 (0.0007) [2023-10-08 08:51:39,022][53885] Updated weights for policy 1, policy_version 26082 (0.0010) [2023-10-08 08:51:39,399][53885] Updated weights for policy 1, policy_version 26092 (0.0007) [2023-10-08 08:51:39,765][53885] Updated weights for policy 1, policy_version 26102 (0.0008) [2023-10-08 08:51:40,134][53885] Updated weights for policy 1, policy_version 26112 (0.0008) [2023-10-08 08:51:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 53608448. Throughput: 0: 1828.3, 1: 1830.1. Samples: 13408058. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 08:51:42,016][52710] Avg episode reward: [(0, '25.410'), (1, '25.310')] [2023-10-08 08:51:42,174][53852] Updated weights for policy 0, policy_version 26250 (0.0007) [2023-10-08 08:51:42,550][53852] Updated weights for policy 0, policy_version 26260 (0.0007) [2023-10-08 08:51:42,922][53852] Updated weights for policy 0, policy_version 26270 (0.0007) [2023-10-08 08:51:43,765][53885] Updated weights for policy 1, policy_version 26122 (0.0007) [2023-10-08 08:51:44,130][53885] Updated weights for policy 1, policy_version 26132 (0.0011) [2023-10-08 08:51:44,507][53885] Updated weights for policy 1, policy_version 26142 (0.0010) [2023-10-08 08:51:46,486][53852] Updated weights for policy 0, policy_version 26280 (0.0008) [2023-10-08 08:51:46,856][53852] Updated weights for policy 0, policy_version 26290 (0.0007) [2023-10-08 08:51:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 53673984. Throughput: 0: 1831.6, 1: 1840.6. Samples: 13430332. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 08:51:47,016][52710] Avg episode reward: [(0, '26.700'), (1, '25.840')] [2023-10-08 08:51:47,221][53852] Updated weights for policy 0, policy_version 26300 (0.0007) [2023-10-08 08:51:48,040][53885] Updated weights for policy 1, policy_version 26152 (0.0009) [2023-10-08 08:51:48,403][53885] Updated weights for policy 1, policy_version 26162 (0.0009) [2023-10-08 08:51:48,776][53885] Updated weights for policy 1, policy_version 26172 (0.0010) [2023-10-08 08:51:50,796][53852] Updated weights for policy 0, policy_version 26310 (0.0007) [2023-10-08 08:51:51,161][53852] Updated weights for policy 0, policy_version 26320 (0.0007) [2023-10-08 08:51:51,526][53852] Updated weights for policy 0, policy_version 26330 (0.0007) [2023-10-08 08:51:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 53772288. Throughput: 0: 1828.7, 1: 1839.9. Samples: 13452410. Policy #0 lag: (min: 31.0, avg: 35.3, max: 63.0) [2023-10-08 08:51:52,016][52710] Avg episode reward: [(0, '28.810'), (1, '28.860')] [2023-10-08 08:51:52,028][53500] Saving new best policy, reward=28.810! [2023-10-08 08:51:52,585][53885] Updated weights for policy 1, policy_version 26182 (0.0008) [2023-10-08 08:51:52,962][53885] Updated weights for policy 1, policy_version 26192 (0.0007) [2023-10-08 08:51:53,330][53885] Updated weights for policy 1, policy_version 26202 (0.0010) [2023-10-08 08:51:55,146][53852] Updated weights for policy 0, policy_version 26340 (0.0010) [2023-10-08 08:51:55,517][53852] Updated weights for policy 0, policy_version 26350 (0.0009) [2023-10-08 08:51:55,887][53852] Updated weights for policy 0, policy_version 26360 (0.0011) [2023-10-08 08:51:56,768][53885] Updated weights for policy 1, policy_version 26212 (0.0007) [2023-10-08 08:51:57,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53837824. Throughput: 0: 1835.2, 1: 1842.3. Samples: 13463742. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 08:51:57,016][52710] Avg episode reward: [(0, '27.700'), (1, '28.030')] [2023-10-08 08:51:57,141][53885] Updated weights for policy 1, policy_version 26222 (0.0009) [2023-10-08 08:51:57,519][53885] Updated weights for policy 1, policy_version 26232 (0.0008) [2023-10-08 08:51:59,531][53852] Updated weights for policy 0, policy_version 26370 (0.0010) [2023-10-08 08:51:59,897][53852] Updated weights for policy 0, policy_version 26380 (0.0009) [2023-10-08 08:52:00,273][53852] Updated weights for policy 0, policy_version 26390 (0.0010) [2023-10-08 08:52:00,639][53852] Updated weights for policy 0, policy_version 26400 (0.0009) [2023-10-08 08:52:01,166][53885] Updated weights for policy 1, policy_version 26242 (0.0008) [2023-10-08 08:52:01,537][53885] Updated weights for policy 1, policy_version 26252 (0.0008) [2023-10-08 08:52:01,910][53885] Updated weights for policy 1, policy_version 26262 (0.0008) [2023-10-08 08:52:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53903360. Throughput: 0: 1832.7, 1: 1846.0. Samples: 13485674. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 08:52:02,016][52710] Avg episode reward: [(0, '27.340'), (1, '27.700')] [2023-10-08 08:52:02,281][53885] Updated weights for policy 1, policy_version 26272 (0.0008) [2023-10-08 08:52:04,453][53852] Updated weights for policy 0, policy_version 26410 (0.0008) [2023-10-08 08:52:04,818][53852] Updated weights for policy 0, policy_version 26420 (0.0008) [2023-10-08 08:52:05,182][53852] Updated weights for policy 0, policy_version 26430 (0.0009) [2023-10-08 08:52:06,088][53885] Updated weights for policy 1, policy_version 26282 (0.0007) [2023-10-08 08:52:06,453][53885] Updated weights for policy 1, policy_version 26292 (0.0007) [2023-10-08 08:52:06,814][53885] Updated weights for policy 1, policy_version 26302 (0.0008) [2023-10-08 08:52:07,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54001664. Throughput: 0: 1835.6, 1: 1831.7. Samples: 13507328. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 08:52:07,016][52710] Avg episode reward: [(0, '27.120'), (1, '28.380')] [2023-10-08 08:52:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000026432_27066368.pth... [2023-10-08 08:52:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000026304_26935296.pth... [2023-10-08 08:52:07,059][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000024704_25296896.pth [2023-10-08 08:52:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000024576_25165824.pth [2023-10-08 08:52:08,709][53852] Updated weights for policy 0, policy_version 26440 (0.0008) [2023-10-08 08:52:09,075][53852] Updated weights for policy 0, policy_version 26450 (0.0010) [2023-10-08 08:52:09,461][53852] Updated weights for policy 0, policy_version 26460 (0.0009) [2023-10-08 08:52:10,435][53885] Updated weights for policy 1, policy_version 26312 (0.0008) [2023-10-08 08:52:10,795][53885] Updated weights for policy 1, policy_version 26322 (0.0009) [2023-10-08 08:52:11,168][53885] Updated weights for policy 1, policy_version 26332 (0.0008) [2023-10-08 08:52:12,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54067200. Throughput: 0: 1830.4, 1: 1845.4. Samples: 13518578. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 08:52:12,016][52710] Avg episode reward: [(0, '24.630'), (1, '28.110')] [2023-10-08 08:52:13,148][53852] Updated weights for policy 0, policy_version 26470 (0.0008) [2023-10-08 08:52:13,518][53852] Updated weights for policy 0, policy_version 26480 (0.0010) [2023-10-08 08:52:13,884][53852] Updated weights for policy 0, policy_version 26490 (0.0008) [2023-10-08 08:52:14,585][53885] Updated weights for policy 1, policy_version 26342 (0.0007) [2023-10-08 08:52:14,968][53885] Updated weights for policy 1, policy_version 26352 (0.0009) [2023-10-08 08:52:15,335][53885] Updated weights for policy 1, policy_version 26362 (0.0009) [2023-10-08 08:52:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54132736. Throughput: 0: 1839.0, 1: 1836.1. Samples: 13540454. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) [2023-10-08 08:52:17,016][52710] Avg episode reward: [(0, '24.740'), (1, '27.470')] [2023-10-08 08:52:17,734][53852] Updated weights for policy 0, policy_version 26500 (0.0009) [2023-10-08 08:52:18,097][53852] Updated weights for policy 0, policy_version 26510 (0.0010) [2023-10-08 08:52:18,460][53852] Updated weights for policy 0, policy_version 26520 (0.0011) [2023-10-08 08:52:19,009][53885] Updated weights for policy 1, policy_version 26372 (0.0010) [2023-10-08 08:52:19,387][53885] Updated weights for policy 1, policy_version 26382 (0.0010) [2023-10-08 08:52:19,762][53885] Updated weights for policy 1, policy_version 26392 (0.0010) [2023-10-08 08:52:22,011][53852] Updated weights for policy 0, policy_version 26530 (0.0010) [2023-10-08 08:52:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 54198272. Throughput: 0: 1838.3, 1: 1851.9. Samples: 13563296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-10-08 08:52:22,016][52710] Avg episode reward: [(0, '25.770'), (1, '27.570')] [2023-10-08 08:52:22,383][53852] Updated weights for policy 0, policy_version 26540 (0.0007) [2023-10-08 08:52:22,752][53852] Updated weights for policy 0, policy_version 26550 (0.0007) [2023-10-08 08:52:23,122][53852] Updated weights for policy 0, policy_version 26560 (0.0007) [2023-10-08 08:52:23,405][53885] Updated weights for policy 1, policy_version 26402 (0.0010) [2023-10-08 08:52:23,775][53885] Updated weights for policy 1, policy_version 26412 (0.0007) [2023-10-08 08:52:24,139][53885] Updated weights for policy 1, policy_version 26422 (0.0009) [2023-10-08 08:52:24,507][53885] Updated weights for policy 1, policy_version 26432 (0.0008) [2023-10-08 08:52:26,796][53852] Updated weights for policy 0, policy_version 26570 (0.0008) [2023-10-08 08:52:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 54263808. Throughput: 0: 1842.4, 1: 1829.6. Samples: 13573294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-10-08 08:52:27,016][52710] Avg episode reward: [(0, '26.320'), (1, '26.340')] [2023-10-08 08:52:27,174][53852] Updated weights for policy 0, policy_version 26580 (0.0007) [2023-10-08 08:52:27,549][53852] Updated weights for policy 0, policy_version 26590 (0.0009) [2023-10-08 08:52:28,140][53885] Updated weights for policy 1, policy_version 26442 (0.0009) [2023-10-08 08:52:28,505][53885] Updated weights for policy 1, policy_version 26452 (0.0008) [2023-10-08 08:52:28,871][53885] Updated weights for policy 1, policy_version 26462 (0.0007) [2023-10-08 08:52:31,291][53852] Updated weights for policy 0, policy_version 26600 (0.0010) [2023-10-08 08:52:31,663][53852] Updated weights for policy 0, policy_version 26610 (0.0010) [2023-10-08 08:52:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54329344. Throughput: 0: 1839.4, 1: 1851.1. Samples: 13596406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-10-08 08:52:32,016][52710] Avg episode reward: [(0, '26.610'), (1, '29.080')] [2023-10-08 08:52:32,041][53852] Updated weights for policy 0, policy_version 26620 (0.0010) [2023-10-08 08:52:32,475][53885] Updated weights for policy 1, policy_version 26472 (0.0009) [2023-10-08 08:52:32,835][53885] Updated weights for policy 1, policy_version 26482 (0.0008) [2023-10-08 08:52:33,201][53885] Updated weights for policy 1, policy_version 26492 (0.0007) [2023-10-08 08:52:35,590][53852] Updated weights for policy 0, policy_version 26630 (0.0007) [2023-10-08 08:52:35,966][53852] Updated weights for policy 0, policy_version 26640 (0.0007) [2023-10-08 08:52:36,340][53852] Updated weights for policy 0, policy_version 26650 (0.0007) [2023-10-08 08:52:36,828][53885] Updated weights for policy 1, policy_version 26502 (0.0008) [2023-10-08 08:52:37,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54427648. Throughput: 0: 1834.5, 1: 1848.7. Samples: 13618152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) [2023-10-08 08:52:37,015][52710] Avg episode reward: [(0, '27.850'), (1, '27.050')] [2023-10-08 08:52:37,204][53885] Updated weights for policy 1, policy_version 26512 (0.0008) [2023-10-08 08:52:37,578][53885] Updated weights for policy 1, policy_version 26522 (0.0008) [2023-10-08 08:52:40,057][53852] Updated weights for policy 0, policy_version 26660 (0.0008) [2023-10-08 08:52:40,435][53852] Updated weights for policy 0, policy_version 26670 (0.0007) [2023-10-08 08:52:40,800][53852] Updated weights for policy 0, policy_version 26680 (0.0007) [2023-10-08 08:52:41,292][53885] Updated weights for policy 1, policy_version 26532 (0.0010) [2023-10-08 08:52:41,666][53885] Updated weights for policy 1, policy_version 26542 (0.0010) [2023-10-08 08:52:42,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54493184. Throughput: 0: 1837.9, 1: 1846.0. Samples: 13629518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:52:42,016][52710] Avg episode reward: [(0, '27.370'), (1, '28.670')] [2023-10-08 08:52:42,026][53885] Updated weights for policy 1, policy_version 26552 (0.0009) [2023-10-08 08:52:44,433][53852] Updated weights for policy 0, policy_version 26690 (0.0008) [2023-10-08 08:52:44,791][53852] Updated weights for policy 0, policy_version 26700 (0.0010) [2023-10-08 08:52:45,167][53852] Updated weights for policy 0, policy_version 26710 (0.0010) [2023-10-08 08:52:45,526][53852] Updated weights for policy 0, policy_version 26720 (0.0010) [2023-10-08 08:52:45,762][53885] Updated weights for policy 1, policy_version 26562 (0.0010) [2023-10-08 08:52:46,140][53885] Updated weights for policy 1, policy_version 26572 (0.0008) [2023-10-08 08:52:46,500][53885] Updated weights for policy 1, policy_version 26582 (0.0008) [2023-10-08 08:52:46,861][53885] Updated weights for policy 1, policy_version 26592 (0.0009) [2023-10-08 08:52:47,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 54591488. Throughput: 0: 1837.4, 1: 1840.2. Samples: 13651164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:52:47,016][52710] Avg episode reward: [(0, '27.660'), (1, '28.170')] [2023-10-08 08:52:49,167][53852] Updated weights for policy 0, policy_version 26730 (0.0010) [2023-10-08 08:52:49,540][53852] Updated weights for policy 0, policy_version 26740 (0.0009) [2023-10-08 08:52:49,905][53852] Updated weights for policy 0, policy_version 26750 (0.0008) [2023-10-08 08:52:50,544][53885] Updated weights for policy 1, policy_version 26602 (0.0007) [2023-10-08 08:52:50,907][53885] Updated weights for policy 1, policy_version 26612 (0.0008) [2023-10-08 08:52:51,274][53885] Updated weights for policy 1, policy_version 26622 (0.0007) [2023-10-08 08:52:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54657024. Throughput: 0: 1840.3, 1: 1832.2. Samples: 13672594. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:52:52,016][52710] Avg episode reward: [(0, '29.000'), (1, '28.530')] [2023-10-08 08:52:52,027][53500] Saving new best policy, reward=29.000! [2023-10-08 08:52:53,533][53852] Updated weights for policy 0, policy_version 26760 (0.0009) [2023-10-08 08:52:53,904][53852] Updated weights for policy 0, policy_version 26770 (0.0011) [2023-10-08 08:52:54,275][53852] Updated weights for policy 0, policy_version 26780 (0.0008) [2023-10-08 08:52:54,939][53885] Updated weights for policy 1, policy_version 26632 (0.0008) [2023-10-08 08:52:55,300][53885] Updated weights for policy 1, policy_version 26642 (0.0008) [2023-10-08 08:52:55,662][53885] Updated weights for policy 1, policy_version 26652 (0.0007) [2023-10-08 08:52:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54722560. Throughput: 0: 1833.1, 1: 1843.1. Samples: 13684008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:52:57,016][52710] Avg episode reward: [(0, '27.730'), (1, '29.610')] [2023-10-08 08:52:57,719][53852] Updated weights for policy 0, policy_version 26790 (0.0008) [2023-10-08 08:52:58,091][53852] Updated weights for policy 0, policy_version 26800 (0.0009) [2023-10-08 08:52:58,470][53852] Updated weights for policy 0, policy_version 26810 (0.0008) [2023-10-08 08:52:59,249][53885] Updated weights for policy 1, policy_version 26662 (0.0007) [2023-10-08 08:52:59,622][53885] Updated weights for policy 1, policy_version 26672 (0.0009) [2023-10-08 08:52:59,993][53885] Updated weights for policy 1, policy_version 26682 (0.0009) [2023-10-08 08:53:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 54788096. Throughput: 0: 1837.5, 1: 1833.1. Samples: 13705630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:53:02,016][52710] Avg episode reward: [(0, '28.480'), (1, '27.500')] [2023-10-08 08:53:02,133][53852] Updated weights for policy 0, policy_version 26820 (0.0008) [2023-10-08 08:53:02,501][53852] Updated weights for policy 0, policy_version 26830 (0.0007) [2023-10-08 08:53:02,871][53852] Updated weights for policy 0, policy_version 26840 (0.0011) [2023-10-08 08:53:03,618][53885] Updated weights for policy 1, policy_version 26692 (0.0009) [2023-10-08 08:53:03,993][53885] Updated weights for policy 1, policy_version 26702 (0.0008) [2023-10-08 08:53:04,369][53885] Updated weights for policy 1, policy_version 26712 (0.0008) [2023-10-08 08:53:06,613][53852] Updated weights for policy 0, policy_version 26850 (0.0008) [2023-10-08 08:53:06,991][53852] Updated weights for policy 0, policy_version 26860 (0.0010) [2023-10-08 08:53:07,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54853632. Throughput: 0: 1842.0, 1: 1833.3. Samples: 13728682. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 08:53:07,015][52710] Avg episode reward: [(0, '28.210'), (1, '29.030')] [2023-10-08 08:53:07,361][53852] Updated weights for policy 0, policy_version 26870 (0.0009) [2023-10-08 08:53:07,738][53852] Updated weights for policy 0, policy_version 26880 (0.0011) [2023-10-08 08:53:08,121][53885] Updated weights for policy 1, policy_version 26722 (0.0007) [2023-10-08 08:53:08,532][53885] Updated weights for policy 1, policy_version 26732 (0.0007) [2023-10-08 08:53:08,901][53885] Updated weights for policy 1, policy_version 26742 (0.0008) [2023-10-08 08:53:09,267][53885] Updated weights for policy 1, policy_version 26752 (0.0008) [2023-10-08 08:53:11,392][53852] Updated weights for policy 0, policy_version 26890 (0.0008) [2023-10-08 08:53:11,774][53852] Updated weights for policy 0, policy_version 26900 (0.0008) [2023-10-08 08:53:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 54919168. Throughput: 0: 1836.8, 1: 1831.0. Samples: 13738346. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 08:53:12,016][52710] Avg episode reward: [(0, '26.560'), (1, '29.450')] [2023-10-08 08:53:12,146][53852] Updated weights for policy 0, policy_version 26910 (0.0010) [2023-10-08 08:53:12,999][53885] Updated weights for policy 1, policy_version 26762 (0.0008) [2023-10-08 08:53:13,372][53885] Updated weights for policy 1, policy_version 26772 (0.0007) [2023-10-08 08:53:13,737][53885] Updated weights for policy 1, policy_version 26782 (0.0008) [2023-10-08 08:53:15,822][53852] Updated weights for policy 0, policy_version 26920 (0.0008) [2023-10-08 08:53:16,194][53852] Updated weights for policy 0, policy_version 26930 (0.0008) [2023-10-08 08:53:16,568][53852] Updated weights for policy 0, policy_version 26940 (0.0009) [2023-10-08 08:53:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 55017472. Throughput: 0: 1834.0, 1: 1828.2. Samples: 13761204. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 08:53:17,016][52710] Avg episode reward: [(0, '26.890'), (1, '30.310')] [2023-10-08 08:53:17,320][53885] Updated weights for policy 1, policy_version 26792 (0.0007) [2023-10-08 08:53:17,681][53885] Updated weights for policy 1, policy_version 26802 (0.0008) [2023-10-08 08:53:18,046][53885] Updated weights for policy 1, policy_version 26812 (0.0008) [2023-10-08 08:53:20,031][53852] Updated weights for policy 0, policy_version 26950 (0.0007) [2023-10-08 08:53:20,405][53852] Updated weights for policy 0, policy_version 26960 (0.0008) [2023-10-08 08:53:20,773][53852] Updated weights for policy 0, policy_version 26970 (0.0008) [2023-10-08 08:53:21,701][53885] Updated weights for policy 1, policy_version 26822 (0.0009) [2023-10-08 08:53:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55083008. Throughput: 0: 1836.1, 1: 1827.6. Samples: 13783020. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) [2023-10-08 08:53:22,016][52710] Avg episode reward: [(0, '27.620'), (1, '30.630')] [2023-10-08 08:53:22,060][53885] Updated weights for policy 1, policy_version 26832 (0.0010) [2023-10-08 08:53:22,434][53885] Updated weights for policy 1, policy_version 26842 (0.0009) [2023-10-08 08:53:24,524][53852] Updated weights for policy 0, policy_version 26980 (0.0007) [2023-10-08 08:53:24,904][53852] Updated weights for policy 0, policy_version 26990 (0.0008) [2023-10-08 08:53:25,275][53852] Updated weights for policy 0, policy_version 27000 (0.0008) [2023-10-08 08:53:26,184][53885] Updated weights for policy 1, policy_version 26852 (0.0012) [2023-10-08 08:53:26,561][53885] Updated weights for policy 1, policy_version 26862 (0.0009) [2023-10-08 08:53:26,930][53885] Updated weights for policy 1, policy_version 26872 (0.0008) [2023-10-08 08:53:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 55148544. Throughput: 0: 1830.8, 1: 1830.7. Samples: 13794284. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:53:27,016][52710] Avg episode reward: [(0, '25.810'), (1, '29.990')] [2023-10-08 08:53:28,913][53852] Updated weights for policy 0, policy_version 27010 (0.0008) [2023-10-08 08:53:29,286][53852] Updated weights for policy 0, policy_version 27020 (0.0011) [2023-10-08 08:53:29,652][53852] Updated weights for policy 0, policy_version 27030 (0.0010) [2023-10-08 08:53:30,019][53852] Updated weights for policy 0, policy_version 27040 (0.0010) [2023-10-08 08:53:30,679][53885] Updated weights for policy 1, policy_version 26882 (0.0008) [2023-10-08 08:53:31,046][53885] Updated weights for policy 1, policy_version 26892 (0.0009) [2023-10-08 08:53:31,408][53885] Updated weights for policy 1, policy_version 26902 (0.0008) [2023-10-08 08:53:31,769][53885] Updated weights for policy 1, policy_version 26912 (0.0007) [2023-10-08 08:53:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 55246848. Throughput: 0: 1830.2, 1: 1831.2. Samples: 13815928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:53:32,016][52710] Avg episode reward: [(0, '27.400'), (1, '29.700')] [2023-10-08 08:53:33,682][53852] Updated weights for policy 0, policy_version 27050 (0.0007) [2023-10-08 08:53:34,052][53852] Updated weights for policy 0, policy_version 27060 (0.0011) [2023-10-08 08:53:34,427][53852] Updated weights for policy 0, policy_version 27070 (0.0009) [2023-10-08 08:53:35,522][53885] Updated weights for policy 1, policy_version 26922 (0.0008) [2023-10-08 08:53:35,882][53885] Updated weights for policy 1, policy_version 26932 (0.0007) [2023-10-08 08:53:36,249][53885] Updated weights for policy 1, policy_version 26942 (0.0010) [2023-10-08 08:53:37,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55312384. Throughput: 0: 1831.7, 1: 1827.9. Samples: 13837276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:53:37,016][52710] Avg episode reward: [(0, '25.390'), (1, '28.940')] [2023-10-08 08:53:38,100][53852] Updated weights for policy 0, policy_version 27080 (0.0008) [2023-10-08 08:53:38,473][53852] Updated weights for policy 0, policy_version 27090 (0.0008) [2023-10-08 08:53:38,831][53852] Updated weights for policy 0, policy_version 27100 (0.0007) [2023-10-08 08:53:40,140][53885] Updated weights for policy 1, policy_version 26952 (0.0010) [2023-10-08 08:53:40,510][53885] Updated weights for policy 1, policy_version 26962 (0.0007) [2023-10-08 08:53:40,889][53885] Updated weights for policy 1, policy_version 26972 (0.0010) [2023-10-08 08:53:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55377920. Throughput: 0: 1840.2, 1: 1820.8. Samples: 13848752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:53:42,016][52710] Avg episode reward: [(0, '24.980'), (1, '27.300')] [2023-10-08 08:53:42,462][53852] Updated weights for policy 0, policy_version 27110 (0.0009) [2023-10-08 08:53:42,838][53852] Updated weights for policy 0, policy_version 27120 (0.0007) [2023-10-08 08:53:43,209][53852] Updated weights for policy 0, policy_version 27130 (0.0008) [2023-10-08 08:53:44,440][53885] Updated weights for policy 1, policy_version 26982 (0.0011) [2023-10-08 08:53:44,812][53885] Updated weights for policy 1, policy_version 26992 (0.0008) [2023-10-08 08:53:45,172][53885] Updated weights for policy 1, policy_version 27002 (0.0007) [2023-10-08 08:53:46,632][53852] Updated weights for policy 0, policy_version 27140 (0.0007) [2023-10-08 08:53:47,000][53852] Updated weights for policy 0, policy_version 27150 (0.0007) [2023-10-08 08:53:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 55443456. Throughput: 0: 1840.4, 1: 1823.0. Samples: 13870482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:53:47,016][52710] Avg episode reward: [(0, '26.710'), (1, '25.990')] [2023-10-08 08:53:47,378][53852] Updated weights for policy 0, policy_version 27160 (0.0008) [2023-10-08 08:53:48,878][53885] Updated weights for policy 1, policy_version 27012 (0.0009) [2023-10-08 08:53:49,247][53885] Updated weights for policy 1, policy_version 27022 (0.0011) [2023-10-08 08:53:49,616][53885] Updated weights for policy 1, policy_version 27032 (0.0008) [2023-10-08 08:53:51,001][53852] Updated weights for policy 0, policy_version 27170 (0.0008) [2023-10-08 08:53:51,367][53852] Updated weights for policy 0, policy_version 27180 (0.0008) [2023-10-08 08:53:51,743][53852] Updated weights for policy 0, policy_version 27190 (0.0008) [2023-10-08 08:53:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 55508992. Throughput: 0: 1825.5, 1: 1819.1. Samples: 13892692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 08:53:52,016][52710] Avg episode reward: [(0, '24.660'), (1, '27.830')] [2023-10-08 08:53:52,103][53852] Updated weights for policy 0, policy_version 27200 (0.0010) [2023-10-08 08:53:53,285][53885] Updated weights for policy 1, policy_version 27042 (0.0007) [2023-10-08 08:53:53,678][53885] Updated weights for policy 1, policy_version 27052 (0.0007) [2023-10-08 08:53:54,051][53885] Updated weights for policy 1, policy_version 27062 (0.0007) [2023-10-08 08:53:54,423][53885] Updated weights for policy 1, policy_version 27072 (0.0007) [2023-10-08 08:53:55,757][53852] Updated weights for policy 0, policy_version 27210 (0.0008) [2023-10-08 08:53:56,130][53852] Updated weights for policy 0, policy_version 27220 (0.0007) [2023-10-08 08:53:56,506][53852] Updated weights for policy 0, policy_version 27230 (0.0008) [2023-10-08 08:53:57,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 55607296. Throughput: 0: 1844.7, 1: 1821.3. Samples: 13903318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 08:53:57,015][52710] Avg episode reward: [(0, '27.850'), (1, '25.970')] [2023-10-08 08:53:58,084][53885] Updated weights for policy 1, policy_version 27082 (0.0007) [2023-10-08 08:53:58,449][53885] Updated weights for policy 1, policy_version 27092 (0.0008) [2023-10-08 08:53:58,812][53885] Updated weights for policy 1, policy_version 27102 (0.0009) [2023-10-08 08:54:00,220][53852] Updated weights for policy 0, policy_version 27240 (0.0009) [2023-10-08 08:54:00,590][53852] Updated weights for policy 0, policy_version 27250 (0.0007) [2023-10-08 08:54:00,958][53852] Updated weights for policy 0, policy_version 27260 (0.0009) [2023-10-08 08:54:02,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55672832. Throughput: 0: 1833.5, 1: 1816.4. Samples: 13925450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 08:54:02,016][52710] Avg episode reward: [(0, '27.950'), (1, '24.540')] [2023-10-08 08:54:02,519][53885] Updated weights for policy 1, policy_version 27112 (0.0007) [2023-10-08 08:54:02,897][53885] Updated weights for policy 1, policy_version 27122 (0.0008) [2023-10-08 08:54:03,274][53885] Updated weights for policy 1, policy_version 27132 (0.0009) [2023-10-08 08:54:04,807][53852] Updated weights for policy 0, policy_version 27270 (0.0008) [2023-10-08 08:54:05,179][53852] Updated weights for policy 0, policy_version 27280 (0.0007) [2023-10-08 08:54:05,557][53852] Updated weights for policy 0, policy_version 27290 (0.0009) [2023-10-08 08:54:06,896][53885] Updated weights for policy 1, policy_version 27142 (0.0008) [2023-10-08 08:54:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 55738368. Throughput: 0: 1842.9, 1: 1812.9. Samples: 13947532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) [2023-10-08 08:54:07,016][52710] Avg episode reward: [(0, '26.520'), (1, '27.110')] [2023-10-08 08:54:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000027296_27951104.pth... [2023-10-08 08:54:07,064][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000025568_26181632.pth [2023-10-08 08:54:07,261][53885] Updated weights for policy 1, policy_version 27152 (0.0007) [2023-10-08 08:54:07,629][53885] Updated weights for policy 1, policy_version 27162 (0.0008) [2023-10-08 08:54:07,839][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000027168_27820032.pth... [2023-10-08 08:54:07,878][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000025440_26050560.pth [2023-10-08 08:54:09,104][53852] Updated weights for policy 0, policy_version 27300 (0.0007) [2023-10-08 08:54:09,484][53852] Updated weights for policy 0, policy_version 27310 (0.0009) [2023-10-08 08:54:09,848][53852] Updated weights for policy 0, policy_version 27320 (0.0007) [2023-10-08 08:54:11,287][53885] Updated weights for policy 1, policy_version 27172 (0.0007) [2023-10-08 08:54:11,658][53885] Updated weights for policy 1, policy_version 27182 (0.0009) [2023-10-08 08:54:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 55803904. Throughput: 0: 1834.2, 1: 1811.3. Samples: 13958330. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:54:12,015][52710] Avg episode reward: [(0, '26.320'), (1, '25.950')] [2023-10-08 08:54:12,028][53885] Updated weights for policy 1, policy_version 27192 (0.0008) [2023-10-08 08:54:13,496][53852] Updated weights for policy 0, policy_version 27330 (0.0009) [2023-10-08 08:54:13,859][53852] Updated weights for policy 0, policy_version 27340 (0.0010) [2023-10-08 08:54:14,225][53852] Updated weights for policy 0, policy_version 27350 (0.0010) [2023-10-08 08:54:14,602][53852] Updated weights for policy 0, policy_version 27360 (0.0008) [2023-10-08 08:54:15,884][53885] Updated weights for policy 1, policy_version 27202 (0.0010) [2023-10-08 08:54:16,266][53885] Updated weights for policy 1, policy_version 27212 (0.0010) [2023-10-08 08:54:16,634][53885] Updated weights for policy 1, policy_version 27222 (0.0009) [2023-10-08 08:54:16,995][53885] Updated weights for policy 1, policy_version 27232 (0.0008) [2023-10-08 08:54:17,015][52710] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55902208. Throughput: 0: 1840.9, 1: 1812.7. Samples: 13980342. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:54:17,016][52710] Avg episode reward: [(0, '27.490'), (1, '27.740')] [2023-10-08 08:54:18,447][53852] Updated weights for policy 0, policy_version 27370 (0.0011) [2023-10-08 08:54:18,814][53852] Updated weights for policy 0, policy_version 27380 (0.0008) [2023-10-08 08:54:19,187][53852] Updated weights for policy 0, policy_version 27390 (0.0010) [2023-10-08 08:54:20,615][53885] Updated weights for policy 1, policy_version 27242 (0.0007) [2023-10-08 08:54:20,975][53885] Updated weights for policy 1, policy_version 27252 (0.0010) [2023-10-08 08:54:21,341][53885] Updated weights for policy 1, policy_version 27262 (0.0010) [2023-10-08 08:54:22,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 55967744. Throughput: 0: 1838.2, 1: 1819.4. Samples: 14001866. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:54:22,016][52710] Avg episode reward: [(0, '27.120'), (1, '29.270')] [2023-10-08 08:54:22,680][53852] Updated weights for policy 0, policy_version 27400 (0.0007) [2023-10-08 08:54:23,052][53852] Updated weights for policy 0, policy_version 27410 (0.0007) [2023-10-08 08:54:23,419][53852] Updated weights for policy 0, policy_version 27420 (0.0007) [2023-10-08 08:54:24,839][53885] Updated weights for policy 1, policy_version 27272 (0.0010) [2023-10-08 08:54:25,201][53885] Updated weights for policy 1, policy_version 27282 (0.0010) [2023-10-08 08:54:25,560][53885] Updated weights for policy 1, policy_version 27292 (0.0010) [2023-10-08 08:54:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56033280. Throughput: 0: 1834.1, 1: 1827.3. Samples: 14013516. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:54:27,016][52710] Avg episode reward: [(0, '26.570'), (1, '26.500')] [2023-10-08 08:54:27,128][53852] Updated weights for policy 0, policy_version 27430 (0.0008) [2023-10-08 08:54:27,514][53852] Updated weights for policy 0, policy_version 27440 (0.0008) [2023-10-08 08:54:27,884][53852] Updated weights for policy 0, policy_version 27450 (0.0009) [2023-10-08 08:54:29,307][53885] Updated weights for policy 1, policy_version 27302 (0.0009) [2023-10-08 08:54:29,668][53885] Updated weights for policy 1, policy_version 27312 (0.0010) [2023-10-08 08:54:30,041][53885] Updated weights for policy 1, policy_version 27322 (0.0009) [2023-10-08 08:54:31,516][53852] Updated weights for policy 0, policy_version 27460 (0.0008) [2023-10-08 08:54:31,895][53852] Updated weights for policy 0, policy_version 27470 (0.0009) [2023-10-08 08:54:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 56098816. Throughput: 0: 1834.0, 1: 1822.1. Samples: 14035008. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) [2023-10-08 08:54:32,016][52710] Avg episode reward: [(0, '27.950'), (1, '26.900')] [2023-10-08 08:54:32,263][53852] Updated weights for policy 0, policy_version 27480 (0.0008) [2023-10-08 08:54:33,763][53885] Updated weights for policy 1, policy_version 27332 (0.0009) [2023-10-08 08:54:34,131][53885] Updated weights for policy 1, policy_version 27342 (0.0007) [2023-10-08 08:54:34,491][53885] Updated weights for policy 1, policy_version 27352 (0.0011) [2023-10-08 08:54:36,079][53852] Updated weights for policy 0, policy_version 27490 (0.0008) [2023-10-08 08:54:36,444][53852] Updated weights for policy 0, policy_version 27500 (0.0008) [2023-10-08 08:54:36,812][53852] Updated weights for policy 0, policy_version 27510 (0.0007) [2023-10-08 08:54:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 56164352. Throughput: 0: 1833.4, 1: 1828.3. Samples: 14057468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:54:37,016][52710] Avg episode reward: [(0, '27.590'), (1, '27.450')] [2023-10-08 08:54:37,177][53852] Updated weights for policy 0, policy_version 27520 (0.0008) [2023-10-08 08:54:38,090][53885] Updated weights for policy 1, policy_version 27362 (0.0009) [2023-10-08 08:54:38,503][53885] Updated weights for policy 1, policy_version 27372 (0.0009) [2023-10-08 08:54:38,883][53885] Updated weights for policy 1, policy_version 27382 (0.0011) [2023-10-08 08:54:39,250][53885] Updated weights for policy 1, policy_version 27392 (0.0010) [2023-10-08 08:54:40,851][53852] Updated weights for policy 0, policy_version 27530 (0.0007) [2023-10-08 08:54:41,212][53852] Updated weights for policy 0, policy_version 27540 (0.0010) [2023-10-08 08:54:41,587][53852] Updated weights for policy 0, policy_version 27550 (0.0010) [2023-10-08 08:54:42,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 56262656. Throughput: 0: 1833.7, 1: 1828.6. Samples: 14068122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:54:42,016][52710] Avg episode reward: [(0, '25.980'), (1, '27.130')] [2023-10-08 08:54:42,849][53885] Updated weights for policy 1, policy_version 27402 (0.0007) [2023-10-08 08:54:43,213][53885] Updated weights for policy 1, policy_version 27412 (0.0009) [2023-10-08 08:54:43,579][53885] Updated weights for policy 1, policy_version 27422 (0.0008) [2023-10-08 08:54:45,259][53852] Updated weights for policy 0, policy_version 27560 (0.0008) [2023-10-08 08:54:45,623][53852] Updated weights for policy 0, policy_version 27570 (0.0008) [2023-10-08 08:54:45,997][53852] Updated weights for policy 0, policy_version 27580 (0.0007) [2023-10-08 08:54:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56328192. Throughput: 0: 1832.8, 1: 1834.9. Samples: 14090500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:54:47,016][52710] Avg episode reward: [(0, '28.420'), (1, '27.930')] [2023-10-08 08:54:47,106][53885] Updated weights for policy 1, policy_version 27432 (0.0007) [2023-10-08 08:54:47,461][53885] Updated weights for policy 1, policy_version 27442 (0.0009) [2023-10-08 08:54:47,837][53885] Updated weights for policy 1, policy_version 27452 (0.0009) [2023-10-08 08:54:49,773][53852] Updated weights for policy 0, policy_version 27590 (0.0007) [2023-10-08 08:54:50,149][53852] Updated weights for policy 0, policy_version 27600 (0.0007) [2023-10-08 08:54:50,534][53852] Updated weights for policy 0, policy_version 27610 (0.0010) [2023-10-08 08:54:51,609][53885] Updated weights for policy 1, policy_version 27462 (0.0009) [2023-10-08 08:54:51,974][53885] Updated weights for policy 1, policy_version 27472 (0.0009) [2023-10-08 08:54:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56393728. Throughput: 0: 1828.8, 1: 1835.4. Samples: 14112418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:54:52,016][52710] Avg episode reward: [(0, '27.120'), (1, '26.700')] [2023-10-08 08:54:52,345][53885] Updated weights for policy 1, policy_version 27482 (0.0009) [2023-10-08 08:54:54,023][53852] Updated weights for policy 0, policy_version 27620 (0.0010) [2023-10-08 08:54:54,393][53852] Updated weights for policy 0, policy_version 27630 (0.0008) [2023-10-08 08:54:54,754][53852] Updated weights for policy 0, policy_version 27640 (0.0009) [2023-10-08 08:54:56,075][53885] Updated weights for policy 1, policy_version 27492 (0.0008) [2023-10-08 08:54:56,444][53885] Updated weights for policy 1, policy_version 27502 (0.0010) [2023-10-08 08:54:56,805][53885] Updated weights for policy 1, policy_version 27512 (0.0007) [2023-10-08 08:54:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 56459264. Throughput: 0: 1828.3, 1: 1840.2. Samples: 14123412. Policy #0 lag: (min: 18.0, avg: 24.1, max: 50.0) [2023-10-08 08:54:57,016][52710] Avg episode reward: [(0, '27.360'), (1, '30.080')] [2023-10-08 08:54:58,411][53852] Updated weights for policy 0, policy_version 27650 (0.0007) [2023-10-08 08:54:58,779][53852] Updated weights for policy 0, policy_version 27660 (0.0009) [2023-10-08 08:54:59,147][53852] Updated weights for policy 0, policy_version 27670 (0.0011) [2023-10-08 08:54:59,519][53852] Updated weights for policy 0, policy_version 27680 (0.0007) [2023-10-08 08:55:00,435][53885] Updated weights for policy 1, policy_version 27522 (0.0007) [2023-10-08 08:55:00,800][53885] Updated weights for policy 1, policy_version 27532 (0.0008) [2023-10-08 08:55:01,167][53885] Updated weights for policy 1, policy_version 27542 (0.0007) [2023-10-08 08:55:01,547][53885] Updated weights for policy 1, policy_version 27552 (0.0008) [2023-10-08 08:55:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 56557568. Throughput: 0: 1839.3, 1: 1830.4. Samples: 14145478. Policy #0 lag: (min: 18.0, avg: 24.1, max: 50.0) [2023-10-08 08:55:02,016][52710] Avg episode reward: [(0, '25.950'), (1, '31.560')] [2023-10-08 08:55:03,079][53852] Updated weights for policy 0, policy_version 27690 (0.0010) [2023-10-08 08:55:03,452][53852] Updated weights for policy 0, policy_version 27700 (0.0008) [2023-10-08 08:55:03,822][53852] Updated weights for policy 0, policy_version 27710 (0.0008) [2023-10-08 08:55:05,266][53885] Updated weights for policy 1, policy_version 27562 (0.0010) [2023-10-08 08:55:05,638][53885] Updated weights for policy 1, policy_version 27572 (0.0008) [2023-10-08 08:55:06,004][53885] Updated weights for policy 1, policy_version 27582 (0.0008) [2023-10-08 08:55:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 56623104. Throughput: 0: 1842.8, 1: 1830.9. Samples: 14167180. Policy #0 lag: (min: 18.0, avg: 24.1, max: 50.0) [2023-10-08 08:55:07,016][52710] Avg episode reward: [(0, '26.990'), (1, '30.210')] [2023-10-08 08:55:07,421][53852] Updated weights for policy 0, policy_version 27720 (0.0010) [2023-10-08 08:55:07,791][53852] Updated weights for policy 0, policy_version 27730 (0.0009) [2023-10-08 08:55:08,156][53852] Updated weights for policy 0, policy_version 27740 (0.0009) [2023-10-08 08:55:09,817][53885] Updated weights for policy 1, policy_version 27592 (0.0007) [2023-10-08 08:55:10,180][53885] Updated weights for policy 1, policy_version 27602 (0.0008) [2023-10-08 08:55:10,550][53885] Updated weights for policy 1, policy_version 27612 (0.0009) [2023-10-08 08:55:11,734][53852] Updated weights for policy 0, policy_version 27750 (0.0008) [2023-10-08 08:55:12,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56688640. Throughput: 0: 1843.3, 1: 1823.9. Samples: 14178540. Policy #0 lag: (min: 18.0, avg: 24.1, max: 50.0) [2023-10-08 08:55:12,015][52710] Avg episode reward: [(0, '27.010'), (1, '29.200')] [2023-10-08 08:55:12,109][53852] Updated weights for policy 0, policy_version 27760 (0.0009) [2023-10-08 08:55:12,476][53852] Updated weights for policy 0, policy_version 27770 (0.0009) [2023-10-08 08:55:14,326][53885] Updated weights for policy 1, policy_version 27622 (0.0008) [2023-10-08 08:55:14,689][53885] Updated weights for policy 1, policy_version 27632 (0.0009) [2023-10-08 08:55:15,063][53885] Updated weights for policy 1, policy_version 27642 (0.0009) [2023-10-08 08:55:16,010][53852] Updated weights for policy 0, policy_version 27780 (0.0007) [2023-10-08 08:55:16,394][53852] Updated weights for policy 0, policy_version 27790 (0.0009) [2023-10-08 08:55:16,760][53852] Updated weights for policy 0, policy_version 27800 (0.0009) [2023-10-08 08:55:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 56754176. Throughput: 0: 1845.1, 1: 1823.7. Samples: 14200104. Policy #0 lag: (min: 18.0, avg: 24.1, max: 50.0) [2023-10-08 08:55:17,015][52710] Avg episode reward: [(0, '29.090'), (1, '28.100')] [2023-10-08 08:55:17,048][53500] Saving new best policy, reward=29.090! [2023-10-08 08:55:18,603][53885] Updated weights for policy 1, policy_version 27652 (0.0009) [2023-10-08 08:55:18,968][53885] Updated weights for policy 1, policy_version 27662 (0.0011) [2023-10-08 08:55:19,339][53885] Updated weights for policy 1, policy_version 27672 (0.0011) [2023-10-08 08:55:20,507][53852] Updated weights for policy 0, policy_version 27810 (0.0008) [2023-10-08 08:55:20,883][53852] Updated weights for policy 0, policy_version 27820 (0.0009) [2023-10-08 08:55:21,249][53852] Updated weights for policy 0, policy_version 27830 (0.0008) [2023-10-08 08:55:21,624][53852] Updated weights for policy 0, policy_version 27840 (0.0008) [2023-10-08 08:55:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 56852480. Throughput: 0: 1824.5, 1: 1825.0. Samples: 14221694. Policy #0 lag: (min: 9.0, avg: 22.5, max: 41.0) [2023-10-08 08:55:22,016][52710] Avg episode reward: [(0, '25.680'), (1, '24.630')] [2023-10-08 08:55:23,022][53885] Updated weights for policy 1, policy_version 27682 (0.0010) [2023-10-08 08:55:23,398][53885] Updated weights for policy 1, policy_version 27692 (0.0009) [2023-10-08 08:55:23,749][53885] Updated weights for policy 1, policy_version 27702 (0.0010) [2023-10-08 08:55:24,117][53885] Updated weights for policy 1, policy_version 27712 (0.0009) [2023-10-08 08:55:25,157][53852] Updated weights for policy 0, policy_version 27850 (0.0008) [2023-10-08 08:55:25,527][53852] Updated weights for policy 0, policy_version 27860 (0.0009) [2023-10-08 08:55:25,889][53852] Updated weights for policy 0, policy_version 27870 (0.0010) [2023-10-08 08:55:27,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 56918016. Throughput: 0: 1841.2, 1: 1821.6. Samples: 14232950. Policy #0 lag: (min: 9.0, avg: 22.5, max: 41.0) [2023-10-08 08:55:27,015][52710] Avg episode reward: [(0, '27.140'), (1, '18.540')] [2023-10-08 08:55:27,716][53885] Updated weights for policy 1, policy_version 27722 (0.0007) [2023-10-08 08:55:28,086][53885] Updated weights for policy 1, policy_version 27732 (0.0010) [2023-10-08 08:55:28,455][53885] Updated weights for policy 1, policy_version 27742 (0.0008) [2023-10-08 08:55:29,473][53852] Updated weights for policy 0, policy_version 27880 (0.0008) [2023-10-08 08:55:29,842][53852] Updated weights for policy 0, policy_version 27890 (0.0008) [2023-10-08 08:55:30,219][53852] Updated weights for policy 0, policy_version 27900 (0.0008) [2023-10-08 08:55:31,983][53885] Updated weights for policy 1, policy_version 27752 (0.0008) [2023-10-08 08:55:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 56983552. Throughput: 0: 1824.5, 1: 1823.4. Samples: 14254656. Policy #0 lag: (min: 9.0, avg: 22.5, max: 41.0) [2023-10-08 08:55:32,015][52710] Avg episode reward: [(0, '28.270'), (1, '15.360')] [2023-10-08 08:55:32,356][53885] Updated weights for policy 1, policy_version 27762 (0.0008) [2023-10-08 08:55:32,727][53885] Updated weights for policy 1, policy_version 27772 (0.0010) [2023-10-08 08:55:33,699][53852] Updated weights for policy 0, policy_version 27910 (0.0008) [2023-10-08 08:55:34,061][53852] Updated weights for policy 0, policy_version 27920 (0.0010) [2023-10-08 08:55:34,435][53852] Updated weights for policy 0, policy_version 27930 (0.0008) [2023-10-08 08:55:36,439][53885] Updated weights for policy 1, policy_version 27782 (0.0008) [2023-10-08 08:55:36,801][53885] Updated weights for policy 1, policy_version 27792 (0.0008) [2023-10-08 08:55:37,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57049088. Throughput: 0: 1852.0, 1: 1810.3. Samples: 14277220. Policy #0 lag: (min: 9.0, avg: 22.5, max: 41.0) [2023-10-08 08:55:37,016][52710] Avg episode reward: [(0, '27.460'), (1, '20.710')] [2023-10-08 08:55:37,178][53885] Updated weights for policy 1, policy_version 27802 (0.0008) [2023-10-08 08:55:38,240][53852] Updated weights for policy 0, policy_version 27940 (0.0008) [2023-10-08 08:55:38,627][53852] Updated weights for policy 0, policy_version 27950 (0.0007) [2023-10-08 08:55:38,998][53852] Updated weights for policy 0, policy_version 27960 (0.0009) [2023-10-08 08:55:40,954][53885] Updated weights for policy 1, policy_version 27812 (0.0009) [2023-10-08 08:55:41,323][53885] Updated weights for policy 1, policy_version 27822 (0.0010) [2023-10-08 08:55:41,697][53885] Updated weights for policy 1, policy_version 27832 (0.0008) [2023-10-08 08:55:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 57147392. Throughput: 0: 1831.3, 1: 1814.1. Samples: 14287454. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 08:55:42,016][52710] Avg episode reward: [(0, '27.100'), (1, '22.220')] [2023-10-08 08:55:42,600][53852] Updated weights for policy 0, policy_version 27970 (0.0008) [2023-10-08 08:55:42,971][53852] Updated weights for policy 0, policy_version 27980 (0.0009) [2023-10-08 08:55:43,344][53852] Updated weights for policy 0, policy_version 27990 (0.0011) [2023-10-08 08:55:43,721][53852] Updated weights for policy 0, policy_version 28000 (0.0009) [2023-10-08 08:55:45,478][53885] Updated weights for policy 1, policy_version 27842 (0.0008) [2023-10-08 08:55:45,845][53885] Updated weights for policy 1, policy_version 27852 (0.0007) [2023-10-08 08:55:46,218][53885] Updated weights for policy 1, policy_version 27862 (0.0007) [2023-10-08 08:55:46,587][53885] Updated weights for policy 1, policy_version 27872 (0.0008) [2023-10-08 08:55:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57212928. Throughput: 0: 1844.0, 1: 1817.8. Samples: 14310256. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 08:55:47,016][52710] Avg episode reward: [(0, '26.690'), (1, '25.570')] [2023-10-08 08:55:47,466][53852] Updated weights for policy 0, policy_version 28010 (0.0010) [2023-10-08 08:55:47,832][53852] Updated weights for policy 0, policy_version 28020 (0.0007) [2023-10-08 08:55:48,212][53852] Updated weights for policy 0, policy_version 28030 (0.0009) [2023-10-08 08:55:50,172][53885] Updated weights for policy 1, policy_version 27882 (0.0009) [2023-10-08 08:55:50,537][53885] Updated weights for policy 1, policy_version 27892 (0.0010) [2023-10-08 08:55:50,913][53885] Updated weights for policy 1, policy_version 27902 (0.0007) [2023-10-08 08:55:51,608][53852] Updated weights for policy 0, policy_version 28040 (0.0010) [2023-10-08 08:55:51,990][53852] Updated weights for policy 0, policy_version 28050 (0.0010) [2023-10-08 08:55:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57278464. Throughput: 0: 1843.4, 1: 1820.5. Samples: 14332056. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 08:55:52,016][52710] Avg episode reward: [(0, '25.740'), (1, '28.040')] [2023-10-08 08:55:52,352][53852] Updated weights for policy 0, policy_version 28060 (0.0008) [2023-10-08 08:55:54,589][53885] Updated weights for policy 1, policy_version 27912 (0.0008) [2023-10-08 08:55:54,966][53885] Updated weights for policy 1, policy_version 27922 (0.0008) [2023-10-08 08:55:55,326][53885] Updated weights for policy 1, policy_version 27932 (0.0010) [2023-10-08 08:55:56,061][53852] Updated weights for policy 0, policy_version 28070 (0.0008) [2023-10-08 08:55:56,424][53852] Updated weights for policy 0, policy_version 28080 (0.0007) [2023-10-08 08:55:56,800][53852] Updated weights for policy 0, policy_version 28090 (0.0008) [2023-10-08 08:55:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 57344000. Throughput: 0: 1850.2, 1: 1816.5. Samples: 14343542. Policy #0 lag: (min: 31.0, avg: 35.4, max: 63.0) [2023-10-08 08:55:57,016][52710] Avg episode reward: [(0, '28.170'), (1, '28.110')] [2023-10-08 08:55:58,886][53885] Updated weights for policy 1, policy_version 27942 (0.0010) [2023-10-08 08:55:59,259][53885] Updated weights for policy 1, policy_version 27952 (0.0007) [2023-10-08 08:55:59,626][53885] Updated weights for policy 1, policy_version 27962 (0.0009) [2023-10-08 08:56:00,513][53852] Updated weights for policy 0, policy_version 28100 (0.0009) [2023-10-08 08:56:00,881][53852] Updated weights for policy 0, policy_version 28110 (0.0010) [2023-10-08 08:56:01,241][53852] Updated weights for policy 0, policy_version 28120 (0.0011) [2023-10-08 08:56:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57442304. Throughput: 0: 1843.1, 1: 1834.2. Samples: 14365582. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:56:02,016][52710] Avg episode reward: [(0, '30.080'), (1, '28.250')] [2023-10-08 08:56:02,017][53500] Saving new best policy, reward=30.080! [2023-10-08 08:56:03,270][53885] Updated weights for policy 1, policy_version 27972 (0.0008) [2023-10-08 08:56:03,645][53885] Updated weights for policy 1, policy_version 27982 (0.0011) [2023-10-08 08:56:04,011][53885] Updated weights for policy 1, policy_version 27992 (0.0011) [2023-10-08 08:56:05,092][53852] Updated weights for policy 0, policy_version 28130 (0.0008) [2023-10-08 08:56:05,470][53852] Updated weights for policy 0, policy_version 28140 (0.0007) [2023-10-08 08:56:05,830][53852] Updated weights for policy 0, policy_version 28150 (0.0011) [2023-10-08 08:56:06,195][53852] Updated weights for policy 0, policy_version 28160 (0.0010) [2023-10-08 08:56:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 57507840. Throughput: 0: 1847.0, 1: 1827.2. Samples: 14387032. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:56:07,016][52710] Avg episode reward: [(0, '29.070'), (1, '29.680')] [2023-10-08 08:56:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000028160_28835840.pth... [2023-10-08 08:56:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000028000_28672000.pth... [2023-10-08 08:56:07,056][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000026432_27066368.pth [2023-10-08 08:56:07,066][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000026304_26935296.pth [2023-10-08 08:56:07,847][53885] Updated weights for policy 1, policy_version 28002 (0.0007) [2023-10-08 08:56:08,217][53885] Updated weights for policy 1, policy_version 28012 (0.0009) [2023-10-08 08:56:08,587][53885] Updated weights for policy 1, policy_version 28022 (0.0008) [2023-10-08 08:56:08,950][53885] Updated weights for policy 1, policy_version 28032 (0.0007) [2023-10-08 08:56:09,763][53852] Updated weights for policy 0, policy_version 28170 (0.0010) [2023-10-08 08:56:10,139][53852] Updated weights for policy 0, policy_version 28180 (0.0009) [2023-10-08 08:56:10,502][53852] Updated weights for policy 0, policy_version 28190 (0.0008) [2023-10-08 08:56:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57573376. Throughput: 0: 1846.0, 1: 1830.2. Samples: 14398380. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:56:12,015][52710] Avg episode reward: [(0, '28.360'), (1, '27.820')] [2023-10-08 08:56:12,605][53885] Updated weights for policy 1, policy_version 28042 (0.0007) [2023-10-08 08:56:12,969][53885] Updated weights for policy 1, policy_version 28052 (0.0007) [2023-10-08 08:56:13,335][53885] Updated weights for policy 1, policy_version 28062 (0.0007) [2023-10-08 08:56:14,040][53852] Updated weights for policy 0, policy_version 28200 (0.0009) [2023-10-08 08:56:14,405][53852] Updated weights for policy 0, policy_version 28210 (0.0007) [2023-10-08 08:56:14,770][53852] Updated weights for policy 0, policy_version 28220 (0.0007) [2023-10-08 08:56:16,966][53885] Updated weights for policy 1, policy_version 28072 (0.0007) [2023-10-08 08:56:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57638912. Throughput: 0: 1848.8, 1: 1833.2. Samples: 14420348. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:56:17,016][52710] Avg episode reward: [(0, '29.270'), (1, '28.750')] [2023-10-08 08:56:17,327][53885] Updated weights for policy 1, policy_version 28082 (0.0008) [2023-10-08 08:56:17,702][53885] Updated weights for policy 1, policy_version 28092 (0.0007) [2023-10-08 08:56:18,209][53852] Updated weights for policy 0, policy_version 28230 (0.0007) [2023-10-08 08:56:18,581][53852] Updated weights for policy 0, policy_version 28240 (0.0007) [2023-10-08 08:56:18,961][53852] Updated weights for policy 0, policy_version 28250 (0.0009) [2023-10-08 08:56:21,394][53885] Updated weights for policy 1, policy_version 28102 (0.0007) [2023-10-08 08:56:21,758][53885] Updated weights for policy 1, policy_version 28112 (0.0009) [2023-10-08 08:56:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 57704448. Throughput: 0: 1851.2, 1: 1835.0. Samples: 14443100. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:56:22,016][52710] Avg episode reward: [(0, '27.310'), (1, '28.110')] [2023-10-08 08:56:22,125][53885] Updated weights for policy 1, policy_version 28122 (0.0008) [2023-10-08 08:56:22,608][53852] Updated weights for policy 0, policy_version 28260 (0.0008) [2023-10-08 08:56:23,004][53852] Updated weights for policy 0, policy_version 28270 (0.0007) [2023-10-08 08:56:23,378][53852] Updated weights for policy 0, policy_version 28280 (0.0007) [2023-10-08 08:56:25,742][53885] Updated weights for policy 1, policy_version 28132 (0.0008) [2023-10-08 08:56:26,107][53885] Updated weights for policy 1, policy_version 28142 (0.0007) [2023-10-08 08:56:26,484][53885] Updated weights for policy 1, policy_version 28152 (0.0007) [2023-10-08 08:56:26,915][53852] Updated weights for policy 0, policy_version 28290 (0.0007) [2023-10-08 08:56:27,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57802752. Throughput: 0: 1851.2, 1: 1843.0. Samples: 14453690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:56:27,015][52710] Avg episode reward: [(0, '28.140'), (1, '29.710')] [2023-10-08 08:56:27,287][53852] Updated weights for policy 0, policy_version 28300 (0.0009) [2023-10-08 08:56:27,654][53852] Updated weights for policy 0, policy_version 28310 (0.0008) [2023-10-08 08:56:28,025][53852] Updated weights for policy 0, policy_version 28320 (0.0007) [2023-10-08 08:56:29,860][53885] Updated weights for policy 1, policy_version 28162 (0.0007) [2023-10-08 08:56:30,241][53885] Updated weights for policy 1, policy_version 28172 (0.0008) [2023-10-08 08:56:30,610][53885] Updated weights for policy 1, policy_version 28182 (0.0007) [2023-10-08 08:56:30,969][53885] Updated weights for policy 1, policy_version 28192 (0.0009) [2023-10-08 08:56:31,769][53852] Updated weights for policy 0, policy_version 28330 (0.0008) [2023-10-08 08:56:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57868288. Throughput: 0: 1849.7, 1: 1829.6. Samples: 14475822. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:56:32,016][52710] Avg episode reward: [(0, '29.370'), (1, '31.060')] [2023-10-08 08:56:32,146][53852] Updated weights for policy 0, policy_version 28340 (0.0008) [2023-10-08 08:56:32,507][53852] Updated weights for policy 0, policy_version 28350 (0.0010) [2023-10-08 08:56:34,561][53885] Updated weights for policy 1, policy_version 28202 (0.0010) [2023-10-08 08:56:34,939][53885] Updated weights for policy 1, policy_version 28212 (0.0009) [2023-10-08 08:56:35,302][53885] Updated weights for policy 1, policy_version 28222 (0.0008) [2023-10-08 08:56:36,177][53852] Updated weights for policy 0, policy_version 28360 (0.0008) [2023-10-08 08:56:36,544][53852] Updated weights for policy 0, policy_version 28370 (0.0007) [2023-10-08 08:56:36,913][53852] Updated weights for policy 0, policy_version 28380 (0.0008) [2023-10-08 08:56:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 57933824. Throughput: 0: 1829.7, 1: 1844.3. Samples: 14497388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:56:37,016][52710] Avg episode reward: [(0, '28.290'), (1, '28.560')] [2023-10-08 08:56:39,108][53885] Updated weights for policy 1, policy_version 28232 (0.0008) [2023-10-08 08:56:39,463][53885] Updated weights for policy 1, policy_version 28242 (0.0007) [2023-10-08 08:56:39,832][53885] Updated weights for policy 1, policy_version 28252 (0.0008) [2023-10-08 08:56:40,711][53852] Updated weights for policy 0, policy_version 28390 (0.0008) [2023-10-08 08:56:41,073][53852] Updated weights for policy 0, policy_version 28400 (0.0010) [2023-10-08 08:56:41,442][53852] Updated weights for policy 0, policy_version 28410 (0.0010) [2023-10-08 08:56:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 58032128. Throughput: 0: 1835.9, 1: 1829.6. Samples: 14508488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:56:42,016][52710] Avg episode reward: [(0, '28.100'), (1, '29.310')] [2023-10-08 08:56:43,552][53885] Updated weights for policy 1, policy_version 28262 (0.0008) [2023-10-08 08:56:43,922][53885] Updated weights for policy 1, policy_version 28272 (0.0012) [2023-10-08 08:56:44,294][53885] Updated weights for policy 1, policy_version 28282 (0.0008) [2023-10-08 08:56:45,130][53852] Updated weights for policy 0, policy_version 28420 (0.0010) [2023-10-08 08:56:45,501][53852] Updated weights for policy 0, policy_version 28430 (0.0008) [2023-10-08 08:56:45,874][53852] Updated weights for policy 0, policy_version 28440 (0.0007) [2023-10-08 08:56:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58097664. Throughput: 0: 1826.4, 1: 1837.9. Samples: 14530472. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 08:56:47,016][52710] Avg episode reward: [(0, '28.450'), (1, '30.360')] [2023-10-08 08:56:48,011][53885] Updated weights for policy 1, policy_version 28292 (0.0009) [2023-10-08 08:56:48,380][53885] Updated weights for policy 1, policy_version 28302 (0.0011) [2023-10-08 08:56:48,748][53885] Updated weights for policy 1, policy_version 28312 (0.0009) [2023-10-08 08:56:49,402][53852] Updated weights for policy 0, policy_version 28450 (0.0007) [2023-10-08 08:56:49,774][53852] Updated weights for policy 0, policy_version 28460 (0.0007) [2023-10-08 08:56:50,144][53852] Updated weights for policy 0, policy_version 28470 (0.0008) [2023-10-08 08:56:50,510][53852] Updated weights for policy 0, policy_version 28480 (0.0010) [2023-10-08 08:56:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58163200. Throughput: 0: 1842.9, 1: 1839.5. Samples: 14552742. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 08:56:52,016][52710] Avg episode reward: [(0, '26.770'), (1, '28.330')] [2023-10-08 08:56:52,410][53885] Updated weights for policy 1, policy_version 28322 (0.0008) [2023-10-08 08:56:52,784][53885] Updated weights for policy 1, policy_version 28332 (0.0008) [2023-10-08 08:56:53,152][53885] Updated weights for policy 1, policy_version 28342 (0.0007) [2023-10-08 08:56:53,516][53885] Updated weights for policy 1, policy_version 28352 (0.0009) [2023-10-08 08:56:54,180][53852] Updated weights for policy 0, policy_version 28490 (0.0009) [2023-10-08 08:56:54,551][53852] Updated weights for policy 0, policy_version 28500 (0.0008) [2023-10-08 08:56:54,919][53852] Updated weights for policy 0, policy_version 28510 (0.0009) [2023-10-08 08:56:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 58228736. Throughput: 0: 1827.2, 1: 1843.2. Samples: 14563546. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 08:56:57,016][52710] Avg episode reward: [(0, '26.210'), (1, '29.080')] [2023-10-08 08:56:57,248][53885] Updated weights for policy 1, policy_version 28362 (0.0009) [2023-10-08 08:56:57,623][53885] Updated weights for policy 1, policy_version 28372 (0.0008) [2023-10-08 08:56:57,995][53885] Updated weights for policy 1, policy_version 28382 (0.0011) [2023-10-08 08:56:58,496][53852] Updated weights for policy 0, policy_version 28520 (0.0008) [2023-10-08 08:56:58,871][53852] Updated weights for policy 0, policy_version 28530 (0.0010) [2023-10-08 08:56:59,243][53852] Updated weights for policy 0, policy_version 28540 (0.0009) [2023-10-08 08:57:01,536][53885] Updated weights for policy 1, policy_version 28392 (0.0008) [2023-10-08 08:57:01,892][53885] Updated weights for policy 1, policy_version 28402 (0.0009) [2023-10-08 08:57:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 58294272. Throughput: 0: 1841.8, 1: 1839.3. Samples: 14586000. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 08:57:02,016][52710] Avg episode reward: [(0, '23.590'), (1, '28.050')] [2023-10-08 08:57:02,266][53885] Updated weights for policy 1, policy_version 28412 (0.0007) [2023-10-08 08:57:02,917][53852] Updated weights for policy 0, policy_version 28550 (0.0008) [2023-10-08 08:57:03,290][53852] Updated weights for policy 0, policy_version 28560 (0.0008) [2023-10-08 08:57:03,652][53852] Updated weights for policy 0, policy_version 28570 (0.0008) [2023-10-08 08:57:05,906][53885] Updated weights for policy 1, policy_version 28422 (0.0009) [2023-10-08 08:57:06,266][53885] Updated weights for policy 1, policy_version 28432 (0.0008) [2023-10-08 08:57:06,635][53885] Updated weights for policy 1, policy_version 28442 (0.0008) [2023-10-08 08:57:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 58392576. Throughput: 0: 1838.5, 1: 1827.0. Samples: 14608046. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 08:57:07,016][52710] Avg episode reward: [(0, '23.610'), (1, '29.680')] [2023-10-08 08:57:07,129][53852] Updated weights for policy 0, policy_version 28580 (0.0007) [2023-10-08 08:57:07,495][53852] Updated weights for policy 0, policy_version 28590 (0.0007) [2023-10-08 08:57:07,866][53852] Updated weights for policy 0, policy_version 28600 (0.0007) [2023-10-08 08:57:10,435][53885] Updated weights for policy 1, policy_version 28452 (0.0009) [2023-10-08 08:57:10,798][53885] Updated weights for policy 1, policy_version 28462 (0.0007) [2023-10-08 08:57:11,166][53885] Updated weights for policy 1, policy_version 28472 (0.0008) [2023-10-08 08:57:11,431][53852] Updated weights for policy 0, policy_version 28610 (0.0009) [2023-10-08 08:57:11,804][53852] Updated weights for policy 0, policy_version 28620 (0.0010) [2023-10-08 08:57:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58458112. Throughput: 0: 1843.5, 1: 1835.1. Samples: 14619226. Policy #0 lag: (min: 17.0, avg: 27.2, max: 49.0) [2023-10-08 08:57:12,016][52710] Avg episode reward: [(0, '24.220'), (1, '30.080')] [2023-10-08 08:57:12,172][53852] Updated weights for policy 0, policy_version 28630 (0.0009) [2023-10-08 08:57:12,542][53852] Updated weights for policy 0, policy_version 28640 (0.0007) [2023-10-08 08:57:14,891][53885] Updated weights for policy 1, policy_version 28482 (0.0009) [2023-10-08 08:57:15,269][53885] Updated weights for policy 1, policy_version 28492 (0.0009) [2023-10-08 08:57:15,638][53885] Updated weights for policy 1, policy_version 28502 (0.0008) [2023-10-08 08:57:15,997][53885] Updated weights for policy 1, policy_version 28512 (0.0008) [2023-10-08 08:57:16,356][53852] Updated weights for policy 0, policy_version 28650 (0.0007) [2023-10-08 08:57:16,725][53852] Updated weights for policy 0, policy_version 28660 (0.0007) [2023-10-08 08:57:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58523648. Throughput: 0: 1846.8, 1: 1829.1. Samples: 14641236. Policy #0 lag: (min: 17.0, avg: 27.2, max: 49.0) [2023-10-08 08:57:17,016][52710] Avg episode reward: [(0, '25.280'), (1, '29.010')] [2023-10-08 08:57:17,084][53852] Updated weights for policy 0, policy_version 28670 (0.0007) [2023-10-08 08:57:19,559][53885] Updated weights for policy 1, policy_version 28522 (0.0010) [2023-10-08 08:57:19,927][53885] Updated weights for policy 1, policy_version 28532 (0.0007) [2023-10-08 08:57:20,296][53885] Updated weights for policy 1, policy_version 28542 (0.0007) [2023-10-08 08:57:20,724][53852] Updated weights for policy 0, policy_version 28680 (0.0007) [2023-10-08 08:57:21,089][53852] Updated weights for policy 0, policy_version 28690 (0.0010) [2023-10-08 08:57:21,469][53852] Updated weights for policy 0, policy_version 28700 (0.0009) [2023-10-08 08:57:22,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 58621952. Throughput: 0: 1835.5, 1: 1833.4. Samples: 14662492. Policy #0 lag: (min: 17.0, avg: 27.2, max: 49.0) [2023-10-08 08:57:22,016][52710] Avg episode reward: [(0, '28.120'), (1, '29.740')] [2023-10-08 08:57:23,876][53885] Updated weights for policy 1, policy_version 28552 (0.0008) [2023-10-08 08:57:24,252][53885] Updated weights for policy 1, policy_version 28562 (0.0010) [2023-10-08 08:57:24,613][53885] Updated weights for policy 1, policy_version 28572 (0.0010) [2023-10-08 08:57:25,105][53852] Updated weights for policy 0, policy_version 28710 (0.0009) [2023-10-08 08:57:25,472][53852] Updated weights for policy 0, policy_version 28720 (0.0010) [2023-10-08 08:57:25,850][53852] Updated weights for policy 0, policy_version 28730 (0.0008) [2023-10-08 08:57:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 58687488. Throughput: 0: 1851.5, 1: 1831.6. Samples: 14674228. Policy #0 lag: (min: 17.0, avg: 27.2, max: 49.0) [2023-10-08 08:57:27,016][52710] Avg episode reward: [(0, '28.390'), (1, '29.380')] [2023-10-08 08:57:28,161][53885] Updated weights for policy 1, policy_version 28582 (0.0009) [2023-10-08 08:57:28,534][53885] Updated weights for policy 1, policy_version 28592 (0.0008) [2023-10-08 08:57:28,910][53885] Updated weights for policy 1, policy_version 28602 (0.0008) [2023-10-08 08:57:29,618][53852] Updated weights for policy 0, policy_version 28740 (0.0010) [2023-10-08 08:57:29,987][53852] Updated weights for policy 0, policy_version 28750 (0.0009) [2023-10-08 08:57:30,355][53852] Updated weights for policy 0, policy_version 28760 (0.0008) [2023-10-08 08:57:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58753024. Throughput: 0: 1834.8, 1: 1842.4. Samples: 14695946. Policy #0 lag: (min: 3.0, avg: 6.2, max: 35.0) [2023-10-08 08:57:32,015][52710] Avg episode reward: [(0, '28.160'), (1, '30.350')] [2023-10-08 08:57:32,406][53885] Updated weights for policy 1, policy_version 28612 (0.0009) [2023-10-08 08:57:32,766][53885] Updated weights for policy 1, policy_version 28622 (0.0009) [2023-10-08 08:57:33,133][53885] Updated weights for policy 1, policy_version 28632 (0.0011) [2023-10-08 08:57:33,808][53852] Updated weights for policy 0, policy_version 28770 (0.0008) [2023-10-08 08:57:34,178][53852] Updated weights for policy 0, policy_version 28780 (0.0009) [2023-10-08 08:57:34,550][53852] Updated weights for policy 0, policy_version 28790 (0.0011) [2023-10-08 08:57:34,918][53852] Updated weights for policy 0, policy_version 28800 (0.0010) [2023-10-08 08:57:36,709][53885] Updated weights for policy 1, policy_version 28642 (0.0010) [2023-10-08 08:57:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58818560. Throughput: 0: 1847.3, 1: 1850.7. Samples: 14719152. Policy #0 lag: (min: 3.0, avg: 6.2, max: 35.0) [2023-10-08 08:57:37,015][52710] Avg episode reward: [(0, '30.410'), (1, '30.390')] [2023-10-08 08:57:37,022][53500] Saving new best policy, reward=30.410! [2023-10-08 08:57:37,082][53885] Updated weights for policy 1, policy_version 28652 (0.0007) [2023-10-08 08:57:37,451][53885] Updated weights for policy 1, policy_version 28662 (0.0007) [2023-10-08 08:57:37,813][53885] Updated weights for policy 1, policy_version 28672 (0.0007) [2023-10-08 08:57:38,627][53852] Updated weights for policy 0, policy_version 28810 (0.0007) [2023-10-08 08:57:38,989][53852] Updated weights for policy 0, policy_version 28820 (0.0008) [2023-10-08 08:57:39,366][53852] Updated weights for policy 0, policy_version 28830 (0.0009) [2023-10-08 08:57:41,470][53885] Updated weights for policy 1, policy_version 28682 (0.0007) [2023-10-08 08:57:41,842][53885] Updated weights for policy 1, policy_version 28692 (0.0009) [2023-10-08 08:57:42,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 58884096. Throughput: 0: 1829.9, 1: 1852.3. Samples: 14729248. Policy #0 lag: (min: 3.0, avg: 6.2, max: 35.0) [2023-10-08 08:57:42,016][52710] Avg episode reward: [(0, '30.400'), (1, '32.240')] [2023-10-08 08:57:42,206][53885] Updated weights for policy 1, policy_version 28702 (0.0008) [2023-10-08 08:57:43,172][53852] Updated weights for policy 0, policy_version 28840 (0.0008) [2023-10-08 08:57:43,538][53852] Updated weights for policy 0, policy_version 28850 (0.0009) [2023-10-08 08:57:43,910][53852] Updated weights for policy 0, policy_version 28860 (0.0010) [2023-10-08 08:57:45,916][53885] Updated weights for policy 1, policy_version 28712 (0.0009) [2023-10-08 08:57:46,287][53885] Updated weights for policy 1, policy_version 28722 (0.0009) [2023-10-08 08:57:46,652][53885] Updated weights for policy 1, policy_version 28732 (0.0007) [2023-10-08 08:57:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 58982400. Throughput: 0: 1835.9, 1: 1849.3. Samples: 14751836. Policy #0 lag: (min: 3.0, avg: 6.2, max: 35.0) [2023-10-08 08:57:47,016][52710] Avg episode reward: [(0, '28.370'), (1, '30.690')] [2023-10-08 08:57:47,545][53852] Updated weights for policy 0, policy_version 28870 (0.0009) [2023-10-08 08:57:47,924][53852] Updated weights for policy 0, policy_version 28880 (0.0009) [2023-10-08 08:57:48,286][53852] Updated weights for policy 0, policy_version 28890 (0.0008) [2023-10-08 08:57:50,288][53885] Updated weights for policy 1, policy_version 28742 (0.0010) [2023-10-08 08:57:50,656][53885] Updated weights for policy 1, policy_version 28752 (0.0009) [2023-10-08 08:57:51,019][53885] Updated weights for policy 1, policy_version 28762 (0.0007) [2023-10-08 08:57:51,968][53852] Updated weights for policy 0, policy_version 28900 (0.0007) [2023-10-08 08:57:52,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59047936. Throughput: 0: 1837.6, 1: 1838.8. Samples: 14773486. Policy #0 lag: (min: 3.0, avg: 6.2, max: 35.0) [2023-10-08 08:57:52,016][52710] Avg episode reward: [(0, '30.000'), (1, '29.230')] [2023-10-08 08:57:52,341][53852] Updated weights for policy 0, policy_version 28910 (0.0009) [2023-10-08 08:57:52,712][53852] Updated weights for policy 0, policy_version 28920 (0.0008) [2023-10-08 08:57:54,827][53885] Updated weights for policy 1, policy_version 28772 (0.0007) [2023-10-08 08:57:55,199][53885] Updated weights for policy 1, policy_version 28782 (0.0007) [2023-10-08 08:57:55,570][53885] Updated weights for policy 1, policy_version 28792 (0.0007) [2023-10-08 08:57:56,187][53852] Updated weights for policy 0, policy_version 28930 (0.0007) [2023-10-08 08:57:56,561][53852] Updated weights for policy 0, policy_version 28940 (0.0007) [2023-10-08 08:57:56,929][53852] Updated weights for policy 0, policy_version 28950 (0.0007) [2023-10-08 08:57:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 59113472. Throughput: 0: 1839.5, 1: 1848.6. Samples: 14785192. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 08:57:57,016][52710] Avg episode reward: [(0, '29.930'), (1, '30.420')] [2023-10-08 08:57:57,294][53852] Updated weights for policy 0, policy_version 28960 (0.0007) [2023-10-08 08:57:59,129][53885] Updated weights for policy 1, policy_version 28802 (0.0008) [2023-10-08 08:57:59,507][53885] Updated weights for policy 1, policy_version 28812 (0.0007) [2023-10-08 08:57:59,877][53885] Updated weights for policy 1, policy_version 28822 (0.0009) [2023-10-08 08:58:00,242][53885] Updated weights for policy 1, policy_version 28832 (0.0008) [2023-10-08 08:58:01,043][53852] Updated weights for policy 0, policy_version 28970 (0.0007) [2023-10-08 08:58:01,415][53852] Updated weights for policy 0, policy_version 28980 (0.0007) [2023-10-08 08:58:01,790][53852] Updated weights for policy 0, policy_version 28990 (0.0009) [2023-10-08 08:58:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 59211776. Throughput: 0: 1840.2, 1: 1843.7. Samples: 14807010. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 08:58:02,016][52710] Avg episode reward: [(0, '28.870'), (1, '25.550')] [2023-10-08 08:58:03,918][53885] Updated weights for policy 1, policy_version 28842 (0.0009) [2023-10-08 08:58:04,277][53885] Updated weights for policy 1, policy_version 28852 (0.0009) [2023-10-08 08:58:04,655][53885] Updated weights for policy 1, policy_version 28862 (0.0010) [2023-10-08 08:58:05,199][53852] Updated weights for policy 0, policy_version 29000 (0.0009) [2023-10-08 08:58:05,565][53852] Updated weights for policy 0, policy_version 29010 (0.0010) [2023-10-08 08:58:05,939][53852] Updated weights for policy 0, policy_version 29020 (0.0007) [2023-10-08 08:58:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 59277312. Throughput: 0: 1838.8, 1: 1851.8. Samples: 14828570. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 08:58:07,016][52710] Avg episode reward: [(0, '27.940'), (1, '30.450')] [2023-10-08 08:58:07,026][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000028864_29556736.pth... [2023-10-08 08:58:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000029024_29720576.pth... [2023-10-08 08:58:07,063][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000027168_27820032.pth [2023-10-08 08:58:07,066][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000027296_27951104.pth [2023-10-08 08:58:08,317][53885] Updated weights for policy 1, policy_version 28872 (0.0009) [2023-10-08 08:58:08,692][53885] Updated weights for policy 1, policy_version 28882 (0.0009) [2023-10-08 08:58:09,062][53885] Updated weights for policy 1, policy_version 28892 (0.0009) [2023-10-08 08:58:09,590][53852] Updated weights for policy 0, policy_version 29030 (0.0007) [2023-10-08 08:58:09,954][53852] Updated weights for policy 0, policy_version 29040 (0.0009) [2023-10-08 08:58:10,334][53852] Updated weights for policy 0, policy_version 29050 (0.0010) [2023-10-08 08:58:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59342848. Throughput: 0: 1839.4, 1: 1842.5. Samples: 14839912. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 08:58:12,016][52710] Avg episode reward: [(0, '30.370'), (1, '30.940')] [2023-10-08 08:58:12,771][53885] Updated weights for policy 1, policy_version 28902 (0.0010) [2023-10-08 08:58:13,142][53885] Updated weights for policy 1, policy_version 28912 (0.0009) [2023-10-08 08:58:13,520][53885] Updated weights for policy 1, policy_version 28922 (0.0008) [2023-10-08 08:58:14,010][53852] Updated weights for policy 0, policy_version 29060 (0.0008) [2023-10-08 08:58:14,387][53852] Updated weights for policy 0, policy_version 29070 (0.0007) [2023-10-08 08:58:14,747][53852] Updated weights for policy 0, policy_version 29080 (0.0008) [2023-10-08 08:58:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59408384. Throughput: 0: 1838.9, 1: 1844.6. Samples: 14861706. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:58:17,016][52710] Avg episode reward: [(0, '27.500'), (1, '28.400')] [2023-10-08 08:58:17,053][53885] Updated weights for policy 1, policy_version 28932 (0.0008) [2023-10-08 08:58:17,414][53885] Updated weights for policy 1, policy_version 28942 (0.0007) [2023-10-08 08:58:17,780][53885] Updated weights for policy 1, policy_version 28952 (0.0008) [2023-10-08 08:58:18,372][53852] Updated weights for policy 0, policy_version 29090 (0.0007) [2023-10-08 08:58:18,748][53852] Updated weights for policy 0, policy_version 29100 (0.0008) [2023-10-08 08:58:19,126][53852] Updated weights for policy 0, policy_version 29110 (0.0009) [2023-10-08 08:58:19,487][53852] Updated weights for policy 0, policy_version 29120 (0.0009) [2023-10-08 08:58:21,314][53885] Updated weights for policy 1, policy_version 28962 (0.0011) [2023-10-08 08:58:21,680][53885] Updated weights for policy 1, policy_version 28972 (0.0007) [2023-10-08 08:58:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 59473920. Throughput: 0: 1848.0, 1: 1827.4. Samples: 14884548. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:58:22,016][52710] Avg episode reward: [(0, '27.860'), (1, '29.680')] [2023-10-08 08:58:22,052][53885] Updated weights for policy 1, policy_version 28982 (0.0009) [2023-10-08 08:58:22,422][53885] Updated weights for policy 1, policy_version 28992 (0.0009) [2023-10-08 08:58:23,122][53852] Updated weights for policy 0, policy_version 29130 (0.0008) [2023-10-08 08:58:23,507][53852] Updated weights for policy 0, policy_version 29140 (0.0008) [2023-10-08 08:58:23,874][53852] Updated weights for policy 0, policy_version 29150 (0.0007) [2023-10-08 08:58:26,170][53885] Updated weights for policy 1, policy_version 29002 (0.0009) [2023-10-08 08:58:26,545][53885] Updated weights for policy 1, policy_version 29012 (0.0008) [2023-10-08 08:58:26,904][53885] Updated weights for policy 1, policy_version 29022 (0.0007) [2023-10-08 08:58:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59572224. Throughput: 0: 1850.3, 1: 1834.6. Samples: 14895068. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:58:27,016][52710] Avg episode reward: [(0, '29.640'), (1, '30.340')] [2023-10-08 08:58:27,575][53852] Updated weights for policy 0, policy_version 29160 (0.0008) [2023-10-08 08:58:27,943][53852] Updated weights for policy 0, policy_version 29170 (0.0008) [2023-10-08 08:58:28,321][53852] Updated weights for policy 0, policy_version 29180 (0.0008) [2023-10-08 08:58:30,735][53885] Updated weights for policy 1, policy_version 29032 (0.0009) [2023-10-08 08:58:31,111][53885] Updated weights for policy 1, policy_version 29042 (0.0009) [2023-10-08 08:58:31,477][53885] Updated weights for policy 1, policy_version 29052 (0.0009) [2023-10-08 08:58:31,816][53852] Updated weights for policy 0, policy_version 29190 (0.0007) [2023-10-08 08:58:32,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 59637760. Throughput: 0: 1864.7, 1: 1828.4. Samples: 14918026. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:58:32,016][52710] Avg episode reward: [(0, '27.890'), (1, '29.710')] [2023-10-08 08:58:32,194][53852] Updated weights for policy 0, policy_version 29200 (0.0009) [2023-10-08 08:58:32,565][53852] Updated weights for policy 0, policy_version 29210 (0.0008) [2023-10-08 08:58:35,019][53885] Updated weights for policy 1, policy_version 29062 (0.0009) [2023-10-08 08:58:35,388][53885] Updated weights for policy 1, policy_version 29072 (0.0008) [2023-10-08 08:58:35,769][53885] Updated weights for policy 1, policy_version 29082 (0.0009) [2023-10-08 08:58:36,046][53852] Updated weights for policy 0, policy_version 29220 (0.0008) [2023-10-08 08:58:36,409][53852] Updated weights for policy 0, policy_version 29230 (0.0007) [2023-10-08 08:58:36,779][53852] Updated weights for policy 0, policy_version 29240 (0.0007) [2023-10-08 08:58:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59703296. Throughput: 0: 1846.1, 1: 1834.0. Samples: 14939092. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 08:58:37,015][52710] Avg episode reward: [(0, '28.150'), (1, '29.760')] [2023-10-08 08:58:39,391][53885] Updated weights for policy 1, policy_version 29092 (0.0009) [2023-10-08 08:58:39,757][53885] Updated weights for policy 1, policy_version 29102 (0.0007) [2023-10-08 08:58:40,123][53885] Updated weights for policy 1, policy_version 29112 (0.0009) [2023-10-08 08:58:40,406][53852] Updated weights for policy 0, policy_version 29250 (0.0007) [2023-10-08 08:58:40,766][53852] Updated weights for policy 0, policy_version 29260 (0.0007) [2023-10-08 08:58:41,144][53852] Updated weights for policy 0, policy_version 29270 (0.0008) [2023-10-08 08:58:41,517][53852] Updated weights for policy 0, policy_version 29280 (0.0007) [2023-10-08 08:58:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 59801600. Throughput: 0: 1858.0, 1: 1823.5. Samples: 14950860. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:58:42,016][52710] Avg episode reward: [(0, '28.700'), (1, '29.610')] [2023-10-08 08:58:43,807][53885] Updated weights for policy 1, policy_version 29122 (0.0008) [2023-10-08 08:58:44,174][53885] Updated weights for policy 1, policy_version 29132 (0.0007) [2023-10-08 08:58:44,551][53885] Updated weights for policy 1, policy_version 29142 (0.0007) [2023-10-08 08:58:44,911][53885] Updated weights for policy 1, policy_version 29152 (0.0007) [2023-10-08 08:58:45,325][53852] Updated weights for policy 0, policy_version 29290 (0.0008) [2023-10-08 08:58:45,691][53852] Updated weights for policy 0, policy_version 29300 (0.0010) [2023-10-08 08:58:46,065][53852] Updated weights for policy 0, policy_version 29310 (0.0008) [2023-10-08 08:58:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 59867136. Throughput: 0: 1837.3, 1: 1827.3. Samples: 14971918. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:58:47,016][52710] Avg episode reward: [(0, '27.750'), (1, '30.460')] [2023-10-08 08:58:48,631][53885] Updated weights for policy 1, policy_version 29162 (0.0009) [2023-10-08 08:58:49,006][53885] Updated weights for policy 1, policy_version 29172 (0.0009) [2023-10-08 08:58:49,365][53885] Updated weights for policy 1, policy_version 29182 (0.0007) [2023-10-08 08:58:49,758][53852] Updated weights for policy 0, policy_version 29320 (0.0008) [2023-10-08 08:58:50,136][53852] Updated weights for policy 0, policy_version 29330 (0.0007) [2023-10-08 08:58:50,514][53852] Updated weights for policy 0, policy_version 29340 (0.0010) [2023-10-08 08:58:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59932672. Throughput: 0: 1848.5, 1: 1828.4. Samples: 14994030. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:58:52,016][52710] Avg episode reward: [(0, '26.940'), (1, '31.120')] [2023-10-08 08:58:52,873][53885] Updated weights for policy 1, policy_version 29192 (0.0008) [2023-10-08 08:58:53,233][53885] Updated weights for policy 1, policy_version 29202 (0.0008) [2023-10-08 08:58:53,603][53885] Updated weights for policy 1, policy_version 29212 (0.0008) [2023-10-08 08:58:53,954][53852] Updated weights for policy 0, policy_version 29350 (0.0009) [2023-10-08 08:58:54,327][53852] Updated weights for policy 0, policy_version 29360 (0.0010) [2023-10-08 08:58:54,700][53852] Updated weights for policy 0, policy_version 29370 (0.0008) [2023-10-08 08:58:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 59998208. Throughput: 0: 1834.7, 1: 1827.2. Samples: 15004694. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 08:58:57,015][52710] Avg episode reward: [(0, '28.570'), (1, '32.280')] [2023-10-08 08:58:57,405][53885] Updated weights for policy 1, policy_version 29222 (0.0008) [2023-10-08 08:58:57,774][53885] Updated weights for policy 1, policy_version 29232 (0.0007) [2023-10-08 08:58:58,136][53885] Updated weights for policy 1, policy_version 29242 (0.0009) [2023-10-08 08:58:58,385][53852] Updated weights for policy 0, policy_version 29380 (0.0008) [2023-10-08 08:58:58,758][53852] Updated weights for policy 0, policy_version 29390 (0.0008) [2023-10-08 08:58:59,133][53852] Updated weights for policy 0, policy_version 29400 (0.0008) [2023-10-08 08:59:01,675][53885] Updated weights for policy 1, policy_version 29252 (0.0007) [2023-10-08 08:59:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 60063744. Throughput: 0: 1846.2, 1: 1825.7. Samples: 15026942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:02,016][52710] Avg episode reward: [(0, '27.000'), (1, '32.500')] [2023-10-08 08:59:02,051][53885] Updated weights for policy 1, policy_version 29262 (0.0008) [2023-10-08 08:59:02,423][53885] Updated weights for policy 1, policy_version 29272 (0.0008) [2023-10-08 08:59:02,794][53852] Updated weights for policy 0, policy_version 29410 (0.0008) [2023-10-08 08:59:03,169][53852] Updated weights for policy 0, policy_version 29420 (0.0008) [2023-10-08 08:59:03,529][53852] Updated weights for policy 0, policy_version 29430 (0.0009) [2023-10-08 08:59:03,905][53852] Updated weights for policy 0, policy_version 29440 (0.0007) [2023-10-08 08:59:06,089][53885] Updated weights for policy 1, policy_version 29282 (0.0007) [2023-10-08 08:59:06,461][53885] Updated weights for policy 1, policy_version 29292 (0.0008) [2023-10-08 08:59:06,831][53885] Updated weights for policy 1, policy_version 29302 (0.0008) [2023-10-08 08:59:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 60129280. Throughput: 0: 1840.2, 1: 1822.5. Samples: 15049372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:07,016][52710] Avg episode reward: [(0, '28.910'), (1, '31.500')] [2023-10-08 08:59:07,202][53885] Updated weights for policy 1, policy_version 29312 (0.0007) [2023-10-08 08:59:07,578][53852] Updated weights for policy 0, policy_version 29450 (0.0008) [2023-10-08 08:59:07,947][53852] Updated weights for policy 0, policy_version 29460 (0.0008) [2023-10-08 08:59:08,321][53852] Updated weights for policy 0, policy_version 29470 (0.0007) [2023-10-08 08:59:10,999][53885] Updated weights for policy 1, policy_version 29322 (0.0011) [2023-10-08 08:59:11,371][53885] Updated weights for policy 1, policy_version 29332 (0.0011) [2023-10-08 08:59:11,733][53885] Updated weights for policy 1, policy_version 29342 (0.0009) [2023-10-08 08:59:11,867][53852] Updated weights for policy 0, policy_version 29480 (0.0009) [2023-10-08 08:59:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60227584. Throughput: 0: 1840.8, 1: 1828.4. Samples: 15060180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:12,015][52710] Avg episode reward: [(0, '29.010'), (1, '32.500')] [2023-10-08 08:59:12,232][53852] Updated weights for policy 0, policy_version 29490 (0.0008) [2023-10-08 08:59:12,609][53852] Updated weights for policy 0, policy_version 29500 (0.0007) [2023-10-08 08:59:15,597][53885] Updated weights for policy 1, policy_version 29352 (0.0009) [2023-10-08 08:59:15,977][53885] Updated weights for policy 1, policy_version 29362 (0.0010) [2023-10-08 08:59:16,298][53852] Updated weights for policy 0, policy_version 29510 (0.0008) [2023-10-08 08:59:16,343][53885] Updated weights for policy 1, policy_version 29372 (0.0008) [2023-10-08 08:59:16,672][53852] Updated weights for policy 0, policy_version 29520 (0.0007) [2023-10-08 08:59:17,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60293120. Throughput: 0: 1837.7, 1: 1821.6. Samples: 15082698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:17,016][52710] Avg episode reward: [(0, '26.770'), (1, '30.980')] [2023-10-08 08:59:17,037][53852] Updated weights for policy 0, policy_version 29530 (0.0008) [2023-10-08 08:59:20,003][53885] Updated weights for policy 1, policy_version 29382 (0.0008) [2023-10-08 08:59:20,379][53885] Updated weights for policy 1, policy_version 29392 (0.0009) [2023-10-08 08:59:20,662][53852] Updated weights for policy 0, policy_version 29540 (0.0007) [2023-10-08 08:59:20,732][53885] Updated weights for policy 1, policy_version 29402 (0.0010) [2023-10-08 08:59:21,040][53852] Updated weights for policy 0, policy_version 29550 (0.0008) [2023-10-08 08:59:21,407][53852] Updated weights for policy 0, policy_version 29560 (0.0007) [2023-10-08 08:59:22,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 60391424. Throughput: 0: 1822.4, 1: 1828.4. Samples: 15103374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:22,016][52710] Avg episode reward: [(0, '27.030'), (1, '27.060')] [2023-10-08 08:59:24,328][53885] Updated weights for policy 1, policy_version 29412 (0.0010) [2023-10-08 08:59:24,704][53885] Updated weights for policy 1, policy_version 29422 (0.0008) [2023-10-08 08:59:25,074][53885] Updated weights for policy 1, policy_version 29432 (0.0008) [2023-10-08 08:59:25,088][53852] Updated weights for policy 0, policy_version 29570 (0.0009) [2023-10-08 08:59:25,464][53852] Updated weights for policy 0, policy_version 29580 (0.0008) [2023-10-08 08:59:25,829][53852] Updated weights for policy 0, policy_version 29590 (0.0009) [2023-10-08 08:59:26,209][53852] Updated weights for policy 0, policy_version 29600 (0.0008) [2023-10-08 08:59:27,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 60456960. Throughput: 0: 1831.7, 1: 1830.8. Samples: 15115672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:27,016][52710] Avg episode reward: [(0, '27.030'), (1, '29.610')] [2023-10-08 08:59:28,779][53885] Updated weights for policy 1, policy_version 29442 (0.0009) [2023-10-08 08:59:29,149][53885] Updated weights for policy 1, policy_version 29452 (0.0007) [2023-10-08 08:59:29,522][53885] Updated weights for policy 1, policy_version 29462 (0.0009) [2023-10-08 08:59:29,856][53852] Updated weights for policy 0, policy_version 29610 (0.0008) [2023-10-08 08:59:29,889][53885] Updated weights for policy 1, policy_version 29472 (0.0008) [2023-10-08 08:59:30,227][53852] Updated weights for policy 0, policy_version 29620 (0.0009) [2023-10-08 08:59:30,592][53852] Updated weights for policy 0, policy_version 29630 (0.0008) [2023-10-08 08:59:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 60522496. Throughput: 0: 1824.3, 1: 1830.5. Samples: 15136382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:32,016][52710] Avg episode reward: [(0, '27.080'), (1, '29.780')] [2023-10-08 08:59:33,512][53885] Updated weights for policy 1, policy_version 29482 (0.0009) [2023-10-08 08:59:33,881][53885] Updated weights for policy 1, policy_version 29492 (0.0007) [2023-10-08 08:59:34,257][53885] Updated weights for policy 1, policy_version 29502 (0.0008) [2023-10-08 08:59:34,325][53852] Updated weights for policy 0, policy_version 29640 (0.0007) [2023-10-08 08:59:34,703][53852] Updated weights for policy 0, policy_version 29650 (0.0007) [2023-10-08 08:59:35,071][53852] Updated weights for policy 0, policy_version 29660 (0.0008) [2023-10-08 08:59:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60588032. Throughput: 0: 1839.5, 1: 1824.0. Samples: 15158886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:37,015][52710] Avg episode reward: [(0, '28.790'), (1, '33.050')] [2023-10-08 08:59:37,024][53594] Saving new best policy, reward=33.050! [2023-10-08 08:59:38,051][53885] Updated weights for policy 1, policy_version 29512 (0.0010) [2023-10-08 08:59:38,410][53885] Updated weights for policy 1, policy_version 29522 (0.0009) [2023-10-08 08:59:38,732][53852] Updated weights for policy 0, policy_version 29670 (0.0008) [2023-10-08 08:59:38,771][53885] Updated weights for policy 1, policy_version 29532 (0.0008) [2023-10-08 08:59:39,106][53852] Updated weights for policy 0, policy_version 29680 (0.0007) [2023-10-08 08:59:39,476][53852] Updated weights for policy 0, policy_version 29690 (0.0008) [2023-10-08 08:59:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 60653568. Throughput: 0: 1827.5, 1: 1827.3. Samples: 15169160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 08:59:42,016][52710] Avg episode reward: [(0, '27.500'), (1, '32.440')] [2023-10-08 08:59:42,484][53885] Updated weights for policy 1, policy_version 29542 (0.0010) [2023-10-08 08:59:42,857][53885] Updated weights for policy 1, policy_version 29552 (0.0009) [2023-10-08 08:59:43,133][53852] Updated weights for policy 0, policy_version 29700 (0.0009) [2023-10-08 08:59:43,230][53885] Updated weights for policy 1, policy_version 29562 (0.0008) [2023-10-08 08:59:43,514][53852] Updated weights for policy 0, policy_version 29710 (0.0008) [2023-10-08 08:59:43,881][53852] Updated weights for policy 0, policy_version 29720 (0.0010) [2023-10-08 08:59:46,848][53885] Updated weights for policy 1, policy_version 29572 (0.0008) [2023-10-08 08:59:47,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 60719104. Throughput: 0: 1837.7, 1: 1818.7. Samples: 15191484. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 08:59:47,016][52710] Avg episode reward: [(0, '30.910'), (1, '29.520')] [2023-10-08 08:59:47,017][53500] Saving new best policy, reward=30.910! [2023-10-08 08:59:47,221][53885] Updated weights for policy 1, policy_version 29582 (0.0008) [2023-10-08 08:59:47,496][53852] Updated weights for policy 0, policy_version 29730 (0.0007) [2023-10-08 08:59:47,592][53885] Updated weights for policy 1, policy_version 29592 (0.0010) [2023-10-08 08:59:47,869][53852] Updated weights for policy 0, policy_version 29740 (0.0008) [2023-10-08 08:59:48,233][53852] Updated weights for policy 0, policy_version 29750 (0.0008) [2023-10-08 08:59:48,605][53852] Updated weights for policy 0, policy_version 29760 (0.0007) [2023-10-08 08:59:51,346][53885] Updated weights for policy 1, policy_version 29602 (0.0008) [2023-10-08 08:59:51,719][53885] Updated weights for policy 1, policy_version 29612 (0.0008) [2023-10-08 08:59:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 60784640. Throughput: 0: 1840.2, 1: 1819.4. Samples: 15214054. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 08:59:52,016][52710] Avg episode reward: [(0, '27.640'), (1, '32.470')] [2023-10-08 08:59:52,083][53885] Updated weights for policy 1, policy_version 29622 (0.0007) [2023-10-08 08:59:52,301][53852] Updated weights for policy 0, policy_version 29770 (0.0009) [2023-10-08 08:59:52,451][53885] Updated weights for policy 1, policy_version 29632 (0.0007) [2023-10-08 08:59:52,670][53852] Updated weights for policy 0, policy_version 29780 (0.0007) [2023-10-08 08:59:53,042][53852] Updated weights for policy 0, policy_version 29790 (0.0007) [2023-10-08 08:59:55,952][53885] Updated weights for policy 1, policy_version 29642 (0.0011) [2023-10-08 08:59:56,313][53885] Updated weights for policy 1, policy_version 29652 (0.0009) [2023-10-08 08:59:56,622][53852] Updated weights for policy 0, policy_version 29800 (0.0007) [2023-10-08 08:59:56,687][53885] Updated weights for policy 1, policy_version 29662 (0.0008) [2023-10-08 08:59:56,988][53852] Updated weights for policy 0, policy_version 29810 (0.0008) [2023-10-08 08:59:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 60882944. Throughput: 0: 1836.1, 1: 1816.0. Samples: 15224528. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 08:59:57,016][52710] Avg episode reward: [(0, '27.250'), (1, '27.090')] [2023-10-08 08:59:57,357][53852] Updated weights for policy 0, policy_version 29820 (0.0008) [2023-10-08 09:00:00,528][53885] Updated weights for policy 1, policy_version 29672 (0.0008) [2023-10-08 09:00:00,897][53885] Updated weights for policy 1, policy_version 29682 (0.0009) [2023-10-08 09:00:00,998][53852] Updated weights for policy 0, policy_version 29830 (0.0007) [2023-10-08 09:00:01,268][53885] Updated weights for policy 1, policy_version 29692 (0.0008) [2023-10-08 09:00:01,359][53852] Updated weights for policy 0, policy_version 29840 (0.0007) [2023-10-08 09:00:01,729][53852] Updated weights for policy 0, policy_version 29850 (0.0008) [2023-10-08 09:00:02,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 60981248. Throughput: 0: 1842.2, 1: 1814.6. Samples: 15247252. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 09:00:02,016][52710] Avg episode reward: [(0, '27.870'), (1, '27.530')] [2023-10-08 09:00:04,865][53885] Updated weights for policy 1, policy_version 29702 (0.0009) [2023-10-08 09:00:05,231][53885] Updated weights for policy 1, policy_version 29712 (0.0009) [2023-10-08 09:00:05,441][53852] Updated weights for policy 0, policy_version 29860 (0.0009) [2023-10-08 09:00:05,599][53885] Updated weights for policy 1, policy_version 29722 (0.0007) [2023-10-08 09:00:05,802][53852] Updated weights for policy 0, policy_version 29870 (0.0007) [2023-10-08 09:00:06,176][53852] Updated weights for policy 0, policy_version 29880 (0.0009) [2023-10-08 09:00:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 61046784. Throughput: 0: 1836.0, 1: 1815.7. Samples: 15267698. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-08 09:00:07,016][52710] Avg episode reward: [(0, '27.850'), (1, '30.060')] [2023-10-08 09:00:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000029728_30441472.pth... [2023-10-08 09:00:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000029888_30605312.pth... [2023-10-08 09:00:07,059][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000028000_28672000.pth [2023-10-08 09:00:07,067][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000028160_28835840.pth [2023-10-08 09:00:09,392][53885] Updated weights for policy 1, policy_version 29732 (0.0009) [2023-10-08 09:00:09,753][53885] Updated weights for policy 1, policy_version 29742 (0.0010) [2023-10-08 09:00:09,968][53852] Updated weights for policy 0, policy_version 29890 (0.0008) [2023-10-08 09:00:10,131][53885] Updated weights for policy 1, policy_version 29752 (0.0008) [2023-10-08 09:00:10,342][53852] Updated weights for policy 0, policy_version 29900 (0.0008) [2023-10-08 09:00:10,703][53852] Updated weights for policy 0, policy_version 29910 (0.0010) [2023-10-08 09:00:11,074][53852] Updated weights for policy 0, policy_version 29920 (0.0010) [2023-10-08 09:00:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 61112320. Throughput: 0: 1835.5, 1: 1804.9. Samples: 15279490. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-08 09:00:12,016][52710] Avg episode reward: [(0, '27.170'), (1, '28.380')] [2023-10-08 09:00:13,920][53885] Updated weights for policy 1, policy_version 29762 (0.0009) [2023-10-08 09:00:14,290][53885] Updated weights for policy 1, policy_version 29772 (0.0009) [2023-10-08 09:00:14,661][53885] Updated weights for policy 1, policy_version 29782 (0.0009) [2023-10-08 09:00:14,790][53852] Updated weights for policy 0, policy_version 29930 (0.0008) [2023-10-08 09:00:15,026][53885] Updated weights for policy 1, policy_version 29792 (0.0008) [2023-10-08 09:00:15,166][53852] Updated weights for policy 0, policy_version 29940 (0.0009) [2023-10-08 09:00:15,528][53852] Updated weights for policy 0, policy_version 29950 (0.0008) [2023-10-08 09:00:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61177856. Throughput: 0: 1832.0, 1: 1800.4. Samples: 15299842. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-08 09:00:17,016][52710] Avg episode reward: [(0, '27.950'), (1, '28.720')] [2023-10-08 09:00:18,761][53885] Updated weights for policy 1, policy_version 29802 (0.0007) [2023-10-08 09:00:19,067][53852] Updated weights for policy 0, policy_version 29960 (0.0007) [2023-10-08 09:00:19,131][53885] Updated weights for policy 1, policy_version 29812 (0.0007) [2023-10-08 09:00:19,438][53852] Updated weights for policy 0, policy_version 29970 (0.0007) [2023-10-08 09:00:19,491][53885] Updated weights for policy 1, policy_version 29822 (0.0007) [2023-10-08 09:00:19,814][53852] Updated weights for policy 0, policy_version 29980 (0.0007) [2023-10-08 09:00:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 61243392. Throughput: 0: 1833.9, 1: 1810.4. Samples: 15322882. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-08 09:00:22,016][52710] Avg episode reward: [(0, '26.180'), (1, '29.970')] [2023-10-08 09:00:23,092][53885] Updated weights for policy 1, policy_version 29832 (0.0008) [2023-10-08 09:00:23,465][53885] Updated weights for policy 1, policy_version 29842 (0.0008) [2023-10-08 09:00:23,563][53852] Updated weights for policy 0, policy_version 29990 (0.0007) [2023-10-08 09:00:23,837][53885] Updated weights for policy 1, policy_version 29852 (0.0009) [2023-10-08 09:00:23,921][53852] Updated weights for policy 0, policy_version 30000 (0.0007) [2023-10-08 09:00:24,290][53852] Updated weights for policy 0, policy_version 30010 (0.0008) [2023-10-08 09:00:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 61308928. Throughput: 0: 1831.7, 1: 1807.7. Samples: 15332932. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) [2023-10-08 09:00:27,016][52710] Avg episode reward: [(0, '25.310'), (1, '29.320')] [2023-10-08 09:00:27,591][53885] Updated weights for policy 1, policy_version 29862 (0.0009) [2023-10-08 09:00:27,960][53885] Updated weights for policy 1, policy_version 29872 (0.0009) [2023-10-08 09:00:27,967][53852] Updated weights for policy 0, policy_version 30020 (0.0008) [2023-10-08 09:00:28,319][53885] Updated weights for policy 1, policy_version 29882 (0.0009) [2023-10-08 09:00:28,345][53852] Updated weights for policy 0, policy_version 30030 (0.0009) [2023-10-08 09:00:28,715][53852] Updated weights for policy 0, policy_version 30040 (0.0010) [2023-10-08 09:00:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 61374464. Throughput: 0: 1836.8, 1: 1810.0. Samples: 15355588. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:00:32,016][52710] Avg episode reward: [(0, '26.810'), (1, '30.410')] [2023-10-08 09:00:32,087][53885] Updated weights for policy 1, policy_version 29892 (0.0009) [2023-10-08 09:00:32,241][53852] Updated weights for policy 0, policy_version 30050 (0.0009) [2023-10-08 09:00:32,462][53885] Updated weights for policy 1, policy_version 29902 (0.0007) [2023-10-08 09:00:32,603][53852] Updated weights for policy 0, policy_version 30060 (0.0009) [2023-10-08 09:00:32,827][53885] Updated weights for policy 1, policy_version 29912 (0.0008) [2023-10-08 09:00:32,975][53852] Updated weights for policy 0, policy_version 30070 (0.0008) [2023-10-08 09:00:33,341][53852] Updated weights for policy 0, policy_version 30080 (0.0009) [2023-10-08 09:00:36,490][53885] Updated weights for policy 1, policy_version 29922 (0.0007) [2023-10-08 09:00:36,857][53885] Updated weights for policy 1, policy_version 29932 (0.0009) [2023-10-08 09:00:36,942][53852] Updated weights for policy 0, policy_version 30090 (0.0008) [2023-10-08 09:00:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 61440000. Throughput: 0: 1833.4, 1: 1815.2. Samples: 15378242. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:00:37,016][52710] Avg episode reward: [(0, '26.780'), (1, '31.070')] [2023-10-08 09:00:37,221][53885] Updated weights for policy 1, policy_version 29942 (0.0008) [2023-10-08 09:00:37,318][53852] Updated weights for policy 0, policy_version 30100 (0.0007) [2023-10-08 09:00:37,577][53885] Updated weights for policy 1, policy_version 29952 (0.0007) [2023-10-08 09:00:37,686][53852] Updated weights for policy 0, policy_version 30110 (0.0007) [2023-10-08 09:00:41,191][53885] Updated weights for policy 1, policy_version 29962 (0.0008) [2023-10-08 09:00:41,390][53852] Updated weights for policy 0, policy_version 30120 (0.0007) [2023-10-08 09:00:41,546][53885] Updated weights for policy 1, policy_version 29972 (0.0008) [2023-10-08 09:00:41,769][53852] Updated weights for policy 0, policy_version 30130 (0.0009) [2023-10-08 09:00:41,916][53885] Updated weights for policy 1, policy_version 29982 (0.0007) [2023-10-08 09:00:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61538304. Throughput: 0: 1832.2, 1: 1809.6. Samples: 15388412. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:00:42,016][52710] Avg episode reward: [(0, '27.350'), (1, '28.060')] [2023-10-08 09:00:42,133][53852] Updated weights for policy 0, policy_version 30140 (0.0009) [2023-10-08 09:00:45,698][53885] Updated weights for policy 1, policy_version 29992 (0.0009) [2023-10-08 09:00:45,894][53852] Updated weights for policy 0, policy_version 30150 (0.0008) [2023-10-08 09:00:46,067][53885] Updated weights for policy 1, policy_version 30002 (0.0008) [2023-10-08 09:00:46,266][53852] Updated weights for policy 0, policy_version 30160 (0.0010) [2023-10-08 09:00:46,435][53885] Updated weights for policy 1, policy_version 30012 (0.0008) [2023-10-08 09:00:46,630][53852] Updated weights for policy 0, policy_version 30170 (0.0009) [2023-10-08 09:00:47,015][52710] Fps is (10 sec: 19661.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 61636608. Throughput: 0: 1822.6, 1: 1811.4. Samples: 15410784. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:00:47,016][52710] Avg episode reward: [(0, '26.800'), (1, '26.640')] [2023-10-08 09:00:50,107][53885] Updated weights for policy 1, policy_version 30022 (0.0007) [2023-10-08 09:00:50,235][53852] Updated weights for policy 0, policy_version 30180 (0.0010) [2023-10-08 09:00:50,484][53885] Updated weights for policy 1, policy_version 30032 (0.0009) [2023-10-08 09:00:50,594][53852] Updated weights for policy 0, policy_version 30190 (0.0008) [2023-10-08 09:00:50,850][53885] Updated weights for policy 1, policy_version 30042 (0.0007) [2023-10-08 09:00:50,959][53852] Updated weights for policy 0, policy_version 30200 (0.0007) [2023-10-08 09:00:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 61702144. Throughput: 0: 1818.0, 1: 1807.0. Samples: 15430822. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-08 09:00:52,016][52710] Avg episode reward: [(0, '26.950'), (1, '30.480')] [2023-10-08 09:00:54,543][53885] Updated weights for policy 1, policy_version 30052 (0.0008) [2023-10-08 09:00:54,702][53852] Updated weights for policy 0, policy_version 30210 (0.0010) [2023-10-08 09:00:54,908][53885] Updated weights for policy 1, policy_version 30062 (0.0008) [2023-10-08 09:00:55,060][53852] Updated weights for policy 0, policy_version 30220 (0.0009) [2023-10-08 09:00:55,274][53885] Updated weights for policy 1, policy_version 30072 (0.0008) [2023-10-08 09:00:55,429][53852] Updated weights for policy 0, policy_version 30230 (0.0008) [2023-10-08 09:00:55,794][53852] Updated weights for policy 0, policy_version 30240 (0.0010) [2023-10-08 09:00:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61767680. Throughput: 0: 1824.3, 1: 1819.2. Samples: 15443450. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-08 09:00:57,016][52710] Avg episode reward: [(0, '26.540'), (1, '31.150')] [2023-10-08 09:00:58,996][53885] Updated weights for policy 1, policy_version 30082 (0.0010) [2023-10-08 09:00:59,363][53885] Updated weights for policy 1, policy_version 30092 (0.0010) [2023-10-08 09:00:59,546][53852] Updated weights for policy 0, policy_version 30250 (0.0007) [2023-10-08 09:00:59,727][53885] Updated weights for policy 1, policy_version 30102 (0.0007) [2023-10-08 09:00:59,906][53852] Updated weights for policy 0, policy_version 30260 (0.0007) [2023-10-08 09:01:00,088][53885] Updated weights for policy 1, policy_version 30112 (0.0008) [2023-10-08 09:01:00,265][53852] Updated weights for policy 0, policy_version 30270 (0.0007) [2023-10-08 09:01:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 61833216. Throughput: 0: 1816.0, 1: 1821.4. Samples: 15463524. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-08 09:01:02,016][52710] Avg episode reward: [(0, '27.020'), (1, '29.750')] [2023-10-08 09:01:03,756][53885] Updated weights for policy 1, policy_version 30122 (0.0009) [2023-10-08 09:01:03,841][53852] Updated weights for policy 0, policy_version 30280 (0.0008) [2023-10-08 09:01:04,123][53885] Updated weights for policy 1, policy_version 30132 (0.0008) [2023-10-08 09:01:04,209][53852] Updated weights for policy 0, policy_version 30290 (0.0008) [2023-10-08 09:01:04,494][53885] Updated weights for policy 1, policy_version 30142 (0.0007) [2023-10-08 09:01:04,577][53852] Updated weights for policy 0, policy_version 30300 (0.0009) [2023-10-08 09:01:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 61898752. Throughput: 0: 1818.1, 1: 1815.8. Samples: 15486408. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-08 09:01:07,016][52710] Avg episode reward: [(0, '25.610'), (1, '30.540')] [2023-10-08 09:01:08,202][53885] Updated weights for policy 1, policy_version 30152 (0.0008) [2023-10-08 09:01:08,299][53852] Updated weights for policy 0, policy_version 30310 (0.0008) [2023-10-08 09:01:08,574][53885] Updated weights for policy 1, policy_version 30162 (0.0007) [2023-10-08 09:01:08,673][53852] Updated weights for policy 0, policy_version 30320 (0.0007) [2023-10-08 09:01:08,933][53885] Updated weights for policy 1, policy_version 30172 (0.0008) [2023-10-08 09:01:09,038][53852] Updated weights for policy 0, policy_version 30330 (0.0008) [2023-10-08 09:01:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 61964288. Throughput: 0: 1820.6, 1: 1812.7. Samples: 15496428. Policy #0 lag: (min: 26.0, avg: 37.2, max: 58.0) [2023-10-08 09:01:12,016][52710] Avg episode reward: [(0, '28.050'), (1, '33.750')] [2023-10-08 09:01:12,016][53594] Saving new best policy, reward=33.750! [2023-10-08 09:01:12,750][53885] Updated weights for policy 1, policy_version 30182 (0.0008) [2023-10-08 09:01:12,766][53852] Updated weights for policy 0, policy_version 30340 (0.0007) [2023-10-08 09:01:13,118][53885] Updated weights for policy 1, policy_version 30192 (0.0007) [2023-10-08 09:01:13,126][53852] Updated weights for policy 0, policy_version 30350 (0.0008) [2023-10-08 09:01:13,484][53885] Updated weights for policy 1, policy_version 30202 (0.0007) [2023-10-08 09:01:13,496][53852] Updated weights for policy 0, policy_version 30360 (0.0008) [2023-10-08 09:01:17,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 62029824. Throughput: 0: 1822.6, 1: 1814.2. Samples: 15519244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:01:17,016][52710] Avg episode reward: [(0, '26.700'), (1, '31.880')] [2023-10-08 09:01:17,189][53852] Updated weights for policy 0, policy_version 30370 (0.0009) [2023-10-08 09:01:17,204][53885] Updated weights for policy 1, policy_version 30212 (0.0008) [2023-10-08 09:01:17,553][53852] Updated weights for policy 0, policy_version 30380 (0.0007) [2023-10-08 09:01:17,571][53885] Updated weights for policy 1, policy_version 30222 (0.0007) [2023-10-08 09:01:17,925][53852] Updated weights for policy 0, policy_version 30390 (0.0008) [2023-10-08 09:01:17,932][53885] Updated weights for policy 1, policy_version 30232 (0.0007) [2023-10-08 09:01:18,297][53852] Updated weights for policy 0, policy_version 30400 (0.0010) [2023-10-08 09:01:21,676][53885] Updated weights for policy 1, policy_version 30242 (0.0007) [2023-10-08 09:01:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 62095360. Throughput: 0: 1815.2, 1: 1824.6. Samples: 15542036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:01:22,016][52710] Avg episode reward: [(0, '28.580'), (1, '32.740')] [2023-10-08 09:01:22,052][53885] Updated weights for policy 1, policy_version 30252 (0.0009) [2023-10-08 09:01:22,199][53852] Updated weights for policy 0, policy_version 30410 (0.0007) [2023-10-08 09:01:22,417][53885] Updated weights for policy 1, policy_version 30262 (0.0007) [2023-10-08 09:01:22,572][53852] Updated weights for policy 0, policy_version 30420 (0.0008) [2023-10-08 09:01:22,795][53885] Updated weights for policy 1, policy_version 30272 (0.0007) [2023-10-08 09:01:22,952][53852] Updated weights for policy 0, policy_version 30430 (0.0009) [2023-10-08 09:01:26,521][53885] Updated weights for policy 1, policy_version 30282 (0.0007) [2023-10-08 09:01:26,593][53852] Updated weights for policy 0, policy_version 30440 (0.0007) [2023-10-08 09:01:26,880][53885] Updated weights for policy 1, policy_version 30292 (0.0007) [2023-10-08 09:01:26,969][53852] Updated weights for policy 0, policy_version 30450 (0.0007) [2023-10-08 09:01:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 62160896. Throughput: 0: 1816.4, 1: 1816.9. Samples: 15551914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:01:27,016][52710] Avg episode reward: [(0, '26.870'), (1, '31.630')] [2023-10-08 09:01:27,244][53885] Updated weights for policy 1, policy_version 30302 (0.0007) [2023-10-08 09:01:27,345][53852] Updated weights for policy 0, policy_version 30460 (0.0007) [2023-10-08 09:01:30,923][53885] Updated weights for policy 1, policy_version 30312 (0.0008) [2023-10-08 09:01:30,953][53852] Updated weights for policy 0, policy_version 30470 (0.0007) [2023-10-08 09:01:31,292][53885] Updated weights for policy 1, policy_version 30322 (0.0008) [2023-10-08 09:01:31,321][53852] Updated weights for policy 0, policy_version 30480 (0.0007) [2023-10-08 09:01:31,659][53885] Updated weights for policy 1, policy_version 30332 (0.0008) [2023-10-08 09:01:31,693][53852] Updated weights for policy 0, policy_version 30490 (0.0009) [2023-10-08 09:01:32,015][52710] Fps is (10 sec: 19661.5, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 62291968. Throughput: 0: 1819.5, 1: 1827.7. Samples: 15574908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:01:32,015][52710] Avg episode reward: [(0, '28.400'), (1, '30.240')] [2023-10-08 09:01:35,234][53852] Updated weights for policy 0, policy_version 30500 (0.0011) [2023-10-08 09:01:35,383][53885] Updated weights for policy 1, policy_version 30342 (0.0008) [2023-10-08 09:01:35,601][53852] Updated weights for policy 0, policy_version 30510 (0.0007) [2023-10-08 09:01:35,745][53885] Updated weights for policy 1, policy_version 30352 (0.0009) [2023-10-08 09:01:35,968][53852] Updated weights for policy 0, policy_version 30520 (0.0007) [2023-10-08 09:01:36,099][53885] Updated weights for policy 1, policy_version 30362 (0.0009) [2023-10-08 09:01:37,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 62357504. Throughput: 0: 1827.7, 1: 1819.5. Samples: 15594948. Policy #0 lag: (min: 16.0, avg: 32.2, max: 48.0) [2023-10-08 09:01:37,016][52710] Avg episode reward: [(0, '27.110'), (1, '30.460')] [2023-10-08 09:01:39,604][53852] Updated weights for policy 0, policy_version 30530 (0.0008) [2023-10-08 09:01:39,756][53885] Updated weights for policy 1, policy_version 30372 (0.0008) [2023-10-08 09:01:39,972][53852] Updated weights for policy 0, policy_version 30540 (0.0008) [2023-10-08 09:01:40,111][53885] Updated weights for policy 1, policy_version 30382 (0.0009) [2023-10-08 09:01:40,347][53852] Updated weights for policy 0, policy_version 30550 (0.0008) [2023-10-08 09:01:40,477][53885] Updated weights for policy 1, policy_version 30392 (0.0007) [2023-10-08 09:01:40,710][53852] Updated weights for policy 0, policy_version 30560 (0.0008) [2023-10-08 09:01:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 62423040. Throughput: 0: 1832.4, 1: 1827.9. Samples: 15608166. Policy #0 lag: (min: 16.0, avg: 32.2, max: 48.0) [2023-10-08 09:01:42,015][52710] Avg episode reward: [(0, '27.510'), (1, '31.710')] [2023-10-08 09:01:44,128][53885] Updated weights for policy 1, policy_version 30402 (0.0009) [2023-10-08 09:01:44,496][53885] Updated weights for policy 1, policy_version 30412 (0.0007) [2023-10-08 09:01:44,505][53852] Updated weights for policy 0, policy_version 30570 (0.0007) [2023-10-08 09:01:44,865][53885] Updated weights for policy 1, policy_version 30422 (0.0007) [2023-10-08 09:01:44,880][53852] Updated weights for policy 0, policy_version 30580 (0.0008) [2023-10-08 09:01:45,240][53885] Updated weights for policy 1, policy_version 30432 (0.0007) [2023-10-08 09:01:45,254][53852] Updated weights for policy 0, policy_version 30590 (0.0008) [2023-10-08 09:01:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 62488576. Throughput: 0: 1833.6, 1: 1822.1. Samples: 15628032. Policy #0 lag: (min: 16.0, avg: 32.2, max: 48.0) [2023-10-08 09:01:47,016][52710] Avg episode reward: [(0, '29.240'), (1, '29.440')] [2023-10-08 09:01:48,855][53852] Updated weights for policy 0, policy_version 30600 (0.0008) [2023-10-08 09:01:48,902][53885] Updated weights for policy 1, policy_version 30442 (0.0008) [2023-10-08 09:01:49,229][53852] Updated weights for policy 0, policy_version 30610 (0.0007) [2023-10-08 09:01:49,277][53885] Updated weights for policy 1, policy_version 30452 (0.0008) [2023-10-08 09:01:49,602][53852] Updated weights for policy 0, policy_version 30620 (0.0008) [2023-10-08 09:01:49,642][53885] Updated weights for policy 1, policy_version 30462 (0.0008) [2023-10-08 09:01:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 62554112. Throughput: 0: 1837.9, 1: 1822.3. Samples: 15651120. Policy #0 lag: (min: 16.0, avg: 32.2, max: 48.0) [2023-10-08 09:01:52,016][52710] Avg episode reward: [(0, '30.770'), (1, '29.410')] [2023-10-08 09:01:53,210][53852] Updated weights for policy 0, policy_version 30630 (0.0007) [2023-10-08 09:01:53,282][53885] Updated weights for policy 1, policy_version 30472 (0.0007) [2023-10-08 09:01:53,581][53852] Updated weights for policy 0, policy_version 30640 (0.0007) [2023-10-08 09:01:53,645][53885] Updated weights for policy 1, policy_version 30482 (0.0007) [2023-10-08 09:01:53,955][53852] Updated weights for policy 0, policy_version 30650 (0.0008) [2023-10-08 09:01:54,009][53885] Updated weights for policy 1, policy_version 30492 (0.0008) [2023-10-08 09:01:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 62619648. Throughput: 0: 1836.6, 1: 1826.6. Samples: 15661272. Policy #0 lag: (min: 16.0, avg: 32.2, max: 48.0) [2023-10-08 09:01:57,016][52710] Avg episode reward: [(0, '28.610'), (1, '28.500')] [2023-10-08 09:01:57,632][53885] Updated weights for policy 1, policy_version 30502 (0.0007) [2023-10-08 09:01:57,635][53852] Updated weights for policy 0, policy_version 30660 (0.0008) [2023-10-08 09:01:57,991][53885] Updated weights for policy 1, policy_version 30512 (0.0008) [2023-10-08 09:01:58,002][53852] Updated weights for policy 0, policy_version 30670 (0.0007) [2023-10-08 09:01:58,362][53885] Updated weights for policy 1, policy_version 30522 (0.0008) [2023-10-08 09:01:58,364][53852] Updated weights for policy 0, policy_version 30680 (0.0007) [2023-10-08 09:02:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 62685184. Throughput: 0: 1835.8, 1: 1829.7. Samples: 15684192. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 09:02:02,016][52710] Avg episode reward: [(0, '28.370'), (1, '30.300')] [2023-10-08 09:02:02,073][53852] Updated weights for policy 0, policy_version 30690 (0.0007) [2023-10-08 09:02:02,144][53885] Updated weights for policy 1, policy_version 30532 (0.0009) [2023-10-08 09:02:02,446][53852] Updated weights for policy 0, policy_version 30700 (0.0008) [2023-10-08 09:02:02,512][53885] Updated weights for policy 1, policy_version 30542 (0.0008) [2023-10-08 09:02:02,817][53852] Updated weights for policy 0, policy_version 30710 (0.0007) [2023-10-08 09:02:02,876][53885] Updated weights for policy 1, policy_version 30552 (0.0007) [2023-10-08 09:02:03,179][53852] Updated weights for policy 0, policy_version 30720 (0.0009) [2023-10-08 09:02:06,511][53885] Updated weights for policy 1, policy_version 30562 (0.0007) [2023-10-08 09:02:06,871][53852] Updated weights for policy 0, policy_version 30730 (0.0007) [2023-10-08 09:02:06,872][53885] Updated weights for policy 1, policy_version 30572 (0.0008) [2023-10-08 09:02:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 62750720. Throughput: 0: 1834.5, 1: 1816.2. Samples: 15706320. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 09:02:07,016][52710] Avg episode reward: [(0, '31.310'), (1, '30.310')] [2023-10-08 09:02:07,237][53885] Updated weights for policy 1, policy_version 30582 (0.0009) [2023-10-08 09:02:07,239][53852] Updated weights for policy 0, policy_version 30740 (0.0007) [2023-10-08 09:02:07,601][53885] Updated weights for policy 1, policy_version 30592 (0.0007) [2023-10-08 09:02:07,602][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000030592_31326208.pth... [2023-10-08 09:02:07,607][53852] Updated weights for policy 0, policy_version 30750 (0.0009) [2023-10-08 09:02:07,634][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000028864_29556736.pth [2023-10-08 09:02:07,677][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000030752_31490048.pth... [2023-10-08 09:02:07,706][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000029024_29720576.pth [2023-10-08 09:02:07,710][53500] Saving new best policy, reward=31.310! [2023-10-08 09:02:11,344][53852] Updated weights for policy 0, policy_version 30760 (0.0008) [2023-10-08 09:02:11,386][53885] Updated weights for policy 1, policy_version 30602 (0.0009) [2023-10-08 09:02:11,699][53852] Updated weights for policy 0, policy_version 30770 (0.0007) [2023-10-08 09:02:11,764][53885] Updated weights for policy 1, policy_version 30612 (0.0008) [2023-10-08 09:02:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 62816256. Throughput: 0: 1836.1, 1: 1819.2. Samples: 15716402. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 09:02:12,016][52710] Avg episode reward: [(0, '27.660'), (1, '30.120')] [2023-10-08 09:02:12,063][53852] Updated weights for policy 0, policy_version 30780 (0.0007) [2023-10-08 09:02:12,129][53885] Updated weights for policy 1, policy_version 30622 (0.0009) [2023-10-08 09:02:15,655][53852] Updated weights for policy 0, policy_version 30790 (0.0008) [2023-10-08 09:02:15,888][53885] Updated weights for policy 1, policy_version 30632 (0.0008) [2023-10-08 09:02:16,030][53852] Updated weights for policy 0, policy_version 30800 (0.0007) [2023-10-08 09:02:16,262][53885] Updated weights for policy 1, policy_version 30642 (0.0007) [2023-10-08 09:02:16,399][53852] Updated weights for policy 0, policy_version 30810 (0.0008) [2023-10-08 09:02:16,625][53885] Updated weights for policy 1, policy_version 30652 (0.0007) [2023-10-08 09:02:17,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 62947328. Throughput: 0: 1829.5, 1: 1817.6. Samples: 15739026. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 09:02:17,016][52710] Avg episode reward: [(0, '26.730'), (1, '31.000')] [2023-10-08 09:02:20,091][53852] Updated weights for policy 0, policy_version 30820 (0.0007) [2023-10-08 09:02:20,290][53885] Updated weights for policy 1, policy_version 30662 (0.0010) [2023-10-08 09:02:20,456][53852] Updated weights for policy 0, policy_version 30830 (0.0007) [2023-10-08 09:02:20,649][53885] Updated weights for policy 1, policy_version 30672 (0.0009) [2023-10-08 09:02:20,835][53852] Updated weights for policy 0, policy_version 30840 (0.0009) [2023-10-08 09:02:21,014][53885] Updated weights for policy 1, policy_version 30682 (0.0007) [2023-10-08 09:02:22,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 63012864. Throughput: 0: 1831.1, 1: 1815.7. Samples: 15759054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:02:22,015][52710] Avg episode reward: [(0, '28.350'), (1, '28.480')] [2023-10-08 09:02:24,514][53852] Updated weights for policy 0, policy_version 30850 (0.0008) [2023-10-08 09:02:24,802][53885] Updated weights for policy 1, policy_version 30692 (0.0008) [2023-10-08 09:02:24,881][53852] Updated weights for policy 0, policy_version 30860 (0.0009) [2023-10-08 09:02:25,179][53885] Updated weights for policy 1, policy_version 30702 (0.0008) [2023-10-08 09:02:25,251][53852] Updated weights for policy 0, policy_version 30870 (0.0007) [2023-10-08 09:02:25,545][53885] Updated weights for policy 1, policy_version 30712 (0.0009) [2023-10-08 09:02:25,622][53852] Updated weights for policy 0, policy_version 30880 (0.0008) [2023-10-08 09:02:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 63078400. Throughput: 0: 1824.2, 1: 1813.7. Samples: 15771872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:02:27,016][52710] Avg episode reward: [(0, '27.710'), (1, '26.490')] [2023-10-08 09:02:29,072][53885] Updated weights for policy 1, policy_version 30722 (0.0008) [2023-10-08 09:02:29,193][53852] Updated weights for policy 0, policy_version 30890 (0.0008) [2023-10-08 09:02:29,445][53885] Updated weights for policy 1, policy_version 30732 (0.0007) [2023-10-08 09:02:29,567][53852] Updated weights for policy 0, policy_version 30900 (0.0008) [2023-10-08 09:02:29,817][53885] Updated weights for policy 1, policy_version 30742 (0.0007) [2023-10-08 09:02:29,939][53852] Updated weights for policy 0, policy_version 30910 (0.0009) [2023-10-08 09:02:30,189][53885] Updated weights for policy 1, policy_version 30752 (0.0007) [2023-10-08 09:02:32,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 63143936. Throughput: 0: 1831.4, 1: 1812.6. Samples: 15792012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:02:32,016][52710] Avg episode reward: [(0, '26.230'), (1, '26.850')] [2023-10-08 09:02:33,582][53852] Updated weights for policy 0, policy_version 30920 (0.0007) [2023-10-08 09:02:33,751][53885] Updated weights for policy 1, policy_version 30762 (0.0008) [2023-10-08 09:02:33,944][53852] Updated weights for policy 0, policy_version 30930 (0.0007) [2023-10-08 09:02:34,124][53885] Updated weights for policy 1, policy_version 30772 (0.0008) [2023-10-08 09:02:34,317][53852] Updated weights for policy 0, policy_version 30940 (0.0007) [2023-10-08 09:02:34,491][53885] Updated weights for policy 1, policy_version 30782 (0.0008) [2023-10-08 09:02:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 63209472. Throughput: 0: 1828.5, 1: 1815.6. Samples: 15815108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:02:37,016][52710] Avg episode reward: [(0, '28.810'), (1, '24.530')] [2023-10-08 09:02:37,939][53852] Updated weights for policy 0, policy_version 30950 (0.0009) [2023-10-08 09:02:38,263][53885] Updated weights for policy 1, policy_version 30792 (0.0010) [2023-10-08 09:02:38,312][53852] Updated weights for policy 0, policy_version 30960 (0.0010) [2023-10-08 09:02:38,620][53885] Updated weights for policy 1, policy_version 30802 (0.0010) [2023-10-08 09:02:38,685][53852] Updated weights for policy 0, policy_version 30970 (0.0008) [2023-10-08 09:02:38,991][53885] Updated weights for policy 1, policy_version 30812 (0.0007) [2023-10-08 09:02:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63275008. Throughput: 0: 1824.9, 1: 1813.9. Samples: 15825016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:02:42,015][52710] Avg episode reward: [(0, '28.880'), (1, '27.340')] [2023-10-08 09:02:42,404][53852] Updated weights for policy 0, policy_version 30980 (0.0007) [2023-10-08 09:02:42,779][53852] Updated weights for policy 0, policy_version 30990 (0.0009) [2023-10-08 09:02:42,796][53885] Updated weights for policy 1, policy_version 30822 (0.0007) [2023-10-08 09:02:43,146][53852] Updated weights for policy 0, policy_version 31000 (0.0009) [2023-10-08 09:02:43,169][53885] Updated weights for policy 1, policy_version 30832 (0.0007) [2023-10-08 09:02:43,535][53885] Updated weights for policy 1, policy_version 30842 (0.0008) [2023-10-08 09:02:46,880][53852] Updated weights for policy 0, policy_version 31010 (0.0007) [2023-10-08 09:02:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63340544. Throughput: 0: 1825.2, 1: 1810.0. Samples: 15847772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:02:47,016][52710] Avg episode reward: [(0, '26.250'), (1, '28.920')] [2023-10-08 09:02:47,159][53885] Updated weights for policy 1, policy_version 30852 (0.0008) [2023-10-08 09:02:47,272][53852] Updated weights for policy 0, policy_version 31020 (0.0007) [2023-10-08 09:02:47,523][53885] Updated weights for policy 1, policy_version 30862 (0.0009) [2023-10-08 09:02:47,635][53852] Updated weights for policy 0, policy_version 31030 (0.0008) [2023-10-08 09:02:47,897][53885] Updated weights for policy 1, policy_version 30872 (0.0010) [2023-10-08 09:02:48,009][53852] Updated weights for policy 0, policy_version 31040 (0.0009) [2023-10-08 09:02:51,656][53852] Updated weights for policy 0, policy_version 31050 (0.0009) [2023-10-08 09:02:51,740][53885] Updated weights for policy 1, policy_version 30882 (0.0010) [2023-10-08 09:02:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63406080. Throughput: 0: 1822.2, 1: 1818.1. Samples: 15870134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:02:52,016][52710] Avg episode reward: [(0, '30.090'), (1, '29.300')] [2023-10-08 09:02:52,025][53852] Updated weights for policy 0, policy_version 31060 (0.0008) [2023-10-08 09:02:52,115][53885] Updated weights for policy 1, policy_version 30892 (0.0007) [2023-10-08 09:02:52,398][53852] Updated weights for policy 0, policy_version 31070 (0.0008) [2023-10-08 09:02:52,478][53885] Updated weights for policy 1, policy_version 30902 (0.0007) [2023-10-08 09:02:52,841][53885] Updated weights for policy 1, policy_version 30912 (0.0008) [2023-10-08 09:02:56,197][53852] Updated weights for policy 0, policy_version 31080 (0.0009) [2023-10-08 09:02:56,386][53885] Updated weights for policy 1, policy_version 30922 (0.0007) [2023-10-08 09:02:56,566][53852] Updated weights for policy 0, policy_version 31090 (0.0007) [2023-10-08 09:02:56,764][53885] Updated weights for policy 1, policy_version 30932 (0.0007) [2023-10-08 09:02:56,945][53852] Updated weights for policy 0, policy_version 31100 (0.0009) [2023-10-08 09:02:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 63471616. Throughput: 0: 1827.1, 1: 1819.2. Samples: 15880486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:02:57,016][52710] Avg episode reward: [(0, '29.270'), (1, '32.250')] [2023-10-08 09:02:57,132][53885] Updated weights for policy 1, policy_version 30942 (0.0008) [2023-10-08 09:03:00,497][53852] Updated weights for policy 0, policy_version 31110 (0.0007) [2023-10-08 09:03:00,837][53885] Updated weights for policy 1, policy_version 30952 (0.0008) [2023-10-08 09:03:00,861][53852] Updated weights for policy 0, policy_version 31120 (0.0008) [2023-10-08 09:03:01,212][53885] Updated weights for policy 1, policy_version 30962 (0.0008) [2023-10-08 09:03:01,234][53852] Updated weights for policy 0, policy_version 31130 (0.0007) [2023-10-08 09:03:01,582][53885] Updated weights for policy 1, policy_version 30972 (0.0007) [2023-10-08 09:03:02,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 63602688. Throughput: 0: 1823.0, 1: 1817.5. Samples: 15902850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:03:02,015][52710] Avg episode reward: [(0, '28.890'), (1, '31.000')] [2023-10-08 09:03:04,870][53852] Updated weights for policy 0, policy_version 31140 (0.0007) [2023-10-08 09:03:05,251][53852] Updated weights for policy 0, policy_version 31150 (0.0008) [2023-10-08 09:03:05,261][53885] Updated weights for policy 1, policy_version 30982 (0.0009) [2023-10-08 09:03:05,616][53852] Updated weights for policy 0, policy_version 31160 (0.0009) [2023-10-08 09:03:05,623][53885] Updated weights for policy 1, policy_version 30992 (0.0009) [2023-10-08 09:03:05,997][53885] Updated weights for policy 1, policy_version 31002 (0.0008) [2023-10-08 09:03:07,015][52710] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 63668224. Throughput: 0: 1824.1, 1: 1819.1. Samples: 15923002. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 09:03:07,016][52710] Avg episode reward: [(0, '27.990'), (1, '32.360')] [2023-10-08 09:03:09,264][53852] Updated weights for policy 0, policy_version 31170 (0.0007) [2023-10-08 09:03:09,635][53852] Updated weights for policy 0, policy_version 31180 (0.0007) [2023-10-08 09:03:09,795][53885] Updated weights for policy 1, policy_version 31012 (0.0007) [2023-10-08 09:03:10,013][53852] Updated weights for policy 0, policy_version 31190 (0.0007) [2023-10-08 09:03:10,162][53885] Updated weights for policy 1, policy_version 31022 (0.0009) [2023-10-08 09:03:10,377][53852] Updated weights for policy 0, policy_version 31200 (0.0007) [2023-10-08 09:03:10,523][53885] Updated weights for policy 1, policy_version 31032 (0.0009) [2023-10-08 09:03:12,015][52710] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 63733760. Throughput: 0: 1818.5, 1: 1819.5. Samples: 15935582. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 09:03:12,016][52710] Avg episode reward: [(0, '30.740'), (1, '29.890')] [2023-10-08 09:03:13,872][53852] Updated weights for policy 0, policy_version 31210 (0.0011) [2023-10-08 09:03:14,222][53885] Updated weights for policy 1, policy_version 31042 (0.0008) [2023-10-08 09:03:14,251][53852] Updated weights for policy 0, policy_version 31220 (0.0010) [2023-10-08 09:03:14,590][53885] Updated weights for policy 1, policy_version 31052 (0.0008) [2023-10-08 09:03:14,627][53852] Updated weights for policy 0, policy_version 31230 (0.0009) [2023-10-08 09:03:14,962][53885] Updated weights for policy 1, policy_version 31062 (0.0008) [2023-10-08 09:03:15,326][53885] Updated weights for policy 1, policy_version 31072 (0.0008) [2023-10-08 09:03:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 63799296. Throughput: 0: 1826.3, 1: 1815.8. Samples: 15955904. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 09:03:17,016][52710] Avg episode reward: [(0, '29.050'), (1, '29.420')] [2023-10-08 09:03:18,294][53852] Updated weights for policy 0, policy_version 31240 (0.0010) [2023-10-08 09:03:18,661][53852] Updated weights for policy 0, policy_version 31250 (0.0009) [2023-10-08 09:03:19,034][53852] Updated weights for policy 0, policy_version 31260 (0.0007) [2023-10-08 09:03:19,042][53885] Updated weights for policy 1, policy_version 31082 (0.0010) [2023-10-08 09:03:19,412][53885] Updated weights for policy 1, policy_version 31092 (0.0009) [2023-10-08 09:03:19,786][53885] Updated weights for policy 1, policy_version 31102 (0.0011) [2023-10-08 09:03:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 63864832. Throughput: 0: 1827.3, 1: 1810.5. Samples: 15978810. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 09:03:22,016][52710] Avg episode reward: [(0, '29.220'), (1, '30.760')] [2023-10-08 09:03:22,738][53852] Updated weights for policy 0, policy_version 31270 (0.0009) [2023-10-08 09:03:23,102][53852] Updated weights for policy 0, policy_version 31280 (0.0008) [2023-10-08 09:03:23,463][53885] Updated weights for policy 1, policy_version 31112 (0.0008) [2023-10-08 09:03:23,470][53852] Updated weights for policy 0, policy_version 31290 (0.0008) [2023-10-08 09:03:23,837][53885] Updated weights for policy 1, policy_version 31122 (0.0008) [2023-10-08 09:03:24,200][53885] Updated weights for policy 1, policy_version 31132 (0.0009) [2023-10-08 09:03:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63930368. Throughput: 0: 1827.8, 1: 1811.5. Samples: 15988786. Policy #0 lag: (min: 31.0, avg: 31.8, max: 50.0) [2023-10-08 09:03:27,016][52710] Avg episode reward: [(0, '28.530'), (1, '28.680')] [2023-10-08 09:03:27,129][53852] Updated weights for policy 0, policy_version 31300 (0.0008) [2023-10-08 09:03:27,497][53852] Updated weights for policy 0, policy_version 31310 (0.0008) [2023-10-08 09:03:27,861][53885] Updated weights for policy 1, policy_version 31142 (0.0007) [2023-10-08 09:03:27,870][53852] Updated weights for policy 0, policy_version 31320 (0.0010) [2023-10-08 09:03:28,231][53885] Updated weights for policy 1, policy_version 31152 (0.0008) [2023-10-08 09:03:28,607][53885] Updated weights for policy 1, policy_version 31162 (0.0008) [2023-10-08 09:03:31,710][53852] Updated weights for policy 0, policy_version 31330 (0.0008) [2023-10-08 09:03:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63995904. Throughput: 0: 1828.3, 1: 1815.9. Samples: 16011760. Policy #0 lag: (min: 0.0, avg: 13.1, max: 32.0) [2023-10-08 09:03:32,016][52710] Avg episode reward: [(0, '29.590'), (1, '29.280')] [2023-10-08 09:03:32,108][53852] Updated weights for policy 0, policy_version 31340 (0.0007) [2023-10-08 09:03:32,259][53885] Updated weights for policy 1, policy_version 31172 (0.0008) [2023-10-08 09:03:32,476][53852] Updated weights for policy 0, policy_version 31350 (0.0007) [2023-10-08 09:03:32,625][53885] Updated weights for policy 1, policy_version 31182 (0.0009) [2023-10-08 09:03:32,852][53852] Updated weights for policy 0, policy_version 31360 (0.0008) [2023-10-08 09:03:32,999][53885] Updated weights for policy 1, policy_version 31192 (0.0008) [2023-10-08 09:03:36,635][53852] Updated weights for policy 0, policy_version 31370 (0.0008) [2023-10-08 09:03:36,641][53885] Updated weights for policy 1, policy_version 31202 (0.0007) [2023-10-08 09:03:37,003][53885] Updated weights for policy 1, policy_version 31212 (0.0007) [2023-10-08 09:03:37,008][53852] Updated weights for policy 0, policy_version 31380 (0.0008) [2023-10-08 09:03:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 64061440. Throughput: 0: 1825.2, 1: 1817.3. Samples: 16034044. Policy #0 lag: (min: 0.0, avg: 13.1, max: 32.0) [2023-10-08 09:03:37,015][52710] Avg episode reward: [(0, '28.990'), (1, '28.370')] [2023-10-08 09:03:37,372][53885] Updated weights for policy 1, policy_version 31222 (0.0008) [2023-10-08 09:03:37,376][53852] Updated weights for policy 0, policy_version 31390 (0.0007) [2023-10-08 09:03:37,734][53885] Updated weights for policy 1, policy_version 31232 (0.0008) [2023-10-08 09:03:41,125][53852] Updated weights for policy 0, policy_version 31400 (0.0008) [2023-10-08 09:03:41,489][53852] Updated weights for policy 0, policy_version 31410 (0.0008) [2023-10-08 09:03:41,500][53885] Updated weights for policy 1, policy_version 31242 (0.0009) [2023-10-08 09:03:41,857][53852] Updated weights for policy 0, policy_version 31420 (0.0007) [2023-10-08 09:03:41,871][53885] Updated weights for policy 1, policy_version 31252 (0.0008) [2023-10-08 09:03:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64159744. Throughput: 0: 1826.0, 1: 1813.5. Samples: 16044262. Policy #0 lag: (min: 0.0, avg: 13.1, max: 32.0) [2023-10-08 09:03:42,015][52710] Avg episode reward: [(0, '30.170'), (1, '31.280')] [2023-10-08 09:03:42,242][53885] Updated weights for policy 1, policy_version 31262 (0.0010) [2023-10-08 09:03:45,480][53852] Updated weights for policy 0, policy_version 31430 (0.0008) [2023-10-08 09:03:45,851][53852] Updated weights for policy 0, policy_version 31440 (0.0007) [2023-10-08 09:03:46,142][53885] Updated weights for policy 1, policy_version 31272 (0.0007) [2023-10-08 09:03:46,212][53852] Updated weights for policy 0, policy_version 31450 (0.0007) [2023-10-08 09:03:46,520][53885] Updated weights for policy 1, policy_version 31282 (0.0007) [2023-10-08 09:03:46,896][53885] Updated weights for policy 1, policy_version 31292 (0.0007) [2023-10-08 09:03:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64225280. Throughput: 0: 1827.1, 1: 1823.0. Samples: 16067106. Policy #0 lag: (min: 0.0, avg: 13.1, max: 32.0) [2023-10-08 09:03:47,016][52710] Avg episode reward: [(0, '29.620'), (1, '30.650')] [2023-10-08 09:03:49,942][53852] Updated weights for policy 0, policy_version 31460 (0.0008) [2023-10-08 09:03:50,309][53852] Updated weights for policy 0, policy_version 31470 (0.0008) [2023-10-08 09:03:50,512][53885] Updated weights for policy 1, policy_version 31302 (0.0008) [2023-10-08 09:03:50,674][53852] Updated weights for policy 0, policy_version 31480 (0.0008) [2023-10-08 09:03:50,884][53885] Updated weights for policy 1, policy_version 31312 (0.0008) [2023-10-08 09:03:51,259][53885] Updated weights for policy 1, policy_version 31322 (0.0010) [2023-10-08 09:03:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 64323584. Throughput: 0: 1829.4, 1: 1818.5. Samples: 16087154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:03:52,016][52710] Avg episode reward: [(0, '30.800'), (1, '27.440')] [2023-10-08 09:03:54,283][53852] Updated weights for policy 0, policy_version 31490 (0.0009) [2023-10-08 09:03:54,648][53852] Updated weights for policy 0, policy_version 31500 (0.0008) [2023-10-08 09:03:54,736][53885] Updated weights for policy 1, policy_version 31332 (0.0007) [2023-10-08 09:03:55,015][53852] Updated weights for policy 0, policy_version 31510 (0.0007) [2023-10-08 09:03:55,099][53885] Updated weights for policy 1, policy_version 31342 (0.0008) [2023-10-08 09:03:55,386][53852] Updated weights for policy 0, policy_version 31520 (0.0008) [2023-10-08 09:03:55,466][53885] Updated weights for policy 1, policy_version 31352 (0.0008) [2023-10-08 09:03:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 64389120. Throughput: 0: 1826.0, 1: 1824.2. Samples: 16099844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:03:57,016][52710] Avg episode reward: [(0, '26.580'), (1, '29.400')] [2023-10-08 09:03:58,974][53885] Updated weights for policy 1, policy_version 31362 (0.0007) [2023-10-08 09:03:58,985][53852] Updated weights for policy 0, policy_version 31530 (0.0010) [2023-10-08 09:03:59,335][53885] Updated weights for policy 1, policy_version 31372 (0.0009) [2023-10-08 09:03:59,348][53852] Updated weights for policy 0, policy_version 31540 (0.0009) [2023-10-08 09:03:59,701][53885] Updated weights for policy 1, policy_version 31382 (0.0007) [2023-10-08 09:03:59,724][53852] Updated weights for policy 0, policy_version 31550 (0.0007) [2023-10-08 09:04:00,066][53885] Updated weights for policy 1, policy_version 31392 (0.0008) [2023-10-08 09:04:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 64454656. Throughput: 0: 1824.7, 1: 1836.0. Samples: 16120634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:04:02,016][52710] Avg episode reward: [(0, '26.870'), (1, '28.490')] [2023-10-08 09:04:03,260][53852] Updated weights for policy 0, policy_version 31560 (0.0007) [2023-10-08 09:04:03,623][53852] Updated weights for policy 0, policy_version 31570 (0.0007) [2023-10-08 09:04:03,637][53885] Updated weights for policy 1, policy_version 31402 (0.0007) [2023-10-08 09:04:03,995][53852] Updated weights for policy 0, policy_version 31580 (0.0007) [2023-10-08 09:04:04,010][53885] Updated weights for policy 1, policy_version 31412 (0.0007) [2023-10-08 09:04:04,370][53885] Updated weights for policy 1, policy_version 31422 (0.0008) [2023-10-08 09:04:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64520192. Throughput: 0: 1825.6, 1: 1841.6. Samples: 16143830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:04:07,016][52710] Avg episode reward: [(0, '27.370'), (1, '28.620')] [2023-10-08 09:04:07,023][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000031584_32342016.pth... [2023-10-08 09:04:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000031424_32178176.pth... [2023-10-08 09:04:07,053][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000029888_30605312.pth [2023-10-08 09:04:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000029728_30441472.pth [2023-10-08 09:04:07,616][53852] Updated weights for policy 0, policy_version 31590 (0.0007) [2023-10-08 09:04:07,982][53852] Updated weights for policy 0, policy_version 31600 (0.0008) [2023-10-08 09:04:08,062][53885] Updated weights for policy 1, policy_version 31432 (0.0008) [2023-10-08 09:04:08,354][53852] Updated weights for policy 0, policy_version 31610 (0.0008) [2023-10-08 09:04:08,434][53885] Updated weights for policy 1, policy_version 31442 (0.0007) [2023-10-08 09:04:08,802][53885] Updated weights for policy 1, policy_version 31452 (0.0007) [2023-10-08 09:04:12,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64585728. Throughput: 0: 1827.4, 1: 1840.8. Samples: 16153854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:04:12,016][52710] Avg episode reward: [(0, '29.120'), (1, '29.910')] [2023-10-08 09:04:12,092][53852] Updated weights for policy 0, policy_version 31620 (0.0007) [2023-10-08 09:04:12,427][53885] Updated weights for policy 1, policy_version 31462 (0.0007) [2023-10-08 09:04:12,460][53852] Updated weights for policy 0, policy_version 31630 (0.0008) [2023-10-08 09:04:12,791][53885] Updated weights for policy 1, policy_version 31472 (0.0008) [2023-10-08 09:04:12,819][53852] Updated weights for policy 0, policy_version 31640 (0.0007) [2023-10-08 09:04:13,156][53885] Updated weights for policy 1, policy_version 31482 (0.0007) [2023-10-08 09:04:16,445][53852] Updated weights for policy 0, policy_version 31650 (0.0007) [2023-10-08 09:04:16,822][53852] Updated weights for policy 0, policy_version 31660 (0.0007) [2023-10-08 09:04:16,927][53885] Updated weights for policy 1, policy_version 31492 (0.0010) [2023-10-08 09:04:17,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 64651264. Throughput: 0: 1828.5, 1: 1842.3. Samples: 16176948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:04:17,016][52710] Avg episode reward: [(0, '28.680'), (1, '29.970')] [2023-10-08 09:04:17,204][53852] Updated weights for policy 0, policy_version 31670 (0.0007) [2023-10-08 09:04:17,295][53885] Updated weights for policy 1, policy_version 31502 (0.0008) [2023-10-08 09:04:17,572][53852] Updated weights for policy 0, policy_version 31680 (0.0009) [2023-10-08 09:04:17,662][53885] Updated weights for policy 1, policy_version 31512 (0.0009) [2023-10-08 09:04:21,152][53852] Updated weights for policy 0, policy_version 31690 (0.0010) [2023-10-08 09:04:21,400][53885] Updated weights for policy 1, policy_version 31522 (0.0008) [2023-10-08 09:04:21,518][53852] Updated weights for policy 0, policy_version 31700 (0.0009) [2023-10-08 09:04:21,757][53885] Updated weights for policy 1, policy_version 31532 (0.0008) [2023-10-08 09:04:21,884][53852] Updated weights for policy 0, policy_version 31710 (0.0008) [2023-10-08 09:04:22,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64749568. Throughput: 0: 1823.3, 1: 1836.4. Samples: 16198732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:04:22,016][52710] Avg episode reward: [(0, '31.060'), (1, '29.030')] [2023-10-08 09:04:22,123][53885] Updated weights for policy 1, policy_version 31542 (0.0009) [2023-10-08 09:04:22,486][53885] Updated weights for policy 1, policy_version 31552 (0.0011) [2023-10-08 09:04:25,486][53852] Updated weights for policy 0, policy_version 31720 (0.0008) [2023-10-08 09:04:25,854][53852] Updated weights for policy 0, policy_version 31730 (0.0008) [2023-10-08 09:04:26,129][53885] Updated weights for policy 1, policy_version 31562 (0.0008) [2023-10-08 09:04:26,229][53852] Updated weights for policy 0, policy_version 31740 (0.0009) [2023-10-08 09:04:26,503][53885] Updated weights for policy 1, policy_version 31572 (0.0008) [2023-10-08 09:04:26,874][53885] Updated weights for policy 1, policy_version 31582 (0.0010) [2023-10-08 09:04:27,015][52710] Fps is (10 sec: 19661.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 64847872. Throughput: 0: 1835.4, 1: 1842.7. Samples: 16209776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:04:27,015][52710] Avg episode reward: [(0, '31.500'), (1, '30.790')] [2023-10-08 09:04:27,016][53500] Saving new best policy, reward=31.500! [2023-10-08 09:04:29,957][53852] Updated weights for policy 0, policy_version 31750 (0.0009) [2023-10-08 09:04:30,332][53852] Updated weights for policy 0, policy_version 31760 (0.0010) [2023-10-08 09:04:30,499][53885] Updated weights for policy 1, policy_version 31592 (0.0008) [2023-10-08 09:04:30,692][53852] Updated weights for policy 0, policy_version 31770 (0.0008) [2023-10-08 09:04:30,877][53885] Updated weights for policy 1, policy_version 31602 (0.0007) [2023-10-08 09:04:31,244][53885] Updated weights for policy 1, policy_version 31612 (0.0011) [2023-10-08 09:04:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 64913408. Throughput: 0: 1819.7, 1: 1827.6. Samples: 16231232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:04:32,016][52710] Avg episode reward: [(0, '32.420'), (1, '28.460')] [2023-10-08 09:04:32,018][53500] Saving new best policy, reward=32.420! [2023-10-08 09:04:34,234][53852] Updated weights for policy 0, policy_version 31780 (0.0008) [2023-10-08 09:04:34,606][53852] Updated weights for policy 0, policy_version 31790 (0.0011) [2023-10-08 09:04:34,976][53852] Updated weights for policy 0, policy_version 31800 (0.0009) [2023-10-08 09:04:35,028][53885] Updated weights for policy 1, policy_version 31622 (0.0009) [2023-10-08 09:04:35,390][53885] Updated weights for policy 1, policy_version 31632 (0.0010) [2023-10-08 09:04:35,760][53885] Updated weights for policy 1, policy_version 31642 (0.0009) [2023-10-08 09:04:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 64978944. Throughput: 0: 1839.5, 1: 1838.3. Samples: 16252652. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-08 09:04:37,016][52710] Avg episode reward: [(0, '30.240'), (1, '31.720')] [2023-10-08 09:04:38,563][53852] Updated weights for policy 0, policy_version 31810 (0.0007) [2023-10-08 09:04:38,931][53852] Updated weights for policy 0, policy_version 31820 (0.0009) [2023-10-08 09:04:39,304][53852] Updated weights for policy 0, policy_version 31830 (0.0009) [2023-10-08 09:04:39,440][53885] Updated weights for policy 1, policy_version 31652 (0.0009) [2023-10-08 09:04:39,684][53852] Updated weights for policy 0, policy_version 31840 (0.0007) [2023-10-08 09:04:39,813][53885] Updated weights for policy 1, policy_version 31662 (0.0008) [2023-10-08 09:04:40,196][53885] Updated weights for policy 1, policy_version 31672 (0.0011) [2023-10-08 09:04:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65044480. Throughput: 0: 1825.5, 1: 1827.9. Samples: 16264248. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-08 09:04:42,015][52710] Avg episode reward: [(0, '27.600'), (1, '33.300')] [2023-10-08 09:04:43,533][53852] Updated weights for policy 0, policy_version 31850 (0.0010) [2023-10-08 09:04:43,898][53852] Updated weights for policy 0, policy_version 31860 (0.0010) [2023-10-08 09:04:43,920][53885] Updated weights for policy 1, policy_version 31682 (0.0011) [2023-10-08 09:04:44,265][53852] Updated weights for policy 0, policy_version 31870 (0.0007) [2023-10-08 09:04:44,278][53885] Updated weights for policy 1, policy_version 31692 (0.0009) [2023-10-08 09:04:44,645][53885] Updated weights for policy 1, policy_version 31702 (0.0009) [2023-10-08 09:04:45,014][53885] Updated weights for policy 1, policy_version 31712 (0.0008) [2023-10-08 09:04:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65110016. Throughput: 0: 1838.3, 1: 1827.4. Samples: 16285592. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-08 09:04:47,016][52710] Avg episode reward: [(0, '30.140'), (1, '29.780')] [2023-10-08 09:04:48,025][53852] Updated weights for policy 0, policy_version 31880 (0.0008) [2023-10-08 09:04:48,397][53852] Updated weights for policy 0, policy_version 31890 (0.0007) [2023-10-08 09:04:48,699][53885] Updated weights for policy 1, policy_version 31722 (0.0008) [2023-10-08 09:04:48,775][53852] Updated weights for policy 0, policy_version 31900 (0.0009) [2023-10-08 09:04:49,073][53885] Updated weights for policy 1, policy_version 31732 (0.0008) [2023-10-08 09:04:49,432][53885] Updated weights for policy 1, policy_version 31742 (0.0007) [2023-10-08 09:04:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65175552. Throughput: 0: 1836.5, 1: 1823.0. Samples: 16308508. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-08 09:04:52,015][52710] Avg episode reward: [(0, '29.980'), (1, '30.950')] [2023-10-08 09:04:52,392][53852] Updated weights for policy 0, policy_version 31910 (0.0009) [2023-10-08 09:04:52,767][53852] Updated weights for policy 0, policy_version 31920 (0.0010) [2023-10-08 09:04:53,129][53852] Updated weights for policy 0, policy_version 31930 (0.0008) [2023-10-08 09:04:53,173][53885] Updated weights for policy 1, policy_version 31752 (0.0007) [2023-10-08 09:04:53,550][53885] Updated weights for policy 1, policy_version 31762 (0.0011) [2023-10-08 09:04:53,918][53885] Updated weights for policy 1, policy_version 31772 (0.0008) [2023-10-08 09:04:56,734][53852] Updated weights for policy 0, policy_version 31940 (0.0007) [2023-10-08 09:04:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65241088. Throughput: 0: 1834.7, 1: 1823.8. Samples: 16318488. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) [2023-10-08 09:04:57,015][52710] Avg episode reward: [(0, '29.730'), (1, '31.700')] [2023-10-08 09:04:57,102][53852] Updated weights for policy 0, policy_version 31950 (0.0009) [2023-10-08 09:04:57,391][53885] Updated weights for policy 1, policy_version 31782 (0.0007) [2023-10-08 09:04:57,479][53852] Updated weights for policy 0, policy_version 31960 (0.0009) [2023-10-08 09:04:57,766][53885] Updated weights for policy 1, policy_version 31792 (0.0009) [2023-10-08 09:04:58,130][53885] Updated weights for policy 1, policy_version 31802 (0.0009) [2023-10-08 09:05:01,254][53852] Updated weights for policy 0, policy_version 31970 (0.0009) [2023-10-08 09:05:01,634][53852] Updated weights for policy 0, policy_version 31980 (0.0007) [2023-10-08 09:05:01,818][53885] Updated weights for policy 1, policy_version 31812 (0.0008) [2023-10-08 09:05:01,994][53852] Updated weights for policy 0, policy_version 31990 (0.0007) [2023-10-08 09:05:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65306624. Throughput: 0: 1836.0, 1: 1821.9. Samples: 16341552. Policy #0 lag: (min: 6.0, avg: 6.9, max: 27.0) [2023-10-08 09:05:02,016][52710] Avg episode reward: [(0, '33.120'), (1, '30.320')] [2023-10-08 09:05:02,195][53885] Updated weights for policy 1, policy_version 31822 (0.0007) [2023-10-08 09:05:02,353][53500] Saving new best policy, reward=33.120! [2023-10-08 09:05:02,355][53852] Updated weights for policy 0, policy_version 32000 (0.0007) [2023-10-08 09:05:02,572][53885] Updated weights for policy 1, policy_version 31832 (0.0009) [2023-10-08 09:05:06,099][53852] Updated weights for policy 0, policy_version 32010 (0.0007) [2023-10-08 09:05:06,265][53885] Updated weights for policy 1, policy_version 31842 (0.0009) [2023-10-08 09:05:06,469][53852] Updated weights for policy 0, policy_version 32020 (0.0007) [2023-10-08 09:05:06,639][53885] Updated weights for policy 1, policy_version 31852 (0.0008) [2023-10-08 09:05:06,837][53852] Updated weights for policy 0, policy_version 32030 (0.0007) [2023-10-08 09:05:07,006][53885] Updated weights for policy 1, policy_version 31862 (0.0008) [2023-10-08 09:05:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65404928. Throughput: 0: 1834.9, 1: 1820.2. Samples: 16363210. Policy #0 lag: (min: 6.0, avg: 6.9, max: 27.0) [2023-10-08 09:05:07,016][52710] Avg episode reward: [(0, '32.940'), (1, '29.440')] [2023-10-08 09:05:07,368][53885] Updated weights for policy 1, policy_version 31872 (0.0008) [2023-10-08 09:05:10,612][53852] Updated weights for policy 0, policy_version 32040 (0.0009) [2023-10-08 09:05:10,979][53852] Updated weights for policy 0, policy_version 32050 (0.0008) [2023-10-08 09:05:11,022][53885] Updated weights for policy 1, policy_version 31882 (0.0008) [2023-10-08 09:05:11,349][53852] Updated weights for policy 0, policy_version 32060 (0.0010) [2023-10-08 09:05:11,387][53885] Updated weights for policy 1, policy_version 31892 (0.0009) [2023-10-08 09:05:11,758][53885] Updated weights for policy 1, policy_version 31902 (0.0007) [2023-10-08 09:05:12,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 65503232. Throughput: 0: 1837.8, 1: 1828.7. Samples: 16374768. Policy #0 lag: (min: 6.0, avg: 6.9, max: 27.0) [2023-10-08 09:05:12,016][52710] Avg episode reward: [(0, '29.550'), (1, '28.920')] [2023-10-08 09:05:15,037][53852] Updated weights for policy 0, policy_version 32070 (0.0007) [2023-10-08 09:05:15,334][53885] Updated weights for policy 1, policy_version 31912 (0.0007) [2023-10-08 09:05:15,410][53852] Updated weights for policy 0, policy_version 32080 (0.0008) [2023-10-08 09:05:15,716][53885] Updated weights for policy 1, policy_version 31922 (0.0007) [2023-10-08 09:05:15,773][53852] Updated weights for policy 0, policy_version 32090 (0.0008) [2023-10-08 09:05:16,074][53885] Updated weights for policy 1, policy_version 31932 (0.0008) [2023-10-08 09:05:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 65568768. Throughput: 0: 1836.5, 1: 1824.6. Samples: 16395980. Policy #0 lag: (min: 6.0, avg: 6.9, max: 27.0) [2023-10-08 09:05:17,015][52710] Avg episode reward: [(0, '29.980'), (1, '27.690')] [2023-10-08 09:05:19,542][53885] Updated weights for policy 1, policy_version 31942 (0.0007) [2023-10-08 09:05:19,596][53852] Updated weights for policy 0, policy_version 32100 (0.0008) [2023-10-08 09:05:19,911][53885] Updated weights for policy 1, policy_version 31952 (0.0007) [2023-10-08 09:05:19,955][53852] Updated weights for policy 0, policy_version 32110 (0.0009) [2023-10-08 09:05:20,282][53885] Updated weights for policy 1, policy_version 31962 (0.0008) [2023-10-08 09:05:20,326][53852] Updated weights for policy 0, policy_version 32120 (0.0009) [2023-10-08 09:05:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65634304. Throughput: 0: 1820.4, 1: 1836.2. Samples: 16417200. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:05:22,016][52710] Avg episode reward: [(0, '30.690'), (1, '30.310')] [2023-10-08 09:05:24,052][53885] Updated weights for policy 1, policy_version 31972 (0.0008) [2023-10-08 09:05:24,126][53852] Updated weights for policy 0, policy_version 32130 (0.0009) [2023-10-08 09:05:24,417][53885] Updated weights for policy 1, policy_version 31982 (0.0008) [2023-10-08 09:05:24,495][53852] Updated weights for policy 0, policy_version 32140 (0.0008) [2023-10-08 09:05:24,777][53885] Updated weights for policy 1, policy_version 31992 (0.0007) [2023-10-08 09:05:24,864][53852] Updated weights for policy 0, policy_version 32150 (0.0009) [2023-10-08 09:05:25,228][53852] Updated weights for policy 0, policy_version 32160 (0.0009) [2023-10-08 09:05:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 65699840. Throughput: 0: 1828.9, 1: 1820.3. Samples: 16428464. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:05:27,015][52710] Avg episode reward: [(0, '27.700'), (1, '30.750')] [2023-10-08 09:05:28,573][53885] Updated weights for policy 1, policy_version 32002 (0.0008) [2023-10-08 09:05:28,824][53852] Updated weights for policy 0, policy_version 32170 (0.0007) [2023-10-08 09:05:28,940][53885] Updated weights for policy 1, policy_version 32012 (0.0008) [2023-10-08 09:05:29,184][53852] Updated weights for policy 0, policy_version 32180 (0.0007) [2023-10-08 09:05:29,307][53885] Updated weights for policy 1, policy_version 32022 (0.0007) [2023-10-08 09:05:29,553][53852] Updated weights for policy 0, policy_version 32190 (0.0007) [2023-10-08 09:05:29,666][53885] Updated weights for policy 1, policy_version 32032 (0.0008) [2023-10-08 09:05:32,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 65765376. Throughput: 0: 1820.8, 1: 1828.1. Samples: 16449792. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:05:32,015][52710] Avg episode reward: [(0, '30.630'), (1, '29.190')] [2023-10-08 09:05:32,878][53852] Updated weights for policy 0, policy_version 32200 (0.0010) [2023-10-08 09:05:33,243][53852] Updated weights for policy 0, policy_version 32210 (0.0008) [2023-10-08 09:05:33,376][53885] Updated weights for policy 1, policy_version 32042 (0.0007) [2023-10-08 09:05:33,610][53852] Updated weights for policy 0, policy_version 32220 (0.0008) [2023-10-08 09:05:33,739][53885] Updated weights for policy 1, policy_version 32052 (0.0007) [2023-10-08 09:05:34,103][53885] Updated weights for policy 1, policy_version 32062 (0.0007) [2023-10-08 09:05:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65830912. Throughput: 0: 1823.5, 1: 1828.0. Samples: 16472830. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:05:37,016][52710] Avg episode reward: [(0, '26.070'), (1, '30.890')] [2023-10-08 09:05:37,071][53852] Updated weights for policy 0, policy_version 32230 (0.0008) [2023-10-08 09:05:37,444][53852] Updated weights for policy 0, policy_version 32240 (0.0009) [2023-10-08 09:05:37,753][53885] Updated weights for policy 1, policy_version 32072 (0.0007) [2023-10-08 09:05:37,822][53852] Updated weights for policy 0, policy_version 32250 (0.0008) [2023-10-08 09:05:38,129][53885] Updated weights for policy 1, policy_version 32082 (0.0009) [2023-10-08 09:05:38,491][53885] Updated weights for policy 1, policy_version 32092 (0.0009) [2023-10-08 09:05:41,506][53852] Updated weights for policy 0, policy_version 32260 (0.0008) [2023-10-08 09:05:41,875][53852] Updated weights for policy 0, policy_version 32270 (0.0008) [2023-10-08 09:05:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 65896448. Throughput: 0: 1823.9, 1: 1826.0. Samples: 16482736. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) [2023-10-08 09:05:42,016][52710] Avg episode reward: [(0, '26.470'), (1, '32.090')] [2023-10-08 09:05:42,202][53885] Updated weights for policy 1, policy_version 32102 (0.0008) [2023-10-08 09:05:42,243][53852] Updated weights for policy 0, policy_version 32280 (0.0007) [2023-10-08 09:05:42,576][53885] Updated weights for policy 1, policy_version 32112 (0.0009) [2023-10-08 09:05:42,952][53885] Updated weights for policy 1, policy_version 32122 (0.0009) [2023-10-08 09:05:45,841][53852] Updated weights for policy 0, policy_version 32290 (0.0008) [2023-10-08 09:05:46,217][53852] Updated weights for policy 0, policy_version 32300 (0.0008) [2023-10-08 09:05:46,586][53852] Updated weights for policy 0, policy_version 32310 (0.0008) [2023-10-08 09:05:46,643][53885] Updated weights for policy 1, policy_version 32132 (0.0009) [2023-10-08 09:05:46,949][53852] Updated weights for policy 0, policy_version 32320 (0.0009) [2023-10-08 09:05:47,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65994752. Throughput: 0: 1826.6, 1: 1823.9. Samples: 16505822. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 09:05:47,016][52710] Avg episode reward: [(0, '27.120'), (1, '31.330')] [2023-10-08 09:05:47,017][53885] Updated weights for policy 1, policy_version 32142 (0.0007) [2023-10-08 09:05:47,379][53885] Updated weights for policy 1, policy_version 32152 (0.0010) [2023-10-08 09:05:50,634][53852] Updated weights for policy 0, policy_version 32330 (0.0008) [2023-10-08 09:05:50,934][53885] Updated weights for policy 1, policy_version 32162 (0.0009) [2023-10-08 09:05:51,007][53852] Updated weights for policy 0, policy_version 32340 (0.0008) [2023-10-08 09:05:51,297][53885] Updated weights for policy 1, policy_version 32172 (0.0009) [2023-10-08 09:05:51,376][53852] Updated weights for policy 0, policy_version 32350 (0.0008) [2023-10-08 09:05:51,673][53885] Updated weights for policy 1, policy_version 32182 (0.0008) [2023-10-08 09:05:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66060288. Throughput: 0: 1811.6, 1: 1816.8. Samples: 16526490. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 09:05:52,016][52710] Avg episode reward: [(0, '28.160'), (1, '33.230')] [2023-10-08 09:05:52,034][53885] Updated weights for policy 1, policy_version 32192 (0.0011) [2023-10-08 09:05:55,070][53852] Updated weights for policy 0, policy_version 32360 (0.0009) [2023-10-08 09:05:55,447][53852] Updated weights for policy 0, policy_version 32370 (0.0008) [2023-10-08 09:05:55,725][53885] Updated weights for policy 1, policy_version 32202 (0.0007) [2023-10-08 09:05:55,811][53852] Updated weights for policy 0, policy_version 32380 (0.0008) [2023-10-08 09:05:56,090][53885] Updated weights for policy 1, policy_version 32212 (0.0009) [2023-10-08 09:05:56,455][53885] Updated weights for policy 1, policy_version 32222 (0.0010) [2023-10-08 09:05:57,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 66158592. Throughput: 0: 1830.1, 1: 1823.8. Samples: 16539194. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 09:05:57,016][52710] Avg episode reward: [(0, '27.730'), (1, '30.840')] [2023-10-08 09:05:59,464][53852] Updated weights for policy 0, policy_version 32390 (0.0007) [2023-10-08 09:05:59,825][53852] Updated weights for policy 0, policy_version 32400 (0.0007) [2023-10-08 09:06:00,130][53885] Updated weights for policy 1, policy_version 32232 (0.0008) [2023-10-08 09:06:00,196][53852] Updated weights for policy 0, policy_version 32410 (0.0008) [2023-10-08 09:06:00,501][53885] Updated weights for policy 1, policy_version 32242 (0.0007) [2023-10-08 09:06:00,870][53885] Updated weights for policy 1, policy_version 32252 (0.0008) [2023-10-08 09:06:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 66224128. Throughput: 0: 1819.2, 1: 1816.0. Samples: 16559562. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 09:06:02,016][52710] Avg episode reward: [(0, '27.460'), (1, '30.190')] [2023-10-08 09:06:03,968][53852] Updated weights for policy 0, policy_version 32420 (0.0010) [2023-10-08 09:06:04,342][53852] Updated weights for policy 0, policy_version 32430 (0.0008) [2023-10-08 09:06:04,616][53885] Updated weights for policy 1, policy_version 32262 (0.0008) [2023-10-08 09:06:04,711][53852] Updated weights for policy 0, policy_version 32440 (0.0007) [2023-10-08 09:06:04,983][53885] Updated weights for policy 1, policy_version 32272 (0.0007) [2023-10-08 09:06:05,349][53885] Updated weights for policy 1, policy_version 32282 (0.0009) [2023-10-08 09:06:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66289664. Throughput: 0: 1843.8, 1: 1816.9. Samples: 16581930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:07,015][52710] Avg episode reward: [(0, '30.190'), (1, '30.940')] [2023-10-08 09:06:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000032288_33062912.pth... [2023-10-08 09:06:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000032448_33226752.pth... [2023-10-08 09:06:07,064][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000030752_31490048.pth [2023-10-08 09:06:07,069][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000030592_31326208.pth [2023-10-08 09:06:08,256][53852] Updated weights for policy 0, policy_version 32450 (0.0008) [2023-10-08 09:06:08,631][53852] Updated weights for policy 0, policy_version 32460 (0.0007) [2023-10-08 09:06:08,992][53885] Updated weights for policy 1, policy_version 32292 (0.0008) [2023-10-08 09:06:08,998][53852] Updated weights for policy 0, policy_version 32470 (0.0007) [2023-10-08 09:06:09,368][53885] Updated weights for policy 1, policy_version 32302 (0.0008) [2023-10-08 09:06:09,373][53852] Updated weights for policy 0, policy_version 32480 (0.0008) [2023-10-08 09:06:09,736][53885] Updated weights for policy 1, policy_version 32312 (0.0008) [2023-10-08 09:06:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 66355200. Throughput: 0: 1829.2, 1: 1818.4. Samples: 16592608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:12,016][52710] Avg episode reward: [(0, '30.830'), (1, '30.390')] [2023-10-08 09:06:13,089][53852] Updated weights for policy 0, policy_version 32490 (0.0007) [2023-10-08 09:06:13,456][53852] Updated weights for policy 0, policy_version 32500 (0.0008) [2023-10-08 09:06:13,622][53885] Updated weights for policy 1, policy_version 32322 (0.0009) [2023-10-08 09:06:13,833][53852] Updated weights for policy 0, policy_version 32510 (0.0007) [2023-10-08 09:06:13,988][53885] Updated weights for policy 1, policy_version 32332 (0.0008) [2023-10-08 09:06:14,350][53885] Updated weights for policy 1, policy_version 32342 (0.0008) [2023-10-08 09:06:14,724][53885] Updated weights for policy 1, policy_version 32352 (0.0009) [2023-10-08 09:06:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 66420736. Throughput: 0: 1851.2, 1: 1816.5. Samples: 16614838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:17,016][52710] Avg episode reward: [(0, '31.370'), (1, '30.480')] [2023-10-08 09:06:17,269][53852] Updated weights for policy 0, policy_version 32520 (0.0008) [2023-10-08 09:06:17,631][53852] Updated weights for policy 0, policy_version 32530 (0.0007) [2023-10-08 09:06:17,997][53852] Updated weights for policy 0, policy_version 32540 (0.0008) [2023-10-08 09:06:18,439][53885] Updated weights for policy 1, policy_version 32362 (0.0008) [2023-10-08 09:06:18,799][53885] Updated weights for policy 1, policy_version 32372 (0.0009) [2023-10-08 09:06:19,159][53885] Updated weights for policy 1, policy_version 32382 (0.0008) [2023-10-08 09:06:21,602][53852] Updated weights for policy 0, policy_version 32550 (0.0008) [2023-10-08 09:06:21,978][53852] Updated weights for policy 0, policy_version 32560 (0.0008) [2023-10-08 09:06:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 66486272. Throughput: 0: 1848.0, 1: 1815.2. Samples: 16637672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:22,016][52710] Avg episode reward: [(0, '28.790'), (1, '31.100')] [2023-10-08 09:06:22,360][53852] Updated weights for policy 0, policy_version 32570 (0.0007) [2023-10-08 09:06:22,995][53885] Updated weights for policy 1, policy_version 32392 (0.0010) [2023-10-08 09:06:23,361][53885] Updated weights for policy 1, policy_version 32402 (0.0008) [2023-10-08 09:06:23,728][53885] Updated weights for policy 1, policy_version 32412 (0.0008) [2023-10-08 09:06:26,017][53852] Updated weights for policy 0, policy_version 32580 (0.0008) [2023-10-08 09:06:26,394][53852] Updated weights for policy 0, policy_version 32590 (0.0008) [2023-10-08 09:06:26,769][53852] Updated weights for policy 0, policy_version 32600 (0.0008) [2023-10-08 09:06:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 66551808. Throughput: 0: 1855.9, 1: 1816.6. Samples: 16647998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:27,016][52710] Avg episode reward: [(0, '30.360'), (1, '33.440')] [2023-10-08 09:06:27,211][53885] Updated weights for policy 1, policy_version 32422 (0.0009) [2023-10-08 09:06:27,582][53885] Updated weights for policy 1, policy_version 32432 (0.0010) [2023-10-08 09:06:27,941][53885] Updated weights for policy 1, policy_version 32442 (0.0008) [2023-10-08 09:06:30,396][53852] Updated weights for policy 0, policy_version 32610 (0.0007) [2023-10-08 09:06:30,765][53852] Updated weights for policy 0, policy_version 32620 (0.0007) [2023-10-08 09:06:31,139][53852] Updated weights for policy 0, policy_version 32630 (0.0008) [2023-10-08 09:06:31,503][53852] Updated weights for policy 0, policy_version 32640 (0.0008) [2023-10-08 09:06:31,642][53885] Updated weights for policy 1, policy_version 32452 (0.0009) [2023-10-08 09:06:31,997][53885] Updated weights for policy 1, policy_version 32462 (0.0010) [2023-10-08 09:06:32,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66650112. Throughput: 0: 1844.6, 1: 1824.2. Samples: 16670916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:32,015][52710] Avg episode reward: [(0, '30.090'), (1, '30.570')] [2023-10-08 09:06:32,366][53885] Updated weights for policy 1, policy_version 32472 (0.0009) [2023-10-08 09:06:35,158][53852] Updated weights for policy 0, policy_version 32650 (0.0007) [2023-10-08 09:06:35,525][53852] Updated weights for policy 0, policy_version 32660 (0.0008) [2023-10-08 09:06:35,893][53852] Updated weights for policy 0, policy_version 32670 (0.0008) [2023-10-08 09:06:35,943][53885] Updated weights for policy 1, policy_version 32482 (0.0009) [2023-10-08 09:06:36,309][53885] Updated weights for policy 1, policy_version 32492 (0.0009) [2023-10-08 09:06:36,689][53885] Updated weights for policy 1, policy_version 32502 (0.0011) [2023-10-08 09:06:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66715648. Throughput: 0: 1855.4, 1: 1825.7. Samples: 16692140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:37,016][52710] Avg episode reward: [(0, '29.170'), (1, '32.430')] [2023-10-08 09:06:37,054][53885] Updated weights for policy 1, policy_version 32512 (0.0010) [2023-10-08 09:06:39,599][53852] Updated weights for policy 0, policy_version 32680 (0.0007) [2023-10-08 09:06:39,979][53852] Updated weights for policy 0, policy_version 32690 (0.0007) [2023-10-08 09:06:40,339][53852] Updated weights for policy 0, policy_version 32700 (0.0008) [2023-10-08 09:06:40,685][53885] Updated weights for policy 1, policy_version 32522 (0.0009) [2023-10-08 09:06:41,063][53885] Updated weights for policy 1, policy_version 32532 (0.0010) [2023-10-08 09:06:41,438][53885] Updated weights for policy 1, policy_version 32542 (0.0010) [2023-10-08 09:06:42,015][52710] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 66813952. Throughput: 0: 1841.0, 1: 1828.7. Samples: 16704332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:42,016][52710] Avg episode reward: [(0, '29.540'), (1, '32.810')] [2023-10-08 09:06:44,063][53852] Updated weights for policy 0, policy_version 32710 (0.0007) [2023-10-08 09:06:44,429][53852] Updated weights for policy 0, policy_version 32720 (0.0007) [2023-10-08 09:06:44,806][53852] Updated weights for policy 0, policy_version 32730 (0.0009) [2023-10-08 09:06:45,143][53885] Updated weights for policy 1, policy_version 32552 (0.0008) [2023-10-08 09:06:45,528][53885] Updated weights for policy 1, policy_version 32562 (0.0009) [2023-10-08 09:06:45,889][53885] Updated weights for policy 1, policy_version 32572 (0.0009) [2023-10-08 09:06:47,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66879488. Throughput: 0: 1845.6, 1: 1828.8. Samples: 16724910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:47,016][52710] Avg episode reward: [(0, '30.310'), (1, '32.510')] [2023-10-08 09:06:48,553][53852] Updated weights for policy 0, policy_version 32740 (0.0008) [2023-10-08 09:06:48,925][53852] Updated weights for policy 0, policy_version 32750 (0.0009) [2023-10-08 09:06:49,293][53852] Updated weights for policy 0, policy_version 32760 (0.0007) [2023-10-08 09:06:49,477][53885] Updated weights for policy 1, policy_version 32582 (0.0007) [2023-10-08 09:06:49,846][53885] Updated weights for policy 1, policy_version 32592 (0.0008) [2023-10-08 09:06:50,207][53885] Updated weights for policy 1, policy_version 32602 (0.0008) [2023-10-08 09:06:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66945024. Throughput: 0: 1841.2, 1: 1833.4. Samples: 16747288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:52,016][52710] Avg episode reward: [(0, '28.840'), (1, '34.050')] [2023-10-08 09:06:52,025][53594] Saving new best policy, reward=34.050! [2023-10-08 09:06:52,870][53852] Updated weights for policy 0, policy_version 32770 (0.0008) [2023-10-08 09:06:53,233][53852] Updated weights for policy 0, policy_version 32780 (0.0007) [2023-10-08 09:06:53,611][53852] Updated weights for policy 0, policy_version 32790 (0.0010) [2023-10-08 09:06:53,901][53885] Updated weights for policy 1, policy_version 32612 (0.0009) [2023-10-08 09:06:53,976][53852] Updated weights for policy 0, policy_version 32800 (0.0009) [2023-10-08 09:06:54,265][53885] Updated weights for policy 1, policy_version 32622 (0.0010) [2023-10-08 09:06:54,637][53885] Updated weights for policy 1, policy_version 32632 (0.0009) [2023-10-08 09:06:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 67010560. Throughput: 0: 1840.8, 1: 1828.4. Samples: 16757722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:06:57,016][52710] Avg episode reward: [(0, '30.940'), (1, '31.440')] [2023-10-08 09:06:57,566][53852] Updated weights for policy 0, policy_version 32810 (0.0007) [2023-10-08 09:06:57,934][53852] Updated weights for policy 0, policy_version 32820 (0.0007) [2023-10-08 09:06:58,311][53852] Updated weights for policy 0, policy_version 32830 (0.0009) [2023-10-08 09:06:58,389][53885] Updated weights for policy 1, policy_version 32642 (0.0007) [2023-10-08 09:06:58,766][53885] Updated weights for policy 1, policy_version 32652 (0.0009) [2023-10-08 09:06:59,132][53885] Updated weights for policy 1, policy_version 32662 (0.0007) [2023-10-08 09:06:59,497][53885] Updated weights for policy 1, policy_version 32672 (0.0007) [2023-10-08 09:07:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 67076096. Throughput: 0: 1837.6, 1: 1834.7. Samples: 16780094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:07:02,017][52710] Avg episode reward: [(0, '28.870'), (1, '33.580')] [2023-10-08 09:07:02,167][53852] Updated weights for policy 0, policy_version 32840 (0.0008) [2023-10-08 09:07:02,540][53852] Updated weights for policy 0, policy_version 32850 (0.0007) [2023-10-08 09:07:02,909][53852] Updated weights for policy 0, policy_version 32860 (0.0008) [2023-10-08 09:07:03,214][53885] Updated weights for policy 1, policy_version 32682 (0.0008) [2023-10-08 09:07:03,576][53885] Updated weights for policy 1, policy_version 32692 (0.0011) [2023-10-08 09:07:03,937][53885] Updated weights for policy 1, policy_version 32702 (0.0010) [2023-10-08 09:07:06,590][53852] Updated weights for policy 0, policy_version 32870 (0.0009) [2023-10-08 09:07:06,971][53852] Updated weights for policy 0, policy_version 32880 (0.0008) [2023-10-08 09:07:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 67141632. Throughput: 0: 1830.2, 1: 1838.2. Samples: 16802750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:07:07,016][52710] Avg episode reward: [(0, '28.290'), (1, '34.700')] [2023-10-08 09:07:07,025][53594] Saving new best policy, reward=34.700! [2023-10-08 09:07:07,341][53852] Updated weights for policy 0, policy_version 32890 (0.0007) [2023-10-08 09:07:07,655][53885] Updated weights for policy 1, policy_version 32712 (0.0008) [2023-10-08 09:07:08,023][53885] Updated weights for policy 1, policy_version 32722 (0.0007) [2023-10-08 09:07:08,389][53885] Updated weights for policy 1, policy_version 32732 (0.0007) [2023-10-08 09:07:10,902][53852] Updated weights for policy 0, policy_version 32900 (0.0007) [2023-10-08 09:07:11,273][53852] Updated weights for policy 0, policy_version 32910 (0.0007) [2023-10-08 09:07:11,646][53852] Updated weights for policy 0, policy_version 32920 (0.0007) [2023-10-08 09:07:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67239936. Throughput: 0: 1831.0, 1: 1837.7. Samples: 16813090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:07:12,016][52710] Avg episode reward: [(0, '27.800'), (1, '33.580')] [2023-10-08 09:07:12,077][53885] Updated weights for policy 1, policy_version 32742 (0.0008) [2023-10-08 09:07:12,444][53885] Updated weights for policy 1, policy_version 32752 (0.0010) [2023-10-08 09:07:12,814][53885] Updated weights for policy 1, policy_version 32762 (0.0007) [2023-10-08 09:07:15,298][53852] Updated weights for policy 0, policy_version 32930 (0.0009) [2023-10-08 09:07:15,667][53852] Updated weights for policy 0, policy_version 32940 (0.0008) [2023-10-08 09:07:16,038][53852] Updated weights for policy 0, policy_version 32950 (0.0009) [2023-10-08 09:07:16,389][53885] Updated weights for policy 1, policy_version 32772 (0.0009) [2023-10-08 09:07:16,403][53852] Updated weights for policy 0, policy_version 32960 (0.0010) [2023-10-08 09:07:16,763][53885] Updated weights for policy 1, policy_version 32782 (0.0007) [2023-10-08 09:07:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67305472. Throughput: 0: 1828.0, 1: 1834.2. Samples: 16835718. Policy #0 lag: (min: 6.0, avg: 7.3, max: 31.0) [2023-10-08 09:07:17,016][52710] Avg episode reward: [(0, '28.230'), (1, '30.260')] [2023-10-08 09:07:17,136][53885] Updated weights for policy 1, policy_version 32792 (0.0010) [2023-10-08 09:07:19,970][53852] Updated weights for policy 0, policy_version 32970 (0.0009) [2023-10-08 09:07:20,342][53852] Updated weights for policy 0, policy_version 32980 (0.0008) [2023-10-08 09:07:20,712][53852] Updated weights for policy 0, policy_version 32990 (0.0008) [2023-10-08 09:07:20,789][53885] Updated weights for policy 1, policy_version 32802 (0.0009) [2023-10-08 09:07:21,158][53885] Updated weights for policy 1, policy_version 32812 (0.0011) [2023-10-08 09:07:21,526][53885] Updated weights for policy 1, policy_version 32822 (0.0009) [2023-10-08 09:07:21,893][53885] Updated weights for policy 1, policy_version 32832 (0.0009) [2023-10-08 09:07:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 67403776. Throughput: 0: 1832.1, 1: 1826.8. Samples: 16856792. Policy #0 lag: (min: 6.0, avg: 7.3, max: 31.0) [2023-10-08 09:07:22,016][52710] Avg episode reward: [(0, '27.110'), (1, '32.210')] [2023-10-08 09:07:24,327][53852] Updated weights for policy 0, policy_version 33000 (0.0010) [2023-10-08 09:07:24,701][53852] Updated weights for policy 0, policy_version 33010 (0.0008) [2023-10-08 09:07:25,073][53852] Updated weights for policy 0, policy_version 33020 (0.0007) [2023-10-08 09:07:25,577][53885] Updated weights for policy 1, policy_version 32842 (0.0008) [2023-10-08 09:07:25,948][53885] Updated weights for policy 1, policy_version 32852 (0.0007) [2023-10-08 09:07:26,320][53885] Updated weights for policy 1, policy_version 32862 (0.0009) [2023-10-08 09:07:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 67469312. Throughput: 0: 1823.8, 1: 1828.6. Samples: 16868692. Policy #0 lag: (min: 6.0, avg: 7.3, max: 31.0) [2023-10-08 09:07:27,016][52710] Avg episode reward: [(0, '25.850'), (1, '33.230')] [2023-10-08 09:07:28,766][53852] Updated weights for policy 0, policy_version 33030 (0.0007) [2023-10-08 09:07:29,145][53852] Updated weights for policy 0, policy_version 33040 (0.0007) [2023-10-08 09:07:29,513][53852] Updated weights for policy 0, policy_version 33050 (0.0007) [2023-10-08 09:07:30,009][53885] Updated weights for policy 1, policy_version 32872 (0.0007) [2023-10-08 09:07:30,381][53885] Updated weights for policy 1, policy_version 32882 (0.0009) [2023-10-08 09:07:30,740][53885] Updated weights for policy 1, policy_version 32892 (0.0007) [2023-10-08 09:07:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 67534848. Throughput: 0: 1836.8, 1: 1828.8. Samples: 16889862. Policy #0 lag: (min: 6.0, avg: 7.3, max: 31.0) [2023-10-08 09:07:32,016][52710] Avg episode reward: [(0, '29.330'), (1, '32.690')] [2023-10-08 09:07:33,042][53852] Updated weights for policy 0, policy_version 33060 (0.0007) [2023-10-08 09:07:33,415][53852] Updated weights for policy 0, policy_version 33070 (0.0007) [2023-10-08 09:07:33,798][53852] Updated weights for policy 0, policy_version 33080 (0.0007) [2023-10-08 09:07:34,566][53885] Updated weights for policy 1, policy_version 32902 (0.0008) [2023-10-08 09:07:34,947][53885] Updated weights for policy 1, policy_version 32912 (0.0007) [2023-10-08 09:07:35,318][53885] Updated weights for policy 1, policy_version 32922 (0.0008) [2023-10-08 09:07:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 67600384. Throughput: 0: 1846.1, 1: 1826.0. Samples: 16912530. Policy #0 lag: (min: 6.0, avg: 7.3, max: 31.0) [2023-10-08 09:07:37,015][52710] Avg episode reward: [(0, '27.320'), (1, '31.070')] [2023-10-08 09:07:37,246][53852] Updated weights for policy 0, policy_version 33090 (0.0007) [2023-10-08 09:07:37,616][53852] Updated weights for policy 0, policy_version 33100 (0.0009) [2023-10-08 09:07:37,979][53852] Updated weights for policy 0, policy_version 33110 (0.0009) [2023-10-08 09:07:38,352][53852] Updated weights for policy 0, policy_version 33120 (0.0009) [2023-10-08 09:07:38,948][53885] Updated weights for policy 1, policy_version 32932 (0.0009) [2023-10-08 09:07:39,312][53885] Updated weights for policy 1, policy_version 32942 (0.0008) [2023-10-08 09:07:39,679][53885] Updated weights for policy 1, policy_version 32952 (0.0010) [2023-10-08 09:07:41,920][53852] Updated weights for policy 0, policy_version 33130 (0.0009) [2023-10-08 09:07:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 67665920. Throughput: 0: 1846.4, 1: 1828.5. Samples: 16923092. Policy #0 lag: (min: 1.0, avg: 19.8, max: 33.0) [2023-10-08 09:07:42,016][52710] Avg episode reward: [(0, '27.350'), (1, '29.690')] [2023-10-08 09:07:42,286][53852] Updated weights for policy 0, policy_version 33140 (0.0008) [2023-10-08 09:07:42,652][53852] Updated weights for policy 0, policy_version 33150 (0.0008) [2023-10-08 09:07:43,398][53885] Updated weights for policy 1, policy_version 32962 (0.0008) [2023-10-08 09:07:43,768][53885] Updated weights for policy 1, policy_version 32972 (0.0008) [2023-10-08 09:07:44,137][53885] Updated weights for policy 1, policy_version 32982 (0.0008) [2023-10-08 09:07:44,503][53885] Updated weights for policy 1, policy_version 32992 (0.0007) [2023-10-08 09:07:46,467][53852] Updated weights for policy 0, policy_version 33160 (0.0009) [2023-10-08 09:07:46,843][53852] Updated weights for policy 0, policy_version 33170 (0.0008) [2023-10-08 09:07:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 67731456. Throughput: 0: 1846.1, 1: 1830.7. Samples: 16945546. Policy #0 lag: (min: 1.0, avg: 19.8, max: 33.0) [2023-10-08 09:07:47,016][52710] Avg episode reward: [(0, '28.230'), (1, '34.050')] [2023-10-08 09:07:47,224][53852] Updated weights for policy 0, policy_version 33180 (0.0009) [2023-10-08 09:07:48,117][53885] Updated weights for policy 1, policy_version 33002 (0.0011) [2023-10-08 09:07:48,474][53885] Updated weights for policy 1, policy_version 33012 (0.0008) [2023-10-08 09:07:48,842][53885] Updated weights for policy 1, policy_version 33022 (0.0010) [2023-10-08 09:07:50,868][53852] Updated weights for policy 0, policy_version 33190 (0.0008) [2023-10-08 09:07:51,246][53852] Updated weights for policy 0, policy_version 33200 (0.0009) [2023-10-08 09:07:51,625][53852] Updated weights for policy 0, policy_version 33210 (0.0008) [2023-10-08 09:07:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 67829760. Throughput: 0: 1824.6, 1: 1832.6. Samples: 16967322. Policy #0 lag: (min: 1.0, avg: 19.8, max: 33.0) [2023-10-08 09:07:52,016][52710] Avg episode reward: [(0, '26.130'), (1, '30.280')] [2023-10-08 09:07:52,558][53885] Updated weights for policy 1, policy_version 33032 (0.0007) [2023-10-08 09:07:52,930][53885] Updated weights for policy 1, policy_version 33042 (0.0008) [2023-10-08 09:07:53,293][53885] Updated weights for policy 1, policy_version 33052 (0.0008) [2023-10-08 09:07:55,180][53852] Updated weights for policy 0, policy_version 33220 (0.0009) [2023-10-08 09:07:55,555][53852] Updated weights for policy 0, policy_version 33230 (0.0009) [2023-10-08 09:07:55,916][53852] Updated weights for policy 0, policy_version 33240 (0.0009) [2023-10-08 09:07:56,892][53885] Updated weights for policy 1, policy_version 33062 (0.0009) [2023-10-08 09:07:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67895296. Throughput: 0: 1848.0, 1: 1833.1. Samples: 16978740. Policy #0 lag: (min: 1.0, avg: 19.8, max: 33.0) [2023-10-08 09:07:57,016][52710] Avg episode reward: [(0, '28.830'), (1, '31.860')] [2023-10-08 09:07:57,264][53885] Updated weights for policy 1, policy_version 33072 (0.0010) [2023-10-08 09:07:57,635][53885] Updated weights for policy 1, policy_version 33082 (0.0010) [2023-10-08 09:07:59,438][53852] Updated weights for policy 0, policy_version 33250 (0.0009) [2023-10-08 09:07:59,803][53852] Updated weights for policy 0, policy_version 33260 (0.0010) [2023-10-08 09:08:00,167][53852] Updated weights for policy 0, policy_version 33270 (0.0010) [2023-10-08 09:08:00,536][53852] Updated weights for policy 0, policy_version 33280 (0.0007) [2023-10-08 09:08:01,306][53885] Updated weights for policy 1, policy_version 33092 (0.0008) [2023-10-08 09:08:01,677][53885] Updated weights for policy 1, policy_version 33102 (0.0008) [2023-10-08 09:08:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67960832. Throughput: 0: 1823.7, 1: 1833.6. Samples: 17000300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:08:02,016][52710] Avg episode reward: [(0, '29.080'), (1, '32.990')] [2023-10-08 09:08:02,041][53885] Updated weights for policy 1, policy_version 33112 (0.0008) [2023-10-08 09:08:04,178][53852] Updated weights for policy 0, policy_version 33290 (0.0009) [2023-10-08 09:08:04,543][53852] Updated weights for policy 0, policy_version 33300 (0.0010) [2023-10-08 09:08:04,919][53852] Updated weights for policy 0, policy_version 33310 (0.0008) [2023-10-08 09:08:05,678][53885] Updated weights for policy 1, policy_version 33122 (0.0009) [2023-10-08 09:08:06,045][53885] Updated weights for policy 1, policy_version 33132 (0.0010) [2023-10-08 09:08:06,422][53885] Updated weights for policy 1, policy_version 33142 (0.0009) [2023-10-08 09:08:06,783][53885] Updated weights for policy 1, policy_version 33152 (0.0010) [2023-10-08 09:08:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 68059136. Throughput: 0: 1847.7, 1: 1827.6. Samples: 17022178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:08:07,016][52710] Avg episode reward: [(0, '29.450'), (1, '30.750')] [2023-10-08 09:08:07,026][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000033152_33947648.pth... [2023-10-08 09:08:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000033312_34111488.pth... [2023-10-08 09:08:07,055][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000031424_32178176.pth [2023-10-08 09:08:07,065][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000031584_32342016.pth [2023-10-08 09:08:08,576][53852] Updated weights for policy 0, policy_version 33320 (0.0010) [2023-10-08 09:08:08,950][53852] Updated weights for policy 0, policy_version 33330 (0.0010) [2023-10-08 09:08:09,322][53852] Updated weights for policy 0, policy_version 33340 (0.0008) [2023-10-08 09:08:10,413][53885] Updated weights for policy 1, policy_version 33162 (0.0009) [2023-10-08 09:08:10,779][53885] Updated weights for policy 1, policy_version 33172 (0.0010) [2023-10-08 09:08:11,153][53885] Updated weights for policy 1, policy_version 33182 (0.0008) [2023-10-08 09:08:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68124672. Throughput: 0: 1828.3, 1: 1830.0. Samples: 17033316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:08:12,016][52710] Avg episode reward: [(0, '28.380'), (1, '29.670')] [2023-10-08 09:08:13,050][53852] Updated weights for policy 0, policy_version 33350 (0.0007) [2023-10-08 09:08:13,414][53852] Updated weights for policy 0, policy_version 33360 (0.0008) [2023-10-08 09:08:13,789][53852] Updated weights for policy 0, policy_version 33370 (0.0008) [2023-10-08 09:08:14,979][53885] Updated weights for policy 1, policy_version 33192 (0.0008) [2023-10-08 09:08:15,351][53885] Updated weights for policy 1, policy_version 33202 (0.0009) [2023-10-08 09:08:15,726][53885] Updated weights for policy 1, policy_version 33212 (0.0011) [2023-10-08 09:08:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68190208. Throughput: 0: 1856.1, 1: 1822.4. Samples: 17055392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:08:17,016][52710] Avg episode reward: [(0, '29.120'), (1, '33.290')] [2023-10-08 09:08:17,318][53852] Updated weights for policy 0, policy_version 33380 (0.0009) [2023-10-08 09:08:17,711][53852] Updated weights for policy 0, policy_version 33390 (0.0008) [2023-10-08 09:08:18,079][53852] Updated weights for policy 0, policy_version 33400 (0.0007) [2023-10-08 09:08:19,384][53885] Updated weights for policy 1, policy_version 33222 (0.0009) [2023-10-08 09:08:19,762][53885] Updated weights for policy 1, policy_version 33232 (0.0008) [2023-10-08 09:08:20,122][53885] Updated weights for policy 1, policy_version 33242 (0.0008) [2023-10-08 09:08:21,723][53852] Updated weights for policy 0, policy_version 33410 (0.0008) [2023-10-08 09:08:22,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 68255744. Throughput: 0: 1850.3, 1: 1825.0. Samples: 17077918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:08:22,017][52710] Avg episode reward: [(0, '28.820'), (1, '30.180')] [2023-10-08 09:08:22,101][53852] Updated weights for policy 0, policy_version 33420 (0.0007) [2023-10-08 09:08:22,461][53852] Updated weights for policy 0, policy_version 33430 (0.0009) [2023-10-08 09:08:22,833][53852] Updated weights for policy 0, policy_version 33440 (0.0009) [2023-10-08 09:08:23,770][53885] Updated weights for policy 1, policy_version 33252 (0.0008) [2023-10-08 09:08:24,132][53885] Updated weights for policy 1, policy_version 33262 (0.0008) [2023-10-08 09:08:24,498][53885] Updated weights for policy 1, policy_version 33272 (0.0007) [2023-10-08 09:08:26,492][53852] Updated weights for policy 0, policy_version 33450 (0.0009) [2023-10-08 09:08:26,860][53852] Updated weights for policy 0, policy_version 33460 (0.0008) [2023-10-08 09:08:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 68321280. Throughput: 0: 1850.7, 1: 1820.4. Samples: 17088290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:08:27,016][52710] Avg episode reward: [(0, '28.120'), (1, '31.710')] [2023-10-08 09:08:27,221][53852] Updated weights for policy 0, policy_version 33470 (0.0007) [2023-10-08 09:08:28,030][53885] Updated weights for policy 1, policy_version 33282 (0.0008) [2023-10-08 09:08:28,398][53885] Updated weights for policy 1, policy_version 33292 (0.0007) [2023-10-08 09:08:28,772][53885] Updated weights for policy 1, policy_version 33302 (0.0008) [2023-10-08 09:08:29,132][53885] Updated weights for policy 1, policy_version 33312 (0.0007) [2023-10-08 09:08:30,764][53852] Updated weights for policy 0, policy_version 33480 (0.0008) [2023-10-08 09:08:31,124][53852] Updated weights for policy 0, policy_version 33490 (0.0009) [2023-10-08 09:08:31,496][53852] Updated weights for policy 0, policy_version 33500 (0.0009) [2023-10-08 09:08:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 68419584. Throughput: 0: 1853.9, 1: 1826.9. Samples: 17111182. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-08 09:08:32,015][52710] Avg episode reward: [(0, '28.740'), (1, '29.330')] [2023-10-08 09:08:32,766][53885] Updated weights for policy 1, policy_version 33322 (0.0009) [2023-10-08 09:08:33,133][53885] Updated weights for policy 1, policy_version 33332 (0.0008) [2023-10-08 09:08:33,517][53885] Updated weights for policy 1, policy_version 33342 (0.0009) [2023-10-08 09:08:35,151][53852] Updated weights for policy 0, policy_version 33510 (0.0008) [2023-10-08 09:08:35,522][53852] Updated weights for policy 0, policy_version 33520 (0.0008) [2023-10-08 09:08:35,884][53852] Updated weights for policy 0, policy_version 33530 (0.0008) [2023-10-08 09:08:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68485120. Throughput: 0: 1850.9, 1: 1825.7. Samples: 17132768. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-08 09:08:37,016][52710] Avg episode reward: [(0, '30.250'), (1, '32.520')] [2023-10-08 09:08:37,249][53885] Updated weights for policy 1, policy_version 33352 (0.0009) [2023-10-08 09:08:37,622][53885] Updated weights for policy 1, policy_version 33362 (0.0008) [2023-10-08 09:08:37,977][53885] Updated weights for policy 1, policy_version 33372 (0.0009) [2023-10-08 09:08:39,279][53852] Updated weights for policy 0, policy_version 33540 (0.0011) [2023-10-08 09:08:39,648][53852] Updated weights for policy 0, policy_version 33550 (0.0008) [2023-10-08 09:08:40,023][53852] Updated weights for policy 0, policy_version 33560 (0.0007) [2023-10-08 09:08:41,616][53885] Updated weights for policy 1, policy_version 33382 (0.0009) [2023-10-08 09:08:41,980][53885] Updated weights for policy 1, policy_version 33392 (0.0009) [2023-10-08 09:08:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68550656. Throughput: 0: 1846.1, 1: 1824.0. Samples: 17143894. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-08 09:08:42,016][52710] Avg episode reward: [(0, '28.760'), (1, '32.410')] [2023-10-08 09:08:42,340][53885] Updated weights for policy 1, policy_version 33402 (0.0008) [2023-10-08 09:08:43,815][53852] Updated weights for policy 0, policy_version 33570 (0.0010) [2023-10-08 09:08:44,176][53852] Updated weights for policy 0, policy_version 33580 (0.0011) [2023-10-08 09:08:44,554][53852] Updated weights for policy 0, policy_version 33590 (0.0010) [2023-10-08 09:08:44,931][53852] Updated weights for policy 0, policy_version 33600 (0.0008) [2023-10-08 09:08:46,103][53885] Updated weights for policy 1, policy_version 33412 (0.0007) [2023-10-08 09:08:46,469][53885] Updated weights for policy 1, policy_version 33422 (0.0008) [2023-10-08 09:08:46,845][53885] Updated weights for policy 1, policy_version 33432 (0.0007) [2023-10-08 09:08:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68616192. Throughput: 0: 1858.6, 1: 1823.2. Samples: 17165980. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-08 09:08:47,015][52710] Avg episode reward: [(0, '28.560'), (1, '31.120')] [2023-10-08 09:08:48,487][53852] Updated weights for policy 0, policy_version 33610 (0.0008) [2023-10-08 09:08:48,857][53852] Updated weights for policy 0, policy_version 33620 (0.0007) [2023-10-08 09:08:49,235][53852] Updated weights for policy 0, policy_version 33630 (0.0007) [2023-10-08 09:08:50,540][53885] Updated weights for policy 1, policy_version 33442 (0.0007) [2023-10-08 09:08:50,898][53885] Updated weights for policy 1, policy_version 33452 (0.0011) [2023-10-08 09:08:51,264][53885] Updated weights for policy 1, policy_version 33462 (0.0007) [2023-10-08 09:08:51,634][53885] Updated weights for policy 1, policy_version 33472 (0.0008) [2023-10-08 09:08:52,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68714496. Throughput: 0: 1857.9, 1: 1815.8. Samples: 17187492. Policy #0 lag: (min: 14.0, avg: 21.8, max: 46.0) [2023-10-08 09:08:52,016][52710] Avg episode reward: [(0, '30.310'), (1, '32.030')] [2023-10-08 09:08:52,869][53852] Updated weights for policy 0, policy_version 33640 (0.0009) [2023-10-08 09:08:53,235][53852] Updated weights for policy 0, policy_version 33650 (0.0007) [2023-10-08 09:08:53,602][53852] Updated weights for policy 0, policy_version 33660 (0.0007) [2023-10-08 09:08:55,216][53885] Updated weights for policy 1, policy_version 33482 (0.0011) [2023-10-08 09:08:55,589][53885] Updated weights for policy 1, policy_version 33492 (0.0012) [2023-10-08 09:08:55,950][53885] Updated weights for policy 1, policy_version 33502 (0.0007) [2023-10-08 09:08:57,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68780032. Throughput: 0: 1862.8, 1: 1820.4. Samples: 17199058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:08:57,016][52710] Avg episode reward: [(0, '29.050'), (1, '32.640')] [2023-10-08 09:08:57,194][53852] Updated weights for policy 0, policy_version 33670 (0.0009) [2023-10-08 09:08:57,572][53852] Updated weights for policy 0, policy_version 33680 (0.0008) [2023-10-08 09:08:57,936][53852] Updated weights for policy 0, policy_version 33690 (0.0008) [2023-10-08 09:08:59,606][53885] Updated weights for policy 1, policy_version 33512 (0.0007) [2023-10-08 09:08:59,970][53885] Updated weights for policy 1, policy_version 33522 (0.0008) [2023-10-08 09:09:00,341][53885] Updated weights for policy 1, policy_version 33532 (0.0009) [2023-10-08 09:09:01,742][53852] Updated weights for policy 0, policy_version 33700 (0.0009) [2023-10-08 09:09:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68845568. Throughput: 0: 1852.7, 1: 1816.1. Samples: 17220488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:09:02,016][52710] Avg episode reward: [(0, '30.130'), (1, '32.320')] [2023-10-08 09:09:02,127][53852] Updated weights for policy 0, policy_version 33710 (0.0007) [2023-10-08 09:09:02,493][53852] Updated weights for policy 0, policy_version 33720 (0.0008) [2023-10-08 09:09:04,101][53885] Updated weights for policy 1, policy_version 33542 (0.0010) [2023-10-08 09:09:04,472][53885] Updated weights for policy 1, policy_version 33552 (0.0008) [2023-10-08 09:09:04,839][53885] Updated weights for policy 1, policy_version 33562 (0.0010) [2023-10-08 09:09:06,099][53852] Updated weights for policy 0, policy_version 33730 (0.0007) [2023-10-08 09:09:06,457][53852] Updated weights for policy 0, policy_version 33740 (0.0009) [2023-10-08 09:09:06,827][53852] Updated weights for policy 0, policy_version 33750 (0.0008) [2023-10-08 09:09:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 68911104. Throughput: 0: 1836.9, 1: 1822.1. Samples: 17242572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:09:07,016][52710] Avg episode reward: [(0, '31.850'), (1, '32.330')] [2023-10-08 09:09:07,199][53852] Updated weights for policy 0, policy_version 33760 (0.0009) [2023-10-08 09:09:08,444][53885] Updated weights for policy 1, policy_version 33572 (0.0010) [2023-10-08 09:09:08,807][53885] Updated weights for policy 1, policy_version 33582 (0.0009) [2023-10-08 09:09:09,175][53885] Updated weights for policy 1, policy_version 33592 (0.0007) [2023-10-08 09:09:10,826][53852] Updated weights for policy 0, policy_version 33770 (0.0011) [2023-10-08 09:09:11,199][53852] Updated weights for policy 0, policy_version 33780 (0.0010) [2023-10-08 09:09:11,574][53852] Updated weights for policy 0, policy_version 33790 (0.0009) [2023-10-08 09:09:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69009408. Throughput: 0: 1855.7, 1: 1815.6. Samples: 17253500. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:09:12,016][52710] Avg episode reward: [(0, '30.420'), (1, '31.750')] [2023-10-08 09:09:13,047][53885] Updated weights for policy 1, policy_version 33602 (0.0010) [2023-10-08 09:09:13,403][53885] Updated weights for policy 1, policy_version 33612 (0.0009) [2023-10-08 09:09:13,773][53885] Updated weights for policy 1, policy_version 33622 (0.0010) [2023-10-08 09:09:14,139][53885] Updated weights for policy 1, policy_version 33632 (0.0009) [2023-10-08 09:09:15,185][53852] Updated weights for policy 0, policy_version 33800 (0.0010) [2023-10-08 09:09:15,543][53852] Updated weights for policy 0, policy_version 33810 (0.0011) [2023-10-08 09:09:15,919][53852] Updated weights for policy 0, policy_version 33820 (0.0011) [2023-10-08 09:09:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69074944. Throughput: 0: 1834.2, 1: 1820.3. Samples: 17275636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:09:17,016][52710] Avg episode reward: [(0, '28.930'), (1, '32.120')] [2023-10-08 09:09:17,900][53885] Updated weights for policy 1, policy_version 33642 (0.0009) [2023-10-08 09:09:18,268][53885] Updated weights for policy 1, policy_version 33652 (0.0012) [2023-10-08 09:09:18,638][53885] Updated weights for policy 1, policy_version 33662 (0.0009) [2023-10-08 09:09:19,480][53852] Updated weights for policy 0, policy_version 33830 (0.0008) [2023-10-08 09:09:19,849][53852] Updated weights for policy 0, policy_version 33840 (0.0007) [2023-10-08 09:09:20,211][53852] Updated weights for policy 0, policy_version 33850 (0.0008) [2023-10-08 09:09:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69140480. Throughput: 0: 1850.8, 1: 1816.7. Samples: 17297804. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 09:09:22,016][52710] Avg episode reward: [(0, '29.210'), (1, '32.910')] [2023-10-08 09:09:22,410][53885] Updated weights for policy 1, policy_version 33672 (0.0008) [2023-10-08 09:09:22,787][53885] Updated weights for policy 1, policy_version 33682 (0.0008) [2023-10-08 09:09:23,145][53885] Updated weights for policy 1, policy_version 33692 (0.0007) [2023-10-08 09:09:23,862][53852] Updated weights for policy 0, policy_version 33860 (0.0009) [2023-10-08 09:09:24,227][53852] Updated weights for policy 0, policy_version 33870 (0.0008) [2023-10-08 09:09:24,602][53852] Updated weights for policy 0, policy_version 33880 (0.0010) [2023-10-08 09:09:26,673][53885] Updated weights for policy 1, policy_version 33702 (0.0007) [2023-10-08 09:09:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69206016. Throughput: 0: 1836.0, 1: 1817.4. Samples: 17308294. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 09:09:27,016][52710] Avg episode reward: [(0, '29.620'), (1, '33.200')] [2023-10-08 09:09:27,031][53885] Updated weights for policy 1, policy_version 33712 (0.0009) [2023-10-08 09:09:27,401][53885] Updated weights for policy 1, policy_version 33722 (0.0010) [2023-10-08 09:09:28,052][53852] Updated weights for policy 0, policy_version 33890 (0.0007) [2023-10-08 09:09:28,424][53852] Updated weights for policy 0, policy_version 33900 (0.0008) [2023-10-08 09:09:28,798][53852] Updated weights for policy 0, policy_version 33910 (0.0008) [2023-10-08 09:09:29,167][53852] Updated weights for policy 0, policy_version 33920 (0.0009) [2023-10-08 09:09:31,010][53885] Updated weights for policy 1, policy_version 33732 (0.0009) [2023-10-08 09:09:31,372][53885] Updated weights for policy 1, policy_version 33742 (0.0009) [2023-10-08 09:09:31,746][53885] Updated weights for policy 1, policy_version 33752 (0.0011) [2023-10-08 09:09:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69271552. Throughput: 0: 1853.5, 1: 1819.0. Samples: 17331242. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 09:09:32,016][52710] Avg episode reward: [(0, '30.610'), (1, '32.790')] [2023-10-08 09:09:32,731][53852] Updated weights for policy 0, policy_version 33930 (0.0008) [2023-10-08 09:09:33,114][53852] Updated weights for policy 0, policy_version 33940 (0.0009) [2023-10-08 09:09:33,479][53852] Updated weights for policy 0, policy_version 33950 (0.0007) [2023-10-08 09:09:35,426][53885] Updated weights for policy 1, policy_version 33762 (0.0009) [2023-10-08 09:09:35,789][53885] Updated weights for policy 1, policy_version 33772 (0.0007) [2023-10-08 09:09:36,154][53885] Updated weights for policy 1, policy_version 33782 (0.0007) [2023-10-08 09:09:36,521][53885] Updated weights for policy 1, policy_version 33792 (0.0008) [2023-10-08 09:09:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69369856. Throughput: 0: 1852.4, 1: 1818.1. Samples: 17352662. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 09:09:37,016][52710] Avg episode reward: [(0, '28.370'), (1, '33.040')] [2023-10-08 09:09:37,048][53852] Updated weights for policy 0, policy_version 33960 (0.0008) [2023-10-08 09:09:37,410][53852] Updated weights for policy 0, policy_version 33970 (0.0007) [2023-10-08 09:09:37,783][53852] Updated weights for policy 0, policy_version 33980 (0.0008) [2023-10-08 09:09:40,327][53885] Updated weights for policy 1, policy_version 33802 (0.0010) [2023-10-08 09:09:40,697][53885] Updated weights for policy 1, policy_version 33812 (0.0010) [2023-10-08 09:09:41,074][53885] Updated weights for policy 1, policy_version 33822 (0.0007) [2023-10-08 09:09:41,535][53852] Updated weights for policy 0, policy_version 33990 (0.0008) [2023-10-08 09:09:41,901][53852] Updated weights for policy 0, policy_version 34000 (0.0008) [2023-10-08 09:09:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69435392. Throughput: 0: 1853.3, 1: 1818.4. Samples: 17364282. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 09:09:42,016][52710] Avg episode reward: [(0, '28.250'), (1, '32.180')] [2023-10-08 09:09:42,264][53852] Updated weights for policy 0, policy_version 34010 (0.0009) [2023-10-08 09:09:44,822][53885] Updated weights for policy 1, policy_version 33832 (0.0008) [2023-10-08 09:09:45,188][53885] Updated weights for policy 1, policy_version 33842 (0.0010) [2023-10-08 09:09:45,560][53885] Updated weights for policy 1, policy_version 33852 (0.0011) [2023-10-08 09:09:46,037][53852] Updated weights for policy 0, policy_version 34020 (0.0008) [2023-10-08 09:09:46,403][53852] Updated weights for policy 0, policy_version 34030 (0.0007) [2023-10-08 09:09:46,775][53852] Updated weights for policy 0, policy_version 34040 (0.0007) [2023-10-08 09:09:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69500928. Throughput: 0: 1857.8, 1: 1818.2. Samples: 17385906. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) [2023-10-08 09:09:47,015][52710] Avg episode reward: [(0, '30.820'), (1, '32.170')] [2023-10-08 09:09:49,285][53885] Updated weights for policy 1, policy_version 33862 (0.0011) [2023-10-08 09:09:49,655][53885] Updated weights for policy 1, policy_version 33872 (0.0008) [2023-10-08 09:09:50,017][53885] Updated weights for policy 1, policy_version 33882 (0.0007) [2023-10-08 09:09:50,525][53852] Updated weights for policy 0, policy_version 34050 (0.0007) [2023-10-08 09:09:50,938][53852] Updated weights for policy 0, policy_version 34060 (0.0010) [2023-10-08 09:09:51,310][53852] Updated weights for policy 0, policy_version 34070 (0.0011) [2023-10-08 09:09:51,681][53852] Updated weights for policy 0, policy_version 34080 (0.0008) [2023-10-08 09:09:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69599232. Throughput: 0: 1840.0, 1: 1817.5. Samples: 17407158. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 09:09:52,016][52710] Avg episode reward: [(0, '31.890'), (1, '32.220')] [2023-10-08 09:09:53,707][53885] Updated weights for policy 1, policy_version 33892 (0.0007) [2023-10-08 09:09:54,066][53885] Updated weights for policy 1, policy_version 33902 (0.0009) [2023-10-08 09:09:54,438][53885] Updated weights for policy 1, policy_version 33912 (0.0010) [2023-10-08 09:09:55,342][53852] Updated weights for policy 0, policy_version 34090 (0.0007) [2023-10-08 09:09:55,708][53852] Updated weights for policy 0, policy_version 34100 (0.0007) [2023-10-08 09:09:56,075][53852] Updated weights for policy 0, policy_version 34110 (0.0007) [2023-10-08 09:09:57,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 69664768. Throughput: 0: 1846.0, 1: 1823.4. Samples: 17418624. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 09:09:57,016][52710] Avg episode reward: [(0, '29.840'), (1, '31.640')] [2023-10-08 09:09:58,237][53885] Updated weights for policy 1, policy_version 33922 (0.0009) [2023-10-08 09:09:58,600][53885] Updated weights for policy 1, policy_version 33932 (0.0010) [2023-10-08 09:09:58,978][53885] Updated weights for policy 1, policy_version 33942 (0.0008) [2023-10-08 09:09:59,340][53885] Updated weights for policy 1, policy_version 33952 (0.0008) [2023-10-08 09:09:59,558][53852] Updated weights for policy 0, policy_version 34120 (0.0008) [2023-10-08 09:09:59,936][53852] Updated weights for policy 0, policy_version 34130 (0.0010) [2023-10-08 09:10:00,299][53852] Updated weights for policy 0, policy_version 34140 (0.0011) [2023-10-08 09:10:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69730304. Throughput: 0: 1838.8, 1: 1817.2. Samples: 17440154. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 09:10:02,015][52710] Avg episode reward: [(0, '27.760'), (1, '31.340')] [2023-10-08 09:10:03,085][53885] Updated weights for policy 1, policy_version 33962 (0.0007) [2023-10-08 09:10:03,451][53885] Updated weights for policy 1, policy_version 33972 (0.0007) [2023-10-08 09:10:03,811][53885] Updated weights for policy 1, policy_version 33982 (0.0008) [2023-10-08 09:10:04,035][53852] Updated weights for policy 0, policy_version 34150 (0.0009) [2023-10-08 09:10:04,395][53852] Updated weights for policy 0, policy_version 34160 (0.0008) [2023-10-08 09:10:04,769][53852] Updated weights for policy 0, policy_version 34170 (0.0008) [2023-10-08 09:10:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69795840. Throughput: 0: 1851.3, 1: 1820.1. Samples: 17463018. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 09:10:07,016][52710] Avg episode reward: [(0, '25.730'), (1, '30.580')] [2023-10-08 09:10:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000034176_34996224.pth... [2023-10-08 09:10:07,061][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000032448_33226752.pth [2023-10-08 09:10:07,065][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000034176_34996224.pth [2023-10-08 09:10:07,324][53885] Updated weights for policy 1, policy_version 33992 (0.0007) [2023-10-08 09:10:07,702][53885] Updated weights for policy 1, policy_version 34002 (0.0007) [2023-10-08 09:10:08,071][53885] Updated weights for policy 1, policy_version 34012 (0.0008) [2023-10-08 09:10:08,209][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000034016_34832384.pth... [2023-10-08 09:10:08,242][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000032288_33062912.pth [2023-10-08 09:10:08,246][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000034016_34832384.pth [2023-10-08 09:10:08,400][53852] Updated weights for policy 0, policy_version 34180 (0.0009) [2023-10-08 09:10:08,768][53852] Updated weights for policy 0, policy_version 34190 (0.0008) [2023-10-08 09:10:09,129][53852] Updated weights for policy 0, policy_version 34200 (0.0008) [2023-10-08 09:10:11,785][53885] Updated weights for policy 1, policy_version 34022 (0.0008) [2023-10-08 09:10:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69861376. Throughput: 0: 1840.3, 1: 1823.2. Samples: 17473154. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 09:10:12,016][52710] Avg episode reward: [(0, '26.830'), (1, '32.920')] [2023-10-08 09:10:12,150][53885] Updated weights for policy 1, policy_version 34032 (0.0007) [2023-10-08 09:10:12,514][53885] Updated weights for policy 1, policy_version 34042 (0.0010) [2023-10-08 09:10:12,818][53852] Updated weights for policy 0, policy_version 34210 (0.0007) [2023-10-08 09:10:13,197][53852] Updated weights for policy 0, policy_version 34220 (0.0008) [2023-10-08 09:10:13,576][53852] Updated weights for policy 0, policy_version 34230 (0.0007) [2023-10-08 09:10:13,943][53852] Updated weights for policy 0, policy_version 34240 (0.0009) [2023-10-08 09:10:16,387][53885] Updated weights for policy 1, policy_version 34052 (0.0008) [2023-10-08 09:10:16,756][53885] Updated weights for policy 1, policy_version 34062 (0.0009) [2023-10-08 09:10:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69926912. Throughput: 0: 1840.2, 1: 1817.0. Samples: 17495818. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:10:17,016][52710] Avg episode reward: [(0, '25.320'), (1, '32.210')] [2023-10-08 09:10:17,121][53885] Updated weights for policy 1, policy_version 34072 (0.0009) [2023-10-08 09:10:17,545][53852] Updated weights for policy 0, policy_version 34250 (0.0008) [2023-10-08 09:10:17,923][53852] Updated weights for policy 0, policy_version 34260 (0.0007) [2023-10-08 09:10:18,298][53852] Updated weights for policy 0, policy_version 34270 (0.0009) [2023-10-08 09:10:20,822][53885] Updated weights for policy 1, policy_version 34082 (0.0007) [2023-10-08 09:10:21,191][53885] Updated weights for policy 1, policy_version 34092 (0.0007) [2023-10-08 09:10:21,555][53885] Updated weights for policy 1, policy_version 34102 (0.0008) [2023-10-08 09:10:21,748][53852] Updated weights for policy 0, policy_version 34280 (0.0009) [2023-10-08 09:10:21,928][53885] Updated weights for policy 1, policy_version 34112 (0.0008) [2023-10-08 09:10:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70025216. Throughput: 0: 1839.1, 1: 1827.3. Samples: 17517652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:10:22,016][52710] Avg episode reward: [(0, '24.510'), (1, '32.500')] [2023-10-08 09:10:22,114][53852] Updated weights for policy 0, policy_version 34290 (0.0009) [2023-10-08 09:10:22,495][53852] Updated weights for policy 0, policy_version 34300 (0.0008) [2023-10-08 09:10:25,681][53885] Updated weights for policy 1, policy_version 34122 (0.0010) [2023-10-08 09:10:26,046][53885] Updated weights for policy 1, policy_version 34132 (0.0009) [2023-10-08 09:10:26,156][53852] Updated weights for policy 0, policy_version 34310 (0.0009) [2023-10-08 09:10:26,407][53885] Updated weights for policy 1, policy_version 34142 (0.0008) [2023-10-08 09:10:26,533][53852] Updated weights for policy 0, policy_version 34320 (0.0009) [2023-10-08 09:10:26,906][53852] Updated weights for policy 0, policy_version 34330 (0.0009) [2023-10-08 09:10:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70090752. Throughput: 0: 1839.1, 1: 1816.9. Samples: 17528800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:10:27,016][52710] Avg episode reward: [(0, '28.730'), (1, '32.880')] [2023-10-08 09:10:30,126][53885] Updated weights for policy 1, policy_version 34152 (0.0010) [2023-10-08 09:10:30,494][53885] Updated weights for policy 1, policy_version 34162 (0.0008) [2023-10-08 09:10:30,575][53852] Updated weights for policy 0, policy_version 34340 (0.0007) [2023-10-08 09:10:30,860][53885] Updated weights for policy 1, policy_version 34172 (0.0008) [2023-10-08 09:10:30,936][53852] Updated weights for policy 0, policy_version 34350 (0.0008) [2023-10-08 09:10:31,312][53852] Updated weights for policy 0, policy_version 34360 (0.0008) [2023-10-08 09:10:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 70189056. Throughput: 0: 1831.5, 1: 1829.4. Samples: 17550648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:10:32,016][52710] Avg episode reward: [(0, '27.300'), (1, '31.840')] [2023-10-08 09:10:34,666][53885] Updated weights for policy 1, policy_version 34182 (0.0011) [2023-10-08 09:10:34,996][53852] Updated weights for policy 0, policy_version 34370 (0.0007) [2023-10-08 09:10:35,043][53885] Updated weights for policy 1, policy_version 34192 (0.0008) [2023-10-08 09:10:35,349][53852] Updated weights for policy 0, policy_version 34380 (0.0007) [2023-10-08 09:10:35,414][53885] Updated weights for policy 1, policy_version 34202 (0.0008) [2023-10-08 09:10:35,714][53852] Updated weights for policy 0, policy_version 34390 (0.0010) [2023-10-08 09:10:36,083][53852] Updated weights for policy 0, policy_version 34400 (0.0010) [2023-10-08 09:10:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 70254592. Throughput: 0: 1831.1, 1: 1822.0. Samples: 17571544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:10:37,016][52710] Avg episode reward: [(0, '28.210'), (1, '30.280')] [2023-10-08 09:10:38,827][53885] Updated weights for policy 1, policy_version 34212 (0.0008) [2023-10-08 09:10:39,190][53885] Updated weights for policy 1, policy_version 34222 (0.0009) [2023-10-08 09:10:39,566][53885] Updated weights for policy 1, policy_version 34232 (0.0009) [2023-10-08 09:10:39,902][53852] Updated weights for policy 0, policy_version 34410 (0.0007) [2023-10-08 09:10:40,279][53852] Updated weights for policy 0, policy_version 34420 (0.0009) [2023-10-08 09:10:40,650][53852] Updated weights for policy 0, policy_version 34430 (0.0009) [2023-10-08 09:10:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70320128. Throughput: 0: 1833.9, 1: 1825.4. Samples: 17583292. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 09:10:42,016][52710] Avg episode reward: [(0, '29.700'), (1, '31.250')] [2023-10-08 09:10:43,200][53885] Updated weights for policy 1, policy_version 34242 (0.0009) [2023-10-08 09:10:43,577][53885] Updated weights for policy 1, policy_version 34252 (0.0010) [2023-10-08 09:10:43,939][53885] Updated weights for policy 1, policy_version 34262 (0.0010) [2023-10-08 09:10:44,305][53885] Updated weights for policy 1, policy_version 34272 (0.0007) [2023-10-08 09:10:44,319][53852] Updated weights for policy 0, policy_version 34440 (0.0008) [2023-10-08 09:10:44,689][53852] Updated weights for policy 0, policy_version 34450 (0.0007) [2023-10-08 09:10:45,067][53852] Updated weights for policy 0, policy_version 34460 (0.0008) [2023-10-08 09:10:47,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 70385664. Throughput: 0: 1822.1, 1: 1822.9. Samples: 17604180. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 09:10:47,017][52710] Avg episode reward: [(0, '30.600'), (1, '27.610')] [2023-10-08 09:10:47,860][53885] Updated weights for policy 1, policy_version 34282 (0.0008) [2023-10-08 09:10:48,222][53885] Updated weights for policy 1, policy_version 34292 (0.0008) [2023-10-08 09:10:48,581][53852] Updated weights for policy 0, policy_version 34470 (0.0008) [2023-10-08 09:10:48,597][53885] Updated weights for policy 1, policy_version 34302 (0.0007) [2023-10-08 09:10:48,950][53852] Updated weights for policy 0, policy_version 34480 (0.0008) [2023-10-08 09:10:49,329][53852] Updated weights for policy 0, policy_version 34490 (0.0007) [2023-10-08 09:10:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70451200. Throughput: 0: 1829.9, 1: 1823.4. Samples: 17627416. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 09:10:52,016][52710] Avg episode reward: [(0, '29.290'), (1, '28.140')] [2023-10-08 09:10:52,413][53885] Updated weights for policy 1, policy_version 34312 (0.0009) [2023-10-08 09:10:52,775][53885] Updated weights for policy 1, policy_version 34322 (0.0009) [2023-10-08 09:10:52,904][53852] Updated weights for policy 0, policy_version 34500 (0.0007) [2023-10-08 09:10:53,150][53885] Updated weights for policy 1, policy_version 34332 (0.0007) [2023-10-08 09:10:53,274][53852] Updated weights for policy 0, policy_version 34510 (0.0008) [2023-10-08 09:10:53,639][53852] Updated weights for policy 0, policy_version 34520 (0.0008) [2023-10-08 09:10:56,823][53885] Updated weights for policy 1, policy_version 34342 (0.0009) [2023-10-08 09:10:57,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70516736. Throughput: 0: 1830.7, 1: 1816.8. Samples: 17637292. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 09:10:57,016][52710] Avg episode reward: [(0, '32.270'), (1, '32.160')] [2023-10-08 09:10:57,118][53852] Updated weights for policy 0, policy_version 34530 (0.0007) [2023-10-08 09:10:57,193][53885] Updated weights for policy 1, policy_version 34352 (0.0010) [2023-10-08 09:10:57,493][53852] Updated weights for policy 0, policy_version 34540 (0.0008) [2023-10-08 09:10:57,561][53885] Updated weights for policy 1, policy_version 34362 (0.0009) [2023-10-08 09:10:57,867][53852] Updated weights for policy 0, policy_version 34550 (0.0007) [2023-10-08 09:10:58,243][53852] Updated weights for policy 0, policy_version 34560 (0.0009) [2023-10-08 09:11:01,216][53885] Updated weights for policy 1, policy_version 34372 (0.0007) [2023-10-08 09:11:01,585][53885] Updated weights for policy 1, policy_version 34382 (0.0007) [2023-10-08 09:11:01,809][53852] Updated weights for policy 0, policy_version 34570 (0.0008) [2023-10-08 09:11:01,958][53885] Updated weights for policy 1, policy_version 34392 (0.0007) [2023-10-08 09:11:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 70582272. Throughput: 0: 1839.0, 1: 1817.2. Samples: 17660348. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) [2023-10-08 09:11:02,016][52710] Avg episode reward: [(0, '29.160'), (1, '26.850')] [2023-10-08 09:11:02,164][53852] Updated weights for policy 0, policy_version 34580 (0.0007) [2023-10-08 09:11:02,537][53852] Updated weights for policy 0, policy_version 34590 (0.0010) [2023-10-08 09:11:05,635][53885] Updated weights for policy 1, policy_version 34402 (0.0008) [2023-10-08 09:11:05,993][53885] Updated weights for policy 1, policy_version 34412 (0.0008) [2023-10-08 09:11:06,098][53852] Updated weights for policy 0, policy_version 34600 (0.0007) [2023-10-08 09:11:06,360][53885] Updated weights for policy 1, policy_version 34422 (0.0008) [2023-10-08 09:11:06,474][53852] Updated weights for policy 0, policy_version 34610 (0.0008) [2023-10-08 09:11:06,727][53885] Updated weights for policy 1, policy_version 34432 (0.0008) [2023-10-08 09:11:06,848][53852] Updated weights for policy 0, policy_version 34620 (0.0008) [2023-10-08 09:11:07,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 70713344. Throughput: 0: 1823.8, 1: 1808.2. Samples: 17681094. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-08 09:11:07,015][52710] Avg episode reward: [(0, '29.820'), (1, '23.430')] [2023-10-08 09:11:10,476][53885] Updated weights for policy 1, policy_version 34442 (0.0009) [2023-10-08 09:11:10,561][53852] Updated weights for policy 0, policy_version 34630 (0.0008) [2023-10-08 09:11:10,839][53885] Updated weights for policy 1, policy_version 34452 (0.0007) [2023-10-08 09:11:10,932][53852] Updated weights for policy 0, policy_version 34640 (0.0008) [2023-10-08 09:11:11,202][53885] Updated weights for policy 1, policy_version 34462 (0.0007) [2023-10-08 09:11:11,289][53852] Updated weights for policy 0, policy_version 34650 (0.0009) [2023-10-08 09:11:12,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 70778880. Throughput: 0: 1839.1, 1: 1812.5. Samples: 17693120. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-08 09:11:12,016][52710] Avg episode reward: [(0, '29.750'), (1, '22.030')] [2023-10-08 09:11:15,001][53885] Updated weights for policy 1, policy_version 34472 (0.0007) [2023-10-08 09:11:15,078][53852] Updated weights for policy 0, policy_version 34660 (0.0010) [2023-10-08 09:11:15,371][53885] Updated weights for policy 1, policy_version 34482 (0.0008) [2023-10-08 09:11:15,453][53852] Updated weights for policy 0, policy_version 34670 (0.0008) [2023-10-08 09:11:15,743][53885] Updated weights for policy 1, policy_version 34492 (0.0008) [2023-10-08 09:11:15,825][53852] Updated weights for policy 0, policy_version 34680 (0.0009) [2023-10-08 09:11:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 70844416. Throughput: 0: 1827.5, 1: 1805.3. Samples: 17714128. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-08 09:11:17,016][52710] Avg episode reward: [(0, '29.230'), (1, '20.490')] [2023-10-08 09:11:19,465][53852] Updated weights for policy 0, policy_version 34690 (0.0009) [2023-10-08 09:11:19,502][53885] Updated weights for policy 1, policy_version 34502 (0.0009) [2023-10-08 09:11:19,826][53852] Updated weights for policy 0, policy_version 34700 (0.0008) [2023-10-08 09:11:19,872][53885] Updated weights for policy 1, policy_version 34512 (0.0008) [2023-10-08 09:11:20,197][53852] Updated weights for policy 0, policy_version 34710 (0.0008) [2023-10-08 09:11:20,233][53885] Updated weights for policy 1, policy_version 34522 (0.0008) [2023-10-08 09:11:20,569][53852] Updated weights for policy 0, policy_version 34720 (0.0010) [2023-10-08 09:11:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 70909952. Throughput: 0: 1841.0, 1: 1802.8. Samples: 17735512. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-08 09:11:22,016][52710] Avg episode reward: [(0, '26.490'), (1, '24.430')] [2023-10-08 09:11:24,186][53852] Updated weights for policy 0, policy_version 34730 (0.0008) [2023-10-08 09:11:24,222][53885] Updated weights for policy 1, policy_version 34532 (0.0009) [2023-10-08 09:11:24,552][53852] Updated weights for policy 0, policy_version 34740 (0.0008) [2023-10-08 09:11:24,593][53885] Updated weights for policy 1, policy_version 34542 (0.0007) [2023-10-08 09:11:24,930][53852] Updated weights for policy 0, policy_version 34750 (0.0009) [2023-10-08 09:11:24,950][53885] Updated weights for policy 1, policy_version 34552 (0.0007) [2023-10-08 09:11:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70975488. Throughput: 0: 1827.4, 1: 1807.7. Samples: 17746872. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-08 09:11:27,016][52710] Avg episode reward: [(0, '19.650'), (1, '27.470')] [2023-10-08 09:11:28,583][53885] Updated weights for policy 1, policy_version 34562 (0.0009) [2023-10-08 09:11:28,773][53852] Updated weights for policy 0, policy_version 34760 (0.0008) [2023-10-08 09:11:28,946][53885] Updated weights for policy 1, policy_version 34572 (0.0008) [2023-10-08 09:11:29,130][53852] Updated weights for policy 0, policy_version 34770 (0.0007) [2023-10-08 09:11:29,308][53885] Updated weights for policy 1, policy_version 34582 (0.0010) [2023-10-08 09:11:29,498][53852] Updated weights for policy 0, policy_version 34780 (0.0007) [2023-10-08 09:11:29,686][53885] Updated weights for policy 1, policy_version 34592 (0.0009) [2023-10-08 09:11:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 71041024. Throughput: 0: 1841.8, 1: 1797.3. Samples: 17767938. Policy #0 lag: (min: 18.0, avg: 21.0, max: 50.0) [2023-10-08 09:11:32,016][52710] Avg episode reward: [(0, '10.340'), (1, '29.520')] [2023-10-08 09:11:33,355][53852] Updated weights for policy 0, policy_version 34790 (0.0007) [2023-10-08 09:11:33,445][53885] Updated weights for policy 1, policy_version 34602 (0.0008) [2023-10-08 09:11:33,724][53852] Updated weights for policy 0, policy_version 34800 (0.0007) [2023-10-08 09:11:33,812][53885] Updated weights for policy 1, policy_version 34612 (0.0007) [2023-10-08 09:11:34,094][53852] Updated weights for policy 0, policy_version 34810 (0.0007) [2023-10-08 09:11:34,173][53885] Updated weights for policy 1, policy_version 34622 (0.0008) [2023-10-08 09:11:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 71106560. Throughput: 0: 1831.7, 1: 1794.2. Samples: 17790580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:11:37,016][52710] Avg episode reward: [(0, '6.350'), (1, '30.390')] [2023-10-08 09:11:37,928][53852] Updated weights for policy 0, policy_version 34820 (0.0008) [2023-10-08 09:11:37,976][53885] Updated weights for policy 1, policy_version 34632 (0.0009) [2023-10-08 09:11:38,291][53852] Updated weights for policy 0, policy_version 34830 (0.0008) [2023-10-08 09:11:38,355][53885] Updated weights for policy 1, policy_version 34642 (0.0010) [2023-10-08 09:11:38,665][53852] Updated weights for policy 0, policy_version 34840 (0.0007) [2023-10-08 09:11:38,723][53885] Updated weights for policy 1, policy_version 34652 (0.0008) [2023-10-08 09:11:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71172096. Throughput: 0: 1828.1, 1: 1798.6. Samples: 17800494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:11:42,016][52710] Avg episode reward: [(0, '7.450'), (1, '32.250')] [2023-10-08 09:11:42,433][53852] Updated weights for policy 0, policy_version 34850 (0.0009) [2023-10-08 09:11:42,467][53885] Updated weights for policy 1, policy_version 34662 (0.0007) [2023-10-08 09:11:42,799][53852] Updated weights for policy 0, policy_version 34860 (0.0007) [2023-10-08 09:11:42,839][53885] Updated weights for policy 1, policy_version 34672 (0.0007) [2023-10-08 09:11:43,166][53852] Updated weights for policy 0, policy_version 34870 (0.0008) [2023-10-08 09:11:43,212][53885] Updated weights for policy 1, policy_version 34682 (0.0007) [2023-10-08 09:11:43,536][53852] Updated weights for policy 0, policy_version 34880 (0.0007) [2023-10-08 09:11:46,917][53885] Updated weights for policy 1, policy_version 34692 (0.0007) [2023-10-08 09:11:47,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71237632. Throughput: 0: 1815.9, 1: 1800.3. Samples: 17823076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:11:47,016][52710] Avg episode reward: [(0, '7.470'), (1, '30.680')] [2023-10-08 09:11:47,283][53885] Updated weights for policy 1, policy_version 34702 (0.0008) [2023-10-08 09:11:47,322][53852] Updated weights for policy 0, policy_version 34890 (0.0007) [2023-10-08 09:11:47,643][53885] Updated weights for policy 1, policy_version 34712 (0.0008) [2023-10-08 09:11:47,681][53852] Updated weights for policy 0, policy_version 34900 (0.0008) [2023-10-08 09:11:48,059][53852] Updated weights for policy 0, policy_version 34910 (0.0008) [2023-10-08 09:11:51,320][53885] Updated weights for policy 1, policy_version 34722 (0.0008) [2023-10-08 09:11:51,684][53885] Updated weights for policy 1, policy_version 34732 (0.0008) [2023-10-08 09:11:51,688][53852] Updated weights for policy 0, policy_version 34920 (0.0007) [2023-10-08 09:11:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71303168. Throughput: 0: 1828.7, 1: 1821.6. Samples: 17845358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:11:52,016][52710] Avg episode reward: [(0, '7.860'), (1, '31.080')] [2023-10-08 09:11:52,045][53885] Updated weights for policy 1, policy_version 34742 (0.0008) [2023-10-08 09:11:52,046][53852] Updated weights for policy 0, policy_version 34930 (0.0007) [2023-10-08 09:11:52,410][53852] Updated weights for policy 0, policy_version 34940 (0.0007) [2023-10-08 09:11:52,419][53885] Updated weights for policy 1, policy_version 34752 (0.0008) [2023-10-08 09:11:55,965][53852] Updated weights for policy 0, policy_version 34950 (0.0007) [2023-10-08 09:11:56,212][53885] Updated weights for policy 1, policy_version 34762 (0.0007) [2023-10-08 09:11:56,342][53852] Updated weights for policy 0, policy_version 34960 (0.0009) [2023-10-08 09:11:56,579][53885] Updated weights for policy 1, policy_version 34772 (0.0008) [2023-10-08 09:11:56,705][53852] Updated weights for policy 0, policy_version 34970 (0.0010) [2023-10-08 09:11:56,948][53885] Updated weights for policy 1, policy_version 34782 (0.0008) [2023-10-08 09:11:57,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 71434240. Throughput: 0: 1813.2, 1: 1799.8. Samples: 17855704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:11:57,016][52710] Avg episode reward: [(0, '9.490'), (1, '29.680')] [2023-10-08 09:12:00,296][53852] Updated weights for policy 0, policy_version 34980 (0.0009) [2023-10-08 09:12:00,629][53885] Updated weights for policy 1, policy_version 34792 (0.0008) [2023-10-08 09:12:00,668][53852] Updated weights for policy 0, policy_version 34990 (0.0009) [2023-10-08 09:12:01,000][53885] Updated weights for policy 1, policy_version 34802 (0.0008) [2023-10-08 09:12:01,038][53852] Updated weights for policy 0, policy_version 35000 (0.0008) [2023-10-08 09:12:01,364][53885] Updated weights for policy 1, policy_version 34812 (0.0008) [2023-10-08 09:12:02,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 71499776. Throughput: 0: 1818.6, 1: 1819.2. Samples: 17877830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:02,016][52710] Avg episode reward: [(0, '9.590'), (1, '33.180')] [2023-10-08 09:12:04,570][53852] Updated weights for policy 0, policy_version 35010 (0.0008) [2023-10-08 09:12:04,947][53852] Updated weights for policy 0, policy_version 35020 (0.0008) [2023-10-08 09:12:05,014][53885] Updated weights for policy 1, policy_version 34822 (0.0009) [2023-10-08 09:12:05,319][53852] Updated weights for policy 0, policy_version 35030 (0.0009) [2023-10-08 09:12:05,391][53885] Updated weights for policy 1, policy_version 34832 (0.0008) [2023-10-08 09:12:05,689][53852] Updated weights for policy 0, policy_version 35040 (0.0008) [2023-10-08 09:12:05,762][53885] Updated weights for policy 1, policy_version 34842 (0.0009) [2023-10-08 09:12:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 71565312. Throughput: 0: 1814.2, 1: 1806.4. Samples: 17898438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:07,016][52710] Avg episode reward: [(0, '10.820'), (1, '30.380')] [2023-10-08 09:12:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000035040_35880960.pth... [2023-10-08 09:12:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000034848_35684352.pth... [2023-10-08 09:12:07,062][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000033312_34111488.pth [2023-10-08 09:12:07,065][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000033152_33947648.pth [2023-10-08 09:12:09,297][53885] Updated weights for policy 1, policy_version 34852 (0.0007) [2023-10-08 09:12:09,459][53852] Updated weights for policy 0, policy_version 35050 (0.0008) [2023-10-08 09:12:09,659][53885] Updated weights for policy 1, policy_version 34862 (0.0008) [2023-10-08 09:12:09,835][53852] Updated weights for policy 0, policy_version 35060 (0.0007) [2023-10-08 09:12:10,028][53885] Updated weights for policy 1, policy_version 34872 (0.0008) [2023-10-08 09:12:10,197][53852] Updated weights for policy 0, policy_version 35070 (0.0007) [2023-10-08 09:12:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 71630848. Throughput: 0: 1819.7, 1: 1816.5. Samples: 17910502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:12,016][52710] Avg episode reward: [(0, '11.210'), (1, '28.370')] [2023-10-08 09:12:13,797][53885] Updated weights for policy 1, policy_version 34882 (0.0009) [2023-10-08 09:12:13,829][53852] Updated weights for policy 0, policy_version 35080 (0.0007) [2023-10-08 09:12:14,153][53885] Updated weights for policy 1, policy_version 34892 (0.0008) [2023-10-08 09:12:14,198][53852] Updated weights for policy 0, policy_version 35090 (0.0007) [2023-10-08 09:12:14,517][53885] Updated weights for policy 1, policy_version 34902 (0.0007) [2023-10-08 09:12:14,562][53852] Updated weights for policy 0, policy_version 35100 (0.0008) [2023-10-08 09:12:14,893][53885] Updated weights for policy 1, policy_version 34912 (0.0008) [2023-10-08 09:12:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71696384. Throughput: 0: 1819.6, 1: 1807.9. Samples: 17931178. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:17,016][52710] Avg episode reward: [(0, '11.520'), (1, '31.140')] [2023-10-08 09:12:18,401][53852] Updated weights for policy 0, policy_version 35110 (0.0007) [2023-10-08 09:12:18,613][53885] Updated weights for policy 1, policy_version 34922 (0.0007) [2023-10-08 09:12:18,785][53852] Updated weights for policy 0, policy_version 35120 (0.0009) [2023-10-08 09:12:18,982][53885] Updated weights for policy 1, policy_version 34932 (0.0009) [2023-10-08 09:12:19,166][53852] Updated weights for policy 0, policy_version 35130 (0.0010) [2023-10-08 09:12:19,341][53885] Updated weights for policy 1, policy_version 34942 (0.0008) [2023-10-08 09:12:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71761920. Throughput: 0: 1815.8, 1: 1808.3. Samples: 17953662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:22,016][52710] Avg episode reward: [(0, '10.480'), (1, '31.320')] [2023-10-08 09:12:22,853][53852] Updated weights for policy 0, policy_version 35140 (0.0008) [2023-10-08 09:12:23,141][53885] Updated weights for policy 1, policy_version 34952 (0.0009) [2023-10-08 09:12:23,227][53852] Updated weights for policy 0, policy_version 35150 (0.0009) [2023-10-08 09:12:23,508][53885] Updated weights for policy 1, policy_version 34962 (0.0007) [2023-10-08 09:12:23,589][53852] Updated weights for policy 0, policy_version 35160 (0.0008) [2023-10-08 09:12:23,872][53885] Updated weights for policy 1, policy_version 34972 (0.0007) [2023-10-08 09:12:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71827456. Throughput: 0: 1817.4, 1: 1806.8. Samples: 17963580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:27,015][52710] Avg episode reward: [(0, '12.210'), (1, '32.160')] [2023-10-08 09:12:27,326][53852] Updated weights for policy 0, policy_version 35170 (0.0008) [2023-10-08 09:12:27,584][53885] Updated weights for policy 1, policy_version 34982 (0.0008) [2023-10-08 09:12:27,699][53852] Updated weights for policy 0, policy_version 35180 (0.0009) [2023-10-08 09:12:27,949][53885] Updated weights for policy 1, policy_version 34992 (0.0008) [2023-10-08 09:12:28,066][53852] Updated weights for policy 0, policy_version 35190 (0.0009) [2023-10-08 09:12:28,303][53885] Updated weights for policy 1, policy_version 35002 (0.0008) [2023-10-08 09:12:28,436][53852] Updated weights for policy 0, policy_version 35200 (0.0009) [2023-10-08 09:12:31,935][53885] Updated weights for policy 1, policy_version 35012 (0.0009) [2023-10-08 09:12:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71892992. Throughput: 0: 1821.8, 1: 1812.0. Samples: 17986596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:32,016][52710] Avg episode reward: [(0, '12.090'), (1, '31.810')] [2023-10-08 09:12:32,077][53852] Updated weights for policy 0, policy_version 35210 (0.0008) [2023-10-08 09:12:32,300][53885] Updated weights for policy 1, policy_version 35022 (0.0007) [2023-10-08 09:12:32,437][53852] Updated weights for policy 0, policy_version 35220 (0.0008) [2023-10-08 09:12:32,653][53885] Updated weights for policy 1, policy_version 35032 (0.0007) [2023-10-08 09:12:32,806][53852] Updated weights for policy 0, policy_version 35230 (0.0008) [2023-10-08 09:12:36,234][53885] Updated weights for policy 1, policy_version 35042 (0.0007) [2023-10-08 09:12:36,519][53852] Updated weights for policy 0, policy_version 35240 (0.0007) [2023-10-08 09:12:36,604][53885] Updated weights for policy 1, policy_version 35052 (0.0007) [2023-10-08 09:12:36,876][53852] Updated weights for policy 0, policy_version 35250 (0.0008) [2023-10-08 09:12:36,976][53885] Updated weights for policy 1, policy_version 35062 (0.0008) [2023-10-08 09:12:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71958528. Throughput: 0: 1811.2, 1: 1813.1. Samples: 18008450. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:37,016][52710] Avg episode reward: [(0, '13.110'), (1, '32.810')] [2023-10-08 09:12:37,242][53852] Updated weights for policy 0, policy_version 35260 (0.0009) [2023-10-08 09:12:37,342][53885] Updated weights for policy 1, policy_version 35072 (0.0007) [2023-10-08 09:12:40,864][53852] Updated weights for policy 0, policy_version 35270 (0.0009) [2023-10-08 09:12:40,893][53885] Updated weights for policy 1, policy_version 35082 (0.0009) [2023-10-08 09:12:41,238][53852] Updated weights for policy 0, policy_version 35280 (0.0009) [2023-10-08 09:12:41,260][53885] Updated weights for policy 1, policy_version 35092 (0.0009) [2023-10-08 09:12:41,592][53852] Updated weights for policy 0, policy_version 35290 (0.0008) [2023-10-08 09:12:41,625][53885] Updated weights for policy 1, policy_version 35102 (0.0008) [2023-10-08 09:12:42,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 72089600. Throughput: 0: 1819.3, 1: 1824.5. Samples: 18019676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:42,016][52710] Avg episode reward: [(0, '13.030'), (1, '30.180')] [2023-10-08 09:12:45,182][53852] Updated weights for policy 0, policy_version 35300 (0.0010) [2023-10-08 09:12:45,356][53885] Updated weights for policy 1, policy_version 35112 (0.0008) [2023-10-08 09:12:45,557][53852] Updated weights for policy 0, policy_version 35310 (0.0009) [2023-10-08 09:12:45,724][53885] Updated weights for policy 1, policy_version 35122 (0.0007) [2023-10-08 09:12:45,929][53852] Updated weights for policy 0, policy_version 35320 (0.0009) [2023-10-08 09:12:46,086][53885] Updated weights for policy 1, policy_version 35132 (0.0008) [2023-10-08 09:12:47,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72155136. Throughput: 0: 1818.8, 1: 1817.6. Samples: 18041468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:47,016][52710] Avg episode reward: [(0, '15.530'), (1, '31.150')] [2023-10-08 09:12:49,648][53852] Updated weights for policy 0, policy_version 35330 (0.0009) [2023-10-08 09:12:49,883][53885] Updated weights for policy 1, policy_version 35142 (0.0007) [2023-10-08 09:12:50,023][53852] Updated weights for policy 0, policy_version 35340 (0.0007) [2023-10-08 09:12:50,270][53885] Updated weights for policy 1, policy_version 35152 (0.0007) [2023-10-08 09:12:50,382][53852] Updated weights for policy 0, policy_version 35350 (0.0008) [2023-10-08 09:12:50,638][53885] Updated weights for policy 1, policy_version 35162 (0.0007) [2023-10-08 09:12:50,752][53852] Updated weights for policy 0, policy_version 35360 (0.0007) [2023-10-08 09:12:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72220672. Throughput: 0: 1820.3, 1: 1825.3. Samples: 18062490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:52,016][52710] Avg episode reward: [(0, '15.830'), (1, '31.060')] [2023-10-08 09:12:54,133][53885] Updated weights for policy 1, policy_version 35172 (0.0009) [2023-10-08 09:12:54,498][53852] Updated weights for policy 0, policy_version 35370 (0.0007) [2023-10-08 09:12:54,499][53885] Updated weights for policy 1, policy_version 35182 (0.0008) [2023-10-08 09:12:54,864][53885] Updated weights for policy 1, policy_version 35192 (0.0007) [2023-10-08 09:12:54,866][53852] Updated weights for policy 0, policy_version 35380 (0.0007) [2023-10-08 09:12:55,245][53852] Updated weights for policy 0, policy_version 35390 (0.0008) [2023-10-08 09:12:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 72286208. Throughput: 0: 1819.9, 1: 1820.0. Samples: 18074296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:12:57,016][52710] Avg episode reward: [(0, '15.190'), (1, '29.160')] [2023-10-08 09:12:58,713][53885] Updated weights for policy 1, policy_version 35202 (0.0007) [2023-10-08 09:12:58,725][53852] Updated weights for policy 0, policy_version 35400 (0.0009) [2023-10-08 09:12:59,087][53885] Updated weights for policy 1, policy_version 35212 (0.0007) [2023-10-08 09:12:59,092][53852] Updated weights for policy 0, policy_version 35410 (0.0007) [2023-10-08 09:12:59,454][53885] Updated weights for policy 1, policy_version 35222 (0.0008) [2023-10-08 09:12:59,464][53852] Updated weights for policy 0, policy_version 35420 (0.0007) [2023-10-08 09:12:59,822][53885] Updated weights for policy 1, policy_version 35232 (0.0007) [2023-10-08 09:13:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72351744. Throughput: 0: 1824.3, 1: 1830.1. Samples: 18095624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:02,016][52710] Avg episode reward: [(0, '15.240'), (1, '32.040')] [2023-10-08 09:13:03,170][53852] Updated weights for policy 0, policy_version 35430 (0.0009) [2023-10-08 09:13:03,398][53885] Updated weights for policy 1, policy_version 35242 (0.0007) [2023-10-08 09:13:03,541][53852] Updated weights for policy 0, policy_version 35440 (0.0009) [2023-10-08 09:13:03,766][53885] Updated weights for policy 1, policy_version 35252 (0.0007) [2023-10-08 09:13:03,918][53852] Updated weights for policy 0, policy_version 35450 (0.0008) [2023-10-08 09:13:04,131][53885] Updated weights for policy 1, policy_version 35262 (0.0007) [2023-10-08 09:13:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72417280. Throughput: 0: 1824.1, 1: 1838.0. Samples: 18118454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:07,016][52710] Avg episode reward: [(0, '15.770'), (1, '34.020')] [2023-10-08 09:13:07,654][53885] Updated weights for policy 1, policy_version 35272 (0.0008) [2023-10-08 09:13:07,695][53852] Updated weights for policy 0, policy_version 35460 (0.0008) [2023-10-08 09:13:08,015][53885] Updated weights for policy 1, policy_version 35282 (0.0009) [2023-10-08 09:13:08,067][53852] Updated weights for policy 0, policy_version 35470 (0.0009) [2023-10-08 09:13:08,388][53885] Updated weights for policy 1, policy_version 35292 (0.0009) [2023-10-08 09:13:08,429][53852] Updated weights for policy 0, policy_version 35480 (0.0008) [2023-10-08 09:13:11,962][53885] Updated weights for policy 1, policy_version 35302 (0.0008) [2023-10-08 09:13:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72482816. Throughput: 0: 1825.7, 1: 1840.9. Samples: 18128578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:12,016][52710] Avg episode reward: [(0, '16.590'), (1, '32.150')] [2023-10-08 09:13:12,165][53852] Updated weights for policy 0, policy_version 35490 (0.0007) [2023-10-08 09:13:12,314][53885] Updated weights for policy 1, policy_version 35312 (0.0009) [2023-10-08 09:13:12,528][53852] Updated weights for policy 0, policy_version 35500 (0.0007) [2023-10-08 09:13:12,680][53885] Updated weights for policy 1, policy_version 35322 (0.0007) [2023-10-08 09:13:12,901][53852] Updated weights for policy 0, policy_version 35510 (0.0008) [2023-10-08 09:13:13,269][53852] Updated weights for policy 0, policy_version 35520 (0.0009) [2023-10-08 09:13:16,225][53885] Updated weights for policy 1, policy_version 35332 (0.0007) [2023-10-08 09:13:16,599][53885] Updated weights for policy 1, policy_version 35342 (0.0007) [2023-10-08 09:13:16,961][53852] Updated weights for policy 0, policy_version 35530 (0.0007) [2023-10-08 09:13:16,962][53885] Updated weights for policy 1, policy_version 35352 (0.0008) [2023-10-08 09:13:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 72548352. Throughput: 0: 1830.9, 1: 1843.9. Samples: 18151958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:17,016][52710] Avg episode reward: [(0, '16.410'), (1, '32.320')] [2023-10-08 09:13:17,330][53852] Updated weights for policy 0, policy_version 35540 (0.0008) [2023-10-08 09:13:17,706][53852] Updated weights for policy 0, policy_version 35550 (0.0007) [2023-10-08 09:13:20,643][53885] Updated weights for policy 1, policy_version 35362 (0.0007) [2023-10-08 09:13:21,019][53885] Updated weights for policy 1, policy_version 35372 (0.0009) [2023-10-08 09:13:21,381][53885] Updated weights for policy 1, policy_version 35382 (0.0008) [2023-10-08 09:13:21,501][53852] Updated weights for policy 0, policy_version 35560 (0.0007) [2023-10-08 09:13:21,752][53885] Updated weights for policy 1, policy_version 35392 (0.0007) [2023-10-08 09:13:21,871][53852] Updated weights for policy 0, policy_version 35570 (0.0007) [2023-10-08 09:13:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 72646656. Throughput: 0: 1830.1, 1: 1827.7. Samples: 18173054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:22,016][52710] Avg episode reward: [(0, '17.450'), (1, '30.720')] [2023-10-08 09:13:22,245][53852] Updated weights for policy 0, policy_version 35580 (0.0008) [2023-10-08 09:13:25,688][53885] Updated weights for policy 1, policy_version 35402 (0.0007) [2023-10-08 09:13:25,993][53852] Updated weights for policy 0, policy_version 35590 (0.0009) [2023-10-08 09:13:26,054][53885] Updated weights for policy 1, policy_version 35412 (0.0007) [2023-10-08 09:13:26,352][53852] Updated weights for policy 0, policy_version 35600 (0.0007) [2023-10-08 09:13:26,418][53885] Updated weights for policy 1, policy_version 35422 (0.0008) [2023-10-08 09:13:26,723][53852] Updated weights for policy 0, policy_version 35610 (0.0007) [2023-10-08 09:13:27,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72744960. Throughput: 0: 1827.1, 1: 1835.7. Samples: 18184504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:27,016][52710] Avg episode reward: [(0, '20.920'), (1, '34.970')] [2023-10-08 09:13:27,017][53594] Saving new best policy, reward=34.970! [2023-10-08 09:13:30,088][53885] Updated weights for policy 1, policy_version 35432 (0.0009) [2023-10-08 09:13:30,455][53885] Updated weights for policy 1, policy_version 35442 (0.0008) [2023-10-08 09:13:30,489][53852] Updated weights for policy 0, policy_version 35620 (0.0008) [2023-10-08 09:13:30,815][53885] Updated weights for policy 1, policy_version 35452 (0.0007) [2023-10-08 09:13:30,851][53852] Updated weights for policy 0, policy_version 35630 (0.0008) [2023-10-08 09:13:31,215][53852] Updated weights for policy 0, policy_version 35640 (0.0007) [2023-10-08 09:13:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 72810496. Throughput: 0: 1828.5, 1: 1828.1. Samples: 18206016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:32,016][52710] Avg episode reward: [(0, '21.730'), (1, '32.190')] [2023-10-08 09:13:34,626][53885] Updated weights for policy 1, policy_version 35462 (0.0009) [2023-10-08 09:13:34,977][53852] Updated weights for policy 0, policy_version 35650 (0.0010) [2023-10-08 09:13:35,004][53885] Updated weights for policy 1, policy_version 35472 (0.0008) [2023-10-08 09:13:35,341][53852] Updated weights for policy 0, policy_version 35660 (0.0008) [2023-10-08 09:13:35,375][53885] Updated weights for policy 1, policy_version 35482 (0.0008) [2023-10-08 09:13:35,713][53852] Updated weights for policy 0, policy_version 35670 (0.0010) [2023-10-08 09:13:36,080][53852] Updated weights for policy 0, policy_version 35680 (0.0007) [2023-10-08 09:13:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 72876032. Throughput: 0: 1815.1, 1: 1844.3. Samples: 18227164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:37,015][52710] Avg episode reward: [(0, '23.770'), (1, '32.690')] [2023-10-08 09:13:38,925][53885] Updated weights for policy 1, policy_version 35492 (0.0008) [2023-10-08 09:13:39,286][53885] Updated weights for policy 1, policy_version 35502 (0.0010) [2023-10-08 09:13:39,653][53885] Updated weights for policy 1, policy_version 35512 (0.0008) [2023-10-08 09:13:39,720][53852] Updated weights for policy 0, policy_version 35690 (0.0008) [2023-10-08 09:13:40,093][53852] Updated weights for policy 0, policy_version 35700 (0.0007) [2023-10-08 09:13:40,454][53852] Updated weights for policy 0, policy_version 35710 (0.0009) [2023-10-08 09:13:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 72941568. Throughput: 0: 1822.1, 1: 1834.0. Samples: 18238820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:42,015][52710] Avg episode reward: [(0, '26.260'), (1, '31.740')] [2023-10-08 09:13:43,414][53885] Updated weights for policy 1, policy_version 35522 (0.0008) [2023-10-08 09:13:43,785][53885] Updated weights for policy 1, policy_version 35532 (0.0008) [2023-10-08 09:13:44,144][53885] Updated weights for policy 1, policy_version 35542 (0.0009) [2023-10-08 09:13:44,193][53852] Updated weights for policy 0, policy_version 35720 (0.0007) [2023-10-08 09:13:44,511][53885] Updated weights for policy 1, policy_version 35552 (0.0008) [2023-10-08 09:13:44,554][53852] Updated weights for policy 0, policy_version 35730 (0.0007) [2023-10-08 09:13:44,936][53852] Updated weights for policy 0, policy_version 35740 (0.0007) [2023-10-08 09:13:47,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73007104. Throughput: 0: 1811.3, 1: 1839.6. Samples: 18259916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:13:47,016][52710] Avg episode reward: [(0, '25.070'), (1, '32.230')] [2023-10-08 09:13:48,202][53885] Updated weights for policy 1, policy_version 35562 (0.0009) [2023-10-08 09:13:48,567][53885] Updated weights for policy 1, policy_version 35572 (0.0008) [2023-10-08 09:13:48,694][53852] Updated weights for policy 0, policy_version 35750 (0.0007) [2023-10-08 09:13:48,931][53885] Updated weights for policy 1, policy_version 35582 (0.0007) [2023-10-08 09:13:49,075][53852] Updated weights for policy 0, policy_version 35760 (0.0007) [2023-10-08 09:13:49,452][53852] Updated weights for policy 0, policy_version 35770 (0.0007) [2023-10-08 09:13:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73072640. Throughput: 0: 1813.6, 1: 1838.4. Samples: 18282796. Policy #0 lag: (min: 19.0, avg: 19.1, max: 25.0) [2023-10-08 09:13:52,016][52710] Avg episode reward: [(0, '23.680'), (1, '32.320')] [2023-10-08 09:13:52,447][53885] Updated weights for policy 1, policy_version 35592 (0.0009) [2023-10-08 09:13:52,811][53885] Updated weights for policy 1, policy_version 35602 (0.0009) [2023-10-08 09:13:53,016][53852] Updated weights for policy 0, policy_version 35780 (0.0007) [2023-10-08 09:13:53,177][53885] Updated weights for policy 1, policy_version 35612 (0.0007) [2023-10-08 09:13:53,395][53852] Updated weights for policy 0, policy_version 35790 (0.0009) [2023-10-08 09:13:53,776][53852] Updated weights for policy 0, policy_version 35800 (0.0010) [2023-10-08 09:13:56,795][53885] Updated weights for policy 1, policy_version 35622 (0.0008) [2023-10-08 09:13:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 73138176. Throughput: 0: 1814.5, 1: 1840.2. Samples: 18293040. Policy #0 lag: (min: 19.0, avg: 19.1, max: 25.0) [2023-10-08 09:13:57,016][52710] Avg episode reward: [(0, '25.240'), (1, '29.060')] [2023-10-08 09:13:57,171][53885] Updated weights for policy 1, policy_version 35632 (0.0010) [2023-10-08 09:13:57,482][53852] Updated weights for policy 0, policy_version 35810 (0.0008) [2023-10-08 09:13:57,540][53885] Updated weights for policy 1, policy_version 35642 (0.0008) [2023-10-08 09:13:57,853][53852] Updated weights for policy 0, policy_version 35820 (0.0008) [2023-10-08 09:13:58,226][53852] Updated weights for policy 0, policy_version 35830 (0.0007) [2023-10-08 09:13:58,594][53852] Updated weights for policy 0, policy_version 35840 (0.0007) [2023-10-08 09:14:01,323][53885] Updated weights for policy 1, policy_version 35652 (0.0007) [2023-10-08 09:14:01,699][53885] Updated weights for policy 1, policy_version 35662 (0.0008) [2023-10-08 09:14:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73203712. Throughput: 0: 1813.7, 1: 1830.7. Samples: 18315958. Policy #0 lag: (min: 19.0, avg: 19.1, max: 25.0) [2023-10-08 09:14:02,016][52710] Avg episode reward: [(0, '25.540'), (1, '29.490')] [2023-10-08 09:14:02,055][53852] Updated weights for policy 0, policy_version 35850 (0.0009) [2023-10-08 09:14:02,071][53885] Updated weights for policy 1, policy_version 35672 (0.0007) [2023-10-08 09:14:02,421][53852] Updated weights for policy 0, policy_version 35860 (0.0008) [2023-10-08 09:14:02,784][53852] Updated weights for policy 0, policy_version 35870 (0.0010) [2023-10-08 09:14:05,742][53885] Updated weights for policy 1, policy_version 35682 (0.0007) [2023-10-08 09:14:06,105][53885] Updated weights for policy 1, policy_version 35692 (0.0008) [2023-10-08 09:14:06,439][53852] Updated weights for policy 0, policy_version 35880 (0.0008) [2023-10-08 09:14:06,477][53885] Updated weights for policy 1, policy_version 35702 (0.0008) [2023-10-08 09:14:06,803][53852] Updated weights for policy 0, policy_version 35890 (0.0009) [2023-10-08 09:14:06,844][53885] Updated weights for policy 1, policy_version 35712 (0.0008) [2023-10-08 09:14:07,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 73302016. Throughput: 0: 1816.8, 1: 1831.4. Samples: 18337222. Policy #0 lag: (min: 19.0, avg: 19.1, max: 25.0) [2023-10-08 09:14:07,016][52710] Avg episode reward: [(0, '25.140'), (1, '29.960')] [2023-10-08 09:14:07,030][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000035712_36569088.pth... [2023-10-08 09:14:07,070][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000034016_34832384.pth [2023-10-08 09:14:07,174][53852] Updated weights for policy 0, policy_version 35900 (0.0008) [2023-10-08 09:14:07,322][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000035904_36765696.pth... [2023-10-08 09:14:07,360][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000034176_34996224.pth [2023-10-08 09:14:10,474][53885] Updated weights for policy 1, policy_version 35722 (0.0008) [2023-10-08 09:14:10,838][53885] Updated weights for policy 1, policy_version 35732 (0.0007) [2023-10-08 09:14:10,870][53852] Updated weights for policy 0, policy_version 35910 (0.0008) [2023-10-08 09:14:11,211][53885] Updated weights for policy 1, policy_version 35742 (0.0007) [2023-10-08 09:14:11,247][53852] Updated weights for policy 0, policy_version 35920 (0.0009) [2023-10-08 09:14:11,624][53852] Updated weights for policy 0, policy_version 35930 (0.0010) [2023-10-08 09:14:12,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 73400320. Throughput: 0: 1817.7, 1: 1833.9. Samples: 18348828. Policy #0 lag: (min: 19.0, avg: 19.1, max: 25.0) [2023-10-08 09:14:12,016][52710] Avg episode reward: [(0, '26.830'), (1, '28.530')] [2023-10-08 09:14:14,878][53885] Updated weights for policy 1, policy_version 35752 (0.0007) [2023-10-08 09:14:15,247][53885] Updated weights for policy 1, policy_version 35762 (0.0009) [2023-10-08 09:14:15,444][53852] Updated weights for policy 0, policy_version 35940 (0.0009) [2023-10-08 09:14:15,615][53885] Updated weights for policy 1, policy_version 35772 (0.0009) [2023-10-08 09:14:15,814][53852] Updated weights for policy 0, policy_version 35950 (0.0008) [2023-10-08 09:14:16,186][53852] Updated weights for policy 0, policy_version 35960 (0.0007) [2023-10-08 09:14:17,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 73465856. Throughput: 0: 1816.1, 1: 1828.5. Samples: 18370022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:17,016][52710] Avg episode reward: [(0, '27.180'), (1, '28.830')] [2023-10-08 09:14:19,275][53885] Updated weights for policy 1, policy_version 35782 (0.0009) [2023-10-08 09:14:19,655][53885] Updated weights for policy 1, policy_version 35792 (0.0007) [2023-10-08 09:14:19,791][53852] Updated weights for policy 0, policy_version 35970 (0.0007) [2023-10-08 09:14:20,031][53885] Updated weights for policy 1, policy_version 35802 (0.0007) [2023-10-08 09:14:20,173][53852] Updated weights for policy 0, policy_version 35980 (0.0009) [2023-10-08 09:14:20,542][53852] Updated weights for policy 0, policy_version 35990 (0.0010) [2023-10-08 09:14:20,906][53852] Updated weights for policy 0, policy_version 36000 (0.0009) [2023-10-08 09:14:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 73531392. Throughput: 0: 1828.6, 1: 1828.8. Samples: 18391748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:22,016][52710] Avg episode reward: [(0, '26.480'), (1, '29.650')] [2023-10-08 09:14:23,625][53885] Updated weights for policy 1, policy_version 35812 (0.0007) [2023-10-08 09:14:23,985][53885] Updated weights for policy 1, policy_version 35822 (0.0008) [2023-10-08 09:14:24,354][53885] Updated weights for policy 1, policy_version 35832 (0.0010) [2023-10-08 09:14:24,459][53852] Updated weights for policy 0, policy_version 36010 (0.0010) [2023-10-08 09:14:24,828][53852] Updated weights for policy 0, policy_version 36020 (0.0008) [2023-10-08 09:14:25,200][53852] Updated weights for policy 0, policy_version 36030 (0.0011) [2023-10-08 09:14:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 73596928. Throughput: 0: 1824.1, 1: 1821.6. Samples: 18402878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:27,016][52710] Avg episode reward: [(0, '25.400'), (1, '33.090')] [2023-10-08 09:14:27,924][53885] Updated weights for policy 1, policy_version 35842 (0.0009) [2023-10-08 09:14:28,278][53885] Updated weights for policy 1, policy_version 35852 (0.0007) [2023-10-08 09:14:28,646][53885] Updated weights for policy 1, policy_version 35862 (0.0007) [2023-10-08 09:14:28,861][53852] Updated weights for policy 0, policy_version 36040 (0.0009) [2023-10-08 09:14:29,005][53885] Updated weights for policy 1, policy_version 35872 (0.0007) [2023-10-08 09:14:29,231][53852] Updated weights for policy 0, policy_version 36050 (0.0010) [2023-10-08 09:14:29,605][53852] Updated weights for policy 0, policy_version 36060 (0.0008) [2023-10-08 09:14:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 73662464. Throughput: 0: 1828.0, 1: 1838.1. Samples: 18424894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:32,016][52710] Avg episode reward: [(0, '25.250'), (1, '28.650')] [2023-10-08 09:14:32,572][53885] Updated weights for policy 1, policy_version 35882 (0.0007) [2023-10-08 09:14:32,936][53885] Updated weights for policy 1, policy_version 35892 (0.0008) [2023-10-08 09:14:33,301][53885] Updated weights for policy 1, policy_version 35902 (0.0007) [2023-10-08 09:14:33,321][53852] Updated weights for policy 0, policy_version 36070 (0.0007) [2023-10-08 09:14:33,709][53852] Updated weights for policy 0, policy_version 36080 (0.0007) [2023-10-08 09:14:34,080][53852] Updated weights for policy 0, policy_version 36090 (0.0008) [2023-10-08 09:14:36,891][53885] Updated weights for policy 1, policy_version 35912 (0.0007) [2023-10-08 09:14:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73728000. Throughput: 0: 1827.7, 1: 1842.2. Samples: 18447942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:37,015][52710] Avg episode reward: [(0, '25.440'), (1, '31.350')] [2023-10-08 09:14:37,259][53885] Updated weights for policy 1, policy_version 35922 (0.0007) [2023-10-08 09:14:37,627][53885] Updated weights for policy 1, policy_version 35932 (0.0008) [2023-10-08 09:14:37,748][53852] Updated weights for policy 0, policy_version 36100 (0.0009) [2023-10-08 09:14:38,116][53852] Updated weights for policy 0, policy_version 36110 (0.0009) [2023-10-08 09:14:38,485][53852] Updated weights for policy 0, policy_version 36120 (0.0008) [2023-10-08 09:14:41,237][53885] Updated weights for policy 1, policy_version 35942 (0.0009) [2023-10-08 09:14:41,598][53885] Updated weights for policy 1, policy_version 35952 (0.0008) [2023-10-08 09:14:41,962][53885] Updated weights for policy 1, policy_version 35962 (0.0009) [2023-10-08 09:14:41,995][53852] Updated weights for policy 0, policy_version 36130 (0.0007) [2023-10-08 09:14:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73793536. Throughput: 0: 1828.8, 1: 1842.7. Samples: 18458256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:42,016][52710] Avg episode reward: [(0, '25.600'), (1, '32.830')] [2023-10-08 09:14:42,362][53852] Updated weights for policy 0, policy_version 36140 (0.0007) [2023-10-08 09:14:42,733][53852] Updated weights for policy 0, policy_version 36150 (0.0009) [2023-10-08 09:14:43,100][53852] Updated weights for policy 0, policy_version 36160 (0.0010) [2023-10-08 09:14:45,608][53885] Updated weights for policy 1, policy_version 35972 (0.0009) [2023-10-08 09:14:45,974][53885] Updated weights for policy 1, policy_version 35982 (0.0009) [2023-10-08 09:14:46,343][53885] Updated weights for policy 1, policy_version 35992 (0.0009) [2023-10-08 09:14:46,691][53852] Updated weights for policy 0, policy_version 36170 (0.0007) [2023-10-08 09:14:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 73891840. Throughput: 0: 1834.5, 1: 1840.8. Samples: 18481346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:47,015][52710] Avg episode reward: [(0, '27.650'), (1, '31.510')] [2023-10-08 09:14:47,057][53852] Updated weights for policy 0, policy_version 36180 (0.0007) [2023-10-08 09:14:47,439][53852] Updated weights for policy 0, policy_version 36190 (0.0009) [2023-10-08 09:14:50,149][53885] Updated weights for policy 1, policy_version 36002 (0.0009) [2023-10-08 09:14:50,515][53885] Updated weights for policy 1, policy_version 36012 (0.0007) [2023-10-08 09:14:50,880][53885] Updated weights for policy 1, policy_version 36022 (0.0007) [2023-10-08 09:14:51,147][53852] Updated weights for policy 0, policy_version 36200 (0.0008) [2023-10-08 09:14:51,247][53885] Updated weights for policy 1, policy_version 36032 (0.0009) [2023-10-08 09:14:51,526][53852] Updated weights for policy 0, policy_version 36210 (0.0008) [2023-10-08 09:14:51,900][53852] Updated weights for policy 0, policy_version 36220 (0.0007) [2023-10-08 09:14:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73957376. Throughput: 0: 1822.0, 1: 1831.2. Samples: 18501616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:52,016][52710] Avg episode reward: [(0, '29.670'), (1, '30.770')] [2023-10-08 09:14:54,977][53885] Updated weights for policy 1, policy_version 36042 (0.0007) [2023-10-08 09:14:55,343][53885] Updated weights for policy 1, policy_version 36052 (0.0007) [2023-10-08 09:14:55,575][53852] Updated weights for policy 0, policy_version 36230 (0.0009) [2023-10-08 09:14:55,708][53885] Updated weights for policy 1, policy_version 36062 (0.0009) [2023-10-08 09:14:55,953][53852] Updated weights for policy 0, policy_version 36240 (0.0009) [2023-10-08 09:14:56,313][53852] Updated weights for policy 0, policy_version 36250 (0.0010) [2023-10-08 09:14:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 74055680. Throughput: 0: 1832.9, 1: 1837.5. Samples: 18513996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:14:57,016][52710] Avg episode reward: [(0, '29.810'), (1, '32.490')] [2023-10-08 09:14:59,465][53885] Updated weights for policy 1, policy_version 36072 (0.0008) [2023-10-08 09:14:59,828][53885] Updated weights for policy 1, policy_version 36082 (0.0008) [2023-10-08 09:14:59,878][53852] Updated weights for policy 0, policy_version 36260 (0.0008) [2023-10-08 09:15:00,194][53885] Updated weights for policy 1, policy_version 36092 (0.0011) [2023-10-08 09:15:00,250][53852] Updated weights for policy 0, policy_version 36270 (0.0008) [2023-10-08 09:15:00,622][53852] Updated weights for policy 0, policy_version 36280 (0.0010) [2023-10-08 09:15:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 74121216. Throughput: 0: 1821.1, 1: 1832.1. Samples: 18534416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:15:02,017][52710] Avg episode reward: [(0, '30.490'), (1, '32.030')] [2023-10-08 09:15:03,883][53885] Updated weights for policy 1, policy_version 36102 (0.0008) [2023-10-08 09:15:04,247][53885] Updated weights for policy 1, policy_version 36112 (0.0007) [2023-10-08 09:15:04,326][53852] Updated weights for policy 0, policy_version 36290 (0.0008) [2023-10-08 09:15:04,621][53885] Updated weights for policy 1, policy_version 36122 (0.0009) [2023-10-08 09:15:04,700][53852] Updated weights for policy 0, policy_version 36300 (0.0008) [2023-10-08 09:15:05,065][53852] Updated weights for policy 0, policy_version 36310 (0.0010) [2023-10-08 09:15:05,438][53852] Updated weights for policy 0, policy_version 36320 (0.0007) [2023-10-08 09:15:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74186752. Throughput: 0: 1829.2, 1: 1835.8. Samples: 18556676. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:15:07,016][52710] Avg episode reward: [(0, '28.360'), (1, '27.660')] [2023-10-08 09:15:08,409][53885] Updated weights for policy 1, policy_version 36132 (0.0007) [2023-10-08 09:15:08,788][53885] Updated weights for policy 1, policy_version 36142 (0.0009) [2023-10-08 09:15:09,126][53852] Updated weights for policy 0, policy_version 36330 (0.0009) [2023-10-08 09:15:09,153][53885] Updated weights for policy 1, policy_version 36152 (0.0008) [2023-10-08 09:15:09,491][53852] Updated weights for policy 0, policy_version 36340 (0.0009) [2023-10-08 09:15:09,865][53852] Updated weights for policy 0, policy_version 36350 (0.0009) [2023-10-08 09:15:12,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 74252288. Throughput: 0: 1823.4, 1: 1826.8. Samples: 18567134. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:15:12,016][52710] Avg episode reward: [(0, '29.300'), (1, '28.300')] [2023-10-08 09:15:12,807][53885] Updated weights for policy 1, policy_version 36162 (0.0008) [2023-10-08 09:15:13,177][53885] Updated weights for policy 1, policy_version 36172 (0.0009) [2023-10-08 09:15:13,544][53885] Updated weights for policy 1, policy_version 36182 (0.0008) [2023-10-08 09:15:13,645][53852] Updated weights for policy 0, policy_version 36360 (0.0007) [2023-10-08 09:15:13,913][53885] Updated weights for policy 1, policy_version 36192 (0.0009) [2023-10-08 09:15:14,013][53852] Updated weights for policy 0, policy_version 36370 (0.0008) [2023-10-08 09:15:14,382][53852] Updated weights for policy 0, policy_version 36380 (0.0008) [2023-10-08 09:15:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 74317824. Throughput: 0: 1829.1, 1: 1818.0. Samples: 18589010. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:15:17,016][52710] Avg episode reward: [(0, '29.440'), (1, '23.970')] [2023-10-08 09:15:17,615][53885] Updated weights for policy 1, policy_version 36202 (0.0008) [2023-10-08 09:15:17,979][53885] Updated weights for policy 1, policy_version 36212 (0.0008) [2023-10-08 09:15:18,172][53852] Updated weights for policy 0, policy_version 36390 (0.0008) [2023-10-08 09:15:18,342][53885] Updated weights for policy 1, policy_version 36222 (0.0008) [2023-10-08 09:15:18,546][53852] Updated weights for policy 0, policy_version 36400 (0.0008) [2023-10-08 09:15:18,918][53852] Updated weights for policy 0, policy_version 36410 (0.0007) [2023-10-08 09:15:21,943][53885] Updated weights for policy 1, policy_version 36232 (0.0010) [2023-10-08 09:15:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 74383360. Throughput: 0: 1835.8, 1: 1814.8. Samples: 18612220. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:15:22,016][52710] Avg episode reward: [(0, '26.200'), (1, '22.870')] [2023-10-08 09:15:22,310][53885] Updated weights for policy 1, policy_version 36242 (0.0011) [2023-10-08 09:15:22,532][53852] Updated weights for policy 0, policy_version 36420 (0.0007) [2023-10-08 09:15:22,674][53885] Updated weights for policy 1, policy_version 36252 (0.0007) [2023-10-08 09:15:22,902][53852] Updated weights for policy 0, policy_version 36430 (0.0008) [2023-10-08 09:15:23,269][53852] Updated weights for policy 0, policy_version 36440 (0.0007) [2023-10-08 09:15:26,494][53885] Updated weights for policy 1, policy_version 36262 (0.0008) [2023-10-08 09:15:26,852][53852] Updated weights for policy 0, policy_version 36450 (0.0008) [2023-10-08 09:15:26,860][53885] Updated weights for policy 1, policy_version 36272 (0.0009) [2023-10-08 09:15:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74448896. Throughput: 0: 1836.0, 1: 1812.5. Samples: 18622438. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:15:27,015][52710] Avg episode reward: [(0, '28.160'), (1, '24.680')] [2023-10-08 09:15:27,215][53852] Updated weights for policy 0, policy_version 36460 (0.0007) [2023-10-08 09:15:27,224][53885] Updated weights for policy 1, policy_version 36282 (0.0007) [2023-10-08 09:15:27,580][53852] Updated weights for policy 0, policy_version 36470 (0.0009) [2023-10-08 09:15:27,958][53852] Updated weights for policy 0, policy_version 36480 (0.0008) [2023-10-08 09:15:30,875][53885] Updated weights for policy 1, policy_version 36292 (0.0007) [2023-10-08 09:15:31,234][53885] Updated weights for policy 1, policy_version 36302 (0.0007) [2023-10-08 09:15:31,597][53885] Updated weights for policy 1, policy_version 36312 (0.0007) [2023-10-08 09:15:31,605][53852] Updated weights for policy 0, policy_version 36490 (0.0007) [2023-10-08 09:15:31,968][53852] Updated weights for policy 0, policy_version 36500 (0.0007) [2023-10-08 09:15:32,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 74547200. Throughput: 0: 1832.5, 1: 1813.2. Samples: 18645400. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:15:32,015][52710] Avg episode reward: [(0, '31.810'), (1, '26.300')] [2023-10-08 09:15:32,344][53852] Updated weights for policy 0, policy_version 36510 (0.0009) [2023-10-08 09:15:35,259][53885] Updated weights for policy 1, policy_version 36322 (0.0009) [2023-10-08 09:15:35,629][53885] Updated weights for policy 1, policy_version 36332 (0.0011) [2023-10-08 09:15:35,999][53885] Updated weights for policy 1, policy_version 36342 (0.0008) [2023-10-08 09:15:36,065][53852] Updated weights for policy 0, policy_version 36520 (0.0009) [2023-10-08 09:15:36,359][53885] Updated weights for policy 1, policy_version 36352 (0.0007) [2023-10-08 09:15:36,442][53852] Updated weights for policy 0, policy_version 36530 (0.0010) [2023-10-08 09:15:36,806][53852] Updated weights for policy 0, policy_version 36540 (0.0009) [2023-10-08 09:15:37,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 74645504. Throughput: 0: 1832.8, 1: 1816.1. Samples: 18665816. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:15:37,016][52710] Avg episode reward: [(0, '30.120'), (1, '29.500')] [2023-10-08 09:15:40,267][53885] Updated weights for policy 1, policy_version 36362 (0.0011) [2023-10-08 09:15:40,533][53852] Updated weights for policy 0, policy_version 36550 (0.0008) [2023-10-08 09:15:40,621][53885] Updated weights for policy 1, policy_version 36372 (0.0009) [2023-10-08 09:15:40,896][53852] Updated weights for policy 0, policy_version 36560 (0.0007) [2023-10-08 09:15:40,981][53885] Updated weights for policy 1, policy_version 36382 (0.0008) [2023-10-08 09:15:41,258][53852] Updated weights for policy 0, policy_version 36570 (0.0007) [2023-10-08 09:15:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 74711040. Throughput: 0: 1831.9, 1: 1810.6. Samples: 18677910. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:15:42,016][52710] Avg episode reward: [(0, '31.660'), (1, '31.900')] [2023-10-08 09:15:44,879][53885] Updated weights for policy 1, policy_version 36392 (0.0008) [2023-10-08 09:15:45,062][53852] Updated weights for policy 0, policy_version 36580 (0.0008) [2023-10-08 09:15:45,247][53885] Updated weights for policy 1, policy_version 36402 (0.0007) [2023-10-08 09:15:45,434][53852] Updated weights for policy 0, policy_version 36590 (0.0008) [2023-10-08 09:15:45,616][53885] Updated weights for policy 1, policy_version 36412 (0.0007) [2023-10-08 09:15:45,811][53852] Updated weights for policy 0, policy_version 36600 (0.0008) [2023-10-08 09:15:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74776576. Throughput: 0: 1834.8, 1: 1805.6. Samples: 18698230. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:15:47,016][52710] Avg episode reward: [(0, '32.630'), (1, '31.880')] [2023-10-08 09:15:49,334][53852] Updated weights for policy 0, policy_version 36610 (0.0008) [2023-10-08 09:15:49,363][53885] Updated weights for policy 1, policy_version 36422 (0.0007) [2023-10-08 09:15:49,703][53852] Updated weights for policy 0, policy_version 36620 (0.0009) [2023-10-08 09:15:49,722][53885] Updated weights for policy 1, policy_version 36432 (0.0008) [2023-10-08 09:15:50,064][53852] Updated weights for policy 0, policy_version 36630 (0.0008) [2023-10-08 09:15:50,086][53885] Updated weights for policy 1, policy_version 36442 (0.0007) [2023-10-08 09:15:50,427][53852] Updated weights for policy 0, policy_version 36640 (0.0009) [2023-10-08 09:15:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74842112. Throughput: 0: 1833.5, 1: 1796.1. Samples: 18720010. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:15:52,016][52710] Avg episode reward: [(0, '30.840'), (1, '30.790')] [2023-10-08 09:15:53,837][53885] Updated weights for policy 1, policy_version 36452 (0.0008) [2023-10-08 09:15:54,116][53852] Updated weights for policy 0, policy_version 36650 (0.0008) [2023-10-08 09:15:54,195][53885] Updated weights for policy 1, policy_version 36462 (0.0008) [2023-10-08 09:15:54,484][53852] Updated weights for policy 0, policy_version 36660 (0.0008) [2023-10-08 09:15:54,558][53885] Updated weights for policy 1, policy_version 36472 (0.0007) [2023-10-08 09:15:54,863][53852] Updated weights for policy 0, policy_version 36670 (0.0008) [2023-10-08 09:15:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 74907648. Throughput: 0: 1829.7, 1: 1812.6. Samples: 18731036. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:15:57,016][52710] Avg episode reward: [(0, '31.050'), (1, '33.580')] [2023-10-08 09:15:58,356][53885] Updated weights for policy 1, policy_version 36482 (0.0008) [2023-10-08 09:15:58,414][53852] Updated weights for policy 0, policy_version 36680 (0.0007) [2023-10-08 09:15:58,728][53885] Updated weights for policy 1, policy_version 36492 (0.0007) [2023-10-08 09:15:58,783][53852] Updated weights for policy 0, policy_version 36690 (0.0008) [2023-10-08 09:15:59,095][53885] Updated weights for policy 1, policy_version 36502 (0.0008) [2023-10-08 09:15:59,147][53852] Updated weights for policy 0, policy_version 36700 (0.0010) [2023-10-08 09:15:59,462][53885] Updated weights for policy 1, policy_version 36512 (0.0008) [2023-10-08 09:16:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74973184. Throughput: 0: 1841.2, 1: 1806.4. Samples: 18753150. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:16:02,016][52710] Avg episode reward: [(0, '30.280'), (1, '32.630')] [2023-10-08 09:16:02,816][53852] Updated weights for policy 0, policy_version 36710 (0.0007) [2023-10-08 09:16:03,050][53885] Updated weights for policy 1, policy_version 36522 (0.0009) [2023-10-08 09:16:03,197][53852] Updated weights for policy 0, policy_version 36720 (0.0008) [2023-10-08 09:16:03,425][53885] Updated weights for policy 1, policy_version 36532 (0.0007) [2023-10-08 09:16:03,561][53852] Updated weights for policy 0, policy_version 36730 (0.0007) [2023-10-08 09:16:03,787][53885] Updated weights for policy 1, policy_version 36542 (0.0007) [2023-10-08 09:16:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 75038720. Throughput: 0: 1842.7, 1: 1806.3. Samples: 18776422. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 09:16:07,017][52710] Avg episode reward: [(0, '30.070'), (1, '30.160')] [2023-10-08 09:16:07,091][53852] Updated weights for policy 0, policy_version 36740 (0.0008) [2023-10-08 09:16:07,338][53885] Updated weights for policy 1, policy_version 36552 (0.0008) [2023-10-08 09:16:07,459][53852] Updated weights for policy 0, policy_version 36750 (0.0008) [2023-10-08 09:16:07,701][53885] Updated weights for policy 1, policy_version 36562 (0.0007) [2023-10-08 09:16:07,827][53852] Updated weights for policy 0, policy_version 36760 (0.0007) [2023-10-08 09:16:08,067][53885] Updated weights for policy 1, policy_version 36572 (0.0007) [2023-10-08 09:16:08,114][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000036768_37650432.pth... [2023-10-08 09:16:08,153][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000035040_35880960.pth [2023-10-08 09:16:08,210][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000036576_37453824.pth... [2023-10-08 09:16:08,250][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000034848_35684352.pth [2023-10-08 09:16:11,403][53852] Updated weights for policy 0, policy_version 36770 (0.0007) [2023-10-08 09:16:11,735][53885] Updated weights for policy 1, policy_version 36582 (0.0009) [2023-10-08 09:16:11,773][53852] Updated weights for policy 0, policy_version 36780 (0.0008) [2023-10-08 09:16:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75104256. Throughput: 0: 1840.9, 1: 1803.1. Samples: 18786420. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 09:16:12,015][52710] Avg episode reward: [(0, '30.970'), (1, '31.360')] [2023-10-08 09:16:12,095][53885] Updated weights for policy 1, policy_version 36592 (0.0007) [2023-10-08 09:16:12,144][53852] Updated weights for policy 0, policy_version 36790 (0.0008) [2023-10-08 09:16:12,466][53885] Updated weights for policy 1, policy_version 36602 (0.0009) [2023-10-08 09:16:12,505][53852] Updated weights for policy 0, policy_version 36800 (0.0008) [2023-10-08 09:16:16,199][53852] Updated weights for policy 0, policy_version 36810 (0.0008) [2023-10-08 09:16:16,364][53885] Updated weights for policy 1, policy_version 36612 (0.0008) [2023-10-08 09:16:16,564][53852] Updated weights for policy 0, policy_version 36820 (0.0007) [2023-10-08 09:16:16,730][53885] Updated weights for policy 1, policy_version 36622 (0.0008) [2023-10-08 09:16:16,926][53852] Updated weights for policy 0, policy_version 36830 (0.0007) [2023-10-08 09:16:17,015][52710] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75202560. Throughput: 0: 1835.3, 1: 1800.3. Samples: 18809000. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 09:16:17,016][52710] Avg episode reward: [(0, '31.300'), (1, '29.350')] [2023-10-08 09:16:17,097][53885] Updated weights for policy 1, policy_version 36632 (0.0007) [2023-10-08 09:16:20,544][53852] Updated weights for policy 0, policy_version 36840 (0.0007) [2023-10-08 09:16:20,563][53885] Updated weights for policy 1, policy_version 36642 (0.0009) [2023-10-08 09:16:20,918][53852] Updated weights for policy 0, policy_version 36850 (0.0008) [2023-10-08 09:16:20,930][53885] Updated weights for policy 1, policy_version 36652 (0.0010) [2023-10-08 09:16:21,283][53852] Updated weights for policy 0, policy_version 36860 (0.0007) [2023-10-08 09:16:21,291][53885] Updated weights for policy 1, policy_version 36662 (0.0007) [2023-10-08 09:16:21,659][53885] Updated weights for policy 1, policy_version 36672 (0.0008) [2023-10-08 09:16:22,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 75300864. Throughput: 0: 1825.7, 1: 1809.1. Samples: 18829380. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 09:16:22,016][52710] Avg episode reward: [(0, '30.310'), (1, '29.290')] [2023-10-08 09:16:25,064][53852] Updated weights for policy 0, policy_version 36870 (0.0008) [2023-10-08 09:16:25,342][53885] Updated weights for policy 1, policy_version 36682 (0.0008) [2023-10-08 09:16:25,428][53852] Updated weights for policy 0, policy_version 36880 (0.0008) [2023-10-08 09:16:25,711][53885] Updated weights for policy 1, policy_version 36692 (0.0008) [2023-10-08 09:16:25,792][53852] Updated weights for policy 0, policy_version 36890 (0.0007) [2023-10-08 09:16:26,075][53885] Updated weights for policy 1, policy_version 36702 (0.0007) [2023-10-08 09:16:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 75366400. Throughput: 0: 1836.3, 1: 1814.3. Samples: 18842184. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) [2023-10-08 09:16:27,016][52710] Avg episode reward: [(0, '30.830'), (1, '31.440')] [2023-10-08 09:16:29,472][53852] Updated weights for policy 0, policy_version 36900 (0.0007) [2023-10-08 09:16:29,784][53885] Updated weights for policy 1, policy_version 36712 (0.0007) [2023-10-08 09:16:29,844][53852] Updated weights for policy 0, policy_version 36910 (0.0007) [2023-10-08 09:16:30,153][53885] Updated weights for policy 1, policy_version 36722 (0.0008) [2023-10-08 09:16:30,208][53852] Updated weights for policy 0, policy_version 36920 (0.0008) [2023-10-08 09:16:30,513][53885] Updated weights for policy 1, policy_version 36732 (0.0008) [2023-10-08 09:16:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75431936. Throughput: 0: 1820.1, 1: 1824.6. Samples: 18862244. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:16:32,016][52710] Avg episode reward: [(0, '30.810'), (1, '30.760')] [2023-10-08 09:16:33,835][53852] Updated weights for policy 0, policy_version 36930 (0.0008) [2023-10-08 09:16:34,196][53885] Updated weights for policy 1, policy_version 36742 (0.0008) [2023-10-08 09:16:34,205][53852] Updated weights for policy 0, policy_version 36940 (0.0009) [2023-10-08 09:16:34,556][53885] Updated weights for policy 1, policy_version 36752 (0.0008) [2023-10-08 09:16:34,569][53852] Updated weights for policy 0, policy_version 36950 (0.0008) [2023-10-08 09:16:34,928][53885] Updated weights for policy 1, policy_version 36762 (0.0007) [2023-10-08 09:16:34,941][53852] Updated weights for policy 0, policy_version 36960 (0.0008) [2023-10-08 09:16:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 75497472. Throughput: 0: 1841.6, 1: 1831.0. Samples: 18885274. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:16:37,016][52710] Avg episode reward: [(0, '29.280'), (1, '31.440')] [2023-10-08 09:16:38,605][53852] Updated weights for policy 0, policy_version 36970 (0.0008) [2023-10-08 09:16:38,713][53885] Updated weights for policy 1, policy_version 36772 (0.0009) [2023-10-08 09:16:38,968][53852] Updated weights for policy 0, policy_version 36980 (0.0009) [2023-10-08 09:16:39,098][53885] Updated weights for policy 1, policy_version 36782 (0.0007) [2023-10-08 09:16:39,345][53852] Updated weights for policy 0, policy_version 36990 (0.0007) [2023-10-08 09:16:39,467][53885] Updated weights for policy 1, policy_version 36792 (0.0007) [2023-10-08 09:16:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 75563008. Throughput: 0: 1824.8, 1: 1820.6. Samples: 18895078. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:16:42,016][52710] Avg episode reward: [(0, '31.400'), (1, '33.030')] [2023-10-08 09:16:43,110][53852] Updated weights for policy 0, policy_version 37000 (0.0009) [2023-10-08 09:16:43,208][53885] Updated weights for policy 1, policy_version 36802 (0.0008) [2023-10-08 09:16:43,475][53852] Updated weights for policy 0, policy_version 37010 (0.0009) [2023-10-08 09:16:43,578][53885] Updated weights for policy 1, policy_version 36812 (0.0010) [2023-10-08 09:16:43,840][53852] Updated weights for policy 0, policy_version 37020 (0.0007) [2023-10-08 09:16:43,953][53885] Updated weights for policy 1, policy_version 36822 (0.0008) [2023-10-08 09:16:44,321][53885] Updated weights for policy 1, policy_version 36832 (0.0010) [2023-10-08 09:16:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 75628544. Throughput: 0: 1829.9, 1: 1825.7. Samples: 18917650. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:16:47,016][52710] Avg episode reward: [(0, '28.900'), (1, '30.420')] [2023-10-08 09:16:47,439][53852] Updated weights for policy 0, policy_version 37030 (0.0010) [2023-10-08 09:16:47,805][53852] Updated weights for policy 0, policy_version 37040 (0.0010) [2023-10-08 09:16:47,971][53885] Updated weights for policy 1, policy_version 36842 (0.0007) [2023-10-08 09:16:48,172][53852] Updated weights for policy 0, policy_version 37050 (0.0009) [2023-10-08 09:16:48,341][53885] Updated weights for policy 1, policy_version 36852 (0.0008) [2023-10-08 09:16:48,701][53885] Updated weights for policy 1, policy_version 36862 (0.0008) [2023-10-08 09:16:51,838][53852] Updated weights for policy 0, policy_version 37060 (0.0007) [2023-10-08 09:16:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75694080. Throughput: 0: 1833.1, 1: 1823.3. Samples: 18940960. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:16:52,016][52710] Avg episode reward: [(0, '29.770'), (1, '34.240')] [2023-10-08 09:16:52,207][53852] Updated weights for policy 0, policy_version 37070 (0.0008) [2023-10-08 09:16:52,358][53885] Updated weights for policy 1, policy_version 36872 (0.0007) [2023-10-08 09:16:52,573][53852] Updated weights for policy 0, policy_version 37080 (0.0010) [2023-10-08 09:16:52,737][53885] Updated weights for policy 1, policy_version 36882 (0.0008) [2023-10-08 09:16:53,100][53885] Updated weights for policy 1, policy_version 36892 (0.0009) [2023-10-08 09:16:56,216][53852] Updated weights for policy 0, policy_version 37090 (0.0008) [2023-10-08 09:16:56,590][53852] Updated weights for policy 0, policy_version 37100 (0.0007) [2023-10-08 09:16:56,786][53885] Updated weights for policy 1, policy_version 36902 (0.0007) [2023-10-08 09:16:56,962][53852] Updated weights for policy 0, policy_version 37110 (0.0007) [2023-10-08 09:16:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75759616. Throughput: 0: 1829.4, 1: 1825.1. Samples: 18950874. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:16:57,016][52710] Avg episode reward: [(0, '30.120'), (1, '30.850')] [2023-10-08 09:16:57,149][53885] Updated weights for policy 1, policy_version 36912 (0.0008) [2023-10-08 09:16:57,326][53852] Updated weights for policy 0, policy_version 37120 (0.0008) [2023-10-08 09:16:57,503][53885] Updated weights for policy 1, policy_version 36922 (0.0007) [2023-10-08 09:17:00,981][53852] Updated weights for policy 0, policy_version 37130 (0.0008) [2023-10-08 09:17:01,290][53885] Updated weights for policy 1, policy_version 36932 (0.0007) [2023-10-08 09:17:01,356][53852] Updated weights for policy 0, policy_version 37140 (0.0007) [2023-10-08 09:17:01,649][53885] Updated weights for policy 1, policy_version 36942 (0.0007) [2023-10-08 09:17:01,710][53852] Updated weights for policy 0, policy_version 37150 (0.0008) [2023-10-08 09:17:02,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 75857920. Throughput: 0: 1834.4, 1: 1830.1. Samples: 18973904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:17:02,016][52710] Avg episode reward: [(0, '31.350'), (1, '30.450')] [2023-10-08 09:17:02,030][53885] Updated weights for policy 1, policy_version 36952 (0.0008) [2023-10-08 09:17:05,381][53852] Updated weights for policy 0, policy_version 37160 (0.0009) [2023-10-08 09:17:05,662][53885] Updated weights for policy 1, policy_version 36962 (0.0008) [2023-10-08 09:17:05,751][53852] Updated weights for policy 0, policy_version 37170 (0.0008) [2023-10-08 09:17:06,022][53885] Updated weights for policy 1, policy_version 36972 (0.0008) [2023-10-08 09:17:06,123][53852] Updated weights for policy 0, policy_version 37180 (0.0007) [2023-10-08 09:17:06,391][53885] Updated weights for policy 1, policy_version 36982 (0.0010) [2023-10-08 09:17:06,764][53885] Updated weights for policy 1, policy_version 36992 (0.0011) [2023-10-08 09:17:07,015][52710] Fps is (10 sec: 19660.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 75956224. Throughput: 0: 1834.5, 1: 1829.9. Samples: 18994276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:17:07,016][52710] Avg episode reward: [(0, '30.540'), (1, '29.730')] [2023-10-08 09:17:09,592][53852] Updated weights for policy 0, policy_version 37190 (0.0009) [2023-10-08 09:17:09,966][53852] Updated weights for policy 0, policy_version 37200 (0.0008) [2023-10-08 09:17:10,327][53885] Updated weights for policy 1, policy_version 37002 (0.0008) [2023-10-08 09:17:10,333][53852] Updated weights for policy 0, policy_version 37210 (0.0008) [2023-10-08 09:17:10,698][53885] Updated weights for policy 1, policy_version 37012 (0.0008) [2023-10-08 09:17:11,063][53885] Updated weights for policy 1, policy_version 37022 (0.0009) [2023-10-08 09:17:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 76021760. Throughput: 0: 1836.0, 1: 1825.0. Samples: 19006930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:17:12,016][52710] Avg episode reward: [(0, '32.050'), (1, '29.580')] [2023-10-08 09:17:14,113][53852] Updated weights for policy 0, policy_version 37220 (0.0007) [2023-10-08 09:17:14,490][53852] Updated weights for policy 0, policy_version 37230 (0.0007) [2023-10-08 09:17:14,696][53885] Updated weights for policy 1, policy_version 37032 (0.0008) [2023-10-08 09:17:14,855][53852] Updated weights for policy 0, policy_version 37240 (0.0007) [2023-10-08 09:17:15,055][53885] Updated weights for policy 1, policy_version 37042 (0.0009) [2023-10-08 09:17:15,436][53885] Updated weights for policy 1, policy_version 37052 (0.0010) [2023-10-08 09:17:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76087296. Throughput: 0: 1841.3, 1: 1820.5. Samples: 19027026. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:17:17,016][52710] Avg episode reward: [(0, '32.860'), (1, '29.520')] [2023-10-08 09:17:18,621][53852] Updated weights for policy 0, policy_version 37250 (0.0009) [2023-10-08 09:17:18,982][53852] Updated weights for policy 0, policy_version 37260 (0.0007) [2023-10-08 09:17:19,050][53885] Updated weights for policy 1, policy_version 37062 (0.0008) [2023-10-08 09:17:19,356][53852] Updated weights for policy 0, policy_version 37270 (0.0009) [2023-10-08 09:17:19,413][53885] Updated weights for policy 1, policy_version 37072 (0.0009) [2023-10-08 09:17:19,729][53852] Updated weights for policy 0, policy_version 37280 (0.0008) [2023-10-08 09:17:19,781][53885] Updated weights for policy 1, policy_version 37082 (0.0009) [2023-10-08 09:17:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76152832. Throughput: 0: 1830.6, 1: 1821.3. Samples: 19049612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:17:22,016][52710] Avg episode reward: [(0, '31.660'), (1, '28.680')] [2023-10-08 09:17:23,380][53852] Updated weights for policy 0, policy_version 37290 (0.0011) [2023-10-08 09:17:23,601][53885] Updated weights for policy 1, policy_version 37092 (0.0007) [2023-10-08 09:17:23,750][53852] Updated weights for policy 0, policy_version 37300 (0.0007) [2023-10-08 09:17:23,966][53885] Updated weights for policy 1, policy_version 37102 (0.0008) [2023-10-08 09:17:24,122][53852] Updated weights for policy 0, policy_version 37310 (0.0008) [2023-10-08 09:17:24,334][53885] Updated weights for policy 1, policy_version 37112 (0.0010) [2023-10-08 09:17:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76218368. Throughput: 0: 1837.7, 1: 1819.7. Samples: 19059658. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 09:17:27,015][52710] Avg episode reward: [(0, '30.080'), (1, '32.500')] [2023-10-08 09:17:27,697][53852] Updated weights for policy 0, policy_version 37320 (0.0008) [2023-10-08 09:17:28,073][53852] Updated weights for policy 0, policy_version 37330 (0.0009) [2023-10-08 09:17:28,170][53885] Updated weights for policy 1, policy_version 37122 (0.0010) [2023-10-08 09:17:28,432][53852] Updated weights for policy 0, policy_version 37340 (0.0008) [2023-10-08 09:17:28,538][53885] Updated weights for policy 1, policy_version 37132 (0.0009) [2023-10-08 09:17:28,911][53885] Updated weights for policy 1, policy_version 37142 (0.0008) [2023-10-08 09:17:29,267][53885] Updated weights for policy 1, policy_version 37152 (0.0009) [2023-10-08 09:17:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 76283904. Throughput: 0: 1845.1, 1: 1816.9. Samples: 19082440. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 09:17:32,016][52710] Avg episode reward: [(0, '31.990'), (1, '30.790')] [2023-10-08 09:17:32,069][53852] Updated weights for policy 0, policy_version 37350 (0.0007) [2023-10-08 09:17:32,445][53852] Updated weights for policy 0, policy_version 37360 (0.0008) [2023-10-08 09:17:32,816][53852] Updated weights for policy 0, policy_version 37370 (0.0007) [2023-10-08 09:17:33,068][53885] Updated weights for policy 1, policy_version 37162 (0.0008) [2023-10-08 09:17:33,437][53885] Updated weights for policy 1, policy_version 37172 (0.0008) [2023-10-08 09:17:33,809][53885] Updated weights for policy 1, policy_version 37182 (0.0007) [2023-10-08 09:17:36,450][53852] Updated weights for policy 0, policy_version 37380 (0.0007) [2023-10-08 09:17:36,849][53852] Updated weights for policy 0, policy_version 37390 (0.0007) [2023-10-08 09:17:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 76349440. Throughput: 0: 1833.7, 1: 1815.2. Samples: 19105160. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 09:17:37,016][52710] Avg episode reward: [(0, '30.990'), (1, '31.720')] [2023-10-08 09:17:37,234][53852] Updated weights for policy 0, policy_version 37400 (0.0008) [2023-10-08 09:17:37,598][53885] Updated weights for policy 1, policy_version 37192 (0.0007) [2023-10-08 09:17:37,959][53885] Updated weights for policy 1, policy_version 37202 (0.0009) [2023-10-08 09:17:38,332][53885] Updated weights for policy 1, policy_version 37212 (0.0009) [2023-10-08 09:17:40,800][53852] Updated weights for policy 0, policy_version 37410 (0.0008) [2023-10-08 09:17:41,170][53852] Updated weights for policy 0, policy_version 37420 (0.0007) [2023-10-08 09:17:41,536][53852] Updated weights for policy 0, policy_version 37430 (0.0007) [2023-10-08 09:17:41,863][53885] Updated weights for policy 1, policy_version 37222 (0.0008) [2023-10-08 09:17:41,901][53852] Updated weights for policy 0, policy_version 37440 (0.0007) [2023-10-08 09:17:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76447744. Throughput: 0: 1840.7, 1: 1815.9. Samples: 19115420. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 09:17:42,015][52710] Avg episode reward: [(0, '30.920'), (1, '30.690')] [2023-10-08 09:17:42,230][53885] Updated weights for policy 1, policy_version 37232 (0.0009) [2023-10-08 09:17:42,605][53885] Updated weights for policy 1, policy_version 37242 (0.0008) [2023-10-08 09:17:45,572][53852] Updated weights for policy 0, policy_version 37450 (0.0011) [2023-10-08 09:17:45,945][53852] Updated weights for policy 0, policy_version 37460 (0.0010) [2023-10-08 09:17:46,311][53852] Updated weights for policy 0, policy_version 37470 (0.0007) [2023-10-08 09:17:46,318][53885] Updated weights for policy 1, policy_version 37252 (0.0008) [2023-10-08 09:17:46,684][53885] Updated weights for policy 1, policy_version 37262 (0.0007) [2023-10-08 09:17:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76513280. Throughput: 0: 1830.1, 1: 1812.5. Samples: 19137816. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 09:17:47,016][52710] Avg episode reward: [(0, '32.040'), (1, '30.110')] [2023-10-08 09:17:47,067][53885] Updated weights for policy 1, policy_version 37272 (0.0008) [2023-10-08 09:17:49,833][53852] Updated weights for policy 0, policy_version 37480 (0.0010) [2023-10-08 09:17:50,198][53852] Updated weights for policy 0, policy_version 37490 (0.0011) [2023-10-08 09:17:50,555][53852] Updated weights for policy 0, policy_version 37500 (0.0011) [2023-10-08 09:17:50,692][53885] Updated weights for policy 1, policy_version 37282 (0.0010) [2023-10-08 09:17:51,066][53885] Updated weights for policy 1, policy_version 37292 (0.0008) [2023-10-08 09:17:51,432][53885] Updated weights for policy 1, policy_version 37302 (0.0008) [2023-10-08 09:17:51,802][53885] Updated weights for policy 1, policy_version 37312 (0.0009) [2023-10-08 09:17:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 76611584. Throughput: 0: 1843.1, 1: 1812.1. Samples: 19158760. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 09:17:52,016][52710] Avg episode reward: [(0, '31.480'), (1, '29.170')] [2023-10-08 09:17:54,381][53852] Updated weights for policy 0, policy_version 37510 (0.0009) [2023-10-08 09:17:54,753][53852] Updated weights for policy 0, policy_version 37520 (0.0011) [2023-10-08 09:17:55,123][53852] Updated weights for policy 0, policy_version 37530 (0.0010) [2023-10-08 09:17:55,418][53885] Updated weights for policy 1, policy_version 37322 (0.0009) [2023-10-08 09:17:55,784][53885] Updated weights for policy 1, policy_version 37332 (0.0010) [2023-10-08 09:17:56,156][53885] Updated weights for policy 1, policy_version 37342 (0.0010) [2023-10-08 09:17:57,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 76677120. Throughput: 0: 1830.5, 1: 1813.5. Samples: 19170908. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 09:17:57,016][52710] Avg episode reward: [(0, '30.100'), (1, '27.650')] [2023-10-08 09:17:58,769][53852] Updated weights for policy 0, policy_version 37540 (0.0008) [2023-10-08 09:17:59,149][53852] Updated weights for policy 0, policy_version 37550 (0.0008) [2023-10-08 09:17:59,518][53852] Updated weights for policy 0, policy_version 37560 (0.0007) [2023-10-08 09:17:59,793][53885] Updated weights for policy 1, policy_version 37352 (0.0009) [2023-10-08 09:18:00,162][53885] Updated weights for policy 1, policy_version 37362 (0.0009) [2023-10-08 09:18:00,531][53885] Updated weights for policy 1, policy_version 37372 (0.0008) [2023-10-08 09:18:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76742656. Throughput: 0: 1838.3, 1: 1816.7. Samples: 19191500. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 09:18:02,016][52710] Avg episode reward: [(0, '28.370'), (1, '28.340')] [2023-10-08 09:18:03,128][53852] Updated weights for policy 0, policy_version 37570 (0.0008) [2023-10-08 09:18:03,492][53852] Updated weights for policy 0, policy_version 37580 (0.0007) [2023-10-08 09:18:03,858][53852] Updated weights for policy 0, policy_version 37590 (0.0009) [2023-10-08 09:18:04,167][53885] Updated weights for policy 1, policy_version 37382 (0.0009) [2023-10-08 09:18:04,222][53852] Updated weights for policy 0, policy_version 37600 (0.0008) [2023-10-08 09:18:04,526][53885] Updated weights for policy 1, policy_version 37392 (0.0009) [2023-10-08 09:18:04,887][53885] Updated weights for policy 1, policy_version 37402 (0.0007) [2023-10-08 09:18:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76808192. Throughput: 0: 1851.6, 1: 1814.5. Samples: 19214586. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 09:18:07,016][52710] Avg episode reward: [(0, '31.200'), (1, '24.490')] [2023-10-08 09:18:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000037408_38305792.pth... [2023-10-08 09:18:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000037600_38502400.pth... [2023-10-08 09:18:07,055][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000035712_36569088.pth [2023-10-08 09:18:07,062][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000035904_36765696.pth [2023-10-08 09:18:07,915][53852] Updated weights for policy 0, policy_version 37610 (0.0011) [2023-10-08 09:18:08,281][53852] Updated weights for policy 0, policy_version 37620 (0.0008) [2023-10-08 09:18:08,650][53852] Updated weights for policy 0, policy_version 37630 (0.0007) [2023-10-08 09:18:08,744][53885] Updated weights for policy 1, policy_version 37412 (0.0007) [2023-10-08 09:18:09,110][53885] Updated weights for policy 1, policy_version 37422 (0.0007) [2023-10-08 09:18:09,479][53885] Updated weights for policy 1, policy_version 37432 (0.0008) [2023-10-08 09:18:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 76873728. Throughput: 0: 1848.0, 1: 1820.4. Samples: 19224734. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 09:18:12,016][52710] Avg episode reward: [(0, '28.990'), (1, '24.770')] [2023-10-08 09:18:12,136][53852] Updated weights for policy 0, policy_version 37640 (0.0007) [2023-10-08 09:18:12,506][53852] Updated weights for policy 0, policy_version 37650 (0.0008) [2023-10-08 09:18:12,872][53852] Updated weights for policy 0, policy_version 37660 (0.0009) [2023-10-08 09:18:13,200][53885] Updated weights for policy 1, policy_version 37442 (0.0008) [2023-10-08 09:18:13,573][53885] Updated weights for policy 1, policy_version 37452 (0.0008) [2023-10-08 09:18:13,945][53885] Updated weights for policy 1, policy_version 37462 (0.0010) [2023-10-08 09:18:14,302][53885] Updated weights for policy 1, policy_version 37472 (0.0010) [2023-10-08 09:18:16,552][53852] Updated weights for policy 0, policy_version 37670 (0.0008) [2023-10-08 09:18:16,924][53852] Updated weights for policy 0, policy_version 37680 (0.0008) [2023-10-08 09:18:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76939264. Throughput: 0: 1848.8, 1: 1816.6. Samples: 19247384. Policy #0 lag: (min: 8.0, avg: 30.6, max: 40.0) [2023-10-08 09:18:17,016][52710] Avg episode reward: [(0, '29.400'), (1, '28.890')] [2023-10-08 09:18:17,284][53852] Updated weights for policy 0, policy_version 37690 (0.0007) [2023-10-08 09:18:17,917][53885] Updated weights for policy 1, policy_version 37482 (0.0009) [2023-10-08 09:18:18,293][53885] Updated weights for policy 1, policy_version 37492 (0.0008) [2023-10-08 09:18:18,653][53885] Updated weights for policy 1, policy_version 37502 (0.0007) [2023-10-08 09:18:20,822][53852] Updated weights for policy 0, policy_version 37700 (0.0008) [2023-10-08 09:18:21,194][53852] Updated weights for policy 0, policy_version 37710 (0.0009) [2023-10-08 09:18:21,570][53852] Updated weights for policy 0, policy_version 37720 (0.0008) [2023-10-08 09:18:22,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 77037568. Throughput: 0: 1827.2, 1: 1819.4. Samples: 19269256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:18:22,015][52710] Avg episode reward: [(0, '28.160'), (1, '25.460')] [2023-10-08 09:18:22,298][53885] Updated weights for policy 1, policy_version 37512 (0.0010) [2023-10-08 09:18:22,672][53885] Updated weights for policy 1, policy_version 37522 (0.0008) [2023-10-08 09:18:23,032][53885] Updated weights for policy 1, policy_version 37532 (0.0010) [2023-10-08 09:18:25,182][53852] Updated weights for policy 0, policy_version 37730 (0.0009) [2023-10-08 09:18:25,568][53852] Updated weights for policy 0, policy_version 37740 (0.0009) [2023-10-08 09:18:25,945][53852] Updated weights for policy 0, policy_version 37750 (0.0009) [2023-10-08 09:18:26,319][53852] Updated weights for policy 0, policy_version 37760 (0.0010) [2023-10-08 09:18:26,755][53885] Updated weights for policy 1, policy_version 37542 (0.0009) [2023-10-08 09:18:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 77103104. Throughput: 0: 1847.9, 1: 1816.9. Samples: 19280334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:18:27,016][52710] Avg episode reward: [(0, '31.440'), (1, '26.700')] [2023-10-08 09:18:27,134][53885] Updated weights for policy 1, policy_version 37552 (0.0009) [2023-10-08 09:18:27,505][53885] Updated weights for policy 1, policy_version 37562 (0.0010) [2023-10-08 09:18:29,994][53852] Updated weights for policy 0, policy_version 37770 (0.0008) [2023-10-08 09:18:30,363][53852] Updated weights for policy 0, policy_version 37780 (0.0010) [2023-10-08 09:18:30,739][53852] Updated weights for policy 0, policy_version 37790 (0.0008) [2023-10-08 09:18:31,221][53885] Updated weights for policy 1, policy_version 37572 (0.0011) [2023-10-08 09:18:31,583][53885] Updated weights for policy 1, policy_version 37582 (0.0007) [2023-10-08 09:18:31,960][53885] Updated weights for policy 1, policy_version 37592 (0.0009) [2023-10-08 09:18:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77168640. Throughput: 0: 1827.9, 1: 1822.0. Samples: 19302058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:18:32,016][52710] Avg episode reward: [(0, '29.390'), (1, '31.990')] [2023-10-08 09:18:34,391][53852] Updated weights for policy 0, policy_version 37800 (0.0010) [2023-10-08 09:18:34,759][53852] Updated weights for policy 0, policy_version 37810 (0.0009) [2023-10-08 09:18:35,136][53852] Updated weights for policy 0, policy_version 37820 (0.0007) [2023-10-08 09:18:35,531][53885] Updated weights for policy 1, policy_version 37602 (0.0008) [2023-10-08 09:18:35,901][53885] Updated weights for policy 1, policy_version 37612 (0.0009) [2023-10-08 09:18:36,264][53885] Updated weights for policy 1, policy_version 37622 (0.0007) [2023-10-08 09:18:36,632][53885] Updated weights for policy 1, policy_version 37632 (0.0008) [2023-10-08 09:18:37,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 77266944. Throughput: 0: 1840.1, 1: 1821.0. Samples: 19323512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:18:37,015][52710] Avg episode reward: [(0, '28.550'), (1, '30.480')] [2023-10-08 09:18:38,745][53852] Updated weights for policy 0, policy_version 37830 (0.0008) [2023-10-08 09:18:39,122][53852] Updated weights for policy 0, policy_version 37840 (0.0007) [2023-10-08 09:18:39,494][53852] Updated weights for policy 0, policy_version 37850 (0.0007) [2023-10-08 09:18:40,215][53885] Updated weights for policy 1, policy_version 37642 (0.0008) [2023-10-08 09:18:40,590][53885] Updated weights for policy 1, policy_version 37652 (0.0011) [2023-10-08 09:18:40,968][53885] Updated weights for policy 1, policy_version 37662 (0.0010) [2023-10-08 09:18:42,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77332480. Throughput: 0: 1827.7, 1: 1824.7. Samples: 19335266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:18:42,015][52710] Avg episode reward: [(0, '29.800'), (1, '31.970')] [2023-10-08 09:18:43,380][53852] Updated weights for policy 0, policy_version 37860 (0.0008) [2023-10-08 09:18:43,737][53852] Updated weights for policy 0, policy_version 37870 (0.0009) [2023-10-08 09:18:44,101][53852] Updated weights for policy 0, policy_version 37880 (0.0009) [2023-10-08 09:18:44,597][53885] Updated weights for policy 1, policy_version 37672 (0.0010) [2023-10-08 09:18:44,956][53885] Updated weights for policy 1, policy_version 37682 (0.0009) [2023-10-08 09:18:45,327][53885] Updated weights for policy 1, policy_version 37692 (0.0009) [2023-10-08 09:18:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77398016. Throughput: 0: 1842.1, 1: 1818.7. Samples: 19356238. Policy #0 lag: (min: 25.0, avg: 35.1, max: 57.0) [2023-10-08 09:18:47,016][52710] Avg episode reward: [(0, '27.420'), (1, '30.740')] [2023-10-08 09:18:47,672][53852] Updated weights for policy 0, policy_version 37890 (0.0007) [2023-10-08 09:18:48,033][53852] Updated weights for policy 0, policy_version 37900 (0.0007) [2023-10-08 09:18:48,410][53852] Updated weights for policy 0, policy_version 37910 (0.0010) [2023-10-08 09:18:48,778][53852] Updated weights for policy 0, policy_version 37920 (0.0010) [2023-10-08 09:18:48,946][53885] Updated weights for policy 1, policy_version 37702 (0.0010) [2023-10-08 09:18:49,314][53885] Updated weights for policy 1, policy_version 37712 (0.0010) [2023-10-08 09:18:49,692][53885] Updated weights for policy 1, policy_version 37722 (0.0010) [2023-10-08 09:18:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 77463552. Throughput: 0: 1835.8, 1: 1825.6. Samples: 19379352. Policy #0 lag: (min: 25.0, avg: 35.1, max: 57.0) [2023-10-08 09:18:52,016][52710] Avg episode reward: [(0, '28.570'), (1, '31.590')] [2023-10-08 09:18:52,491][53852] Updated weights for policy 0, policy_version 37930 (0.0008) [2023-10-08 09:18:52,859][53852] Updated weights for policy 0, policy_version 37940 (0.0009) [2023-10-08 09:18:53,223][53852] Updated weights for policy 0, policy_version 37950 (0.0009) [2023-10-08 09:18:53,484][53885] Updated weights for policy 1, policy_version 37732 (0.0009) [2023-10-08 09:18:53,879][53885] Updated weights for policy 1, policy_version 37742 (0.0009) [2023-10-08 09:18:54,252][53885] Updated weights for policy 1, policy_version 37752 (0.0008) [2023-10-08 09:18:56,852][53852] Updated weights for policy 0, policy_version 37960 (0.0007) [2023-10-08 09:18:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 77529088. Throughput: 0: 1837.6, 1: 1820.0. Samples: 19389328. Policy #0 lag: (min: 25.0, avg: 35.1, max: 57.0) [2023-10-08 09:18:57,015][52710] Avg episode reward: [(0, '27.190'), (1, '32.550')] [2023-10-08 09:18:57,213][53852] Updated weights for policy 0, policy_version 37970 (0.0007) [2023-10-08 09:18:57,583][53852] Updated weights for policy 0, policy_version 37980 (0.0009) [2023-10-08 09:18:57,992][53885] Updated weights for policy 1, policy_version 37762 (0.0007) [2023-10-08 09:18:58,359][53885] Updated weights for policy 1, policy_version 37772 (0.0009) [2023-10-08 09:18:58,733][53885] Updated weights for policy 1, policy_version 37782 (0.0009) [2023-10-08 09:18:59,101][53885] Updated weights for policy 1, policy_version 37792 (0.0010) [2023-10-08 09:19:01,299][53852] Updated weights for policy 0, policy_version 37990 (0.0009) [2023-10-08 09:19:01,666][53852] Updated weights for policy 0, policy_version 38000 (0.0009) [2023-10-08 09:19:02,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77594624. Throughput: 0: 1830.1, 1: 1829.2. Samples: 19412056. Policy #0 lag: (min: 25.0, avg: 35.1, max: 57.0) [2023-10-08 09:19:02,016][52710] Avg episode reward: [(0, '29.900'), (1, '30.830')] [2023-10-08 09:19:02,044][53852] Updated weights for policy 0, policy_version 38010 (0.0008) [2023-10-08 09:19:02,859][53885] Updated weights for policy 1, policy_version 37802 (0.0008) [2023-10-08 09:19:03,222][53885] Updated weights for policy 1, policy_version 37812 (0.0012) [2023-10-08 09:19:03,589][53885] Updated weights for policy 1, policy_version 37822 (0.0008) [2023-10-08 09:19:05,687][53852] Updated weights for policy 0, policy_version 38020 (0.0008) [2023-10-08 09:19:06,062][53852] Updated weights for policy 0, policy_version 38030 (0.0009) [2023-10-08 09:19:06,431][53852] Updated weights for policy 0, policy_version 38040 (0.0008) [2023-10-08 09:19:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77692928. Throughput: 0: 1827.9, 1: 1819.0. Samples: 19433364. Policy #0 lag: (min: 25.0, avg: 35.1, max: 57.0) [2023-10-08 09:19:07,016][52710] Avg episode reward: [(0, '28.740'), (1, '31.520')] [2023-10-08 09:19:07,356][53885] Updated weights for policy 1, policy_version 37832 (0.0008) [2023-10-08 09:19:07,723][53885] Updated weights for policy 1, policy_version 37842 (0.0009) [2023-10-08 09:19:08,091][53885] Updated weights for policy 1, policy_version 37852 (0.0009) [2023-10-08 09:19:10,058][53852] Updated weights for policy 0, policy_version 38050 (0.0010) [2023-10-08 09:19:10,459][53852] Updated weights for policy 0, policy_version 38060 (0.0009) [2023-10-08 09:19:10,822][53852] Updated weights for policy 0, policy_version 38070 (0.0010) [2023-10-08 09:19:11,202][53852] Updated weights for policy 0, policy_version 38080 (0.0011) [2023-10-08 09:19:11,965][53885] Updated weights for policy 1, policy_version 37862 (0.0008) [2023-10-08 09:19:12,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77758464. Throughput: 0: 1831.3, 1: 1817.2. Samples: 19444514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:19:12,015][52710] Avg episode reward: [(0, '27.630'), (1, '32.250')] [2023-10-08 09:19:12,335][53885] Updated weights for policy 1, policy_version 37872 (0.0008) [2023-10-08 09:19:12,700][53885] Updated weights for policy 1, policy_version 37882 (0.0007) [2023-10-08 09:19:14,881][53852] Updated weights for policy 0, policy_version 38090 (0.0007) [2023-10-08 09:19:15,260][53852] Updated weights for policy 0, policy_version 38100 (0.0008) [2023-10-08 09:19:15,634][53852] Updated weights for policy 0, policy_version 38110 (0.0008) [2023-10-08 09:19:16,251][53885] Updated weights for policy 1, policy_version 37892 (0.0008) [2023-10-08 09:19:16,628][53885] Updated weights for policy 1, policy_version 37902 (0.0010) [2023-10-08 09:19:16,997][53885] Updated weights for policy 1, policy_version 37912 (0.0010) [2023-10-08 09:19:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77824000. Throughput: 0: 1822.6, 1: 1818.6. Samples: 19465912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:19:17,016][52710] Avg episode reward: [(0, '25.610'), (1, '32.780')] [2023-10-08 09:19:19,368][53852] Updated weights for policy 0, policy_version 38120 (0.0009) [2023-10-08 09:19:19,742][53852] Updated weights for policy 0, policy_version 38130 (0.0008) [2023-10-08 09:19:20,116][53852] Updated weights for policy 0, policy_version 38140 (0.0007) [2023-10-08 09:19:20,585][53885] Updated weights for policy 1, policy_version 37922 (0.0008) [2023-10-08 09:19:20,953][53885] Updated weights for policy 1, policy_version 37932 (0.0011) [2023-10-08 09:19:21,324][53885] Updated weights for policy 1, policy_version 37942 (0.0011) [2023-10-08 09:19:21,691][53885] Updated weights for policy 1, policy_version 37952 (0.0008) [2023-10-08 09:19:22,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 77922304. Throughput: 0: 1821.8, 1: 1812.1. Samples: 19487040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:19:22,016][52710] Avg episode reward: [(0, '28.240'), (1, '31.580')] [2023-10-08 09:19:23,774][53852] Updated weights for policy 0, policy_version 38150 (0.0009) [2023-10-08 09:19:24,140][53852] Updated weights for policy 0, policy_version 38160 (0.0008) [2023-10-08 09:19:24,507][53852] Updated weights for policy 0, policy_version 38170 (0.0007) [2023-10-08 09:19:25,539][53885] Updated weights for policy 1, policy_version 37962 (0.0010) [2023-10-08 09:19:25,907][53885] Updated weights for policy 1, policy_version 37972 (0.0009) [2023-10-08 09:19:26,268][53885] Updated weights for policy 1, policy_version 37982 (0.0009) [2023-10-08 09:19:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77987840. Throughput: 0: 1821.6, 1: 1805.0. Samples: 19498466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:19:27,016][52710] Avg episode reward: [(0, '26.850'), (1, '28.680')] [2023-10-08 09:19:28,236][53852] Updated weights for policy 0, policy_version 38180 (0.0007) [2023-10-08 09:19:28,609][53852] Updated weights for policy 0, policy_version 38190 (0.0008) [2023-10-08 09:19:28,986][53852] Updated weights for policy 0, policy_version 38200 (0.0010) [2023-10-08 09:19:30,074][53885] Updated weights for policy 1, policy_version 37992 (0.0008) [2023-10-08 09:19:30,445][53885] Updated weights for policy 1, policy_version 38002 (0.0010) [2023-10-08 09:19:30,817][53885] Updated weights for policy 1, policy_version 38012 (0.0009) [2023-10-08 09:19:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 78053376. Throughput: 0: 1821.6, 1: 1821.7. Samples: 19520188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:19:32,016][52710] Avg episode reward: [(0, '24.610'), (1, '32.820')] [2023-10-08 09:19:32,597][53852] Updated weights for policy 0, policy_version 38210 (0.0007) [2023-10-08 09:19:32,971][53852] Updated weights for policy 0, policy_version 38220 (0.0007) [2023-10-08 09:19:33,330][53852] Updated weights for policy 0, policy_version 38230 (0.0010) [2023-10-08 09:19:33,700][53852] Updated weights for policy 0, policy_version 38240 (0.0010) [2023-10-08 09:19:34,391][53885] Updated weights for policy 1, policy_version 38022 (0.0008) [2023-10-08 09:19:34,763][53885] Updated weights for policy 1, policy_version 38032 (0.0008) [2023-10-08 09:19:35,140][53885] Updated weights for policy 1, policy_version 38042 (0.0007) [2023-10-08 09:19:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 78118912. Throughput: 0: 1816.1, 1: 1811.5. Samples: 19542598. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:19:37,016][52710] Avg episode reward: [(0, '22.650'), (1, '30.600')] [2023-10-08 09:19:37,436][53852] Updated weights for policy 0, policy_version 38250 (0.0010) [2023-10-08 09:19:37,807][53852] Updated weights for policy 0, policy_version 38260 (0.0007) [2023-10-08 09:19:38,184][53852] Updated weights for policy 0, policy_version 38270 (0.0007) [2023-10-08 09:19:38,838][53885] Updated weights for policy 1, policy_version 38052 (0.0010) [2023-10-08 09:19:39,211][53885] Updated weights for policy 1, policy_version 38062 (0.0008) [2023-10-08 09:19:39,572][53885] Updated weights for policy 1, policy_version 38072 (0.0010) [2023-10-08 09:19:41,717][53852] Updated weights for policy 0, policy_version 38280 (0.0010) [2023-10-08 09:19:42,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78184448. Throughput: 0: 1814.0, 1: 1822.5. Samples: 19552970. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 09:19:42,015][52710] Avg episode reward: [(0, '18.820'), (1, '30.120')] [2023-10-08 09:19:42,085][53852] Updated weights for policy 0, policy_version 38290 (0.0008) [2023-10-08 09:19:42,452][53852] Updated weights for policy 0, policy_version 38300 (0.0008) [2023-10-08 09:19:43,189][53885] Updated weights for policy 1, policy_version 38082 (0.0011) [2023-10-08 09:19:43,558][53885] Updated weights for policy 1, policy_version 38092 (0.0009) [2023-10-08 09:19:43,930][53885] Updated weights for policy 1, policy_version 38102 (0.0009) [2023-10-08 09:19:44,300][53885] Updated weights for policy 1, policy_version 38112 (0.0008) [2023-10-08 09:19:46,189][53852] Updated weights for policy 0, policy_version 38310 (0.0008) [2023-10-08 09:19:46,563][53852] Updated weights for policy 0, policy_version 38320 (0.0008) [2023-10-08 09:19:46,925][53852] Updated weights for policy 0, policy_version 38330 (0.0008) [2023-10-08 09:19:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78249984. Throughput: 0: 1819.3, 1: 1813.0. Samples: 19575512. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 09:19:47,016][52710] Avg episode reward: [(0, '18.750'), (1, '31.230')] [2023-10-08 09:19:48,101][53885] Updated weights for policy 1, policy_version 38122 (0.0008) [2023-10-08 09:19:48,464][53885] Updated weights for policy 1, policy_version 38132 (0.0009) [2023-10-08 09:19:48,831][53885] Updated weights for policy 1, policy_version 38142 (0.0009) [2023-10-08 09:19:50,636][53852] Updated weights for policy 0, policy_version 38340 (0.0008) [2023-10-08 09:19:51,001][53852] Updated weights for policy 0, policy_version 38350 (0.0008) [2023-10-08 09:19:51,374][53852] Updated weights for policy 0, policy_version 38360 (0.0008) [2023-10-08 09:19:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78348288. Throughput: 0: 1815.6, 1: 1824.2. Samples: 19597152. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 09:19:52,016][52710] Avg episode reward: [(0, '16.690'), (1, '32.710')] [2023-10-08 09:19:52,389][53885] Updated weights for policy 1, policy_version 38152 (0.0007) [2023-10-08 09:19:52,757][53885] Updated weights for policy 1, policy_version 38162 (0.0008) [2023-10-08 09:19:53,121][53885] Updated weights for policy 1, policy_version 38172 (0.0007) [2023-10-08 09:19:55,036][53852] Updated weights for policy 0, policy_version 38370 (0.0008) [2023-10-08 09:19:55,437][53852] Updated weights for policy 0, policy_version 38380 (0.0007) [2023-10-08 09:19:55,807][53852] Updated weights for policy 0, policy_version 38390 (0.0007) [2023-10-08 09:19:56,185][53852] Updated weights for policy 0, policy_version 38400 (0.0007) [2023-10-08 09:19:56,812][53885] Updated weights for policy 1, policy_version 38182 (0.0007) [2023-10-08 09:19:57,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 78413824. Throughput: 0: 1817.9, 1: 1824.2. Samples: 19608412. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 09:19:57,016][52710] Avg episode reward: [(0, '16.910'), (1, '30.220')] [2023-10-08 09:19:57,179][53885] Updated weights for policy 1, policy_version 38192 (0.0009) [2023-10-08 09:19:57,554][53885] Updated weights for policy 1, policy_version 38202 (0.0007) [2023-10-08 09:19:59,817][53852] Updated weights for policy 0, policy_version 38410 (0.0008) [2023-10-08 09:20:00,179][53852] Updated weights for policy 0, policy_version 38420 (0.0008) [2023-10-08 09:20:00,552][53852] Updated weights for policy 0, policy_version 38430 (0.0010) [2023-10-08 09:20:01,305][53885] Updated weights for policy 1, policy_version 38212 (0.0009) [2023-10-08 09:20:01,670][53885] Updated weights for policy 1, policy_version 38222 (0.0008) [2023-10-08 09:20:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78479360. Throughput: 0: 1822.0, 1: 1821.9. Samples: 19629888. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) [2023-10-08 09:20:02,016][52710] Avg episode reward: [(0, '17.500'), (1, '30.620')] [2023-10-08 09:20:02,035][53885] Updated weights for policy 1, policy_version 38232 (0.0010) [2023-10-08 09:20:04,096][53852] Updated weights for policy 0, policy_version 38440 (0.0008) [2023-10-08 09:20:04,456][53852] Updated weights for policy 0, policy_version 38450 (0.0008) [2023-10-08 09:20:04,833][53852] Updated weights for policy 0, policy_version 38460 (0.0007) [2023-10-08 09:20:05,819][53885] Updated weights for policy 1, policy_version 38242 (0.0010) [2023-10-08 09:20:06,190][53885] Updated weights for policy 1, policy_version 38252 (0.0007) [2023-10-08 09:20:06,544][53885] Updated weights for policy 1, policy_version 38262 (0.0007) [2023-10-08 09:20:06,913][53885] Updated weights for policy 1, policy_version 38272 (0.0010) [2023-10-08 09:20:07,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78577664. Throughput: 0: 1828.8, 1: 1830.9. Samples: 19651726. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-08 09:20:07,016][52710] Avg episode reward: [(0, '18.200'), (1, '34.600')] [2023-10-08 09:20:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000038272_39190528.pth... [2023-10-08 09:20:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000038464_39387136.pth... [2023-10-08 09:20:07,063][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000036768_37650432.pth [2023-10-08 09:20:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000036576_37453824.pth [2023-10-08 09:20:08,702][53852] Updated weights for policy 0, policy_version 38470 (0.0007) [2023-10-08 09:20:09,070][53852] Updated weights for policy 0, policy_version 38480 (0.0007) [2023-10-08 09:20:09,439][53852] Updated weights for policy 0, policy_version 38490 (0.0007) [2023-10-08 09:20:10,545][53885] Updated weights for policy 1, policy_version 38282 (0.0007) [2023-10-08 09:20:10,912][53885] Updated weights for policy 1, policy_version 38292 (0.0007) [2023-10-08 09:20:11,275][53885] Updated weights for policy 1, policy_version 38302 (0.0008) [2023-10-08 09:20:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78643200. Throughput: 0: 1822.9, 1: 1831.0. Samples: 19662892. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-08 09:20:12,016][52710] Avg episode reward: [(0, '18.990'), (1, '35.960')] [2023-10-08 09:20:12,017][53594] Saving new best policy, reward=35.960! [2023-10-08 09:20:13,066][53852] Updated weights for policy 0, policy_version 38500 (0.0007) [2023-10-08 09:20:13,440][53852] Updated weights for policy 0, policy_version 38510 (0.0009) [2023-10-08 09:20:13,813][53852] Updated weights for policy 0, policy_version 38520 (0.0009) [2023-10-08 09:20:15,041][53885] Updated weights for policy 1, policy_version 38312 (0.0009) [2023-10-08 09:20:15,407][53885] Updated weights for policy 1, policy_version 38322 (0.0008) [2023-10-08 09:20:15,775][53885] Updated weights for policy 1, policy_version 38332 (0.0008) [2023-10-08 09:20:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78708736. Throughput: 0: 1830.8, 1: 1821.3. Samples: 19684534. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-08 09:20:17,016][52710] Avg episode reward: [(0, '19.580'), (1, '30.920')] [2023-10-08 09:20:17,409][53852] Updated weights for policy 0, policy_version 38530 (0.0008) [2023-10-08 09:20:17,780][53852] Updated weights for policy 0, policy_version 38540 (0.0010) [2023-10-08 09:20:18,147][53852] Updated weights for policy 0, policy_version 38550 (0.0010) [2023-10-08 09:20:18,513][53852] Updated weights for policy 0, policy_version 38560 (0.0008) [2023-10-08 09:20:19,394][53885] Updated weights for policy 1, policy_version 38342 (0.0008) [2023-10-08 09:20:19,754][53885] Updated weights for policy 1, policy_version 38352 (0.0007) [2023-10-08 09:20:20,117][53885] Updated weights for policy 1, policy_version 38362 (0.0008) [2023-10-08 09:20:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 78774272. Throughput: 0: 1835.7, 1: 1821.9. Samples: 19707192. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-08 09:20:22,016][52710] Avg episode reward: [(0, '22.280'), (1, '30.890')] [2023-10-08 09:20:22,294][53852] Updated weights for policy 0, policy_version 38570 (0.0008) [2023-10-08 09:20:22,673][53852] Updated weights for policy 0, policy_version 38580 (0.0008) [2023-10-08 09:20:23,034][53852] Updated weights for policy 0, policy_version 38590 (0.0008) [2023-10-08 09:20:23,864][53885] Updated weights for policy 1, policy_version 38372 (0.0008) [2023-10-08 09:20:24,258][53885] Updated weights for policy 1, policy_version 38382 (0.0010) [2023-10-08 09:20:24,622][53885] Updated weights for policy 1, policy_version 38392 (0.0009) [2023-10-08 09:20:26,713][53852] Updated weights for policy 0, policy_version 38600 (0.0009) [2023-10-08 09:20:27,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 78839808. Throughput: 0: 1835.8, 1: 1823.9. Samples: 19717656. Policy #0 lag: (min: 31.0, avg: 31.7, max: 49.0) [2023-10-08 09:20:27,016][52710] Avg episode reward: [(0, '23.640'), (1, '34.410')] [2023-10-08 09:20:27,080][53852] Updated weights for policy 0, policy_version 38610 (0.0009) [2023-10-08 09:20:27,448][53852] Updated weights for policy 0, policy_version 38620 (0.0007) [2023-10-08 09:20:28,199][53885] Updated weights for policy 1, policy_version 38402 (0.0009) [2023-10-08 09:20:28,554][53885] Updated weights for policy 1, policy_version 38412 (0.0010) [2023-10-08 09:20:28,923][53885] Updated weights for policy 1, policy_version 38422 (0.0007) [2023-10-08 09:20:29,285][53885] Updated weights for policy 1, policy_version 38432 (0.0007) [2023-10-08 09:20:30,877][53852] Updated weights for policy 0, policy_version 38630 (0.0009) [2023-10-08 09:20:31,246][53852] Updated weights for policy 0, policy_version 38640 (0.0008) [2023-10-08 09:20:31,617][53852] Updated weights for policy 0, policy_version 38650 (0.0009) [2023-10-08 09:20:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78938112. Throughput: 0: 1835.9, 1: 1829.1. Samples: 19740440. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) [2023-10-08 09:20:32,016][52710] Avg episode reward: [(0, '24.500'), (1, '32.850')] [2023-10-08 09:20:32,757][53885] Updated weights for policy 1, policy_version 38442 (0.0009) [2023-10-08 09:20:33,132][53885] Updated weights for policy 1, policy_version 38452 (0.0007) [2023-10-08 09:20:33,490][53885] Updated weights for policy 1, policy_version 38462 (0.0010) [2023-10-08 09:20:35,234][53852] Updated weights for policy 0, policy_version 38660 (0.0008) [2023-10-08 09:20:35,597][53852] Updated weights for policy 0, policy_version 38670 (0.0008) [2023-10-08 09:20:35,973][53852] Updated weights for policy 0, policy_version 38680 (0.0007) [2023-10-08 09:20:37,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79003648. Throughput: 0: 1836.7, 1: 1828.7. Samples: 19762096. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) [2023-10-08 09:20:37,016][52710] Avg episode reward: [(0, '26.210'), (1, '32.060')] [2023-10-08 09:20:37,184][53885] Updated weights for policy 1, policy_version 38472 (0.0008) [2023-10-08 09:20:37,549][53885] Updated weights for policy 1, policy_version 38482 (0.0008) [2023-10-08 09:20:37,926][53885] Updated weights for policy 1, policy_version 38492 (0.0007) [2023-10-08 09:20:39,610][53852] Updated weights for policy 0, policy_version 38690 (0.0009) [2023-10-08 09:20:40,007][53852] Updated weights for policy 0, policy_version 38700 (0.0010) [2023-10-08 09:20:40,380][53852] Updated weights for policy 0, policy_version 38710 (0.0009) [2023-10-08 09:20:40,742][53852] Updated weights for policy 0, policy_version 38720 (0.0007) [2023-10-08 09:20:41,606][53885] Updated weights for policy 1, policy_version 38502 (0.0008) [2023-10-08 09:20:41,963][53885] Updated weights for policy 1, policy_version 38512 (0.0007) [2023-10-08 09:20:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79069184. Throughput: 0: 1843.6, 1: 1830.4. Samples: 19773742. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) [2023-10-08 09:20:42,016][52710] Avg episode reward: [(0, '23.430'), (1, '34.410')] [2023-10-08 09:20:42,340][53885] Updated weights for policy 1, policy_version 38522 (0.0007) [2023-10-08 09:20:44,345][53852] Updated weights for policy 0, policy_version 38730 (0.0008) [2023-10-08 09:20:44,709][53852] Updated weights for policy 0, policy_version 38740 (0.0010) [2023-10-08 09:20:45,082][53852] Updated weights for policy 0, policy_version 38750 (0.0012) [2023-10-08 09:20:45,961][53885] Updated weights for policy 1, policy_version 38532 (0.0009) [2023-10-08 09:20:46,340][53885] Updated weights for policy 1, policy_version 38542 (0.0009) [2023-10-08 09:20:46,705][53885] Updated weights for policy 1, policy_version 38552 (0.0009) [2023-10-08 09:20:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 79167488. Throughput: 0: 1839.7, 1: 1832.5. Samples: 19795140. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) [2023-10-08 09:20:47,016][52710] Avg episode reward: [(0, '22.680'), (1, '32.590')] [2023-10-08 09:20:48,657][53852] Updated weights for policy 0, policy_version 38760 (0.0010) [2023-10-08 09:20:49,036][53852] Updated weights for policy 0, policy_version 38770 (0.0010) [2023-10-08 09:20:49,401][53852] Updated weights for policy 0, policy_version 38780 (0.0010) [2023-10-08 09:20:50,523][53885] Updated weights for policy 1, policy_version 38562 (0.0008) [2023-10-08 09:20:50,890][53885] Updated weights for policy 1, policy_version 38572 (0.0007) [2023-10-08 09:20:51,274][53885] Updated weights for policy 1, policy_version 38582 (0.0008) [2023-10-08 09:20:51,638][53885] Updated weights for policy 1, policy_version 38592 (0.0008) [2023-10-08 09:20:52,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79233024. Throughput: 0: 1845.6, 1: 1825.8. Samples: 19816936. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) [2023-10-08 09:20:52,016][52710] Avg episode reward: [(0, '21.460'), (1, '32.010')] [2023-10-08 09:20:53,060][53852] Updated weights for policy 0, policy_version 38790 (0.0007) [2023-10-08 09:20:53,423][53852] Updated weights for policy 0, policy_version 38800 (0.0007) [2023-10-08 09:20:53,800][53852] Updated weights for policy 0, policy_version 38810 (0.0008) [2023-10-08 09:20:55,313][53885] Updated weights for policy 1, policy_version 38602 (0.0008) [2023-10-08 09:20:55,684][53885] Updated weights for policy 1, policy_version 38612 (0.0008) [2023-10-08 09:20:56,054][53885] Updated weights for policy 1, policy_version 38622 (0.0008) [2023-10-08 09:20:57,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79298560. Throughput: 0: 1842.7, 1: 1833.0. Samples: 19828302. Policy #0 lag: (min: 1.0, avg: 18.9, max: 33.0) [2023-10-08 09:20:57,016][52710] Avg episode reward: [(0, '21.020'), (1, '32.250')] [2023-10-08 09:20:57,428][53852] Updated weights for policy 0, policy_version 38820 (0.0008) [2023-10-08 09:20:57,817][53852] Updated weights for policy 0, policy_version 38830 (0.0008) [2023-10-08 09:20:58,196][53852] Updated weights for policy 0, policy_version 38840 (0.0007) [2023-10-08 09:20:59,681][53885] Updated weights for policy 1, policy_version 38632 (0.0008) [2023-10-08 09:21:00,050][53885] Updated weights for policy 1, policy_version 38642 (0.0007) [2023-10-08 09:21:00,418][53885] Updated weights for policy 1, policy_version 38652 (0.0008) [2023-10-08 09:21:01,755][53852] Updated weights for policy 0, policy_version 38850 (0.0008) [2023-10-08 09:21:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79364096. Throughput: 0: 1849.5, 1: 1827.0. Samples: 19849976. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 09:21:02,016][52710] Avg episode reward: [(0, '23.140'), (1, '29.920')] [2023-10-08 09:21:02,132][53852] Updated weights for policy 0, policy_version 38860 (0.0009) [2023-10-08 09:21:02,505][53852] Updated weights for policy 0, policy_version 38870 (0.0007) [2023-10-08 09:21:02,874][53852] Updated weights for policy 0, policy_version 38880 (0.0007) [2023-10-08 09:21:04,143][53885] Updated weights for policy 1, policy_version 38662 (0.0009) [2023-10-08 09:21:04,500][53885] Updated weights for policy 1, policy_version 38672 (0.0009) [2023-10-08 09:21:04,868][53885] Updated weights for policy 1, policy_version 38682 (0.0008) [2023-10-08 09:21:06,651][53852] Updated weights for policy 0, policy_version 38890 (0.0010) [2023-10-08 09:21:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 79429632. Throughput: 0: 1835.4, 1: 1833.3. Samples: 19872284. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 09:21:07,016][52710] Avg episode reward: [(0, '24.900'), (1, '31.510')] [2023-10-08 09:21:07,030][53852] Updated weights for policy 0, policy_version 38900 (0.0010) [2023-10-08 09:21:07,403][53852] Updated weights for policy 0, policy_version 38910 (0.0010) [2023-10-08 09:21:08,592][53885] Updated weights for policy 1, policy_version 38692 (0.0007) [2023-10-08 09:21:08,983][53885] Updated weights for policy 1, policy_version 38702 (0.0009) [2023-10-08 09:21:09,343][53885] Updated weights for policy 1, policy_version 38712 (0.0007) [2023-10-08 09:21:11,057][53852] Updated weights for policy 0, policy_version 38920 (0.0010) [2023-10-08 09:21:11,430][53852] Updated weights for policy 0, policy_version 38930 (0.0007) [2023-10-08 09:21:11,794][53852] Updated weights for policy 0, policy_version 38940 (0.0007) [2023-10-08 09:21:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79527936. Throughput: 0: 1842.1, 1: 1822.9. Samples: 19882582. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 09:21:12,016][52710] Avg episode reward: [(0, '23.760'), (1, '28.070')] [2023-10-08 09:21:12,981][53885] Updated weights for policy 1, policy_version 38722 (0.0007) [2023-10-08 09:21:13,350][53885] Updated weights for policy 1, policy_version 38732 (0.0008) [2023-10-08 09:21:13,716][53885] Updated weights for policy 1, policy_version 38742 (0.0008) [2023-10-08 09:21:14,086][53885] Updated weights for policy 1, policy_version 38752 (0.0007) [2023-10-08 09:21:15,512][53852] Updated weights for policy 0, policy_version 38950 (0.0009) [2023-10-08 09:21:15,893][53852] Updated weights for policy 0, policy_version 38960 (0.0007) [2023-10-08 09:21:16,256][53852] Updated weights for policy 0, policy_version 38970 (0.0008) [2023-10-08 09:21:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79593472. Throughput: 0: 1830.8, 1: 1828.4. Samples: 19905104. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 09:21:17,016][52710] Avg episode reward: [(0, '24.640'), (1, '29.980')] [2023-10-08 09:21:17,887][53885] Updated weights for policy 1, policy_version 38762 (0.0008) [2023-10-08 09:21:18,253][53885] Updated weights for policy 1, policy_version 38772 (0.0008) [2023-10-08 09:21:18,624][53885] Updated weights for policy 1, policy_version 38782 (0.0007) [2023-10-08 09:21:19,878][53852] Updated weights for policy 0, policy_version 38980 (0.0008) [2023-10-08 09:21:20,247][53852] Updated weights for policy 0, policy_version 38990 (0.0008) [2023-10-08 09:21:20,619][53852] Updated weights for policy 0, policy_version 39000 (0.0008) [2023-10-08 09:21:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79659008. Throughput: 0: 1836.5, 1: 1826.3. Samples: 19926920. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) [2023-10-08 09:21:22,015][52710] Avg episode reward: [(0, '28.000'), (1, '28.970')] [2023-10-08 09:21:22,215][53885] Updated weights for policy 1, policy_version 38792 (0.0011) [2023-10-08 09:21:22,578][53885] Updated weights for policy 1, policy_version 38802 (0.0010) [2023-10-08 09:21:22,946][53885] Updated weights for policy 1, policy_version 38812 (0.0009) [2023-10-08 09:21:24,288][53852] Updated weights for policy 0, policy_version 39010 (0.0010) [2023-10-08 09:21:24,659][53852] Updated weights for policy 0, policy_version 39020 (0.0010) [2023-10-08 09:21:25,031][53852] Updated weights for policy 0, policy_version 39030 (0.0008) [2023-10-08 09:21:25,403][53852] Updated weights for policy 0, policy_version 39040 (0.0007) [2023-10-08 09:21:26,742][53885] Updated weights for policy 1, policy_version 38822 (0.0008) [2023-10-08 09:21:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 79724544. Throughput: 0: 1824.1, 1: 1824.5. Samples: 19937930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:21:27,016][52710] Avg episode reward: [(0, '30.020'), (1, '30.260')] [2023-10-08 09:21:27,112][53885] Updated weights for policy 1, policy_version 38832 (0.0008) [2023-10-08 09:21:27,486][53885] Updated weights for policy 1, policy_version 38842 (0.0009) [2023-10-08 09:21:29,016][53852] Updated weights for policy 0, policy_version 39050 (0.0010) [2023-10-08 09:21:29,386][53852] Updated weights for policy 0, policy_version 39060 (0.0011) [2023-10-08 09:21:29,755][53852] Updated weights for policy 0, policy_version 39070 (0.0009) [2023-10-08 09:21:31,158][53885] Updated weights for policy 1, policy_version 38852 (0.0010) [2023-10-08 09:21:31,519][53885] Updated weights for policy 1, policy_version 38862 (0.0007) [2023-10-08 09:21:31,881][53885] Updated weights for policy 1, policy_version 38872 (0.0010) [2023-10-08 09:21:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79790080. Throughput: 0: 1836.5, 1: 1825.2. Samples: 19959916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:21:32,016][52710] Avg episode reward: [(0, '28.900'), (1, '30.890')] [2023-10-08 09:21:33,483][53852] Updated weights for policy 0, policy_version 39080 (0.0008) [2023-10-08 09:21:33,847][53852] Updated weights for policy 0, policy_version 39090 (0.0009) [2023-10-08 09:21:34,219][53852] Updated weights for policy 0, policy_version 39100 (0.0008) [2023-10-08 09:21:35,386][53885] Updated weights for policy 1, policy_version 38882 (0.0009) [2023-10-08 09:21:35,758][53885] Updated weights for policy 1, policy_version 38892 (0.0008) [2023-10-08 09:21:36,131][53885] Updated weights for policy 1, policy_version 38902 (0.0008) [2023-10-08 09:21:36,488][53885] Updated weights for policy 1, policy_version 38912 (0.0010) [2023-10-08 09:21:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79888384. Throughput: 0: 1835.7, 1: 1824.4. Samples: 19981640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:21:37,016][52710] Avg episode reward: [(0, '29.400'), (1, '31.960')] [2023-10-08 09:21:37,897][53852] Updated weights for policy 0, policy_version 39110 (0.0008) [2023-10-08 09:21:38,271][53852] Updated weights for policy 0, policy_version 39120 (0.0007) [2023-10-08 09:21:38,637][53852] Updated weights for policy 0, policy_version 39130 (0.0008) [2023-10-08 09:21:40,271][53885] Updated weights for policy 1, policy_version 38922 (0.0010) [2023-10-08 09:21:40,635][53885] Updated weights for policy 1, policy_version 38932 (0.0009) [2023-10-08 09:21:41,005][53885] Updated weights for policy 1, policy_version 38942 (0.0008) [2023-10-08 09:21:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79953920. Throughput: 0: 1836.2, 1: 1825.8. Samples: 19993092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:21:42,015][52710] Avg episode reward: [(0, '31.210'), (1, '31.500')] [2023-10-08 09:21:42,254][53852] Updated weights for policy 0, policy_version 39140 (0.0007) [2023-10-08 09:21:42,631][53852] Updated weights for policy 0, policy_version 39150 (0.0009) [2023-10-08 09:21:42,999][53852] Updated weights for policy 0, policy_version 39160 (0.0011) [2023-10-08 09:21:44,688][53885] Updated weights for policy 1, policy_version 38952 (0.0008) [2023-10-08 09:21:45,051][53885] Updated weights for policy 1, policy_version 38962 (0.0009) [2023-10-08 09:21:45,416][53885] Updated weights for policy 1, policy_version 38972 (0.0008) [2023-10-08 09:21:46,688][53852] Updated weights for policy 0, policy_version 39170 (0.0008) [2023-10-08 09:21:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 80019456. Throughput: 0: 1832.5, 1: 1821.9. Samples: 20014422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:21:47,015][52710] Avg episode reward: [(0, '29.780'), (1, '32.380')] [2023-10-08 09:21:47,067][53852] Updated weights for policy 0, policy_version 39180 (0.0008) [2023-10-08 09:21:47,441][53852] Updated weights for policy 0, policy_version 39190 (0.0010) [2023-10-08 09:21:47,807][53852] Updated weights for policy 0, policy_version 39200 (0.0007) [2023-10-08 09:21:49,056][53885] Updated weights for policy 1, policy_version 38982 (0.0009) [2023-10-08 09:21:49,430][53885] Updated weights for policy 1, policy_version 38992 (0.0009) [2023-10-08 09:21:49,793][53885] Updated weights for policy 1, policy_version 39002 (0.0007) [2023-10-08 09:21:51,441][53852] Updated weights for policy 0, policy_version 39210 (0.0007) [2023-10-08 09:21:51,816][53852] Updated weights for policy 0, policy_version 39220 (0.0008) [2023-10-08 09:21:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 80084992. Throughput: 0: 1831.6, 1: 1822.2. Samples: 20036702. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:21:52,016][52710] Avg episode reward: [(0, '28.800'), (1, '33.920')] [2023-10-08 09:21:52,197][53852] Updated weights for policy 0, policy_version 39230 (0.0010) [2023-10-08 09:21:53,597][53885] Updated weights for policy 1, policy_version 39012 (0.0009) [2023-10-08 09:21:53,990][53885] Updated weights for policy 1, policy_version 39022 (0.0010) [2023-10-08 09:21:54,355][53885] Updated weights for policy 1, policy_version 39032 (0.0007) [2023-10-08 09:21:55,760][53852] Updated weights for policy 0, policy_version 39240 (0.0010) [2023-10-08 09:21:56,137][53852] Updated weights for policy 0, policy_version 39250 (0.0011) [2023-10-08 09:21:56,508][53852] Updated weights for policy 0, policy_version 39260 (0.0009) [2023-10-08 09:21:57,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80183296. Throughput: 0: 1842.0, 1: 1822.4. Samples: 20047478. Policy #0 lag: (min: 23.0, avg: 24.6, max: 50.0) [2023-10-08 09:21:57,016][52710] Avg episode reward: [(0, '28.890'), (1, '29.580')] [2023-10-08 09:21:58,075][53885] Updated weights for policy 1, policy_version 39042 (0.0007) [2023-10-08 09:21:58,439][53885] Updated weights for policy 1, policy_version 39052 (0.0009) [2023-10-08 09:21:58,819][53885] Updated weights for policy 1, policy_version 39062 (0.0009) [2023-10-08 09:21:59,187][53885] Updated weights for policy 1, policy_version 39072 (0.0009) [2023-10-08 09:22:00,097][53852] Updated weights for policy 0, policy_version 39270 (0.0007) [2023-10-08 09:22:00,467][53852] Updated weights for policy 0, policy_version 39280 (0.0009) [2023-10-08 09:22:00,844][53852] Updated weights for policy 0, policy_version 39290 (0.0010) [2023-10-08 09:22:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80248832. Throughput: 0: 1830.5, 1: 1823.6. Samples: 20069540. Policy #0 lag: (min: 23.0, avg: 24.6, max: 50.0) [2023-10-08 09:22:02,016][52710] Avg episode reward: [(0, '29.720'), (1, '29.940')] [2023-10-08 09:22:02,767][53885] Updated weights for policy 1, policy_version 39082 (0.0007) [2023-10-08 09:22:03,131][53885] Updated weights for policy 1, policy_version 39092 (0.0007) [2023-10-08 09:22:03,500][53885] Updated weights for policy 1, policy_version 39102 (0.0008) [2023-10-08 09:22:04,510][53852] Updated weights for policy 0, policy_version 39300 (0.0009) [2023-10-08 09:22:04,887][53852] Updated weights for policy 0, policy_version 39310 (0.0010) [2023-10-08 09:22:05,259][53852] Updated weights for policy 0, policy_version 39320 (0.0009) [2023-10-08 09:22:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80314368. Throughput: 0: 1841.2, 1: 1825.3. Samples: 20091914. Policy #0 lag: (min: 23.0, avg: 24.6, max: 50.0) [2023-10-08 09:22:07,016][52710] Avg episode reward: [(0, '29.850'), (1, '29.760')] [2023-10-08 09:22:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000039328_40271872.pth... [2023-10-08 09:22:07,038][53885] Updated weights for policy 1, policy_version 39112 (0.0007) [2023-10-08 09:22:07,061][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000037600_38502400.pth [2023-10-08 09:22:07,405][53885] Updated weights for policy 1, policy_version 39122 (0.0008) [2023-10-08 09:22:07,774][53885] Updated weights for policy 1, policy_version 39132 (0.0008) [2023-10-08 09:22:07,925][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000039136_40075264.pth... [2023-10-08 09:22:07,963][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000037408_38305792.pth [2023-10-08 09:22:08,715][53852] Updated weights for policy 0, policy_version 39330 (0.0007) [2023-10-08 09:22:09,085][53852] Updated weights for policy 0, policy_version 39340 (0.0007) [2023-10-08 09:22:09,454][53852] Updated weights for policy 0, policy_version 39350 (0.0007) [2023-10-08 09:22:09,819][53852] Updated weights for policy 0, policy_version 39360 (0.0008) [2023-10-08 09:22:11,557][53885] Updated weights for policy 1, policy_version 39142 (0.0010) [2023-10-08 09:22:11,919][53885] Updated weights for policy 1, policy_version 39152 (0.0008) [2023-10-08 09:22:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80379904. Throughput: 0: 1828.8, 1: 1827.5. Samples: 20102460. Policy #0 lag: (min: 23.0, avg: 24.6, max: 50.0) [2023-10-08 09:22:12,016][52710] Avg episode reward: [(0, '31.190'), (1, '26.230')] [2023-10-08 09:22:12,295][53885] Updated weights for policy 1, policy_version 39162 (0.0007) [2023-10-08 09:22:13,521][53852] Updated weights for policy 0, policy_version 39370 (0.0009) [2023-10-08 09:22:13,887][53852] Updated weights for policy 0, policy_version 39380 (0.0008) [2023-10-08 09:22:14,260][53852] Updated weights for policy 0, policy_version 39390 (0.0010) [2023-10-08 09:22:15,953][53885] Updated weights for policy 1, policy_version 39172 (0.0008) [2023-10-08 09:22:16,323][53885] Updated weights for policy 1, policy_version 39182 (0.0008) [2023-10-08 09:22:16,684][53885] Updated weights for policy 1, policy_version 39192 (0.0009) [2023-10-08 09:22:17,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 80478208. Throughput: 0: 1834.3, 1: 1826.3. Samples: 20124640. Policy #0 lag: (min: 23.0, avg: 24.6, max: 50.0) [2023-10-08 09:22:17,016][52710] Avg episode reward: [(0, '29.660'), (1, '28.300')] [2023-10-08 09:22:18,015][53852] Updated weights for policy 0, policy_version 39400 (0.0007) [2023-10-08 09:22:18,386][53852] Updated weights for policy 0, policy_version 39410 (0.0007) [2023-10-08 09:22:18,768][53852] Updated weights for policy 0, policy_version 39420 (0.0008) [2023-10-08 09:22:20,252][53885] Updated weights for policy 1, policy_version 39202 (0.0010) [2023-10-08 09:22:20,626][53885] Updated weights for policy 1, policy_version 39212 (0.0009) [2023-10-08 09:22:20,996][53885] Updated weights for policy 1, policy_version 39222 (0.0008) [2023-10-08 09:22:21,355][53885] Updated weights for policy 1, policy_version 39232 (0.0009) [2023-10-08 09:22:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80543744. Throughput: 0: 1833.3, 1: 1826.0. Samples: 20146308. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) [2023-10-08 09:22:22,016][52710] Avg episode reward: [(0, '28.300'), (1, '32.910')] [2023-10-08 09:22:22,449][53852] Updated weights for policy 0, policy_version 39430 (0.0009) [2023-10-08 09:22:22,811][53852] Updated weights for policy 0, policy_version 39440 (0.0008) [2023-10-08 09:22:23,184][53852] Updated weights for policy 0, policy_version 39450 (0.0010) [2023-10-08 09:22:24,988][53885] Updated weights for policy 1, policy_version 39242 (0.0008) [2023-10-08 09:22:25,355][53885] Updated weights for policy 1, policy_version 39252 (0.0008) [2023-10-08 09:22:25,725][53885] Updated weights for policy 1, policy_version 39262 (0.0008) [2023-10-08 09:22:26,911][53852] Updated weights for policy 0, policy_version 39460 (0.0008) [2023-10-08 09:22:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80609280. Throughput: 0: 1834.6, 1: 1829.7. Samples: 20157986. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) [2023-10-08 09:22:27,016][52710] Avg episode reward: [(0, '29.610'), (1, '29.990')] [2023-10-08 09:22:27,293][53852] Updated weights for policy 0, policy_version 39470 (0.0009) [2023-10-08 09:22:27,662][53852] Updated weights for policy 0, policy_version 39480 (0.0011) [2023-10-08 09:22:29,192][53885] Updated weights for policy 1, policy_version 39272 (0.0007) [2023-10-08 09:22:29,567][53885] Updated weights for policy 1, policy_version 39282 (0.0007) [2023-10-08 09:22:29,932][53885] Updated weights for policy 1, policy_version 39292 (0.0009) [2023-10-08 09:22:31,249][53852] Updated weights for policy 0, policy_version 39490 (0.0009) [2023-10-08 09:22:31,621][53852] Updated weights for policy 0, policy_version 39500 (0.0008) [2023-10-08 09:22:31,986][53852] Updated weights for policy 0, policy_version 39510 (0.0007) [2023-10-08 09:22:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80674816. Throughput: 0: 1833.0, 1: 1838.2. Samples: 20179624. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) [2023-10-08 09:22:32,016][52710] Avg episode reward: [(0, '29.460'), (1, '33.530')] [2023-10-08 09:22:32,354][53852] Updated weights for policy 0, policy_version 39520 (0.0008) [2023-10-08 09:22:33,610][53885] Updated weights for policy 1, policy_version 39302 (0.0010) [2023-10-08 09:22:33,986][53885] Updated weights for policy 1, policy_version 39312 (0.0010) [2023-10-08 09:22:34,354][53885] Updated weights for policy 1, policy_version 39322 (0.0008) [2023-10-08 09:22:35,981][53852] Updated weights for policy 0, policy_version 39530 (0.0007) [2023-10-08 09:22:36,352][53852] Updated weights for policy 0, policy_version 39540 (0.0007) [2023-10-08 09:22:36,724][53852] Updated weights for policy 0, policy_version 39550 (0.0007) [2023-10-08 09:22:37,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80773120. Throughput: 0: 1820.2, 1: 1841.2. Samples: 20201466. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) [2023-10-08 09:22:37,015][52710] Avg episode reward: [(0, '30.940'), (1, '29.790')] [2023-10-08 09:22:37,884][53885] Updated weights for policy 1, policy_version 39332 (0.0007) [2023-10-08 09:22:38,251][53885] Updated weights for policy 1, policy_version 39342 (0.0008) [2023-10-08 09:22:38,617][53885] Updated weights for policy 1, policy_version 39352 (0.0007) [2023-10-08 09:22:40,212][53852] Updated weights for policy 0, policy_version 39560 (0.0008) [2023-10-08 09:22:40,569][53852] Updated weights for policy 0, policy_version 39570 (0.0008) [2023-10-08 09:22:40,937][53852] Updated weights for policy 0, policy_version 39580 (0.0007) [2023-10-08 09:22:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80838656. Throughput: 0: 1835.3, 1: 1841.8. Samples: 20212944. Policy #0 lag: (min: 31.0, avg: 31.5, max: 45.0) [2023-10-08 09:22:42,015][52710] Avg episode reward: [(0, '30.210'), (1, '29.840')] [2023-10-08 09:22:42,323][53885] Updated weights for policy 1, policy_version 39362 (0.0007) [2023-10-08 09:22:42,684][53885] Updated weights for policy 1, policy_version 39372 (0.0007) [2023-10-08 09:22:43,061][53885] Updated weights for policy 1, policy_version 39382 (0.0009) [2023-10-08 09:22:43,421][53885] Updated weights for policy 1, policy_version 39392 (0.0008) [2023-10-08 09:22:44,624][53852] Updated weights for policy 0, policy_version 39590 (0.0010) [2023-10-08 09:22:44,999][53852] Updated weights for policy 0, policy_version 39600 (0.0010) [2023-10-08 09:22:45,370][53852] Updated weights for policy 0, policy_version 39610 (0.0010) [2023-10-08 09:22:47,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 80904192. Throughput: 0: 1821.4, 1: 1846.4. Samples: 20234592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:22:47,016][52710] Avg episode reward: [(0, '31.850'), (1, '30.330')] [2023-10-08 09:22:47,117][53885] Updated weights for policy 1, policy_version 39402 (0.0009) [2023-10-08 09:22:47,501][53885] Updated weights for policy 1, policy_version 39412 (0.0011) [2023-10-08 09:22:47,866][53885] Updated weights for policy 1, policy_version 39422 (0.0009) [2023-10-08 09:22:48,984][53852] Updated weights for policy 0, policy_version 39620 (0.0008) [2023-10-08 09:22:49,353][53852] Updated weights for policy 0, policy_version 39630 (0.0008) [2023-10-08 09:22:49,725][53852] Updated weights for policy 0, policy_version 39640 (0.0007) [2023-10-08 09:22:51,497][53885] Updated weights for policy 1, policy_version 39432 (0.0010) [2023-10-08 09:22:51,855][53885] Updated weights for policy 1, policy_version 39442 (0.0009) [2023-10-08 09:22:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80969728. Throughput: 0: 1837.4, 1: 1828.2. Samples: 20256864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:22:52,016][52710] Avg episode reward: [(0, '31.870'), (1, '29.330')] [2023-10-08 09:22:52,226][53885] Updated weights for policy 1, policy_version 39452 (0.0009) [2023-10-08 09:22:53,440][53852] Updated weights for policy 0, policy_version 39650 (0.0008) [2023-10-08 09:22:53,818][53852] Updated weights for policy 0, policy_version 39660 (0.0008) [2023-10-08 09:22:54,196][53852] Updated weights for policy 0, policy_version 39670 (0.0009) [2023-10-08 09:22:54,552][53852] Updated weights for policy 0, policy_version 39680 (0.0010) [2023-10-08 09:22:55,955][53885] Updated weights for policy 1, policy_version 39462 (0.0008) [2023-10-08 09:22:56,327][53885] Updated weights for policy 1, policy_version 39472 (0.0009) [2023-10-08 09:22:56,692][53885] Updated weights for policy 1, policy_version 39482 (0.0009) [2023-10-08 09:22:57,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81068032. Throughput: 0: 1824.0, 1: 1841.6. Samples: 20267408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:22:57,016][52710] Avg episode reward: [(0, '28.120'), (1, '30.630')] [2023-10-08 09:22:58,168][53852] Updated weights for policy 0, policy_version 39690 (0.0008) [2023-10-08 09:22:58,545][53852] Updated weights for policy 0, policy_version 39700 (0.0008) [2023-10-08 09:22:58,906][53852] Updated weights for policy 0, policy_version 39710 (0.0008) [2023-10-08 09:23:00,367][53885] Updated weights for policy 1, policy_version 39492 (0.0008) [2023-10-08 09:23:00,734][53885] Updated weights for policy 1, policy_version 39502 (0.0008) [2023-10-08 09:23:01,112][53885] Updated weights for policy 1, policy_version 39512 (0.0008) [2023-10-08 09:23:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81133568. Throughput: 0: 1849.3, 1: 1830.3. Samples: 20290224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:23:02,016][52710] Avg episode reward: [(0, '30.400'), (1, '31.590')] [2023-10-08 09:23:02,452][53852] Updated weights for policy 0, policy_version 39720 (0.0009) [2023-10-08 09:23:02,823][53852] Updated weights for policy 0, policy_version 39730 (0.0009) [2023-10-08 09:23:03,191][53852] Updated weights for policy 0, policy_version 39740 (0.0009) [2023-10-08 09:23:04,796][53885] Updated weights for policy 1, policy_version 39522 (0.0011) [2023-10-08 09:23:05,150][53885] Updated weights for policy 1, policy_version 39532 (0.0011) [2023-10-08 09:23:05,520][53885] Updated weights for policy 1, policy_version 39542 (0.0011) [2023-10-08 09:23:05,885][53885] Updated weights for policy 1, policy_version 39552 (0.0010) [2023-10-08 09:23:06,936][53852] Updated weights for policy 0, policy_version 39750 (0.0008) [2023-10-08 09:23:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81199104. Throughput: 0: 1848.4, 1: 1833.5. Samples: 20311994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:23:07,016][52710] Avg episode reward: [(0, '31.850'), (1, '30.110')] [2023-10-08 09:23:07,323][53852] Updated weights for policy 0, policy_version 39760 (0.0008) [2023-10-08 09:23:07,701][53852] Updated weights for policy 0, policy_version 39770 (0.0007) [2023-10-08 09:23:09,452][53885] Updated weights for policy 1, policy_version 39562 (0.0009) [2023-10-08 09:23:09,821][53885] Updated weights for policy 1, policy_version 39572 (0.0010) [2023-10-08 09:23:10,181][53885] Updated weights for policy 1, policy_version 39582 (0.0009) [2023-10-08 09:23:11,453][53852] Updated weights for policy 0, policy_version 39780 (0.0009) [2023-10-08 09:23:11,819][53852] Updated weights for policy 0, policy_version 39790 (0.0008) [2023-10-08 09:23:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81264640. Throughput: 0: 1841.7, 1: 1820.9. Samples: 20322804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:23:12,016][52710] Avg episode reward: [(0, '28.920'), (1, '30.760')] [2023-10-08 09:23:12,195][53852] Updated weights for policy 0, policy_version 39800 (0.0009) [2023-10-08 09:23:13,890][53885] Updated weights for policy 1, policy_version 39592 (0.0008) [2023-10-08 09:23:14,258][53885] Updated weights for policy 1, policy_version 39602 (0.0007) [2023-10-08 09:23:14,620][53885] Updated weights for policy 1, policy_version 39612 (0.0007) [2023-10-08 09:23:16,016][53852] Updated weights for policy 0, policy_version 39810 (0.0007) [2023-10-08 09:23:16,379][53852] Updated weights for policy 0, policy_version 39820 (0.0007) [2023-10-08 09:23:16,751][53852] Updated weights for policy 0, policy_version 39830 (0.0007) [2023-10-08 09:23:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81330176. Throughput: 0: 1838.2, 1: 1832.5. Samples: 20344804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 09:23:17,016][52710] Avg episode reward: [(0, '29.420'), (1, '28.490')] [2023-10-08 09:23:17,115][53852] Updated weights for policy 0, policy_version 39840 (0.0007) [2023-10-08 09:23:18,280][53885] Updated weights for policy 1, policy_version 39622 (0.0008) [2023-10-08 09:23:18,639][53885] Updated weights for policy 1, policy_version 39632 (0.0010) [2023-10-08 09:23:19,002][53885] Updated weights for policy 1, policy_version 39642 (0.0009) [2023-10-08 09:23:20,836][53852] Updated weights for policy 0, policy_version 39850 (0.0009) [2023-10-08 09:23:21,201][53852] Updated weights for policy 0, policy_version 39860 (0.0011) [2023-10-08 09:23:21,577][53852] Updated weights for policy 0, policy_version 39870 (0.0010) [2023-10-08 09:23:22,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81428480. Throughput: 0: 1831.2, 1: 1828.4. Samples: 20366152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 09:23:22,016][52710] Avg episode reward: [(0, '33.940'), (1, '29.330')] [2023-10-08 09:23:22,025][53500] Saving new best policy, reward=33.940! [2023-10-08 09:23:22,899][53885] Updated weights for policy 1, policy_version 39652 (0.0009) [2023-10-08 09:23:23,278][53885] Updated weights for policy 1, policy_version 39662 (0.0008) [2023-10-08 09:23:23,646][53885] Updated weights for policy 1, policy_version 39672 (0.0009) [2023-10-08 09:23:25,124][53852] Updated weights for policy 0, policy_version 39880 (0.0008) [2023-10-08 09:23:25,490][53852] Updated weights for policy 0, policy_version 39890 (0.0009) [2023-10-08 09:23:25,869][53852] Updated weights for policy 0, policy_version 39900 (0.0008) [2023-10-08 09:23:27,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 81494016. Throughput: 0: 1834.0, 1: 1823.9. Samples: 20377552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 09:23:27,016][52710] Avg episode reward: [(0, '31.120'), (1, '30.860')] [2023-10-08 09:23:27,401][53885] Updated weights for policy 1, policy_version 39682 (0.0010) [2023-10-08 09:23:27,773][53885] Updated weights for policy 1, policy_version 39692 (0.0010) [2023-10-08 09:23:28,137][53885] Updated weights for policy 1, policy_version 39702 (0.0008) [2023-10-08 09:23:28,513][53885] Updated weights for policy 1, policy_version 39712 (0.0008) [2023-10-08 09:23:29,546][53852] Updated weights for policy 0, policy_version 39910 (0.0008) [2023-10-08 09:23:29,915][53852] Updated weights for policy 0, policy_version 39920 (0.0008) [2023-10-08 09:23:30,291][53852] Updated weights for policy 0, policy_version 39930 (0.0007) [2023-10-08 09:23:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81559552. Throughput: 0: 1830.3, 1: 1819.8. Samples: 20398844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 09:23:32,016][52710] Avg episode reward: [(0, '30.920'), (1, '27.290')] [2023-10-08 09:23:32,337][53885] Updated weights for policy 1, policy_version 39722 (0.0007) [2023-10-08 09:23:32,707][53885] Updated weights for policy 1, policy_version 39732 (0.0008) [2023-10-08 09:23:33,084][53885] Updated weights for policy 1, policy_version 39742 (0.0009) [2023-10-08 09:23:33,874][53852] Updated weights for policy 0, policy_version 39940 (0.0007) [2023-10-08 09:23:34,247][53852] Updated weights for policy 0, policy_version 39950 (0.0008) [2023-10-08 09:23:34,617][53852] Updated weights for policy 0, policy_version 39960 (0.0009) [2023-10-08 09:23:36,690][53885] Updated weights for policy 1, policy_version 39752 (0.0009) [2023-10-08 09:23:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 81625088. Throughput: 0: 1834.0, 1: 1826.3. Samples: 20421580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) [2023-10-08 09:23:37,016][52710] Avg episode reward: [(0, '32.300'), (1, '27.520')] [2023-10-08 09:23:37,054][53885] Updated weights for policy 1, policy_version 39762 (0.0011) [2023-10-08 09:23:37,424][53885] Updated weights for policy 1, policy_version 39772 (0.0010) [2023-10-08 09:23:38,293][53852] Updated weights for policy 0, policy_version 39970 (0.0008) [2023-10-08 09:23:38,669][53852] Updated weights for policy 0, policy_version 39980 (0.0010) [2023-10-08 09:23:39,036][53852] Updated weights for policy 0, policy_version 39990 (0.0007) [2023-10-08 09:23:39,409][53852] Updated weights for policy 0, policy_version 40000 (0.0007) [2023-10-08 09:23:40,921][53885] Updated weights for policy 1, policy_version 39782 (0.0010) [2023-10-08 09:23:41,284][53885] Updated weights for policy 1, policy_version 39792 (0.0011) [2023-10-08 09:23:41,657][53885] Updated weights for policy 1, policy_version 39802 (0.0008) [2023-10-08 09:23:42,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81723392. Throughput: 0: 1833.3, 1: 1821.4. Samples: 20431872. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:23:42,015][52710] Avg episode reward: [(0, '31.130'), (1, '31.040')] [2023-10-08 09:23:43,047][53852] Updated weights for policy 0, policy_version 40010 (0.0007) [2023-10-08 09:23:43,421][53852] Updated weights for policy 0, policy_version 40020 (0.0008) [2023-10-08 09:23:43,800][53852] Updated weights for policy 0, policy_version 40030 (0.0008) [2023-10-08 09:23:45,270][53885] Updated weights for policy 1, policy_version 39812 (0.0008) [2023-10-08 09:23:45,635][53885] Updated weights for policy 1, policy_version 39822 (0.0009) [2023-10-08 09:23:45,992][53885] Updated weights for policy 1, policy_version 39832 (0.0007) [2023-10-08 09:23:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81788928. Throughput: 0: 1824.3, 1: 1826.5. Samples: 20454510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:23:47,016][52710] Avg episode reward: [(0, '28.140'), (1, '30.740')] [2023-10-08 09:23:47,515][53852] Updated weights for policy 0, policy_version 40040 (0.0009) [2023-10-08 09:23:47,888][53852] Updated weights for policy 0, policy_version 40050 (0.0008) [2023-10-08 09:23:48,254][53852] Updated weights for policy 0, policy_version 40060 (0.0008) [2023-10-08 09:23:49,652][53885] Updated weights for policy 1, policy_version 39842 (0.0007) [2023-10-08 09:23:50,024][53885] Updated weights for policy 1, policy_version 39852 (0.0007) [2023-10-08 09:23:50,395][53885] Updated weights for policy 1, policy_version 39862 (0.0010) [2023-10-08 09:23:50,763][53885] Updated weights for policy 1, policy_version 39872 (0.0009) [2023-10-08 09:23:51,911][53852] Updated weights for policy 0, policy_version 40070 (0.0009) [2023-10-08 09:23:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81854464. Throughput: 0: 1823.3, 1: 1837.5. Samples: 20476730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:23:52,016][52710] Avg episode reward: [(0, '30.390'), (1, '32.980')] [2023-10-08 09:23:52,289][53852] Updated weights for policy 0, policy_version 40080 (0.0007) [2023-10-08 09:23:52,661][53852] Updated weights for policy 0, policy_version 40090 (0.0008) [2023-10-08 09:23:54,371][53885] Updated weights for policy 1, policy_version 39882 (0.0008) [2023-10-08 09:23:54,732][53885] Updated weights for policy 1, policy_version 39892 (0.0008) [2023-10-08 09:23:55,112][53885] Updated weights for policy 1, policy_version 39902 (0.0011) [2023-10-08 09:23:56,330][53852] Updated weights for policy 0, policy_version 40100 (0.0010) [2023-10-08 09:23:56,697][53852] Updated weights for policy 0, policy_version 40110 (0.0008) [2023-10-08 09:23:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 81920000. Throughput: 0: 1827.5, 1: 1826.5. Samples: 20487234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:23:57,016][52710] Avg episode reward: [(0, '32.200'), (1, '35.310')] [2023-10-08 09:23:57,074][53852] Updated weights for policy 0, policy_version 40120 (0.0009) [2023-10-08 09:23:58,666][53885] Updated weights for policy 1, policy_version 39912 (0.0008) [2023-10-08 09:23:59,032][53885] Updated weights for policy 1, policy_version 39922 (0.0008) [2023-10-08 09:23:59,412][53885] Updated weights for policy 1, policy_version 39932 (0.0009) [2023-10-08 09:24:00,704][53852] Updated weights for policy 0, policy_version 40130 (0.0009) [2023-10-08 09:24:01,082][53852] Updated weights for policy 0, policy_version 40140 (0.0009) [2023-10-08 09:24:01,449][53852] Updated weights for policy 0, policy_version 40150 (0.0008) [2023-10-08 09:24:01,818][53852] Updated weights for policy 0, policy_version 40160 (0.0008) [2023-10-08 09:24:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82018304. Throughput: 0: 1830.8, 1: 1830.0. Samples: 20509542. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:24:02,016][52710] Avg episode reward: [(0, '27.490'), (1, '31.220')] [2023-10-08 09:24:03,009][53885] Updated weights for policy 1, policy_version 39942 (0.0009) [2023-10-08 09:24:03,378][53885] Updated weights for policy 1, policy_version 39952 (0.0009) [2023-10-08 09:24:03,746][53885] Updated weights for policy 1, policy_version 39962 (0.0007) [2023-10-08 09:24:05,356][53852] Updated weights for policy 0, policy_version 40170 (0.0008) [2023-10-08 09:24:05,731][53852] Updated weights for policy 0, policy_version 40180 (0.0010) [2023-10-08 09:24:06,092][53852] Updated weights for policy 0, policy_version 40190 (0.0007) [2023-10-08 09:24:07,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 82083840. Throughput: 0: 1834.5, 1: 1837.6. Samples: 20531396. Policy #0 lag: (min: 31.0, avg: 32.8, max: 59.0) [2023-10-08 09:24:07,016][52710] Avg episode reward: [(0, '29.870'), (1, '33.240')] [2023-10-08 09:24:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000040192_41156608.pth... [2023-10-08 09:24:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000039968_40927232.pth... [2023-10-08 09:24:07,060][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000038464_39387136.pth [2023-10-08 09:24:07,067][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000038272_39190528.pth [2023-10-08 09:24:07,447][53885] Updated weights for policy 1, policy_version 39972 (0.0008) [2023-10-08 09:24:07,815][53885] Updated weights for policy 1, policy_version 39982 (0.0007) [2023-10-08 09:24:08,172][53885] Updated weights for policy 1, policy_version 39992 (0.0008) [2023-10-08 09:24:09,730][53852] Updated weights for policy 0, policy_version 40200 (0.0008) [2023-10-08 09:24:10,100][53852] Updated weights for policy 0, policy_version 40210 (0.0009) [2023-10-08 09:24:10,478][53852] Updated weights for policy 0, policy_version 40220 (0.0011) [2023-10-08 09:24:11,944][53885] Updated weights for policy 1, policy_version 40002 (0.0008) [2023-10-08 09:24:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82149376. Throughput: 0: 1836.0, 1: 1837.3. Samples: 20542848. Policy #0 lag: (min: 31.0, avg: 32.8, max: 59.0) [2023-10-08 09:24:12,016][52710] Avg episode reward: [(0, '28.630'), (1, '32.860')] [2023-10-08 09:24:12,320][53885] Updated weights for policy 1, policy_version 40012 (0.0009) [2023-10-08 09:24:12,691][53885] Updated weights for policy 1, policy_version 40022 (0.0009) [2023-10-08 09:24:13,064][53885] Updated weights for policy 1, policy_version 40032 (0.0008) [2023-10-08 09:24:14,091][53852] Updated weights for policy 0, policy_version 40230 (0.0009) [2023-10-08 09:24:14,459][53852] Updated weights for policy 0, policy_version 40240 (0.0008) [2023-10-08 09:24:14,838][53852] Updated weights for policy 0, policy_version 40250 (0.0008) [2023-10-08 09:24:16,796][53885] Updated weights for policy 1, policy_version 40042 (0.0010) [2023-10-08 09:24:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82214912. Throughput: 0: 1840.6, 1: 1838.6. Samples: 20564408. Policy #0 lag: (min: 31.0, avg: 32.8, max: 59.0) [2023-10-08 09:24:17,016][52710] Avg episode reward: [(0, '29.150'), (1, '31.350')] [2023-10-08 09:24:17,167][53885] Updated weights for policy 1, policy_version 40052 (0.0011) [2023-10-08 09:24:17,534][53885] Updated weights for policy 1, policy_version 40062 (0.0010) [2023-10-08 09:24:18,454][53852] Updated weights for policy 0, policy_version 40260 (0.0008) [2023-10-08 09:24:18,832][53852] Updated weights for policy 0, policy_version 40270 (0.0008) [2023-10-08 09:24:19,203][53852] Updated weights for policy 0, policy_version 40280 (0.0009) [2023-10-08 09:24:21,288][53885] Updated weights for policy 1, policy_version 40072 (0.0010) [2023-10-08 09:24:21,664][53885] Updated weights for policy 1, policy_version 40082 (0.0009) [2023-10-08 09:24:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 82280448. Throughput: 0: 1839.4, 1: 1821.7. Samples: 20586330. Policy #0 lag: (min: 31.0, avg: 32.8, max: 59.0) [2023-10-08 09:24:22,016][52710] Avg episode reward: [(0, '30.330'), (1, '31.950')] [2023-10-08 09:24:22,025][53885] Updated weights for policy 1, policy_version 40092 (0.0009) [2023-10-08 09:24:22,867][53852] Updated weights for policy 0, policy_version 40290 (0.0011) [2023-10-08 09:24:23,230][53852] Updated weights for policy 0, policy_version 40300 (0.0009) [2023-10-08 09:24:23,594][53852] Updated weights for policy 0, policy_version 40310 (0.0009) [2023-10-08 09:24:23,961][53852] Updated weights for policy 0, policy_version 40320 (0.0010) [2023-10-08 09:24:25,562][53885] Updated weights for policy 1, policy_version 40102 (0.0009) [2023-10-08 09:24:25,929][53885] Updated weights for policy 1, policy_version 40112 (0.0010) [2023-10-08 09:24:26,304][53885] Updated weights for policy 1, policy_version 40122 (0.0009) [2023-10-08 09:24:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82378752. Throughput: 0: 1838.0, 1: 1835.6. Samples: 20597184. Policy #0 lag: (min: 31.0, avg: 32.8, max: 59.0) [2023-10-08 09:24:27,016][52710] Avg episode reward: [(0, '30.010'), (1, '33.290')] [2023-10-08 09:24:27,730][53852] Updated weights for policy 0, policy_version 40330 (0.0008) [2023-10-08 09:24:28,088][53852] Updated weights for policy 0, policy_version 40340 (0.0008) [2023-10-08 09:24:28,458][53852] Updated weights for policy 0, policy_version 40350 (0.0009) [2023-10-08 09:24:30,000][53885] Updated weights for policy 1, policy_version 40132 (0.0009) [2023-10-08 09:24:30,366][53885] Updated weights for policy 1, policy_version 40142 (0.0008) [2023-10-08 09:24:30,734][53885] Updated weights for policy 1, policy_version 40152 (0.0007) [2023-10-08 09:24:32,006][53852] Updated weights for policy 0, policy_version 40360 (0.0007) [2023-10-08 09:24:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 82444288. Throughput: 0: 1842.0, 1: 1821.9. Samples: 20619386. Policy #0 lag: (min: 31.0, avg: 32.8, max: 59.0) [2023-10-08 09:24:32,016][52710] Avg episode reward: [(0, '30.660'), (1, '31.570')] [2023-10-08 09:24:32,367][53852] Updated weights for policy 0, policy_version 40370 (0.0010) [2023-10-08 09:24:32,734][53852] Updated weights for policy 0, policy_version 40380 (0.0010) [2023-10-08 09:24:34,364][53885] Updated weights for policy 1, policy_version 40162 (0.0009) [2023-10-08 09:24:34,742][53885] Updated weights for policy 1, policy_version 40172 (0.0009) [2023-10-08 09:24:35,098][53885] Updated weights for policy 1, policy_version 40182 (0.0008) [2023-10-08 09:24:35,467][53885] Updated weights for policy 1, policy_version 40192 (0.0009) [2023-10-08 09:24:36,503][53852] Updated weights for policy 0, policy_version 40390 (0.0010) [2023-10-08 09:24:36,878][53852] Updated weights for policy 0, policy_version 40400 (0.0009) [2023-10-08 09:24:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82509824. Throughput: 0: 1830.7, 1: 1828.0. Samples: 20641370. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-10-08 09:24:37,016][52710] Avg episode reward: [(0, '30.210'), (1, '31.360')] [2023-10-08 09:24:37,250][53852] Updated weights for policy 0, policy_version 40410 (0.0009) [2023-10-08 09:24:39,065][53885] Updated weights for policy 1, policy_version 40202 (0.0007) [2023-10-08 09:24:39,439][53885] Updated weights for policy 1, policy_version 40212 (0.0007) [2023-10-08 09:24:39,798][53885] Updated weights for policy 1, policy_version 40222 (0.0008) [2023-10-08 09:24:40,857][53852] Updated weights for policy 0, policy_version 40420 (0.0009) [2023-10-08 09:24:41,239][53852] Updated weights for policy 0, policy_version 40430 (0.0010) [2023-10-08 09:24:41,609][53852] Updated weights for policy 0, policy_version 40440 (0.0009) [2023-10-08 09:24:42,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 82608128. Throughput: 0: 1842.7, 1: 1829.3. Samples: 20652470. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-10-08 09:24:42,015][52710] Avg episode reward: [(0, '28.410'), (1, '29.760')] [2023-10-08 09:24:43,581][53885] Updated weights for policy 1, policy_version 40232 (0.0008) [2023-10-08 09:24:43,940][53885] Updated weights for policy 1, policy_version 40242 (0.0008) [2023-10-08 09:24:44,305][53885] Updated weights for policy 1, policy_version 40252 (0.0009) [2023-10-08 09:24:45,216][53852] Updated weights for policy 0, policy_version 40450 (0.0007) [2023-10-08 09:24:45,579][53852] Updated weights for policy 0, policy_version 40460 (0.0009) [2023-10-08 09:24:45,947][53852] Updated weights for policy 0, policy_version 40470 (0.0009) [2023-10-08 09:24:46,313][53852] Updated weights for policy 0, policy_version 40480 (0.0008) [2023-10-08 09:24:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82673664. Throughput: 0: 1835.5, 1: 1834.8. Samples: 20674706. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-10-08 09:24:47,016][52710] Avg episode reward: [(0, '31.890'), (1, '31.920')] [2023-10-08 09:24:47,975][53885] Updated weights for policy 1, policy_version 40262 (0.0010) [2023-10-08 09:24:48,334][53885] Updated weights for policy 1, policy_version 40272 (0.0007) [2023-10-08 09:24:48,703][53885] Updated weights for policy 1, policy_version 40282 (0.0008) [2023-10-08 09:24:49,815][53852] Updated weights for policy 0, policy_version 40490 (0.0009) [2023-10-08 09:24:50,183][53852] Updated weights for policy 0, policy_version 40500 (0.0007) [2023-10-08 09:24:50,553][53852] Updated weights for policy 0, policy_version 40510 (0.0007) [2023-10-08 09:24:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82739200. Throughput: 0: 1848.9, 1: 1835.6. Samples: 20697198. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-10-08 09:24:52,016][52710] Avg episode reward: [(0, '30.230'), (1, '30.110')] [2023-10-08 09:24:52,384][53885] Updated weights for policy 1, policy_version 40292 (0.0011) [2023-10-08 09:24:52,752][53885] Updated weights for policy 1, policy_version 40302 (0.0009) [2023-10-08 09:24:53,122][53885] Updated weights for policy 1, policy_version 40312 (0.0009) [2023-10-08 09:24:54,057][53852] Updated weights for policy 0, policy_version 40520 (0.0009) [2023-10-08 09:24:54,434][53852] Updated weights for policy 0, policy_version 40530 (0.0008) [2023-10-08 09:24:54,803][53852] Updated weights for policy 0, policy_version 40540 (0.0008) [2023-10-08 09:24:56,529][53885] Updated weights for policy 1, policy_version 40322 (0.0007) [2023-10-08 09:24:56,896][53885] Updated weights for policy 1, policy_version 40332 (0.0007) [2023-10-08 09:24:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82804736. Throughput: 0: 1828.6, 1: 1840.3. Samples: 20707950. Policy #0 lag: (min: 11.0, avg: 17.1, max: 43.0) [2023-10-08 09:24:57,016][52710] Avg episode reward: [(0, '30.150'), (1, '27.790')] [2023-10-08 09:24:57,266][53885] Updated weights for policy 1, policy_version 40342 (0.0007) [2023-10-08 09:24:57,640][53885] Updated weights for policy 1, policy_version 40352 (0.0007) [2023-10-08 09:24:58,591][53852] Updated weights for policy 0, policy_version 40550 (0.0008) [2023-10-08 09:24:58,962][53852] Updated weights for policy 0, policy_version 40560 (0.0009) [2023-10-08 09:24:59,329][53852] Updated weights for policy 0, policy_version 40570 (0.0008) [2023-10-08 09:25:01,336][53885] Updated weights for policy 1, policy_version 40362 (0.0009) [2023-10-08 09:25:01,712][53885] Updated weights for policy 1, policy_version 40372 (0.0010) [2023-10-08 09:25:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82870272. Throughput: 0: 1842.0, 1: 1843.2. Samples: 20730244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:25:02,016][52710] Avg episode reward: [(0, '30.070'), (1, '29.720')] [2023-10-08 09:25:02,073][53885] Updated weights for policy 1, policy_version 40382 (0.0011) [2023-10-08 09:25:03,032][53852] Updated weights for policy 0, policy_version 40580 (0.0007) [2023-10-08 09:25:03,398][53852] Updated weights for policy 0, policy_version 40590 (0.0008) [2023-10-08 09:25:03,769][53852] Updated weights for policy 0, policy_version 40600 (0.0008) [2023-10-08 09:25:05,924][53885] Updated weights for policy 1, policy_version 40392 (0.0007) [2023-10-08 09:25:06,297][53885] Updated weights for policy 1, policy_version 40402 (0.0010) [2023-10-08 09:25:06,680][53885] Updated weights for policy 1, policy_version 40412 (0.0007) [2023-10-08 09:25:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82968576. Throughput: 0: 1843.6, 1: 1838.0. Samples: 20752004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:25:07,016][52710] Avg episode reward: [(0, '33.120'), (1, '31.430')] [2023-10-08 09:25:07,467][53852] Updated weights for policy 0, policy_version 40610 (0.0009) [2023-10-08 09:25:07,830][53852] Updated weights for policy 0, policy_version 40620 (0.0008) [2023-10-08 09:25:08,202][53852] Updated weights for policy 0, policy_version 40630 (0.0007) [2023-10-08 09:25:08,576][53852] Updated weights for policy 0, policy_version 40640 (0.0007) [2023-10-08 09:25:10,349][53885] Updated weights for policy 1, policy_version 40422 (0.0008) [2023-10-08 09:25:10,724][53885] Updated weights for policy 1, policy_version 40432 (0.0008) [2023-10-08 09:25:11,080][53885] Updated weights for policy 1, policy_version 40442 (0.0009) [2023-10-08 09:25:12,003][53852] Updated weights for policy 0, policy_version 40650 (0.0007) [2023-10-08 09:25:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83034112. Throughput: 0: 1845.5, 1: 1839.6. Samples: 20763012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:25:12,016][52710] Avg episode reward: [(0, '31.250'), (1, '28.290')] [2023-10-08 09:25:12,363][53852] Updated weights for policy 0, policy_version 40660 (0.0007) [2023-10-08 09:25:12,735][53852] Updated weights for policy 0, policy_version 40670 (0.0008) [2023-10-08 09:25:14,761][53885] Updated weights for policy 1, policy_version 40452 (0.0008) [2023-10-08 09:25:15,121][53885] Updated weights for policy 1, policy_version 40462 (0.0010) [2023-10-08 09:25:15,494][53885] Updated weights for policy 1, policy_version 40472 (0.0009) [2023-10-08 09:25:16,307][53852] Updated weights for policy 0, policy_version 40680 (0.0009) [2023-10-08 09:25:16,682][53852] Updated weights for policy 0, policy_version 40690 (0.0009) [2023-10-08 09:25:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83099648. Throughput: 0: 1850.7, 1: 1830.4. Samples: 20785034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:25:17,016][52710] Avg episode reward: [(0, '33.100'), (1, '30.300')] [2023-10-08 09:25:17,059][53852] Updated weights for policy 0, policy_version 40700 (0.0009) [2023-10-08 09:25:19,116][53885] Updated weights for policy 1, policy_version 40482 (0.0008) [2023-10-08 09:25:19,483][53885] Updated weights for policy 1, policy_version 40492 (0.0010) [2023-10-08 09:25:19,854][53885] Updated weights for policy 1, policy_version 40502 (0.0008) [2023-10-08 09:25:20,224][53885] Updated weights for policy 1, policy_version 40512 (0.0008) [2023-10-08 09:25:20,726][53852] Updated weights for policy 0, policy_version 40710 (0.0008) [2023-10-08 09:25:21,088][53852] Updated weights for policy 0, policy_version 40720 (0.0010) [2023-10-08 09:25:21,467][53852] Updated weights for policy 0, policy_version 40730 (0.0008) [2023-10-08 09:25:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 83197952. Throughput: 0: 1830.4, 1: 1835.3. Samples: 20806324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:25:22,015][52710] Avg episode reward: [(0, '32.680'), (1, '33.970')] [2023-10-08 09:25:23,971][53885] Updated weights for policy 1, policy_version 40522 (0.0009) [2023-10-08 09:25:24,336][53885] Updated weights for policy 1, policy_version 40532 (0.0008) [2023-10-08 09:25:24,701][53885] Updated weights for policy 1, policy_version 40542 (0.0009) [2023-10-08 09:25:24,995][53852] Updated weights for policy 0, policy_version 40740 (0.0009) [2023-10-08 09:25:25,363][53852] Updated weights for policy 0, policy_version 40750 (0.0009) [2023-10-08 09:25:25,733][53852] Updated weights for policy 0, policy_version 40760 (0.0008) [2023-10-08 09:25:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83263488. Throughput: 0: 1856.4, 1: 1824.2. Samples: 20818096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:25:27,016][52710] Avg episode reward: [(0, '31.830'), (1, '31.450')] [2023-10-08 09:25:28,193][53885] Updated weights for policy 1, policy_version 40552 (0.0009) [2023-10-08 09:25:28,552][53885] Updated weights for policy 1, policy_version 40562 (0.0010) [2023-10-08 09:25:28,918][53885] Updated weights for policy 1, policy_version 40572 (0.0008) [2023-10-08 09:25:29,411][53852] Updated weights for policy 0, policy_version 40770 (0.0007) [2023-10-08 09:25:29,795][53852] Updated weights for policy 0, policy_version 40780 (0.0007) [2023-10-08 09:25:30,170][53852] Updated weights for policy 0, policy_version 40790 (0.0008) [2023-10-08 09:25:30,535][53852] Updated weights for policy 0, policy_version 40800 (0.0007) [2023-10-08 09:25:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 83329024. Throughput: 0: 1830.2, 1: 1832.8. Samples: 20839542. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-08 09:25:32,015][52710] Avg episode reward: [(0, '29.950'), (1, '29.470')] [2023-10-08 09:25:32,722][53885] Updated weights for policy 1, policy_version 40582 (0.0009) [2023-10-08 09:25:33,096][53885] Updated weights for policy 1, policy_version 40592 (0.0008) [2023-10-08 09:25:33,461][53885] Updated weights for policy 1, policy_version 40602 (0.0007) [2023-10-08 09:25:34,229][53852] Updated weights for policy 0, policy_version 40810 (0.0008) [2023-10-08 09:25:34,595][53852] Updated weights for policy 0, policy_version 40820 (0.0007) [2023-10-08 09:25:34,964][53852] Updated weights for policy 0, policy_version 40830 (0.0007) [2023-10-08 09:25:37,000][53885] Updated weights for policy 1, policy_version 40612 (0.0009) [2023-10-08 09:25:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83394560. Throughput: 0: 1844.7, 1: 1826.7. Samples: 20862412. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-08 09:25:37,016][52710] Avg episode reward: [(0, '32.420'), (1, '30.210')] [2023-10-08 09:25:37,374][53885] Updated weights for policy 1, policy_version 40622 (0.0010) [2023-10-08 09:25:37,726][53885] Updated weights for policy 1, policy_version 40632 (0.0009) [2023-10-08 09:25:38,550][53852] Updated weights for policy 0, policy_version 40840 (0.0010) [2023-10-08 09:25:38,907][53852] Updated weights for policy 0, policy_version 40850 (0.0011) [2023-10-08 09:25:39,283][53852] Updated weights for policy 0, policy_version 40860 (0.0008) [2023-10-08 09:25:41,425][53885] Updated weights for policy 1, policy_version 40642 (0.0010) [2023-10-08 09:25:41,784][53885] Updated weights for policy 1, policy_version 40652 (0.0009) [2023-10-08 09:25:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83460096. Throughput: 0: 1832.2, 1: 1824.2. Samples: 20872488. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-08 09:25:42,015][52710] Avg episode reward: [(0, '29.630'), (1, '28.430')] [2023-10-08 09:25:42,152][53885] Updated weights for policy 1, policy_version 40662 (0.0009) [2023-10-08 09:25:42,512][53885] Updated weights for policy 1, policy_version 40672 (0.0008) [2023-10-08 09:25:43,044][53852] Updated weights for policy 0, policy_version 40870 (0.0009) [2023-10-08 09:25:43,413][53852] Updated weights for policy 0, policy_version 40880 (0.0008) [2023-10-08 09:25:43,777][53852] Updated weights for policy 0, policy_version 40890 (0.0008) [2023-10-08 09:25:46,318][53885] Updated weights for policy 1, policy_version 40682 (0.0008) [2023-10-08 09:25:46,693][53885] Updated weights for policy 1, policy_version 40692 (0.0009) [2023-10-08 09:25:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 83525632. Throughput: 0: 1846.4, 1: 1816.7. Samples: 20895084. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-08 09:25:47,016][52710] Avg episode reward: [(0, '28.560'), (1, '27.540')] [2023-10-08 09:25:47,061][53885] Updated weights for policy 1, policy_version 40702 (0.0009) [2023-10-08 09:25:47,392][53852] Updated weights for policy 0, policy_version 40900 (0.0008) [2023-10-08 09:25:47,759][53852] Updated weights for policy 0, policy_version 40910 (0.0008) [2023-10-08 09:25:48,136][53852] Updated weights for policy 0, policy_version 40920 (0.0010) [2023-10-08 09:25:50,763][53885] Updated weights for policy 1, policy_version 40712 (0.0009) [2023-10-08 09:25:51,144][53885] Updated weights for policy 1, policy_version 40722 (0.0010) [2023-10-08 09:25:51,513][53885] Updated weights for policy 1, policy_version 40732 (0.0007) [2023-10-08 09:25:51,797][53852] Updated weights for policy 0, policy_version 40930 (0.0009) [2023-10-08 09:25:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83623936. Throughput: 0: 1847.5, 1: 1812.0. Samples: 20916678. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-08 09:25:52,016][52710] Avg episode reward: [(0, '32.340'), (1, '25.090')] [2023-10-08 09:25:52,162][53852] Updated weights for policy 0, policy_version 40940 (0.0010) [2023-10-08 09:25:52,533][53852] Updated weights for policy 0, policy_version 40950 (0.0010) [2023-10-08 09:25:52,912][53852] Updated weights for policy 0, policy_version 40960 (0.0011) [2023-10-08 09:25:55,064][53885] Updated weights for policy 1, policy_version 40742 (0.0007) [2023-10-08 09:25:55,428][53885] Updated weights for policy 1, policy_version 40752 (0.0009) [2023-10-08 09:25:55,797][53885] Updated weights for policy 1, policy_version 40762 (0.0009) [2023-10-08 09:25:56,579][53852] Updated weights for policy 0, policy_version 40970 (0.0009) [2023-10-08 09:25:56,956][53852] Updated weights for policy 0, policy_version 40980 (0.0010) [2023-10-08 09:25:57,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 83689472. Throughput: 0: 1845.8, 1: 1825.6. Samples: 20928226. Policy #0 lag: (min: 31.0, avg: 40.8, max: 63.0) [2023-10-08 09:25:57,016][52710] Avg episode reward: [(0, '28.910'), (1, '29.110')] [2023-10-08 09:25:57,322][53852] Updated weights for policy 0, policy_version 40990 (0.0010) [2023-10-08 09:25:59,370][53885] Updated weights for policy 1, policy_version 40772 (0.0009) [2023-10-08 09:25:59,735][53885] Updated weights for policy 1, policy_version 40782 (0.0007) [2023-10-08 09:26:00,109][53885] Updated weights for policy 1, policy_version 40792 (0.0009) [2023-10-08 09:26:00,838][53852] Updated weights for policy 0, policy_version 41000 (0.0010) [2023-10-08 09:26:01,204][53852] Updated weights for policy 0, policy_version 41010 (0.0011) [2023-10-08 09:26:01,571][53852] Updated weights for policy 0, policy_version 41020 (0.0009) [2023-10-08 09:26:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 83787776. Throughput: 0: 1846.7, 1: 1825.3. Samples: 20950276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:02,015][52710] Avg episode reward: [(0, '29.460'), (1, '27.890')] [2023-10-08 09:26:03,921][53885] Updated weights for policy 1, policy_version 40802 (0.0009) [2023-10-08 09:26:04,288][53885] Updated weights for policy 1, policy_version 40812 (0.0010) [2023-10-08 09:26:04,663][53885] Updated weights for policy 1, policy_version 40822 (0.0007) [2023-10-08 09:26:05,028][53885] Updated weights for policy 1, policy_version 40832 (0.0007) [2023-10-08 09:26:05,236][53852] Updated weights for policy 0, policy_version 41030 (0.0008) [2023-10-08 09:26:05,608][53852] Updated weights for policy 0, policy_version 41040 (0.0007) [2023-10-08 09:26:05,974][53852] Updated weights for policy 0, policy_version 41050 (0.0007) [2023-10-08 09:26:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83853312. Throughput: 0: 1842.2, 1: 1829.9. Samples: 20971568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:07,016][52710] Avg episode reward: [(0, '31.570'), (1, '28.480')] [2023-10-08 09:26:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000041056_42041344.pth... [2023-10-08 09:26:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000040832_41811968.pth... [2023-10-08 09:26:07,064][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000039328_40271872.pth [2023-10-08 09:26:07,066][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000039136_40075264.pth [2023-10-08 09:26:08,784][53885] Updated weights for policy 1, policy_version 40842 (0.0009) [2023-10-08 09:26:09,156][53885] Updated weights for policy 1, policy_version 40852 (0.0009) [2023-10-08 09:26:09,526][53885] Updated weights for policy 1, policy_version 40862 (0.0007) [2023-10-08 09:26:09,598][53852] Updated weights for policy 0, policy_version 41060 (0.0007) [2023-10-08 09:26:09,959][53852] Updated weights for policy 0, policy_version 41070 (0.0008) [2023-10-08 09:26:10,339][53852] Updated weights for policy 0, policy_version 41080 (0.0010) [2023-10-08 09:26:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83918848. Throughput: 0: 1843.0, 1: 1822.8. Samples: 20983056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:12,015][52710] Avg episode reward: [(0, '31.430'), (1, '30.230')] [2023-10-08 09:26:13,099][53885] Updated weights for policy 1, policy_version 40872 (0.0008) [2023-10-08 09:26:13,468][53885] Updated weights for policy 1, policy_version 40882 (0.0007) [2023-10-08 09:26:13,841][53885] Updated weights for policy 1, policy_version 40892 (0.0008) [2023-10-08 09:26:13,975][53852] Updated weights for policy 0, policy_version 41090 (0.0008) [2023-10-08 09:26:14,346][53852] Updated weights for policy 0, policy_version 41100 (0.0007) [2023-10-08 09:26:14,730][53852] Updated weights for policy 0, policy_version 41110 (0.0009) [2023-10-08 09:26:15,107][53852] Updated weights for policy 0, policy_version 41120 (0.0007) [2023-10-08 09:26:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 83984384. Throughput: 0: 1842.4, 1: 1824.2. Samples: 21004540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:17,016][52710] Avg episode reward: [(0, '28.070'), (1, '31.080')] [2023-10-08 09:26:17,667][53885] Updated weights for policy 1, policy_version 40902 (0.0009) [2023-10-08 09:26:18,039][53885] Updated weights for policy 1, policy_version 40912 (0.0008) [2023-10-08 09:26:18,410][53885] Updated weights for policy 1, policy_version 40922 (0.0011) [2023-10-08 09:26:18,942][53852] Updated weights for policy 0, policy_version 41130 (0.0010) [2023-10-08 09:26:19,318][53852] Updated weights for policy 0, policy_version 41140 (0.0008) [2023-10-08 09:26:19,692][53852] Updated weights for policy 0, policy_version 41150 (0.0009) [2023-10-08 09:26:22,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 84049920. Throughput: 0: 1835.9, 1: 1820.0. Samples: 21026926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:22,016][52710] Avg episode reward: [(0, '29.470'), (1, '34.530')] [2023-10-08 09:26:22,109][53885] Updated weights for policy 1, policy_version 40932 (0.0009) [2023-10-08 09:26:22,484][53885] Updated weights for policy 1, policy_version 40942 (0.0009) [2023-10-08 09:26:22,849][53885] Updated weights for policy 1, policy_version 40952 (0.0009) [2023-10-08 09:26:23,298][53852] Updated weights for policy 0, policy_version 41160 (0.0007) [2023-10-08 09:26:23,670][53852] Updated weights for policy 0, policy_version 41170 (0.0007) [2023-10-08 09:26:24,039][53852] Updated weights for policy 0, policy_version 41180 (0.0007) [2023-10-08 09:26:26,544][53885] Updated weights for policy 1, policy_version 40962 (0.0007) [2023-10-08 09:26:26,917][53885] Updated weights for policy 1, policy_version 40972 (0.0007) [2023-10-08 09:26:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 84115456. Throughput: 0: 1830.8, 1: 1819.6. Samples: 21036756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:27,016][52710] Avg episode reward: [(0, '29.040'), (1, '31.180')] [2023-10-08 09:26:27,277][53885] Updated weights for policy 1, policy_version 40982 (0.0007) [2023-10-08 09:26:27,642][53885] Updated weights for policy 1, policy_version 40992 (0.0008) [2023-10-08 09:26:27,813][53852] Updated weights for policy 0, policy_version 41190 (0.0008) [2023-10-08 09:26:28,173][53852] Updated weights for policy 0, policy_version 41200 (0.0009) [2023-10-08 09:26:28,554][53852] Updated weights for policy 0, policy_version 41210 (0.0011) [2023-10-08 09:26:31,173][53885] Updated weights for policy 1, policy_version 41002 (0.0007) [2023-10-08 09:26:31,540][53885] Updated weights for policy 1, policy_version 41012 (0.0010) [2023-10-08 09:26:31,911][53885] Updated weights for policy 1, policy_version 41022 (0.0009) [2023-10-08 09:26:32,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84213760. Throughput: 0: 1833.7, 1: 1830.8. Samples: 21059988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:32,016][52710] Avg episode reward: [(0, '28.130'), (1, '30.150')] [2023-10-08 09:26:32,272][53852] Updated weights for policy 0, policy_version 41220 (0.0009) [2023-10-08 09:26:32,641][53852] Updated weights for policy 0, policy_version 41230 (0.0008) [2023-10-08 09:26:33,020][53852] Updated weights for policy 0, policy_version 41240 (0.0009) [2023-10-08 09:26:35,592][53885] Updated weights for policy 1, policy_version 41032 (0.0009) [2023-10-08 09:26:35,975][53885] Updated weights for policy 1, policy_version 41042 (0.0009) [2023-10-08 09:26:36,344][53885] Updated weights for policy 1, policy_version 41052 (0.0010) [2023-10-08 09:26:36,707][53852] Updated weights for policy 0, policy_version 41250 (0.0009) [2023-10-08 09:26:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84279296. Throughput: 0: 1830.3, 1: 1832.3. Samples: 21081494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:37,016][52710] Avg episode reward: [(0, '26.330'), (1, '33.880')] [2023-10-08 09:26:37,080][53852] Updated weights for policy 0, policy_version 41260 (0.0008) [2023-10-08 09:26:37,452][53852] Updated weights for policy 0, policy_version 41270 (0.0009) [2023-10-08 09:26:37,820][53852] Updated weights for policy 0, policy_version 41280 (0.0009) [2023-10-08 09:26:39,941][53885] Updated weights for policy 1, policy_version 41062 (0.0009) [2023-10-08 09:26:40,312][53885] Updated weights for policy 1, policy_version 41072 (0.0010) [2023-10-08 09:26:40,684][53885] Updated weights for policy 1, policy_version 41082 (0.0008) [2023-10-08 09:26:41,363][53852] Updated weights for policy 0, policy_version 41290 (0.0007) [2023-10-08 09:26:41,734][53852] Updated weights for policy 0, policy_version 41300 (0.0007) [2023-10-08 09:26:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84344832. Throughput: 0: 1832.6, 1: 1831.9. Samples: 21093128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:42,015][52710] Avg episode reward: [(0, '27.170'), (1, '31.670')] [2023-10-08 09:26:42,104][53852] Updated weights for policy 0, policy_version 41310 (0.0008) [2023-10-08 09:26:44,354][53885] Updated weights for policy 1, policy_version 41092 (0.0007) [2023-10-08 09:26:44,709][53885] Updated weights for policy 1, policy_version 41102 (0.0009) [2023-10-08 09:26:45,073][53885] Updated weights for policy 1, policy_version 41112 (0.0010) [2023-10-08 09:26:45,702][53852] Updated weights for policy 0, policy_version 41320 (0.0011) [2023-10-08 09:26:46,086][53852] Updated weights for policy 0, policy_version 41330 (0.0011) [2023-10-08 09:26:46,456][53852] Updated weights for policy 0, policy_version 41340 (0.0008) [2023-10-08 09:26:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 84443136. Throughput: 0: 1828.1, 1: 1820.7. Samples: 21114470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:47,016][52710] Avg episode reward: [(0, '29.200'), (1, '29.990')] [2023-10-08 09:26:48,863][53885] Updated weights for policy 1, policy_version 41122 (0.0010) [2023-10-08 09:26:49,232][53885] Updated weights for policy 1, policy_version 41132 (0.0009) [2023-10-08 09:26:49,595][53885] Updated weights for policy 1, policy_version 41142 (0.0007) [2023-10-08 09:26:49,970][53885] Updated weights for policy 1, policy_version 41152 (0.0007) [2023-10-08 09:26:50,192][53852] Updated weights for policy 0, policy_version 41350 (0.0010) [2023-10-08 09:26:50,559][53852] Updated weights for policy 0, policy_version 41360 (0.0009) [2023-10-08 09:26:50,934][53852] Updated weights for policy 0, policy_version 41370 (0.0009) [2023-10-08 09:26:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84508672. Throughput: 0: 1830.8, 1: 1820.8. Samples: 21135888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:52,015][52710] Avg episode reward: [(0, '26.650'), (1, '31.670')] [2023-10-08 09:26:53,518][53885] Updated weights for policy 1, policy_version 41162 (0.0008) [2023-10-08 09:26:53,884][53885] Updated weights for policy 1, policy_version 41172 (0.0009) [2023-10-08 09:26:54,264][53885] Updated weights for policy 1, policy_version 41182 (0.0010) [2023-10-08 09:26:54,419][53852] Updated weights for policy 0, policy_version 41380 (0.0009) [2023-10-08 09:26:54,791][53852] Updated weights for policy 0, policy_version 41390 (0.0007) [2023-10-08 09:26:55,163][53852] Updated weights for policy 0, policy_version 41400 (0.0007) [2023-10-08 09:26:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84574208. Throughput: 0: 1823.1, 1: 1823.5. Samples: 21147158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:26:57,016][52710] Avg episode reward: [(0, '27.430'), (1, '31.340')] [2023-10-08 09:26:57,882][53885] Updated weights for policy 1, policy_version 41192 (0.0008) [2023-10-08 09:26:58,256][53885] Updated weights for policy 1, policy_version 41202 (0.0008) [2023-10-08 09:26:58,633][53885] Updated weights for policy 1, policy_version 41212 (0.0008) [2023-10-08 09:26:58,730][53852] Updated weights for policy 0, policy_version 41410 (0.0007) [2023-10-08 09:26:59,094][53852] Updated weights for policy 0, policy_version 41420 (0.0008) [2023-10-08 09:26:59,478][53852] Updated weights for policy 0, policy_version 41430 (0.0009) [2023-10-08 09:26:59,846][53852] Updated weights for policy 0, policy_version 41440 (0.0007) [2023-10-08 09:27:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 84639744. Throughput: 0: 1835.1, 1: 1825.7. Samples: 21169276. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-08 09:27:02,015][52710] Avg episode reward: [(0, '28.770'), (1, '27.100')] [2023-10-08 09:27:02,300][53885] Updated weights for policy 1, policy_version 41222 (0.0010) [2023-10-08 09:27:02,663][53885] Updated weights for policy 1, policy_version 41232 (0.0009) [2023-10-08 09:27:03,030][53885] Updated weights for policy 1, policy_version 41242 (0.0009) [2023-10-08 09:27:03,438][53852] Updated weights for policy 0, policy_version 41450 (0.0009) [2023-10-08 09:27:03,805][53852] Updated weights for policy 0, policy_version 41460 (0.0009) [2023-10-08 09:27:04,176][53852] Updated weights for policy 0, policy_version 41470 (0.0008) [2023-10-08 09:27:06,806][53885] Updated weights for policy 1, policy_version 41252 (0.0007) [2023-10-08 09:27:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 84705280. Throughput: 0: 1846.2, 1: 1827.4. Samples: 21192238. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-08 09:27:07,016][52710] Avg episode reward: [(0, '31.910'), (1, '25.010')] [2023-10-08 09:27:07,178][53885] Updated weights for policy 1, policy_version 41262 (0.0007) [2023-10-08 09:27:07,545][53885] Updated weights for policy 1, policy_version 41272 (0.0011) [2023-10-08 09:27:07,980][53852] Updated weights for policy 0, policy_version 41480 (0.0009) [2023-10-08 09:27:08,352][53852] Updated weights for policy 0, policy_version 41490 (0.0009) [2023-10-08 09:27:08,718][53852] Updated weights for policy 0, policy_version 41500 (0.0007) [2023-10-08 09:27:11,207][53885] Updated weights for policy 1, policy_version 41282 (0.0009) [2023-10-08 09:27:11,581][53885] Updated weights for policy 1, policy_version 41292 (0.0007) [2023-10-08 09:27:11,947][53885] Updated weights for policy 1, policy_version 41302 (0.0007) [2023-10-08 09:27:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84770816. Throughput: 0: 1845.7, 1: 1827.8. Samples: 21202064. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-08 09:27:12,015][52710] Avg episode reward: [(0, '29.430'), (1, '23.770')] [2023-10-08 09:27:12,309][53885] Updated weights for policy 1, policy_version 41312 (0.0009) [2023-10-08 09:27:12,354][53852] Updated weights for policy 0, policy_version 41510 (0.0008) [2023-10-08 09:27:12,722][53852] Updated weights for policy 0, policy_version 41520 (0.0008) [2023-10-08 09:27:13,107][53852] Updated weights for policy 0, policy_version 41530 (0.0009) [2023-10-08 09:27:15,953][53885] Updated weights for policy 1, policy_version 41322 (0.0009) [2023-10-08 09:27:16,311][53885] Updated weights for policy 1, policy_version 41332 (0.0007) [2023-10-08 09:27:16,678][53885] Updated weights for policy 1, policy_version 41342 (0.0008) [2023-10-08 09:27:16,724][53852] Updated weights for policy 0, policy_version 41540 (0.0009) [2023-10-08 09:27:17,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 84869120. Throughput: 0: 1848.8, 1: 1822.6. Samples: 21225200. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-08 09:27:17,015][52710] Avg episode reward: [(0, '29.930'), (1, '20.880')] [2023-10-08 09:27:17,100][53852] Updated weights for policy 0, policy_version 41550 (0.0010) [2023-10-08 09:27:17,469][53852] Updated weights for policy 0, policy_version 41560 (0.0009) [2023-10-08 09:27:20,324][53885] Updated weights for policy 1, policy_version 41352 (0.0010) [2023-10-08 09:27:20,687][53885] Updated weights for policy 1, policy_version 41362 (0.0007) [2023-10-08 09:27:21,058][53885] Updated weights for policy 1, policy_version 41372 (0.0007) [2023-10-08 09:27:21,154][53852] Updated weights for policy 0, policy_version 41570 (0.0009) [2023-10-08 09:27:21,528][53852] Updated weights for policy 0, policy_version 41580 (0.0008) [2023-10-08 09:27:21,892][53852] Updated weights for policy 0, policy_version 41590 (0.0009) [2023-10-08 09:27:22,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84934656. Throughput: 0: 1833.1, 1: 1831.7. Samples: 21246410. Policy #0 lag: (min: 17.0, avg: 29.7, max: 49.0) [2023-10-08 09:27:22,016][52710] Avg episode reward: [(0, '31.540'), (1, '24.630')] [2023-10-08 09:27:22,261][53852] Updated weights for policy 0, policy_version 41600 (0.0008) [2023-10-08 09:27:24,654][53885] Updated weights for policy 1, policy_version 41382 (0.0009) [2023-10-08 09:27:25,023][53885] Updated weights for policy 1, policy_version 41392 (0.0007) [2023-10-08 09:27:25,393][53885] Updated weights for policy 1, policy_version 41402 (0.0009) [2023-10-08 09:27:25,889][53852] Updated weights for policy 0, policy_version 41610 (0.0009) [2023-10-08 09:27:26,264][53852] Updated weights for policy 0, policy_version 41620 (0.0008) [2023-10-08 09:27:26,632][53852] Updated weights for policy 0, policy_version 41630 (0.0007) [2023-10-08 09:27:27,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 85032960. Throughput: 0: 1845.0, 1: 1825.2. Samples: 21258288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:27:27,016][52710] Avg episode reward: [(0, '27.620'), (1, '22.960')] [2023-10-08 09:27:28,977][53885] Updated weights for policy 1, policy_version 41412 (0.0007) [2023-10-08 09:27:29,350][53885] Updated weights for policy 1, policy_version 41422 (0.0008) [2023-10-08 09:27:29,706][53885] Updated weights for policy 1, policy_version 41432 (0.0009) [2023-10-08 09:27:30,274][53852] Updated weights for policy 0, policy_version 41640 (0.0009) [2023-10-08 09:27:30,632][53852] Updated weights for policy 0, policy_version 41650 (0.0008) [2023-10-08 09:27:31,011][53852] Updated weights for policy 0, policy_version 41660 (0.0011) [2023-10-08 09:27:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85098496. Throughput: 0: 1829.0, 1: 1839.7. Samples: 21279562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:27:32,016][52710] Avg episode reward: [(0, '28.620'), (1, '26.670')] [2023-10-08 09:27:33,354][53885] Updated weights for policy 1, policy_version 41442 (0.0008) [2023-10-08 09:27:33,725][53885] Updated weights for policy 1, policy_version 41452 (0.0007) [2023-10-08 09:27:34,093][53885] Updated weights for policy 1, policy_version 41462 (0.0010) [2023-10-08 09:27:34,462][53885] Updated weights for policy 1, policy_version 41472 (0.0009) [2023-10-08 09:27:34,663][53852] Updated weights for policy 0, policy_version 41670 (0.0009) [2023-10-08 09:27:35,028][53852] Updated weights for policy 0, policy_version 41680 (0.0007) [2023-10-08 09:27:35,406][53852] Updated weights for policy 0, policy_version 41690 (0.0007) [2023-10-08 09:27:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85164032. Throughput: 0: 1848.5, 1: 1846.5. Samples: 21302162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:27:37,016][52710] Avg episode reward: [(0, '32.640'), (1, '25.860')] [2023-10-08 09:27:38,097][53885] Updated weights for policy 1, policy_version 41482 (0.0008) [2023-10-08 09:27:38,457][53885] Updated weights for policy 1, policy_version 41492 (0.0008) [2023-10-08 09:27:38,831][53885] Updated weights for policy 1, policy_version 41502 (0.0007) [2023-10-08 09:27:38,948][53852] Updated weights for policy 0, policy_version 41700 (0.0008) [2023-10-08 09:27:39,325][53852] Updated weights for policy 0, policy_version 41710 (0.0008) [2023-10-08 09:27:39,701][53852] Updated weights for policy 0, policy_version 41720 (0.0007) [2023-10-08 09:27:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85229568. Throughput: 0: 1835.7, 1: 1843.7. Samples: 21312730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:27:42,016][52710] Avg episode reward: [(0, '32.280'), (1, '26.740')] [2023-10-08 09:27:42,493][53885] Updated weights for policy 1, policy_version 41512 (0.0009) [2023-10-08 09:27:42,868][53885] Updated weights for policy 1, policy_version 41522 (0.0009) [2023-10-08 09:27:43,241][53885] Updated weights for policy 1, policy_version 41532 (0.0008) [2023-10-08 09:27:43,436][53852] Updated weights for policy 0, policy_version 41730 (0.0007) [2023-10-08 09:27:43,795][53852] Updated weights for policy 0, policy_version 41740 (0.0008) [2023-10-08 09:27:44,155][53852] Updated weights for policy 0, policy_version 41750 (0.0007) [2023-10-08 09:27:44,525][53852] Updated weights for policy 0, policy_version 41760 (0.0008) [2023-10-08 09:27:46,794][53885] Updated weights for policy 1, policy_version 41542 (0.0010) [2023-10-08 09:27:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 85295104. Throughput: 0: 1838.3, 1: 1841.5. Samples: 21334868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:27:47,016][52710] Avg episode reward: [(0, '32.860'), (1, '25.730')] [2023-10-08 09:27:47,165][53885] Updated weights for policy 1, policy_version 41552 (0.0009) [2023-10-08 09:27:47,531][53885] Updated weights for policy 1, policy_version 41562 (0.0008) [2023-10-08 09:27:48,259][53852] Updated weights for policy 0, policy_version 41770 (0.0010) [2023-10-08 09:27:48,625][53852] Updated weights for policy 0, policy_version 41780 (0.0009) [2023-10-08 09:27:48,995][53852] Updated weights for policy 0, policy_version 41790 (0.0010) [2023-10-08 09:27:51,434][53885] Updated weights for policy 1, policy_version 41572 (0.0009) [2023-10-08 09:27:51,814][53885] Updated weights for policy 1, policy_version 41582 (0.0009) [2023-10-08 09:27:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 85360640. Throughput: 0: 1832.8, 1: 1838.0. Samples: 21357428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:27:52,016][52710] Avg episode reward: [(0, '31.360'), (1, '27.810')] [2023-10-08 09:27:52,170][53885] Updated weights for policy 1, policy_version 41592 (0.0010) [2023-10-08 09:27:52,665][53852] Updated weights for policy 0, policy_version 41800 (0.0007) [2023-10-08 09:27:53,039][53852] Updated weights for policy 0, policy_version 41810 (0.0009) [2023-10-08 09:27:53,418][53852] Updated weights for policy 0, policy_version 41820 (0.0008) [2023-10-08 09:27:55,911][53885] Updated weights for policy 1, policy_version 41602 (0.0008) [2023-10-08 09:27:56,276][53885] Updated weights for policy 1, policy_version 41612 (0.0009) [2023-10-08 09:27:56,647][53885] Updated weights for policy 1, policy_version 41622 (0.0007) [2023-10-08 09:27:57,015][53885] Updated weights for policy 1, policy_version 41632 (0.0009) [2023-10-08 09:27:57,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85458944. Throughput: 0: 1835.1, 1: 1851.6. Samples: 21367966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:27:57,016][52710] Avg episode reward: [(0, '29.690'), (1, '28.990')] [2023-10-08 09:27:57,094][53852] Updated weights for policy 0, policy_version 41830 (0.0007) [2023-10-08 09:27:57,476][53852] Updated weights for policy 0, policy_version 41840 (0.0008) [2023-10-08 09:27:57,857][53852] Updated weights for policy 0, policy_version 41850 (0.0007) [2023-10-08 09:28:00,787][53885] Updated weights for policy 1, policy_version 41642 (0.0011) [2023-10-08 09:28:01,159][53885] Updated weights for policy 1, policy_version 41652 (0.0011) [2023-10-08 09:28:01,524][53885] Updated weights for policy 1, policy_version 41662 (0.0009) [2023-10-08 09:28:01,591][53852] Updated weights for policy 0, policy_version 41860 (0.0007) [2023-10-08 09:28:01,971][53852] Updated weights for policy 0, policy_version 41870 (0.0007) [2023-10-08 09:28:02,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85524480. Throughput: 0: 1833.3, 1: 1841.6. Samples: 21390570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:02,015][52710] Avg episode reward: [(0, '30.900'), (1, '25.950')] [2023-10-08 09:28:02,334][53852] Updated weights for policy 0, policy_version 41880 (0.0007) [2023-10-08 09:28:05,102][53885] Updated weights for policy 1, policy_version 41672 (0.0007) [2023-10-08 09:28:05,479][53885] Updated weights for policy 1, policy_version 41682 (0.0008) [2023-10-08 09:28:05,843][53885] Updated weights for policy 1, policy_version 41692 (0.0009) [2023-10-08 09:28:05,905][53852] Updated weights for policy 0, policy_version 41890 (0.0007) [2023-10-08 09:28:06,275][53852] Updated weights for policy 0, policy_version 41900 (0.0010) [2023-10-08 09:28:06,658][53852] Updated weights for policy 0, policy_version 41910 (0.0009) [2023-10-08 09:28:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85590016. Throughput: 0: 1826.4, 1: 1841.1. Samples: 21411446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:07,016][52710] Avg episode reward: [(0, '30.570'), (1, '28.020')] [2023-10-08 09:28:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000041920_42926080.pth... [2023-10-08 09:28:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000041696_42696704.pth... [2023-10-08 09:28:07,029][53852] Updated weights for policy 0, policy_version 41920 (0.0010) [2023-10-08 09:28:07,064][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000040192_41156608.pth [2023-10-08 09:28:07,067][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000039968_40927232.pth [2023-10-08 09:28:09,492][53885] Updated weights for policy 1, policy_version 41702 (0.0009) [2023-10-08 09:28:09,861][53885] Updated weights for policy 1, policy_version 41712 (0.0011) [2023-10-08 09:28:10,228][53885] Updated weights for policy 1, policy_version 41722 (0.0011) [2023-10-08 09:28:10,659][53852] Updated weights for policy 0, policy_version 41930 (0.0009) [2023-10-08 09:28:11,032][53852] Updated weights for policy 0, policy_version 41940 (0.0010) [2023-10-08 09:28:11,413][53852] Updated weights for policy 0, policy_version 41950 (0.0010) [2023-10-08 09:28:12,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 85688320. Throughput: 0: 1831.1, 1: 1835.1. Samples: 21423266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:12,016][52710] Avg episode reward: [(0, '29.650'), (1, '25.330')] [2023-10-08 09:28:13,633][53885] Updated weights for policy 1, policy_version 41732 (0.0007) [2023-10-08 09:28:13,998][53885] Updated weights for policy 1, policy_version 41742 (0.0009) [2023-10-08 09:28:14,364][53885] Updated weights for policy 1, policy_version 41752 (0.0007) [2023-10-08 09:28:15,005][53852] Updated weights for policy 0, policy_version 41960 (0.0007) [2023-10-08 09:28:15,371][53852] Updated weights for policy 0, policy_version 41970 (0.0009) [2023-10-08 09:28:15,744][53852] Updated weights for policy 0, policy_version 41980 (0.0009) [2023-10-08 09:28:17,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85753856. Throughput: 0: 1824.1, 1: 1838.2. Samples: 21444362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:17,015][52710] Avg episode reward: [(0, '32.140'), (1, '30.430')] [2023-10-08 09:28:17,995][53885] Updated weights for policy 1, policy_version 41762 (0.0009) [2023-10-08 09:28:18,368][53885] Updated weights for policy 1, policy_version 41772 (0.0008) [2023-10-08 09:28:18,738][53885] Updated weights for policy 1, policy_version 41782 (0.0009) [2023-10-08 09:28:19,103][53885] Updated weights for policy 1, policy_version 41792 (0.0009) [2023-10-08 09:28:19,355][53852] Updated weights for policy 0, policy_version 41990 (0.0009) [2023-10-08 09:28:19,725][53852] Updated weights for policy 0, policy_version 42000 (0.0008) [2023-10-08 09:28:20,109][53852] Updated weights for policy 0, policy_version 42010 (0.0007) [2023-10-08 09:28:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 85819392. Throughput: 0: 1823.3, 1: 1833.4. Samples: 21466716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:22,015][52710] Avg episode reward: [(0, '29.450'), (1, '29.970')] [2023-10-08 09:28:22,780][53885] Updated weights for policy 1, policy_version 41802 (0.0008) [2023-10-08 09:28:23,143][53885] Updated weights for policy 1, policy_version 41812 (0.0008) [2023-10-08 09:28:23,511][53885] Updated weights for policy 1, policy_version 41822 (0.0009) [2023-10-08 09:28:23,873][53852] Updated weights for policy 0, policy_version 42020 (0.0009) [2023-10-08 09:28:24,244][53852] Updated weights for policy 0, policy_version 42030 (0.0009) [2023-10-08 09:28:24,610][53852] Updated weights for policy 0, policy_version 42040 (0.0009) [2023-10-08 09:28:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 85884928. Throughput: 0: 1819.8, 1: 1838.4. Samples: 21477350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:27,015][52710] Avg episode reward: [(0, '30.040'), (1, '30.710')] [2023-10-08 09:28:27,218][53885] Updated weights for policy 1, policy_version 41832 (0.0010) [2023-10-08 09:28:27,581][53885] Updated weights for policy 1, policy_version 41842 (0.0009) [2023-10-08 09:28:27,953][53885] Updated weights for policy 1, policy_version 41852 (0.0007) [2023-10-08 09:28:28,106][53852] Updated weights for policy 0, policy_version 42050 (0.0008) [2023-10-08 09:28:28,467][53852] Updated weights for policy 0, policy_version 42060 (0.0008) [2023-10-08 09:28:28,842][53852] Updated weights for policy 0, policy_version 42070 (0.0010) [2023-10-08 09:28:29,217][53852] Updated weights for policy 0, policy_version 42080 (0.0007) [2023-10-08 09:28:31,626][53885] Updated weights for policy 1, policy_version 41862 (0.0008) [2023-10-08 09:28:31,994][53885] Updated weights for policy 1, policy_version 41872 (0.0009) [2023-10-08 09:28:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 85950464. Throughput: 0: 1828.1, 1: 1837.4. Samples: 21499814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:32,016][52710] Avg episode reward: [(0, '30.280'), (1, '27.730')] [2023-10-08 09:28:32,362][53885] Updated weights for policy 1, policy_version 41882 (0.0010) [2023-10-08 09:28:32,950][53852] Updated weights for policy 0, policy_version 42090 (0.0010) [2023-10-08 09:28:33,322][53852] Updated weights for policy 0, policy_version 42100 (0.0010) [2023-10-08 09:28:33,682][53852] Updated weights for policy 0, policy_version 42110 (0.0009) [2023-10-08 09:28:35,938][53885] Updated weights for policy 1, policy_version 41892 (0.0009) [2023-10-08 09:28:36,311][53885] Updated weights for policy 1, policy_version 41902 (0.0008) [2023-10-08 09:28:36,678][53885] Updated weights for policy 1, policy_version 41912 (0.0009) [2023-10-08 09:28:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 86048768. Throughput: 0: 1829.2, 1: 1829.8. Samples: 21522080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:37,015][52710] Avg episode reward: [(0, '31.910'), (1, '28.290')] [2023-10-08 09:28:37,252][53852] Updated weights for policy 0, policy_version 42120 (0.0008) [2023-10-08 09:28:37,611][53852] Updated weights for policy 0, policy_version 42130 (0.0010) [2023-10-08 09:28:37,973][53852] Updated weights for policy 0, policy_version 42140 (0.0010) [2023-10-08 09:28:40,394][53885] Updated weights for policy 1, policy_version 41922 (0.0007) [2023-10-08 09:28:40,766][53885] Updated weights for policy 1, policy_version 41932 (0.0009) [2023-10-08 09:28:41,129][53885] Updated weights for policy 1, policy_version 41942 (0.0007) [2023-10-08 09:28:41,497][53885] Updated weights for policy 1, policy_version 41952 (0.0007) [2023-10-08 09:28:41,640][53852] Updated weights for policy 0, policy_version 42150 (0.0009) [2023-10-08 09:28:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86114304. Throughput: 0: 1831.1, 1: 1837.7. Samples: 21533062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:42,015][52710] Avg episode reward: [(0, '30.280'), (1, '28.500')] [2023-10-08 09:28:42,016][53852] Updated weights for policy 0, policy_version 42160 (0.0007) [2023-10-08 09:28:42,394][53852] Updated weights for policy 0, policy_version 42170 (0.0007) [2023-10-08 09:28:45,145][53885] Updated weights for policy 1, policy_version 41962 (0.0011) [2023-10-08 09:28:45,507][53885] Updated weights for policy 1, policy_version 41972 (0.0009) [2023-10-08 09:28:45,875][53885] Updated weights for policy 1, policy_version 41982 (0.0007) [2023-10-08 09:28:46,176][53852] Updated weights for policy 0, policy_version 42180 (0.0010) [2023-10-08 09:28:46,546][53852] Updated weights for policy 0, policy_version 42190 (0.0010) [2023-10-08 09:28:46,921][53852] Updated weights for policy 0, policy_version 42200 (0.0009) [2023-10-08 09:28:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86179840. Throughput: 0: 1834.6, 1: 1824.2. Samples: 21555218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:47,016][52710] Avg episode reward: [(0, '33.180'), (1, '27.080')] [2023-10-08 09:28:49,520][53885] Updated weights for policy 1, policy_version 41992 (0.0008) [2023-10-08 09:28:49,893][53885] Updated weights for policy 1, policy_version 42002 (0.0008) [2023-10-08 09:28:50,257][53885] Updated weights for policy 1, policy_version 42012 (0.0008) [2023-10-08 09:28:50,520][53852] Updated weights for policy 0, policy_version 42210 (0.0009) [2023-10-08 09:28:50,899][53852] Updated weights for policy 0, policy_version 42220 (0.0008) [2023-10-08 09:28:51,264][53852] Updated weights for policy 0, policy_version 42230 (0.0007) [2023-10-08 09:28:51,632][53852] Updated weights for policy 0, policy_version 42240 (0.0008) [2023-10-08 09:28:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 86278144. Throughput: 0: 1825.2, 1: 1840.4. Samples: 21576400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:52,016][52710] Avg episode reward: [(0, '31.670'), (1, '25.640')] [2023-10-08 09:28:53,978][53885] Updated weights for policy 1, policy_version 42022 (0.0008) [2023-10-08 09:28:54,336][53885] Updated weights for policy 1, policy_version 42032 (0.0009) [2023-10-08 09:28:54,708][53885] Updated weights for policy 1, policy_version 42042 (0.0008) [2023-10-08 09:28:55,232][53852] Updated weights for policy 0, policy_version 42250 (0.0010) [2023-10-08 09:28:55,598][53852] Updated weights for policy 0, policy_version 42260 (0.0010) [2023-10-08 09:28:55,958][53852] Updated weights for policy 0, policy_version 42270 (0.0010) [2023-10-08 09:28:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86343680. Throughput: 0: 1838.5, 1: 1825.6. Samples: 21588152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:28:57,016][52710] Avg episode reward: [(0, '31.290'), (1, '30.300')] [2023-10-08 09:28:58,424][53885] Updated weights for policy 1, policy_version 42052 (0.0009) [2023-10-08 09:28:58,803][53885] Updated weights for policy 1, policy_version 42062 (0.0009) [2023-10-08 09:28:59,168][53885] Updated weights for policy 1, policy_version 42072 (0.0009) [2023-10-08 09:28:59,596][53852] Updated weights for policy 0, policy_version 42280 (0.0007) [2023-10-08 09:28:59,971][53852] Updated weights for policy 0, policy_version 42290 (0.0007) [2023-10-08 09:29:00,350][53852] Updated weights for policy 0, policy_version 42300 (0.0009) [2023-10-08 09:29:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 86409216. Throughput: 0: 1825.6, 1: 1837.5. Samples: 21609204. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-08 09:29:02,016][52710] Avg episode reward: [(0, '31.470'), (1, '28.620')] [2023-10-08 09:29:02,836][53885] Updated weights for policy 1, policy_version 42082 (0.0008) [2023-10-08 09:29:03,209][53885] Updated weights for policy 1, policy_version 42092 (0.0008) [2023-10-08 09:29:03,574][53885] Updated weights for policy 1, policy_version 42102 (0.0007) [2023-10-08 09:29:03,940][53885] Updated weights for policy 1, policy_version 42112 (0.0009) [2023-10-08 09:29:04,018][53852] Updated weights for policy 0, policy_version 42310 (0.0009) [2023-10-08 09:29:04,383][53852] Updated weights for policy 0, policy_version 42320 (0.0009) [2023-10-08 09:29:04,759][53852] Updated weights for policy 0, policy_version 42330 (0.0008) [2023-10-08 09:29:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86474752. Throughput: 0: 1837.9, 1: 1839.1. Samples: 21632186. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-08 09:29:07,016][52710] Avg episode reward: [(0, '30.470'), (1, '29.690')] [2023-10-08 09:29:07,465][53885] Updated weights for policy 1, policy_version 42122 (0.0010) [2023-10-08 09:29:07,832][53885] Updated weights for policy 1, policy_version 42132 (0.0010) [2023-10-08 09:29:08,195][53885] Updated weights for policy 1, policy_version 42142 (0.0007) [2023-10-08 09:29:08,405][53852] Updated weights for policy 0, policy_version 42340 (0.0008) [2023-10-08 09:29:08,783][53852] Updated weights for policy 0, policy_version 42350 (0.0010) [2023-10-08 09:29:09,151][53852] Updated weights for policy 0, policy_version 42360 (0.0010) [2023-10-08 09:29:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 86540288. Throughput: 0: 1826.3, 1: 1838.1. Samples: 21642248. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-08 09:29:12,016][52710] Avg episode reward: [(0, '30.460'), (1, '29.090')] [2023-10-08 09:29:12,030][53885] Updated weights for policy 1, policy_version 42152 (0.0008) [2023-10-08 09:29:12,393][53885] Updated weights for policy 1, policy_version 42162 (0.0008) [2023-10-08 09:29:12,756][53885] Updated weights for policy 1, policy_version 42172 (0.0007) [2023-10-08 09:29:12,954][53852] Updated weights for policy 0, policy_version 42370 (0.0009) [2023-10-08 09:29:13,338][53852] Updated weights for policy 0, policy_version 42380 (0.0009) [2023-10-08 09:29:13,700][53852] Updated weights for policy 0, policy_version 42390 (0.0008) [2023-10-08 09:29:14,077][53852] Updated weights for policy 0, policy_version 42400 (0.0009) [2023-10-08 09:29:16,461][53885] Updated weights for policy 1, policy_version 42182 (0.0007) [2023-10-08 09:29:16,832][53885] Updated weights for policy 1, policy_version 42192 (0.0009) [2023-10-08 09:29:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 86605824. Throughput: 0: 1832.0, 1: 1831.4. Samples: 21664670. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-08 09:29:17,016][52710] Avg episode reward: [(0, '31.330'), (1, '28.240')] [2023-10-08 09:29:17,203][53885] Updated weights for policy 1, policy_version 42202 (0.0010) [2023-10-08 09:29:17,806][53852] Updated weights for policy 0, policy_version 42410 (0.0007) [2023-10-08 09:29:18,172][53852] Updated weights for policy 0, policy_version 42420 (0.0009) [2023-10-08 09:29:18,552][53852] Updated weights for policy 0, policy_version 42430 (0.0010) [2023-10-08 09:29:20,925][53885] Updated weights for policy 1, policy_version 42212 (0.0009) [2023-10-08 09:29:21,288][53885] Updated weights for policy 1, policy_version 42222 (0.0010) [2023-10-08 09:29:21,661][53885] Updated weights for policy 1, policy_version 42232 (0.0010) [2023-10-08 09:29:22,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86704128. Throughput: 0: 1832.0, 1: 1816.4. Samples: 21686258. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-08 09:29:22,016][52710] Avg episode reward: [(0, '32.190'), (1, '28.310')] [2023-10-08 09:29:22,076][53852] Updated weights for policy 0, policy_version 42440 (0.0008) [2023-10-08 09:29:22,442][53852] Updated weights for policy 0, policy_version 42450 (0.0010) [2023-10-08 09:29:22,809][53852] Updated weights for policy 0, policy_version 42460 (0.0008) [2023-10-08 09:29:25,510][53885] Updated weights for policy 1, policy_version 42242 (0.0008) [2023-10-08 09:29:25,867][53885] Updated weights for policy 1, policy_version 42252 (0.0009) [2023-10-08 09:29:26,242][53885] Updated weights for policy 1, policy_version 42262 (0.0010) [2023-10-08 09:29:26,491][53852] Updated weights for policy 0, policy_version 42470 (0.0009) [2023-10-08 09:29:26,601][53885] Updated weights for policy 1, policy_version 42272 (0.0009) [2023-10-08 09:29:26,865][53852] Updated weights for policy 0, policy_version 42480 (0.0007) [2023-10-08 09:29:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86769664. Throughput: 0: 1827.1, 1: 1815.6. Samples: 21696988. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-08 09:29:27,016][52710] Avg episode reward: [(0, '28.840'), (1, '28.940')] [2023-10-08 09:29:27,244][53852] Updated weights for policy 0, policy_version 42490 (0.0009) [2023-10-08 09:29:30,466][53885] Updated weights for policy 1, policy_version 42282 (0.0009) [2023-10-08 09:29:30,840][53885] Updated weights for policy 1, policy_version 42292 (0.0010) [2023-10-08 09:29:31,070][53852] Updated weights for policy 0, policy_version 42500 (0.0008) [2023-10-08 09:29:31,214][53885] Updated weights for policy 1, policy_version 42302 (0.0008) [2023-10-08 09:29:31,465][53852] Updated weights for policy 0, policy_version 42510 (0.0008) [2023-10-08 09:29:31,833][53852] Updated weights for policy 0, policy_version 42520 (0.0009) [2023-10-08 09:29:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 86835200. Throughput: 0: 1822.4, 1: 1815.3. Samples: 21718914. Policy #0 lag: (min: 9.0, avg: 25.0, max: 41.0) [2023-10-08 09:29:32,016][52710] Avg episode reward: [(0, '32.920'), (1, '25.640')] [2023-10-08 09:29:34,984][53885] Updated weights for policy 1, policy_version 42312 (0.0008) [2023-10-08 09:29:35,345][53885] Updated weights for policy 1, policy_version 42322 (0.0007) [2023-10-08 09:29:35,554][53852] Updated weights for policy 0, policy_version 42530 (0.0009) [2023-10-08 09:29:35,717][53885] Updated weights for policy 1, policy_version 42332 (0.0008) [2023-10-08 09:29:35,922][53852] Updated weights for policy 0, policy_version 42540 (0.0009) [2023-10-08 09:29:36,307][53852] Updated weights for policy 0, policy_version 42550 (0.0009) [2023-10-08 09:29:36,678][53852] Updated weights for policy 0, policy_version 42560 (0.0008) [2023-10-08 09:29:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 86933504. Throughput: 0: 1815.4, 1: 1798.5. Samples: 21739028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:29:37,016][52710] Avg episode reward: [(0, '29.770'), (1, '25.980')] [2023-10-08 09:29:39,279][53885] Updated weights for policy 1, policy_version 42342 (0.0008) [2023-10-08 09:29:39,652][53885] Updated weights for policy 1, policy_version 42352 (0.0008) [2023-10-08 09:29:40,033][53885] Updated weights for policy 1, policy_version 42362 (0.0009) [2023-10-08 09:29:40,329][53852] Updated weights for policy 0, policy_version 42570 (0.0009) [2023-10-08 09:29:40,682][53852] Updated weights for policy 0, policy_version 42580 (0.0009) [2023-10-08 09:29:41,056][53852] Updated weights for policy 0, policy_version 42590 (0.0010) [2023-10-08 09:29:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 86999040. Throughput: 0: 1813.5, 1: 1809.5. Samples: 21751184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:29:42,016][52710] Avg episode reward: [(0, '30.850'), (1, '23.690')] [2023-10-08 09:29:43,623][53885] Updated weights for policy 1, policy_version 42372 (0.0010) [2023-10-08 09:29:43,985][53885] Updated weights for policy 1, policy_version 42382 (0.0008) [2023-10-08 09:29:44,338][53885] Updated weights for policy 1, policy_version 42392 (0.0007) [2023-10-08 09:29:44,704][53852] Updated weights for policy 0, policy_version 42600 (0.0007) [2023-10-08 09:29:45,073][53852] Updated weights for policy 0, policy_version 42610 (0.0008) [2023-10-08 09:29:45,445][53852] Updated weights for policy 0, policy_version 42620 (0.0010) [2023-10-08 09:29:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87064576. Throughput: 0: 1819.9, 1: 1802.5. Samples: 21772210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:29:47,016][52710] Avg episode reward: [(0, '33.350'), (1, '25.990')] [2023-10-08 09:29:48,032][53885] Updated weights for policy 1, policy_version 42402 (0.0008) [2023-10-08 09:29:48,435][53885] Updated weights for policy 1, policy_version 42412 (0.0010) [2023-10-08 09:29:48,810][53885] Updated weights for policy 1, policy_version 42422 (0.0011) [2023-10-08 09:29:48,993][53852] Updated weights for policy 0, policy_version 42630 (0.0010) [2023-10-08 09:29:49,171][53885] Updated weights for policy 1, policy_version 42432 (0.0010) [2023-10-08 09:29:49,363][53852] Updated weights for policy 0, policy_version 42640 (0.0009) [2023-10-08 09:29:49,726][53852] Updated weights for policy 0, policy_version 42650 (0.0007) [2023-10-08 09:29:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 87130112. Throughput: 0: 1819.7, 1: 1798.3. Samples: 21794994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:29:52,015][52710] Avg episode reward: [(0, '33.080'), (1, '27.990')] [2023-10-08 09:29:52,880][53885] Updated weights for policy 1, policy_version 42442 (0.0007) [2023-10-08 09:29:53,241][53885] Updated weights for policy 1, policy_version 42452 (0.0007) [2023-10-08 09:29:53,323][53852] Updated weights for policy 0, policy_version 42660 (0.0008) [2023-10-08 09:29:53,609][53885] Updated weights for policy 1, policy_version 42462 (0.0007) [2023-10-08 09:29:53,699][53852] Updated weights for policy 0, policy_version 42670 (0.0007) [2023-10-08 09:29:54,064][53852] Updated weights for policy 0, policy_version 42680 (0.0009) [2023-10-08 09:29:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 87195648. Throughput: 0: 1824.8, 1: 1796.3. Samples: 21805198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:29:57,016][52710] Avg episode reward: [(0, '29.400'), (1, '29.710')] [2023-10-08 09:29:57,335][53885] Updated weights for policy 1, policy_version 42472 (0.0007) [2023-10-08 09:29:57,702][53852] Updated weights for policy 0, policy_version 42690 (0.0009) [2023-10-08 09:29:57,706][53885] Updated weights for policy 1, policy_version 42482 (0.0009) [2023-10-08 09:29:58,064][53885] Updated weights for policy 1, policy_version 42492 (0.0007) [2023-10-08 09:29:58,077][53852] Updated weights for policy 0, policy_version 42700 (0.0007) [2023-10-08 09:29:58,439][53852] Updated weights for policy 0, policy_version 42710 (0.0008) [2023-10-08 09:29:58,804][53852] Updated weights for policy 0, policy_version 42720 (0.0007) [2023-10-08 09:30:01,925][53885] Updated weights for policy 1, policy_version 42502 (0.0008) [2023-10-08 09:30:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87261184. Throughput: 0: 1832.5, 1: 1801.7. Samples: 21828210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:30:02,016][52710] Avg episode reward: [(0, '32.750'), (1, '27.570')] [2023-10-08 09:30:02,294][53885] Updated weights for policy 1, policy_version 42512 (0.0007) [2023-10-08 09:30:02,591][53852] Updated weights for policy 0, policy_version 42730 (0.0009) [2023-10-08 09:30:02,655][53885] Updated weights for policy 1, policy_version 42522 (0.0008) [2023-10-08 09:30:02,964][53852] Updated weights for policy 0, policy_version 42740 (0.0008) [2023-10-08 09:30:03,343][53852] Updated weights for policy 0, policy_version 42750 (0.0008) [2023-10-08 09:30:06,296][53885] Updated weights for policy 1, policy_version 42532 (0.0008) [2023-10-08 09:30:06,669][53885] Updated weights for policy 1, policy_version 42542 (0.0007) [2023-10-08 09:30:06,962][53852] Updated weights for policy 0, policy_version 42760 (0.0008) [2023-10-08 09:30:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87326720. Throughput: 0: 1833.7, 1: 1818.9. Samples: 21850624. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-08 09:30:07,016][52710] Avg episode reward: [(0, '32.350'), (1, '31.550')] [2023-10-08 09:30:07,037][53885] Updated weights for policy 1, policy_version 42552 (0.0007) [2023-10-08 09:30:07,332][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000042560_43581440.pth... [2023-10-08 09:30:07,333][53852] Updated weights for policy 0, policy_version 42770 (0.0008) [2023-10-08 09:30:07,361][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000040832_41811968.pth [2023-10-08 09:30:07,365][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000042560_43581440.pth [2023-10-08 09:30:07,695][53852] Updated weights for policy 0, policy_version 42780 (0.0009) [2023-10-08 09:30:07,842][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000042784_43810816.pth... [2023-10-08 09:30:07,870][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000041056_42041344.pth [2023-10-08 09:30:07,874][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000042784_43810816.pth [2023-10-08 09:30:10,823][53885] Updated weights for policy 1, policy_version 42562 (0.0008) [2023-10-08 09:30:11,194][53885] Updated weights for policy 1, policy_version 42572 (0.0007) [2023-10-08 09:30:11,339][53852] Updated weights for policy 0, policy_version 42790 (0.0007) [2023-10-08 09:30:11,555][53885] Updated weights for policy 1, policy_version 42582 (0.0008) [2023-10-08 09:30:11,711][53852] Updated weights for policy 0, policy_version 42800 (0.0009) [2023-10-08 09:30:11,918][53885] Updated weights for policy 1, policy_version 42592 (0.0008) [2023-10-08 09:30:12,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87425024. Throughput: 0: 1836.0, 1: 1806.8. Samples: 21860914. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-08 09:30:12,016][52710] Avg episode reward: [(0, '29.200'), (1, '29.660')] [2023-10-08 09:30:12,081][53852] Updated weights for policy 0, policy_version 42810 (0.0010) [2023-10-08 09:30:15,500][53885] Updated weights for policy 1, policy_version 42602 (0.0009) [2023-10-08 09:30:15,833][53852] Updated weights for policy 0, policy_version 42820 (0.0010) [2023-10-08 09:30:15,868][53885] Updated weights for policy 1, policy_version 42612 (0.0008) [2023-10-08 09:30:16,213][53852] Updated weights for policy 0, policy_version 42830 (0.0007) [2023-10-08 09:30:16,237][53885] Updated weights for policy 1, policy_version 42622 (0.0007) [2023-10-08 09:30:16,582][53852] Updated weights for policy 0, policy_version 42840 (0.0010) [2023-10-08 09:30:17,015][52710] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 87523328. Throughput: 0: 1833.3, 1: 1814.0. Samples: 21883042. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-08 09:30:17,015][52710] Avg episode reward: [(0, '30.940'), (1, '26.370')] [2023-10-08 09:30:19,921][53885] Updated weights for policy 1, policy_version 42632 (0.0010) [2023-10-08 09:30:20,285][53885] Updated weights for policy 1, policy_version 42642 (0.0011) [2023-10-08 09:30:20,320][53852] Updated weights for policy 0, policy_version 42850 (0.0009) [2023-10-08 09:30:20,650][53885] Updated weights for policy 1, policy_version 42652 (0.0007) [2023-10-08 09:30:20,686][53852] Updated weights for policy 0, policy_version 42860 (0.0009) [2023-10-08 09:30:21,063][53852] Updated weights for policy 0, policy_version 42870 (0.0008) [2023-10-08 09:30:21,426][53852] Updated weights for policy 0, policy_version 42880 (0.0007) [2023-10-08 09:30:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87588864. Throughput: 0: 1832.9, 1: 1819.3. Samples: 21903378. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-08 09:30:22,016][52710] Avg episode reward: [(0, '27.650'), (1, '30.330')] [2023-10-08 09:30:24,291][53885] Updated weights for policy 1, policy_version 42662 (0.0009) [2023-10-08 09:30:24,656][53885] Updated weights for policy 1, policy_version 42672 (0.0007) [2023-10-08 09:30:25,030][53885] Updated weights for policy 1, policy_version 42682 (0.0008) [2023-10-08 09:30:25,136][53852] Updated weights for policy 0, policy_version 42890 (0.0008) [2023-10-08 09:30:25,511][53852] Updated weights for policy 0, policy_version 42900 (0.0009) [2023-10-08 09:30:25,875][53852] Updated weights for policy 0, policy_version 42910 (0.0009) [2023-10-08 09:30:27,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87654400. Throughput: 0: 1836.2, 1: 1820.4. Samples: 21915732. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-08 09:30:27,016][52710] Avg episode reward: [(0, '29.640'), (1, '30.960')] [2023-10-08 09:30:28,742][53885] Updated weights for policy 1, policy_version 42692 (0.0009) [2023-10-08 09:30:29,118][53885] Updated weights for policy 1, policy_version 42702 (0.0007) [2023-10-08 09:30:29,478][53885] Updated weights for policy 1, policy_version 42712 (0.0008) [2023-10-08 09:30:29,674][53852] Updated weights for policy 0, policy_version 42920 (0.0009) [2023-10-08 09:30:30,048][53852] Updated weights for policy 0, policy_version 42930 (0.0009) [2023-10-08 09:30:30,407][53852] Updated weights for policy 0, policy_version 42940 (0.0008) [2023-10-08 09:30:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87719936. Throughput: 0: 1825.3, 1: 1818.7. Samples: 21936192. Policy #0 lag: (min: 31.0, avg: 36.9, max: 63.0) [2023-10-08 09:30:32,016][52710] Avg episode reward: [(0, '29.120'), (1, '29.810')] [2023-10-08 09:30:33,199][53885] Updated weights for policy 1, policy_version 42722 (0.0008) [2023-10-08 09:30:33,608][53885] Updated weights for policy 1, policy_version 42732 (0.0008) [2023-10-08 09:30:33,978][53885] Updated weights for policy 1, policy_version 42742 (0.0008) [2023-10-08 09:30:33,997][53852] Updated weights for policy 0, policy_version 42950 (0.0007) [2023-10-08 09:30:34,334][53885] Updated weights for policy 1, policy_version 42752 (0.0007) [2023-10-08 09:30:34,364][53852] Updated weights for policy 0, policy_version 42960 (0.0007) [2023-10-08 09:30:34,740][53852] Updated weights for policy 0, policy_version 42970 (0.0007) [2023-10-08 09:30:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 87785472. Throughput: 0: 1827.0, 1: 1816.7. Samples: 21958962. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:30:37,016][52710] Avg episode reward: [(0, '30.140'), (1, '30.900')] [2023-10-08 09:30:37,969][53885] Updated weights for policy 1, policy_version 42762 (0.0009) [2023-10-08 09:30:38,327][53885] Updated weights for policy 1, policy_version 42772 (0.0010) [2023-10-08 09:30:38,462][53852] Updated weights for policy 0, policy_version 42980 (0.0008) [2023-10-08 09:30:38,693][53885] Updated weights for policy 1, policy_version 42782 (0.0007) [2023-10-08 09:30:38,829][53852] Updated weights for policy 0, policy_version 42990 (0.0007) [2023-10-08 09:30:39,201][53852] Updated weights for policy 0, policy_version 43000 (0.0009) [2023-10-08 09:30:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 87851008. Throughput: 0: 1819.2, 1: 1817.4. Samples: 21968844. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:30:42,016][52710] Avg episode reward: [(0, '32.260'), (1, '31.190')] [2023-10-08 09:30:42,473][53885] Updated weights for policy 1, policy_version 42792 (0.0007) [2023-10-08 09:30:42,837][53885] Updated weights for policy 1, policy_version 42802 (0.0008) [2023-10-08 09:30:42,876][53852] Updated weights for policy 0, policy_version 43010 (0.0009) [2023-10-08 09:30:43,201][53885] Updated weights for policy 1, policy_version 42812 (0.0007) [2023-10-08 09:30:43,250][53852] Updated weights for policy 0, policy_version 43020 (0.0009) [2023-10-08 09:30:43,623][53852] Updated weights for policy 0, policy_version 43030 (0.0008) [2023-10-08 09:30:43,981][53852] Updated weights for policy 0, policy_version 43040 (0.0008) [2023-10-08 09:30:46,844][53885] Updated weights for policy 1, policy_version 42822 (0.0007) [2023-10-08 09:30:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 87916544. Throughput: 0: 1808.3, 1: 1817.4. Samples: 21991368. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:30:47,015][52710] Avg episode reward: [(0, '32.150'), (1, '32.090')] [2023-10-08 09:30:47,204][53885] Updated weights for policy 1, policy_version 42832 (0.0008) [2023-10-08 09:30:47,582][53885] Updated weights for policy 1, policy_version 42842 (0.0009) [2023-10-08 09:30:47,823][53852] Updated weights for policy 0, policy_version 43050 (0.0008) [2023-10-08 09:30:48,198][53852] Updated weights for policy 0, policy_version 43060 (0.0009) [2023-10-08 09:30:48,567][53852] Updated weights for policy 0, policy_version 43070 (0.0009) [2023-10-08 09:30:51,318][53885] Updated weights for policy 1, policy_version 42852 (0.0007) [2023-10-08 09:30:51,684][53885] Updated weights for policy 1, policy_version 42862 (0.0010) [2023-10-08 09:30:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 87982080. Throughput: 0: 1810.1, 1: 1816.4. Samples: 22013816. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:30:52,016][52710] Avg episode reward: [(0, '30.700'), (1, '30.430')] [2023-10-08 09:30:52,053][53885] Updated weights for policy 1, policy_version 42872 (0.0009) [2023-10-08 09:30:52,068][53852] Updated weights for policy 0, policy_version 43080 (0.0008) [2023-10-08 09:30:52,442][53852] Updated weights for policy 0, policy_version 43090 (0.0008) [2023-10-08 09:30:52,816][53852] Updated weights for policy 0, policy_version 43100 (0.0008) [2023-10-08 09:30:55,775][53885] Updated weights for policy 1, policy_version 42882 (0.0009) [2023-10-08 09:30:56,136][53885] Updated weights for policy 1, policy_version 42892 (0.0007) [2023-10-08 09:30:56,507][53885] Updated weights for policy 1, policy_version 42902 (0.0008) [2023-10-08 09:30:56,550][53852] Updated weights for policy 0, policy_version 43110 (0.0008) [2023-10-08 09:30:56,869][53885] Updated weights for policy 1, policy_version 42912 (0.0009) [2023-10-08 09:30:56,920][53852] Updated weights for policy 0, policy_version 43120 (0.0008) [2023-10-08 09:30:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 88080384. Throughput: 0: 1806.7, 1: 1819.9. Samples: 22024112. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:30:57,015][52710] Avg episode reward: [(0, '33.770'), (1, '31.110')] [2023-10-08 09:30:57,289][53852] Updated weights for policy 0, policy_version 43130 (0.0008) [2023-10-08 09:31:00,521][53885] Updated weights for policy 1, policy_version 42922 (0.0009) [2023-10-08 09:31:00,895][53885] Updated weights for policy 1, policy_version 42932 (0.0008) [2023-10-08 09:31:00,981][53852] Updated weights for policy 0, policy_version 43140 (0.0008) [2023-10-08 09:31:01,258][53885] Updated weights for policy 1, policy_version 42942 (0.0009) [2023-10-08 09:31:01,368][53852] Updated weights for policy 0, policy_version 43150 (0.0008) [2023-10-08 09:31:01,735][53852] Updated weights for policy 0, policy_version 43160 (0.0009) [2023-10-08 09:31:02,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 88145920. Throughput: 0: 1811.6, 1: 1821.0. Samples: 22046508. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) [2023-10-08 09:31:02,016][52710] Avg episode reward: [(0, '30.670'), (1, '31.540')] [2023-10-08 09:31:04,977][53885] Updated weights for policy 1, policy_version 42952 (0.0009) [2023-10-08 09:31:05,339][53885] Updated weights for policy 1, policy_version 42962 (0.0009) [2023-10-08 09:31:05,440][53852] Updated weights for policy 0, policy_version 43170 (0.0010) [2023-10-08 09:31:05,706][53885] Updated weights for policy 1, policy_version 42972 (0.0009) [2023-10-08 09:31:05,799][53852] Updated weights for policy 0, policy_version 43180 (0.0009) [2023-10-08 09:31:06,166][53852] Updated weights for policy 0, policy_version 43190 (0.0008) [2023-10-08 09:31:06,536][53852] Updated weights for policy 0, policy_version 43200 (0.0007) [2023-10-08 09:31:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 88244224. Throughput: 0: 1813.3, 1: 1819.3. Samples: 22066846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:31:07,016][52710] Avg episode reward: [(0, '28.900'), (1, '29.770')] [2023-10-08 09:31:09,406][53885] Updated weights for policy 1, policy_version 42982 (0.0008) [2023-10-08 09:31:09,772][53885] Updated weights for policy 1, policy_version 42992 (0.0008) [2023-10-08 09:31:10,126][53885] Updated weights for policy 1, policy_version 43002 (0.0008) [2023-10-08 09:31:10,183][53852] Updated weights for policy 0, policy_version 43210 (0.0008) [2023-10-08 09:31:10,556][53852] Updated weights for policy 0, policy_version 43220 (0.0008) [2023-10-08 09:31:10,927][53852] Updated weights for policy 0, policy_version 43230 (0.0010) [2023-10-08 09:31:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88309760. Throughput: 0: 1812.5, 1: 1821.3. Samples: 22079256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:31:12,016][52710] Avg episode reward: [(0, '30.920'), (1, '32.530')] [2023-10-08 09:31:13,781][53885] Updated weights for policy 1, policy_version 43012 (0.0009) [2023-10-08 09:31:14,147][53885] Updated weights for policy 1, policy_version 43022 (0.0009) [2023-10-08 09:31:14,402][53852] Updated weights for policy 0, policy_version 43240 (0.0007) [2023-10-08 09:31:14,513][53885] Updated weights for policy 1, policy_version 43032 (0.0008) [2023-10-08 09:31:14,781][53852] Updated weights for policy 0, policy_version 43250 (0.0009) [2023-10-08 09:31:15,150][53852] Updated weights for policy 0, policy_version 43260 (0.0008) [2023-10-08 09:31:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 88375296. Throughput: 0: 1819.5, 1: 1818.7. Samples: 22099912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:31:17,016][52710] Avg episode reward: [(0, '29.670'), (1, '31.010')] [2023-10-08 09:31:18,240][53885] Updated weights for policy 1, policy_version 43042 (0.0009) [2023-10-08 09:31:18,640][53885] Updated weights for policy 1, policy_version 43052 (0.0009) [2023-10-08 09:31:18,712][53852] Updated weights for policy 0, policy_version 43270 (0.0008) [2023-10-08 09:31:19,005][53885] Updated weights for policy 1, policy_version 43062 (0.0009) [2023-10-08 09:31:19,085][53852] Updated weights for policy 0, policy_version 43280 (0.0008) [2023-10-08 09:31:19,370][53885] Updated weights for policy 1, policy_version 43072 (0.0007) [2023-10-08 09:31:19,460][53852] Updated weights for policy 0, policy_version 43290 (0.0007) [2023-10-08 09:31:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 88440832. Throughput: 0: 1818.5, 1: 1824.5. Samples: 22122896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:31:22,016][52710] Avg episode reward: [(0, '29.450'), (1, '30.960')] [2023-10-08 09:31:23,004][53885] Updated weights for policy 1, policy_version 43082 (0.0009) [2023-10-08 09:31:23,073][53852] Updated weights for policy 0, policy_version 43300 (0.0010) [2023-10-08 09:31:23,363][53885] Updated weights for policy 1, policy_version 43092 (0.0008) [2023-10-08 09:31:23,455][53852] Updated weights for policy 0, policy_version 43310 (0.0007) [2023-10-08 09:31:23,736][53885] Updated weights for policy 1, policy_version 43102 (0.0008) [2023-10-08 09:31:23,828][53852] Updated weights for policy 0, policy_version 43320 (0.0008) [2023-10-08 09:31:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88506368. Throughput: 0: 1825.1, 1: 1821.7. Samples: 22132948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:31:27,016][52710] Avg episode reward: [(0, '31.250'), (1, '31.330')] [2023-10-08 09:31:27,344][53885] Updated weights for policy 1, policy_version 43112 (0.0009) [2023-10-08 09:31:27,554][53852] Updated weights for policy 0, policy_version 43330 (0.0007) [2023-10-08 09:31:27,713][53885] Updated weights for policy 1, policy_version 43122 (0.0009) [2023-10-08 09:31:27,925][53852] Updated weights for policy 0, policy_version 43340 (0.0008) [2023-10-08 09:31:28,072][53885] Updated weights for policy 1, policy_version 43132 (0.0008) [2023-10-08 09:31:28,300][53852] Updated weights for policy 0, policy_version 43350 (0.0009) [2023-10-08 09:31:28,659][53852] Updated weights for policy 0, policy_version 43360 (0.0007) [2023-10-08 09:31:31,695][53885] Updated weights for policy 1, policy_version 43142 (0.0008) [2023-10-08 09:31:32,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88571904. Throughput: 0: 1832.6, 1: 1820.8. Samples: 22155770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:31:32,015][52710] Avg episode reward: [(0, '28.220'), (1, '31.540')] [2023-10-08 09:31:32,066][53885] Updated weights for policy 1, policy_version 43152 (0.0009) [2023-10-08 09:31:32,373][53852] Updated weights for policy 0, policy_version 43370 (0.0007) [2023-10-08 09:31:32,433][53885] Updated weights for policy 1, policy_version 43162 (0.0009) [2023-10-08 09:31:32,747][53852] Updated weights for policy 0, policy_version 43380 (0.0007) [2023-10-08 09:31:33,128][53852] Updated weights for policy 0, policy_version 43390 (0.0008) [2023-10-08 09:31:36,213][53885] Updated weights for policy 1, policy_version 43172 (0.0008) [2023-10-08 09:31:36,584][53885] Updated weights for policy 1, policy_version 43182 (0.0009) [2023-10-08 09:31:36,744][53852] Updated weights for policy 0, policy_version 43400 (0.0007) [2023-10-08 09:31:36,949][53885] Updated weights for policy 1, policy_version 43192 (0.0008) [2023-10-08 09:31:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88637440. Throughput: 0: 1826.4, 1: 1817.8. Samples: 22177804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:31:37,016][52710] Avg episode reward: [(0, '30.300'), (1, '32.530')] [2023-10-08 09:31:37,112][53852] Updated weights for policy 0, policy_version 43410 (0.0007) [2023-10-08 09:31:37,474][53852] Updated weights for policy 0, policy_version 43420 (0.0009) [2023-10-08 09:31:40,635][53885] Updated weights for policy 1, policy_version 43202 (0.0007) [2023-10-08 09:31:41,003][53885] Updated weights for policy 1, policy_version 43212 (0.0008) [2023-10-08 09:31:41,218][53852] Updated weights for policy 0, policy_version 43430 (0.0008) [2023-10-08 09:31:41,368][53885] Updated weights for policy 1, policy_version 43222 (0.0008) [2023-10-08 09:31:41,589][53852] Updated weights for policy 0, policy_version 43440 (0.0008) [2023-10-08 09:31:41,731][53885] Updated weights for policy 1, policy_version 43232 (0.0007) [2023-10-08 09:31:41,950][53852] Updated weights for policy 0, policy_version 43450 (0.0010) [2023-10-08 09:31:42,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88735744. Throughput: 0: 1832.8, 1: 1826.6. Samples: 22188786. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 09:31:42,016][52710] Avg episode reward: [(0, '32.030'), (1, '31.510')] [2023-10-08 09:31:45,443][53885] Updated weights for policy 1, policy_version 43242 (0.0008) [2023-10-08 09:31:45,733][53852] Updated weights for policy 0, policy_version 43460 (0.0009) [2023-10-08 09:31:45,800][53885] Updated weights for policy 1, policy_version 43252 (0.0009) [2023-10-08 09:31:46,121][53852] Updated weights for policy 0, policy_version 43470 (0.0009) [2023-10-08 09:31:46,167][53885] Updated weights for policy 1, policy_version 43262 (0.0007) [2023-10-08 09:31:46,486][53852] Updated weights for policy 0, policy_version 43480 (0.0009) [2023-10-08 09:31:47,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 88834048. Throughput: 0: 1832.0, 1: 1828.0. Samples: 22211208. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 09:31:47,016][52710] Avg episode reward: [(0, '31.160'), (1, '31.950')] [2023-10-08 09:31:49,799][53885] Updated weights for policy 1, policy_version 43272 (0.0007) [2023-10-08 09:31:50,168][53885] Updated weights for policy 1, policy_version 43282 (0.0008) [2023-10-08 09:31:50,221][53852] Updated weights for policy 0, policy_version 43490 (0.0008) [2023-10-08 09:31:50,533][53885] Updated weights for policy 1, policy_version 43292 (0.0009) [2023-10-08 09:31:50,595][53852] Updated weights for policy 0, policy_version 43500 (0.0009) [2023-10-08 09:31:50,961][53852] Updated weights for policy 0, policy_version 43510 (0.0007) [2023-10-08 09:31:51,322][53852] Updated weights for policy 0, policy_version 43520 (0.0008) [2023-10-08 09:31:52,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 88899584. Throughput: 0: 1834.3, 1: 1830.8. Samples: 22231772. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 09:31:52,016][52710] Avg episode reward: [(0, '32.410'), (1, '29.370')] [2023-10-08 09:31:54,239][53885] Updated weights for policy 1, policy_version 43302 (0.0008) [2023-10-08 09:31:54,618][53885] Updated weights for policy 1, policy_version 43312 (0.0008) [2023-10-08 09:31:54,990][53885] Updated weights for policy 1, policy_version 43322 (0.0007) [2023-10-08 09:31:55,001][53852] Updated weights for policy 0, policy_version 43530 (0.0007) [2023-10-08 09:31:55,374][53852] Updated weights for policy 0, policy_version 43540 (0.0010) [2023-10-08 09:31:55,729][53852] Updated weights for policy 0, policy_version 43550 (0.0010) [2023-10-08 09:31:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 88965120. Throughput: 0: 1839.8, 1: 1827.7. Samples: 22244292. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 09:31:57,015][52710] Avg episode reward: [(0, '31.430'), (1, '30.300')] [2023-10-08 09:31:58,446][53885] Updated weights for policy 1, policy_version 43332 (0.0008) [2023-10-08 09:31:58,824][53885] Updated weights for policy 1, policy_version 43342 (0.0008) [2023-10-08 09:31:59,177][53885] Updated weights for policy 1, policy_version 43352 (0.0009) [2023-10-08 09:31:59,364][53852] Updated weights for policy 0, policy_version 43560 (0.0008) [2023-10-08 09:31:59,734][53852] Updated weights for policy 0, policy_version 43570 (0.0008) [2023-10-08 09:32:00,101][53852] Updated weights for policy 0, policy_version 43580 (0.0007) [2023-10-08 09:32:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 89030656. Throughput: 0: 1836.6, 1: 1838.8. Samples: 22265308. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 09:32:02,016][52710] Avg episode reward: [(0, '30.300'), (1, '32.300')] [2023-10-08 09:32:02,959][53885] Updated weights for policy 1, policy_version 43362 (0.0008) [2023-10-08 09:32:03,326][53885] Updated weights for policy 1, policy_version 43372 (0.0008) [2023-10-08 09:32:03,698][53885] Updated weights for policy 1, policy_version 43382 (0.0007) [2023-10-08 09:32:03,796][53852] Updated weights for policy 0, policy_version 43590 (0.0009) [2023-10-08 09:32:04,060][53885] Updated weights for policy 1, policy_version 43392 (0.0008) [2023-10-08 09:32:04,172][53852] Updated weights for policy 0, policy_version 43600 (0.0009) [2023-10-08 09:32:04,537][53852] Updated weights for policy 0, policy_version 43610 (0.0007) [2023-10-08 09:32:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 89096192. Throughput: 0: 1834.0, 1: 1836.2. Samples: 22288054. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 09:32:07,016][52710] Avg episode reward: [(0, '27.490'), (1, '27.430')] [2023-10-08 09:32:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000043392_44433408.pth... [2023-10-08 09:32:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000043616_44662784.pth... [2023-10-08 09:32:07,054][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000041920_42926080.pth [2023-10-08 09:32:07,059][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000041696_42696704.pth [2023-10-08 09:32:07,720][53885] Updated weights for policy 1, policy_version 43402 (0.0007) [2023-10-08 09:32:08,089][53885] Updated weights for policy 1, policy_version 43412 (0.0007) [2023-10-08 09:32:08,148][53852] Updated weights for policy 0, policy_version 43620 (0.0008) [2023-10-08 09:32:08,447][53885] Updated weights for policy 1, policy_version 43422 (0.0008) [2023-10-08 09:32:08,507][53852] Updated weights for policy 0, policy_version 43630 (0.0008) [2023-10-08 09:32:08,877][53852] Updated weights for policy 0, policy_version 43640 (0.0009) [2023-10-08 09:32:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89161728. Throughput: 0: 1831.6, 1: 1837.9. Samples: 22298074. Policy #0 lag: (min: 27.0, avg: 42.1, max: 59.0) [2023-10-08 09:32:12,016][52710] Avg episode reward: [(0, '26.230'), (1, '28.860')] [2023-10-08 09:32:12,078][53885] Updated weights for policy 1, policy_version 43432 (0.0009) [2023-10-08 09:32:12,441][53885] Updated weights for policy 1, policy_version 43442 (0.0010) [2023-10-08 09:32:12,524][53852] Updated weights for policy 0, policy_version 43650 (0.0009) [2023-10-08 09:32:12,807][53885] Updated weights for policy 1, policy_version 43452 (0.0009) [2023-10-08 09:32:12,892][53852] Updated weights for policy 0, policy_version 43660 (0.0008) [2023-10-08 09:32:13,257][53852] Updated weights for policy 0, policy_version 43670 (0.0008) [2023-10-08 09:32:13,636][53852] Updated weights for policy 0, policy_version 43680 (0.0008) [2023-10-08 09:32:16,452][53885] Updated weights for policy 1, policy_version 43462 (0.0008) [2023-10-08 09:32:16,820][53885] Updated weights for policy 1, policy_version 43472 (0.0007) [2023-10-08 09:32:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89227264. Throughput: 0: 1825.6, 1: 1838.4. Samples: 22320652. Policy #0 lag: (min: 27.0, avg: 42.1, max: 59.0) [2023-10-08 09:32:17,016][52710] Avg episode reward: [(0, '29.240'), (1, '32.940')] [2023-10-08 09:32:17,187][53885] Updated weights for policy 1, policy_version 43482 (0.0007) [2023-10-08 09:32:17,323][53852] Updated weights for policy 0, policy_version 43690 (0.0007) [2023-10-08 09:32:17,697][53852] Updated weights for policy 0, policy_version 43700 (0.0008) [2023-10-08 09:32:18,074][53852] Updated weights for policy 0, policy_version 43710 (0.0010) [2023-10-08 09:32:20,797][53885] Updated weights for policy 1, policy_version 43492 (0.0009) [2023-10-08 09:32:21,163][53885] Updated weights for policy 1, policy_version 43502 (0.0010) [2023-10-08 09:32:21,530][53885] Updated weights for policy 1, policy_version 43512 (0.0008) [2023-10-08 09:32:21,638][53852] Updated weights for policy 0, policy_version 43720 (0.0009) [2023-10-08 09:32:22,009][53852] Updated weights for policy 0, policy_version 43730 (0.0011) [2023-10-08 09:32:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89325568. Throughput: 0: 1827.5, 1: 1832.4. Samples: 22342498. Policy #0 lag: (min: 27.0, avg: 42.1, max: 59.0) [2023-10-08 09:32:22,016][52710] Avg episode reward: [(0, '27.470'), (1, '26.540')] [2023-10-08 09:32:22,387][53852] Updated weights for policy 0, policy_version 43740 (0.0010) [2023-10-08 09:32:25,174][53885] Updated weights for policy 1, policy_version 43522 (0.0009) [2023-10-08 09:32:25,540][53885] Updated weights for policy 1, policy_version 43532 (0.0009) [2023-10-08 09:32:25,912][53885] Updated weights for policy 1, policy_version 43542 (0.0007) [2023-10-08 09:32:26,070][53852] Updated weights for policy 0, policy_version 43750 (0.0008) [2023-10-08 09:32:26,276][53885] Updated weights for policy 1, policy_version 43552 (0.0007) [2023-10-08 09:32:26,424][53852] Updated weights for policy 0, policy_version 43760 (0.0007) [2023-10-08 09:32:26,795][53852] Updated weights for policy 0, policy_version 43770 (0.0007) [2023-10-08 09:32:27,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 89423872. Throughput: 0: 1828.5, 1: 1835.1. Samples: 22353644. Policy #0 lag: (min: 27.0, avg: 42.1, max: 59.0) [2023-10-08 09:32:27,015][52710] Avg episode reward: [(0, '27.930'), (1, '28.870')] [2023-10-08 09:32:30,177][53885] Updated weights for policy 1, policy_version 43562 (0.0011) [2023-10-08 09:32:30,514][53852] Updated weights for policy 0, policy_version 43780 (0.0007) [2023-10-08 09:32:30,539][53885] Updated weights for policy 1, policy_version 43572 (0.0010) [2023-10-08 09:32:30,888][53852] Updated weights for policy 0, policy_version 43790 (0.0011) [2023-10-08 09:32:30,897][53885] Updated weights for policy 1, policy_version 43582 (0.0008) [2023-10-08 09:32:31,251][53852] Updated weights for policy 0, policy_version 43800 (0.0008) [2023-10-08 09:32:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 89489408. Throughput: 0: 1822.3, 1: 1824.4. Samples: 22375310. Policy #0 lag: (min: 27.0, avg: 42.1, max: 59.0) [2023-10-08 09:32:32,016][52710] Avg episode reward: [(0, '26.830'), (1, '32.640')] [2023-10-08 09:32:34,627][53885] Updated weights for policy 1, policy_version 43592 (0.0008) [2023-10-08 09:32:34,989][53885] Updated weights for policy 1, policy_version 43602 (0.0008) [2023-10-08 09:32:35,058][53852] Updated weights for policy 0, policy_version 43810 (0.0009) [2023-10-08 09:32:35,347][53885] Updated weights for policy 1, policy_version 43612 (0.0008) [2023-10-08 09:32:35,451][53852] Updated weights for policy 0, policy_version 43820 (0.0008) [2023-10-08 09:32:35,830][53852] Updated weights for policy 0, policy_version 43830 (0.0008) [2023-10-08 09:32:36,206][53852] Updated weights for policy 0, policy_version 43840 (0.0011) [2023-10-08 09:32:37,015][52710] Fps is (10 sec: 13106.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 89554944. Throughput: 0: 1827.2, 1: 1828.1. Samples: 22396260. Policy #0 lag: (min: 27.0, avg: 42.1, max: 59.0) [2023-10-08 09:32:37,016][52710] Avg episode reward: [(0, '32.020'), (1, '29.750')] [2023-10-08 09:32:39,071][53885] Updated weights for policy 1, policy_version 43622 (0.0009) [2023-10-08 09:32:39,441][53885] Updated weights for policy 1, policy_version 43632 (0.0010) [2023-10-08 09:32:39,814][53852] Updated weights for policy 0, policy_version 43850 (0.0008) [2023-10-08 09:32:39,818][53885] Updated weights for policy 1, policy_version 43642 (0.0008) [2023-10-08 09:32:40,183][53852] Updated weights for policy 0, policy_version 43860 (0.0008) [2023-10-08 09:32:40,554][53852] Updated weights for policy 0, policy_version 43870 (0.0008) [2023-10-08 09:32:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89620480. Throughput: 0: 1819.0, 1: 1821.8. Samples: 22408126. Policy #0 lag: (min: 7.0, avg: 7.2, max: 16.0) [2023-10-08 09:32:42,016][52710] Avg episode reward: [(0, '29.890'), (1, '28.250')] [2023-10-08 09:32:43,483][53885] Updated weights for policy 1, policy_version 43652 (0.0009) [2023-10-08 09:32:43,851][53885] Updated weights for policy 1, policy_version 43662 (0.0009) [2023-10-08 09:32:44,119][53852] Updated weights for policy 0, policy_version 43880 (0.0008) [2023-10-08 09:32:44,222][53885] Updated weights for policy 1, policy_version 43672 (0.0007) [2023-10-08 09:32:44,484][53852] Updated weights for policy 0, policy_version 43890 (0.0007) [2023-10-08 09:32:44,852][53852] Updated weights for policy 0, policy_version 43900 (0.0007) [2023-10-08 09:32:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 89686016. Throughput: 0: 1823.3, 1: 1820.2. Samples: 22429264. Policy #0 lag: (min: 7.0, avg: 7.2, max: 16.0) [2023-10-08 09:32:47,015][52710] Avg episode reward: [(0, '31.030'), (1, '32.740')] [2023-10-08 09:32:48,012][53885] Updated weights for policy 1, policy_version 43682 (0.0010) [2023-10-08 09:32:48,384][53885] Updated weights for policy 1, policy_version 43692 (0.0011) [2023-10-08 09:32:48,537][53852] Updated weights for policy 0, policy_version 43910 (0.0009) [2023-10-08 09:32:48,756][53885] Updated weights for policy 1, policy_version 43702 (0.0007) [2023-10-08 09:32:48,906][53852] Updated weights for policy 0, policy_version 43920 (0.0009) [2023-10-08 09:32:49,118][53885] Updated weights for policy 1, policy_version 43712 (0.0008) [2023-10-08 09:32:49,270][53852] Updated weights for policy 0, policy_version 43930 (0.0008) [2023-10-08 09:32:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89751552. Throughput: 0: 1822.6, 1: 1818.0. Samples: 22451880. Policy #0 lag: (min: 7.0, avg: 7.2, max: 16.0) [2023-10-08 09:32:52,015][52710] Avg episode reward: [(0, '28.000'), (1, '30.040')] [2023-10-08 09:32:52,853][53885] Updated weights for policy 1, policy_version 43722 (0.0008) [2023-10-08 09:32:52,875][53852] Updated weights for policy 0, policy_version 43940 (0.0008) [2023-10-08 09:32:53,230][53885] Updated weights for policy 1, policy_version 43732 (0.0008) [2023-10-08 09:32:53,247][53852] Updated weights for policy 0, policy_version 43950 (0.0007) [2023-10-08 09:32:53,595][53885] Updated weights for policy 1, policy_version 43742 (0.0007) [2023-10-08 09:32:53,609][53852] Updated weights for policy 0, policy_version 43960 (0.0008) [2023-10-08 09:32:57,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 89817088. Throughput: 0: 1822.6, 1: 1813.2. Samples: 22461686. Policy #0 lag: (min: 7.0, avg: 7.2, max: 16.0) [2023-10-08 09:32:57,016][52710] Avg episode reward: [(0, '29.620'), (1, '33.050')] [2023-10-08 09:32:57,167][53852] Updated weights for policy 0, policy_version 43970 (0.0009) [2023-10-08 09:32:57,311][53885] Updated weights for policy 1, policy_version 43752 (0.0008) [2023-10-08 09:32:57,547][53852] Updated weights for policy 0, policy_version 43980 (0.0007) [2023-10-08 09:32:57,672][53885] Updated weights for policy 1, policy_version 43762 (0.0007) [2023-10-08 09:32:57,908][53852] Updated weights for policy 0, policy_version 43990 (0.0008) [2023-10-08 09:32:58,038][53885] Updated weights for policy 1, policy_version 43772 (0.0007) [2023-10-08 09:32:58,281][53852] Updated weights for policy 0, policy_version 44000 (0.0008) [2023-10-08 09:33:01,646][53885] Updated weights for policy 1, policy_version 43782 (0.0009) [2023-10-08 09:33:01,986][53852] Updated weights for policy 0, policy_version 44010 (0.0008) [2023-10-08 09:33:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 89882624. Throughput: 0: 1832.9, 1: 1815.6. Samples: 22484836. Policy #0 lag: (min: 7.0, avg: 7.2, max: 16.0) [2023-10-08 09:33:02,016][52710] Avg episode reward: [(0, '29.890'), (1, '32.480')] [2023-10-08 09:33:02,019][53885] Updated weights for policy 1, policy_version 43792 (0.0009) [2023-10-08 09:33:02,368][53852] Updated weights for policy 0, policy_version 44020 (0.0008) [2023-10-08 09:33:02,382][53885] Updated weights for policy 1, policy_version 43802 (0.0007) [2023-10-08 09:33:02,742][53852] Updated weights for policy 0, policy_version 44030 (0.0008) [2023-10-08 09:33:05,986][53885] Updated weights for policy 1, policy_version 43812 (0.0007) [2023-10-08 09:33:06,355][53885] Updated weights for policy 1, policy_version 43822 (0.0008) [2023-10-08 09:33:06,397][53852] Updated weights for policy 0, policy_version 44040 (0.0008) [2023-10-08 09:33:06,717][53885] Updated weights for policy 1, policy_version 43832 (0.0007) [2023-10-08 09:33:06,758][53852] Updated weights for policy 0, policy_version 44050 (0.0007) [2023-10-08 09:33:07,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89980928. Throughput: 0: 1822.2, 1: 1816.8. Samples: 22506250. Policy #0 lag: (min: 7.0, avg: 7.2, max: 16.0) [2023-10-08 09:33:07,016][52710] Avg episode reward: [(0, '29.470'), (1, '28.350')] [2023-10-08 09:33:07,127][53852] Updated weights for policy 0, policy_version 44060 (0.0008) [2023-10-08 09:33:10,398][53885] Updated weights for policy 1, policy_version 43842 (0.0007) [2023-10-08 09:33:10,767][53885] Updated weights for policy 1, policy_version 43852 (0.0007) [2023-10-08 09:33:10,858][53852] Updated weights for policy 0, policy_version 44070 (0.0008) [2023-10-08 09:33:11,127][53885] Updated weights for policy 1, policy_version 43862 (0.0009) [2023-10-08 09:33:11,220][53852] Updated weights for policy 0, policy_version 44080 (0.0008) [2023-10-08 09:33:11,496][53885] Updated weights for policy 1, policy_version 43872 (0.0007) [2023-10-08 09:33:11,597][53852] Updated weights for policy 0, policy_version 44090 (0.0007) [2023-10-08 09:33:12,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 90079232. Throughput: 0: 1832.4, 1: 1810.6. Samples: 22517580. Policy #0 lag: (min: 24.0, avg: 38.9, max: 56.0) [2023-10-08 09:33:12,016][52710] Avg episode reward: [(0, '28.910'), (1, '29.610')] [2023-10-08 09:33:15,240][53885] Updated weights for policy 1, policy_version 43882 (0.0007) [2023-10-08 09:33:15,294][53852] Updated weights for policy 0, policy_version 44100 (0.0010) [2023-10-08 09:33:15,610][53885] Updated weights for policy 1, policy_version 43892 (0.0009) [2023-10-08 09:33:15,662][53852] Updated weights for policy 0, policy_version 44110 (0.0008) [2023-10-08 09:33:15,978][53885] Updated weights for policy 1, policy_version 43902 (0.0007) [2023-10-08 09:33:16,033][53852] Updated weights for policy 0, policy_version 44120 (0.0008) [2023-10-08 09:33:17,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 90144768. Throughput: 0: 1824.4, 1: 1816.9. Samples: 22539168. Policy #0 lag: (min: 24.0, avg: 38.9, max: 56.0) [2023-10-08 09:33:17,016][52710] Avg episode reward: [(0, '30.860'), (1, '27.790')] [2023-10-08 09:33:19,501][53885] Updated weights for policy 1, policy_version 43912 (0.0008) [2023-10-08 09:33:19,861][53852] Updated weights for policy 0, policy_version 44130 (0.0008) [2023-10-08 09:33:19,868][53885] Updated weights for policy 1, policy_version 43922 (0.0008) [2023-10-08 09:33:20,221][53885] Updated weights for policy 1, policy_version 43932 (0.0008) [2023-10-08 09:33:20,248][53852] Updated weights for policy 0, policy_version 44140 (0.0008) [2023-10-08 09:33:20,615][53852] Updated weights for policy 0, policy_version 44150 (0.0009) [2023-10-08 09:33:20,976][53852] Updated weights for policy 0, policy_version 44160 (0.0008) [2023-10-08 09:33:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90210304. Throughput: 0: 1829.5, 1: 1823.4. Samples: 22560642. Policy #0 lag: (min: 24.0, avg: 38.9, max: 56.0) [2023-10-08 09:33:22,016][52710] Avg episode reward: [(0, '28.010'), (1, '29.990')] [2023-10-08 09:33:23,945][53885] Updated weights for policy 1, policy_version 43942 (0.0007) [2023-10-08 09:33:24,300][53885] Updated weights for policy 1, policy_version 43952 (0.0009) [2023-10-08 09:33:24,444][53852] Updated weights for policy 0, policy_version 44170 (0.0008) [2023-10-08 09:33:24,662][53885] Updated weights for policy 1, policy_version 43962 (0.0008) [2023-10-08 09:33:24,820][53852] Updated weights for policy 0, policy_version 44180 (0.0008) [2023-10-08 09:33:25,185][53852] Updated weights for policy 0, policy_version 44190 (0.0008) [2023-10-08 09:33:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 90275840. Throughput: 0: 1828.6, 1: 1819.6. Samples: 22572298. Policy #0 lag: (min: 24.0, avg: 38.9, max: 56.0) [2023-10-08 09:33:27,016][52710] Avg episode reward: [(0, '29.400'), (1, '29.460')] [2023-10-08 09:33:28,294][53885] Updated weights for policy 1, policy_version 43972 (0.0008) [2023-10-08 09:33:28,658][53885] Updated weights for policy 1, policy_version 43982 (0.0007) [2023-10-08 09:33:28,924][53852] Updated weights for policy 0, policy_version 44200 (0.0008) [2023-10-08 09:33:29,031][53885] Updated weights for policy 1, policy_version 43992 (0.0008) [2023-10-08 09:33:29,293][53852] Updated weights for policy 0, policy_version 44210 (0.0007) [2023-10-08 09:33:29,660][53852] Updated weights for policy 0, policy_version 44220 (0.0007) [2023-10-08 09:33:32,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 90341376. Throughput: 0: 1835.1, 1: 1824.9. Samples: 22593966. Policy #0 lag: (min: 24.0, avg: 38.9, max: 56.0) [2023-10-08 09:33:32,017][52710] Avg episode reward: [(0, '30.960'), (1, '32.540')] [2023-10-08 09:33:32,777][53885] Updated weights for policy 1, policy_version 44002 (0.0009) [2023-10-08 09:33:33,144][53885] Updated weights for policy 1, policy_version 44012 (0.0009) [2023-10-08 09:33:33,338][53852] Updated weights for policy 0, policy_version 44230 (0.0008) [2023-10-08 09:33:33,513][53885] Updated weights for policy 1, policy_version 44022 (0.0008) [2023-10-08 09:33:33,706][53852] Updated weights for policy 0, policy_version 44240 (0.0008) [2023-10-08 09:33:33,881][53885] Updated weights for policy 1, policy_version 44032 (0.0007) [2023-10-08 09:33:34,071][53852] Updated weights for policy 0, policy_version 44250 (0.0009) [2023-10-08 09:33:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90406912. Throughput: 0: 1836.0, 1: 1831.3. Samples: 22616906. Policy #0 lag: (min: 24.0, avg: 38.9, max: 56.0) [2023-10-08 09:33:37,016][52710] Avg episode reward: [(0, '28.760'), (1, '31.400')] [2023-10-08 09:33:37,699][53885] Updated weights for policy 1, policy_version 44042 (0.0009) [2023-10-08 09:33:37,874][53852] Updated weights for policy 0, policy_version 44260 (0.0007) [2023-10-08 09:33:38,076][53885] Updated weights for policy 1, policy_version 44052 (0.0008) [2023-10-08 09:33:38,239][53852] Updated weights for policy 0, policy_version 44270 (0.0009) [2023-10-08 09:33:38,443][53885] Updated weights for policy 1, policy_version 44062 (0.0008) [2023-10-08 09:33:38,606][53852] Updated weights for policy 0, policy_version 44280 (0.0009) [2023-10-08 09:33:41,932][53885] Updated weights for policy 1, policy_version 44072 (0.0007) [2023-10-08 09:33:42,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90472448. Throughput: 0: 1833.8, 1: 1836.6. Samples: 22626854. Policy #0 lag: (min: 24.0, avg: 38.9, max: 56.0) [2023-10-08 09:33:42,015][52710] Avg episode reward: [(0, '27.240'), (1, '31.340')] [2023-10-08 09:33:42,286][53885] Updated weights for policy 1, policy_version 44082 (0.0007) [2023-10-08 09:33:42,369][53852] Updated weights for policy 0, policy_version 44290 (0.0009) [2023-10-08 09:33:42,668][53885] Updated weights for policy 1, policy_version 44092 (0.0007) [2023-10-08 09:33:42,735][53852] Updated weights for policy 0, policy_version 44300 (0.0009) [2023-10-08 09:33:43,106][53852] Updated weights for policy 0, policy_version 44310 (0.0009) [2023-10-08 09:33:43,471][53852] Updated weights for policy 0, policy_version 44320 (0.0009) [2023-10-08 09:33:46,275][53885] Updated weights for policy 1, policy_version 44102 (0.0008) [2023-10-08 09:33:46,651][53885] Updated weights for policy 1, policy_version 44112 (0.0011) [2023-10-08 09:33:47,010][53885] Updated weights for policy 1, policy_version 44122 (0.0008) [2023-10-08 09:33:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 90537984. Throughput: 0: 1827.6, 1: 1836.0. Samples: 22649702. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 09:33:47,016][52710] Avg episode reward: [(0, '31.500'), (1, '34.250')] [2023-10-08 09:33:47,133][53852] Updated weights for policy 0, policy_version 44330 (0.0007) [2023-10-08 09:33:47,498][53852] Updated weights for policy 0, policy_version 44340 (0.0008) [2023-10-08 09:33:47,863][53852] Updated weights for policy 0, policy_version 44350 (0.0009) [2023-10-08 09:33:50,702][53885] Updated weights for policy 1, policy_version 44132 (0.0009) [2023-10-08 09:33:51,066][53885] Updated weights for policy 1, policy_version 44142 (0.0007) [2023-10-08 09:33:51,427][53885] Updated weights for policy 1, policy_version 44152 (0.0007) [2023-10-08 09:33:51,533][53852] Updated weights for policy 0, policy_version 44360 (0.0007) [2023-10-08 09:33:51,911][53852] Updated weights for policy 0, policy_version 44370 (0.0009) [2023-10-08 09:33:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90636288. Throughput: 0: 1834.7, 1: 1830.4. Samples: 22671178. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 09:33:52,015][52710] Avg episode reward: [(0, '30.910'), (1, '31.780')] [2023-10-08 09:33:52,272][53852] Updated weights for policy 0, policy_version 44380 (0.0008) [2023-10-08 09:33:55,034][53885] Updated weights for policy 1, policy_version 44162 (0.0008) [2023-10-08 09:33:55,408][53885] Updated weights for policy 1, policy_version 44172 (0.0009) [2023-10-08 09:33:55,777][53852] Updated weights for policy 0, policy_version 44390 (0.0007) [2023-10-08 09:33:55,782][53885] Updated weights for policy 1, policy_version 44182 (0.0007) [2023-10-08 09:33:56,143][53852] Updated weights for policy 0, policy_version 44400 (0.0007) [2023-10-08 09:33:56,152][53885] Updated weights for policy 1, policy_version 44192 (0.0009) [2023-10-08 09:33:56,512][53852] Updated weights for policy 0, policy_version 44410 (0.0011) [2023-10-08 09:33:57,015][52710] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 90734592. Throughput: 0: 1835.7, 1: 1842.9. Samples: 22683118. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 09:33:57,015][52710] Avg episode reward: [(0, '30.640'), (1, '32.500')] [2023-10-08 09:33:59,843][53885] Updated weights for policy 1, policy_version 44202 (0.0007) [2023-10-08 09:34:00,097][53852] Updated weights for policy 0, policy_version 44420 (0.0007) [2023-10-08 09:34:00,216][53885] Updated weights for policy 1, policy_version 44212 (0.0008) [2023-10-08 09:34:00,468][53852] Updated weights for policy 0, policy_version 44430 (0.0007) [2023-10-08 09:34:00,579][53885] Updated weights for policy 1, policy_version 44222 (0.0008) [2023-10-08 09:34:00,831][53852] Updated weights for policy 0, policy_version 44440 (0.0008) [2023-10-08 09:34:02,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 90800128. Throughput: 0: 1834.4, 1: 1829.9. Samples: 22704062. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 09:34:02,016][52710] Avg episode reward: [(0, '32.120'), (1, '32.700')] [2023-10-08 09:34:04,078][53885] Updated weights for policy 1, policy_version 44232 (0.0008) [2023-10-08 09:34:04,348][53852] Updated weights for policy 0, policy_version 44450 (0.0007) [2023-10-08 09:34:04,445][53885] Updated weights for policy 1, policy_version 44242 (0.0007) [2023-10-08 09:34:04,723][53852] Updated weights for policy 0, policy_version 44460 (0.0008) [2023-10-08 09:34:04,797][53885] Updated weights for policy 1, policy_version 44252 (0.0010) [2023-10-08 09:34:05,094][53852] Updated weights for policy 0, policy_version 44470 (0.0008) [2023-10-08 09:34:05,456][53852] Updated weights for policy 0, policy_version 44480 (0.0008) [2023-10-08 09:34:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90865664. Throughput: 0: 1844.0, 1: 1836.3. Samples: 22726256. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 09:34:07,016][52710] Avg episode reward: [(0, '34.810'), (1, '34.170')] [2023-10-08 09:34:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000044480_45547520.pth... [2023-10-08 09:34:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000044256_45318144.pth... [2023-10-08 09:34:07,058][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000042784_43810816.pth [2023-10-08 09:34:07,061][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000042560_43581440.pth [2023-10-08 09:34:07,061][53500] Saving new best policy, reward=34.810! [2023-10-08 09:34:08,570][53885] Updated weights for policy 1, policy_version 44262 (0.0009) [2023-10-08 09:34:08,946][53885] Updated weights for policy 1, policy_version 44272 (0.0008) [2023-10-08 09:34:09,249][53852] Updated weights for policy 0, policy_version 44490 (0.0007) [2023-10-08 09:34:09,324][53885] Updated weights for policy 1, policy_version 44282 (0.0007) [2023-10-08 09:34:09,619][53852] Updated weights for policy 0, policy_version 44500 (0.0008) [2023-10-08 09:34:09,982][53852] Updated weights for policy 0, policy_version 44510 (0.0011) [2023-10-08 09:34:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 90931200. Throughput: 0: 1827.1, 1: 1824.1. Samples: 22736602. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) [2023-10-08 09:34:12,016][52710] Avg episode reward: [(0, '30.360'), (1, '31.490')] [2023-10-08 09:34:13,089][53885] Updated weights for policy 1, policy_version 44292 (0.0008) [2023-10-08 09:34:13,459][53885] Updated weights for policy 1, policy_version 44302 (0.0008) [2023-10-08 09:34:13,640][53852] Updated weights for policy 0, policy_version 44520 (0.0009) [2023-10-08 09:34:13,823][53885] Updated weights for policy 1, policy_version 44312 (0.0008) [2023-10-08 09:34:14,001][53852] Updated weights for policy 0, policy_version 44530 (0.0010) [2023-10-08 09:34:14,372][53852] Updated weights for policy 0, policy_version 44540 (0.0007) [2023-10-08 09:34:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90996736. Throughput: 0: 1834.8, 1: 1827.6. Samples: 22758772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:17,015][52710] Avg episode reward: [(0, '31.300'), (1, '31.660')] [2023-10-08 09:34:17,457][53885] Updated weights for policy 1, policy_version 44322 (0.0009) [2023-10-08 09:34:17,818][53885] Updated weights for policy 1, policy_version 44332 (0.0008) [2023-10-08 09:34:18,002][53852] Updated weights for policy 0, policy_version 44550 (0.0007) [2023-10-08 09:34:18,178][53885] Updated weights for policy 1, policy_version 44342 (0.0008) [2023-10-08 09:34:18,372][53852] Updated weights for policy 0, policy_version 44560 (0.0009) [2023-10-08 09:34:18,542][53885] Updated weights for policy 1, policy_version 44352 (0.0008) [2023-10-08 09:34:18,739][53852] Updated weights for policy 0, policy_version 44570 (0.0011) [2023-10-08 09:34:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91062272. Throughput: 0: 1839.5, 1: 1820.7. Samples: 22781616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:22,016][52710] Avg episode reward: [(0, '27.990'), (1, '34.230')] [2023-10-08 09:34:22,336][53885] Updated weights for policy 1, policy_version 44362 (0.0008) [2023-10-08 09:34:22,362][53852] Updated weights for policy 0, policy_version 44580 (0.0009) [2023-10-08 09:34:22,705][53885] Updated weights for policy 1, policy_version 44372 (0.0008) [2023-10-08 09:34:22,734][53852] Updated weights for policy 0, policy_version 44590 (0.0008) [2023-10-08 09:34:23,061][53885] Updated weights for policy 1, policy_version 44382 (0.0008) [2023-10-08 09:34:23,095][53852] Updated weights for policy 0, policy_version 44600 (0.0009) [2023-10-08 09:34:26,751][53852] Updated weights for policy 0, policy_version 44610 (0.0010) [2023-10-08 09:34:26,866][53885] Updated weights for policy 1, policy_version 44392 (0.0008) [2023-10-08 09:34:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91127808. Throughput: 0: 1839.7, 1: 1821.8. Samples: 22791620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:27,015][52710] Avg episode reward: [(0, '29.990'), (1, '32.310')] [2023-10-08 09:34:27,110][53852] Updated weights for policy 0, policy_version 44620 (0.0007) [2023-10-08 09:34:27,242][53885] Updated weights for policy 1, policy_version 44402 (0.0009) [2023-10-08 09:34:27,483][53852] Updated weights for policy 0, policy_version 44630 (0.0007) [2023-10-08 09:34:27,608][53885] Updated weights for policy 1, policy_version 44412 (0.0007) [2023-10-08 09:34:27,851][53852] Updated weights for policy 0, policy_version 44640 (0.0008) [2023-10-08 09:34:31,303][53885] Updated weights for policy 1, policy_version 44422 (0.0008) [2023-10-08 09:34:31,613][53852] Updated weights for policy 0, policy_version 44650 (0.0009) [2023-10-08 09:34:31,664][53885] Updated weights for policy 1, policy_version 44432 (0.0009) [2023-10-08 09:34:31,984][53852] Updated weights for policy 0, policy_version 44660 (0.0009) [2023-10-08 09:34:32,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91193344. Throughput: 0: 1842.1, 1: 1816.5. Samples: 22814342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:32,016][52710] Avg episode reward: [(0, '27.520'), (1, '28.760')] [2023-10-08 09:34:32,039][53885] Updated weights for policy 1, policy_version 44442 (0.0008) [2023-10-08 09:34:32,357][53852] Updated weights for policy 0, policy_version 44670 (0.0008) [2023-10-08 09:34:35,621][53885] Updated weights for policy 1, policy_version 44452 (0.0009) [2023-10-08 09:34:36,000][53885] Updated weights for policy 1, policy_version 44462 (0.0007) [2023-10-08 09:34:36,010][53852] Updated weights for policy 0, policy_version 44680 (0.0010) [2023-10-08 09:34:36,366][53885] Updated weights for policy 1, policy_version 44472 (0.0008) [2023-10-08 09:34:36,379][53852] Updated weights for policy 0, policy_version 44690 (0.0009) [2023-10-08 09:34:36,756][53852] Updated weights for policy 0, policy_version 44700 (0.0010) [2023-10-08 09:34:37,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 91324416. Throughput: 0: 1824.7, 1: 1813.2. Samples: 22834884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:37,016][52710] Avg episode reward: [(0, '29.540'), (1, '32.460')] [2023-10-08 09:34:40,137][53885] Updated weights for policy 1, policy_version 44482 (0.0008) [2023-10-08 09:34:40,402][53852] Updated weights for policy 0, policy_version 44710 (0.0009) [2023-10-08 09:34:40,497][53885] Updated weights for policy 1, policy_version 44492 (0.0007) [2023-10-08 09:34:40,772][53852] Updated weights for policy 0, policy_version 44720 (0.0008) [2023-10-08 09:34:40,873][53885] Updated weights for policy 1, policy_version 44502 (0.0008) [2023-10-08 09:34:41,147][53852] Updated weights for policy 0, policy_version 44730 (0.0008) [2023-10-08 09:34:41,235][53885] Updated weights for policy 1, policy_version 44512 (0.0008) [2023-10-08 09:34:42,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 91389952. Throughput: 0: 1831.6, 1: 1813.3. Samples: 22847138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:42,016][52710] Avg episode reward: [(0, '29.860'), (1, '30.220')] [2023-10-08 09:34:44,699][53852] Updated weights for policy 0, policy_version 44740 (0.0007) [2023-10-08 09:34:45,060][53852] Updated weights for policy 0, policy_version 44750 (0.0008) [2023-10-08 09:34:45,118][53885] Updated weights for policy 1, policy_version 44522 (0.0007) [2023-10-08 09:34:45,423][53852] Updated weights for policy 0, policy_version 44760 (0.0009) [2023-10-08 09:34:45,492][53885] Updated weights for policy 1, policy_version 44532 (0.0009) [2023-10-08 09:34:45,857][53885] Updated weights for policy 1, policy_version 44542 (0.0007) [2023-10-08 09:34:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 91455488. Throughput: 0: 1824.9, 1: 1815.3. Samples: 22867868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:47,015][52710] Avg episode reward: [(0, '30.890'), (1, '31.250')] [2023-10-08 09:34:49,118][53852] Updated weights for policy 0, policy_version 44770 (0.0009) [2023-10-08 09:34:49,491][53852] Updated weights for policy 0, policy_version 44780 (0.0008) [2023-10-08 09:34:49,579][53885] Updated weights for policy 1, policy_version 44552 (0.0009) [2023-10-08 09:34:49,868][53852] Updated weights for policy 0, policy_version 44790 (0.0007) [2023-10-08 09:34:49,948][53885] Updated weights for policy 1, policy_version 44562 (0.0009) [2023-10-08 09:34:50,235][53852] Updated weights for policy 0, policy_version 44800 (0.0008) [2023-10-08 09:34:50,323][53885] Updated weights for policy 1, policy_version 44572 (0.0009) [2023-10-08 09:34:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91521024. Throughput: 0: 1834.7, 1: 1808.3. Samples: 22890192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:52,016][52710] Avg episode reward: [(0, '31.560'), (1, '32.340')] [2023-10-08 09:34:53,894][53852] Updated weights for policy 0, policy_version 44810 (0.0007) [2023-10-08 09:34:54,050][53885] Updated weights for policy 1, policy_version 44582 (0.0008) [2023-10-08 09:34:54,262][53852] Updated weights for policy 0, policy_version 44820 (0.0007) [2023-10-08 09:34:54,425][53885] Updated weights for policy 1, policy_version 44592 (0.0009) [2023-10-08 09:34:54,627][53852] Updated weights for policy 0, policy_version 44830 (0.0007) [2023-10-08 09:34:54,786][53885] Updated weights for policy 1, policy_version 44602 (0.0009) [2023-10-08 09:34:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 91586560. Throughput: 0: 1825.4, 1: 1821.6. Samples: 22900716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:34:57,016][52710] Avg episode reward: [(0, '32.250'), (1, '31.740')] [2023-10-08 09:34:58,369][53852] Updated weights for policy 0, policy_version 44840 (0.0008) [2023-10-08 09:34:58,557][53885] Updated weights for policy 1, policy_version 44612 (0.0010) [2023-10-08 09:34:58,741][53852] Updated weights for policy 0, policy_version 44850 (0.0007) [2023-10-08 09:34:58,927][53885] Updated weights for policy 1, policy_version 44622 (0.0009) [2023-10-08 09:34:59,113][53852] Updated weights for policy 0, policy_version 44860 (0.0007) [2023-10-08 09:34:59,298][53885] Updated weights for policy 1, policy_version 44632 (0.0010) [2023-10-08 09:35:02,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 91652096. Throughput: 0: 1834.7, 1: 1803.5. Samples: 22922494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:35:02,016][52710] Avg episode reward: [(0, '30.510'), (1, '30.610')] [2023-10-08 09:35:02,847][53852] Updated weights for policy 0, policy_version 44870 (0.0007) [2023-10-08 09:35:03,160][53885] Updated weights for policy 1, policy_version 44642 (0.0009) [2023-10-08 09:35:03,212][53852] Updated weights for policy 0, policy_version 44880 (0.0007) [2023-10-08 09:35:03,530][53885] Updated weights for policy 1, policy_version 44652 (0.0007) [2023-10-08 09:35:03,593][53852] Updated weights for policy 0, policy_version 44890 (0.0007) [2023-10-08 09:35:03,905][53885] Updated weights for policy 1, policy_version 44662 (0.0007) [2023-10-08 09:35:04,265][53885] Updated weights for policy 1, policy_version 44672 (0.0007) [2023-10-08 09:35:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91717632. Throughput: 0: 1833.8, 1: 1801.8. Samples: 22945220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:35:07,016][52710] Avg episode reward: [(0, '29.840'), (1, '32.560')] [2023-10-08 09:35:07,213][53852] Updated weights for policy 0, policy_version 44900 (0.0007) [2023-10-08 09:35:07,573][53852] Updated weights for policy 0, policy_version 44910 (0.0007) [2023-10-08 09:35:07,949][53885] Updated weights for policy 1, policy_version 44682 (0.0008) [2023-10-08 09:35:07,951][53852] Updated weights for policy 0, policy_version 44920 (0.0008) [2023-10-08 09:35:08,308][53885] Updated weights for policy 1, policy_version 44692 (0.0007) [2023-10-08 09:35:08,682][53885] Updated weights for policy 1, policy_version 44702 (0.0011) [2023-10-08 09:35:11,538][53852] Updated weights for policy 0, policy_version 44930 (0.0009) [2023-10-08 09:35:11,903][53852] Updated weights for policy 0, policy_version 44940 (0.0008) [2023-10-08 09:35:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91783168. Throughput: 0: 1833.5, 1: 1798.9. Samples: 22955080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:35:12,016][52710] Avg episode reward: [(0, '30.560'), (1, '33.830')] [2023-10-08 09:35:12,274][53852] Updated weights for policy 0, policy_version 44950 (0.0008) [2023-10-08 09:35:12,405][53885] Updated weights for policy 1, policy_version 44712 (0.0009) [2023-10-08 09:35:12,639][53852] Updated weights for policy 0, policy_version 44960 (0.0007) [2023-10-08 09:35:12,773][53885] Updated weights for policy 1, policy_version 44722 (0.0008) [2023-10-08 09:35:13,143][53885] Updated weights for policy 1, policy_version 44732 (0.0008) [2023-10-08 09:35:16,222][53852] Updated weights for policy 0, policy_version 44970 (0.0008) [2023-10-08 09:35:16,601][53852] Updated weights for policy 0, policy_version 44980 (0.0008) [2023-10-08 09:35:16,698][53885] Updated weights for policy 1, policy_version 44742 (0.0007) [2023-10-08 09:35:16,974][53852] Updated weights for policy 0, policy_version 44990 (0.0008) [2023-10-08 09:35:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 91848704. Throughput: 0: 1836.7, 1: 1801.9. Samples: 22978078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:35:17,016][52710] Avg episode reward: [(0, '29.930'), (1, '34.180')] [2023-10-08 09:35:17,071][53885] Updated weights for policy 1, policy_version 44752 (0.0007) [2023-10-08 09:35:17,448][53885] Updated weights for policy 1, policy_version 44762 (0.0007) [2023-10-08 09:35:20,735][53852] Updated weights for policy 0, policy_version 45000 (0.0010) [2023-10-08 09:35:20,996][53885] Updated weights for policy 1, policy_version 44772 (0.0008) [2023-10-08 09:35:21,097][53852] Updated weights for policy 0, policy_version 45010 (0.0009) [2023-10-08 09:35:21,360][53885] Updated weights for policy 1, policy_version 44782 (0.0009) [2023-10-08 09:35:21,476][53852] Updated weights for policy 0, policy_version 45020 (0.0007) [2023-10-08 09:35:21,729][53885] Updated weights for policy 1, policy_version 44792 (0.0009) [2023-10-08 09:35:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91947008. Throughput: 0: 1821.6, 1: 1815.6. Samples: 22998556. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-08 09:35:22,016][52710] Avg episode reward: [(0, '31.270'), (1, '32.010')] [2023-10-08 09:35:25,224][53852] Updated weights for policy 0, policy_version 45030 (0.0007) [2023-10-08 09:35:25,396][53885] Updated weights for policy 1, policy_version 44802 (0.0008) [2023-10-08 09:35:25,584][53852] Updated weights for policy 0, policy_version 45040 (0.0008) [2023-10-08 09:35:25,763][53885] Updated weights for policy 1, policy_version 44812 (0.0008) [2023-10-08 09:35:25,954][53852] Updated weights for policy 0, policy_version 45050 (0.0009) [2023-10-08 09:35:26,140][53885] Updated weights for policy 1, policy_version 44822 (0.0009) [2023-10-08 09:35:26,509][53885] Updated weights for policy 1, policy_version 44832 (0.0010) [2023-10-08 09:35:27,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 92045312. Throughput: 0: 1828.1, 1: 1805.6. Samples: 23010654. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-08 09:35:27,015][52710] Avg episode reward: [(0, '31.420'), (1, '31.910')] [2023-10-08 09:35:29,581][53852] Updated weights for policy 0, policy_version 45060 (0.0008) [2023-10-08 09:35:29,950][53852] Updated weights for policy 0, policy_version 45070 (0.0009) [2023-10-08 09:35:30,225][53885] Updated weights for policy 1, policy_version 44842 (0.0007) [2023-10-08 09:35:30,310][53852] Updated weights for policy 0, policy_version 45080 (0.0008) [2023-10-08 09:35:30,589][53885] Updated weights for policy 1, policy_version 44852 (0.0007) [2023-10-08 09:35:30,951][53885] Updated weights for policy 1, policy_version 44862 (0.0008) [2023-10-08 09:35:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 92110848. Throughput: 0: 1819.5, 1: 1813.2. Samples: 23031342. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-08 09:35:32,016][52710] Avg episode reward: [(0, '29.930'), (1, '35.110')] [2023-10-08 09:35:34,087][53852] Updated weights for policy 0, policy_version 45090 (0.0008) [2023-10-08 09:35:34,452][53852] Updated weights for policy 0, policy_version 45100 (0.0007) [2023-10-08 09:35:34,629][53885] Updated weights for policy 1, policy_version 44872 (0.0008) [2023-10-08 09:35:34,813][53852] Updated weights for policy 0, policy_version 45110 (0.0008) [2023-10-08 09:35:34,999][53885] Updated weights for policy 1, policy_version 44882 (0.0008) [2023-10-08 09:35:35,186][53852] Updated weights for policy 0, policy_version 45120 (0.0008) [2023-10-08 09:35:35,359][53885] Updated weights for policy 1, policy_version 44892 (0.0009) [2023-10-08 09:35:37,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 92176384. Throughput: 0: 1822.8, 1: 1809.0. Samples: 23053624. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-08 09:35:37,016][52710] Avg episode reward: [(0, '29.270'), (1, '34.480')] [2023-10-08 09:35:38,724][53852] Updated weights for policy 0, policy_version 45130 (0.0009) [2023-10-08 09:35:39,042][53885] Updated weights for policy 1, policy_version 44902 (0.0007) [2023-10-08 09:35:39,082][53852] Updated weights for policy 0, policy_version 45140 (0.0008) [2023-10-08 09:35:39,406][53885] Updated weights for policy 1, policy_version 44912 (0.0007) [2023-10-08 09:35:39,459][53852] Updated weights for policy 0, policy_version 45150 (0.0008) [2023-10-08 09:35:39,765][53885] Updated weights for policy 1, policy_version 44922 (0.0009) [2023-10-08 09:35:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 92241920. Throughput: 0: 1821.6, 1: 1809.3. Samples: 23064104. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-08 09:35:42,017][52710] Avg episode reward: [(0, '29.460'), (1, '33.950')] [2023-10-08 09:35:43,210][53852] Updated weights for policy 0, policy_version 45160 (0.0009) [2023-10-08 09:35:43,576][53852] Updated weights for policy 0, policy_version 45170 (0.0009) [2023-10-08 09:35:43,700][53885] Updated weights for policy 1, policy_version 44932 (0.0009) [2023-10-08 09:35:43,945][53852] Updated weights for policy 0, policy_version 45180 (0.0009) [2023-10-08 09:35:44,060][53885] Updated weights for policy 1, policy_version 44942 (0.0009) [2023-10-08 09:35:44,430][53885] Updated weights for policy 1, policy_version 44952 (0.0010) [2023-10-08 09:35:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 92307456. Throughput: 0: 1822.7, 1: 1807.3. Samples: 23085844. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) [2023-10-08 09:35:47,016][52710] Avg episode reward: [(0, '28.500'), (1, '33.940')] [2023-10-08 09:35:47,512][53852] Updated weights for policy 0, policy_version 45190 (0.0008) [2023-10-08 09:35:47,892][53852] Updated weights for policy 0, policy_version 45200 (0.0008) [2023-10-08 09:35:48,098][53885] Updated weights for policy 1, policy_version 44962 (0.0008) [2023-10-08 09:35:48,256][53852] Updated weights for policy 0, policy_version 45210 (0.0009) [2023-10-08 09:35:48,461][53885] Updated weights for policy 1, policy_version 44972 (0.0007) [2023-10-08 09:35:48,829][53885] Updated weights for policy 1, policy_version 44982 (0.0007) [2023-10-08 09:35:49,198][53885] Updated weights for policy 1, policy_version 44992 (0.0011) [2023-10-08 09:35:51,872][53852] Updated weights for policy 0, policy_version 45220 (0.0008) [2023-10-08 09:35:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 92372992. Throughput: 0: 1826.7, 1: 1811.8. Samples: 23108952. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 09:35:52,016][52710] Avg episode reward: [(0, '29.870'), (1, '29.890')] [2023-10-08 09:35:52,241][53852] Updated weights for policy 0, policy_version 45230 (0.0007) [2023-10-08 09:35:52,612][53852] Updated weights for policy 0, policy_version 45240 (0.0009) [2023-10-08 09:35:52,947][53885] Updated weights for policy 1, policy_version 45002 (0.0007) [2023-10-08 09:35:53,307][53885] Updated weights for policy 1, policy_version 45012 (0.0008) [2023-10-08 09:35:53,681][53885] Updated weights for policy 1, policy_version 45022 (0.0007) [2023-10-08 09:35:56,337][53852] Updated weights for policy 0, policy_version 45250 (0.0009) [2023-10-08 09:35:56,701][53852] Updated weights for policy 0, policy_version 45260 (0.0009) [2023-10-08 09:35:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 92438528. Throughput: 0: 1825.3, 1: 1813.5. Samples: 23118826. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 09:35:57,016][52710] Avg episode reward: [(0, '31.100'), (1, '30.720')] [2023-10-08 09:35:57,069][53852] Updated weights for policy 0, policy_version 45270 (0.0008) [2023-10-08 09:35:57,300][53885] Updated weights for policy 1, policy_version 45032 (0.0007) [2023-10-08 09:35:57,445][53852] Updated weights for policy 0, policy_version 45280 (0.0007) [2023-10-08 09:35:57,675][53885] Updated weights for policy 1, policy_version 45042 (0.0011) [2023-10-08 09:35:58,051][53885] Updated weights for policy 1, policy_version 45052 (0.0011) [2023-10-08 09:36:01,140][53852] Updated weights for policy 0, policy_version 45290 (0.0009) [2023-10-08 09:36:01,512][53852] Updated weights for policy 0, policy_version 45300 (0.0007) [2023-10-08 09:36:01,722][53885] Updated weights for policy 1, policy_version 45062 (0.0010) [2023-10-08 09:36:01,878][53852] Updated weights for policy 0, policy_version 45310 (0.0010) [2023-10-08 09:36:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92536832. Throughput: 0: 1820.3, 1: 1817.7. Samples: 23141788. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 09:36:02,016][52710] Avg episode reward: [(0, '31.860'), (1, '30.190')] [2023-10-08 09:36:02,099][53885] Updated weights for policy 1, policy_version 45072 (0.0009) [2023-10-08 09:36:02,457][53885] Updated weights for policy 1, policy_version 45082 (0.0010) [2023-10-08 09:36:05,650][53852] Updated weights for policy 0, policy_version 45320 (0.0008) [2023-10-08 09:36:06,018][53852] Updated weights for policy 0, policy_version 45330 (0.0009) [2023-10-08 09:36:06,168][53885] Updated weights for policy 1, policy_version 45092 (0.0008) [2023-10-08 09:36:06,387][53852] Updated weights for policy 0, policy_version 45340 (0.0008) [2023-10-08 09:36:06,537][53885] Updated weights for policy 1, policy_version 45102 (0.0008) [2023-10-08 09:36:06,906][53885] Updated weights for policy 1, policy_version 45112 (0.0009) [2023-10-08 09:36:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92602368. Throughput: 0: 1824.3, 1: 1820.0. Samples: 23162548. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 09:36:07,016][52710] Avg episode reward: [(0, '31.530'), (1, '32.280')] [2023-10-08 09:36:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000045344_46432256.pth... [2023-10-08 09:36:07,055][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000043616_44662784.pth [2023-10-08 09:36:07,198][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000045120_46202880.pth... [2023-10-08 09:36:07,238][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000043392_44433408.pth [2023-10-08 09:36:09,865][53852] Updated weights for policy 0, policy_version 45350 (0.0008) [2023-10-08 09:36:10,228][53852] Updated weights for policy 0, policy_version 45360 (0.0008) [2023-10-08 09:36:10,451][53885] Updated weights for policy 1, policy_version 45122 (0.0007) [2023-10-08 09:36:10,602][53852] Updated weights for policy 0, policy_version 45370 (0.0007) [2023-10-08 09:36:10,814][53885] Updated weights for policy 1, policy_version 45132 (0.0008) [2023-10-08 09:36:11,176][53885] Updated weights for policy 1, policy_version 45142 (0.0008) [2023-10-08 09:36:11,540][53885] Updated weights for policy 1, policy_version 45152 (0.0008) [2023-10-08 09:36:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 92700672. Throughput: 0: 1834.4, 1: 1820.1. Samples: 23175106. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 09:36:12,016][52710] Avg episode reward: [(0, '32.420'), (1, '32.180')] [2023-10-08 09:36:14,208][53852] Updated weights for policy 0, policy_version 45380 (0.0009) [2023-10-08 09:36:14,577][53852] Updated weights for policy 0, policy_version 45390 (0.0008) [2023-10-08 09:36:14,947][53852] Updated weights for policy 0, policy_version 45400 (0.0009) [2023-10-08 09:36:15,196][53885] Updated weights for policy 1, policy_version 45162 (0.0007) [2023-10-08 09:36:15,562][53885] Updated weights for policy 1, policy_version 45172 (0.0009) [2023-10-08 09:36:15,933][53885] Updated weights for policy 1, policy_version 45182 (0.0008) [2023-10-08 09:36:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 92766208. Throughput: 0: 1835.6, 1: 1818.8. Samples: 23195792. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) [2023-10-08 09:36:17,016][52710] Avg episode reward: [(0, '28.410'), (1, '32.980')] [2023-10-08 09:36:18,488][53852] Updated weights for policy 0, policy_version 45410 (0.0008) [2023-10-08 09:36:18,859][53852] Updated weights for policy 0, policy_version 45420 (0.0007) [2023-10-08 09:36:19,234][53852] Updated weights for policy 0, policy_version 45430 (0.0007) [2023-10-08 09:36:19,564][53885] Updated weights for policy 1, policy_version 45192 (0.0007) [2023-10-08 09:36:19,603][53852] Updated weights for policy 0, policy_version 45440 (0.0009) [2023-10-08 09:36:19,935][53885] Updated weights for policy 1, policy_version 45202 (0.0008) [2023-10-08 09:36:20,302][53885] Updated weights for policy 1, policy_version 45212 (0.0007) [2023-10-08 09:36:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92831744. Throughput: 0: 1841.6, 1: 1823.5. Samples: 23218552. Policy #0 lag: (min: 9.0, avg: 15.7, max: 41.0) [2023-10-08 09:36:22,016][52710] Avg episode reward: [(0, '30.450'), (1, '31.710')] [2023-10-08 09:36:23,247][53852] Updated weights for policy 0, policy_version 45450 (0.0007) [2023-10-08 09:36:23,607][53852] Updated weights for policy 0, policy_version 45460 (0.0008) [2023-10-08 09:36:23,971][53885] Updated weights for policy 1, policy_version 45222 (0.0008) [2023-10-08 09:36:23,983][53852] Updated weights for policy 0, policy_version 45470 (0.0008) [2023-10-08 09:36:24,344][53885] Updated weights for policy 1, policy_version 45232 (0.0009) [2023-10-08 09:36:24,712][53885] Updated weights for policy 1, policy_version 45242 (0.0008) [2023-10-08 09:36:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 92897280. Throughput: 0: 1840.9, 1: 1824.4. Samples: 23229042. Policy #0 lag: (min: 9.0, avg: 15.7, max: 41.0) [2023-10-08 09:36:27,016][52710] Avg episode reward: [(0, '29.810'), (1, '34.280')] [2023-10-08 09:36:27,662][53852] Updated weights for policy 0, policy_version 45480 (0.0010) [2023-10-08 09:36:28,034][53852] Updated weights for policy 0, policy_version 45490 (0.0007) [2023-10-08 09:36:28,356][53885] Updated weights for policy 1, policy_version 45252 (0.0008) [2023-10-08 09:36:28,394][53852] Updated weights for policy 0, policy_version 45500 (0.0007) [2023-10-08 09:36:28,726][53885] Updated weights for policy 1, policy_version 45262 (0.0008) [2023-10-08 09:36:29,087][53885] Updated weights for policy 1, policy_version 45272 (0.0008) [2023-10-08 09:36:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 92962816. Throughput: 0: 1842.8, 1: 1834.2. Samples: 23251306. Policy #0 lag: (min: 9.0, avg: 15.7, max: 41.0) [2023-10-08 09:36:32,016][52710] Avg episode reward: [(0, '29.000'), (1, '33.920')] [2023-10-08 09:36:32,118][53852] Updated weights for policy 0, policy_version 45510 (0.0009) [2023-10-08 09:36:32,493][53852] Updated weights for policy 0, policy_version 45520 (0.0008) [2023-10-08 09:36:32,791][53885] Updated weights for policy 1, policy_version 45282 (0.0007) [2023-10-08 09:36:32,860][53852] Updated weights for policy 0, policy_version 45530 (0.0008) [2023-10-08 09:36:33,157][53885] Updated weights for policy 1, policy_version 45292 (0.0007) [2023-10-08 09:36:33,519][53885] Updated weights for policy 1, policy_version 45302 (0.0009) [2023-10-08 09:36:33,884][53885] Updated weights for policy 1, policy_version 45312 (0.0009) [2023-10-08 09:36:36,564][53852] Updated weights for policy 0, policy_version 45540 (0.0009) [2023-10-08 09:36:36,940][53852] Updated weights for policy 0, policy_version 45550 (0.0008) [2023-10-08 09:36:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 93028352. Throughput: 0: 1827.4, 1: 1839.4. Samples: 23273958. Policy #0 lag: (min: 9.0, avg: 15.7, max: 41.0) [2023-10-08 09:36:37,015][52710] Avg episode reward: [(0, '31.160'), (1, '31.980')] [2023-10-08 09:36:37,317][53852] Updated weights for policy 0, policy_version 45560 (0.0009) [2023-10-08 09:36:37,536][53885] Updated weights for policy 1, policy_version 45322 (0.0007) [2023-10-08 09:36:37,900][53885] Updated weights for policy 1, policy_version 45332 (0.0009) [2023-10-08 09:36:38,278][53885] Updated weights for policy 1, policy_version 45342 (0.0010) [2023-10-08 09:36:41,128][53852] Updated weights for policy 0, policy_version 45570 (0.0008) [2023-10-08 09:36:41,501][53852] Updated weights for policy 0, policy_version 45580 (0.0009) [2023-10-08 09:36:41,864][53852] Updated weights for policy 0, policy_version 45590 (0.0008) [2023-10-08 09:36:42,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93093888. Throughput: 0: 1831.9, 1: 1836.2. Samples: 23283890. Policy #0 lag: (min: 9.0, avg: 15.7, max: 41.0) [2023-10-08 09:36:42,016][52710] Avg episode reward: [(0, '31.190'), (1, '32.390')] [2023-10-08 09:36:42,130][53885] Updated weights for policy 1, policy_version 45352 (0.0008) [2023-10-08 09:36:42,238][53852] Updated weights for policy 0, policy_version 45600 (0.0008) [2023-10-08 09:36:42,501][53885] Updated weights for policy 1, policy_version 45362 (0.0007) [2023-10-08 09:36:42,875][53885] Updated weights for policy 1, policy_version 45372 (0.0007) [2023-10-08 09:36:45,953][53852] Updated weights for policy 0, policy_version 45610 (0.0008) [2023-10-08 09:36:46,315][53852] Updated weights for policy 0, policy_version 45620 (0.0010) [2023-10-08 09:36:46,620][53885] Updated weights for policy 1, policy_version 45382 (0.0008) [2023-10-08 09:36:46,695][53852] Updated weights for policy 0, policy_version 45630 (0.0007) [2023-10-08 09:36:46,997][53885] Updated weights for policy 1, policy_version 45392 (0.0009) [2023-10-08 09:36:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93192192. Throughput: 0: 1828.7, 1: 1832.2. Samples: 23306528. Policy #0 lag: (min: 9.0, avg: 15.7, max: 41.0) [2023-10-08 09:36:47,016][52710] Avg episode reward: [(0, '31.240'), (1, '32.170')] [2023-10-08 09:36:47,364][53885] Updated weights for policy 1, policy_version 45402 (0.0009) [2023-10-08 09:36:50,259][53852] Updated weights for policy 0, policy_version 45640 (0.0008) [2023-10-08 09:36:50,627][53852] Updated weights for policy 0, policy_version 45650 (0.0010) [2023-10-08 09:36:50,990][53852] Updated weights for policy 0, policy_version 45660 (0.0009) [2023-10-08 09:36:51,047][53885] Updated weights for policy 1, policy_version 45412 (0.0008) [2023-10-08 09:36:51,411][53885] Updated weights for policy 1, policy_version 45422 (0.0009) [2023-10-08 09:36:51,774][53885] Updated weights for policy 1, policy_version 45432 (0.0010) [2023-10-08 09:36:52,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93257728. Throughput: 0: 1832.5, 1: 1829.4. Samples: 23327330. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-08 09:36:52,015][52710] Avg episode reward: [(0, '29.790'), (1, '26.650')] [2023-10-08 09:36:54,795][53852] Updated weights for policy 0, policy_version 45670 (0.0009) [2023-10-08 09:36:55,175][53852] Updated weights for policy 0, policy_version 45680 (0.0008) [2023-10-08 09:36:55,339][53885] Updated weights for policy 1, policy_version 45442 (0.0010) [2023-10-08 09:36:55,546][53852] Updated weights for policy 0, policy_version 45690 (0.0009) [2023-10-08 09:36:55,702][53885] Updated weights for policy 1, policy_version 45452 (0.0011) [2023-10-08 09:36:56,072][53885] Updated weights for policy 1, policy_version 45462 (0.0009) [2023-10-08 09:36:56,439][53885] Updated weights for policy 1, policy_version 45472 (0.0010) [2023-10-08 09:36:57,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 93356032. Throughput: 0: 1827.9, 1: 1829.3. Samples: 23339680. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-08 09:36:57,016][52710] Avg episode reward: [(0, '29.900'), (1, '28.360')] [2023-10-08 09:36:59,185][53852] Updated weights for policy 0, policy_version 45700 (0.0009) [2023-10-08 09:36:59,547][53852] Updated weights for policy 0, policy_version 45710 (0.0009) [2023-10-08 09:36:59,922][53852] Updated weights for policy 0, policy_version 45720 (0.0007) [2023-10-08 09:37:00,023][53885] Updated weights for policy 1, policy_version 45482 (0.0007) [2023-10-08 09:37:00,383][53885] Updated weights for policy 1, policy_version 45492 (0.0009) [2023-10-08 09:37:00,752][53885] Updated weights for policy 1, policy_version 45502 (0.0007) [2023-10-08 09:37:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93421568. Throughput: 0: 1822.7, 1: 1828.0. Samples: 23360074. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-08 09:37:02,016][52710] Avg episode reward: [(0, '32.180'), (1, '30.290')] [2023-10-08 09:37:03,596][53852] Updated weights for policy 0, policy_version 45730 (0.0008) [2023-10-08 09:37:03,955][53852] Updated weights for policy 0, policy_version 45740 (0.0007) [2023-10-08 09:37:04,302][53885] Updated weights for policy 1, policy_version 45512 (0.0008) [2023-10-08 09:37:04,316][53852] Updated weights for policy 0, policy_version 45750 (0.0007) [2023-10-08 09:37:04,668][53885] Updated weights for policy 1, policy_version 45522 (0.0007) [2023-10-08 09:37:04,683][53852] Updated weights for policy 0, policy_version 45760 (0.0007) [2023-10-08 09:37:05,035][53885] Updated weights for policy 1, policy_version 45532 (0.0007) [2023-10-08 09:37:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93487104. Throughput: 0: 1817.9, 1: 1833.6. Samples: 23382872. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-08 09:37:07,016][52710] Avg episode reward: [(0, '30.480'), (1, '31.740')] [2023-10-08 09:37:08,409][53852] Updated weights for policy 0, policy_version 45770 (0.0008) [2023-10-08 09:37:08,753][53885] Updated weights for policy 1, policy_version 45542 (0.0009) [2023-10-08 09:37:08,790][53852] Updated weights for policy 0, policy_version 45780 (0.0008) [2023-10-08 09:37:09,117][53885] Updated weights for policy 1, policy_version 45552 (0.0008) [2023-10-08 09:37:09,155][53852] Updated weights for policy 0, policy_version 45790 (0.0008) [2023-10-08 09:37:09,481][53885] Updated weights for policy 1, policy_version 45562 (0.0010) [2023-10-08 09:37:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 93552640. Throughput: 0: 1814.4, 1: 1830.0. Samples: 23393038. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-08 09:37:12,016][52710] Avg episode reward: [(0, '30.230'), (1, '30.870')] [2023-10-08 09:37:12,909][53852] Updated weights for policy 0, policy_version 45800 (0.0010) [2023-10-08 09:37:13,161][53885] Updated weights for policy 1, policy_version 45572 (0.0009) [2023-10-08 09:37:13,283][53852] Updated weights for policy 0, policy_version 45810 (0.0009) [2023-10-08 09:37:13,515][53885] Updated weights for policy 1, policy_version 45582 (0.0007) [2023-10-08 09:37:13,648][53852] Updated weights for policy 0, policy_version 45820 (0.0009) [2023-10-08 09:37:13,881][53885] Updated weights for policy 1, policy_version 45592 (0.0008) [2023-10-08 09:37:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 93618176. Throughput: 0: 1818.4, 1: 1838.8. Samples: 23415882. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-08 09:37:17,016][52710] Avg episode reward: [(0, '32.110'), (1, '29.220')] [2023-10-08 09:37:17,317][53852] Updated weights for policy 0, policy_version 45830 (0.0009) [2023-10-08 09:37:17,536][53885] Updated weights for policy 1, policy_version 45602 (0.0008) [2023-10-08 09:37:17,690][53852] Updated weights for policy 0, policy_version 45840 (0.0007) [2023-10-08 09:37:17,902][53885] Updated weights for policy 1, policy_version 45612 (0.0008) [2023-10-08 09:37:18,055][53852] Updated weights for policy 0, policy_version 45850 (0.0007) [2023-10-08 09:37:18,258][53885] Updated weights for policy 1, policy_version 45622 (0.0009) [2023-10-08 09:37:18,619][53885] Updated weights for policy 1, policy_version 45632 (0.0008) [2023-10-08 09:37:21,778][53852] Updated weights for policy 0, policy_version 45860 (0.0007) [2023-10-08 09:37:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93683712. Throughput: 0: 1823.6, 1: 1836.3. Samples: 23438656. Policy #0 lag: (min: 23.0, avg: 23.1, max: 28.0) [2023-10-08 09:37:22,015][52710] Avg episode reward: [(0, '33.030'), (1, '32.290')] [2023-10-08 09:37:22,153][53852] Updated weights for policy 0, policy_version 45870 (0.0007) [2023-10-08 09:37:22,368][53885] Updated weights for policy 1, policy_version 45642 (0.0007) [2023-10-08 09:37:22,510][53852] Updated weights for policy 0, policy_version 45880 (0.0007) [2023-10-08 09:37:22,735][53885] Updated weights for policy 1, policy_version 45652 (0.0007) [2023-10-08 09:37:23,107][53885] Updated weights for policy 1, policy_version 45662 (0.0007) [2023-10-08 09:37:26,222][53852] Updated weights for policy 0, policy_version 45890 (0.0008) [2023-10-08 09:37:26,595][53852] Updated weights for policy 0, policy_version 45900 (0.0010) [2023-10-08 09:37:26,888][53885] Updated weights for policy 1, policy_version 45672 (0.0009) [2023-10-08 09:37:26,956][53852] Updated weights for policy 0, policy_version 45910 (0.0009) [2023-10-08 09:37:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 93749248. Throughput: 0: 1822.3, 1: 1837.4. Samples: 23448574. Policy #0 lag: (min: 2.0, avg: 7.7, max: 34.0) [2023-10-08 09:37:27,016][52710] Avg episode reward: [(0, '31.230'), (1, '30.980')] [2023-10-08 09:37:27,266][53885] Updated weights for policy 1, policy_version 45682 (0.0009) [2023-10-08 09:37:27,330][53852] Updated weights for policy 0, policy_version 45920 (0.0009) [2023-10-08 09:37:27,637][53885] Updated weights for policy 1, policy_version 45692 (0.0008) [2023-10-08 09:37:31,217][53852] Updated weights for policy 0, policy_version 45930 (0.0008) [2023-10-08 09:37:31,420][53885] Updated weights for policy 1, policy_version 45702 (0.0008) [2023-10-08 09:37:31,589][53852] Updated weights for policy 0, policy_version 45940 (0.0009) [2023-10-08 09:37:31,807][53885] Updated weights for policy 1, policy_version 45712 (0.0008) [2023-10-08 09:37:31,959][53852] Updated weights for policy 0, policy_version 45950 (0.0009) [2023-10-08 09:37:32,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 93814784. Throughput: 0: 1820.2, 1: 1837.7. Samples: 23471134. Policy #0 lag: (min: 2.0, avg: 7.7, max: 34.0) [2023-10-08 09:37:32,016][52710] Avg episode reward: [(0, '32.020'), (1, '28.860')] [2023-10-08 09:37:32,178][53885] Updated weights for policy 1, policy_version 45722 (0.0009) [2023-10-08 09:37:35,527][53852] Updated weights for policy 0, policy_version 45960 (0.0009) [2023-10-08 09:37:35,828][53885] Updated weights for policy 1, policy_version 45732 (0.0007) [2023-10-08 09:37:35,908][53852] Updated weights for policy 0, policy_version 45970 (0.0007) [2023-10-08 09:37:36,193][53885] Updated weights for policy 1, policy_version 45742 (0.0009) [2023-10-08 09:37:36,286][53852] Updated weights for policy 0, policy_version 45980 (0.0007) [2023-10-08 09:37:36,573][53885] Updated weights for policy 1, policy_version 45752 (0.0010) [2023-10-08 09:37:37,015][52710] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 93945856. Throughput: 0: 1815.2, 1: 1825.1. Samples: 23491140. Policy #0 lag: (min: 2.0, avg: 7.7, max: 34.0) [2023-10-08 09:37:37,015][52710] Avg episode reward: [(0, '32.320'), (1, '31.590')] [2023-10-08 09:37:39,976][53852] Updated weights for policy 0, policy_version 45990 (0.0009) [2023-10-08 09:37:40,129][53885] Updated weights for policy 1, policy_version 45762 (0.0007) [2023-10-08 09:37:40,342][53852] Updated weights for policy 0, policy_version 46000 (0.0008) [2023-10-08 09:37:40,495][53885] Updated weights for policy 1, policy_version 45772 (0.0008) [2023-10-08 09:37:40,713][53852] Updated weights for policy 0, policy_version 46010 (0.0009) [2023-10-08 09:37:40,861][53885] Updated weights for policy 1, policy_version 45782 (0.0009) [2023-10-08 09:37:41,226][53885] Updated weights for policy 1, policy_version 45792 (0.0009) [2023-10-08 09:37:42,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 94011392. Throughput: 0: 1812.6, 1: 1833.2. Samples: 23503744. Policy #0 lag: (min: 2.0, avg: 7.7, max: 34.0) [2023-10-08 09:37:42,016][52710] Avg episode reward: [(0, '30.710'), (1, '32.040')] [2023-10-08 09:37:44,358][53852] Updated weights for policy 0, policy_version 46020 (0.0010) [2023-10-08 09:37:44,728][53852] Updated weights for policy 0, policy_version 46030 (0.0008) [2023-10-08 09:37:44,941][53885] Updated weights for policy 1, policy_version 45802 (0.0007) [2023-10-08 09:37:45,106][53852] Updated weights for policy 0, policy_version 46040 (0.0008) [2023-10-08 09:37:45,309][53885] Updated weights for policy 1, policy_version 45812 (0.0007) [2023-10-08 09:37:45,677][53885] Updated weights for policy 1, policy_version 45822 (0.0009) [2023-10-08 09:37:47,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 94076928. Throughput: 0: 1813.7, 1: 1828.0. Samples: 23523952. Policy #0 lag: (min: 2.0, avg: 7.7, max: 34.0) [2023-10-08 09:37:47,016][52710] Avg episode reward: [(0, '32.550'), (1, '28.700')] [2023-10-08 09:37:48,665][53852] Updated weights for policy 0, policy_version 46050 (0.0010) [2023-10-08 09:37:49,026][53852] Updated weights for policy 0, policy_version 46060 (0.0010) [2023-10-08 09:37:49,391][53885] Updated weights for policy 1, policy_version 45832 (0.0007) [2023-10-08 09:37:49,397][53852] Updated weights for policy 0, policy_version 46070 (0.0008) [2023-10-08 09:37:49,756][53885] Updated weights for policy 1, policy_version 45842 (0.0007) [2023-10-08 09:37:49,771][53852] Updated weights for policy 0, policy_version 46080 (0.0008) [2023-10-08 09:37:50,120][53885] Updated weights for policy 1, policy_version 45852 (0.0007) [2023-10-08 09:37:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94142464. Throughput: 0: 1811.7, 1: 1817.5. Samples: 23546184. Policy #0 lag: (min: 2.0, avg: 7.7, max: 34.0) [2023-10-08 09:37:52,016][52710] Avg episode reward: [(0, '31.050'), (1, '28.460')] [2023-10-08 09:37:53,515][53852] Updated weights for policy 0, policy_version 46090 (0.0010) [2023-10-08 09:37:53,737][53885] Updated weights for policy 1, policy_version 45862 (0.0007) [2023-10-08 09:37:53,888][53852] Updated weights for policy 0, policy_version 46100 (0.0007) [2023-10-08 09:37:54,102][53885] Updated weights for policy 1, policy_version 45872 (0.0008) [2023-10-08 09:37:54,257][53852] Updated weights for policy 0, policy_version 46110 (0.0007) [2023-10-08 09:37:54,473][53885] Updated weights for policy 1, policy_version 45882 (0.0009) [2023-10-08 09:37:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 94208000. Throughput: 0: 1813.3, 1: 1815.5. Samples: 23556332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:37:57,016][52710] Avg episode reward: [(0, '29.670'), (1, '27.360')] [2023-10-08 09:37:57,746][53852] Updated weights for policy 0, policy_version 46120 (0.0010) [2023-10-08 09:37:58,115][53852] Updated weights for policy 0, policy_version 46130 (0.0009) [2023-10-08 09:37:58,164][53885] Updated weights for policy 1, policy_version 45892 (0.0008) [2023-10-08 09:37:58,489][53852] Updated weights for policy 0, policy_version 46140 (0.0007) [2023-10-08 09:37:58,535][53885] Updated weights for policy 1, policy_version 45902 (0.0009) [2023-10-08 09:37:58,902][53885] Updated weights for policy 1, policy_version 45912 (0.0011) [2023-10-08 09:38:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94273536. Throughput: 0: 1817.3, 1: 1817.5. Samples: 23579448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:38:02,015][52710] Avg episode reward: [(0, '28.280'), (1, '24.730')] [2023-10-08 09:38:02,078][53852] Updated weights for policy 0, policy_version 46150 (0.0008) [2023-10-08 09:38:02,453][53852] Updated weights for policy 0, policy_version 46160 (0.0009) [2023-10-08 09:38:02,693][53885] Updated weights for policy 1, policy_version 45922 (0.0008) [2023-10-08 09:38:02,823][53852] Updated weights for policy 0, policy_version 46170 (0.0007) [2023-10-08 09:38:03,063][53885] Updated weights for policy 1, policy_version 45932 (0.0007) [2023-10-08 09:38:03,434][53885] Updated weights for policy 1, policy_version 45942 (0.0009) [2023-10-08 09:38:03,797][53885] Updated weights for policy 1, policy_version 45952 (0.0011) [2023-10-08 09:38:06,485][53852] Updated weights for policy 0, policy_version 46180 (0.0009) [2023-10-08 09:38:06,861][53852] Updated weights for policy 0, policy_version 46190 (0.0009) [2023-10-08 09:38:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94339072. Throughput: 0: 1818.5, 1: 1815.0. Samples: 23602162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:38:07,016][52710] Avg episode reward: [(0, '28.300'), (1, '25.230')] [2023-10-08 09:38:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000045952_47054848.pth... [2023-10-08 09:38:07,056][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000044256_45318144.pth [2023-10-08 09:38:07,233][53852] Updated weights for policy 0, policy_version 46200 (0.0008) [2023-10-08 09:38:07,530][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000046208_47316992.pth... [2023-10-08 09:38:07,536][53885] Updated weights for policy 1, policy_version 45962 (0.0008) [2023-10-08 09:38:07,566][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000044480_45547520.pth [2023-10-08 09:38:07,910][53885] Updated weights for policy 1, policy_version 45972 (0.0008) [2023-10-08 09:38:08,277][53885] Updated weights for policy 1, policy_version 45982 (0.0008) [2023-10-08 09:38:10,853][53852] Updated weights for policy 0, policy_version 46210 (0.0008) [2023-10-08 09:38:11,216][53852] Updated weights for policy 0, policy_version 46220 (0.0007) [2023-10-08 09:38:11,586][53852] Updated weights for policy 0, policy_version 46230 (0.0008) [2023-10-08 09:38:11,883][53885] Updated weights for policy 1, policy_version 45992 (0.0008) [2023-10-08 09:38:11,957][53852] Updated weights for policy 0, policy_version 46240 (0.0007) [2023-10-08 09:38:12,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 94437376. Throughput: 0: 1823.7, 1: 1818.3. Samples: 23612466. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:38:12,016][52710] Avg episode reward: [(0, '28.970'), (1, '25.930')] [2023-10-08 09:38:12,240][53885] Updated weights for policy 1, policy_version 46002 (0.0011) [2023-10-08 09:38:12,606][53885] Updated weights for policy 1, policy_version 46012 (0.0010) [2023-10-08 09:38:15,617][53852] Updated weights for policy 0, policy_version 46250 (0.0010) [2023-10-08 09:38:15,985][53852] Updated weights for policy 0, policy_version 46260 (0.0008) [2023-10-08 09:38:16,310][53885] Updated weights for policy 1, policy_version 46022 (0.0010) [2023-10-08 09:38:16,359][53852] Updated weights for policy 0, policy_version 46270 (0.0007) [2023-10-08 09:38:16,694][53885] Updated weights for policy 1, policy_version 46032 (0.0009) [2023-10-08 09:38:17,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 94502912. Throughput: 0: 1820.8, 1: 1821.2. Samples: 23635024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:38:17,016][52710] Avg episode reward: [(0, '30.320'), (1, '26.730')] [2023-10-08 09:38:17,072][53885] Updated weights for policy 1, policy_version 46042 (0.0008) [2023-10-08 09:38:19,930][53852] Updated weights for policy 0, policy_version 46280 (0.0009) [2023-10-08 09:38:20,309][53852] Updated weights for policy 0, policy_version 46290 (0.0008) [2023-10-08 09:38:20,670][53852] Updated weights for policy 0, policy_version 46300 (0.0009) [2023-10-08 09:38:20,778][53885] Updated weights for policy 1, policy_version 46052 (0.0007) [2023-10-08 09:38:21,148][53885] Updated weights for policy 1, policy_version 46062 (0.0009) [2023-10-08 09:38:21,529][53885] Updated weights for policy 1, policy_version 46072 (0.0007) [2023-10-08 09:38:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 94601216. Throughput: 0: 1835.9, 1: 1823.3. Samples: 23655804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:38:22,016][52710] Avg episode reward: [(0, '28.750'), (1, '27.120')] [2023-10-08 09:38:24,269][53852] Updated weights for policy 0, policy_version 46310 (0.0007) [2023-10-08 09:38:24,644][53852] Updated weights for policy 0, policy_version 46320 (0.0008) [2023-10-08 09:38:25,025][53852] Updated weights for policy 0, policy_version 46330 (0.0010) [2023-10-08 09:38:25,144][53885] Updated weights for policy 1, policy_version 46082 (0.0008) [2023-10-08 09:38:25,512][53885] Updated weights for policy 1, policy_version 46092 (0.0009) [2023-10-08 09:38:25,878][53885] Updated weights for policy 1, policy_version 46102 (0.0008) [2023-10-08 09:38:26,252][53885] Updated weights for policy 1, policy_version 46112 (0.0010) [2023-10-08 09:38:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 94666752. Throughput: 0: 1828.7, 1: 1821.9. Samples: 23668022. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:38:27,016][52710] Avg episode reward: [(0, '32.060'), (1, '27.610')] [2023-10-08 09:38:28,693][53852] Updated weights for policy 0, policy_version 46340 (0.0008) [2023-10-08 09:38:29,060][53852] Updated weights for policy 0, policy_version 46350 (0.0007) [2023-10-08 09:38:29,428][53852] Updated weights for policy 0, policy_version 46360 (0.0007) [2023-10-08 09:38:29,854][53885] Updated weights for policy 1, policy_version 46122 (0.0009) [2023-10-08 09:38:30,218][53885] Updated weights for policy 1, policy_version 46132 (0.0011) [2023-10-08 09:38:30,588][53885] Updated weights for policy 1, policy_version 46142 (0.0008) [2023-10-08 09:38:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 94732288. Throughput: 0: 1844.1, 1: 1817.3. Samples: 23688716. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:38:32,016][52710] Avg episode reward: [(0, '28.690'), (1, '30.280')] [2023-10-08 09:38:33,074][53852] Updated weights for policy 0, policy_version 46370 (0.0009) [2023-10-08 09:38:33,439][53852] Updated weights for policy 0, policy_version 46380 (0.0008) [2023-10-08 09:38:33,804][53852] Updated weights for policy 0, policy_version 46390 (0.0009) [2023-10-08 09:38:34,167][53852] Updated weights for policy 0, policy_version 46400 (0.0008) [2023-10-08 09:38:34,169][53885] Updated weights for policy 1, policy_version 46152 (0.0008) [2023-10-08 09:38:34,537][53885] Updated weights for policy 1, policy_version 46162 (0.0011) [2023-10-08 09:38:34,909][53885] Updated weights for policy 1, policy_version 46172 (0.0008) [2023-10-08 09:38:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 94797824. Throughput: 0: 1854.2, 1: 1826.9. Samples: 23711834. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:38:37,016][52710] Avg episode reward: [(0, '29.790'), (1, '26.150')] [2023-10-08 09:38:37,853][53852] Updated weights for policy 0, policy_version 46410 (0.0007) [2023-10-08 09:38:38,229][53852] Updated weights for policy 0, policy_version 46420 (0.0008) [2023-10-08 09:38:38,570][53885] Updated weights for policy 1, policy_version 46182 (0.0008) [2023-10-08 09:38:38,595][53852] Updated weights for policy 0, policy_version 46430 (0.0007) [2023-10-08 09:38:38,931][53885] Updated weights for policy 1, policy_version 46192 (0.0008) [2023-10-08 09:38:39,292][53885] Updated weights for policy 1, policy_version 46202 (0.0008) [2023-10-08 09:38:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 94863360. Throughput: 0: 1854.6, 1: 1825.4. Samples: 23721932. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:38:42,016][52710] Avg episode reward: [(0, '30.730'), (1, '26.260')] [2023-10-08 09:38:42,415][53852] Updated weights for policy 0, policy_version 46440 (0.0007) [2023-10-08 09:38:42,791][53852] Updated weights for policy 0, policy_version 46450 (0.0007) [2023-10-08 09:38:43,055][53885] Updated weights for policy 1, policy_version 46212 (0.0007) [2023-10-08 09:38:43,159][53852] Updated weights for policy 0, policy_version 46460 (0.0008) [2023-10-08 09:38:43,417][53885] Updated weights for policy 1, policy_version 46222 (0.0007) [2023-10-08 09:38:43,787][53885] Updated weights for policy 1, policy_version 46232 (0.0008) [2023-10-08 09:38:46,879][53852] Updated weights for policy 0, policy_version 46470 (0.0007) [2023-10-08 09:38:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94928896. Throughput: 0: 1848.8, 1: 1824.5. Samples: 23744748. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:38:47,016][52710] Avg episode reward: [(0, '31.030'), (1, '30.420')] [2023-10-08 09:38:47,248][53852] Updated weights for policy 0, policy_version 46480 (0.0007) [2023-10-08 09:38:47,538][53885] Updated weights for policy 1, policy_version 46242 (0.0009) [2023-10-08 09:38:47,623][53852] Updated weights for policy 0, policy_version 46490 (0.0007) [2023-10-08 09:38:47,906][53885] Updated weights for policy 1, policy_version 46252 (0.0009) [2023-10-08 09:38:48,257][53885] Updated weights for policy 1, policy_version 46262 (0.0009) [2023-10-08 09:38:48,630][53885] Updated weights for policy 1, policy_version 46272 (0.0009) [2023-10-08 09:38:51,216][53852] Updated weights for policy 0, policy_version 46500 (0.0010) [2023-10-08 09:38:51,587][53852] Updated weights for policy 0, policy_version 46510 (0.0009) [2023-10-08 09:38:51,962][53852] Updated weights for policy 0, policy_version 46520 (0.0008) [2023-10-08 09:38:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 94994432. Throughput: 0: 1833.8, 1: 1826.4. Samples: 23766870. Policy #0 lag: (min: 31.0, avg: 34.8, max: 63.0) [2023-10-08 09:38:52,016][52710] Avg episode reward: [(0, '29.320'), (1, '29.500')] [2023-10-08 09:38:52,196][53885] Updated weights for policy 1, policy_version 46282 (0.0007) [2023-10-08 09:38:52,561][53885] Updated weights for policy 1, policy_version 46292 (0.0009) [2023-10-08 09:38:52,933][53885] Updated weights for policy 1, policy_version 46302 (0.0009) [2023-10-08 09:38:55,658][53852] Updated weights for policy 0, policy_version 46530 (0.0008) [2023-10-08 09:38:56,018][53852] Updated weights for policy 0, policy_version 46540 (0.0007) [2023-10-08 09:38:56,379][53852] Updated weights for policy 0, policy_version 46550 (0.0009) [2023-10-08 09:38:56,483][53885] Updated weights for policy 1, policy_version 46312 (0.0008) [2023-10-08 09:38:56,750][53852] Updated weights for policy 0, policy_version 46560 (0.0007) [2023-10-08 09:38:56,846][53885] Updated weights for policy 1, policy_version 46322 (0.0007) [2023-10-08 09:38:57,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 95092736. Throughput: 0: 1839.1, 1: 1827.5. Samples: 23777460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:38:57,016][52710] Avg episode reward: [(0, '30.160'), (1, '30.790')] [2023-10-08 09:38:57,215][53885] Updated weights for policy 1, policy_version 46332 (0.0008) [2023-10-08 09:39:00,474][53852] Updated weights for policy 0, policy_version 46570 (0.0008) [2023-10-08 09:39:00,851][53852] Updated weights for policy 0, policy_version 46580 (0.0009) [2023-10-08 09:39:00,898][53885] Updated weights for policy 1, policy_version 46342 (0.0009) [2023-10-08 09:39:01,217][53852] Updated weights for policy 0, policy_version 46590 (0.0009) [2023-10-08 09:39:01,259][53885] Updated weights for policy 1, policy_version 46352 (0.0008) [2023-10-08 09:39:01,633][53885] Updated weights for policy 1, policy_version 46362 (0.0009) [2023-10-08 09:39:02,015][52710] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 95191040. Throughput: 0: 1833.0, 1: 1832.4. Samples: 23799964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:39:02,016][52710] Avg episode reward: [(0, '29.810'), (1, '33.990')] [2023-10-08 09:39:04,937][53852] Updated weights for policy 0, policy_version 46600 (0.0007) [2023-10-08 09:39:05,268][53885] Updated weights for policy 1, policy_version 46372 (0.0007) [2023-10-08 09:39:05,314][53852] Updated weights for policy 0, policy_version 46610 (0.0007) [2023-10-08 09:39:05,656][53885] Updated weights for policy 1, policy_version 46382 (0.0008) [2023-10-08 09:39:05,679][53852] Updated weights for policy 0, policy_version 46620 (0.0008) [2023-10-08 09:39:06,024][53885] Updated weights for policy 1, policy_version 46392 (0.0008) [2023-10-08 09:39:07,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 95256576. Throughput: 0: 1831.8, 1: 1824.2. Samples: 23820324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:39:07,016][52710] Avg episode reward: [(0, '32.130'), (1, '28.180')] [2023-10-08 09:39:09,207][53852] Updated weights for policy 0, policy_version 46630 (0.0008) [2023-10-08 09:39:09,575][53852] Updated weights for policy 0, policy_version 46640 (0.0010) [2023-10-08 09:39:09,717][53885] Updated weights for policy 1, policy_version 46402 (0.0007) [2023-10-08 09:39:09,943][53852] Updated weights for policy 0, policy_version 46650 (0.0009) [2023-10-08 09:39:10,085][53885] Updated weights for policy 1, policy_version 46412 (0.0008) [2023-10-08 09:39:10,447][53885] Updated weights for policy 1, policy_version 46422 (0.0008) [2023-10-08 09:39:10,815][53885] Updated weights for policy 1, policy_version 46432 (0.0007) [2023-10-08 09:39:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 95322112. Throughput: 0: 1825.8, 1: 1831.3. Samples: 23832592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:39:12,015][52710] Avg episode reward: [(0, '30.090'), (1, '28.080')] [2023-10-08 09:39:13,601][53852] Updated weights for policy 0, policy_version 46660 (0.0007) [2023-10-08 09:39:13,960][53852] Updated weights for policy 0, policy_version 46670 (0.0008) [2023-10-08 09:39:14,346][53852] Updated weights for policy 0, policy_version 46680 (0.0008) [2023-10-08 09:39:14,424][53885] Updated weights for policy 1, policy_version 46442 (0.0009) [2023-10-08 09:39:14,803][53885] Updated weights for policy 1, policy_version 46452 (0.0007) [2023-10-08 09:39:15,170][53885] Updated weights for policy 1, policy_version 46462 (0.0008) [2023-10-08 09:39:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 95387648. Throughput: 0: 1824.8, 1: 1832.9. Samples: 23853308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:39:17,015][52710] Avg episode reward: [(0, '29.820'), (1, '32.950')] [2023-10-08 09:39:17,990][53852] Updated weights for policy 0, policy_version 46690 (0.0008) [2023-10-08 09:39:18,361][53852] Updated weights for policy 0, policy_version 46700 (0.0009) [2023-10-08 09:39:18,737][53852] Updated weights for policy 0, policy_version 46710 (0.0010) [2023-10-08 09:39:19,008][53885] Updated weights for policy 1, policy_version 46472 (0.0009) [2023-10-08 09:39:19,113][53852] Updated weights for policy 0, policy_version 46720 (0.0008) [2023-10-08 09:39:19,382][53885] Updated weights for policy 1, policy_version 46482 (0.0009) [2023-10-08 09:39:19,756][53885] Updated weights for policy 1, policy_version 46492 (0.0008) [2023-10-08 09:39:22,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 95453184. Throughput: 0: 1817.3, 1: 1840.4. Samples: 23876432. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:39:22,016][52710] Avg episode reward: [(0, '28.640'), (1, '31.900')] [2023-10-08 09:39:22,851][53852] Updated weights for policy 0, policy_version 46730 (0.0009) [2023-10-08 09:39:23,212][53852] Updated weights for policy 0, policy_version 46740 (0.0008) [2023-10-08 09:39:23,302][53885] Updated weights for policy 1, policy_version 46502 (0.0007) [2023-10-08 09:39:23,588][53852] Updated weights for policy 0, policy_version 46750 (0.0008) [2023-10-08 09:39:23,678][53885] Updated weights for policy 1, policy_version 46512 (0.0007) [2023-10-08 09:39:24,039][53885] Updated weights for policy 1, policy_version 46522 (0.0008) [2023-10-08 09:39:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 95518720. Throughput: 0: 1820.6, 1: 1838.0. Samples: 23886566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:39:27,016][52710] Avg episode reward: [(0, '30.880'), (1, '30.750')] [2023-10-08 09:39:27,178][53852] Updated weights for policy 0, policy_version 46760 (0.0007) [2023-10-08 09:39:27,548][53852] Updated weights for policy 0, policy_version 46770 (0.0010) [2023-10-08 09:39:27,731][53885] Updated weights for policy 1, policy_version 46532 (0.0007) [2023-10-08 09:39:27,921][53852] Updated weights for policy 0, policy_version 46780 (0.0007) [2023-10-08 09:39:28,101][53885] Updated weights for policy 1, policy_version 46542 (0.0008) [2023-10-08 09:39:28,466][53885] Updated weights for policy 1, policy_version 46552 (0.0009) [2023-10-08 09:39:31,549][53852] Updated weights for policy 0, policy_version 46790 (0.0007) [2023-10-08 09:39:31,924][53852] Updated weights for policy 0, policy_version 46800 (0.0008) [2023-10-08 09:39:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95584256. Throughput: 0: 1822.7, 1: 1838.4. Samples: 23909496. Policy #0 lag: (min: 7.0, avg: 11.3, max: 39.0) [2023-10-08 09:39:32,016][52710] Avg episode reward: [(0, '30.280'), (1, '32.490')] [2023-10-08 09:39:32,153][53885] Updated weights for policy 1, policy_version 46562 (0.0007) [2023-10-08 09:39:32,289][53852] Updated weights for policy 0, policy_version 46810 (0.0008) [2023-10-08 09:39:32,522][53885] Updated weights for policy 1, policy_version 46572 (0.0007) [2023-10-08 09:39:32,889][53885] Updated weights for policy 1, policy_version 46582 (0.0010) [2023-10-08 09:39:33,266][53885] Updated weights for policy 1, policy_version 46592 (0.0009) [2023-10-08 09:39:36,112][53852] Updated weights for policy 0, policy_version 46820 (0.0009) [2023-10-08 09:39:36,486][53852] Updated weights for policy 0, policy_version 46830 (0.0009) [2023-10-08 09:39:36,851][53852] Updated weights for policy 0, policy_version 46840 (0.0010) [2023-10-08 09:39:36,985][53885] Updated weights for policy 1, policy_version 46602 (0.0007) [2023-10-08 09:39:37,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 95649792. Throughput: 0: 1821.9, 1: 1835.6. Samples: 23931458. Policy #0 lag: (min: 7.0, avg: 11.3, max: 39.0) [2023-10-08 09:39:37,017][52710] Avg episode reward: [(0, '29.670'), (1, '26.290')] [2023-10-08 09:39:37,349][53885] Updated weights for policy 1, policy_version 46612 (0.0008) [2023-10-08 09:39:37,724][53885] Updated weights for policy 1, policy_version 46622 (0.0008) [2023-10-08 09:39:40,586][53852] Updated weights for policy 0, policy_version 46850 (0.0007) [2023-10-08 09:39:40,958][53852] Updated weights for policy 0, policy_version 46860 (0.0008) [2023-10-08 09:39:41,286][53885] Updated weights for policy 1, policy_version 46632 (0.0009) [2023-10-08 09:39:41,329][53852] Updated weights for policy 0, policy_version 46870 (0.0007) [2023-10-08 09:39:41,649][53885] Updated weights for policy 1, policy_version 46642 (0.0008) [2023-10-08 09:39:41,695][53852] Updated weights for policy 0, policy_version 46880 (0.0008) [2023-10-08 09:39:42,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 95748096. Throughput: 0: 1827.9, 1: 1837.0. Samples: 23942380. Policy #0 lag: (min: 7.0, avg: 11.3, max: 39.0) [2023-10-08 09:39:42,015][52710] Avg episode reward: [(0, '30.320'), (1, '31.860')] [2023-10-08 09:39:42,019][53885] Updated weights for policy 1, policy_version 46652 (0.0007) [2023-10-08 09:39:45,333][53852] Updated weights for policy 0, policy_version 46890 (0.0009) [2023-10-08 09:39:45,707][53852] Updated weights for policy 0, policy_version 46900 (0.0007) [2023-10-08 09:39:45,879][53885] Updated weights for policy 1, policy_version 46662 (0.0008) [2023-10-08 09:39:46,076][53852] Updated weights for policy 0, policy_version 46910 (0.0009) [2023-10-08 09:39:46,239][53885] Updated weights for policy 1, policy_version 46672 (0.0007) [2023-10-08 09:39:46,609][53885] Updated weights for policy 1, policy_version 46682 (0.0008) [2023-10-08 09:39:47,015][52710] Fps is (10 sec: 19661.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 95846400. Throughput: 0: 1828.5, 1: 1828.2. Samples: 23964518. Policy #0 lag: (min: 7.0, avg: 11.3, max: 39.0) [2023-10-08 09:39:47,016][52710] Avg episode reward: [(0, '34.410'), (1, '30.250')] [2023-10-08 09:39:49,685][53852] Updated weights for policy 0, policy_version 46920 (0.0009) [2023-10-08 09:39:50,058][53852] Updated weights for policy 0, policy_version 46930 (0.0007) [2023-10-08 09:39:50,401][53885] Updated weights for policy 1, policy_version 46692 (0.0008) [2023-10-08 09:39:50,424][53852] Updated weights for policy 0, policy_version 46940 (0.0010) [2023-10-08 09:39:50,784][53885] Updated weights for policy 1, policy_version 46702 (0.0009) [2023-10-08 09:39:51,158][53885] Updated weights for policy 1, policy_version 46712 (0.0007) [2023-10-08 09:39:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 95911936. Throughput: 0: 1832.8, 1: 1829.1. Samples: 23985110. Policy #0 lag: (min: 7.0, avg: 11.3, max: 39.0) [2023-10-08 09:39:52,015][52710] Avg episode reward: [(0, '32.400'), (1, '27.540')] [2023-10-08 09:39:54,063][53852] Updated weights for policy 0, policy_version 46950 (0.0007) [2023-10-08 09:39:54,440][53852] Updated weights for policy 0, policy_version 46960 (0.0007) [2023-10-08 09:39:54,686][53885] Updated weights for policy 1, policy_version 46722 (0.0008) [2023-10-08 09:39:54,801][53852] Updated weights for policy 0, policy_version 46970 (0.0008) [2023-10-08 09:39:55,057][53885] Updated weights for policy 1, policy_version 46732 (0.0007) [2023-10-08 09:39:55,425][53885] Updated weights for policy 1, policy_version 46742 (0.0008) [2023-10-08 09:39:55,797][53885] Updated weights for policy 1, policy_version 46752 (0.0007) [2023-10-08 09:39:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95977472. Throughput: 0: 1828.4, 1: 1829.4. Samples: 23997192. Policy #0 lag: (min: 7.0, avg: 11.3, max: 39.0) [2023-10-08 09:39:57,016][52710] Avg episode reward: [(0, '30.040'), (1, '32.180')] [2023-10-08 09:39:58,497][53852] Updated weights for policy 0, policy_version 46980 (0.0008) [2023-10-08 09:39:58,867][53852] Updated weights for policy 0, policy_version 46990 (0.0008) [2023-10-08 09:39:59,243][53852] Updated weights for policy 0, policy_version 47000 (0.0008) [2023-10-08 09:39:59,265][53885] Updated weights for policy 1, policy_version 46762 (0.0007) [2023-10-08 09:39:59,641][53885] Updated weights for policy 1, policy_version 46772 (0.0007) [2023-10-08 09:40:00,026][53885] Updated weights for policy 1, policy_version 46782 (0.0009) [2023-10-08 09:40:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 96043008. Throughput: 0: 1833.9, 1: 1826.1. Samples: 24018010. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 09:40:02,016][52710] Avg episode reward: [(0, '33.490'), (1, '32.810')] [2023-10-08 09:40:02,889][53852] Updated weights for policy 0, policy_version 47010 (0.0008) [2023-10-08 09:40:03,276][53852] Updated weights for policy 0, policy_version 47020 (0.0008) [2023-10-08 09:40:03,639][53852] Updated weights for policy 0, policy_version 47030 (0.0009) [2023-10-08 09:40:03,726][53885] Updated weights for policy 1, policy_version 46792 (0.0009) [2023-10-08 09:40:04,005][53852] Updated weights for policy 0, policy_version 47040 (0.0008) [2023-10-08 09:40:04,085][53885] Updated weights for policy 1, policy_version 46802 (0.0007) [2023-10-08 09:40:04,456][53885] Updated weights for policy 1, policy_version 46812 (0.0008) [2023-10-08 09:40:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 96108544. Throughput: 0: 1828.0, 1: 1826.5. Samples: 24040886. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 09:40:07,016][52710] Avg episode reward: [(0, '29.920'), (1, '29.810')] [2023-10-08 09:40:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000047040_48168960.pth... [2023-10-08 09:40:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000046816_47939584.pth... [2023-10-08 09:40:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000045120_46202880.pth [2023-10-08 09:40:07,065][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000045344_46432256.pth [2023-10-08 09:40:07,664][53852] Updated weights for policy 0, policy_version 47050 (0.0009) [2023-10-08 09:40:08,033][53852] Updated weights for policy 0, policy_version 47060 (0.0008) [2023-10-08 09:40:08,130][53885] Updated weights for policy 1, policy_version 46822 (0.0008) [2023-10-08 09:40:08,396][53852] Updated weights for policy 0, policy_version 47070 (0.0008) [2023-10-08 09:40:08,486][53885] Updated weights for policy 1, policy_version 46832 (0.0009) [2023-10-08 09:40:08,862][53885] Updated weights for policy 1, policy_version 46842 (0.0008) [2023-10-08 09:40:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 96174080. Throughput: 0: 1826.2, 1: 1825.1. Samples: 24050876. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 09:40:12,016][52710] Avg episode reward: [(0, '30.160'), (1, '32.940')] [2023-10-08 09:40:12,198][53852] Updated weights for policy 0, policy_version 47080 (0.0009) [2023-10-08 09:40:12,572][53852] Updated weights for policy 0, policy_version 47090 (0.0008) [2023-10-08 09:40:12,612][53885] Updated weights for policy 1, policy_version 46852 (0.0008) [2023-10-08 09:40:12,943][53852] Updated weights for policy 0, policy_version 47100 (0.0010) [2023-10-08 09:40:12,979][53885] Updated weights for policy 1, policy_version 46862 (0.0008) [2023-10-08 09:40:13,355][53885] Updated weights for policy 1, policy_version 46872 (0.0008) [2023-10-08 09:40:16,820][53852] Updated weights for policy 0, policy_version 47110 (0.0010) [2023-10-08 09:40:16,995][53885] Updated weights for policy 1, policy_version 46882 (0.0011) [2023-10-08 09:40:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 96239616. Throughput: 0: 1816.5, 1: 1826.7. Samples: 24073442. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 09:40:17,016][52710] Avg episode reward: [(0, '29.810'), (1, '32.830')] [2023-10-08 09:40:17,186][53852] Updated weights for policy 0, policy_version 47120 (0.0008) [2023-10-08 09:40:17,360][53885] Updated weights for policy 1, policy_version 46892 (0.0008) [2023-10-08 09:40:17,544][53852] Updated weights for policy 0, policy_version 47130 (0.0007) [2023-10-08 09:40:17,723][53885] Updated weights for policy 1, policy_version 46902 (0.0007) [2023-10-08 09:40:18,094][53885] Updated weights for policy 1, policy_version 46912 (0.0008) [2023-10-08 09:40:21,197][53852] Updated weights for policy 0, policy_version 47140 (0.0007) [2023-10-08 09:40:21,560][53852] Updated weights for policy 0, policy_version 47150 (0.0007) [2023-10-08 09:40:21,794][53885] Updated weights for policy 1, policy_version 46922 (0.0007) [2023-10-08 09:40:21,927][53852] Updated weights for policy 0, policy_version 47160 (0.0007) [2023-10-08 09:40:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 96305152. Throughput: 0: 1820.9, 1: 1826.4. Samples: 24095588. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 09:40:22,016][52710] Avg episode reward: [(0, '30.410'), (1, '31.110')] [2023-10-08 09:40:22,154][53885] Updated weights for policy 1, policy_version 46932 (0.0008) [2023-10-08 09:40:22,524][53885] Updated weights for policy 1, policy_version 46942 (0.0009) [2023-10-08 09:40:25,584][53852] Updated weights for policy 0, policy_version 47170 (0.0007) [2023-10-08 09:40:25,955][53852] Updated weights for policy 0, policy_version 47180 (0.0007) [2023-10-08 09:40:26,071][53885] Updated weights for policy 1, policy_version 46952 (0.0008) [2023-10-08 09:40:26,318][53852] Updated weights for policy 0, policy_version 47190 (0.0007) [2023-10-08 09:40:26,436][53885] Updated weights for policy 1, policy_version 46962 (0.0007) [2023-10-08 09:40:26,687][53852] Updated weights for policy 0, policy_version 47200 (0.0007) [2023-10-08 09:40:26,805][53885] Updated weights for policy 1, policy_version 46972 (0.0008) [2023-10-08 09:40:27,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 96436224. Throughput: 0: 1819.1, 1: 1832.3. Samples: 24106692. Policy #0 lag: (min: 31.0, avg: 31.3, max: 44.0) [2023-10-08 09:40:27,015][52710] Avg episode reward: [(0, '30.180'), (1, '29.240')] [2023-10-08 09:40:30,253][53852] Updated weights for policy 0, policy_version 47210 (0.0007) [2023-10-08 09:40:30,443][53885] Updated weights for policy 1, policy_version 46982 (0.0007) [2023-10-08 09:40:30,623][53852] Updated weights for policy 0, policy_version 47220 (0.0007) [2023-10-08 09:40:30,802][53885] Updated weights for policy 1, policy_version 46992 (0.0008) [2023-10-08 09:40:30,990][53852] Updated weights for policy 0, policy_version 47230 (0.0007) [2023-10-08 09:40:31,170][53885] Updated weights for policy 1, policy_version 47002 (0.0009) [2023-10-08 09:40:32,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 96501760. Throughput: 0: 1819.2, 1: 1824.7. Samples: 24128494. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) [2023-10-08 09:40:32,016][52710] Avg episode reward: [(0, '28.740'), (1, '33.510')] [2023-10-08 09:40:34,582][53852] Updated weights for policy 0, policy_version 47240 (0.0008) [2023-10-08 09:40:34,814][53885] Updated weights for policy 1, policy_version 47012 (0.0009) [2023-10-08 09:40:34,960][53852] Updated weights for policy 0, policy_version 47250 (0.0008) [2023-10-08 09:40:35,184][53885] Updated weights for policy 1, policy_version 47022 (0.0009) [2023-10-08 09:40:35,328][53852] Updated weights for policy 0, policy_version 47260 (0.0008) [2023-10-08 09:40:35,540][53885] Updated weights for policy 1, policy_version 47032 (0.0008) [2023-10-08 09:40:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 96567296. Throughput: 0: 1818.0, 1: 1836.3. Samples: 24149554. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) [2023-10-08 09:40:37,015][52710] Avg episode reward: [(0, '29.490'), (1, '28.650')] [2023-10-08 09:40:39,119][53852] Updated weights for policy 0, policy_version 47270 (0.0008) [2023-10-08 09:40:39,316][53885] Updated weights for policy 1, policy_version 47042 (0.0010) [2023-10-08 09:40:39,485][53852] Updated weights for policy 0, policy_version 47280 (0.0008) [2023-10-08 09:40:39,727][53885] Updated weights for policy 1, policy_version 47052 (0.0007) [2023-10-08 09:40:39,855][53852] Updated weights for policy 0, policy_version 47290 (0.0009) [2023-10-08 09:40:40,091][53885] Updated weights for policy 1, policy_version 47062 (0.0007) [2023-10-08 09:40:40,454][53885] Updated weights for policy 1, policy_version 47072 (0.0010) [2023-10-08 09:40:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96632832. Throughput: 0: 1824.2, 1: 1825.2. Samples: 24161414. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) [2023-10-08 09:40:42,016][52710] Avg episode reward: [(0, '30.630'), (1, '29.660')] [2023-10-08 09:40:43,391][53852] Updated weights for policy 0, policy_version 47300 (0.0008) [2023-10-08 09:40:43,756][53852] Updated weights for policy 0, policy_version 47310 (0.0007) [2023-10-08 09:40:44,074][53885] Updated weights for policy 1, policy_version 47082 (0.0007) [2023-10-08 09:40:44,131][53852] Updated weights for policy 0, policy_version 47320 (0.0008) [2023-10-08 09:40:44,446][53885] Updated weights for policy 1, policy_version 47092 (0.0008) [2023-10-08 09:40:44,816][53885] Updated weights for policy 1, policy_version 47102 (0.0009) [2023-10-08 09:40:47,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 96698368. Throughput: 0: 1820.9, 1: 1829.5. Samples: 24182276. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) [2023-10-08 09:40:47,017][52710] Avg episode reward: [(0, '29.230'), (1, '28.950')] [2023-10-08 09:40:47,772][53852] Updated weights for policy 0, policy_version 47330 (0.0008) [2023-10-08 09:40:48,135][53852] Updated weights for policy 0, policy_version 47340 (0.0009) [2023-10-08 09:40:48,455][53885] Updated weights for policy 1, policy_version 47112 (0.0007) [2023-10-08 09:40:48,500][53852] Updated weights for policy 0, policy_version 47350 (0.0009) [2023-10-08 09:40:48,818][53885] Updated weights for policy 1, policy_version 47122 (0.0008) [2023-10-08 09:40:48,861][53852] Updated weights for policy 0, policy_version 47360 (0.0007) [2023-10-08 09:40:49,188][53885] Updated weights for policy 1, policy_version 47132 (0.0008) [2023-10-08 09:40:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 96763904. Throughput: 0: 1830.7, 1: 1826.9. Samples: 24205476. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) [2023-10-08 09:40:52,016][52710] Avg episode reward: [(0, '30.890'), (1, '28.230')] [2023-10-08 09:40:52,553][53852] Updated weights for policy 0, policy_version 47370 (0.0010) [2023-10-08 09:40:52,926][53852] Updated weights for policy 0, policy_version 47380 (0.0008) [2023-10-08 09:40:52,940][53885] Updated weights for policy 1, policy_version 47142 (0.0008) [2023-10-08 09:40:53,307][53885] Updated weights for policy 1, policy_version 47152 (0.0007) [2023-10-08 09:40:53,309][53852] Updated weights for policy 0, policy_version 47390 (0.0008) [2023-10-08 09:40:53,666][53885] Updated weights for policy 1, policy_version 47162 (0.0007) [2023-10-08 09:40:57,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96829440. Throughput: 0: 1827.6, 1: 1827.3. Samples: 24215346. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) [2023-10-08 09:40:57,015][52710] Avg episode reward: [(0, '20.960'), (1, '27.030')] [2023-10-08 09:40:57,054][53852] Updated weights for policy 0, policy_version 47400 (0.0007) [2023-10-08 09:40:57,420][53885] Updated weights for policy 1, policy_version 47172 (0.0010) [2023-10-08 09:40:57,434][53852] Updated weights for policy 0, policy_version 47410 (0.0009) [2023-10-08 09:40:57,795][53885] Updated weights for policy 1, policy_version 47182 (0.0010) [2023-10-08 09:40:57,797][53852] Updated weights for policy 0, policy_version 47420 (0.0007) [2023-10-08 09:40:58,162][53885] Updated weights for policy 1, policy_version 47192 (0.0010) [2023-10-08 09:41:01,476][53852] Updated weights for policy 0, policy_version 47430 (0.0007) [2023-10-08 09:41:01,843][53852] Updated weights for policy 0, policy_version 47440 (0.0007) [2023-10-08 09:41:01,876][53885] Updated weights for policy 1, policy_version 47202 (0.0008) [2023-10-08 09:41:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96894976. Throughput: 0: 1832.6, 1: 1829.3. Samples: 24238228. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) [2023-10-08 09:41:02,016][52710] Avg episode reward: [(0, '12.850'), (1, '30.520')] [2023-10-08 09:41:02,210][53852] Updated weights for policy 0, policy_version 47450 (0.0007) [2023-10-08 09:41:02,237][53885] Updated weights for policy 1, policy_version 47212 (0.0007) [2023-10-08 09:41:02,609][53885] Updated weights for policy 1, policy_version 47222 (0.0007) [2023-10-08 09:41:02,981][53885] Updated weights for policy 1, policy_version 47232 (0.0009) [2023-10-08 09:41:05,941][53852] Updated weights for policy 0, policy_version 47460 (0.0008) [2023-10-08 09:41:06,327][53852] Updated weights for policy 0, policy_version 47470 (0.0008) [2023-10-08 09:41:06,608][53885] Updated weights for policy 1, policy_version 47242 (0.0007) [2023-10-08 09:41:06,694][53852] Updated weights for policy 0, policy_version 47480 (0.0007) [2023-10-08 09:41:06,980][53885] Updated weights for policy 1, policy_version 47252 (0.0007) [2023-10-08 09:41:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 96993280. Throughput: 0: 1824.8, 1: 1821.4. Samples: 24259664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:41:07,015][52710] Avg episode reward: [(0, '7.430'), (1, '30.840')] [2023-10-08 09:41:07,355][53885] Updated weights for policy 1, policy_version 47262 (0.0008) [2023-10-08 09:41:10,308][53852] Updated weights for policy 0, policy_version 47490 (0.0008) [2023-10-08 09:41:10,673][53852] Updated weights for policy 0, policy_version 47500 (0.0007) [2023-10-08 09:41:11,044][53852] Updated weights for policy 0, policy_version 47510 (0.0009) [2023-10-08 09:41:11,103][53885] Updated weights for policy 1, policy_version 47272 (0.0008) [2023-10-08 09:41:11,403][53852] Updated weights for policy 0, policy_version 47520 (0.0007) [2023-10-08 09:41:11,464][53885] Updated weights for policy 1, policy_version 47282 (0.0007) [2023-10-08 09:41:11,831][53885] Updated weights for policy 1, policy_version 47292 (0.0009) [2023-10-08 09:41:12,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 97091584. Throughput: 0: 1830.0, 1: 1819.8. Samples: 24270932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:41:12,016][52710] Avg episode reward: [(0, '8.200'), (1, '31.260')] [2023-10-08 09:41:15,130][53852] Updated weights for policy 0, policy_version 47530 (0.0009) [2023-10-08 09:41:15,495][53852] Updated weights for policy 0, policy_version 47540 (0.0009) [2023-10-08 09:41:15,556][53885] Updated weights for policy 1, policy_version 47302 (0.0007) [2023-10-08 09:41:15,865][53852] Updated weights for policy 0, policy_version 47550 (0.0010) [2023-10-08 09:41:15,907][53885] Updated weights for policy 1, policy_version 47312 (0.0009) [2023-10-08 09:41:16,280][53885] Updated weights for policy 1, policy_version 47322 (0.0009) [2023-10-08 09:41:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 97157120. Throughput: 0: 1824.9, 1: 1822.7. Samples: 24292636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:41:17,016][52710] Avg episode reward: [(0, '8.400'), (1, '31.350')] [2023-10-08 09:41:19,644][53852] Updated weights for policy 0, policy_version 47560 (0.0009) [2023-10-08 09:41:19,935][53885] Updated weights for policy 1, policy_version 47332 (0.0007) [2023-10-08 09:41:20,027][53852] Updated weights for policy 0, policy_version 47570 (0.0008) [2023-10-08 09:41:20,302][53885] Updated weights for policy 1, policy_version 47342 (0.0008) [2023-10-08 09:41:20,385][53852] Updated weights for policy 0, policy_version 47580 (0.0009) [2023-10-08 09:41:20,675][53885] Updated weights for policy 1, policy_version 47352 (0.0010) [2023-10-08 09:41:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 97222656. Throughput: 0: 1824.7, 1: 1821.2. Samples: 24313618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:41:22,016][52710] Avg episode reward: [(0, '8.940'), (1, '31.560')] [2023-10-08 09:41:24,052][53852] Updated weights for policy 0, policy_version 47590 (0.0008) [2023-10-08 09:41:24,412][53852] Updated weights for policy 0, policy_version 47600 (0.0008) [2023-10-08 09:41:24,552][53885] Updated weights for policy 1, policy_version 47362 (0.0010) [2023-10-08 09:41:24,777][53852] Updated weights for policy 0, policy_version 47610 (0.0009) [2023-10-08 09:41:24,953][53885] Updated weights for policy 1, policy_version 47372 (0.0009) [2023-10-08 09:41:25,321][53885] Updated weights for policy 1, policy_version 47382 (0.0007) [2023-10-08 09:41:25,682][53885] Updated weights for policy 1, policy_version 47392 (0.0008) [2023-10-08 09:41:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 97288192. Throughput: 0: 1821.6, 1: 1821.7. Samples: 24325360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:41:27,015][52710] Avg episode reward: [(0, '9.280'), (1, '29.900')] [2023-10-08 09:41:28,536][53852] Updated weights for policy 0, policy_version 47620 (0.0008) [2023-10-08 09:41:28,906][53852] Updated weights for policy 0, policy_version 47630 (0.0009) [2023-10-08 09:41:29,272][53885] Updated weights for policy 1, policy_version 47402 (0.0007) [2023-10-08 09:41:29,280][53852] Updated weights for policy 0, policy_version 47640 (0.0009) [2023-10-08 09:41:29,639][53885] Updated weights for policy 1, policy_version 47412 (0.0007) [2023-10-08 09:41:30,021][53885] Updated weights for policy 1, policy_version 47422 (0.0011) [2023-10-08 09:41:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 97353728. Throughput: 0: 1822.4, 1: 1818.5. Samples: 24346112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:41:32,016][52710] Avg episode reward: [(0, '9.400'), (1, '30.650')] [2023-10-08 09:41:33,050][53852] Updated weights for policy 0, policy_version 47650 (0.0009) [2023-10-08 09:41:33,426][53852] Updated weights for policy 0, policy_version 47660 (0.0008) [2023-10-08 09:41:33,745][53885] Updated weights for policy 1, policy_version 47432 (0.0008) [2023-10-08 09:41:33,798][53852] Updated weights for policy 0, policy_version 47670 (0.0008) [2023-10-08 09:41:34,106][53885] Updated weights for policy 1, policy_version 47442 (0.0011) [2023-10-08 09:41:34,169][53852] Updated weights for policy 0, policy_version 47680 (0.0007) [2023-10-08 09:41:34,481][53885] Updated weights for policy 1, policy_version 47452 (0.0009) [2023-10-08 09:41:37,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 97419264. Throughput: 0: 1815.6, 1: 1817.1. Samples: 24368948. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 09:41:37,016][52710] Avg episode reward: [(0, '10.680'), (1, '33.060')] [2023-10-08 09:41:37,763][53852] Updated weights for policy 0, policy_version 47690 (0.0007) [2023-10-08 09:41:38,124][53852] Updated weights for policy 0, policy_version 47700 (0.0008) [2023-10-08 09:41:38,132][53885] Updated weights for policy 1, policy_version 47462 (0.0007) [2023-10-08 09:41:38,487][53852] Updated weights for policy 0, policy_version 47710 (0.0007) [2023-10-08 09:41:38,498][53885] Updated weights for policy 1, policy_version 47472 (0.0008) [2023-10-08 09:41:38,864][53885] Updated weights for policy 1, policy_version 47482 (0.0009) [2023-10-08 09:41:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97484800. Throughput: 0: 1818.9, 1: 1814.8. Samples: 24378864. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 09:41:42,016][52710] Avg episode reward: [(0, '9.570'), (1, '28.300')] [2023-10-08 09:41:42,250][53852] Updated weights for policy 0, policy_version 47720 (0.0010) [2023-10-08 09:41:42,559][53885] Updated weights for policy 1, policy_version 47492 (0.0010) [2023-10-08 09:41:42,620][53852] Updated weights for policy 0, policy_version 47730 (0.0007) [2023-10-08 09:41:42,924][53885] Updated weights for policy 1, policy_version 47502 (0.0008) [2023-10-08 09:41:42,993][53852] Updated weights for policy 0, policy_version 47740 (0.0008) [2023-10-08 09:41:43,295][53885] Updated weights for policy 1, policy_version 47512 (0.0007) [2023-10-08 09:41:46,670][53852] Updated weights for policy 0, policy_version 47750 (0.0007) [2023-10-08 09:41:46,990][53885] Updated weights for policy 1, policy_version 47522 (0.0007) [2023-10-08 09:41:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97550336. Throughput: 0: 1815.8, 1: 1814.8. Samples: 24401606. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 09:41:47,016][52710] Avg episode reward: [(0, '10.060'), (1, '28.730')] [2023-10-08 09:41:47,038][53852] Updated weights for policy 0, policy_version 47760 (0.0010) [2023-10-08 09:41:47,359][53885] Updated weights for policy 1, policy_version 47532 (0.0008) [2023-10-08 09:41:47,413][53852] Updated weights for policy 0, policy_version 47770 (0.0008) [2023-10-08 09:41:47,724][53885] Updated weights for policy 1, policy_version 47542 (0.0008) [2023-10-08 09:41:48,085][53885] Updated weights for policy 1, policy_version 47552 (0.0008) [2023-10-08 09:41:51,150][53852] Updated weights for policy 0, policy_version 47780 (0.0009) [2023-10-08 09:41:51,541][53852] Updated weights for policy 0, policy_version 47790 (0.0008) [2023-10-08 09:41:51,861][53885] Updated weights for policy 1, policy_version 47562 (0.0009) [2023-10-08 09:41:51,909][53852] Updated weights for policy 0, policy_version 47800 (0.0007) [2023-10-08 09:41:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 97615872. Throughput: 0: 1822.1, 1: 1818.8. Samples: 24423504. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 09:41:52,016][52710] Avg episode reward: [(0, '10.090'), (1, '28.960')] [2023-10-08 09:41:52,230][53885] Updated weights for policy 1, policy_version 47572 (0.0008) [2023-10-08 09:41:52,605][53885] Updated weights for policy 1, policy_version 47582 (0.0009) [2023-10-08 09:41:55,459][53852] Updated weights for policy 0, policy_version 47810 (0.0009) [2023-10-08 09:41:55,825][53852] Updated weights for policy 0, policy_version 47820 (0.0008) [2023-10-08 09:41:56,196][53852] Updated weights for policy 0, policy_version 47830 (0.0007) [2023-10-08 09:41:56,285][53885] Updated weights for policy 1, policy_version 47592 (0.0008) [2023-10-08 09:41:56,555][53852] Updated weights for policy 0, policy_version 47840 (0.0007) [2023-10-08 09:41:56,655][53885] Updated weights for policy 1, policy_version 47602 (0.0008) [2023-10-08 09:41:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 97714176. Throughput: 0: 1822.7, 1: 1811.9. Samples: 24434492. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 09:41:57,016][52710] Avg episode reward: [(0, '12.170'), (1, '27.020')] [2023-10-08 09:41:57,030][53885] Updated weights for policy 1, policy_version 47612 (0.0009) [2023-10-08 09:42:00,359][53852] Updated weights for policy 0, policy_version 47850 (0.0011) [2023-10-08 09:42:00,731][53852] Updated weights for policy 0, policy_version 47860 (0.0007) [2023-10-08 09:42:00,733][53885] Updated weights for policy 1, policy_version 47622 (0.0009) [2023-10-08 09:42:01,097][53885] Updated weights for policy 1, policy_version 47632 (0.0008) [2023-10-08 09:42:01,098][53852] Updated weights for policy 0, policy_version 47870 (0.0007) [2023-10-08 09:42:01,468][53885] Updated weights for policy 1, policy_version 47642 (0.0008) [2023-10-08 09:42:02,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 97812480. Throughput: 0: 1820.0, 1: 1813.5. Samples: 24456142. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) [2023-10-08 09:42:02,016][52710] Avg episode reward: [(0, '11.750'), (1, '29.520')] [2023-10-08 09:42:04,645][53852] Updated weights for policy 0, policy_version 47880 (0.0007) [2023-10-08 09:42:05,006][53852] Updated weights for policy 0, policy_version 47890 (0.0007) [2023-10-08 09:42:05,222][53885] Updated weights for policy 1, policy_version 47652 (0.0009) [2023-10-08 09:42:05,380][53852] Updated weights for policy 0, policy_version 47900 (0.0007) [2023-10-08 09:42:05,590][53885] Updated weights for policy 1, policy_version 47662 (0.0008) [2023-10-08 09:42:05,962][53885] Updated weights for policy 1, policy_version 47672 (0.0009) [2023-10-08 09:42:07,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97878016. Throughput: 0: 1828.1, 1: 1806.0. Samples: 24477154. Policy #0 lag: (min: 20.0, avg: 31.1, max: 52.0) [2023-10-08 09:42:07,016][52710] Avg episode reward: [(0, '10.980'), (1, '33.220')] [2023-10-08 09:42:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000047680_48824320.pth... [2023-10-08 09:42:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000047904_49053696.pth... [2023-10-08 09:42:07,065][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000045952_47054848.pth [2023-10-08 09:42:07,067][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000046208_47316992.pth [2023-10-08 09:42:09,010][53852] Updated weights for policy 0, policy_version 47910 (0.0007) [2023-10-08 09:42:09,378][53852] Updated weights for policy 0, policy_version 47920 (0.0007) [2023-10-08 09:42:09,690][53885] Updated weights for policy 1, policy_version 47682 (0.0007) [2023-10-08 09:42:09,749][53852] Updated weights for policy 0, policy_version 47930 (0.0008) [2023-10-08 09:42:10,100][53885] Updated weights for policy 1, policy_version 47692 (0.0008) [2023-10-08 09:42:10,463][53885] Updated weights for policy 1, policy_version 47702 (0.0010) [2023-10-08 09:42:10,827][53885] Updated weights for policy 1, policy_version 47712 (0.0010) [2023-10-08 09:42:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 97943552. Throughput: 0: 1825.2, 1: 1815.4. Samples: 24489184. Policy #0 lag: (min: 20.0, avg: 31.1, max: 52.0) [2023-10-08 09:42:12,016][52710] Avg episode reward: [(0, '10.540'), (1, '29.420')] [2023-10-08 09:42:13,393][53852] Updated weights for policy 0, policy_version 47940 (0.0008) [2023-10-08 09:42:13,773][53852] Updated weights for policy 0, policy_version 47950 (0.0009) [2023-10-08 09:42:14,141][53852] Updated weights for policy 0, policy_version 47960 (0.0008) [2023-10-08 09:42:14,341][53885] Updated weights for policy 1, policy_version 47722 (0.0007) [2023-10-08 09:42:14,721][53885] Updated weights for policy 1, policy_version 47732 (0.0010) [2023-10-08 09:42:15,082][53885] Updated weights for policy 1, policy_version 47742 (0.0010) [2023-10-08 09:42:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 98009088. Throughput: 0: 1830.3, 1: 1812.5. Samples: 24510038. Policy #0 lag: (min: 20.0, avg: 31.1, max: 52.0) [2023-10-08 09:42:17,016][52710] Avg episode reward: [(0, '11.230'), (1, '33.330')] [2023-10-08 09:42:17,769][53852] Updated weights for policy 0, policy_version 47970 (0.0007) [2023-10-08 09:42:18,134][53852] Updated weights for policy 0, policy_version 47980 (0.0007) [2023-10-08 09:42:18,506][53852] Updated weights for policy 0, policy_version 47990 (0.0008) [2023-10-08 09:42:18,819][53885] Updated weights for policy 1, policy_version 47752 (0.0008) [2023-10-08 09:42:18,873][53852] Updated weights for policy 0, policy_version 48000 (0.0009) [2023-10-08 09:42:19,191][53885] Updated weights for policy 1, policy_version 47762 (0.0007) [2023-10-08 09:42:19,560][53885] Updated weights for policy 1, policy_version 47772 (0.0008) [2023-10-08 09:42:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 98074624. Throughput: 0: 1831.0, 1: 1811.3. Samples: 24532854. Policy #0 lag: (min: 20.0, avg: 31.1, max: 52.0) [2023-10-08 09:42:22,016][52710] Avg episode reward: [(0, '12.780'), (1, '29.690')] [2023-10-08 09:42:22,505][53852] Updated weights for policy 0, policy_version 48010 (0.0009) [2023-10-08 09:42:22,879][53852] Updated weights for policy 0, policy_version 48020 (0.0007) [2023-10-08 09:42:23,099][53885] Updated weights for policy 1, policy_version 47782 (0.0008) [2023-10-08 09:42:23,245][53852] Updated weights for policy 0, policy_version 48030 (0.0007) [2023-10-08 09:42:23,459][53885] Updated weights for policy 1, policy_version 47792 (0.0009) [2023-10-08 09:42:23,830][53885] Updated weights for policy 1, policy_version 47802 (0.0007) [2023-10-08 09:42:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 98140160. Throughput: 0: 1828.8, 1: 1813.0. Samples: 24542748. Policy #0 lag: (min: 20.0, avg: 31.1, max: 52.0) [2023-10-08 09:42:27,016][52710] Avg episode reward: [(0, '13.870'), (1, '26.020')] [2023-10-08 09:42:27,052][53852] Updated weights for policy 0, policy_version 48040 (0.0008) [2023-10-08 09:42:27,426][53852] Updated weights for policy 0, policy_version 48050 (0.0009) [2023-10-08 09:42:27,514][53885] Updated weights for policy 1, policy_version 47812 (0.0011) [2023-10-08 09:42:27,800][53852] Updated weights for policy 0, policy_version 48060 (0.0007) [2023-10-08 09:42:27,888][53885] Updated weights for policy 1, policy_version 47822 (0.0010) [2023-10-08 09:42:28,264][53885] Updated weights for policy 1, policy_version 47832 (0.0010) [2023-10-08 09:42:31,419][53852] Updated weights for policy 0, policy_version 48070 (0.0008) [2023-10-08 09:42:31,788][53852] Updated weights for policy 0, policy_version 48080 (0.0010) [2023-10-08 09:42:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98205696. Throughput: 0: 1829.2, 1: 1808.3. Samples: 24565292. Policy #0 lag: (min: 20.0, avg: 31.1, max: 52.0) [2023-10-08 09:42:32,016][52710] Avg episode reward: [(0, '13.260'), (1, '31.340')] [2023-10-08 09:42:32,101][53885] Updated weights for policy 1, policy_version 47842 (0.0010) [2023-10-08 09:42:32,158][53852] Updated weights for policy 0, policy_version 48090 (0.0007) [2023-10-08 09:42:32,474][53885] Updated weights for policy 1, policy_version 47852 (0.0008) [2023-10-08 09:42:32,848][53885] Updated weights for policy 1, policy_version 47862 (0.0007) [2023-10-08 09:42:33,216][53885] Updated weights for policy 1, policy_version 47872 (0.0007) [2023-10-08 09:42:35,928][53852] Updated weights for policy 0, policy_version 48100 (0.0007) [2023-10-08 09:42:36,297][53852] Updated weights for policy 0, policy_version 48110 (0.0008) [2023-10-08 09:42:36,669][53852] Updated weights for policy 0, policy_version 48120 (0.0007) [2023-10-08 09:42:36,794][53885] Updated weights for policy 1, policy_version 47882 (0.0009) [2023-10-08 09:42:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98304000. Throughput: 0: 1815.8, 1: 1816.8. Samples: 24586974. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:42:37,016][52710] Avg episode reward: [(0, '11.690'), (1, '30.040')] [2023-10-08 09:42:37,160][53885] Updated weights for policy 1, policy_version 47892 (0.0008) [2023-10-08 09:42:37,528][53885] Updated weights for policy 1, policy_version 47902 (0.0011) [2023-10-08 09:42:40,374][53852] Updated weights for policy 0, policy_version 48130 (0.0008) [2023-10-08 09:42:40,742][53852] Updated weights for policy 0, policy_version 48140 (0.0008) [2023-10-08 09:42:41,108][53852] Updated weights for policy 0, policy_version 48150 (0.0010) [2023-10-08 09:42:41,341][53885] Updated weights for policy 1, policy_version 47912 (0.0008) [2023-10-08 09:42:41,484][53852] Updated weights for policy 0, policy_version 48160 (0.0009) [2023-10-08 09:42:41,709][53885] Updated weights for policy 1, policy_version 47922 (0.0007) [2023-10-08 09:42:42,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98369536. Throughput: 0: 1813.1, 1: 1816.3. Samples: 24597814. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:42:42,016][52710] Avg episode reward: [(0, '11.630'), (1, '27.730')] [2023-10-08 09:42:42,079][53885] Updated weights for policy 1, policy_version 47932 (0.0007) [2023-10-08 09:42:45,350][53852] Updated weights for policy 0, policy_version 48170 (0.0010) [2023-10-08 09:42:45,651][53885] Updated weights for policy 1, policy_version 47942 (0.0008) [2023-10-08 09:42:45,723][53852] Updated weights for policy 0, policy_version 48180 (0.0009) [2023-10-08 09:42:46,029][53885] Updated weights for policy 1, policy_version 47952 (0.0008) [2023-10-08 09:42:46,095][53852] Updated weights for policy 0, policy_version 48190 (0.0007) [2023-10-08 09:42:46,391][53885] Updated weights for policy 1, policy_version 47962 (0.0008) [2023-10-08 09:42:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 98467840. Throughput: 0: 1812.9, 1: 1824.1. Samples: 24619806. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:42:47,016][52710] Avg episode reward: [(0, '12.460'), (1, '32.500')] [2023-10-08 09:42:49,840][53852] Updated weights for policy 0, policy_version 48200 (0.0007) [2023-10-08 09:42:50,214][53852] Updated weights for policy 0, policy_version 48210 (0.0008) [2023-10-08 09:42:50,234][53885] Updated weights for policy 1, policy_version 47972 (0.0010) [2023-10-08 09:42:50,583][53852] Updated weights for policy 0, policy_version 48220 (0.0007) [2023-10-08 09:42:50,601][53885] Updated weights for policy 1, policy_version 47982 (0.0008) [2023-10-08 09:42:50,967][53885] Updated weights for policy 1, policy_version 47992 (0.0008) [2023-10-08 09:42:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 98533376. Throughput: 0: 1805.7, 1: 1819.8. Samples: 24640302. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:42:52,015][52710] Avg episode reward: [(0, '13.320'), (1, '25.890')] [2023-10-08 09:42:54,396][53852] Updated weights for policy 0, policy_version 48230 (0.0008) [2023-10-08 09:42:54,700][53885] Updated weights for policy 1, policy_version 48002 (0.0008) [2023-10-08 09:42:54,757][53852] Updated weights for policy 0, policy_version 48240 (0.0008) [2023-10-08 09:42:55,111][53885] Updated weights for policy 1, policy_version 48012 (0.0009) [2023-10-08 09:42:55,126][53852] Updated weights for policy 0, policy_version 48250 (0.0010) [2023-10-08 09:42:55,482][53885] Updated weights for policy 1, policy_version 48022 (0.0008) [2023-10-08 09:42:55,843][53885] Updated weights for policy 1, policy_version 48032 (0.0009) [2023-10-08 09:42:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98598912. Throughput: 0: 1812.2, 1: 1823.4. Samples: 24652788. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:42:57,015][52710] Avg episode reward: [(0, '13.510'), (1, '30.300')] [2023-10-08 09:42:58,933][53852] Updated weights for policy 0, policy_version 48260 (0.0009) [2023-10-08 09:42:59,295][53852] Updated weights for policy 0, policy_version 48270 (0.0007) [2023-10-08 09:42:59,379][53885] Updated weights for policy 1, policy_version 48042 (0.0008) [2023-10-08 09:42:59,662][53852] Updated weights for policy 0, policy_version 48280 (0.0007) [2023-10-08 09:42:59,740][53885] Updated weights for policy 1, policy_version 48052 (0.0009) [2023-10-08 09:43:00,104][53885] Updated weights for policy 1, policy_version 48062 (0.0009) [2023-10-08 09:43:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 98664448. Throughput: 0: 1794.3, 1: 1822.6. Samples: 24672800. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:43:02,016][52710] Avg episode reward: [(0, '13.380'), (1, '31.460')] [2023-10-08 09:43:03,346][53852] Updated weights for policy 0, policy_version 48290 (0.0007) [2023-10-08 09:43:03,718][53852] Updated weights for policy 0, policy_version 48300 (0.0007) [2023-10-08 09:43:03,951][53885] Updated weights for policy 1, policy_version 48072 (0.0009) [2023-10-08 09:43:04,088][53852] Updated weights for policy 0, policy_version 48310 (0.0008) [2023-10-08 09:43:04,327][53885] Updated weights for policy 1, policy_version 48082 (0.0008) [2023-10-08 09:43:04,459][53852] Updated weights for policy 0, policy_version 48320 (0.0008) [2023-10-08 09:43:04,692][53885] Updated weights for policy 1, policy_version 48092 (0.0009) [2023-10-08 09:43:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 98729984. Throughput: 0: 1793.5, 1: 1828.8. Samples: 24695860. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 09:43:07,016][52710] Avg episode reward: [(0, '13.250'), (1, '28.470')] [2023-10-08 09:43:08,122][53852] Updated weights for policy 0, policy_version 48330 (0.0008) [2023-10-08 09:43:08,282][53885] Updated weights for policy 1, policy_version 48102 (0.0008) [2023-10-08 09:43:08,481][53852] Updated weights for policy 0, policy_version 48340 (0.0007) [2023-10-08 09:43:08,652][53885] Updated weights for policy 1, policy_version 48112 (0.0007) [2023-10-08 09:43:08,850][53852] Updated weights for policy 0, policy_version 48350 (0.0008) [2023-10-08 09:43:09,024][53885] Updated weights for policy 1, policy_version 48122 (0.0008) [2023-10-08 09:43:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 98795520. Throughput: 0: 1795.0, 1: 1828.0. Samples: 24705782. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 09:43:12,016][52710] Avg episode reward: [(0, '12.370'), (1, '33.090')] [2023-10-08 09:43:12,542][53885] Updated weights for policy 1, policy_version 48132 (0.0008) [2023-10-08 09:43:12,576][53852] Updated weights for policy 0, policy_version 48360 (0.0008) [2023-10-08 09:43:12,914][53885] Updated weights for policy 1, policy_version 48142 (0.0008) [2023-10-08 09:43:12,957][53852] Updated weights for policy 0, policy_version 48370 (0.0008) [2023-10-08 09:43:13,275][53885] Updated weights for policy 1, policy_version 48152 (0.0007) [2023-10-08 09:43:13,318][53852] Updated weights for policy 0, policy_version 48380 (0.0007) [2023-10-08 09:43:16,988][53885] Updated weights for policy 1, policy_version 48162 (0.0008) [2023-10-08 09:43:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 98861056. Throughput: 0: 1793.6, 1: 1831.7. Samples: 24728434. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 09:43:17,016][52710] Avg episode reward: [(0, '11.990'), (1, '28.540')] [2023-10-08 09:43:17,047][53852] Updated weights for policy 0, policy_version 48390 (0.0009) [2023-10-08 09:43:17,363][53885] Updated weights for policy 1, policy_version 48172 (0.0007) [2023-10-08 09:43:17,416][53852] Updated weights for policy 0, policy_version 48400 (0.0009) [2023-10-08 09:43:17,728][53885] Updated weights for policy 1, policy_version 48182 (0.0007) [2023-10-08 09:43:17,788][53852] Updated weights for policy 0, policy_version 48410 (0.0008) [2023-10-08 09:43:18,088][53885] Updated weights for policy 1, policy_version 48192 (0.0009) [2023-10-08 09:43:21,564][53852] Updated weights for policy 0, policy_version 48420 (0.0010) [2023-10-08 09:43:21,845][53885] Updated weights for policy 1, policy_version 48202 (0.0007) [2023-10-08 09:43:21,958][53852] Updated weights for policy 0, policy_version 48430 (0.0008) [2023-10-08 09:43:22,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 98926592. Throughput: 0: 1815.5, 1: 1821.7. Samples: 24750646. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 09:43:22,015][52710] Avg episode reward: [(0, '13.450'), (1, '28.540')] [2023-10-08 09:43:22,217][53885] Updated weights for policy 1, policy_version 48212 (0.0009) [2023-10-08 09:43:22,312][53852] Updated weights for policy 0, policy_version 48440 (0.0009) [2023-10-08 09:43:22,587][53885] Updated weights for policy 1, policy_version 48222 (0.0007) [2023-10-08 09:43:26,055][53852] Updated weights for policy 0, policy_version 48450 (0.0008) [2023-10-08 09:43:26,310][53885] Updated weights for policy 1, policy_version 48232 (0.0007) [2023-10-08 09:43:26,427][53852] Updated weights for policy 0, policy_version 48460 (0.0007) [2023-10-08 09:43:26,676][53885] Updated weights for policy 1, policy_version 48242 (0.0007) [2023-10-08 09:43:26,792][53852] Updated weights for policy 0, policy_version 48470 (0.0007) [2023-10-08 09:43:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 98992128. Throughput: 0: 1797.0, 1: 1822.9. Samples: 24760710. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 09:43:27,015][52710] Avg episode reward: [(0, '12.980'), (1, '30.280')] [2023-10-08 09:43:27,049][53885] Updated weights for policy 1, policy_version 48252 (0.0009) [2023-10-08 09:43:27,160][53852] Updated weights for policy 0, policy_version 48480 (0.0008) [2023-10-08 09:43:30,641][53885] Updated weights for policy 1, policy_version 48262 (0.0007) [2023-10-08 09:43:30,850][53852] Updated weights for policy 0, policy_version 48490 (0.0007) [2023-10-08 09:43:31,004][53885] Updated weights for policy 1, policy_version 48272 (0.0008) [2023-10-08 09:43:31,221][53852] Updated weights for policy 0, policy_version 48500 (0.0009) [2023-10-08 09:43:31,371][53885] Updated weights for policy 1, policy_version 48282 (0.0008) [2023-10-08 09:43:31,580][53852] Updated weights for policy 0, policy_version 48510 (0.0008) [2023-10-08 09:43:32,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 99123200. Throughput: 0: 1819.3, 1: 1815.8. Samples: 24783386. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 09:43:32,016][52710] Avg episode reward: [(0, '12.880'), (1, '26.040')] [2023-10-08 09:43:35,070][53885] Updated weights for policy 1, policy_version 48292 (0.0007) [2023-10-08 09:43:35,202][53852] Updated weights for policy 0, policy_version 48520 (0.0007) [2023-10-08 09:43:35,434][53885] Updated weights for policy 1, policy_version 48302 (0.0009) [2023-10-08 09:43:35,572][53852] Updated weights for policy 0, policy_version 48530 (0.0009) [2023-10-08 09:43:35,797][53885] Updated weights for policy 1, policy_version 48312 (0.0009) [2023-10-08 09:43:35,933][53852] Updated weights for policy 0, policy_version 48540 (0.0008) [2023-10-08 09:43:37,015][52710] Fps is (10 sec: 19660.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99188736. Throughput: 0: 1806.9, 1: 1826.8. Samples: 24803820. Policy #0 lag: (min: 31.0, avg: 36.6, max: 63.0) [2023-10-08 09:43:37,016][52710] Avg episode reward: [(0, '12.470'), (1, '25.850')] [2023-10-08 09:43:39,509][53885] Updated weights for policy 1, policy_version 48322 (0.0008) [2023-10-08 09:43:39,513][53852] Updated weights for policy 0, policy_version 48550 (0.0009) [2023-10-08 09:43:39,883][53852] Updated weights for policy 0, policy_version 48560 (0.0007) [2023-10-08 09:43:39,892][53885] Updated weights for policy 1, policy_version 48332 (0.0007) [2023-10-08 09:43:40,257][53885] Updated weights for policy 1, policy_version 48342 (0.0009) [2023-10-08 09:43:40,261][53852] Updated weights for policy 0, policy_version 48570 (0.0007) [2023-10-08 09:43:40,627][53885] Updated weights for policy 1, policy_version 48352 (0.0008) [2023-10-08 09:43:42,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99254272. Throughput: 0: 1814.9, 1: 1817.9. Samples: 24816266. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:43:42,016][52710] Avg episode reward: [(0, '11.780'), (1, '30.210')] [2023-10-08 09:43:44,057][53852] Updated weights for policy 0, policy_version 48580 (0.0007) [2023-10-08 09:43:44,406][53885] Updated weights for policy 1, policy_version 48362 (0.0007) [2023-10-08 09:43:44,425][53852] Updated weights for policy 0, policy_version 48590 (0.0007) [2023-10-08 09:43:44,781][53885] Updated weights for policy 1, policy_version 48372 (0.0007) [2023-10-08 09:43:44,799][53852] Updated weights for policy 0, policy_version 48600 (0.0007) [2023-10-08 09:43:45,146][53885] Updated weights for policy 1, policy_version 48382 (0.0010) [2023-10-08 09:43:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 99319808. Throughput: 0: 1811.7, 1: 1816.5. Samples: 24836068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:43:47,016][52710] Avg episode reward: [(0, '10.120'), (1, '27.820')] [2023-10-08 09:43:48,448][53852] Updated weights for policy 0, policy_version 48610 (0.0008) [2023-10-08 09:43:48,829][53852] Updated weights for policy 0, policy_version 48620 (0.0007) [2023-10-08 09:43:48,884][53885] Updated weights for policy 1, policy_version 48392 (0.0008) [2023-10-08 09:43:49,189][53852] Updated weights for policy 0, policy_version 48630 (0.0008) [2023-10-08 09:43:49,260][53885] Updated weights for policy 1, policy_version 48402 (0.0008) [2023-10-08 09:43:49,551][53852] Updated weights for policy 0, policy_version 48640 (0.0007) [2023-10-08 09:43:49,621][53885] Updated weights for policy 1, policy_version 48412 (0.0010) [2023-10-08 09:43:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99385344. Throughput: 0: 1808.7, 1: 1814.4. Samples: 24858898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:43:52,015][52710] Avg episode reward: [(0, '12.090'), (1, '28.100')] [2023-10-08 09:43:53,223][53885] Updated weights for policy 1, policy_version 48422 (0.0008) [2023-10-08 09:43:53,399][53852] Updated weights for policy 0, policy_version 48650 (0.0008) [2023-10-08 09:43:53,587][53885] Updated weights for policy 1, policy_version 48432 (0.0007) [2023-10-08 09:43:53,773][53852] Updated weights for policy 0, policy_version 48660 (0.0009) [2023-10-08 09:43:53,961][53885] Updated weights for policy 1, policy_version 48442 (0.0008) [2023-10-08 09:43:54,138][53852] Updated weights for policy 0, policy_version 48670 (0.0009) [2023-10-08 09:43:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99450880. Throughput: 0: 1808.9, 1: 1813.6. Samples: 24868794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:43:57,016][52710] Avg episode reward: [(0, '12.880'), (1, '31.000')] [2023-10-08 09:43:57,702][53885] Updated weights for policy 1, policy_version 48452 (0.0007) [2023-10-08 09:43:57,855][53852] Updated weights for policy 0, policy_version 48680 (0.0008) [2023-10-08 09:43:58,069][53885] Updated weights for policy 1, policy_version 48462 (0.0008) [2023-10-08 09:43:58,225][53852] Updated weights for policy 0, policy_version 48690 (0.0008) [2023-10-08 09:43:58,426][53885] Updated weights for policy 1, policy_version 48472 (0.0008) [2023-10-08 09:43:58,596][53852] Updated weights for policy 0, policy_version 48700 (0.0008) [2023-10-08 09:44:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99516416. Throughput: 0: 1805.8, 1: 1810.0. Samples: 24891144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:44:02,016][52710] Avg episode reward: [(0, '10.590'), (1, '33.620')] [2023-10-08 09:44:02,236][53885] Updated weights for policy 1, policy_version 48482 (0.0008) [2023-10-08 09:44:02,328][53852] Updated weights for policy 0, policy_version 48710 (0.0008) [2023-10-08 09:44:02,604][53885] Updated weights for policy 1, policy_version 48492 (0.0007) [2023-10-08 09:44:02,694][53852] Updated weights for policy 0, policy_version 48720 (0.0010) [2023-10-08 09:44:02,978][53885] Updated weights for policy 1, policy_version 48502 (0.0009) [2023-10-08 09:44:03,062][53852] Updated weights for policy 0, policy_version 48730 (0.0007) [2023-10-08 09:44:03,343][53885] Updated weights for policy 1, policy_version 48512 (0.0009) [2023-10-08 09:44:06,858][53852] Updated weights for policy 0, policy_version 48740 (0.0008) [2023-10-08 09:44:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99581952. Throughput: 0: 1811.6, 1: 1815.5. Samples: 24913864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:44:07,016][52710] Avg episode reward: [(0, '11.190'), (1, '31.720')] [2023-10-08 09:44:07,035][53885] Updated weights for policy 1, policy_version 48522 (0.0008) [2023-10-08 09:44:07,244][53852] Updated weights for policy 0, policy_version 48750 (0.0008) [2023-10-08 09:44:07,391][53885] Updated weights for policy 1, policy_version 48532 (0.0007) [2023-10-08 09:44:07,610][53852] Updated weights for policy 0, policy_version 48760 (0.0010) [2023-10-08 09:44:07,770][53885] Updated weights for policy 1, policy_version 48542 (0.0010) [2023-10-08 09:44:07,837][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000048544_49709056.pth... [2023-10-08 09:44:07,870][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000046816_47939584.pth [2023-10-08 09:44:07,902][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000048768_49938432.pth... [2023-10-08 09:44:07,941][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000047040_48168960.pth [2023-10-08 09:44:11,403][53852] Updated weights for policy 0, policy_version 48770 (0.0009) [2023-10-08 09:44:11,487][53885] Updated weights for policy 1, policy_version 48552 (0.0007) [2023-10-08 09:44:11,779][53852] Updated weights for policy 0, policy_version 48780 (0.0010) [2023-10-08 09:44:11,857][53885] Updated weights for policy 1, policy_version 48562 (0.0008) [2023-10-08 09:44:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 99647488. Throughput: 0: 1808.9, 1: 1810.7. Samples: 24923592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:44:12,015][52710] Avg episode reward: [(0, '11.520'), (1, '30.770')] [2023-10-08 09:44:12,153][53852] Updated weights for policy 0, policy_version 48790 (0.0008) [2023-10-08 09:44:12,213][53885] Updated weights for policy 1, policy_version 48572 (0.0009) [2023-10-08 09:44:12,514][53852] Updated weights for policy 0, policy_version 48800 (0.0007) [2023-10-08 09:44:15,919][53885] Updated weights for policy 1, policy_version 48582 (0.0008) [2023-10-08 09:44:16,157][53852] Updated weights for policy 0, policy_version 48810 (0.0008) [2023-10-08 09:44:16,293][53885] Updated weights for policy 1, policy_version 48592 (0.0008) [2023-10-08 09:44:16,531][53852] Updated weights for policy 0, policy_version 48820 (0.0008) [2023-10-08 09:44:16,664][53885] Updated weights for policy 1, policy_version 48602 (0.0008) [2023-10-08 09:44:16,904][53852] Updated weights for policy 0, policy_version 48830 (0.0008) [2023-10-08 09:44:17,015][52710] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 99778560. Throughput: 0: 1803.8, 1: 1818.5. Samples: 24946388. Policy #0 lag: (min: 7.0, avg: 22.1, max: 39.0) [2023-10-08 09:44:17,015][52710] Avg episode reward: [(0, '13.590'), (1, '33.630')] [2023-10-08 09:44:20,265][53885] Updated weights for policy 1, policy_version 48612 (0.0008) [2023-10-08 09:44:20,618][53852] Updated weights for policy 0, policy_version 48840 (0.0008) [2023-10-08 09:44:20,632][53885] Updated weights for policy 1, policy_version 48622 (0.0008) [2023-10-08 09:44:20,979][53852] Updated weights for policy 0, policy_version 48850 (0.0007) [2023-10-08 09:44:21,001][53885] Updated weights for policy 1, policy_version 48632 (0.0007) [2023-10-08 09:44:21,358][53852] Updated weights for policy 0, policy_version 48860 (0.0008) [2023-10-08 09:44:22,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 99844096. Throughput: 0: 1801.8, 1: 1810.4. Samples: 24966370. Policy #0 lag: (min: 7.0, avg: 22.1, max: 39.0) [2023-10-08 09:44:22,016][52710] Avg episode reward: [(0, '15.470'), (1, '34.510')] [2023-10-08 09:44:24,700][53885] Updated weights for policy 1, policy_version 48642 (0.0007) [2023-10-08 09:44:24,963][53852] Updated weights for policy 0, policy_version 48870 (0.0007) [2023-10-08 09:44:25,101][53885] Updated weights for policy 1, policy_version 48652 (0.0009) [2023-10-08 09:44:25,331][53852] Updated weights for policy 0, policy_version 48880 (0.0007) [2023-10-08 09:44:25,466][53885] Updated weights for policy 1, policy_version 48662 (0.0008) [2023-10-08 09:44:25,706][53852] Updated weights for policy 0, policy_version 48890 (0.0008) [2023-10-08 09:44:25,828][53885] Updated weights for policy 1, policy_version 48672 (0.0008) [2023-10-08 09:44:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 99909632. Throughput: 0: 1804.7, 1: 1816.4. Samples: 24979212. Policy #0 lag: (min: 7.0, avg: 22.1, max: 39.0) [2023-10-08 09:44:27,016][52710] Avg episode reward: [(0, '15.130'), (1, '30.530')] [2023-10-08 09:44:29,410][53852] Updated weights for policy 0, policy_version 48900 (0.0008) [2023-10-08 09:44:29,423][53885] Updated weights for policy 1, policy_version 48682 (0.0007) [2023-10-08 09:44:29,780][53852] Updated weights for policy 0, policy_version 48910 (0.0007) [2023-10-08 09:44:29,795][53885] Updated weights for policy 1, policy_version 48692 (0.0008) [2023-10-08 09:44:30,146][53852] Updated weights for policy 0, policy_version 48920 (0.0007) [2023-10-08 09:44:30,153][53885] Updated weights for policy 1, policy_version 48702 (0.0008) [2023-10-08 09:44:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 99975168. Throughput: 0: 1803.6, 1: 1821.6. Samples: 24999200. Policy #0 lag: (min: 7.0, avg: 22.1, max: 39.0) [2023-10-08 09:44:32,015][52710] Avg episode reward: [(0, '15.880'), (1, '34.860')] [2023-10-08 09:44:33,801][53852] Updated weights for policy 0, policy_version 48930 (0.0010) [2023-10-08 09:44:33,898][53885] Updated weights for policy 1, policy_version 48712 (0.0007) [2023-10-08 09:44:34,164][53852] Updated weights for policy 0, policy_version 48940 (0.0009) [2023-10-08 09:44:34,268][53885] Updated weights for policy 1, policy_version 48722 (0.0008) [2023-10-08 09:44:34,533][53852] Updated weights for policy 0, policy_version 48950 (0.0009) [2023-10-08 09:44:34,638][53885] Updated weights for policy 1, policy_version 48732 (0.0009) [2023-10-08 09:44:34,903][53852] Updated weights for policy 0, policy_version 48960 (0.0007) [2023-10-08 09:44:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100040704. Throughput: 0: 1807.2, 1: 1817.6. Samples: 25022010. Policy #0 lag: (min: 7.0, avg: 22.1, max: 39.0) [2023-10-08 09:44:37,016][52710] Avg episode reward: [(0, '16.240'), (1, '31.780')] [2023-10-08 09:44:38,358][53885] Updated weights for policy 1, policy_version 48742 (0.0009) [2023-10-08 09:44:38,577][53852] Updated weights for policy 0, policy_version 48970 (0.0009) [2023-10-08 09:44:38,729][53885] Updated weights for policy 1, policy_version 48752 (0.0008) [2023-10-08 09:44:38,947][53852] Updated weights for policy 0, policy_version 48980 (0.0009) [2023-10-08 09:44:39,091][53885] Updated weights for policy 1, policy_version 48762 (0.0007) [2023-10-08 09:44:39,313][53852] Updated weights for policy 0, policy_version 48990 (0.0009) [2023-10-08 09:44:42,016][52710] Fps is (10 sec: 13106.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 100106240. Throughput: 0: 1804.6, 1: 1816.9. Samples: 25031762. Policy #0 lag: (min: 7.0, avg: 22.1, max: 39.0) [2023-10-08 09:44:42,016][52710] Avg episode reward: [(0, '16.430'), (1, '27.320')] [2023-10-08 09:44:42,724][53885] Updated weights for policy 1, policy_version 48772 (0.0008) [2023-10-08 09:44:43,061][53852] Updated weights for policy 0, policy_version 49000 (0.0008) [2023-10-08 09:44:43,086][53885] Updated weights for policy 1, policy_version 48782 (0.0007) [2023-10-08 09:44:43,429][53852] Updated weights for policy 0, policy_version 49010 (0.0007) [2023-10-08 09:44:43,455][53885] Updated weights for policy 1, policy_version 48792 (0.0007) [2023-10-08 09:44:43,798][53852] Updated weights for policy 0, policy_version 49020 (0.0009) [2023-10-08 09:44:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100171776. Throughput: 0: 1809.0, 1: 1821.9. Samples: 25054534. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:44:47,015][52710] Avg episode reward: [(0, '15.090'), (1, '30.680')] [2023-10-08 09:44:47,175][53885] Updated weights for policy 1, policy_version 48802 (0.0008) [2023-10-08 09:44:47,471][53852] Updated weights for policy 0, policy_version 49030 (0.0008) [2023-10-08 09:44:47,538][53885] Updated weights for policy 1, policy_version 48812 (0.0009) [2023-10-08 09:44:47,840][53852] Updated weights for policy 0, policy_version 49040 (0.0009) [2023-10-08 09:44:47,904][53885] Updated weights for policy 1, policy_version 48822 (0.0008) [2023-10-08 09:44:48,201][53852] Updated weights for policy 0, policy_version 49050 (0.0008) [2023-10-08 09:44:48,275][53885] Updated weights for policy 1, policy_version 48832 (0.0008) [2023-10-08 09:44:51,948][53885] Updated weights for policy 1, policy_version 48842 (0.0007) [2023-10-08 09:44:52,015][52710] Fps is (10 sec: 13107.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 100237312. Throughput: 0: 1811.3, 1: 1819.1. Samples: 25077234. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:44:52,016][52710] Avg episode reward: [(0, '16.220'), (1, '32.320')] [2023-10-08 09:44:52,058][53852] Updated weights for policy 0, policy_version 49060 (0.0008) [2023-10-08 09:44:52,313][53885] Updated weights for policy 1, policy_version 48852 (0.0007) [2023-10-08 09:44:52,430][53852] Updated weights for policy 0, policy_version 49070 (0.0009) [2023-10-08 09:44:52,681][53885] Updated weights for policy 1, policy_version 48862 (0.0007) [2023-10-08 09:44:52,801][53852] Updated weights for policy 0, policy_version 49080 (0.0010) [2023-10-08 09:44:56,373][53852] Updated weights for policy 0, policy_version 49090 (0.0008) [2023-10-08 09:44:56,456][53885] Updated weights for policy 1, policy_version 48872 (0.0008) [2023-10-08 09:44:56,742][53852] Updated weights for policy 0, policy_version 49100 (0.0007) [2023-10-08 09:44:56,812][53885] Updated weights for policy 1, policy_version 48882 (0.0008) [2023-10-08 09:44:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100302848. Throughput: 0: 1810.3, 1: 1822.0. Samples: 25087046. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:44:57,015][52710] Avg episode reward: [(0, '18.520'), (1, '29.340')] [2023-10-08 09:44:57,115][53852] Updated weights for policy 0, policy_version 49110 (0.0007) [2023-10-08 09:44:57,177][53885] Updated weights for policy 1, policy_version 48892 (0.0008) [2023-10-08 09:44:57,481][53852] Updated weights for policy 0, policy_version 49120 (0.0007) [2023-10-08 09:45:00,850][53885] Updated weights for policy 1, policy_version 48902 (0.0008) [2023-10-08 09:45:01,218][53885] Updated weights for policy 1, policy_version 48912 (0.0007) [2023-10-08 09:45:01,268][53852] Updated weights for policy 0, policy_version 49130 (0.0007) [2023-10-08 09:45:01,595][53885] Updated weights for policy 1, policy_version 48922 (0.0009) [2023-10-08 09:45:01,631][53852] Updated weights for policy 0, policy_version 49140 (0.0007) [2023-10-08 09:45:02,003][53852] Updated weights for policy 0, policy_version 49150 (0.0008) [2023-10-08 09:45:02,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 100401152. Throughput: 0: 1814.5, 1: 1820.3. Samples: 25109952. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:45:02,015][52710] Avg episode reward: [(0, '19.460'), (1, '29.150')] [2023-10-08 09:45:05,347][53885] Updated weights for policy 1, policy_version 48932 (0.0009) [2023-10-08 09:45:05,710][53885] Updated weights for policy 1, policy_version 48942 (0.0009) [2023-10-08 09:45:05,790][53852] Updated weights for policy 0, policy_version 49160 (0.0009) [2023-10-08 09:45:06,083][53885] Updated weights for policy 1, policy_version 48952 (0.0007) [2023-10-08 09:45:06,155][53852] Updated weights for policy 0, policy_version 49170 (0.0008) [2023-10-08 09:45:06,528][53852] Updated weights for policy 0, policy_version 49180 (0.0007) [2023-10-08 09:45:07,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 100499456. Throughput: 0: 1812.2, 1: 1817.9. Samples: 25129726. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:45:07,016][52710] Avg episode reward: [(0, '20.650'), (1, '30.600')] [2023-10-08 09:45:09,814][53885] Updated weights for policy 1, policy_version 48962 (0.0008) [2023-10-08 09:45:10,056][53852] Updated weights for policy 0, policy_version 49190 (0.0007) [2023-10-08 09:45:10,221][53885] Updated weights for policy 1, policy_version 48972 (0.0009) [2023-10-08 09:45:10,437][53852] Updated weights for policy 0, policy_version 49200 (0.0007) [2023-10-08 09:45:10,595][53885] Updated weights for policy 1, policy_version 48982 (0.0007) [2023-10-08 09:45:10,792][53852] Updated weights for policy 0, policy_version 49210 (0.0008) [2023-10-08 09:45:10,958][53885] Updated weights for policy 1, policy_version 48992 (0.0008) [2023-10-08 09:45:12,015][52710] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 100564992. Throughput: 0: 1812.1, 1: 1810.5. Samples: 25142228. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:45:12,016][52710] Avg episode reward: [(0, '18.680'), (1, '28.820')] [2023-10-08 09:45:14,493][53852] Updated weights for policy 0, policy_version 49220 (0.0009) [2023-10-08 09:45:14,734][53885] Updated weights for policy 1, policy_version 49002 (0.0007) [2023-10-08 09:45:14,860][53852] Updated weights for policy 0, policy_version 49230 (0.0007) [2023-10-08 09:45:15,097][53885] Updated weights for policy 1, policy_version 49012 (0.0008) [2023-10-08 09:45:15,222][53852] Updated weights for policy 0, policy_version 49240 (0.0007) [2023-10-08 09:45:15,463][53885] Updated weights for policy 1, policy_version 49022 (0.0007) [2023-10-08 09:45:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 100630528. Throughput: 0: 1813.6, 1: 1800.2. Samples: 25161820. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:45:17,016][52710] Avg episode reward: [(0, '16.660'), (1, '28.840')] [2023-10-08 09:45:19,025][53852] Updated weights for policy 0, policy_version 49250 (0.0008) [2023-10-08 09:45:19,206][53885] Updated weights for policy 1, policy_version 49032 (0.0008) [2023-10-08 09:45:19,396][53852] Updated weights for policy 0, policy_version 49260 (0.0007) [2023-10-08 09:45:19,577][53885] Updated weights for policy 1, policy_version 49042 (0.0007) [2023-10-08 09:45:19,769][53852] Updated weights for policy 0, policy_version 49270 (0.0009) [2023-10-08 09:45:19,941][53885] Updated weights for policy 1, policy_version 49052 (0.0009) [2023-10-08 09:45:20,141][53852] Updated weights for policy 0, policy_version 49280 (0.0010) [2023-10-08 09:45:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100696064. Throughput: 0: 1809.0, 1: 1802.3. Samples: 25184520. Policy #0 lag: (min: 21.0, avg: 21.3, max: 33.0) [2023-10-08 09:45:22,015][52710] Avg episode reward: [(0, '18.510'), (1, '29.050')] [2023-10-08 09:45:23,786][53885] Updated weights for policy 1, policy_version 49062 (0.0009) [2023-10-08 09:45:23,891][53852] Updated weights for policy 0, policy_version 49290 (0.0009) [2023-10-08 09:45:24,157][53885] Updated weights for policy 1, policy_version 49072 (0.0009) [2023-10-08 09:45:24,269][53852] Updated weights for policy 0, policy_version 49300 (0.0008) [2023-10-08 09:45:24,513][53885] Updated weights for policy 1, policy_version 49082 (0.0007) [2023-10-08 09:45:24,634][53852] Updated weights for policy 0, policy_version 49310 (0.0007) [2023-10-08 09:45:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 100761600. Throughput: 0: 1813.9, 1: 1805.7. Samples: 25194642. Policy #0 lag: (min: 21.0, avg: 21.3, max: 33.0) [2023-10-08 09:45:27,016][52710] Avg episode reward: [(0, '18.390'), (1, '30.000')] [2023-10-08 09:45:28,192][53885] Updated weights for policy 1, policy_version 49092 (0.0008) [2023-10-08 09:45:28,356][53852] Updated weights for policy 0, policy_version 49320 (0.0008) [2023-10-08 09:45:28,564][53885] Updated weights for policy 1, policy_version 49102 (0.0008) [2023-10-08 09:45:28,720][53852] Updated weights for policy 0, policy_version 49330 (0.0009) [2023-10-08 09:45:28,933][53885] Updated weights for policy 1, policy_version 49112 (0.0008) [2023-10-08 09:45:29,098][53852] Updated weights for policy 0, policy_version 49340 (0.0009) [2023-10-08 09:45:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 100827136. Throughput: 0: 1815.1, 1: 1798.6. Samples: 25217150. Policy #0 lag: (min: 21.0, avg: 21.3, max: 33.0) [2023-10-08 09:45:32,016][52710] Avg episode reward: [(0, '17.290'), (1, '30.960')] [2023-10-08 09:45:32,633][53885] Updated weights for policy 1, policy_version 49122 (0.0007) [2023-10-08 09:45:32,737][53852] Updated weights for policy 0, policy_version 49350 (0.0007) [2023-10-08 09:45:32,996][53885] Updated weights for policy 1, policy_version 49132 (0.0008) [2023-10-08 09:45:33,107][53852] Updated weights for policy 0, policy_version 49360 (0.0008) [2023-10-08 09:45:33,359][53885] Updated weights for policy 1, policy_version 49142 (0.0008) [2023-10-08 09:45:33,474][53852] Updated weights for policy 0, policy_version 49370 (0.0008) [2023-10-08 09:45:33,729][53885] Updated weights for policy 1, policy_version 49152 (0.0008) [2023-10-08 09:45:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 100892672. Throughput: 0: 1818.4, 1: 1802.5. Samples: 25240178. Policy #0 lag: (min: 21.0, avg: 21.3, max: 33.0) [2023-10-08 09:45:37,016][52710] Avg episode reward: [(0, '16.900'), (1, '28.940')] [2023-10-08 09:45:37,229][53852] Updated weights for policy 0, policy_version 49380 (0.0007) [2023-10-08 09:45:37,286][53885] Updated weights for policy 1, policy_version 49162 (0.0008) [2023-10-08 09:45:37,610][53852] Updated weights for policy 0, policy_version 49390 (0.0008) [2023-10-08 09:45:37,657][53885] Updated weights for policy 1, policy_version 49172 (0.0007) [2023-10-08 09:45:37,985][53852] Updated weights for policy 0, policy_version 49400 (0.0008) [2023-10-08 09:45:38,026][53885] Updated weights for policy 1, policy_version 49182 (0.0008) [2023-10-08 09:45:41,634][53852] Updated weights for policy 0, policy_version 49410 (0.0007) [2023-10-08 09:45:41,873][53885] Updated weights for policy 1, policy_version 49192 (0.0008) [2023-10-08 09:45:41,995][53852] Updated weights for policy 0, policy_version 49420 (0.0007) [2023-10-08 09:45:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14440.2). Total num frames: 100958208. Throughput: 0: 1821.6, 1: 1797.7. Samples: 25249916. Policy #0 lag: (min: 21.0, avg: 21.3, max: 33.0) [2023-10-08 09:45:42,016][52710] Avg episode reward: [(0, '18.440'), (1, '31.730')] [2023-10-08 09:45:42,236][53885] Updated weights for policy 1, policy_version 49202 (0.0009) [2023-10-08 09:45:42,361][53852] Updated weights for policy 0, policy_version 49430 (0.0007) [2023-10-08 09:45:42,604][53885] Updated weights for policy 1, policy_version 49212 (0.0007) [2023-10-08 09:45:42,723][53852] Updated weights for policy 0, policy_version 49440 (0.0007) [2023-10-08 09:45:46,358][53885] Updated weights for policy 1, policy_version 49222 (0.0007) [2023-10-08 09:45:46,462][53852] Updated weights for policy 0, policy_version 49450 (0.0007) [2023-10-08 09:45:46,722][53885] Updated weights for policy 1, policy_version 49232 (0.0007) [2023-10-08 09:45:46,829][53852] Updated weights for policy 0, policy_version 49460 (0.0007) [2023-10-08 09:45:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 101023744. Throughput: 0: 1823.1, 1: 1795.7. Samples: 25272798. Policy #0 lag: (min: 21.0, avg: 21.3, max: 33.0) [2023-10-08 09:45:47,016][52710] Avg episode reward: [(0, '20.160'), (1, '31.260')] [2023-10-08 09:45:47,093][53885] Updated weights for policy 1, policy_version 49242 (0.0008) [2023-10-08 09:45:47,196][53852] Updated weights for policy 0, policy_version 49470 (0.0007) [2023-10-08 09:45:50,614][53885] Updated weights for policy 1, policy_version 49252 (0.0009) [2023-10-08 09:45:50,801][53852] Updated weights for policy 0, policy_version 49480 (0.0007) [2023-10-08 09:45:50,982][53885] Updated weights for policy 1, policy_version 49262 (0.0008) [2023-10-08 09:45:51,175][53852] Updated weights for policy 0, policy_version 49490 (0.0007) [2023-10-08 09:45:51,347][53885] Updated weights for policy 1, policy_version 49272 (0.0008) [2023-10-08 09:45:51,535][53852] Updated weights for policy 0, policy_version 49500 (0.0009) [2023-10-08 09:45:52,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101154816. Throughput: 0: 1825.6, 1: 1806.4. Samples: 25293170. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-10-08 09:45:52,016][52710] Avg episode reward: [(0, '19.890'), (1, '32.240')] [2023-10-08 09:45:55,140][53852] Updated weights for policy 0, policy_version 49510 (0.0010) [2023-10-08 09:45:55,292][53885] Updated weights for policy 1, policy_version 49282 (0.0008) [2023-10-08 09:45:55,515][53852] Updated weights for policy 0, policy_version 49520 (0.0008) [2023-10-08 09:45:55,689][53885] Updated weights for policy 1, policy_version 49292 (0.0009) [2023-10-08 09:45:55,877][53852] Updated weights for policy 0, policy_version 49530 (0.0008) [2023-10-08 09:45:56,053][53885] Updated weights for policy 1, policy_version 49302 (0.0009) [2023-10-08 09:45:56,424][53885] Updated weights for policy 1, policy_version 49312 (0.0009) [2023-10-08 09:45:57,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101220352. Throughput: 0: 1824.0, 1: 1807.3. Samples: 25305636. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-10-08 09:45:57,015][52710] Avg episode reward: [(0, '19.500'), (1, '28.880')] [2023-10-08 09:45:59,378][53852] Updated weights for policy 0, policy_version 49540 (0.0009) [2023-10-08 09:45:59,757][53852] Updated weights for policy 0, policy_version 49550 (0.0008) [2023-10-08 09:45:59,963][53885] Updated weights for policy 1, policy_version 49322 (0.0008) [2023-10-08 09:46:00,124][53852] Updated weights for policy 0, policy_version 49560 (0.0008) [2023-10-08 09:46:00,336][53885] Updated weights for policy 1, policy_version 49332 (0.0007) [2023-10-08 09:46:00,703][53885] Updated weights for policy 1, policy_version 49342 (0.0007) [2023-10-08 09:46:02,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 101285888. Throughput: 0: 1825.2, 1: 1819.7. Samples: 25325842. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-10-08 09:46:02,016][52710] Avg episode reward: [(0, '18.940'), (1, '31.010')] [2023-10-08 09:46:03,736][53852] Updated weights for policy 0, policy_version 49570 (0.0007) [2023-10-08 09:46:04,094][53852] Updated weights for policy 0, policy_version 49580 (0.0007) [2023-10-08 09:46:04,438][53885] Updated weights for policy 1, policy_version 49352 (0.0008) [2023-10-08 09:46:04,473][53852] Updated weights for policy 0, policy_version 49590 (0.0008) [2023-10-08 09:46:04,808][53885] Updated weights for policy 1, policy_version 49362 (0.0008) [2023-10-08 09:46:04,839][53852] Updated weights for policy 0, policy_version 49600 (0.0008) [2023-10-08 09:46:05,180][53885] Updated weights for policy 1, policy_version 49372 (0.0010) [2023-10-08 09:46:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 101351424. Throughput: 0: 1836.1, 1: 1808.7. Samples: 25348538. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-10-08 09:46:07,016][52710] Avg episode reward: [(0, '19.110'), (1, '30.260')] [2023-10-08 09:46:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000049376_50561024.pth... [2023-10-08 09:46:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000049600_50790400.pth... [2023-10-08 09:46:07,060][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000047680_48824320.pth [2023-10-08 09:46:07,072][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000047904_49053696.pth [2023-10-08 09:46:08,517][53852] Updated weights for policy 0, policy_version 49610 (0.0009) [2023-10-08 09:46:08,883][53852] Updated weights for policy 0, policy_version 49620 (0.0007) [2023-10-08 09:46:08,940][53885] Updated weights for policy 1, policy_version 49382 (0.0008) [2023-10-08 09:46:09,248][53852] Updated weights for policy 0, policy_version 49630 (0.0007) [2023-10-08 09:46:09,300][53885] Updated weights for policy 1, policy_version 49392 (0.0007) [2023-10-08 09:46:09,680][53885] Updated weights for policy 1, policy_version 49402 (0.0009) [2023-10-08 09:46:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101416960. Throughput: 0: 1831.3, 1: 1816.8. Samples: 25358804. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-10-08 09:46:12,016][52710] Avg episode reward: [(0, '19.970'), (1, '28.690')] [2023-10-08 09:46:13,043][53852] Updated weights for policy 0, policy_version 49640 (0.0007) [2023-10-08 09:46:13,405][53852] Updated weights for policy 0, policy_version 49650 (0.0007) [2023-10-08 09:46:13,422][53885] Updated weights for policy 1, policy_version 49412 (0.0010) [2023-10-08 09:46:13,770][53852] Updated weights for policy 0, policy_version 49660 (0.0007) [2023-10-08 09:46:13,790][53885] Updated weights for policy 1, policy_version 49422 (0.0008) [2023-10-08 09:46:14,155][53885] Updated weights for policy 1, policy_version 49432 (0.0007) [2023-10-08 09:46:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101482496. Throughput: 0: 1831.2, 1: 1807.2. Samples: 25380874. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-10-08 09:46:17,016][52710] Avg episode reward: [(0, '21.000'), (1, '30.170')] [2023-10-08 09:46:17,402][53852] Updated weights for policy 0, policy_version 49670 (0.0007) [2023-10-08 09:46:17,774][53852] Updated weights for policy 0, policy_version 49680 (0.0009) [2023-10-08 09:46:17,979][53885] Updated weights for policy 1, policy_version 49442 (0.0010) [2023-10-08 09:46:18,133][53852] Updated weights for policy 0, policy_version 49690 (0.0007) [2023-10-08 09:46:18,351][53885] Updated weights for policy 1, policy_version 49452 (0.0007) [2023-10-08 09:46:18,717][53885] Updated weights for policy 1, policy_version 49462 (0.0007) [2023-10-08 09:46:19,088][53885] Updated weights for policy 1, policy_version 49472 (0.0009) [2023-10-08 09:46:21,943][53852] Updated weights for policy 0, policy_version 49700 (0.0008) [2023-10-08 09:46:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101548032. Throughput: 0: 1823.5, 1: 1805.4. Samples: 25403478. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-10-08 09:46:22,015][52710] Avg episode reward: [(0, '22.280'), (1, '30.390')] [2023-10-08 09:46:22,331][53852] Updated weights for policy 0, policy_version 49710 (0.0008) [2023-10-08 09:46:22,708][53852] Updated weights for policy 0, policy_version 49720 (0.0007) [2023-10-08 09:46:22,827][53885] Updated weights for policy 1, policy_version 49482 (0.0008) [2023-10-08 09:46:23,187][53885] Updated weights for policy 1, policy_version 49492 (0.0008) [2023-10-08 09:46:23,555][53885] Updated weights for policy 1, policy_version 49502 (0.0007) [2023-10-08 09:46:26,367][53852] Updated weights for policy 0, policy_version 49730 (0.0008) [2023-10-08 09:46:26,746][53852] Updated weights for policy 0, policy_version 49740 (0.0008) [2023-10-08 09:46:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 101613568. Throughput: 0: 1818.0, 1: 1807.0. Samples: 25413040. Policy #0 lag: (min: 31.0, avg: 39.3, max: 63.0) [2023-10-08 09:46:27,016][52710] Avg episode reward: [(0, '21.700'), (1, '30.290')] [2023-10-08 09:46:27,112][53852] Updated weights for policy 0, policy_version 49750 (0.0007) [2023-10-08 09:46:27,313][53885] Updated weights for policy 1, policy_version 49512 (0.0008) [2023-10-08 09:46:27,483][53852] Updated weights for policy 0, policy_version 49760 (0.0008) [2023-10-08 09:46:27,677][53885] Updated weights for policy 1, policy_version 49522 (0.0008) [2023-10-08 09:46:28,038][53885] Updated weights for policy 1, policy_version 49532 (0.0009) [2023-10-08 09:46:31,219][53852] Updated weights for policy 0, policy_version 49770 (0.0008) [2023-10-08 09:46:31,589][53852] Updated weights for policy 0, policy_version 49780 (0.0007) [2023-10-08 09:46:31,682][53885] Updated weights for policy 1, policy_version 49542 (0.0010) [2023-10-08 09:46:31,960][53852] Updated weights for policy 0, policy_version 49790 (0.0008) [2023-10-08 09:46:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 101679104. Throughput: 0: 1817.6, 1: 1803.5. Samples: 25435750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:46:32,015][52710] Avg episode reward: [(0, '21.290'), (1, '31.330')] [2023-10-08 09:46:32,054][53885] Updated weights for policy 1, policy_version 49552 (0.0007) [2023-10-08 09:46:32,411][53885] Updated weights for policy 1, policy_version 49562 (0.0008) [2023-10-08 09:46:35,586][53852] Updated weights for policy 0, policy_version 49800 (0.0009) [2023-10-08 09:46:35,953][53852] Updated weights for policy 0, policy_version 49810 (0.0007) [2023-10-08 09:46:36,129][53885] Updated weights for policy 1, policy_version 49572 (0.0008) [2023-10-08 09:46:36,319][53852] Updated weights for policy 0, policy_version 49820 (0.0007) [2023-10-08 09:46:36,499][53885] Updated weights for policy 1, policy_version 49582 (0.0008) [2023-10-08 09:46:36,858][53885] Updated weights for policy 1, policy_version 49592 (0.0011) [2023-10-08 09:46:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 101777408. Throughput: 0: 1822.7, 1: 1809.6. Samples: 25456626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:46:37,016][52710] Avg episode reward: [(0, '22.050'), (1, '30.970')] [2023-10-08 09:46:40,006][53852] Updated weights for policy 0, policy_version 49830 (0.0008) [2023-10-08 09:46:40,376][53852] Updated weights for policy 0, policy_version 49840 (0.0009) [2023-10-08 09:46:40,735][53885] Updated weights for policy 1, policy_version 49602 (0.0009) [2023-10-08 09:46:40,757][53852] Updated weights for policy 0, policy_version 49850 (0.0010) [2023-10-08 09:46:41,148][53885] Updated weights for policy 1, policy_version 49612 (0.0009) [2023-10-08 09:46:41,517][53885] Updated weights for policy 1, policy_version 49622 (0.0011) [2023-10-08 09:46:41,889][53885] Updated weights for policy 1, policy_version 49632 (0.0010) [2023-10-08 09:46:42,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101875712. Throughput: 0: 1827.1, 1: 1796.3. Samples: 25468694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:46:42,016][52710] Avg episode reward: [(0, '21.470'), (1, '31.460')] [2023-10-08 09:46:44,328][53852] Updated weights for policy 0, policy_version 49860 (0.0008) [2023-10-08 09:46:44,686][53852] Updated weights for policy 0, policy_version 49870 (0.0008) [2023-10-08 09:46:45,062][53852] Updated weights for policy 0, policy_version 49880 (0.0008) [2023-10-08 09:46:45,516][53885] Updated weights for policy 1, policy_version 49642 (0.0008) [2023-10-08 09:46:45,885][53885] Updated weights for policy 1, policy_version 49652 (0.0008) [2023-10-08 09:46:46,256][53885] Updated weights for policy 1, policy_version 49662 (0.0011) [2023-10-08 09:46:47,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101941248. Throughput: 0: 1825.0, 1: 1815.0. Samples: 25489644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:46:47,016][52710] Avg episode reward: [(0, '21.070'), (1, '32.190')] [2023-10-08 09:46:48,722][53852] Updated weights for policy 0, policy_version 49890 (0.0008) [2023-10-08 09:46:49,095][53852] Updated weights for policy 0, policy_version 49900 (0.0011) [2023-10-08 09:46:49,470][53852] Updated weights for policy 0, policy_version 49910 (0.0009) [2023-10-08 09:46:49,843][53852] Updated weights for policy 0, policy_version 49920 (0.0010) [2023-10-08 09:46:49,976][53885] Updated weights for policy 1, policy_version 49672 (0.0008) [2023-10-08 09:46:50,347][53885] Updated weights for policy 1, policy_version 49682 (0.0009) [2023-10-08 09:46:50,727][53885] Updated weights for policy 1, policy_version 49692 (0.0009) [2023-10-08 09:46:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102006784. Throughput: 0: 1818.9, 1: 1800.9. Samples: 25511430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:46:52,016][52710] Avg episode reward: [(0, '19.990'), (1, '32.930')] [2023-10-08 09:46:53,396][53852] Updated weights for policy 0, policy_version 49930 (0.0007) [2023-10-08 09:46:53,760][53852] Updated weights for policy 0, policy_version 49940 (0.0007) [2023-10-08 09:46:54,134][53852] Updated weights for policy 0, policy_version 49950 (0.0010) [2023-10-08 09:46:54,171][53885] Updated weights for policy 1, policy_version 49702 (0.0010) [2023-10-08 09:46:54,541][53885] Updated weights for policy 1, policy_version 49712 (0.0010) [2023-10-08 09:46:54,912][53885] Updated weights for policy 1, policy_version 49722 (0.0010) [2023-10-08 09:46:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102072320. Throughput: 0: 1822.0, 1: 1813.2. Samples: 25522388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:46:57,016][52710] Avg episode reward: [(0, '19.440'), (1, '32.150')] [2023-10-08 09:46:57,738][53852] Updated weights for policy 0, policy_version 49960 (0.0011) [2023-10-08 09:46:58,110][53852] Updated weights for policy 0, policy_version 49970 (0.0009) [2023-10-08 09:46:58,472][53852] Updated weights for policy 0, policy_version 49980 (0.0009) [2023-10-08 09:46:58,592][53885] Updated weights for policy 1, policy_version 49732 (0.0009) [2023-10-08 09:46:58,958][53885] Updated weights for policy 1, policy_version 49742 (0.0008) [2023-10-08 09:46:59,323][53885] Updated weights for policy 1, policy_version 49752 (0.0008) [2023-10-08 09:47:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102137856. Throughput: 0: 1828.0, 1: 1809.6. Samples: 25544566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:47:02,016][52710] Avg episode reward: [(0, '19.970'), (1, '32.680')] [2023-10-08 09:47:02,235][53852] Updated weights for policy 0, policy_version 49990 (0.0010) [2023-10-08 09:47:02,609][53852] Updated weights for policy 0, policy_version 50000 (0.0011) [2023-10-08 09:47:02,977][53852] Updated weights for policy 0, policy_version 50010 (0.0008) [2023-10-08 09:47:03,022][53885] Updated weights for policy 1, policy_version 49762 (0.0008) [2023-10-08 09:47:03,388][53885] Updated weights for policy 1, policy_version 49772 (0.0008) [2023-10-08 09:47:03,759][53885] Updated weights for policy 1, policy_version 49782 (0.0010) [2023-10-08 09:47:04,128][53885] Updated weights for policy 1, policy_version 49792 (0.0011) [2023-10-08 09:47:06,702][53852] Updated weights for policy 0, policy_version 50020 (0.0010) [2023-10-08 09:47:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102203392. Throughput: 0: 1829.5, 1: 1813.8. Samples: 25567426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:47:07,016][52710] Avg episode reward: [(0, '19.690'), (1, '32.470')] [2023-10-08 09:47:07,076][53852] Updated weights for policy 0, policy_version 50030 (0.0008) [2023-10-08 09:47:07,449][53852] Updated weights for policy 0, policy_version 50040 (0.0007) [2023-10-08 09:47:07,813][53885] Updated weights for policy 1, policy_version 49802 (0.0008) [2023-10-08 09:47:08,180][53885] Updated weights for policy 1, policy_version 49812 (0.0008) [2023-10-08 09:47:08,546][53885] Updated weights for policy 1, policy_version 49822 (0.0007) [2023-10-08 09:47:11,132][53852] Updated weights for policy 0, policy_version 50050 (0.0007) [2023-10-08 09:47:11,516][53852] Updated weights for policy 0, policy_version 50060 (0.0007) [2023-10-08 09:47:11,894][53852] Updated weights for policy 0, policy_version 50070 (0.0008) [2023-10-08 09:47:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102268928. Throughput: 0: 1836.0, 1: 1820.3. Samples: 25577576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:47:12,015][52710] Avg episode reward: [(0, '21.970'), (1, '33.360')] [2023-10-08 09:47:12,145][53885] Updated weights for policy 1, policy_version 49832 (0.0008) [2023-10-08 09:47:12,251][53852] Updated weights for policy 0, policy_version 50080 (0.0007) [2023-10-08 09:47:12,508][53885] Updated weights for policy 1, policy_version 49842 (0.0008) [2023-10-08 09:47:12,876][53885] Updated weights for policy 1, policy_version 49852 (0.0009) [2023-10-08 09:47:15,913][53852] Updated weights for policy 0, policy_version 50090 (0.0009) [2023-10-08 09:47:16,285][53852] Updated weights for policy 0, policy_version 50100 (0.0007) [2023-10-08 09:47:16,558][53885] Updated weights for policy 1, policy_version 49862 (0.0008) [2023-10-08 09:47:16,655][53852] Updated weights for policy 0, policy_version 50110 (0.0008) [2023-10-08 09:47:16,921][53885] Updated weights for policy 1, policy_version 49872 (0.0008) [2023-10-08 09:47:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 102367232. Throughput: 0: 1833.1, 1: 1825.0. Samples: 25600364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:47:17,015][52710] Avg episode reward: [(0, '22.300'), (1, '32.650')] [2023-10-08 09:47:17,284][53885] Updated weights for policy 1, policy_version 49882 (0.0009) [2023-10-08 09:47:20,298][53852] Updated weights for policy 0, policy_version 50120 (0.0007) [2023-10-08 09:47:20,662][53852] Updated weights for policy 0, policy_version 50130 (0.0007) [2023-10-08 09:47:20,954][53885] Updated weights for policy 1, policy_version 49892 (0.0008) [2023-10-08 09:47:21,037][53852] Updated weights for policy 0, policy_version 50140 (0.0008) [2023-10-08 09:47:21,328][53885] Updated weights for policy 1, policy_version 49902 (0.0008) [2023-10-08 09:47:21,703][53885] Updated weights for policy 1, policy_version 49912 (0.0009) [2023-10-08 09:47:22,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 102465536. Throughput: 0: 1834.2, 1: 1823.1. Samples: 25621206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:47:22,016][52710] Avg episode reward: [(0, '21.050'), (1, '34.120')] [2023-10-08 09:47:24,697][53852] Updated weights for policy 0, policy_version 50150 (0.0008) [2023-10-08 09:47:25,060][53852] Updated weights for policy 0, policy_version 50160 (0.0009) [2023-10-08 09:47:25,437][53852] Updated weights for policy 0, policy_version 50170 (0.0008) [2023-10-08 09:47:25,448][53885] Updated weights for policy 1, policy_version 49922 (0.0009) [2023-10-08 09:47:25,866][53885] Updated weights for policy 1, policy_version 49932 (0.0009) [2023-10-08 09:47:26,233][53885] Updated weights for policy 1, policy_version 49942 (0.0007) [2023-10-08 09:47:26,595][53885] Updated weights for policy 1, policy_version 49952 (0.0008) [2023-10-08 09:47:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 102531072. Throughput: 0: 1830.9, 1: 1828.9. Samples: 25633386. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:47:27,016][52710] Avg episode reward: [(0, '23.770'), (1, '35.360')] [2023-10-08 09:47:28,890][53852] Updated weights for policy 0, policy_version 50180 (0.0008) [2023-10-08 09:47:29,256][53852] Updated weights for policy 0, policy_version 50190 (0.0008) [2023-10-08 09:47:29,628][53852] Updated weights for policy 0, policy_version 50200 (0.0009) [2023-10-08 09:47:30,266][53885] Updated weights for policy 1, policy_version 49962 (0.0008) [2023-10-08 09:47:30,631][53885] Updated weights for policy 1, policy_version 49972 (0.0009) [2023-10-08 09:47:31,007][53885] Updated weights for policy 1, policy_version 49982 (0.0009) [2023-10-08 09:47:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 102596608. Throughput: 0: 1835.6, 1: 1821.6. Samples: 25654216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:47:32,016][52710] Avg episode reward: [(0, '23.030'), (1, '29.840')] [2023-10-08 09:47:33,330][53852] Updated weights for policy 0, policy_version 50210 (0.0009) [2023-10-08 09:47:33,711][53852] Updated weights for policy 0, policy_version 50220 (0.0008) [2023-10-08 09:47:34,082][53852] Updated weights for policy 0, policy_version 50230 (0.0008) [2023-10-08 09:47:34,439][53852] Updated weights for policy 0, policy_version 50240 (0.0007) [2023-10-08 09:47:34,594][53885] Updated weights for policy 1, policy_version 49992 (0.0010) [2023-10-08 09:47:34,959][53885] Updated weights for policy 1, policy_version 50002 (0.0009) [2023-10-08 09:47:35,322][53885] Updated weights for policy 1, policy_version 50012 (0.0010) [2023-10-08 09:47:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 102662144. Throughput: 0: 1840.4, 1: 1832.5. Samples: 25676714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:47:37,016][52710] Avg episode reward: [(0, '22.490'), (1, '32.070')] [2023-10-08 09:47:38,078][53852] Updated weights for policy 0, policy_version 50250 (0.0007) [2023-10-08 09:47:38,448][53852] Updated weights for policy 0, policy_version 50260 (0.0008) [2023-10-08 09:47:38,818][53852] Updated weights for policy 0, policy_version 50270 (0.0009) [2023-10-08 09:47:39,069][53885] Updated weights for policy 1, policy_version 50022 (0.0009) [2023-10-08 09:47:39,438][53885] Updated weights for policy 1, policy_version 50032 (0.0011) [2023-10-08 09:47:39,805][53885] Updated weights for policy 1, policy_version 50042 (0.0010) [2023-10-08 09:47:42,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102727680. Throughput: 0: 1840.1, 1: 1824.9. Samples: 25687312. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 09:47:42,015][52710] Avg episode reward: [(0, '24.950'), (1, '32.890')] [2023-10-08 09:47:42,509][53852] Updated weights for policy 0, policy_version 50280 (0.0010) [2023-10-08 09:47:42,883][53852] Updated weights for policy 0, policy_version 50290 (0.0010) [2023-10-08 09:47:43,258][53852] Updated weights for policy 0, policy_version 50300 (0.0007) [2023-10-08 09:47:43,535][53885] Updated weights for policy 1, policy_version 50052 (0.0009) [2023-10-08 09:47:43,909][53885] Updated weights for policy 1, policy_version 50062 (0.0009) [2023-10-08 09:47:44,270][53885] Updated weights for policy 1, policy_version 50072 (0.0009) [2023-10-08 09:47:46,794][53852] Updated weights for policy 0, policy_version 50310 (0.0009) [2023-10-08 09:47:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102793216. Throughput: 0: 1838.7, 1: 1827.9. Samples: 25709562. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 09:47:47,016][52710] Avg episode reward: [(0, '23.020'), (1, '31.560')] [2023-10-08 09:47:47,160][53852] Updated weights for policy 0, policy_version 50320 (0.0008) [2023-10-08 09:47:47,526][53852] Updated weights for policy 0, policy_version 50330 (0.0008) [2023-10-08 09:47:47,884][53885] Updated weights for policy 1, policy_version 50082 (0.0008) [2023-10-08 09:47:48,260][53885] Updated weights for policy 1, policy_version 50092 (0.0010) [2023-10-08 09:47:48,622][53885] Updated weights for policy 1, policy_version 50102 (0.0010) [2023-10-08 09:47:48,999][53885] Updated weights for policy 1, policy_version 50112 (0.0008) [2023-10-08 09:47:51,143][53852] Updated weights for policy 0, policy_version 50340 (0.0010) [2023-10-08 09:47:51,511][53852] Updated weights for policy 0, policy_version 50350 (0.0010) [2023-10-08 09:47:51,888][53852] Updated weights for policy 0, policy_version 50360 (0.0009) [2023-10-08 09:47:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 102858752. Throughput: 0: 1827.3, 1: 1823.8. Samples: 25731726. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 09:47:52,015][52710] Avg episode reward: [(0, '22.370'), (1, '32.640')] [2023-10-08 09:47:52,791][53885] Updated weights for policy 1, policy_version 50122 (0.0009) [2023-10-08 09:47:53,158][53885] Updated weights for policy 1, policy_version 50132 (0.0007) [2023-10-08 09:47:53,524][53885] Updated weights for policy 1, policy_version 50142 (0.0009) [2023-10-08 09:47:55,555][53852] Updated weights for policy 0, policy_version 50370 (0.0009) [2023-10-08 09:47:55,921][53852] Updated weights for policy 0, policy_version 50380 (0.0007) [2023-10-08 09:47:56,296][53852] Updated weights for policy 0, policy_version 50390 (0.0009) [2023-10-08 09:47:56,664][53852] Updated weights for policy 0, policy_version 50400 (0.0009) [2023-10-08 09:47:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 102957056. Throughput: 0: 1839.5, 1: 1823.3. Samples: 25742402. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 09:47:57,015][52710] Avg episode reward: [(0, '23.700'), (1, '33.620')] [2023-10-08 09:47:57,222][53885] Updated weights for policy 1, policy_version 50152 (0.0010) [2023-10-08 09:47:57,591][53885] Updated weights for policy 1, policy_version 50162 (0.0008) [2023-10-08 09:47:57,959][53885] Updated weights for policy 1, policy_version 50172 (0.0007) [2023-10-08 09:48:00,416][53852] Updated weights for policy 0, policy_version 50410 (0.0009) [2023-10-08 09:48:00,798][53852] Updated weights for policy 0, policy_version 50420 (0.0008) [2023-10-08 09:48:01,159][53852] Updated weights for policy 0, policy_version 50430 (0.0009) [2023-10-08 09:48:01,641][53885] Updated weights for policy 1, policy_version 50182 (0.0008) [2023-10-08 09:48:02,004][53885] Updated weights for policy 1, policy_version 50192 (0.0008) [2023-10-08 09:48:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103022592. Throughput: 0: 1829.9, 1: 1823.0. Samples: 25764746. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 09:48:02,016][52710] Avg episode reward: [(0, '22.780'), (1, '31.000')] [2023-10-08 09:48:02,371][53885] Updated weights for policy 1, policy_version 50202 (0.0007) [2023-10-08 09:48:04,734][53852] Updated weights for policy 0, policy_version 50440 (0.0008) [2023-10-08 09:48:05,106][53852] Updated weights for policy 0, policy_version 50450 (0.0009) [2023-10-08 09:48:05,467][53852] Updated weights for policy 0, policy_version 50460 (0.0007) [2023-10-08 09:48:05,935][53885] Updated weights for policy 1, policy_version 50212 (0.0009) [2023-10-08 09:48:06,297][53885] Updated weights for policy 1, policy_version 50222 (0.0009) [2023-10-08 09:48:06,666][53885] Updated weights for policy 1, policy_version 50232 (0.0008) [2023-10-08 09:48:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 103120896. Throughput: 0: 1839.1, 1: 1822.5. Samples: 25785978. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 09:48:07,016][52710] Avg episode reward: [(0, '24.380'), (1, '30.770')] [2023-10-08 09:48:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000050240_51445760.pth... [2023-10-08 09:48:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000050464_51675136.pth... [2023-10-08 09:48:07,063][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000048768_49938432.pth [2023-10-08 09:48:07,065][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000048544_49709056.pth [2023-10-08 09:48:09,242][53852] Updated weights for policy 0, policy_version 50470 (0.0007) [2023-10-08 09:48:09,616][53852] Updated weights for policy 0, policy_version 50480 (0.0007) [2023-10-08 09:48:09,980][53852] Updated weights for policy 0, policy_version 50490 (0.0007) [2023-10-08 09:48:10,327][53885] Updated weights for policy 1, policy_version 50242 (0.0011) [2023-10-08 09:48:10,696][53885] Updated weights for policy 1, policy_version 50252 (0.0010) [2023-10-08 09:48:11,066][53885] Updated weights for policy 1, policy_version 50262 (0.0008) [2023-10-08 09:48:11,419][53885] Updated weights for policy 1, policy_version 50272 (0.0007) [2023-10-08 09:48:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 103186432. Throughput: 0: 1827.6, 1: 1822.0. Samples: 25797620. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 09:48:12,016][52710] Avg episode reward: [(0, '25.500'), (1, '33.250')] [2023-10-08 09:48:13,732][53852] Updated weights for policy 0, policy_version 50500 (0.0009) [2023-10-08 09:48:14,113][53852] Updated weights for policy 0, policy_version 50510 (0.0008) [2023-10-08 09:48:14,474][53852] Updated weights for policy 0, policy_version 50520 (0.0007) [2023-10-08 09:48:15,130][53885] Updated weights for policy 1, policy_version 50282 (0.0007) [2023-10-08 09:48:15,499][53885] Updated weights for policy 1, policy_version 50292 (0.0008) [2023-10-08 09:48:15,865][53885] Updated weights for policy 1, policy_version 50302 (0.0008) [2023-10-08 09:48:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103251968. Throughput: 0: 1835.1, 1: 1820.7. Samples: 25818728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:17,015][52710] Avg episode reward: [(0, '21.910'), (1, '30.240')] [2023-10-08 09:48:18,142][53852] Updated weights for policy 0, policy_version 50530 (0.0007) [2023-10-08 09:48:18,517][53852] Updated weights for policy 0, policy_version 50540 (0.0008) [2023-10-08 09:48:18,883][53852] Updated weights for policy 0, policy_version 50550 (0.0009) [2023-10-08 09:48:19,255][53852] Updated weights for policy 0, policy_version 50560 (0.0009) [2023-10-08 09:48:19,361][53885] Updated weights for policy 1, policy_version 50312 (0.0007) [2023-10-08 09:48:19,738][53885] Updated weights for policy 1, policy_version 50322 (0.0007) [2023-10-08 09:48:20,106][53885] Updated weights for policy 1, policy_version 50332 (0.0007) [2023-10-08 09:48:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 103317504. Throughput: 0: 1830.2, 1: 1826.0. Samples: 25841246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:22,016][52710] Avg episode reward: [(0, '22.200'), (1, '30.630')] [2023-10-08 09:48:22,908][53852] Updated weights for policy 0, policy_version 50570 (0.0008) [2023-10-08 09:48:23,286][53852] Updated weights for policy 0, policy_version 50580 (0.0008) [2023-10-08 09:48:23,661][53852] Updated weights for policy 0, policy_version 50590 (0.0009) [2023-10-08 09:48:23,737][53885] Updated weights for policy 1, policy_version 50342 (0.0007) [2023-10-08 09:48:24,109][53885] Updated weights for policy 1, policy_version 50352 (0.0007) [2023-10-08 09:48:24,478][53885] Updated weights for policy 1, policy_version 50362 (0.0009) [2023-10-08 09:48:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103383040. Throughput: 0: 1828.6, 1: 1818.1. Samples: 25851414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:27,016][52710] Avg episode reward: [(0, '22.290'), (1, '31.180')] [2023-10-08 09:48:27,335][53852] Updated weights for policy 0, policy_version 50600 (0.0011) [2023-10-08 09:48:27,707][53852] Updated weights for policy 0, policy_version 50610 (0.0009) [2023-10-08 09:48:28,081][53852] Updated weights for policy 0, policy_version 50620 (0.0008) [2023-10-08 09:48:28,139][53885] Updated weights for policy 1, policy_version 50372 (0.0008) [2023-10-08 09:48:28,508][53885] Updated weights for policy 1, policy_version 50382 (0.0009) [2023-10-08 09:48:28,873][53885] Updated weights for policy 1, policy_version 50392 (0.0008) [2023-10-08 09:48:31,689][53852] Updated weights for policy 0, policy_version 50630 (0.0009) [2023-10-08 09:48:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103448576. Throughput: 0: 1824.0, 1: 1829.8. Samples: 25873986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:32,016][52710] Avg episode reward: [(0, '21.800'), (1, '31.340')] [2023-10-08 09:48:32,051][53852] Updated weights for policy 0, policy_version 50640 (0.0008) [2023-10-08 09:48:32,417][53852] Updated weights for policy 0, policy_version 50650 (0.0011) [2023-10-08 09:48:32,573][53885] Updated weights for policy 1, policy_version 50402 (0.0010) [2023-10-08 09:48:32,944][53885] Updated weights for policy 1, policy_version 50412 (0.0009) [2023-10-08 09:48:33,311][53885] Updated weights for policy 1, policy_version 50422 (0.0007) [2023-10-08 09:48:33,681][53885] Updated weights for policy 1, policy_version 50432 (0.0009) [2023-10-08 09:48:36,139][53852] Updated weights for policy 0, policy_version 50660 (0.0007) [2023-10-08 09:48:36,507][53852] Updated weights for policy 0, policy_version 50670 (0.0008) [2023-10-08 09:48:36,884][53852] Updated weights for policy 0, policy_version 50680 (0.0007) [2023-10-08 09:48:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103514112. Throughput: 0: 1823.2, 1: 1832.0. Samples: 25896210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:37,016][52710] Avg episode reward: [(0, '22.560'), (1, '29.790')] [2023-10-08 09:48:37,419][53885] Updated weights for policy 1, policy_version 50442 (0.0008) [2023-10-08 09:48:37,786][53885] Updated weights for policy 1, policy_version 50452 (0.0007) [2023-10-08 09:48:38,151][53885] Updated weights for policy 1, policy_version 50462 (0.0007) [2023-10-08 09:48:40,562][53852] Updated weights for policy 0, policy_version 50690 (0.0008) [2023-10-08 09:48:40,924][53852] Updated weights for policy 0, policy_version 50700 (0.0009) [2023-10-08 09:48:41,284][53852] Updated weights for policy 0, policy_version 50710 (0.0007) [2023-10-08 09:48:41,653][53852] Updated weights for policy 0, policy_version 50720 (0.0008) [2023-10-08 09:48:41,843][53885] Updated weights for policy 1, policy_version 50472 (0.0008) [2023-10-08 09:48:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 103612416. Throughput: 0: 1826.1, 1: 1829.0. Samples: 25906882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:42,016][52710] Avg episode reward: [(0, '21.630'), (1, '33.320')] [2023-10-08 09:48:42,209][53885] Updated weights for policy 1, policy_version 50482 (0.0008) [2023-10-08 09:48:42,575][53885] Updated weights for policy 1, policy_version 50492 (0.0007) [2023-10-08 09:48:45,440][53852] Updated weights for policy 0, policy_version 50730 (0.0009) [2023-10-08 09:48:45,820][53852] Updated weights for policy 0, policy_version 50740 (0.0007) [2023-10-08 09:48:46,143][53885] Updated weights for policy 1, policy_version 50502 (0.0008) [2023-10-08 09:48:46,192][53852] Updated weights for policy 0, policy_version 50750 (0.0007) [2023-10-08 09:48:46,513][53885] Updated weights for policy 1, policy_version 50512 (0.0010) [2023-10-08 09:48:46,886][53885] Updated weights for policy 1, policy_version 50522 (0.0009) [2023-10-08 09:48:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103677952. Throughput: 0: 1821.8, 1: 1834.4. Samples: 25929274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:47,016][52710] Avg episode reward: [(0, '22.840'), (1, '28.780')] [2023-10-08 09:48:49,767][53852] Updated weights for policy 0, policy_version 50760 (0.0007) [2023-10-08 09:48:50,141][53852] Updated weights for policy 0, policy_version 50770 (0.0009) [2023-10-08 09:48:50,509][53852] Updated weights for policy 0, policy_version 50780 (0.0009) [2023-10-08 09:48:50,556][53885] Updated weights for policy 1, policy_version 50532 (0.0008) [2023-10-08 09:48:50,919][53885] Updated weights for policy 1, policy_version 50542 (0.0011) [2023-10-08 09:48:51,291][53885] Updated weights for policy 1, policy_version 50552 (0.0010) [2023-10-08 09:48:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 103776256. Throughput: 0: 1818.7, 1: 1823.7. Samples: 25949886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:52,016][52710] Avg episode reward: [(0, '24.470'), (1, '31.900')] [2023-10-08 09:48:54,190][53852] Updated weights for policy 0, policy_version 50790 (0.0009) [2023-10-08 09:48:54,556][53852] Updated weights for policy 0, policy_version 50800 (0.0009) [2023-10-08 09:48:54,934][53852] Updated weights for policy 0, policy_version 50810 (0.0009) [2023-10-08 09:48:54,952][53885] Updated weights for policy 1, policy_version 50562 (0.0010) [2023-10-08 09:48:55,325][53885] Updated weights for policy 1, policy_version 50572 (0.0009) [2023-10-08 09:48:55,695][53885] Updated weights for policy 1, policy_version 50582 (0.0010) [2023-10-08 09:48:56,067][53885] Updated weights for policy 1, policy_version 50592 (0.0009) [2023-10-08 09:48:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103841792. Throughput: 0: 1818.1, 1: 1841.0. Samples: 25962280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:48:57,015][52710] Avg episode reward: [(0, '23.380'), (1, '33.470')] [2023-10-08 09:48:58,755][53852] Updated weights for policy 0, policy_version 50820 (0.0008) [2023-10-08 09:48:59,131][53852] Updated weights for policy 0, policy_version 50830 (0.0009) [2023-10-08 09:48:59,503][53852] Updated weights for policy 0, policy_version 50840 (0.0009) [2023-10-08 09:48:59,943][53885] Updated weights for policy 1, policy_version 50602 (0.0009) [2023-10-08 09:49:00,321][53885] Updated weights for policy 1, policy_version 50612 (0.0008) [2023-10-08 09:49:00,684][53885] Updated weights for policy 1, policy_version 50622 (0.0008) [2023-10-08 09:49:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103907328. Throughput: 0: 1819.0, 1: 1828.4. Samples: 25982864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:02,016][52710] Avg episode reward: [(0, '24.470'), (1, '30.750')] [2023-10-08 09:49:03,049][53852] Updated weights for policy 0, policy_version 50850 (0.0008) [2023-10-08 09:49:03,413][53852] Updated weights for policy 0, policy_version 50860 (0.0009) [2023-10-08 09:49:03,786][53852] Updated weights for policy 0, policy_version 50870 (0.0009) [2023-10-08 09:49:04,155][53852] Updated weights for policy 0, policy_version 50880 (0.0009) [2023-10-08 09:49:04,180][53885] Updated weights for policy 1, policy_version 50632 (0.0009) [2023-10-08 09:49:04,557][53885] Updated weights for policy 1, policy_version 50642 (0.0010) [2023-10-08 09:49:04,918][53885] Updated weights for policy 1, policy_version 50652 (0.0011) [2023-10-08 09:49:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 103972864. Throughput: 0: 1824.0, 1: 1834.5. Samples: 26005876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:07,016][52710] Avg episode reward: [(0, '24.180'), (1, '32.640')] [2023-10-08 09:49:07,727][53852] Updated weights for policy 0, policy_version 50890 (0.0008) [2023-10-08 09:49:08,099][53852] Updated weights for policy 0, policy_version 50900 (0.0007) [2023-10-08 09:49:08,470][53852] Updated weights for policy 0, policy_version 50910 (0.0007) [2023-10-08 09:49:08,623][53885] Updated weights for policy 1, policy_version 50662 (0.0010) [2023-10-08 09:49:08,994][53885] Updated weights for policy 1, policy_version 50672 (0.0011) [2023-10-08 09:49:09,366][53885] Updated weights for policy 1, policy_version 50682 (0.0011) [2023-10-08 09:49:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104038400. Throughput: 0: 1828.8, 1: 1829.2. Samples: 26016022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:12,016][52710] Avg episode reward: [(0, '24.660'), (1, '32.170')] [2023-10-08 09:49:12,147][53852] Updated weights for policy 0, policy_version 50920 (0.0007) [2023-10-08 09:49:12,515][53852] Updated weights for policy 0, policy_version 50930 (0.0007) [2023-10-08 09:49:12,875][53852] Updated weights for policy 0, policy_version 50940 (0.0007) [2023-10-08 09:49:13,039][53885] Updated weights for policy 1, policy_version 50692 (0.0009) [2023-10-08 09:49:13,414][53885] Updated weights for policy 1, policy_version 50702 (0.0007) [2023-10-08 09:49:13,771][53885] Updated weights for policy 1, policy_version 50712 (0.0007) [2023-10-08 09:49:16,525][53852] Updated weights for policy 0, policy_version 50950 (0.0008) [2023-10-08 09:49:16,896][53852] Updated weights for policy 0, policy_version 50960 (0.0008) [2023-10-08 09:49:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 104103936. Throughput: 0: 1830.3, 1: 1835.5. Samples: 26038946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:17,016][52710] Avg episode reward: [(0, '26.050'), (1, '33.050')] [2023-10-08 09:49:17,267][53852] Updated weights for policy 0, policy_version 50970 (0.0007) [2023-10-08 09:49:17,356][53885] Updated weights for policy 1, policy_version 50722 (0.0008) [2023-10-08 09:49:17,720][53885] Updated weights for policy 1, policy_version 50732 (0.0009) [2023-10-08 09:49:18,083][53885] Updated weights for policy 1, policy_version 50742 (0.0007) [2023-10-08 09:49:18,443][53885] Updated weights for policy 1, policy_version 50752 (0.0007) [2023-10-08 09:49:20,795][53852] Updated weights for policy 0, policy_version 50980 (0.0008) [2023-10-08 09:49:21,170][53852] Updated weights for policy 0, policy_version 50990 (0.0007) [2023-10-08 09:49:21,539][53852] Updated weights for policy 0, policy_version 51000 (0.0007) [2023-10-08 09:49:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104202240. Throughput: 0: 1826.1, 1: 1838.3. Samples: 26061106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:22,016][52710] Avg episode reward: [(0, '26.710'), (1, '32.900')] [2023-10-08 09:49:22,164][53885] Updated weights for policy 1, policy_version 50762 (0.0007) [2023-10-08 09:49:22,537][53885] Updated weights for policy 1, policy_version 50772 (0.0009) [2023-10-08 09:49:22,903][53885] Updated weights for policy 1, policy_version 50782 (0.0009) [2023-10-08 09:49:25,182][53852] Updated weights for policy 0, policy_version 51010 (0.0008) [2023-10-08 09:49:25,558][53852] Updated weights for policy 0, policy_version 51020 (0.0010) [2023-10-08 09:49:25,918][53852] Updated weights for policy 0, policy_version 51030 (0.0011) [2023-10-08 09:49:26,288][53852] Updated weights for policy 0, policy_version 51040 (0.0007) [2023-10-08 09:49:26,594][53885] Updated weights for policy 1, policy_version 50792 (0.0009) [2023-10-08 09:49:26,962][53885] Updated weights for policy 1, policy_version 50802 (0.0010) [2023-10-08 09:49:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104267776. Throughput: 0: 1835.0, 1: 1837.8. Samples: 26072160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:27,016][52710] Avg episode reward: [(0, '25.000'), (1, '34.010')] [2023-10-08 09:49:27,337][53885] Updated weights for policy 1, policy_version 50812 (0.0010) [2023-10-08 09:49:30,009][53852] Updated weights for policy 0, policy_version 51050 (0.0007) [2023-10-08 09:49:30,381][53852] Updated weights for policy 0, policy_version 51060 (0.0009) [2023-10-08 09:49:30,747][53852] Updated weights for policy 0, policy_version 51070 (0.0011) [2023-10-08 09:49:31,046][53885] Updated weights for policy 1, policy_version 50822 (0.0008) [2023-10-08 09:49:31,420][53885] Updated weights for policy 1, policy_version 50832 (0.0007) [2023-10-08 09:49:31,789][53885] Updated weights for policy 1, policy_version 50842 (0.0008) [2023-10-08 09:49:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 104366080. Throughput: 0: 1829.6, 1: 1835.2. Samples: 26094192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:32,016][52710] Avg episode reward: [(0, '22.490'), (1, '33.450')] [2023-10-08 09:49:34,355][53852] Updated weights for policy 0, policy_version 51080 (0.0008) [2023-10-08 09:49:34,726][53852] Updated weights for policy 0, policy_version 51090 (0.0007) [2023-10-08 09:49:35,102][53852] Updated weights for policy 0, policy_version 51100 (0.0007) [2023-10-08 09:49:35,275][53885] Updated weights for policy 1, policy_version 50852 (0.0008) [2023-10-08 09:49:35,639][53885] Updated weights for policy 1, policy_version 50862 (0.0009) [2023-10-08 09:49:36,005][53885] Updated weights for policy 1, policy_version 50872 (0.0009) [2023-10-08 09:49:37,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 104431616. Throughput: 0: 1848.3, 1: 1837.2. Samples: 26115734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:37,016][52710] Avg episode reward: [(0, '19.750'), (1, '31.600')] [2023-10-08 09:49:38,698][53852] Updated weights for policy 0, policy_version 51110 (0.0009) [2023-10-08 09:49:39,075][53852] Updated weights for policy 0, policy_version 51120 (0.0011) [2023-10-08 09:49:39,447][53852] Updated weights for policy 0, policy_version 51130 (0.0010) [2023-10-08 09:49:39,561][53885] Updated weights for policy 1, policy_version 50882 (0.0009) [2023-10-08 09:49:39,938][53885] Updated weights for policy 1, policy_version 50892 (0.0007) [2023-10-08 09:49:40,303][53885] Updated weights for policy 1, policy_version 50902 (0.0009) [2023-10-08 09:49:40,669][53885] Updated weights for policy 1, policy_version 50912 (0.0008) [2023-10-08 09:49:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 104497152. Throughput: 0: 1833.5, 1: 1836.4. Samples: 26127426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:42,016][52710] Avg episode reward: [(0, '18.150'), (1, '29.510')] [2023-10-08 09:49:43,120][53852] Updated weights for policy 0, policy_version 51140 (0.0010) [2023-10-08 09:49:43,491][53852] Updated weights for policy 0, policy_version 51150 (0.0008) [2023-10-08 09:49:43,861][53852] Updated weights for policy 0, policy_version 51160 (0.0008) [2023-10-08 09:49:44,381][53885] Updated weights for policy 1, policy_version 50922 (0.0008) [2023-10-08 09:49:44,743][53885] Updated weights for policy 1, policy_version 50932 (0.0007) [2023-10-08 09:49:45,112][53885] Updated weights for policy 1, policy_version 50942 (0.0007) [2023-10-08 09:49:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 104562688. Throughput: 0: 1854.5, 1: 1839.5. Samples: 26149094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:47,016][52710] Avg episode reward: [(0, '18.890'), (1, '36.280')] [2023-10-08 09:49:47,018][53594] Saving new best policy, reward=36.280! [2023-10-08 09:49:47,412][53852] Updated weights for policy 0, policy_version 51170 (0.0007) [2023-10-08 09:49:47,790][53852] Updated weights for policy 0, policy_version 51180 (0.0010) [2023-10-08 09:49:48,151][53852] Updated weights for policy 0, policy_version 51190 (0.0007) [2023-10-08 09:49:48,535][53852] Updated weights for policy 0, policy_version 51200 (0.0009) [2023-10-08 09:49:48,917][53885] Updated weights for policy 1, policy_version 50952 (0.0009) [2023-10-08 09:49:49,288][53885] Updated weights for policy 1, policy_version 50962 (0.0008) [2023-10-08 09:49:49,642][53885] Updated weights for policy 1, policy_version 50972 (0.0008) [2023-10-08 09:49:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 104628224. Throughput: 0: 1850.7, 1: 1834.8. Samples: 26171726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:52,015][52710] Avg episode reward: [(0, '22.220'), (1, '30.390')] [2023-10-08 09:49:52,147][53852] Updated weights for policy 0, policy_version 51210 (0.0008) [2023-10-08 09:49:52,523][53852] Updated weights for policy 0, policy_version 51220 (0.0008) [2023-10-08 09:49:52,885][53852] Updated weights for policy 0, policy_version 51230 (0.0007) [2023-10-08 09:49:53,372][53885] Updated weights for policy 1, policy_version 50982 (0.0007) [2023-10-08 09:49:53,732][53885] Updated weights for policy 1, policy_version 50992 (0.0009) [2023-10-08 09:49:54,096][53885] Updated weights for policy 1, policy_version 51002 (0.0009) [2023-10-08 09:49:56,579][53852] Updated weights for policy 0, policy_version 51240 (0.0008) [2023-10-08 09:49:56,945][53852] Updated weights for policy 0, policy_version 51250 (0.0007) [2023-10-08 09:49:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 104693760. Throughput: 0: 1849.6, 1: 1832.5. Samples: 26181718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:49:57,016][52710] Avg episode reward: [(0, '23.380'), (1, '30.550')] [2023-10-08 09:49:57,313][53852] Updated weights for policy 0, policy_version 51260 (0.0007) [2023-10-08 09:49:57,806][53885] Updated weights for policy 1, policy_version 51012 (0.0009) [2023-10-08 09:49:58,166][53885] Updated weights for policy 1, policy_version 51022 (0.0008) [2023-10-08 09:49:58,536][53885] Updated weights for policy 1, policy_version 51032 (0.0007) [2023-10-08 09:50:01,044][53852] Updated weights for policy 0, policy_version 51270 (0.0008) [2023-10-08 09:50:01,424][53852] Updated weights for policy 0, policy_version 51280 (0.0007) [2023-10-08 09:50:01,782][53852] Updated weights for policy 0, policy_version 51290 (0.0007) [2023-10-08 09:50:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104792064. Throughput: 0: 1850.9, 1: 1831.5. Samples: 26204656. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-08 09:50:02,015][52710] Avg episode reward: [(0, '21.890'), (1, '31.960')] [2023-10-08 09:50:02,235][53885] Updated weights for policy 1, policy_version 51042 (0.0009) [2023-10-08 09:50:02,603][53885] Updated weights for policy 1, policy_version 51052 (0.0007) [2023-10-08 09:50:02,969][53885] Updated weights for policy 1, policy_version 51062 (0.0010) [2023-10-08 09:50:03,330][53885] Updated weights for policy 1, policy_version 51072 (0.0009) [2023-10-08 09:50:05,523][53852] Updated weights for policy 0, policy_version 51300 (0.0011) [2023-10-08 09:50:05,885][53852] Updated weights for policy 0, policy_version 51310 (0.0011) [2023-10-08 09:50:06,259][53852] Updated weights for policy 0, policy_version 51320 (0.0010) [2023-10-08 09:50:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104857600. Throughput: 0: 1836.0, 1: 1828.2. Samples: 26225994. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-08 09:50:07,016][52710] Avg episode reward: [(0, '24.670'), (1, '31.170')] [2023-10-08 09:50:07,017][53885] Updated weights for policy 1, policy_version 51082 (0.0008) [2023-10-08 09:50:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000051328_52559872.pth... [2023-10-08 09:50:07,058][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000049600_50790400.pth [2023-10-08 09:50:07,064][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000051328_52559872.pth [2023-10-08 09:50:07,381][53885] Updated weights for policy 1, policy_version 51092 (0.0007) [2023-10-08 09:50:07,746][53885] Updated weights for policy 1, policy_version 51102 (0.0008) [2023-10-08 09:50:07,817][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000051104_52330496.pth... [2023-10-08 09:50:07,855][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000049376_50561024.pth [2023-10-08 09:50:07,859][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000051104_52330496.pth [2023-10-08 09:50:09,900][53852] Updated weights for policy 0, policy_version 51330 (0.0009) [2023-10-08 09:50:10,273][53852] Updated weights for policy 0, policy_version 51340 (0.0008) [2023-10-08 09:50:10,640][53852] Updated weights for policy 0, policy_version 51350 (0.0008) [2023-10-08 09:50:11,003][53852] Updated weights for policy 0, policy_version 51360 (0.0008) [2023-10-08 09:50:11,334][53885] Updated weights for policy 1, policy_version 51112 (0.0010) [2023-10-08 09:50:11,697][53885] Updated weights for policy 1, policy_version 51122 (0.0008) [2023-10-08 09:50:12,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104923136. Throughput: 0: 1843.6, 1: 1828.2. Samples: 26237388. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-08 09:50:12,016][52710] Avg episode reward: [(0, '24.620'), (1, '31.180')] [2023-10-08 09:50:12,068][53885] Updated weights for policy 1, policy_version 51132 (0.0008) [2023-10-08 09:50:14,524][53852] Updated weights for policy 0, policy_version 51370 (0.0007) [2023-10-08 09:50:14,891][53852] Updated weights for policy 0, policy_version 51380 (0.0007) [2023-10-08 09:50:15,261][53852] Updated weights for policy 0, policy_version 51390 (0.0007) [2023-10-08 09:50:15,933][53885] Updated weights for policy 1, policy_version 51142 (0.0009) [2023-10-08 09:50:16,296][53885] Updated weights for policy 1, policy_version 51152 (0.0008) [2023-10-08 09:50:16,667][53885] Updated weights for policy 1, policy_version 51162 (0.0011) [2023-10-08 09:50:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 105021440. Throughput: 0: 1834.9, 1: 1825.1. Samples: 26258888. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-08 09:50:17,015][52710] Avg episode reward: [(0, '23.610'), (1, '34.750')] [2023-10-08 09:50:18,834][53852] Updated weights for policy 0, policy_version 51400 (0.0010) [2023-10-08 09:50:19,214][53852] Updated weights for policy 0, policy_version 51410 (0.0010) [2023-10-08 09:50:19,577][53852] Updated weights for policy 0, policy_version 51420 (0.0008) [2023-10-08 09:50:20,318][53885] Updated weights for policy 1, policy_version 51172 (0.0010) [2023-10-08 09:50:20,688][53885] Updated weights for policy 1, policy_version 51182 (0.0008) [2023-10-08 09:50:21,055][53885] Updated weights for policy 1, policy_version 51192 (0.0007) [2023-10-08 09:50:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105086976. Throughput: 0: 1838.3, 1: 1820.0. Samples: 26280362. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-08 09:50:22,016][52710] Avg episode reward: [(0, '24.270'), (1, '33.530')] [2023-10-08 09:50:23,337][53852] Updated weights for policy 0, policy_version 51430 (0.0008) [2023-10-08 09:50:23,701][53852] Updated weights for policy 0, policy_version 51440 (0.0008) [2023-10-08 09:50:24,066][53852] Updated weights for policy 0, policy_version 51450 (0.0011) [2023-10-08 09:50:24,806][53885] Updated weights for policy 1, policy_version 51202 (0.0007) [2023-10-08 09:50:25,181][53885] Updated weights for policy 1, policy_version 51212 (0.0008) [2023-10-08 09:50:25,552][53885] Updated weights for policy 1, policy_version 51222 (0.0009) [2023-10-08 09:50:25,917][53885] Updated weights for policy 1, policy_version 51232 (0.0007) [2023-10-08 09:50:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105152512. Throughput: 0: 1827.8, 1: 1820.8. Samples: 26291614. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-08 09:50:27,016][52710] Avg episode reward: [(0, '24.390'), (1, '33.270')] [2023-10-08 09:50:27,980][53852] Updated weights for policy 0, policy_version 51460 (0.0010) [2023-10-08 09:50:28,359][53852] Updated weights for policy 0, policy_version 51470 (0.0008) [2023-10-08 09:50:28,739][53852] Updated weights for policy 0, policy_version 51480 (0.0007) [2023-10-08 09:50:29,641][53885] Updated weights for policy 1, policy_version 51242 (0.0012) [2023-10-08 09:50:30,029][53885] Updated weights for policy 1, policy_version 51252 (0.0010) [2023-10-08 09:50:30,390][53885] Updated weights for policy 1, policy_version 51262 (0.0009) [2023-10-08 09:50:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105218048. Throughput: 0: 1823.5, 1: 1815.1. Samples: 26312830. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-08 09:50:32,015][52710] Avg episode reward: [(0, '24.310'), (1, '31.260')] [2023-10-08 09:50:32,357][53852] Updated weights for policy 0, policy_version 51490 (0.0008) [2023-10-08 09:50:32,727][53852] Updated weights for policy 0, policy_version 51500 (0.0007) [2023-10-08 09:50:33,103][53852] Updated weights for policy 0, policy_version 51510 (0.0011) [2023-10-08 09:50:33,481][53852] Updated weights for policy 0, policy_version 51520 (0.0010) [2023-10-08 09:50:34,139][53885] Updated weights for policy 1, policy_version 51272 (0.0009) [2023-10-08 09:50:34,519][53885] Updated weights for policy 1, policy_version 51282 (0.0012) [2023-10-08 09:50:34,883][53885] Updated weights for policy 1, policy_version 51292 (0.0009) [2023-10-08 09:50:36,996][53852] Updated weights for policy 0, policy_version 51530 (0.0010) [2023-10-08 09:50:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105283584. Throughput: 0: 1828.0, 1: 1819.7. Samples: 26335874. Policy #0 lag: (min: 31.0, avg: 32.1, max: 54.0) [2023-10-08 09:50:37,016][52710] Avg episode reward: [(0, '23.960'), (1, '32.170')] [2023-10-08 09:50:37,365][53852] Updated weights for policy 0, policy_version 51540 (0.0010) [2023-10-08 09:50:37,733][53852] Updated weights for policy 0, policy_version 51550 (0.0011) [2023-10-08 09:50:38,482][53885] Updated weights for policy 1, policy_version 51302 (0.0010) [2023-10-08 09:50:38,849][53885] Updated weights for policy 1, policy_version 51312 (0.0009) [2023-10-08 09:50:39,220][53885] Updated weights for policy 1, policy_version 51322 (0.0007) [2023-10-08 09:50:41,285][53852] Updated weights for policy 0, policy_version 51560 (0.0008) [2023-10-08 09:50:41,651][53852] Updated weights for policy 0, policy_version 51570 (0.0009) [2023-10-08 09:50:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105349120. Throughput: 0: 1828.4, 1: 1823.3. Samples: 26346044. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:50:42,016][52710] Avg episode reward: [(0, '26.640'), (1, '33.590')] [2023-10-08 09:50:42,019][53852] Updated weights for policy 0, policy_version 51580 (0.0008) [2023-10-08 09:50:42,899][53885] Updated weights for policy 1, policy_version 51332 (0.0008) [2023-10-08 09:50:43,271][53885] Updated weights for policy 1, policy_version 51342 (0.0011) [2023-10-08 09:50:43,640][53885] Updated weights for policy 1, policy_version 51352 (0.0008) [2023-10-08 09:50:45,767][53852] Updated weights for policy 0, policy_version 51590 (0.0008) [2023-10-08 09:50:46,133][53852] Updated weights for policy 0, policy_version 51600 (0.0009) [2023-10-08 09:50:46,505][53852] Updated weights for policy 0, policy_version 51610 (0.0008) [2023-10-08 09:50:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 105447424. Throughput: 0: 1824.0, 1: 1824.4. Samples: 26368838. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:50:47,016][52710] Avg episode reward: [(0, '24.440'), (1, '31.230')] [2023-10-08 09:50:47,160][53885] Updated weights for policy 1, policy_version 51362 (0.0008) [2023-10-08 09:50:47,529][53885] Updated weights for policy 1, policy_version 51372 (0.0007) [2023-10-08 09:50:47,895][53885] Updated weights for policy 1, policy_version 51382 (0.0008) [2023-10-08 09:50:48,264][53885] Updated weights for policy 1, policy_version 51392 (0.0008) [2023-10-08 09:50:50,091][53852] Updated weights for policy 0, policy_version 51620 (0.0007) [2023-10-08 09:50:50,463][53852] Updated weights for policy 0, policy_version 51630 (0.0009) [2023-10-08 09:50:50,829][53852] Updated weights for policy 0, policy_version 51640 (0.0010) [2023-10-08 09:50:51,894][53885] Updated weights for policy 1, policy_version 51402 (0.0008) [2023-10-08 09:50:52,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 105512960. Throughput: 0: 1828.5, 1: 1828.3. Samples: 26390550. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:50:52,016][52710] Avg episode reward: [(0, '25.780'), (1, '29.560')] [2023-10-08 09:50:52,256][53885] Updated weights for policy 1, policy_version 51412 (0.0009) [2023-10-08 09:50:52,622][53885] Updated weights for policy 1, policy_version 51422 (0.0007) [2023-10-08 09:50:54,464][53852] Updated weights for policy 0, policy_version 51650 (0.0011) [2023-10-08 09:50:54,846][53852] Updated weights for policy 0, policy_version 51660 (0.0012) [2023-10-08 09:50:55,215][53852] Updated weights for policy 0, policy_version 51670 (0.0008) [2023-10-08 09:50:55,583][53852] Updated weights for policy 0, policy_version 51680 (0.0007) [2023-10-08 09:50:56,297][53885] Updated weights for policy 1, policy_version 51432 (0.0007) [2023-10-08 09:50:56,672][53885] Updated weights for policy 1, policy_version 51442 (0.0008) [2023-10-08 09:50:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 105578496. Throughput: 0: 1827.6, 1: 1830.5. Samples: 26401998. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:50:57,016][52710] Avg episode reward: [(0, '24.600'), (1, '30.330')] [2023-10-08 09:50:57,041][53885] Updated weights for policy 1, policy_version 51452 (0.0009) [2023-10-08 09:50:59,216][53852] Updated weights for policy 0, policy_version 51690 (0.0010) [2023-10-08 09:50:59,579][53852] Updated weights for policy 0, policy_version 51700 (0.0010) [2023-10-08 09:50:59,955][53852] Updated weights for policy 0, policy_version 51710 (0.0008) [2023-10-08 09:51:00,796][53885] Updated weights for policy 1, policy_version 51462 (0.0010) [2023-10-08 09:51:01,160][53885] Updated weights for policy 1, policy_version 51472 (0.0009) [2023-10-08 09:51:01,533][53885] Updated weights for policy 1, policy_version 51482 (0.0008) [2023-10-08 09:51:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105676800. Throughput: 0: 1832.0, 1: 1827.9. Samples: 26423586. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:51:02,016][52710] Avg episode reward: [(0, '24.070'), (1, '27.230')] [2023-10-08 09:51:03,530][53852] Updated weights for policy 0, policy_version 51720 (0.0010) [2023-10-08 09:51:03,903][53852] Updated weights for policy 0, policy_version 51730 (0.0009) [2023-10-08 09:51:04,270][53852] Updated weights for policy 0, policy_version 51740 (0.0008) [2023-10-08 09:51:05,207][53885] Updated weights for policy 1, policy_version 51492 (0.0008) [2023-10-08 09:51:05,571][53885] Updated weights for policy 1, policy_version 51502 (0.0008) [2023-10-08 09:51:05,938][53885] Updated weights for policy 1, policy_version 51512 (0.0008) [2023-10-08 09:51:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 105742336. Throughput: 0: 1842.1, 1: 1830.7. Samples: 26445636. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:51:07,016][52710] Avg episode reward: [(0, '25.150'), (1, '30.460')] [2023-10-08 09:51:07,686][53852] Updated weights for policy 0, policy_version 51750 (0.0009) [2023-10-08 09:51:08,047][53852] Updated weights for policy 0, policy_version 51760 (0.0009) [2023-10-08 09:51:08,428][53852] Updated weights for policy 0, policy_version 51770 (0.0008) [2023-10-08 09:51:09,496][53885] Updated weights for policy 1, policy_version 51522 (0.0008) [2023-10-08 09:51:09,866][53885] Updated weights for policy 1, policy_version 51532 (0.0009) [2023-10-08 09:51:10,239][53885] Updated weights for policy 1, policy_version 51542 (0.0009) [2023-10-08 09:51:10,611][53885] Updated weights for policy 1, policy_version 51552 (0.0007) [2023-10-08 09:51:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 105807872. Throughput: 0: 1851.1, 1: 1827.7. Samples: 26457162. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:51:12,015][52710] Avg episode reward: [(0, '23.050'), (1, '32.250')] [2023-10-08 09:51:12,150][53852] Updated weights for policy 0, policy_version 51780 (0.0008) [2023-10-08 09:51:12,534][53852] Updated weights for policy 0, policy_version 51790 (0.0008) [2023-10-08 09:51:12,905][53852] Updated weights for policy 0, policy_version 51800 (0.0009) [2023-10-08 09:51:14,257][53885] Updated weights for policy 1, policy_version 51562 (0.0008) [2023-10-08 09:51:14,630][53885] Updated weights for policy 1, policy_version 51572 (0.0008) [2023-10-08 09:51:15,012][53885] Updated weights for policy 1, policy_version 51582 (0.0009) [2023-10-08 09:51:16,519][53852] Updated weights for policy 0, policy_version 51810 (0.0009) [2023-10-08 09:51:16,898][53852] Updated weights for policy 0, policy_version 51820 (0.0008) [2023-10-08 09:51:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 105873408. Throughput: 0: 1854.0, 1: 1831.4. Samples: 26478674. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 09:51:17,016][52710] Avg episode reward: [(0, '22.180'), (1, '27.350')] [2023-10-08 09:51:17,263][53852] Updated weights for policy 0, policy_version 51830 (0.0009) [2023-10-08 09:51:17,631][53852] Updated weights for policy 0, policy_version 51840 (0.0009) [2023-10-08 09:51:18,842][53885] Updated weights for policy 1, policy_version 51592 (0.0007) [2023-10-08 09:51:19,214][53885] Updated weights for policy 1, policy_version 51602 (0.0007) [2023-10-08 09:51:19,584][53885] Updated weights for policy 1, policy_version 51612 (0.0009) [2023-10-08 09:51:21,300][53852] Updated weights for policy 0, policy_version 51850 (0.0008) [2023-10-08 09:51:21,670][53852] Updated weights for policy 0, policy_version 51860 (0.0009) [2023-10-08 09:51:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 105938944. Throughput: 0: 1832.3, 1: 1833.4. Samples: 26500830. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 09:51:22,015][52710] Avg episode reward: [(0, '20.970'), (1, '32.340')] [2023-10-08 09:51:22,035][53852] Updated weights for policy 0, policy_version 51870 (0.0009) [2023-10-08 09:51:23,184][53885] Updated weights for policy 1, policy_version 51622 (0.0008) [2023-10-08 09:51:23,546][53885] Updated weights for policy 1, policy_version 51632 (0.0008) [2023-10-08 09:51:23,914][53885] Updated weights for policy 1, policy_version 51642 (0.0008) [2023-10-08 09:51:25,678][53852] Updated weights for policy 0, policy_version 51880 (0.0008) [2023-10-08 09:51:26,058][53852] Updated weights for policy 0, policy_version 51890 (0.0008) [2023-10-08 09:51:26,426][53852] Updated weights for policy 0, policy_version 51900 (0.0007) [2023-10-08 09:51:27,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 106037248. Throughput: 0: 1850.2, 1: 1829.6. Samples: 26511636. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 09:51:27,016][52710] Avg episode reward: [(0, '19.420'), (1, '31.630')] [2023-10-08 09:51:27,584][53885] Updated weights for policy 1, policy_version 51652 (0.0009) [2023-10-08 09:51:27,947][53885] Updated weights for policy 1, policy_version 51662 (0.0010) [2023-10-08 09:51:28,307][53885] Updated weights for policy 1, policy_version 51672 (0.0010) [2023-10-08 09:51:30,163][53852] Updated weights for policy 0, policy_version 51910 (0.0008) [2023-10-08 09:51:30,536][53852] Updated weights for policy 0, policy_version 51920 (0.0010) [2023-10-08 09:51:30,905][53852] Updated weights for policy 0, policy_version 51930 (0.0008) [2023-10-08 09:51:31,812][53885] Updated weights for policy 1, policy_version 51682 (0.0010) [2023-10-08 09:51:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106102784. Throughput: 0: 1838.7, 1: 1831.8. Samples: 26534010. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 09:51:32,016][52710] Avg episode reward: [(0, '20.700'), (1, '30.900')] [2023-10-08 09:51:32,179][53885] Updated weights for policy 1, policy_version 51692 (0.0008) [2023-10-08 09:51:32,542][53885] Updated weights for policy 1, policy_version 51702 (0.0008) [2023-10-08 09:51:32,905][53885] Updated weights for policy 1, policy_version 51712 (0.0009) [2023-10-08 09:51:34,648][53852] Updated weights for policy 0, policy_version 51940 (0.0008) [2023-10-08 09:51:35,021][53852] Updated weights for policy 0, policy_version 51950 (0.0007) [2023-10-08 09:51:35,378][53852] Updated weights for policy 0, policy_version 51960 (0.0009) [2023-10-08 09:51:36,402][53885] Updated weights for policy 1, policy_version 51722 (0.0008) [2023-10-08 09:51:36,772][53885] Updated weights for policy 1, policy_version 51732 (0.0010) [2023-10-08 09:51:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 106168320. Throughput: 0: 1852.6, 1: 1812.4. Samples: 26555476. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 09:51:37,016][52710] Avg episode reward: [(0, '21.850'), (1, '30.160')] [2023-10-08 09:51:37,134][53885] Updated weights for policy 1, policy_version 51742 (0.0007) [2023-10-08 09:51:38,865][53852] Updated weights for policy 0, policy_version 51970 (0.0008) [2023-10-08 09:51:39,231][53852] Updated weights for policy 0, policy_version 51980 (0.0008) [2023-10-08 09:51:39,606][53852] Updated weights for policy 0, policy_version 51990 (0.0008) [2023-10-08 09:51:39,971][53852] Updated weights for policy 0, policy_version 52000 (0.0010) [2023-10-08 09:51:40,761][53885] Updated weights for policy 1, policy_version 51752 (0.0008) [2023-10-08 09:51:41,126][53885] Updated weights for policy 1, policy_version 51762 (0.0007) [2023-10-08 09:51:41,485][53885] Updated weights for policy 1, policy_version 51772 (0.0008) [2023-10-08 09:51:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106266624. Throughput: 0: 1834.9, 1: 1830.0. Samples: 26566920. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 09:51:42,016][52710] Avg episode reward: [(0, '23.690'), (1, '32.230')] [2023-10-08 09:51:43,601][53852] Updated weights for policy 0, policy_version 52010 (0.0008) [2023-10-08 09:51:43,967][53852] Updated weights for policy 0, policy_version 52020 (0.0007) [2023-10-08 09:51:44,347][53852] Updated weights for policy 0, policy_version 52030 (0.0009) [2023-10-08 09:51:45,128][53885] Updated weights for policy 1, policy_version 51782 (0.0008) [2023-10-08 09:51:45,497][53885] Updated weights for policy 1, policy_version 51792 (0.0010) [2023-10-08 09:51:45,867][53885] Updated weights for policy 1, policy_version 51802 (0.0010) [2023-10-08 09:51:47,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106332160. Throughput: 0: 1845.1, 1: 1816.6. Samples: 26588362. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) [2023-10-08 09:51:47,015][52710] Avg episode reward: [(0, '21.800'), (1, '32.210')] [2023-10-08 09:51:48,011][53852] Updated weights for policy 0, policy_version 52040 (0.0009) [2023-10-08 09:51:48,381][53852] Updated weights for policy 0, policy_version 52050 (0.0011) [2023-10-08 09:51:48,750][53852] Updated weights for policy 0, policy_version 52060 (0.0008) [2023-10-08 09:51:49,637][53885] Updated weights for policy 1, policy_version 51812 (0.0007) [2023-10-08 09:51:50,003][53885] Updated weights for policy 1, policy_version 51822 (0.0007) [2023-10-08 09:51:50,378][53885] Updated weights for policy 1, policy_version 51832 (0.0008) [2023-10-08 09:51:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 106397696. Throughput: 0: 1833.2, 1: 1833.7. Samples: 26610648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:51:52,016][52710] Avg episode reward: [(0, '21.440'), (1, '31.490')] [2023-10-08 09:51:52,405][53852] Updated weights for policy 0, policy_version 52070 (0.0007) [2023-10-08 09:51:52,781][53852] Updated weights for policy 0, policy_version 52080 (0.0007) [2023-10-08 09:51:53,146][53852] Updated weights for policy 0, policy_version 52090 (0.0008) [2023-10-08 09:51:54,129][53885] Updated weights for policy 1, policy_version 51842 (0.0007) [2023-10-08 09:51:54,498][53885] Updated weights for policy 1, policy_version 51852 (0.0007) [2023-10-08 09:51:54,868][53885] Updated weights for policy 1, policy_version 51862 (0.0009) [2023-10-08 09:51:55,238][53885] Updated weights for policy 1, policy_version 51872 (0.0008) [2023-10-08 09:51:56,861][53852] Updated weights for policy 0, policy_version 52100 (0.0007) [2023-10-08 09:51:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106463232. Throughput: 0: 1830.0, 1: 1818.9. Samples: 26621364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:51:57,016][52710] Avg episode reward: [(0, '22.300'), (1, '32.070')] [2023-10-08 09:51:57,219][53852] Updated weights for policy 0, policy_version 52110 (0.0007) [2023-10-08 09:51:57,599][53852] Updated weights for policy 0, policy_version 52120 (0.0007) [2023-10-08 09:51:59,049][53885] Updated weights for policy 1, policy_version 51882 (0.0009) [2023-10-08 09:51:59,424][53885] Updated weights for policy 1, policy_version 51892 (0.0009) [2023-10-08 09:51:59,792][53885] Updated weights for policy 1, policy_version 51902 (0.0007) [2023-10-08 09:52:01,246][53852] Updated weights for policy 0, policy_version 52130 (0.0009) [2023-10-08 09:52:01,611][53852] Updated weights for policy 0, policy_version 52140 (0.0008) [2023-10-08 09:52:01,994][53852] Updated weights for policy 0, policy_version 52150 (0.0009) [2023-10-08 09:52:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 106528768. Throughput: 0: 1835.2, 1: 1831.3. Samples: 26643666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:52:02,016][52710] Avg episode reward: [(0, '21.810'), (1, '31.210')] [2023-10-08 09:52:02,353][53852] Updated weights for policy 0, policy_version 52160 (0.0008) [2023-10-08 09:52:03,424][53885] Updated weights for policy 1, policy_version 51912 (0.0010) [2023-10-08 09:52:03,787][53885] Updated weights for policy 1, policy_version 51922 (0.0010) [2023-10-08 09:52:04,149][53885] Updated weights for policy 1, policy_version 51932 (0.0009) [2023-10-08 09:52:05,950][53852] Updated weights for policy 0, policy_version 52170 (0.0007) [2023-10-08 09:52:06,320][53852] Updated weights for policy 0, policy_version 52180 (0.0009) [2023-10-08 09:52:06,686][53852] Updated weights for policy 0, policy_version 52190 (0.0010) [2023-10-08 09:52:07,015][52710] Fps is (10 sec: 16383.3, 60 sec: 14745.6, 300 sec: 14773.3). Total num frames: 106627072. Throughput: 0: 1828.9, 1: 1834.6. Samples: 26665690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:52:07,016][52710] Avg episode reward: [(0, '23.750'), (1, '31.470')] [2023-10-08 09:52:07,030][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000051936_53182464.pth... [2023-10-08 09:52:07,030][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000052192_53444608.pth... [2023-10-08 09:52:07,059][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000050240_51445760.pth [2023-10-08 09:52:07,074][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000050464_51675136.pth [2023-10-08 09:52:07,779][53885] Updated weights for policy 1, policy_version 51942 (0.0008) [2023-10-08 09:52:08,142][53885] Updated weights for policy 1, policy_version 51952 (0.0009) [2023-10-08 09:52:08,506][53885] Updated weights for policy 1, policy_version 51962 (0.0007) [2023-10-08 09:52:10,349][53852] Updated weights for policy 0, policy_version 52200 (0.0009) [2023-10-08 09:52:10,717][53852] Updated weights for policy 0, policy_version 52210 (0.0008) [2023-10-08 09:52:11,080][53852] Updated weights for policy 0, policy_version 52220 (0.0009) [2023-10-08 09:52:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106692608. Throughput: 0: 1835.6, 1: 1837.2. Samples: 26676914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:52:12,015][52710] Avg episode reward: [(0, '22.990'), (1, '34.780')] [2023-10-08 09:52:12,219][53885] Updated weights for policy 1, policy_version 51972 (0.0008) [2023-10-08 09:52:12,587][53885] Updated weights for policy 1, policy_version 51982 (0.0008) [2023-10-08 09:52:12,958][53885] Updated weights for policy 1, policy_version 51992 (0.0008) [2023-10-08 09:52:14,649][53852] Updated weights for policy 0, policy_version 52230 (0.0008) [2023-10-08 09:52:15,019][53852] Updated weights for policy 0, policy_version 52240 (0.0008) [2023-10-08 09:52:15,394][53852] Updated weights for policy 0, policy_version 52250 (0.0008) [2023-10-08 09:52:16,499][53885] Updated weights for policy 1, policy_version 52002 (0.0008) [2023-10-08 09:52:16,862][53885] Updated weights for policy 1, policy_version 52012 (0.0009) [2023-10-08 09:52:17,015][52710] Fps is (10 sec: 13107.8, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 106758144. Throughput: 0: 1827.1, 1: 1829.6. Samples: 26698560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:52:17,015][52710] Avg episode reward: [(0, '22.410'), (1, '30.550')] [2023-10-08 09:52:17,222][53885] Updated weights for policy 1, policy_version 52022 (0.0009) [2023-10-08 09:52:17,587][53885] Updated weights for policy 1, policy_version 52032 (0.0011) [2023-10-08 09:52:19,082][53852] Updated weights for policy 0, policy_version 52260 (0.0010) [2023-10-08 09:52:19,467][53852] Updated weights for policy 0, policy_version 52270 (0.0009) [2023-10-08 09:52:19,831][53852] Updated weights for policy 0, policy_version 52280 (0.0008) [2023-10-08 09:52:21,140][53885] Updated weights for policy 1, policy_version 52042 (0.0007) [2023-10-08 09:52:21,511][53885] Updated weights for policy 1, policy_version 52052 (0.0010) [2023-10-08 09:52:21,876][53885] Updated weights for policy 1, policy_version 52062 (0.0010) [2023-10-08 09:52:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106856448. Throughput: 0: 1834.9, 1: 1831.3. Samples: 26720454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:52:22,016][52710] Avg episode reward: [(0, '21.680'), (1, '32.750')] [2023-10-08 09:52:23,382][53852] Updated weights for policy 0, policy_version 52290 (0.0008) [2023-10-08 09:52:23,749][53852] Updated weights for policy 0, policy_version 52300 (0.0010) [2023-10-08 09:52:24,132][53852] Updated weights for policy 0, policy_version 52310 (0.0011) [2023-10-08 09:52:24,502][53852] Updated weights for policy 0, policy_version 52320 (0.0009) [2023-10-08 09:52:25,523][53885] Updated weights for policy 1, policy_version 52072 (0.0010) [2023-10-08 09:52:25,901][53885] Updated weights for policy 1, policy_version 52082 (0.0008) [2023-10-08 09:52:26,260][53885] Updated weights for policy 1, policy_version 52092 (0.0008) [2023-10-08 09:52:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106921984. Throughput: 0: 1824.4, 1: 1836.3. Samples: 26731650. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 09:52:27,016][52710] Avg episode reward: [(0, '19.050'), (1, '35.790')] [2023-10-08 09:52:28,112][53852] Updated weights for policy 0, policy_version 52330 (0.0008) [2023-10-08 09:52:28,488][53852] Updated weights for policy 0, policy_version 52340 (0.0009) [2023-10-08 09:52:28,859][53852] Updated weights for policy 0, policy_version 52350 (0.0009) [2023-10-08 09:52:30,063][53885] Updated weights for policy 1, policy_version 52102 (0.0008) [2023-10-08 09:52:30,423][53885] Updated weights for policy 1, policy_version 52112 (0.0008) [2023-10-08 09:52:30,799][53885] Updated weights for policy 1, policy_version 52122 (0.0008) [2023-10-08 09:52:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106987520. Throughput: 0: 1851.8, 1: 1833.9. Samples: 26754220. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 09:52:32,016][52710] Avg episode reward: [(0, '19.400'), (1, '31.350')] [2023-10-08 09:52:32,345][53852] Updated weights for policy 0, policy_version 52360 (0.0009) [2023-10-08 09:52:32,712][53852] Updated weights for policy 0, policy_version 52370 (0.0008) [2023-10-08 09:52:33,093][53852] Updated weights for policy 0, policy_version 52380 (0.0009) [2023-10-08 09:52:34,475][53885] Updated weights for policy 1, policy_version 52132 (0.0010) [2023-10-08 09:52:34,851][53885] Updated weights for policy 1, policy_version 52142 (0.0008) [2023-10-08 09:52:35,211][53885] Updated weights for policy 1, policy_version 52152 (0.0008) [2023-10-08 09:52:36,729][53852] Updated weights for policy 0, policy_version 52390 (0.0009) [2023-10-08 09:52:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 107053056. Throughput: 0: 1851.3, 1: 1834.6. Samples: 26776514. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 09:52:37,016][52710] Avg episode reward: [(0, '20.500'), (1, '30.530')] [2023-10-08 09:52:37,097][53852] Updated weights for policy 0, policy_version 52400 (0.0008) [2023-10-08 09:52:37,486][53852] Updated weights for policy 0, policy_version 52410 (0.0007) [2023-10-08 09:52:38,883][53885] Updated weights for policy 1, policy_version 52162 (0.0009) [2023-10-08 09:52:39,253][53885] Updated weights for policy 1, policy_version 52172 (0.0008) [2023-10-08 09:52:39,634][53885] Updated weights for policy 1, policy_version 52182 (0.0009) [2023-10-08 09:52:39,999][53885] Updated weights for policy 1, policy_version 52192 (0.0008) [2023-10-08 09:52:41,281][53852] Updated weights for policy 0, policy_version 52420 (0.0008) [2023-10-08 09:52:41,651][53852] Updated weights for policy 0, policy_version 52430 (0.0009) [2023-10-08 09:52:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 107118592. Throughput: 0: 1850.4, 1: 1827.5. Samples: 26786868. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 09:52:42,016][52710] Avg episode reward: [(0, '19.230'), (1, '31.250')] [2023-10-08 09:52:42,022][53852] Updated weights for policy 0, policy_version 52440 (0.0009) [2023-10-08 09:52:43,643][53885] Updated weights for policy 1, policy_version 52202 (0.0009) [2023-10-08 09:52:44,015][53885] Updated weights for policy 1, policy_version 52212 (0.0008) [2023-10-08 09:52:44,388][53885] Updated weights for policy 1, policy_version 52222 (0.0008) [2023-10-08 09:52:45,725][53852] Updated weights for policy 0, policy_version 52450 (0.0009) [2023-10-08 09:52:46,138][53852] Updated weights for policy 0, policy_version 52460 (0.0009) [2023-10-08 09:52:46,508][53852] Updated weights for policy 0, policy_version 52470 (0.0008) [2023-10-08 09:52:46,875][53852] Updated weights for policy 0, policy_version 52480 (0.0008) [2023-10-08 09:52:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 107216896. Throughput: 0: 1845.2, 1: 1833.0. Samples: 26809184. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 09:52:47,015][52710] Avg episode reward: [(0, '21.010'), (1, '30.800')] [2023-10-08 09:52:48,211][53885] Updated weights for policy 1, policy_version 52232 (0.0008) [2023-10-08 09:52:48,581][53885] Updated weights for policy 1, policy_version 52242 (0.0010) [2023-10-08 09:52:48,948][53885] Updated weights for policy 1, policy_version 52252 (0.0010) [2023-10-08 09:52:50,563][53852] Updated weights for policy 0, policy_version 52490 (0.0008) [2023-10-08 09:52:50,933][53852] Updated weights for policy 0, policy_version 52500 (0.0009) [2023-10-08 09:52:51,306][53852] Updated weights for policy 0, policy_version 52510 (0.0008) [2023-10-08 09:52:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107282432. Throughput: 0: 1830.1, 1: 1833.0. Samples: 26830528. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 09:52:52,016][52710] Avg episode reward: [(0, '18.430'), (1, '31.240')] [2023-10-08 09:52:52,625][53885] Updated weights for policy 1, policy_version 52262 (0.0010) [2023-10-08 09:52:53,009][53885] Updated weights for policy 1, policy_version 52272 (0.0008) [2023-10-08 09:52:53,368][53885] Updated weights for policy 1, policy_version 52282 (0.0011) [2023-10-08 09:52:54,969][53852] Updated weights for policy 0, policy_version 52520 (0.0007) [2023-10-08 09:52:55,336][53852] Updated weights for policy 0, policy_version 52530 (0.0008) [2023-10-08 09:52:55,703][53852] Updated weights for policy 0, policy_version 52540 (0.0007) [2023-10-08 09:52:56,951][53885] Updated weights for policy 1, policy_version 52292 (0.0010) [2023-10-08 09:52:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107347968. Throughput: 0: 1842.5, 1: 1825.9. Samples: 26841992. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) [2023-10-08 09:52:57,016][52710] Avg episode reward: [(0, '19.090'), (1, '29.890')] [2023-10-08 09:52:57,313][53885] Updated weights for policy 1, policy_version 52302 (0.0011) [2023-10-08 09:52:57,675][53885] Updated weights for policy 1, policy_version 52312 (0.0010) [2023-10-08 09:52:59,225][53852] Updated weights for policy 0, policy_version 52550 (0.0007) [2023-10-08 09:52:59,594][53852] Updated weights for policy 0, policy_version 52560 (0.0009) [2023-10-08 09:52:59,973][53852] Updated weights for policy 0, policy_version 52570 (0.0009) [2023-10-08 09:53:01,395][53885] Updated weights for policy 1, policy_version 52322 (0.0012) [2023-10-08 09:53:01,780][53885] Updated weights for policy 1, policy_version 52332 (0.0009) [2023-10-08 09:53:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 107413504. Throughput: 0: 1838.6, 1: 1832.0. Samples: 26863734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:53:02,016][52710] Avg episode reward: [(0, '21.170'), (1, '32.470')] [2023-10-08 09:53:02,147][53885] Updated weights for policy 1, policy_version 52342 (0.0008) [2023-10-08 09:53:02,507][53885] Updated weights for policy 1, policy_version 52352 (0.0007) [2023-10-08 09:53:03,656][53852] Updated weights for policy 0, policy_version 52580 (0.0010) [2023-10-08 09:53:04,026][53852] Updated weights for policy 0, policy_version 52590 (0.0010) [2023-10-08 09:53:04,395][53852] Updated weights for policy 0, policy_version 52600 (0.0009) [2023-10-08 09:53:06,064][53885] Updated weights for policy 1, policy_version 52362 (0.0008) [2023-10-08 09:53:06,428][53885] Updated weights for policy 1, policy_version 52372 (0.0008) [2023-10-08 09:53:06,794][53885] Updated weights for policy 1, policy_version 52382 (0.0008) [2023-10-08 09:53:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 107511808. Throughput: 0: 1848.3, 1: 1823.9. Samples: 26885704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:53:07,016][52710] Avg episode reward: [(0, '23.340'), (1, '33.730')] [2023-10-08 09:53:07,918][53852] Updated weights for policy 0, policy_version 52610 (0.0009) [2023-10-08 09:53:08,284][53852] Updated weights for policy 0, policy_version 52620 (0.0007) [2023-10-08 09:53:08,659][53852] Updated weights for policy 0, policy_version 52630 (0.0007) [2023-10-08 09:53:09,023][53852] Updated weights for policy 0, policy_version 52640 (0.0008) [2023-10-08 09:53:10,583][53885] Updated weights for policy 1, policy_version 52392 (0.0009) [2023-10-08 09:53:10,945][53885] Updated weights for policy 1, policy_version 52402 (0.0009) [2023-10-08 09:53:11,323][53885] Updated weights for policy 1, policy_version 52412 (0.0007) [2023-10-08 09:53:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107577344. Throughput: 0: 1846.5, 1: 1824.0. Samples: 26896824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:53:12,015][52710] Avg episode reward: [(0, '24.160'), (1, '29.390')] [2023-10-08 09:53:12,693][53852] Updated weights for policy 0, policy_version 52650 (0.0010) [2023-10-08 09:53:13,073][53852] Updated weights for policy 0, policy_version 52660 (0.0009) [2023-10-08 09:53:13,431][53852] Updated weights for policy 0, policy_version 52670 (0.0010) [2023-10-08 09:53:14,829][53885] Updated weights for policy 1, policy_version 52422 (0.0009) [2023-10-08 09:53:15,199][53885] Updated weights for policy 1, policy_version 52432 (0.0009) [2023-10-08 09:53:15,562][53885] Updated weights for policy 1, policy_version 52442 (0.0010) [2023-10-08 09:53:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107642880. Throughput: 0: 1837.2, 1: 1821.3. Samples: 26918850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:53:17,016][52710] Avg episode reward: [(0, '26.010'), (1, '27.250')] [2023-10-08 09:53:17,198][53852] Updated weights for policy 0, policy_version 52680 (0.0008) [2023-10-08 09:53:17,570][53852] Updated weights for policy 0, policy_version 52690 (0.0007) [2023-10-08 09:53:17,946][53852] Updated weights for policy 0, policy_version 52700 (0.0007) [2023-10-08 09:53:19,302][53885] Updated weights for policy 1, policy_version 52452 (0.0009) [2023-10-08 09:53:19,668][53885] Updated weights for policy 1, policy_version 52462 (0.0011) [2023-10-08 09:53:20,036][53885] Updated weights for policy 1, policy_version 52472 (0.0009) [2023-10-08 09:53:21,605][53852] Updated weights for policy 0, policy_version 52710 (0.0008) [2023-10-08 09:53:21,967][53852] Updated weights for policy 0, policy_version 52720 (0.0008) [2023-10-08 09:53:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 107708416. Throughput: 0: 1833.0, 1: 1828.4. Samples: 26941274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:53:22,015][52710] Avg episode reward: [(0, '22.840'), (1, '26.640')] [2023-10-08 09:53:22,347][53852] Updated weights for policy 0, policy_version 52730 (0.0007) [2023-10-08 09:53:23,845][53885] Updated weights for policy 1, policy_version 52482 (0.0008) [2023-10-08 09:53:24,215][53885] Updated weights for policy 1, policy_version 52492 (0.0009) [2023-10-08 09:53:24,583][53885] Updated weights for policy 1, policy_version 52502 (0.0007) [2023-10-08 09:53:24,945][53885] Updated weights for policy 1, policy_version 52512 (0.0008) [2023-10-08 09:53:25,990][53852] Updated weights for policy 0, policy_version 52740 (0.0009) [2023-10-08 09:53:26,362][53852] Updated weights for policy 0, policy_version 52750 (0.0010) [2023-10-08 09:53:26,724][53852] Updated weights for policy 0, policy_version 52760 (0.0008) [2023-10-08 09:53:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 107773952. Throughput: 0: 1840.9, 1: 1830.5. Samples: 26952080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:53:27,016][52710] Avg episode reward: [(0, '25.170'), (1, '22.110')] [2023-10-08 09:53:28,650][53885] Updated weights for policy 1, policy_version 52522 (0.0009) [2023-10-08 09:53:29,012][53885] Updated weights for policy 1, policy_version 52532 (0.0009) [2023-10-08 09:53:29,390][53885] Updated weights for policy 1, policy_version 52542 (0.0009) [2023-10-08 09:53:30,377][53852] Updated weights for policy 0, policy_version 52770 (0.0007) [2023-10-08 09:53:30,744][53852] Updated weights for policy 0, policy_version 52780 (0.0008) [2023-10-08 09:53:31,113][53852] Updated weights for policy 0, policy_version 52790 (0.0008) [2023-10-08 09:53:31,481][53852] Updated weights for policy 0, policy_version 52800 (0.0007) [2023-10-08 09:53:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 107872256. Throughput: 0: 1831.4, 1: 1830.6. Samples: 26973974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:53:32,015][52710] Avg episode reward: [(0, '23.740'), (1, '25.440')] [2023-10-08 09:53:33,020][53885] Updated weights for policy 1, policy_version 52552 (0.0008) [2023-10-08 09:53:33,391][53885] Updated weights for policy 1, policy_version 52562 (0.0007) [2023-10-08 09:53:33,758][53885] Updated weights for policy 1, policy_version 52572 (0.0009) [2023-10-08 09:53:35,339][53852] Updated weights for policy 0, policy_version 52810 (0.0007) [2023-10-08 09:53:35,708][53852] Updated weights for policy 0, policy_version 52820 (0.0010) [2023-10-08 09:53:36,083][53852] Updated weights for policy 0, policy_version 52830 (0.0008) [2023-10-08 09:53:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 107937792. Throughput: 0: 1834.8, 1: 1832.3. Samples: 26995546. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 09:53:37,015][52710] Avg episode reward: [(0, '23.720'), (1, '25.380')] [2023-10-08 09:53:37,462][53885] Updated weights for policy 1, policy_version 52582 (0.0007) [2023-10-08 09:53:37,839][53885] Updated weights for policy 1, policy_version 52592 (0.0007) [2023-10-08 09:53:38,194][53885] Updated weights for policy 1, policy_version 52602 (0.0007) [2023-10-08 09:53:39,720][53852] Updated weights for policy 0, policy_version 52840 (0.0009) [2023-10-08 09:53:40,095][53852] Updated weights for policy 0, policy_version 52850 (0.0011) [2023-10-08 09:53:40,461][53852] Updated weights for policy 0, policy_version 52860 (0.0011) [2023-10-08 09:53:41,647][53885] Updated weights for policy 1, policy_version 52612 (0.0009) [2023-10-08 09:53:42,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108003328. Throughput: 0: 1830.1, 1: 1836.5. Samples: 27006990. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 09:53:42,016][52710] Avg episode reward: [(0, '21.850'), (1, '25.210')] [2023-10-08 09:53:42,025][53885] Updated weights for policy 1, policy_version 52622 (0.0009) [2023-10-08 09:53:42,393][53885] Updated weights for policy 1, policy_version 52632 (0.0009) [2023-10-08 09:53:44,348][53852] Updated weights for policy 0, policy_version 52870 (0.0008) [2023-10-08 09:53:44,721][53852] Updated weights for policy 0, policy_version 52880 (0.0007) [2023-10-08 09:53:45,104][53852] Updated weights for policy 0, policy_version 52890 (0.0010) [2023-10-08 09:53:46,274][53885] Updated weights for policy 1, policy_version 52642 (0.0008) [2023-10-08 09:53:46,642][53885] Updated weights for policy 1, policy_version 52652 (0.0009) [2023-10-08 09:53:47,007][53885] Updated weights for policy 1, policy_version 52662 (0.0008) [2023-10-08 09:53:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 108068864. Throughput: 0: 1827.1, 1: 1834.4. Samples: 27028502. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 09:53:47,016][52710] Avg episode reward: [(0, '24.290'), (1, '29.630')] [2023-10-08 09:53:47,376][53885] Updated weights for policy 1, policy_version 52672 (0.0009) [2023-10-08 09:53:48,682][53852] Updated weights for policy 0, policy_version 52900 (0.0010) [2023-10-08 09:53:49,040][53852] Updated weights for policy 0, policy_version 52910 (0.0010) [2023-10-08 09:53:49,407][53852] Updated weights for policy 0, policy_version 52920 (0.0011) [2023-10-08 09:53:50,837][53885] Updated weights for policy 1, policy_version 52682 (0.0010) [2023-10-08 09:53:51,200][53885] Updated weights for policy 1, policy_version 52692 (0.0010) [2023-10-08 09:53:51,565][53885] Updated weights for policy 1, policy_version 52702 (0.0010) [2023-10-08 09:53:52,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 108167168. Throughput: 0: 1825.3, 1: 1827.6. Samples: 27050084. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 09:53:52,015][52710] Avg episode reward: [(0, '24.840'), (1, '29.060')] [2023-10-08 09:53:52,944][53852] Updated weights for policy 0, policy_version 52930 (0.0009) [2023-10-08 09:53:53,308][53852] Updated weights for policy 0, policy_version 52940 (0.0007) [2023-10-08 09:53:53,684][53852] Updated weights for policy 0, policy_version 52950 (0.0009) [2023-10-08 09:53:54,051][53852] Updated weights for policy 0, policy_version 52960 (0.0009) [2023-10-08 09:53:55,296][53885] Updated weights for policy 1, policy_version 52712 (0.0009) [2023-10-08 09:53:55,673][53885] Updated weights for policy 1, policy_version 52722 (0.0008) [2023-10-08 09:53:56,042][53885] Updated weights for policy 1, policy_version 52732 (0.0007) [2023-10-08 09:53:57,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108232704. Throughput: 0: 1824.5, 1: 1837.2. Samples: 27061600. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 09:53:57,016][52710] Avg episode reward: [(0, '22.640'), (1, '29.270')] [2023-10-08 09:53:57,588][53852] Updated weights for policy 0, policy_version 52970 (0.0009) [2023-10-08 09:53:57,967][53852] Updated weights for policy 0, policy_version 52980 (0.0009) [2023-10-08 09:53:58,332][53852] Updated weights for policy 0, policy_version 52990 (0.0008) [2023-10-08 09:53:59,639][53885] Updated weights for policy 1, policy_version 52742 (0.0007) [2023-10-08 09:54:00,017][53885] Updated weights for policy 1, policy_version 52752 (0.0008) [2023-10-08 09:54:00,390][53885] Updated weights for policy 1, policy_version 52762 (0.0010) [2023-10-08 09:54:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108298240. Throughput: 0: 1827.2, 1: 1828.9. Samples: 27083378. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 09:54:02,015][52710] Avg episode reward: [(0, '25.770'), (1, '33.530')] [2023-10-08 09:54:02,099][53852] Updated weights for policy 0, policy_version 53000 (0.0010) [2023-10-08 09:54:02,468][53852] Updated weights for policy 0, policy_version 53010 (0.0007) [2023-10-08 09:54:02,831][53852] Updated weights for policy 0, policy_version 53020 (0.0008) [2023-10-08 09:54:04,266][53885] Updated weights for policy 1, policy_version 52772 (0.0008) [2023-10-08 09:54:04,636][53885] Updated weights for policy 1, policy_version 52782 (0.0009) [2023-10-08 09:54:05,002][53885] Updated weights for policy 1, policy_version 52792 (0.0009) [2023-10-08 09:54:06,380][53852] Updated weights for policy 0, policy_version 53030 (0.0009) [2023-10-08 09:54:06,743][53852] Updated weights for policy 0, policy_version 53040 (0.0009) [2023-10-08 09:54:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 108363776. Throughput: 0: 1819.9, 1: 1829.5. Samples: 27105496. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) [2023-10-08 09:54:07,016][52710] Avg episode reward: [(0, '22.100'), (1, '32.310')] [2023-10-08 09:54:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000052800_54067200.pth... [2023-10-08 09:54:07,058][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000051104_52330496.pth [2023-10-08 09:54:07,111][53852] Updated weights for policy 0, policy_version 53050 (0.0007) [2023-10-08 09:54:07,328][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000053056_54329344.pth... [2023-10-08 09:54:07,357][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000051328_52559872.pth [2023-10-08 09:54:08,691][53885] Updated weights for policy 1, policy_version 52802 (0.0009) [2023-10-08 09:54:09,061][53885] Updated weights for policy 1, policy_version 52812 (0.0008) [2023-10-08 09:54:09,425][53885] Updated weights for policy 1, policy_version 52822 (0.0007) [2023-10-08 09:54:09,786][53885] Updated weights for policy 1, policy_version 52832 (0.0007) [2023-10-08 09:54:10,817][53852] Updated weights for policy 0, policy_version 53060 (0.0009) [2023-10-08 09:54:11,197][53852] Updated weights for policy 0, policy_version 53070 (0.0009) [2023-10-08 09:54:11,564][53852] Updated weights for policy 0, policy_version 53080 (0.0009) [2023-10-08 09:54:12,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 108462080. Throughput: 0: 1824.8, 1: 1824.3. Samples: 27116290. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 09:54:12,016][52710] Avg episode reward: [(0, '22.980'), (1, '30.030')] [2023-10-08 09:54:13,348][53885] Updated weights for policy 1, policy_version 52842 (0.0009) [2023-10-08 09:54:13,708][53885] Updated weights for policy 1, policy_version 52852 (0.0009) [2023-10-08 09:54:14,078][53885] Updated weights for policy 1, policy_version 52862 (0.0008) [2023-10-08 09:54:15,171][53852] Updated weights for policy 0, policy_version 53090 (0.0009) [2023-10-08 09:54:15,546][53852] Updated weights for policy 0, policy_version 53100 (0.0009) [2023-10-08 09:54:15,913][53852] Updated weights for policy 0, policy_version 53110 (0.0010) [2023-10-08 09:54:16,284][53852] Updated weights for policy 0, policy_version 53120 (0.0009) [2023-10-08 09:54:17,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108527616. Throughput: 0: 1820.9, 1: 1832.6. Samples: 27138382. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 09:54:17,015][52710] Avg episode reward: [(0, '25.550'), (1, '28.340')] [2023-10-08 09:54:17,855][53885] Updated weights for policy 1, policy_version 52872 (0.0010) [2023-10-08 09:54:18,231][53885] Updated weights for policy 1, policy_version 52882 (0.0009) [2023-10-08 09:54:18,595][53885] Updated weights for policy 1, policy_version 52892 (0.0007) [2023-10-08 09:54:20,138][53852] Updated weights for policy 0, policy_version 53130 (0.0008) [2023-10-08 09:54:20,524][53852] Updated weights for policy 0, policy_version 53140 (0.0008) [2023-10-08 09:54:20,893][53852] Updated weights for policy 0, policy_version 53150 (0.0009) [2023-10-08 09:54:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 108593152. Throughput: 0: 1829.5, 1: 1827.0. Samples: 27160088. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 09:54:22,016][52710] Avg episode reward: [(0, '25.210'), (1, '27.210')] [2023-10-08 09:54:22,432][53885] Updated weights for policy 1, policy_version 52902 (0.0008) [2023-10-08 09:54:22,815][53885] Updated weights for policy 1, policy_version 52912 (0.0010) [2023-10-08 09:54:23,182][53885] Updated weights for policy 1, policy_version 52922 (0.0009) [2023-10-08 09:54:24,508][53852] Updated weights for policy 0, policy_version 53160 (0.0009) [2023-10-08 09:54:24,872][53852] Updated weights for policy 0, policy_version 53170 (0.0008) [2023-10-08 09:54:25,255][53852] Updated weights for policy 0, policy_version 53180 (0.0008) [2023-10-08 09:54:26,633][53885] Updated weights for policy 1, policy_version 52932 (0.0009) [2023-10-08 09:54:26,985][53885] Updated weights for policy 1, policy_version 52942 (0.0010) [2023-10-08 09:54:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108658688. Throughput: 0: 1821.5, 1: 1824.0. Samples: 27171038. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 09:54:27,016][52710] Avg episode reward: [(0, '23.890'), (1, '29.260')] [2023-10-08 09:54:27,351][53885] Updated weights for policy 1, policy_version 52952 (0.0009) [2023-10-08 09:54:28,867][53852] Updated weights for policy 0, policy_version 53190 (0.0008) [2023-10-08 09:54:29,237][53852] Updated weights for policy 0, policy_version 53200 (0.0011) [2023-10-08 09:54:29,600][53852] Updated weights for policy 0, policy_version 53210 (0.0010) [2023-10-08 09:54:31,019][53885] Updated weights for policy 1, policy_version 52962 (0.0009) [2023-10-08 09:54:31,387][53885] Updated weights for policy 1, policy_version 52972 (0.0010) [2023-10-08 09:54:31,756][53885] Updated weights for policy 1, policy_version 52982 (0.0010) [2023-10-08 09:54:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 108724224. Throughput: 0: 1828.2, 1: 1827.8. Samples: 27193020. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 09:54:32,016][52710] Avg episode reward: [(0, '28.380'), (1, '28.440')] [2023-10-08 09:54:32,135][53885] Updated weights for policy 1, policy_version 52992 (0.0010) [2023-10-08 09:54:33,327][53852] Updated weights for policy 0, policy_version 53220 (0.0010) [2023-10-08 09:54:33,705][53852] Updated weights for policy 0, policy_version 53230 (0.0009) [2023-10-08 09:54:34,079][53852] Updated weights for policy 0, policy_version 53240 (0.0009) [2023-10-08 09:54:35,842][53885] Updated weights for policy 1, policy_version 53002 (0.0008) [2023-10-08 09:54:36,207][53885] Updated weights for policy 1, policy_version 53012 (0.0008) [2023-10-08 09:54:36,574][53885] Updated weights for policy 1, policy_version 53022 (0.0008) [2023-10-08 09:54:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 108822528. Throughput: 0: 1826.7, 1: 1827.2. Samples: 27214512. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 09:54:37,016][52710] Avg episode reward: [(0, '25.920'), (1, '30.130')] [2023-10-08 09:54:37,858][53852] Updated weights for policy 0, policy_version 53250 (0.0009) [2023-10-08 09:54:38,230][53852] Updated weights for policy 0, policy_version 53260 (0.0007) [2023-10-08 09:54:38,603][53852] Updated weights for policy 0, policy_version 53270 (0.0007) [2023-10-08 09:54:38,966][53852] Updated weights for policy 0, policy_version 53280 (0.0009) [2023-10-08 09:54:40,224][53885] Updated weights for policy 1, policy_version 53032 (0.0009) [2023-10-08 09:54:40,587][53885] Updated weights for policy 1, policy_version 53042 (0.0007) [2023-10-08 09:54:40,951][53885] Updated weights for policy 1, policy_version 53052 (0.0008) [2023-10-08 09:54:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 108888064. Throughput: 0: 1826.6, 1: 1829.3. Samples: 27226116. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 09:54:42,016][52710] Avg episode reward: [(0, '26.240'), (1, '32.010')] [2023-10-08 09:54:42,517][53852] Updated weights for policy 0, policy_version 53290 (0.0007) [2023-10-08 09:54:42,884][53852] Updated weights for policy 0, policy_version 53300 (0.0007) [2023-10-08 09:54:43,259][53852] Updated weights for policy 0, policy_version 53310 (0.0007) [2023-10-08 09:54:44,466][53885] Updated weights for policy 1, policy_version 53062 (0.0008) [2023-10-08 09:54:44,839][53885] Updated weights for policy 1, policy_version 53072 (0.0008) [2023-10-08 09:54:45,210][53885] Updated weights for policy 1, policy_version 53082 (0.0009) [2023-10-08 09:54:46,927][53852] Updated weights for policy 0, policy_version 53320 (0.0007) [2023-10-08 09:54:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 108953600. Throughput: 0: 1826.1, 1: 1826.4. Samples: 27247744. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 09:54:47,016][52710] Avg episode reward: [(0, '26.780'), (1, '31.340')] [2023-10-08 09:54:47,293][53852] Updated weights for policy 0, policy_version 53330 (0.0009) [2023-10-08 09:54:47,659][53852] Updated weights for policy 0, policy_version 53340 (0.0009) [2023-10-08 09:54:48,933][53885] Updated weights for policy 1, policy_version 53092 (0.0009) [2023-10-08 09:54:49,296][53885] Updated weights for policy 1, policy_version 53102 (0.0007) [2023-10-08 09:54:49,669][53885] Updated weights for policy 1, policy_version 53112 (0.0008) [2023-10-08 09:54:51,370][53852] Updated weights for policy 0, policy_version 53350 (0.0010) [2023-10-08 09:54:51,740][53852] Updated weights for policy 0, policy_version 53360 (0.0010) [2023-10-08 09:54:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 109019136. Throughput: 0: 1825.9, 1: 1833.2. Samples: 27270154. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-08 09:54:52,015][52710] Avg episode reward: [(0, '26.010'), (1, '29.490')] [2023-10-08 09:54:52,114][53852] Updated weights for policy 0, policy_version 53370 (0.0010) [2023-10-08 09:54:53,375][53885] Updated weights for policy 1, policy_version 53122 (0.0007) [2023-10-08 09:54:53,740][53885] Updated weights for policy 1, policy_version 53132 (0.0007) [2023-10-08 09:54:54,107][53885] Updated weights for policy 1, policy_version 53142 (0.0010) [2023-10-08 09:54:54,474][53885] Updated weights for policy 1, policy_version 53152 (0.0011) [2023-10-08 09:54:55,848][53852] Updated weights for policy 0, policy_version 53380 (0.0007) [2023-10-08 09:54:56,221][53852] Updated weights for policy 0, policy_version 53390 (0.0007) [2023-10-08 09:54:56,591][53852] Updated weights for policy 0, policy_version 53400 (0.0007) [2023-10-08 09:54:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109117440. Throughput: 0: 1826.5, 1: 1823.2. Samples: 27280526. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-08 09:54:57,016][52710] Avg episode reward: [(0, '25.580'), (1, '31.130')] [2023-10-08 09:54:58,130][53885] Updated weights for policy 1, policy_version 53162 (0.0011) [2023-10-08 09:54:58,494][53885] Updated weights for policy 1, policy_version 53172 (0.0010) [2023-10-08 09:54:58,860][53885] Updated weights for policy 1, policy_version 53182 (0.0008) [2023-10-08 09:55:00,347][53852] Updated weights for policy 0, policy_version 53410 (0.0010) [2023-10-08 09:55:00,721][53852] Updated weights for policy 0, policy_version 53420 (0.0009) [2023-10-08 09:55:01,090][53852] Updated weights for policy 0, policy_version 53430 (0.0011) [2023-10-08 09:55:01,466][53852] Updated weights for policy 0, policy_version 53440 (0.0010) [2023-10-08 09:55:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109182976. Throughput: 0: 1825.7, 1: 1827.6. Samples: 27302782. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-08 09:55:02,016][52710] Avg episode reward: [(0, '23.790'), (1, '30.860')] [2023-10-08 09:55:02,534][53885] Updated weights for policy 1, policy_version 53192 (0.0009) [2023-10-08 09:55:02,897][53885] Updated weights for policy 1, policy_version 53202 (0.0010) [2023-10-08 09:55:03,261][53885] Updated weights for policy 1, policy_version 53212 (0.0008) [2023-10-08 09:55:05,272][53852] Updated weights for policy 0, policy_version 53450 (0.0008) [2023-10-08 09:55:05,650][53852] Updated weights for policy 0, policy_version 53460 (0.0007) [2023-10-08 09:55:06,014][53852] Updated weights for policy 0, policy_version 53470 (0.0008) [2023-10-08 09:55:06,931][53885] Updated weights for policy 1, policy_version 53222 (0.0009) [2023-10-08 09:55:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 109248512. Throughput: 0: 1822.8, 1: 1829.8. Samples: 27324454. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-08 09:55:07,016][52710] Avg episode reward: [(0, '23.740'), (1, '32.270')] [2023-10-08 09:55:07,301][53885] Updated weights for policy 1, policy_version 53232 (0.0007) [2023-10-08 09:55:07,663][53885] Updated weights for policy 1, policy_version 53242 (0.0009) [2023-10-08 09:55:09,549][53852] Updated weights for policy 0, policy_version 53480 (0.0010) [2023-10-08 09:55:09,911][53852] Updated weights for policy 0, policy_version 53490 (0.0009) [2023-10-08 09:55:10,278][53852] Updated weights for policy 0, policy_version 53500 (0.0008) [2023-10-08 09:55:11,324][53885] Updated weights for policy 1, policy_version 53252 (0.0008) [2023-10-08 09:55:11,692][53885] Updated weights for policy 1, policy_version 53262 (0.0007) [2023-10-08 09:55:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 109314048. Throughput: 0: 1825.4, 1: 1836.2. Samples: 27335810. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-08 09:55:12,015][52710] Avg episode reward: [(0, '25.330'), (1, '30.170')] [2023-10-08 09:55:12,065][53885] Updated weights for policy 1, policy_version 53272 (0.0008) [2023-10-08 09:55:13,878][53852] Updated weights for policy 0, policy_version 53510 (0.0007) [2023-10-08 09:55:14,258][53852] Updated weights for policy 0, policy_version 53520 (0.0008) [2023-10-08 09:55:14,627][53852] Updated weights for policy 0, policy_version 53530 (0.0009) [2023-10-08 09:55:15,633][53885] Updated weights for policy 1, policy_version 53282 (0.0008) [2023-10-08 09:55:15,998][53885] Updated weights for policy 1, policy_version 53292 (0.0008) [2023-10-08 09:55:16,371][53885] Updated weights for policy 1, policy_version 53302 (0.0009) [2023-10-08 09:55:16,749][53885] Updated weights for policy 1, policy_version 53312 (0.0010) [2023-10-08 09:55:17,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 109412352. Throughput: 0: 1825.8, 1: 1834.5. Samples: 27357734. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-08 09:55:17,016][52710] Avg episode reward: [(0, '25.090'), (1, '33.570')] [2023-10-08 09:55:18,140][53852] Updated weights for policy 0, policy_version 53540 (0.0008) [2023-10-08 09:55:18,511][53852] Updated weights for policy 0, policy_version 53550 (0.0009) [2023-10-08 09:55:18,882][53852] Updated weights for policy 0, policy_version 53560 (0.0008) [2023-10-08 09:55:20,228][53885] Updated weights for policy 1, policy_version 53322 (0.0008) [2023-10-08 09:55:20,596][53885] Updated weights for policy 1, policy_version 53332 (0.0008) [2023-10-08 09:55:20,963][53885] Updated weights for policy 1, policy_version 53342 (0.0008) [2023-10-08 09:55:22,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109477888. Throughput: 0: 1831.9, 1: 1839.0. Samples: 27379704. Policy #0 lag: (min: 9.0, avg: 28.9, max: 41.0) [2023-10-08 09:55:22,016][52710] Avg episode reward: [(0, '26.660'), (1, '34.730')] [2023-10-08 09:55:22,631][53852] Updated weights for policy 0, policy_version 53570 (0.0009) [2023-10-08 09:55:23,005][53852] Updated weights for policy 0, policy_version 53580 (0.0008) [2023-10-08 09:55:23,384][53852] Updated weights for policy 0, policy_version 53590 (0.0007) [2023-10-08 09:55:23,756][53852] Updated weights for policy 0, policy_version 53600 (0.0008) [2023-10-08 09:55:24,686][53885] Updated weights for policy 1, policy_version 53352 (0.0011) [2023-10-08 09:55:25,054][53885] Updated weights for policy 1, policy_version 53362 (0.0009) [2023-10-08 09:55:25,416][53885] Updated weights for policy 1, policy_version 53372 (0.0010) [2023-10-08 09:55:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109543424. Throughput: 0: 1827.4, 1: 1831.5. Samples: 27390764. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) [2023-10-08 09:55:27,016][52710] Avg episode reward: [(0, '29.440'), (1, '31.370')] [2023-10-08 09:55:27,409][53852] Updated weights for policy 0, policy_version 53610 (0.0010) [2023-10-08 09:55:27,781][53852] Updated weights for policy 0, policy_version 53620 (0.0007) [2023-10-08 09:55:28,149][53852] Updated weights for policy 0, policy_version 53630 (0.0008) [2023-10-08 09:55:29,261][53885] Updated weights for policy 1, policy_version 53382 (0.0008) [2023-10-08 09:55:29,628][53885] Updated weights for policy 1, policy_version 53392 (0.0007) [2023-10-08 09:55:29,995][53885] Updated weights for policy 1, policy_version 53402 (0.0007) [2023-10-08 09:55:31,715][53852] Updated weights for policy 0, policy_version 53640 (0.0008) [2023-10-08 09:55:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109608960. Throughput: 0: 1829.9, 1: 1830.4. Samples: 27412454. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) [2023-10-08 09:55:32,016][52710] Avg episode reward: [(0, '28.720'), (1, '33.130')] [2023-10-08 09:55:32,072][53852] Updated weights for policy 0, policy_version 53650 (0.0008) [2023-10-08 09:55:32,453][53852] Updated weights for policy 0, policy_version 53660 (0.0008) [2023-10-08 09:55:33,554][53885] Updated weights for policy 1, policy_version 53412 (0.0008) [2023-10-08 09:55:33,934][53885] Updated weights for policy 1, policy_version 53422 (0.0010) [2023-10-08 09:55:34,301][53885] Updated weights for policy 1, policy_version 53432 (0.0012) [2023-10-08 09:55:36,094][53852] Updated weights for policy 0, policy_version 53670 (0.0008) [2023-10-08 09:55:36,461][53852] Updated weights for policy 0, policy_version 53680 (0.0007) [2023-10-08 09:55:36,835][53852] Updated weights for policy 0, policy_version 53690 (0.0009) [2023-10-08 09:55:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 109674496. Throughput: 0: 1820.7, 1: 1835.9. Samples: 27434706. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) [2023-10-08 09:55:37,016][52710] Avg episode reward: [(0, '27.210'), (1, '35.400')] [2023-10-08 09:55:37,966][53885] Updated weights for policy 1, policy_version 53442 (0.0009) [2023-10-08 09:55:38,333][53885] Updated weights for policy 1, policy_version 53452 (0.0009) [2023-10-08 09:55:38,709][53885] Updated weights for policy 1, policy_version 53462 (0.0009) [2023-10-08 09:55:39,090][53885] Updated weights for policy 1, policy_version 53472 (0.0007) [2023-10-08 09:55:40,600][53852] Updated weights for policy 0, policy_version 53700 (0.0009) [2023-10-08 09:55:40,978][53852] Updated weights for policy 0, policy_version 53710 (0.0008) [2023-10-08 09:55:41,344][53852] Updated weights for policy 0, policy_version 53720 (0.0007) [2023-10-08 09:55:42,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109772800. Throughput: 0: 1828.6, 1: 1835.3. Samples: 27445400. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) [2023-10-08 09:55:42,015][52710] Avg episode reward: [(0, '26.730'), (1, '35.250')] [2023-10-08 09:55:42,811][53885] Updated weights for policy 1, policy_version 53482 (0.0007) [2023-10-08 09:55:43,186][53885] Updated weights for policy 1, policy_version 53492 (0.0009) [2023-10-08 09:55:43,551][53885] Updated weights for policy 1, policy_version 53502 (0.0008) [2023-10-08 09:55:44,998][53852] Updated weights for policy 0, policy_version 53730 (0.0008) [2023-10-08 09:55:45,371][53852] Updated weights for policy 0, policy_version 53740 (0.0008) [2023-10-08 09:55:45,742][53852] Updated weights for policy 0, policy_version 53750 (0.0007) [2023-10-08 09:55:46,107][53852] Updated weights for policy 0, policy_version 53760 (0.0009) [2023-10-08 09:55:47,011][53885] Updated weights for policy 1, policy_version 53512 (0.0008) [2023-10-08 09:55:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109838336. Throughput: 0: 1826.7, 1: 1838.3. Samples: 27467706. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) [2023-10-08 09:55:47,016][52710] Avg episode reward: [(0, '26.010'), (1, '31.300')] [2023-10-08 09:55:47,373][53885] Updated weights for policy 1, policy_version 53522 (0.0009) [2023-10-08 09:55:47,739][53885] Updated weights for policy 1, policy_version 53532 (0.0010) [2023-10-08 09:55:49,700][53852] Updated weights for policy 0, policy_version 53770 (0.0008) [2023-10-08 09:55:50,064][53852] Updated weights for policy 0, policy_version 53780 (0.0011) [2023-10-08 09:55:50,438][53852] Updated weights for policy 0, policy_version 53790 (0.0008) [2023-10-08 09:55:51,586][53885] Updated weights for policy 1, policy_version 53542 (0.0010) [2023-10-08 09:55:51,948][53885] Updated weights for policy 1, policy_version 53552 (0.0010) [2023-10-08 09:55:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109903872. Throughput: 0: 1839.0, 1: 1830.4. Samples: 27489578. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) [2023-10-08 09:55:52,015][52710] Avg episode reward: [(0, '25.840'), (1, '33.190')] [2023-10-08 09:55:52,317][53885] Updated weights for policy 1, policy_version 53562 (0.0009) [2023-10-08 09:55:54,080][53852] Updated weights for policy 0, policy_version 53800 (0.0008) [2023-10-08 09:55:54,453][53852] Updated weights for policy 0, policy_version 53810 (0.0007) [2023-10-08 09:55:54,820][53852] Updated weights for policy 0, policy_version 53820 (0.0007) [2023-10-08 09:55:56,074][53885] Updated weights for policy 1, policy_version 53572 (0.0010) [2023-10-08 09:55:56,449][53885] Updated weights for policy 1, policy_version 53582 (0.0009) [2023-10-08 09:55:56,815][53885] Updated weights for policy 1, policy_version 53592 (0.0009) [2023-10-08 09:55:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 109969408. Throughput: 0: 1823.0, 1: 1830.3. Samples: 27500210. Policy #0 lag: (min: 31.0, avg: 44.9, max: 63.0) [2023-10-08 09:55:57,015][52710] Avg episode reward: [(0, '26.090'), (1, '33.350')] [2023-10-08 09:55:58,372][53852] Updated weights for policy 0, policy_version 53830 (0.0009) [2023-10-08 09:55:58,741][53852] Updated weights for policy 0, policy_version 53840 (0.0010) [2023-10-08 09:55:59,110][53852] Updated weights for policy 0, policy_version 53850 (0.0007) [2023-10-08 09:56:00,402][53885] Updated weights for policy 1, policy_version 53602 (0.0008) [2023-10-08 09:56:00,772][53885] Updated weights for policy 1, policy_version 53612 (0.0008) [2023-10-08 09:56:01,136][53885] Updated weights for policy 1, policy_version 53622 (0.0008) [2023-10-08 09:56:01,508][53885] Updated weights for policy 1, policy_version 53632 (0.0009) [2023-10-08 09:56:02,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110067712. Throughput: 0: 1834.8, 1: 1825.9. Samples: 27522464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:02,016][52710] Avg episode reward: [(0, '27.280'), (1, '31.130')] [2023-10-08 09:56:02,866][53852] Updated weights for policy 0, policy_version 53860 (0.0010) [2023-10-08 09:56:03,234][53852] Updated weights for policy 0, policy_version 53870 (0.0008) [2023-10-08 09:56:03,598][53852] Updated weights for policy 0, policy_version 53880 (0.0008) [2023-10-08 09:56:05,121][53885] Updated weights for policy 1, policy_version 53642 (0.0008) [2023-10-08 09:56:05,484][53885] Updated weights for policy 1, policy_version 53652 (0.0008) [2023-10-08 09:56:05,850][53885] Updated weights for policy 1, policy_version 53662 (0.0009) [2023-10-08 09:56:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 110133248. Throughput: 0: 1838.0, 1: 1825.5. Samples: 27544562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:07,016][52710] Avg episode reward: [(0, '27.640'), (1, '30.140')] [2023-10-08 09:56:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000053664_54951936.pth... [2023-10-08 09:56:07,061][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000051936_53182464.pth [2023-10-08 09:56:07,096][53852] Updated weights for policy 0, policy_version 53890 (0.0008) [2023-10-08 09:56:07,474][53852] Updated weights for policy 0, policy_version 53900 (0.0009) [2023-10-08 09:56:07,839][53852] Updated weights for policy 0, policy_version 53910 (0.0009) [2023-10-08 09:56:08,207][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000053920_55214080.pth... [2023-10-08 09:56:08,209][53852] Updated weights for policy 0, policy_version 53920 (0.0009) [2023-10-08 09:56:08,246][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000052192_53444608.pth [2023-10-08 09:56:09,685][53885] Updated weights for policy 1, policy_version 53672 (0.0009) [2023-10-08 09:56:10,050][53885] Updated weights for policy 1, policy_version 53682 (0.0010) [2023-10-08 09:56:10,416][53885] Updated weights for policy 1, policy_version 53692 (0.0010) [2023-10-08 09:56:11,755][53852] Updated weights for policy 0, policy_version 53930 (0.0008) [2023-10-08 09:56:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110198784. Throughput: 0: 1841.5, 1: 1822.0. Samples: 27555622. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:12,016][52710] Avg episode reward: [(0, '27.100'), (1, '31.770')] [2023-10-08 09:56:12,135][53852] Updated weights for policy 0, policy_version 53940 (0.0011) [2023-10-08 09:56:12,500][53852] Updated weights for policy 0, policy_version 53950 (0.0009) [2023-10-08 09:56:14,264][53885] Updated weights for policy 1, policy_version 53702 (0.0009) [2023-10-08 09:56:14,632][53885] Updated weights for policy 1, policy_version 53712 (0.0008) [2023-10-08 09:56:15,010][53885] Updated weights for policy 1, policy_version 53722 (0.0007) [2023-10-08 09:56:16,393][53852] Updated weights for policy 0, policy_version 53960 (0.0008) [2023-10-08 09:56:16,765][53852] Updated weights for policy 0, policy_version 53970 (0.0008) [2023-10-08 09:56:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 110264320. Throughput: 0: 1832.0, 1: 1824.5. Samples: 27576996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:17,016][52710] Avg episode reward: [(0, '25.580'), (1, '30.490')] [2023-10-08 09:56:17,132][53852] Updated weights for policy 0, policy_version 53980 (0.0007) [2023-10-08 09:56:18,651][53885] Updated weights for policy 1, policy_version 53732 (0.0010) [2023-10-08 09:56:19,017][53885] Updated weights for policy 1, policy_version 53742 (0.0009) [2023-10-08 09:56:19,385][53885] Updated weights for policy 1, policy_version 53752 (0.0008) [2023-10-08 09:56:20,714][53852] Updated weights for policy 0, policy_version 53990 (0.0007) [2023-10-08 09:56:21,086][53852] Updated weights for policy 0, policy_version 54000 (0.0007) [2023-10-08 09:56:21,464][53852] Updated weights for policy 0, policy_version 54010 (0.0008) [2023-10-08 09:56:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110362624. Throughput: 0: 1825.7, 1: 1818.5. Samples: 27598696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:22,016][52710] Avg episode reward: [(0, '23.600'), (1, '32.500')] [2023-10-08 09:56:23,125][53885] Updated weights for policy 1, policy_version 53762 (0.0008) [2023-10-08 09:56:23,491][53885] Updated weights for policy 1, policy_version 53772 (0.0010) [2023-10-08 09:56:23,862][53885] Updated weights for policy 1, policy_version 53782 (0.0007) [2023-10-08 09:56:24,221][53885] Updated weights for policy 1, policy_version 53792 (0.0007) [2023-10-08 09:56:25,201][53852] Updated weights for policy 0, policy_version 54020 (0.0009) [2023-10-08 09:56:25,564][53852] Updated weights for policy 0, policy_version 54030 (0.0008) [2023-10-08 09:56:25,941][53852] Updated weights for policy 0, policy_version 54040 (0.0009) [2023-10-08 09:56:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110428160. Throughput: 0: 1834.3, 1: 1818.5. Samples: 27609774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:27,016][52710] Avg episode reward: [(0, '23.610'), (1, '30.220')] [2023-10-08 09:56:27,757][53885] Updated weights for policy 1, policy_version 53802 (0.0010) [2023-10-08 09:56:28,130][53885] Updated weights for policy 1, policy_version 53812 (0.0010) [2023-10-08 09:56:28,500][53885] Updated weights for policy 1, policy_version 53822 (0.0007) [2023-10-08 09:56:29,501][53852] Updated weights for policy 0, policy_version 54050 (0.0009) [2023-10-08 09:56:29,871][53852] Updated weights for policy 0, policy_version 54060 (0.0010) [2023-10-08 09:56:30,245][53852] Updated weights for policy 0, policy_version 54070 (0.0008) [2023-10-08 09:56:30,615][53852] Updated weights for policy 0, policy_version 54080 (0.0008) [2023-10-08 09:56:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110493696. Throughput: 0: 1822.2, 1: 1823.7. Samples: 27631772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:32,016][52710] Avg episode reward: [(0, '26.220'), (1, '33.370')] [2023-10-08 09:56:32,212][53885] Updated weights for policy 1, policy_version 53832 (0.0009) [2023-10-08 09:56:32,584][53885] Updated weights for policy 1, policy_version 53842 (0.0010) [2023-10-08 09:56:32,958][53885] Updated weights for policy 1, policy_version 53852 (0.0008) [2023-10-08 09:56:34,092][53852] Updated weights for policy 0, policy_version 54090 (0.0008) [2023-10-08 09:56:34,467][53852] Updated weights for policy 0, policy_version 54100 (0.0009) [2023-10-08 09:56:34,845][53852] Updated weights for policy 0, policy_version 54110 (0.0008) [2023-10-08 09:56:36,578][53885] Updated weights for policy 1, policy_version 53862 (0.0007) [2023-10-08 09:56:36,953][53885] Updated weights for policy 1, policy_version 53872 (0.0007) [2023-10-08 09:56:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 110559232. Throughput: 0: 1842.8, 1: 1825.0. Samples: 27654626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:37,016][52710] Avg episode reward: [(0, '25.300'), (1, '33.210')] [2023-10-08 09:56:37,314][53885] Updated weights for policy 1, policy_version 53882 (0.0007) [2023-10-08 09:56:38,510][53852] Updated weights for policy 0, policy_version 54120 (0.0008) [2023-10-08 09:56:38,885][53852] Updated weights for policy 0, policy_version 54130 (0.0007) [2023-10-08 09:56:39,255][53852] Updated weights for policy 0, policy_version 54140 (0.0008) [2023-10-08 09:56:40,996][53885] Updated weights for policy 1, policy_version 53892 (0.0010) [2023-10-08 09:56:41,361][53885] Updated weights for policy 1, policy_version 53902 (0.0010) [2023-10-08 09:56:41,728][53885] Updated weights for policy 1, policy_version 53912 (0.0009) [2023-10-08 09:56:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 110624768. Throughput: 0: 1832.6, 1: 1831.7. Samples: 27665106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:42,016][52710] Avg episode reward: [(0, '25.830'), (1, '34.270')] [2023-10-08 09:56:42,977][53852] Updated weights for policy 0, policy_version 54150 (0.0008) [2023-10-08 09:56:43,338][53852] Updated weights for policy 0, policy_version 54160 (0.0011) [2023-10-08 09:56:43,711][53852] Updated weights for policy 0, policy_version 54170 (0.0011) [2023-10-08 09:56:45,427][53885] Updated weights for policy 1, policy_version 53922 (0.0008) [2023-10-08 09:56:45,824][53885] Updated weights for policy 1, policy_version 53932 (0.0010) [2023-10-08 09:56:46,198][53885] Updated weights for policy 1, policy_version 53942 (0.0007) [2023-10-08 09:56:46,565][53885] Updated weights for policy 1, policy_version 53952 (0.0009) [2023-10-08 09:56:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110723072. Throughput: 0: 1848.7, 1: 1822.9. Samples: 27687686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:47,015][52710] Avg episode reward: [(0, '27.390'), (1, '32.060')] [2023-10-08 09:56:47,306][53852] Updated weights for policy 0, policy_version 54180 (0.0010) [2023-10-08 09:56:47,681][53852] Updated weights for policy 0, policy_version 54190 (0.0009) [2023-10-08 09:56:48,046][53852] Updated weights for policy 0, policy_version 54200 (0.0007) [2023-10-08 09:56:50,303][53885] Updated weights for policy 1, policy_version 53962 (0.0008) [2023-10-08 09:56:50,671][53885] Updated weights for policy 1, policy_version 53972 (0.0009) [2023-10-08 09:56:51,049][53885] Updated weights for policy 1, policy_version 53982 (0.0009) [2023-10-08 09:56:51,766][53852] Updated weights for policy 0, policy_version 54210 (0.0007) [2023-10-08 09:56:52,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110788608. Throughput: 0: 1845.1, 1: 1819.7. Samples: 27709478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:52,016][52710] Avg episode reward: [(0, '27.430'), (1, '34.770')] [2023-10-08 09:56:52,146][53852] Updated weights for policy 0, policy_version 54220 (0.0007) [2023-10-08 09:56:52,504][53852] Updated weights for policy 0, policy_version 54230 (0.0007) [2023-10-08 09:56:52,879][53852] Updated weights for policy 0, policy_version 54240 (0.0007) [2023-10-08 09:56:54,773][53885] Updated weights for policy 1, policy_version 53992 (0.0008) [2023-10-08 09:56:55,148][53885] Updated weights for policy 1, policy_version 54002 (0.0007) [2023-10-08 09:56:55,505][53885] Updated weights for policy 1, policy_version 54012 (0.0009) [2023-10-08 09:56:56,490][53852] Updated weights for policy 0, policy_version 54250 (0.0007) [2023-10-08 09:56:56,865][53852] Updated weights for policy 0, policy_version 54260 (0.0010) [2023-10-08 09:56:57,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 110854144. Throughput: 0: 1843.2, 1: 1824.6. Samples: 27720674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:56:57,016][52710] Avg episode reward: [(0, '26.470'), (1, '32.250')] [2023-10-08 09:56:57,239][53852] Updated weights for policy 0, policy_version 54270 (0.0012) [2023-10-08 09:56:59,215][53885] Updated weights for policy 1, policy_version 54022 (0.0009) [2023-10-08 09:56:59,590][53885] Updated weights for policy 1, policy_version 54032 (0.0010) [2023-10-08 09:56:59,965][53885] Updated weights for policy 1, policy_version 54042 (0.0008) [2023-10-08 09:57:00,900][53852] Updated weights for policy 0, policy_version 54280 (0.0010) [2023-10-08 09:57:01,271][53852] Updated weights for policy 0, policy_version 54290 (0.0011) [2023-10-08 09:57:01,639][53852] Updated weights for policy 0, policy_version 54300 (0.0008) [2023-10-08 09:57:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 110952448. Throughput: 0: 1850.7, 1: 1823.4. Samples: 27742332. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:57:02,016][52710] Avg episode reward: [(0, '28.050'), (1, '33.050')] [2023-10-08 09:57:03,635][53885] Updated weights for policy 1, policy_version 54052 (0.0008) [2023-10-08 09:57:04,003][53885] Updated weights for policy 1, policy_version 54062 (0.0009) [2023-10-08 09:57:04,366][53885] Updated weights for policy 1, policy_version 54072 (0.0010) [2023-10-08 09:57:05,403][53852] Updated weights for policy 0, policy_version 54310 (0.0009) [2023-10-08 09:57:05,775][53852] Updated weights for policy 0, policy_version 54320 (0.0009) [2023-10-08 09:57:06,142][53852] Updated weights for policy 0, policy_version 54330 (0.0008) [2023-10-08 09:57:07,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111017984. Throughput: 0: 1843.3, 1: 1827.3. Samples: 27763870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:57:07,016][52710] Avg episode reward: [(0, '28.170'), (1, '31.680')] [2023-10-08 09:57:08,079][53885] Updated weights for policy 1, policy_version 54082 (0.0009) [2023-10-08 09:57:08,454][53885] Updated weights for policy 1, policy_version 54092 (0.0010) [2023-10-08 09:57:08,821][53885] Updated weights for policy 1, policy_version 54102 (0.0009) [2023-10-08 09:57:09,188][53885] Updated weights for policy 1, policy_version 54112 (0.0008) [2023-10-08 09:57:09,772][53852] Updated weights for policy 0, policy_version 54340 (0.0009) [2023-10-08 09:57:10,151][53852] Updated weights for policy 0, policy_version 54350 (0.0010) [2023-10-08 09:57:10,512][53852] Updated weights for policy 0, policy_version 54360 (0.0008) [2023-10-08 09:57:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111083520. Throughput: 0: 1851.3, 1: 1827.4. Samples: 27775316. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:57:12,015][52710] Avg episode reward: [(0, '26.410'), (1, '32.350')] [2023-10-08 09:57:12,765][53885] Updated weights for policy 1, policy_version 54122 (0.0008) [2023-10-08 09:57:13,136][53885] Updated weights for policy 1, policy_version 54132 (0.0010) [2023-10-08 09:57:13,497][53885] Updated weights for policy 1, policy_version 54142 (0.0010) [2023-10-08 09:57:14,311][53852] Updated weights for policy 0, policy_version 54370 (0.0010) [2023-10-08 09:57:14,675][53852] Updated weights for policy 0, policy_version 54380 (0.0008) [2023-10-08 09:57:15,037][53852] Updated weights for policy 0, policy_version 54390 (0.0007) [2023-10-08 09:57:15,410][53852] Updated weights for policy 0, policy_version 54400 (0.0010) [2023-10-08 09:57:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111149056. Throughput: 0: 1841.4, 1: 1823.1. Samples: 27796676. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:57:17,016][52710] Avg episode reward: [(0, '26.910'), (1, '31.260')] [2023-10-08 09:57:17,093][53885] Updated weights for policy 1, policy_version 54152 (0.0008) [2023-10-08 09:57:17,460][53885] Updated weights for policy 1, policy_version 54162 (0.0009) [2023-10-08 09:57:17,824][53885] Updated weights for policy 1, policy_version 54172 (0.0007) [2023-10-08 09:57:18,992][53852] Updated weights for policy 0, policy_version 54410 (0.0007) [2023-10-08 09:57:19,366][53852] Updated weights for policy 0, policy_version 54420 (0.0010) [2023-10-08 09:57:19,734][53852] Updated weights for policy 0, policy_version 54430 (0.0007) [2023-10-08 09:57:21,533][53885] Updated weights for policy 1, policy_version 54182 (0.0008) [2023-10-08 09:57:21,895][53885] Updated weights for policy 1, policy_version 54192 (0.0010) [2023-10-08 09:57:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111214592. Throughput: 0: 1843.6, 1: 1820.2. Samples: 27819500. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:57:22,016][52710] Avg episode reward: [(0, '24.580'), (1, '29.580')] [2023-10-08 09:57:22,262][53885] Updated weights for policy 1, policy_version 54202 (0.0008) [2023-10-08 09:57:23,242][53852] Updated weights for policy 0, policy_version 54440 (0.0009) [2023-10-08 09:57:23,616][53852] Updated weights for policy 0, policy_version 54450 (0.0007) [2023-10-08 09:57:23,976][53852] Updated weights for policy 0, policy_version 54460 (0.0009) [2023-10-08 09:57:26,009][53885] Updated weights for policy 1, policy_version 54212 (0.0007) [2023-10-08 09:57:26,379][53885] Updated weights for policy 1, policy_version 54222 (0.0008) [2023-10-08 09:57:26,755][53885] Updated weights for policy 1, policy_version 54232 (0.0007) [2023-10-08 09:57:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111280128. Throughput: 0: 1846.0, 1: 1821.2. Samples: 27830128. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:57:27,016][52710] Avg episode reward: [(0, '27.440'), (1, '29.500')] [2023-10-08 09:57:27,466][53852] Updated weights for policy 0, policy_version 54470 (0.0008) [2023-10-08 09:57:27,840][53852] Updated weights for policy 0, policy_version 54480 (0.0008) [2023-10-08 09:57:28,214][53852] Updated weights for policy 0, policy_version 54490 (0.0009) [2023-10-08 09:57:30,438][53885] Updated weights for policy 1, policy_version 54242 (0.0008) [2023-10-08 09:57:30,835][53885] Updated weights for policy 1, policy_version 54252 (0.0007) [2023-10-08 09:57:31,201][53885] Updated weights for policy 1, policy_version 54262 (0.0008) [2023-10-08 09:57:31,576][53885] Updated weights for policy 1, policy_version 54272 (0.0009) [2023-10-08 09:57:31,884][53852] Updated weights for policy 0, policy_version 54500 (0.0009) [2023-10-08 09:57:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111378432. Throughput: 0: 1843.8, 1: 1824.3. Samples: 27852752. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:57:32,016][52710] Avg episode reward: [(0, '27.980'), (1, '32.720')] [2023-10-08 09:57:32,273][53852] Updated weights for policy 0, policy_version 54510 (0.0010) [2023-10-08 09:57:32,657][53852] Updated weights for policy 0, policy_version 54520 (0.0008) [2023-10-08 09:57:35,209][53885] Updated weights for policy 1, policy_version 54282 (0.0008) [2023-10-08 09:57:35,568][53885] Updated weights for policy 1, policy_version 54292 (0.0009) [2023-10-08 09:57:35,934][53885] Updated weights for policy 1, policy_version 54302 (0.0007) [2023-10-08 09:57:36,326][53852] Updated weights for policy 0, policy_version 54530 (0.0007) [2023-10-08 09:57:36,702][53852] Updated weights for policy 0, policy_version 54540 (0.0008) [2023-10-08 09:57:37,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111443968. Throughput: 0: 1832.8, 1: 1825.4. Samples: 27874096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:57:37,016][52710] Avg episode reward: [(0, '27.240'), (1, '30.440')] [2023-10-08 09:57:37,069][53852] Updated weights for policy 0, policy_version 54550 (0.0007) [2023-10-08 09:57:37,432][53852] Updated weights for policy 0, policy_version 54560 (0.0007) [2023-10-08 09:57:39,513][53885] Updated weights for policy 1, policy_version 54312 (0.0007) [2023-10-08 09:57:39,888][53885] Updated weights for policy 1, policy_version 54322 (0.0007) [2023-10-08 09:57:40,247][53885] Updated weights for policy 1, policy_version 54332 (0.0009) [2023-10-08 09:57:41,166][53852] Updated weights for policy 0, policy_version 54570 (0.0010) [2023-10-08 09:57:41,541][53852] Updated weights for policy 0, policy_version 54580 (0.0009) [2023-10-08 09:57:41,904][53852] Updated weights for policy 0, policy_version 54590 (0.0008) [2023-10-08 09:57:42,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 111542272. Throughput: 0: 1839.7, 1: 1820.9. Samples: 27885400. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 09:57:42,016][52710] Avg episode reward: [(0, '27.110'), (1, '34.630')] [2023-10-08 09:57:44,015][53885] Updated weights for policy 1, policy_version 54342 (0.0009) [2023-10-08 09:57:44,387][53885] Updated weights for policy 1, policy_version 54352 (0.0009) [2023-10-08 09:57:44,761][53885] Updated weights for policy 1, policy_version 54362 (0.0007) [2023-10-08 09:57:45,509][53852] Updated weights for policy 0, policy_version 54600 (0.0008) [2023-10-08 09:57:45,879][53852] Updated weights for policy 0, policy_version 54610 (0.0010) [2023-10-08 09:57:46,256][53852] Updated weights for policy 0, policy_version 54620 (0.0007) [2023-10-08 09:57:47,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 111607808. Throughput: 0: 1827.9, 1: 1825.8. Samples: 27906752. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 09:57:47,016][52710] Avg episode reward: [(0, '24.260'), (1, '30.630')] [2023-10-08 09:57:48,359][53885] Updated weights for policy 1, policy_version 54372 (0.0008) [2023-10-08 09:57:48,734][53885] Updated weights for policy 1, policy_version 54382 (0.0011) [2023-10-08 09:57:49,104][53885] Updated weights for policy 1, policy_version 54392 (0.0009) [2023-10-08 09:57:49,860][53852] Updated weights for policy 0, policy_version 54630 (0.0007) [2023-10-08 09:57:50,237][53852] Updated weights for policy 0, policy_version 54640 (0.0009) [2023-10-08 09:57:50,609][53852] Updated weights for policy 0, policy_version 54650 (0.0008) [2023-10-08 09:57:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111673344. Throughput: 0: 1839.4, 1: 1827.6. Samples: 27928886. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 09:57:52,016][52710] Avg episode reward: [(0, '24.880'), (1, '29.510')] [2023-10-08 09:57:52,688][53885] Updated weights for policy 1, policy_version 54402 (0.0008) [2023-10-08 09:57:53,058][53885] Updated weights for policy 1, policy_version 54412 (0.0009) [2023-10-08 09:57:53,430][53885] Updated weights for policy 1, policy_version 54422 (0.0007) [2023-10-08 09:57:53,804][53885] Updated weights for policy 1, policy_version 54432 (0.0009) [2023-10-08 09:57:54,145][53852] Updated weights for policy 0, policy_version 54660 (0.0009) [2023-10-08 09:57:54,527][53852] Updated weights for policy 0, policy_version 54670 (0.0009) [2023-10-08 09:57:54,897][53852] Updated weights for policy 0, policy_version 54680 (0.0011) [2023-10-08 09:57:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 111738880. Throughput: 0: 1827.4, 1: 1828.7. Samples: 27939840. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 09:57:57,015][52710] Avg episode reward: [(0, '28.630'), (1, '32.300')] [2023-10-08 09:57:57,487][53885] Updated weights for policy 1, policy_version 54442 (0.0009) [2023-10-08 09:57:57,866][53885] Updated weights for policy 1, policy_version 54452 (0.0007) [2023-10-08 09:57:58,238][53885] Updated weights for policy 1, policy_version 54462 (0.0008) [2023-10-08 09:57:58,584][53852] Updated weights for policy 0, policy_version 54690 (0.0009) [2023-10-08 09:57:58,952][53852] Updated weights for policy 0, policy_version 54700 (0.0008) [2023-10-08 09:57:59,329][53852] Updated weights for policy 0, policy_version 54710 (0.0007) [2023-10-08 09:57:59,700][53852] Updated weights for policy 0, policy_version 54720 (0.0008) [2023-10-08 09:58:01,891][53885] Updated weights for policy 1, policy_version 54472 (0.0008) [2023-10-08 09:58:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111804416. Throughput: 0: 1847.6, 1: 1831.3. Samples: 27962228. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 09:58:02,015][52710] Avg episode reward: [(0, '25.680'), (1, '28.100')] [2023-10-08 09:58:02,270][53885] Updated weights for policy 1, policy_version 54482 (0.0008) [2023-10-08 09:58:02,634][53885] Updated weights for policy 1, policy_version 54492 (0.0009) [2023-10-08 09:58:03,340][53852] Updated weights for policy 0, policy_version 54730 (0.0007) [2023-10-08 09:58:03,707][53852] Updated weights for policy 0, policy_version 54740 (0.0007) [2023-10-08 09:58:04,085][53852] Updated weights for policy 0, policy_version 54750 (0.0008) [2023-10-08 09:58:06,306][53885] Updated weights for policy 1, policy_version 54502 (0.0008) [2023-10-08 09:58:06,670][53885] Updated weights for policy 1, policy_version 54512 (0.0009) [2023-10-08 09:58:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 111869952. Throughput: 0: 1841.5, 1: 1826.0. Samples: 27984540. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 09:58:07,016][52710] Avg episode reward: [(0, '25.590'), (1, '27.470')] [2023-10-08 09:58:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000054752_56066048.pth... [2023-10-08 09:58:07,042][53885] Updated weights for policy 1, policy_version 54522 (0.0011) [2023-10-08 09:58:07,063][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000053056_54329344.pth [2023-10-08 09:58:07,255][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000054528_55836672.pth... [2023-10-08 09:58:07,296][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000052800_54067200.pth [2023-10-08 09:58:07,763][53852] Updated weights for policy 0, policy_version 54760 (0.0008) [2023-10-08 09:58:08,143][53852] Updated weights for policy 0, policy_version 54770 (0.0008) [2023-10-08 09:58:08,519][53852] Updated weights for policy 0, policy_version 54780 (0.0007) [2023-10-08 09:58:10,520][53885] Updated weights for policy 1, policy_version 54532 (0.0010) [2023-10-08 09:58:10,880][53885] Updated weights for policy 1, policy_version 54542 (0.0010) [2023-10-08 09:58:11,245][53885] Updated weights for policy 1, policy_version 54552 (0.0010) [2023-10-08 09:58:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 111968256. Throughput: 0: 1837.8, 1: 1833.0. Samples: 27995312. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 09:58:12,015][52710] Avg episode reward: [(0, '26.430'), (1, '30.950')] [2023-10-08 09:58:12,162][53852] Updated weights for policy 0, policy_version 54790 (0.0010) [2023-10-08 09:58:12,533][53852] Updated weights for policy 0, policy_version 54800 (0.0008) [2023-10-08 09:58:12,897][53852] Updated weights for policy 0, policy_version 54810 (0.0009) [2023-10-08 09:58:14,985][53885] Updated weights for policy 1, policy_version 54562 (0.0008) [2023-10-08 09:58:15,354][53885] Updated weights for policy 1, policy_version 54572 (0.0009) [2023-10-08 09:58:15,715][53885] Updated weights for policy 1, policy_version 54582 (0.0009) [2023-10-08 09:58:16,093][53885] Updated weights for policy 1, policy_version 54592 (0.0009) [2023-10-08 09:58:16,463][53852] Updated weights for policy 0, policy_version 54820 (0.0008) [2023-10-08 09:58:16,834][53852] Updated weights for policy 0, policy_version 54830 (0.0008) [2023-10-08 09:58:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112033792. Throughput: 0: 1838.0, 1: 1822.7. Samples: 28017482. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) [2023-10-08 09:58:17,016][52710] Avg episode reward: [(0, '22.380'), (1, '30.570')] [2023-10-08 09:58:17,205][53852] Updated weights for policy 0, policy_version 54840 (0.0010) [2023-10-08 09:58:19,677][53885] Updated weights for policy 1, policy_version 54602 (0.0010) [2023-10-08 09:58:20,040][53885] Updated weights for policy 1, policy_version 54612 (0.0010) [2023-10-08 09:58:20,412][53885] Updated weights for policy 1, policy_version 54622 (0.0010) [2023-10-08 09:58:20,836][53852] Updated weights for policy 0, policy_version 54850 (0.0009) [2023-10-08 09:58:21,236][53852] Updated weights for policy 0, policy_version 54860 (0.0008) [2023-10-08 09:58:21,599][53852] Updated weights for policy 0, policy_version 54870 (0.0007) [2023-10-08 09:58:21,963][53852] Updated weights for policy 0, policy_version 54880 (0.0008) [2023-10-08 09:58:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 112132096. Throughput: 0: 1823.9, 1: 1845.4. Samples: 28039214. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-08 09:58:22,015][52710] Avg episode reward: [(0, '23.110'), (1, '32.370')] [2023-10-08 09:58:23,852][53885] Updated weights for policy 1, policy_version 54632 (0.0008) [2023-10-08 09:58:24,218][53885] Updated weights for policy 1, policy_version 54642 (0.0008) [2023-10-08 09:58:24,581][53885] Updated weights for policy 1, policy_version 54652 (0.0010) [2023-10-08 09:58:25,584][53852] Updated weights for policy 0, policy_version 54890 (0.0008) [2023-10-08 09:58:25,953][53852] Updated weights for policy 0, policy_version 54900 (0.0008) [2023-10-08 09:58:26,322][53852] Updated weights for policy 0, policy_version 54910 (0.0007) [2023-10-08 09:58:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 112197632. Throughput: 0: 1848.1, 1: 1831.7. Samples: 28050992. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-08 09:58:27,016][52710] Avg episode reward: [(0, '25.240'), (1, '33.110')] [2023-10-08 09:58:28,156][53885] Updated weights for policy 1, policy_version 54662 (0.0009) [2023-10-08 09:58:28,523][53885] Updated weights for policy 1, policy_version 54672 (0.0008) [2023-10-08 09:58:28,899][53885] Updated weights for policy 1, policy_version 54682 (0.0011) [2023-10-08 09:58:29,998][53852] Updated weights for policy 0, policy_version 54920 (0.0010) [2023-10-08 09:58:30,363][53852] Updated weights for policy 0, policy_version 54930 (0.0010) [2023-10-08 09:58:30,735][53852] Updated weights for policy 0, policy_version 54940 (0.0008) [2023-10-08 09:58:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112263168. Throughput: 0: 1833.1, 1: 1862.3. Samples: 28073044. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-08 09:58:32,016][52710] Avg episode reward: [(0, '23.040'), (1, '29.220')] [2023-10-08 09:58:32,508][53885] Updated weights for policy 1, policy_version 54692 (0.0007) [2023-10-08 09:58:32,875][53885] Updated weights for policy 1, policy_version 54702 (0.0007) [2023-10-08 09:58:33,251][53885] Updated weights for policy 1, policy_version 54712 (0.0007) [2023-10-08 09:58:34,498][53852] Updated weights for policy 0, policy_version 54950 (0.0008) [2023-10-08 09:58:34,865][53852] Updated weights for policy 0, policy_version 54960 (0.0010) [2023-10-08 09:58:35,243][53852] Updated weights for policy 0, policy_version 54970 (0.0010) [2023-10-08 09:58:36,893][53885] Updated weights for policy 1, policy_version 54722 (0.0007) [2023-10-08 09:58:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112328704. Throughput: 0: 1840.7, 1: 1856.0. Samples: 28095236. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-08 09:58:37,015][52710] Avg episode reward: [(0, '26.310'), (1, '34.170')] [2023-10-08 09:58:37,259][53885] Updated weights for policy 1, policy_version 54732 (0.0007) [2023-10-08 09:58:37,627][53885] Updated weights for policy 1, policy_version 54742 (0.0008) [2023-10-08 09:58:37,996][53885] Updated weights for policy 1, policy_version 54752 (0.0008) [2023-10-08 09:58:38,902][53852] Updated weights for policy 0, policy_version 54980 (0.0009) [2023-10-08 09:58:39,277][53852] Updated weights for policy 0, policy_version 54990 (0.0008) [2023-10-08 09:58:39,645][53852] Updated weights for policy 0, policy_version 55000 (0.0008) [2023-10-08 09:58:41,738][53885] Updated weights for policy 1, policy_version 54762 (0.0009) [2023-10-08 09:58:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 112394240. Throughput: 0: 1830.7, 1: 1856.7. Samples: 28105772. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-08 09:58:42,016][52710] Avg episode reward: [(0, '25.690'), (1, '31.880')] [2023-10-08 09:58:42,101][53885] Updated weights for policy 1, policy_version 54772 (0.0007) [2023-10-08 09:58:42,472][53885] Updated weights for policy 1, policy_version 54782 (0.0008) [2023-10-08 09:58:43,363][53852] Updated weights for policy 0, policy_version 55010 (0.0007) [2023-10-08 09:58:43,736][53852] Updated weights for policy 0, policy_version 55020 (0.0007) [2023-10-08 09:58:44,099][53852] Updated weights for policy 0, policy_version 55030 (0.0007) [2023-10-08 09:58:44,467][53852] Updated weights for policy 0, policy_version 55040 (0.0007) [2023-10-08 09:58:46,247][53885] Updated weights for policy 1, policy_version 54792 (0.0007) [2023-10-08 09:58:46,606][53885] Updated weights for policy 1, policy_version 54802 (0.0007) [2023-10-08 09:58:46,965][53885] Updated weights for policy 1, policy_version 54812 (0.0008) [2023-10-08 09:58:47,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112459776. Throughput: 0: 1833.9, 1: 1849.1. Samples: 28127964. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-08 09:58:47,016][52710] Avg episode reward: [(0, '24.030'), (1, '31.470')] [2023-10-08 09:58:48,105][53852] Updated weights for policy 0, policy_version 55050 (0.0008) [2023-10-08 09:58:48,474][53852] Updated weights for policy 0, policy_version 55060 (0.0008) [2023-10-08 09:58:48,837][53852] Updated weights for policy 0, policy_version 55070 (0.0008) [2023-10-08 09:58:50,570][53885] Updated weights for policy 1, policy_version 54822 (0.0007) [2023-10-08 09:58:50,949][53885] Updated weights for policy 1, policy_version 54832 (0.0010) [2023-10-08 09:58:51,311][53885] Updated weights for policy 1, policy_version 54842 (0.0008) [2023-10-08 09:58:52,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112558080. Throughput: 0: 1840.2, 1: 1829.8. Samples: 28149690. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-08 09:58:52,016][52710] Avg episode reward: [(0, '24.670'), (1, '33.290')] [2023-10-08 09:58:52,521][53852] Updated weights for policy 0, policy_version 55080 (0.0007) [2023-10-08 09:58:52,890][53852] Updated weights for policy 0, policy_version 55090 (0.0008) [2023-10-08 09:58:53,254][53852] Updated weights for policy 0, policy_version 55100 (0.0009) [2023-10-08 09:58:54,866][53885] Updated weights for policy 1, policy_version 54852 (0.0007) [2023-10-08 09:58:55,226][53885] Updated weights for policy 1, policy_version 54862 (0.0008) [2023-10-08 09:58:55,594][53885] Updated weights for policy 1, policy_version 54872 (0.0008) [2023-10-08 09:58:56,970][53852] Updated weights for policy 0, policy_version 55110 (0.0009) [2023-10-08 09:58:57,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112623616. Throughput: 0: 1835.9, 1: 1850.1. Samples: 28161182. Policy #0 lag: (min: 2.0, avg: 2.2, max: 11.0) [2023-10-08 09:58:57,015][52710] Avg episode reward: [(0, '26.180'), (1, '32.220')] [2023-10-08 09:58:57,337][53852] Updated weights for policy 0, policy_version 55120 (0.0007) [2023-10-08 09:58:57,708][53852] Updated weights for policy 0, policy_version 55130 (0.0007) [2023-10-08 09:58:59,191][53885] Updated weights for policy 1, policy_version 54882 (0.0008) [2023-10-08 09:58:59,560][53885] Updated weights for policy 1, policy_version 54892 (0.0007) [2023-10-08 09:58:59,924][53885] Updated weights for policy 1, policy_version 54902 (0.0011) [2023-10-08 09:59:00,297][53885] Updated weights for policy 1, policy_version 54912 (0.0008) [2023-10-08 09:59:01,405][53852] Updated weights for policy 0, policy_version 55140 (0.0007) [2023-10-08 09:59:01,785][53852] Updated weights for policy 0, policy_version 55150 (0.0009) [2023-10-08 09:59:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112689152. Throughput: 0: 1835.7, 1: 1835.4. Samples: 28182682. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-08 09:59:02,016][52710] Avg episode reward: [(0, '24.140'), (1, '32.690')] [2023-10-08 09:59:02,157][53852] Updated weights for policy 0, policy_version 55160 (0.0008) [2023-10-08 09:59:04,263][53885] Updated weights for policy 1, policy_version 54922 (0.0007) [2023-10-08 09:59:04,636][53885] Updated weights for policy 1, policy_version 54932 (0.0009) [2023-10-08 09:59:05,002][53885] Updated weights for policy 1, policy_version 54942 (0.0008) [2023-10-08 09:59:05,779][53852] Updated weights for policy 0, policy_version 55170 (0.0007) [2023-10-08 09:59:06,146][53852] Updated weights for policy 0, policy_version 55180 (0.0009) [2023-10-08 09:59:06,522][53852] Updated weights for policy 0, policy_version 55190 (0.0009) [2023-10-08 09:59:06,886][53852] Updated weights for policy 0, policy_version 55200 (0.0008) [2023-10-08 09:59:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 112787456. Throughput: 0: 1835.8, 1: 1840.3. Samples: 28204636. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-08 09:59:07,015][52710] Avg episode reward: [(0, '25.190'), (1, '35.290')] [2023-10-08 09:59:08,645][53885] Updated weights for policy 1, policy_version 54952 (0.0007) [2023-10-08 09:59:09,012][53885] Updated weights for policy 1, policy_version 54962 (0.0010) [2023-10-08 09:59:09,376][53885] Updated weights for policy 1, policy_version 54972 (0.0009) [2023-10-08 09:59:10,395][53852] Updated weights for policy 0, policy_version 55210 (0.0009) [2023-10-08 09:59:10,775][53852] Updated weights for policy 0, policy_version 55220 (0.0008) [2023-10-08 09:59:11,135][53852] Updated weights for policy 0, policy_version 55230 (0.0007) [2023-10-08 09:59:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 112852992. Throughput: 0: 1832.4, 1: 1824.4. Samples: 28215546. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-08 09:59:12,016][52710] Avg episode reward: [(0, '23.390'), (1, '32.180')] [2023-10-08 09:59:13,155][53885] Updated weights for policy 1, policy_version 54982 (0.0009) [2023-10-08 09:59:13,536][53885] Updated weights for policy 1, policy_version 54992 (0.0008) [2023-10-08 09:59:13,902][53885] Updated weights for policy 1, policy_version 55002 (0.0008) [2023-10-08 09:59:14,767][53852] Updated weights for policy 0, policy_version 55240 (0.0010) [2023-10-08 09:59:15,140][53852] Updated weights for policy 0, policy_version 55250 (0.0008) [2023-10-08 09:59:15,518][53852] Updated weights for policy 0, policy_version 55260 (0.0007) [2023-10-08 09:59:17,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 112918528. Throughput: 0: 1831.2, 1: 1815.8. Samples: 28237158. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-08 09:59:17,017][52710] Avg episode reward: [(0, '22.530'), (1, '31.180')] [2023-10-08 09:59:17,645][53885] Updated weights for policy 1, policy_version 55012 (0.0008) [2023-10-08 09:59:18,012][53885] Updated weights for policy 1, policy_version 55022 (0.0009) [2023-10-08 09:59:18,378][53885] Updated weights for policy 1, policy_version 55032 (0.0009) [2023-10-08 09:59:19,169][53852] Updated weights for policy 0, policy_version 55270 (0.0007) [2023-10-08 09:59:19,547][53852] Updated weights for policy 0, policy_version 55280 (0.0007) [2023-10-08 09:59:19,918][53852] Updated weights for policy 0, policy_version 55290 (0.0007) [2023-10-08 09:59:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 112984064. Throughput: 0: 1841.2, 1: 1818.0. Samples: 28259896. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-08 09:59:22,015][52710] Avg episode reward: [(0, '22.680'), (1, '34.580')] [2023-10-08 09:59:22,139][53885] Updated weights for policy 1, policy_version 55042 (0.0007) [2023-10-08 09:59:22,502][53885] Updated weights for policy 1, policy_version 55052 (0.0008) [2023-10-08 09:59:22,866][53885] Updated weights for policy 1, policy_version 55062 (0.0007) [2023-10-08 09:59:23,238][53885] Updated weights for policy 1, policy_version 55072 (0.0007) [2023-10-08 09:59:23,587][53852] Updated weights for policy 0, policy_version 55300 (0.0009) [2023-10-08 09:59:23,953][53852] Updated weights for policy 0, policy_version 55310 (0.0011) [2023-10-08 09:59:24,323][53852] Updated weights for policy 0, policy_version 55320 (0.0010) [2023-10-08 09:59:26,957][53885] Updated weights for policy 1, policy_version 55082 (0.0008) [2023-10-08 09:59:27,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 113049600. Throughput: 0: 1830.1, 1: 1823.0. Samples: 28270162. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-08 09:59:27,016][52710] Avg episode reward: [(0, '25.150'), (1, '32.100')] [2023-10-08 09:59:27,332][53885] Updated weights for policy 1, policy_version 55092 (0.0008) [2023-10-08 09:59:27,696][53885] Updated weights for policy 1, policy_version 55102 (0.0008) [2023-10-08 09:59:27,969][53852] Updated weights for policy 0, policy_version 55330 (0.0009) [2023-10-08 09:59:28,344][53852] Updated weights for policy 0, policy_version 55340 (0.0009) [2023-10-08 09:59:28,710][53852] Updated weights for policy 0, policy_version 55350 (0.0008) [2023-10-08 09:59:29,078][53852] Updated weights for policy 0, policy_version 55360 (0.0008) [2023-10-08 09:59:31,461][53885] Updated weights for policy 1, policy_version 55112 (0.0010) [2023-10-08 09:59:31,836][53885] Updated weights for policy 1, policy_version 55122 (0.0008) [2023-10-08 09:59:32,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 113115136. Throughput: 0: 1846.2, 1: 1822.5. Samples: 28293056. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) [2023-10-08 09:59:32,017][52710] Avg episode reward: [(0, '25.630'), (1, '31.010')] [2023-10-08 09:59:32,204][53885] Updated weights for policy 1, policy_version 55132 (0.0007) [2023-10-08 09:59:32,620][53852] Updated weights for policy 0, policy_version 55370 (0.0009) [2023-10-08 09:59:32,996][53852] Updated weights for policy 0, policy_version 55380 (0.0008) [2023-10-08 09:59:33,364][53852] Updated weights for policy 0, policy_version 55390 (0.0008) [2023-10-08 09:59:35,803][53885] Updated weights for policy 1, policy_version 55142 (0.0007) [2023-10-08 09:59:36,176][53885] Updated weights for policy 1, policy_version 55152 (0.0008) [2023-10-08 09:59:36,547][53885] Updated weights for policy 1, policy_version 55162 (0.0008) [2023-10-08 09:59:37,000][53852] Updated weights for policy 0, policy_version 55400 (0.0008) [2023-10-08 09:59:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 113213440. Throughput: 0: 1842.5, 1: 1832.7. Samples: 28315072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:59:37,016][52710] Avg episode reward: [(0, '25.200'), (1, '33.250')] [2023-10-08 09:59:37,380][53852] Updated weights for policy 0, policy_version 55410 (0.0009) [2023-10-08 09:59:37,741][53852] Updated weights for policy 0, policy_version 55420 (0.0009) [2023-10-08 09:59:40,101][53885] Updated weights for policy 1, policy_version 55172 (0.0008) [2023-10-08 09:59:40,473][53885] Updated weights for policy 1, policy_version 55182 (0.0009) [2023-10-08 09:59:40,839][53885] Updated weights for policy 1, policy_version 55192 (0.0008) [2023-10-08 09:59:41,278][53852] Updated weights for policy 0, policy_version 55430 (0.0009) [2023-10-08 09:59:41,647][53852] Updated weights for policy 0, policy_version 55440 (0.0009) [2023-10-08 09:59:42,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113278976. Throughput: 0: 1844.2, 1: 1824.0. Samples: 28326250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:59:42,016][52710] Avg episode reward: [(0, '22.960'), (1, '33.400')] [2023-10-08 09:59:42,019][53852] Updated weights for policy 0, policy_version 55450 (0.0010) [2023-10-08 09:59:44,602][53885] Updated weights for policy 1, policy_version 55202 (0.0008) [2023-10-08 09:59:44,974][53885] Updated weights for policy 1, policy_version 55212 (0.0007) [2023-10-08 09:59:45,340][53885] Updated weights for policy 1, policy_version 55222 (0.0008) [2023-10-08 09:59:45,626][53852] Updated weights for policy 0, policy_version 55460 (0.0008) [2023-10-08 09:59:45,705][53885] Updated weights for policy 1, policy_version 55232 (0.0007) [2023-10-08 09:59:45,993][53852] Updated weights for policy 0, policy_version 55470 (0.0008) [2023-10-08 09:59:46,364][53852] Updated weights for policy 0, policy_version 55480 (0.0007) [2023-10-08 09:59:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 113377280. Throughput: 0: 1849.7, 1: 1825.6. Samples: 28348072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:59:47,016][52710] Avg episode reward: [(0, '19.990'), (1, '31.460')] [2023-10-08 09:59:49,321][53885] Updated weights for policy 1, policy_version 55242 (0.0008) [2023-10-08 09:59:49,685][53885] Updated weights for policy 1, policy_version 55252 (0.0008) [2023-10-08 09:59:49,881][53852] Updated weights for policy 0, policy_version 55490 (0.0007) [2023-10-08 09:59:50,060][53885] Updated weights for policy 1, policy_version 55262 (0.0007) [2023-10-08 09:59:50,259][53852] Updated weights for policy 0, policy_version 55500 (0.0007) [2023-10-08 09:59:50,623][53852] Updated weights for policy 0, policy_version 55510 (0.0008) [2023-10-08 09:59:50,993][53852] Updated weights for policy 0, policy_version 55520 (0.0007) [2023-10-08 09:59:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113442816. Throughput: 0: 1842.4, 1: 1820.4. Samples: 28369460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:59:52,016][52710] Avg episode reward: [(0, '21.420'), (1, '33.440')] [2023-10-08 09:59:53,693][53885] Updated weights for policy 1, policy_version 55272 (0.0007) [2023-10-08 09:59:54,057][53885] Updated weights for policy 1, policy_version 55282 (0.0009) [2023-10-08 09:59:54,417][53885] Updated weights for policy 1, policy_version 55292 (0.0008) [2023-10-08 09:59:54,640][53852] Updated weights for policy 0, policy_version 55530 (0.0009) [2023-10-08 09:59:55,007][53852] Updated weights for policy 0, policy_version 55540 (0.0008) [2023-10-08 09:59:55,375][53852] Updated weights for policy 0, policy_version 55550 (0.0008) [2023-10-08 09:59:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113508352. Throughput: 0: 1844.7, 1: 1829.0. Samples: 28380862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 09:59:57,016][52710] Avg episode reward: [(0, '24.470'), (1, '32.680')] [2023-10-08 09:59:58,023][53885] Updated weights for policy 1, policy_version 55302 (0.0008) [2023-10-08 09:59:58,385][53885] Updated weights for policy 1, policy_version 55312 (0.0008) [2023-10-08 09:59:58,748][53885] Updated weights for policy 1, policy_version 55322 (0.0008) [2023-10-08 09:59:59,012][53852] Updated weights for policy 0, policy_version 55560 (0.0008) [2023-10-08 09:59:59,386][53852] Updated weights for policy 0, policy_version 55570 (0.0007) [2023-10-08 09:59:59,754][53852] Updated weights for policy 0, policy_version 55580 (0.0008) [2023-10-08 10:00:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113573888. Throughput: 0: 1841.6, 1: 1838.9. Samples: 28402782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:02,016][52710] Avg episode reward: [(0, '22.740'), (1, '36.480')] [2023-10-08 10:00:02,017][53594] Saving new best policy, reward=36.480! [2023-10-08 10:00:02,321][53885] Updated weights for policy 1, policy_version 55332 (0.0009) [2023-10-08 10:00:02,692][53885] Updated weights for policy 1, policy_version 55342 (0.0007) [2023-10-08 10:00:03,065][53885] Updated weights for policy 1, policy_version 55352 (0.0008) [2023-10-08 10:00:03,420][53852] Updated weights for policy 0, policy_version 55590 (0.0007) [2023-10-08 10:00:03,783][53852] Updated weights for policy 0, policy_version 55600 (0.0008) [2023-10-08 10:00:04,160][53852] Updated weights for policy 0, policy_version 55610 (0.0007) [2023-10-08 10:00:06,737][53885] Updated weights for policy 1, policy_version 55362 (0.0007) [2023-10-08 10:00:07,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 113639424. Throughput: 0: 1848.9, 1: 1840.6. Samples: 28425924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:07,016][52710] Avg episode reward: [(0, '24.080'), (1, '33.450')] [2023-10-08 10:00:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000055616_56950784.pth... [2023-10-08 10:00:07,058][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000053920_55214080.pth [2023-10-08 10:00:07,108][53885] Updated weights for policy 1, policy_version 55372 (0.0009) [2023-10-08 10:00:07,475][53885] Updated weights for policy 1, policy_version 55382 (0.0008) [2023-10-08 10:00:07,742][53852] Updated weights for policy 0, policy_version 55620 (0.0007) [2023-10-08 10:00:07,840][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000055392_56721408.pth... [2023-10-08 10:00:07,842][53885] Updated weights for policy 1, policy_version 55392 (0.0008) [2023-10-08 10:00:07,873][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000053664_54951936.pth [2023-10-08 10:00:08,114][53852] Updated weights for policy 0, policy_version 55630 (0.0009) [2023-10-08 10:00:08,483][53852] Updated weights for policy 0, policy_version 55640 (0.0007) [2023-10-08 10:00:11,485][53885] Updated weights for policy 1, policy_version 55402 (0.0009) [2023-10-08 10:00:11,853][53885] Updated weights for policy 1, policy_version 55412 (0.0008) [2023-10-08 10:00:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 113704960. Throughput: 0: 1847.1, 1: 1834.8. Samples: 28435844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:12,016][52710] Avg episode reward: [(0, '25.560'), (1, '31.820')] [2023-10-08 10:00:12,214][53885] Updated weights for policy 1, policy_version 55422 (0.0010) [2023-10-08 10:00:12,225][53852] Updated weights for policy 0, policy_version 55650 (0.0007) [2023-10-08 10:00:12,597][53852] Updated weights for policy 0, policy_version 55660 (0.0011) [2023-10-08 10:00:12,963][53852] Updated weights for policy 0, policy_version 55670 (0.0008) [2023-10-08 10:00:13,336][53852] Updated weights for policy 0, policy_version 55680 (0.0009) [2023-10-08 10:00:15,861][53885] Updated weights for policy 1, policy_version 55432 (0.0007) [2023-10-08 10:00:16,226][53885] Updated weights for policy 1, policy_version 55442 (0.0008) [2023-10-08 10:00:16,597][53885] Updated weights for policy 1, policy_version 55452 (0.0007) [2023-10-08 10:00:17,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113803264. Throughput: 0: 1840.7, 1: 1837.2. Samples: 28458558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:17,016][52710] Avg episode reward: [(0, '24.400'), (1, '33.210')] [2023-10-08 10:00:17,051][53852] Updated weights for policy 0, policy_version 55690 (0.0009) [2023-10-08 10:00:17,421][53852] Updated weights for policy 0, policy_version 55700 (0.0007) [2023-10-08 10:00:17,789][53852] Updated weights for policy 0, policy_version 55710 (0.0007) [2023-10-08 10:00:20,226][53885] Updated weights for policy 1, policy_version 55462 (0.0008) [2023-10-08 10:00:20,584][53885] Updated weights for policy 1, policy_version 55472 (0.0007) [2023-10-08 10:00:20,957][53885] Updated weights for policy 1, policy_version 55482 (0.0008) [2023-10-08 10:00:21,365][53852] Updated weights for policy 0, policy_version 55720 (0.0007) [2023-10-08 10:00:21,729][53852] Updated weights for policy 0, policy_version 55730 (0.0008) [2023-10-08 10:00:22,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 113868800. Throughput: 0: 1833.5, 1: 1825.0. Samples: 28479706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:22,016][52710] Avg episode reward: [(0, '25.190'), (1, '32.410')] [2023-10-08 10:00:22,100][53852] Updated weights for policy 0, policy_version 55740 (0.0011) [2023-10-08 10:00:24,660][53885] Updated weights for policy 1, policy_version 55492 (0.0007) [2023-10-08 10:00:25,034][53885] Updated weights for policy 1, policy_version 55502 (0.0008) [2023-10-08 10:00:25,405][53885] Updated weights for policy 1, policy_version 55512 (0.0008) [2023-10-08 10:00:25,773][53852] Updated weights for policy 0, policy_version 55750 (0.0008) [2023-10-08 10:00:26,143][53852] Updated weights for policy 0, policy_version 55760 (0.0007) [2023-10-08 10:00:26,515][53852] Updated weights for policy 0, policy_version 55770 (0.0007) [2023-10-08 10:00:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 113967104. Throughput: 0: 1847.4, 1: 1829.0. Samples: 28491688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:27,016][52710] Avg episode reward: [(0, '23.880'), (1, '29.370')] [2023-10-08 10:00:29,223][53885] Updated weights for policy 1, policy_version 55522 (0.0010) [2023-10-08 10:00:29,598][53885] Updated weights for policy 1, policy_version 55532 (0.0008) [2023-10-08 10:00:29,964][53885] Updated weights for policy 1, policy_version 55542 (0.0010) [2023-10-08 10:00:30,168][53852] Updated weights for policy 0, policy_version 55780 (0.0009) [2023-10-08 10:00:30,334][53885] Updated weights for policy 1, policy_version 55552 (0.0009) [2023-10-08 10:00:30,528][53852] Updated weights for policy 0, policy_version 55790 (0.0010) [2023-10-08 10:00:30,894][53852] Updated weights for policy 0, policy_version 55800 (0.0009) [2023-10-08 10:00:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 114032640. Throughput: 0: 1828.3, 1: 1826.8. Samples: 28512550. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:32,016][52710] Avg episode reward: [(0, '25.440'), (1, '30.480')] [2023-10-08 10:00:34,089][53885] Updated weights for policy 1, policy_version 55562 (0.0011) [2023-10-08 10:00:34,472][53885] Updated weights for policy 1, policy_version 55572 (0.0010) [2023-10-08 10:00:34,603][53852] Updated weights for policy 0, policy_version 55810 (0.0007) [2023-10-08 10:00:34,830][53885] Updated weights for policy 1, policy_version 55582 (0.0010) [2023-10-08 10:00:34,979][53852] Updated weights for policy 0, policy_version 55820 (0.0009) [2023-10-08 10:00:35,364][53852] Updated weights for policy 0, policy_version 55830 (0.0008) [2023-10-08 10:00:35,725][53852] Updated weights for policy 0, policy_version 55840 (0.0010) [2023-10-08 10:00:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114098176. Throughput: 0: 1835.5, 1: 1830.5. Samples: 28534428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:37,016][52710] Avg episode reward: [(0, '25.930'), (1, '33.680')] [2023-10-08 10:00:38,482][53885] Updated weights for policy 1, policy_version 55592 (0.0009) [2023-10-08 10:00:38,871][53885] Updated weights for policy 1, policy_version 55602 (0.0010) [2023-10-08 10:00:39,222][53885] Updated weights for policy 1, policy_version 55612 (0.0009) [2023-10-08 10:00:39,407][53852] Updated weights for policy 0, policy_version 55850 (0.0009) [2023-10-08 10:00:39,768][53852] Updated weights for policy 0, policy_version 55860 (0.0009) [2023-10-08 10:00:40,141][53852] Updated weights for policy 0, policy_version 55870 (0.0007) [2023-10-08 10:00:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114163712. Throughput: 0: 1827.3, 1: 1823.5. Samples: 28545150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:42,016][52710] Avg episode reward: [(0, '25.170'), (1, '29.280')] [2023-10-08 10:00:42,945][53885] Updated weights for policy 1, policy_version 55622 (0.0008) [2023-10-08 10:00:43,322][53885] Updated weights for policy 1, policy_version 55632 (0.0008) [2023-10-08 10:00:43,692][53885] Updated weights for policy 1, policy_version 55642 (0.0007) [2023-10-08 10:00:43,782][53852] Updated weights for policy 0, policy_version 55880 (0.0008) [2023-10-08 10:00:44,155][53852] Updated weights for policy 0, policy_version 55890 (0.0010) [2023-10-08 10:00:44,529][53852] Updated weights for policy 0, policy_version 55900 (0.0010) [2023-10-08 10:00:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114229248. Throughput: 0: 1833.4, 1: 1820.7. Samples: 28567216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:00:47,016][52710] Avg episode reward: [(0, '23.680'), (1, '34.020')] [2023-10-08 10:00:47,298][53885] Updated weights for policy 1, policy_version 55652 (0.0009) [2023-10-08 10:00:47,657][53885] Updated weights for policy 1, policy_version 55662 (0.0007) [2023-10-08 10:00:48,037][53885] Updated weights for policy 1, policy_version 55672 (0.0008) [2023-10-08 10:00:48,211][53852] Updated weights for policy 0, policy_version 55910 (0.0009) [2023-10-08 10:00:48,583][53852] Updated weights for policy 0, policy_version 55920 (0.0008) [2023-10-08 10:00:48,944][53852] Updated weights for policy 0, policy_version 55930 (0.0009) [2023-10-08 10:00:51,780][53885] Updated weights for policy 1, policy_version 55682 (0.0007) [2023-10-08 10:00:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114294784. Throughput: 0: 1829.8, 1: 1816.2. Samples: 28589994. Policy #0 lag: (min: 28.0, avg: 36.5, max: 60.0) [2023-10-08 10:00:52,016][52710] Avg episode reward: [(0, '25.730'), (1, '34.270')] [2023-10-08 10:00:52,145][53885] Updated weights for policy 1, policy_version 55692 (0.0007) [2023-10-08 10:00:52,513][53885] Updated weights for policy 1, policy_version 55702 (0.0007) [2023-10-08 10:00:52,685][53852] Updated weights for policy 0, policy_version 55940 (0.0009) [2023-10-08 10:00:52,873][53885] Updated weights for policy 1, policy_version 55712 (0.0007) [2023-10-08 10:00:53,058][53852] Updated weights for policy 0, policy_version 55950 (0.0007) [2023-10-08 10:00:53,420][53852] Updated weights for policy 0, policy_version 55960 (0.0007) [2023-10-08 10:00:56,575][53885] Updated weights for policy 1, policy_version 55722 (0.0009) [2023-10-08 10:00:56,948][53885] Updated weights for policy 1, policy_version 55732 (0.0007) [2023-10-08 10:00:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 114360320. Throughput: 0: 1828.1, 1: 1818.7. Samples: 28599950. Policy #0 lag: (min: 28.0, avg: 36.5, max: 60.0) [2023-10-08 10:00:57,016][52710] Avg episode reward: [(0, '24.850'), (1, '31.380')] [2023-10-08 10:00:57,023][53852] Updated weights for policy 0, policy_version 55970 (0.0008) [2023-10-08 10:00:57,304][53885] Updated weights for policy 1, policy_version 55742 (0.0007) [2023-10-08 10:00:57,392][53852] Updated weights for policy 0, policy_version 55980 (0.0007) [2023-10-08 10:00:57,764][53852] Updated weights for policy 0, policy_version 55990 (0.0009) [2023-10-08 10:00:58,130][53852] Updated weights for policy 0, policy_version 56000 (0.0008) [2023-10-08 10:01:01,001][53885] Updated weights for policy 1, policy_version 55752 (0.0009) [2023-10-08 10:01:01,370][53885] Updated weights for policy 1, policy_version 55762 (0.0007) [2023-10-08 10:01:01,692][53852] Updated weights for policy 0, policy_version 56010 (0.0007) [2023-10-08 10:01:01,740][53885] Updated weights for policy 1, policy_version 55772 (0.0007) [2023-10-08 10:01:02,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114458624. Throughput: 0: 1835.0, 1: 1817.6. Samples: 28622922. Policy #0 lag: (min: 28.0, avg: 36.5, max: 60.0) [2023-10-08 10:01:02,015][52710] Avg episode reward: [(0, '25.650'), (1, '33.560')] [2023-10-08 10:01:02,057][53852] Updated weights for policy 0, policy_version 56020 (0.0008) [2023-10-08 10:01:02,434][53852] Updated weights for policy 0, policy_version 56030 (0.0009) [2023-10-08 10:01:05,328][53885] Updated weights for policy 1, policy_version 55782 (0.0007) [2023-10-08 10:01:05,701][53885] Updated weights for policy 1, policy_version 55792 (0.0007) [2023-10-08 10:01:06,055][53852] Updated weights for policy 0, policy_version 56040 (0.0009) [2023-10-08 10:01:06,066][53885] Updated weights for policy 1, policy_version 55802 (0.0010) [2023-10-08 10:01:06,434][53852] Updated weights for policy 0, policy_version 56050 (0.0009) [2023-10-08 10:01:06,793][53852] Updated weights for policy 0, policy_version 56060 (0.0007) [2023-10-08 10:01:07,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 114556928. Throughput: 0: 1815.8, 1: 1820.2. Samples: 28643324. Policy #0 lag: (min: 28.0, avg: 36.5, max: 60.0) [2023-10-08 10:01:07,016][52710] Avg episode reward: [(0, '25.670'), (1, '32.370')] [2023-10-08 10:01:09,725][53885] Updated weights for policy 1, policy_version 55812 (0.0008) [2023-10-08 10:01:10,097][53885] Updated weights for policy 1, policy_version 55822 (0.0008) [2023-10-08 10:01:10,376][53852] Updated weights for policy 0, policy_version 56070 (0.0007) [2023-10-08 10:01:10,466][53885] Updated weights for policy 1, policy_version 55832 (0.0008) [2023-10-08 10:01:10,739][53852] Updated weights for policy 0, policy_version 56080 (0.0008) [2023-10-08 10:01:11,104][53852] Updated weights for policy 0, policy_version 56090 (0.0009) [2023-10-08 10:01:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 114622464. Throughput: 0: 1826.9, 1: 1825.3. Samples: 28656038. Policy #0 lag: (min: 28.0, avg: 36.5, max: 60.0) [2023-10-08 10:01:12,016][52710] Avg episode reward: [(0, '27.210'), (1, '32.330')] [2023-10-08 10:01:13,940][53885] Updated weights for policy 1, policy_version 55842 (0.0009) [2023-10-08 10:01:14,298][53885] Updated weights for policy 1, policy_version 55852 (0.0009) [2023-10-08 10:01:14,673][53885] Updated weights for policy 1, policy_version 55862 (0.0007) [2023-10-08 10:01:14,804][53852] Updated weights for policy 0, policy_version 56100 (0.0008) [2023-10-08 10:01:15,026][53885] Updated weights for policy 1, policy_version 55872 (0.0008) [2023-10-08 10:01:15,175][53852] Updated weights for policy 0, policy_version 56110 (0.0007) [2023-10-08 10:01:15,543][53852] Updated weights for policy 0, policy_version 56120 (0.0009) [2023-10-08 10:01:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114688000. Throughput: 0: 1817.0, 1: 1829.4. Samples: 28676636. Policy #0 lag: (min: 28.0, avg: 36.5, max: 60.0) [2023-10-08 10:01:17,016][52710] Avg episode reward: [(0, '26.860'), (1, '34.100')] [2023-10-08 10:01:18,652][53885] Updated weights for policy 1, policy_version 55882 (0.0008) [2023-10-08 10:01:19,017][53885] Updated weights for policy 1, policy_version 55892 (0.0010) [2023-10-08 10:01:19,174][53852] Updated weights for policy 0, policy_version 56130 (0.0009) [2023-10-08 10:01:19,386][53885] Updated weights for policy 1, policy_version 55902 (0.0008) [2023-10-08 10:01:19,538][53852] Updated weights for policy 0, policy_version 56140 (0.0007) [2023-10-08 10:01:19,904][53852] Updated weights for policy 0, policy_version 56150 (0.0007) [2023-10-08 10:01:20,273][53852] Updated weights for policy 0, policy_version 56160 (0.0007) [2023-10-08 10:01:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114753536. Throughput: 0: 1832.0, 1: 1831.8. Samples: 28699296. Policy #0 lag: (min: 28.0, avg: 36.5, max: 60.0) [2023-10-08 10:01:22,016][52710] Avg episode reward: [(0, '27.420'), (1, '30.550')] [2023-10-08 10:01:23,201][53885] Updated weights for policy 1, policy_version 55912 (0.0008) [2023-10-08 10:01:23,579][53885] Updated weights for policy 1, policy_version 55922 (0.0008) [2023-10-08 10:01:23,944][53885] Updated weights for policy 1, policy_version 55932 (0.0009) [2023-10-08 10:01:24,183][53852] Updated weights for policy 0, policy_version 56170 (0.0010) [2023-10-08 10:01:24,547][53852] Updated weights for policy 0, policy_version 56180 (0.0011) [2023-10-08 10:01:24,926][53852] Updated weights for policy 0, policy_version 56190 (0.0010) [2023-10-08 10:01:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114819072. Throughput: 0: 1821.3, 1: 1831.7. Samples: 28709538. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-08 10:01:27,016][52710] Avg episode reward: [(0, '25.340'), (1, '31.570')] [2023-10-08 10:01:27,713][53885] Updated weights for policy 1, policy_version 55942 (0.0009) [2023-10-08 10:01:28,082][53885] Updated weights for policy 1, policy_version 55952 (0.0008) [2023-10-08 10:01:28,448][53885] Updated weights for policy 1, policy_version 55962 (0.0007) [2023-10-08 10:01:28,578][53852] Updated weights for policy 0, policy_version 56200 (0.0008) [2023-10-08 10:01:28,948][53852] Updated weights for policy 0, policy_version 56210 (0.0008) [2023-10-08 10:01:29,319][53852] Updated weights for policy 0, policy_version 56220 (0.0008) [2023-10-08 10:01:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114884608. Throughput: 0: 1828.4, 1: 1827.6. Samples: 28731738. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-08 10:01:32,016][52710] Avg episode reward: [(0, '25.190'), (1, '32.580')] [2023-10-08 10:01:32,064][53885] Updated weights for policy 1, policy_version 55972 (0.0008) [2023-10-08 10:01:32,429][53885] Updated weights for policy 1, policy_version 55982 (0.0009) [2023-10-08 10:01:32,797][53885] Updated weights for policy 1, policy_version 55992 (0.0009) [2023-10-08 10:01:32,910][53852] Updated weights for policy 0, policy_version 56230 (0.0007) [2023-10-08 10:01:33,271][53852] Updated weights for policy 0, policy_version 56240 (0.0007) [2023-10-08 10:01:33,644][53852] Updated weights for policy 0, policy_version 56250 (0.0011) [2023-10-08 10:01:36,331][53885] Updated weights for policy 1, policy_version 56002 (0.0010) [2023-10-08 10:01:36,698][53885] Updated weights for policy 1, policy_version 56012 (0.0009) [2023-10-08 10:01:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 114950144. Throughput: 0: 1827.0, 1: 1821.1. Samples: 28754160. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-08 10:01:37,015][52710] Avg episode reward: [(0, '25.370'), (1, '33.230')] [2023-10-08 10:01:37,059][53885] Updated weights for policy 1, policy_version 56022 (0.0008) [2023-10-08 10:01:37,354][53852] Updated weights for policy 0, policy_version 56260 (0.0009) [2023-10-08 10:01:37,428][53885] Updated weights for policy 1, policy_version 56032 (0.0008) [2023-10-08 10:01:37,731][53852] Updated weights for policy 0, policy_version 56270 (0.0007) [2023-10-08 10:01:38,109][53852] Updated weights for policy 0, policy_version 56280 (0.0009) [2023-10-08 10:01:41,200][53885] Updated weights for policy 1, policy_version 56042 (0.0011) [2023-10-08 10:01:41,572][53885] Updated weights for policy 1, policy_version 56052 (0.0008) [2023-10-08 10:01:41,888][53852] Updated weights for policy 0, policy_version 56290 (0.0009) [2023-10-08 10:01:41,940][53885] Updated weights for policy 1, policy_version 56062 (0.0007) [2023-10-08 10:01:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115048448. Throughput: 0: 1824.4, 1: 1831.0. Samples: 28764442. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-08 10:01:42,016][52710] Avg episode reward: [(0, '25.080'), (1, '34.470')] [2023-10-08 10:01:42,267][53852] Updated weights for policy 0, policy_version 56300 (0.0007) [2023-10-08 10:01:42,643][53852] Updated weights for policy 0, policy_version 56310 (0.0007) [2023-10-08 10:01:43,007][53852] Updated weights for policy 0, policy_version 56320 (0.0007) [2023-10-08 10:01:45,469][53885] Updated weights for policy 1, policy_version 56072 (0.0008) [2023-10-08 10:01:45,836][53885] Updated weights for policy 1, policy_version 56082 (0.0011) [2023-10-08 10:01:46,202][53885] Updated weights for policy 1, policy_version 56092 (0.0010) [2023-10-08 10:01:46,576][53852] Updated weights for policy 0, policy_version 56330 (0.0007) [2023-10-08 10:01:46,949][53852] Updated weights for policy 0, policy_version 56340 (0.0008) [2023-10-08 10:01:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115113984. Throughput: 0: 1824.1, 1: 1824.8. Samples: 28787122. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-08 10:01:47,016][52710] Avg episode reward: [(0, '24.380'), (1, '34.500')] [2023-10-08 10:01:47,323][53852] Updated weights for policy 0, policy_version 56350 (0.0009) [2023-10-08 10:01:49,859][53885] Updated weights for policy 1, policy_version 56102 (0.0009) [2023-10-08 10:01:50,228][53885] Updated weights for policy 1, policy_version 56112 (0.0009) [2023-10-08 10:01:50,591][53885] Updated weights for policy 1, policy_version 56122 (0.0010) [2023-10-08 10:01:50,942][53852] Updated weights for policy 0, policy_version 56360 (0.0010) [2023-10-08 10:01:51,306][53852] Updated weights for policy 0, policy_version 56370 (0.0008) [2023-10-08 10:01:51,680][53852] Updated weights for policy 0, policy_version 56380 (0.0008) [2023-10-08 10:01:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 115212288. Throughput: 0: 1825.4, 1: 1837.3. Samples: 28808146. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-08 10:01:52,016][52710] Avg episode reward: [(0, '24.960'), (1, '32.540')] [2023-10-08 10:01:54,241][53885] Updated weights for policy 1, policy_version 56132 (0.0009) [2023-10-08 10:01:54,607][53885] Updated weights for policy 1, policy_version 56142 (0.0009) [2023-10-08 10:01:54,977][53885] Updated weights for policy 1, policy_version 56152 (0.0009) [2023-10-08 10:01:55,271][53852] Updated weights for policy 0, policy_version 56390 (0.0009) [2023-10-08 10:01:55,637][53852] Updated weights for policy 0, policy_version 56400 (0.0010) [2023-10-08 10:01:56,013][53852] Updated weights for policy 0, policy_version 56410 (0.0008) [2023-10-08 10:01:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 115277824. Throughput: 0: 1827.7, 1: 1819.0. Samples: 28820140. Policy #0 lag: (min: 6.0, avg: 9.5, max: 38.0) [2023-10-08 10:01:57,015][52710] Avg episode reward: [(0, '24.230'), (1, '33.000')] [2023-10-08 10:01:58,713][53885] Updated weights for policy 1, policy_version 56162 (0.0007) [2023-10-08 10:01:59,082][53885] Updated weights for policy 1, policy_version 56172 (0.0007) [2023-10-08 10:01:59,449][53885] Updated weights for policy 1, policy_version 56182 (0.0008) [2023-10-08 10:01:59,645][53852] Updated weights for policy 0, policy_version 56420 (0.0008) [2023-10-08 10:01:59,813][53885] Updated weights for policy 1, policy_version 56192 (0.0008) [2023-10-08 10:02:00,013][53852] Updated weights for policy 0, policy_version 56430 (0.0008) [2023-10-08 10:02:00,394][53852] Updated weights for policy 0, policy_version 56440 (0.0008) [2023-10-08 10:02:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115343360. Throughput: 0: 1823.2, 1: 1825.7. Samples: 28840836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:02:02,016][52710] Avg episode reward: [(0, '25.490'), (1, '33.190')] [2023-10-08 10:02:03,342][53885] Updated weights for policy 1, policy_version 56202 (0.0007) [2023-10-08 10:02:03,708][53885] Updated weights for policy 1, policy_version 56212 (0.0007) [2023-10-08 10:02:04,069][53885] Updated weights for policy 1, policy_version 56222 (0.0007) [2023-10-08 10:02:04,107][53852] Updated weights for policy 0, policy_version 56450 (0.0009) [2023-10-08 10:02:04,483][53852] Updated weights for policy 0, policy_version 56460 (0.0007) [2023-10-08 10:02:04,852][53852] Updated weights for policy 0, policy_version 56470 (0.0007) [2023-10-08 10:02:05,224][53852] Updated weights for policy 0, policy_version 56480 (0.0008) [2023-10-08 10:02:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 115408896. Throughput: 0: 1826.2, 1: 1832.0. Samples: 28863916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:02:07,016][52710] Avg episode reward: [(0, '23.770'), (1, '32.340')] [2023-10-08 10:02:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000056480_57835520.pth... [2023-10-08 10:02:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000056224_57573376.pth... [2023-10-08 10:02:07,063][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000054752_56066048.pth [2023-10-08 10:02:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000054528_55836672.pth [2023-10-08 10:02:07,777][53885] Updated weights for policy 1, policy_version 56232 (0.0008) [2023-10-08 10:02:08,153][53885] Updated weights for policy 1, policy_version 56242 (0.0009) [2023-10-08 10:02:08,518][53885] Updated weights for policy 1, policy_version 56252 (0.0008) [2023-10-08 10:02:08,885][53852] Updated weights for policy 0, policy_version 56490 (0.0008) [2023-10-08 10:02:09,252][53852] Updated weights for policy 0, policy_version 56500 (0.0009) [2023-10-08 10:02:09,626][53852] Updated weights for policy 0, policy_version 56510 (0.0009) [2023-10-08 10:02:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 115474432. Throughput: 0: 1822.7, 1: 1836.4. Samples: 28874196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:02:12,016][52710] Avg episode reward: [(0, '23.970'), (1, '32.310')] [2023-10-08 10:02:12,386][53885] Updated weights for policy 1, policy_version 56262 (0.0010) [2023-10-08 10:02:12,752][53885] Updated weights for policy 1, policy_version 56272 (0.0009) [2023-10-08 10:02:13,122][53885] Updated weights for policy 1, policy_version 56282 (0.0007) [2023-10-08 10:02:13,318][53852] Updated weights for policy 0, policy_version 56520 (0.0008) [2023-10-08 10:02:13,689][53852] Updated weights for policy 0, policy_version 56530 (0.0008) [2023-10-08 10:02:14,053][53852] Updated weights for policy 0, policy_version 56540 (0.0008) [2023-10-08 10:02:16,777][53885] Updated weights for policy 1, policy_version 56292 (0.0008) [2023-10-08 10:02:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 115539968. Throughput: 0: 1825.1, 1: 1836.7. Samples: 28896520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:02:17,016][52710] Avg episode reward: [(0, '25.240'), (1, '34.080')] [2023-10-08 10:02:17,170][53885] Updated weights for policy 1, policy_version 56302 (0.0007) [2023-10-08 10:02:17,537][53885] Updated weights for policy 1, policy_version 56312 (0.0008) [2023-10-08 10:02:17,797][53852] Updated weights for policy 0, policy_version 56550 (0.0008) [2023-10-08 10:02:18,168][53852] Updated weights for policy 0, policy_version 56560 (0.0008) [2023-10-08 10:02:18,544][53852] Updated weights for policy 0, policy_version 56570 (0.0009) [2023-10-08 10:02:21,397][53885] Updated weights for policy 1, policy_version 56322 (0.0009) [2023-10-08 10:02:21,753][53885] Updated weights for policy 1, policy_version 56332 (0.0008) [2023-10-08 10:02:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 115605504. Throughput: 0: 1830.7, 1: 1832.2. Samples: 28918992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:02:22,016][52710] Avg episode reward: [(0, '22.130'), (1, '32.020')] [2023-10-08 10:02:22,045][53852] Updated weights for policy 0, policy_version 56580 (0.0009) [2023-10-08 10:02:22,122][53885] Updated weights for policy 1, policy_version 56342 (0.0008) [2023-10-08 10:02:22,420][53852] Updated weights for policy 0, policy_version 56590 (0.0008) [2023-10-08 10:02:22,487][53885] Updated weights for policy 1, policy_version 56352 (0.0008) [2023-10-08 10:02:22,797][53852] Updated weights for policy 0, policy_version 56600 (0.0007) [2023-10-08 10:02:26,224][53885] Updated weights for policy 1, policy_version 56362 (0.0008) [2023-10-08 10:02:26,514][53852] Updated weights for policy 0, policy_version 56610 (0.0007) [2023-10-08 10:02:26,594][53885] Updated weights for policy 1, policy_version 56372 (0.0008) [2023-10-08 10:02:26,886][53852] Updated weights for policy 0, policy_version 56620 (0.0007) [2023-10-08 10:02:26,967][53885] Updated weights for policy 1, policy_version 56382 (0.0010) [2023-10-08 10:02:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 115671040. Throughput: 0: 1836.2, 1: 1827.5. Samples: 28929308. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:02:27,016][52710] Avg episode reward: [(0, '25.250'), (1, '34.240')] [2023-10-08 10:02:27,256][53852] Updated weights for policy 0, policy_version 56630 (0.0010) [2023-10-08 10:02:27,614][53852] Updated weights for policy 0, policy_version 56640 (0.0010) [2023-10-08 10:02:30,704][53885] Updated weights for policy 1, policy_version 56392 (0.0009) [2023-10-08 10:02:31,066][53885] Updated weights for policy 1, policy_version 56402 (0.0010) [2023-10-08 10:02:31,417][53852] Updated weights for policy 0, policy_version 56650 (0.0007) [2023-10-08 10:02:31,433][53885] Updated weights for policy 1, policy_version 56412 (0.0008) [2023-10-08 10:02:31,793][53852] Updated weights for policy 0, policy_version 56660 (0.0008) [2023-10-08 10:02:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115769344. Throughput: 0: 1833.2, 1: 1828.5. Samples: 28951900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:02:32,016][52710] Avg episode reward: [(0, '24.870'), (1, '33.420')] [2023-10-08 10:02:32,158][53852] Updated weights for policy 0, policy_version 56670 (0.0010) [2023-10-08 10:02:35,162][53885] Updated weights for policy 1, policy_version 56422 (0.0010) [2023-10-08 10:02:35,529][53885] Updated weights for policy 1, policy_version 56432 (0.0008) [2023-10-08 10:02:35,865][53852] Updated weights for policy 0, policy_version 56680 (0.0008) [2023-10-08 10:02:35,889][53885] Updated weights for policy 1, policy_version 56442 (0.0008) [2023-10-08 10:02:36,241][53852] Updated weights for policy 0, policy_version 56690 (0.0009) [2023-10-08 10:02:36,605][53852] Updated weights for policy 0, policy_version 56700 (0.0010) [2023-10-08 10:02:37,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 115867648. Throughput: 0: 1827.4, 1: 1815.5. Samples: 28972074. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 10:02:37,015][52710] Avg episode reward: [(0, '25.470'), (1, '33.550')] [2023-10-08 10:02:39,616][53885] Updated weights for policy 1, policy_version 56452 (0.0009) [2023-10-08 10:02:39,985][53885] Updated weights for policy 1, policy_version 56462 (0.0011) [2023-10-08 10:02:40,359][53885] Updated weights for policy 1, policy_version 56472 (0.0009) [2023-10-08 10:02:40,399][53852] Updated weights for policy 0, policy_version 56710 (0.0007) [2023-10-08 10:02:40,778][53852] Updated weights for policy 0, policy_version 56720 (0.0010) [2023-10-08 10:02:41,134][53852] Updated weights for policy 0, policy_version 56730 (0.0011) [2023-10-08 10:02:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115933184. Throughput: 0: 1824.1, 1: 1826.6. Samples: 28984420. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 10:02:42,016][52710] Avg episode reward: [(0, '24.060'), (1, '32.890')] [2023-10-08 10:02:43,883][53885] Updated weights for policy 1, policy_version 56482 (0.0008) [2023-10-08 10:02:44,253][53885] Updated weights for policy 1, policy_version 56492 (0.0008) [2023-10-08 10:02:44,618][53885] Updated weights for policy 1, policy_version 56502 (0.0007) [2023-10-08 10:02:44,850][53852] Updated weights for policy 0, policy_version 56740 (0.0009) [2023-10-08 10:02:44,983][53885] Updated weights for policy 1, policy_version 56512 (0.0007) [2023-10-08 10:02:45,226][53852] Updated weights for policy 0, policy_version 56750 (0.0007) [2023-10-08 10:02:45,595][53852] Updated weights for policy 0, policy_version 56760 (0.0008) [2023-10-08 10:02:47,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 115998720. Throughput: 0: 1827.2, 1: 1823.2. Samples: 29005108. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 10:02:47,016][52710] Avg episode reward: [(0, '26.530'), (1, '34.000')] [2023-10-08 10:02:48,589][53885] Updated weights for policy 1, policy_version 56522 (0.0009) [2023-10-08 10:02:48,957][53885] Updated weights for policy 1, policy_version 56532 (0.0008) [2023-10-08 10:02:49,319][53885] Updated weights for policy 1, policy_version 56542 (0.0007) [2023-10-08 10:02:49,324][53852] Updated weights for policy 0, policy_version 56770 (0.0008) [2023-10-08 10:02:49,693][53852] Updated weights for policy 0, policy_version 56780 (0.0008) [2023-10-08 10:02:50,068][53852] Updated weights for policy 0, policy_version 56790 (0.0008) [2023-10-08 10:02:50,434][53852] Updated weights for policy 0, policy_version 56800 (0.0009) [2023-10-08 10:02:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 116064256. Throughput: 0: 1815.9, 1: 1821.9. Samples: 29027616. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 10:02:52,016][52710] Avg episode reward: [(0, '25.630'), (1, '30.990')] [2023-10-08 10:02:52,907][53885] Updated weights for policy 1, policy_version 56552 (0.0007) [2023-10-08 10:02:53,269][53885] Updated weights for policy 1, policy_version 56562 (0.0008) [2023-10-08 10:02:53,630][53885] Updated weights for policy 1, policy_version 56572 (0.0009) [2023-10-08 10:02:54,172][53852] Updated weights for policy 0, policy_version 56810 (0.0010) [2023-10-08 10:02:54,533][53852] Updated weights for policy 0, policy_version 56820 (0.0009) [2023-10-08 10:02:54,901][53852] Updated weights for policy 0, policy_version 56830 (0.0007) [2023-10-08 10:02:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 116129792. Throughput: 0: 1819.9, 1: 1820.7. Samples: 29038024. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 10:02:57,016][52710] Avg episode reward: [(0, '25.920'), (1, '30.520')] [2023-10-08 10:02:57,377][53885] Updated weights for policy 1, policy_version 56582 (0.0009) [2023-10-08 10:02:57,749][53885] Updated weights for policy 1, policy_version 56592 (0.0008) [2023-10-08 10:02:58,113][53885] Updated weights for policy 1, policy_version 56602 (0.0008) [2023-10-08 10:02:58,524][53852] Updated weights for policy 0, policy_version 56840 (0.0009) [2023-10-08 10:02:58,885][53852] Updated weights for policy 0, policy_version 56850 (0.0007) [2023-10-08 10:02:59,257][53852] Updated weights for policy 0, policy_version 56860 (0.0007) [2023-10-08 10:03:01,675][53885] Updated weights for policy 1, policy_version 56612 (0.0008) [2023-10-08 10:03:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 116195328. Throughput: 0: 1821.0, 1: 1818.8. Samples: 29060310. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 10:03:02,016][52710] Avg episode reward: [(0, '25.340'), (1, '32.390')] [2023-10-08 10:03:02,058][53885] Updated weights for policy 1, policy_version 56622 (0.0009) [2023-10-08 10:03:02,430][53885] Updated weights for policy 1, policy_version 56632 (0.0011) [2023-10-08 10:03:02,990][53852] Updated weights for policy 0, policy_version 56870 (0.0009) [2023-10-08 10:03:03,362][53852] Updated weights for policy 0, policy_version 56880 (0.0010) [2023-10-08 10:03:03,740][53852] Updated weights for policy 0, policy_version 56890 (0.0009) [2023-10-08 10:03:06,215][53885] Updated weights for policy 1, policy_version 56642 (0.0009) [2023-10-08 10:03:06,575][53885] Updated weights for policy 1, policy_version 56652 (0.0007) [2023-10-08 10:03:06,950][53885] Updated weights for policy 1, policy_version 56662 (0.0008) [2023-10-08 10:03:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116260864. Throughput: 0: 1818.9, 1: 1817.9. Samples: 29082646. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 10:03:07,015][52710] Avg episode reward: [(0, '24.580'), (1, '34.920')] [2023-10-08 10:03:07,317][53885] Updated weights for policy 1, policy_version 56672 (0.0007) [2023-10-08 10:03:07,344][53852] Updated weights for policy 0, policy_version 56900 (0.0010) [2023-10-08 10:03:07,715][53852] Updated weights for policy 0, policy_version 56910 (0.0008) [2023-10-08 10:03:08,088][53852] Updated weights for policy 0, policy_version 56920 (0.0010) [2023-10-08 10:03:10,927][53885] Updated weights for policy 1, policy_version 56682 (0.0008) [2023-10-08 10:03:11,294][53885] Updated weights for policy 1, policy_version 56692 (0.0009) [2023-10-08 10:03:11,654][53885] Updated weights for policy 1, policy_version 56702 (0.0007) [2023-10-08 10:03:11,845][53852] Updated weights for policy 0, policy_version 56930 (0.0009) [2023-10-08 10:03:12,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116359168. Throughput: 0: 1816.2, 1: 1825.9. Samples: 29093202. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) [2023-10-08 10:03:12,016][52710] Avg episode reward: [(0, '26.530'), (1, '33.310')] [2023-10-08 10:03:12,220][53852] Updated weights for policy 0, policy_version 56940 (0.0010) [2023-10-08 10:03:12,594][53852] Updated weights for policy 0, policy_version 56950 (0.0009) [2023-10-08 10:03:12,972][53852] Updated weights for policy 0, policy_version 56960 (0.0008) [2023-10-08 10:03:15,425][53885] Updated weights for policy 1, policy_version 56712 (0.0008) [2023-10-08 10:03:15,788][53885] Updated weights for policy 1, policy_version 56722 (0.0011) [2023-10-08 10:03:16,157][53885] Updated weights for policy 1, policy_version 56732 (0.0007) [2023-10-08 10:03:16,562][53852] Updated weights for policy 0, policy_version 56970 (0.0007) [2023-10-08 10:03:16,937][53852] Updated weights for policy 0, policy_version 56980 (0.0008) [2023-10-08 10:03:17,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116424704. Throughput: 0: 1819.6, 1: 1817.1. Samples: 29115554. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-08 10:03:17,016][52710] Avg episode reward: [(0, '27.680'), (1, '32.560')] [2023-10-08 10:03:17,312][53852] Updated weights for policy 0, policy_version 56990 (0.0008) [2023-10-08 10:03:19,799][53885] Updated weights for policy 1, policy_version 56742 (0.0007) [2023-10-08 10:03:20,171][53885] Updated weights for policy 1, policy_version 56752 (0.0007) [2023-10-08 10:03:20,538][53885] Updated weights for policy 1, policy_version 56762 (0.0007) [2023-10-08 10:03:20,921][53852] Updated weights for policy 0, policy_version 57000 (0.0008) [2023-10-08 10:03:21,287][53852] Updated weights for policy 0, policy_version 57010 (0.0010) [2023-10-08 10:03:21,662][53852] Updated weights for policy 0, policy_version 57020 (0.0008) [2023-10-08 10:03:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 116523008. Throughput: 0: 1825.4, 1: 1834.7. Samples: 29136782. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-08 10:03:22,016][52710] Avg episode reward: [(0, '26.750'), (1, '31.120')] [2023-10-08 10:03:24,200][53885] Updated weights for policy 1, policy_version 56772 (0.0008) [2023-10-08 10:03:24,574][53885] Updated weights for policy 1, policy_version 56782 (0.0008) [2023-10-08 10:03:24,946][53885] Updated weights for policy 1, policy_version 56792 (0.0007) [2023-10-08 10:03:25,251][53852] Updated weights for policy 0, policy_version 57030 (0.0009) [2023-10-08 10:03:25,615][53852] Updated weights for policy 0, policy_version 57040 (0.0008) [2023-10-08 10:03:25,994][53852] Updated weights for policy 0, policy_version 57050 (0.0009) [2023-10-08 10:03:27,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 116588544. Throughput: 0: 1829.4, 1: 1823.5. Samples: 29148802. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-08 10:03:27,016][52710] Avg episode reward: [(0, '27.460'), (1, '33.920')] [2023-10-08 10:03:28,539][53885] Updated weights for policy 1, policy_version 56802 (0.0007) [2023-10-08 10:03:28,910][53885] Updated weights for policy 1, policy_version 56812 (0.0008) [2023-10-08 10:03:29,274][53885] Updated weights for policy 1, policy_version 56822 (0.0009) [2023-10-08 10:03:29,643][53885] Updated weights for policy 1, policy_version 56832 (0.0010) [2023-10-08 10:03:29,667][53852] Updated weights for policy 0, policy_version 57060 (0.0008) [2023-10-08 10:03:30,041][53852] Updated weights for policy 0, policy_version 57070 (0.0011) [2023-10-08 10:03:30,406][53852] Updated weights for policy 0, policy_version 57080 (0.0011) [2023-10-08 10:03:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116654080. Throughput: 0: 1826.1, 1: 1832.2. Samples: 29169728. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-08 10:03:32,016][52710] Avg episode reward: [(0, '27.010'), (1, '31.690')] [2023-10-08 10:03:33,371][53885] Updated weights for policy 1, policy_version 56842 (0.0008) [2023-10-08 10:03:33,741][53885] Updated weights for policy 1, policy_version 56852 (0.0008) [2023-10-08 10:03:33,953][53852] Updated weights for policy 0, policy_version 57090 (0.0010) [2023-10-08 10:03:34,102][53885] Updated weights for policy 1, policy_version 56862 (0.0008) [2023-10-08 10:03:34,324][53852] Updated weights for policy 0, policy_version 57100 (0.0008) [2023-10-08 10:03:34,695][53852] Updated weights for policy 0, policy_version 57110 (0.0007) [2023-10-08 10:03:35,071][53852] Updated weights for policy 0, policy_version 57120 (0.0007) [2023-10-08 10:03:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 116719616. Throughput: 0: 1842.6, 1: 1827.6. Samples: 29192774. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-08 10:03:37,016][52710] Avg episode reward: [(0, '28.060'), (1, '32.310')] [2023-10-08 10:03:37,765][53885] Updated weights for policy 1, policy_version 56872 (0.0008) [2023-10-08 10:03:38,128][53885] Updated weights for policy 1, policy_version 56882 (0.0008) [2023-10-08 10:03:38,493][53885] Updated weights for policy 1, policy_version 56892 (0.0009) [2023-10-08 10:03:38,508][53852] Updated weights for policy 0, policy_version 57130 (0.0008) [2023-10-08 10:03:38,878][53852] Updated weights for policy 0, policy_version 57140 (0.0010) [2023-10-08 10:03:39,248][53852] Updated weights for policy 0, policy_version 57150 (0.0008) [2023-10-08 10:03:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 116785152. Throughput: 0: 1833.9, 1: 1825.8. Samples: 29202712. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-08 10:03:42,016][52710] Avg episode reward: [(0, '26.120'), (1, '32.120')] [2023-10-08 10:03:42,158][53885] Updated weights for policy 1, policy_version 56902 (0.0009) [2023-10-08 10:03:42,514][53885] Updated weights for policy 1, policy_version 56912 (0.0008) [2023-10-08 10:03:42,889][53885] Updated weights for policy 1, policy_version 56922 (0.0008) [2023-10-08 10:03:42,955][53852] Updated weights for policy 0, policy_version 57160 (0.0007) [2023-10-08 10:03:43,316][53852] Updated weights for policy 0, policy_version 57170 (0.0008) [2023-10-08 10:03:43,691][53852] Updated weights for policy 0, policy_version 57180 (0.0007) [2023-10-08 10:03:46,619][53885] Updated weights for policy 1, policy_version 56932 (0.0008) [2023-10-08 10:03:47,005][53885] Updated weights for policy 1, policy_version 56942 (0.0007) [2023-10-08 10:03:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116850688. Throughput: 0: 1844.1, 1: 1827.3. Samples: 29225524. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-08 10:03:47,016][52710] Avg episode reward: [(0, '27.400'), (1, '34.740')] [2023-10-08 10:03:47,340][53852] Updated weights for policy 0, policy_version 57190 (0.0009) [2023-10-08 10:03:47,375][53885] Updated weights for policy 1, policy_version 56952 (0.0007) [2023-10-08 10:03:47,705][53852] Updated weights for policy 0, policy_version 57200 (0.0007) [2023-10-08 10:03:48,089][53852] Updated weights for policy 0, policy_version 57210 (0.0009) [2023-10-08 10:03:50,978][53885] Updated weights for policy 1, policy_version 56962 (0.0008) [2023-10-08 10:03:51,347][53885] Updated weights for policy 1, policy_version 56972 (0.0007) [2023-10-08 10:03:51,717][53885] Updated weights for policy 1, policy_version 56982 (0.0007) [2023-10-08 10:03:51,852][53852] Updated weights for policy 0, policy_version 57220 (0.0008) [2023-10-08 10:03:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116916224. Throughput: 0: 1844.1, 1: 1820.7. Samples: 29247562. Policy #0 lag: (min: 26.0, avg: 26.8, max: 45.0) [2023-10-08 10:03:52,016][52710] Avg episode reward: [(0, '27.510'), (1, '32.340')] [2023-10-08 10:03:52,081][53885] Updated weights for policy 1, policy_version 56992 (0.0008) [2023-10-08 10:03:52,244][53852] Updated weights for policy 0, policy_version 57230 (0.0008) [2023-10-08 10:03:52,617][53852] Updated weights for policy 0, policy_version 57240 (0.0007) [2023-10-08 10:03:55,832][53885] Updated weights for policy 1, policy_version 57002 (0.0008) [2023-10-08 10:03:56,205][53885] Updated weights for policy 1, policy_version 57012 (0.0008) [2023-10-08 10:03:56,369][53852] Updated weights for policy 0, policy_version 57250 (0.0008) [2023-10-08 10:03:56,579][53885] Updated weights for policy 1, policy_version 57022 (0.0008) [2023-10-08 10:03:56,743][53852] Updated weights for policy 0, policy_version 57260 (0.0007) [2023-10-08 10:03:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 117014528. Throughput: 0: 1841.3, 1: 1824.9. Samples: 29258180. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:03:57,015][52710] Avg episode reward: [(0, '26.630'), (1, '32.500')] [2023-10-08 10:03:57,102][53852] Updated weights for policy 0, policy_version 57270 (0.0010) [2023-10-08 10:03:57,469][53852] Updated weights for policy 0, policy_version 57280 (0.0009) [2023-10-08 10:04:00,299][53885] Updated weights for policy 1, policy_version 57032 (0.0009) [2023-10-08 10:04:00,658][53885] Updated weights for policy 1, policy_version 57042 (0.0008) [2023-10-08 10:04:01,028][53885] Updated weights for policy 1, policy_version 57052 (0.0008) [2023-10-08 10:04:01,142][53852] Updated weights for policy 0, policy_version 57290 (0.0007) [2023-10-08 10:04:01,510][53852] Updated weights for policy 0, policy_version 57300 (0.0007) [2023-10-08 10:04:01,881][53852] Updated weights for policy 0, policy_version 57310 (0.0007) [2023-10-08 10:04:02,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 117112832. Throughput: 0: 1838.5, 1: 1824.7. Samples: 29280396. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:04:02,015][52710] Avg episode reward: [(0, '24.580'), (1, '31.540')] [2023-10-08 10:04:04,647][53885] Updated weights for policy 1, policy_version 57062 (0.0008) [2023-10-08 10:04:05,013][53885] Updated weights for policy 1, policy_version 57072 (0.0008) [2023-10-08 10:04:05,375][53885] Updated weights for policy 1, policy_version 57082 (0.0007) [2023-10-08 10:04:05,411][53852] Updated weights for policy 0, policy_version 57320 (0.0007) [2023-10-08 10:04:05,782][53852] Updated weights for policy 0, policy_version 57330 (0.0009) [2023-10-08 10:04:06,160][53852] Updated weights for policy 0, policy_version 57340 (0.0007) [2023-10-08 10:04:07,015][52710] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 117178368. Throughput: 0: 1832.6, 1: 1820.7. Samples: 29301180. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:04:07,016][52710] Avg episode reward: [(0, '28.360'), (1, '32.370')] [2023-10-08 10:04:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000057344_58720256.pth... [2023-10-08 10:04:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000057088_58458112.pth... [2023-10-08 10:04:07,062][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000055392_56721408.pth [2023-10-08 10:04:07,066][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000055616_56950784.pth [2023-10-08 10:04:09,176][53885] Updated weights for policy 1, policy_version 57092 (0.0007) [2023-10-08 10:04:09,540][53885] Updated weights for policy 1, policy_version 57102 (0.0009) [2023-10-08 10:04:09,778][53852] Updated weights for policy 0, policy_version 57350 (0.0007) [2023-10-08 10:04:09,909][53885] Updated weights for policy 1, policy_version 57112 (0.0009) [2023-10-08 10:04:10,150][53852] Updated weights for policy 0, policy_version 57360 (0.0007) [2023-10-08 10:04:10,507][53852] Updated weights for policy 0, policy_version 57370 (0.0007) [2023-10-08 10:04:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117243904. Throughput: 0: 1838.4, 1: 1816.5. Samples: 29313270. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:04:12,015][52710] Avg episode reward: [(0, '26.880'), (1, '35.410')] [2023-10-08 10:04:13,707][53885] Updated weights for policy 1, policy_version 57122 (0.0009) [2023-10-08 10:04:14,074][53885] Updated weights for policy 1, policy_version 57132 (0.0010) [2023-10-08 10:04:14,279][53852] Updated weights for policy 0, policy_version 57380 (0.0008) [2023-10-08 10:04:14,443][53885] Updated weights for policy 1, policy_version 57142 (0.0008) [2023-10-08 10:04:14,650][53852] Updated weights for policy 0, policy_version 57390 (0.0008) [2023-10-08 10:04:14,803][53885] Updated weights for policy 1, policy_version 57152 (0.0008) [2023-10-08 10:04:15,018][53852] Updated weights for policy 0, policy_version 57400 (0.0007) [2023-10-08 10:04:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117309440. Throughput: 0: 1833.4, 1: 1809.4. Samples: 29333656. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:04:17,016][52710] Avg episode reward: [(0, '27.960'), (1, '33.290')] [2023-10-08 10:04:18,368][53885] Updated weights for policy 1, policy_version 57162 (0.0007) [2023-10-08 10:04:18,635][53852] Updated weights for policy 0, policy_version 57410 (0.0008) [2023-10-08 10:04:18,734][53885] Updated weights for policy 1, policy_version 57172 (0.0007) [2023-10-08 10:04:19,004][53852] Updated weights for policy 0, policy_version 57420 (0.0009) [2023-10-08 10:04:19,108][53885] Updated weights for policy 1, policy_version 57182 (0.0009) [2023-10-08 10:04:19,368][53852] Updated weights for policy 0, policy_version 57430 (0.0007) [2023-10-08 10:04:19,739][53852] Updated weights for policy 0, policy_version 57440 (0.0008) [2023-10-08 10:04:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 117374976. Throughput: 0: 1835.8, 1: 1810.3. Samples: 29356850. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:04:22,016][52710] Avg episode reward: [(0, '29.490'), (1, '33.050')] [2023-10-08 10:04:22,804][53885] Updated weights for policy 1, policy_version 57192 (0.0009) [2023-10-08 10:04:23,176][53885] Updated weights for policy 1, policy_version 57202 (0.0008) [2023-10-08 10:04:23,499][53852] Updated weights for policy 0, policy_version 57450 (0.0007) [2023-10-08 10:04:23,545][53885] Updated weights for policy 1, policy_version 57212 (0.0009) [2023-10-08 10:04:23,856][53852] Updated weights for policy 0, policy_version 57460 (0.0007) [2023-10-08 10:04:24,231][53852] Updated weights for policy 0, policy_version 57470 (0.0009) [2023-10-08 10:04:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 117440512. Throughput: 0: 1829.8, 1: 1816.0. Samples: 29366772. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:04:27,015][52710] Avg episode reward: [(0, '29.830'), (1, '35.230')] [2023-10-08 10:04:27,182][53885] Updated weights for policy 1, policy_version 57222 (0.0007) [2023-10-08 10:04:27,539][53885] Updated weights for policy 1, policy_version 57232 (0.0007) [2023-10-08 10:04:27,909][53885] Updated weights for policy 1, policy_version 57242 (0.0007) [2023-10-08 10:04:27,913][53852] Updated weights for policy 0, policy_version 57480 (0.0009) [2023-10-08 10:04:28,286][53852] Updated weights for policy 0, policy_version 57490 (0.0009) [2023-10-08 10:04:28,651][53852] Updated weights for policy 0, policy_version 57500 (0.0007) [2023-10-08 10:04:31,640][53885] Updated weights for policy 1, policy_version 57252 (0.0008) [2023-10-08 10:04:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117506048. Throughput: 0: 1825.7, 1: 1817.9. Samples: 29389490. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-08 10:04:32,016][52710] Avg episode reward: [(0, '28.990'), (1, '32.590')] [2023-10-08 10:04:32,023][53885] Updated weights for policy 1, policy_version 57262 (0.0010) [2023-10-08 10:04:32,391][53885] Updated weights for policy 1, policy_version 57272 (0.0010) [2023-10-08 10:04:32,417][53852] Updated weights for policy 0, policy_version 57510 (0.0007) [2023-10-08 10:04:32,791][53852] Updated weights for policy 0, policy_version 57520 (0.0009) [2023-10-08 10:04:33,165][53852] Updated weights for policy 0, policy_version 57530 (0.0007) [2023-10-08 10:04:36,179][53885] Updated weights for policy 1, policy_version 57282 (0.0009) [2023-10-08 10:04:36,544][53885] Updated weights for policy 1, policy_version 57292 (0.0009) [2023-10-08 10:04:36,806][53852] Updated weights for policy 0, policy_version 57540 (0.0007) [2023-10-08 10:04:36,912][53885] Updated weights for policy 1, policy_version 57302 (0.0007) [2023-10-08 10:04:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117571584. Throughput: 0: 1824.6, 1: 1816.0. Samples: 29411386. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-08 10:04:37,016][52710] Avg episode reward: [(0, '29.610'), (1, '32.370')] [2023-10-08 10:04:37,191][53852] Updated weights for policy 0, policy_version 57550 (0.0008) [2023-10-08 10:04:37,279][53885] Updated weights for policy 1, policy_version 57312 (0.0008) [2023-10-08 10:04:37,572][53852] Updated weights for policy 0, policy_version 57560 (0.0007) [2023-10-08 10:04:41,076][53885] Updated weights for policy 1, policy_version 57322 (0.0009) [2023-10-08 10:04:41,354][53852] Updated weights for policy 0, policy_version 57570 (0.0008) [2023-10-08 10:04:41,439][53885] Updated weights for policy 1, policy_version 57332 (0.0010) [2023-10-08 10:04:41,730][53852] Updated weights for policy 0, policy_version 57580 (0.0008) [2023-10-08 10:04:41,811][53885] Updated weights for policy 1, policy_version 57342 (0.0009) [2023-10-08 10:04:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117669888. Throughput: 0: 1821.5, 1: 1808.5. Samples: 29421530. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-08 10:04:42,016][52710] Avg episode reward: [(0, '28.560'), (1, '33.690')] [2023-10-08 10:04:42,091][53852] Updated weights for policy 0, policy_version 57590 (0.0009) [2023-10-08 10:04:42,463][53852] Updated weights for policy 0, policy_version 57600 (0.0008) [2023-10-08 10:04:45,546][53885] Updated weights for policy 1, policy_version 57352 (0.0008) [2023-10-08 10:04:45,906][53885] Updated weights for policy 1, policy_version 57362 (0.0007) [2023-10-08 10:04:46,123][53852] Updated weights for policy 0, policy_version 57610 (0.0007) [2023-10-08 10:04:46,276][53885] Updated weights for policy 1, policy_version 57372 (0.0007) [2023-10-08 10:04:46,487][53852] Updated weights for policy 0, policy_version 57620 (0.0007) [2023-10-08 10:04:46,865][53852] Updated weights for policy 0, policy_version 57630 (0.0008) [2023-10-08 10:04:47,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 117768192. Throughput: 0: 1825.8, 1: 1813.2. Samples: 29444152. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-08 10:04:47,016][52710] Avg episode reward: [(0, '29.370'), (1, '35.150')] [2023-10-08 10:04:49,970][53885] Updated weights for policy 1, policy_version 57382 (0.0008) [2023-10-08 10:04:50,336][53885] Updated weights for policy 1, policy_version 57392 (0.0009) [2023-10-08 10:04:50,386][53852] Updated weights for policy 0, policy_version 57640 (0.0008) [2023-10-08 10:04:50,705][53885] Updated weights for policy 1, policy_version 57402 (0.0009) [2023-10-08 10:04:50,751][53852] Updated weights for policy 0, policy_version 57650 (0.0009) [2023-10-08 10:04:51,129][53852] Updated weights for policy 0, policy_version 57660 (0.0008) [2023-10-08 10:04:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 117833728. Throughput: 0: 1822.2, 1: 1805.6. Samples: 29464430. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-08 10:04:52,017][52710] Avg episode reward: [(0, '28.520'), (1, '32.820')] [2023-10-08 10:04:54,442][53885] Updated weights for policy 1, policy_version 57412 (0.0008) [2023-10-08 10:04:54,806][53885] Updated weights for policy 1, policy_version 57422 (0.0008) [2023-10-08 10:04:54,937][53852] Updated weights for policy 0, policy_version 57670 (0.0008) [2023-10-08 10:04:55,168][53885] Updated weights for policy 1, policy_version 57432 (0.0008) [2023-10-08 10:04:55,314][53852] Updated weights for policy 0, policy_version 57680 (0.0008) [2023-10-08 10:04:55,678][53852] Updated weights for policy 0, policy_version 57690 (0.0009) [2023-10-08 10:04:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117899264. Throughput: 0: 1821.0, 1: 1812.7. Samples: 29476786. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-08 10:04:57,016][52710] Avg episode reward: [(0, '26.540'), (1, '32.560')] [2023-10-08 10:04:58,981][53885] Updated weights for policy 1, policy_version 57442 (0.0007) [2023-10-08 10:04:59,076][53852] Updated weights for policy 0, policy_version 57700 (0.0009) [2023-10-08 10:04:59,349][53885] Updated weights for policy 1, policy_version 57452 (0.0007) [2023-10-08 10:04:59,441][53852] Updated weights for policy 0, policy_version 57710 (0.0008) [2023-10-08 10:04:59,709][53885] Updated weights for policy 1, policy_version 57462 (0.0008) [2023-10-08 10:04:59,808][53852] Updated weights for policy 0, policy_version 57720 (0.0008) [2023-10-08 10:05:00,079][53885] Updated weights for policy 1, policy_version 57472 (0.0009) [2023-10-08 10:05:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 117964800. Throughput: 0: 1820.1, 1: 1805.3. Samples: 29496798. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-08 10:05:02,016][52710] Avg episode reward: [(0, '27.380'), (1, '33.270')] [2023-10-08 10:05:03,674][53852] Updated weights for policy 0, policy_version 57730 (0.0007) [2023-10-08 10:05:03,833][53885] Updated weights for policy 1, policy_version 57482 (0.0009) [2023-10-08 10:05:04,043][53852] Updated weights for policy 0, policy_version 57740 (0.0008) [2023-10-08 10:05:04,201][53885] Updated weights for policy 1, policy_version 57492 (0.0008) [2023-10-08 10:05:04,418][53852] Updated weights for policy 0, policy_version 57750 (0.0007) [2023-10-08 10:05:04,573][53885] Updated weights for policy 1, policy_version 57502 (0.0008) [2023-10-08 10:05:04,791][53852] Updated weights for policy 0, policy_version 57760 (0.0008) [2023-10-08 10:05:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 118030336. Throughput: 0: 1811.2, 1: 1803.5. Samples: 29519510. Policy #0 lag: (min: 31.0, avg: 31.9, max: 52.0) [2023-10-08 10:05:07,016][52710] Avg episode reward: [(0, '27.330'), (1, '32.850')] [2023-10-08 10:05:08,123][53885] Updated weights for policy 1, policy_version 57512 (0.0007) [2023-10-08 10:05:08,348][53852] Updated weights for policy 0, policy_version 57770 (0.0008) [2023-10-08 10:05:08,490][53885] Updated weights for policy 1, policy_version 57522 (0.0008) [2023-10-08 10:05:08,719][53852] Updated weights for policy 0, policy_version 57780 (0.0008) [2023-10-08 10:05:08,856][53885] Updated weights for policy 1, policy_version 57532 (0.0007) [2023-10-08 10:05:09,091][53852] Updated weights for policy 0, policy_version 57790 (0.0007) [2023-10-08 10:05:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118095872. Throughput: 0: 1817.4, 1: 1800.9. Samples: 29529596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:12,015][52710] Avg episode reward: [(0, '27.460'), (1, '34.010')] [2023-10-08 10:05:12,439][53885] Updated weights for policy 1, policy_version 57542 (0.0008) [2023-10-08 10:05:12,782][53852] Updated weights for policy 0, policy_version 57800 (0.0007) [2023-10-08 10:05:12,799][53885] Updated weights for policy 1, policy_version 57552 (0.0008) [2023-10-08 10:05:13,158][53852] Updated weights for policy 0, policy_version 57810 (0.0007) [2023-10-08 10:05:13,164][53885] Updated weights for policy 1, policy_version 57562 (0.0007) [2023-10-08 10:05:13,529][53852] Updated weights for policy 0, policy_version 57820 (0.0008) [2023-10-08 10:05:16,976][53885] Updated weights for policy 1, policy_version 57572 (0.0008) [2023-10-08 10:05:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118161408. Throughput: 0: 1820.5, 1: 1800.9. Samples: 29552452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:17,015][52710] Avg episode reward: [(0, '30.360'), (1, '35.700')] [2023-10-08 10:05:17,259][53852] Updated weights for policy 0, policy_version 57830 (0.0009) [2023-10-08 10:05:17,368][53885] Updated weights for policy 1, policy_version 57582 (0.0008) [2023-10-08 10:05:17,621][53852] Updated weights for policy 0, policy_version 57840 (0.0009) [2023-10-08 10:05:17,738][53885] Updated weights for policy 1, policy_version 57592 (0.0008) [2023-10-08 10:05:17,998][53852] Updated weights for policy 0, policy_version 57850 (0.0008) [2023-10-08 10:05:21,373][53885] Updated weights for policy 1, policy_version 57602 (0.0009) [2023-10-08 10:05:21,741][53885] Updated weights for policy 1, policy_version 57612 (0.0008) [2023-10-08 10:05:21,755][53852] Updated weights for policy 0, policy_version 57860 (0.0008) [2023-10-08 10:05:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 118226944. Throughput: 0: 1822.7, 1: 1819.1. Samples: 29575268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:22,016][52710] Avg episode reward: [(0, '30.400'), (1, '32.920')] [2023-10-08 10:05:22,116][53885] Updated weights for policy 1, policy_version 57622 (0.0008) [2023-10-08 10:05:22,130][53852] Updated weights for policy 0, policy_version 57870 (0.0008) [2023-10-08 10:05:22,471][53885] Updated weights for policy 1, policy_version 57632 (0.0009) [2023-10-08 10:05:22,499][53852] Updated weights for policy 0, policy_version 57880 (0.0008) [2023-10-08 10:05:26,197][53885] Updated weights for policy 1, policy_version 57642 (0.0008) [2023-10-08 10:05:26,233][53852] Updated weights for policy 0, policy_version 57890 (0.0007) [2023-10-08 10:05:26,558][53885] Updated weights for policy 1, policy_version 57652 (0.0008) [2023-10-08 10:05:26,632][53852] Updated weights for policy 0, policy_version 57900 (0.0007) [2023-10-08 10:05:26,924][53885] Updated weights for policy 1, policy_version 57662 (0.0007) [2023-10-08 10:05:27,005][53852] Updated weights for policy 0, policy_version 57910 (0.0009) [2023-10-08 10:05:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 118325248. Throughput: 0: 1831.9, 1: 1816.0. Samples: 29585686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:27,016][52710] Avg episode reward: [(0, '29.430'), (1, '34.170')] [2023-10-08 10:05:27,371][53852] Updated weights for policy 0, policy_version 57920 (0.0008) [2023-10-08 10:05:30,636][53885] Updated weights for policy 1, policy_version 57672 (0.0007) [2023-10-08 10:05:30,894][53852] Updated weights for policy 0, policy_version 57930 (0.0010) [2023-10-08 10:05:31,003][53885] Updated weights for policy 1, policy_version 57682 (0.0008) [2023-10-08 10:05:31,254][53852] Updated weights for policy 0, policy_version 57940 (0.0009) [2023-10-08 10:05:31,367][53885] Updated weights for policy 1, policy_version 57692 (0.0007) [2023-10-08 10:05:31,625][53852] Updated weights for policy 0, policy_version 57950 (0.0008) [2023-10-08 10:05:32,015][52710] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 118423552. Throughput: 0: 1826.6, 1: 1821.4. Samples: 29608314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:32,016][52710] Avg episode reward: [(0, '29.370'), (1, '35.490')] [2023-10-08 10:05:34,996][53885] Updated weights for policy 1, policy_version 57702 (0.0008) [2023-10-08 10:05:35,238][53852] Updated weights for policy 0, policy_version 57960 (0.0010) [2023-10-08 10:05:35,360][53885] Updated weights for policy 1, policy_version 57712 (0.0008) [2023-10-08 10:05:35,612][53852] Updated weights for policy 0, policy_version 57970 (0.0009) [2023-10-08 10:05:35,720][53885] Updated weights for policy 1, policy_version 57722 (0.0008) [2023-10-08 10:05:35,976][53852] Updated weights for policy 0, policy_version 57980 (0.0007) [2023-10-08 10:05:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 118489088. Throughput: 0: 1830.6, 1: 1819.6. Samples: 29628690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:37,016][52710] Avg episode reward: [(0, '31.720'), (1, '34.730')] [2023-10-08 10:05:39,487][53885] Updated weights for policy 1, policy_version 57732 (0.0008) [2023-10-08 10:05:39,681][53852] Updated weights for policy 0, policy_version 57990 (0.0008) [2023-10-08 10:05:39,856][53885] Updated weights for policy 1, policy_version 57742 (0.0008) [2023-10-08 10:05:40,052][53852] Updated weights for policy 0, policy_version 58000 (0.0008) [2023-10-08 10:05:40,229][53885] Updated weights for policy 1, policy_version 57752 (0.0008) [2023-10-08 10:05:40,408][53852] Updated weights for policy 0, policy_version 58010 (0.0008) [2023-10-08 10:05:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118554624. Throughput: 0: 1827.8, 1: 1822.5. Samples: 29641048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:42,015][52710] Avg episode reward: [(0, '27.330'), (1, '32.540')] [2023-10-08 10:05:43,907][53885] Updated weights for policy 1, policy_version 57762 (0.0007) [2023-10-08 10:05:44,173][53852] Updated weights for policy 0, policy_version 58020 (0.0008) [2023-10-08 10:05:44,270][53885] Updated weights for policy 1, policy_version 57772 (0.0008) [2023-10-08 10:05:44,548][53852] Updated weights for policy 0, policy_version 58030 (0.0008) [2023-10-08 10:05:44,648][53885] Updated weights for policy 1, policy_version 57782 (0.0008) [2023-10-08 10:05:44,913][53852] Updated weights for policy 0, policy_version 58040 (0.0009) [2023-10-08 10:05:45,021][53885] Updated weights for policy 1, policy_version 57792 (0.0008) [2023-10-08 10:05:47,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 118620160. Throughput: 0: 1826.7, 1: 1828.9. Samples: 29661298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:47,016][52710] Avg episode reward: [(0, '27.200'), (1, '33.810')] [2023-10-08 10:05:48,586][53885] Updated weights for policy 1, policy_version 57802 (0.0007) [2023-10-08 10:05:48,787][53852] Updated weights for policy 0, policy_version 58050 (0.0008) [2023-10-08 10:05:48,941][53885] Updated weights for policy 1, policy_version 57812 (0.0008) [2023-10-08 10:05:49,160][53852] Updated weights for policy 0, policy_version 58060 (0.0008) [2023-10-08 10:05:49,313][53885] Updated weights for policy 1, policy_version 57822 (0.0009) [2023-10-08 10:05:49,525][53852] Updated weights for policy 0, policy_version 58070 (0.0007) [2023-10-08 10:05:49,898][53852] Updated weights for policy 0, policy_version 58080 (0.0007) [2023-10-08 10:05:52,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 118685696. Throughput: 0: 1826.9, 1: 1837.0. Samples: 29684388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:52,016][52710] Avg episode reward: [(0, '27.900'), (1, '33.840')] [2023-10-08 10:05:52,905][53885] Updated weights for policy 1, policy_version 57832 (0.0008) [2023-10-08 10:05:53,275][53885] Updated weights for policy 1, policy_version 57842 (0.0008) [2023-10-08 10:05:53,572][53852] Updated weights for policy 0, policy_version 58090 (0.0008) [2023-10-08 10:05:53,639][53885] Updated weights for policy 1, policy_version 57852 (0.0008) [2023-10-08 10:05:53,931][53852] Updated weights for policy 0, policy_version 58100 (0.0010) [2023-10-08 10:05:54,312][53852] Updated weights for policy 0, policy_version 58110 (0.0009) [2023-10-08 10:05:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118751232. Throughput: 0: 1825.6, 1: 1835.5. Samples: 29694346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:05:57,016][52710] Avg episode reward: [(0, '24.110'), (1, '33.380')] [2023-10-08 10:05:57,488][53885] Updated weights for policy 1, policy_version 57862 (0.0010) [2023-10-08 10:05:57,844][53852] Updated weights for policy 0, policy_version 58120 (0.0011) [2023-10-08 10:05:57,859][53885] Updated weights for policy 1, policy_version 57872 (0.0008) [2023-10-08 10:05:58,216][53852] Updated weights for policy 0, policy_version 58130 (0.0007) [2023-10-08 10:05:58,220][53885] Updated weights for policy 1, policy_version 57882 (0.0008) [2023-10-08 10:05:58,583][53852] Updated weights for policy 0, policy_version 58140 (0.0007) [2023-10-08 10:06:01,883][53885] Updated weights for policy 1, policy_version 57892 (0.0007) [2023-10-08 10:06:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118816768. Throughput: 0: 1825.3, 1: 1832.5. Samples: 29717056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:06:02,016][52710] Avg episode reward: [(0, '24.580'), (1, '33.990')] [2023-10-08 10:06:02,183][53852] Updated weights for policy 0, policy_version 58150 (0.0007) [2023-10-08 10:06:02,264][53885] Updated weights for policy 1, policy_version 57902 (0.0008) [2023-10-08 10:06:02,549][53852] Updated weights for policy 0, policy_version 58160 (0.0008) [2023-10-08 10:06:02,631][53885] Updated weights for policy 1, policy_version 57912 (0.0007) [2023-10-08 10:06:02,918][53852] Updated weights for policy 0, policy_version 58170 (0.0007) [2023-10-08 10:06:06,436][53885] Updated weights for policy 1, policy_version 57922 (0.0008) [2023-10-08 10:06:06,547][53852] Updated weights for policy 0, policy_version 58180 (0.0009) [2023-10-08 10:06:06,813][53885] Updated weights for policy 1, policy_version 57932 (0.0008) [2023-10-08 10:06:06,909][53852] Updated weights for policy 0, policy_version 58190 (0.0007) [2023-10-08 10:06:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118882304. Throughput: 0: 1824.2, 1: 1822.9. Samples: 29739390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:06:07,016][52710] Avg episode reward: [(0, '26.290'), (1, '34.750')] [2023-10-08 10:06:07,176][53885] Updated weights for policy 1, policy_version 57942 (0.0010) [2023-10-08 10:06:07,283][53852] Updated weights for policy 0, policy_version 58200 (0.0007) [2023-10-08 10:06:07,531][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000057952_59342848.pth... [2023-10-08 10:06:07,535][53885] Updated weights for policy 1, policy_version 57952 (0.0009) [2023-10-08 10:06:07,563][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000056224_57573376.pth [2023-10-08 10:06:07,571][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000058208_59604992.pth... [2023-10-08 10:06:07,600][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000056480_57835520.pth [2023-10-08 10:06:10,919][53852] Updated weights for policy 0, policy_version 58210 (0.0007) [2023-10-08 10:06:11,327][53885] Updated weights for policy 1, policy_version 57962 (0.0008) [2023-10-08 10:06:11,333][53852] Updated weights for policy 0, policy_version 58220 (0.0008) [2023-10-08 10:06:11,688][53885] Updated weights for policy 1, policy_version 57972 (0.0008) [2023-10-08 10:06:11,701][53852] Updated weights for policy 0, policy_version 58230 (0.0007) [2023-10-08 10:06:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 118947840. Throughput: 0: 1826.7, 1: 1820.6. Samples: 29749814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:06:12,016][52710] Avg episode reward: [(0, '25.020'), (1, '34.330')] [2023-10-08 10:06:12,060][53885] Updated weights for policy 1, policy_version 57982 (0.0008) [2023-10-08 10:06:12,061][53852] Updated weights for policy 0, policy_version 58240 (0.0007) [2023-10-08 10:06:15,596][53852] Updated weights for policy 0, policy_version 58250 (0.0008) [2023-10-08 10:06:15,665][53885] Updated weights for policy 1, policy_version 57992 (0.0007) [2023-10-08 10:06:15,969][53852] Updated weights for policy 0, policy_version 58260 (0.0007) [2023-10-08 10:06:16,042][53885] Updated weights for policy 1, policy_version 58002 (0.0007) [2023-10-08 10:06:16,336][53852] Updated weights for policy 0, policy_version 58270 (0.0009) [2023-10-08 10:06:16,405][53885] Updated weights for policy 1, policy_version 58012 (0.0008) [2023-10-08 10:06:17,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 119078912. Throughput: 0: 1827.7, 1: 1818.3. Samples: 29772388. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:06:17,016][52710] Avg episode reward: [(0, '27.340'), (1, '33.970')] [2023-10-08 10:06:19,618][53852] Updated weights for policy 0, policy_version 58280 (0.0007) [2023-10-08 10:06:19,991][53852] Updated weights for policy 0, policy_version 58290 (0.0008) [2023-10-08 10:06:20,121][53885] Updated weights for policy 1, policy_version 58022 (0.0009) [2023-10-08 10:06:20,355][53852] Updated weights for policy 0, policy_version 58300 (0.0010) [2023-10-08 10:06:20,484][53885] Updated weights for policy 1, policy_version 58032 (0.0007) [2023-10-08 10:06:20,852][53885] Updated weights for policy 1, policy_version 58042 (0.0010) [2023-10-08 10:06:22,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 119144448. Throughput: 0: 1839.2, 1: 1812.4. Samples: 29793010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:06:22,016][52710] Avg episode reward: [(0, '27.310'), (1, '32.110')] [2023-10-08 10:06:24,032][53852] Updated weights for policy 0, policy_version 58310 (0.0009) [2023-10-08 10:06:24,408][53852] Updated weights for policy 0, policy_version 58320 (0.0008) [2023-10-08 10:06:24,674][53885] Updated weights for policy 1, policy_version 58052 (0.0009) [2023-10-08 10:06:24,780][53852] Updated weights for policy 0, policy_version 58330 (0.0008) [2023-10-08 10:06:25,043][53885] Updated weights for policy 1, policy_version 58062 (0.0008) [2023-10-08 10:06:25,405][53885] Updated weights for policy 1, policy_version 58072 (0.0008) [2023-10-08 10:06:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119209984. Throughput: 0: 1826.7, 1: 1816.1. Samples: 29804974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:06:27,016][52710] Avg episode reward: [(0, '29.800'), (1, '34.400')] [2023-10-08 10:06:28,309][53852] Updated weights for policy 0, policy_version 58340 (0.0009) [2023-10-08 10:06:28,676][53852] Updated weights for policy 0, policy_version 58350 (0.0008) [2023-10-08 10:06:29,042][53852] Updated weights for policy 0, policy_version 58360 (0.0010) [2023-10-08 10:06:29,110][53885] Updated weights for policy 1, policy_version 58082 (0.0008) [2023-10-08 10:06:29,475][53885] Updated weights for policy 1, policy_version 58092 (0.0007) [2023-10-08 10:06:29,841][53885] Updated weights for policy 1, policy_version 58102 (0.0007) [2023-10-08 10:06:30,206][53885] Updated weights for policy 1, policy_version 58112 (0.0009) [2023-10-08 10:06:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 119275520. Throughput: 0: 1852.4, 1: 1806.6. Samples: 29825956. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:06:32,015][52710] Avg episode reward: [(0, '30.240'), (1, '37.450')] [2023-10-08 10:06:32,016][53594] Saving new best policy, reward=37.450! [2023-10-08 10:06:32,739][53852] Updated weights for policy 0, policy_version 58370 (0.0009) [2023-10-08 10:06:33,109][53852] Updated weights for policy 0, policy_version 58380 (0.0007) [2023-10-08 10:06:33,479][53852] Updated weights for policy 0, policy_version 58390 (0.0007) [2023-10-08 10:06:33,839][53852] Updated weights for policy 0, policy_version 58400 (0.0007) [2023-10-08 10:06:33,892][53885] Updated weights for policy 1, policy_version 58122 (0.0008) [2023-10-08 10:06:34,252][53885] Updated weights for policy 1, policy_version 58132 (0.0007) [2023-10-08 10:06:34,627][53885] Updated weights for policy 1, policy_version 58142 (0.0008) [2023-10-08 10:06:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119341056. Throughput: 0: 1860.6, 1: 1801.1. Samples: 29849164. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:06:37,016][52710] Avg episode reward: [(0, '29.440'), (1, '35.980')] [2023-10-08 10:06:37,466][53852] Updated weights for policy 0, policy_version 58410 (0.0008) [2023-10-08 10:06:37,832][53852] Updated weights for policy 0, policy_version 58420 (0.0008) [2023-10-08 10:06:38,200][53852] Updated weights for policy 0, policy_version 58430 (0.0008) [2023-10-08 10:06:38,280][53885] Updated weights for policy 1, policy_version 58152 (0.0007) [2023-10-08 10:06:38,642][53885] Updated weights for policy 1, policy_version 58162 (0.0008) [2023-10-08 10:06:39,014][53885] Updated weights for policy 1, policy_version 58172 (0.0008) [2023-10-08 10:06:41,889][53852] Updated weights for policy 0, policy_version 58440 (0.0009) [2023-10-08 10:06:42,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 119406592. Throughput: 0: 1861.5, 1: 1805.5. Samples: 29859358. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:06:42,016][52710] Avg episode reward: [(0, '28.540'), (1, '31.940')] [2023-10-08 10:06:42,258][53852] Updated weights for policy 0, policy_version 58450 (0.0008) [2023-10-08 10:06:42,628][53852] Updated weights for policy 0, policy_version 58460 (0.0007) [2023-10-08 10:06:42,775][53885] Updated weights for policy 1, policy_version 58182 (0.0007) [2023-10-08 10:06:43,147][53885] Updated weights for policy 1, policy_version 58192 (0.0010) [2023-10-08 10:06:43,514][53885] Updated weights for policy 1, policy_version 58202 (0.0008) [2023-10-08 10:06:46,435][53852] Updated weights for policy 0, policy_version 58470 (0.0007) [2023-10-08 10:06:46,806][53852] Updated weights for policy 0, policy_version 58480 (0.0009) [2023-10-08 10:06:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 119472128. Throughput: 0: 1860.0, 1: 1810.8. Samples: 29882242. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:06:47,015][52710] Avg episode reward: [(0, '29.430'), (1, '34.910')] [2023-10-08 10:06:47,110][53885] Updated weights for policy 1, policy_version 58212 (0.0008) [2023-10-08 10:06:47,175][53852] Updated weights for policy 0, policy_version 58490 (0.0007) [2023-10-08 10:06:47,502][53885] Updated weights for policy 1, policy_version 58222 (0.0010) [2023-10-08 10:06:47,869][53885] Updated weights for policy 1, policy_version 58232 (0.0009) [2023-10-08 10:06:50,944][53852] Updated weights for policy 0, policy_version 58500 (0.0008) [2023-10-08 10:06:51,315][53852] Updated weights for policy 0, policy_version 58510 (0.0008) [2023-10-08 10:06:51,566][53885] Updated weights for policy 1, policy_version 58242 (0.0008) [2023-10-08 10:06:51,679][53852] Updated weights for policy 0, policy_version 58520 (0.0008) [2023-10-08 10:06:51,931][53885] Updated weights for policy 1, policy_version 58252 (0.0007) [2023-10-08 10:06:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 119570432. Throughput: 0: 1836.6, 1: 1824.4. Samples: 29904136. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:06:52,015][52710] Avg episode reward: [(0, '30.170'), (1, '30.860')] [2023-10-08 10:06:52,294][53885] Updated weights for policy 1, policy_version 58262 (0.0008) [2023-10-08 10:06:52,659][53885] Updated weights for policy 1, policy_version 58272 (0.0008) [2023-10-08 10:06:55,179][53852] Updated weights for policy 0, policy_version 58530 (0.0008) [2023-10-08 10:06:55,554][53852] Updated weights for policy 0, policy_version 58540 (0.0009) [2023-10-08 10:06:55,920][53852] Updated weights for policy 0, policy_version 58550 (0.0009) [2023-10-08 10:06:56,288][53852] Updated weights for policy 0, policy_version 58560 (0.0007) [2023-10-08 10:06:56,323][53885] Updated weights for policy 1, policy_version 58282 (0.0009) [2023-10-08 10:06:56,695][53885] Updated weights for policy 1, policy_version 58292 (0.0008) [2023-10-08 10:06:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119635968. Throughput: 0: 1858.1, 1: 1825.6. Samples: 29915580. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:06:57,015][52710] Avg episode reward: [(0, '29.810'), (1, '30.910')] [2023-10-08 10:06:57,055][53885] Updated weights for policy 1, policy_version 58302 (0.0008) [2023-10-08 10:07:00,046][53852] Updated weights for policy 0, policy_version 58570 (0.0009) [2023-10-08 10:07:00,407][53852] Updated weights for policy 0, policy_version 58580 (0.0007) [2023-10-08 10:07:00,533][53885] Updated weights for policy 1, policy_version 58312 (0.0008) [2023-10-08 10:07:00,782][53852] Updated weights for policy 0, policy_version 58590 (0.0007) [2023-10-08 10:07:00,902][53885] Updated weights for policy 1, policy_version 58322 (0.0008) [2023-10-08 10:07:01,274][53885] Updated weights for policy 1, policy_version 58332 (0.0007) [2023-10-08 10:07:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 119734272. Throughput: 0: 1832.0, 1: 1826.7. Samples: 29937028. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:07:02,016][52710] Avg episode reward: [(0, '29.200'), (1, '34.130')] [2023-10-08 10:07:04,346][53852] Updated weights for policy 0, policy_version 58600 (0.0007) [2023-10-08 10:07:04,706][53852] Updated weights for policy 0, policy_version 58610 (0.0007) [2023-10-08 10:07:04,879][53885] Updated weights for policy 1, policy_version 58342 (0.0007) [2023-10-08 10:07:05,073][53852] Updated weights for policy 0, policy_version 58620 (0.0007) [2023-10-08 10:07:05,249][53885] Updated weights for policy 1, policy_version 58352 (0.0007) [2023-10-08 10:07:05,616][53885] Updated weights for policy 1, policy_version 58362 (0.0008) [2023-10-08 10:07:07,015][52710] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 119799808. Throughput: 0: 1840.8, 1: 1844.1. Samples: 29958834. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:07:07,016][52710] Avg episode reward: [(0, '28.230'), (1, '36.400')] [2023-10-08 10:07:08,841][53852] Updated weights for policy 0, policy_version 58630 (0.0009) [2023-10-08 10:07:09,165][53885] Updated weights for policy 1, policy_version 58372 (0.0008) [2023-10-08 10:07:09,196][53852] Updated weights for policy 0, policy_version 58640 (0.0008) [2023-10-08 10:07:09,520][53885] Updated weights for policy 1, policy_version 58382 (0.0009) [2023-10-08 10:07:09,560][53852] Updated weights for policy 0, policy_version 58650 (0.0007) [2023-10-08 10:07:09,889][53885] Updated weights for policy 1, policy_version 58392 (0.0008) [2023-10-08 10:07:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 119865344. Throughput: 0: 1829.9, 1: 1833.6. Samples: 29969832. Policy #0 lag: (min: 26.0, avg: 27.8, max: 56.0) [2023-10-08 10:07:12,016][52710] Avg episode reward: [(0, '28.670'), (1, '32.560')] [2023-10-08 10:07:13,162][53852] Updated weights for policy 0, policy_version 58660 (0.0008) [2023-10-08 10:07:13,529][53852] Updated weights for policy 0, policy_version 58670 (0.0009) [2023-10-08 10:07:13,655][53885] Updated weights for policy 1, policy_version 58402 (0.0007) [2023-10-08 10:07:13,898][53852] Updated weights for policy 0, policy_version 58680 (0.0009) [2023-10-08 10:07:14,025][53885] Updated weights for policy 1, policy_version 58412 (0.0008) [2023-10-08 10:07:14,391][53885] Updated weights for policy 1, policy_version 58422 (0.0007) [2023-10-08 10:07:14,759][53885] Updated weights for policy 1, policy_version 58432 (0.0008) [2023-10-08 10:07:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 119930880. Throughput: 0: 1831.6, 1: 1846.5. Samples: 29991470. Policy #0 lag: (min: 26.0, avg: 27.8, max: 56.0) [2023-10-08 10:07:17,015][52710] Avg episode reward: [(0, '25.770'), (1, '30.330')] [2023-10-08 10:07:17,534][53852] Updated weights for policy 0, policy_version 58690 (0.0007) [2023-10-08 10:07:17,899][53852] Updated weights for policy 0, policy_version 58700 (0.0007) [2023-10-08 10:07:18,263][53852] Updated weights for policy 0, policy_version 58710 (0.0007) [2023-10-08 10:07:18,381][53885] Updated weights for policy 1, policy_version 58442 (0.0009) [2023-10-08 10:07:18,626][53852] Updated weights for policy 0, policy_version 58720 (0.0010) [2023-10-08 10:07:18,743][53885] Updated weights for policy 1, policy_version 58452 (0.0008) [2023-10-08 10:07:19,105][53885] Updated weights for policy 1, policy_version 58462 (0.0009) [2023-10-08 10:07:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 119996416. Throughput: 0: 1830.6, 1: 1848.2. Samples: 30014708. Policy #0 lag: (min: 26.0, avg: 27.8, max: 56.0) [2023-10-08 10:07:22,015][52710] Avg episode reward: [(0, '25.860'), (1, '34.200')] [2023-10-08 10:07:22,309][53852] Updated weights for policy 0, policy_version 58730 (0.0009) [2023-10-08 10:07:22,669][53852] Updated weights for policy 0, policy_version 58740 (0.0007) [2023-10-08 10:07:22,814][53885] Updated weights for policy 1, policy_version 58472 (0.0008) [2023-10-08 10:07:23,028][53852] Updated weights for policy 0, policy_version 58750 (0.0007) [2023-10-08 10:07:23,180][53885] Updated weights for policy 1, policy_version 58482 (0.0007) [2023-10-08 10:07:23,544][53885] Updated weights for policy 1, policy_version 58492 (0.0008) [2023-10-08 10:07:26,708][53852] Updated weights for policy 0, policy_version 58760 (0.0009) [2023-10-08 10:07:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120061952. Throughput: 0: 1828.7, 1: 1843.5. Samples: 30024606. Policy #0 lag: (min: 26.0, avg: 27.8, max: 56.0) [2023-10-08 10:07:27,016][52710] Avg episode reward: [(0, '26.420'), (1, '31.640')] [2023-10-08 10:07:27,074][53852] Updated weights for policy 0, policy_version 58770 (0.0009) [2023-10-08 10:07:27,349][53885] Updated weights for policy 1, policy_version 58502 (0.0009) [2023-10-08 10:07:27,446][53852] Updated weights for policy 0, policy_version 58780 (0.0008) [2023-10-08 10:07:27,715][53885] Updated weights for policy 1, policy_version 58512 (0.0008) [2023-10-08 10:07:28,078][53885] Updated weights for policy 1, policy_version 58522 (0.0008) [2023-10-08 10:07:31,056][53852] Updated weights for policy 0, policy_version 58790 (0.0009) [2023-10-08 10:07:31,429][53852] Updated weights for policy 0, policy_version 58800 (0.0009) [2023-10-08 10:07:31,802][53852] Updated weights for policy 0, policy_version 58810 (0.0009) [2023-10-08 10:07:31,883][53885] Updated weights for policy 1, policy_version 58532 (0.0010) [2023-10-08 10:07:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 120127488. Throughput: 0: 1837.9, 1: 1838.9. Samples: 30047700. Policy #0 lag: (min: 26.0, avg: 27.8, max: 56.0) [2023-10-08 10:07:32,016][52710] Avg episode reward: [(0, '26.890'), (1, '31.010')] [2023-10-08 10:07:32,270][53885] Updated weights for policy 1, policy_version 58542 (0.0009) [2023-10-08 10:07:32,638][53885] Updated weights for policy 1, policy_version 58552 (0.0007) [2023-10-08 10:07:35,485][53852] Updated weights for policy 0, policy_version 58820 (0.0007) [2023-10-08 10:07:35,849][53852] Updated weights for policy 0, policy_version 58830 (0.0007) [2023-10-08 10:07:36,220][53852] Updated weights for policy 0, policy_version 58840 (0.0008) [2023-10-08 10:07:36,260][53885] Updated weights for policy 1, policy_version 58562 (0.0007) [2023-10-08 10:07:36,634][53885] Updated weights for policy 1, policy_version 58572 (0.0008) [2023-10-08 10:07:36,990][53885] Updated weights for policy 1, policy_version 58582 (0.0008) [2023-10-08 10:07:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120225792. Throughput: 0: 1825.6, 1: 1827.7. Samples: 30068534. Policy #0 lag: (min: 26.0, avg: 27.8, max: 56.0) [2023-10-08 10:07:37,016][52710] Avg episode reward: [(0, '26.690'), (1, '33.890')] [2023-10-08 10:07:37,357][53885] Updated weights for policy 1, policy_version 58592 (0.0007) [2023-10-08 10:07:39,927][53852] Updated weights for policy 0, policy_version 58850 (0.0008) [2023-10-08 10:07:40,292][53852] Updated weights for policy 0, policy_version 58860 (0.0008) [2023-10-08 10:07:40,669][53852] Updated weights for policy 0, policy_version 58870 (0.0008) [2023-10-08 10:07:40,938][53885] Updated weights for policy 1, policy_version 58602 (0.0008) [2023-10-08 10:07:41,035][53852] Updated weights for policy 0, policy_version 58880 (0.0007) [2023-10-08 10:07:41,303][53885] Updated weights for policy 1, policy_version 58612 (0.0008) [2023-10-08 10:07:41,661][53885] Updated weights for policy 1, policy_version 58622 (0.0008) [2023-10-08 10:07:42,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 120324096. Throughput: 0: 1830.1, 1: 1832.0. Samples: 30080376. Policy #0 lag: (min: 26.0, avg: 27.8, max: 56.0) [2023-10-08 10:07:42,016][52710] Avg episode reward: [(0, '27.510'), (1, '36.180')] [2023-10-08 10:07:44,598][53852] Updated weights for policy 0, policy_version 58890 (0.0007) [2023-10-08 10:07:44,971][53852] Updated weights for policy 0, policy_version 58900 (0.0007) [2023-10-08 10:07:45,341][53852] Updated weights for policy 0, policy_version 58910 (0.0008) [2023-10-08 10:07:45,363][53885] Updated weights for policy 1, policy_version 58632 (0.0007) [2023-10-08 10:07:45,739][53885] Updated weights for policy 1, policy_version 58642 (0.0008) [2023-10-08 10:07:46,100][53885] Updated weights for policy 1, policy_version 58652 (0.0009) [2023-10-08 10:07:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 120389632. Throughput: 0: 1824.9, 1: 1824.0. Samples: 30101230. Policy #0 lag: (min: 26.0, avg: 27.8, max: 56.0) [2023-10-08 10:07:47,016][52710] Avg episode reward: [(0, '28.900'), (1, '35.190')] [2023-10-08 10:07:48,874][53852] Updated weights for policy 0, policy_version 58920 (0.0009) [2023-10-08 10:07:49,247][53852] Updated weights for policy 0, policy_version 58930 (0.0011) [2023-10-08 10:07:49,607][53852] Updated weights for policy 0, policy_version 58940 (0.0009) [2023-10-08 10:07:49,676][53885] Updated weights for policy 1, policy_version 58662 (0.0008) [2023-10-08 10:07:50,042][53885] Updated weights for policy 1, policy_version 58672 (0.0007) [2023-10-08 10:07:50,409][53885] Updated weights for policy 1, policy_version 58682 (0.0010) [2023-10-08 10:07:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120455168. Throughput: 0: 1841.5, 1: 1823.8. Samples: 30123774. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 10:07:52,016][52710] Avg episode reward: [(0, '27.060'), (1, '36.300')] [2023-10-08 10:07:53,310][53852] Updated weights for policy 0, policy_version 58950 (0.0009) [2023-10-08 10:07:53,676][53852] Updated weights for policy 0, policy_version 58960 (0.0009) [2023-10-08 10:07:54,000][53885] Updated weights for policy 1, policy_version 58692 (0.0008) [2023-10-08 10:07:54,045][53852] Updated weights for policy 0, policy_version 58970 (0.0007) [2023-10-08 10:07:54,364][53885] Updated weights for policy 1, policy_version 58702 (0.0008) [2023-10-08 10:07:54,729][53885] Updated weights for policy 1, policy_version 58712 (0.0008) [2023-10-08 10:07:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120520704. Throughput: 0: 1834.7, 1: 1821.2. Samples: 30134348. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 10:07:57,015][52710] Avg episode reward: [(0, '27.580'), (1, '36.880')] [2023-10-08 10:07:57,698][53852] Updated weights for policy 0, policy_version 58980 (0.0007) [2023-10-08 10:07:58,063][53852] Updated weights for policy 0, policy_version 58990 (0.0007) [2023-10-08 10:07:58,435][53852] Updated weights for policy 0, policy_version 59000 (0.0007) [2023-10-08 10:07:58,460][53885] Updated weights for policy 1, policy_version 58722 (0.0007) [2023-10-08 10:07:58,829][53885] Updated weights for policy 1, policy_version 58732 (0.0008) [2023-10-08 10:07:59,195][53885] Updated weights for policy 1, policy_version 58742 (0.0018) [2023-10-08 10:07:59,556][53885] Updated weights for policy 1, policy_version 58752 (0.0008) [2023-10-08 10:08:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 120586240. Throughput: 0: 1848.7, 1: 1828.5. Samples: 30156944. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 10:08:02,015][52710] Avg episode reward: [(0, '27.610'), (1, '33.470')] [2023-10-08 10:08:02,065][53852] Updated weights for policy 0, policy_version 59010 (0.0008) [2023-10-08 10:08:02,437][53852] Updated weights for policy 0, policy_version 59020 (0.0007) [2023-10-08 10:08:02,803][53852] Updated weights for policy 0, policy_version 59030 (0.0007) [2023-10-08 10:08:03,175][53852] Updated weights for policy 0, policy_version 59040 (0.0008) [2023-10-08 10:08:03,276][53885] Updated weights for policy 1, policy_version 58762 (0.0007) [2023-10-08 10:08:03,639][53885] Updated weights for policy 1, policy_version 58772 (0.0007) [2023-10-08 10:08:04,001][53885] Updated weights for policy 1, policy_version 58782 (0.0008) [2023-10-08 10:08:06,829][53852] Updated weights for policy 0, policy_version 59050 (0.0007) [2023-10-08 10:08:07,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120651776. Throughput: 0: 1845.4, 1: 1825.1. Samples: 30179884. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 10:08:07,016][52710] Avg episode reward: [(0, '27.940'), (1, '35.870')] [2023-10-08 10:08:07,026][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000058784_60194816.pth... [2023-10-08 10:08:07,063][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000057088_58458112.pth [2023-10-08 10:08:07,206][53852] Updated weights for policy 0, policy_version 59060 (0.0008) [2023-10-08 10:08:07,577][53852] Updated weights for policy 0, policy_version 59070 (0.0007) [2023-10-08 10:08:07,650][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000059072_60489728.pth... [2023-10-08 10:08:07,671][53885] Updated weights for policy 1, policy_version 58792 (0.0008) [2023-10-08 10:08:07,683][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000057344_58720256.pth [2023-10-08 10:08:08,030][53885] Updated weights for policy 1, policy_version 58802 (0.0009) [2023-10-08 10:08:08,402][53885] Updated weights for policy 1, policy_version 58812 (0.0008) [2023-10-08 10:08:11,158][53852] Updated weights for policy 0, policy_version 59080 (0.0007) [2023-10-08 10:08:11,534][53852] Updated weights for policy 0, policy_version 59090 (0.0008) [2023-10-08 10:08:11,903][53852] Updated weights for policy 0, policy_version 59100 (0.0009) [2023-10-08 10:08:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 120717312. Throughput: 0: 1850.1, 1: 1826.0. Samples: 30190032. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 10:08:12,015][52710] Avg episode reward: [(0, '26.660'), (1, '35.640')] [2023-10-08 10:08:12,050][53885] Updated weights for policy 1, policy_version 58822 (0.0010) [2023-10-08 10:08:12,422][53885] Updated weights for policy 1, policy_version 58832 (0.0007) [2023-10-08 10:08:12,786][53885] Updated weights for policy 1, policy_version 58842 (0.0008) [2023-10-08 10:08:15,477][53852] Updated weights for policy 0, policy_version 59110 (0.0009) [2023-10-08 10:08:15,850][53852] Updated weights for policy 0, policy_version 59120 (0.0011) [2023-10-08 10:08:16,214][53852] Updated weights for policy 0, policy_version 59130 (0.0009) [2023-10-08 10:08:16,625][53885] Updated weights for policy 1, policy_version 58852 (0.0008) [2023-10-08 10:08:17,012][53885] Updated weights for policy 1, policy_version 58862 (0.0009) [2023-10-08 10:08:17,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120815616. Throughput: 0: 1840.1, 1: 1828.6. Samples: 30212792. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 10:08:17,015][52710] Avg episode reward: [(0, '28.490'), (1, '31.000')] [2023-10-08 10:08:17,383][53885] Updated weights for policy 1, policy_version 58872 (0.0007) [2023-10-08 10:08:19,802][53852] Updated weights for policy 0, policy_version 59140 (0.0007) [2023-10-08 10:08:20,179][53852] Updated weights for policy 0, policy_version 59150 (0.0009) [2023-10-08 10:08:20,536][53852] Updated weights for policy 0, policy_version 59160 (0.0009) [2023-10-08 10:08:21,002][53885] Updated weights for policy 1, policy_version 58882 (0.0008) [2023-10-08 10:08:21,368][53885] Updated weights for policy 1, policy_version 58892 (0.0010) [2023-10-08 10:08:21,737][53885] Updated weights for policy 1, policy_version 58902 (0.0010) [2023-10-08 10:08:22,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 120881152. Throughput: 0: 1851.7, 1: 1818.0. Samples: 30233670. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 10:08:22,016][52710] Avg episode reward: [(0, '28.300'), (1, '34.600')] [2023-10-08 10:08:22,096][53885] Updated weights for policy 1, policy_version 58912 (0.0011) [2023-10-08 10:08:24,218][53852] Updated weights for policy 0, policy_version 59170 (0.0010) [2023-10-08 10:08:24,589][53852] Updated weights for policy 0, policy_version 59180 (0.0007) [2023-10-08 10:08:24,959][53852] Updated weights for policy 0, policy_version 59190 (0.0009) [2023-10-08 10:08:25,322][53852] Updated weights for policy 0, policy_version 59200 (0.0010) [2023-10-08 10:08:25,782][53885] Updated weights for policy 1, policy_version 58922 (0.0010) [2023-10-08 10:08:26,158][53885] Updated weights for policy 1, policy_version 58932 (0.0011) [2023-10-08 10:08:26,516][53885] Updated weights for policy 1, policy_version 58942 (0.0009) [2023-10-08 10:08:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 120979456. Throughput: 0: 1846.6, 1: 1826.5. Samples: 30245666. Policy #0 lag: (min: 31.0, avg: 36.3, max: 63.0) [2023-10-08 10:08:27,016][52710] Avg episode reward: [(0, '27.880'), (1, '38.150')] [2023-10-08 10:08:27,016][53594] Saving new best policy, reward=38.150! [2023-10-08 10:08:28,973][53852] Updated weights for policy 0, policy_version 59210 (0.0010) [2023-10-08 10:08:29,344][53852] Updated weights for policy 0, policy_version 59220 (0.0010) [2023-10-08 10:08:29,713][53852] Updated weights for policy 0, policy_version 59230 (0.0008) [2023-10-08 10:08:29,975][53885] Updated weights for policy 1, policy_version 58952 (0.0008) [2023-10-08 10:08:30,333][53885] Updated weights for policy 1, policy_version 58962 (0.0008) [2023-10-08 10:08:30,709][53885] Updated weights for policy 1, policy_version 58972 (0.0007) [2023-10-08 10:08:32,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 121044992. Throughput: 0: 1855.0, 1: 1821.1. Samples: 30266654. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:08:32,015][52710] Avg episode reward: [(0, '28.170'), (1, '33.030')] [2023-10-08 10:08:33,283][53852] Updated weights for policy 0, policy_version 59240 (0.0009) [2023-10-08 10:08:33,659][53852] Updated weights for policy 0, policy_version 59250 (0.0007) [2023-10-08 10:08:34,033][53852] Updated weights for policy 0, policy_version 59260 (0.0009) [2023-10-08 10:08:34,456][53885] Updated weights for policy 1, policy_version 58982 (0.0009) [2023-10-08 10:08:34,824][53885] Updated weights for policy 1, policy_version 58992 (0.0010) [2023-10-08 10:08:35,199][53885] Updated weights for policy 1, policy_version 59002 (0.0009) [2023-10-08 10:08:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121110528. Throughput: 0: 1853.3, 1: 1829.3. Samples: 30289492. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:08:37,016][52710] Avg episode reward: [(0, '28.240'), (1, '34.670')] [2023-10-08 10:08:37,714][53852] Updated weights for policy 0, policy_version 59270 (0.0008) [2023-10-08 10:08:38,082][53852] Updated weights for policy 0, policy_version 59280 (0.0009) [2023-10-08 10:08:38,451][53852] Updated weights for policy 0, policy_version 59290 (0.0009) [2023-10-08 10:08:38,820][53885] Updated weights for policy 1, policy_version 59012 (0.0008) [2023-10-08 10:08:39,180][53885] Updated weights for policy 1, policy_version 59022 (0.0007) [2023-10-08 10:08:39,555][53885] Updated weights for policy 1, policy_version 59032 (0.0007) [2023-10-08 10:08:42,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 121176064. Throughput: 0: 1853.7, 1: 1825.2. Samples: 30299900. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:08:42,016][52710] Avg episode reward: [(0, '27.940'), (1, '37.730')] [2023-10-08 10:08:42,215][53852] Updated weights for policy 0, policy_version 59300 (0.0007) [2023-10-08 10:08:42,591][53852] Updated weights for policy 0, policy_version 59310 (0.0007) [2023-10-08 10:08:42,966][53852] Updated weights for policy 0, policy_version 59320 (0.0007) [2023-10-08 10:08:43,236][53885] Updated weights for policy 1, policy_version 59042 (0.0008) [2023-10-08 10:08:43,600][53885] Updated weights for policy 1, policy_version 59052 (0.0009) [2023-10-08 10:08:43,964][53885] Updated weights for policy 1, policy_version 59062 (0.0011) [2023-10-08 10:08:44,335][53885] Updated weights for policy 1, policy_version 59072 (0.0008) [2023-10-08 10:08:46,496][53852] Updated weights for policy 0, policy_version 59330 (0.0009) [2023-10-08 10:08:46,864][53852] Updated weights for policy 0, policy_version 59340 (0.0009) [2023-10-08 10:08:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 121241600. Throughput: 0: 1851.0, 1: 1829.6. Samples: 30322572. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:08:47,016][52710] Avg episode reward: [(0, '27.560'), (1, '34.280')] [2023-10-08 10:08:47,240][53852] Updated weights for policy 0, policy_version 59350 (0.0009) [2023-10-08 10:08:47,601][53852] Updated weights for policy 0, policy_version 59360 (0.0011) [2023-10-08 10:08:48,095][53885] Updated weights for policy 1, policy_version 59082 (0.0009) [2023-10-08 10:08:48,477][53885] Updated weights for policy 1, policy_version 59092 (0.0007) [2023-10-08 10:08:48,840][53885] Updated weights for policy 1, policy_version 59102 (0.0007) [2023-10-08 10:08:51,110][53852] Updated weights for policy 0, policy_version 59370 (0.0009) [2023-10-08 10:08:51,475][53852] Updated weights for policy 0, policy_version 59380 (0.0010) [2023-10-08 10:08:51,839][53852] Updated weights for policy 0, policy_version 59390 (0.0009) [2023-10-08 10:08:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121339904. Throughput: 0: 1831.0, 1: 1822.0. Samples: 30344266. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:08:52,016][52710] Avg episode reward: [(0, '30.590'), (1, '35.070')] [2023-10-08 10:08:52,594][53885] Updated weights for policy 1, policy_version 59112 (0.0008) [2023-10-08 10:08:52,951][53885] Updated weights for policy 1, policy_version 59122 (0.0008) [2023-10-08 10:08:53,328][53885] Updated weights for policy 1, policy_version 59132 (0.0009) [2023-10-08 10:08:55,429][53852] Updated weights for policy 0, policy_version 59400 (0.0011) [2023-10-08 10:08:55,804][53852] Updated weights for policy 0, policy_version 59410 (0.0010) [2023-10-08 10:08:56,179][53852] Updated weights for policy 0, policy_version 59420 (0.0010) [2023-10-08 10:08:57,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121405440. Throughput: 0: 1854.2, 1: 1820.7. Samples: 30355402. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:08:57,015][52710] Avg episode reward: [(0, '29.130'), (1, '34.810')] [2023-10-08 10:08:57,108][53885] Updated weights for policy 1, policy_version 59142 (0.0009) [2023-10-08 10:08:57,481][53885] Updated weights for policy 1, policy_version 59152 (0.0011) [2023-10-08 10:08:57,853][53885] Updated weights for policy 1, policy_version 59162 (0.0010) [2023-10-08 10:08:59,909][53852] Updated weights for policy 0, policy_version 59430 (0.0007) [2023-10-08 10:09:00,280][53852] Updated weights for policy 0, policy_version 59440 (0.0009) [2023-10-08 10:09:00,650][53852] Updated weights for policy 0, policy_version 59450 (0.0009) [2023-10-08 10:09:01,637][53885] Updated weights for policy 1, policy_version 59172 (0.0009) [2023-10-08 10:09:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 121470976. Throughput: 0: 1832.1, 1: 1819.2. Samples: 30377102. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:09:02,016][52710] Avg episode reward: [(0, '28.700'), (1, '33.180')] [2023-10-08 10:09:02,032][53885] Updated weights for policy 1, policy_version 59182 (0.0010) [2023-10-08 10:09:02,407][53885] Updated weights for policy 1, policy_version 59192 (0.0008) [2023-10-08 10:09:04,283][53852] Updated weights for policy 0, policy_version 59460 (0.0009) [2023-10-08 10:09:04,650][53852] Updated weights for policy 0, policy_version 59470 (0.0008) [2023-10-08 10:09:05,011][53852] Updated weights for policy 0, policy_version 59480 (0.0010) [2023-10-08 10:09:06,194][53885] Updated weights for policy 1, policy_version 59202 (0.0010) [2023-10-08 10:09:06,563][53885] Updated weights for policy 1, policy_version 59212 (0.0010) [2023-10-08 10:09:06,948][53885] Updated weights for policy 1, policy_version 59222 (0.0010) [2023-10-08 10:09:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 121536512. Throughput: 0: 1855.9, 1: 1821.5. Samples: 30399154. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:09:07,016][52710] Avg episode reward: [(0, '27.360'), (1, '32.280')] [2023-10-08 10:09:07,308][53885] Updated weights for policy 1, policy_version 59232 (0.0009) [2023-10-08 10:09:08,456][53852] Updated weights for policy 0, policy_version 59490 (0.0008) [2023-10-08 10:09:08,829][53852] Updated weights for policy 0, policy_version 59500 (0.0007) [2023-10-08 10:09:09,209][53852] Updated weights for policy 0, policy_version 59510 (0.0007) [2023-10-08 10:09:09,569][53852] Updated weights for policy 0, policy_version 59520 (0.0009) [2023-10-08 10:09:11,023][53885] Updated weights for policy 1, policy_version 59242 (0.0007) [2023-10-08 10:09:11,393][53885] Updated weights for policy 1, policy_version 59252 (0.0007) [2023-10-08 10:09:11,760][53885] Updated weights for policy 1, policy_version 59262 (0.0007) [2023-10-08 10:09:12,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 121634816. Throughput: 0: 1833.6, 1: 1814.9. Samples: 30409852. Policy #0 lag: (min: 13.0, avg: 37.5, max: 40.0) [2023-10-08 10:09:12,016][52710] Avg episode reward: [(0, '29.460'), (1, '34.400')] [2023-10-08 10:09:13,211][53852] Updated weights for policy 0, policy_version 59530 (0.0009) [2023-10-08 10:09:13,583][53852] Updated weights for policy 0, policy_version 59540 (0.0008) [2023-10-08 10:09:13,947][53852] Updated weights for policy 0, policy_version 59550 (0.0007) [2023-10-08 10:09:15,306][53885] Updated weights for policy 1, policy_version 59272 (0.0009) [2023-10-08 10:09:15,670][53885] Updated weights for policy 1, policy_version 59282 (0.0009) [2023-10-08 10:09:16,033][53885] Updated weights for policy 1, policy_version 59292 (0.0010) [2023-10-08 10:09:17,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121700352. Throughput: 0: 1855.5, 1: 1828.7. Samples: 30432442. Policy #0 lag: (min: 13.0, avg: 37.5, max: 40.0) [2023-10-08 10:09:17,016][52710] Avg episode reward: [(0, '27.270'), (1, '34.800')] [2023-10-08 10:09:17,706][53852] Updated weights for policy 0, policy_version 59560 (0.0009) [2023-10-08 10:09:18,088][53852] Updated weights for policy 0, policy_version 59570 (0.0010) [2023-10-08 10:09:18,447][53852] Updated weights for policy 0, policy_version 59580 (0.0007) [2023-10-08 10:09:19,680][53885] Updated weights for policy 1, policy_version 59302 (0.0009) [2023-10-08 10:09:20,049][53885] Updated weights for policy 1, policy_version 59312 (0.0009) [2023-10-08 10:09:20,415][53885] Updated weights for policy 1, policy_version 59322 (0.0011) [2023-10-08 10:09:21,875][53852] Updated weights for policy 0, policy_version 59590 (0.0010) [2023-10-08 10:09:22,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 121765888. Throughput: 0: 1854.5, 1: 1820.3. Samples: 30454856. Policy #0 lag: (min: 13.0, avg: 37.5, max: 40.0) [2023-10-08 10:09:22,016][52710] Avg episode reward: [(0, '28.410'), (1, '33.090')] [2023-10-08 10:09:22,246][53852] Updated weights for policy 0, policy_version 59600 (0.0010) [2023-10-08 10:09:22,622][53852] Updated weights for policy 0, policy_version 59610 (0.0007) [2023-10-08 10:09:24,064][53885] Updated weights for policy 1, policy_version 59332 (0.0008) [2023-10-08 10:09:24,438][53885] Updated weights for policy 1, policy_version 59342 (0.0008) [2023-10-08 10:09:24,802][53885] Updated weights for policy 1, policy_version 59352 (0.0009) [2023-10-08 10:09:26,170][53852] Updated weights for policy 0, policy_version 59620 (0.0007) [2023-10-08 10:09:26,539][53852] Updated weights for policy 0, policy_version 59630 (0.0007) [2023-10-08 10:09:26,921][53852] Updated weights for policy 0, policy_version 59640 (0.0008) [2023-10-08 10:09:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 121831424. Throughput: 0: 1859.9, 1: 1825.1. Samples: 30465724. Policy #0 lag: (min: 13.0, avg: 37.5, max: 40.0) [2023-10-08 10:09:27,016][52710] Avg episode reward: [(0, '29.260'), (1, '36.570')] [2023-10-08 10:09:28,481][53885] Updated weights for policy 1, policy_version 59362 (0.0008) [2023-10-08 10:09:28,848][53885] Updated weights for policy 1, policy_version 59372 (0.0009) [2023-10-08 10:09:29,207][53885] Updated weights for policy 1, policy_version 59382 (0.0009) [2023-10-08 10:09:29,575][53885] Updated weights for policy 1, policy_version 59392 (0.0007) [2023-10-08 10:09:30,646][53852] Updated weights for policy 0, policy_version 59650 (0.0008) [2023-10-08 10:09:31,026][53852] Updated weights for policy 0, policy_version 59660 (0.0007) [2023-10-08 10:09:31,401][53852] Updated weights for policy 0, policy_version 59670 (0.0007) [2023-10-08 10:09:31,772][53852] Updated weights for policy 0, policy_version 59680 (0.0009) [2023-10-08 10:09:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 121929728. Throughput: 0: 1858.7, 1: 1820.2. Samples: 30488122. Policy #0 lag: (min: 13.0, avg: 37.5, max: 40.0) [2023-10-08 10:09:32,016][52710] Avg episode reward: [(0, '28.130'), (1, '34.230')] [2023-10-08 10:09:33,165][53885] Updated weights for policy 1, policy_version 59402 (0.0009) [2023-10-08 10:09:33,535][53885] Updated weights for policy 1, policy_version 59412 (0.0008) [2023-10-08 10:09:33,904][53885] Updated weights for policy 1, policy_version 59422 (0.0007) [2023-10-08 10:09:35,309][53852] Updated weights for policy 0, policy_version 59690 (0.0009) [2023-10-08 10:09:35,674][53852] Updated weights for policy 0, policy_version 59700 (0.0009) [2023-10-08 10:09:36,047][53852] Updated weights for policy 0, policy_version 59710 (0.0010) [2023-10-08 10:09:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121995264. Throughput: 0: 1848.7, 1: 1836.5. Samples: 30510100. Policy #0 lag: (min: 13.0, avg: 37.5, max: 40.0) [2023-10-08 10:09:37,016][52710] Avg episode reward: [(0, '27.100'), (1, '31.810')] [2023-10-08 10:09:37,516][53885] Updated weights for policy 1, policy_version 59432 (0.0009) [2023-10-08 10:09:37,891][53885] Updated weights for policy 1, policy_version 59442 (0.0009) [2023-10-08 10:09:38,258][53885] Updated weights for policy 1, policy_version 59452 (0.0009) [2023-10-08 10:09:39,709][53852] Updated weights for policy 0, policy_version 59720 (0.0008) [2023-10-08 10:09:40,084][53852] Updated weights for policy 0, policy_version 59730 (0.0009) [2023-10-08 10:09:40,453][53852] Updated weights for policy 0, policy_version 59740 (0.0010) [2023-10-08 10:09:41,941][53885] Updated weights for policy 1, policy_version 59462 (0.0010) [2023-10-08 10:09:42,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122060800. Throughput: 0: 1852.9, 1: 1834.5. Samples: 30521338. Policy #0 lag: (min: 13.0, avg: 37.5, max: 40.0) [2023-10-08 10:09:42,016][52710] Avg episode reward: [(0, '27.740'), (1, '34.650')] [2023-10-08 10:09:42,303][53885] Updated weights for policy 1, policy_version 59472 (0.0009) [2023-10-08 10:09:42,678][53885] Updated weights for policy 1, policy_version 59482 (0.0011) [2023-10-08 10:09:44,008][53852] Updated weights for policy 0, policy_version 59750 (0.0009) [2023-10-08 10:09:44,381][53852] Updated weights for policy 0, policy_version 59760 (0.0010) [2023-10-08 10:09:44,768][53852] Updated weights for policy 0, policy_version 59770 (0.0010) [2023-10-08 10:09:46,351][53885] Updated weights for policy 1, policy_version 59492 (0.0008) [2023-10-08 10:09:46,742][53885] Updated weights for policy 1, policy_version 59502 (0.0009) [2023-10-08 10:09:47,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 122126336. Throughput: 0: 1848.7, 1: 1838.3. Samples: 30543016. Policy #0 lag: (min: 13.0, avg: 37.5, max: 40.0) [2023-10-08 10:09:47,015][52710] Avg episode reward: [(0, '27.280'), (1, '34.070')] [2023-10-08 10:09:47,113][53885] Updated weights for policy 1, policy_version 59512 (0.0008) [2023-10-08 10:09:48,510][53852] Updated weights for policy 0, policy_version 59780 (0.0009) [2023-10-08 10:09:48,873][53852] Updated weights for policy 0, policy_version 59790 (0.0009) [2023-10-08 10:09:49,238][53852] Updated weights for policy 0, policy_version 59800 (0.0007) [2023-10-08 10:09:50,671][53885] Updated weights for policy 1, policy_version 59522 (0.0010) [2023-10-08 10:09:51,036][53885] Updated weights for policy 1, policy_version 59532 (0.0010) [2023-10-08 10:09:51,406][53885] Updated weights for policy 1, policy_version 59542 (0.0008) [2023-10-08 10:09:51,774][53885] Updated weights for policy 1, policy_version 59552 (0.0007) [2023-10-08 10:09:52,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 122224640. Throughput: 0: 1847.2, 1: 1827.0. Samples: 30564494. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:09:52,016][52710] Avg episode reward: [(0, '25.850'), (1, '31.090')] [2023-10-08 10:09:52,997][53852] Updated weights for policy 0, policy_version 59810 (0.0008) [2023-10-08 10:09:53,359][53852] Updated weights for policy 0, policy_version 59820 (0.0011) [2023-10-08 10:09:53,741][53852] Updated weights for policy 0, policy_version 59830 (0.0007) [2023-10-08 10:09:54,108][53852] Updated weights for policy 0, policy_version 59840 (0.0007) [2023-10-08 10:09:55,298][53885] Updated weights for policy 1, policy_version 59562 (0.0011) [2023-10-08 10:09:55,666][53885] Updated weights for policy 1, policy_version 59572 (0.0010) [2023-10-08 10:09:56,031][53885] Updated weights for policy 1, policy_version 59582 (0.0008) [2023-10-08 10:09:57,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 122290176. Throughput: 0: 1841.8, 1: 1846.1. Samples: 30575810. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:09:57,016][52710] Avg episode reward: [(0, '27.100'), (1, '36.100')] [2023-10-08 10:09:57,831][53852] Updated weights for policy 0, policy_version 59850 (0.0007) [2023-10-08 10:09:58,194][53852] Updated weights for policy 0, policy_version 59860 (0.0009) [2023-10-08 10:09:58,557][53852] Updated weights for policy 0, policy_version 59870 (0.0007) [2023-10-08 10:09:59,700][53885] Updated weights for policy 1, policy_version 59592 (0.0007) [2023-10-08 10:10:00,068][53885] Updated weights for policy 1, policy_version 59602 (0.0007) [2023-10-08 10:10:00,439][53885] Updated weights for policy 1, policy_version 59612 (0.0009) [2023-10-08 10:10:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122355712. Throughput: 0: 1843.5, 1: 1822.2. Samples: 30597396. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:10:02,016][52710] Avg episode reward: [(0, '25.900'), (1, '38.390')] [2023-10-08 10:10:02,016][53594] Saving new best policy, reward=38.390! [2023-10-08 10:10:02,289][53852] Updated weights for policy 0, policy_version 59880 (0.0008) [2023-10-08 10:10:02,659][53852] Updated weights for policy 0, policy_version 59890 (0.0008) [2023-10-08 10:10:03,033][53852] Updated weights for policy 0, policy_version 59900 (0.0007) [2023-10-08 10:10:04,056][53885] Updated weights for policy 1, policy_version 59622 (0.0009) [2023-10-08 10:10:04,427][53885] Updated weights for policy 1, policy_version 59632 (0.0008) [2023-10-08 10:10:04,790][53885] Updated weights for policy 1, policy_version 59642 (0.0009) [2023-10-08 10:10:06,740][53852] Updated weights for policy 0, policy_version 59910 (0.0008) [2023-10-08 10:10:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122421248. Throughput: 0: 1834.5, 1: 1834.6. Samples: 30619966. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:10:07,016][52710] Avg episode reward: [(0, '25.680'), (1, '32.440')] [2023-10-08 10:10:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000059648_61079552.pth... [2023-10-08 10:10:07,055][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000057952_59342848.pth [2023-10-08 10:10:07,059][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000059648_61079552.pth [2023-10-08 10:10:07,127][53852] Updated weights for policy 0, policy_version 59920 (0.0007) [2023-10-08 10:10:07,486][53852] Updated weights for policy 0, policy_version 59930 (0.0007) [2023-10-08 10:10:07,707][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000059936_61374464.pth... [2023-10-08 10:10:07,735][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000058208_59604992.pth [2023-10-08 10:10:07,739][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000059936_61374464.pth [2023-10-08 10:10:08,560][53885] Updated weights for policy 1, policy_version 59652 (0.0008) [2023-10-08 10:10:08,924][53885] Updated weights for policy 1, policy_version 59662 (0.0008) [2023-10-08 10:10:09,298][53885] Updated weights for policy 1, policy_version 59672 (0.0009) [2023-10-08 10:10:11,017][53852] Updated weights for policy 0, policy_version 59940 (0.0008) [2023-10-08 10:10:11,388][53852] Updated weights for policy 0, policy_version 59950 (0.0008) [2023-10-08 10:10:11,765][53852] Updated weights for policy 0, policy_version 59960 (0.0010) [2023-10-08 10:10:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 122486784. Throughput: 0: 1835.9, 1: 1822.3. Samples: 30630342. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:10:12,016][52710] Avg episode reward: [(0, '25.960'), (1, '34.340')] [2023-10-08 10:10:13,108][53885] Updated weights for policy 1, policy_version 59682 (0.0007) [2023-10-08 10:10:13,471][53885] Updated weights for policy 1, policy_version 59692 (0.0008) [2023-10-08 10:10:13,841][53885] Updated weights for policy 1, policy_version 59702 (0.0007) [2023-10-08 10:10:14,204][53885] Updated weights for policy 1, policy_version 59712 (0.0011) [2023-10-08 10:10:15,346][53852] Updated weights for policy 0, policy_version 59970 (0.0010) [2023-10-08 10:10:15,718][53852] Updated weights for policy 0, policy_version 59980 (0.0010) [2023-10-08 10:10:16,092][53852] Updated weights for policy 0, policy_version 59990 (0.0008) [2023-10-08 10:10:16,470][53852] Updated weights for policy 0, policy_version 60000 (0.0007) [2023-10-08 10:10:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 122585088. Throughput: 0: 1830.8, 1: 1832.3. Samples: 30652962. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:10:17,016][52710] Avg episode reward: [(0, '27.620'), (1, '35.800')] [2023-10-08 10:10:17,821][53885] Updated weights for policy 1, policy_version 59722 (0.0007) [2023-10-08 10:10:18,188][53885] Updated weights for policy 1, policy_version 59732 (0.0009) [2023-10-08 10:10:18,558][53885] Updated weights for policy 1, policy_version 59742 (0.0008) [2023-10-08 10:10:20,113][53852] Updated weights for policy 0, policy_version 60010 (0.0007) [2023-10-08 10:10:20,473][53852] Updated weights for policy 0, policy_version 60020 (0.0007) [2023-10-08 10:10:20,842][53852] Updated weights for policy 0, policy_version 60030 (0.0007) [2023-10-08 10:10:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122650624. Throughput: 0: 1832.5, 1: 1830.5. Samples: 30674934. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:10:22,016][52710] Avg episode reward: [(0, '26.660'), (1, '32.770')] [2023-10-08 10:10:22,256][53885] Updated weights for policy 1, policy_version 59752 (0.0008) [2023-10-08 10:10:22,625][53885] Updated weights for policy 1, policy_version 59762 (0.0007) [2023-10-08 10:10:22,994][53885] Updated weights for policy 1, policy_version 59772 (0.0008) [2023-10-08 10:10:24,547][53852] Updated weights for policy 0, policy_version 60040 (0.0009) [2023-10-08 10:10:24,925][53852] Updated weights for policy 0, policy_version 60050 (0.0008) [2023-10-08 10:10:25,286][53852] Updated weights for policy 0, policy_version 60060 (0.0008) [2023-10-08 10:10:26,717][53885] Updated weights for policy 1, policy_version 59782 (0.0009) [2023-10-08 10:10:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 122716160. Throughput: 0: 1828.0, 1: 1831.0. Samples: 30685994. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) [2023-10-08 10:10:27,016][52710] Avg episode reward: [(0, '26.370'), (1, '35.080')] [2023-10-08 10:10:27,079][53885] Updated weights for policy 1, policy_version 59792 (0.0007) [2023-10-08 10:10:27,444][53885] Updated weights for policy 1, policy_version 59802 (0.0009) [2023-10-08 10:10:28,853][53852] Updated weights for policy 0, policy_version 60070 (0.0008) [2023-10-08 10:10:29,226][53852] Updated weights for policy 0, policy_version 60080 (0.0008) [2023-10-08 10:10:29,600][53852] Updated weights for policy 0, policy_version 60090 (0.0008) [2023-10-08 10:10:31,181][53885] Updated weights for policy 1, policy_version 59812 (0.0008) [2023-10-08 10:10:31,553][53885] Updated weights for policy 1, policy_version 59822 (0.0007) [2023-10-08 10:10:31,926][53885] Updated weights for policy 1, policy_version 59832 (0.0009) [2023-10-08 10:10:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 122781696. Throughput: 0: 1833.3, 1: 1827.9. Samples: 30707768. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:10:32,016][52710] Avg episode reward: [(0, '26.390'), (1, '37.220')] [2023-10-08 10:10:33,207][53852] Updated weights for policy 0, policy_version 60100 (0.0010) [2023-10-08 10:10:33,574][53852] Updated weights for policy 0, policy_version 60110 (0.0007) [2023-10-08 10:10:33,941][53852] Updated weights for policy 0, policy_version 60120 (0.0008) [2023-10-08 10:10:35,604][53885] Updated weights for policy 1, policy_version 59842 (0.0008) [2023-10-08 10:10:36,016][53885] Updated weights for policy 1, policy_version 59852 (0.0009) [2023-10-08 10:10:36,388][53885] Updated weights for policy 1, policy_version 59862 (0.0010) [2023-10-08 10:10:36,762][53885] Updated weights for policy 1, policy_version 59872 (0.0007) [2023-10-08 10:10:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122880000. Throughput: 0: 1838.9, 1: 1827.8. Samples: 30729496. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:10:37,016][52710] Avg episode reward: [(0, '28.900'), (1, '33.380')] [2023-10-08 10:10:37,554][53852] Updated weights for policy 0, policy_version 60130 (0.0007) [2023-10-08 10:10:37,926][53852] Updated weights for policy 0, policy_version 60140 (0.0010) [2023-10-08 10:10:38,292][53852] Updated weights for policy 0, policy_version 60150 (0.0008) [2023-10-08 10:10:38,661][53852] Updated weights for policy 0, policy_version 60160 (0.0007) [2023-10-08 10:10:40,245][53885] Updated weights for policy 1, policy_version 59882 (0.0007) [2023-10-08 10:10:40,607][53885] Updated weights for policy 1, policy_version 59892 (0.0011) [2023-10-08 10:10:40,979][53885] Updated weights for policy 1, policy_version 59902 (0.0008) [2023-10-08 10:10:42,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 122945536. Throughput: 0: 1838.7, 1: 1829.7. Samples: 30740888. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:10:42,016][52710] Avg episode reward: [(0, '27.540'), (1, '35.270')] [2023-10-08 10:10:42,293][53852] Updated weights for policy 0, policy_version 60170 (0.0007) [2023-10-08 10:10:42,664][53852] Updated weights for policy 0, policy_version 60180 (0.0007) [2023-10-08 10:10:43,049][53852] Updated weights for policy 0, policy_version 60190 (0.0007) [2023-10-08 10:10:44,485][53885] Updated weights for policy 1, policy_version 59912 (0.0010) [2023-10-08 10:10:44,850][53885] Updated weights for policy 1, policy_version 59922 (0.0010) [2023-10-08 10:10:45,226][53885] Updated weights for policy 1, policy_version 59932 (0.0007) [2023-10-08 10:10:46,957][53852] Updated weights for policy 0, policy_version 60200 (0.0008) [2023-10-08 10:10:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123011072. Throughput: 0: 1834.6, 1: 1829.6. Samples: 30762282. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:10:47,015][52710] Avg episode reward: [(0, '28.060'), (1, '35.980')] [2023-10-08 10:10:47,327][53852] Updated weights for policy 0, policy_version 60210 (0.0008) [2023-10-08 10:10:47,699][53852] Updated weights for policy 0, policy_version 60220 (0.0009) [2023-10-08 10:10:48,737][53885] Updated weights for policy 1, policy_version 59942 (0.0008) [2023-10-08 10:10:49,099][53885] Updated weights for policy 1, policy_version 59952 (0.0011) [2023-10-08 10:10:49,475][53885] Updated weights for policy 1, policy_version 59962 (0.0010) [2023-10-08 10:10:51,330][53852] Updated weights for policy 0, policy_version 60230 (0.0007) [2023-10-08 10:10:51,717][53852] Updated weights for policy 0, policy_version 60240 (0.0008) [2023-10-08 10:10:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 123076608. Throughput: 0: 1822.8, 1: 1837.8. Samples: 30784696. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:10:52,016][52710] Avg episode reward: [(0, '28.300'), (1, '35.360')] [2023-10-08 10:10:52,087][53852] Updated weights for policy 0, policy_version 60250 (0.0007) [2023-10-08 10:10:53,130][53885] Updated weights for policy 1, policy_version 59972 (0.0009) [2023-10-08 10:10:53,502][53885] Updated weights for policy 1, policy_version 59982 (0.0010) [2023-10-08 10:10:53,875][53885] Updated weights for policy 1, policy_version 59992 (0.0009) [2023-10-08 10:10:55,724][53852] Updated weights for policy 0, policy_version 60260 (0.0007) [2023-10-08 10:10:56,094][53852] Updated weights for policy 0, policy_version 60270 (0.0007) [2023-10-08 10:10:56,466][53852] Updated weights for policy 0, policy_version 60280 (0.0009) [2023-10-08 10:10:57,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 123174912. Throughput: 0: 1832.6, 1: 1833.7. Samples: 30795324. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:10:57,016][52710] Avg episode reward: [(0, '27.940'), (1, '33.260')] [2023-10-08 10:10:57,777][53885] Updated weights for policy 1, policy_version 60002 (0.0009) [2023-10-08 10:10:58,143][53885] Updated weights for policy 1, policy_version 60012 (0.0007) [2023-10-08 10:10:58,504][53885] Updated weights for policy 1, policy_version 60022 (0.0007) [2023-10-08 10:10:58,873][53885] Updated weights for policy 1, policy_version 60032 (0.0007) [2023-10-08 10:10:59,993][53852] Updated weights for policy 0, policy_version 60290 (0.0009) [2023-10-08 10:11:00,370][53852] Updated weights for policy 0, policy_version 60300 (0.0008) [2023-10-08 10:11:00,739][53852] Updated weights for policy 0, policy_version 60310 (0.0007) [2023-10-08 10:11:01,101][53852] Updated weights for policy 0, policy_version 60320 (0.0007) [2023-10-08 10:11:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 123240448. Throughput: 0: 1821.3, 1: 1837.2. Samples: 30817596. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:11:02,016][52710] Avg episode reward: [(0, '26.370'), (1, '37.100')] [2023-10-08 10:11:02,597][53885] Updated weights for policy 1, policy_version 60042 (0.0011) [2023-10-08 10:11:02,966][53885] Updated weights for policy 1, policy_version 60052 (0.0007) [2023-10-08 10:11:03,339][53885] Updated weights for policy 1, policy_version 60062 (0.0007) [2023-10-08 10:11:04,784][53852] Updated weights for policy 0, policy_version 60330 (0.0009) [2023-10-08 10:11:05,152][53852] Updated weights for policy 0, policy_version 60340 (0.0008) [2023-10-08 10:11:05,513][53852] Updated weights for policy 0, policy_version 60350 (0.0007) [2023-10-08 10:11:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 123305984. Throughput: 0: 1837.5, 1: 1829.7. Samples: 30839960. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) [2023-10-08 10:11:07,016][52710] Avg episode reward: [(0, '27.410'), (1, '34.530')] [2023-10-08 10:11:07,115][53885] Updated weights for policy 1, policy_version 60072 (0.0009) [2023-10-08 10:11:07,486][53885] Updated weights for policy 1, policy_version 60082 (0.0007) [2023-10-08 10:11:07,841][53885] Updated weights for policy 1, policy_version 60092 (0.0007) [2023-10-08 10:11:09,084][53852] Updated weights for policy 0, policy_version 60360 (0.0011) [2023-10-08 10:11:09,457][53852] Updated weights for policy 0, policy_version 60370 (0.0010) [2023-10-08 10:11:09,824][53852] Updated weights for policy 0, policy_version 60380 (0.0010) [2023-10-08 10:11:11,313][53885] Updated weights for policy 1, policy_version 60102 (0.0008) [2023-10-08 10:11:11,681][53885] Updated weights for policy 1, policy_version 60112 (0.0009) [2023-10-08 10:11:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 123371520. Throughput: 0: 1825.9, 1: 1829.3. Samples: 30850478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:11:12,016][52710] Avg episode reward: [(0, '27.750'), (1, '34.410')] [2023-10-08 10:11:12,054][53885] Updated weights for policy 1, policy_version 60122 (0.0008) [2023-10-08 10:11:13,510][53852] Updated weights for policy 0, policy_version 60390 (0.0010) [2023-10-08 10:11:13,886][53852] Updated weights for policy 0, policy_version 60400 (0.0010) [2023-10-08 10:11:14,244][53852] Updated weights for policy 0, policy_version 60410 (0.0008) [2023-10-08 10:11:15,635][53885] Updated weights for policy 1, policy_version 60132 (0.0009) [2023-10-08 10:11:16,006][53885] Updated weights for policy 1, policy_version 60142 (0.0011) [2023-10-08 10:11:16,368][53885] Updated weights for policy 1, policy_version 60152 (0.0009) [2023-10-08 10:11:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123469824. Throughput: 0: 1834.2, 1: 1832.0. Samples: 30872744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:11:17,016][52710] Avg episode reward: [(0, '26.790'), (1, '36.570')] [2023-10-08 10:11:17,757][53852] Updated weights for policy 0, policy_version 60420 (0.0007) [2023-10-08 10:11:18,121][53852] Updated weights for policy 0, policy_version 60430 (0.0011) [2023-10-08 10:11:18,502][53852] Updated weights for policy 0, policy_version 60440 (0.0008) [2023-10-08 10:11:19,967][53885] Updated weights for policy 1, policy_version 60162 (0.0008) [2023-10-08 10:11:20,333][53885] Updated weights for policy 1, policy_version 60172 (0.0009) [2023-10-08 10:11:20,697][53885] Updated weights for policy 1, policy_version 60182 (0.0009) [2023-10-08 10:11:21,062][53885] Updated weights for policy 1, policy_version 60192 (0.0011) [2023-10-08 10:11:22,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123535360. Throughput: 0: 1836.4, 1: 1836.4. Samples: 30894772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:11:22,016][52710] Avg episode reward: [(0, '26.460'), (1, '33.920')] [2023-10-08 10:11:22,253][53852] Updated weights for policy 0, policy_version 60450 (0.0010) [2023-10-08 10:11:22,627][53852] Updated weights for policy 0, policy_version 60460 (0.0007) [2023-10-08 10:11:22,999][53852] Updated weights for policy 0, policy_version 60470 (0.0007) [2023-10-08 10:11:23,365][53852] Updated weights for policy 0, policy_version 60480 (0.0008) [2023-10-08 10:11:24,856][53885] Updated weights for policy 1, policy_version 60202 (0.0010) [2023-10-08 10:11:25,232][53885] Updated weights for policy 1, policy_version 60212 (0.0010) [2023-10-08 10:11:25,593][53885] Updated weights for policy 1, policy_version 60222 (0.0010) [2023-10-08 10:11:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123600896. Throughput: 0: 1833.5, 1: 1833.6. Samples: 30905906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:11:27,015][52710] Avg episode reward: [(0, '28.580'), (1, '33.550')] [2023-10-08 10:11:27,158][53852] Updated weights for policy 0, policy_version 60490 (0.0009) [2023-10-08 10:11:27,536][53852] Updated weights for policy 0, policy_version 60500 (0.0007) [2023-10-08 10:11:27,909][53852] Updated weights for policy 0, policy_version 60510 (0.0007) [2023-10-08 10:11:29,180][53885] Updated weights for policy 1, policy_version 60232 (0.0008) [2023-10-08 10:11:29,560][53885] Updated weights for policy 1, policy_version 60242 (0.0008) [2023-10-08 10:11:29,927][53885] Updated weights for policy 1, policy_version 60252 (0.0008) [2023-10-08 10:11:31,552][53852] Updated weights for policy 0, policy_version 60520 (0.0007) [2023-10-08 10:11:31,921][53852] Updated weights for policy 0, policy_version 60530 (0.0010) [2023-10-08 10:11:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123666432. Throughput: 0: 1840.1, 1: 1832.7. Samples: 30927556. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:11:32,016][52710] Avg episode reward: [(0, '26.250'), (1, '34.320')] [2023-10-08 10:11:32,292][53852] Updated weights for policy 0, policy_version 60540 (0.0010) [2023-10-08 10:11:33,643][53885] Updated weights for policy 1, policy_version 60262 (0.0009) [2023-10-08 10:11:34,002][53885] Updated weights for policy 1, policy_version 60272 (0.0009) [2023-10-08 10:11:34,380][53885] Updated weights for policy 1, policy_version 60282 (0.0008) [2023-10-08 10:11:35,919][53852] Updated weights for policy 0, policy_version 60550 (0.0008) [2023-10-08 10:11:36,302][53852] Updated weights for policy 0, policy_version 60560 (0.0007) [2023-10-08 10:11:36,674][53852] Updated weights for policy 0, policy_version 60570 (0.0009) [2023-10-08 10:11:37,015][52710] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 123764736. Throughput: 0: 1833.3, 1: 1829.3. Samples: 30949516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:11:37,017][52710] Avg episode reward: [(0, '25.090'), (1, '34.040')] [2023-10-08 10:11:38,170][53885] Updated weights for policy 1, policy_version 60292 (0.0010) [2023-10-08 10:11:38,536][53885] Updated weights for policy 1, policy_version 60302 (0.0007) [2023-10-08 10:11:38,904][53885] Updated weights for policy 1, policy_version 60312 (0.0007) [2023-10-08 10:11:40,075][53852] Updated weights for policy 0, policy_version 60580 (0.0010) [2023-10-08 10:11:40,449][53852] Updated weights for policy 0, policy_version 60590 (0.0008) [2023-10-08 10:11:40,822][53852] Updated weights for policy 0, policy_version 60600 (0.0009) [2023-10-08 10:11:42,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 123830272. Throughput: 0: 1847.5, 1: 1830.4. Samples: 30960832. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:11:42,016][52710] Avg episode reward: [(0, '26.060'), (1, '34.300')] [2023-10-08 10:11:42,406][53885] Updated weights for policy 1, policy_version 60322 (0.0009) [2023-10-08 10:11:42,776][53885] Updated weights for policy 1, policy_version 60332 (0.0008) [2023-10-08 10:11:43,150][53885] Updated weights for policy 1, policy_version 60342 (0.0011) [2023-10-08 10:11:43,507][53885] Updated weights for policy 1, policy_version 60352 (0.0010) [2023-10-08 10:11:44,552][53852] Updated weights for policy 0, policy_version 60610 (0.0009) [2023-10-08 10:11:44,914][53852] Updated weights for policy 0, policy_version 60620 (0.0007) [2023-10-08 10:11:45,288][53852] Updated weights for policy 0, policy_version 60630 (0.0007) [2023-10-08 10:11:45,649][53852] Updated weights for policy 0, policy_version 60640 (0.0008) [2023-10-08 10:11:47,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123895808. Throughput: 0: 1829.4, 1: 1832.3. Samples: 30982372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:11:47,016][52710] Avg episode reward: [(0, '26.200'), (1, '33.680')] [2023-10-08 10:11:47,163][53885] Updated weights for policy 1, policy_version 60362 (0.0009) [2023-10-08 10:11:47,533][53885] Updated weights for policy 1, policy_version 60372 (0.0010) [2023-10-08 10:11:47,906][53885] Updated weights for policy 1, policy_version 60382 (0.0010) [2023-10-08 10:11:49,297][53852] Updated weights for policy 0, policy_version 60650 (0.0007) [2023-10-08 10:11:49,677][53852] Updated weights for policy 0, policy_version 60660 (0.0007) [2023-10-08 10:11:50,056][53852] Updated weights for policy 0, policy_version 60670 (0.0007) [2023-10-08 10:11:51,729][53885] Updated weights for policy 1, policy_version 60392 (0.0008) [2023-10-08 10:11:52,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123961344. Throughput: 0: 1840.4, 1: 1830.2. Samples: 31005138. Policy #0 lag: (min: 29.0, avg: 29.2, max: 37.0) [2023-10-08 10:11:52,016][52710] Avg episode reward: [(0, '26.110'), (1, '33.280')] [2023-10-08 10:11:52,103][53885] Updated weights for policy 1, policy_version 60402 (0.0007) [2023-10-08 10:11:52,465][53885] Updated weights for policy 1, policy_version 60412 (0.0010) [2023-10-08 10:11:53,734][53852] Updated weights for policy 0, policy_version 60680 (0.0010) [2023-10-08 10:11:54,099][53852] Updated weights for policy 0, policy_version 60690 (0.0008) [2023-10-08 10:11:54,468][53852] Updated weights for policy 0, policy_version 60700 (0.0007) [2023-10-08 10:11:55,964][53885] Updated weights for policy 1, policy_version 60422 (0.0009) [2023-10-08 10:11:56,339][53885] Updated weights for policy 1, policy_version 60432 (0.0008) [2023-10-08 10:11:56,706][53885] Updated weights for policy 1, policy_version 60442 (0.0009) [2023-10-08 10:11:57,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124059648. Throughput: 0: 1828.8, 1: 1842.8. Samples: 31015702. Policy #0 lag: (min: 29.0, avg: 29.2, max: 37.0) [2023-10-08 10:11:57,016][52710] Avg episode reward: [(0, '29.640'), (1, '34.490')] [2023-10-08 10:11:58,102][53852] Updated weights for policy 0, policy_version 60710 (0.0008) [2023-10-08 10:11:58,473][53852] Updated weights for policy 0, policy_version 60720 (0.0008) [2023-10-08 10:11:58,848][53852] Updated weights for policy 0, policy_version 60730 (0.0010) [2023-10-08 10:12:00,476][53885] Updated weights for policy 1, policy_version 60452 (0.0008) [2023-10-08 10:12:00,841][53885] Updated weights for policy 1, policy_version 60462 (0.0008) [2023-10-08 10:12:01,214][53885] Updated weights for policy 1, policy_version 60472 (0.0008) [2023-10-08 10:12:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124125184. Throughput: 0: 1838.9, 1: 1831.6. Samples: 31037914. Policy #0 lag: (min: 29.0, avg: 29.2, max: 37.0) [2023-10-08 10:12:02,016][52710] Avg episode reward: [(0, '28.140'), (1, '34.330')] [2023-10-08 10:12:02,561][53852] Updated weights for policy 0, policy_version 60740 (0.0011) [2023-10-08 10:12:02,932][53852] Updated weights for policy 0, policy_version 60750 (0.0010) [2023-10-08 10:12:03,309][53852] Updated weights for policy 0, policy_version 60760 (0.0012) [2023-10-08 10:12:04,956][53885] Updated weights for policy 1, policy_version 60482 (0.0007) [2023-10-08 10:12:05,328][53885] Updated weights for policy 1, policy_version 60492 (0.0009) [2023-10-08 10:12:05,697][53885] Updated weights for policy 1, policy_version 60502 (0.0009) [2023-10-08 10:12:06,053][53885] Updated weights for policy 1, policy_version 60512 (0.0011) [2023-10-08 10:12:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124190720. Throughput: 0: 1833.7, 1: 1826.6. Samples: 31059486. Policy #0 lag: (min: 29.0, avg: 29.2, max: 37.0) [2023-10-08 10:12:07,016][52710] Avg episode reward: [(0, '28.140'), (1, '31.210')] [2023-10-08 10:12:07,026][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000060512_61964288.pth... [2023-10-08 10:12:07,061][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000058784_60194816.pth [2023-10-08 10:12:07,067][53852] Updated weights for policy 0, policy_version 60770 (0.0008) [2023-10-08 10:12:07,449][53852] Updated weights for policy 0, policy_version 60780 (0.0010) [2023-10-08 10:12:07,818][53852] Updated weights for policy 0, policy_version 60790 (0.0008) [2023-10-08 10:12:08,177][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000060800_62259200.pth... [2023-10-08 10:12:08,182][53852] Updated weights for policy 0, policy_version 60800 (0.0008) [2023-10-08 10:12:08,206][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000059072_60489728.pth [2023-10-08 10:12:09,823][53885] Updated weights for policy 1, policy_version 60522 (0.0008) [2023-10-08 10:12:10,184][53885] Updated weights for policy 1, policy_version 60532 (0.0007) [2023-10-08 10:12:10,556][53885] Updated weights for policy 1, policy_version 60542 (0.0010) [2023-10-08 10:12:11,719][53852] Updated weights for policy 0, policy_version 60810 (0.0007) [2023-10-08 10:12:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124256256. Throughput: 0: 1841.1, 1: 1821.9. Samples: 31070742. Policy #0 lag: (min: 29.0, avg: 29.2, max: 37.0) [2023-10-08 10:12:12,016][52710] Avg episode reward: [(0, '28.350'), (1, '32.960')] [2023-10-08 10:12:12,089][53852] Updated weights for policy 0, policy_version 60820 (0.0007) [2023-10-08 10:12:12,464][53852] Updated weights for policy 0, policy_version 60830 (0.0007) [2023-10-08 10:12:14,343][53885] Updated weights for policy 1, policy_version 60552 (0.0010) [2023-10-08 10:12:14,716][53885] Updated weights for policy 1, policy_version 60562 (0.0007) [2023-10-08 10:12:15,080][53885] Updated weights for policy 1, policy_version 60572 (0.0008) [2023-10-08 10:12:15,953][53852] Updated weights for policy 0, policy_version 60840 (0.0010) [2023-10-08 10:12:16,318][53852] Updated weights for policy 0, policy_version 60850 (0.0011) [2023-10-08 10:12:16,690][53852] Updated weights for policy 0, policy_version 60860 (0.0008) [2023-10-08 10:12:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 124354560. Throughput: 0: 1841.8, 1: 1819.4. Samples: 31092310. Policy #0 lag: (min: 29.0, avg: 29.2, max: 37.0) [2023-10-08 10:12:17,016][52710] Avg episode reward: [(0, '30.420'), (1, '30.560')] [2023-10-08 10:12:18,677][53885] Updated weights for policy 1, policy_version 60582 (0.0008) [2023-10-08 10:12:19,045][53885] Updated weights for policy 1, policy_version 60592 (0.0011) [2023-10-08 10:12:19,405][53885] Updated weights for policy 1, policy_version 60602 (0.0009) [2023-10-08 10:12:20,313][53852] Updated weights for policy 0, policy_version 60870 (0.0007) [2023-10-08 10:12:20,689][53852] Updated weights for policy 0, policy_version 60880 (0.0007) [2023-10-08 10:12:21,056][53852] Updated weights for policy 0, policy_version 60890 (0.0009) [2023-10-08 10:12:22,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 124420096. Throughput: 0: 1829.6, 1: 1825.0. Samples: 31113970. Policy #0 lag: (min: 29.0, avg: 29.2, max: 37.0) [2023-10-08 10:12:22,016][52710] Avg episode reward: [(0, '27.610'), (1, '28.150')] [2023-10-08 10:12:23,129][53885] Updated weights for policy 1, policy_version 60612 (0.0010) [2023-10-08 10:12:23,499][53885] Updated weights for policy 1, policy_version 60622 (0.0011) [2023-10-08 10:12:23,871][53885] Updated weights for policy 1, policy_version 60632 (0.0010) [2023-10-08 10:12:24,879][53852] Updated weights for policy 0, policy_version 60900 (0.0009) [2023-10-08 10:12:25,264][53852] Updated weights for policy 0, policy_version 60910 (0.0008) [2023-10-08 10:12:25,641][53852] Updated weights for policy 0, policy_version 60920 (0.0008) [2023-10-08 10:12:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 124485632. Throughput: 0: 1837.6, 1: 1821.6. Samples: 31125496. Policy #0 lag: (min: 29.0, avg: 29.2, max: 37.0) [2023-10-08 10:12:27,016][52710] Avg episode reward: [(0, '27.860'), (1, '28.020')] [2023-10-08 10:12:27,512][53885] Updated weights for policy 1, policy_version 60642 (0.0009) [2023-10-08 10:12:27,886][53885] Updated weights for policy 1, policy_version 60652 (0.0009) [2023-10-08 10:12:28,248][53885] Updated weights for policy 1, policy_version 60662 (0.0008) [2023-10-08 10:12:28,626][53885] Updated weights for policy 1, policy_version 60672 (0.0008) [2023-10-08 10:12:29,383][53852] Updated weights for policy 0, policy_version 60930 (0.0008) [2023-10-08 10:12:29,750][53852] Updated weights for policy 0, policy_version 60940 (0.0009) [2023-10-08 10:12:30,130][53852] Updated weights for policy 0, policy_version 60950 (0.0008) [2023-10-08 10:12:30,498][53852] Updated weights for policy 0, policy_version 60960 (0.0009) [2023-10-08 10:12:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124551168. Throughput: 0: 1830.9, 1: 1823.2. Samples: 31146808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:12:32,016][52710] Avg episode reward: [(0, '30.750'), (1, '31.230')] [2023-10-08 10:12:32,350][53885] Updated weights for policy 1, policy_version 60682 (0.0007) [2023-10-08 10:12:32,724][53885] Updated weights for policy 1, policy_version 60692 (0.0008) [2023-10-08 10:12:33,086][53885] Updated weights for policy 1, policy_version 60702 (0.0008) [2023-10-08 10:12:34,128][53852] Updated weights for policy 0, policy_version 60970 (0.0007) [2023-10-08 10:12:34,495][53852] Updated weights for policy 0, policy_version 60980 (0.0007) [2023-10-08 10:12:34,877][53852] Updated weights for policy 0, policy_version 60990 (0.0007) [2023-10-08 10:12:36,718][53885] Updated weights for policy 1, policy_version 60712 (0.0007) [2023-10-08 10:12:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124616704. Throughput: 0: 1832.3, 1: 1817.1. Samples: 31169360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:12:37,016][52710] Avg episode reward: [(0, '29.670'), (1, '32.210')] [2023-10-08 10:12:37,084][53885] Updated weights for policy 1, policy_version 60722 (0.0007) [2023-10-08 10:12:37,456][53885] Updated weights for policy 1, policy_version 60732 (0.0009) [2023-10-08 10:12:38,396][53852] Updated weights for policy 0, policy_version 61000 (0.0009) [2023-10-08 10:12:38,778][53852] Updated weights for policy 0, policy_version 61010 (0.0009) [2023-10-08 10:12:39,143][53852] Updated weights for policy 0, policy_version 61020 (0.0009) [2023-10-08 10:12:41,112][53885] Updated weights for policy 1, policy_version 60742 (0.0010) [2023-10-08 10:12:41,483][53885] Updated weights for policy 1, policy_version 60752 (0.0009) [2023-10-08 10:12:41,851][53885] Updated weights for policy 1, policy_version 60762 (0.0008) [2023-10-08 10:12:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124682240. Throughput: 0: 1832.9, 1: 1815.0. Samples: 31179858. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:12:42,016][52710] Avg episode reward: [(0, '27.540'), (1, '32.390')] [2023-10-08 10:12:42,842][53852] Updated weights for policy 0, policy_version 61030 (0.0009) [2023-10-08 10:12:43,205][53852] Updated weights for policy 0, policy_version 61040 (0.0007) [2023-10-08 10:12:43,573][53852] Updated weights for policy 0, policy_version 61050 (0.0010) [2023-10-08 10:12:45,416][53885] Updated weights for policy 1, policy_version 60772 (0.0008) [2023-10-08 10:12:45,777][53885] Updated weights for policy 1, policy_version 60782 (0.0008) [2023-10-08 10:12:46,155][53885] Updated weights for policy 1, policy_version 60792 (0.0007) [2023-10-08 10:12:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124780544. Throughput: 0: 1842.1, 1: 1822.2. Samples: 31202808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:12:47,016][52710] Avg episode reward: [(0, '30.050'), (1, '30.850')] [2023-10-08 10:12:47,093][53852] Updated weights for policy 0, policy_version 61060 (0.0007) [2023-10-08 10:12:47,465][53852] Updated weights for policy 0, policy_version 61070 (0.0008) [2023-10-08 10:12:47,837][53852] Updated weights for policy 0, policy_version 61080 (0.0008) [2023-10-08 10:12:49,677][53885] Updated weights for policy 1, policy_version 60802 (0.0007) [2023-10-08 10:12:50,040][53885] Updated weights for policy 1, policy_version 60812 (0.0008) [2023-10-08 10:12:50,412][53885] Updated weights for policy 1, policy_version 60822 (0.0009) [2023-10-08 10:12:50,772][53885] Updated weights for policy 1, policy_version 60832 (0.0008) [2023-10-08 10:12:51,412][53852] Updated weights for policy 0, policy_version 61090 (0.0008) [2023-10-08 10:12:51,782][53852] Updated weights for policy 0, policy_version 61100 (0.0009) [2023-10-08 10:12:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124846080. Throughput: 0: 1840.7, 1: 1835.2. Samples: 31224902. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:12:52,015][52710] Avg episode reward: [(0, '28.830'), (1, '32.090')] [2023-10-08 10:12:52,155][53852] Updated weights for policy 0, policy_version 61110 (0.0009) [2023-10-08 10:12:52,528][53852] Updated weights for policy 0, policy_version 61120 (0.0010) [2023-10-08 10:12:54,492][53885] Updated weights for policy 1, policy_version 60842 (0.0007) [2023-10-08 10:12:54,860][53885] Updated weights for policy 1, policy_version 60852 (0.0008) [2023-10-08 10:12:55,233][53885] Updated weights for policy 1, policy_version 60862 (0.0008) [2023-10-08 10:12:56,272][53852] Updated weights for policy 0, policy_version 61130 (0.0007) [2023-10-08 10:12:56,635][53852] Updated weights for policy 0, policy_version 61140 (0.0008) [2023-10-08 10:12:57,004][53852] Updated weights for policy 0, policy_version 61150 (0.0010) [2023-10-08 10:12:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124911616. Throughput: 0: 1845.9, 1: 1824.7. Samples: 31235920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:12:57,016][52710] Avg episode reward: [(0, '26.490'), (1, '34.330')] [2023-10-08 10:12:58,960][53885] Updated weights for policy 1, policy_version 60872 (0.0009) [2023-10-08 10:12:59,334][53885] Updated weights for policy 1, policy_version 60882 (0.0008) [2023-10-08 10:12:59,704][53885] Updated weights for policy 1, policy_version 60892 (0.0009) [2023-10-08 10:13:00,656][53852] Updated weights for policy 0, policy_version 61160 (0.0009) [2023-10-08 10:13:01,026][53852] Updated weights for policy 0, policy_version 61170 (0.0008) [2023-10-08 10:13:01,396][53852] Updated weights for policy 0, policy_version 61180 (0.0008) [2023-10-08 10:13:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 125009920. Throughput: 0: 1835.1, 1: 1836.6. Samples: 31257538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:02,016][52710] Avg episode reward: [(0, '27.230'), (1, '29.620')] [2023-10-08 10:13:03,271][53885] Updated weights for policy 1, policy_version 60902 (0.0007) [2023-10-08 10:13:03,633][53885] Updated weights for policy 1, policy_version 60912 (0.0008) [2023-10-08 10:13:03,997][53885] Updated weights for policy 1, policy_version 60922 (0.0008) [2023-10-08 10:13:05,010][53852] Updated weights for policy 0, policy_version 61190 (0.0010) [2023-10-08 10:13:05,377][53852] Updated weights for policy 0, policy_version 61200 (0.0010) [2023-10-08 10:13:05,748][53852] Updated weights for policy 0, policy_version 61210 (0.0009) [2023-10-08 10:13:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 125075456. Throughput: 0: 1841.0, 1: 1839.4. Samples: 31279588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:07,016][52710] Avg episode reward: [(0, '29.950'), (1, '31.190')] [2023-10-08 10:13:07,734][53885] Updated weights for policy 1, policy_version 60932 (0.0009) [2023-10-08 10:13:08,106][53885] Updated weights for policy 1, policy_version 60942 (0.0011) [2023-10-08 10:13:08,470][53885] Updated weights for policy 1, policy_version 60952 (0.0008) [2023-10-08 10:13:09,483][53852] Updated weights for policy 0, policy_version 61220 (0.0008) [2023-10-08 10:13:09,874][53852] Updated weights for policy 0, policy_version 61230 (0.0008) [2023-10-08 10:13:10,253][53852] Updated weights for policy 0, policy_version 61240 (0.0007) [2023-10-08 10:13:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125140992. Throughput: 0: 1830.9, 1: 1840.5. Samples: 31290708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:12,016][52710] Avg episode reward: [(0, '28.840'), (1, '34.160')] [2023-10-08 10:13:12,213][53885] Updated weights for policy 1, policy_version 60962 (0.0007) [2023-10-08 10:13:12,570][53885] Updated weights for policy 1, policy_version 60972 (0.0007) [2023-10-08 10:13:12,939][53885] Updated weights for policy 1, policy_version 60982 (0.0008) [2023-10-08 10:13:13,304][53885] Updated weights for policy 1, policy_version 60992 (0.0009) [2023-10-08 10:13:13,935][53852] Updated weights for policy 0, policy_version 61250 (0.0008) [2023-10-08 10:13:14,310][53852] Updated weights for policy 0, policy_version 61260 (0.0010) [2023-10-08 10:13:14,684][53852] Updated weights for policy 0, policy_version 61270 (0.0010) [2023-10-08 10:13:15,055][53852] Updated weights for policy 0, policy_version 61280 (0.0008) [2023-10-08 10:13:17,005][53885] Updated weights for policy 1, policy_version 61002 (0.0008) [2023-10-08 10:13:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 125206528. Throughput: 0: 1839.4, 1: 1838.1. Samples: 31312296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:17,016][52710] Avg episode reward: [(0, '30.950'), (1, '33.300')] [2023-10-08 10:13:17,378][53885] Updated weights for policy 1, policy_version 61012 (0.0009) [2023-10-08 10:13:17,735][53885] Updated weights for policy 1, policy_version 61022 (0.0007) [2023-10-08 10:13:18,572][53852] Updated weights for policy 0, policy_version 61290 (0.0009) [2023-10-08 10:13:18,940][53852] Updated weights for policy 0, policy_version 61300 (0.0010) [2023-10-08 10:13:19,306][53852] Updated weights for policy 0, policy_version 61310 (0.0009) [2023-10-08 10:13:21,388][53885] Updated weights for policy 1, policy_version 61032 (0.0007) [2023-10-08 10:13:21,755][53885] Updated weights for policy 1, policy_version 61042 (0.0008) [2023-10-08 10:13:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 125272064. Throughput: 0: 1842.4, 1: 1834.2. Samples: 31334808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:22,016][52710] Avg episode reward: [(0, '30.900'), (1, '31.730')] [2023-10-08 10:13:22,120][53885] Updated weights for policy 1, policy_version 61052 (0.0007) [2023-10-08 10:13:22,986][53852] Updated weights for policy 0, policy_version 61320 (0.0010) [2023-10-08 10:13:23,355][53852] Updated weights for policy 0, policy_version 61330 (0.0010) [2023-10-08 10:13:23,724][53852] Updated weights for policy 0, policy_version 61340 (0.0010) [2023-10-08 10:13:25,534][53885] Updated weights for policy 1, policy_version 61062 (0.0009) [2023-10-08 10:13:25,906][53885] Updated weights for policy 1, policy_version 61072 (0.0007) [2023-10-08 10:13:26,270][53885] Updated weights for policy 1, policy_version 61082 (0.0007) [2023-10-08 10:13:27,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 125370368. Throughput: 0: 1837.0, 1: 1848.7. Samples: 31345718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:27,016][52710] Avg episode reward: [(0, '30.930'), (1, '36.220')] [2023-10-08 10:13:27,386][53852] Updated weights for policy 0, policy_version 61350 (0.0009) [2023-10-08 10:13:27,758][53852] Updated weights for policy 0, policy_version 61360 (0.0009) [2023-10-08 10:13:28,128][53852] Updated weights for policy 0, policy_version 61370 (0.0008) [2023-10-08 10:13:29,894][53885] Updated weights for policy 1, policy_version 61092 (0.0009) [2023-10-08 10:13:30,261][53885] Updated weights for policy 1, policy_version 61102 (0.0009) [2023-10-08 10:13:30,620][53885] Updated weights for policy 1, policy_version 61112 (0.0007) [2023-10-08 10:13:31,794][53852] Updated weights for policy 0, policy_version 61380 (0.0008) [2023-10-08 10:13:32,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125435904. Throughput: 0: 1837.4, 1: 1833.7. Samples: 31368004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:32,016][52710] Avg episode reward: [(0, '30.020'), (1, '29.380')] [2023-10-08 10:13:32,162][53852] Updated weights for policy 0, policy_version 61390 (0.0009) [2023-10-08 10:13:32,533][53852] Updated weights for policy 0, policy_version 61400 (0.0008) [2023-10-08 10:13:34,161][53885] Updated weights for policy 1, policy_version 61122 (0.0010) [2023-10-08 10:13:34,531][53885] Updated weights for policy 1, policy_version 61132 (0.0007) [2023-10-08 10:13:34,892][53885] Updated weights for policy 1, policy_version 61142 (0.0007) [2023-10-08 10:13:35,253][53885] Updated weights for policy 1, policy_version 61152 (0.0008) [2023-10-08 10:13:36,233][53852] Updated weights for policy 0, policy_version 61410 (0.0007) [2023-10-08 10:13:36,595][53852] Updated weights for policy 0, policy_version 61420 (0.0008) [2023-10-08 10:13:36,962][53852] Updated weights for policy 0, policy_version 61430 (0.0008) [2023-10-08 10:13:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125501440. Throughput: 0: 1828.9, 1: 1847.5. Samples: 31390344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:37,016][52710] Avg episode reward: [(0, '30.060'), (1, '31.990')] [2023-10-08 10:13:37,328][53852] Updated weights for policy 0, policy_version 61440 (0.0009) [2023-10-08 10:13:38,769][53885] Updated weights for policy 1, policy_version 61162 (0.0009) [2023-10-08 10:13:39,133][53885] Updated weights for policy 1, policy_version 61172 (0.0011) [2023-10-08 10:13:39,505][53885] Updated weights for policy 1, policy_version 61182 (0.0008) [2023-10-08 10:13:40,929][53852] Updated weights for policy 0, policy_version 61450 (0.0011) [2023-10-08 10:13:41,301][53852] Updated weights for policy 0, policy_version 61460 (0.0008) [2023-10-08 10:13:41,667][53852] Updated weights for policy 0, policy_version 61470 (0.0008) [2023-10-08 10:13:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 125599744. Throughput: 0: 1835.9, 1: 1834.4. Samples: 31401082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:42,016][52710] Avg episode reward: [(0, '30.890'), (1, '32.420')] [2023-10-08 10:13:43,224][53885] Updated weights for policy 1, policy_version 61192 (0.0010) [2023-10-08 10:13:43,596][53885] Updated weights for policy 1, policy_version 61202 (0.0011) [2023-10-08 10:13:43,954][53885] Updated weights for policy 1, policy_version 61212 (0.0010) [2023-10-08 10:13:45,286][53852] Updated weights for policy 0, policy_version 61480 (0.0009) [2023-10-08 10:13:45,655][53852] Updated weights for policy 0, policy_version 61490 (0.0009) [2023-10-08 10:13:46,020][53852] Updated weights for policy 0, policy_version 61500 (0.0007) [2023-10-08 10:13:47,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125665280. Throughput: 0: 1826.8, 1: 1854.8. Samples: 31423212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:13:47,016][52710] Avg episode reward: [(0, '28.770'), (1, '29.170')] [2023-10-08 10:13:47,726][53885] Updated weights for policy 1, policy_version 61222 (0.0010) [2023-10-08 10:13:48,110][53885] Updated weights for policy 1, policy_version 61232 (0.0011) [2023-10-08 10:13:48,473][53885] Updated weights for policy 1, policy_version 61242 (0.0010) [2023-10-08 10:13:49,614][53852] Updated weights for policy 0, policy_version 61510 (0.0008) [2023-10-08 10:13:49,972][53852] Updated weights for policy 0, policy_version 61520 (0.0009) [2023-10-08 10:13:50,347][53852] Updated weights for policy 0, policy_version 61530 (0.0011) [2023-10-08 10:13:51,975][53885] Updated weights for policy 1, policy_version 61252 (0.0010) [2023-10-08 10:13:52,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125730816. Throughput: 0: 1841.0, 1: 1847.0. Samples: 31445550. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 10:13:52,015][52710] Avg episode reward: [(0, '29.030'), (1, '30.470')] [2023-10-08 10:13:52,339][53885] Updated weights for policy 1, policy_version 61262 (0.0008) [2023-10-08 10:13:52,697][53885] Updated weights for policy 1, policy_version 61272 (0.0007) [2023-10-08 10:13:54,131][53852] Updated weights for policy 0, policy_version 61540 (0.0009) [2023-10-08 10:13:54,522][53852] Updated weights for policy 0, policy_version 61550 (0.0007) [2023-10-08 10:13:54,885][53852] Updated weights for policy 0, policy_version 61560 (0.0007) [2023-10-08 10:13:56,389][53885] Updated weights for policy 1, policy_version 61282 (0.0009) [2023-10-08 10:13:56,744][53885] Updated weights for policy 1, policy_version 61292 (0.0011) [2023-10-08 10:13:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125796352. Throughput: 0: 1830.4, 1: 1850.4. Samples: 31456346. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 10:13:57,016][52710] Avg episode reward: [(0, '26.550'), (1, '31.530')] [2023-10-08 10:13:57,121][53885] Updated weights for policy 1, policy_version 61302 (0.0009) [2023-10-08 10:13:57,481][53885] Updated weights for policy 1, policy_version 61312 (0.0009) [2023-10-08 10:13:58,607][53852] Updated weights for policy 0, policy_version 61570 (0.0008) [2023-10-08 10:13:58,977][53852] Updated weights for policy 0, policy_version 61580 (0.0008) [2023-10-08 10:13:59,351][53852] Updated weights for policy 0, policy_version 61590 (0.0010) [2023-10-08 10:13:59,723][53852] Updated weights for policy 0, policy_version 61600 (0.0009) [2023-10-08 10:14:01,038][53885] Updated weights for policy 1, policy_version 61322 (0.0007) [2023-10-08 10:14:01,409][53885] Updated weights for policy 1, policy_version 61332 (0.0008) [2023-10-08 10:14:01,778][53885] Updated weights for policy 1, policy_version 61342 (0.0010) [2023-10-08 10:14:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 125894656. Throughput: 0: 1837.7, 1: 1852.6. Samples: 31478362. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 10:14:02,016][52710] Avg episode reward: [(0, '28.530'), (1, '28.710')] [2023-10-08 10:14:03,326][53852] Updated weights for policy 0, policy_version 61610 (0.0007) [2023-10-08 10:14:03,694][53852] Updated weights for policy 0, policy_version 61620 (0.0007) [2023-10-08 10:14:04,057][53852] Updated weights for policy 0, policy_version 61630 (0.0009) [2023-10-08 10:14:05,382][53885] Updated weights for policy 1, policy_version 61352 (0.0010) [2023-10-08 10:14:05,755][53885] Updated weights for policy 1, policy_version 61362 (0.0009) [2023-10-08 10:14:06,128][53885] Updated weights for policy 1, policy_version 61372 (0.0009) [2023-10-08 10:14:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125960192. Throughput: 0: 1837.7, 1: 1833.4. Samples: 31500008. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 10:14:07,016][52710] Avg episode reward: [(0, '27.800'), (1, '32.910')] [2023-10-08 10:14:07,030][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000061376_62849024.pth... [2023-10-08 10:14:07,030][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000061632_63111168.pth... [2023-10-08 10:14:07,065][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000059936_61374464.pth [2023-10-08 10:14:07,070][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000059648_61079552.pth [2023-10-08 10:14:07,695][53852] Updated weights for policy 0, policy_version 61640 (0.0008) [2023-10-08 10:14:08,065][53852] Updated weights for policy 0, policy_version 61650 (0.0008) [2023-10-08 10:14:08,441][53852] Updated weights for policy 0, policy_version 61660 (0.0008) [2023-10-08 10:14:09,918][53885] Updated weights for policy 1, policy_version 61382 (0.0008) [2023-10-08 10:14:10,280][53885] Updated weights for policy 1, policy_version 61392 (0.0008) [2023-10-08 10:14:10,651][53885] Updated weights for policy 1, policy_version 61402 (0.0010) [2023-10-08 10:14:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126025728. Throughput: 0: 1841.4, 1: 1845.3. Samples: 31511622. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 10:14:12,016][52710] Avg episode reward: [(0, '30.270'), (1, '30.960')] [2023-10-08 10:14:12,076][53852] Updated weights for policy 0, policy_version 61670 (0.0009) [2023-10-08 10:14:12,441][53852] Updated weights for policy 0, policy_version 61680 (0.0010) [2023-10-08 10:14:12,811][53852] Updated weights for policy 0, policy_version 61690 (0.0008) [2023-10-08 10:14:14,367][53885] Updated weights for policy 1, policy_version 61412 (0.0009) [2023-10-08 10:14:14,727][53885] Updated weights for policy 1, policy_version 61422 (0.0007) [2023-10-08 10:14:15,101][53885] Updated weights for policy 1, policy_version 61432 (0.0009) [2023-10-08 10:14:16,319][53852] Updated weights for policy 0, policy_version 61700 (0.0007) [2023-10-08 10:14:16,691][53852] Updated weights for policy 0, policy_version 61710 (0.0007) [2023-10-08 10:14:17,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126091264. Throughput: 0: 1842.2, 1: 1826.2. Samples: 31533084. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 10:14:17,015][52710] Avg episode reward: [(0, '28.730'), (1, '33.860')] [2023-10-08 10:14:17,060][53852] Updated weights for policy 0, policy_version 61720 (0.0008) [2023-10-08 10:14:18,748][53885] Updated weights for policy 1, policy_version 61442 (0.0007) [2023-10-08 10:14:19,114][53885] Updated weights for policy 1, policy_version 61452 (0.0007) [2023-10-08 10:14:19,472][53885] Updated weights for policy 1, policy_version 61462 (0.0008) [2023-10-08 10:14:19,839][53885] Updated weights for policy 1, policy_version 61472 (0.0009) [2023-10-08 10:14:20,801][53852] Updated weights for policy 0, policy_version 61730 (0.0007) [2023-10-08 10:14:21,169][53852] Updated weights for policy 0, policy_version 61740 (0.0008) [2023-10-08 10:14:21,535][53852] Updated weights for policy 0, policy_version 61750 (0.0007) [2023-10-08 10:14:21,911][53852] Updated weights for policy 0, policy_version 61760 (0.0008) [2023-10-08 10:14:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 126189568. Throughput: 0: 1826.1, 1: 1835.8. Samples: 31555128. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 10:14:22,015][52710] Avg episode reward: [(0, '29.250'), (1, '36.250')] [2023-10-08 10:14:23,440][53885] Updated weights for policy 1, policy_version 61482 (0.0008) [2023-10-08 10:14:23,799][53885] Updated weights for policy 1, policy_version 61492 (0.0008) [2023-10-08 10:14:24,167][53885] Updated weights for policy 1, policy_version 61502 (0.0008) [2023-10-08 10:14:25,613][53852] Updated weights for policy 0, policy_version 61770 (0.0007) [2023-10-08 10:14:25,983][53852] Updated weights for policy 0, policy_version 61780 (0.0007) [2023-10-08 10:14:26,351][53852] Updated weights for policy 0, policy_version 61790 (0.0007) [2023-10-08 10:14:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126255104. Throughput: 0: 1835.3, 1: 1833.6. Samples: 31566184. Policy #0 lag: (min: 31.0, avg: 34.3, max: 63.0) [2023-10-08 10:14:27,016][52710] Avg episode reward: [(0, '31.050'), (1, '35.470')] [2023-10-08 10:14:27,781][53885] Updated weights for policy 1, policy_version 61512 (0.0008) [2023-10-08 10:14:28,155][53885] Updated weights for policy 1, policy_version 61522 (0.0009) [2023-10-08 10:14:28,513][53885] Updated weights for policy 1, policy_version 61532 (0.0008) [2023-10-08 10:14:30,047][53852] Updated weights for policy 0, policy_version 61800 (0.0009) [2023-10-08 10:14:30,423][53852] Updated weights for policy 0, policy_version 61810 (0.0008) [2023-10-08 10:14:30,790][53852] Updated weights for policy 0, policy_version 61820 (0.0008) [2023-10-08 10:14:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126320640. Throughput: 0: 1829.1, 1: 1841.4. Samples: 31588384. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 10:14:32,016][52710] Avg episode reward: [(0, '29.120'), (1, '33.780')] [2023-10-08 10:14:32,264][53885] Updated weights for policy 1, policy_version 61542 (0.0009) [2023-10-08 10:14:32,645][53885] Updated weights for policy 1, policy_version 61552 (0.0009) [2023-10-08 10:14:33,013][53885] Updated weights for policy 1, policy_version 61562 (0.0007) [2023-10-08 10:14:34,359][53852] Updated weights for policy 0, policy_version 61830 (0.0009) [2023-10-08 10:14:34,727][53852] Updated weights for policy 0, policy_version 61840 (0.0008) [2023-10-08 10:14:35,103][53852] Updated weights for policy 0, policy_version 61850 (0.0009) [2023-10-08 10:14:36,708][53885] Updated weights for policy 1, policy_version 61572 (0.0008) [2023-10-08 10:14:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 126386176. Throughput: 0: 1835.2, 1: 1838.0. Samples: 31610844. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 10:14:37,016][52710] Avg episode reward: [(0, '31.620'), (1, '31.750')] [2023-10-08 10:14:37,100][53885] Updated weights for policy 1, policy_version 61582 (0.0007) [2023-10-08 10:14:37,464][53885] Updated weights for policy 1, policy_version 61592 (0.0008) [2023-10-08 10:14:38,719][53852] Updated weights for policy 0, policy_version 61860 (0.0007) [2023-10-08 10:14:39,085][53852] Updated weights for policy 0, policy_version 61870 (0.0007) [2023-10-08 10:14:39,454][53852] Updated weights for policy 0, policy_version 61880 (0.0007) [2023-10-08 10:14:40,886][53885] Updated weights for policy 1, policy_version 61602 (0.0008) [2023-10-08 10:14:41,270][53885] Updated weights for policy 1, policy_version 61612 (0.0009) [2023-10-08 10:14:41,629][53885] Updated weights for policy 1, policy_version 61622 (0.0008) [2023-10-08 10:14:41,994][53885] Updated weights for policy 1, policy_version 61632 (0.0007) [2023-10-08 10:14:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 126484480. Throughput: 0: 1828.5, 1: 1840.8. Samples: 31621464. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 10:14:42,015][52710] Avg episode reward: [(0, '28.150'), (1, '33.220')] [2023-10-08 10:14:43,085][53852] Updated weights for policy 0, policy_version 61890 (0.0009) [2023-10-08 10:14:43,452][53852] Updated weights for policy 0, policy_version 61900 (0.0009) [2023-10-08 10:14:43,815][53852] Updated weights for policy 0, policy_version 61910 (0.0008) [2023-10-08 10:14:44,186][53852] Updated weights for policy 0, policy_version 61920 (0.0010) [2023-10-08 10:14:45,741][53885] Updated weights for policy 1, policy_version 61642 (0.0010) [2023-10-08 10:14:46,117][53885] Updated weights for policy 1, policy_version 61652 (0.0008) [2023-10-08 10:14:46,481][53885] Updated weights for policy 1, policy_version 61662 (0.0008) [2023-10-08 10:14:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126550016. Throughput: 0: 1849.5, 1: 1833.6. Samples: 31644104. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 10:14:47,016][52710] Avg episode reward: [(0, '29.540'), (1, '29.430')] [2023-10-08 10:14:47,737][53852] Updated weights for policy 0, policy_version 61930 (0.0009) [2023-10-08 10:14:48,101][53852] Updated weights for policy 0, policy_version 61940 (0.0007) [2023-10-08 10:14:48,470][53852] Updated weights for policy 0, policy_version 61950 (0.0009) [2023-10-08 10:14:50,073][53885] Updated weights for policy 1, policy_version 61672 (0.0008) [2023-10-08 10:14:50,450][53885] Updated weights for policy 1, policy_version 61682 (0.0008) [2023-10-08 10:14:50,817][53885] Updated weights for policy 1, policy_version 61692 (0.0007) [2023-10-08 10:14:51,975][53852] Updated weights for policy 0, policy_version 61960 (0.0007) [2023-10-08 10:14:52,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 126615552. Throughput: 0: 1853.5, 1: 1839.4. Samples: 31666188. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 10:14:52,016][52710] Avg episode reward: [(0, '30.490'), (1, '29.830')] [2023-10-08 10:14:52,339][53852] Updated weights for policy 0, policy_version 61970 (0.0007) [2023-10-08 10:14:52,711][53852] Updated weights for policy 0, policy_version 61980 (0.0008) [2023-10-08 10:14:54,604][53885] Updated weights for policy 1, policy_version 61702 (0.0009) [2023-10-08 10:14:54,964][53885] Updated weights for policy 1, policy_version 61712 (0.0008) [2023-10-08 10:14:55,338][53885] Updated weights for policy 1, policy_version 61722 (0.0009) [2023-10-08 10:14:56,523][53852] Updated weights for policy 0, policy_version 61990 (0.0007) [2023-10-08 10:14:56,888][53852] Updated weights for policy 0, policy_version 62000 (0.0008) [2023-10-08 10:14:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 126681088. Throughput: 0: 1850.0, 1: 1830.4. Samples: 31677244. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 10:14:57,015][52710] Avg episode reward: [(0, '29.070'), (1, '31.650')] [2023-10-08 10:14:57,263][53852] Updated weights for policy 0, policy_version 62010 (0.0007) [2023-10-08 10:14:58,909][53885] Updated weights for policy 1, policy_version 61732 (0.0010) [2023-10-08 10:14:59,272][53885] Updated weights for policy 1, policy_version 61742 (0.0009) [2023-10-08 10:14:59,635][53885] Updated weights for policy 1, policy_version 61752 (0.0008) [2023-10-08 10:15:00,897][53852] Updated weights for policy 0, policy_version 62020 (0.0007) [2023-10-08 10:15:01,268][53852] Updated weights for policy 0, policy_version 62030 (0.0009) [2023-10-08 10:15:01,638][53852] Updated weights for policy 0, policy_version 62040 (0.0011) [2023-10-08 10:15:02,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 126779392. Throughput: 0: 1845.7, 1: 1840.1. Samples: 31698948. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 10:15:02,016][52710] Avg episode reward: [(0, '27.210'), (1, '30.910')] [2023-10-08 10:15:03,267][53885] Updated weights for policy 1, policy_version 61762 (0.0008) [2023-10-08 10:15:03,641][53885] Updated weights for policy 1, policy_version 61772 (0.0009) [2023-10-08 10:15:04,003][53885] Updated weights for policy 1, policy_version 61782 (0.0010) [2023-10-08 10:15:04,371][53885] Updated weights for policy 1, policy_version 61792 (0.0009) [2023-10-08 10:15:05,308][53852] Updated weights for policy 0, policy_version 62050 (0.0009) [2023-10-08 10:15:05,677][53852] Updated weights for policy 0, policy_version 62060 (0.0009) [2023-10-08 10:15:06,043][53852] Updated weights for policy 0, policy_version 62070 (0.0010) [2023-10-08 10:15:06,414][53852] Updated weights for policy 0, policy_version 62080 (0.0008) [2023-10-08 10:15:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 126844928. Throughput: 0: 1836.4, 1: 1842.9. Samples: 31720698. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 10:15:07,016][52710] Avg episode reward: [(0, '29.520'), (1, '32.150')] [2023-10-08 10:15:08,006][53885] Updated weights for policy 1, policy_version 61802 (0.0008) [2023-10-08 10:15:08,368][53885] Updated weights for policy 1, policy_version 61812 (0.0009) [2023-10-08 10:15:08,729][53885] Updated weights for policy 1, policy_version 61822 (0.0007) [2023-10-08 10:15:10,121][53852] Updated weights for policy 0, policy_version 62090 (0.0009) [2023-10-08 10:15:10,491][53852] Updated weights for policy 0, policy_version 62100 (0.0009) [2023-10-08 10:15:10,851][53852] Updated weights for policy 0, policy_version 62110 (0.0009) [2023-10-08 10:15:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126910464. Throughput: 0: 1845.6, 1: 1840.6. Samples: 31732066. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 10:15:12,016][52710] Avg episode reward: [(0, '27.190'), (1, '32.190')] [2023-10-08 10:15:12,363][53885] Updated weights for policy 1, policy_version 61832 (0.0008) [2023-10-08 10:15:12,736][53885] Updated weights for policy 1, policy_version 61842 (0.0009) [2023-10-08 10:15:13,104][53885] Updated weights for policy 1, policy_version 61852 (0.0011) [2023-10-08 10:15:14,405][53852] Updated weights for policy 0, policy_version 62120 (0.0010) [2023-10-08 10:15:14,772][53852] Updated weights for policy 0, policy_version 62130 (0.0010) [2023-10-08 10:15:15,142][53852] Updated weights for policy 0, policy_version 62140 (0.0011) [2023-10-08 10:15:16,893][53885] Updated weights for policy 1, policy_version 61862 (0.0009) [2023-10-08 10:15:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126976000. Throughput: 0: 1834.7, 1: 1836.6. Samples: 31753594. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 10:15:17,016][52710] Avg episode reward: [(0, '26.920'), (1, '33.240')] [2023-10-08 10:15:17,260][53885] Updated weights for policy 1, policy_version 61872 (0.0008) [2023-10-08 10:15:17,628][53885] Updated weights for policy 1, policy_version 61882 (0.0007) [2023-10-08 10:15:18,847][53852] Updated weights for policy 0, policy_version 62150 (0.0009) [2023-10-08 10:15:19,209][53852] Updated weights for policy 0, policy_version 62160 (0.0007) [2023-10-08 10:15:19,586][53852] Updated weights for policy 0, policy_version 62170 (0.0007) [2023-10-08 10:15:21,353][53885] Updated weights for policy 1, policy_version 61892 (0.0008) [2023-10-08 10:15:21,723][53885] Updated weights for policy 1, policy_version 61902 (0.0009) [2023-10-08 10:15:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 127041536. Throughput: 0: 1841.8, 1: 1832.4. Samples: 31776186. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 10:15:22,016][52710] Avg episode reward: [(0, '26.980'), (1, '31.650')] [2023-10-08 10:15:22,090][53885] Updated weights for policy 1, policy_version 61912 (0.0009) [2023-10-08 10:15:23,218][53852] Updated weights for policy 0, policy_version 62180 (0.0008) [2023-10-08 10:15:23,595][53852] Updated weights for policy 0, policy_version 62190 (0.0008) [2023-10-08 10:15:23,970][53852] Updated weights for policy 0, policy_version 62200 (0.0009) [2023-10-08 10:15:25,721][53885] Updated weights for policy 1, policy_version 61922 (0.0007) [2023-10-08 10:15:26,092][53885] Updated weights for policy 1, policy_version 61932 (0.0009) [2023-10-08 10:15:26,465][53885] Updated weights for policy 1, policy_version 61942 (0.0008) [2023-10-08 10:15:26,836][53885] Updated weights for policy 1, policy_version 61952 (0.0007) [2023-10-08 10:15:27,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 127139840. Throughput: 0: 1830.8, 1: 1844.6. Samples: 31786858. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 10:15:27,015][52710] Avg episode reward: [(0, '27.920'), (1, '28.060')] [2023-10-08 10:15:27,538][53852] Updated weights for policy 0, policy_version 62210 (0.0009) [2023-10-08 10:15:27,914][53852] Updated weights for policy 0, policy_version 62220 (0.0007) [2023-10-08 10:15:28,281][53852] Updated weights for policy 0, policy_version 62230 (0.0008) [2023-10-08 10:15:28,650][53852] Updated weights for policy 0, policy_version 62240 (0.0009) [2023-10-08 10:15:30,341][53885] Updated weights for policy 1, policy_version 61962 (0.0011) [2023-10-08 10:15:30,709][53885] Updated weights for policy 1, policy_version 61972 (0.0009) [2023-10-08 10:15:31,071][53885] Updated weights for policy 1, policy_version 61982 (0.0011) [2023-10-08 10:15:32,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127205376. Throughput: 0: 1833.4, 1: 1827.2. Samples: 31808830. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 10:15:32,016][52710] Avg episode reward: [(0, '26.670'), (1, '29.230')] [2023-10-08 10:15:32,640][53852] Updated weights for policy 0, policy_version 62250 (0.0007) [2023-10-08 10:15:33,020][53852] Updated weights for policy 0, policy_version 62260 (0.0011) [2023-10-08 10:15:33,397][53852] Updated weights for policy 0, policy_version 62270 (0.0010) [2023-10-08 10:15:34,834][53885] Updated weights for policy 1, policy_version 61992 (0.0008) [2023-10-08 10:15:35,211][53885] Updated weights for policy 1, policy_version 62002 (0.0009) [2023-10-08 10:15:35,576][53885] Updated weights for policy 1, policy_version 62012 (0.0008) [2023-10-08 10:15:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127270912. Throughput: 0: 1819.7, 1: 1834.9. Samples: 31830644. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 10:15:37,015][52710] Avg episode reward: [(0, '27.900'), (1, '31.430')] [2023-10-08 10:15:37,049][53852] Updated weights for policy 0, policy_version 62280 (0.0008) [2023-10-08 10:15:37,425][53852] Updated weights for policy 0, policy_version 62290 (0.0007) [2023-10-08 10:15:37,799][53852] Updated weights for policy 0, policy_version 62300 (0.0011) [2023-10-08 10:15:39,271][53885] Updated weights for policy 1, policy_version 62022 (0.0010) [2023-10-08 10:15:39,650][53885] Updated weights for policy 1, policy_version 62032 (0.0009) [2023-10-08 10:15:40,010][53885] Updated weights for policy 1, policy_version 62042 (0.0009) [2023-10-08 10:15:41,405][53852] Updated weights for policy 0, policy_version 62310 (0.0010) [2023-10-08 10:15:41,769][53852] Updated weights for policy 0, policy_version 62320 (0.0011) [2023-10-08 10:15:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 127336448. Throughput: 0: 1819.3, 1: 1822.9. Samples: 31841142. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 10:15:42,016][52710] Avg episode reward: [(0, '28.340'), (1, '29.980')] [2023-10-08 10:15:42,142][53852] Updated weights for policy 0, policy_version 62330 (0.0011) [2023-10-08 10:15:43,718][53885] Updated weights for policy 1, policy_version 62052 (0.0009) [2023-10-08 10:15:44,085][53885] Updated weights for policy 1, policy_version 62062 (0.0011) [2023-10-08 10:15:44,449][53885] Updated weights for policy 1, policy_version 62072 (0.0008) [2023-10-08 10:15:45,699][53852] Updated weights for policy 0, policy_version 62340 (0.0009) [2023-10-08 10:15:46,074][53852] Updated weights for policy 0, policy_version 62350 (0.0008) [2023-10-08 10:15:46,436][53852] Updated weights for policy 0, policy_version 62360 (0.0009) [2023-10-08 10:15:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 127434752. Throughput: 0: 1824.3, 1: 1826.5. Samples: 31863236. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) [2023-10-08 10:15:47,016][52710] Avg episode reward: [(0, '28.740'), (1, '33.850')] [2023-10-08 10:15:48,201][53885] Updated weights for policy 1, policy_version 62082 (0.0007) [2023-10-08 10:15:48,560][53885] Updated weights for policy 1, policy_version 62092 (0.0007) [2023-10-08 10:15:48,931][53885] Updated weights for policy 1, policy_version 62102 (0.0008) [2023-10-08 10:15:49,293][53885] Updated weights for policy 1, policy_version 62112 (0.0008) [2023-10-08 10:15:50,147][53852] Updated weights for policy 0, policy_version 62370 (0.0007) [2023-10-08 10:15:50,517][53852] Updated weights for policy 0, policy_version 62380 (0.0007) [2023-10-08 10:15:50,884][53852] Updated weights for policy 0, policy_version 62390 (0.0007) [2023-10-08 10:15:51,255][53852] Updated weights for policy 0, policy_version 62400 (0.0007) [2023-10-08 10:15:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 127500288. Throughput: 0: 1822.4, 1: 1819.1. Samples: 31884564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:15:52,015][52710] Avg episode reward: [(0, '29.180'), (1, '30.680')] [2023-10-08 10:15:52,924][53885] Updated weights for policy 1, policy_version 62122 (0.0007) [2023-10-08 10:15:53,301][53885] Updated weights for policy 1, policy_version 62132 (0.0007) [2023-10-08 10:15:53,663][53885] Updated weights for policy 1, policy_version 62142 (0.0009) [2023-10-08 10:15:55,017][53852] Updated weights for policy 0, policy_version 62410 (0.0008) [2023-10-08 10:15:55,379][53852] Updated weights for policy 0, policy_version 62420 (0.0008) [2023-10-08 10:15:55,751][53852] Updated weights for policy 0, policy_version 62430 (0.0011) [2023-10-08 10:15:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127565824. Throughput: 0: 1823.4, 1: 1821.2. Samples: 31896070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:15:57,016][52710] Avg episode reward: [(0, '29.630'), (1, '31.090')] [2023-10-08 10:15:57,380][53885] Updated weights for policy 1, policy_version 62152 (0.0010) [2023-10-08 10:15:57,746][53885] Updated weights for policy 1, policy_version 62162 (0.0008) [2023-10-08 10:15:58,105][53885] Updated weights for policy 1, policy_version 62172 (0.0008) [2023-10-08 10:15:59,350][53852] Updated weights for policy 0, policy_version 62440 (0.0008) [2023-10-08 10:15:59,714][53852] Updated weights for policy 0, policy_version 62450 (0.0009) [2023-10-08 10:16:00,089][53852] Updated weights for policy 0, policy_version 62460 (0.0010) [2023-10-08 10:16:01,898][53885] Updated weights for policy 1, policy_version 62182 (0.0007) [2023-10-08 10:16:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 127631360. Throughput: 0: 1823.7, 1: 1816.7. Samples: 31917412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:16:02,016][52710] Avg episode reward: [(0, '27.740'), (1, '35.980')] [2023-10-08 10:16:02,258][53885] Updated weights for policy 1, policy_version 62192 (0.0008) [2023-10-08 10:16:02,627][53885] Updated weights for policy 1, policy_version 62202 (0.0010) [2023-10-08 10:16:03,882][53852] Updated weights for policy 0, policy_version 62470 (0.0009) [2023-10-08 10:16:04,257][53852] Updated weights for policy 0, policy_version 62480 (0.0007) [2023-10-08 10:16:04,632][53852] Updated weights for policy 0, policy_version 62490 (0.0007) [2023-10-08 10:16:06,407][53885] Updated weights for policy 1, policy_version 62212 (0.0008) [2023-10-08 10:16:06,779][53885] Updated weights for policy 1, policy_version 62222 (0.0009) [2023-10-08 10:16:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 127696896. Throughput: 0: 1818.4, 1: 1812.9. Samples: 31939596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:16:07,016][52710] Avg episode reward: [(0, '28.580'), (1, '32.110')] [2023-10-08 10:16:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000062496_63995904.pth... [2023-10-08 10:16:07,060][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000060800_62259200.pth [2023-10-08 10:16:07,154][53885] Updated weights for policy 1, policy_version 62232 (0.0008) [2023-10-08 10:16:07,451][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000062240_63733760.pth... [2023-10-08 10:16:07,488][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000060512_61964288.pth [2023-10-08 10:16:08,212][53852] Updated weights for policy 0, policy_version 62500 (0.0008) [2023-10-08 10:16:08,586][53852] Updated weights for policy 0, policy_version 62510 (0.0009) [2023-10-08 10:16:08,946][53852] Updated weights for policy 0, policy_version 62520 (0.0008) [2023-10-08 10:16:10,926][53885] Updated weights for policy 1, policy_version 62242 (0.0008) [2023-10-08 10:16:11,309][53885] Updated weights for policy 1, policy_version 62252 (0.0011) [2023-10-08 10:16:11,672][53885] Updated weights for policy 1, policy_version 62262 (0.0011) [2023-10-08 10:16:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 127762432. Throughput: 0: 1822.8, 1: 1803.7. Samples: 31950052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:16:12,016][52710] Avg episode reward: [(0, '28.930'), (1, '31.100')] [2023-10-08 10:16:12,044][53885] Updated weights for policy 1, policy_version 62272 (0.0010) [2023-10-08 10:16:12,585][53852] Updated weights for policy 0, policy_version 62530 (0.0008) [2023-10-08 10:16:12,948][53852] Updated weights for policy 0, policy_version 62540 (0.0008) [2023-10-08 10:16:13,318][53852] Updated weights for policy 0, policy_version 62550 (0.0008) [2023-10-08 10:16:13,687][53852] Updated weights for policy 0, policy_version 62560 (0.0008) [2023-10-08 10:16:15,836][53885] Updated weights for policy 1, policy_version 62282 (0.0008) [2023-10-08 10:16:16,202][53885] Updated weights for policy 1, policy_version 62292 (0.0008) [2023-10-08 10:16:16,566][53885] Updated weights for policy 1, policy_version 62302 (0.0008) [2023-10-08 10:16:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127860736. Throughput: 0: 1826.5, 1: 1810.2. Samples: 31972480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:16:17,016][52710] Avg episode reward: [(0, '29.870'), (1, '34.150')] [2023-10-08 10:16:17,328][53852] Updated weights for policy 0, policy_version 62570 (0.0008) [2023-10-08 10:16:17,701][53852] Updated weights for policy 0, policy_version 62580 (0.0011) [2023-10-08 10:16:18,067][53852] Updated weights for policy 0, policy_version 62590 (0.0009) [2023-10-08 10:16:20,247][53885] Updated weights for policy 1, policy_version 62312 (0.0007) [2023-10-08 10:16:20,623][53885] Updated weights for policy 1, policy_version 62322 (0.0010) [2023-10-08 10:16:20,992][53885] Updated weights for policy 1, policy_version 62332 (0.0007) [2023-10-08 10:16:21,780][53852] Updated weights for policy 0, policy_version 62600 (0.0008) [2023-10-08 10:16:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 127926272. Throughput: 0: 1828.8, 1: 1800.6. Samples: 31993966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:16:22,016][52710] Avg episode reward: [(0, '29.490'), (1, '34.280')] [2023-10-08 10:16:22,153][53852] Updated weights for policy 0, policy_version 62610 (0.0007) [2023-10-08 10:16:22,528][53852] Updated weights for policy 0, policy_version 62620 (0.0007) [2023-10-08 10:16:24,631][53885] Updated weights for policy 1, policy_version 62342 (0.0009) [2023-10-08 10:16:25,005][53885] Updated weights for policy 1, policy_version 62352 (0.0010) [2023-10-08 10:16:25,371][53885] Updated weights for policy 1, policy_version 62362 (0.0009) [2023-10-08 10:16:25,950][53852] Updated weights for policy 0, policy_version 62630 (0.0007) [2023-10-08 10:16:26,314][53852] Updated weights for policy 0, policy_version 62640 (0.0010) [2023-10-08 10:16:26,694][53852] Updated weights for policy 0, policy_version 62650 (0.0010) [2023-10-08 10:16:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 128024576. Throughput: 0: 1833.5, 1: 1817.9. Samples: 32005452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:16:27,016][52710] Avg episode reward: [(0, '27.880'), (1, '32.690')] [2023-10-08 10:16:29,095][53885] Updated weights for policy 1, policy_version 62372 (0.0008) [2023-10-08 10:16:29,465][53885] Updated weights for policy 1, policy_version 62382 (0.0008) [2023-10-08 10:16:29,837][53885] Updated weights for policy 1, policy_version 62392 (0.0007) [2023-10-08 10:16:30,382][53852] Updated weights for policy 0, policy_version 62660 (0.0008) [2023-10-08 10:16:30,749][53852] Updated weights for policy 0, policy_version 62670 (0.0007) [2023-10-08 10:16:31,120][53852] Updated weights for policy 0, policy_version 62680 (0.0007) [2023-10-08 10:16:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128090112. Throughput: 0: 1820.5, 1: 1814.1. Samples: 32026790. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-10-08 10:16:32,016][52710] Avg episode reward: [(0, '30.580'), (1, '33.510')] [2023-10-08 10:16:33,380][53885] Updated weights for policy 1, policy_version 62402 (0.0008) [2023-10-08 10:16:33,740][53885] Updated weights for policy 1, policy_version 62412 (0.0008) [2023-10-08 10:16:34,114][53885] Updated weights for policy 1, policy_version 62422 (0.0007) [2023-10-08 10:16:34,485][53885] Updated weights for policy 1, policy_version 62432 (0.0007) [2023-10-08 10:16:34,783][53852] Updated weights for policy 0, policy_version 62690 (0.0007) [2023-10-08 10:16:35,147][53852] Updated weights for policy 0, policy_version 62700 (0.0010) [2023-10-08 10:16:35,520][53852] Updated weights for policy 0, policy_version 62710 (0.0009) [2023-10-08 10:16:35,887][53852] Updated weights for policy 0, policy_version 62720 (0.0007) [2023-10-08 10:16:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 128155648. Throughput: 0: 1828.9, 1: 1824.3. Samples: 32048958. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-10-08 10:16:37,016][52710] Avg episode reward: [(0, '29.240'), (1, '37.680')] [2023-10-08 10:16:37,967][53885] Updated weights for policy 1, policy_version 62442 (0.0008) [2023-10-08 10:16:38,331][53885] Updated weights for policy 1, policy_version 62452 (0.0009) [2023-10-08 10:16:38,703][53885] Updated weights for policy 1, policy_version 62462 (0.0008) [2023-10-08 10:16:39,569][53852] Updated weights for policy 0, policy_version 62730 (0.0008) [2023-10-08 10:16:39,933][53852] Updated weights for policy 0, policy_version 62740 (0.0007) [2023-10-08 10:16:40,311][53852] Updated weights for policy 0, policy_version 62750 (0.0008) [2023-10-08 10:16:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128221184. Throughput: 0: 1821.6, 1: 1823.3. Samples: 32060092. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-10-08 10:16:42,016][52710] Avg episode reward: [(0, '28.290'), (1, '34.000')] [2023-10-08 10:16:42,527][53885] Updated weights for policy 1, policy_version 62472 (0.0007) [2023-10-08 10:16:42,888][53885] Updated weights for policy 1, policy_version 62482 (0.0008) [2023-10-08 10:16:43,261][53885] Updated weights for policy 1, policy_version 62492 (0.0008) [2023-10-08 10:16:43,993][53852] Updated weights for policy 0, policy_version 62760 (0.0010) [2023-10-08 10:16:44,373][53852] Updated weights for policy 0, policy_version 62770 (0.0008) [2023-10-08 10:16:44,739][53852] Updated weights for policy 0, policy_version 62780 (0.0007) [2023-10-08 10:16:46,882][53885] Updated weights for policy 1, policy_version 62502 (0.0007) [2023-10-08 10:16:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 128286720. Throughput: 0: 1830.5, 1: 1821.1. Samples: 32081732. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-10-08 10:16:47,016][52710] Avg episode reward: [(0, '28.550'), (1, '35.910')] [2023-10-08 10:16:47,244][53885] Updated weights for policy 1, policy_version 62512 (0.0008) [2023-10-08 10:16:47,608][53885] Updated weights for policy 1, policy_version 62522 (0.0010) [2023-10-08 10:16:48,480][53852] Updated weights for policy 0, policy_version 62790 (0.0009) [2023-10-08 10:16:48,840][53852] Updated weights for policy 0, policy_version 62800 (0.0010) [2023-10-08 10:16:49,206][53852] Updated weights for policy 0, policy_version 62810 (0.0009) [2023-10-08 10:16:51,280][53885] Updated weights for policy 1, policy_version 62532 (0.0009) [2023-10-08 10:16:51,640][53885] Updated weights for policy 1, policy_version 62542 (0.0011) [2023-10-08 10:16:52,013][53885] Updated weights for policy 1, policy_version 62552 (0.0007) [2023-10-08 10:16:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128352256. Throughput: 0: 1837.1, 1: 1816.4. Samples: 32104002. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-10-08 10:16:52,015][52710] Avg episode reward: [(0, '28.030'), (1, '34.650')] [2023-10-08 10:16:52,931][53852] Updated weights for policy 0, policy_version 62820 (0.0008) [2023-10-08 10:16:53,304][53852] Updated weights for policy 0, policy_version 62830 (0.0009) [2023-10-08 10:16:53,669][53852] Updated weights for policy 0, policy_version 62840 (0.0009) [2023-10-08 10:16:55,899][53885] Updated weights for policy 1, policy_version 62562 (0.0011) [2023-10-08 10:16:56,311][53885] Updated weights for policy 1, policy_version 62572 (0.0008) [2023-10-08 10:16:56,682][53885] Updated weights for policy 1, policy_version 62582 (0.0009) [2023-10-08 10:16:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 128417792. Throughput: 0: 1835.0, 1: 1817.4. Samples: 32114410. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-10-08 10:16:57,016][52710] Avg episode reward: [(0, '28.760'), (1, '34.600')] [2023-10-08 10:16:57,054][53885] Updated weights for policy 1, policy_version 62592 (0.0009) [2023-10-08 10:16:57,413][53852] Updated weights for policy 0, policy_version 62850 (0.0010) [2023-10-08 10:16:57,787][53852] Updated weights for policy 0, policy_version 62860 (0.0007) [2023-10-08 10:16:58,158][53852] Updated weights for policy 0, policy_version 62870 (0.0008) [2023-10-08 10:16:58,521][53852] Updated weights for policy 0, policy_version 62880 (0.0008) [2023-10-08 10:17:00,698][53885] Updated weights for policy 1, policy_version 62602 (0.0009) [2023-10-08 10:17:01,062][53885] Updated weights for policy 1, policy_version 62612 (0.0008) [2023-10-08 10:17:01,429][53885] Updated weights for policy 1, policy_version 62622 (0.0009) [2023-10-08 10:17:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128516096. Throughput: 0: 1827.2, 1: 1820.8. Samples: 32136642. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-10-08 10:17:02,016][52710] Avg episode reward: [(0, '30.750'), (1, '31.090')] [2023-10-08 10:17:02,345][53852] Updated weights for policy 0, policy_version 62890 (0.0007) [2023-10-08 10:17:02,723][53852] Updated weights for policy 0, policy_version 62900 (0.0007) [2023-10-08 10:17:03,097][53852] Updated weights for policy 0, policy_version 62910 (0.0007) [2023-10-08 10:17:04,982][53885] Updated weights for policy 1, policy_version 62632 (0.0009) [2023-10-08 10:17:05,351][53885] Updated weights for policy 1, policy_version 62642 (0.0010) [2023-10-08 10:17:05,723][53885] Updated weights for policy 1, policy_version 62652 (0.0007) [2023-10-08 10:17:06,552][53852] Updated weights for policy 0, policy_version 62920 (0.0007) [2023-10-08 10:17:06,921][53852] Updated weights for policy 0, policy_version 62930 (0.0007) [2023-10-08 10:17:07,015][52710] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 128581632. Throughput: 0: 1829.2, 1: 1827.8. Samples: 32158534. Policy #0 lag: (min: 8.0, avg: 33.4, max: 40.0) [2023-10-08 10:17:07,017][52710] Avg episode reward: [(0, '30.510'), (1, '33.280')] [2023-10-08 10:17:07,301][53852] Updated weights for policy 0, policy_version 62940 (0.0008) [2023-10-08 10:17:09,421][53885] Updated weights for policy 1, policy_version 62662 (0.0008) [2023-10-08 10:17:09,789][53885] Updated weights for policy 1, policy_version 62672 (0.0010) [2023-10-08 10:17:10,150][53885] Updated weights for policy 1, policy_version 62682 (0.0009) [2023-10-08 10:17:10,868][53852] Updated weights for policy 0, policy_version 62950 (0.0009) [2023-10-08 10:17:11,235][53852] Updated weights for policy 0, policy_version 62960 (0.0008) [2023-10-08 10:17:11,605][53852] Updated weights for policy 0, policy_version 62970 (0.0010) [2023-10-08 10:17:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 128679936. Throughput: 0: 1835.3, 1: 1821.2. Samples: 32169996. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 10:17:12,016][52710] Avg episode reward: [(0, '30.400'), (1, '35.230')] [2023-10-08 10:17:13,721][53885] Updated weights for policy 1, policy_version 62692 (0.0010) [2023-10-08 10:17:14,091][53885] Updated weights for policy 1, policy_version 62702 (0.0009) [2023-10-08 10:17:14,454][53885] Updated weights for policy 1, policy_version 62712 (0.0010) [2023-10-08 10:17:15,243][53852] Updated weights for policy 0, policy_version 62980 (0.0009) [2023-10-08 10:17:15,614][53852] Updated weights for policy 0, policy_version 62990 (0.0009) [2023-10-08 10:17:15,987][53852] Updated weights for policy 0, policy_version 63000 (0.0010) [2023-10-08 10:17:17,015][52710] Fps is (10 sec: 16384.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 128745472. Throughput: 0: 1834.2, 1: 1829.5. Samples: 32191656. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 10:17:17,016][52710] Avg episode reward: [(0, '31.430'), (1, '33.400')] [2023-10-08 10:17:18,258][53885] Updated weights for policy 1, policy_version 62722 (0.0008) [2023-10-08 10:17:18,620][53885] Updated weights for policy 1, policy_version 62732 (0.0010) [2023-10-08 10:17:18,990][53885] Updated weights for policy 1, policy_version 62742 (0.0008) [2023-10-08 10:17:19,355][53885] Updated weights for policy 1, policy_version 62752 (0.0007) [2023-10-08 10:17:19,657][53852] Updated weights for policy 0, policy_version 63010 (0.0010) [2023-10-08 10:17:20,030][53852] Updated weights for policy 0, policy_version 63020 (0.0008) [2023-10-08 10:17:20,409][53852] Updated weights for policy 0, policy_version 63030 (0.0009) [2023-10-08 10:17:20,777][53852] Updated weights for policy 0, policy_version 63040 (0.0009) [2023-10-08 10:17:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 128811008. Throughput: 0: 1843.2, 1: 1820.8. Samples: 32213836. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 10:17:22,016][52710] Avg episode reward: [(0, '30.760'), (1, '34.990')] [2023-10-08 10:17:22,891][53885] Updated weights for policy 1, policy_version 62762 (0.0010) [2023-10-08 10:17:23,257][53885] Updated weights for policy 1, policy_version 62772 (0.0007) [2023-10-08 10:17:23,635][53885] Updated weights for policy 1, policy_version 62782 (0.0009) [2023-10-08 10:17:24,383][53852] Updated weights for policy 0, policy_version 63050 (0.0011) [2023-10-08 10:17:24,754][53852] Updated weights for policy 0, policy_version 63060 (0.0007) [2023-10-08 10:17:25,125][53852] Updated weights for policy 0, policy_version 63070 (0.0009) [2023-10-08 10:17:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 128876544. Throughput: 0: 1836.8, 1: 1823.2. Samples: 32224788. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 10:17:27,016][52710] Avg episode reward: [(0, '29.620'), (1, '34.790')] [2023-10-08 10:17:27,318][53885] Updated weights for policy 1, policy_version 62792 (0.0011) [2023-10-08 10:17:27,685][53885] Updated weights for policy 1, policy_version 62802 (0.0011) [2023-10-08 10:17:28,043][53885] Updated weights for policy 1, policy_version 62812 (0.0007) [2023-10-08 10:17:28,711][53852] Updated weights for policy 0, policy_version 63080 (0.0007) [2023-10-08 10:17:29,076][53852] Updated weights for policy 0, policy_version 63090 (0.0008) [2023-10-08 10:17:29,447][53852] Updated weights for policy 0, policy_version 63100 (0.0008) [2023-10-08 10:17:31,773][53885] Updated weights for policy 1, policy_version 62822 (0.0010) [2023-10-08 10:17:32,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 128942080. Throughput: 0: 1842.8, 1: 1821.9. Samples: 32246642. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 10:17:32,016][52710] Avg episode reward: [(0, '30.020'), (1, '33.020')] [2023-10-08 10:17:32,142][53885] Updated weights for policy 1, policy_version 62832 (0.0010) [2023-10-08 10:17:32,511][53885] Updated weights for policy 1, policy_version 62842 (0.0008) [2023-10-08 10:17:33,013][53852] Updated weights for policy 0, policy_version 63110 (0.0009) [2023-10-08 10:17:33,374][53852] Updated weights for policy 0, policy_version 63120 (0.0008) [2023-10-08 10:17:33,747][53852] Updated weights for policy 0, policy_version 63130 (0.0011) [2023-10-08 10:17:36,179][53885] Updated weights for policy 1, policy_version 62852 (0.0009) [2023-10-08 10:17:36,539][53885] Updated weights for policy 1, policy_version 62862 (0.0010) [2023-10-08 10:17:36,910][53885] Updated weights for policy 1, policy_version 62872 (0.0009) [2023-10-08 10:17:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 129007616. Throughput: 0: 1849.4, 1: 1818.0. Samples: 32269038. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 10:17:37,016][52710] Avg episode reward: [(0, '27.570'), (1, '34.090')] [2023-10-08 10:17:37,325][53852] Updated weights for policy 0, policy_version 63140 (0.0010) [2023-10-08 10:17:37,706][53852] Updated weights for policy 0, policy_version 63150 (0.0008) [2023-10-08 10:17:38,064][53852] Updated weights for policy 0, policy_version 63160 (0.0009) [2023-10-08 10:17:40,653][53885] Updated weights for policy 1, policy_version 62882 (0.0010) [2023-10-08 10:17:41,042][53885] Updated weights for policy 1, policy_version 62892 (0.0008) [2023-10-08 10:17:41,407][53885] Updated weights for policy 1, policy_version 62902 (0.0008) [2023-10-08 10:17:41,686][53852] Updated weights for policy 0, policy_version 63170 (0.0009) [2023-10-08 10:17:41,775][53885] Updated weights for policy 1, policy_version 62912 (0.0008) [2023-10-08 10:17:42,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 129105920. Throughput: 0: 1850.2, 1: 1822.4. Samples: 32279676. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 10:17:42,015][52710] Avg episode reward: [(0, '29.040'), (1, '35.460')] [2023-10-08 10:17:42,048][53852] Updated weights for policy 0, policy_version 63180 (0.0008) [2023-10-08 10:17:42,420][53852] Updated weights for policy 0, policy_version 63190 (0.0010) [2023-10-08 10:17:42,785][53852] Updated weights for policy 0, policy_version 63200 (0.0010) [2023-10-08 10:17:45,475][53885] Updated weights for policy 1, policy_version 62922 (0.0009) [2023-10-08 10:17:45,846][53885] Updated weights for policy 1, policy_version 62932 (0.0009) [2023-10-08 10:17:46,209][53885] Updated weights for policy 1, policy_version 62942 (0.0009) [2023-10-08 10:17:46,666][53852] Updated weights for policy 0, policy_version 63210 (0.0007) [2023-10-08 10:17:47,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129171456. Throughput: 0: 1849.7, 1: 1822.4. Samples: 32301886. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) [2023-10-08 10:17:47,015][52710] Avg episode reward: [(0, '30.070'), (1, '34.720')] [2023-10-08 10:17:47,035][53852] Updated weights for policy 0, policy_version 63220 (0.0009) [2023-10-08 10:17:47,394][53852] Updated weights for policy 0, policy_version 63230 (0.0010) [2023-10-08 10:17:49,865][53885] Updated weights for policy 1, policy_version 62952 (0.0008) [2023-10-08 10:17:50,238][53885] Updated weights for policy 1, policy_version 62962 (0.0007) [2023-10-08 10:17:50,614][53885] Updated weights for policy 1, policy_version 62972 (0.0008) [2023-10-08 10:17:51,088][53852] Updated weights for policy 0, policy_version 63240 (0.0008) [2023-10-08 10:17:51,462][53852] Updated weights for policy 0, policy_version 63250 (0.0007) [2023-10-08 10:17:51,838][53852] Updated weights for policy 0, policy_version 63260 (0.0007) [2023-10-08 10:17:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 129269760. Throughput: 0: 1830.6, 1: 1825.3. Samples: 32323046. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:17:52,015][52710] Avg episode reward: [(0, '29.940'), (1, '34.820')] [2023-10-08 10:17:54,308][53885] Updated weights for policy 1, policy_version 62982 (0.0010) [2023-10-08 10:17:54,678][53885] Updated weights for policy 1, policy_version 62992 (0.0008) [2023-10-08 10:17:55,045][53885] Updated weights for policy 1, policy_version 63002 (0.0009) [2023-10-08 10:17:55,340][53852] Updated weights for policy 0, policy_version 63270 (0.0008) [2023-10-08 10:17:55,713][53852] Updated weights for policy 0, policy_version 63280 (0.0008) [2023-10-08 10:17:56,081][53852] Updated weights for policy 0, policy_version 63290 (0.0008) [2023-10-08 10:17:57,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 129335296. Throughput: 0: 1843.7, 1: 1823.7. Samples: 32335028. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:17:57,016][52710] Avg episode reward: [(0, '29.330'), (1, '38.580')] [2023-10-08 10:17:57,018][53594] Saving new best policy, reward=38.580! [2023-10-08 10:17:58,872][53885] Updated weights for policy 1, policy_version 63012 (0.0007) [2023-10-08 10:17:59,243][53885] Updated weights for policy 1, policy_version 63022 (0.0010) [2023-10-08 10:17:59,579][53852] Updated weights for policy 0, policy_version 63300 (0.0008) [2023-10-08 10:17:59,598][53885] Updated weights for policy 1, policy_version 63032 (0.0009) [2023-10-08 10:17:59,946][53852] Updated weights for policy 0, policy_version 63310 (0.0009) [2023-10-08 10:18:00,328][53852] Updated weights for policy 0, policy_version 63320 (0.0008) [2023-10-08 10:18:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129400832. Throughput: 0: 1828.2, 1: 1819.5. Samples: 32355800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:18:02,016][52710] Avg episode reward: [(0, '29.840'), (1, '33.460')] [2023-10-08 10:18:03,308][53885] Updated weights for policy 1, policy_version 63042 (0.0007) [2023-10-08 10:18:03,681][53885] Updated weights for policy 1, policy_version 63052 (0.0010) [2023-10-08 10:18:04,048][53885] Updated weights for policy 1, policy_version 63062 (0.0008) [2023-10-08 10:18:04,067][53852] Updated weights for policy 0, policy_version 63330 (0.0009) [2023-10-08 10:18:04,414][53885] Updated weights for policy 1, policy_version 63072 (0.0009) [2023-10-08 10:18:04,431][53852] Updated weights for policy 0, policy_version 63340 (0.0007) [2023-10-08 10:18:04,797][53852] Updated weights for policy 0, policy_version 63350 (0.0008) [2023-10-08 10:18:05,168][53852] Updated weights for policy 0, policy_version 63360 (0.0007) [2023-10-08 10:18:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 129466368. Throughput: 0: 1841.8, 1: 1818.4. Samples: 32378544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:18:07,016][52710] Avg episode reward: [(0, '31.880'), (1, '34.880')] [2023-10-08 10:18:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000063072_64585728.pth... [2023-10-08 10:18:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000063360_64880640.pth... [2023-10-08 10:18:07,065][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000061376_62849024.pth [2023-10-08 10:18:07,065][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000061632_63111168.pth [2023-10-08 10:18:08,184][53885] Updated weights for policy 1, policy_version 63082 (0.0007) [2023-10-08 10:18:08,557][53885] Updated weights for policy 1, policy_version 63092 (0.0007) [2023-10-08 10:18:08,908][53852] Updated weights for policy 0, policy_version 63370 (0.0008) [2023-10-08 10:18:08,921][53885] Updated weights for policy 1, policy_version 63102 (0.0007) [2023-10-08 10:18:09,277][53852] Updated weights for policy 0, policy_version 63380 (0.0009) [2023-10-08 10:18:09,653][53852] Updated weights for policy 0, policy_version 63390 (0.0008) [2023-10-08 10:18:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 129531904. Throughput: 0: 1828.1, 1: 1817.5. Samples: 32388840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:18:12,015][52710] Avg episode reward: [(0, '30.420'), (1, '33.040')] [2023-10-08 10:18:12,607][53885] Updated weights for policy 1, policy_version 63112 (0.0010) [2023-10-08 10:18:12,977][53885] Updated weights for policy 1, policy_version 63122 (0.0010) [2023-10-08 10:18:13,201][53852] Updated weights for policy 0, policy_version 63400 (0.0007) [2023-10-08 10:18:13,341][53885] Updated weights for policy 1, policy_version 63132 (0.0008) [2023-10-08 10:18:13,576][53852] Updated weights for policy 0, policy_version 63410 (0.0007) [2023-10-08 10:18:13,947][53852] Updated weights for policy 0, policy_version 63420 (0.0009) [2023-10-08 10:18:17,009][53885] Updated weights for policy 1, policy_version 63142 (0.0008) [2023-10-08 10:18:17,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 129597440. Throughput: 0: 1839.6, 1: 1820.1. Samples: 32411330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:18:17,017][52710] Avg episode reward: [(0, '29.440'), (1, '32.410')] [2023-10-08 10:18:17,383][53885] Updated weights for policy 1, policy_version 63152 (0.0008) [2023-10-08 10:18:17,566][53852] Updated weights for policy 0, policy_version 63430 (0.0008) [2023-10-08 10:18:17,745][53885] Updated weights for policy 1, policy_version 63162 (0.0008) [2023-10-08 10:18:17,942][53852] Updated weights for policy 0, policy_version 63440 (0.0009) [2023-10-08 10:18:18,304][53852] Updated weights for policy 0, policy_version 63450 (0.0009) [2023-10-08 10:18:21,357][53885] Updated weights for policy 1, policy_version 63172 (0.0008) [2023-10-08 10:18:21,733][53885] Updated weights for policy 1, policy_version 63182 (0.0010) [2023-10-08 10:18:22,013][53852] Updated weights for policy 0, policy_version 63460 (0.0008) [2023-10-08 10:18:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129662976. Throughput: 0: 1833.4, 1: 1827.2. Samples: 32433766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:18:22,016][52710] Avg episode reward: [(0, '29.760'), (1, '32.800')] [2023-10-08 10:18:22,089][53885] Updated weights for policy 1, policy_version 63192 (0.0007) [2023-10-08 10:18:22,377][53852] Updated weights for policy 0, policy_version 63470 (0.0007) [2023-10-08 10:18:22,746][53852] Updated weights for policy 0, policy_version 63480 (0.0007) [2023-10-08 10:18:25,823][53885] Updated weights for policy 1, policy_version 63202 (0.0008) [2023-10-08 10:18:26,236][53885] Updated weights for policy 1, policy_version 63212 (0.0007) [2023-10-08 10:18:26,267][53852] Updated weights for policy 0, policy_version 63490 (0.0008) [2023-10-08 10:18:26,603][53885] Updated weights for policy 1, policy_version 63222 (0.0007) [2023-10-08 10:18:26,640][53852] Updated weights for policy 0, policy_version 63500 (0.0008) [2023-10-08 10:18:26,963][53885] Updated weights for policy 1, policy_version 63232 (0.0007) [2023-10-08 10:18:27,003][53852] Updated weights for policy 0, policy_version 63510 (0.0007) [2023-10-08 10:18:27,015][52710] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129761280. Throughput: 0: 1832.8, 1: 1824.7. Samples: 32444262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:18:27,016][52710] Avg episode reward: [(0, '28.980'), (1, '36.300')] [2023-10-08 10:18:27,372][53852] Updated weights for policy 0, policy_version 63520 (0.0007) [2023-10-08 10:18:30,682][53885] Updated weights for policy 1, policy_version 63242 (0.0007) [2023-10-08 10:18:30,899][53852] Updated weights for policy 0, policy_version 63530 (0.0008) [2023-10-08 10:18:31,045][53885] Updated weights for policy 1, policy_version 63252 (0.0007) [2023-10-08 10:18:31,266][53852] Updated weights for policy 0, policy_version 63540 (0.0008) [2023-10-08 10:18:31,411][53885] Updated weights for policy 1, policy_version 63262 (0.0007) [2023-10-08 10:18:31,637][53852] Updated weights for policy 0, policy_version 63550 (0.0007) [2023-10-08 10:18:32,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 129859584. Throughput: 0: 1844.1, 1: 1823.4. Samples: 32466926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:18:32,016][52710] Avg episode reward: [(0, '26.320'), (1, '32.070')] [2023-10-08 10:18:35,074][53885] Updated weights for policy 1, policy_version 63272 (0.0008) [2023-10-08 10:18:35,354][53852] Updated weights for policy 0, policy_version 63560 (0.0008) [2023-10-08 10:18:35,439][53885] Updated weights for policy 1, policy_version 63282 (0.0007) [2023-10-08 10:18:35,729][53852] Updated weights for policy 0, policy_version 63570 (0.0008) [2023-10-08 10:18:35,803][53885] Updated weights for policy 1, policy_version 63292 (0.0007) [2023-10-08 10:18:36,098][53852] Updated weights for policy 0, policy_version 63580 (0.0009) [2023-10-08 10:18:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 129925120. Throughput: 0: 1832.4, 1: 1815.7. Samples: 32487214. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 10:18:37,016][52710] Avg episode reward: [(0, '30.230'), (1, '35.680')] [2023-10-08 10:18:39,441][53885] Updated weights for policy 1, policy_version 63302 (0.0009) [2023-10-08 10:18:39,746][53852] Updated weights for policy 0, policy_version 63590 (0.0010) [2023-10-08 10:18:39,811][53885] Updated weights for policy 1, policy_version 63312 (0.0009) [2023-10-08 10:18:40,120][53852] Updated weights for policy 0, policy_version 63600 (0.0007) [2023-10-08 10:18:40,185][53885] Updated weights for policy 1, policy_version 63322 (0.0007) [2023-10-08 10:18:40,492][53852] Updated weights for policy 0, policy_version 63610 (0.0009) [2023-10-08 10:18:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 129990656. Throughput: 0: 1841.7, 1: 1817.7. Samples: 32499700. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 10:18:42,015][52710] Avg episode reward: [(0, '27.590'), (1, '36.180')] [2023-10-08 10:18:43,887][53885] Updated weights for policy 1, policy_version 63332 (0.0008) [2023-10-08 10:18:44,140][53852] Updated weights for policy 0, policy_version 63620 (0.0009) [2023-10-08 10:18:44,253][53885] Updated weights for policy 1, policy_version 63342 (0.0007) [2023-10-08 10:18:44,501][53852] Updated weights for policy 0, policy_version 63630 (0.0007) [2023-10-08 10:18:44,619][53885] Updated weights for policy 1, policy_version 63352 (0.0007) [2023-10-08 10:18:44,869][53852] Updated weights for policy 0, policy_version 63640 (0.0009) [2023-10-08 10:18:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130056192. Throughput: 0: 1831.5, 1: 1817.5. Samples: 32520004. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 10:18:47,016][52710] Avg episode reward: [(0, '30.440'), (1, '34.130')] [2023-10-08 10:18:48,367][53885] Updated weights for policy 1, policy_version 63362 (0.0007) [2023-10-08 10:18:48,502][53852] Updated weights for policy 0, policy_version 63650 (0.0009) [2023-10-08 10:18:48,735][53885] Updated weights for policy 1, policy_version 63372 (0.0008) [2023-10-08 10:18:48,877][53852] Updated weights for policy 0, policy_version 63660 (0.0007) [2023-10-08 10:18:49,096][53885] Updated weights for policy 1, policy_version 63382 (0.0009) [2023-10-08 10:18:49,241][53852] Updated weights for policy 0, policy_version 63670 (0.0007) [2023-10-08 10:18:49,459][53885] Updated weights for policy 1, policy_version 63392 (0.0008) [2023-10-08 10:18:49,615][53852] Updated weights for policy 0, policy_version 63680 (0.0008) [2023-10-08 10:18:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 130121728. Throughput: 0: 1840.4, 1: 1814.2. Samples: 32542998. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 10:18:52,016][52710] Avg episode reward: [(0, '32.140'), (1, '35.620')] [2023-10-08 10:18:53,163][53885] Updated weights for policy 1, policy_version 63402 (0.0008) [2023-10-08 10:18:53,214][53852] Updated weights for policy 0, policy_version 63690 (0.0007) [2023-10-08 10:18:53,520][53885] Updated weights for policy 1, policy_version 63412 (0.0007) [2023-10-08 10:18:53,590][53852] Updated weights for policy 0, policy_version 63700 (0.0008) [2023-10-08 10:18:53,890][53885] Updated weights for policy 1, policy_version 63422 (0.0007) [2023-10-08 10:18:53,955][53852] Updated weights for policy 0, policy_version 63710 (0.0008) [2023-10-08 10:18:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130187264. Throughput: 0: 1832.8, 1: 1812.8. Samples: 32552894. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 10:18:57,016][52710] Avg episode reward: [(0, '27.760'), (1, '32.460')] [2023-10-08 10:18:57,588][53885] Updated weights for policy 1, policy_version 63432 (0.0008) [2023-10-08 10:18:57,819][53852] Updated weights for policy 0, policy_version 63720 (0.0008) [2023-10-08 10:18:57,953][53885] Updated weights for policy 1, policy_version 63442 (0.0008) [2023-10-08 10:18:58,188][53852] Updated weights for policy 0, policy_version 63730 (0.0007) [2023-10-08 10:18:58,319][53885] Updated weights for policy 1, policy_version 63452 (0.0009) [2023-10-08 10:18:58,567][53852] Updated weights for policy 0, policy_version 63740 (0.0008) [2023-10-08 10:19:02,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 130252800. Throughput: 0: 1840.5, 1: 1810.9. Samples: 32575644. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 10:19:02,017][52710] Avg episode reward: [(0, '28.510'), (1, '30.490')] [2023-10-08 10:19:02,158][53885] Updated weights for policy 1, policy_version 63462 (0.0007) [2023-10-08 10:19:02,230][53852] Updated weights for policy 0, policy_version 63750 (0.0007) [2023-10-08 10:19:02,524][53885] Updated weights for policy 1, policy_version 63472 (0.0009) [2023-10-08 10:19:02,606][53852] Updated weights for policy 0, policy_version 63760 (0.0009) [2023-10-08 10:19:02,892][53885] Updated weights for policy 1, policy_version 63482 (0.0008) [2023-10-08 10:19:02,972][53852] Updated weights for policy 0, policy_version 63770 (0.0008) [2023-10-08 10:19:06,513][53885] Updated weights for policy 1, policy_version 63492 (0.0008) [2023-10-08 10:19:06,713][53852] Updated weights for policy 0, policy_version 63780 (0.0009) [2023-10-08 10:19:06,887][53885] Updated weights for policy 1, policy_version 63502 (0.0007) [2023-10-08 10:19:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130318336. Throughput: 0: 1838.5, 1: 1815.5. Samples: 32598194. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 10:19:07,016][52710] Avg episode reward: [(0, '33.720'), (1, '33.730')] [2023-10-08 10:19:07,082][53852] Updated weights for policy 0, policy_version 63790 (0.0009) [2023-10-08 10:19:07,258][53885] Updated weights for policy 1, policy_version 63512 (0.0008) [2023-10-08 10:19:07,450][53852] Updated weights for policy 0, policy_version 63800 (0.0008) [2023-10-08 10:19:11,020][53885] Updated weights for policy 1, policy_version 63522 (0.0009) [2023-10-08 10:19:11,097][53852] Updated weights for policy 0, policy_version 63810 (0.0007) [2023-10-08 10:19:11,427][53885] Updated weights for policy 1, policy_version 63532 (0.0007) [2023-10-08 10:19:11,459][53852] Updated weights for policy 0, policy_version 63820 (0.0008) [2023-10-08 10:19:11,797][53885] Updated weights for policy 1, policy_version 63542 (0.0008) [2023-10-08 10:19:11,836][53852] Updated weights for policy 0, policy_version 63830 (0.0007) [2023-10-08 10:19:12,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130383872. Throughput: 0: 1842.2, 1: 1811.7. Samples: 32608688. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) [2023-10-08 10:19:12,016][52710] Avg episode reward: [(0, '31.810'), (1, '29.740')] [2023-10-08 10:19:12,161][53885] Updated weights for policy 1, policy_version 63552 (0.0007) [2023-10-08 10:19:12,211][53852] Updated weights for policy 0, policy_version 63840 (0.0007) [2023-10-08 10:19:15,818][53885] Updated weights for policy 1, policy_version 63562 (0.0007) [2023-10-08 10:19:16,094][53852] Updated weights for policy 0, policy_version 63850 (0.0007) [2023-10-08 10:19:16,189][53885] Updated weights for policy 1, policy_version 63572 (0.0008) [2023-10-08 10:19:16,455][53852] Updated weights for policy 0, policy_version 63860 (0.0007) [2023-10-08 10:19:16,556][53885] Updated weights for policy 1, policy_version 63582 (0.0007) [2023-10-08 10:19:16,830][53852] Updated weights for policy 0, policy_version 63870 (0.0009) [2023-10-08 10:19:17,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 130514944. Throughput: 0: 1826.0, 1: 1813.2. Samples: 32630692. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 10:19:17,016][52710] Avg episode reward: [(0, '29.110'), (1, '27.820')] [2023-10-08 10:19:20,321][53885] Updated weights for policy 1, policy_version 63592 (0.0009) [2023-10-08 10:19:20,534][53852] Updated weights for policy 0, policy_version 63880 (0.0008) [2023-10-08 10:19:20,687][53885] Updated weights for policy 1, policy_version 63602 (0.0008) [2023-10-08 10:19:20,903][53852] Updated weights for policy 0, policy_version 63890 (0.0008) [2023-10-08 10:19:21,055][53885] Updated weights for policy 1, policy_version 63612 (0.0007) [2023-10-08 10:19:21,272][53852] Updated weights for policy 0, policy_version 63900 (0.0007) [2023-10-08 10:19:22,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 130580480. Throughput: 0: 1824.0, 1: 1810.9. Samples: 32650784. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 10:19:22,016][52710] Avg episode reward: [(0, '30.590'), (1, '32.110')] [2023-10-08 10:19:24,771][53885] Updated weights for policy 1, policy_version 63622 (0.0007) [2023-10-08 10:19:24,979][53852] Updated weights for policy 0, policy_version 63910 (0.0007) [2023-10-08 10:19:25,130][53885] Updated weights for policy 1, policy_version 63632 (0.0008) [2023-10-08 10:19:25,358][53852] Updated weights for policy 0, policy_version 63920 (0.0009) [2023-10-08 10:19:25,492][53885] Updated weights for policy 1, policy_version 63642 (0.0008) [2023-10-08 10:19:25,729][53852] Updated weights for policy 0, policy_version 63930 (0.0008) [2023-10-08 10:19:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130646016. Throughput: 0: 1828.2, 1: 1819.7. Samples: 32663856. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 10:19:27,016][52710] Avg episode reward: [(0, '31.220'), (1, '32.290')] [2023-10-08 10:19:29,069][53885] Updated weights for policy 1, policy_version 63652 (0.0008) [2023-10-08 10:19:29,164][53852] Updated weights for policy 0, policy_version 63940 (0.0008) [2023-10-08 10:19:29,436][53885] Updated weights for policy 1, policy_version 63662 (0.0009) [2023-10-08 10:19:29,526][53852] Updated weights for policy 0, policy_version 63950 (0.0007) [2023-10-08 10:19:29,797][53885] Updated weights for policy 1, policy_version 63672 (0.0008) [2023-10-08 10:19:29,894][53852] Updated weights for policy 0, policy_version 63960 (0.0009) [2023-10-08 10:19:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 130711552. Throughput: 0: 1828.1, 1: 1807.6. Samples: 32683610. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 10:19:32,016][52710] Avg episode reward: [(0, '31.350'), (1, '29.940')] [2023-10-08 10:19:33,500][53852] Updated weights for policy 0, policy_version 63970 (0.0009) [2023-10-08 10:19:33,671][53885] Updated weights for policy 1, policy_version 63682 (0.0008) [2023-10-08 10:19:33,872][53852] Updated weights for policy 0, policy_version 63980 (0.0007) [2023-10-08 10:19:34,038][53885] Updated weights for policy 1, policy_version 63692 (0.0007) [2023-10-08 10:19:34,246][53852] Updated weights for policy 0, policy_version 63990 (0.0008) [2023-10-08 10:19:34,408][53885] Updated weights for policy 1, policy_version 63702 (0.0008) [2023-10-08 10:19:34,609][53852] Updated weights for policy 0, policy_version 64000 (0.0009) [2023-10-08 10:19:34,780][53885] Updated weights for policy 1, policy_version 63712 (0.0007) [2023-10-08 10:19:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 130777088. Throughput: 0: 1820.5, 1: 1809.3. Samples: 32706340. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 10:19:37,016][52710] Avg episode reward: [(0, '30.810'), (1, '32.320')] [2023-10-08 10:19:38,304][53852] Updated weights for policy 0, policy_version 64010 (0.0007) [2023-10-08 10:19:38,381][53885] Updated weights for policy 1, policy_version 63722 (0.0007) [2023-10-08 10:19:38,671][53852] Updated weights for policy 0, policy_version 64020 (0.0007) [2023-10-08 10:19:38,744][53885] Updated weights for policy 1, policy_version 63732 (0.0007) [2023-10-08 10:19:39,039][53852] Updated weights for policy 0, policy_version 64030 (0.0008) [2023-10-08 10:19:39,108][53885] Updated weights for policy 1, policy_version 63742 (0.0008) [2023-10-08 10:19:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130842624. Throughput: 0: 1824.6, 1: 1809.9. Samples: 32716444. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 10:19:42,015][52710] Avg episode reward: [(0, '30.990'), (1, '32.180')] [2023-10-08 10:19:42,593][53852] Updated weights for policy 0, policy_version 64040 (0.0009) [2023-10-08 10:19:42,937][53885] Updated weights for policy 1, policy_version 63752 (0.0007) [2023-10-08 10:19:42,966][53852] Updated weights for policy 0, policy_version 64050 (0.0007) [2023-10-08 10:19:43,307][53885] Updated weights for policy 1, policy_version 63762 (0.0007) [2023-10-08 10:19:43,334][53852] Updated weights for policy 0, policy_version 64060 (0.0008) [2023-10-08 10:19:43,667][53885] Updated weights for policy 1, policy_version 63772 (0.0010) [2023-10-08 10:19:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 130908160. Throughput: 0: 1823.3, 1: 1812.3. Samples: 32739244. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 10:19:47,016][52710] Avg episode reward: [(0, '32.950'), (1, '35.070')] [2023-10-08 10:19:47,110][53852] Updated weights for policy 0, policy_version 64070 (0.0007) [2023-10-08 10:19:47,354][53885] Updated weights for policy 1, policy_version 63782 (0.0008) [2023-10-08 10:19:47,468][53852] Updated weights for policy 0, policy_version 64080 (0.0009) [2023-10-08 10:19:47,711][53885] Updated weights for policy 1, policy_version 63792 (0.0008) [2023-10-08 10:19:47,837][53852] Updated weights for policy 0, policy_version 64090 (0.0008) [2023-10-08 10:19:48,092][53885] Updated weights for policy 1, policy_version 63802 (0.0008) [2023-10-08 10:19:51,572][53852] Updated weights for policy 0, policy_version 64100 (0.0009) [2023-10-08 10:19:51,812][53885] Updated weights for policy 1, policy_version 63812 (0.0009) [2023-10-08 10:19:51,935][53852] Updated weights for policy 0, policy_version 64110 (0.0009) [2023-10-08 10:19:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 130973696. Throughput: 0: 1816.7, 1: 1814.5. Samples: 32761602. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) [2023-10-08 10:19:52,016][52710] Avg episode reward: [(0, '32.870'), (1, '32.210')] [2023-10-08 10:19:52,168][53885] Updated weights for policy 1, policy_version 63822 (0.0007) [2023-10-08 10:19:52,307][53852] Updated weights for policy 0, policy_version 64120 (0.0008) [2023-10-08 10:19:52,527][53885] Updated weights for policy 1, policy_version 63832 (0.0007) [2023-10-08 10:19:55,850][53852] Updated weights for policy 0, policy_version 64130 (0.0007) [2023-10-08 10:19:56,230][53852] Updated weights for policy 0, policy_version 64140 (0.0009) [2023-10-08 10:19:56,242][53885] Updated weights for policy 1, policy_version 63842 (0.0009) [2023-10-08 10:19:56,592][53852] Updated weights for policy 0, policy_version 64150 (0.0007) [2023-10-08 10:19:56,652][53885] Updated weights for policy 1, policy_version 63852 (0.0008) [2023-10-08 10:19:56,963][53852] Updated weights for policy 0, policy_version 64160 (0.0007) [2023-10-08 10:19:57,009][53885] Updated weights for policy 1, policy_version 63862 (0.0009) [2023-10-08 10:19:57,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131072000. Throughput: 0: 1818.0, 1: 1806.5. Samples: 32771790. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:19:57,016][52710] Avg episode reward: [(0, '30.750'), (1, '32.560')] [2023-10-08 10:19:57,372][53885] Updated weights for policy 1, policy_version 63872 (0.0009) [2023-10-08 10:20:00,574][53852] Updated weights for policy 0, policy_version 64170 (0.0009) [2023-10-08 10:20:00,944][53852] Updated weights for policy 0, policy_version 64180 (0.0008) [2023-10-08 10:20:01,015][53885] Updated weights for policy 1, policy_version 63882 (0.0009) [2023-10-08 10:20:01,308][53852] Updated weights for policy 0, policy_version 64190 (0.0007) [2023-10-08 10:20:01,374][53885] Updated weights for policy 1, policy_version 63892 (0.0007) [2023-10-08 10:20:01,745][53885] Updated weights for policy 1, policy_version 63902 (0.0007) [2023-10-08 10:20:02,015][52710] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 131170304. Throughput: 0: 1819.1, 1: 1818.0. Samples: 32794360. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:20:02,016][52710] Avg episode reward: [(0, '34.170'), (1, '32.330')] [2023-10-08 10:20:05,050][53852] Updated weights for policy 0, policy_version 64200 (0.0008) [2023-10-08 10:20:05,417][53852] Updated weights for policy 0, policy_version 64210 (0.0009) [2023-10-08 10:20:05,451][53885] Updated weights for policy 1, policy_version 63912 (0.0008) [2023-10-08 10:20:05,787][53852] Updated weights for policy 0, policy_version 64220 (0.0009) [2023-10-08 10:20:05,826][53885] Updated weights for policy 1, policy_version 63922 (0.0008) [2023-10-08 10:20:06,185][53885] Updated weights for policy 1, policy_version 63932 (0.0009) [2023-10-08 10:20:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 131235840. Throughput: 0: 1827.4, 1: 1816.4. Samples: 32814756. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:20:07,016][52710] Avg episode reward: [(0, '30.790'), (1, '35.210')] [2023-10-08 10:20:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000063936_65470464.pth... [2023-10-08 10:20:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000064224_65765376.pth... [2023-10-08 10:20:07,063][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000062496_63995904.pth [2023-10-08 10:20:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000062240_63733760.pth [2023-10-08 10:20:09,730][53852] Updated weights for policy 0, policy_version 64230 (0.0008) [2023-10-08 10:20:09,991][53885] Updated weights for policy 1, policy_version 63942 (0.0008) [2023-10-08 10:20:10,113][53852] Updated weights for policy 0, policy_version 64240 (0.0007) [2023-10-08 10:20:10,351][53885] Updated weights for policy 1, policy_version 63952 (0.0008) [2023-10-08 10:20:10,486][53852] Updated weights for policy 0, policy_version 64250 (0.0007) [2023-10-08 10:20:10,722][53885] Updated weights for policy 1, policy_version 63962 (0.0008) [2023-10-08 10:20:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 131301376. Throughput: 0: 1817.6, 1: 1814.3. Samples: 32827294. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:20:12,016][52710] Avg episode reward: [(0, '30.010'), (1, '31.270')] [2023-10-08 10:20:14,050][53852] Updated weights for policy 0, policy_version 64260 (0.0008) [2023-10-08 10:20:14,422][53852] Updated weights for policy 0, policy_version 64270 (0.0009) [2023-10-08 10:20:14,485][53885] Updated weights for policy 1, policy_version 63972 (0.0008) [2023-10-08 10:20:14,786][53852] Updated weights for policy 0, policy_version 64280 (0.0008) [2023-10-08 10:20:14,857][53885] Updated weights for policy 1, policy_version 63982 (0.0009) [2023-10-08 10:20:15,226][53885] Updated weights for policy 1, policy_version 63992 (0.0007) [2023-10-08 10:20:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 131366912. Throughput: 0: 1821.6, 1: 1814.3. Samples: 32847226. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:20:17,016][52710] Avg episode reward: [(0, '30.740'), (1, '31.490')] [2023-10-08 10:20:18,578][53852] Updated weights for policy 0, policy_version 64290 (0.0007) [2023-10-08 10:20:18,912][53885] Updated weights for policy 1, policy_version 64002 (0.0009) [2023-10-08 10:20:18,940][53852] Updated weights for policy 0, policy_version 64300 (0.0008) [2023-10-08 10:20:19,277][53885] Updated weights for policy 1, policy_version 64012 (0.0008) [2023-10-08 10:20:19,318][53852] Updated weights for policy 0, policy_version 64310 (0.0008) [2023-10-08 10:20:19,643][53885] Updated weights for policy 1, policy_version 64022 (0.0008) [2023-10-08 10:20:19,678][53852] Updated weights for policy 0, policy_version 64320 (0.0007) [2023-10-08 10:20:20,007][53885] Updated weights for policy 1, policy_version 64032 (0.0010) [2023-10-08 10:20:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131432448. Throughput: 0: 1824.4, 1: 1817.7. Samples: 32870238. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:20:22,016][52710] Avg episode reward: [(0, '32.190'), (1, '33.350')] [2023-10-08 10:20:23,329][53852] Updated weights for policy 0, policy_version 64330 (0.0007) [2023-10-08 10:20:23,646][53885] Updated weights for policy 1, policy_version 64042 (0.0007) [2023-10-08 10:20:23,693][53852] Updated weights for policy 0, policy_version 64340 (0.0008) [2023-10-08 10:20:24,013][53885] Updated weights for policy 1, policy_version 64052 (0.0007) [2023-10-08 10:20:24,066][53852] Updated weights for policy 0, policy_version 64350 (0.0008) [2023-10-08 10:20:24,383][53885] Updated weights for policy 1, policy_version 64062 (0.0008) [2023-10-08 10:20:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 131497984. Throughput: 0: 1822.6, 1: 1815.5. Samples: 32880158. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:20:27,016][52710] Avg episode reward: [(0, '30.040'), (1, '33.640')] [2023-10-08 10:20:27,700][53852] Updated weights for policy 0, policy_version 64360 (0.0008) [2023-10-08 10:20:28,065][53852] Updated weights for policy 0, policy_version 64370 (0.0007) [2023-10-08 10:20:28,089][53885] Updated weights for policy 1, policy_version 64072 (0.0007) [2023-10-08 10:20:28,430][53852] Updated weights for policy 0, policy_version 64380 (0.0009) [2023-10-08 10:20:28,455][53885] Updated weights for policy 1, policy_version 64082 (0.0007) [2023-10-08 10:20:28,830][53885] Updated weights for policy 1, policy_version 64092 (0.0007) [2023-10-08 10:20:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131563520. Throughput: 0: 1824.7, 1: 1813.0. Samples: 32902938. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:20:32,015][52710] Avg episode reward: [(0, '31.820'), (1, '36.810')] [2023-10-08 10:20:32,220][53852] Updated weights for policy 0, policy_version 64390 (0.0008) [2023-10-08 10:20:32,587][53852] Updated weights for policy 0, policy_version 64400 (0.0008) [2023-10-08 10:20:32,622][53885] Updated weights for policy 1, policy_version 64102 (0.0010) [2023-10-08 10:20:32,961][53852] Updated weights for policy 0, policy_version 64410 (0.0008) [2023-10-08 10:20:32,984][53885] Updated weights for policy 1, policy_version 64112 (0.0008) [2023-10-08 10:20:33,357][53885] Updated weights for policy 1, policy_version 64122 (0.0007) [2023-10-08 10:20:36,674][53852] Updated weights for policy 0, policy_version 64420 (0.0009) [2023-10-08 10:20:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131629056. Throughput: 0: 1826.9, 1: 1812.6. Samples: 32925380. Policy #0 lag: (min: 26.0, avg: 30.6, max: 58.0) [2023-10-08 10:20:37,016][52710] Avg episode reward: [(0, '32.590'), (1, '38.720')] [2023-10-08 10:20:37,047][53852] Updated weights for policy 0, policy_version 64430 (0.0008) [2023-10-08 10:20:37,058][53885] Updated weights for policy 1, policy_version 64132 (0.0007) [2023-10-08 10:20:37,412][53852] Updated weights for policy 0, policy_version 64440 (0.0009) [2023-10-08 10:20:37,418][53885] Updated weights for policy 1, policy_version 64142 (0.0007) [2023-10-08 10:20:37,781][53885] Updated weights for policy 1, policy_version 64152 (0.0007) [2023-10-08 10:20:38,074][53594] Saving new best policy, reward=38.720! [2023-10-08 10:20:41,066][53852] Updated weights for policy 0, policy_version 64450 (0.0008) [2023-10-08 10:20:41,427][53852] Updated weights for policy 0, policy_version 64460 (0.0007) [2023-10-08 10:20:41,479][53885] Updated weights for policy 1, policy_version 64162 (0.0009) [2023-10-08 10:20:41,805][53852] Updated weights for policy 0, policy_version 64470 (0.0008) [2023-10-08 10:20:41,883][53885] Updated weights for policy 1, policy_version 64172 (0.0008) [2023-10-08 10:20:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 131694592. Throughput: 0: 1820.0, 1: 1812.7. Samples: 32935260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:20:42,016][52710] Avg episode reward: [(0, '30.030'), (1, '37.950')] [2023-10-08 10:20:42,177][53852] Updated weights for policy 0, policy_version 64480 (0.0009) [2023-10-08 10:20:42,257][53885] Updated weights for policy 1, policy_version 64182 (0.0008) [2023-10-08 10:20:42,619][53885] Updated weights for policy 1, policy_version 64192 (0.0009) [2023-10-08 10:20:45,877][53852] Updated weights for policy 0, policy_version 64490 (0.0011) [2023-10-08 10:20:46,237][53852] Updated weights for policy 0, policy_version 64500 (0.0008) [2023-10-08 10:20:46,284][53885] Updated weights for policy 1, policy_version 64202 (0.0009) [2023-10-08 10:20:46,606][53852] Updated weights for policy 0, policy_version 64510 (0.0008) [2023-10-08 10:20:46,643][53885] Updated weights for policy 1, policy_version 64212 (0.0007) [2023-10-08 10:20:47,007][53885] Updated weights for policy 1, policy_version 64222 (0.0008) [2023-10-08 10:20:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131792896. Throughput: 0: 1826.3, 1: 1809.1. Samples: 32957950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:20:47,016][52710] Avg episode reward: [(0, '27.690'), (1, '38.440')] [2023-10-08 10:20:50,401][53852] Updated weights for policy 0, policy_version 64520 (0.0009) [2023-10-08 10:20:50,521][53885] Updated weights for policy 1, policy_version 64232 (0.0009) [2023-10-08 10:20:50,767][53852] Updated weights for policy 0, policy_version 64530 (0.0008) [2023-10-08 10:20:50,891][53885] Updated weights for policy 1, policy_version 64242 (0.0009) [2023-10-08 10:20:51,133][53852] Updated weights for policy 0, policy_version 64540 (0.0007) [2023-10-08 10:20:51,258][53885] Updated weights for policy 1, policy_version 64252 (0.0009) [2023-10-08 10:20:52,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 131891200. Throughput: 0: 1817.1, 1: 1807.0. Samples: 32977840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:20:52,016][52710] Avg episode reward: [(0, '28.570'), (1, '38.600')] [2023-10-08 10:20:54,871][53852] Updated weights for policy 0, policy_version 64550 (0.0008) [2023-10-08 10:20:55,109][53885] Updated weights for policy 1, policy_version 64262 (0.0007) [2023-10-08 10:20:55,252][53852] Updated weights for policy 0, policy_version 64560 (0.0007) [2023-10-08 10:20:55,471][53885] Updated weights for policy 1, policy_version 64272 (0.0007) [2023-10-08 10:20:55,618][53852] Updated weights for policy 0, policy_version 64570 (0.0007) [2023-10-08 10:20:55,836][53885] Updated weights for policy 1, policy_version 64282 (0.0009) [2023-10-08 10:20:57,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131956736. Throughput: 0: 1821.8, 1: 1805.4. Samples: 32990518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:20:57,016][52710] Avg episode reward: [(0, '29.840'), (1, '33.220')] [2023-10-08 10:20:59,363][53852] Updated weights for policy 0, policy_version 64580 (0.0007) [2023-10-08 10:20:59,736][53852] Updated weights for policy 0, policy_version 64590 (0.0007) [2023-10-08 10:20:59,764][53885] Updated weights for policy 1, policy_version 64292 (0.0008) [2023-10-08 10:21:00,108][53852] Updated weights for policy 0, policy_version 64600 (0.0007) [2023-10-08 10:21:00,125][53885] Updated weights for policy 1, policy_version 64302 (0.0008) [2023-10-08 10:21:00,505][53885] Updated weights for policy 1, policy_version 64312 (0.0009) [2023-10-08 10:21:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 132022272. Throughput: 0: 1813.1, 1: 1813.4. Samples: 33010420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:02,016][52710] Avg episode reward: [(0, '30.390'), (1, '34.440')] [2023-10-08 10:21:03,756][53852] Updated weights for policy 0, policy_version 64610 (0.0008) [2023-10-08 10:21:04,126][53852] Updated weights for policy 0, policy_version 64620 (0.0008) [2023-10-08 10:21:04,210][53885] Updated weights for policy 1, policy_version 64322 (0.0009) [2023-10-08 10:21:04,489][53852] Updated weights for policy 0, policy_version 64630 (0.0007) [2023-10-08 10:21:04,571][53885] Updated weights for policy 1, policy_version 64332 (0.0007) [2023-10-08 10:21:04,861][53852] Updated weights for policy 0, policy_version 64640 (0.0008) [2023-10-08 10:21:04,938][53885] Updated weights for policy 1, policy_version 64342 (0.0008) [2023-10-08 10:21:05,308][53885] Updated weights for policy 1, policy_version 64352 (0.0010) [2023-10-08 10:21:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 132087808. Throughput: 0: 1816.2, 1: 1800.7. Samples: 33032996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:07,016][52710] Avg episode reward: [(0, '31.020'), (1, '38.250')] [2023-10-08 10:21:08,602][53852] Updated weights for policy 0, policy_version 64650 (0.0009) [2023-10-08 10:21:08,957][53885] Updated weights for policy 1, policy_version 64362 (0.0008) [2023-10-08 10:21:08,969][53852] Updated weights for policy 0, policy_version 64660 (0.0009) [2023-10-08 10:21:09,328][53885] Updated weights for policy 1, policy_version 64372 (0.0007) [2023-10-08 10:21:09,334][53852] Updated weights for policy 0, policy_version 64670 (0.0009) [2023-10-08 10:21:09,702][53885] Updated weights for policy 1, policy_version 64382 (0.0011) [2023-10-08 10:21:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132153344. Throughput: 0: 1812.1, 1: 1810.9. Samples: 33043196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:12,015][52710] Avg episode reward: [(0, '30.990'), (1, '37.400')] [2023-10-08 10:21:12,896][53852] Updated weights for policy 0, policy_version 64680 (0.0008) [2023-10-08 10:21:13,221][53885] Updated weights for policy 1, policy_version 64392 (0.0009) [2023-10-08 10:21:13,266][53852] Updated weights for policy 0, policy_version 64690 (0.0007) [2023-10-08 10:21:13,575][53885] Updated weights for policy 1, policy_version 64402 (0.0008) [2023-10-08 10:21:13,641][53852] Updated weights for policy 0, policy_version 64700 (0.0007) [2023-10-08 10:21:13,937][53885] Updated weights for policy 1, policy_version 64412 (0.0010) [2023-10-08 10:21:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132218880. Throughput: 0: 1809.6, 1: 1810.8. Samples: 33065856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:17,016][52710] Avg episode reward: [(0, '30.430'), (1, '34.500')] [2023-10-08 10:21:17,494][53852] Updated weights for policy 0, policy_version 64710 (0.0008) [2023-10-08 10:21:17,738][53885] Updated weights for policy 1, policy_version 64422 (0.0010) [2023-10-08 10:21:17,858][53852] Updated weights for policy 0, policy_version 64720 (0.0008) [2023-10-08 10:21:18,097][53885] Updated weights for policy 1, policy_version 64432 (0.0009) [2023-10-08 10:21:18,235][53852] Updated weights for policy 0, policy_version 64730 (0.0007) [2023-10-08 10:21:18,456][53885] Updated weights for policy 1, policy_version 64442 (0.0008) [2023-10-08 10:21:21,979][53852] Updated weights for policy 0, policy_version 64740 (0.0009) [2023-10-08 10:21:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132284416. Throughput: 0: 1809.1, 1: 1813.2. Samples: 33088384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:22,016][52710] Avg episode reward: [(0, '31.470'), (1, '37.890')] [2023-10-08 10:21:22,129][53885] Updated weights for policy 1, policy_version 64452 (0.0008) [2023-10-08 10:21:22,345][53852] Updated weights for policy 0, policy_version 64750 (0.0008) [2023-10-08 10:21:22,489][53885] Updated weights for policy 1, policy_version 64462 (0.0007) [2023-10-08 10:21:22,714][53852] Updated weights for policy 0, policy_version 64760 (0.0008) [2023-10-08 10:21:22,853][53885] Updated weights for policy 1, policy_version 64472 (0.0008) [2023-10-08 10:21:26,276][53852] Updated weights for policy 0, policy_version 64770 (0.0008) [2023-10-08 10:21:26,618][53885] Updated weights for policy 1, policy_version 64482 (0.0007) [2023-10-08 10:21:26,651][53852] Updated weights for policy 0, policy_version 64780 (0.0009) [2023-10-08 10:21:27,010][53852] Updated weights for policy 0, policy_version 64790 (0.0008) [2023-10-08 10:21:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132349952. Throughput: 0: 1811.8, 1: 1813.8. Samples: 33098410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:27,015][52710] Avg episode reward: [(0, '29.740'), (1, '36.950')] [2023-10-08 10:21:27,027][53885] Updated weights for policy 1, policy_version 64492 (0.0008) [2023-10-08 10:21:27,375][53852] Updated weights for policy 0, policy_version 64800 (0.0008) [2023-10-08 10:21:27,402][53885] Updated weights for policy 1, policy_version 64502 (0.0008) [2023-10-08 10:21:27,762][53885] Updated weights for policy 1, policy_version 64512 (0.0011) [2023-10-08 10:21:31,072][53852] Updated weights for policy 0, policy_version 64810 (0.0009) [2023-10-08 10:21:31,411][53885] Updated weights for policy 1, policy_version 64522 (0.0008) [2023-10-08 10:21:31,439][53852] Updated weights for policy 0, policy_version 64820 (0.0008) [2023-10-08 10:21:31,774][53885] Updated weights for policy 1, policy_version 64532 (0.0007) [2023-10-08 10:21:31,806][53852] Updated weights for policy 0, policy_version 64830 (0.0008) [2023-10-08 10:21:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132448256. Throughput: 0: 1813.7, 1: 1812.9. Samples: 33121150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:32,016][52710] Avg episode reward: [(0, '30.910'), (1, '36.100')] [2023-10-08 10:21:32,140][53885] Updated weights for policy 1, policy_version 64542 (0.0008) [2023-10-08 10:21:35,372][53852] Updated weights for policy 0, policy_version 64840 (0.0008) [2023-10-08 10:21:35,739][53852] Updated weights for policy 0, policy_version 64850 (0.0008) [2023-10-08 10:21:35,790][53885] Updated weights for policy 1, policy_version 64552 (0.0007) [2023-10-08 10:21:36,116][53852] Updated weights for policy 0, policy_version 64860 (0.0009) [2023-10-08 10:21:36,161][53885] Updated weights for policy 1, policy_version 64562 (0.0007) [2023-10-08 10:21:36,526][53885] Updated weights for policy 1, policy_version 64572 (0.0008) [2023-10-08 10:21:37,015][52710] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 132546560. Throughput: 0: 1815.9, 1: 1817.7. Samples: 33141350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:37,016][52710] Avg episode reward: [(0, '32.170'), (1, '34.050')] [2023-10-08 10:21:39,843][53852] Updated weights for policy 0, policy_version 64870 (0.0009) [2023-10-08 10:21:40,185][53885] Updated weights for policy 1, policy_version 64582 (0.0007) [2023-10-08 10:21:40,217][53852] Updated weights for policy 0, policy_version 64880 (0.0007) [2023-10-08 10:21:40,556][53885] Updated weights for policy 1, policy_version 64592 (0.0008) [2023-10-08 10:21:40,577][53852] Updated weights for policy 0, policy_version 64890 (0.0007) [2023-10-08 10:21:40,913][53885] Updated weights for policy 1, policy_version 64602 (0.0009) [2023-10-08 10:21:42,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 132612096. Throughput: 0: 1821.2, 1: 1819.3. Samples: 33154340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:42,016][52710] Avg episode reward: [(0, '31.940'), (1, '34.580')] [2023-10-08 10:21:44,197][53852] Updated weights for policy 0, policy_version 64900 (0.0007) [2023-10-08 10:21:44,511][53885] Updated weights for policy 1, policy_version 64612 (0.0009) [2023-10-08 10:21:44,566][53852] Updated weights for policy 0, policy_version 64910 (0.0008) [2023-10-08 10:21:44,872][53885] Updated weights for policy 1, policy_version 64622 (0.0009) [2023-10-08 10:21:44,940][53852] Updated weights for policy 0, policy_version 64920 (0.0007) [2023-10-08 10:21:45,234][53885] Updated weights for policy 1, policy_version 64632 (0.0009) [2023-10-08 10:21:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132677632. Throughput: 0: 1825.2, 1: 1811.2. Samples: 33174060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:47,016][52710] Avg episode reward: [(0, '32.800'), (1, '35.020')] [2023-10-08 10:21:48,736][53852] Updated weights for policy 0, policy_version 64930 (0.0007) [2023-10-08 10:21:49,022][53885] Updated weights for policy 1, policy_version 64642 (0.0010) [2023-10-08 10:21:49,094][53852] Updated weights for policy 0, policy_version 64940 (0.0008) [2023-10-08 10:21:49,390][53885] Updated weights for policy 1, policy_version 64652 (0.0007) [2023-10-08 10:21:49,467][53852] Updated weights for policy 0, policy_version 64950 (0.0007) [2023-10-08 10:21:49,757][53885] Updated weights for policy 1, policy_version 64662 (0.0007) [2023-10-08 10:21:49,838][53852] Updated weights for policy 0, policy_version 64960 (0.0008) [2023-10-08 10:21:50,136][53885] Updated weights for policy 1, policy_version 64672 (0.0009) [2023-10-08 10:21:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 132743168. Throughput: 0: 1816.6, 1: 1816.4. Samples: 33196480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:52,015][52710] Avg episode reward: [(0, '31.300'), (1, '34.830')] [2023-10-08 10:21:53,552][53852] Updated weights for policy 0, policy_version 64970 (0.0010) [2023-10-08 10:21:53,899][53885] Updated weights for policy 1, policy_version 64682 (0.0007) [2023-10-08 10:21:53,921][53852] Updated weights for policy 0, policy_version 64980 (0.0008) [2023-10-08 10:21:54,259][53885] Updated weights for policy 1, policy_version 64692 (0.0008) [2023-10-08 10:21:54,294][53852] Updated weights for policy 0, policy_version 64990 (0.0009) [2023-10-08 10:21:54,636][53885] Updated weights for policy 1, policy_version 64702 (0.0008) [2023-10-08 10:21:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132808704. Throughput: 0: 1818.0, 1: 1815.6. Samples: 33206712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:21:57,016][52710] Avg episode reward: [(0, '30.990'), (1, '33.840')] [2023-10-08 10:21:57,815][53852] Updated weights for policy 0, policy_version 65000 (0.0008) [2023-10-08 10:21:58,185][53885] Updated weights for policy 1, policy_version 64712 (0.0009) [2023-10-08 10:21:58,187][53852] Updated weights for policy 0, policy_version 65010 (0.0008) [2023-10-08 10:21:58,548][53852] Updated weights for policy 0, policy_version 65020 (0.0010) [2023-10-08 10:21:58,555][53885] Updated weights for policy 1, policy_version 64722 (0.0007) [2023-10-08 10:21:58,920][53885] Updated weights for policy 1, policy_version 64732 (0.0009) [2023-10-08 10:22:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 132874240. Throughput: 0: 1826.9, 1: 1817.2. Samples: 33229842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:02,016][52710] Avg episode reward: [(0, '32.150'), (1, '38.250')] [2023-10-08 10:22:02,202][53852] Updated weights for policy 0, policy_version 65030 (0.0007) [2023-10-08 10:22:02,574][53852] Updated weights for policy 0, policy_version 65040 (0.0008) [2023-10-08 10:22:02,595][53885] Updated weights for policy 1, policy_version 64742 (0.0007) [2023-10-08 10:22:02,944][53852] Updated weights for policy 0, policy_version 65050 (0.0008) [2023-10-08 10:22:02,969][53885] Updated weights for policy 1, policy_version 64752 (0.0007) [2023-10-08 10:22:03,349][53885] Updated weights for policy 1, policy_version 64762 (0.0009) [2023-10-08 10:22:06,543][53852] Updated weights for policy 0, policy_version 65060 (0.0009) [2023-10-08 10:22:06,918][53852] Updated weights for policy 0, policy_version 65070 (0.0007) [2023-10-08 10:22:07,014][53885] Updated weights for policy 1, policy_version 64772 (0.0009) [2023-10-08 10:22:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132939776. Throughput: 0: 1832.6, 1: 1819.3. Samples: 33252722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:07,016][52710] Avg episode reward: [(0, '30.980'), (1, '36.550')] [2023-10-08 10:22:07,278][53852] Updated weights for policy 0, policy_version 65080 (0.0009) [2023-10-08 10:22:07,374][53885] Updated weights for policy 1, policy_version 64782 (0.0008) [2023-10-08 10:22:07,570][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000065088_66650112.pth... [2023-10-08 10:22:07,603][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000063360_64880640.pth [2023-10-08 10:22:07,724][53885] Updated weights for policy 1, policy_version 64792 (0.0007) [2023-10-08 10:22:08,018][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000064800_66355200.pth... [2023-10-08 10:22:08,059][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000063072_64585728.pth [2023-10-08 10:22:10,883][53852] Updated weights for policy 0, policy_version 65090 (0.0008) [2023-10-08 10:22:11,249][53852] Updated weights for policy 0, policy_version 65100 (0.0010) [2023-10-08 10:22:11,585][53885] Updated weights for policy 1, policy_version 64802 (0.0009) [2023-10-08 10:22:11,617][53852] Updated weights for policy 0, policy_version 65110 (0.0009) [2023-10-08 10:22:11,960][53885] Updated weights for policy 1, policy_version 64812 (0.0007) [2023-10-08 10:22:11,985][53852] Updated weights for policy 0, policy_version 65120 (0.0007) [2023-10-08 10:22:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 133038080. Throughput: 0: 1835.9, 1: 1819.4. Samples: 33262900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:12,016][52710] Avg episode reward: [(0, '31.100'), (1, '33.940')] [2023-10-08 10:22:12,333][53885] Updated weights for policy 1, policy_version 64822 (0.0007) [2023-10-08 10:22:12,702][53885] Updated weights for policy 1, policy_version 64832 (0.0007) [2023-10-08 10:22:15,686][53852] Updated weights for policy 0, policy_version 65130 (0.0009) [2023-10-08 10:22:16,069][53852] Updated weights for policy 0, policy_version 65140 (0.0008) [2023-10-08 10:22:16,435][53852] Updated weights for policy 0, policy_version 65150 (0.0008) [2023-10-08 10:22:16,494][53885] Updated weights for policy 1, policy_version 64842 (0.0008) [2023-10-08 10:22:16,854][53885] Updated weights for policy 1, policy_version 64852 (0.0007) [2023-10-08 10:22:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133103616. Throughput: 0: 1825.8, 1: 1824.2. Samples: 33285402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:17,015][52710] Avg episode reward: [(0, '33.990'), (1, '34.190')] [2023-10-08 10:22:17,223][53885] Updated weights for policy 1, policy_version 64862 (0.0008) [2023-10-08 10:22:20,102][53852] Updated weights for policy 0, policy_version 65160 (0.0009) [2023-10-08 10:22:20,486][53852] Updated weights for policy 0, policy_version 65170 (0.0009) [2023-10-08 10:22:20,843][53885] Updated weights for policy 1, policy_version 64872 (0.0008) [2023-10-08 10:22:20,858][53852] Updated weights for policy 0, policy_version 65180 (0.0008) [2023-10-08 10:22:21,209][53885] Updated weights for policy 1, policy_version 64882 (0.0009) [2023-10-08 10:22:21,586][53885] Updated weights for policy 1, policy_version 64892 (0.0010) [2023-10-08 10:22:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 133201920. Throughput: 0: 1829.6, 1: 1825.8. Samples: 33305842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:22,016][52710] Avg episode reward: [(0, '34.270'), (1, '36.710')] [2023-10-08 10:22:24,699][53852] Updated weights for policy 0, policy_version 65190 (0.0011) [2023-10-08 10:22:25,092][53852] Updated weights for policy 0, policy_version 65200 (0.0008) [2023-10-08 10:22:25,277][53885] Updated weights for policy 1, policy_version 64902 (0.0008) [2023-10-08 10:22:25,468][53852] Updated weights for policy 0, policy_version 65210 (0.0008) [2023-10-08 10:22:25,652][53885] Updated weights for policy 1, policy_version 64912 (0.0008) [2023-10-08 10:22:26,023][53885] Updated weights for policy 1, policy_version 64922 (0.0007) [2023-10-08 10:22:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 133267456. Throughput: 0: 1820.6, 1: 1821.4. Samples: 33318230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:27,016][52710] Avg episode reward: [(0, '34.060'), (1, '32.620')] [2023-10-08 10:22:29,013][53852] Updated weights for policy 0, policy_version 65220 (0.0007) [2023-10-08 10:22:29,384][53852] Updated weights for policy 0, policy_version 65230 (0.0007) [2023-10-08 10:22:29,760][53852] Updated weights for policy 0, policy_version 65240 (0.0007) [2023-10-08 10:22:29,807][53885] Updated weights for policy 1, policy_version 64932 (0.0008) [2023-10-08 10:22:30,173][53885] Updated weights for policy 1, policy_version 64942 (0.0008) [2023-10-08 10:22:30,539][53885] Updated weights for policy 1, policy_version 64952 (0.0008) [2023-10-08 10:22:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 133332992. Throughput: 0: 1828.0, 1: 1828.7. Samples: 33338612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:32,016][52710] Avg episode reward: [(0, '33.640'), (1, '36.690')] [2023-10-08 10:22:33,393][53852] Updated weights for policy 0, policy_version 65250 (0.0008) [2023-10-08 10:22:33,763][53852] Updated weights for policy 0, policy_version 65260 (0.0008) [2023-10-08 10:22:34,130][53852] Updated weights for policy 0, policy_version 65270 (0.0009) [2023-10-08 10:22:34,183][53885] Updated weights for policy 1, policy_version 64962 (0.0009) [2023-10-08 10:22:34,502][53852] Updated weights for policy 0, policy_version 65280 (0.0008) [2023-10-08 10:22:34,544][53885] Updated weights for policy 1, policy_version 64972 (0.0008) [2023-10-08 10:22:34,914][53885] Updated weights for policy 1, policy_version 64982 (0.0008) [2023-10-08 10:22:35,278][53885] Updated weights for policy 1, policy_version 64992 (0.0009) [2023-10-08 10:22:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 133398528. Throughput: 0: 1836.1, 1: 1825.5. Samples: 33361254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:37,016][52710] Avg episode reward: [(0, '33.320'), (1, '34.500')] [2023-10-08 10:22:38,258][53852] Updated weights for policy 0, policy_version 65290 (0.0008) [2023-10-08 10:22:38,624][53852] Updated weights for policy 0, policy_version 65300 (0.0007) [2023-10-08 10:22:38,941][53885] Updated weights for policy 1, policy_version 65002 (0.0007) [2023-10-08 10:22:38,997][53852] Updated weights for policy 0, policy_version 65310 (0.0008) [2023-10-08 10:22:39,305][53885] Updated weights for policy 1, policy_version 65012 (0.0007) [2023-10-08 10:22:39,673][53885] Updated weights for policy 1, policy_version 65022 (0.0009) [2023-10-08 10:22:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133464064. Throughput: 0: 1835.3, 1: 1824.4. Samples: 33371396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:42,016][52710] Avg episode reward: [(0, '32.590'), (1, '35.650')] [2023-10-08 10:22:42,654][53852] Updated weights for policy 0, policy_version 65320 (0.0009) [2023-10-08 10:22:43,033][53852] Updated weights for policy 0, policy_version 65330 (0.0009) [2023-10-08 10:22:43,265][53885] Updated weights for policy 1, policy_version 65032 (0.0007) [2023-10-08 10:22:43,400][53852] Updated weights for policy 0, policy_version 65340 (0.0007) [2023-10-08 10:22:43,629][53885] Updated weights for policy 1, policy_version 65042 (0.0007) [2023-10-08 10:22:43,994][53885] Updated weights for policy 1, policy_version 65052 (0.0009) [2023-10-08 10:22:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 133529600. Throughput: 0: 1821.8, 1: 1820.1. Samples: 33393726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:47,016][52710] Avg episode reward: [(0, '30.780'), (1, '37.920')] [2023-10-08 10:22:47,184][53852] Updated weights for policy 0, policy_version 65350 (0.0007) [2023-10-08 10:22:47,550][53852] Updated weights for policy 0, policy_version 65360 (0.0007) [2023-10-08 10:22:47,699][53885] Updated weights for policy 1, policy_version 65062 (0.0010) [2023-10-08 10:22:47,917][53852] Updated weights for policy 0, policy_version 65370 (0.0008) [2023-10-08 10:22:48,059][53885] Updated weights for policy 1, policy_version 65072 (0.0010) [2023-10-08 10:22:48,428][53885] Updated weights for policy 1, policy_version 65082 (0.0010) [2023-10-08 10:22:51,498][53852] Updated weights for policy 0, policy_version 65380 (0.0007) [2023-10-08 10:22:51,866][53852] Updated weights for policy 0, policy_version 65390 (0.0009) [2023-10-08 10:22:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 133595136. Throughput: 0: 1819.3, 1: 1819.2. Samples: 33416454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:52,016][52710] Avg episode reward: [(0, '29.770'), (1, '34.510')] [2023-10-08 10:22:52,126][53885] Updated weights for policy 1, policy_version 65092 (0.0009) [2023-10-08 10:22:52,230][53852] Updated weights for policy 0, policy_version 65400 (0.0008) [2023-10-08 10:22:52,502][53885] Updated weights for policy 1, policy_version 65102 (0.0007) [2023-10-08 10:22:52,861][53885] Updated weights for policy 1, policy_version 65112 (0.0007) [2023-10-08 10:22:55,850][53852] Updated weights for policy 0, policy_version 65410 (0.0008) [2023-10-08 10:22:56,216][53852] Updated weights for policy 0, policy_version 65420 (0.0010) [2023-10-08 10:22:56,581][53852] Updated weights for policy 0, policy_version 65430 (0.0007) [2023-10-08 10:22:56,654][53885] Updated weights for policy 1, policy_version 65122 (0.0009) [2023-10-08 10:22:56,949][53852] Updated weights for policy 0, policy_version 65440 (0.0007) [2023-10-08 10:22:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133693440. Throughput: 0: 1823.1, 1: 1819.3. Samples: 33426808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:22:57,016][52710] Avg episode reward: [(0, '31.500'), (1, '33.470')] [2023-10-08 10:22:57,028][53885] Updated weights for policy 1, policy_version 65132 (0.0008) [2023-10-08 10:22:57,397][53885] Updated weights for policy 1, policy_version 65142 (0.0008) [2023-10-08 10:22:57,766][53885] Updated weights for policy 1, policy_version 65152 (0.0009) [2023-10-08 10:23:00,521][53852] Updated weights for policy 0, policy_version 65450 (0.0009) [2023-10-08 10:23:00,880][53852] Updated weights for policy 0, policy_version 65460 (0.0008) [2023-10-08 10:23:01,247][53852] Updated weights for policy 0, policy_version 65470 (0.0008) [2023-10-08 10:23:01,349][53885] Updated weights for policy 1, policy_version 65162 (0.0008) [2023-10-08 10:23:01,712][53885] Updated weights for policy 1, policy_version 65172 (0.0008) [2023-10-08 10:23:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133758976. Throughput: 0: 1821.9, 1: 1821.1. Samples: 33449338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:23:02,016][52710] Avg episode reward: [(0, '28.830'), (1, '36.350')] [2023-10-08 10:23:02,087][53885] Updated weights for policy 1, policy_version 65182 (0.0009) [2023-10-08 10:23:04,916][53852] Updated weights for policy 0, policy_version 65480 (0.0009) [2023-10-08 10:23:05,288][53852] Updated weights for policy 0, policy_version 65490 (0.0008) [2023-10-08 10:23:05,656][53852] Updated weights for policy 0, policy_version 65500 (0.0008) [2023-10-08 10:23:05,723][53885] Updated weights for policy 1, policy_version 65192 (0.0009) [2023-10-08 10:23:06,088][53885] Updated weights for policy 1, policy_version 65202 (0.0009) [2023-10-08 10:23:06,462][53885] Updated weights for policy 1, policy_version 65212 (0.0008) [2023-10-08 10:23:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 133857280. Throughput: 0: 1831.0, 1: 1813.2. Samples: 33469828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:23:07,016][52710] Avg episode reward: [(0, '32.230'), (1, '32.790')] [2023-10-08 10:23:09,608][53852] Updated weights for policy 0, policy_version 65510 (0.0010) [2023-10-08 10:23:09,986][53852] Updated weights for policy 0, policy_version 65520 (0.0010) [2023-10-08 10:23:10,052][53885] Updated weights for policy 1, policy_version 65222 (0.0008) [2023-10-08 10:23:10,347][53852] Updated weights for policy 0, policy_version 65530 (0.0009) [2023-10-08 10:23:10,412][53885] Updated weights for policy 1, policy_version 65232 (0.0008) [2023-10-08 10:23:10,783][53885] Updated weights for policy 1, policy_version 65242 (0.0009) [2023-10-08 10:23:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133922816. Throughput: 0: 1823.9, 1: 1821.1. Samples: 33482254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:23:12,016][52710] Avg episode reward: [(0, '29.980'), (1, '31.240')] [2023-10-08 10:23:13,965][53852] Updated weights for policy 0, policy_version 65540 (0.0008) [2023-10-08 10:23:14,328][53852] Updated weights for policy 0, policy_version 65550 (0.0009) [2023-10-08 10:23:14,417][53885] Updated weights for policy 1, policy_version 65252 (0.0007) [2023-10-08 10:23:14,699][53852] Updated weights for policy 0, policy_version 65560 (0.0007) [2023-10-08 10:23:14,779][53885] Updated weights for policy 1, policy_version 65262 (0.0007) [2023-10-08 10:23:15,148][53885] Updated weights for policy 1, policy_version 65272 (0.0007) [2023-10-08 10:23:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 133988352. Throughput: 0: 1821.0, 1: 1820.0. Samples: 33502454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:23:17,016][52710] Avg episode reward: [(0, '29.720'), (1, '35.300')] [2023-10-08 10:23:18,436][53852] Updated weights for policy 0, policy_version 65570 (0.0008) [2023-10-08 10:23:18,811][53852] Updated weights for policy 0, policy_version 65580 (0.0008) [2023-10-08 10:23:18,933][53885] Updated weights for policy 1, policy_version 65282 (0.0010) [2023-10-08 10:23:19,174][53852] Updated weights for policy 0, policy_version 65590 (0.0007) [2023-10-08 10:23:19,299][53885] Updated weights for policy 1, policy_version 65292 (0.0008) [2023-10-08 10:23:19,541][53852] Updated weights for policy 0, policy_version 65600 (0.0008) [2023-10-08 10:23:19,664][53885] Updated weights for policy 1, policy_version 65302 (0.0007) [2023-10-08 10:23:20,031][53885] Updated weights for policy 1, policy_version 65312 (0.0007) [2023-10-08 10:23:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 134053888. Throughput: 0: 1822.2, 1: 1826.4. Samples: 33525438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:23:22,016][52710] Avg episode reward: [(0, '29.560'), (1, '36.560')] [2023-10-08 10:23:23,340][53852] Updated weights for policy 0, policy_version 65610 (0.0007) [2023-10-08 10:23:23,615][53885] Updated weights for policy 1, policy_version 65322 (0.0008) [2023-10-08 10:23:23,714][53852] Updated weights for policy 0, policy_version 65620 (0.0007) [2023-10-08 10:23:23,973][53885] Updated weights for policy 1, policy_version 65332 (0.0009) [2023-10-08 10:23:24,088][53852] Updated weights for policy 0, policy_version 65630 (0.0007) [2023-10-08 10:23:24,336][53885] Updated weights for policy 1, policy_version 65342 (0.0009) [2023-10-08 10:23:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134119424. Throughput: 0: 1824.4, 1: 1818.4. Samples: 33535318. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:23:27,015][52710] Avg episode reward: [(0, '31.100'), (1, '31.950')] [2023-10-08 10:23:27,747][53852] Updated weights for policy 0, policy_version 65640 (0.0008) [2023-10-08 10:23:27,962][53885] Updated weights for policy 1, policy_version 65352 (0.0009) [2023-10-08 10:23:28,131][53852] Updated weights for policy 0, policy_version 65650 (0.0009) [2023-10-08 10:23:28,339][53885] Updated weights for policy 1, policy_version 65362 (0.0008) [2023-10-08 10:23:28,489][53852] Updated weights for policy 0, policy_version 65660 (0.0008) [2023-10-08 10:23:28,706][53885] Updated weights for policy 1, policy_version 65372 (0.0007) [2023-10-08 10:23:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134184960. Throughput: 0: 1827.2, 1: 1830.4. Samples: 33558322. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:23:32,016][52710] Avg episode reward: [(0, '26.610'), (1, '36.230')] [2023-10-08 10:23:32,166][53852] Updated weights for policy 0, policy_version 65670 (0.0007) [2023-10-08 10:23:32,408][53885] Updated weights for policy 1, policy_version 65382 (0.0007) [2023-10-08 10:23:32,533][53852] Updated weights for policy 0, policy_version 65680 (0.0007) [2023-10-08 10:23:32,770][53885] Updated weights for policy 1, policy_version 65392 (0.0007) [2023-10-08 10:23:32,895][53852] Updated weights for policy 0, policy_version 65690 (0.0007) [2023-10-08 10:23:33,135][53885] Updated weights for policy 1, policy_version 65402 (0.0008) [2023-10-08 10:23:36,565][53852] Updated weights for policy 0, policy_version 65700 (0.0008) [2023-10-08 10:23:36,692][53885] Updated weights for policy 1, policy_version 65412 (0.0010) [2023-10-08 10:23:36,931][53852] Updated weights for policy 0, policy_version 65710 (0.0007) [2023-10-08 10:23:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134250496. Throughput: 0: 1823.5, 1: 1829.8. Samples: 33580850. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:23:37,016][52710] Avg episode reward: [(0, '30.470'), (1, '35.060')] [2023-10-08 10:23:37,054][53885] Updated weights for policy 1, policy_version 65422 (0.0009) [2023-10-08 10:23:37,297][53852] Updated weights for policy 0, policy_version 65720 (0.0007) [2023-10-08 10:23:37,428][53885] Updated weights for policy 1, policy_version 65432 (0.0007) [2023-10-08 10:23:40,916][53852] Updated weights for policy 0, policy_version 65730 (0.0008) [2023-10-08 10:23:41,221][53885] Updated weights for policy 1, policy_version 65442 (0.0007) [2023-10-08 10:23:41,290][53852] Updated weights for policy 0, policy_version 65740 (0.0007) [2023-10-08 10:23:41,582][53885] Updated weights for policy 1, policy_version 65452 (0.0007) [2023-10-08 10:23:41,651][53852] Updated weights for policy 0, policy_version 65750 (0.0008) [2023-10-08 10:23:41,935][53885] Updated weights for policy 1, policy_version 65462 (0.0007) [2023-10-08 10:23:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134316032. Throughput: 0: 1820.0, 1: 1830.0. Samples: 33591058. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:23:42,015][52710] Avg episode reward: [(0, '31.110'), (1, '32.360')] [2023-10-08 10:23:42,019][53852] Updated weights for policy 0, policy_version 65760 (0.0008) [2023-10-08 10:23:42,301][53885] Updated weights for policy 1, policy_version 65472 (0.0008) [2023-10-08 10:23:45,642][53852] Updated weights for policy 0, policy_version 65770 (0.0009) [2023-10-08 10:23:46,009][53852] Updated weights for policy 0, policy_version 65780 (0.0009) [2023-10-08 10:23:46,308][53885] Updated weights for policy 1, policy_version 65482 (0.0008) [2023-10-08 10:23:46,383][53852] Updated weights for policy 0, policy_version 65790 (0.0009) [2023-10-08 10:23:46,666][53885] Updated weights for policy 1, policy_version 65492 (0.0008) [2023-10-08 10:23:47,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134414336. Throughput: 0: 1821.0, 1: 1821.8. Samples: 33613264. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:23:47,016][52710] Avg episode reward: [(0, '27.580'), (1, '30.070')] [2023-10-08 10:23:47,040][53885] Updated weights for policy 1, policy_version 65502 (0.0010) [2023-10-08 10:23:50,210][53852] Updated weights for policy 0, policy_version 65800 (0.0008) [2023-10-08 10:23:50,585][53852] Updated weights for policy 0, policy_version 65810 (0.0007) [2023-10-08 10:23:50,706][53885] Updated weights for policy 1, policy_version 65512 (0.0008) [2023-10-08 10:23:50,950][53852] Updated weights for policy 0, policy_version 65820 (0.0007) [2023-10-08 10:23:51,076][53885] Updated weights for policy 1, policy_version 65522 (0.0009) [2023-10-08 10:23:51,437][53885] Updated weights for policy 1, policy_version 65532 (0.0011) [2023-10-08 10:23:52,015][52710] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 134512640. Throughput: 0: 1810.8, 1: 1819.2. Samples: 33633182. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:23:52,016][52710] Avg episode reward: [(0, '26.790'), (1, '37.060')] [2023-10-08 10:23:54,630][53852] Updated weights for policy 0, policy_version 65830 (0.0008) [2023-10-08 10:23:55,013][53852] Updated weights for policy 0, policy_version 65840 (0.0009) [2023-10-08 10:23:55,167][53885] Updated weights for policy 1, policy_version 65542 (0.0009) [2023-10-08 10:23:55,390][53852] Updated weights for policy 0, policy_version 65850 (0.0008) [2023-10-08 10:23:55,535][53885] Updated weights for policy 1, policy_version 65552 (0.0009) [2023-10-08 10:23:55,901][53885] Updated weights for policy 1, policy_version 65562 (0.0008) [2023-10-08 10:23:57,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134578176. Throughput: 0: 1821.3, 1: 1816.4. Samples: 33645954. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:23:57,016][52710] Avg episode reward: [(0, '32.760'), (1, '34.720')] [2023-10-08 10:23:58,896][53852] Updated weights for policy 0, policy_version 65860 (0.0008) [2023-10-08 10:23:59,262][53852] Updated weights for policy 0, policy_version 65870 (0.0008) [2023-10-08 10:23:59,536][53885] Updated weights for policy 1, policy_version 65572 (0.0009) [2023-10-08 10:23:59,635][53852] Updated weights for policy 0, policy_version 65880 (0.0009) [2023-10-08 10:23:59,906][53885] Updated weights for policy 1, policy_version 65582 (0.0009) [2023-10-08 10:24:00,260][53885] Updated weights for policy 1, policy_version 65592 (0.0008) [2023-10-08 10:24:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 134643712. Throughput: 0: 1823.8, 1: 1814.9. Samples: 33666198. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:24:02,016][52710] Avg episode reward: [(0, '29.090'), (1, '34.770')] [2023-10-08 10:24:03,445][53852] Updated weights for policy 0, policy_version 65890 (0.0007) [2023-10-08 10:24:03,811][53852] Updated weights for policy 0, policy_version 65900 (0.0011) [2023-10-08 10:24:03,859][53885] Updated weights for policy 1, policy_version 65602 (0.0009) [2023-10-08 10:24:04,181][53852] Updated weights for policy 0, policy_version 65910 (0.0008) [2023-10-08 10:24:04,227][53885] Updated weights for policy 1, policy_version 65612 (0.0008) [2023-10-08 10:24:04,547][53852] Updated weights for policy 0, policy_version 65920 (0.0007) [2023-10-08 10:24:04,600][53885] Updated weights for policy 1, policy_version 65622 (0.0008) [2023-10-08 10:24:04,975][53885] Updated weights for policy 1, policy_version 65632 (0.0008) [2023-10-08 10:24:07,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 134709248. Throughput: 0: 1818.3, 1: 1818.1. Samples: 33689074. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:24:07,015][52710] Avg episode reward: [(0, '31.510'), (1, '40.680')] [2023-10-08 10:24:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000065632_67207168.pth... [2023-10-08 10:24:07,023][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000065920_67502080.pth... [2023-10-08 10:24:07,052][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000063936_65470464.pth [2023-10-08 10:24:07,056][53594] Saving new best policy, reward=40.680! [2023-10-08 10:24:07,061][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000064224_65765376.pth [2023-10-08 10:24:08,206][53852] Updated weights for policy 0, policy_version 65930 (0.0007) [2023-10-08 10:24:08,572][53852] Updated weights for policy 0, policy_version 65940 (0.0009) [2023-10-08 10:24:08,858][53885] Updated weights for policy 1, policy_version 65642 (0.0009) [2023-10-08 10:24:08,939][53852] Updated weights for policy 0, policy_version 65950 (0.0008) [2023-10-08 10:24:09,223][53885] Updated weights for policy 1, policy_version 65652 (0.0008) [2023-10-08 10:24:09,593][53885] Updated weights for policy 1, policy_version 65662 (0.0007) [2023-10-08 10:24:12,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134774784. Throughput: 0: 1819.9, 1: 1822.5. Samples: 33699228. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:24:12,017][52710] Avg episode reward: [(0, '30.180'), (1, '35.460')] [2023-10-08 10:24:12,651][53852] Updated weights for policy 0, policy_version 65960 (0.0008) [2023-10-08 10:24:13,027][53852] Updated weights for policy 0, policy_version 65970 (0.0010) [2023-10-08 10:24:13,238][53885] Updated weights for policy 1, policy_version 65672 (0.0010) [2023-10-08 10:24:13,403][53852] Updated weights for policy 0, policy_version 65980 (0.0007) [2023-10-08 10:24:13,611][53885] Updated weights for policy 1, policy_version 65682 (0.0009) [2023-10-08 10:24:13,976][53885] Updated weights for policy 1, policy_version 65692 (0.0007) [2023-10-08 10:24:16,918][53852] Updated weights for policy 0, policy_version 65990 (0.0007) [2023-10-08 10:24:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134840320. Throughput: 0: 1824.6, 1: 1815.0. Samples: 33722102. Policy #0 lag: (min: 18.0, avg: 21.5, max: 50.0) [2023-10-08 10:24:17,015][52710] Avg episode reward: [(0, '30.490'), (1, '36.130')] [2023-10-08 10:24:17,282][53852] Updated weights for policy 0, policy_version 66000 (0.0007) [2023-10-08 10:24:17,652][53852] Updated weights for policy 0, policy_version 66010 (0.0007) [2023-10-08 10:24:17,758][53885] Updated weights for policy 1, policy_version 65702 (0.0008) [2023-10-08 10:24:18,131][53885] Updated weights for policy 1, policy_version 65712 (0.0008) [2023-10-08 10:24:18,505][53885] Updated weights for policy 1, policy_version 65722 (0.0009) [2023-10-08 10:24:21,304][53852] Updated weights for policy 0, policy_version 66020 (0.0008) [2023-10-08 10:24:21,677][53852] Updated weights for policy 0, policy_version 66030 (0.0007) [2023-10-08 10:24:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 134905856. Throughput: 0: 1820.3, 1: 1815.1. Samples: 33744444. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:24:22,015][52710] Avg episode reward: [(0, '29.000'), (1, '41.220')] [2023-10-08 10:24:22,038][53852] Updated weights for policy 0, policy_version 66040 (0.0010) [2023-10-08 10:24:22,227][53885] Updated weights for policy 1, policy_version 65732 (0.0007) [2023-10-08 10:24:22,590][53885] Updated weights for policy 1, policy_version 65742 (0.0009) [2023-10-08 10:24:22,970][53885] Updated weights for policy 1, policy_version 65752 (0.0008) [2023-10-08 10:24:23,260][53594] Saving new best policy, reward=41.220! [2023-10-08 10:24:25,783][53852] Updated weights for policy 0, policy_version 66050 (0.0008) [2023-10-08 10:24:26,160][53852] Updated weights for policy 0, policy_version 66060 (0.0009) [2023-10-08 10:24:26,531][53852] Updated weights for policy 0, policy_version 66070 (0.0007) [2023-10-08 10:24:26,538][53885] Updated weights for policy 1, policy_version 65762 (0.0009) [2023-10-08 10:24:26,890][53852] Updated weights for policy 0, policy_version 66080 (0.0008) [2023-10-08 10:24:26,900][53885] Updated weights for policy 1, policy_version 65772 (0.0007) [2023-10-08 10:24:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 135004160. Throughput: 0: 1822.5, 1: 1818.4. Samples: 33754902. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:24:27,016][52710] Avg episode reward: [(0, '30.530'), (1, '35.840')] [2023-10-08 10:24:27,264][53885] Updated weights for policy 1, policy_version 65782 (0.0008) [2023-10-08 10:24:27,632][53885] Updated weights for policy 1, policy_version 65792 (0.0007) [2023-10-08 10:24:30,433][53852] Updated weights for policy 0, policy_version 66090 (0.0010) [2023-10-08 10:24:30,807][53852] Updated weights for policy 0, policy_version 66100 (0.0010) [2023-10-08 10:24:31,186][53852] Updated weights for policy 0, policy_version 66110 (0.0008) [2023-10-08 10:24:31,384][53885] Updated weights for policy 1, policy_version 65802 (0.0009) [2023-10-08 10:24:31,752][53885] Updated weights for policy 1, policy_version 65812 (0.0008) [2023-10-08 10:24:32,015][52710] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 135069696. Throughput: 0: 1823.1, 1: 1828.7. Samples: 33777594. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:24:32,017][52710] Avg episode reward: [(0, '29.930'), (1, '37.110')] [2023-10-08 10:24:32,116][53885] Updated weights for policy 1, policy_version 65822 (0.0009) [2023-10-08 10:24:34,735][53852] Updated weights for policy 0, policy_version 66120 (0.0008) [2023-10-08 10:24:35,097][53852] Updated weights for policy 0, policy_version 66130 (0.0007) [2023-10-08 10:24:35,466][53852] Updated weights for policy 0, policy_version 66140 (0.0008) [2023-10-08 10:24:35,735][53885] Updated weights for policy 1, policy_version 65832 (0.0007) [2023-10-08 10:24:36,108][53885] Updated weights for policy 1, policy_version 65842 (0.0008) [2023-10-08 10:24:36,469][53885] Updated weights for policy 1, policy_version 65852 (0.0009) [2023-10-08 10:24:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 135168000. Throughput: 0: 1836.9, 1: 1832.2. Samples: 33798292. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:24:37,016][52710] Avg episode reward: [(0, '32.680'), (1, '39.460')] [2023-10-08 10:24:39,246][53852] Updated weights for policy 0, policy_version 66150 (0.0007) [2023-10-08 10:24:39,618][53852] Updated weights for policy 0, policy_version 66160 (0.0010) [2023-10-08 10:24:39,988][53852] Updated weights for policy 0, policy_version 66170 (0.0009) [2023-10-08 10:24:40,048][53885] Updated weights for policy 1, policy_version 65862 (0.0009) [2023-10-08 10:24:40,421][53885] Updated weights for policy 1, policy_version 65872 (0.0008) [2023-10-08 10:24:40,791][53885] Updated weights for policy 1, policy_version 65882 (0.0008) [2023-10-08 10:24:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 135233536. Throughput: 0: 1824.3, 1: 1831.7. Samples: 33810476. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:24:42,016][52710] Avg episode reward: [(0, '34.390'), (1, '37.970')] [2023-10-08 10:24:43,775][53852] Updated weights for policy 0, policy_version 66180 (0.0008) [2023-10-08 10:24:44,149][53852] Updated weights for policy 0, policy_version 66190 (0.0009) [2023-10-08 10:24:44,376][53885] Updated weights for policy 1, policy_version 65892 (0.0010) [2023-10-08 10:24:44,518][53852] Updated weights for policy 0, policy_version 66200 (0.0007) [2023-10-08 10:24:44,739][53885] Updated weights for policy 1, policy_version 65902 (0.0007) [2023-10-08 10:24:45,109][53885] Updated weights for policy 1, policy_version 65912 (0.0009) [2023-10-08 10:24:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135299072. Throughput: 0: 1826.6, 1: 1828.8. Samples: 33830688. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:24:47,016][52710] Avg episode reward: [(0, '28.790'), (1, '34.510')] [2023-10-08 10:24:48,172][53852] Updated weights for policy 0, policy_version 66210 (0.0008) [2023-10-08 10:24:48,536][53852] Updated weights for policy 0, policy_version 66220 (0.0009) [2023-10-08 10:24:48,887][53885] Updated weights for policy 1, policy_version 65922 (0.0008) [2023-10-08 10:24:48,907][53852] Updated weights for policy 0, policy_version 66230 (0.0009) [2023-10-08 10:24:49,257][53885] Updated weights for policy 1, policy_version 65932 (0.0007) [2023-10-08 10:24:49,272][53852] Updated weights for policy 0, policy_version 66240 (0.0008) [2023-10-08 10:24:49,632][53885] Updated weights for policy 1, policy_version 65942 (0.0008) [2023-10-08 10:24:49,988][53885] Updated weights for policy 1, policy_version 65952 (0.0008) [2023-10-08 10:24:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135364608. Throughput: 0: 1837.2, 1: 1823.2. Samples: 33853792. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:24:52,015][52710] Avg episode reward: [(0, '30.780'), (1, '36.610')] [2023-10-08 10:24:52,782][53852] Updated weights for policy 0, policy_version 66250 (0.0010) [2023-10-08 10:24:53,146][53852] Updated weights for policy 0, policy_version 66260 (0.0008) [2023-10-08 10:24:53,509][53852] Updated weights for policy 0, policy_version 66270 (0.0007) [2023-10-08 10:24:53,624][53885] Updated weights for policy 1, policy_version 65962 (0.0008) [2023-10-08 10:24:53,990][53885] Updated weights for policy 1, policy_version 65972 (0.0009) [2023-10-08 10:24:54,362][53885] Updated weights for policy 1, policy_version 65982 (0.0007) [2023-10-08 10:24:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 135430144. Throughput: 0: 1837.3, 1: 1821.7. Samples: 33863880. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:24:57,015][52710] Avg episode reward: [(0, '31.680'), (1, '36.430')] [2023-10-08 10:24:57,117][53852] Updated weights for policy 0, policy_version 66280 (0.0009) [2023-10-08 10:24:57,474][53852] Updated weights for policy 0, policy_version 66290 (0.0008) [2023-10-08 10:24:57,847][53852] Updated weights for policy 0, policy_version 66300 (0.0009) [2023-10-08 10:24:58,025][53885] Updated weights for policy 1, policy_version 65992 (0.0008) [2023-10-08 10:24:58,394][53885] Updated weights for policy 1, policy_version 66002 (0.0008) [2023-10-08 10:24:58,771][53885] Updated weights for policy 1, policy_version 66012 (0.0009) [2023-10-08 10:25:01,623][53852] Updated weights for policy 0, policy_version 66310 (0.0009) [2023-10-08 10:25:01,992][53852] Updated weights for policy 0, policy_version 66320 (0.0008) [2023-10-08 10:25:02,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 135495680. Throughput: 0: 1840.1, 1: 1824.7. Samples: 33887022. Policy #0 lag: (min: 15.0, avg: 30.5, max: 32.0) [2023-10-08 10:25:02,016][52710] Avg episode reward: [(0, '31.560'), (1, '32.670')] [2023-10-08 10:25:02,362][53852] Updated weights for policy 0, policy_version 66330 (0.0009) [2023-10-08 10:25:02,475][53885] Updated weights for policy 1, policy_version 66022 (0.0007) [2023-10-08 10:25:02,841][53885] Updated weights for policy 1, policy_version 66032 (0.0009) [2023-10-08 10:25:03,218][53885] Updated weights for policy 1, policy_version 66042 (0.0008) [2023-10-08 10:25:06,168][53852] Updated weights for policy 0, policy_version 66340 (0.0008) [2023-10-08 10:25:06,535][53852] Updated weights for policy 0, policy_version 66350 (0.0007) [2023-10-08 10:25:06,913][53852] Updated weights for policy 0, policy_version 66360 (0.0007) [2023-10-08 10:25:06,924][53885] Updated weights for policy 1, policy_version 66052 (0.0007) [2023-10-08 10:25:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 135561216. Throughput: 0: 1831.1, 1: 1832.9. Samples: 33909322. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:07,016][52710] Avg episode reward: [(0, '30.310'), (1, '32.750')] [2023-10-08 10:25:07,293][53885] Updated weights for policy 1, policy_version 66062 (0.0008) [2023-10-08 10:25:07,669][53885] Updated weights for policy 1, policy_version 66072 (0.0008) [2023-10-08 10:25:10,606][53852] Updated weights for policy 0, policy_version 66370 (0.0007) [2023-10-08 10:25:10,978][53852] Updated weights for policy 0, policy_version 66380 (0.0009) [2023-10-08 10:25:11,339][53852] Updated weights for policy 0, policy_version 66390 (0.0009) [2023-10-08 10:25:11,396][53885] Updated weights for policy 1, policy_version 66082 (0.0008) [2023-10-08 10:25:11,706][53852] Updated weights for policy 0, policy_version 66400 (0.0007) [2023-10-08 10:25:11,759][53885] Updated weights for policy 1, policy_version 66092 (0.0007) [2023-10-08 10:25:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135659520. Throughput: 0: 1835.2, 1: 1827.5. Samples: 33919724. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:12,016][52710] Avg episode reward: [(0, '31.830'), (1, '36.730')] [2023-10-08 10:25:12,124][53885] Updated weights for policy 1, policy_version 66102 (0.0009) [2023-10-08 10:25:12,484][53885] Updated weights for policy 1, policy_version 66112 (0.0007) [2023-10-08 10:25:15,291][53852] Updated weights for policy 0, policy_version 66410 (0.0007) [2023-10-08 10:25:15,666][53852] Updated weights for policy 0, policy_version 66420 (0.0011) [2023-10-08 10:25:16,040][53852] Updated weights for policy 0, policy_version 66430 (0.0007) [2023-10-08 10:25:16,369][53885] Updated weights for policy 1, policy_version 66122 (0.0008) [2023-10-08 10:25:16,735][53885] Updated weights for policy 1, policy_version 66132 (0.0008) [2023-10-08 10:25:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135725056. Throughput: 0: 1830.5, 1: 1819.2. Samples: 33941826. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:17,016][52710] Avg episode reward: [(0, '31.830'), (1, '33.180')] [2023-10-08 10:25:17,099][53885] Updated weights for policy 1, policy_version 66142 (0.0007) [2023-10-08 10:25:19,602][53852] Updated weights for policy 0, policy_version 66440 (0.0008) [2023-10-08 10:25:19,973][53852] Updated weights for policy 0, policy_version 66450 (0.0008) [2023-10-08 10:25:20,348][53852] Updated weights for policy 0, policy_version 66460 (0.0008) [2023-10-08 10:25:20,663][53885] Updated weights for policy 1, policy_version 66152 (0.0008) [2023-10-08 10:25:21,036][53885] Updated weights for policy 1, policy_version 66162 (0.0008) [2023-10-08 10:25:21,402][53885] Updated weights for policy 1, policy_version 66172 (0.0007) [2023-10-08 10:25:22,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 135823360. Throughput: 0: 1832.5, 1: 1819.1. Samples: 33962612. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:22,015][52710] Avg episode reward: [(0, '33.230'), (1, '35.980')] [2023-10-08 10:25:23,925][53852] Updated weights for policy 0, policy_version 66470 (0.0010) [2023-10-08 10:25:24,291][53852] Updated weights for policy 0, policy_version 66480 (0.0010) [2023-10-08 10:25:24,659][53852] Updated weights for policy 0, policy_version 66490 (0.0010) [2023-10-08 10:25:24,910][53885] Updated weights for policy 1, policy_version 66182 (0.0007) [2023-10-08 10:25:25,279][53885] Updated weights for policy 1, policy_version 66192 (0.0007) [2023-10-08 10:25:25,650][53885] Updated weights for policy 1, policy_version 66202 (0.0008) [2023-10-08 10:25:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135888896. Throughput: 0: 1824.3, 1: 1821.4. Samples: 33974530. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:27,016][52710] Avg episode reward: [(0, '31.410'), (1, '36.220')] [2023-10-08 10:25:28,318][53852] Updated weights for policy 0, policy_version 66500 (0.0011) [2023-10-08 10:25:28,699][53852] Updated weights for policy 0, policy_version 66510 (0.0008) [2023-10-08 10:25:29,073][53852] Updated weights for policy 0, policy_version 66520 (0.0009) [2023-10-08 10:25:29,352][53885] Updated weights for policy 1, policy_version 66212 (0.0008) [2023-10-08 10:25:29,713][53885] Updated weights for policy 1, policy_version 66222 (0.0009) [2023-10-08 10:25:30,077][53885] Updated weights for policy 1, policy_version 66232 (0.0010) [2023-10-08 10:25:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 135954432. Throughput: 0: 1844.7, 1: 1821.8. Samples: 33995680. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:32,016][52710] Avg episode reward: [(0, '31.190'), (1, '34.130')] [2023-10-08 10:25:32,719][53852] Updated weights for policy 0, policy_version 66530 (0.0007) [2023-10-08 10:25:33,085][53852] Updated weights for policy 0, policy_version 66540 (0.0007) [2023-10-08 10:25:33,461][53852] Updated weights for policy 0, policy_version 66550 (0.0008) [2023-10-08 10:25:33,680][53885] Updated weights for policy 1, policy_version 66242 (0.0009) [2023-10-08 10:25:33,830][53852] Updated weights for policy 0, policy_version 66560 (0.0009) [2023-10-08 10:25:34,052][53885] Updated weights for policy 1, policy_version 66252 (0.0008) [2023-10-08 10:25:34,409][53885] Updated weights for policy 1, policy_version 66262 (0.0008) [2023-10-08 10:25:34,783][53885] Updated weights for policy 1, policy_version 66272 (0.0010) [2023-10-08 10:25:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 136019968. Throughput: 0: 1840.9, 1: 1827.7. Samples: 34018878. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:37,016][52710] Avg episode reward: [(0, '30.370'), (1, '33.830')] [2023-10-08 10:25:37,313][53852] Updated weights for policy 0, policy_version 66570 (0.0008) [2023-10-08 10:25:37,671][53852] Updated weights for policy 0, policy_version 66580 (0.0007) [2023-10-08 10:25:38,049][53852] Updated weights for policy 0, policy_version 66590 (0.0010) [2023-10-08 10:25:38,449][53885] Updated weights for policy 1, policy_version 66282 (0.0011) [2023-10-08 10:25:38,818][53885] Updated weights for policy 1, policy_version 66292 (0.0008) [2023-10-08 10:25:39,187][53885] Updated weights for policy 1, policy_version 66302 (0.0008) [2023-10-08 10:25:41,675][53852] Updated weights for policy 0, policy_version 66600 (0.0010) [2023-10-08 10:25:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136085504. Throughput: 0: 1840.7, 1: 1828.1. Samples: 34028976. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:42,015][52710] Avg episode reward: [(0, '29.400'), (1, '34.310')] [2023-10-08 10:25:42,050][53852] Updated weights for policy 0, policy_version 66610 (0.0010) [2023-10-08 10:25:42,421][53852] Updated weights for policy 0, policy_version 66620 (0.0009) [2023-10-08 10:25:42,910][53885] Updated weights for policy 1, policy_version 66312 (0.0008) [2023-10-08 10:25:43,273][53885] Updated weights for policy 1, policy_version 66322 (0.0011) [2023-10-08 10:25:43,633][53885] Updated weights for policy 1, policy_version 66332 (0.0008) [2023-10-08 10:25:46,091][53852] Updated weights for policy 0, policy_version 66630 (0.0007) [2023-10-08 10:25:46,465][53852] Updated weights for policy 0, policy_version 66640 (0.0009) [2023-10-08 10:25:46,839][53852] Updated weights for policy 0, policy_version 66650 (0.0008) [2023-10-08 10:25:47,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 136151040. Throughput: 0: 1839.7, 1: 1827.6. Samples: 34052048. Policy #0 lag: (min: 31.0, avg: 37.6, max: 63.0) [2023-10-08 10:25:47,015][52710] Avg episode reward: [(0, '29.860'), (1, '31.210')] [2023-10-08 10:25:47,301][53885] Updated weights for policy 1, policy_version 66342 (0.0008) [2023-10-08 10:25:47,665][53885] Updated weights for policy 1, policy_version 66352 (0.0008) [2023-10-08 10:25:48,027][53885] Updated weights for policy 1, policy_version 66362 (0.0008) [2023-10-08 10:25:50,498][53852] Updated weights for policy 0, policy_version 66660 (0.0009) [2023-10-08 10:25:50,870][53852] Updated weights for policy 0, policy_version 66670 (0.0008) [2023-10-08 10:25:51,231][53852] Updated weights for policy 0, policy_version 66680 (0.0007) [2023-10-08 10:25:51,737][53885] Updated weights for policy 1, policy_version 66372 (0.0009) [2023-10-08 10:25:52,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 136249344. Throughput: 0: 1826.2, 1: 1824.6. Samples: 34073608. Policy #0 lag: (min: 17.0, avg: 34.4, max: 49.0) [2023-10-08 10:25:52,016][52710] Avg episode reward: [(0, '32.070'), (1, '34.150')] [2023-10-08 10:25:52,105][53885] Updated weights for policy 1, policy_version 66382 (0.0009) [2023-10-08 10:25:52,480][53885] Updated weights for policy 1, policy_version 66392 (0.0008) [2023-10-08 10:25:54,869][53852] Updated weights for policy 0, policy_version 66690 (0.0009) [2023-10-08 10:25:55,232][53852] Updated weights for policy 0, policy_version 66700 (0.0010) [2023-10-08 10:25:55,607][53852] Updated weights for policy 0, policy_version 66710 (0.0008) [2023-10-08 10:25:55,976][53852] Updated weights for policy 0, policy_version 66720 (0.0008) [2023-10-08 10:25:56,091][53885] Updated weights for policy 1, policy_version 66402 (0.0009) [2023-10-08 10:25:56,460][53885] Updated weights for policy 1, policy_version 66412 (0.0007) [2023-10-08 10:25:56,820][53885] Updated weights for policy 1, policy_version 66422 (0.0011) [2023-10-08 10:25:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136314880. Throughput: 0: 1846.6, 1: 1826.2. Samples: 34085002. Policy #0 lag: (min: 17.0, avg: 34.4, max: 49.0) [2023-10-08 10:25:57,016][52710] Avg episode reward: [(0, '31.920'), (1, '34.430')] [2023-10-08 10:25:57,189][53885] Updated weights for policy 1, policy_version 66432 (0.0008) [2023-10-08 10:25:59,569][53852] Updated weights for policy 0, policy_version 66730 (0.0009) [2023-10-08 10:25:59,933][53852] Updated weights for policy 0, policy_version 66740 (0.0007) [2023-10-08 10:26:00,299][53852] Updated weights for policy 0, policy_version 66750 (0.0009) [2023-10-08 10:26:00,967][53885] Updated weights for policy 1, policy_version 66442 (0.0010) [2023-10-08 10:26:01,330][53885] Updated weights for policy 1, policy_version 66452 (0.0011) [2023-10-08 10:26:01,694][53885] Updated weights for policy 1, policy_version 66462 (0.0008) [2023-10-08 10:26:02,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 136413184. Throughput: 0: 1834.2, 1: 1830.8. Samples: 34106750. Policy #0 lag: (min: 17.0, avg: 34.4, max: 49.0) [2023-10-08 10:26:02,016][52710] Avg episode reward: [(0, '32.850'), (1, '34.770')] [2023-10-08 10:26:03,855][53852] Updated weights for policy 0, policy_version 66760 (0.0008) [2023-10-08 10:26:04,221][53852] Updated weights for policy 0, policy_version 66770 (0.0008) [2023-10-08 10:26:04,591][53852] Updated weights for policy 0, policy_version 66780 (0.0009) [2023-10-08 10:26:05,277][53885] Updated weights for policy 1, policy_version 66472 (0.0010) [2023-10-08 10:26:05,645][53885] Updated weights for policy 1, policy_version 66482 (0.0009) [2023-10-08 10:26:06,010][53885] Updated weights for policy 1, policy_version 66492 (0.0007) [2023-10-08 10:26:07,015][52710] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 136478720. Throughput: 0: 1857.7, 1: 1832.7. Samples: 34128682. Policy #0 lag: (min: 17.0, avg: 34.4, max: 49.0) [2023-10-08 10:26:07,017][52710] Avg episode reward: [(0, '32.650'), (1, '31.800')] [2023-10-08 10:26:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000066784_68386816.pth... [2023-10-08 10:26:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000066496_68091904.pth... [2023-10-08 10:26:07,057][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000065088_66650112.pth [2023-10-08 10:26:07,068][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000064800_66355200.pth [2023-10-08 10:26:08,128][53852] Updated weights for policy 0, policy_version 66790 (0.0008) [2023-10-08 10:26:08,509][53852] Updated weights for policy 0, policy_version 66800 (0.0011) [2023-10-08 10:26:08,876][53852] Updated weights for policy 0, policy_version 66810 (0.0009) [2023-10-08 10:26:09,639][53885] Updated weights for policy 1, policy_version 66502 (0.0008) [2023-10-08 10:26:10,007][53885] Updated weights for policy 1, policy_version 66512 (0.0007) [2023-10-08 10:26:10,386][53885] Updated weights for policy 1, policy_version 66522 (0.0008) [2023-10-08 10:26:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136544256. Throughput: 0: 1848.8, 1: 1828.3. Samples: 34139998. Policy #0 lag: (min: 17.0, avg: 34.4, max: 49.0) [2023-10-08 10:26:12,016][52710] Avg episode reward: [(0, '32.570'), (1, '34.910')] [2023-10-08 10:26:12,582][53852] Updated weights for policy 0, policy_version 66820 (0.0008) [2023-10-08 10:26:12,957][53852] Updated weights for policy 0, policy_version 66830 (0.0010) [2023-10-08 10:26:13,324][53852] Updated weights for policy 0, policy_version 66840 (0.0011) [2023-10-08 10:26:14,085][53885] Updated weights for policy 1, policy_version 66532 (0.0011) [2023-10-08 10:26:14,446][53885] Updated weights for policy 1, policy_version 66542 (0.0009) [2023-10-08 10:26:14,815][53885] Updated weights for policy 1, policy_version 66552 (0.0007) [2023-10-08 10:26:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136609792. Throughput: 0: 1855.0, 1: 1831.5. Samples: 34161572. Policy #0 lag: (min: 17.0, avg: 34.4, max: 49.0) [2023-10-08 10:26:17,016][52710] Avg episode reward: [(0, '32.550'), (1, '30.940')] [2023-10-08 10:26:17,080][53852] Updated weights for policy 0, policy_version 66850 (0.0010) [2023-10-08 10:26:17,453][53852] Updated weights for policy 0, policy_version 66860 (0.0007) [2023-10-08 10:26:17,815][53852] Updated weights for policy 0, policy_version 66870 (0.0007) [2023-10-08 10:26:18,184][53852] Updated weights for policy 0, policy_version 66880 (0.0008) [2023-10-08 10:26:18,301][53885] Updated weights for policy 1, policy_version 66562 (0.0008) [2023-10-08 10:26:18,668][53885] Updated weights for policy 1, policy_version 66572 (0.0008) [2023-10-08 10:26:19,038][53885] Updated weights for policy 1, policy_version 66582 (0.0008) [2023-10-08 10:26:19,409][53885] Updated weights for policy 1, policy_version 66592 (0.0008) [2023-10-08 10:26:21,802][53852] Updated weights for policy 0, policy_version 66890 (0.0008) [2023-10-08 10:26:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 136675328. Throughput: 0: 1847.2, 1: 1834.7. Samples: 34184562. Policy #0 lag: (min: 17.0, avg: 34.4, max: 49.0) [2023-10-08 10:26:22,017][52710] Avg episode reward: [(0, '34.320'), (1, '29.770')] [2023-10-08 10:26:22,173][53852] Updated weights for policy 0, policy_version 66900 (0.0009) [2023-10-08 10:26:22,541][53852] Updated weights for policy 0, policy_version 66910 (0.0009) [2023-10-08 10:26:23,094][53885] Updated weights for policy 1, policy_version 66602 (0.0011) [2023-10-08 10:26:23,464][53885] Updated weights for policy 1, policy_version 66612 (0.0007) [2023-10-08 10:26:23,822][53885] Updated weights for policy 1, policy_version 66622 (0.0007) [2023-10-08 10:26:26,049][53852] Updated weights for policy 0, policy_version 66920 (0.0008) [2023-10-08 10:26:26,418][53852] Updated weights for policy 0, policy_version 66930 (0.0007) [2023-10-08 10:26:26,789][53852] Updated weights for policy 0, policy_version 66940 (0.0007) [2023-10-08 10:26:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136773632. Throughput: 0: 1852.5, 1: 1834.5. Samples: 34194892. Policy #0 lag: (min: 17.0, avg: 34.4, max: 49.0) [2023-10-08 10:26:27,015][52710] Avg episode reward: [(0, '32.440'), (1, '30.860')] [2023-10-08 10:26:27,347][53885] Updated weights for policy 1, policy_version 66632 (0.0007) [2023-10-08 10:26:27,722][53885] Updated weights for policy 1, policy_version 66642 (0.0007) [2023-10-08 10:26:28,100][53885] Updated weights for policy 1, policy_version 66652 (0.0008) [2023-10-08 10:26:30,559][53852] Updated weights for policy 0, policy_version 66950 (0.0008) [2023-10-08 10:26:30,930][53852] Updated weights for policy 0, policy_version 66960 (0.0009) [2023-10-08 10:26:31,299][53852] Updated weights for policy 0, policy_version 66970 (0.0012) [2023-10-08 10:26:31,863][53885] Updated weights for policy 1, policy_version 66662 (0.0009) [2023-10-08 10:26:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136839168. Throughput: 0: 1840.7, 1: 1838.7. Samples: 34217626. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:26:32,016][52710] Avg episode reward: [(0, '31.850'), (1, '27.690')] [2023-10-08 10:26:32,225][53885] Updated weights for policy 1, policy_version 66672 (0.0010) [2023-10-08 10:26:32,592][53885] Updated weights for policy 1, policy_version 66682 (0.0008) [2023-10-08 10:26:35,013][53852] Updated weights for policy 0, policy_version 66980 (0.0009) [2023-10-08 10:26:35,382][53852] Updated weights for policy 0, policy_version 66990 (0.0008) [2023-10-08 10:26:35,741][53852] Updated weights for policy 0, policy_version 67000 (0.0009) [2023-10-08 10:26:36,191][53885] Updated weights for policy 1, policy_version 66692 (0.0008) [2023-10-08 10:26:36,561][53885] Updated weights for policy 1, policy_version 66702 (0.0007) [2023-10-08 10:26:36,933][53885] Updated weights for policy 1, policy_version 66712 (0.0007) [2023-10-08 10:26:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136904704. Throughput: 0: 1843.8, 1: 1824.2. Samples: 34238666. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:26:37,016][52710] Avg episode reward: [(0, '32.260'), (1, '28.210')] [2023-10-08 10:26:39,382][53852] Updated weights for policy 0, policy_version 67010 (0.0009) [2023-10-08 10:26:39,750][53852] Updated weights for policy 0, policy_version 67020 (0.0007) [2023-10-08 10:26:40,124][53852] Updated weights for policy 0, policy_version 67030 (0.0007) [2023-10-08 10:26:40,484][53852] Updated weights for policy 0, policy_version 67040 (0.0007) [2023-10-08 10:26:40,612][53885] Updated weights for policy 1, policy_version 66722 (0.0009) [2023-10-08 10:26:40,979][53885] Updated weights for policy 1, policy_version 66732 (0.0008) [2023-10-08 10:26:41,352][53885] Updated weights for policy 1, policy_version 66742 (0.0007) [2023-10-08 10:26:41,719][53885] Updated weights for policy 1, policy_version 66752 (0.0008) [2023-10-08 10:26:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 137003008. Throughput: 0: 1838.3, 1: 1840.4. Samples: 34250546. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:26:42,016][52710] Avg episode reward: [(0, '28.590'), (1, '31.640')] [2023-10-08 10:26:44,203][53852] Updated weights for policy 0, policy_version 67050 (0.0009) [2023-10-08 10:26:44,581][53852] Updated weights for policy 0, policy_version 67060 (0.0009) [2023-10-08 10:26:44,952][53852] Updated weights for policy 0, policy_version 67070 (0.0009) [2023-10-08 10:26:45,385][53885] Updated weights for policy 1, policy_version 66762 (0.0010) [2023-10-08 10:26:45,746][53885] Updated weights for policy 1, policy_version 66772 (0.0010) [2023-10-08 10:26:46,120][53885] Updated weights for policy 1, policy_version 66782 (0.0011) [2023-10-08 10:26:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 137068544. Throughput: 0: 1836.3, 1: 1831.1. Samples: 34271782. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:26:47,016][52710] Avg episode reward: [(0, '31.080'), (1, '29.770')] [2023-10-08 10:26:48,758][53852] Updated weights for policy 0, policy_version 67080 (0.0009) [2023-10-08 10:26:49,128][53852] Updated weights for policy 0, policy_version 67090 (0.0010) [2023-10-08 10:26:49,499][53852] Updated weights for policy 0, policy_version 67100 (0.0009) [2023-10-08 10:26:49,899][53885] Updated weights for policy 1, policy_version 66792 (0.0008) [2023-10-08 10:26:50,271][53885] Updated weights for policy 1, policy_version 66802 (0.0011) [2023-10-08 10:26:50,647][53885] Updated weights for policy 1, policy_version 66812 (0.0009) [2023-10-08 10:26:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 137134080. Throughput: 0: 1826.2, 1: 1836.9. Samples: 34293520. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:26:52,016][52710] Avg episode reward: [(0, '32.070'), (1, '29.790')] [2023-10-08 10:26:53,140][53852] Updated weights for policy 0, policy_version 67110 (0.0010) [2023-10-08 10:26:53,516][53852] Updated weights for policy 0, policy_version 67120 (0.0007) [2023-10-08 10:26:53,880][53852] Updated weights for policy 0, policy_version 67130 (0.0007) [2023-10-08 10:26:54,228][53885] Updated weights for policy 1, policy_version 66822 (0.0008) [2023-10-08 10:26:54,593][53885] Updated weights for policy 1, policy_version 66832 (0.0007) [2023-10-08 10:26:54,971][53885] Updated weights for policy 1, policy_version 66842 (0.0009) [2023-10-08 10:26:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137199616. Throughput: 0: 1826.7, 1: 1827.3. Samples: 34304428. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:26:57,015][52710] Avg episode reward: [(0, '30.280'), (1, '30.030')] [2023-10-08 10:26:57,493][53852] Updated weights for policy 0, policy_version 67140 (0.0007) [2023-10-08 10:26:57,863][53852] Updated weights for policy 0, policy_version 67150 (0.0007) [2023-10-08 10:26:58,240][53852] Updated weights for policy 0, policy_version 67160 (0.0010) [2023-10-08 10:26:58,768][53885] Updated weights for policy 1, policy_version 66852 (0.0008) [2023-10-08 10:26:59,134][53885] Updated weights for policy 1, policy_version 66862 (0.0008) [2023-10-08 10:26:59,504][53885] Updated weights for policy 1, policy_version 66872 (0.0008) [2023-10-08 10:27:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 137265152. Throughput: 0: 1833.1, 1: 1834.2. Samples: 34326598. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:27:02,015][52710] Avg episode reward: [(0, '32.100'), (1, '28.740')] [2023-10-08 10:27:02,058][53852] Updated weights for policy 0, policy_version 67170 (0.0009) [2023-10-08 10:27:02,464][53852] Updated weights for policy 0, policy_version 67180 (0.0008) [2023-10-08 10:27:02,837][53852] Updated weights for policy 0, policy_version 67190 (0.0010) [2023-10-08 10:27:03,205][53852] Updated weights for policy 0, policy_version 67200 (0.0009) [2023-10-08 10:27:03,270][53885] Updated weights for policy 1, policy_version 66882 (0.0009) [2023-10-08 10:27:03,636][53885] Updated weights for policy 1, policy_version 66892 (0.0010) [2023-10-08 10:27:04,004][53885] Updated weights for policy 1, policy_version 66902 (0.0010) [2023-10-08 10:27:04,371][53885] Updated weights for policy 1, policy_version 66912 (0.0010) [2023-10-08 10:27:06,743][53852] Updated weights for policy 0, policy_version 67210 (0.0009) [2023-10-08 10:27:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137330688. Throughput: 0: 1823.3, 1: 1833.9. Samples: 34349134. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:27:07,016][52710] Avg episode reward: [(0, '33.640'), (1, '31.960')] [2023-10-08 10:27:07,102][53852] Updated weights for policy 0, policy_version 67220 (0.0008) [2023-10-08 10:27:07,476][53852] Updated weights for policy 0, policy_version 67230 (0.0007) [2023-10-08 10:27:07,971][53885] Updated weights for policy 1, policy_version 66922 (0.0007) [2023-10-08 10:27:08,341][53885] Updated weights for policy 1, policy_version 66932 (0.0009) [2023-10-08 10:27:08,706][53885] Updated weights for policy 1, policy_version 66942 (0.0009) [2023-10-08 10:27:11,256][53852] Updated weights for policy 0, policy_version 67240 (0.0008) [2023-10-08 10:27:11,624][53852] Updated weights for policy 0, policy_version 67250 (0.0007) [2023-10-08 10:27:11,993][53852] Updated weights for policy 0, policy_version 67260 (0.0009) [2023-10-08 10:27:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137396224. Throughput: 0: 1823.7, 1: 1831.7. Samples: 34359386. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) [2023-10-08 10:27:12,016][52710] Avg episode reward: [(0, '32.490'), (1, '33.210')] [2023-10-08 10:27:12,493][53885] Updated weights for policy 1, policy_version 66952 (0.0009) [2023-10-08 10:27:12,855][53885] Updated weights for policy 1, policy_version 66962 (0.0007) [2023-10-08 10:27:13,218][53885] Updated weights for policy 1, policy_version 66972 (0.0007) [2023-10-08 10:27:15,447][53852] Updated weights for policy 0, policy_version 67270 (0.0008) [2023-10-08 10:27:15,817][53852] Updated weights for policy 0, policy_version 67280 (0.0007) [2023-10-08 10:27:16,188][53852] Updated weights for policy 0, policy_version 67290 (0.0007) [2023-10-08 10:27:16,942][53885] Updated weights for policy 1, policy_version 66982 (0.0009) [2023-10-08 10:27:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 137494528. Throughput: 0: 1832.7, 1: 1826.1. Samples: 34382272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:17,015][52710] Avg episode reward: [(0, '30.450'), (1, '31.450')] [2023-10-08 10:27:17,308][53885] Updated weights for policy 1, policy_version 66992 (0.0007) [2023-10-08 10:27:17,667][53885] Updated weights for policy 1, policy_version 67002 (0.0009) [2023-10-08 10:27:19,725][53852] Updated weights for policy 0, policy_version 67300 (0.0010) [2023-10-08 10:27:20,084][53852] Updated weights for policy 0, policy_version 67310 (0.0007) [2023-10-08 10:27:20,444][53852] Updated weights for policy 0, policy_version 67320 (0.0008) [2023-10-08 10:27:21,345][53885] Updated weights for policy 1, policy_version 67012 (0.0007) [2023-10-08 10:27:21,715][53885] Updated weights for policy 1, policy_version 67022 (0.0007) [2023-10-08 10:27:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 137560064. Throughput: 0: 1840.7, 1: 1827.2. Samples: 34403720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:22,016][52710] Avg episode reward: [(0, '31.350'), (1, '31.550')] [2023-10-08 10:27:22,081][53885] Updated weights for policy 1, policy_version 67032 (0.0008) [2023-10-08 10:27:24,010][53852] Updated weights for policy 0, policy_version 67330 (0.0010) [2023-10-08 10:27:24,375][53852] Updated weights for policy 0, policy_version 67340 (0.0009) [2023-10-08 10:27:24,752][53852] Updated weights for policy 0, policy_version 67350 (0.0008) [2023-10-08 10:27:25,116][53852] Updated weights for policy 0, policy_version 67360 (0.0009) [2023-10-08 10:27:25,732][53885] Updated weights for policy 1, policy_version 67042 (0.0009) [2023-10-08 10:27:26,103][53885] Updated weights for policy 1, policy_version 67052 (0.0009) [2023-10-08 10:27:26,475][53885] Updated weights for policy 1, policy_version 67062 (0.0007) [2023-10-08 10:27:26,837][53885] Updated weights for policy 1, policy_version 67072 (0.0008) [2023-10-08 10:27:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137658368. Throughput: 0: 1838.4, 1: 1822.9. Samples: 34415302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:27,015][52710] Avg episode reward: [(0, '32.640'), (1, '32.970')] [2023-10-08 10:27:28,710][53852] Updated weights for policy 0, policy_version 67370 (0.0010) [2023-10-08 10:27:29,080][53852] Updated weights for policy 0, policy_version 67380 (0.0008) [2023-10-08 10:27:29,450][53852] Updated weights for policy 0, policy_version 67390 (0.0007) [2023-10-08 10:27:30,506][53885] Updated weights for policy 1, policy_version 67082 (0.0010) [2023-10-08 10:27:30,868][53885] Updated weights for policy 1, policy_version 67092 (0.0008) [2023-10-08 10:27:31,231][53885] Updated weights for policy 1, policy_version 67102 (0.0008) [2023-10-08 10:27:32,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 137723904. Throughput: 0: 1854.8, 1: 1822.2. Samples: 34437246. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:32,016][52710] Avg episode reward: [(0, '29.580'), (1, '32.820')] [2023-10-08 10:27:33,051][53852] Updated weights for policy 0, policy_version 67400 (0.0008) [2023-10-08 10:27:33,418][53852] Updated weights for policy 0, policy_version 67410 (0.0007) [2023-10-08 10:27:33,788][53852] Updated weights for policy 0, policy_version 67420 (0.0008) [2023-10-08 10:27:34,965][53885] Updated weights for policy 1, policy_version 67112 (0.0009) [2023-10-08 10:27:35,335][53885] Updated weights for policy 1, policy_version 67122 (0.0008) [2023-10-08 10:27:35,709][53885] Updated weights for policy 1, policy_version 67132 (0.0009) [2023-10-08 10:27:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137789440. Throughput: 0: 1855.9, 1: 1823.8. Samples: 34459106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:37,016][52710] Avg episode reward: [(0, '30.870'), (1, '34.010')] [2023-10-08 10:27:37,343][53852] Updated weights for policy 0, policy_version 67430 (0.0008) [2023-10-08 10:27:37,714][53852] Updated weights for policy 0, policy_version 67440 (0.0007) [2023-10-08 10:27:38,084][53852] Updated weights for policy 0, policy_version 67450 (0.0007) [2023-10-08 10:27:39,327][53885] Updated weights for policy 1, policy_version 67142 (0.0011) [2023-10-08 10:27:39,696][53885] Updated weights for policy 1, policy_version 67152 (0.0009) [2023-10-08 10:27:40,072][53885] Updated weights for policy 1, policy_version 67162 (0.0011) [2023-10-08 10:27:41,703][53852] Updated weights for policy 0, policy_version 67460 (0.0009) [2023-10-08 10:27:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 137854976. Throughput: 0: 1858.8, 1: 1821.8. Samples: 34470052. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:42,016][52710] Avg episode reward: [(0, '31.940'), (1, '32.830')] [2023-10-08 10:27:42,079][53852] Updated weights for policy 0, policy_version 67470 (0.0007) [2023-10-08 10:27:42,443][53852] Updated weights for policy 0, policy_version 67480 (0.0007) [2023-10-08 10:27:43,813][53885] Updated weights for policy 1, policy_version 67172 (0.0009) [2023-10-08 10:27:44,177][53885] Updated weights for policy 1, policy_version 67182 (0.0008) [2023-10-08 10:27:44,556][53885] Updated weights for policy 1, policy_version 67192 (0.0008) [2023-10-08 10:27:46,094][53852] Updated weights for policy 0, policy_version 67490 (0.0008) [2023-10-08 10:27:46,460][53852] Updated weights for policy 0, policy_version 67500 (0.0008) [2023-10-08 10:27:46,826][53852] Updated weights for policy 0, policy_version 67510 (0.0008) [2023-10-08 10:27:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 137920512. Throughput: 0: 1855.7, 1: 1820.4. Samples: 34492022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:47,016][52710] Avg episode reward: [(0, '29.980'), (1, '33.900')] [2023-10-08 10:27:47,194][53852] Updated weights for policy 0, policy_version 67520 (0.0008) [2023-10-08 10:27:48,254][53885] Updated weights for policy 1, policy_version 67202 (0.0008) [2023-10-08 10:27:48,625][53885] Updated weights for policy 1, policy_version 67212 (0.0008) [2023-10-08 10:27:48,997][53885] Updated weights for policy 1, policy_version 67222 (0.0007) [2023-10-08 10:27:49,356][53885] Updated weights for policy 1, policy_version 67232 (0.0008) [2023-10-08 10:27:50,955][53852] Updated weights for policy 0, policy_version 67530 (0.0010) [2023-10-08 10:27:51,327][53852] Updated weights for policy 0, policy_version 67540 (0.0009) [2023-10-08 10:27:51,698][53852] Updated weights for policy 0, policy_version 67550 (0.0009) [2023-10-08 10:27:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 138018816. Throughput: 0: 1839.0, 1: 1818.3. Samples: 34513712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:52,016][52710] Avg episode reward: [(0, '30.990'), (1, '29.980')] [2023-10-08 10:27:53,002][53885] Updated weights for policy 1, policy_version 67242 (0.0007) [2023-10-08 10:27:53,374][53885] Updated weights for policy 1, policy_version 67252 (0.0008) [2023-10-08 10:27:53,735][53885] Updated weights for policy 1, policy_version 67262 (0.0008) [2023-10-08 10:27:55,355][53852] Updated weights for policy 0, policy_version 67560 (0.0011) [2023-10-08 10:27:55,713][53852] Updated weights for policy 0, policy_version 67570 (0.0009) [2023-10-08 10:27:56,094][53852] Updated weights for policy 0, policy_version 67580 (0.0010) [2023-10-08 10:27:57,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138084352. Throughput: 0: 1854.0, 1: 1819.5. Samples: 34524694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:27:57,015][52710] Avg episode reward: [(0, '31.040'), (1, '31.700')] [2023-10-08 10:27:57,518][53885] Updated weights for policy 1, policy_version 67272 (0.0007) [2023-10-08 10:27:57,885][53885] Updated weights for policy 1, policy_version 67282 (0.0007) [2023-10-08 10:27:58,252][53885] Updated weights for policy 1, policy_version 67292 (0.0010) [2023-10-08 10:27:59,791][53852] Updated weights for policy 0, policy_version 67590 (0.0009) [2023-10-08 10:28:00,157][53852] Updated weights for policy 0, policy_version 67600 (0.0009) [2023-10-08 10:28:00,529][53852] Updated weights for policy 0, policy_version 67610 (0.0009) [2023-10-08 10:28:01,733][53885] Updated weights for policy 1, policy_version 67302 (0.0009) [2023-10-08 10:28:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138149888. Throughput: 0: 1823.9, 1: 1823.7. Samples: 34546412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:02,016][52710] Avg episode reward: [(0, '28.620'), (1, '35.390')] [2023-10-08 10:28:02,094][53885] Updated weights for policy 1, policy_version 67312 (0.0007) [2023-10-08 10:28:02,470][53885] Updated weights for policy 1, policy_version 67322 (0.0009) [2023-10-08 10:28:04,457][53852] Updated weights for policy 0, policy_version 67620 (0.0008) [2023-10-08 10:28:04,822][53852] Updated weights for policy 0, policy_version 67630 (0.0007) [2023-10-08 10:28:05,199][53852] Updated weights for policy 0, policy_version 67640 (0.0008) [2023-10-08 10:28:06,243][53885] Updated weights for policy 1, policy_version 67332 (0.0010) [2023-10-08 10:28:06,600][53885] Updated weights for policy 1, policy_version 67342 (0.0011) [2023-10-08 10:28:06,978][53885] Updated weights for policy 1, policy_version 67352 (0.0008) [2023-10-08 10:28:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138215424. Throughput: 0: 1835.9, 1: 1824.6. Samples: 34568444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:07,016][52710] Avg episode reward: [(0, '31.840'), (1, '34.780')] [2023-10-08 10:28:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000067648_69271552.pth... [2023-10-08 10:28:07,062][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000065920_67502080.pth [2023-10-08 10:28:07,279][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000067360_68976640.pth... [2023-10-08 10:28:07,314][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000065632_67207168.pth [2023-10-08 10:28:08,798][53852] Updated weights for policy 0, policy_version 67650 (0.0008) [2023-10-08 10:28:09,173][53852] Updated weights for policy 0, policy_version 67660 (0.0007) [2023-10-08 10:28:09,536][53852] Updated weights for policy 0, policy_version 67670 (0.0007) [2023-10-08 10:28:09,905][53852] Updated weights for policy 0, policy_version 67680 (0.0009) [2023-10-08 10:28:10,599][53885] Updated weights for policy 1, policy_version 67362 (0.0010) [2023-10-08 10:28:10,966][53885] Updated weights for policy 1, policy_version 67372 (0.0007) [2023-10-08 10:28:11,331][53885] Updated weights for policy 1, policy_version 67382 (0.0007) [2023-10-08 10:28:11,702][53885] Updated weights for policy 1, policy_version 67392 (0.0009) [2023-10-08 10:28:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 138313728. Throughput: 0: 1824.9, 1: 1830.8. Samples: 34579810. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:12,016][52710] Avg episode reward: [(0, '32.670'), (1, '31.850')] [2023-10-08 10:28:13,517][53852] Updated weights for policy 0, policy_version 67690 (0.0009) [2023-10-08 10:28:13,877][53852] Updated weights for policy 0, policy_version 67700 (0.0008) [2023-10-08 10:28:14,253][53852] Updated weights for policy 0, policy_version 67710 (0.0008) [2023-10-08 10:28:15,188][53885] Updated weights for policy 1, policy_version 67402 (0.0009) [2023-10-08 10:28:15,562][53885] Updated weights for policy 1, policy_version 67412 (0.0010) [2023-10-08 10:28:15,932][53885] Updated weights for policy 1, policy_version 67422 (0.0010) [2023-10-08 10:28:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138379264. Throughput: 0: 1827.6, 1: 1828.3. Samples: 34601758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:17,015][52710] Avg episode reward: [(0, '31.370'), (1, '33.670')] [2023-10-08 10:28:17,951][53852] Updated weights for policy 0, policy_version 67720 (0.0007) [2023-10-08 10:28:18,306][53852] Updated weights for policy 0, policy_version 67730 (0.0008) [2023-10-08 10:28:18,675][53852] Updated weights for policy 0, policy_version 67740 (0.0007) [2023-10-08 10:28:19,608][53885] Updated weights for policy 1, policy_version 67432 (0.0007) [2023-10-08 10:28:19,993][53885] Updated weights for policy 1, policy_version 67442 (0.0008) [2023-10-08 10:28:20,363][53885] Updated weights for policy 1, policy_version 67452 (0.0008) [2023-10-08 10:28:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138444800. Throughput: 0: 1826.7, 1: 1835.1. Samples: 34623884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:22,016][52710] Avg episode reward: [(0, '30.900'), (1, '30.270')] [2023-10-08 10:28:22,186][53852] Updated weights for policy 0, policy_version 67750 (0.0010) [2023-10-08 10:28:22,557][53852] Updated weights for policy 0, policy_version 67760 (0.0010) [2023-10-08 10:28:22,929][53852] Updated weights for policy 0, policy_version 67770 (0.0009) [2023-10-08 10:28:23,919][53885] Updated weights for policy 1, policy_version 67462 (0.0008) [2023-10-08 10:28:24,289][53885] Updated weights for policy 1, policy_version 67472 (0.0008) [2023-10-08 10:28:24,660][53885] Updated weights for policy 1, policy_version 67482 (0.0007) [2023-10-08 10:28:26,572][53852] Updated weights for policy 0, policy_version 67780 (0.0011) [2023-10-08 10:28:26,938][53852] Updated weights for policy 0, policy_version 67790 (0.0008) [2023-10-08 10:28:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 138510336. Throughput: 0: 1822.6, 1: 1831.9. Samples: 34634504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:27,016][52710] Avg episode reward: [(0, '31.190'), (1, '29.680')] [2023-10-08 10:28:27,305][53852] Updated weights for policy 0, policy_version 67800 (0.0007) [2023-10-08 10:28:28,303][53885] Updated weights for policy 1, policy_version 67492 (0.0008) [2023-10-08 10:28:28,682][53885] Updated weights for policy 1, policy_version 67502 (0.0009) [2023-10-08 10:28:29,046][53885] Updated weights for policy 1, policy_version 67512 (0.0009) [2023-10-08 10:28:31,038][53852] Updated weights for policy 0, policy_version 67810 (0.0008) [2023-10-08 10:28:31,417][53852] Updated weights for policy 0, policy_version 67820 (0.0009) [2023-10-08 10:28:31,781][53852] Updated weights for policy 0, policy_version 67830 (0.0009) [2023-10-08 10:28:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 138575872. Throughput: 0: 1820.5, 1: 1849.0. Samples: 34657148. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:32,016][52710] Avg episode reward: [(0, '32.740'), (1, '29.450')] [2023-10-08 10:28:32,154][53852] Updated weights for policy 0, policy_version 67840 (0.0010) [2023-10-08 10:28:32,595][53885] Updated weights for policy 1, policy_version 67522 (0.0008) [2023-10-08 10:28:32,957][53885] Updated weights for policy 1, policy_version 67532 (0.0008) [2023-10-08 10:28:33,321][53885] Updated weights for policy 1, policy_version 67542 (0.0009) [2023-10-08 10:28:33,692][53885] Updated weights for policy 1, policy_version 67552 (0.0010) [2023-10-08 10:28:35,738][53852] Updated weights for policy 0, policy_version 67850 (0.0010) [2023-10-08 10:28:36,103][53852] Updated weights for policy 0, policy_version 67860 (0.0010) [2023-10-08 10:28:36,460][53852] Updated weights for policy 0, policy_version 67870 (0.0008) [2023-10-08 10:28:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 138674176. Throughput: 0: 1818.6, 1: 1851.7. Samples: 34678876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:37,016][52710] Avg episode reward: [(0, '29.790'), (1, '31.830')] [2023-10-08 10:28:37,317][53885] Updated weights for policy 1, policy_version 67562 (0.0009) [2023-10-08 10:28:37,683][53885] Updated weights for policy 1, policy_version 67572 (0.0009) [2023-10-08 10:28:38,049][53885] Updated weights for policy 1, policy_version 67582 (0.0008) [2023-10-08 10:28:39,985][53852] Updated weights for policy 0, policy_version 67880 (0.0009) [2023-10-08 10:28:40,347][53852] Updated weights for policy 0, policy_version 67890 (0.0008) [2023-10-08 10:28:40,710][53852] Updated weights for policy 0, policy_version 67900 (0.0009) [2023-10-08 10:28:41,624][53885] Updated weights for policy 1, policy_version 67592 (0.0007) [2023-10-08 10:28:41,989][53885] Updated weights for policy 1, policy_version 67602 (0.0007) [2023-10-08 10:28:42,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138739712. Throughput: 0: 1832.2, 1: 1853.8. Samples: 34690564. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:28:42,015][52710] Avg episode reward: [(0, '30.290'), (1, '32.040')] [2023-10-08 10:28:42,350][53885] Updated weights for policy 1, policy_version 67612 (0.0007) [2023-10-08 10:28:44,389][53852] Updated weights for policy 0, policy_version 67910 (0.0009) [2023-10-08 10:28:44,762][53852] Updated weights for policy 0, policy_version 67920 (0.0007) [2023-10-08 10:28:45,139][53852] Updated weights for policy 0, policy_version 67930 (0.0007) [2023-10-08 10:28:46,122][53885] Updated weights for policy 1, policy_version 67622 (0.0010) [2023-10-08 10:28:46,492][53885] Updated weights for policy 1, policy_version 67632 (0.0008) [2023-10-08 10:28:46,853][53885] Updated weights for policy 1, policy_version 67642 (0.0010) [2023-10-08 10:28:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138805248. Throughput: 0: 1832.7, 1: 1854.5. Samples: 34712334. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:28:47,016][52710] Avg episode reward: [(0, '30.370'), (1, '30.460')] [2023-10-08 10:28:48,807][53852] Updated weights for policy 0, policy_version 67940 (0.0008) [2023-10-08 10:28:49,175][53852] Updated weights for policy 0, policy_version 67950 (0.0008) [2023-10-08 10:28:49,541][53852] Updated weights for policy 0, policy_version 67960 (0.0009) [2023-10-08 10:28:50,585][53885] Updated weights for policy 1, policy_version 67652 (0.0011) [2023-10-08 10:28:50,945][53885] Updated weights for policy 1, policy_version 67662 (0.0010) [2023-10-08 10:28:51,310][53885] Updated weights for policy 1, policy_version 67672 (0.0007) [2023-10-08 10:28:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138903552. Throughput: 0: 1846.4, 1: 1832.0. Samples: 34733972. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:28:52,016][52710] Avg episode reward: [(0, '31.080'), (1, '33.260')] [2023-10-08 10:28:53,106][53852] Updated weights for policy 0, policy_version 67970 (0.0010) [2023-10-08 10:28:53,484][53852] Updated weights for policy 0, policy_version 67980 (0.0007) [2023-10-08 10:28:53,853][53852] Updated weights for policy 0, policy_version 67990 (0.0009) [2023-10-08 10:28:54,224][53852] Updated weights for policy 0, policy_version 68000 (0.0008) [2023-10-08 10:28:55,000][53885] Updated weights for policy 1, policy_version 67682 (0.0008) [2023-10-08 10:28:55,366][53885] Updated weights for policy 1, policy_version 67692 (0.0008) [2023-10-08 10:28:55,740][53885] Updated weights for policy 1, policy_version 67702 (0.0010) [2023-10-08 10:28:56,101][53885] Updated weights for policy 1, policy_version 67712 (0.0007) [2023-10-08 10:28:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138969088. Throughput: 0: 1833.7, 1: 1850.0. Samples: 34745576. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:28:57,016][52710] Avg episode reward: [(0, '29.470'), (1, '29.570')] [2023-10-08 10:28:58,025][53852] Updated weights for policy 0, policy_version 68010 (0.0007) [2023-10-08 10:28:58,388][53852] Updated weights for policy 0, policy_version 68020 (0.0008) [2023-10-08 10:28:58,753][53852] Updated weights for policy 0, policy_version 68030 (0.0007) [2023-10-08 10:28:59,737][53885] Updated weights for policy 1, policy_version 67722 (0.0008) [2023-10-08 10:29:00,105][53885] Updated weights for policy 1, policy_version 67732 (0.0008) [2023-10-08 10:29:00,478][53885] Updated weights for policy 1, policy_version 67742 (0.0008) [2023-10-08 10:29:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 139034624. Throughput: 0: 1841.6, 1: 1829.8. Samples: 34766972. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:29:02,016][52710] Avg episode reward: [(0, '32.770'), (1, '28.210')] [2023-10-08 10:29:02,418][53852] Updated weights for policy 0, policy_version 68040 (0.0008) [2023-10-08 10:29:02,795][53852] Updated weights for policy 0, policy_version 68050 (0.0008) [2023-10-08 10:29:03,159][53852] Updated weights for policy 0, policy_version 68060 (0.0009) [2023-10-08 10:29:04,075][53885] Updated weights for policy 1, policy_version 67752 (0.0008) [2023-10-08 10:29:04,447][53885] Updated weights for policy 1, policy_version 67762 (0.0007) [2023-10-08 10:29:04,809][53885] Updated weights for policy 1, policy_version 67772 (0.0008) [2023-10-08 10:29:06,695][53852] Updated weights for policy 0, policy_version 68070 (0.0008) [2023-10-08 10:29:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139100160. Throughput: 0: 1842.8, 1: 1849.7. Samples: 34790048. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:29:07,016][52710] Avg episode reward: [(0, '32.210'), (1, '30.100')] [2023-10-08 10:29:07,069][53852] Updated weights for policy 0, policy_version 68080 (0.0007) [2023-10-08 10:29:07,445][53852] Updated weights for policy 0, policy_version 68090 (0.0007) [2023-10-08 10:29:08,582][53885] Updated weights for policy 1, policy_version 67782 (0.0010) [2023-10-08 10:29:08,971][53885] Updated weights for policy 1, policy_version 67792 (0.0008) [2023-10-08 10:29:09,332][53885] Updated weights for policy 1, policy_version 67802 (0.0009) [2023-10-08 10:29:11,152][53852] Updated weights for policy 0, policy_version 68100 (0.0007) [2023-10-08 10:29:11,522][53852] Updated weights for policy 0, policy_version 68110 (0.0007) [2023-10-08 10:29:11,898][53852] Updated weights for policy 0, policy_version 68120 (0.0007) [2023-10-08 10:29:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 139165696. Throughput: 0: 1844.4, 1: 1832.7. Samples: 34799974. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:29:12,016][52710] Avg episode reward: [(0, '32.280'), (1, '32.440')] [2023-10-08 10:29:12,960][53885] Updated weights for policy 1, policy_version 67812 (0.0008) [2023-10-08 10:29:13,331][53885] Updated weights for policy 1, policy_version 67822 (0.0009) [2023-10-08 10:29:13,690][53885] Updated weights for policy 1, policy_version 67832 (0.0007) [2023-10-08 10:29:15,567][53852] Updated weights for policy 0, policy_version 68130 (0.0008) [2023-10-08 10:29:15,932][53852] Updated weights for policy 0, policy_version 68140 (0.0009) [2023-10-08 10:29:16,310][53852] Updated weights for policy 0, policy_version 68150 (0.0008) [2023-10-08 10:29:16,670][53852] Updated weights for policy 0, policy_version 68160 (0.0009) [2023-10-08 10:29:17,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 139264000. Throughput: 0: 1842.0, 1: 1840.6. Samples: 34822864. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:29:17,015][52710] Avg episode reward: [(0, '36.410'), (1, '28.350')] [2023-10-08 10:29:17,016][53500] Saving new best policy, reward=36.410! [2023-10-08 10:29:17,390][53885] Updated weights for policy 1, policy_version 67842 (0.0007) [2023-10-08 10:29:17,754][53885] Updated weights for policy 1, policy_version 67852 (0.0007) [2023-10-08 10:29:18,117][53885] Updated weights for policy 1, policy_version 67862 (0.0007) [2023-10-08 10:29:18,480][53885] Updated weights for policy 1, policy_version 67872 (0.0007) [2023-10-08 10:29:20,409][53852] Updated weights for policy 0, policy_version 68170 (0.0009) [2023-10-08 10:29:20,784][53852] Updated weights for policy 0, policy_version 68180 (0.0009) [2023-10-08 10:29:21,156][53852] Updated weights for policy 0, policy_version 68190 (0.0012) [2023-10-08 10:29:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139329536. Throughput: 0: 1843.1, 1: 1838.4. Samples: 34844542. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:29:22,016][52710] Avg episode reward: [(0, '30.490'), (1, '32.610')] [2023-10-08 10:29:22,142][53885] Updated weights for policy 1, policy_version 67882 (0.0009) [2023-10-08 10:29:22,520][53885] Updated weights for policy 1, policy_version 67892 (0.0008) [2023-10-08 10:29:22,890][53885] Updated weights for policy 1, policy_version 67902 (0.0008) [2023-10-08 10:29:24,800][53852] Updated weights for policy 0, policy_version 68200 (0.0009) [2023-10-08 10:29:25,176][53852] Updated weights for policy 0, policy_version 68210 (0.0010) [2023-10-08 10:29:25,545][53852] Updated weights for policy 0, policy_version 68220 (0.0008) [2023-10-08 10:29:26,543][53885] Updated weights for policy 1, policy_version 67912 (0.0007) [2023-10-08 10:29:26,911][53885] Updated weights for policy 1, policy_version 67922 (0.0010) [2023-10-08 10:29:27,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139395072. Throughput: 0: 1842.9, 1: 1833.5. Samples: 34856002. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 10:29:27,016][52710] Avg episode reward: [(0, '30.540'), (1, '28.290')] [2023-10-08 10:29:27,276][53885] Updated weights for policy 1, policy_version 67932 (0.0009) [2023-10-08 10:29:29,176][53852] Updated weights for policy 0, policy_version 68230 (0.0007) [2023-10-08 10:29:29,552][53852] Updated weights for policy 0, policy_version 68240 (0.0008) [2023-10-08 10:29:29,927][53852] Updated weights for policy 0, policy_version 68250 (0.0009) [2023-10-08 10:29:30,813][53885] Updated weights for policy 1, policy_version 67942 (0.0009) [2023-10-08 10:29:31,174][53885] Updated weights for policy 1, policy_version 67952 (0.0008) [2023-10-08 10:29:31,539][53885] Updated weights for policy 1, policy_version 67962 (0.0010) [2023-10-08 10:29:32,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 139493376. Throughput: 0: 1835.1, 1: 1838.7. Samples: 34877656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:29:32,016][52710] Avg episode reward: [(0, '30.950'), (1, '31.100')] [2023-10-08 10:29:33,753][53852] Updated weights for policy 0, policy_version 68260 (0.0009) [2023-10-08 10:29:34,118][53852] Updated weights for policy 0, policy_version 68270 (0.0010) [2023-10-08 10:29:34,497][53852] Updated weights for policy 0, policy_version 68280 (0.0009) [2023-10-08 10:29:35,196][53885] Updated weights for policy 1, policy_version 67972 (0.0009) [2023-10-08 10:29:35,559][53885] Updated weights for policy 1, policy_version 67982 (0.0009) [2023-10-08 10:29:35,934][53885] Updated weights for policy 1, policy_version 67992 (0.0011) [2023-10-08 10:29:37,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139558912. Throughput: 0: 1830.7, 1: 1839.4. Samples: 34899128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:29:37,016][52710] Avg episode reward: [(0, '27.040'), (1, '31.220')] [2023-10-08 10:29:38,083][53852] Updated weights for policy 0, policy_version 68290 (0.0008) [2023-10-08 10:29:38,456][53852] Updated weights for policy 0, policy_version 68300 (0.0008) [2023-10-08 10:29:38,818][53852] Updated weights for policy 0, policy_version 68310 (0.0007) [2023-10-08 10:29:39,189][53852] Updated weights for policy 0, policy_version 68320 (0.0010) [2023-10-08 10:29:39,560][53885] Updated weights for policy 1, policy_version 68002 (0.0008) [2023-10-08 10:29:39,926][53885] Updated weights for policy 1, policy_version 68012 (0.0007) [2023-10-08 10:29:40,290][53885] Updated weights for policy 1, policy_version 68022 (0.0010) [2023-10-08 10:29:40,662][53885] Updated weights for policy 1, policy_version 68032 (0.0010) [2023-10-08 10:29:42,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139624448. Throughput: 0: 1828.0, 1: 1833.9. Samples: 34910358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:29:42,015][52710] Avg episode reward: [(0, '28.750'), (1, '28.530')] [2023-10-08 10:29:42,925][53852] Updated weights for policy 0, policy_version 68330 (0.0007) [2023-10-08 10:29:43,303][53852] Updated weights for policy 0, policy_version 68340 (0.0008) [2023-10-08 10:29:43,671][53852] Updated weights for policy 0, policy_version 68350 (0.0007) [2023-10-08 10:29:44,389][53885] Updated weights for policy 1, policy_version 68042 (0.0008) [2023-10-08 10:29:44,747][53885] Updated weights for policy 1, policy_version 68052 (0.0010) [2023-10-08 10:29:45,107][53885] Updated weights for policy 1, policy_version 68062 (0.0009) [2023-10-08 10:29:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139689984. Throughput: 0: 1830.2, 1: 1832.2. Samples: 34931780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:29:47,016][52710] Avg episode reward: [(0, '31.760'), (1, '29.610')] [2023-10-08 10:29:47,381][53852] Updated weights for policy 0, policy_version 68360 (0.0010) [2023-10-08 10:29:47,759][53852] Updated weights for policy 0, policy_version 68370 (0.0007) [2023-10-08 10:29:48,124][53852] Updated weights for policy 0, policy_version 68380 (0.0008) [2023-10-08 10:29:48,777][53885] Updated weights for policy 1, policy_version 68072 (0.0008) [2023-10-08 10:29:49,145][53885] Updated weights for policy 1, policy_version 68082 (0.0009) [2023-10-08 10:29:49,512][53885] Updated weights for policy 1, policy_version 68092 (0.0008) [2023-10-08 10:29:51,790][53852] Updated weights for policy 0, policy_version 68390 (0.0010) [2023-10-08 10:29:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 139755520. Throughput: 0: 1829.9, 1: 1829.0. Samples: 34954698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:29:52,016][52710] Avg episode reward: [(0, '31.310'), (1, '32.340')] [2023-10-08 10:29:52,160][53852] Updated weights for policy 0, policy_version 68400 (0.0009) [2023-10-08 10:29:52,531][53852] Updated weights for policy 0, policy_version 68410 (0.0008) [2023-10-08 10:29:53,379][53885] Updated weights for policy 1, policy_version 68102 (0.0009) [2023-10-08 10:29:53,756][53885] Updated weights for policy 1, policy_version 68112 (0.0008) [2023-10-08 10:29:54,127][53885] Updated weights for policy 1, policy_version 68122 (0.0007) [2023-10-08 10:29:56,142][53852] Updated weights for policy 0, policy_version 68420 (0.0009) [2023-10-08 10:29:56,505][53852] Updated weights for policy 0, policy_version 68430 (0.0010) [2023-10-08 10:29:56,880][53852] Updated weights for policy 0, policy_version 68440 (0.0008) [2023-10-08 10:29:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 139821056. Throughput: 0: 1830.0, 1: 1830.9. Samples: 34964716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:29:57,016][52710] Avg episode reward: [(0, '31.300'), (1, '30.750')] [2023-10-08 10:29:57,659][53885] Updated weights for policy 1, policy_version 68132 (0.0007) [2023-10-08 10:29:58,052][53885] Updated weights for policy 1, policy_version 68142 (0.0008) [2023-10-08 10:29:58,416][53885] Updated weights for policy 1, policy_version 68152 (0.0010) [2023-10-08 10:30:00,368][53852] Updated weights for policy 0, policy_version 68450 (0.0007) [2023-10-08 10:30:00,734][53852] Updated weights for policy 0, policy_version 68460 (0.0008) [2023-10-08 10:30:01,101][53852] Updated weights for policy 0, policy_version 68470 (0.0008) [2023-10-08 10:30:01,462][53852] Updated weights for policy 0, policy_version 68480 (0.0008) [2023-10-08 10:30:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 139919360. Throughput: 0: 1825.5, 1: 1833.2. Samples: 34987508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:30:02,016][52710] Avg episode reward: [(0, '32.390'), (1, '31.020')] [2023-10-08 10:30:02,112][53885] Updated weights for policy 1, policy_version 68162 (0.0009) [2023-10-08 10:30:02,476][53885] Updated weights for policy 1, policy_version 68172 (0.0008) [2023-10-08 10:30:02,848][53885] Updated weights for policy 1, policy_version 68182 (0.0010) [2023-10-08 10:30:03,209][53885] Updated weights for policy 1, policy_version 68192 (0.0010) [2023-10-08 10:30:05,076][53852] Updated weights for policy 0, policy_version 68490 (0.0008) [2023-10-08 10:30:05,454][53852] Updated weights for policy 0, policy_version 68500 (0.0008) [2023-10-08 10:30:05,817][53852] Updated weights for policy 0, policy_version 68510 (0.0008) [2023-10-08 10:30:06,871][53885] Updated weights for policy 1, policy_version 68202 (0.0008) [2023-10-08 10:30:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139984896. Throughput: 0: 1835.7, 1: 1825.4. Samples: 35009292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:30:07,016][52710] Avg episode reward: [(0, '30.720'), (1, '31.070')] [2023-10-08 10:30:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000068512_70156288.pth... [2023-10-08 10:30:07,054][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000066784_68386816.pth [2023-10-08 10:30:07,058][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000068512_70156288.pth [2023-10-08 10:30:07,229][53885] Updated weights for policy 1, policy_version 68212 (0.0008) [2023-10-08 10:30:07,605][53885] Updated weights for policy 1, policy_version 68222 (0.0009) [2023-10-08 10:30:07,669][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000068224_69861376.pth... [2023-10-08 10:30:07,706][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000066496_68091904.pth [2023-10-08 10:30:07,710][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000068224_69861376.pth [2023-10-08 10:30:09,468][53852] Updated weights for policy 0, policy_version 68520 (0.0009) [2023-10-08 10:30:09,855][53852] Updated weights for policy 0, policy_version 68530 (0.0008) [2023-10-08 10:30:10,220][53852] Updated weights for policy 0, policy_version 68540 (0.0009) [2023-10-08 10:30:11,331][53885] Updated weights for policy 1, policy_version 68232 (0.0008) [2023-10-08 10:30:11,694][53885] Updated weights for policy 1, policy_version 68242 (0.0008) [2023-10-08 10:30:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 140050432. Throughput: 0: 1822.2, 1: 1829.5. Samples: 35020330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:30:12,016][52710] Avg episode reward: [(0, '30.010'), (1, '29.020')] [2023-10-08 10:30:12,066][53885] Updated weights for policy 1, policy_version 68252 (0.0010) [2023-10-08 10:30:13,837][53852] Updated weights for policy 0, policy_version 68550 (0.0011) [2023-10-08 10:30:14,215][53852] Updated weights for policy 0, policy_version 68560 (0.0009) [2023-10-08 10:30:14,581][53852] Updated weights for policy 0, policy_version 68570 (0.0008) [2023-10-08 10:30:15,669][53885] Updated weights for policy 1, policy_version 68262 (0.0010) [2023-10-08 10:30:16,052][53885] Updated weights for policy 1, policy_version 68272 (0.0009) [2023-10-08 10:30:16,412][53885] Updated weights for policy 1, policy_version 68282 (0.0010) [2023-10-08 10:30:17,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140148736. Throughput: 0: 1828.8, 1: 1819.4. Samples: 35041824. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:17,015][52710] Avg episode reward: [(0, '29.850'), (1, '31.220')] [2023-10-08 10:30:18,246][53852] Updated weights for policy 0, policy_version 68580 (0.0008) [2023-10-08 10:30:18,619][53852] Updated weights for policy 0, policy_version 68590 (0.0008) [2023-10-08 10:30:18,991][53852] Updated weights for policy 0, policy_version 68600 (0.0010) [2023-10-08 10:30:20,126][53885] Updated weights for policy 1, policy_version 68292 (0.0009) [2023-10-08 10:30:20,497][53885] Updated weights for policy 1, policy_version 68302 (0.0009) [2023-10-08 10:30:20,874][53885] Updated weights for policy 1, policy_version 68312 (0.0009) [2023-10-08 10:30:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140214272. Throughput: 0: 1830.6, 1: 1818.9. Samples: 35063356. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:22,016][52710] Avg episode reward: [(0, '28.560'), (1, '30.600')] [2023-10-08 10:30:22,588][53852] Updated weights for policy 0, policy_version 68610 (0.0009) [2023-10-08 10:30:22,966][53852] Updated weights for policy 0, policy_version 68620 (0.0009) [2023-10-08 10:30:23,328][53852] Updated weights for policy 0, policy_version 68630 (0.0010) [2023-10-08 10:30:23,702][53852] Updated weights for policy 0, policy_version 68640 (0.0009) [2023-10-08 10:30:24,582][53885] Updated weights for policy 1, policy_version 68322 (0.0007) [2023-10-08 10:30:24,958][53885] Updated weights for policy 1, policy_version 68332 (0.0007) [2023-10-08 10:30:25,332][53885] Updated weights for policy 1, policy_version 68342 (0.0008) [2023-10-08 10:30:25,695][53885] Updated weights for policy 1, policy_version 68352 (0.0009) [2023-10-08 10:30:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 140279808. Throughput: 0: 1831.2, 1: 1822.0. Samples: 35074754. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:27,015][52710] Avg episode reward: [(0, '32.200'), (1, '32.400')] [2023-10-08 10:30:27,395][53852] Updated weights for policy 0, policy_version 68650 (0.0009) [2023-10-08 10:30:27,780][53852] Updated weights for policy 0, policy_version 68660 (0.0009) [2023-10-08 10:30:28,143][53852] Updated weights for policy 0, policy_version 68670 (0.0008) [2023-10-08 10:30:29,265][53885] Updated weights for policy 1, policy_version 68362 (0.0007) [2023-10-08 10:30:29,635][53885] Updated weights for policy 1, policy_version 68372 (0.0007) [2023-10-08 10:30:30,008][53885] Updated weights for policy 1, policy_version 68382 (0.0007) [2023-10-08 10:30:31,687][53852] Updated weights for policy 0, policy_version 68680 (0.0007) [2023-10-08 10:30:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 140345344. Throughput: 0: 1834.0, 1: 1826.4. Samples: 35096500. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:32,015][52710] Avg episode reward: [(0, '29.840'), (1, '35.050')] [2023-10-08 10:30:32,059][53852] Updated weights for policy 0, policy_version 68690 (0.0007) [2023-10-08 10:30:32,431][53852] Updated weights for policy 0, policy_version 68700 (0.0007) [2023-10-08 10:30:33,750][53885] Updated weights for policy 1, policy_version 68392 (0.0010) [2023-10-08 10:30:34,122][53885] Updated weights for policy 1, policy_version 68402 (0.0009) [2023-10-08 10:30:34,486][53885] Updated weights for policy 1, policy_version 68412 (0.0009) [2023-10-08 10:30:36,139][53852] Updated weights for policy 0, policy_version 68710 (0.0008) [2023-10-08 10:30:36,500][53852] Updated weights for policy 0, policy_version 68720 (0.0009) [2023-10-08 10:30:36,878][53852] Updated weights for policy 0, policy_version 68730 (0.0010) [2023-10-08 10:30:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 140410880. Throughput: 0: 1817.1, 1: 1826.4. Samples: 35118654. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:37,016][52710] Avg episode reward: [(0, '29.540'), (1, '33.200')] [2023-10-08 10:30:38,134][53885] Updated weights for policy 1, policy_version 68422 (0.0008) [2023-10-08 10:30:38,495][53885] Updated weights for policy 1, policy_version 68432 (0.0007) [2023-10-08 10:30:38,863][53885] Updated weights for policy 1, policy_version 68442 (0.0008) [2023-10-08 10:30:40,555][53852] Updated weights for policy 0, policy_version 68740 (0.0011) [2023-10-08 10:30:40,927][53852] Updated weights for policy 0, policy_version 68750 (0.0010) [2023-10-08 10:30:41,293][53852] Updated weights for policy 0, policy_version 68760 (0.0010) [2023-10-08 10:30:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 140509184. Throughput: 0: 1831.6, 1: 1829.1. Samples: 35129450. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:42,016][52710] Avg episode reward: [(0, '32.400'), (1, '33.970')] [2023-10-08 10:30:42,644][53885] Updated weights for policy 1, policy_version 68452 (0.0008) [2023-10-08 10:30:43,007][53885] Updated weights for policy 1, policy_version 68462 (0.0008) [2023-10-08 10:30:43,384][53885] Updated weights for policy 1, policy_version 68472 (0.0009) [2023-10-08 10:30:44,961][53852] Updated weights for policy 0, policy_version 68770 (0.0009) [2023-10-08 10:30:45,339][53852] Updated weights for policy 0, policy_version 68780 (0.0007) [2023-10-08 10:30:45,702][53852] Updated weights for policy 0, policy_version 68790 (0.0008) [2023-10-08 10:30:46,069][53852] Updated weights for policy 0, policy_version 68800 (0.0007) [2023-10-08 10:30:46,976][53885] Updated weights for policy 1, policy_version 68482 (0.0008) [2023-10-08 10:30:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140574720. Throughput: 0: 1824.5, 1: 1825.7. Samples: 35151770. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:47,016][52710] Avg episode reward: [(0, '28.990'), (1, '38.080')] [2023-10-08 10:30:47,337][53885] Updated weights for policy 1, policy_version 68492 (0.0011) [2023-10-08 10:30:47,719][53885] Updated weights for policy 1, policy_version 68502 (0.0010) [2023-10-08 10:30:48,083][53885] Updated weights for policy 1, policy_version 68512 (0.0009) [2023-10-08 10:30:49,585][53852] Updated weights for policy 0, policy_version 68810 (0.0008) [2023-10-08 10:30:49,951][53852] Updated weights for policy 0, policy_version 68820 (0.0009) [2023-10-08 10:30:50,319][53852] Updated weights for policy 0, policy_version 68830 (0.0009) [2023-10-08 10:30:51,771][53885] Updated weights for policy 1, policy_version 68522 (0.0009) [2023-10-08 10:30:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140640256. Throughput: 0: 1833.4, 1: 1823.3. Samples: 35173842. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:52,016][52710] Avg episode reward: [(0, '28.090'), (1, '32.620')] [2023-10-08 10:30:52,136][53885] Updated weights for policy 1, policy_version 68532 (0.0011) [2023-10-08 10:30:52,496][53885] Updated weights for policy 1, policy_version 68542 (0.0010) [2023-10-08 10:30:53,968][53852] Updated weights for policy 0, policy_version 68840 (0.0008) [2023-10-08 10:30:54,329][53852] Updated weights for policy 0, policy_version 68850 (0.0011) [2023-10-08 10:30:54,696][53852] Updated weights for policy 0, policy_version 68860 (0.0009) [2023-10-08 10:30:56,134][53885] Updated weights for policy 1, policy_version 68552 (0.0010) [2023-10-08 10:30:56,494][53885] Updated weights for policy 1, policy_version 68562 (0.0011) [2023-10-08 10:30:56,857][53885] Updated weights for policy 1, policy_version 68572 (0.0008) [2023-10-08 10:30:57,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 140738560. Throughput: 0: 1819.8, 1: 1826.1. Samples: 35184398. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) [2023-10-08 10:30:57,016][52710] Avg episode reward: [(0, '33.530'), (1, '34.310')] [2023-10-08 10:30:58,512][53852] Updated weights for policy 0, policy_version 68870 (0.0008) [2023-10-08 10:30:58,880][53852] Updated weights for policy 0, policy_version 68880 (0.0010) [2023-10-08 10:30:59,243][53852] Updated weights for policy 0, policy_version 68890 (0.0010) [2023-10-08 10:31:00,581][53885] Updated weights for policy 1, policy_version 68582 (0.0007) [2023-10-08 10:31:00,946][53885] Updated weights for policy 1, policy_version 68592 (0.0009) [2023-10-08 10:31:01,312][53885] Updated weights for policy 1, policy_version 68602 (0.0008) [2023-10-08 10:31:02,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140804096. Throughput: 0: 1841.4, 1: 1823.7. Samples: 35206752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:02,016][52710] Avg episode reward: [(0, '31.580'), (1, '35.750')] [2023-10-08 10:31:02,906][53852] Updated weights for policy 0, policy_version 68900 (0.0010) [2023-10-08 10:31:03,275][53852] Updated weights for policy 0, policy_version 68910 (0.0010) [2023-10-08 10:31:03,643][53852] Updated weights for policy 0, policy_version 68920 (0.0008) [2023-10-08 10:31:04,907][53885] Updated weights for policy 1, policy_version 68612 (0.0008) [2023-10-08 10:31:05,276][53885] Updated weights for policy 1, policy_version 68622 (0.0009) [2023-10-08 10:31:05,638][53885] Updated weights for policy 1, policy_version 68632 (0.0010) [2023-10-08 10:31:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140869632. Throughput: 0: 1843.9, 1: 1827.7. Samples: 35228578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:07,016][52710] Avg episode reward: [(0, '31.040'), (1, '33.090')] [2023-10-08 10:31:07,250][53852] Updated weights for policy 0, policy_version 68930 (0.0007) [2023-10-08 10:31:07,612][53852] Updated weights for policy 0, policy_version 68940 (0.0008) [2023-10-08 10:31:07,988][53852] Updated weights for policy 0, policy_version 68950 (0.0007) [2023-10-08 10:31:08,352][53852] Updated weights for policy 0, policy_version 68960 (0.0007) [2023-10-08 10:31:09,226][53885] Updated weights for policy 1, policy_version 68642 (0.0008) [2023-10-08 10:31:09,596][53885] Updated weights for policy 1, policy_version 68652 (0.0011) [2023-10-08 10:31:09,966][53885] Updated weights for policy 1, policy_version 68662 (0.0011) [2023-10-08 10:31:10,338][53885] Updated weights for policy 1, policy_version 68672 (0.0008) [2023-10-08 10:31:11,998][53852] Updated weights for policy 0, policy_version 68970 (0.0010) [2023-10-08 10:31:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140935168. Throughput: 0: 1849.1, 1: 1821.2. Samples: 35239918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:12,016][52710] Avg episode reward: [(0, '31.220'), (1, '30.720')] [2023-10-08 10:31:12,371][53852] Updated weights for policy 0, policy_version 68980 (0.0008) [2023-10-08 10:31:12,747][53852] Updated weights for policy 0, policy_version 68990 (0.0007) [2023-10-08 10:31:14,062][53885] Updated weights for policy 1, policy_version 68682 (0.0007) [2023-10-08 10:31:14,432][53885] Updated weights for policy 1, policy_version 68692 (0.0007) [2023-10-08 10:31:14,807][53885] Updated weights for policy 1, policy_version 68702 (0.0008) [2023-10-08 10:31:16,364][53852] Updated weights for policy 0, policy_version 69000 (0.0009) [2023-10-08 10:31:16,729][53852] Updated weights for policy 0, policy_version 69010 (0.0008) [2023-10-08 10:31:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 141000704. Throughput: 0: 1845.2, 1: 1825.6. Samples: 35261686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:17,016][52710] Avg episode reward: [(0, '35.160'), (1, '31.990')] [2023-10-08 10:31:17,102][53852] Updated weights for policy 0, policy_version 69020 (0.0008) [2023-10-08 10:31:18,350][53885] Updated weights for policy 1, policy_version 68712 (0.0009) [2023-10-08 10:31:18,714][53885] Updated weights for policy 1, policy_version 68722 (0.0007) [2023-10-08 10:31:19,082][53885] Updated weights for policy 1, policy_version 68732 (0.0007) [2023-10-08 10:31:20,822][53852] Updated weights for policy 0, policy_version 69030 (0.0008) [2023-10-08 10:31:21,191][53852] Updated weights for policy 0, policy_version 69040 (0.0009) [2023-10-08 10:31:21,564][53852] Updated weights for policy 0, policy_version 69050 (0.0009) [2023-10-08 10:31:22,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 141099008. Throughput: 0: 1834.9, 1: 1827.6. Samples: 35283468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:22,016][52710] Avg episode reward: [(0, '30.220'), (1, '29.780')] [2023-10-08 10:31:22,908][53885] Updated weights for policy 1, policy_version 68742 (0.0010) [2023-10-08 10:31:23,283][53885] Updated weights for policy 1, policy_version 68752 (0.0011) [2023-10-08 10:31:23,651][53885] Updated weights for policy 1, policy_version 68762 (0.0010) [2023-10-08 10:31:25,156][53852] Updated weights for policy 0, policy_version 69060 (0.0010) [2023-10-08 10:31:25,530][53852] Updated weights for policy 0, policy_version 69070 (0.0009) [2023-10-08 10:31:25,893][53852] Updated weights for policy 0, policy_version 69080 (0.0009) [2023-10-08 10:31:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141164544. Throughput: 0: 1848.6, 1: 1823.9. Samples: 35294712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:27,015][52710] Avg episode reward: [(0, '31.160'), (1, '31.600')] [2023-10-08 10:31:27,393][53885] Updated weights for policy 1, policy_version 68772 (0.0010) [2023-10-08 10:31:27,779][53885] Updated weights for policy 1, policy_version 68782 (0.0009) [2023-10-08 10:31:28,141][53885] Updated weights for policy 1, policy_version 68792 (0.0008) [2023-10-08 10:31:29,374][53852] Updated weights for policy 0, policy_version 69090 (0.0008) [2023-10-08 10:31:29,747][53852] Updated weights for policy 0, policy_version 69100 (0.0008) [2023-10-08 10:31:30,108][53852] Updated weights for policy 0, policy_version 69110 (0.0007) [2023-10-08 10:31:30,473][53852] Updated weights for policy 0, policy_version 69120 (0.0011) [2023-10-08 10:31:31,798][53885] Updated weights for policy 1, policy_version 68802 (0.0009) [2023-10-08 10:31:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 141230080. Throughput: 0: 1833.7, 1: 1828.3. Samples: 35316562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:32,016][52710] Avg episode reward: [(0, '28.650'), (1, '32.690')] [2023-10-08 10:31:32,168][53885] Updated weights for policy 1, policy_version 68812 (0.0010) [2023-10-08 10:31:32,533][53885] Updated weights for policy 1, policy_version 68822 (0.0007) [2023-10-08 10:31:32,895][53885] Updated weights for policy 1, policy_version 68832 (0.0007) [2023-10-08 10:31:34,080][53852] Updated weights for policy 0, policy_version 69130 (0.0011) [2023-10-08 10:31:34,453][53852] Updated weights for policy 0, policy_version 69140 (0.0007) [2023-10-08 10:31:34,814][53852] Updated weights for policy 0, policy_version 69150 (0.0007) [2023-10-08 10:31:36,631][53885] Updated weights for policy 1, policy_version 68842 (0.0009) [2023-10-08 10:31:37,003][53885] Updated weights for policy 1, policy_version 68852 (0.0009) [2023-10-08 10:31:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 141295616. Throughput: 0: 1848.6, 1: 1825.8. Samples: 35339192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:37,016][52710] Avg episode reward: [(0, '29.810'), (1, '33.560')] [2023-10-08 10:31:37,371][53885] Updated weights for policy 1, policy_version 68862 (0.0008) [2023-10-08 10:31:38,481][53852] Updated weights for policy 0, policy_version 69160 (0.0009) [2023-10-08 10:31:38,845][53852] Updated weights for policy 0, policy_version 69170 (0.0007) [2023-10-08 10:31:39,211][53852] Updated weights for policy 0, policy_version 69180 (0.0008) [2023-10-08 10:31:40,987][53885] Updated weights for policy 1, policy_version 68872 (0.0008) [2023-10-08 10:31:41,359][53885] Updated weights for policy 1, policy_version 68882 (0.0008) [2023-10-08 10:31:41,711][53885] Updated weights for policy 1, policy_version 68892 (0.0008) [2023-10-08 10:31:42,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141393920. Throughput: 0: 1843.6, 1: 1831.2. Samples: 35349768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:31:42,016][52710] Avg episode reward: [(0, '28.850'), (1, '33.160')] [2023-10-08 10:31:42,870][53852] Updated weights for policy 0, policy_version 69190 (0.0008) [2023-10-08 10:31:43,241][53852] Updated weights for policy 0, policy_version 69200 (0.0009) [2023-10-08 10:31:43,618][53852] Updated weights for policy 0, policy_version 69210 (0.0008) [2023-10-08 10:31:45,505][53885] Updated weights for policy 1, policy_version 68902 (0.0008) [2023-10-08 10:31:45,859][53885] Updated weights for policy 1, policy_version 68912 (0.0008) [2023-10-08 10:31:46,233][53885] Updated weights for policy 1, policy_version 68922 (0.0008) [2023-10-08 10:31:47,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 141459456. Throughput: 0: 1854.8, 1: 1822.4. Samples: 35372228. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:31:47,017][52710] Avg episode reward: [(0, '29.630'), (1, '36.250')] [2023-10-08 10:31:47,253][53852] Updated weights for policy 0, policy_version 69220 (0.0009) [2023-10-08 10:31:47,647][53852] Updated weights for policy 0, policy_version 69230 (0.0010) [2023-10-08 10:31:48,018][53852] Updated weights for policy 0, policy_version 69240 (0.0007) [2023-10-08 10:31:49,865][53885] Updated weights for policy 1, policy_version 68932 (0.0008) [2023-10-08 10:31:50,226][53885] Updated weights for policy 1, policy_version 68942 (0.0010) [2023-10-08 10:31:50,596][53885] Updated weights for policy 1, policy_version 68952 (0.0008) [2023-10-08 10:31:51,701][53852] Updated weights for policy 0, policy_version 69250 (0.0008) [2023-10-08 10:31:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141524992. Throughput: 0: 1851.6, 1: 1828.9. Samples: 35394202. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:31:52,016][52710] Avg episode reward: [(0, '27.740'), (1, '33.710')] [2023-10-08 10:31:52,069][53852] Updated weights for policy 0, policy_version 69260 (0.0010) [2023-10-08 10:31:52,437][53852] Updated weights for policy 0, policy_version 69270 (0.0010) [2023-10-08 10:31:52,811][53852] Updated weights for policy 0, policy_version 69280 (0.0008) [2023-10-08 10:31:54,148][53885] Updated weights for policy 1, policy_version 68962 (0.0008) [2023-10-08 10:31:54,522][53885] Updated weights for policy 1, policy_version 68972 (0.0007) [2023-10-08 10:31:54,893][53885] Updated weights for policy 1, policy_version 68982 (0.0008) [2023-10-08 10:31:55,260][53885] Updated weights for policy 1, policy_version 68992 (0.0008) [2023-10-08 10:31:56,405][53852] Updated weights for policy 0, policy_version 69290 (0.0008) [2023-10-08 10:31:56,767][53852] Updated weights for policy 0, policy_version 69300 (0.0009) [2023-10-08 10:31:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 141590528. Throughput: 0: 1845.9, 1: 1827.0. Samples: 35405196. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:31:57,016][52710] Avg episode reward: [(0, '31.130'), (1, '33.490')] [2023-10-08 10:31:57,138][53852] Updated weights for policy 0, policy_version 69310 (0.0008) [2023-10-08 10:31:58,726][53885] Updated weights for policy 1, policy_version 69002 (0.0007) [2023-10-08 10:31:59,103][53885] Updated weights for policy 1, policy_version 69012 (0.0010) [2023-10-08 10:31:59,468][53885] Updated weights for policy 1, policy_version 69022 (0.0008) [2023-10-08 10:32:00,680][53852] Updated weights for policy 0, policy_version 69320 (0.0008) [2023-10-08 10:32:01,054][53852] Updated weights for policy 0, policy_version 69330 (0.0010) [2023-10-08 10:32:01,431][53852] Updated weights for policy 0, policy_version 69340 (0.0010) [2023-10-08 10:32:02,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 141688832. Throughput: 0: 1846.1, 1: 1833.9. Samples: 35427288. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:32:02,016][52710] Avg episode reward: [(0, '30.670'), (1, '36.340')] [2023-10-08 10:32:03,261][53885] Updated weights for policy 1, policy_version 69032 (0.0008) [2023-10-08 10:32:03,636][53885] Updated weights for policy 1, policy_version 69042 (0.0009) [2023-10-08 10:32:04,000][53885] Updated weights for policy 1, policy_version 69052 (0.0011) [2023-10-08 10:32:04,948][53852] Updated weights for policy 0, policy_version 69350 (0.0009) [2023-10-08 10:32:05,313][53852] Updated weights for policy 0, policy_version 69360 (0.0010) [2023-10-08 10:32:05,690][53852] Updated weights for policy 0, policy_version 69370 (0.0009) [2023-10-08 10:32:07,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 141754368. Throughput: 0: 1847.9, 1: 1828.3. Samples: 35448896. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:32:07,016][52710] Avg episode reward: [(0, '28.870'), (1, '32.020')] [2023-10-08 10:32:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000069056_70713344.pth... [2023-10-08 10:32:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000069376_71041024.pth... [2023-10-08 10:32:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000067360_68976640.pth [2023-10-08 10:32:07,065][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000067648_69271552.pth [2023-10-08 10:32:07,647][53885] Updated weights for policy 1, policy_version 69062 (0.0009) [2023-10-08 10:32:08,023][53885] Updated weights for policy 1, policy_version 69072 (0.0010) [2023-10-08 10:32:08,388][53885] Updated weights for policy 1, policy_version 69082 (0.0008) [2023-10-08 10:32:09,257][53852] Updated weights for policy 0, policy_version 69380 (0.0009) [2023-10-08 10:32:09,619][53852] Updated weights for policy 0, policy_version 69390 (0.0007) [2023-10-08 10:32:09,991][53852] Updated weights for policy 0, policy_version 69400 (0.0008) [2023-10-08 10:32:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141819904. Throughput: 0: 1846.7, 1: 1830.1. Samples: 35460170. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:32:12,015][52710] Avg episode reward: [(0, '31.220'), (1, '37.400')] [2023-10-08 10:32:12,061][53885] Updated weights for policy 1, policy_version 69092 (0.0007) [2023-10-08 10:32:12,432][53885] Updated weights for policy 1, policy_version 69102 (0.0010) [2023-10-08 10:32:12,799][53885] Updated weights for policy 1, policy_version 69112 (0.0009) [2023-10-08 10:32:13,654][53852] Updated weights for policy 0, policy_version 69410 (0.0007) [2023-10-08 10:32:14,029][53852] Updated weights for policy 0, policy_version 69420 (0.0009) [2023-10-08 10:32:14,401][53852] Updated weights for policy 0, policy_version 69430 (0.0007) [2023-10-08 10:32:14,777][53852] Updated weights for policy 0, policy_version 69440 (0.0008) [2023-10-08 10:32:16,630][53885] Updated weights for policy 1, policy_version 69122 (0.0009) [2023-10-08 10:32:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141885440. Throughput: 0: 1854.2, 1: 1830.7. Samples: 35482382. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:32:17,016][52710] Avg episode reward: [(0, '33.650'), (1, '36.470')] [2023-10-08 10:32:17,039][53885] Updated weights for policy 1, policy_version 69132 (0.0010) [2023-10-08 10:32:17,411][53885] Updated weights for policy 1, policy_version 69142 (0.0010) [2023-10-08 10:32:17,772][53885] Updated weights for policy 1, policy_version 69152 (0.0011) [2023-10-08 10:32:18,229][53852] Updated weights for policy 0, policy_version 69450 (0.0010) [2023-10-08 10:32:18,602][53852] Updated weights for policy 0, policy_version 69460 (0.0009) [2023-10-08 10:32:18,975][53852] Updated weights for policy 0, policy_version 69470 (0.0009) [2023-10-08 10:32:21,512][53885] Updated weights for policy 1, policy_version 69162 (0.0008) [2023-10-08 10:32:21,879][53885] Updated weights for policy 1, policy_version 69172 (0.0008) [2023-10-08 10:32:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 141950976. Throughput: 0: 1856.0, 1: 1819.8. Samples: 35504602. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:32:22,016][52710] Avg episode reward: [(0, '30.380'), (1, '31.630')] [2023-10-08 10:32:22,252][53885] Updated weights for policy 1, policy_version 69182 (0.0007) [2023-10-08 10:32:22,755][53852] Updated weights for policy 0, policy_version 69480 (0.0008) [2023-10-08 10:32:23,121][53852] Updated weights for policy 0, policy_version 69490 (0.0009) [2023-10-08 10:32:23,501][53852] Updated weights for policy 0, policy_version 69500 (0.0009) [2023-10-08 10:32:25,760][53885] Updated weights for policy 1, policy_version 69192 (0.0009) [2023-10-08 10:32:26,125][53885] Updated weights for policy 1, policy_version 69202 (0.0008) [2023-10-08 10:32:26,492][53885] Updated weights for policy 1, policy_version 69212 (0.0007) [2023-10-08 10:32:26,955][53852] Updated weights for policy 0, policy_version 69510 (0.0009) [2023-10-08 10:32:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 142049280. Throughput: 0: 1852.5, 1: 1825.9. Samples: 35515298. Policy #0 lag: (min: 24.0, avg: 48.9, max: 56.0) [2023-10-08 10:32:27,016][52710] Avg episode reward: [(0, '30.200'), (1, '34.400')] [2023-10-08 10:32:27,334][53852] Updated weights for policy 0, policy_version 69520 (0.0008) [2023-10-08 10:32:27,710][53852] Updated weights for policy 0, policy_version 69530 (0.0008) [2023-10-08 10:32:30,182][53885] Updated weights for policy 1, policy_version 69222 (0.0008) [2023-10-08 10:32:30,547][53885] Updated weights for policy 1, policy_version 69232 (0.0007) [2023-10-08 10:32:30,914][53885] Updated weights for policy 1, policy_version 69242 (0.0008) [2023-10-08 10:32:31,538][53852] Updated weights for policy 0, policy_version 69540 (0.0009) [2023-10-08 10:32:31,896][53852] Updated weights for policy 0, policy_version 69550 (0.0008) [2023-10-08 10:32:32,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 142114816. Throughput: 0: 1855.0, 1: 1826.6. Samples: 35537900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:32:32,015][52710] Avg episode reward: [(0, '31.660'), (1, '31.730')] [2023-10-08 10:32:32,265][53852] Updated weights for policy 0, policy_version 69560 (0.0010) [2023-10-08 10:32:34,593][53885] Updated weights for policy 1, policy_version 69252 (0.0008) [2023-10-08 10:32:34,961][53885] Updated weights for policy 1, policy_version 69262 (0.0007) [2023-10-08 10:32:35,324][53885] Updated weights for policy 1, policy_version 69272 (0.0009) [2023-10-08 10:32:36,069][53852] Updated weights for policy 0, policy_version 69570 (0.0009) [2023-10-08 10:32:36,478][53852] Updated weights for policy 0, policy_version 69580 (0.0007) [2023-10-08 10:32:36,858][53852] Updated weights for policy 0, policy_version 69590 (0.0007) [2023-10-08 10:32:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142180352. Throughput: 0: 1834.8, 1: 1836.3. Samples: 35559400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:32:37,016][52710] Avg episode reward: [(0, '32.870'), (1, '30.740')] [2023-10-08 10:32:37,218][53852] Updated weights for policy 0, policy_version 69600 (0.0007) [2023-10-08 10:32:38,947][53885] Updated weights for policy 1, policy_version 69282 (0.0008) [2023-10-08 10:32:39,319][53885] Updated weights for policy 1, policy_version 69292 (0.0009) [2023-10-08 10:32:39,682][53885] Updated weights for policy 1, policy_version 69302 (0.0008) [2023-10-08 10:32:40,048][53885] Updated weights for policy 1, policy_version 69312 (0.0011) [2023-10-08 10:32:40,780][53852] Updated weights for policy 0, policy_version 69610 (0.0008) [2023-10-08 10:32:41,145][53852] Updated weights for policy 0, policy_version 69620 (0.0010) [2023-10-08 10:32:41,518][53852] Updated weights for policy 0, policy_version 69630 (0.0010) [2023-10-08 10:32:42,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 142278656. Throughput: 0: 1850.5, 1: 1823.6. Samples: 35570532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:32:42,016][52710] Avg episode reward: [(0, '31.980'), (1, '31.980')] [2023-10-08 10:32:43,645][53885] Updated weights for policy 1, policy_version 69322 (0.0009) [2023-10-08 10:32:44,014][53885] Updated weights for policy 1, policy_version 69332 (0.0009) [2023-10-08 10:32:44,383][53885] Updated weights for policy 1, policy_version 69342 (0.0008) [2023-10-08 10:32:45,131][53852] Updated weights for policy 0, policy_version 69640 (0.0008) [2023-10-08 10:32:45,496][53852] Updated weights for policy 0, policy_version 69650 (0.0009) [2023-10-08 10:32:45,864][53852] Updated weights for policy 0, policy_version 69660 (0.0011) [2023-10-08 10:32:47,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142344192. Throughput: 0: 1830.5, 1: 1828.3. Samples: 35591938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:32:47,016][52710] Avg episode reward: [(0, '31.180'), (1, '29.210')] [2023-10-08 10:32:48,058][53885] Updated weights for policy 1, policy_version 69352 (0.0008) [2023-10-08 10:32:48,414][53885] Updated weights for policy 1, policy_version 69362 (0.0008) [2023-10-08 10:32:48,788][53885] Updated weights for policy 1, policy_version 69372 (0.0008) [2023-10-08 10:32:49,545][53852] Updated weights for policy 0, policy_version 69670 (0.0009) [2023-10-08 10:32:49,911][53852] Updated weights for policy 0, policy_version 69680 (0.0008) [2023-10-08 10:32:50,294][53852] Updated weights for policy 0, policy_version 69690 (0.0008) [2023-10-08 10:32:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 142409728. Throughput: 0: 1839.7, 1: 1836.0. Samples: 35614306. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:32:52,015][52710] Avg episode reward: [(0, '30.510'), (1, '29.020')] [2023-10-08 10:32:52,335][53885] Updated weights for policy 1, policy_version 69382 (0.0008) [2023-10-08 10:32:52,709][53885] Updated weights for policy 1, policy_version 69392 (0.0011) [2023-10-08 10:32:53,077][53885] Updated weights for policy 1, policy_version 69402 (0.0010) [2023-10-08 10:32:54,077][53852] Updated weights for policy 0, policy_version 69700 (0.0009) [2023-10-08 10:32:54,437][53852] Updated weights for policy 0, policy_version 69710 (0.0010) [2023-10-08 10:32:54,804][53852] Updated weights for policy 0, policy_version 69720 (0.0011) [2023-10-08 10:32:56,704][53885] Updated weights for policy 1, policy_version 69412 (0.0008) [2023-10-08 10:32:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142475264. Throughput: 0: 1827.4, 1: 1837.3. Samples: 35625080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:32:57,016][52710] Avg episode reward: [(0, '34.050'), (1, '33.720')] [2023-10-08 10:32:57,072][53885] Updated weights for policy 1, policy_version 69422 (0.0009) [2023-10-08 10:32:57,436][53885] Updated weights for policy 1, policy_version 69432 (0.0008) [2023-10-08 10:32:58,516][53852] Updated weights for policy 0, policy_version 69730 (0.0007) [2023-10-08 10:32:58,884][53852] Updated weights for policy 0, policy_version 69740 (0.0008) [2023-10-08 10:32:59,255][53852] Updated weights for policy 0, policy_version 69750 (0.0007) [2023-10-08 10:32:59,628][53852] Updated weights for policy 0, policy_version 69760 (0.0010) [2023-10-08 10:33:01,182][53885] Updated weights for policy 1, policy_version 69442 (0.0008) [2023-10-08 10:33:01,549][53885] Updated weights for policy 1, policy_version 69452 (0.0009) [2023-10-08 10:33:01,921][53885] Updated weights for policy 1, policy_version 69462 (0.0008) [2023-10-08 10:33:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 142540800. Throughput: 0: 1831.5, 1: 1831.8. Samples: 35647230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:33:02,016][52710] Avg episode reward: [(0, '29.640'), (1, '33.690')] [2023-10-08 10:33:02,290][53885] Updated weights for policy 1, policy_version 69472 (0.0008) [2023-10-08 10:33:03,300][53852] Updated weights for policy 0, policy_version 69770 (0.0008) [2023-10-08 10:33:03,662][53852] Updated weights for policy 0, policy_version 69780 (0.0008) [2023-10-08 10:33:04,028][53852] Updated weights for policy 0, policy_version 69790 (0.0008) [2023-10-08 10:33:06,102][53885] Updated weights for policy 1, policy_version 69482 (0.0009) [2023-10-08 10:33:06,464][53885] Updated weights for policy 1, policy_version 69492 (0.0008) [2023-10-08 10:33:06,833][53885] Updated weights for policy 1, policy_version 69502 (0.0007) [2023-10-08 10:33:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142639104. Throughput: 0: 1831.6, 1: 1829.3. Samples: 35669342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:33:07,016][52710] Avg episode reward: [(0, '30.920'), (1, '32.590')] [2023-10-08 10:33:07,690][53852] Updated weights for policy 0, policy_version 69800 (0.0009) [2023-10-08 10:33:08,065][53852] Updated weights for policy 0, policy_version 69810 (0.0008) [2023-10-08 10:33:08,433][53852] Updated weights for policy 0, policy_version 69820 (0.0009) [2023-10-08 10:33:10,525][53885] Updated weights for policy 1, policy_version 69512 (0.0007) [2023-10-08 10:33:10,897][53885] Updated weights for policy 1, policy_version 69522 (0.0009) [2023-10-08 10:33:11,262][53885] Updated weights for policy 1, policy_version 69532 (0.0009) [2023-10-08 10:33:11,997][53852] Updated weights for policy 0, policy_version 69830 (0.0008) [2023-10-08 10:33:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142704640. Throughput: 0: 1834.6, 1: 1838.6. Samples: 35680592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:33:12,015][52710] Avg episode reward: [(0, '34.960'), (1, '35.950')] [2023-10-08 10:33:12,364][53852] Updated weights for policy 0, policy_version 69840 (0.0010) [2023-10-08 10:33:12,738][53852] Updated weights for policy 0, policy_version 69850 (0.0010) [2023-10-08 10:33:15,090][53885] Updated weights for policy 1, policy_version 69542 (0.0009) [2023-10-08 10:33:15,450][53885] Updated weights for policy 1, policy_version 69552 (0.0008) [2023-10-08 10:33:15,812][53885] Updated weights for policy 1, policy_version 69562 (0.0011) [2023-10-08 10:33:16,265][53852] Updated weights for policy 0, policy_version 69860 (0.0008) [2023-10-08 10:33:16,628][53852] Updated weights for policy 0, policy_version 69870 (0.0007) [2023-10-08 10:33:17,005][53852] Updated weights for policy 0, policy_version 69880 (0.0008) [2023-10-08 10:33:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 142770176. Throughput: 0: 1835.3, 1: 1832.4. Samples: 35702948. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:17,016][52710] Avg episode reward: [(0, '33.830'), (1, '32.820')] [2023-10-08 10:33:19,421][53885] Updated weights for policy 1, policy_version 69572 (0.0010) [2023-10-08 10:33:19,786][53885] Updated weights for policy 1, policy_version 69582 (0.0007) [2023-10-08 10:33:20,157][53885] Updated weights for policy 1, policy_version 69592 (0.0009) [2023-10-08 10:33:20,699][53852] Updated weights for policy 0, policy_version 69890 (0.0008) [2023-10-08 10:33:21,068][53852] Updated weights for policy 0, policy_version 69900 (0.0009) [2023-10-08 10:33:21,435][53852] Updated weights for policy 0, policy_version 69910 (0.0011) [2023-10-08 10:33:21,807][53852] Updated weights for policy 0, policy_version 69920 (0.0008) [2023-10-08 10:33:22,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 142868480. Throughput: 0: 1830.8, 1: 1835.1. Samples: 35724364. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:22,016][52710] Avg episode reward: [(0, '30.240'), (1, '33.080')] [2023-10-08 10:33:23,726][53885] Updated weights for policy 1, policy_version 69602 (0.0010) [2023-10-08 10:33:24,091][53885] Updated weights for policy 1, policy_version 69612 (0.0007) [2023-10-08 10:33:24,460][53885] Updated weights for policy 1, policy_version 69622 (0.0007) [2023-10-08 10:33:24,823][53885] Updated weights for policy 1, policy_version 69632 (0.0007) [2023-10-08 10:33:25,488][53852] Updated weights for policy 0, policy_version 69930 (0.0010) [2023-10-08 10:33:25,850][53852] Updated weights for policy 0, policy_version 69940 (0.0007) [2023-10-08 10:33:26,222][53852] Updated weights for policy 0, policy_version 69950 (0.0011) [2023-10-08 10:33:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 142934016. Throughput: 0: 1845.9, 1: 1830.4. Samples: 35735966. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:27,016][52710] Avg episode reward: [(0, '33.940'), (1, '34.610')] [2023-10-08 10:33:28,653][53885] Updated weights for policy 1, policy_version 69642 (0.0009) [2023-10-08 10:33:29,023][53885] Updated weights for policy 1, policy_version 69652 (0.0009) [2023-10-08 10:33:29,390][53885] Updated weights for policy 1, policy_version 69662 (0.0009) [2023-10-08 10:33:29,722][53852] Updated weights for policy 0, policy_version 69960 (0.0011) [2023-10-08 10:33:30,092][53852] Updated weights for policy 0, policy_version 69970 (0.0008) [2023-10-08 10:33:30,467][53852] Updated weights for policy 0, policy_version 69980 (0.0010) [2023-10-08 10:33:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142999552. Throughput: 0: 1836.7, 1: 1836.3. Samples: 35757222. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:32,015][52710] Avg episode reward: [(0, '33.860'), (1, '30.750')] [2023-10-08 10:33:33,019][53885] Updated weights for policy 1, policy_version 69672 (0.0008) [2023-10-08 10:33:33,386][53885] Updated weights for policy 1, policy_version 69682 (0.0008) [2023-10-08 10:33:33,762][53885] Updated weights for policy 1, policy_version 69692 (0.0008) [2023-10-08 10:33:34,096][53852] Updated weights for policy 0, policy_version 69990 (0.0011) [2023-10-08 10:33:34,469][53852] Updated weights for policy 0, policy_version 70000 (0.0011) [2023-10-08 10:33:34,837][53852] Updated weights for policy 0, policy_version 70010 (0.0009) [2023-10-08 10:33:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143065088. Throughput: 0: 1855.7, 1: 1829.3. Samples: 35780132. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:37,016][52710] Avg episode reward: [(0, '30.050'), (1, '32.160')] [2023-10-08 10:33:37,347][53885] Updated weights for policy 1, policy_version 69702 (0.0008) [2023-10-08 10:33:37,720][53885] Updated weights for policy 1, policy_version 69712 (0.0008) [2023-10-08 10:33:38,082][53885] Updated weights for policy 1, policy_version 69722 (0.0008) [2023-10-08 10:33:38,432][53852] Updated weights for policy 0, policy_version 70020 (0.0008) [2023-10-08 10:33:38,803][53852] Updated weights for policy 0, policy_version 70030 (0.0009) [2023-10-08 10:33:39,176][53852] Updated weights for policy 0, policy_version 70040 (0.0008) [2023-10-08 10:33:41,906][53885] Updated weights for policy 1, policy_version 69732 (0.0008) [2023-10-08 10:33:42,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 143130624. Throughput: 0: 1838.6, 1: 1830.3. Samples: 35790178. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:42,016][52710] Avg episode reward: [(0, '31.760'), (1, '32.770')] [2023-10-08 10:33:42,279][53885] Updated weights for policy 1, policy_version 69742 (0.0010) [2023-10-08 10:33:42,646][53885] Updated weights for policy 1, policy_version 69752 (0.0009) [2023-10-08 10:33:42,767][53852] Updated weights for policy 0, policy_version 70050 (0.0007) [2023-10-08 10:33:43,133][53852] Updated weights for policy 0, policy_version 70060 (0.0008) [2023-10-08 10:33:43,498][53852] Updated weights for policy 0, policy_version 70070 (0.0009) [2023-10-08 10:33:43,864][53852] Updated weights for policy 0, policy_version 70080 (0.0012) [2023-10-08 10:33:46,296][53885] Updated weights for policy 1, policy_version 69762 (0.0008) [2023-10-08 10:33:46,662][53885] Updated weights for policy 1, policy_version 69772 (0.0008) [2023-10-08 10:33:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 143196160. Throughput: 0: 1856.4, 1: 1829.2. Samples: 35813084. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:47,016][52710] Avg episode reward: [(0, '32.710'), (1, '29.200')] [2023-10-08 10:33:47,035][53885] Updated weights for policy 1, policy_version 69782 (0.0010) [2023-10-08 10:33:47,400][53885] Updated weights for policy 1, policy_version 69792 (0.0009) [2023-10-08 10:33:47,519][53852] Updated weights for policy 0, policy_version 70090 (0.0009) [2023-10-08 10:33:47,873][53852] Updated weights for policy 0, policy_version 70100 (0.0010) [2023-10-08 10:33:48,241][53852] Updated weights for policy 0, policy_version 70110 (0.0010) [2023-10-08 10:33:51,270][53885] Updated weights for policy 1, policy_version 69802 (0.0008) [2023-10-08 10:33:51,634][53885] Updated weights for policy 1, policy_version 69812 (0.0008) [2023-10-08 10:33:51,787][53852] Updated weights for policy 0, policy_version 70120 (0.0008) [2023-10-08 10:33:52,001][53885] Updated weights for policy 1, policy_version 69822 (0.0008) [2023-10-08 10:33:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 143261696. Throughput: 0: 1857.2, 1: 1826.0. Samples: 35835086. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:52,016][52710] Avg episode reward: [(0, '29.210'), (1, '28.520')] [2023-10-08 10:33:52,152][53852] Updated weights for policy 0, policy_version 70130 (0.0008) [2023-10-08 10:33:52,529][53852] Updated weights for policy 0, policy_version 70140 (0.0010) [2023-10-08 10:33:55,675][53885] Updated weights for policy 1, policy_version 69832 (0.0008) [2023-10-08 10:33:56,039][53885] Updated weights for policy 1, policy_version 69842 (0.0009) [2023-10-08 10:33:56,082][53852] Updated weights for policy 0, policy_version 70150 (0.0008) [2023-10-08 10:33:56,399][53885] Updated weights for policy 1, policy_version 69852 (0.0009) [2023-10-08 10:33:56,444][53852] Updated weights for policy 0, policy_version 70160 (0.0007) [2023-10-08 10:33:56,821][53852] Updated weights for policy 0, policy_version 70170 (0.0009) [2023-10-08 10:33:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143360000. Throughput: 0: 1855.6, 1: 1820.7. Samples: 35846026. Policy #0 lag: (min: 31.0, avg: 32.3, max: 55.0) [2023-10-08 10:33:57,016][52710] Avg episode reward: [(0, '29.700'), (1, '32.440')] [2023-10-08 10:33:59,887][53885] Updated weights for policy 1, policy_version 69862 (0.0009) [2023-10-08 10:34:00,251][53885] Updated weights for policy 1, policy_version 69872 (0.0008) [2023-10-08 10:34:00,516][53852] Updated weights for policy 0, policy_version 70180 (0.0007) [2023-10-08 10:34:00,611][53885] Updated weights for policy 1, policy_version 69882 (0.0009) [2023-10-08 10:34:00,886][53852] Updated weights for policy 0, policy_version 70190 (0.0008) [2023-10-08 10:34:01,251][53852] Updated weights for policy 0, policy_version 70200 (0.0008) [2023-10-08 10:34:02,015][52710] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 143458304. Throughput: 0: 1844.5, 1: 1817.9. Samples: 35867756. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-08 10:34:02,016][52710] Avg episode reward: [(0, '31.940'), (1, '32.670')] [2023-10-08 10:34:04,276][53885] Updated weights for policy 1, policy_version 69892 (0.0009) [2023-10-08 10:34:04,645][53885] Updated weights for policy 1, policy_version 69902 (0.0009) [2023-10-08 10:34:04,973][53852] Updated weights for policy 0, policy_version 70210 (0.0007) [2023-10-08 10:34:05,020][53885] Updated weights for policy 1, policy_version 69912 (0.0008) [2023-10-08 10:34:05,334][53852] Updated weights for policy 0, policy_version 70220 (0.0007) [2023-10-08 10:34:05,708][53852] Updated weights for policy 0, policy_version 70230 (0.0010) [2023-10-08 10:34:06,073][53852] Updated weights for policy 0, policy_version 70240 (0.0009) [2023-10-08 10:34:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 143523840. Throughput: 0: 1836.9, 1: 1827.0. Samples: 35889240. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-08 10:34:07,017][52710] Avg episode reward: [(0, '33.990'), (1, '31.750')] [2023-10-08 10:34:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000070240_71925760.pth... [2023-10-08 10:34:07,026][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000069920_71598080.pth... [2023-10-08 10:34:07,055][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000068512_70156288.pth [2023-10-08 10:34:07,065][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000068224_69861376.pth [2023-10-08 10:34:08,706][53885] Updated weights for policy 1, policy_version 69922 (0.0008) [2023-10-08 10:34:09,073][53885] Updated weights for policy 1, policy_version 69932 (0.0008) [2023-10-08 10:34:09,450][53885] Updated weights for policy 1, policy_version 69942 (0.0007) [2023-10-08 10:34:09,778][53852] Updated weights for policy 0, policy_version 70250 (0.0009) [2023-10-08 10:34:09,809][53885] Updated weights for policy 1, policy_version 69952 (0.0008) [2023-10-08 10:34:10,149][53852] Updated weights for policy 0, policy_version 70260 (0.0007) [2023-10-08 10:34:10,514][53852] Updated weights for policy 0, policy_version 70270 (0.0008) [2023-10-08 10:34:12,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 143589376. Throughput: 0: 1840.5, 1: 1826.9. Samples: 35901002. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-08 10:34:12,016][52710] Avg episode reward: [(0, '30.200'), (1, '35.720')] [2023-10-08 10:34:13,454][53885] Updated weights for policy 1, policy_version 69962 (0.0010) [2023-10-08 10:34:13,815][53885] Updated weights for policy 1, policy_version 69972 (0.0007) [2023-10-08 10:34:14,149][53852] Updated weights for policy 0, policy_version 70280 (0.0007) [2023-10-08 10:34:14,183][53885] Updated weights for policy 1, policy_version 69982 (0.0009) [2023-10-08 10:34:14,519][53852] Updated weights for policy 0, policy_version 70290 (0.0007) [2023-10-08 10:34:14,882][53852] Updated weights for policy 0, policy_version 70300 (0.0007) [2023-10-08 10:34:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 143654912. Throughput: 0: 1840.2, 1: 1827.1. Samples: 35922252. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-08 10:34:17,016][52710] Avg episode reward: [(0, '30.870'), (1, '30.190')] [2023-10-08 10:34:17,773][53885] Updated weights for policy 1, policy_version 69992 (0.0008) [2023-10-08 10:34:18,142][53885] Updated weights for policy 1, policy_version 70002 (0.0008) [2023-10-08 10:34:18,463][53852] Updated weights for policy 0, policy_version 70310 (0.0009) [2023-10-08 10:34:18,504][53885] Updated weights for policy 1, policy_version 70012 (0.0008) [2023-10-08 10:34:18,845][53852] Updated weights for policy 0, policy_version 70320 (0.0007) [2023-10-08 10:34:19,216][53852] Updated weights for policy 0, policy_version 70330 (0.0007) [2023-10-08 10:34:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 143720448. Throughput: 0: 1840.3, 1: 1829.4. Samples: 35945266. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-08 10:34:22,016][52710] Avg episode reward: [(0, '29.920'), (1, '33.040')] [2023-10-08 10:34:22,185][53885] Updated weights for policy 1, policy_version 70022 (0.0008) [2023-10-08 10:34:22,561][53885] Updated weights for policy 1, policy_version 70032 (0.0007) [2023-10-08 10:34:22,882][53852] Updated weights for policy 0, policy_version 70340 (0.0008) [2023-10-08 10:34:22,930][53885] Updated weights for policy 1, policy_version 70042 (0.0009) [2023-10-08 10:34:23,244][53852] Updated weights for policy 0, policy_version 70350 (0.0008) [2023-10-08 10:34:23,623][53852] Updated weights for policy 0, policy_version 70360 (0.0008) [2023-10-08 10:34:26,729][53885] Updated weights for policy 1, policy_version 70052 (0.0007) [2023-10-08 10:34:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 143785984. Throughput: 0: 1840.8, 1: 1825.6. Samples: 35955164. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-08 10:34:27,016][52710] Avg episode reward: [(0, '30.500'), (1, '37.670')] [2023-10-08 10:34:27,090][53885] Updated weights for policy 1, policy_version 70062 (0.0009) [2023-10-08 10:34:27,390][53852] Updated weights for policy 0, policy_version 70370 (0.0008) [2023-10-08 10:34:27,463][53885] Updated weights for policy 1, policy_version 70072 (0.0008) [2023-10-08 10:34:27,767][53852] Updated weights for policy 0, policy_version 70380 (0.0007) [2023-10-08 10:34:28,134][53852] Updated weights for policy 0, policy_version 70390 (0.0009) [2023-10-08 10:34:28,503][53852] Updated weights for policy 0, policy_version 70400 (0.0010) [2023-10-08 10:34:31,171][53885] Updated weights for policy 1, policy_version 70082 (0.0009) [2023-10-08 10:34:31,534][53885] Updated weights for policy 1, policy_version 70092 (0.0009) [2023-10-08 10:34:31,912][53885] Updated weights for policy 1, policy_version 70102 (0.0008) [2023-10-08 10:34:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 143851520. Throughput: 0: 1844.4, 1: 1824.4. Samples: 35978180. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-08 10:34:32,016][52710] Avg episode reward: [(0, '29.120'), (1, '33.890')] [2023-10-08 10:34:32,136][53852] Updated weights for policy 0, policy_version 70410 (0.0010) [2023-10-08 10:34:32,285][53885] Updated weights for policy 1, policy_version 70112 (0.0009) [2023-10-08 10:34:32,504][53852] Updated weights for policy 0, policy_version 70420 (0.0007) [2023-10-08 10:34:32,866][53852] Updated weights for policy 0, policy_version 70430 (0.0007) [2023-10-08 10:34:35,991][53885] Updated weights for policy 1, policy_version 70122 (0.0011) [2023-10-08 10:34:36,363][53885] Updated weights for policy 1, policy_version 70132 (0.0007) [2023-10-08 10:34:36,528][53852] Updated weights for policy 0, policy_version 70440 (0.0008) [2023-10-08 10:34:36,721][53885] Updated weights for policy 1, policy_version 70142 (0.0008) [2023-10-08 10:34:36,896][53852] Updated weights for policy 0, policy_version 70450 (0.0009) [2023-10-08 10:34:37,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 143949824. Throughput: 0: 1832.3, 1: 1817.6. Samples: 35999330. Policy #0 lag: (min: 16.0, avg: 43.4, max: 48.0) [2023-10-08 10:34:37,015][52710] Avg episode reward: [(0, '32.020'), (1, '31.140')] [2023-10-08 10:34:37,265][53852] Updated weights for policy 0, policy_version 70460 (0.0008) [2023-10-08 10:34:40,389][53885] Updated weights for policy 1, policy_version 70152 (0.0009) [2023-10-08 10:34:40,752][53885] Updated weights for policy 1, policy_version 70162 (0.0007) [2023-10-08 10:34:40,902][53852] Updated weights for policy 0, policy_version 70470 (0.0008) [2023-10-08 10:34:41,116][53885] Updated weights for policy 1, policy_version 70172 (0.0008) [2023-10-08 10:34:41,269][53852] Updated weights for policy 0, policy_version 70480 (0.0007) [2023-10-08 10:34:41,637][53852] Updated weights for policy 0, policy_version 70490 (0.0008) [2023-10-08 10:34:42,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 144048128. Throughput: 0: 1843.2, 1: 1824.6. Samples: 36011080. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:34:42,016][52710] Avg episode reward: [(0, '31.970'), (1, '33.820')] [2023-10-08 10:34:44,759][53885] Updated weights for policy 1, policy_version 70182 (0.0008) [2023-10-08 10:34:45,134][53885] Updated weights for policy 1, policy_version 70192 (0.0008) [2023-10-08 10:34:45,197][53852] Updated weights for policy 0, policy_version 70500 (0.0009) [2023-10-08 10:34:45,500][53885] Updated weights for policy 1, policy_version 70202 (0.0008) [2023-10-08 10:34:45,565][53852] Updated weights for policy 0, policy_version 70510 (0.0008) [2023-10-08 10:34:45,933][53852] Updated weights for policy 0, policy_version 70520 (0.0008) [2023-10-08 10:34:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 144113664. Throughput: 0: 1837.9, 1: 1820.8. Samples: 36032400. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:34:47,016][52710] Avg episode reward: [(0, '31.710'), (1, '31.660')] [2023-10-08 10:34:48,978][53885] Updated weights for policy 1, policy_version 70212 (0.0010) [2023-10-08 10:34:49,349][53885] Updated weights for policy 1, policy_version 70222 (0.0008) [2023-10-08 10:34:49,526][53852] Updated weights for policy 0, policy_version 70530 (0.0007) [2023-10-08 10:34:49,721][53885] Updated weights for policy 1, policy_version 70232 (0.0008) [2023-10-08 10:34:49,899][53852] Updated weights for policy 0, policy_version 70540 (0.0009) [2023-10-08 10:34:50,273][53852] Updated weights for policy 0, policy_version 70550 (0.0009) [2023-10-08 10:34:50,645][53852] Updated weights for policy 0, policy_version 70560 (0.0009) [2023-10-08 10:34:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 144179200. Throughput: 0: 1849.1, 1: 1822.5. Samples: 36054464. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:34:52,016][52710] Avg episode reward: [(0, '32.110'), (1, '31.410')] [2023-10-08 10:34:53,397][53885] Updated weights for policy 1, policy_version 70242 (0.0009) [2023-10-08 10:34:53,758][53885] Updated weights for policy 1, policy_version 70252 (0.0009) [2023-10-08 10:34:54,124][53885] Updated weights for policy 1, policy_version 70262 (0.0008) [2023-10-08 10:34:54,249][53852] Updated weights for policy 0, policy_version 70570 (0.0009) [2023-10-08 10:34:54,489][53885] Updated weights for policy 1, policy_version 70272 (0.0007) [2023-10-08 10:34:54,622][53852] Updated weights for policy 0, policy_version 70580 (0.0008) [2023-10-08 10:34:54,989][53852] Updated weights for policy 0, policy_version 70590 (0.0008) [2023-10-08 10:34:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144244736. Throughput: 0: 1831.5, 1: 1814.3. Samples: 36065062. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:34:57,016][52710] Avg episode reward: [(0, '32.320'), (1, '33.720')] [2023-10-08 10:34:58,171][53885] Updated weights for policy 1, policy_version 70282 (0.0008) [2023-10-08 10:34:58,535][53885] Updated weights for policy 1, policy_version 70292 (0.0007) [2023-10-08 10:34:58,645][53852] Updated weights for policy 0, policy_version 70600 (0.0009) [2023-10-08 10:34:58,901][53885] Updated weights for policy 1, policy_version 70302 (0.0008) [2023-10-08 10:34:59,020][53852] Updated weights for policy 0, policy_version 70610 (0.0008) [2023-10-08 10:34:59,398][53852] Updated weights for policy 0, policy_version 70620 (0.0008) [2023-10-08 10:35:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 144310272. Throughput: 0: 1844.5, 1: 1823.0. Samples: 36087292. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:35:02,016][52710] Avg episode reward: [(0, '34.280'), (1, '36.210')] [2023-10-08 10:35:02,497][53885] Updated weights for policy 1, policy_version 70312 (0.0009) [2023-10-08 10:35:02,858][53885] Updated weights for policy 1, policy_version 70322 (0.0007) [2023-10-08 10:35:03,227][53885] Updated weights for policy 1, policy_version 70332 (0.0007) [2023-10-08 10:35:03,232][53852] Updated weights for policy 0, policy_version 70630 (0.0008) [2023-10-08 10:35:03,623][53852] Updated weights for policy 0, policy_version 70640 (0.0007) [2023-10-08 10:35:03,987][53852] Updated weights for policy 0, policy_version 70650 (0.0008) [2023-10-08 10:35:06,906][53885] Updated weights for policy 1, policy_version 70342 (0.0010) [2023-10-08 10:35:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 144375808. Throughput: 0: 1836.8, 1: 1825.6. Samples: 36110072. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:35:07,016][52710] Avg episode reward: [(0, '33.240'), (1, '33.870')] [2023-10-08 10:35:07,269][53885] Updated weights for policy 1, policy_version 70352 (0.0007) [2023-10-08 10:35:07,574][53852] Updated weights for policy 0, policy_version 70660 (0.0008) [2023-10-08 10:35:07,645][53885] Updated weights for policy 1, policy_version 70362 (0.0008) [2023-10-08 10:35:07,934][53852] Updated weights for policy 0, policy_version 70670 (0.0008) [2023-10-08 10:35:08,312][53852] Updated weights for policy 0, policy_version 70680 (0.0009) [2023-10-08 10:35:11,487][53885] Updated weights for policy 1, policy_version 70372 (0.0007) [2023-10-08 10:35:11,858][53885] Updated weights for policy 1, policy_version 70382 (0.0008) [2023-10-08 10:35:11,927][53852] Updated weights for policy 0, policy_version 70690 (0.0007) [2023-10-08 10:35:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144441344. Throughput: 0: 1836.9, 1: 1822.1. Samples: 36119818. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:35:12,015][52710] Avg episode reward: [(0, '33.500'), (1, '36.210')] [2023-10-08 10:35:12,223][53885] Updated weights for policy 1, policy_version 70392 (0.0008) [2023-10-08 10:35:12,285][53852] Updated weights for policy 0, policy_version 70700 (0.0008) [2023-10-08 10:35:12,650][53852] Updated weights for policy 0, policy_version 70710 (0.0007) [2023-10-08 10:35:13,021][53852] Updated weights for policy 0, policy_version 70720 (0.0007) [2023-10-08 10:35:15,796][53885] Updated weights for policy 1, policy_version 70402 (0.0008) [2023-10-08 10:35:16,159][53885] Updated weights for policy 1, policy_version 70412 (0.0009) [2023-10-08 10:35:16,509][53852] Updated weights for policy 0, policy_version 70730 (0.0007) [2023-10-08 10:35:16,523][53885] Updated weights for policy 1, policy_version 70422 (0.0007) [2023-10-08 10:35:16,889][53885] Updated weights for policy 1, policy_version 70432 (0.0007) [2023-10-08 10:35:16,890][53852] Updated weights for policy 0, policy_version 70740 (0.0008) [2023-10-08 10:35:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144539648. Throughput: 0: 1836.6, 1: 1825.9. Samples: 36142992. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:35:17,016][52710] Avg episode reward: [(0, '33.560'), (1, '31.130')] [2023-10-08 10:35:17,253][53852] Updated weights for policy 0, policy_version 70750 (0.0010) [2023-10-08 10:35:20,615][53885] Updated weights for policy 1, policy_version 70442 (0.0009) [2023-10-08 10:35:20,914][53852] Updated weights for policy 0, policy_version 70760 (0.0008) [2023-10-08 10:35:20,976][53885] Updated weights for policy 1, policy_version 70452 (0.0007) [2023-10-08 10:35:21,285][53852] Updated weights for policy 0, policy_version 70770 (0.0010) [2023-10-08 10:35:21,346][53885] Updated weights for policy 1, policy_version 70462 (0.0008) [2023-10-08 10:35:21,659][53852] Updated weights for policy 0, policy_version 70780 (0.0010) [2023-10-08 10:35:22,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 144637952. Throughput: 0: 1821.7, 1: 1823.5. Samples: 36163364. Policy #0 lag: (min: 31.0, avg: 34.5, max: 60.0) [2023-10-08 10:35:22,016][52710] Avg episode reward: [(0, '34.080'), (1, '31.980')] [2023-10-08 10:35:25,097][53885] Updated weights for policy 1, policy_version 70472 (0.0007) [2023-10-08 10:35:25,287][53852] Updated weights for policy 0, policy_version 70790 (0.0009) [2023-10-08 10:35:25,452][53885] Updated weights for policy 1, policy_version 70482 (0.0007) [2023-10-08 10:35:25,648][53852] Updated weights for policy 0, policy_version 70800 (0.0007) [2023-10-08 10:35:25,817][53885] Updated weights for policy 1, policy_version 70492 (0.0009) [2023-10-08 10:35:26,021][53852] Updated weights for policy 0, policy_version 70810 (0.0009) [2023-10-08 10:35:27,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 144703488. Throughput: 0: 1838.2, 1: 1829.6. Samples: 36176132. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:35:27,015][52710] Avg episode reward: [(0, '31.690'), (1, '30.450')] [2023-10-08 10:35:29,412][53885] Updated weights for policy 1, policy_version 70502 (0.0008) [2023-10-08 10:35:29,718][53852] Updated weights for policy 0, policy_version 70820 (0.0008) [2023-10-08 10:35:29,787][53885] Updated weights for policy 1, policy_version 70512 (0.0010) [2023-10-08 10:35:30,084][53852] Updated weights for policy 0, policy_version 70830 (0.0008) [2023-10-08 10:35:30,147][53885] Updated weights for policy 1, policy_version 70522 (0.0008) [2023-10-08 10:35:30,454][53852] Updated weights for policy 0, policy_version 70840 (0.0010) [2023-10-08 10:35:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 144769024. Throughput: 0: 1823.3, 1: 1820.8. Samples: 36196382. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:35:32,016][52710] Avg episode reward: [(0, '34.940'), (1, '30.500')] [2023-10-08 10:35:33,816][53885] Updated weights for policy 1, policy_version 70532 (0.0009) [2023-10-08 10:35:34,049][53852] Updated weights for policy 0, policy_version 70850 (0.0009) [2023-10-08 10:35:34,170][53885] Updated weights for policy 1, policy_version 70542 (0.0008) [2023-10-08 10:35:34,426][53852] Updated weights for policy 0, policy_version 70860 (0.0008) [2023-10-08 10:35:34,540][53885] Updated weights for policy 1, policy_version 70552 (0.0009) [2023-10-08 10:35:34,807][53852] Updated weights for policy 0, policy_version 70870 (0.0009) [2023-10-08 10:35:35,181][53852] Updated weights for policy 0, policy_version 70880 (0.0009) [2023-10-08 10:35:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144834560. Throughput: 0: 1840.9, 1: 1816.4. Samples: 36219042. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:35:37,016][52710] Avg episode reward: [(0, '35.940'), (1, '32.730')] [2023-10-08 10:35:38,322][53885] Updated weights for policy 1, policy_version 70562 (0.0009) [2023-10-08 10:35:38,692][53885] Updated weights for policy 1, policy_version 70572 (0.0009) [2023-10-08 10:35:38,778][53852] Updated weights for policy 0, policy_version 70890 (0.0007) [2023-10-08 10:35:39,059][53885] Updated weights for policy 1, policy_version 70582 (0.0007) [2023-10-08 10:35:39,142][53852] Updated weights for policy 0, policy_version 70900 (0.0009) [2023-10-08 10:35:39,427][53885] Updated weights for policy 1, policy_version 70592 (0.0007) [2023-10-08 10:35:39,517][53852] Updated weights for policy 0, policy_version 70910 (0.0008) [2023-10-08 10:35:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 144900096. Throughput: 0: 1828.9, 1: 1814.8. Samples: 36229028. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:35:42,016][52710] Avg episode reward: [(0, '35.630'), (1, '33.440')] [2023-10-08 10:35:43,151][53885] Updated weights for policy 1, policy_version 70602 (0.0007) [2023-10-08 10:35:43,330][53852] Updated weights for policy 0, policy_version 70920 (0.0007) [2023-10-08 10:35:43,518][53885] Updated weights for policy 1, policy_version 70612 (0.0007) [2023-10-08 10:35:43,702][53852] Updated weights for policy 0, policy_version 70930 (0.0007) [2023-10-08 10:35:43,879][53885] Updated weights for policy 1, policy_version 70622 (0.0008) [2023-10-08 10:35:44,060][53852] Updated weights for policy 0, policy_version 70940 (0.0007) [2023-10-08 10:35:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 144965632. Throughput: 0: 1836.8, 1: 1811.0. Samples: 36251442. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:35:47,016][52710] Avg episode reward: [(0, '31.930'), (1, '35.400')] [2023-10-08 10:35:47,675][53885] Updated weights for policy 1, policy_version 70632 (0.0010) [2023-10-08 10:35:47,766][53852] Updated weights for policy 0, policy_version 70950 (0.0008) [2023-10-08 10:35:48,038][53885] Updated weights for policy 1, policy_version 70642 (0.0010) [2023-10-08 10:35:48,142][53852] Updated weights for policy 0, policy_version 70960 (0.0007) [2023-10-08 10:35:48,403][53885] Updated weights for policy 1, policy_version 70652 (0.0007) [2023-10-08 10:35:48,512][53852] Updated weights for policy 0, policy_version 70970 (0.0008) [2023-10-08 10:35:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 145031168. Throughput: 0: 1842.5, 1: 1808.9. Samples: 36274386. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:35:52,016][52710] Avg episode reward: [(0, '31.150'), (1, '35.360')] [2023-10-08 10:35:52,083][53885] Updated weights for policy 1, policy_version 70662 (0.0008) [2023-10-08 10:35:52,326][53852] Updated weights for policy 0, policy_version 70980 (0.0009) [2023-10-08 10:35:52,440][53885] Updated weights for policy 1, policy_version 70672 (0.0007) [2023-10-08 10:35:52,703][53852] Updated weights for policy 0, policy_version 70990 (0.0007) [2023-10-08 10:35:52,802][53885] Updated weights for policy 1, policy_version 70682 (0.0008) [2023-10-08 10:35:53,079][53852] Updated weights for policy 0, policy_version 71000 (0.0008) [2023-10-08 10:35:56,618][53885] Updated weights for policy 1, policy_version 70692 (0.0008) [2023-10-08 10:35:56,768][53852] Updated weights for policy 0, policy_version 71010 (0.0009) [2023-10-08 10:35:56,982][53885] Updated weights for policy 1, policy_version 70702 (0.0007) [2023-10-08 10:35:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 145096704. Throughput: 0: 1838.3, 1: 1813.8. Samples: 36284164. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:35:57,015][52710] Avg episode reward: [(0, '35.380'), (1, '32.250')] [2023-10-08 10:35:57,145][53852] Updated weights for policy 0, policy_version 71020 (0.0009) [2023-10-08 10:35:57,343][53885] Updated weights for policy 1, policy_version 70712 (0.0008) [2023-10-08 10:35:57,509][53852] Updated weights for policy 0, policy_version 71030 (0.0007) [2023-10-08 10:35:57,872][53852] Updated weights for policy 0, policy_version 71040 (0.0007) [2023-10-08 10:36:00,936][53885] Updated weights for policy 1, policy_version 70722 (0.0009) [2023-10-08 10:36:01,302][53885] Updated weights for policy 1, policy_version 70732 (0.0008) [2023-10-08 10:36:01,464][53852] Updated weights for policy 0, policy_version 71050 (0.0009) [2023-10-08 10:36:01,667][53885] Updated weights for policy 1, policy_version 70742 (0.0008) [2023-10-08 10:36:01,836][53852] Updated weights for policy 0, policy_version 71060 (0.0008) [2023-10-08 10:36:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 145162240. Throughput: 0: 1837.1, 1: 1815.6. Samples: 36307362. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:36:02,016][52710] Avg episode reward: [(0, '32.270'), (1, '33.980')] [2023-10-08 10:36:02,028][53885] Updated weights for policy 1, policy_version 70752 (0.0007) [2023-10-08 10:36:02,211][53852] Updated weights for policy 0, policy_version 71070 (0.0008) [2023-10-08 10:36:05,574][53885] Updated weights for policy 1, policy_version 70762 (0.0008) [2023-10-08 10:36:05,710][53852] Updated weights for policy 0, policy_version 71080 (0.0007) [2023-10-08 10:36:05,950][53885] Updated weights for policy 1, policy_version 70772 (0.0007) [2023-10-08 10:36:06,076][53852] Updated weights for policy 0, policy_version 71090 (0.0007) [2023-10-08 10:36:06,310][53885] Updated weights for policy 1, policy_version 70782 (0.0007) [2023-10-08 10:36:06,448][53852] Updated weights for policy 0, policy_version 71100 (0.0007) [2023-10-08 10:36:07,015][52710] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 145293312. Throughput: 0: 1834.0, 1: 1822.3. Samples: 36327894. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) [2023-10-08 10:36:07,016][52710] Avg episode reward: [(0, '30.530'), (1, '33.990')] [2023-10-08 10:36:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000070784_72482816.pth... [2023-10-08 10:36:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000071104_72810496.pth... [2023-10-08 10:36:07,067][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000069056_70713344.pth [2023-10-08 10:36:07,071][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000069376_71041024.pth [2023-10-08 10:36:10,056][53852] Updated weights for policy 0, policy_version 71110 (0.0008) [2023-10-08 10:36:10,259][53885] Updated weights for policy 1, policy_version 70792 (0.0010) [2023-10-08 10:36:10,429][53852] Updated weights for policy 0, policy_version 71120 (0.0007) [2023-10-08 10:36:10,634][53885] Updated weights for policy 1, policy_version 70802 (0.0009) [2023-10-08 10:36:10,783][53852] Updated weights for policy 0, policy_version 71130 (0.0008) [2023-10-08 10:36:11,000][53885] Updated weights for policy 1, policy_version 70812 (0.0008) [2023-10-08 10:36:12,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 145358848. Throughput: 0: 1833.8, 1: 1820.0. Samples: 36340550. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:12,016][52710] Avg episode reward: [(0, '32.460'), (1, '31.900')] [2023-10-08 10:36:14,439][53852] Updated weights for policy 0, policy_version 71140 (0.0008) [2023-10-08 10:36:14,599][53885] Updated weights for policy 1, policy_version 70822 (0.0009) [2023-10-08 10:36:14,816][53852] Updated weights for policy 0, policy_version 71150 (0.0008) [2023-10-08 10:36:14,969][53885] Updated weights for policy 1, policy_version 70832 (0.0008) [2023-10-08 10:36:15,182][53852] Updated weights for policy 0, policy_version 71160 (0.0008) [2023-10-08 10:36:15,327][53885] Updated weights for policy 1, policy_version 70842 (0.0009) [2023-10-08 10:36:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145424384. Throughput: 0: 1829.1, 1: 1815.2. Samples: 36360376. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:17,016][52710] Avg episode reward: [(0, '30.770'), (1, '31.190')] [2023-10-08 10:36:18,709][53852] Updated weights for policy 0, policy_version 71170 (0.0008) [2023-10-08 10:36:18,995][53885] Updated weights for policy 1, policy_version 70852 (0.0009) [2023-10-08 10:36:19,077][53852] Updated weights for policy 0, policy_version 71180 (0.0008) [2023-10-08 10:36:19,363][53885] Updated weights for policy 1, policy_version 70862 (0.0008) [2023-10-08 10:36:19,443][53852] Updated weights for policy 0, policy_version 71190 (0.0007) [2023-10-08 10:36:19,726][53885] Updated weights for policy 1, policy_version 70872 (0.0008) [2023-10-08 10:36:19,803][53852] Updated weights for policy 0, policy_version 71200 (0.0007) [2023-10-08 10:36:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 145489920. Throughput: 0: 1834.0, 1: 1814.0. Samples: 36383198. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:22,016][52710] Avg episode reward: [(0, '28.110'), (1, '35.460')] [2023-10-08 10:36:23,442][53885] Updated weights for policy 1, policy_version 70882 (0.0007) [2023-10-08 10:36:23,461][53852] Updated weights for policy 0, policy_version 71210 (0.0008) [2023-10-08 10:36:23,809][53885] Updated weights for policy 1, policy_version 70892 (0.0009) [2023-10-08 10:36:23,836][53852] Updated weights for policy 0, policy_version 71220 (0.0007) [2023-10-08 10:36:24,174][53885] Updated weights for policy 1, policy_version 70902 (0.0007) [2023-10-08 10:36:24,211][53852] Updated weights for policy 0, policy_version 71230 (0.0009) [2023-10-08 10:36:24,538][53885] Updated weights for policy 1, policy_version 70912 (0.0008) [2023-10-08 10:36:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 145555456. Throughput: 0: 1830.3, 1: 1817.9. Samples: 36393196. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:27,016][52710] Avg episode reward: [(0, '31.220'), (1, '34.950')] [2023-10-08 10:36:27,933][53852] Updated weights for policy 0, policy_version 71240 (0.0009) [2023-10-08 10:36:28,227][53885] Updated weights for policy 1, policy_version 70922 (0.0008) [2023-10-08 10:36:28,314][53852] Updated weights for policy 0, policy_version 71250 (0.0009) [2023-10-08 10:36:28,604][53885] Updated weights for policy 1, policy_version 70932 (0.0008) [2023-10-08 10:36:28,685][53852] Updated weights for policy 0, policy_version 71260 (0.0008) [2023-10-08 10:36:28,974][53885] Updated weights for policy 1, policy_version 70942 (0.0008) [2023-10-08 10:36:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 145620992. Throughput: 0: 1841.9, 1: 1816.9. Samples: 36416088. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:32,016][52710] Avg episode reward: [(0, '29.730'), (1, '32.600')] [2023-10-08 10:36:32,186][53852] Updated weights for policy 0, policy_version 71270 (0.0007) [2023-10-08 10:36:32,559][53852] Updated weights for policy 0, policy_version 71280 (0.0007) [2023-10-08 10:36:32,701][53885] Updated weights for policy 1, policy_version 70952 (0.0008) [2023-10-08 10:36:32,925][53852] Updated weights for policy 0, policy_version 71290 (0.0008) [2023-10-08 10:36:33,073][53885] Updated weights for policy 1, policy_version 70962 (0.0008) [2023-10-08 10:36:33,440][53885] Updated weights for policy 1, policy_version 70972 (0.0010) [2023-10-08 10:36:36,604][53852] Updated weights for policy 0, policy_version 71300 (0.0010) [2023-10-08 10:36:36,968][53852] Updated weights for policy 0, policy_version 71310 (0.0007) [2023-10-08 10:36:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 145686528. Throughput: 0: 1837.3, 1: 1820.7. Samples: 36438998. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:37,016][52710] Avg episode reward: [(0, '28.840'), (1, '36.900')] [2023-10-08 10:36:37,166][53885] Updated weights for policy 1, policy_version 70982 (0.0008) [2023-10-08 10:36:37,339][53852] Updated weights for policy 0, policy_version 71320 (0.0007) [2023-10-08 10:36:37,529][53885] Updated weights for policy 1, policy_version 70992 (0.0010) [2023-10-08 10:36:37,892][53885] Updated weights for policy 1, policy_version 71002 (0.0007) [2023-10-08 10:36:40,951][53852] Updated weights for policy 0, policy_version 71330 (0.0007) [2023-10-08 10:36:41,345][53852] Updated weights for policy 0, policy_version 71340 (0.0010) [2023-10-08 10:36:41,529][53885] Updated weights for policy 1, policy_version 71012 (0.0008) [2023-10-08 10:36:41,705][53852] Updated weights for policy 0, policy_version 71350 (0.0008) [2023-10-08 10:36:41,897][53885] Updated weights for policy 1, policy_version 71022 (0.0008) [2023-10-08 10:36:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 145752064. Throughput: 0: 1845.2, 1: 1819.0. Samples: 36449052. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:42,015][52710] Avg episode reward: [(0, '30.600'), (1, '32.420')] [2023-10-08 10:36:42,081][53852] Updated weights for policy 0, policy_version 71360 (0.0009) [2023-10-08 10:36:42,263][53885] Updated weights for policy 1, policy_version 71032 (0.0007) [2023-10-08 10:36:45,681][53852] Updated weights for policy 0, policy_version 71370 (0.0010) [2023-10-08 10:36:46,047][53852] Updated weights for policy 0, policy_version 71380 (0.0010) [2023-10-08 10:36:46,065][53885] Updated weights for policy 1, policy_version 71042 (0.0008) [2023-10-08 10:36:46,413][53852] Updated weights for policy 0, policy_version 71390 (0.0008) [2023-10-08 10:36:46,437][53885] Updated weights for policy 1, policy_version 71052 (0.0007) [2023-10-08 10:36:46,798][53885] Updated weights for policy 1, policy_version 71062 (0.0008) [2023-10-08 10:36:47,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 145850368. Throughput: 0: 1838.5, 1: 1818.3. Samples: 36471916. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:47,016][52710] Avg episode reward: [(0, '31.670'), (1, '32.100')] [2023-10-08 10:36:47,177][53885] Updated weights for policy 1, policy_version 71072 (0.0008) [2023-10-08 10:36:50,074][53852] Updated weights for policy 0, policy_version 71400 (0.0010) [2023-10-08 10:36:50,447][53852] Updated weights for policy 0, policy_version 71410 (0.0010) [2023-10-08 10:36:50,816][53852] Updated weights for policy 0, policy_version 71420 (0.0008) [2023-10-08 10:36:50,931][53885] Updated weights for policy 1, policy_version 71082 (0.0009) [2023-10-08 10:36:51,297][53885] Updated weights for policy 1, policy_version 71092 (0.0008) [2023-10-08 10:36:51,665][53885] Updated weights for policy 1, policy_version 71102 (0.0009) [2023-10-08 10:36:52,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 145948672. Throughput: 0: 1841.3, 1: 1814.1. Samples: 36492386. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:52,016][52710] Avg episode reward: [(0, '28.830'), (1, '32.720')] [2023-10-08 10:36:54,513][53852] Updated weights for policy 0, policy_version 71430 (0.0009) [2023-10-08 10:36:54,883][53852] Updated weights for policy 0, policy_version 71440 (0.0007) [2023-10-08 10:36:55,247][53852] Updated weights for policy 0, policy_version 71450 (0.0008) [2023-10-08 10:36:55,499][53885] Updated weights for policy 1, policy_version 71112 (0.0009) [2023-10-08 10:36:55,865][53885] Updated weights for policy 1, policy_version 71122 (0.0008) [2023-10-08 10:36:56,231][53885] Updated weights for policy 1, policy_version 71132 (0.0008) [2023-10-08 10:36:57,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 146014208. Throughput: 0: 1837.2, 1: 1806.9. Samples: 36504536. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:36:57,016][52710] Avg episode reward: [(0, '28.830'), (1, '32.670')] [2023-10-08 10:36:58,946][53852] Updated weights for policy 0, policy_version 71460 (0.0009) [2023-10-08 10:36:59,313][53852] Updated weights for policy 0, policy_version 71470 (0.0009) [2023-10-08 10:36:59,684][53852] Updated weights for policy 0, policy_version 71480 (0.0009) [2023-10-08 10:36:59,838][53885] Updated weights for policy 1, policy_version 71142 (0.0007) [2023-10-08 10:37:00,204][53885] Updated weights for policy 1, policy_version 71152 (0.0010) [2023-10-08 10:37:00,575][53885] Updated weights for policy 1, policy_version 71162 (0.0011) [2023-10-08 10:37:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 146079744. Throughput: 0: 1838.3, 1: 1820.2. Samples: 36525010. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:02,016][52710] Avg episode reward: [(0, '32.540'), (1, '31.560')] [2023-10-08 10:37:03,205][53852] Updated weights for policy 0, policy_version 71490 (0.0008) [2023-10-08 10:37:03,585][53852] Updated weights for policy 0, policy_version 71500 (0.0007) [2023-10-08 10:37:03,951][53852] Updated weights for policy 0, policy_version 71510 (0.0008) [2023-10-08 10:37:04,114][53885] Updated weights for policy 1, policy_version 71172 (0.0011) [2023-10-08 10:37:04,319][53852] Updated weights for policy 0, policy_version 71520 (0.0007) [2023-10-08 10:37:04,477][53885] Updated weights for policy 1, policy_version 71182 (0.0010) [2023-10-08 10:37:04,848][53885] Updated weights for policy 1, policy_version 71192 (0.0007) [2023-10-08 10:37:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 146145280. Throughput: 0: 1842.1, 1: 1819.5. Samples: 36547968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:07,016][52710] Avg episode reward: [(0, '31.100'), (1, '30.650')] [2023-10-08 10:37:08,124][53852] Updated weights for policy 0, policy_version 71530 (0.0007) [2023-10-08 10:37:08,497][53852] Updated weights for policy 0, policy_version 71540 (0.0010) [2023-10-08 10:37:08,532][53885] Updated weights for policy 1, policy_version 71202 (0.0008) [2023-10-08 10:37:08,856][53852] Updated weights for policy 0, policy_version 71550 (0.0009) [2023-10-08 10:37:08,906][53885] Updated weights for policy 1, policy_version 71212 (0.0007) [2023-10-08 10:37:09,274][53885] Updated weights for policy 1, policy_version 71222 (0.0008) [2023-10-08 10:37:09,638][53885] Updated weights for policy 1, policy_version 71232 (0.0009) [2023-10-08 10:37:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 146210816. Throughput: 0: 1839.6, 1: 1824.1. Samples: 36558066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:12,016][52710] Avg episode reward: [(0, '31.610'), (1, '33.540')] [2023-10-08 10:37:12,578][53852] Updated weights for policy 0, policy_version 71560 (0.0007) [2023-10-08 10:37:12,947][53852] Updated weights for policy 0, policy_version 71570 (0.0009) [2023-10-08 10:37:13,321][53852] Updated weights for policy 0, policy_version 71580 (0.0008) [2023-10-08 10:37:13,422][53885] Updated weights for policy 1, policy_version 71242 (0.0009) [2023-10-08 10:37:13,785][53885] Updated weights for policy 1, policy_version 71252 (0.0008) [2023-10-08 10:37:14,145][53885] Updated weights for policy 1, policy_version 71262 (0.0009) [2023-10-08 10:37:16,994][53852] Updated weights for policy 0, policy_version 71590 (0.0008) [2023-10-08 10:37:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 146276352. Throughput: 0: 1836.0, 1: 1825.1. Samples: 36580836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:17,016][52710] Avg episode reward: [(0, '30.590'), (1, '32.080')] [2023-10-08 10:37:17,359][53852] Updated weights for policy 0, policy_version 71600 (0.0009) [2023-10-08 10:37:17,736][53852] Updated weights for policy 0, policy_version 71610 (0.0008) [2023-10-08 10:37:17,893][53885] Updated weights for policy 1, policy_version 71272 (0.0007) [2023-10-08 10:37:18,250][53885] Updated weights for policy 1, policy_version 71282 (0.0008) [2023-10-08 10:37:18,610][53885] Updated weights for policy 1, policy_version 71292 (0.0007) [2023-10-08 10:37:21,195][53852] Updated weights for policy 0, policy_version 71620 (0.0007) [2023-10-08 10:37:21,564][53852] Updated weights for policy 0, policy_version 71630 (0.0007) [2023-10-08 10:37:21,937][53852] Updated weights for policy 0, policy_version 71640 (0.0009) [2023-10-08 10:37:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 146341888. Throughput: 0: 1834.2, 1: 1820.0. Samples: 36603440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:22,016][52710] Avg episode reward: [(0, '33.180'), (1, '33.160')] [2023-10-08 10:37:22,265][53885] Updated weights for policy 1, policy_version 71302 (0.0008) [2023-10-08 10:37:22,632][53885] Updated weights for policy 1, policy_version 71312 (0.0009) [2023-10-08 10:37:23,005][53885] Updated weights for policy 1, policy_version 71322 (0.0007) [2023-10-08 10:37:25,560][53852] Updated weights for policy 0, policy_version 71650 (0.0010) [2023-10-08 10:37:25,936][53852] Updated weights for policy 0, policy_version 71660 (0.0008) [2023-10-08 10:37:26,297][53852] Updated weights for policy 0, policy_version 71670 (0.0007) [2023-10-08 10:37:26,571][53885] Updated weights for policy 1, policy_version 71332 (0.0008) [2023-10-08 10:37:26,673][53852] Updated weights for policy 0, policy_version 71680 (0.0007) [2023-10-08 10:37:26,938][53885] Updated weights for policy 1, policy_version 71342 (0.0008) [2023-10-08 10:37:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146440192. Throughput: 0: 1843.8, 1: 1821.5. Samples: 36613992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:27,016][52710] Avg episode reward: [(0, '29.620'), (1, '35.650')] [2023-10-08 10:37:27,299][53885] Updated weights for policy 1, policy_version 71352 (0.0009) [2023-10-08 10:37:30,498][53852] Updated weights for policy 0, policy_version 71690 (0.0008) [2023-10-08 10:37:30,841][53885] Updated weights for policy 1, policy_version 71362 (0.0008) [2023-10-08 10:37:30,870][53852] Updated weights for policy 0, policy_version 71700 (0.0009) [2023-10-08 10:37:31,221][53885] Updated weights for policy 1, policy_version 71372 (0.0009) [2023-10-08 10:37:31,235][53852] Updated weights for policy 0, policy_version 71710 (0.0007) [2023-10-08 10:37:31,593][53885] Updated weights for policy 1, policy_version 71382 (0.0009) [2023-10-08 10:37:31,965][53885] Updated weights for policy 1, policy_version 71392 (0.0009) [2023-10-08 10:37:32,015][52710] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 146538496. Throughput: 0: 1832.4, 1: 1827.8. Samples: 36636626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:32,015][52710] Avg episode reward: [(0, '29.810'), (1, '33.470')] [2023-10-08 10:37:34,722][53852] Updated weights for policy 0, policy_version 71720 (0.0007) [2023-10-08 10:37:35,099][53852] Updated weights for policy 0, policy_version 71730 (0.0010) [2023-10-08 10:37:35,465][53852] Updated weights for policy 0, policy_version 71740 (0.0008) [2023-10-08 10:37:35,557][53885] Updated weights for policy 1, policy_version 71402 (0.0010) [2023-10-08 10:37:35,933][53885] Updated weights for policy 1, policy_version 71412 (0.0011) [2023-10-08 10:37:36,306][53885] Updated weights for policy 1, policy_version 71422 (0.0008) [2023-10-08 10:37:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 146604032. Throughput: 0: 1837.3, 1: 1828.5. Samples: 36657350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:37,016][52710] Avg episode reward: [(0, '30.360'), (1, '34.110')] [2023-10-08 10:37:39,193][53852] Updated weights for policy 0, policy_version 71750 (0.0008) [2023-10-08 10:37:39,565][53852] Updated weights for policy 0, policy_version 71760 (0.0008) [2023-10-08 10:37:39,931][53852] Updated weights for policy 0, policy_version 71770 (0.0007) [2023-10-08 10:37:40,064][53885] Updated weights for policy 1, policy_version 71432 (0.0007) [2023-10-08 10:37:40,438][53885] Updated weights for policy 1, policy_version 71442 (0.0007) [2023-10-08 10:37:40,795][53885] Updated weights for policy 1, policy_version 71452 (0.0009) [2023-10-08 10:37:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 146669568. Throughput: 0: 1829.7, 1: 1841.4. Samples: 36669734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:42,016][52710] Avg episode reward: [(0, '31.530'), (1, '32.630')] [2023-10-08 10:37:43,487][53852] Updated weights for policy 0, policy_version 71780 (0.0007) [2023-10-08 10:37:43,856][53852] Updated weights for policy 0, policy_version 71790 (0.0009) [2023-10-08 10:37:44,212][53852] Updated weights for policy 0, policy_version 71800 (0.0010) [2023-10-08 10:37:44,432][53885] Updated weights for policy 1, policy_version 71462 (0.0008) [2023-10-08 10:37:44,799][53885] Updated weights for policy 1, policy_version 71472 (0.0007) [2023-10-08 10:37:45,159][53885] Updated weights for policy 1, policy_version 71482 (0.0009) [2023-10-08 10:37:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 146735104. Throughput: 0: 1844.0, 1: 1827.9. Samples: 36690242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:47,016][52710] Avg episode reward: [(0, '29.250'), (1, '32.360')] [2023-10-08 10:37:47,908][53852] Updated weights for policy 0, policy_version 71810 (0.0007) [2023-10-08 10:37:48,278][53852] Updated weights for policy 0, policy_version 71820 (0.0007) [2023-10-08 10:37:48,649][53852] Updated weights for policy 0, policy_version 71830 (0.0007) [2023-10-08 10:37:48,715][53885] Updated weights for policy 1, policy_version 71492 (0.0008) [2023-10-08 10:37:49,013][53852] Updated weights for policy 0, policy_version 71840 (0.0009) [2023-10-08 10:37:49,082][53885] Updated weights for policy 1, policy_version 71502 (0.0009) [2023-10-08 10:37:49,443][53885] Updated weights for policy 1, policy_version 71512 (0.0009) [2023-10-08 10:37:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 146800640. Throughput: 0: 1837.1, 1: 1832.5. Samples: 36713102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:52,016][52710] Avg episode reward: [(0, '31.120'), (1, '32.420')] [2023-10-08 10:37:52,665][53852] Updated weights for policy 0, policy_version 71850 (0.0008) [2023-10-08 10:37:53,032][53852] Updated weights for policy 0, policy_version 71860 (0.0008) [2023-10-08 10:37:53,296][53885] Updated weights for policy 1, policy_version 71522 (0.0009) [2023-10-08 10:37:53,408][53852] Updated weights for policy 0, policy_version 71870 (0.0009) [2023-10-08 10:37:53,674][53885] Updated weights for policy 1, policy_version 71532 (0.0008) [2023-10-08 10:37:54,037][53885] Updated weights for policy 1, policy_version 71542 (0.0009) [2023-10-08 10:37:54,400][53885] Updated weights for policy 1, policy_version 71552 (0.0008) [2023-10-08 10:37:56,977][53852] Updated weights for policy 0, policy_version 71880 (0.0010) [2023-10-08 10:37:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 146866176. Throughput: 0: 1839.1, 1: 1825.2. Samples: 36722960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:37:57,015][52710] Avg episode reward: [(0, '33.190'), (1, '33.310')] [2023-10-08 10:37:57,336][53852] Updated weights for policy 0, policy_version 71890 (0.0009) [2023-10-08 10:37:57,705][53852] Updated weights for policy 0, policy_version 71900 (0.0009) [2023-10-08 10:37:58,091][53885] Updated weights for policy 1, policy_version 71562 (0.0008) [2023-10-08 10:37:58,452][53885] Updated weights for policy 1, policy_version 71572 (0.0009) [2023-10-08 10:37:58,826][53885] Updated weights for policy 1, policy_version 71582 (0.0009) [2023-10-08 10:38:01,319][53852] Updated weights for policy 0, policy_version 71910 (0.0007) [2023-10-08 10:38:01,693][53852] Updated weights for policy 0, policy_version 71920 (0.0010) [2023-10-08 10:38:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 146931712. Throughput: 0: 1844.1, 1: 1824.2. Samples: 36745910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:02,016][52710] Avg episode reward: [(0, '32.770'), (1, '30.890')] [2023-10-08 10:38:02,076][53852] Updated weights for policy 0, policy_version 71930 (0.0008) [2023-10-08 10:38:02,549][53885] Updated weights for policy 1, policy_version 71592 (0.0012) [2023-10-08 10:38:02,913][53885] Updated weights for policy 1, policy_version 71602 (0.0009) [2023-10-08 10:38:03,286][53885] Updated weights for policy 1, policy_version 71612 (0.0007) [2023-10-08 10:38:05,655][53852] Updated weights for policy 0, policy_version 71940 (0.0008) [2023-10-08 10:38:06,034][53852] Updated weights for policy 0, policy_version 71950 (0.0007) [2023-10-08 10:38:06,396][53852] Updated weights for policy 0, policy_version 71960 (0.0009) [2023-10-08 10:38:06,879][53885] Updated weights for policy 1, policy_version 71622 (0.0008) [2023-10-08 10:38:07,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147030016. Throughput: 0: 1821.9, 1: 1834.7. Samples: 36767986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:07,016][52710] Avg episode reward: [(0, '29.750'), (1, '31.970')] [2023-10-08 10:38:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000071968_73695232.pth... [2023-10-08 10:38:07,057][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000070240_71925760.pth [2023-10-08 10:38:07,244][53885] Updated weights for policy 1, policy_version 71632 (0.0007) [2023-10-08 10:38:07,617][53885] Updated weights for policy 1, policy_version 71642 (0.0011) [2023-10-08 10:38:07,838][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000071648_73367552.pth... [2023-10-08 10:38:07,874][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000069920_71598080.pth [2023-10-08 10:38:10,173][53852] Updated weights for policy 0, policy_version 71970 (0.0009) [2023-10-08 10:38:10,541][53852] Updated weights for policy 0, policy_version 71980 (0.0007) [2023-10-08 10:38:10,913][53852] Updated weights for policy 0, policy_version 71990 (0.0008) [2023-10-08 10:38:11,278][53852] Updated weights for policy 0, policy_version 72000 (0.0007) [2023-10-08 10:38:11,358][53885] Updated weights for policy 1, policy_version 71652 (0.0009) [2023-10-08 10:38:11,718][53885] Updated weights for policy 1, policy_version 71662 (0.0009) [2023-10-08 10:38:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147095552. Throughput: 0: 1834.0, 1: 1834.1. Samples: 36779056. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:12,016][52710] Avg episode reward: [(0, '32.840'), (1, '36.000')] [2023-10-08 10:38:12,084][53885] Updated weights for policy 1, policy_version 71672 (0.0011) [2023-10-08 10:38:14,994][53852] Updated weights for policy 0, policy_version 72010 (0.0008) [2023-10-08 10:38:15,363][53852] Updated weights for policy 0, policy_version 72020 (0.0008) [2023-10-08 10:38:15,734][53852] Updated weights for policy 0, policy_version 72030 (0.0010) [2023-10-08 10:38:15,870][53885] Updated weights for policy 1, policy_version 71682 (0.0009) [2023-10-08 10:38:16,239][53885] Updated weights for policy 1, policy_version 71692 (0.0008) [2023-10-08 10:38:16,603][53885] Updated weights for policy 1, policy_version 71702 (0.0008) [2023-10-08 10:38:16,979][53885] Updated weights for policy 1, policy_version 71712 (0.0008) [2023-10-08 10:38:17,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 147193856. Throughput: 0: 1820.8, 1: 1826.7. Samples: 36800762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:17,016][52710] Avg episode reward: [(0, '32.560'), (1, '33.860')] [2023-10-08 10:38:19,491][53852] Updated weights for policy 0, policy_version 72040 (0.0009) [2023-10-08 10:38:19,859][53852] Updated weights for policy 0, policy_version 72050 (0.0008) [2023-10-08 10:38:20,232][53852] Updated weights for policy 0, policy_version 72060 (0.0007) [2023-10-08 10:38:20,558][53885] Updated weights for policy 1, policy_version 71722 (0.0007) [2023-10-08 10:38:20,921][53885] Updated weights for policy 1, policy_version 71732 (0.0009) [2023-10-08 10:38:21,289][53885] Updated weights for policy 1, policy_version 71742 (0.0007) [2023-10-08 10:38:22,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 147259392. Throughput: 0: 1831.6, 1: 1826.6. Samples: 36821970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:22,016][52710] Avg episode reward: [(0, '29.400'), (1, '33.650')] [2023-10-08 10:38:23,896][53852] Updated weights for policy 0, policy_version 72070 (0.0010) [2023-10-08 10:38:24,254][53852] Updated weights for policy 0, policy_version 72080 (0.0009) [2023-10-08 10:38:24,621][53852] Updated weights for policy 0, policy_version 72090 (0.0009) [2023-10-08 10:38:25,002][53885] Updated weights for policy 1, policy_version 71752 (0.0007) [2023-10-08 10:38:25,371][53885] Updated weights for policy 1, policy_version 71762 (0.0007) [2023-10-08 10:38:25,737][53885] Updated weights for policy 1, policy_version 71772 (0.0007) [2023-10-08 10:38:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147324928. Throughput: 0: 1819.9, 1: 1826.9. Samples: 36833842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:27,016][52710] Avg episode reward: [(0, '29.310'), (1, '37.440')] [2023-10-08 10:38:28,126][53852] Updated weights for policy 0, policy_version 72100 (0.0008) [2023-10-08 10:38:28,504][53852] Updated weights for policy 0, policy_version 72110 (0.0010) [2023-10-08 10:38:28,877][53852] Updated weights for policy 0, policy_version 72120 (0.0007) [2023-10-08 10:38:29,434][53885] Updated weights for policy 1, policy_version 71782 (0.0009) [2023-10-08 10:38:29,803][53885] Updated weights for policy 1, policy_version 71792 (0.0010) [2023-10-08 10:38:30,177][53885] Updated weights for policy 1, policy_version 71802 (0.0008) [2023-10-08 10:38:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 147390464. Throughput: 0: 1826.4, 1: 1826.4. Samples: 36854616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:32,016][52710] Avg episode reward: [(0, '30.480'), (1, '36.530')] [2023-10-08 10:38:32,554][53852] Updated weights for policy 0, policy_version 72130 (0.0007) [2023-10-08 10:38:32,931][53852] Updated weights for policy 0, policy_version 72140 (0.0007) [2023-10-08 10:38:33,297][53852] Updated weights for policy 0, policy_version 72150 (0.0007) [2023-10-08 10:38:33,666][53852] Updated weights for policy 0, policy_version 72160 (0.0007) [2023-10-08 10:38:33,760][53885] Updated weights for policy 1, policy_version 71812 (0.0009) [2023-10-08 10:38:34,124][53885] Updated weights for policy 1, policy_version 71822 (0.0010) [2023-10-08 10:38:34,483][53885] Updated weights for policy 1, policy_version 71832 (0.0007) [2023-10-08 10:38:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 147456000. Throughput: 0: 1830.1, 1: 1828.0. Samples: 36877718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:37,016][52710] Avg episode reward: [(0, '28.920'), (1, '33.930')] [2023-10-08 10:38:37,383][53852] Updated weights for policy 0, policy_version 72170 (0.0008) [2023-10-08 10:38:37,750][53852] Updated weights for policy 0, policy_version 72180 (0.0010) [2023-10-08 10:38:38,043][53885] Updated weights for policy 1, policy_version 71842 (0.0008) [2023-10-08 10:38:38,119][53852] Updated weights for policy 0, policy_version 72190 (0.0009) [2023-10-08 10:38:38,413][53885] Updated weights for policy 1, policy_version 71852 (0.0010) [2023-10-08 10:38:38,772][53885] Updated weights for policy 1, policy_version 71862 (0.0010) [2023-10-08 10:38:39,144][53885] Updated weights for policy 1, policy_version 71872 (0.0011) [2023-10-08 10:38:41,706][53852] Updated weights for policy 0, policy_version 72200 (0.0009) [2023-10-08 10:38:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 147521536. Throughput: 0: 1833.3, 1: 1829.9. Samples: 36887804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:42,015][52710] Avg episode reward: [(0, '28.910'), (1, '33.450')] [2023-10-08 10:38:42,080][53852] Updated weights for policy 0, policy_version 72210 (0.0008) [2023-10-08 10:38:42,452][53852] Updated weights for policy 0, policy_version 72220 (0.0008) [2023-10-08 10:38:42,941][53885] Updated weights for policy 1, policy_version 71882 (0.0007) [2023-10-08 10:38:43,301][53885] Updated weights for policy 1, policy_version 71892 (0.0007) [2023-10-08 10:38:43,664][53885] Updated weights for policy 1, policy_version 71902 (0.0008) [2023-10-08 10:38:46,259][53852] Updated weights for policy 0, policy_version 72230 (0.0007) [2023-10-08 10:38:46,627][53852] Updated weights for policy 0, policy_version 72240 (0.0007) [2023-10-08 10:38:47,005][53852] Updated weights for policy 0, policy_version 72250 (0.0007) [2023-10-08 10:38:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 147587072. Throughput: 0: 1822.0, 1: 1834.2. Samples: 36910438. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:47,016][52710] Avg episode reward: [(0, '29.880'), (1, '36.280')] [2023-10-08 10:38:47,322][53885] Updated weights for policy 1, policy_version 71912 (0.0008) [2023-10-08 10:38:47,679][53885] Updated weights for policy 1, policy_version 71922 (0.0008) [2023-10-08 10:38:48,045][53885] Updated weights for policy 1, policy_version 71932 (0.0009) [2023-10-08 10:38:50,753][53852] Updated weights for policy 0, policy_version 72260 (0.0008) [2023-10-08 10:38:51,120][53852] Updated weights for policy 0, policy_version 72270 (0.0007) [2023-10-08 10:38:51,494][53852] Updated weights for policy 0, policy_version 72280 (0.0007) [2023-10-08 10:38:51,697][53885] Updated weights for policy 1, policy_version 71942 (0.0007) [2023-10-08 10:38:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147685376. Throughput: 0: 1823.3, 1: 1826.8. Samples: 36932240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:52,016][52710] Avg episode reward: [(0, '31.690'), (1, '32.500')] [2023-10-08 10:38:52,059][53885] Updated weights for policy 1, policy_version 71952 (0.0010) [2023-10-08 10:38:52,434][53885] Updated weights for policy 1, policy_version 71962 (0.0009) [2023-10-08 10:38:55,039][53852] Updated weights for policy 0, policy_version 72290 (0.0007) [2023-10-08 10:38:55,402][53852] Updated weights for policy 0, policy_version 72300 (0.0008) [2023-10-08 10:38:55,772][53852] Updated weights for policy 0, policy_version 72310 (0.0007) [2023-10-08 10:38:56,142][53852] Updated weights for policy 0, policy_version 72320 (0.0007) [2023-10-08 10:38:56,152][53885] Updated weights for policy 1, policy_version 71972 (0.0009) [2023-10-08 10:38:56,518][53885] Updated weights for policy 1, policy_version 71982 (0.0008) [2023-10-08 10:38:56,886][53885] Updated weights for policy 1, policy_version 71992 (0.0011) [2023-10-08 10:38:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 147750912. Throughput: 0: 1832.2, 1: 1827.6. Samples: 36943748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:38:57,016][52710] Avg episode reward: [(0, '32.940'), (1, '33.360')] [2023-10-08 10:38:59,700][53852] Updated weights for policy 0, policy_version 72330 (0.0007) [2023-10-08 10:39:00,067][53852] Updated weights for policy 0, policy_version 72340 (0.0007) [2023-10-08 10:39:00,451][53852] Updated weights for policy 0, policy_version 72350 (0.0008) [2023-10-08 10:39:00,474][53885] Updated weights for policy 1, policy_version 72002 (0.0009) [2023-10-08 10:39:00,835][53885] Updated weights for policy 1, policy_version 72012 (0.0008) [2023-10-08 10:39:01,209][53885] Updated weights for policy 1, policy_version 72022 (0.0008) [2023-10-08 10:39:01,567][53885] Updated weights for policy 1, policy_version 72032 (0.0007) [2023-10-08 10:39:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 147849216. Throughput: 0: 1835.8, 1: 1824.9. Samples: 36965494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:02,016][52710] Avg episode reward: [(0, '30.360'), (1, '34.980')] [2023-10-08 10:39:04,287][53852] Updated weights for policy 0, policy_version 72360 (0.0009) [2023-10-08 10:39:04,661][53852] Updated weights for policy 0, policy_version 72370 (0.0008) [2023-10-08 10:39:05,031][53852] Updated weights for policy 0, policy_version 72380 (0.0009) [2023-10-08 10:39:05,410][53885] Updated weights for policy 1, policy_version 72042 (0.0009) [2023-10-08 10:39:05,773][53885] Updated weights for policy 1, policy_version 72052 (0.0009) [2023-10-08 10:39:06,139][53885] Updated weights for policy 1, policy_version 72062 (0.0010) [2023-10-08 10:39:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147914752. Throughput: 0: 1834.0, 1: 1826.4. Samples: 36986692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:07,016][52710] Avg episode reward: [(0, '33.280'), (1, '34.370')] [2023-10-08 10:39:08,584][53852] Updated weights for policy 0, policy_version 72390 (0.0009) [2023-10-08 10:39:08,958][53852] Updated weights for policy 0, policy_version 72400 (0.0007) [2023-10-08 10:39:09,320][53852] Updated weights for policy 0, policy_version 72410 (0.0008) [2023-10-08 10:39:09,801][53885] Updated weights for policy 1, policy_version 72072 (0.0009) [2023-10-08 10:39:10,178][53885] Updated weights for policy 1, policy_version 72082 (0.0008) [2023-10-08 10:39:10,538][53885] Updated weights for policy 1, policy_version 72092 (0.0009) [2023-10-08 10:39:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147980288. Throughput: 0: 1834.1, 1: 1821.3. Samples: 36998334. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:12,016][52710] Avg episode reward: [(0, '32.310'), (1, '32.790')] [2023-10-08 10:39:12,864][53852] Updated weights for policy 0, policy_version 72420 (0.0008) [2023-10-08 10:39:13,220][53852] Updated weights for policy 0, policy_version 72430 (0.0008) [2023-10-08 10:39:13,598][53852] Updated weights for policy 0, policy_version 72440 (0.0009) [2023-10-08 10:39:14,303][53885] Updated weights for policy 1, policy_version 72102 (0.0008) [2023-10-08 10:39:14,670][53885] Updated weights for policy 1, policy_version 72112 (0.0008) [2023-10-08 10:39:15,036][53885] Updated weights for policy 1, policy_version 72122 (0.0007) [2023-10-08 10:39:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 148045824. Throughput: 0: 1844.5, 1: 1828.9. Samples: 37019922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:17,016][52710] Avg episode reward: [(0, '31.310'), (1, '32.850')] [2023-10-08 10:39:17,170][53852] Updated weights for policy 0, policy_version 72450 (0.0010) [2023-10-08 10:39:17,545][53852] Updated weights for policy 0, policy_version 72460 (0.0008) [2023-10-08 10:39:17,905][53852] Updated weights for policy 0, policy_version 72470 (0.0007) [2023-10-08 10:39:18,269][53852] Updated weights for policy 0, policy_version 72480 (0.0007) [2023-10-08 10:39:18,627][53885] Updated weights for policy 1, policy_version 72132 (0.0008) [2023-10-08 10:39:18,997][53885] Updated weights for policy 1, policy_version 72142 (0.0008) [2023-10-08 10:39:19,361][53885] Updated weights for policy 1, policy_version 72152 (0.0008) [2023-10-08 10:39:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 148111360. Throughput: 0: 1845.6, 1: 1831.2. Samples: 37043172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:22,016][52710] Avg episode reward: [(0, '29.670'), (1, '33.110')] [2023-10-08 10:39:22,069][53852] Updated weights for policy 0, policy_version 72490 (0.0008) [2023-10-08 10:39:22,428][53852] Updated weights for policy 0, policy_version 72500 (0.0009) [2023-10-08 10:39:22,799][53852] Updated weights for policy 0, policy_version 72510 (0.0008) [2023-10-08 10:39:23,086][53885] Updated weights for policy 1, policy_version 72162 (0.0008) [2023-10-08 10:39:23,457][53885] Updated weights for policy 1, policy_version 72172 (0.0009) [2023-10-08 10:39:23,819][53885] Updated weights for policy 1, policy_version 72182 (0.0009) [2023-10-08 10:39:24,194][53885] Updated weights for policy 1, policy_version 72192 (0.0011) [2023-10-08 10:39:26,403][53852] Updated weights for policy 0, policy_version 72520 (0.0009) [2023-10-08 10:39:26,772][53852] Updated weights for policy 0, policy_version 72530 (0.0011) [2023-10-08 10:39:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 148176896. Throughput: 0: 1845.6, 1: 1831.1. Samples: 37053254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:27,015][52710] Avg episode reward: [(0, '32.680'), (1, '32.720')] [2023-10-08 10:39:27,147][53852] Updated weights for policy 0, policy_version 72540 (0.0011) [2023-10-08 10:39:27,799][53885] Updated weights for policy 1, policy_version 72202 (0.0009) [2023-10-08 10:39:28,163][53885] Updated weights for policy 1, policy_version 72212 (0.0010) [2023-10-08 10:39:28,524][53885] Updated weights for policy 1, policy_version 72222 (0.0009) [2023-10-08 10:39:30,861][53852] Updated weights for policy 0, policy_version 72550 (0.0009) [2023-10-08 10:39:31,241][53852] Updated weights for policy 0, policy_version 72560 (0.0008) [2023-10-08 10:39:31,608][53852] Updated weights for policy 0, policy_version 72570 (0.0010) [2023-10-08 10:39:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148275200. Throughput: 0: 1854.9, 1: 1828.8. Samples: 37076206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:32,016][52710] Avg episode reward: [(0, '31.320'), (1, '34.820')] [2023-10-08 10:39:32,132][53885] Updated weights for policy 1, policy_version 72232 (0.0007) [2023-10-08 10:39:32,495][53885] Updated weights for policy 1, policy_version 72242 (0.0007) [2023-10-08 10:39:32,870][53885] Updated weights for policy 1, policy_version 72252 (0.0009) [2023-10-08 10:39:35,258][53852] Updated weights for policy 0, policy_version 72580 (0.0012) [2023-10-08 10:39:35,631][53852] Updated weights for policy 0, policy_version 72590 (0.0009) [2023-10-08 10:39:36,002][53852] Updated weights for policy 0, policy_version 72600 (0.0009) [2023-10-08 10:39:36,556][53885] Updated weights for policy 1, policy_version 72262 (0.0008) [2023-10-08 10:39:36,918][53885] Updated weights for policy 1, policy_version 72272 (0.0007) [2023-10-08 10:39:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148340736. Throughput: 0: 1844.4, 1: 1822.9. Samples: 37097268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:37,016][52710] Avg episode reward: [(0, '30.000'), (1, '38.260')] [2023-10-08 10:39:37,283][53885] Updated weights for policy 1, policy_version 72282 (0.0007) [2023-10-08 10:39:39,666][53852] Updated weights for policy 0, policy_version 72610 (0.0007) [2023-10-08 10:39:40,044][53852] Updated weights for policy 0, policy_version 72620 (0.0008) [2023-10-08 10:39:40,412][53852] Updated weights for policy 0, policy_version 72630 (0.0007) [2023-10-08 10:39:40,781][53852] Updated weights for policy 0, policy_version 72640 (0.0007) [2023-10-08 10:39:41,051][53885] Updated weights for policy 1, policy_version 72292 (0.0009) [2023-10-08 10:39:41,431][53885] Updated weights for policy 1, policy_version 72302 (0.0010) [2023-10-08 10:39:41,797][53885] Updated weights for policy 1, policy_version 72312 (0.0007) [2023-10-08 10:39:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148406272. Throughput: 0: 1848.0, 1: 1826.5. Samples: 37109102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:42,016][52710] Avg episode reward: [(0, '32.670'), (1, '34.370')] [2023-10-08 10:39:44,398][53852] Updated weights for policy 0, policy_version 72650 (0.0007) [2023-10-08 10:39:44,771][53852] Updated weights for policy 0, policy_version 72660 (0.0007) [2023-10-08 10:39:45,151][53852] Updated weights for policy 0, policy_version 72670 (0.0011) [2023-10-08 10:39:45,580][53885] Updated weights for policy 1, policy_version 72322 (0.0007) [2023-10-08 10:39:45,954][53885] Updated weights for policy 1, policy_version 72332 (0.0010) [2023-10-08 10:39:46,333][53885] Updated weights for policy 1, policy_version 72342 (0.0009) [2023-10-08 10:39:46,696][53885] Updated weights for policy 1, policy_version 72352 (0.0009) [2023-10-08 10:39:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 148504576. Throughput: 0: 1836.5, 1: 1824.8. Samples: 37130252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:47,016][52710] Avg episode reward: [(0, '30.080'), (1, '34.100')] [2023-10-08 10:39:48,793][53852] Updated weights for policy 0, policy_version 72680 (0.0008) [2023-10-08 10:39:49,165][53852] Updated weights for policy 0, policy_version 72690 (0.0008) [2023-10-08 10:39:49,542][53852] Updated weights for policy 0, policy_version 72700 (0.0008) [2023-10-08 10:39:50,209][53885] Updated weights for policy 1, policy_version 72362 (0.0008) [2023-10-08 10:39:50,576][53885] Updated weights for policy 1, policy_version 72372 (0.0007) [2023-10-08 10:39:50,945][53885] Updated weights for policy 1, policy_version 72382 (0.0010) [2023-10-08 10:39:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148570112. Throughput: 0: 1841.1, 1: 1829.4. Samples: 37151864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:52,016][52710] Avg episode reward: [(0, '30.960'), (1, '33.360')] [2023-10-08 10:39:53,351][53852] Updated weights for policy 0, policy_version 72710 (0.0009) [2023-10-08 10:39:53,730][53852] Updated weights for policy 0, policy_version 72720 (0.0010) [2023-10-08 10:39:54,104][53852] Updated weights for policy 0, policy_version 72730 (0.0011) [2023-10-08 10:39:54,547][53885] Updated weights for policy 1, policy_version 72392 (0.0008) [2023-10-08 10:39:54,915][53885] Updated weights for policy 1, policy_version 72402 (0.0010) [2023-10-08 10:39:55,279][53885] Updated weights for policy 1, policy_version 72412 (0.0010) [2023-10-08 10:39:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148635648. Throughput: 0: 1830.4, 1: 1823.9. Samples: 37162776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:39:57,016][52710] Avg episode reward: [(0, '31.630'), (1, '34.520')] [2023-10-08 10:39:57,701][53852] Updated weights for policy 0, policy_version 72740 (0.0007) [2023-10-08 10:39:58,066][53852] Updated weights for policy 0, policy_version 72750 (0.0010) [2023-10-08 10:39:58,429][53852] Updated weights for policy 0, policy_version 72760 (0.0010) [2023-10-08 10:39:58,924][53885] Updated weights for policy 1, policy_version 72422 (0.0008) [2023-10-08 10:39:59,296][53885] Updated weights for policy 1, policy_version 72432 (0.0007) [2023-10-08 10:39:59,670][53885] Updated weights for policy 1, policy_version 72442 (0.0007) [2023-10-08 10:40:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 148701184. Throughput: 0: 1832.1, 1: 1830.0. Samples: 37184714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:40:02,016][52710] Avg episode reward: [(0, '29.360'), (1, '33.490')] [2023-10-08 10:40:02,099][53852] Updated weights for policy 0, policy_version 72770 (0.0008) [2023-10-08 10:40:02,471][53852] Updated weights for policy 0, policy_version 72780 (0.0007) [2023-10-08 10:40:02,836][53852] Updated weights for policy 0, policy_version 72790 (0.0007) [2023-10-08 10:40:03,202][53852] Updated weights for policy 0, policy_version 72800 (0.0007) [2023-10-08 10:40:03,402][53885] Updated weights for policy 1, policy_version 72452 (0.0008) [2023-10-08 10:40:03,755][53885] Updated weights for policy 1, policy_version 72462 (0.0012) [2023-10-08 10:40:04,132][53885] Updated weights for policy 1, policy_version 72472 (0.0011) [2023-10-08 10:40:06,921][53852] Updated weights for policy 0, policy_version 72810 (0.0009) [2023-10-08 10:40:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 148766720. Throughput: 0: 1823.8, 1: 1826.5. Samples: 37207436. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:40:07,016][52710] Avg episode reward: [(0, '27.600'), (1, '31.770')] [2023-10-08 10:40:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000072480_74219520.pth... [2023-10-08 10:40:07,063][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000070784_72482816.pth [2023-10-08 10:40:07,275][53852] Updated weights for policy 0, policy_version 72820 (0.0008) [2023-10-08 10:40:07,647][53852] Updated weights for policy 0, policy_version 72830 (0.0008) [2023-10-08 10:40:07,718][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000072832_74579968.pth... [2023-10-08 10:40:07,757][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000071104_72810496.pth [2023-10-08 10:40:07,861][53885] Updated weights for policy 1, policy_version 72482 (0.0010) [2023-10-08 10:40:08,216][53885] Updated weights for policy 1, policy_version 72492 (0.0009) [2023-10-08 10:40:08,581][53885] Updated weights for policy 1, policy_version 72502 (0.0007) [2023-10-08 10:40:08,950][53885] Updated weights for policy 1, policy_version 72512 (0.0008) [2023-10-08 10:40:11,397][53852] Updated weights for policy 0, policy_version 72840 (0.0008) [2023-10-08 10:40:11,764][53852] Updated weights for policy 0, policy_version 72850 (0.0008) [2023-10-08 10:40:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 148832256. Throughput: 0: 1824.0, 1: 1826.1. Samples: 37217508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:40:12,016][52710] Avg episode reward: [(0, '31.390'), (1, '30.680')] [2023-10-08 10:40:12,139][53852] Updated weights for policy 0, policy_version 72860 (0.0007) [2023-10-08 10:40:12,726][53885] Updated weights for policy 1, policy_version 72522 (0.0007) [2023-10-08 10:40:13,100][53885] Updated weights for policy 1, policy_version 72532 (0.0007) [2023-10-08 10:40:13,465][53885] Updated weights for policy 1, policy_version 72542 (0.0007) [2023-10-08 10:40:15,842][53852] Updated weights for policy 0, policy_version 72870 (0.0007) [2023-10-08 10:40:16,206][53852] Updated weights for policy 0, policy_version 72880 (0.0008) [2023-10-08 10:40:16,580][53852] Updated weights for policy 0, policy_version 72890 (0.0008) [2023-10-08 10:40:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148930560. Throughput: 0: 1815.5, 1: 1824.2. Samples: 37239992. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:17,016][52710] Avg episode reward: [(0, '28.920'), (1, '33.140')] [2023-10-08 10:40:17,189][53885] Updated weights for policy 1, policy_version 72552 (0.0008) [2023-10-08 10:40:17,557][53885] Updated weights for policy 1, policy_version 72562 (0.0009) [2023-10-08 10:40:17,926][53885] Updated weights for policy 1, policy_version 72572 (0.0008) [2023-10-08 10:40:20,141][53852] Updated weights for policy 0, policy_version 72900 (0.0007) [2023-10-08 10:40:20,511][53852] Updated weights for policy 0, policy_version 72910 (0.0008) [2023-10-08 10:40:20,870][53852] Updated weights for policy 0, policy_version 72920 (0.0010) [2023-10-08 10:40:21,532][53885] Updated weights for policy 1, policy_version 72582 (0.0008) [2023-10-08 10:40:21,899][53885] Updated weights for policy 1, policy_version 72592 (0.0009) [2023-10-08 10:40:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 148996096. Throughput: 0: 1821.4, 1: 1823.7. Samples: 37261298. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:22,016][52710] Avg episode reward: [(0, '29.340'), (1, '33.670')] [2023-10-08 10:40:22,266][53885] Updated weights for policy 1, policy_version 72602 (0.0009) [2023-10-08 10:40:24,415][53852] Updated weights for policy 0, policy_version 72930 (0.0010) [2023-10-08 10:40:24,784][53852] Updated weights for policy 0, policy_version 72940 (0.0008) [2023-10-08 10:40:25,149][53852] Updated weights for policy 0, policy_version 72950 (0.0007) [2023-10-08 10:40:25,515][53852] Updated weights for policy 0, policy_version 72960 (0.0009) [2023-10-08 10:40:25,991][53885] Updated weights for policy 1, policy_version 72612 (0.0009) [2023-10-08 10:40:26,364][53885] Updated weights for policy 1, policy_version 72622 (0.0007) [2023-10-08 10:40:26,733][53885] Updated weights for policy 1, policy_version 72632 (0.0007) [2023-10-08 10:40:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149061632. Throughput: 0: 1813.6, 1: 1826.9. Samples: 37272926. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:27,016][52710] Avg episode reward: [(0, '30.610'), (1, '31.720')] [2023-10-08 10:40:29,248][53852] Updated weights for policy 0, policy_version 72970 (0.0008) [2023-10-08 10:40:29,626][53852] Updated weights for policy 0, policy_version 72980 (0.0008) [2023-10-08 10:40:30,000][53852] Updated weights for policy 0, policy_version 72990 (0.0010) [2023-10-08 10:40:30,278][53885] Updated weights for policy 1, policy_version 72642 (0.0008) [2023-10-08 10:40:30,644][53885] Updated weights for policy 1, policy_version 72652 (0.0008) [2023-10-08 10:40:31,018][53885] Updated weights for policy 1, policy_version 72662 (0.0008) [2023-10-08 10:40:31,382][53885] Updated weights for policy 1, policy_version 72672 (0.0009) [2023-10-08 10:40:32,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149159936. Throughput: 0: 1820.3, 1: 1823.6. Samples: 37294228. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:32,016][52710] Avg episode reward: [(0, '27.910'), (1, '34.570')] [2023-10-08 10:40:33,568][53852] Updated weights for policy 0, policy_version 73000 (0.0010) [2023-10-08 10:40:33,928][53852] Updated weights for policy 0, policy_version 73010 (0.0011) [2023-10-08 10:40:34,303][53852] Updated weights for policy 0, policy_version 73020 (0.0010) [2023-10-08 10:40:35,061][53885] Updated weights for policy 1, policy_version 72682 (0.0009) [2023-10-08 10:40:35,435][53885] Updated weights for policy 1, policy_version 72692 (0.0009) [2023-10-08 10:40:35,795][53885] Updated weights for policy 1, policy_version 72702 (0.0011) [2023-10-08 10:40:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149225472. Throughput: 0: 1827.7, 1: 1828.4. Samples: 37316386. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:37,016][52710] Avg episode reward: [(0, '30.150'), (1, '35.930')] [2023-10-08 10:40:38,050][53852] Updated weights for policy 0, policy_version 73030 (0.0009) [2023-10-08 10:40:38,418][53852] Updated weights for policy 0, policy_version 73040 (0.0009) [2023-10-08 10:40:38,789][53852] Updated weights for policy 0, policy_version 73050 (0.0009) [2023-10-08 10:40:39,408][53885] Updated weights for policy 1, policy_version 72712 (0.0007) [2023-10-08 10:40:39,765][53885] Updated weights for policy 1, policy_version 72722 (0.0009) [2023-10-08 10:40:40,133][53885] Updated weights for policy 1, policy_version 72732 (0.0011) [2023-10-08 10:40:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149291008. Throughput: 0: 1834.0, 1: 1825.0. Samples: 37327430. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:42,015][52710] Avg episode reward: [(0, '31.500'), (1, '32.580')] [2023-10-08 10:40:42,522][53852] Updated weights for policy 0, policy_version 73060 (0.0010) [2023-10-08 10:40:42,884][53852] Updated weights for policy 0, policy_version 73070 (0.0009) [2023-10-08 10:40:43,254][53852] Updated weights for policy 0, policy_version 73080 (0.0009) [2023-10-08 10:40:43,831][53885] Updated weights for policy 1, policy_version 72742 (0.0009) [2023-10-08 10:40:44,196][53885] Updated weights for policy 1, policy_version 72752 (0.0007) [2023-10-08 10:40:44,572][53885] Updated weights for policy 1, policy_version 72762 (0.0008) [2023-10-08 10:40:46,756][53852] Updated weights for policy 0, policy_version 73090 (0.0007) [2023-10-08 10:40:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 149356544. Throughput: 0: 1837.2, 1: 1830.2. Samples: 37349746. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:47,016][52710] Avg episode reward: [(0, '32.860'), (1, '33.510')] [2023-10-08 10:40:47,135][53852] Updated weights for policy 0, policy_version 73100 (0.0008) [2023-10-08 10:40:47,498][53852] Updated weights for policy 0, policy_version 73110 (0.0007) [2023-10-08 10:40:47,870][53852] Updated weights for policy 0, policy_version 73120 (0.0007) [2023-10-08 10:40:48,214][53885] Updated weights for policy 1, policy_version 72772 (0.0007) [2023-10-08 10:40:48,593][53885] Updated weights for policy 1, policy_version 72782 (0.0007) [2023-10-08 10:40:48,966][53885] Updated weights for policy 1, policy_version 72792 (0.0007) [2023-10-08 10:40:51,381][53852] Updated weights for policy 0, policy_version 73130 (0.0009) [2023-10-08 10:40:51,739][53852] Updated weights for policy 0, policy_version 73140 (0.0010) [2023-10-08 10:40:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 149422080. Throughput: 0: 1835.0, 1: 1830.5. Samples: 37372382. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:52,016][52710] Avg episode reward: [(0, '33.030'), (1, '37.710')] [2023-10-08 10:40:52,101][53852] Updated weights for policy 0, policy_version 73150 (0.0011) [2023-10-08 10:40:52,573][53885] Updated weights for policy 1, policy_version 72802 (0.0008) [2023-10-08 10:40:52,941][53885] Updated weights for policy 1, policy_version 72812 (0.0008) [2023-10-08 10:40:53,310][53885] Updated weights for policy 1, policy_version 72822 (0.0007) [2023-10-08 10:40:53,680][53885] Updated weights for policy 1, policy_version 72832 (0.0007) [2023-10-08 10:40:55,654][53852] Updated weights for policy 0, policy_version 73160 (0.0009) [2023-10-08 10:40:56,029][53852] Updated weights for policy 0, policy_version 73170 (0.0008) [2023-10-08 10:40:56,396][53852] Updated weights for policy 0, policy_version 73180 (0.0007) [2023-10-08 10:40:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 149520384. Throughput: 0: 1849.5, 1: 1832.4. Samples: 37383190. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:40:57,016][52710] Avg episode reward: [(0, '32.820'), (1, '33.990')] [2023-10-08 10:40:57,281][53885] Updated weights for policy 1, policy_version 72842 (0.0007) [2023-10-08 10:40:57,656][53885] Updated weights for policy 1, policy_version 72852 (0.0008) [2023-10-08 10:40:58,018][53885] Updated weights for policy 1, policy_version 72862 (0.0008) [2023-10-08 10:40:59,962][53852] Updated weights for policy 0, policy_version 73190 (0.0009) [2023-10-08 10:41:00,341][53852] Updated weights for policy 0, policy_version 73200 (0.0008) [2023-10-08 10:41:00,716][53852] Updated weights for policy 0, policy_version 73210 (0.0007) [2023-10-08 10:41:01,794][53885] Updated weights for policy 1, policy_version 72872 (0.0011) [2023-10-08 10:41:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149585920. Throughput: 0: 1839.2, 1: 1833.1. Samples: 37405244. Policy #0 lag: (min: 31.0, avg: 32.7, max: 58.0) [2023-10-08 10:41:02,016][52710] Avg episode reward: [(0, '30.420'), (1, '33.440')] [2023-10-08 10:41:02,151][53885] Updated weights for policy 1, policy_version 72882 (0.0010) [2023-10-08 10:41:02,520][53885] Updated weights for policy 1, policy_version 72892 (0.0009) [2023-10-08 10:41:04,393][53852] Updated weights for policy 0, policy_version 73220 (0.0009) [2023-10-08 10:41:04,775][53852] Updated weights for policy 0, policy_version 73230 (0.0007) [2023-10-08 10:41:05,143][53852] Updated weights for policy 0, policy_version 73240 (0.0007) [2023-10-08 10:41:06,381][53885] Updated weights for policy 1, policy_version 72902 (0.0010) [2023-10-08 10:41:06,741][53885] Updated weights for policy 1, policy_version 72912 (0.0011) [2023-10-08 10:41:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149651456. Throughput: 0: 1863.7, 1: 1825.3. Samples: 37427304. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:07,016][52710] Avg episode reward: [(0, '32.570'), (1, '34.850')] [2023-10-08 10:41:07,105][53885] Updated weights for policy 1, policy_version 72922 (0.0010) [2023-10-08 10:41:08,806][53852] Updated weights for policy 0, policy_version 73250 (0.0010) [2023-10-08 10:41:09,174][53852] Updated weights for policy 0, policy_version 73260 (0.0009) [2023-10-08 10:41:09,549][53852] Updated weights for policy 0, policy_version 73270 (0.0008) [2023-10-08 10:41:09,919][53852] Updated weights for policy 0, policy_version 73280 (0.0010) [2023-10-08 10:41:10,848][53885] Updated weights for policy 1, policy_version 72932 (0.0009) [2023-10-08 10:41:11,214][53885] Updated weights for policy 1, policy_version 72942 (0.0007) [2023-10-08 10:41:11,578][53885] Updated weights for policy 1, policy_version 72952 (0.0007) [2023-10-08 10:41:12,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 149749760. Throughput: 0: 1845.5, 1: 1828.1. Samples: 37438236. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:12,016][52710] Avg episode reward: [(0, '30.450'), (1, '34.270')] [2023-10-08 10:41:13,536][53852] Updated weights for policy 0, policy_version 73290 (0.0009) [2023-10-08 10:41:13,895][53852] Updated weights for policy 0, policy_version 73300 (0.0010) [2023-10-08 10:41:14,261][53852] Updated weights for policy 0, policy_version 73310 (0.0011) [2023-10-08 10:41:15,208][53885] Updated weights for policy 1, policy_version 72962 (0.0007) [2023-10-08 10:41:15,579][53885] Updated weights for policy 1, policy_version 72972 (0.0009) [2023-10-08 10:41:15,944][53885] Updated weights for policy 1, policy_version 72982 (0.0008) [2023-10-08 10:41:16,306][53885] Updated weights for policy 1, policy_version 72992 (0.0008) [2023-10-08 10:41:17,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 149815296. Throughput: 0: 1866.0, 1: 1821.9. Samples: 37460184. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:17,015][52710] Avg episode reward: [(0, '31.780'), (1, '33.600')] [2023-10-08 10:41:17,841][53852] Updated weights for policy 0, policy_version 73320 (0.0007) [2023-10-08 10:41:18,206][53852] Updated weights for policy 0, policy_version 73330 (0.0009) [2023-10-08 10:41:18,587][53852] Updated weights for policy 0, policy_version 73340 (0.0008) [2023-10-08 10:41:20,070][53885] Updated weights for policy 1, policy_version 73002 (0.0009) [2023-10-08 10:41:20,431][53885] Updated weights for policy 1, policy_version 73012 (0.0008) [2023-10-08 10:41:20,796][53885] Updated weights for policy 1, policy_version 73022 (0.0009) [2023-10-08 10:41:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149880832. Throughput: 0: 1868.1, 1: 1822.9. Samples: 37482480. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:22,016][52710] Avg episode reward: [(0, '29.130'), (1, '33.870')] [2023-10-08 10:41:22,212][53852] Updated weights for policy 0, policy_version 73350 (0.0009) [2023-10-08 10:41:22,582][53852] Updated weights for policy 0, policy_version 73360 (0.0007) [2023-10-08 10:41:22,951][53852] Updated weights for policy 0, policy_version 73370 (0.0007) [2023-10-08 10:41:24,499][53885] Updated weights for policy 1, policy_version 73032 (0.0008) [2023-10-08 10:41:24,867][53885] Updated weights for policy 1, policy_version 73042 (0.0009) [2023-10-08 10:41:25,226][53885] Updated weights for policy 1, policy_version 73052 (0.0008) [2023-10-08 10:41:26,481][53852] Updated weights for policy 0, policy_version 73380 (0.0008) [2023-10-08 10:41:26,853][53852] Updated weights for policy 0, policy_version 73390 (0.0008) [2023-10-08 10:41:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149946368. Throughput: 0: 1867.7, 1: 1824.6. Samples: 37493584. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:27,016][52710] Avg episode reward: [(0, '33.620'), (1, '30.840')] [2023-10-08 10:41:27,212][53852] Updated weights for policy 0, policy_version 73400 (0.0007) [2023-10-08 10:41:28,870][53885] Updated weights for policy 1, policy_version 73062 (0.0008) [2023-10-08 10:41:29,237][53885] Updated weights for policy 1, policy_version 73072 (0.0008) [2023-10-08 10:41:29,614][53885] Updated weights for policy 1, policy_version 73082 (0.0007) [2023-10-08 10:41:30,854][53852] Updated weights for policy 0, policy_version 73410 (0.0007) [2023-10-08 10:41:31,238][53852] Updated weights for policy 0, policy_version 73420 (0.0008) [2023-10-08 10:41:31,617][53852] Updated weights for policy 0, policy_version 73430 (0.0008) [2023-10-08 10:41:31,980][53852] Updated weights for policy 0, policy_version 73440 (0.0010) [2023-10-08 10:41:32,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 150044672. Throughput: 0: 1865.1, 1: 1827.5. Samples: 37515910. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:32,016][52710] Avg episode reward: [(0, '30.430'), (1, '33.030')] [2023-10-08 10:41:33,231][53885] Updated weights for policy 1, policy_version 73092 (0.0009) [2023-10-08 10:41:33,625][53885] Updated weights for policy 1, policy_version 73102 (0.0011) [2023-10-08 10:41:33,995][53885] Updated weights for policy 1, policy_version 73112 (0.0011) [2023-10-08 10:41:35,406][53852] Updated weights for policy 0, policy_version 73450 (0.0009) [2023-10-08 10:41:35,773][53852] Updated weights for policy 0, policy_version 73460 (0.0009) [2023-10-08 10:41:36,143][53852] Updated weights for policy 0, policy_version 73470 (0.0007) [2023-10-08 10:41:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 150110208. Throughput: 0: 1840.7, 1: 1826.9. Samples: 37537428. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:37,016][52710] Avg episode reward: [(0, '32.610'), (1, '35.580')] [2023-10-08 10:41:37,655][53885] Updated weights for policy 1, policy_version 73122 (0.0011) [2023-10-08 10:41:38,022][53885] Updated weights for policy 1, policy_version 73132 (0.0009) [2023-10-08 10:41:38,389][53885] Updated weights for policy 1, policy_version 73142 (0.0009) [2023-10-08 10:41:38,750][53885] Updated weights for policy 1, policy_version 73152 (0.0008) [2023-10-08 10:41:39,827][53852] Updated weights for policy 0, policy_version 73480 (0.0010) [2023-10-08 10:41:40,197][53852] Updated weights for policy 0, policy_version 73490 (0.0011) [2023-10-08 10:41:40,567][53852] Updated weights for policy 0, policy_version 73500 (0.0011) [2023-10-08 10:41:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150175744. Throughput: 0: 1864.4, 1: 1824.6. Samples: 37549196. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:42,016][52710] Avg episode reward: [(0, '33.900'), (1, '32.450')] [2023-10-08 10:41:42,486][53885] Updated weights for policy 1, policy_version 73162 (0.0008) [2023-10-08 10:41:42,846][53885] Updated weights for policy 1, policy_version 73172 (0.0008) [2023-10-08 10:41:43,216][53885] Updated weights for policy 1, policy_version 73182 (0.0008) [2023-10-08 10:41:44,209][53852] Updated weights for policy 0, policy_version 73510 (0.0008) [2023-10-08 10:41:44,581][53852] Updated weights for policy 0, policy_version 73520 (0.0007) [2023-10-08 10:41:44,949][53852] Updated weights for policy 0, policy_version 73530 (0.0007) [2023-10-08 10:41:46,882][53885] Updated weights for policy 1, policy_version 73192 (0.0009) [2023-10-08 10:41:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150241280. Throughput: 0: 1849.6, 1: 1829.3. Samples: 37570794. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:47,015][52710] Avg episode reward: [(0, '30.200'), (1, '35.050')] [2023-10-08 10:41:47,251][53885] Updated weights for policy 1, policy_version 73202 (0.0007) [2023-10-08 10:41:47,607][53885] Updated weights for policy 1, policy_version 73212 (0.0010) [2023-10-08 10:41:48,360][53852] Updated weights for policy 0, policy_version 73540 (0.0008) [2023-10-08 10:41:48,738][53852] Updated weights for policy 0, policy_version 73550 (0.0009) [2023-10-08 10:41:49,101][53852] Updated weights for policy 0, policy_version 73560 (0.0008) [2023-10-08 10:41:51,294][53885] Updated weights for policy 1, policy_version 73222 (0.0010) [2023-10-08 10:41:51,659][53885] Updated weights for policy 1, policy_version 73232 (0.0008) [2023-10-08 10:41:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150306816. Throughput: 0: 1858.2, 1: 1821.8. Samples: 37592904. Policy #0 lag: (min: 10.0, avg: 10.5, max: 25.0) [2023-10-08 10:41:52,016][52710] Avg episode reward: [(0, '31.270'), (1, '33.520')] [2023-10-08 10:41:52,030][53885] Updated weights for policy 1, policy_version 73242 (0.0007) [2023-10-08 10:41:52,711][53852] Updated weights for policy 0, policy_version 73570 (0.0007) [2023-10-08 10:41:53,076][53852] Updated weights for policy 0, policy_version 73580 (0.0007) [2023-10-08 10:41:53,449][53852] Updated weights for policy 0, policy_version 73590 (0.0007) [2023-10-08 10:41:53,820][53852] Updated weights for policy 0, policy_version 73600 (0.0008) [2023-10-08 10:41:55,674][53885] Updated weights for policy 1, policy_version 73252 (0.0007) [2023-10-08 10:41:56,030][53885] Updated weights for policy 1, policy_version 73262 (0.0009) [2023-10-08 10:41:56,397][53885] Updated weights for policy 1, policy_version 73272 (0.0012) [2023-10-08 10:41:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150405120. Throughput: 0: 1846.2, 1: 1827.7. Samples: 37603560. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:41:57,015][52710] Avg episode reward: [(0, '32.800'), (1, '29.100')] [2023-10-08 10:41:57,597][53852] Updated weights for policy 0, policy_version 73610 (0.0008) [2023-10-08 10:41:57,966][53852] Updated weights for policy 0, policy_version 73620 (0.0007) [2023-10-08 10:41:58,338][53852] Updated weights for policy 0, policy_version 73630 (0.0008) [2023-10-08 10:42:00,101][53885] Updated weights for policy 1, policy_version 73282 (0.0008) [2023-10-08 10:42:00,465][53885] Updated weights for policy 1, policy_version 73292 (0.0010) [2023-10-08 10:42:00,828][53885] Updated weights for policy 1, policy_version 73302 (0.0007) [2023-10-08 10:42:01,204][53885] Updated weights for policy 1, policy_version 73312 (0.0009) [2023-10-08 10:42:01,848][53852] Updated weights for policy 0, policy_version 73640 (0.0010) [2023-10-08 10:42:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150470656. Throughput: 0: 1856.4, 1: 1824.6. Samples: 37625826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:02,016][52710] Avg episode reward: [(0, '31.770'), (1, '30.320')] [2023-10-08 10:42:02,219][53852] Updated weights for policy 0, policy_version 73650 (0.0008) [2023-10-08 10:42:02,583][53852] Updated weights for policy 0, policy_version 73660 (0.0009) [2023-10-08 10:42:04,976][53885] Updated weights for policy 1, policy_version 73322 (0.0008) [2023-10-08 10:42:05,338][53885] Updated weights for policy 1, policy_version 73332 (0.0009) [2023-10-08 10:42:05,704][53885] Updated weights for policy 1, policy_version 73342 (0.0009) [2023-10-08 10:42:06,311][53852] Updated weights for policy 0, policy_version 73670 (0.0009) [2023-10-08 10:42:06,683][53852] Updated weights for policy 0, policy_version 73680 (0.0007) [2023-10-08 10:42:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150536192. Throughput: 0: 1837.0, 1: 1830.8. Samples: 37647530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:07,016][52710] Avg episode reward: [(0, '31.940'), (1, '31.140')] [2023-10-08 10:42:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000073344_75104256.pth... [2023-10-08 10:42:07,055][53852] Updated weights for policy 0, policy_version 73690 (0.0007) [2023-10-08 10:42:07,062][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000071648_73367552.pth [2023-10-08 10:42:07,271][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000073696_75464704.pth... [2023-10-08 10:42:07,310][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000071968_73695232.pth [2023-10-08 10:42:09,227][53885] Updated weights for policy 1, policy_version 73352 (0.0010) [2023-10-08 10:42:09,598][53885] Updated weights for policy 1, policy_version 73362 (0.0007) [2023-10-08 10:42:09,955][53885] Updated weights for policy 1, policy_version 73372 (0.0009) [2023-10-08 10:42:10,705][53852] Updated weights for policy 0, policy_version 73700 (0.0009) [2023-10-08 10:42:11,067][53852] Updated weights for policy 0, policy_version 73710 (0.0009) [2023-10-08 10:42:11,439][53852] Updated weights for policy 0, policy_version 73720 (0.0009) [2023-10-08 10:42:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 150634496. Throughput: 0: 1851.2, 1: 1826.2. Samples: 37659068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:12,016][52710] Avg episode reward: [(0, '32.640'), (1, '32.170')] [2023-10-08 10:42:13,654][53885] Updated weights for policy 1, policy_version 73382 (0.0010) [2023-10-08 10:42:14,029][53885] Updated weights for policy 1, policy_version 73392 (0.0011) [2023-10-08 10:42:14,390][53885] Updated weights for policy 1, policy_version 73402 (0.0011) [2023-10-08 10:42:15,258][53852] Updated weights for policy 0, policy_version 73730 (0.0008) [2023-10-08 10:42:15,666][53852] Updated weights for policy 0, policy_version 73740 (0.0007) [2023-10-08 10:42:16,039][53852] Updated weights for policy 0, policy_version 73750 (0.0007) [2023-10-08 10:42:16,402][53852] Updated weights for policy 0, policy_version 73760 (0.0008) [2023-10-08 10:42:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 150700032. Throughput: 0: 1834.7, 1: 1826.3. Samples: 37680656. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:17,016][52710] Avg episode reward: [(0, '32.760'), (1, '31.860')] [2023-10-08 10:42:18,030][53885] Updated weights for policy 1, policy_version 73412 (0.0008) [2023-10-08 10:42:18,408][53885] Updated weights for policy 1, policy_version 73422 (0.0008) [2023-10-08 10:42:18,767][53885] Updated weights for policy 1, policy_version 73432 (0.0008) [2023-10-08 10:42:19,985][53852] Updated weights for policy 0, policy_version 73770 (0.0010) [2023-10-08 10:42:20,352][53852] Updated weights for policy 0, policy_version 73780 (0.0009) [2023-10-08 10:42:20,716][53852] Updated weights for policy 0, policy_version 73790 (0.0008) [2023-10-08 10:42:22,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150765568. Throughput: 0: 1844.6, 1: 1829.7. Samples: 37702774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:22,016][52710] Avg episode reward: [(0, '29.080'), (1, '33.980')] [2023-10-08 10:42:22,405][53885] Updated weights for policy 1, policy_version 73442 (0.0008) [2023-10-08 10:42:22,782][53885] Updated weights for policy 1, policy_version 73452 (0.0008) [2023-10-08 10:42:23,140][53885] Updated weights for policy 1, policy_version 73462 (0.0009) [2023-10-08 10:42:23,507][53885] Updated weights for policy 1, policy_version 73472 (0.0011) [2023-10-08 10:42:24,322][53852] Updated weights for policy 0, policy_version 73800 (0.0008) [2023-10-08 10:42:24,687][53852] Updated weights for policy 0, policy_version 73810 (0.0007) [2023-10-08 10:42:25,052][53852] Updated weights for policy 0, policy_version 73820 (0.0009) [2023-10-08 10:42:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150831104. Throughput: 0: 1824.5, 1: 1831.2. Samples: 37713704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:27,015][52710] Avg episode reward: [(0, '33.680'), (1, '33.060')] [2023-10-08 10:42:27,103][53885] Updated weights for policy 1, policy_version 73482 (0.0009) [2023-10-08 10:42:27,462][53885] Updated weights for policy 1, policy_version 73492 (0.0010) [2023-10-08 10:42:27,824][53885] Updated weights for policy 1, policy_version 73502 (0.0009) [2023-10-08 10:42:28,635][53852] Updated weights for policy 0, policy_version 73830 (0.0007) [2023-10-08 10:42:29,011][53852] Updated weights for policy 0, policy_version 73840 (0.0009) [2023-10-08 10:42:29,385][53852] Updated weights for policy 0, policy_version 73850 (0.0009) [2023-10-08 10:42:31,627][53885] Updated weights for policy 1, policy_version 73512 (0.0010) [2023-10-08 10:42:31,993][53885] Updated weights for policy 1, policy_version 73522 (0.0009) [2023-10-08 10:42:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150896640. Throughput: 0: 1840.5, 1: 1831.9. Samples: 37736050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:32,016][52710] Avg episode reward: [(0, '31.400'), (1, '36.530')] [2023-10-08 10:42:32,354][53885] Updated weights for policy 1, policy_version 73532 (0.0007) [2023-10-08 10:42:33,075][53852] Updated weights for policy 0, policy_version 73860 (0.0008) [2023-10-08 10:42:33,442][53852] Updated weights for policy 0, policy_version 73870 (0.0010) [2023-10-08 10:42:33,809][53852] Updated weights for policy 0, policy_version 73880 (0.0010) [2023-10-08 10:42:36,032][53885] Updated weights for policy 1, policy_version 73542 (0.0007) [2023-10-08 10:42:36,397][53885] Updated weights for policy 1, policy_version 73552 (0.0009) [2023-10-08 10:42:36,774][53885] Updated weights for policy 1, policy_version 73562 (0.0007) [2023-10-08 10:42:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150994944. Throughput: 0: 1841.9, 1: 1827.6. Samples: 37758030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:37,016][52710] Avg episode reward: [(0, '28.450'), (1, '30.900')] [2023-10-08 10:42:37,421][53852] Updated weights for policy 0, policy_version 73890 (0.0009) [2023-10-08 10:42:37,786][53852] Updated weights for policy 0, policy_version 73900 (0.0008) [2023-10-08 10:42:38,159][53852] Updated weights for policy 0, policy_version 73910 (0.0009) [2023-10-08 10:42:38,531][53852] Updated weights for policy 0, policy_version 73920 (0.0009) [2023-10-08 10:42:40,422][53885] Updated weights for policy 1, policy_version 73572 (0.0009) [2023-10-08 10:42:40,782][53885] Updated weights for policy 1, policy_version 73582 (0.0011) [2023-10-08 10:42:41,158][53885] Updated weights for policy 1, policy_version 73592 (0.0007) [2023-10-08 10:42:42,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151060480. Throughput: 0: 1846.2, 1: 1832.7. Samples: 37769108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:42,016][52710] Avg episode reward: [(0, '31.160'), (1, '34.440')] [2023-10-08 10:42:42,144][53852] Updated weights for policy 0, policy_version 73930 (0.0008) [2023-10-08 10:42:42,511][53852] Updated weights for policy 0, policy_version 73940 (0.0008) [2023-10-08 10:42:42,892][53852] Updated weights for policy 0, policy_version 73950 (0.0009) [2023-10-08 10:42:44,717][53885] Updated weights for policy 1, policy_version 73602 (0.0007) [2023-10-08 10:42:45,079][53885] Updated weights for policy 1, policy_version 73612 (0.0009) [2023-10-08 10:42:45,443][53885] Updated weights for policy 1, policy_version 73622 (0.0008) [2023-10-08 10:42:45,810][53885] Updated weights for policy 1, policy_version 73632 (0.0009) [2023-10-08 10:42:46,543][53852] Updated weights for policy 0, policy_version 73960 (0.0010) [2023-10-08 10:42:46,921][53852] Updated weights for policy 0, policy_version 73970 (0.0009) [2023-10-08 10:42:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151126016. Throughput: 0: 1846.3, 1: 1828.4. Samples: 37791186. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:47,015][52710] Avg episode reward: [(0, '32.880'), (1, '34.570')] [2023-10-08 10:42:47,286][53852] Updated weights for policy 0, policy_version 73980 (0.0007) [2023-10-08 10:42:49,400][53885] Updated weights for policy 1, policy_version 73642 (0.0008) [2023-10-08 10:42:49,765][53885] Updated weights for policy 1, policy_version 73652 (0.0009) [2023-10-08 10:42:50,134][53885] Updated weights for policy 1, policy_version 73662 (0.0010) [2023-10-08 10:42:50,866][53852] Updated weights for policy 0, policy_version 73990 (0.0010) [2023-10-08 10:42:51,241][53852] Updated weights for policy 0, policy_version 74000 (0.0007) [2023-10-08 10:42:51,611][53852] Updated weights for policy 0, policy_version 74010 (0.0008) [2023-10-08 10:42:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 151224320. Throughput: 0: 1838.5, 1: 1833.1. Samples: 37812754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:52,016][52710] Avg episode reward: [(0, '28.920'), (1, '32.960')] [2023-10-08 10:42:53,927][53885] Updated weights for policy 1, policy_version 73672 (0.0008) [2023-10-08 10:42:54,297][53885] Updated weights for policy 1, policy_version 73682 (0.0008) [2023-10-08 10:42:54,656][53885] Updated weights for policy 1, policy_version 73692 (0.0009) [2023-10-08 10:42:55,291][53852] Updated weights for policy 0, policy_version 74020 (0.0008) [2023-10-08 10:42:55,657][53852] Updated weights for policy 0, policy_version 74030 (0.0009) [2023-10-08 10:42:56,023][53852] Updated weights for policy 0, policy_version 74040 (0.0011) [2023-10-08 10:42:57,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 151289856. Throughput: 0: 1848.7, 1: 1819.6. Samples: 37824144. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:42:57,016][52710] Avg episode reward: [(0, '29.630'), (1, '32.870')] [2023-10-08 10:42:58,230][53885] Updated weights for policy 1, policy_version 73702 (0.0007) [2023-10-08 10:42:58,596][53885] Updated weights for policy 1, policy_version 73712 (0.0007) [2023-10-08 10:42:58,954][53885] Updated weights for policy 1, policy_version 73722 (0.0008) [2023-10-08 10:42:59,738][53852] Updated weights for policy 0, policy_version 74050 (0.0009) [2023-10-08 10:43:00,099][53852] Updated weights for policy 0, policy_version 74060 (0.0009) [2023-10-08 10:43:00,465][53852] Updated weights for policy 0, policy_version 74070 (0.0010) [2023-10-08 10:43:00,837][53852] Updated weights for policy 0, policy_version 74080 (0.0010) [2023-10-08 10:43:02,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151355392. Throughput: 0: 1834.2, 1: 1838.1. Samples: 37845910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:43:02,016][52710] Avg episode reward: [(0, '36.150'), (1, '35.450')] [2023-10-08 10:43:02,644][53885] Updated weights for policy 1, policy_version 73732 (0.0009) [2023-10-08 10:43:03,010][53885] Updated weights for policy 1, policy_version 73742 (0.0007) [2023-10-08 10:43:03,376][53885] Updated weights for policy 1, policy_version 73752 (0.0007) [2023-10-08 10:43:04,555][53852] Updated weights for policy 0, policy_version 74090 (0.0009) [2023-10-08 10:43:04,929][53852] Updated weights for policy 0, policy_version 74100 (0.0007) [2023-10-08 10:43:05,295][53852] Updated weights for policy 0, policy_version 74110 (0.0007) [2023-10-08 10:43:06,989][53885] Updated weights for policy 1, policy_version 73762 (0.0008) [2023-10-08 10:43:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151420928. Throughput: 0: 1847.5, 1: 1838.4. Samples: 37868638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:43:07,016][52710] Avg episode reward: [(0, '34.470'), (1, '33.270')] [2023-10-08 10:43:07,352][53885] Updated weights for policy 1, policy_version 73772 (0.0008) [2023-10-08 10:43:07,725][53885] Updated weights for policy 1, policy_version 73782 (0.0007) [2023-10-08 10:43:08,090][53885] Updated weights for policy 1, policy_version 73792 (0.0008) [2023-10-08 10:43:08,751][53852] Updated weights for policy 0, policy_version 74120 (0.0007) [2023-10-08 10:43:09,120][53852] Updated weights for policy 0, policy_version 74130 (0.0008) [2023-10-08 10:43:09,487][53852] Updated weights for policy 0, policy_version 74140 (0.0009) [2023-10-08 10:43:11,747][53885] Updated weights for policy 1, policy_version 73802 (0.0009) [2023-10-08 10:43:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 151486464. Throughput: 0: 1837.5, 1: 1839.1. Samples: 37879156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:43:12,016][52710] Avg episode reward: [(0, '32.480'), (1, '32.740')] [2023-10-08 10:43:12,107][53885] Updated weights for policy 1, policy_version 73812 (0.0009) [2023-10-08 10:43:12,475][53885] Updated weights for policy 1, policy_version 73822 (0.0008) [2023-10-08 10:43:13,278][53852] Updated weights for policy 0, policy_version 74150 (0.0008) [2023-10-08 10:43:13,653][53852] Updated weights for policy 0, policy_version 74160 (0.0008) [2023-10-08 10:43:14,019][53852] Updated weights for policy 0, policy_version 74170 (0.0009) [2023-10-08 10:43:16,259][53885] Updated weights for policy 1, policy_version 73832 (0.0008) [2023-10-08 10:43:16,632][53885] Updated weights for policy 1, policy_version 73842 (0.0008) [2023-10-08 10:43:16,991][53885] Updated weights for policy 1, policy_version 73852 (0.0007) [2023-10-08 10:43:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151552000. Throughput: 0: 1846.6, 1: 1835.8. Samples: 37901758. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:43:17,016][52710] Avg episode reward: [(0, '35.370'), (1, '32.860')] [2023-10-08 10:43:17,595][53852] Updated weights for policy 0, policy_version 74180 (0.0008) [2023-10-08 10:43:17,969][53852] Updated weights for policy 0, policy_version 74190 (0.0007) [2023-10-08 10:43:18,328][53852] Updated weights for policy 0, policy_version 74200 (0.0010) [2023-10-08 10:43:20,738][53885] Updated weights for policy 1, policy_version 73862 (0.0007) [2023-10-08 10:43:21,104][53885] Updated weights for policy 1, policy_version 73872 (0.0007) [2023-10-08 10:43:21,479][53885] Updated weights for policy 1, policy_version 73882 (0.0007) [2023-10-08 10:43:21,930][53852] Updated weights for policy 0, policy_version 74210 (0.0009) [2023-10-08 10:43:22,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151650304. Throughput: 0: 1851.0, 1: 1827.0. Samples: 37923540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:43:22,017][52710] Avg episode reward: [(0, '32.350'), (1, '29.530')] [2023-10-08 10:43:22,302][53852] Updated weights for policy 0, policy_version 74220 (0.0008) [2023-10-08 10:43:22,674][53852] Updated weights for policy 0, policy_version 74230 (0.0008) [2023-10-08 10:43:23,038][53852] Updated weights for policy 0, policy_version 74240 (0.0007) [2023-10-08 10:43:25,080][53885] Updated weights for policy 1, policy_version 73892 (0.0009) [2023-10-08 10:43:25,454][53885] Updated weights for policy 1, policy_version 73902 (0.0009) [2023-10-08 10:43:25,824][53885] Updated weights for policy 1, policy_version 73912 (0.0010) [2023-10-08 10:43:26,676][53852] Updated weights for policy 0, policy_version 74250 (0.0011) [2023-10-08 10:43:27,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 151715840. Throughput: 0: 1846.4, 1: 1838.0. Samples: 37934904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:43:27,016][52710] Avg episode reward: [(0, '34.960'), (1, '31.850')] [2023-10-08 10:43:27,050][53852] Updated weights for policy 0, policy_version 74260 (0.0007) [2023-10-08 10:43:27,415][53852] Updated weights for policy 0, policy_version 74270 (0.0009) [2023-10-08 10:43:29,379][53885] Updated weights for policy 1, policy_version 73922 (0.0009) [2023-10-08 10:43:29,736][53885] Updated weights for policy 1, policy_version 73932 (0.0009) [2023-10-08 10:43:30,099][53885] Updated weights for policy 1, policy_version 73942 (0.0009) [2023-10-08 10:43:30,467][53885] Updated weights for policy 1, policy_version 73952 (0.0009) [2023-10-08 10:43:31,027][53852] Updated weights for policy 0, policy_version 74280 (0.0008) [2023-10-08 10:43:31,402][53852] Updated weights for policy 0, policy_version 74290 (0.0008) [2023-10-08 10:43:31,775][53852] Updated weights for policy 0, policy_version 74300 (0.0009) [2023-10-08 10:43:32,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 151814144. Throughput: 0: 1851.6, 1: 1825.2. Samples: 37956644. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:43:32,016][52710] Avg episode reward: [(0, '33.410'), (1, '29.430')] [2023-10-08 10:43:34,248][53885] Updated weights for policy 1, policy_version 73962 (0.0007) [2023-10-08 10:43:34,625][53885] Updated weights for policy 1, policy_version 73972 (0.0008) [2023-10-08 10:43:34,993][53885] Updated weights for policy 1, policy_version 73982 (0.0008) [2023-10-08 10:43:35,405][53852] Updated weights for policy 0, policy_version 74310 (0.0009) [2023-10-08 10:43:35,768][53852] Updated weights for policy 0, policy_version 74320 (0.0010) [2023-10-08 10:43:36,140][53852] Updated weights for policy 0, policy_version 74330 (0.0009) [2023-10-08 10:43:37,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 151879680. Throughput: 0: 1837.8, 1: 1838.3. Samples: 37978178. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:43:37,016][52710] Avg episode reward: [(0, '34.020'), (1, '30.680')] [2023-10-08 10:43:38,675][53885] Updated weights for policy 1, policy_version 73992 (0.0009) [2023-10-08 10:43:39,056][53885] Updated weights for policy 1, policy_version 74002 (0.0007) [2023-10-08 10:43:39,426][53885] Updated weights for policy 1, policy_version 74012 (0.0010) [2023-10-08 10:43:39,634][53852] Updated weights for policy 0, policy_version 74340 (0.0009) [2023-10-08 10:43:40,002][53852] Updated weights for policy 0, policy_version 74350 (0.0007) [2023-10-08 10:43:40,371][53852] Updated weights for policy 0, policy_version 74360 (0.0009) [2023-10-08 10:43:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 151945216. Throughput: 0: 1852.7, 1: 1828.4. Samples: 37989796. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:43:42,016][52710] Avg episode reward: [(0, '32.520'), (1, '33.480')] [2023-10-08 10:43:43,073][53885] Updated weights for policy 1, policy_version 74022 (0.0010) [2023-10-08 10:43:43,435][53885] Updated weights for policy 1, policy_version 74032 (0.0008) [2023-10-08 10:43:43,808][53885] Updated weights for policy 1, policy_version 74042 (0.0008) [2023-10-08 10:43:44,053][53852] Updated weights for policy 0, policy_version 74370 (0.0011) [2023-10-08 10:43:44,425][53852] Updated weights for policy 0, policy_version 74380 (0.0008) [2023-10-08 10:43:44,788][53852] Updated weights for policy 0, policy_version 74390 (0.0008) [2023-10-08 10:43:45,157][53852] Updated weights for policy 0, policy_version 74400 (0.0009) [2023-10-08 10:43:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 152010752. Throughput: 0: 1842.1, 1: 1825.6. Samples: 38010958. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:43:47,016][52710] Avg episode reward: [(0, '35.210'), (1, '32.060')] [2023-10-08 10:43:47,533][53885] Updated weights for policy 1, policy_version 74052 (0.0008) [2023-10-08 10:43:47,902][53885] Updated weights for policy 1, policy_version 74062 (0.0008) [2023-10-08 10:43:48,262][53885] Updated weights for policy 1, policy_version 74072 (0.0009) [2023-10-08 10:43:48,805][53852] Updated weights for policy 0, policy_version 74410 (0.0007) [2023-10-08 10:43:49,178][53852] Updated weights for policy 0, policy_version 74420 (0.0008) [2023-10-08 10:43:49,540][53852] Updated weights for policy 0, policy_version 74430 (0.0008) [2023-10-08 10:43:51,913][53885] Updated weights for policy 1, policy_version 74082 (0.0007) [2023-10-08 10:43:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 152076288. Throughput: 0: 1849.3, 1: 1824.7. Samples: 38033968. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:43:52,016][52710] Avg episode reward: [(0, '33.720'), (1, '32.290')] [2023-10-08 10:43:52,285][53885] Updated weights for policy 1, policy_version 74092 (0.0007) [2023-10-08 10:43:52,651][53885] Updated weights for policy 1, policy_version 74102 (0.0007) [2023-10-08 10:43:53,026][53885] Updated weights for policy 1, policy_version 74112 (0.0009) [2023-10-08 10:43:53,333][53852] Updated weights for policy 0, policy_version 74440 (0.0007) [2023-10-08 10:43:53,708][53852] Updated weights for policy 0, policy_version 74450 (0.0008) [2023-10-08 10:43:54,069][53852] Updated weights for policy 0, policy_version 74460 (0.0008) [2023-10-08 10:43:56,734][53885] Updated weights for policy 1, policy_version 74122 (0.0008) [2023-10-08 10:43:57,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152141824. Throughput: 0: 1836.1, 1: 1820.8. Samples: 38043716. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:43:57,015][52710] Avg episode reward: [(0, '33.170'), (1, '36.270')] [2023-10-08 10:43:57,104][53885] Updated weights for policy 1, policy_version 74132 (0.0007) [2023-10-08 10:43:57,465][53885] Updated weights for policy 1, policy_version 74142 (0.0008) [2023-10-08 10:43:57,578][53852] Updated weights for policy 0, policy_version 74470 (0.0007) [2023-10-08 10:43:57,953][53852] Updated weights for policy 0, policy_version 74480 (0.0010) [2023-10-08 10:43:58,321][53852] Updated weights for policy 0, policy_version 74490 (0.0010) [2023-10-08 10:44:01,136][53885] Updated weights for policy 1, policy_version 74152 (0.0008) [2023-10-08 10:44:01,500][53885] Updated weights for policy 1, policy_version 74162 (0.0007) [2023-10-08 10:44:01,879][53885] Updated weights for policy 1, policy_version 74172 (0.0010) [2023-10-08 10:44:02,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 152240128. Throughput: 0: 1846.3, 1: 1824.5. Samples: 38066944. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:44:02,016][52710] Avg episode reward: [(0, '33.960'), (1, '36.480')] [2023-10-08 10:44:02,128][53852] Updated weights for policy 0, policy_version 74500 (0.0009) [2023-10-08 10:44:02,496][53852] Updated weights for policy 0, policy_version 74510 (0.0008) [2023-10-08 10:44:02,861][53852] Updated weights for policy 0, policy_version 74520 (0.0010) [2023-10-08 10:44:05,664][53885] Updated weights for policy 1, policy_version 74182 (0.0007) [2023-10-08 10:44:06,039][53885] Updated weights for policy 1, policy_version 74192 (0.0010) [2023-10-08 10:44:06,411][53885] Updated weights for policy 1, policy_version 74202 (0.0009) [2023-10-08 10:44:06,476][53852] Updated weights for policy 0, policy_version 74530 (0.0008) [2023-10-08 10:44:06,839][53852] Updated weights for policy 0, policy_version 74540 (0.0008) [2023-10-08 10:44:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152305664. Throughput: 0: 1835.4, 1: 1821.4. Samples: 38088096. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:44:07,016][52710] Avg episode reward: [(0, '34.850'), (1, '33.050')] [2023-10-08 10:44:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000074208_75988992.pth... [2023-10-08 10:44:07,068][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000072480_74219520.pth [2023-10-08 10:44:07,211][53852] Updated weights for policy 0, policy_version 74550 (0.0008) [2023-10-08 10:44:07,588][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000074560_76349440.pth... [2023-10-08 10:44:07,590][53852] Updated weights for policy 0, policy_version 74560 (0.0008) [2023-10-08 10:44:07,627][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000072832_74579968.pth [2023-10-08 10:44:09,964][53885] Updated weights for policy 1, policy_version 74212 (0.0007) [2023-10-08 10:44:10,332][53885] Updated weights for policy 1, policy_version 74222 (0.0009) [2023-10-08 10:44:10,703][53885] Updated weights for policy 1, policy_version 74232 (0.0008) [2023-10-08 10:44:11,121][53852] Updated weights for policy 0, policy_version 74570 (0.0009) [2023-10-08 10:44:11,489][53852] Updated weights for policy 0, policy_version 74580 (0.0007) [2023-10-08 10:44:11,863][53852] Updated weights for policy 0, policy_version 74590 (0.0007) [2023-10-08 10:44:12,015][52710] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 152403968. Throughput: 0: 1841.5, 1: 1822.1. Samples: 38099764. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:44:12,016][52710] Avg episode reward: [(0, '32.240'), (1, '34.830')] [2023-10-08 10:44:14,558][53885] Updated weights for policy 1, policy_version 74242 (0.0009) [2023-10-08 10:44:14,925][53885] Updated weights for policy 1, policy_version 74252 (0.0007) [2023-10-08 10:44:15,289][53885] Updated weights for policy 1, policy_version 74262 (0.0009) [2023-10-08 10:44:15,487][53852] Updated weights for policy 0, policy_version 74600 (0.0007) [2023-10-08 10:44:15,657][53885] Updated weights for policy 1, policy_version 74272 (0.0008) [2023-10-08 10:44:15,862][53852] Updated weights for policy 0, policy_version 74610 (0.0008) [2023-10-08 10:44:16,225][53852] Updated weights for policy 0, policy_version 74620 (0.0008) [2023-10-08 10:44:17,015][52710] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 152469504. Throughput: 0: 1828.7, 1: 1822.4. Samples: 38120940. Policy #0 lag: (min: 31.0, avg: 43.5, max: 63.0) [2023-10-08 10:44:17,016][52710] Avg episode reward: [(0, '33.810'), (1, '32.740')] [2023-10-08 10:44:19,180][53885] Updated weights for policy 1, policy_version 74282 (0.0011) [2023-10-08 10:44:19,542][53885] Updated weights for policy 1, policy_version 74292 (0.0009) [2023-10-08 10:44:19,904][53885] Updated weights for policy 1, policy_version 74302 (0.0009) [2023-10-08 10:44:19,906][53852] Updated weights for policy 0, policy_version 74630 (0.0008) [2023-10-08 10:44:20,267][53852] Updated weights for policy 0, policy_version 74640 (0.0007) [2023-10-08 10:44:20,641][53852] Updated weights for policy 0, policy_version 74650 (0.0007) [2023-10-08 10:44:22,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 152535040. Throughput: 0: 1840.5, 1: 1816.1. Samples: 38142724. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:44:22,015][52710] Avg episode reward: [(0, '34.820'), (1, '31.670')] [2023-10-08 10:44:23,611][53885] Updated weights for policy 1, policy_version 74312 (0.0007) [2023-10-08 10:44:23,972][53885] Updated weights for policy 1, policy_version 74322 (0.0008) [2023-10-08 10:44:24,142][53852] Updated weights for policy 0, policy_version 74660 (0.0008) [2023-10-08 10:44:24,340][53885] Updated weights for policy 1, policy_version 74332 (0.0007) [2023-10-08 10:44:24,516][53852] Updated weights for policy 0, policy_version 74670 (0.0007) [2023-10-08 10:44:24,888][53852] Updated weights for policy 0, policy_version 74680 (0.0007) [2023-10-08 10:44:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 152600576. Throughput: 0: 1821.3, 1: 1819.4. Samples: 38153628. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:44:27,016][52710] Avg episode reward: [(0, '31.940'), (1, '33.730')] [2023-10-08 10:44:27,981][53885] Updated weights for policy 1, policy_version 74342 (0.0008) [2023-10-08 10:44:28,361][53885] Updated weights for policy 1, policy_version 74352 (0.0009) [2023-10-08 10:44:28,389][53852] Updated weights for policy 0, policy_version 74690 (0.0008) [2023-10-08 10:44:28,726][53885] Updated weights for policy 1, policy_version 74362 (0.0008) [2023-10-08 10:44:28,761][53852] Updated weights for policy 0, policy_version 74700 (0.0007) [2023-10-08 10:44:29,131][53852] Updated weights for policy 0, policy_version 74710 (0.0010) [2023-10-08 10:44:29,486][53852] Updated weights for policy 0, policy_version 74720 (0.0008) [2023-10-08 10:44:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152666112. Throughput: 0: 1845.2, 1: 1824.7. Samples: 38176100. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:44:32,016][52710] Avg episode reward: [(0, '35.450'), (1, '36.070')] [2023-10-08 10:44:32,413][53885] Updated weights for policy 1, policy_version 74372 (0.0009) [2023-10-08 10:44:32,780][53885] Updated weights for policy 1, policy_version 74382 (0.0009) [2023-10-08 10:44:33,085][53852] Updated weights for policy 0, policy_version 74730 (0.0008) [2023-10-08 10:44:33,160][53885] Updated weights for policy 1, policy_version 74392 (0.0008) [2023-10-08 10:44:33,455][53852] Updated weights for policy 0, policy_version 74740 (0.0008) [2023-10-08 10:44:33,827][53852] Updated weights for policy 0, policy_version 74750 (0.0009) [2023-10-08 10:44:36,988][53885] Updated weights for policy 1, policy_version 74402 (0.0007) [2023-10-08 10:44:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152731648. Throughput: 0: 1848.7, 1: 1819.8. Samples: 38199052. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:44:37,016][52710] Avg episode reward: [(0, '36.230'), (1, '31.380')] [2023-10-08 10:44:37,401][53885] Updated weights for policy 1, policy_version 74412 (0.0008) [2023-10-08 10:44:37,590][53852] Updated weights for policy 0, policy_version 74760 (0.0010) [2023-10-08 10:44:37,765][53885] Updated weights for policy 1, policy_version 74422 (0.0008) [2023-10-08 10:44:37,966][53852] Updated weights for policy 0, policy_version 74770 (0.0008) [2023-10-08 10:44:38,128][53885] Updated weights for policy 1, policy_version 74432 (0.0007) [2023-10-08 10:44:38,349][53852] Updated weights for policy 0, policy_version 74780 (0.0009) [2023-10-08 10:44:41,798][53885] Updated weights for policy 1, policy_version 74442 (0.0011) [2023-10-08 10:44:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152797184. Throughput: 0: 1850.6, 1: 1820.5. Samples: 38208916. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:44:42,015][52710] Avg episode reward: [(0, '32.190'), (1, '32.130')] [2023-10-08 10:44:42,061][53852] Updated weights for policy 0, policy_version 74790 (0.0009) [2023-10-08 10:44:42,163][53885] Updated weights for policy 1, policy_version 74452 (0.0008) [2023-10-08 10:44:42,434][53852] Updated weights for policy 0, policy_version 74800 (0.0008) [2023-10-08 10:44:42,535][53885] Updated weights for policy 1, policy_version 74462 (0.0007) [2023-10-08 10:44:42,802][53852] Updated weights for policy 0, policy_version 74810 (0.0007) [2023-10-08 10:44:46,357][53885] Updated weights for policy 1, policy_version 74472 (0.0008) [2023-10-08 10:44:46,501][53852] Updated weights for policy 0, policy_version 74820 (0.0008) [2023-10-08 10:44:46,726][53885] Updated weights for policy 1, policy_version 74482 (0.0007) [2023-10-08 10:44:46,873][53852] Updated weights for policy 0, policy_version 74830 (0.0007) [2023-10-08 10:44:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152862720. Throughput: 0: 1848.9, 1: 1811.9. Samples: 38231682. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:44:47,016][52710] Avg episode reward: [(0, '31.760'), (1, '32.600')] [2023-10-08 10:44:47,088][53885] Updated weights for policy 1, policy_version 74492 (0.0008) [2023-10-08 10:44:47,242][53852] Updated weights for policy 0, policy_version 74840 (0.0008) [2023-10-08 10:44:50,692][53885] Updated weights for policy 1, policy_version 74502 (0.0008) [2023-10-08 10:44:50,938][53852] Updated weights for policy 0, policy_version 74850 (0.0009) [2023-10-08 10:44:51,054][53885] Updated weights for policy 1, policy_version 74512 (0.0007) [2023-10-08 10:44:51,300][53852] Updated weights for policy 0, policy_version 74860 (0.0008) [2023-10-08 10:44:51,423][53885] Updated weights for policy 1, policy_version 74522 (0.0008) [2023-10-08 10:44:51,672][53852] Updated weights for policy 0, policy_version 74870 (0.0009) [2023-10-08 10:44:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152961024. Throughput: 0: 1833.3, 1: 1820.5. Samples: 38252514. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:44:52,016][52710] Avg episode reward: [(0, '34.030'), (1, '32.940')] [2023-10-08 10:44:52,037][53852] Updated weights for policy 0, policy_version 74880 (0.0010) [2023-10-08 10:44:55,100][53885] Updated weights for policy 1, policy_version 74532 (0.0010) [2023-10-08 10:44:55,455][53852] Updated weights for policy 0, policy_version 74890 (0.0007) [2023-10-08 10:44:55,458][53885] Updated weights for policy 1, policy_version 74542 (0.0009) [2023-10-08 10:44:55,817][53852] Updated weights for policy 0, policy_version 74900 (0.0008) [2023-10-08 10:44:55,829][53885] Updated weights for policy 1, policy_version 74552 (0.0008) [2023-10-08 10:44:56,180][53852] Updated weights for policy 0, policy_version 74910 (0.0007) [2023-10-08 10:44:57,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 153059328. Throughput: 0: 1850.9, 1: 1818.4. Samples: 38264880. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:44:57,016][52710] Avg episode reward: [(0, '30.240'), (1, '29.170')] [2023-10-08 10:44:59,545][53885] Updated weights for policy 1, policy_version 74562 (0.0007) [2023-10-08 10:44:59,651][53852] Updated weights for policy 0, policy_version 74920 (0.0008) [2023-10-08 10:44:59,902][53885] Updated weights for policy 1, policy_version 74572 (0.0008) [2023-10-08 10:45:00,015][53852] Updated weights for policy 0, policy_version 74930 (0.0009) [2023-10-08 10:45:00,270][53885] Updated weights for policy 1, policy_version 74582 (0.0008) [2023-10-08 10:45:00,386][53852] Updated weights for policy 0, policy_version 74940 (0.0007) [2023-10-08 10:45:00,639][53885] Updated weights for policy 1, policy_version 74592 (0.0008) [2023-10-08 10:45:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 153124864. Throughput: 0: 1827.4, 1: 1821.4. Samples: 38285136. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:45:02,016][52710] Avg episode reward: [(0, '29.960'), (1, '33.260')] [2023-10-08 10:45:04,065][53852] Updated weights for policy 0, policy_version 74950 (0.0010) [2023-10-08 10:45:04,384][53885] Updated weights for policy 1, policy_version 74602 (0.0007) [2023-10-08 10:45:04,426][53852] Updated weights for policy 0, policy_version 74960 (0.0007) [2023-10-08 10:45:04,752][53885] Updated weights for policy 1, policy_version 74612 (0.0007) [2023-10-08 10:45:04,798][53852] Updated weights for policy 0, policy_version 74970 (0.0008) [2023-10-08 10:45:05,110][53885] Updated weights for policy 1, policy_version 74622 (0.0008) [2023-10-08 10:45:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 153190400. Throughput: 0: 1847.8, 1: 1820.5. Samples: 38307800. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 10:45:07,016][52710] Avg episode reward: [(0, '33.540'), (1, '34.610')] [2023-10-08 10:45:08,602][53852] Updated weights for policy 0, policy_version 74980 (0.0009) [2023-10-08 10:45:08,830][53885] Updated weights for policy 1, policy_version 74632 (0.0011) [2023-10-08 10:45:08,970][53852] Updated weights for policy 0, policy_version 74990 (0.0009) [2023-10-08 10:45:09,206][53885] Updated weights for policy 1, policy_version 74642 (0.0010) [2023-10-08 10:45:09,339][53852] Updated weights for policy 0, policy_version 75000 (0.0008) [2023-10-08 10:45:09,568][53885] Updated weights for policy 1, policy_version 74652 (0.0007) [2023-10-08 10:45:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153255936. Throughput: 0: 1825.9, 1: 1821.6. Samples: 38317762. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:12,016][52710] Avg episode reward: [(0, '35.510'), (1, '32.860')] [2023-10-08 10:45:13,049][53852] Updated weights for policy 0, policy_version 75010 (0.0009) [2023-10-08 10:45:13,291][53885] Updated weights for policy 1, policy_version 74662 (0.0008) [2023-10-08 10:45:13,424][53852] Updated weights for policy 0, policy_version 75020 (0.0007) [2023-10-08 10:45:13,651][53885] Updated weights for policy 1, policy_version 74672 (0.0009) [2023-10-08 10:45:13,789][53852] Updated weights for policy 0, policy_version 75030 (0.0007) [2023-10-08 10:45:14,019][53885] Updated weights for policy 1, policy_version 74682 (0.0009) [2023-10-08 10:45:14,153][53852] Updated weights for policy 0, policy_version 75040 (0.0008) [2023-10-08 10:45:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153321472. Throughput: 0: 1832.6, 1: 1811.6. Samples: 38340090. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:17,016][52710] Avg episode reward: [(0, '32.130'), (1, '33.410')] [2023-10-08 10:45:17,851][53885] Updated weights for policy 1, policy_version 74692 (0.0010) [2023-10-08 10:45:17,872][53852] Updated weights for policy 0, policy_version 75050 (0.0008) [2023-10-08 10:45:18,216][53885] Updated weights for policy 1, policy_version 74702 (0.0009) [2023-10-08 10:45:18,240][53852] Updated weights for policy 0, policy_version 75060 (0.0008) [2023-10-08 10:45:18,585][53885] Updated weights for policy 1, policy_version 74712 (0.0008) [2023-10-08 10:45:18,606][53852] Updated weights for policy 0, policy_version 75070 (0.0008) [2023-10-08 10:45:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153387008. Throughput: 0: 1837.1, 1: 1811.1. Samples: 38363220. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:22,015][52710] Avg episode reward: [(0, '33.360'), (1, '33.890')] [2023-10-08 10:45:22,271][53885] Updated weights for policy 1, policy_version 74722 (0.0009) [2023-10-08 10:45:22,342][53852] Updated weights for policy 0, policy_version 75080 (0.0008) [2023-10-08 10:45:22,665][53885] Updated weights for policy 1, policy_version 74732 (0.0007) [2023-10-08 10:45:22,719][53852] Updated weights for policy 0, policy_version 75090 (0.0008) [2023-10-08 10:45:23,030][53885] Updated weights for policy 1, policy_version 74742 (0.0007) [2023-10-08 10:45:23,087][53852] Updated weights for policy 0, policy_version 75100 (0.0008) [2023-10-08 10:45:23,399][53885] Updated weights for policy 1, policy_version 74752 (0.0009) [2023-10-08 10:45:26,990][53852] Updated weights for policy 0, policy_version 75110 (0.0007) [2023-10-08 10:45:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 153452544. Throughput: 0: 1838.1, 1: 1805.7. Samples: 38372886. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:27,016][52710] Avg episode reward: [(0, '33.360'), (1, '33.940')] [2023-10-08 10:45:27,192][53885] Updated weights for policy 1, policy_version 74762 (0.0007) [2023-10-08 10:45:27,376][53852] Updated weights for policy 0, policy_version 75120 (0.0008) [2023-10-08 10:45:27,559][53885] Updated weights for policy 1, policy_version 74772 (0.0009) [2023-10-08 10:45:27,753][53852] Updated weights for policy 0, policy_version 75130 (0.0007) [2023-10-08 10:45:27,923][53885] Updated weights for policy 1, policy_version 74782 (0.0008) [2023-10-08 10:45:31,391][53852] Updated weights for policy 0, policy_version 75140 (0.0008) [2023-10-08 10:45:31,684][53885] Updated weights for policy 1, policy_version 74792 (0.0009) [2023-10-08 10:45:31,757][53852] Updated weights for policy 0, policy_version 75150 (0.0009) [2023-10-08 10:45:32,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153518080. Throughput: 0: 1831.4, 1: 1811.0. Samples: 38395588. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:32,016][52710] Avg episode reward: [(0, '31.680'), (1, '32.730')] [2023-10-08 10:45:32,053][53885] Updated weights for policy 1, policy_version 74802 (0.0007) [2023-10-08 10:45:32,125][53852] Updated weights for policy 0, policy_version 75160 (0.0008) [2023-10-08 10:45:32,414][53885] Updated weights for policy 1, policy_version 74812 (0.0007) [2023-10-08 10:45:35,764][53852] Updated weights for policy 0, policy_version 75170 (0.0009) [2023-10-08 10:45:36,132][53852] Updated weights for policy 0, policy_version 75180 (0.0010) [2023-10-08 10:45:36,244][53885] Updated weights for policy 1, policy_version 74822 (0.0007) [2023-10-08 10:45:36,499][53852] Updated weights for policy 0, policy_version 75190 (0.0008) [2023-10-08 10:45:36,616][53885] Updated weights for policy 1, policy_version 74832 (0.0007) [2023-10-08 10:45:36,870][53852] Updated weights for policy 0, policy_version 75200 (0.0008) [2023-10-08 10:45:36,976][53885] Updated weights for policy 1, policy_version 74842 (0.0007) [2023-10-08 10:45:37,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153616384. Throughput: 0: 1828.5, 1: 1818.8. Samples: 38416646. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:37,016][52710] Avg episode reward: [(0, '32.810'), (1, '32.410')] [2023-10-08 10:45:40,464][53852] Updated weights for policy 0, policy_version 75210 (0.0010) [2023-10-08 10:45:40,650][53885] Updated weights for policy 1, policy_version 74852 (0.0007) [2023-10-08 10:45:40,830][53852] Updated weights for policy 0, policy_version 75220 (0.0008) [2023-10-08 10:45:41,008][53885] Updated weights for policy 1, policy_version 74862 (0.0008) [2023-10-08 10:45:41,203][53852] Updated weights for policy 0, policy_version 75230 (0.0009) [2023-10-08 10:45:41,378][53885] Updated weights for policy 1, policy_version 74872 (0.0007) [2023-10-08 10:45:42,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 153714688. Throughput: 0: 1827.6, 1: 1802.6. Samples: 38428238. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:42,016][52710] Avg episode reward: [(0, '32.690'), (1, '35.110')] [2023-10-08 10:45:44,864][53852] Updated weights for policy 0, policy_version 75240 (0.0009) [2023-10-08 10:45:45,102][53885] Updated weights for policy 1, policy_version 74882 (0.0008) [2023-10-08 10:45:45,242][53852] Updated weights for policy 0, policy_version 75250 (0.0007) [2023-10-08 10:45:45,463][53885] Updated weights for policy 1, policy_version 74892 (0.0009) [2023-10-08 10:45:45,613][53852] Updated weights for policy 0, policy_version 75260 (0.0009) [2023-10-08 10:45:45,844][53885] Updated weights for policy 1, policy_version 74902 (0.0009) [2023-10-08 10:45:46,206][53885] Updated weights for policy 1, policy_version 74912 (0.0010) [2023-10-08 10:45:47,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 153780224. Throughput: 0: 1829.0, 1: 1815.8. Samples: 38449150. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:47,016][52710] Avg episode reward: [(0, '31.740'), (1, '39.030')] [2023-10-08 10:45:49,260][53852] Updated weights for policy 0, policy_version 75270 (0.0009) [2023-10-08 10:45:49,633][53852] Updated weights for policy 0, policy_version 75280 (0.0009) [2023-10-08 10:45:49,962][53885] Updated weights for policy 1, policy_version 74922 (0.0008) [2023-10-08 10:45:50,003][53852] Updated weights for policy 0, policy_version 75290 (0.0011) [2023-10-08 10:45:50,324][53885] Updated weights for policy 1, policy_version 74932 (0.0010) [2023-10-08 10:45:50,688][53885] Updated weights for policy 1, policy_version 74942 (0.0010) [2023-10-08 10:45:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153845760. Throughput: 0: 1825.3, 1: 1798.5. Samples: 38470866. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:52,016][52710] Avg episode reward: [(0, '31.610'), (1, '35.060')] [2023-10-08 10:45:53,552][53852] Updated weights for policy 0, policy_version 75300 (0.0008) [2023-10-08 10:45:53,918][53852] Updated weights for policy 0, policy_version 75310 (0.0008) [2023-10-08 10:45:54,201][53885] Updated weights for policy 1, policy_version 74952 (0.0008) [2023-10-08 10:45:54,289][53852] Updated weights for policy 0, policy_version 75320 (0.0009) [2023-10-08 10:45:54,562][53885] Updated weights for policy 1, policy_version 74962 (0.0008) [2023-10-08 10:45:54,941][53885] Updated weights for policy 1, policy_version 74972 (0.0008) [2023-10-08 10:45:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153911296. Throughput: 0: 1829.5, 1: 1814.9. Samples: 38481758. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) [2023-10-08 10:45:57,016][52710] Avg episode reward: [(0, '29.890'), (1, '33.980')] [2023-10-08 10:45:57,861][53852] Updated weights for policy 0, policy_version 75330 (0.0008) [2023-10-08 10:45:58,229][53852] Updated weights for policy 0, policy_version 75340 (0.0008) [2023-10-08 10:45:58,539][53885] Updated weights for policy 1, policy_version 74982 (0.0009) [2023-10-08 10:45:58,596][53852] Updated weights for policy 0, policy_version 75350 (0.0007) [2023-10-08 10:45:58,906][53885] Updated weights for policy 1, policy_version 74992 (0.0008) [2023-10-08 10:45:58,970][53852] Updated weights for policy 0, policy_version 75360 (0.0010) [2023-10-08 10:45:59,269][53885] Updated weights for policy 1, policy_version 75002 (0.0009) [2023-10-08 10:46:02,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 153976832. Throughput: 0: 1838.2, 1: 1808.0. Samples: 38504170. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:02,016][52710] Avg episode reward: [(0, '33.600'), (1, '36.620')] [2023-10-08 10:46:02,681][53852] Updated weights for policy 0, policy_version 75370 (0.0009) [2023-10-08 10:46:02,952][53885] Updated weights for policy 1, policy_version 75012 (0.0009) [2023-10-08 10:46:03,054][53852] Updated weights for policy 0, policy_version 75380 (0.0009) [2023-10-08 10:46:03,328][53885] Updated weights for policy 1, policy_version 75022 (0.0008) [2023-10-08 10:46:03,425][53852] Updated weights for policy 0, policy_version 75390 (0.0007) [2023-10-08 10:46:03,691][53885] Updated weights for policy 1, policy_version 75032 (0.0008) [2023-10-08 10:46:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154042368. Throughput: 0: 1829.1, 1: 1814.2. Samples: 38527166. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:07,016][52710] Avg episode reward: [(0, '33.960'), (1, '36.060')] [2023-10-08 10:46:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000075040_76840960.pth... [2023-10-08 10:46:07,062][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000073344_75104256.pth [2023-10-08 10:46:07,298][53885] Updated weights for policy 1, policy_version 75042 (0.0008) [2023-10-08 10:46:07,345][53852] Updated weights for policy 0, policy_version 75400 (0.0007) [2023-10-08 10:46:07,684][53885] Updated weights for policy 1, policy_version 75052 (0.0011) [2023-10-08 10:46:07,708][53852] Updated weights for policy 0, policy_version 75410 (0.0008) [2023-10-08 10:46:08,046][53885] Updated weights for policy 1, policy_version 75062 (0.0008) [2023-10-08 10:46:08,080][53852] Updated weights for policy 0, policy_version 75420 (0.0008) [2023-10-08 10:46:08,224][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000075424_77234176.pth... [2023-10-08 10:46:08,257][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000073696_75464704.pth [2023-10-08 10:46:08,417][53885] Updated weights for policy 1, policy_version 75072 (0.0007) [2023-10-08 10:46:11,845][53852] Updated weights for policy 0, policy_version 75430 (0.0008) [2023-10-08 10:46:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 154107904. Throughput: 0: 1830.5, 1: 1817.7. Samples: 38537054. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:12,016][52710] Avg episode reward: [(0, '32.150'), (1, '32.360')] [2023-10-08 10:46:12,225][53885] Updated weights for policy 1, policy_version 75082 (0.0008) [2023-10-08 10:46:12,228][53852] Updated weights for policy 0, policy_version 75440 (0.0008) [2023-10-08 10:46:12,592][53852] Updated weights for policy 0, policy_version 75450 (0.0007) [2023-10-08 10:46:12,593][53885] Updated weights for policy 1, policy_version 75092 (0.0008) [2023-10-08 10:46:12,960][53885] Updated weights for policy 1, policy_version 75102 (0.0009) [2023-10-08 10:46:16,205][53852] Updated weights for policy 0, policy_version 75460 (0.0007) [2023-10-08 10:46:16,578][53852] Updated weights for policy 0, policy_version 75470 (0.0007) [2023-10-08 10:46:16,612][53885] Updated weights for policy 1, policy_version 75112 (0.0008) [2023-10-08 10:46:16,952][53852] Updated weights for policy 0, policy_version 75480 (0.0008) [2023-10-08 10:46:16,986][53885] Updated weights for policy 1, policy_version 75122 (0.0008) [2023-10-08 10:46:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154173440. Throughput: 0: 1830.7, 1: 1815.3. Samples: 38559658. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:17,016][52710] Avg episode reward: [(0, '33.240'), (1, '36.020')] [2023-10-08 10:46:17,357][53885] Updated weights for policy 1, policy_version 75132 (0.0008) [2023-10-08 10:46:20,575][53852] Updated weights for policy 0, policy_version 75490 (0.0008) [2023-10-08 10:46:20,938][53852] Updated weights for policy 0, policy_version 75500 (0.0008) [2023-10-08 10:46:21,099][53885] Updated weights for policy 1, policy_version 75142 (0.0008) [2023-10-08 10:46:21,309][53852] Updated weights for policy 0, policy_version 75510 (0.0007) [2023-10-08 10:46:21,464][53885] Updated weights for policy 1, policy_version 75152 (0.0008) [2023-10-08 10:46:21,671][53852] Updated weights for policy 0, policy_version 75520 (0.0008) [2023-10-08 10:46:21,825][53885] Updated weights for policy 1, policy_version 75162 (0.0009) [2023-10-08 10:46:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 154271744. Throughput: 0: 1818.2, 1: 1815.0. Samples: 38580140. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:22,016][52710] Avg episode reward: [(0, '34.560'), (1, '36.660')] [2023-10-08 10:46:25,318][53852] Updated weights for policy 0, policy_version 75530 (0.0008) [2023-10-08 10:46:25,614][53885] Updated weights for policy 1, policy_version 75172 (0.0010) [2023-10-08 10:46:25,676][53852] Updated weights for policy 0, policy_version 75540 (0.0009) [2023-10-08 10:46:25,985][53885] Updated weights for policy 1, policy_version 75182 (0.0009) [2023-10-08 10:46:26,047][53852] Updated weights for policy 0, policy_version 75550 (0.0007) [2023-10-08 10:46:26,341][53885] Updated weights for policy 1, policy_version 75192 (0.0009) [2023-10-08 10:46:27,015][52710] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 154370048. Throughput: 0: 1827.5, 1: 1817.1. Samples: 38592244. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:27,016][52710] Avg episode reward: [(0, '28.760'), (1, '32.850')] [2023-10-08 10:46:29,673][53852] Updated weights for policy 0, policy_version 75560 (0.0009) [2023-10-08 10:46:30,037][53852] Updated weights for policy 0, policy_version 75570 (0.0010) [2023-10-08 10:46:30,059][53885] Updated weights for policy 1, policy_version 75202 (0.0007) [2023-10-08 10:46:30,400][53852] Updated weights for policy 0, policy_version 75580 (0.0008) [2023-10-08 10:46:30,424][53885] Updated weights for policy 1, policy_version 75212 (0.0008) [2023-10-08 10:46:30,780][53885] Updated weights for policy 1, policy_version 75222 (0.0009) [2023-10-08 10:46:31,147][53885] Updated weights for policy 1, policy_version 75232 (0.0010) [2023-10-08 10:46:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 154435584. Throughput: 0: 1825.2, 1: 1814.7. Samples: 38612944. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:32,016][52710] Avg episode reward: [(0, '34.310'), (1, '34.040')] [2023-10-08 10:46:34,064][53852] Updated weights for policy 0, policy_version 75590 (0.0007) [2023-10-08 10:46:34,426][53852] Updated weights for policy 0, policy_version 75600 (0.0007) [2023-10-08 10:46:34,774][53885] Updated weights for policy 1, policy_version 75242 (0.0007) [2023-10-08 10:46:34,801][53852] Updated weights for policy 0, policy_version 75610 (0.0007) [2023-10-08 10:46:35,135][53885] Updated weights for policy 1, policy_version 75252 (0.0007) [2023-10-08 10:46:35,500][53885] Updated weights for policy 1, policy_version 75262 (0.0008) [2023-10-08 10:46:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154501120. Throughput: 0: 1832.8, 1: 1821.9. Samples: 38635326. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:37,015][52710] Avg episode reward: [(0, '33.010'), (1, '35.470')] [2023-10-08 10:46:38,352][53852] Updated weights for policy 0, policy_version 75620 (0.0007) [2023-10-08 10:46:38,725][53852] Updated weights for policy 0, policy_version 75630 (0.0009) [2023-10-08 10:46:39,011][53885] Updated weights for policy 1, policy_version 75272 (0.0007) [2023-10-08 10:46:39,095][53852] Updated weights for policy 0, policy_version 75640 (0.0007) [2023-10-08 10:46:39,372][53885] Updated weights for policy 1, policy_version 75282 (0.0007) [2023-10-08 10:46:39,748][53885] Updated weights for policy 1, policy_version 75292 (0.0010) [2023-10-08 10:46:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 154566656. Throughput: 0: 1829.1, 1: 1818.8. Samples: 38645914. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:42,016][52710] Avg episode reward: [(0, '33.460'), (1, '33.940')] [2023-10-08 10:46:42,717][53852] Updated weights for policy 0, policy_version 75650 (0.0008) [2023-10-08 10:46:43,085][53852] Updated weights for policy 0, policy_version 75660 (0.0010) [2023-10-08 10:46:43,450][53885] Updated weights for policy 1, policy_version 75302 (0.0007) [2023-10-08 10:46:43,453][53852] Updated weights for policy 0, policy_version 75670 (0.0008) [2023-10-08 10:46:43,811][53885] Updated weights for policy 1, policy_version 75312 (0.0008) [2023-10-08 10:46:43,821][53852] Updated weights for policy 0, policy_version 75680 (0.0007) [2023-10-08 10:46:44,174][53885] Updated weights for policy 1, policy_version 75322 (0.0011) [2023-10-08 10:46:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 154632192. Throughput: 0: 1829.9, 1: 1819.4. Samples: 38668388. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) [2023-10-08 10:46:47,016][52710] Avg episode reward: [(0, '34.890'), (1, '34.950')] [2023-10-08 10:46:47,402][53852] Updated weights for policy 0, policy_version 75690 (0.0008) [2023-10-08 10:46:47,773][53852] Updated weights for policy 0, policy_version 75700 (0.0010) [2023-10-08 10:46:47,903][53885] Updated weights for policy 1, policy_version 75332 (0.0009) [2023-10-08 10:46:48,147][53852] Updated weights for policy 0, policy_version 75710 (0.0009) [2023-10-08 10:46:48,269][53885] Updated weights for policy 1, policy_version 75342 (0.0008) [2023-10-08 10:46:48,634][53885] Updated weights for policy 1, policy_version 75352 (0.0007) [2023-10-08 10:46:51,782][53852] Updated weights for policy 0, policy_version 75720 (0.0011) [2023-10-08 10:46:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 154697728. Throughput: 0: 1833.8, 1: 1815.0. Samples: 38691364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:46:52,016][52710] Avg episode reward: [(0, '32.510'), (1, '38.590')] [2023-10-08 10:46:52,145][53852] Updated weights for policy 0, policy_version 75730 (0.0011) [2023-10-08 10:46:52,502][53885] Updated weights for policy 1, policy_version 75362 (0.0008) [2023-10-08 10:46:52,510][53852] Updated weights for policy 0, policy_version 75740 (0.0009) [2023-10-08 10:46:52,904][53885] Updated weights for policy 1, policy_version 75372 (0.0007) [2023-10-08 10:46:53,275][53885] Updated weights for policy 1, policy_version 75382 (0.0008) [2023-10-08 10:46:53,643][53885] Updated weights for policy 1, policy_version 75392 (0.0009) [2023-10-08 10:46:56,119][53852] Updated weights for policy 0, policy_version 75750 (0.0009) [2023-10-08 10:46:56,484][53852] Updated weights for policy 0, policy_version 75760 (0.0010) [2023-10-08 10:46:56,854][53852] Updated weights for policy 0, policy_version 75770 (0.0009) [2023-10-08 10:46:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154763264. Throughput: 0: 1837.5, 1: 1814.9. Samples: 38701412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:46:57,016][52710] Avg episode reward: [(0, '32.120'), (1, '35.750')] [2023-10-08 10:46:57,301][53885] Updated weights for policy 1, policy_version 75402 (0.0008) [2023-10-08 10:46:57,678][53885] Updated weights for policy 1, policy_version 75412 (0.0008) [2023-10-08 10:46:58,044][53885] Updated weights for policy 1, policy_version 75422 (0.0008) [2023-10-08 10:47:00,667][53852] Updated weights for policy 0, policy_version 75780 (0.0009) [2023-10-08 10:47:01,063][53852] Updated weights for policy 0, policy_version 75790 (0.0007) [2023-10-08 10:47:01,426][53852] Updated weights for policy 0, policy_version 75800 (0.0008) [2023-10-08 10:47:01,708][53885] Updated weights for policy 1, policy_version 75432 (0.0007) [2023-10-08 10:47:02,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 154861568. Throughput: 0: 1833.7, 1: 1814.9. Samples: 38723846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:02,016][52710] Avg episode reward: [(0, '32.700'), (1, '32.670')] [2023-10-08 10:47:02,075][53885] Updated weights for policy 1, policy_version 75442 (0.0010) [2023-10-08 10:47:02,454][53885] Updated weights for policy 1, policy_version 75452 (0.0010) [2023-10-08 10:47:05,157][53852] Updated weights for policy 0, policy_version 75810 (0.0008) [2023-10-08 10:47:05,523][53852] Updated weights for policy 0, policy_version 75820 (0.0009) [2023-10-08 10:47:05,891][53852] Updated weights for policy 0, policy_version 75830 (0.0007) [2023-10-08 10:47:06,100][53885] Updated weights for policy 1, policy_version 75462 (0.0010) [2023-10-08 10:47:06,257][53852] Updated weights for policy 0, policy_version 75840 (0.0008) [2023-10-08 10:47:06,468][53885] Updated weights for policy 1, policy_version 75472 (0.0007) [2023-10-08 10:47:06,833][53885] Updated weights for policy 1, policy_version 75482 (0.0007) [2023-10-08 10:47:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154927104. Throughput: 0: 1837.5, 1: 1816.9. Samples: 38744588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:07,016][52710] Avg episode reward: [(0, '33.400'), (1, '32.740')] [2023-10-08 10:47:09,782][53852] Updated weights for policy 0, policy_version 75850 (0.0007) [2023-10-08 10:47:10,146][53852] Updated weights for policy 0, policy_version 75860 (0.0007) [2023-10-08 10:47:10,369][53885] Updated weights for policy 1, policy_version 75492 (0.0007) [2023-10-08 10:47:10,519][53852] Updated weights for policy 0, policy_version 75870 (0.0008) [2023-10-08 10:47:10,735][53885] Updated weights for policy 1, policy_version 75502 (0.0008) [2023-10-08 10:47:11,103][53885] Updated weights for policy 1, policy_version 75512 (0.0010) [2023-10-08 10:47:12,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 155025408. Throughput: 0: 1839.8, 1: 1819.4. Samples: 38756908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:12,016][52710] Avg episode reward: [(0, '31.930'), (1, '32.910')] [2023-10-08 10:47:14,129][53852] Updated weights for policy 0, policy_version 75880 (0.0009) [2023-10-08 10:47:14,502][53852] Updated weights for policy 0, policy_version 75890 (0.0009) [2023-10-08 10:47:14,787][53885] Updated weights for policy 1, policy_version 75522 (0.0009) [2023-10-08 10:47:14,868][53852] Updated weights for policy 0, policy_version 75900 (0.0010) [2023-10-08 10:47:15,159][53885] Updated weights for policy 1, policy_version 75532 (0.0007) [2023-10-08 10:47:15,528][53885] Updated weights for policy 1, policy_version 75542 (0.0008) [2023-10-08 10:47:15,892][53885] Updated weights for policy 1, policy_version 75552 (0.0007) [2023-10-08 10:47:17,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155090944. Throughput: 0: 1842.2, 1: 1815.2. Samples: 38777524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:17,016][52710] Avg episode reward: [(0, '34.280'), (1, '29.490')] [2023-10-08 10:47:18,416][53852] Updated weights for policy 0, policy_version 75910 (0.0008) [2023-10-08 10:47:18,792][53852] Updated weights for policy 0, policy_version 75920 (0.0009) [2023-10-08 10:47:19,154][53852] Updated weights for policy 0, policy_version 75930 (0.0007) [2023-10-08 10:47:19,560][53885] Updated weights for policy 1, policy_version 75562 (0.0008) [2023-10-08 10:47:19,925][53885] Updated weights for policy 1, policy_version 75572 (0.0008) [2023-10-08 10:47:20,293][53885] Updated weights for policy 1, policy_version 75582 (0.0011) [2023-10-08 10:47:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155156480. Throughput: 0: 1845.0, 1: 1817.5. Samples: 38800138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:22,016][52710] Avg episode reward: [(0, '31.730'), (1, '33.990')] [2023-10-08 10:47:22,966][53852] Updated weights for policy 0, policy_version 75940 (0.0008) [2023-10-08 10:47:23,335][53852] Updated weights for policy 0, policy_version 75950 (0.0007) [2023-10-08 10:47:23,702][53852] Updated weights for policy 0, policy_version 75960 (0.0009) [2023-10-08 10:47:24,073][53885] Updated weights for policy 1, policy_version 75592 (0.0008) [2023-10-08 10:47:24,442][53885] Updated weights for policy 1, policy_version 75602 (0.0009) [2023-10-08 10:47:24,814][53885] Updated weights for policy 1, policy_version 75612 (0.0010) [2023-10-08 10:47:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 155222016. Throughput: 0: 1846.3, 1: 1817.1. Samples: 38810768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:27,016][52710] Avg episode reward: [(0, '33.140'), (1, '33.110')] [2023-10-08 10:47:27,201][53852] Updated weights for policy 0, policy_version 75970 (0.0007) [2023-10-08 10:47:27,570][53852] Updated weights for policy 0, policy_version 75980 (0.0009) [2023-10-08 10:47:27,937][53852] Updated weights for policy 0, policy_version 75990 (0.0009) [2023-10-08 10:47:28,300][53852] Updated weights for policy 0, policy_version 76000 (0.0008) [2023-10-08 10:47:28,490][53885] Updated weights for policy 1, policy_version 75622 (0.0008) [2023-10-08 10:47:28,860][53885] Updated weights for policy 1, policy_version 75632 (0.0007) [2023-10-08 10:47:29,215][53885] Updated weights for policy 1, policy_version 75642 (0.0011) [2023-10-08 10:47:31,973][53852] Updated weights for policy 0, policy_version 76010 (0.0007) [2023-10-08 10:47:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155287552. Throughput: 0: 1840.8, 1: 1819.8. Samples: 38833116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:32,016][52710] Avg episode reward: [(0, '33.420'), (1, '32.920')] [2023-10-08 10:47:32,343][53852] Updated weights for policy 0, policy_version 76020 (0.0008) [2023-10-08 10:47:32,716][53852] Updated weights for policy 0, policy_version 76030 (0.0007) [2023-10-08 10:47:32,860][53885] Updated weights for policy 1, policy_version 75652 (0.0009) [2023-10-08 10:47:33,216][53885] Updated weights for policy 1, policy_version 75662 (0.0010) [2023-10-08 10:47:33,580][53885] Updated weights for policy 1, policy_version 75672 (0.0009) [2023-10-08 10:47:36,509][53852] Updated weights for policy 0, policy_version 76040 (0.0007) [2023-10-08 10:47:36,878][53852] Updated weights for policy 0, policy_version 76050 (0.0007) [2023-10-08 10:47:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 155353088. Throughput: 0: 1827.6, 1: 1822.1. Samples: 38855602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:37,016][52710] Avg episode reward: [(0, '34.230'), (1, '34.120')] [2023-10-08 10:47:37,252][53852] Updated weights for policy 0, policy_version 76060 (0.0007) [2023-10-08 10:47:37,372][53885] Updated weights for policy 1, policy_version 75682 (0.0008) [2023-10-08 10:47:37,765][53885] Updated weights for policy 1, policy_version 75692 (0.0009) [2023-10-08 10:47:38,135][53885] Updated weights for policy 1, policy_version 75702 (0.0009) [2023-10-08 10:47:38,502][53885] Updated weights for policy 1, policy_version 75712 (0.0009) [2023-10-08 10:47:41,018][53852] Updated weights for policy 0, policy_version 76070 (0.0008) [2023-10-08 10:47:41,384][53852] Updated weights for policy 0, policy_version 76080 (0.0008) [2023-10-08 10:47:41,758][53852] Updated weights for policy 0, policy_version 76090 (0.0009) [2023-10-08 10:47:42,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155451392. Throughput: 0: 1831.0, 1: 1820.7. Samples: 38865738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:42,016][52710] Avg episode reward: [(0, '33.800'), (1, '33.180')] [2023-10-08 10:47:42,165][53885] Updated weights for policy 1, policy_version 75722 (0.0008) [2023-10-08 10:47:42,544][53885] Updated weights for policy 1, policy_version 75732 (0.0009) [2023-10-08 10:47:42,914][53885] Updated weights for policy 1, policy_version 75742 (0.0007) [2023-10-08 10:47:45,345][53852] Updated weights for policy 0, policy_version 76100 (0.0008) [2023-10-08 10:47:45,744][53852] Updated weights for policy 0, policy_version 76110 (0.0009) [2023-10-08 10:47:46,116][53852] Updated weights for policy 0, policy_version 76120 (0.0009) [2023-10-08 10:47:46,412][53885] Updated weights for policy 1, policy_version 75752 (0.0009) [2023-10-08 10:47:46,786][53885] Updated weights for policy 1, policy_version 75762 (0.0009) [2023-10-08 10:47:47,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155516928. Throughput: 0: 1827.8, 1: 1826.5. Samples: 38888290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:47,016][52710] Avg episode reward: [(0, '33.720'), (1, '33.650')] [2023-10-08 10:47:47,158][53885] Updated weights for policy 1, policy_version 75772 (0.0009) [2023-10-08 10:47:49,824][53852] Updated weights for policy 0, policy_version 76130 (0.0009) [2023-10-08 10:47:50,193][53852] Updated weights for policy 0, policy_version 76140 (0.0007) [2023-10-08 10:47:50,569][53852] Updated weights for policy 0, policy_version 76150 (0.0008) [2023-10-08 10:47:50,748][53885] Updated weights for policy 1, policy_version 75782 (0.0008) [2023-10-08 10:47:50,932][53852] Updated weights for policy 0, policy_version 76160 (0.0007) [2023-10-08 10:47:51,117][53885] Updated weights for policy 1, policy_version 75792 (0.0009) [2023-10-08 10:47:51,486][53885] Updated weights for policy 1, policy_version 75802 (0.0007) [2023-10-08 10:47:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155615232. Throughput: 0: 1833.2, 1: 1820.9. Samples: 38909024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:52,016][52710] Avg episode reward: [(0, '32.630'), (1, '33.680')] [2023-10-08 10:47:54,551][53852] Updated weights for policy 0, policy_version 76170 (0.0009) [2023-10-08 10:47:54,911][53852] Updated weights for policy 0, policy_version 76180 (0.0010) [2023-10-08 10:47:55,210][53885] Updated weights for policy 1, policy_version 75812 (0.0008) [2023-10-08 10:47:55,281][53852] Updated weights for policy 0, policy_version 76190 (0.0009) [2023-10-08 10:47:55,580][53885] Updated weights for policy 1, policy_version 75822 (0.0009) [2023-10-08 10:47:55,961][53885] Updated weights for policy 1, policy_version 75832 (0.0009) [2023-10-08 10:47:57,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 155680768. Throughput: 0: 1824.6, 1: 1830.2. Samples: 38921374. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:47:57,015][52710] Avg episode reward: [(0, '30.770'), (1, '33.520')] [2023-10-08 10:47:58,788][53852] Updated weights for policy 0, policy_version 76200 (0.0008) [2023-10-08 10:47:59,146][53852] Updated weights for policy 0, policy_version 76210 (0.0010) [2023-10-08 10:47:59,520][53852] Updated weights for policy 0, policy_version 76220 (0.0008) [2023-10-08 10:47:59,778][53885] Updated weights for policy 1, policy_version 75842 (0.0010) [2023-10-08 10:48:00,135][53885] Updated weights for policy 1, policy_version 75852 (0.0008) [2023-10-08 10:48:00,502][53885] Updated weights for policy 1, policy_version 75862 (0.0007) [2023-10-08 10:48:00,867][53885] Updated weights for policy 1, policy_version 75872 (0.0009) [2023-10-08 10:48:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 155746304. Throughput: 0: 1836.0, 1: 1831.7. Samples: 38942572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:02,016][52710] Avg episode reward: [(0, '31.890'), (1, '33.600')] [2023-10-08 10:48:03,242][53852] Updated weights for policy 0, policy_version 76230 (0.0008) [2023-10-08 10:48:03,615][53852] Updated weights for policy 0, policy_version 76240 (0.0008) [2023-10-08 10:48:03,992][53852] Updated weights for policy 0, policy_version 76250 (0.0008) [2023-10-08 10:48:04,424][53885] Updated weights for policy 1, policy_version 75882 (0.0007) [2023-10-08 10:48:04,788][53885] Updated weights for policy 1, policy_version 75892 (0.0007) [2023-10-08 10:48:05,166][53885] Updated weights for policy 1, policy_version 75902 (0.0009) [2023-10-08 10:48:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 155811840. Throughput: 0: 1829.2, 1: 1835.5. Samples: 38965048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:07,016][52710] Avg episode reward: [(0, '33.270'), (1, '33.310')] [2023-10-08 10:48:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000076256_78086144.pth... [2023-10-08 10:48:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000075904_77725696.pth... [2023-10-08 10:48:07,060][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000074208_75988992.pth [2023-10-08 10:48:07,066][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000074560_76349440.pth [2023-10-08 10:48:07,629][53852] Updated weights for policy 0, policy_version 76260 (0.0007) [2023-10-08 10:48:08,002][53852] Updated weights for policy 0, policy_version 76270 (0.0009) [2023-10-08 10:48:08,378][53852] Updated weights for policy 0, policy_version 76280 (0.0010) [2023-10-08 10:48:08,813][53885] Updated weights for policy 1, policy_version 75912 (0.0009) [2023-10-08 10:48:09,179][53885] Updated weights for policy 1, policy_version 75922 (0.0008) [2023-10-08 10:48:09,549][53885] Updated weights for policy 1, policy_version 75932 (0.0007) [2023-10-08 10:48:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 155877376. Throughput: 0: 1833.7, 1: 1824.5. Samples: 38975384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:12,016][52710] Avg episode reward: [(0, '32.730'), (1, '31.390')] [2023-10-08 10:48:12,094][53852] Updated weights for policy 0, policy_version 76290 (0.0007) [2023-10-08 10:48:12,460][53852] Updated weights for policy 0, policy_version 76300 (0.0009) [2023-10-08 10:48:12,838][53852] Updated weights for policy 0, policy_version 76310 (0.0009) [2023-10-08 10:48:13,198][53852] Updated weights for policy 0, policy_version 76320 (0.0009) [2023-10-08 10:48:13,287][53885] Updated weights for policy 1, policy_version 75942 (0.0008) [2023-10-08 10:48:13,651][53885] Updated weights for policy 1, policy_version 75952 (0.0009) [2023-10-08 10:48:14,012][53885] Updated weights for policy 1, policy_version 75962 (0.0007) [2023-10-08 10:48:16,847][53852] Updated weights for policy 0, policy_version 76330 (0.0008) [2023-10-08 10:48:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155942912. Throughput: 0: 1836.3, 1: 1823.7. Samples: 38997816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:17,016][52710] Avg episode reward: [(0, '32.280'), (1, '31.860')] [2023-10-08 10:48:17,220][53852] Updated weights for policy 0, policy_version 76340 (0.0008) [2023-10-08 10:48:17,597][53852] Updated weights for policy 0, policy_version 76350 (0.0008) [2023-10-08 10:48:17,700][53885] Updated weights for policy 1, policy_version 75972 (0.0009) [2023-10-08 10:48:18,063][53885] Updated weights for policy 1, policy_version 75982 (0.0007) [2023-10-08 10:48:18,435][53885] Updated weights for policy 1, policy_version 75992 (0.0010) [2023-10-08 10:48:21,152][53852] Updated weights for policy 0, policy_version 76360 (0.0008) [2023-10-08 10:48:21,528][53852] Updated weights for policy 0, policy_version 76370 (0.0010) [2023-10-08 10:48:21,899][53852] Updated weights for policy 0, policy_version 76380 (0.0008) [2023-10-08 10:48:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 156008448. Throughput: 0: 1828.9, 1: 1823.1. Samples: 39019942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:22,016][52710] Avg episode reward: [(0, '33.360'), (1, '35.110')] [2023-10-08 10:48:22,197][53885] Updated weights for policy 1, policy_version 76002 (0.0010) [2023-10-08 10:48:22,606][53885] Updated weights for policy 1, policy_version 76012 (0.0009) [2023-10-08 10:48:22,971][53885] Updated weights for policy 1, policy_version 76022 (0.0009) [2023-10-08 10:48:23,337][53885] Updated weights for policy 1, policy_version 76032 (0.0007) [2023-10-08 10:48:25,517][53852] Updated weights for policy 0, policy_version 76390 (0.0009) [2023-10-08 10:48:25,877][53852] Updated weights for policy 0, policy_version 76400 (0.0008) [2023-10-08 10:48:26,254][53852] Updated weights for policy 0, policy_version 76410 (0.0010) [2023-10-08 10:48:26,871][53885] Updated weights for policy 1, policy_version 76042 (0.0008) [2023-10-08 10:48:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156106752. Throughput: 0: 1840.9, 1: 1825.3. Samples: 39030718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:27,016][52710] Avg episode reward: [(0, '33.710'), (1, '34.730')] [2023-10-08 10:48:27,239][53885] Updated weights for policy 1, policy_version 76052 (0.0010) [2023-10-08 10:48:27,604][53885] Updated weights for policy 1, policy_version 76062 (0.0007) [2023-10-08 10:48:29,888][53852] Updated weights for policy 0, policy_version 76420 (0.0009) [2023-10-08 10:48:30,265][53852] Updated weights for policy 0, policy_version 76430 (0.0010) [2023-10-08 10:48:30,644][53852] Updated weights for policy 0, policy_version 76440 (0.0008) [2023-10-08 10:48:31,438][53885] Updated weights for policy 1, policy_version 76072 (0.0008) [2023-10-08 10:48:31,797][53885] Updated weights for policy 1, policy_version 76082 (0.0011) [2023-10-08 10:48:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156172288. Throughput: 0: 1832.1, 1: 1826.9. Samples: 39052946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:32,016][52710] Avg episode reward: [(0, '32.700'), (1, '34.920')] [2023-10-08 10:48:32,169][53885] Updated weights for policy 1, policy_version 76092 (0.0010) [2023-10-08 10:48:34,405][53852] Updated weights for policy 0, policy_version 76450 (0.0009) [2023-10-08 10:48:34,804][53852] Updated weights for policy 0, policy_version 76460 (0.0007) [2023-10-08 10:48:35,182][53852] Updated weights for policy 0, policy_version 76470 (0.0008) [2023-10-08 10:48:35,543][53852] Updated weights for policy 0, policy_version 76480 (0.0008) [2023-10-08 10:48:35,788][53885] Updated weights for policy 1, policy_version 76102 (0.0009) [2023-10-08 10:48:36,155][53885] Updated weights for policy 1, policy_version 76112 (0.0008) [2023-10-08 10:48:36,525][53885] Updated weights for policy 1, policy_version 76122 (0.0010) [2023-10-08 10:48:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 156270592. Throughput: 0: 1841.4, 1: 1821.2. Samples: 39073840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:37,016][52710] Avg episode reward: [(0, '33.340'), (1, '37.350')] [2023-10-08 10:48:39,263][53852] Updated weights for policy 0, policy_version 76490 (0.0008) [2023-10-08 10:48:39,621][53852] Updated weights for policy 0, policy_version 76500 (0.0008) [2023-10-08 10:48:39,990][53852] Updated weights for policy 0, policy_version 76510 (0.0009) [2023-10-08 10:48:40,096][53885] Updated weights for policy 1, policy_version 76132 (0.0009) [2023-10-08 10:48:40,463][53885] Updated weights for policy 1, policy_version 76142 (0.0008) [2023-10-08 10:48:40,836][53885] Updated weights for policy 1, policy_version 76152 (0.0008) [2023-10-08 10:48:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156336128. Throughput: 0: 1828.9, 1: 1822.3. Samples: 39085678. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:42,016][52710] Avg episode reward: [(0, '31.200'), (1, '33.030')] [2023-10-08 10:48:43,799][53852] Updated weights for policy 0, policy_version 76520 (0.0009) [2023-10-08 10:48:44,169][53852] Updated weights for policy 0, policy_version 76530 (0.0011) [2023-10-08 10:48:44,456][53885] Updated weights for policy 1, policy_version 76162 (0.0008) [2023-10-08 10:48:44,542][53852] Updated weights for policy 0, policy_version 76540 (0.0011) [2023-10-08 10:48:44,822][53885] Updated weights for policy 1, policy_version 76172 (0.0008) [2023-10-08 10:48:45,191][53885] Updated weights for policy 1, policy_version 76182 (0.0008) [2023-10-08 10:48:45,558][53885] Updated weights for policy 1, policy_version 76192 (0.0008) [2023-10-08 10:48:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156401664. Throughput: 0: 1830.7, 1: 1817.6. Samples: 39106748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:47,016][52710] Avg episode reward: [(0, '31.590'), (1, '33.200')] [2023-10-08 10:48:48,137][53852] Updated weights for policy 0, policy_version 76550 (0.0008) [2023-10-08 10:48:48,513][53852] Updated weights for policy 0, policy_version 76560 (0.0008) [2023-10-08 10:48:48,880][53852] Updated weights for policy 0, policy_version 76570 (0.0007) [2023-10-08 10:48:49,169][53885] Updated weights for policy 1, policy_version 76202 (0.0007) [2023-10-08 10:48:49,538][53885] Updated weights for policy 1, policy_version 76212 (0.0007) [2023-10-08 10:48:49,901][53885] Updated weights for policy 1, policy_version 76222 (0.0007) [2023-10-08 10:48:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 156467200. Throughput: 0: 1837.1, 1: 1826.7. Samples: 39129920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:52,016][52710] Avg episode reward: [(0, '32.080'), (1, '33.450')] [2023-10-08 10:48:52,478][53852] Updated weights for policy 0, policy_version 76580 (0.0007) [2023-10-08 10:48:52,846][53852] Updated weights for policy 0, policy_version 76590 (0.0007) [2023-10-08 10:48:53,204][53852] Updated weights for policy 0, policy_version 76600 (0.0009) [2023-10-08 10:48:53,514][53885] Updated weights for policy 1, policy_version 76232 (0.0007) [2023-10-08 10:48:53,875][53885] Updated weights for policy 1, policy_version 76242 (0.0010) [2023-10-08 10:48:54,240][53885] Updated weights for policy 1, policy_version 76252 (0.0011) [2023-10-08 10:48:56,811][53852] Updated weights for policy 0, policy_version 76610 (0.0007) [2023-10-08 10:48:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 156532736. Throughput: 0: 1835.0, 1: 1826.2. Samples: 39140138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:48:57,016][52710] Avg episode reward: [(0, '28.600'), (1, '32.170')] [2023-10-08 10:48:57,179][53852] Updated weights for policy 0, policy_version 76620 (0.0009) [2023-10-08 10:48:57,551][53852] Updated weights for policy 0, policy_version 76630 (0.0007) [2023-10-08 10:48:57,911][53852] Updated weights for policy 0, policy_version 76640 (0.0009) [2023-10-08 10:48:57,915][53885] Updated weights for policy 1, policy_version 76262 (0.0008) [2023-10-08 10:48:58,281][53885] Updated weights for policy 1, policy_version 76272 (0.0009) [2023-10-08 10:48:58,655][53885] Updated weights for policy 1, policy_version 76282 (0.0007) [2023-10-08 10:49:01,385][53852] Updated weights for policy 0, policy_version 76650 (0.0007) [2023-10-08 10:49:01,758][53852] Updated weights for policy 0, policy_version 76660 (0.0008) [2023-10-08 10:49:02,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 156598272. Throughput: 0: 1842.9, 1: 1838.0. Samples: 39163458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:49:02,016][52710] Avg episode reward: [(0, '29.710'), (1, '33.290')] [2023-10-08 10:49:02,134][53852] Updated weights for policy 0, policy_version 76670 (0.0008) [2023-10-08 10:49:02,258][53885] Updated weights for policy 1, policy_version 76292 (0.0007) [2023-10-08 10:49:02,630][53885] Updated weights for policy 1, policy_version 76302 (0.0008) [2023-10-08 10:49:03,005][53885] Updated weights for policy 1, policy_version 76312 (0.0008) [2023-10-08 10:49:05,693][53852] Updated weights for policy 0, policy_version 76680 (0.0009) [2023-10-08 10:49:06,069][53852] Updated weights for policy 0, policy_version 76690 (0.0009) [2023-10-08 10:49:06,442][53852] Updated weights for policy 0, policy_version 76700 (0.0009) [2023-10-08 10:49:06,865][53885] Updated weights for policy 1, policy_version 76322 (0.0008) [2023-10-08 10:49:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 156696576. Throughput: 0: 1831.9, 1: 1835.7. Samples: 39184986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:49:07,016][52710] Avg episode reward: [(0, '33.480'), (1, '35.220')] [2023-10-08 10:49:07,287][53885] Updated weights for policy 1, policy_version 76332 (0.0009) [2023-10-08 10:49:07,646][53885] Updated weights for policy 1, policy_version 76342 (0.0010) [2023-10-08 10:49:08,011][53885] Updated weights for policy 1, policy_version 76352 (0.0010) [2023-10-08 10:49:10,143][53852] Updated weights for policy 0, policy_version 76710 (0.0010) [2023-10-08 10:49:10,513][53852] Updated weights for policy 0, policy_version 76720 (0.0008) [2023-10-08 10:49:10,888][53852] Updated weights for policy 0, policy_version 76730 (0.0009) [2023-10-08 10:49:11,721][53885] Updated weights for policy 1, policy_version 76362 (0.0007) [2023-10-08 10:49:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156762112. Throughput: 0: 1847.0, 1: 1833.2. Samples: 39196330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:49:12,016][52710] Avg episode reward: [(0, '32.340'), (1, '30.500')] [2023-10-08 10:49:12,094][53885] Updated weights for policy 1, policy_version 76372 (0.0009) [2023-10-08 10:49:12,464][53885] Updated weights for policy 1, policy_version 76382 (0.0007) [2023-10-08 10:49:14,505][53852] Updated weights for policy 0, policy_version 76740 (0.0010) [2023-10-08 10:49:14,875][53852] Updated weights for policy 0, policy_version 76750 (0.0009) [2023-10-08 10:49:15,254][53852] Updated weights for policy 0, policy_version 76760 (0.0009) [2023-10-08 10:49:16,068][53885] Updated weights for policy 1, policy_version 76392 (0.0008) [2023-10-08 10:49:16,428][53885] Updated weights for policy 1, policy_version 76402 (0.0008) [2023-10-08 10:49:16,802][53885] Updated weights for policy 1, policy_version 76412 (0.0008) [2023-10-08 10:49:17,015][52710] Fps is (10 sec: 16384.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 156860416. Throughput: 0: 1836.1, 1: 1833.8. Samples: 39218090. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:17,015][52710] Avg episode reward: [(0, '28.750'), (1, '33.900')] [2023-10-08 10:49:18,756][53852] Updated weights for policy 0, policy_version 76770 (0.0010) [2023-10-08 10:49:19,164][53852] Updated weights for policy 0, policy_version 76780 (0.0007) [2023-10-08 10:49:19,532][53852] Updated weights for policy 0, policy_version 76790 (0.0008) [2023-10-08 10:49:19,901][53852] Updated weights for policy 0, policy_version 76800 (0.0009) [2023-10-08 10:49:20,504][53885] Updated weights for policy 1, policy_version 76422 (0.0008) [2023-10-08 10:49:20,874][53885] Updated weights for policy 1, policy_version 76432 (0.0008) [2023-10-08 10:49:21,236][53885] Updated weights for policy 1, policy_version 76442 (0.0008) [2023-10-08 10:49:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 156925952. Throughput: 0: 1853.8, 1: 1827.7. Samples: 39239506. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:22,016][52710] Avg episode reward: [(0, '33.320'), (1, '34.660')] [2023-10-08 10:49:23,572][53852] Updated weights for policy 0, policy_version 76810 (0.0008) [2023-10-08 10:49:23,934][53852] Updated weights for policy 0, policy_version 76820 (0.0009) [2023-10-08 10:49:24,303][53852] Updated weights for policy 0, policy_version 76830 (0.0009) [2023-10-08 10:49:25,032][53885] Updated weights for policy 1, policy_version 76452 (0.0009) [2023-10-08 10:49:25,399][53885] Updated weights for policy 1, policy_version 76462 (0.0007) [2023-10-08 10:49:25,770][53885] Updated weights for policy 1, policy_version 76472 (0.0008) [2023-10-08 10:49:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156991488. Throughput: 0: 1838.8, 1: 1832.9. Samples: 39250904. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:27,015][52710] Avg episode reward: [(0, '31.600'), (1, '31.510')] [2023-10-08 10:49:28,023][53852] Updated weights for policy 0, policy_version 76840 (0.0009) [2023-10-08 10:49:28,393][53852] Updated weights for policy 0, policy_version 76850 (0.0008) [2023-10-08 10:49:28,753][53852] Updated weights for policy 0, policy_version 76860 (0.0007) [2023-10-08 10:49:29,381][53885] Updated weights for policy 1, policy_version 76482 (0.0008) [2023-10-08 10:49:29,743][53885] Updated weights for policy 1, policy_version 76492 (0.0009) [2023-10-08 10:49:30,113][53885] Updated weights for policy 1, policy_version 76502 (0.0007) [2023-10-08 10:49:30,470][53885] Updated weights for policy 1, policy_version 76512 (0.0009) [2023-10-08 10:49:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157057024. Throughput: 0: 1856.4, 1: 1829.1. Samples: 39272594. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:32,016][52710] Avg episode reward: [(0, '32.240'), (1, '34.650')] [2023-10-08 10:49:32,453][53852] Updated weights for policy 0, policy_version 76870 (0.0007) [2023-10-08 10:49:32,823][53852] Updated weights for policy 0, policy_version 76880 (0.0008) [2023-10-08 10:49:33,188][53852] Updated weights for policy 0, policy_version 76890 (0.0008) [2023-10-08 10:49:34,112][53885] Updated weights for policy 1, policy_version 76522 (0.0007) [2023-10-08 10:49:34,474][53885] Updated weights for policy 1, policy_version 76532 (0.0007) [2023-10-08 10:49:34,841][53885] Updated weights for policy 1, policy_version 76542 (0.0007) [2023-10-08 10:49:36,764][53852] Updated weights for policy 0, policy_version 76900 (0.0008) [2023-10-08 10:49:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 157122560. Throughput: 0: 1857.2, 1: 1832.1. Samples: 39295934. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:37,016][52710] Avg episode reward: [(0, '32.670'), (1, '34.860')] [2023-10-08 10:49:37,131][53852] Updated weights for policy 0, policy_version 76910 (0.0008) [2023-10-08 10:49:37,499][53852] Updated weights for policy 0, policy_version 76920 (0.0007) [2023-10-08 10:49:38,604][53885] Updated weights for policy 1, policy_version 76552 (0.0008) [2023-10-08 10:49:38,979][53885] Updated weights for policy 1, policy_version 76562 (0.0008) [2023-10-08 10:49:39,343][53885] Updated weights for policy 1, policy_version 76572 (0.0007) [2023-10-08 10:49:41,144][53852] Updated weights for policy 0, policy_version 76930 (0.0007) [2023-10-08 10:49:41,519][53852] Updated weights for policy 0, policy_version 76940 (0.0007) [2023-10-08 10:49:41,888][53852] Updated weights for policy 0, policy_version 76950 (0.0008) [2023-10-08 10:49:42,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 157188096. Throughput: 0: 1853.4, 1: 1827.6. Samples: 39305784. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:42,015][52710] Avg episode reward: [(0, '33.210'), (1, '32.660')] [2023-10-08 10:49:42,256][53852] Updated weights for policy 0, policy_version 76960 (0.0007) [2023-10-08 10:49:42,940][53885] Updated weights for policy 1, policy_version 76582 (0.0008) [2023-10-08 10:49:43,308][53885] Updated weights for policy 1, policy_version 76592 (0.0009) [2023-10-08 10:49:43,685][53885] Updated weights for policy 1, policy_version 76602 (0.0009) [2023-10-08 10:49:45,692][53852] Updated weights for policy 0, policy_version 76970 (0.0011) [2023-10-08 10:49:46,070][53852] Updated weights for policy 0, policy_version 76980 (0.0010) [2023-10-08 10:49:46,445][53852] Updated weights for policy 0, policy_version 76990 (0.0009) [2023-10-08 10:49:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157286400. Throughput: 0: 1846.7, 1: 1826.5. Samples: 39328752. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:47,016][52710] Avg episode reward: [(0, '31.680'), (1, '32.170')] [2023-10-08 10:49:47,329][53885] Updated weights for policy 1, policy_version 76612 (0.0010) [2023-10-08 10:49:47,693][53885] Updated weights for policy 1, policy_version 76622 (0.0008) [2023-10-08 10:49:48,061][53885] Updated weights for policy 1, policy_version 76632 (0.0008) [2023-10-08 10:49:50,078][53852] Updated weights for policy 0, policy_version 77000 (0.0008) [2023-10-08 10:49:50,449][53852] Updated weights for policy 0, policy_version 77010 (0.0011) [2023-10-08 10:49:50,828][53852] Updated weights for policy 0, policy_version 77020 (0.0009) [2023-10-08 10:49:51,792][53885] Updated weights for policy 1, policy_version 76642 (0.0008) [2023-10-08 10:49:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 157351936. Throughput: 0: 1849.5, 1: 1831.1. Samples: 39350612. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:52,015][52710] Avg episode reward: [(0, '31.830'), (1, '34.700')] [2023-10-08 10:49:52,168][53885] Updated weights for policy 1, policy_version 76652 (0.0009) [2023-10-08 10:49:52,533][53885] Updated weights for policy 1, policy_version 76662 (0.0009) [2023-10-08 10:49:52,902][53885] Updated weights for policy 1, policy_version 76672 (0.0007) [2023-10-08 10:49:54,508][53852] Updated weights for policy 0, policy_version 77030 (0.0010) [2023-10-08 10:49:54,876][53852] Updated weights for policy 0, policy_version 77040 (0.0010) [2023-10-08 10:49:55,242][53852] Updated weights for policy 0, policy_version 77050 (0.0009) [2023-10-08 10:49:56,548][53885] Updated weights for policy 1, policy_version 76682 (0.0008) [2023-10-08 10:49:56,915][53885] Updated weights for policy 1, policy_version 76692 (0.0007) [2023-10-08 10:49:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157417472. Throughput: 0: 1835.9, 1: 1834.1. Samples: 39361478. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:49:57,016][52710] Avg episode reward: [(0, '29.130'), (1, '32.070')] [2023-10-08 10:49:57,280][53885] Updated weights for policy 1, policy_version 76702 (0.0008) [2023-10-08 10:49:58,917][53852] Updated weights for policy 0, policy_version 77060 (0.0009) [2023-10-08 10:49:59,290][53852] Updated weights for policy 0, policy_version 77070 (0.0007) [2023-10-08 10:49:59,663][53852] Updated weights for policy 0, policy_version 77080 (0.0010) [2023-10-08 10:50:00,917][53885] Updated weights for policy 1, policy_version 76712 (0.0008) [2023-10-08 10:50:01,288][53885] Updated weights for policy 1, policy_version 76722 (0.0008) [2023-10-08 10:50:01,659][53885] Updated weights for policy 1, policy_version 76732 (0.0008) [2023-10-08 10:50:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 157515776. Throughput: 0: 1840.1, 1: 1831.6. Samples: 39383318. Policy #0 lag: (min: 9.0, avg: 21.4, max: 41.0) [2023-10-08 10:50:02,016][52710] Avg episode reward: [(0, '32.140'), (1, '33.350')] [2023-10-08 10:50:03,244][53852] Updated weights for policy 0, policy_version 77090 (0.0008) [2023-10-08 10:50:03,624][53852] Updated weights for policy 0, policy_version 77100 (0.0008) [2023-10-08 10:50:04,000][53852] Updated weights for policy 0, policy_version 77110 (0.0009) [2023-10-08 10:50:04,360][53852] Updated weights for policy 0, policy_version 77120 (0.0007) [2023-10-08 10:50:05,227][53885] Updated weights for policy 1, policy_version 76742 (0.0009) [2023-10-08 10:50:05,594][53885] Updated weights for policy 1, policy_version 76752 (0.0008) [2023-10-08 10:50:05,976][53885] Updated weights for policy 1, policy_version 76762 (0.0008) [2023-10-08 10:50:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157581312. Throughput: 0: 1843.8, 1: 1837.0. Samples: 39405144. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:07,016][52710] Avg episode reward: [(0, '30.080'), (1, '34.450')] [2023-10-08 10:50:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000077120_78970880.pth... [2023-10-08 10:50:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000076768_78610432.pth... [2023-10-08 10:50:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000075040_76840960.pth [2023-10-08 10:50:07,066][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000075424_77234176.pth [2023-10-08 10:50:07,069][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000076768_78610432.pth [2023-10-08 10:50:07,071][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000077120_78970880.pth [2023-10-08 10:50:08,161][53852] Updated weights for policy 0, policy_version 77130 (0.0009) [2023-10-08 10:50:08,530][53852] Updated weights for policy 0, policy_version 77140 (0.0010) [2023-10-08 10:50:08,906][53852] Updated weights for policy 0, policy_version 77150 (0.0008) [2023-10-08 10:50:09,563][53885] Updated weights for policy 1, policy_version 76772 (0.0008) [2023-10-08 10:50:09,930][53885] Updated weights for policy 1, policy_version 76782 (0.0007) [2023-10-08 10:50:10,295][53885] Updated weights for policy 1, policy_version 76792 (0.0008) [2023-10-08 10:50:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157646848. Throughput: 0: 1839.6, 1: 1834.9. Samples: 39416260. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:12,016][52710] Avg episode reward: [(0, '30.030'), (1, '31.240')] [2023-10-08 10:50:12,443][53852] Updated weights for policy 0, policy_version 77160 (0.0011) [2023-10-08 10:50:12,816][53852] Updated weights for policy 0, policy_version 77170 (0.0011) [2023-10-08 10:50:13,182][53852] Updated weights for policy 0, policy_version 77180 (0.0007) [2023-10-08 10:50:13,934][53885] Updated weights for policy 1, policy_version 76802 (0.0008) [2023-10-08 10:50:14,303][53885] Updated weights for policy 1, policy_version 76812 (0.0007) [2023-10-08 10:50:14,674][53885] Updated weights for policy 1, policy_version 76822 (0.0010) [2023-10-08 10:50:15,036][53885] Updated weights for policy 1, policy_version 76832 (0.0007) [2023-10-08 10:50:16,765][53852] Updated weights for policy 0, policy_version 77190 (0.0007) [2023-10-08 10:50:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 157712384. Throughput: 0: 1844.2, 1: 1842.9. Samples: 39438516. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:17,016][52710] Avg episode reward: [(0, '29.230'), (1, '30.620')] [2023-10-08 10:50:17,119][53852] Updated weights for policy 0, policy_version 77200 (0.0007) [2023-10-08 10:50:17,482][53852] Updated weights for policy 0, policy_version 77210 (0.0007) [2023-10-08 10:50:18,766][53885] Updated weights for policy 1, policy_version 76842 (0.0010) [2023-10-08 10:50:19,136][53885] Updated weights for policy 1, policy_version 76852 (0.0008) [2023-10-08 10:50:19,492][53885] Updated weights for policy 1, policy_version 76862 (0.0008) [2023-10-08 10:50:21,202][53852] Updated weights for policy 0, policy_version 77220 (0.0008) [2023-10-08 10:50:21,573][53852] Updated weights for policy 0, policy_version 77230 (0.0008) [2023-10-08 10:50:21,948][53852] Updated weights for policy 0, policy_version 77240 (0.0009) [2023-10-08 10:50:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 157777920. Throughput: 0: 1826.7, 1: 1843.9. Samples: 39461108. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:22,016][52710] Avg episode reward: [(0, '29.120'), (1, '31.680')] [2023-10-08 10:50:23,090][53885] Updated weights for policy 1, policy_version 76872 (0.0010) [2023-10-08 10:50:23,457][53885] Updated weights for policy 1, policy_version 76882 (0.0008) [2023-10-08 10:50:23,824][53885] Updated weights for policy 1, policy_version 76892 (0.0008) [2023-10-08 10:50:25,611][53852] Updated weights for policy 0, policy_version 77250 (0.0009) [2023-10-08 10:50:25,979][53852] Updated weights for policy 0, policy_version 77260 (0.0008) [2023-10-08 10:50:26,350][53852] Updated weights for policy 0, policy_version 77270 (0.0010) [2023-10-08 10:50:26,726][53852] Updated weights for policy 0, policy_version 77280 (0.0009) [2023-10-08 10:50:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 157876224. Throughput: 0: 1844.3, 1: 1846.1. Samples: 39471854. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:27,016][52710] Avg episode reward: [(0, '30.430'), (1, '34.580')] [2023-10-08 10:50:27,395][53885] Updated weights for policy 1, policy_version 76902 (0.0007) [2023-10-08 10:50:27,761][53885] Updated weights for policy 1, policy_version 76912 (0.0007) [2023-10-08 10:50:28,126][53885] Updated weights for policy 1, policy_version 76922 (0.0008) [2023-10-08 10:50:30,288][53852] Updated weights for policy 0, policy_version 77290 (0.0008) [2023-10-08 10:50:30,653][53852] Updated weights for policy 0, policy_version 77300 (0.0009) [2023-10-08 10:50:31,020][53852] Updated weights for policy 0, policy_version 77310 (0.0008) [2023-10-08 10:50:31,674][53885] Updated weights for policy 1, policy_version 76932 (0.0010) [2023-10-08 10:50:32,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 157941760. Throughput: 0: 1826.4, 1: 1849.2. Samples: 39494152. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:32,016][52710] Avg episode reward: [(0, '30.060'), (1, '29.780')] [2023-10-08 10:50:32,036][53885] Updated weights for policy 1, policy_version 76942 (0.0008) [2023-10-08 10:50:32,402][53885] Updated weights for policy 1, policy_version 76952 (0.0008) [2023-10-08 10:50:34,661][53852] Updated weights for policy 0, policy_version 77320 (0.0007) [2023-10-08 10:50:35,036][53852] Updated weights for policy 0, policy_version 77330 (0.0009) [2023-10-08 10:50:35,403][53852] Updated weights for policy 0, policy_version 77340 (0.0008) [2023-10-08 10:50:36,057][53885] Updated weights for policy 1, policy_version 76962 (0.0007) [2023-10-08 10:50:36,421][53885] Updated weights for policy 1, policy_version 76972 (0.0010) [2023-10-08 10:50:36,789][53885] Updated weights for policy 1, policy_version 76982 (0.0008) [2023-10-08 10:50:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 158007296. Throughput: 0: 1834.8, 1: 1834.4. Samples: 39515730. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:37,016][52710] Avg episode reward: [(0, '30.260'), (1, '34.810')] [2023-10-08 10:50:37,168][53885] Updated weights for policy 1, policy_version 76992 (0.0007) [2023-10-08 10:50:39,027][53852] Updated weights for policy 0, policy_version 77350 (0.0008) [2023-10-08 10:50:39,397][53852] Updated weights for policy 0, policy_version 77360 (0.0008) [2023-10-08 10:50:39,765][53852] Updated weights for policy 0, policy_version 77370 (0.0007) [2023-10-08 10:50:40,840][53885] Updated weights for policy 1, policy_version 77002 (0.0009) [2023-10-08 10:50:41,206][53885] Updated weights for policy 1, policy_version 77012 (0.0010) [2023-10-08 10:50:41,577][53885] Updated weights for policy 1, policy_version 77022 (0.0009) [2023-10-08 10:50:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 158105600. Throughput: 0: 1828.8, 1: 1854.7. Samples: 39527234. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:42,015][52710] Avg episode reward: [(0, '30.110'), (1, '33.230')] [2023-10-08 10:50:43,352][53852] Updated weights for policy 0, policy_version 77380 (0.0007) [2023-10-08 10:50:43,719][53852] Updated weights for policy 0, policy_version 77390 (0.0007) [2023-10-08 10:50:44,085][53852] Updated weights for policy 0, policy_version 77400 (0.0009) [2023-10-08 10:50:45,252][53885] Updated weights for policy 1, policy_version 77032 (0.0008) [2023-10-08 10:50:45,617][53885] Updated weights for policy 1, policy_version 77042 (0.0008) [2023-10-08 10:50:45,987][53885] Updated weights for policy 1, policy_version 77052 (0.0008) [2023-10-08 10:50:47,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158171136. Throughput: 0: 1845.7, 1: 1830.3. Samples: 39548740. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:47,016][52710] Avg episode reward: [(0, '33.900'), (1, '35.590')] [2023-10-08 10:50:47,820][53852] Updated weights for policy 0, policy_version 77410 (0.0009) [2023-10-08 10:50:48,185][53852] Updated weights for policy 0, policy_version 77420 (0.0010) [2023-10-08 10:50:48,564][53852] Updated weights for policy 0, policy_version 77430 (0.0007) [2023-10-08 10:50:48,932][53852] Updated weights for policy 0, policy_version 77440 (0.0010) [2023-10-08 10:50:49,763][53885] Updated weights for policy 1, policy_version 77062 (0.0008) [2023-10-08 10:50:50,130][53885] Updated weights for policy 1, policy_version 77072 (0.0007) [2023-10-08 10:50:50,503][53885] Updated weights for policy 1, policy_version 77082 (0.0007) [2023-10-08 10:50:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158236672. Throughput: 0: 1841.0, 1: 1839.5. Samples: 39570766. Policy #0 lag: (min: 7.0, avg: 8.7, max: 35.0) [2023-10-08 10:50:52,016][52710] Avg episode reward: [(0, '31.370'), (1, '33.080')] [2023-10-08 10:50:52,513][53852] Updated weights for policy 0, policy_version 77450 (0.0007) [2023-10-08 10:50:52,868][53852] Updated weights for policy 0, policy_version 77460 (0.0007) [2023-10-08 10:50:53,240][53852] Updated weights for policy 0, policy_version 77470 (0.0009) [2023-10-08 10:50:54,170][53885] Updated weights for policy 1, policy_version 77092 (0.0009) [2023-10-08 10:50:54,537][53885] Updated weights for policy 1, policy_version 77102 (0.0009) [2023-10-08 10:50:54,896][53885] Updated weights for policy 1, policy_version 77112 (0.0008) [2023-10-08 10:50:56,880][53852] Updated weights for policy 0, policy_version 77480 (0.0008) [2023-10-08 10:50:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158302208. Throughput: 0: 1847.9, 1: 1828.7. Samples: 39581708. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:50:57,016][52710] Avg episode reward: [(0, '31.050'), (1, '36.590')] [2023-10-08 10:50:57,251][53852] Updated weights for policy 0, policy_version 77490 (0.0009) [2023-10-08 10:50:57,623][53852] Updated weights for policy 0, policy_version 77500 (0.0007) [2023-10-08 10:50:58,413][53885] Updated weights for policy 1, policy_version 77122 (0.0008) [2023-10-08 10:50:58,768][53885] Updated weights for policy 1, policy_version 77132 (0.0008) [2023-10-08 10:50:59,130][53885] Updated weights for policy 1, policy_version 77142 (0.0007) [2023-10-08 10:50:59,499][53885] Updated weights for policy 1, policy_version 77152 (0.0010) [2023-10-08 10:51:01,362][53852] Updated weights for policy 0, policy_version 77510 (0.0009) [2023-10-08 10:51:01,729][53852] Updated weights for policy 0, policy_version 77520 (0.0011) [2023-10-08 10:51:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 158367744. Throughput: 0: 1839.1, 1: 1839.1. Samples: 39604036. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:02,015][52710] Avg episode reward: [(0, '31.600'), (1, '35.200')] [2023-10-08 10:51:02,116][53852] Updated weights for policy 0, policy_version 77530 (0.0010) [2023-10-08 10:51:03,321][53885] Updated weights for policy 1, policy_version 77162 (0.0007) [2023-10-08 10:51:03,686][53885] Updated weights for policy 1, policy_version 77172 (0.0008) [2023-10-08 10:51:04,054][53885] Updated weights for policy 1, policy_version 77182 (0.0007) [2023-10-08 10:51:05,774][53852] Updated weights for policy 0, policy_version 77540 (0.0008) [2023-10-08 10:51:06,130][53852] Updated weights for policy 0, policy_version 77550 (0.0007) [2023-10-08 10:51:06,497][53852] Updated weights for policy 0, policy_version 77560 (0.0011) [2023-10-08 10:51:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 158466048. Throughput: 0: 1827.5, 1: 1837.3. Samples: 39626024. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:07,016][52710] Avg episode reward: [(0, '33.810'), (1, '33.800')] [2023-10-08 10:51:07,727][53885] Updated weights for policy 1, policy_version 77192 (0.0008) [2023-10-08 10:51:08,092][53885] Updated weights for policy 1, policy_version 77202 (0.0010) [2023-10-08 10:51:08,459][53885] Updated weights for policy 1, policy_version 77212 (0.0010) [2023-10-08 10:51:10,187][53852] Updated weights for policy 0, policy_version 77570 (0.0009) [2023-10-08 10:51:10,554][53852] Updated weights for policy 0, policy_version 77580 (0.0008) [2023-10-08 10:51:10,932][53852] Updated weights for policy 0, policy_version 77590 (0.0008) [2023-10-08 10:51:11,294][53852] Updated weights for policy 0, policy_version 77600 (0.0007) [2023-10-08 10:51:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 158531584. Throughput: 0: 1837.5, 1: 1835.0. Samples: 39637116. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:12,016][52710] Avg episode reward: [(0, '30.870'), (1, '36.190')] [2023-10-08 10:51:12,124][53885] Updated weights for policy 1, policy_version 77222 (0.0008) [2023-10-08 10:51:12,485][53885] Updated weights for policy 1, policy_version 77232 (0.0009) [2023-10-08 10:51:12,861][53885] Updated weights for policy 1, policy_version 77242 (0.0008) [2023-10-08 10:51:14,939][53852] Updated weights for policy 0, policy_version 77610 (0.0009) [2023-10-08 10:51:15,304][53852] Updated weights for policy 0, policy_version 77620 (0.0008) [2023-10-08 10:51:15,675][53852] Updated weights for policy 0, policy_version 77630 (0.0008) [2023-10-08 10:51:16,461][53885] Updated weights for policy 1, policy_version 77252 (0.0009) [2023-10-08 10:51:16,831][53885] Updated weights for policy 1, policy_version 77262 (0.0010) [2023-10-08 10:51:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 158597120. Throughput: 0: 1829.3, 1: 1839.8. Samples: 39659262. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:17,015][52710] Avg episode reward: [(0, '29.760'), (1, '35.490')] [2023-10-08 10:51:17,201][53885] Updated weights for policy 1, policy_version 77272 (0.0009) [2023-10-08 10:51:19,237][53852] Updated weights for policy 0, policy_version 77640 (0.0008) [2023-10-08 10:51:19,599][53852] Updated weights for policy 0, policy_version 77650 (0.0008) [2023-10-08 10:51:19,966][53852] Updated weights for policy 0, policy_version 77660 (0.0008) [2023-10-08 10:51:20,766][53885] Updated weights for policy 1, policy_version 77282 (0.0010) [2023-10-08 10:51:21,135][53885] Updated weights for policy 1, policy_version 77292 (0.0008) [2023-10-08 10:51:21,500][53885] Updated weights for policy 1, policy_version 77302 (0.0011) [2023-10-08 10:51:21,874][53885] Updated weights for policy 1, policy_version 77312 (0.0011) [2023-10-08 10:51:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 158695424. Throughput: 0: 1847.9, 1: 1827.1. Samples: 39681104. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:22,016][52710] Avg episode reward: [(0, '32.620'), (1, '31.000')] [2023-10-08 10:51:23,787][53852] Updated weights for policy 0, policy_version 77670 (0.0009) [2023-10-08 10:51:24,155][53852] Updated weights for policy 0, policy_version 77680 (0.0007) [2023-10-08 10:51:24,526][53852] Updated weights for policy 0, policy_version 77690 (0.0008) [2023-10-08 10:51:25,344][53885] Updated weights for policy 1, policy_version 77322 (0.0007) [2023-10-08 10:51:25,707][53885] Updated weights for policy 1, policy_version 77332 (0.0007) [2023-10-08 10:51:26,082][53885] Updated weights for policy 1, policy_version 77342 (0.0009) [2023-10-08 10:51:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158760960. Throughput: 0: 1835.4, 1: 1838.6. Samples: 39692562. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:27,016][52710] Avg episode reward: [(0, '28.830'), (1, '34.920')] [2023-10-08 10:51:27,925][53852] Updated weights for policy 0, policy_version 77700 (0.0008) [2023-10-08 10:51:28,287][53852] Updated weights for policy 0, policy_version 77710 (0.0010) [2023-10-08 10:51:28,665][53852] Updated weights for policy 0, policy_version 77720 (0.0007) [2023-10-08 10:51:29,860][53885] Updated weights for policy 1, policy_version 77352 (0.0008) [2023-10-08 10:51:30,227][53885] Updated weights for policy 1, policy_version 77362 (0.0009) [2023-10-08 10:51:30,605][53885] Updated weights for policy 1, policy_version 77372 (0.0008) [2023-10-08 10:51:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158826496. Throughput: 0: 1844.8, 1: 1830.7. Samples: 39714140. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:32,015][52710] Avg episode reward: [(0, '29.450'), (1, '34.020')] [2023-10-08 10:51:32,342][53852] Updated weights for policy 0, policy_version 77730 (0.0008) [2023-10-08 10:51:32,717][53852] Updated weights for policy 0, policy_version 77740 (0.0008) [2023-10-08 10:51:33,079][53852] Updated weights for policy 0, policy_version 77750 (0.0009) [2023-10-08 10:51:33,447][53852] Updated weights for policy 0, policy_version 77760 (0.0011) [2023-10-08 10:51:34,200][53885] Updated weights for policy 1, policy_version 77382 (0.0009) [2023-10-08 10:51:34,558][53885] Updated weights for policy 1, policy_version 77392 (0.0008) [2023-10-08 10:51:34,927][53885] Updated weights for policy 1, policy_version 77402 (0.0008) [2023-10-08 10:51:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158892032. Throughput: 0: 1838.5, 1: 1848.2. Samples: 39736668. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:37,016][52710] Avg episode reward: [(0, '29.660'), (1, '34.460')] [2023-10-08 10:51:37,139][53852] Updated weights for policy 0, policy_version 77770 (0.0009) [2023-10-08 10:51:37,507][53852] Updated weights for policy 0, policy_version 77780 (0.0008) [2023-10-08 10:51:37,879][53852] Updated weights for policy 0, policy_version 77790 (0.0008) [2023-10-08 10:51:38,576][53885] Updated weights for policy 1, policy_version 77412 (0.0008) [2023-10-08 10:51:38,946][53885] Updated weights for policy 1, policy_version 77422 (0.0008) [2023-10-08 10:51:39,315][53885] Updated weights for policy 1, policy_version 77432 (0.0007) [2023-10-08 10:51:41,500][53852] Updated weights for policy 0, policy_version 77800 (0.0007) [2023-10-08 10:51:41,885][53852] Updated weights for policy 0, policy_version 77810 (0.0008) [2023-10-08 10:51:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 158957568. Throughput: 0: 1840.4, 1: 1826.8. Samples: 39746730. Policy #0 lag: (min: 20.0, avg: 25.4, max: 52.0) [2023-10-08 10:51:42,016][52710] Avg episode reward: [(0, '32.180'), (1, '34.600')] [2023-10-08 10:51:42,255][53852] Updated weights for policy 0, policy_version 77820 (0.0008) [2023-10-08 10:51:43,018][53885] Updated weights for policy 1, policy_version 77442 (0.0008) [2023-10-08 10:51:43,371][53885] Updated weights for policy 1, policy_version 77452 (0.0009) [2023-10-08 10:51:43,745][53885] Updated weights for policy 1, policy_version 77462 (0.0007) [2023-10-08 10:51:44,104][53885] Updated weights for policy 1, policy_version 77472 (0.0007) [2023-10-08 10:51:45,920][53852] Updated weights for policy 0, policy_version 77830 (0.0010) [2023-10-08 10:51:46,294][53852] Updated weights for policy 0, policy_version 77840 (0.0010) [2023-10-08 10:51:46,651][53852] Updated weights for policy 0, policy_version 77850 (0.0009) [2023-10-08 10:51:47,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 159055872. Throughput: 0: 1837.0, 1: 1836.6. Samples: 39769346. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:51:47,015][52710] Avg episode reward: [(0, '34.500'), (1, '33.050')] [2023-10-08 10:51:47,865][53885] Updated weights for policy 1, policy_version 77482 (0.0010) [2023-10-08 10:51:48,239][53885] Updated weights for policy 1, policy_version 77492 (0.0010) [2023-10-08 10:51:48,604][53885] Updated weights for policy 1, policy_version 77502 (0.0007) [2023-10-08 10:51:50,307][53852] Updated weights for policy 0, policy_version 77860 (0.0009) [2023-10-08 10:51:50,675][53852] Updated weights for policy 0, policy_version 77870 (0.0007) [2023-10-08 10:51:51,057][53852] Updated weights for policy 0, policy_version 77880 (0.0008) [2023-10-08 10:51:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 159121408. Throughput: 0: 1831.7, 1: 1834.4. Samples: 39790998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:51:52,016][52710] Avg episode reward: [(0, '31.180'), (1, '32.680')] [2023-10-08 10:51:52,309][53885] Updated weights for policy 1, policy_version 77512 (0.0008) [2023-10-08 10:51:52,667][53885] Updated weights for policy 1, policy_version 77522 (0.0008) [2023-10-08 10:51:53,033][53885] Updated weights for policy 1, policy_version 77532 (0.0009) [2023-10-08 10:51:54,724][53852] Updated weights for policy 0, policy_version 77890 (0.0008) [2023-10-08 10:51:55,090][53852] Updated weights for policy 0, policy_version 77900 (0.0007) [2023-10-08 10:51:55,452][53852] Updated weights for policy 0, policy_version 77910 (0.0009) [2023-10-08 10:51:55,819][53852] Updated weights for policy 0, policy_version 77920 (0.0010) [2023-10-08 10:51:56,626][53885] Updated weights for policy 1, policy_version 77542 (0.0009) [2023-10-08 10:51:56,986][53885] Updated weights for policy 1, policy_version 77552 (0.0008) [2023-10-08 10:51:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 159186944. Throughput: 0: 1841.6, 1: 1834.2. Samples: 39802526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:51:57,015][52710] Avg episode reward: [(0, '32.310'), (1, '36.060')] [2023-10-08 10:51:57,357][53885] Updated weights for policy 1, policy_version 77562 (0.0007) [2023-10-08 10:51:59,485][53852] Updated weights for policy 0, policy_version 77930 (0.0010) [2023-10-08 10:51:59,857][53852] Updated weights for policy 0, policy_version 77940 (0.0008) [2023-10-08 10:52:00,225][53852] Updated weights for policy 0, policy_version 77950 (0.0008) [2023-10-08 10:52:01,190][53885] Updated weights for policy 1, policy_version 77572 (0.0008) [2023-10-08 10:52:01,554][53885] Updated weights for policy 1, policy_version 77582 (0.0007) [2023-10-08 10:52:01,920][53885] Updated weights for policy 1, policy_version 77592 (0.0008) [2023-10-08 10:52:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 159252480. Throughput: 0: 1834.4, 1: 1827.3. Samples: 39824038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:52:02,015][52710] Avg episode reward: [(0, '35.650'), (1, '33.750')] [2023-10-08 10:52:03,799][53852] Updated weights for policy 0, policy_version 77960 (0.0008) [2023-10-08 10:52:04,166][53852] Updated weights for policy 0, policy_version 77970 (0.0007) [2023-10-08 10:52:04,536][53852] Updated weights for policy 0, policy_version 77980 (0.0009) [2023-10-08 10:52:05,577][53885] Updated weights for policy 1, policy_version 77602 (0.0009) [2023-10-08 10:52:05,952][53885] Updated weights for policy 1, policy_version 77612 (0.0007) [2023-10-08 10:52:06,329][53885] Updated weights for policy 1, policy_version 77622 (0.0008) [2023-10-08 10:52:06,692][53885] Updated weights for policy 1, policy_version 77632 (0.0009) [2023-10-08 10:52:07,015][52710] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 159350784. Throughput: 0: 1834.1, 1: 1823.5. Samples: 39845696. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:52:07,016][52710] Avg episode reward: [(0, '34.140'), (1, '33.200')] [2023-10-08 10:52:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000077984_79855616.pth... [2023-10-08 10:52:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000077632_79495168.pth... [2023-10-08 10:52:07,068][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000076256_78086144.pth [2023-10-08 10:52:07,068][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000075904_77725696.pth [2023-10-08 10:52:08,281][53852] Updated weights for policy 0, policy_version 77990 (0.0008) [2023-10-08 10:52:08,649][53852] Updated weights for policy 0, policy_version 78000 (0.0008) [2023-10-08 10:52:09,028][53852] Updated weights for policy 0, policy_version 78010 (0.0009) [2023-10-08 10:52:10,405][53885] Updated weights for policy 1, policy_version 77642 (0.0009) [2023-10-08 10:52:10,769][53885] Updated weights for policy 1, policy_version 77652 (0.0009) [2023-10-08 10:52:11,135][53885] Updated weights for policy 1, policy_version 77662 (0.0007) [2023-10-08 10:52:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 159416320. Throughput: 0: 1827.4, 1: 1826.8. Samples: 39857000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:52:12,015][52710] Avg episode reward: [(0, '31.310'), (1, '32.370')] [2023-10-08 10:52:12,675][53852] Updated weights for policy 0, policy_version 78020 (0.0009) [2023-10-08 10:52:13,040][53852] Updated weights for policy 0, policy_version 78030 (0.0007) [2023-10-08 10:52:13,417][53852] Updated weights for policy 0, policy_version 78040 (0.0007) [2023-10-08 10:52:14,693][53885] Updated weights for policy 1, policy_version 77672 (0.0009) [2023-10-08 10:52:15,064][53885] Updated weights for policy 1, policy_version 77682 (0.0008) [2023-10-08 10:52:15,433][53885] Updated weights for policy 1, policy_version 77692 (0.0008) [2023-10-08 10:52:17,002][53852] Updated weights for policy 0, policy_version 78050 (0.0009) [2023-10-08 10:52:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 159481856. Throughput: 0: 1832.4, 1: 1826.6. Samples: 39878794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:52:17,015][52710] Avg episode reward: [(0, '33.200'), (1, '32.650')] [2023-10-08 10:52:17,362][53852] Updated weights for policy 0, policy_version 78060 (0.0008) [2023-10-08 10:52:17,731][53852] Updated weights for policy 0, policy_version 78070 (0.0009) [2023-10-08 10:52:18,101][53852] Updated weights for policy 0, policy_version 78080 (0.0009) [2023-10-08 10:52:19,118][53885] Updated weights for policy 1, policy_version 77702 (0.0008) [2023-10-08 10:52:19,478][53885] Updated weights for policy 1, policy_version 77712 (0.0007) [2023-10-08 10:52:19,845][53885] Updated weights for policy 1, policy_version 77722 (0.0007) [2023-10-08 10:52:21,527][53852] Updated weights for policy 0, policy_version 78090 (0.0008) [2023-10-08 10:52:21,891][53852] Updated weights for policy 0, policy_version 78100 (0.0007) [2023-10-08 10:52:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 159547392. Throughput: 0: 1839.0, 1: 1826.9. Samples: 39901636. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:52:22,016][52710] Avg episode reward: [(0, '32.280'), (1, '34.430')] [2023-10-08 10:52:22,259][53852] Updated weights for policy 0, policy_version 78110 (0.0007) [2023-10-08 10:52:23,483][53885] Updated weights for policy 1, policy_version 77732 (0.0008) [2023-10-08 10:52:23,849][53885] Updated weights for policy 1, policy_version 77742 (0.0008) [2023-10-08 10:52:24,213][53885] Updated weights for policy 1, policy_version 77752 (0.0010) [2023-10-08 10:52:25,838][53852] Updated weights for policy 0, policy_version 78120 (0.0009) [2023-10-08 10:52:26,213][53852] Updated weights for policy 0, policy_version 78130 (0.0008) [2023-10-08 10:52:26,583][53852] Updated weights for policy 0, policy_version 78140 (0.0008) [2023-10-08 10:52:27,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 159645696. Throughput: 0: 1850.7, 1: 1829.5. Samples: 39912340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:52:27,016][52710] Avg episode reward: [(0, '29.870'), (1, '34.480')] [2023-10-08 10:52:27,878][53885] Updated weights for policy 1, policy_version 77762 (0.0010) [2023-10-08 10:52:28,242][53885] Updated weights for policy 1, policy_version 77772 (0.0009) [2023-10-08 10:52:28,614][53885] Updated weights for policy 1, policy_version 77782 (0.0007) [2023-10-08 10:52:28,985][53885] Updated weights for policy 1, policy_version 77792 (0.0007) [2023-10-08 10:52:30,136][53852] Updated weights for policy 0, policy_version 78150 (0.0007) [2023-10-08 10:52:30,519][53852] Updated weights for policy 0, policy_version 78160 (0.0008) [2023-10-08 10:52:30,880][53852] Updated weights for policy 0, policy_version 78170 (0.0009) [2023-10-08 10:52:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 159711232. Throughput: 0: 1842.8, 1: 1832.7. Samples: 39934746. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:52:32,015][52710] Avg episode reward: [(0, '32.020'), (1, '32.160')] [2023-10-08 10:52:32,626][53885] Updated weights for policy 1, policy_version 77802 (0.0009) [2023-10-08 10:52:32,989][53885] Updated weights for policy 1, policy_version 77812 (0.0007) [2023-10-08 10:52:33,353][53885] Updated weights for policy 1, policy_version 77822 (0.0007) [2023-10-08 10:52:34,570][53852] Updated weights for policy 0, policy_version 78180 (0.0009) [2023-10-08 10:52:34,942][53852] Updated weights for policy 0, policy_version 78190 (0.0008) [2023-10-08 10:52:35,306][53852] Updated weights for policy 0, policy_version 78200 (0.0009) [2023-10-08 10:52:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 159776768. Throughput: 0: 1852.6, 1: 1836.0. Samples: 39956982. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:52:37,016][52710] Avg episode reward: [(0, '32.010'), (1, '32.890')] [2023-10-08 10:52:37,110][53885] Updated weights for policy 1, policy_version 77832 (0.0007) [2023-10-08 10:52:37,481][53885] Updated weights for policy 1, policy_version 77842 (0.0008) [2023-10-08 10:52:37,847][53885] Updated weights for policy 1, policy_version 77852 (0.0007) [2023-10-08 10:52:38,953][53852] Updated weights for policy 0, policy_version 78210 (0.0011) [2023-10-08 10:52:39,325][53852] Updated weights for policy 0, policy_version 78220 (0.0009) [2023-10-08 10:52:39,697][53852] Updated weights for policy 0, policy_version 78230 (0.0008) [2023-10-08 10:52:40,060][53852] Updated weights for policy 0, policy_version 78240 (0.0009) [2023-10-08 10:52:41,404][53885] Updated weights for policy 1, policy_version 77862 (0.0008) [2023-10-08 10:52:41,764][53885] Updated weights for policy 1, policy_version 77872 (0.0007) [2023-10-08 10:52:42,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 159842304. Throughput: 0: 1834.0, 1: 1837.7. Samples: 39967752. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:52:42,016][52710] Avg episode reward: [(0, '32.720'), (1, '34.060')] [2023-10-08 10:52:42,129][53885] Updated weights for policy 1, policy_version 77882 (0.0009) [2023-10-08 10:52:43,791][53852] Updated weights for policy 0, policy_version 78250 (0.0008) [2023-10-08 10:52:44,158][53852] Updated weights for policy 0, policy_version 78260 (0.0008) [2023-10-08 10:52:44,531][53852] Updated weights for policy 0, policy_version 78270 (0.0007) [2023-10-08 10:52:45,741][53885] Updated weights for policy 1, policy_version 77892 (0.0008) [2023-10-08 10:52:46,107][53885] Updated weights for policy 1, policy_version 77902 (0.0008) [2023-10-08 10:52:46,486][53885] Updated weights for policy 1, policy_version 77912 (0.0008) [2023-10-08 10:52:47,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 159940608. Throughput: 0: 1845.7, 1: 1843.3. Samples: 39990046. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:52:47,016][52710] Avg episode reward: [(0, '32.280'), (1, '32.660')] [2023-10-08 10:52:48,115][53852] Updated weights for policy 0, policy_version 78280 (0.0008) [2023-10-08 10:52:48,483][53852] Updated weights for policy 0, policy_version 78290 (0.0007) [2023-10-08 10:52:48,851][53852] Updated weights for policy 0, policy_version 78300 (0.0008) [2023-10-08 10:52:50,035][53885] Updated weights for policy 1, policy_version 77922 (0.0008) [2023-10-08 10:52:50,407][53885] Updated weights for policy 1, policy_version 77932 (0.0008) [2023-10-08 10:52:50,770][53885] Updated weights for policy 1, policy_version 77942 (0.0009) [2023-10-08 10:52:51,136][53885] Updated weights for policy 1, policy_version 77952 (0.0007) [2023-10-08 10:52:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160006144. Throughput: 0: 1858.2, 1: 1838.9. Samples: 40012068. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:52:52,016][52710] Avg episode reward: [(0, '31.650'), (1, '33.920')] [2023-10-08 10:52:52,416][53852] Updated weights for policy 0, policy_version 78310 (0.0007) [2023-10-08 10:52:52,791][53852] Updated weights for policy 0, policy_version 78320 (0.0007) [2023-10-08 10:52:53,152][53852] Updated weights for policy 0, policy_version 78330 (0.0007) [2023-10-08 10:52:54,789][53885] Updated weights for policy 1, policy_version 77962 (0.0008) [2023-10-08 10:52:55,159][53885] Updated weights for policy 1, policy_version 77972 (0.0009) [2023-10-08 10:52:55,532][53885] Updated weights for policy 1, policy_version 77982 (0.0008) [2023-10-08 10:52:56,776][53852] Updated weights for policy 0, policy_version 78340 (0.0007) [2023-10-08 10:52:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160071680. Throughput: 0: 1863.8, 1: 1836.4. Samples: 40023510. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:52:57,015][52710] Avg episode reward: [(0, '33.250'), (1, '33.090')] [2023-10-08 10:52:57,142][53852] Updated weights for policy 0, policy_version 78350 (0.0009) [2023-10-08 10:52:57,516][53852] Updated weights for policy 0, policy_version 78360 (0.0010) [2023-10-08 10:52:59,127][53885] Updated weights for policy 1, policy_version 77992 (0.0007) [2023-10-08 10:52:59,485][53885] Updated weights for policy 1, policy_version 78002 (0.0007) [2023-10-08 10:52:59,859][53885] Updated weights for policy 1, policy_version 78012 (0.0007) [2023-10-08 10:53:01,226][53852] Updated weights for policy 0, policy_version 78370 (0.0010) [2023-10-08 10:53:01,594][53852] Updated weights for policy 0, policy_version 78380 (0.0007) [2023-10-08 10:53:01,966][53852] Updated weights for policy 0, policy_version 78390 (0.0008) [2023-10-08 10:53:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160137216. Throughput: 0: 1857.4, 1: 1842.8. Samples: 40045302. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:53:02,016][52710] Avg episode reward: [(0, '32.500'), (1, '35.760')] [2023-10-08 10:53:02,333][53852] Updated weights for policy 0, policy_version 78400 (0.0009) [2023-10-08 10:53:03,661][53885] Updated weights for policy 1, policy_version 78022 (0.0009) [2023-10-08 10:53:04,031][53885] Updated weights for policy 1, policy_version 78032 (0.0009) [2023-10-08 10:53:04,400][53885] Updated weights for policy 1, policy_version 78042 (0.0007) [2023-10-08 10:53:05,900][53852] Updated weights for policy 0, policy_version 78410 (0.0007) [2023-10-08 10:53:06,277][53852] Updated weights for policy 0, policy_version 78420 (0.0009) [2023-10-08 10:53:06,648][53852] Updated weights for policy 0, policy_version 78430 (0.0009) [2023-10-08 10:53:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 160235520. Throughput: 0: 1831.2, 1: 1843.6. Samples: 40067004. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:53:07,016][52710] Avg episode reward: [(0, '30.880'), (1, '34.080')] [2023-10-08 10:53:08,067][53885] Updated weights for policy 1, policy_version 78052 (0.0010) [2023-10-08 10:53:08,435][53885] Updated weights for policy 1, policy_version 78062 (0.0007) [2023-10-08 10:53:08,795][53885] Updated weights for policy 1, policy_version 78072 (0.0010) [2023-10-08 10:53:10,271][53852] Updated weights for policy 0, policy_version 78440 (0.0010) [2023-10-08 10:53:10,646][53852] Updated weights for policy 0, policy_version 78450 (0.0011) [2023-10-08 10:53:11,024][53852] Updated weights for policy 0, policy_version 78460 (0.0007) [2023-10-08 10:53:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 160301056. Throughput: 0: 1845.1, 1: 1840.9. Samples: 40078210. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:53:12,016][52710] Avg episode reward: [(0, '30.680'), (1, '34.540')] [2023-10-08 10:53:12,521][53885] Updated weights for policy 1, policy_version 78082 (0.0011) [2023-10-08 10:53:12,894][53885] Updated weights for policy 1, policy_version 78092 (0.0010) [2023-10-08 10:53:13,270][53885] Updated weights for policy 1, policy_version 78102 (0.0010) [2023-10-08 10:53:13,632][53885] Updated weights for policy 1, policy_version 78112 (0.0011) [2023-10-08 10:53:14,645][53852] Updated weights for policy 0, policy_version 78470 (0.0009) [2023-10-08 10:53:15,017][53852] Updated weights for policy 0, policy_version 78480 (0.0008) [2023-10-08 10:53:15,386][53852] Updated weights for policy 0, policy_version 78490 (0.0008) [2023-10-08 10:53:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 160366592. Throughput: 0: 1825.7, 1: 1839.8. Samples: 40099694. Policy #0 lag: (min: 27.0, avg: 37.7, max: 59.0) [2023-10-08 10:53:17,016][52710] Avg episode reward: [(0, '33.270'), (1, '36.730')] [2023-10-08 10:53:17,379][53885] Updated weights for policy 1, policy_version 78122 (0.0009) [2023-10-08 10:53:17,744][53885] Updated weights for policy 1, policy_version 78132 (0.0008) [2023-10-08 10:53:18,108][53885] Updated weights for policy 1, policy_version 78142 (0.0011) [2023-10-08 10:53:19,096][53852] Updated weights for policy 0, policy_version 78500 (0.0010) [2023-10-08 10:53:19,463][53852] Updated weights for policy 0, policy_version 78510 (0.0008) [2023-10-08 10:53:19,833][53852] Updated weights for policy 0, policy_version 78520 (0.0008) [2023-10-08 10:53:21,675][53885] Updated weights for policy 1, policy_version 78152 (0.0008) [2023-10-08 10:53:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160432128. Throughput: 0: 1850.3, 1: 1826.8. Samples: 40122452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:53:22,016][52710] Avg episode reward: [(0, '30.470'), (1, '35.980')] [2023-10-08 10:53:22,035][53885] Updated weights for policy 1, policy_version 78162 (0.0007) [2023-10-08 10:53:22,407][53885] Updated weights for policy 1, policy_version 78172 (0.0009) [2023-10-08 10:53:23,514][53852] Updated weights for policy 0, policy_version 78530 (0.0007) [2023-10-08 10:53:23,887][53852] Updated weights for policy 0, policy_version 78540 (0.0007) [2023-10-08 10:53:24,259][53852] Updated weights for policy 0, policy_version 78550 (0.0008) [2023-10-08 10:53:24,628][53852] Updated weights for policy 0, policy_version 78560 (0.0010) [2023-10-08 10:53:26,264][53885] Updated weights for policy 1, policy_version 78182 (0.0009) [2023-10-08 10:53:26,627][53885] Updated weights for policy 1, policy_version 78192 (0.0010) [2023-10-08 10:53:26,986][53885] Updated weights for policy 1, policy_version 78202 (0.0008) [2023-10-08 10:53:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 160497664. Throughput: 0: 1838.3, 1: 1832.7. Samples: 40132948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:53:27,015][52710] Avg episode reward: [(0, '31.380'), (1, '34.160')] [2023-10-08 10:53:28,102][53852] Updated weights for policy 0, policy_version 78570 (0.0009) [2023-10-08 10:53:28,473][53852] Updated weights for policy 0, policy_version 78580 (0.0008) [2023-10-08 10:53:28,839][53852] Updated weights for policy 0, policy_version 78590 (0.0008) [2023-10-08 10:53:30,651][53885] Updated weights for policy 1, policy_version 78212 (0.0009) [2023-10-08 10:53:31,023][53885] Updated weights for policy 1, policy_version 78222 (0.0007) [2023-10-08 10:53:31,377][53885] Updated weights for policy 1, policy_version 78232 (0.0007) [2023-10-08 10:53:32,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 160595968. Throughput: 0: 1854.8, 1: 1822.9. Samples: 40155546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:53:32,016][52710] Avg episode reward: [(0, '29.510'), (1, '37.420')] [2023-10-08 10:53:32,532][53852] Updated weights for policy 0, policy_version 78600 (0.0008) [2023-10-08 10:53:32,898][53852] Updated weights for policy 0, policy_version 78610 (0.0009) [2023-10-08 10:53:33,272][53852] Updated weights for policy 0, policy_version 78620 (0.0009) [2023-10-08 10:53:35,017][53885] Updated weights for policy 1, policy_version 78242 (0.0008) [2023-10-08 10:53:35,394][53885] Updated weights for policy 1, policy_version 78252 (0.0010) [2023-10-08 10:53:35,763][53885] Updated weights for policy 1, policy_version 78262 (0.0010) [2023-10-08 10:53:36,129][53885] Updated weights for policy 1, policy_version 78272 (0.0010) [2023-10-08 10:53:36,992][53852] Updated weights for policy 0, policy_version 78630 (0.0009) [2023-10-08 10:53:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160661504. Throughput: 0: 1844.2, 1: 1824.6. Samples: 40177164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:53:37,016][52710] Avg episode reward: [(0, '30.120'), (1, '34.680')] [2023-10-08 10:53:37,349][53852] Updated weights for policy 0, policy_version 78640 (0.0008) [2023-10-08 10:53:37,717][53852] Updated weights for policy 0, policy_version 78650 (0.0007) [2023-10-08 10:53:39,876][53885] Updated weights for policy 1, policy_version 78282 (0.0011) [2023-10-08 10:53:40,245][53885] Updated weights for policy 1, policy_version 78292 (0.0009) [2023-10-08 10:53:40,618][53885] Updated weights for policy 1, policy_version 78302 (0.0008) [2023-10-08 10:53:41,462][53852] Updated weights for policy 0, policy_version 78660 (0.0007) [2023-10-08 10:53:41,832][53852] Updated weights for policy 0, policy_version 78670 (0.0009) [2023-10-08 10:53:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160727040. Throughput: 0: 1840.1, 1: 1826.3. Samples: 40188498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:53:42,016][52710] Avg episode reward: [(0, '32.350'), (1, '33.760')] [2023-10-08 10:53:42,204][53852] Updated weights for policy 0, policy_version 78680 (0.0008) [2023-10-08 10:53:44,187][53885] Updated weights for policy 1, policy_version 78312 (0.0008) [2023-10-08 10:53:44,547][53885] Updated weights for policy 1, policy_version 78322 (0.0008) [2023-10-08 10:53:44,924][53885] Updated weights for policy 1, policy_version 78332 (0.0008) [2023-10-08 10:53:45,822][53852] Updated weights for policy 0, policy_version 78690 (0.0009) [2023-10-08 10:53:46,185][53852] Updated weights for policy 0, policy_version 78700 (0.0008) [2023-10-08 10:53:46,564][53852] Updated weights for policy 0, policy_version 78710 (0.0008) [2023-10-08 10:53:46,929][53852] Updated weights for policy 0, policy_version 78720 (0.0008) [2023-10-08 10:53:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 160825344. Throughput: 0: 1846.6, 1: 1819.7. Samples: 40210286. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:53:47,016][52710] Avg episode reward: [(0, '32.950'), (1, '36.780')] [2023-10-08 10:53:48,720][53885] Updated weights for policy 1, policy_version 78342 (0.0008) [2023-10-08 10:53:49,089][53885] Updated weights for policy 1, policy_version 78352 (0.0010) [2023-10-08 10:53:49,452][53885] Updated weights for policy 1, policy_version 78362 (0.0008) [2023-10-08 10:53:50,535][53852] Updated weights for policy 0, policy_version 78730 (0.0008) [2023-10-08 10:53:50,894][53852] Updated weights for policy 0, policy_version 78740 (0.0009) [2023-10-08 10:53:51,270][53852] Updated weights for policy 0, policy_version 78750 (0.0007) [2023-10-08 10:53:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 160890880. Throughput: 0: 1841.0, 1: 1823.2. Samples: 40231890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:53:52,015][52710] Avg episode reward: [(0, '30.650'), (1, '35.830')] [2023-10-08 10:53:53,099][53885] Updated weights for policy 1, policy_version 78372 (0.0010) [2023-10-08 10:53:53,468][53885] Updated weights for policy 1, policy_version 78382 (0.0011) [2023-10-08 10:53:53,821][53885] Updated weights for policy 1, policy_version 78392 (0.0009) [2023-10-08 10:53:54,735][53852] Updated weights for policy 0, policy_version 78760 (0.0008) [2023-10-08 10:53:55,102][53852] Updated weights for policy 0, policy_version 78770 (0.0007) [2023-10-08 10:53:55,463][53852] Updated weights for policy 0, policy_version 78780 (0.0008) [2023-10-08 10:53:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 160956416. Throughput: 0: 1849.8, 1: 1820.6. Samples: 40243378. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:53:57,016][52710] Avg episode reward: [(0, '29.340'), (1, '36.970')] [2023-10-08 10:53:57,420][53885] Updated weights for policy 1, policy_version 78402 (0.0009) [2023-10-08 10:53:57,786][53885] Updated weights for policy 1, policy_version 78412 (0.0009) [2023-10-08 10:53:58,147][53885] Updated weights for policy 1, policy_version 78422 (0.0009) [2023-10-08 10:53:58,513][53885] Updated weights for policy 1, policy_version 78432 (0.0008) [2023-10-08 10:53:59,080][53852] Updated weights for policy 0, policy_version 78790 (0.0008) [2023-10-08 10:53:59,448][53852] Updated weights for policy 0, policy_version 78800 (0.0007) [2023-10-08 10:53:59,814][53852] Updated weights for policy 0, policy_version 78810 (0.0007) [2023-10-08 10:54:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161021952. Throughput: 0: 1855.8, 1: 1826.3. Samples: 40265392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:54:02,016][52710] Avg episode reward: [(0, '33.160'), (1, '35.060')] [2023-10-08 10:54:02,222][53885] Updated weights for policy 1, policy_version 78442 (0.0008) [2023-10-08 10:54:02,579][53885] Updated weights for policy 1, policy_version 78452 (0.0009) [2023-10-08 10:54:02,941][53885] Updated weights for policy 1, policy_version 78462 (0.0010) [2023-10-08 10:54:03,607][53852] Updated weights for policy 0, policy_version 78820 (0.0008) [2023-10-08 10:54:04,009][53852] Updated weights for policy 0, policy_version 78830 (0.0009) [2023-10-08 10:54:04,374][53852] Updated weights for policy 0, policy_version 78840 (0.0010) [2023-10-08 10:54:06,723][53885] Updated weights for policy 1, policy_version 78472 (0.0007) [2023-10-08 10:54:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 161087488. Throughput: 0: 1844.4, 1: 1825.2. Samples: 40287584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:54:07,016][52710] Avg episode reward: [(0, '31.480'), (1, '36.780')] [2023-10-08 10:54:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000078848_80740352.pth... [2023-10-08 10:54:07,060][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000077120_78970880.pth [2023-10-08 10:54:07,082][53885] Updated weights for policy 1, policy_version 78482 (0.0007) [2023-10-08 10:54:07,448][53885] Updated weights for policy 1, policy_version 78492 (0.0010) [2023-10-08 10:54:07,593][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000078496_80379904.pth... [2023-10-08 10:54:07,632][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000076768_78610432.pth [2023-10-08 10:54:07,977][53852] Updated weights for policy 0, policy_version 78850 (0.0011) [2023-10-08 10:54:08,345][53852] Updated weights for policy 0, policy_version 78860 (0.0008) [2023-10-08 10:54:08,704][53852] Updated weights for policy 0, policy_version 78870 (0.0008) [2023-10-08 10:54:09,077][53852] Updated weights for policy 0, policy_version 78880 (0.0008) [2023-10-08 10:54:11,171][53885] Updated weights for policy 1, policy_version 78502 (0.0010) [2023-10-08 10:54:11,537][53885] Updated weights for policy 1, policy_version 78512 (0.0009) [2023-10-08 10:54:11,896][53885] Updated weights for policy 1, policy_version 78522 (0.0010) [2023-10-08 10:54:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 161153024. Throughput: 0: 1838.0, 1: 1824.6. Samples: 40297766. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:12,016][52710] Avg episode reward: [(0, '31.340'), (1, '34.810')] [2023-10-08 10:54:12,667][53852] Updated weights for policy 0, policy_version 78890 (0.0007) [2023-10-08 10:54:13,036][53852] Updated weights for policy 0, policy_version 78900 (0.0007) [2023-10-08 10:54:13,402][53852] Updated weights for policy 0, policy_version 78910 (0.0008) [2023-10-08 10:54:15,637][53885] Updated weights for policy 1, policy_version 78532 (0.0010) [2023-10-08 10:54:15,999][53885] Updated weights for policy 1, policy_version 78542 (0.0008) [2023-10-08 10:54:16,366][53885] Updated weights for policy 1, policy_version 78552 (0.0008) [2023-10-08 10:54:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161251328. Throughput: 0: 1838.0, 1: 1825.7. Samples: 40320416. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:17,016][52710] Avg episode reward: [(0, '32.570'), (1, '33.400')] [2023-10-08 10:54:17,046][53852] Updated weights for policy 0, policy_version 78920 (0.0008) [2023-10-08 10:54:17,416][53852] Updated weights for policy 0, policy_version 78930 (0.0010) [2023-10-08 10:54:17,795][53852] Updated weights for policy 0, policy_version 78940 (0.0007) [2023-10-08 10:54:20,025][53885] Updated weights for policy 1, policy_version 78562 (0.0008) [2023-10-08 10:54:20,390][53885] Updated weights for policy 1, policy_version 78572 (0.0007) [2023-10-08 10:54:20,766][53885] Updated weights for policy 1, policy_version 78582 (0.0007) [2023-10-08 10:54:21,130][53885] Updated weights for policy 1, policy_version 78592 (0.0008) [2023-10-08 10:54:21,482][53852] Updated weights for policy 0, policy_version 78950 (0.0007) [2023-10-08 10:54:21,847][53852] Updated weights for policy 0, policy_version 78960 (0.0007) [2023-10-08 10:54:22,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161316864. Throughput: 0: 1830.1, 1: 1827.5. Samples: 40341756. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:22,016][52710] Avg episode reward: [(0, '35.100'), (1, '35.580')] [2023-10-08 10:54:22,223][53852] Updated weights for policy 0, policy_version 78970 (0.0009) [2023-10-08 10:54:24,630][53885] Updated weights for policy 1, policy_version 78602 (0.0008) [2023-10-08 10:54:24,996][53885] Updated weights for policy 1, policy_version 78612 (0.0008) [2023-10-08 10:54:25,359][53885] Updated weights for policy 1, policy_version 78622 (0.0009) [2023-10-08 10:54:25,688][53852] Updated weights for policy 0, policy_version 78980 (0.0007) [2023-10-08 10:54:26,056][53852] Updated weights for policy 0, policy_version 78990 (0.0008) [2023-10-08 10:54:26,427][53852] Updated weights for policy 0, policy_version 79000 (0.0008) [2023-10-08 10:54:27,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 161415168. Throughput: 0: 1841.8, 1: 1826.9. Samples: 40353590. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:27,016][52710] Avg episode reward: [(0, '35.260'), (1, '35.680')] [2023-10-08 10:54:28,912][53885] Updated weights for policy 1, policy_version 78632 (0.0007) [2023-10-08 10:54:29,292][53885] Updated weights for policy 1, policy_version 78642 (0.0008) [2023-10-08 10:54:29,659][53885] Updated weights for policy 1, policy_version 78652 (0.0009) [2023-10-08 10:54:30,301][53852] Updated weights for policy 0, policy_version 79010 (0.0009) [2023-10-08 10:54:30,670][53852] Updated weights for policy 0, policy_version 79020 (0.0009) [2023-10-08 10:54:31,032][53852] Updated weights for policy 0, policy_version 79030 (0.0008) [2023-10-08 10:54:31,391][53852] Updated weights for policy 0, policy_version 79040 (0.0008) [2023-10-08 10:54:32,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 161480704. Throughput: 0: 1828.4, 1: 1841.4. Samples: 40375430. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:32,016][52710] Avg episode reward: [(0, '35.070'), (1, '34.250')] [2023-10-08 10:54:33,335][53885] Updated weights for policy 1, policy_version 78662 (0.0009) [2023-10-08 10:54:33,727][53885] Updated weights for policy 1, policy_version 78672 (0.0009) [2023-10-08 10:54:34,091][53885] Updated weights for policy 1, policy_version 78682 (0.0008) [2023-10-08 10:54:35,030][53852] Updated weights for policy 0, policy_version 79050 (0.0009) [2023-10-08 10:54:35,400][53852] Updated weights for policy 0, policy_version 79060 (0.0011) [2023-10-08 10:54:35,772][53852] Updated weights for policy 0, policy_version 79070 (0.0010) [2023-10-08 10:54:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 161546240. Throughput: 0: 1839.1, 1: 1839.9. Samples: 40397446. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:37,016][52710] Avg episode reward: [(0, '35.850'), (1, '34.220')] [2023-10-08 10:54:37,732][53885] Updated weights for policy 1, policy_version 78692 (0.0008) [2023-10-08 10:54:38,101][53885] Updated weights for policy 1, policy_version 78702 (0.0009) [2023-10-08 10:54:38,476][53885] Updated weights for policy 1, policy_version 78712 (0.0009) [2023-10-08 10:54:39,442][53852] Updated weights for policy 0, policy_version 79080 (0.0008) [2023-10-08 10:54:39,809][53852] Updated weights for policy 0, policy_version 79090 (0.0008) [2023-10-08 10:54:40,184][53852] Updated weights for policy 0, policy_version 79100 (0.0009) [2023-10-08 10:54:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161611776. Throughput: 0: 1828.4, 1: 1842.6. Samples: 40408576. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:42,016][52710] Avg episode reward: [(0, '33.510'), (1, '39.660')] [2023-10-08 10:54:42,265][53885] Updated weights for policy 1, policy_version 78722 (0.0009) [2023-10-08 10:54:42,629][53885] Updated weights for policy 1, policy_version 78732 (0.0008) [2023-10-08 10:54:43,007][53885] Updated weights for policy 1, policy_version 78742 (0.0007) [2023-10-08 10:54:43,373][53885] Updated weights for policy 1, policy_version 78752 (0.0008) [2023-10-08 10:54:43,801][53852] Updated weights for policy 0, policy_version 79110 (0.0008) [2023-10-08 10:54:44,163][53852] Updated weights for policy 0, policy_version 79120 (0.0008) [2023-10-08 10:54:44,539][53852] Updated weights for policy 0, policy_version 79130 (0.0007) [2023-10-08 10:54:47,013][53885] Updated weights for policy 1, policy_version 78762 (0.0009) [2023-10-08 10:54:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 161677312. Throughput: 0: 1832.2, 1: 1831.1. Samples: 40430242. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:47,016][52710] Avg episode reward: [(0, '33.660'), (1, '34.070')] [2023-10-08 10:54:47,380][53885] Updated weights for policy 1, policy_version 78772 (0.0008) [2023-10-08 10:54:47,742][53885] Updated weights for policy 1, policy_version 78782 (0.0008) [2023-10-08 10:54:48,286][53852] Updated weights for policy 0, policy_version 79140 (0.0008) [2023-10-08 10:54:48,682][53852] Updated weights for policy 0, policy_version 79150 (0.0011) [2023-10-08 10:54:49,046][53852] Updated weights for policy 0, policy_version 79160 (0.0010) [2023-10-08 10:54:51,405][53885] Updated weights for policy 1, policy_version 78792 (0.0008) [2023-10-08 10:54:51,766][53885] Updated weights for policy 1, policy_version 78802 (0.0010) [2023-10-08 10:54:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 161742848. Throughput: 0: 1836.2, 1: 1828.3. Samples: 40452486. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:52,015][52710] Avg episode reward: [(0, '33.260'), (1, '36.690')] [2023-10-08 10:54:52,137][53885] Updated weights for policy 1, policy_version 78812 (0.0007) [2023-10-08 10:54:52,608][53852] Updated weights for policy 0, policy_version 79170 (0.0010) [2023-10-08 10:54:52,984][53852] Updated weights for policy 0, policy_version 79180 (0.0008) [2023-10-08 10:54:53,357][53852] Updated weights for policy 0, policy_version 79190 (0.0008) [2023-10-08 10:54:53,718][53852] Updated weights for policy 0, policy_version 79200 (0.0007) [2023-10-08 10:54:55,858][53885] Updated weights for policy 1, policy_version 78822 (0.0008) [2023-10-08 10:54:56,228][53885] Updated weights for policy 1, policy_version 78832 (0.0010) [2023-10-08 10:54:56,596][53885] Updated weights for policy 1, policy_version 78842 (0.0008) [2023-10-08 10:54:57,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161841152. Throughput: 0: 1835.7, 1: 1838.8. Samples: 40463116. Policy #0 lag: (min: 22.0, avg: 29.9, max: 54.0) [2023-10-08 10:54:57,015][52710] Avg episode reward: [(0, '31.770'), (1, '36.630')] [2023-10-08 10:54:57,533][53852] Updated weights for policy 0, policy_version 79210 (0.0007) [2023-10-08 10:54:57,912][53852] Updated weights for policy 0, policy_version 79220 (0.0007) [2023-10-08 10:54:58,281][53852] Updated weights for policy 0, policy_version 79230 (0.0008) [2023-10-08 10:55:00,197][53885] Updated weights for policy 1, policy_version 78852 (0.0009) [2023-10-08 10:55:00,568][53885] Updated weights for policy 1, policy_version 78862 (0.0009) [2023-10-08 10:55:00,939][53885] Updated weights for policy 1, policy_version 78872 (0.0009) [2023-10-08 10:55:01,873][53852] Updated weights for policy 0, policy_version 79240 (0.0007) [2023-10-08 10:55:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 161906688. Throughput: 0: 1841.0, 1: 1829.7. Samples: 40485600. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:02,015][52710] Avg episode reward: [(0, '31.880'), (1, '35.460')] [2023-10-08 10:55:02,246][53852] Updated weights for policy 0, policy_version 79250 (0.0009) [2023-10-08 10:55:02,615][53852] Updated weights for policy 0, policy_version 79260 (0.0008) [2023-10-08 10:55:04,518][53885] Updated weights for policy 1, policy_version 78882 (0.0009) [2023-10-08 10:55:04,884][53885] Updated weights for policy 1, policy_version 78892 (0.0011) [2023-10-08 10:55:05,250][53885] Updated weights for policy 1, policy_version 78902 (0.0010) [2023-10-08 10:55:05,617][53885] Updated weights for policy 1, policy_version 78912 (0.0011) [2023-10-08 10:55:06,216][53852] Updated weights for policy 0, policy_version 79270 (0.0009) [2023-10-08 10:55:06,579][53852] Updated weights for policy 0, policy_version 79280 (0.0010) [2023-10-08 10:55:06,948][53852] Updated weights for policy 0, policy_version 79290 (0.0010) [2023-10-08 10:55:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 161972224. Throughput: 0: 1835.2, 1: 1842.6. Samples: 40507258. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:07,016][52710] Avg episode reward: [(0, '32.800'), (1, '36.060')] [2023-10-08 10:55:09,152][53885] Updated weights for policy 1, policy_version 78922 (0.0008) [2023-10-08 10:55:09,526][53885] Updated weights for policy 1, policy_version 78932 (0.0008) [2023-10-08 10:55:09,898][53885] Updated weights for policy 1, policy_version 78942 (0.0009) [2023-10-08 10:55:10,578][53852] Updated weights for policy 0, policy_version 79300 (0.0010) [2023-10-08 10:55:10,963][53852] Updated weights for policy 0, policy_version 79310 (0.0008) [2023-10-08 10:55:11,334][53852] Updated weights for policy 0, policy_version 79320 (0.0010) [2023-10-08 10:55:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 162070528. Throughput: 0: 1838.9, 1: 1823.9. Samples: 40518414. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:12,016][52710] Avg episode reward: [(0, '36.100'), (1, '37.750')] [2023-10-08 10:55:13,574][53885] Updated weights for policy 1, policy_version 78952 (0.0008) [2023-10-08 10:55:13,943][53885] Updated weights for policy 1, policy_version 78962 (0.0008) [2023-10-08 10:55:14,299][53885] Updated weights for policy 1, policy_version 78972 (0.0008) [2023-10-08 10:55:14,936][53852] Updated weights for policy 0, policy_version 79330 (0.0008) [2023-10-08 10:55:15,315][53852] Updated weights for policy 0, policy_version 79340 (0.0010) [2023-10-08 10:55:15,673][53852] Updated weights for policy 0, policy_version 79350 (0.0008) [2023-10-08 10:55:16,045][53852] Updated weights for policy 0, policy_version 79360 (0.0009) [2023-10-08 10:55:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 162136064. Throughput: 0: 1829.5, 1: 1828.3. Samples: 40540032. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:17,016][52710] Avg episode reward: [(0, '32.380'), (1, '36.850')] [2023-10-08 10:55:18,025][53885] Updated weights for policy 1, policy_version 78982 (0.0008) [2023-10-08 10:55:18,407][53885] Updated weights for policy 1, policy_version 78992 (0.0010) [2023-10-08 10:55:18,783][53885] Updated weights for policy 1, policy_version 79002 (0.0009) [2023-10-08 10:55:19,914][53852] Updated weights for policy 0, policy_version 79370 (0.0008) [2023-10-08 10:55:20,290][53852] Updated weights for policy 0, policy_version 79380 (0.0008) [2023-10-08 10:55:20,656][53852] Updated weights for policy 0, policy_version 79390 (0.0008) [2023-10-08 10:55:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 162201600. Throughput: 0: 1832.0, 1: 1832.9. Samples: 40562366. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:22,016][52710] Avg episode reward: [(0, '33.100'), (1, '34.670')] [2023-10-08 10:55:22,308][53885] Updated weights for policy 1, policy_version 79012 (0.0008) [2023-10-08 10:55:22,677][53885] Updated weights for policy 1, policy_version 79022 (0.0009) [2023-10-08 10:55:23,042][53885] Updated weights for policy 1, policy_version 79032 (0.0008) [2023-10-08 10:55:24,186][53852] Updated weights for policy 0, policy_version 79400 (0.0008) [2023-10-08 10:55:24,553][53852] Updated weights for policy 0, policy_version 79410 (0.0007) [2023-10-08 10:55:24,934][53852] Updated weights for policy 0, policy_version 79420 (0.0007) [2023-10-08 10:55:26,678][53885] Updated weights for policy 1, policy_version 79042 (0.0009) [2023-10-08 10:55:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 162267136. Throughput: 0: 1823.1, 1: 1834.5. Samples: 40573166. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:27,016][52710] Avg episode reward: [(0, '33.560'), (1, '36.760')] [2023-10-08 10:55:27,050][53885] Updated weights for policy 1, policy_version 79052 (0.0007) [2023-10-08 10:55:27,416][53885] Updated weights for policy 1, policy_version 79062 (0.0007) [2023-10-08 10:55:27,777][53885] Updated weights for policy 1, policy_version 79072 (0.0007) [2023-10-08 10:55:28,456][53852] Updated weights for policy 0, policy_version 79430 (0.0008) [2023-10-08 10:55:28,826][53852] Updated weights for policy 0, policy_version 79440 (0.0008) [2023-10-08 10:55:29,187][53852] Updated weights for policy 0, policy_version 79450 (0.0007) [2023-10-08 10:55:31,585][53885] Updated weights for policy 1, policy_version 79082 (0.0008) [2023-10-08 10:55:31,962][53885] Updated weights for policy 1, policy_version 79092 (0.0009) [2023-10-08 10:55:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 162332672. Throughput: 0: 1835.9, 1: 1839.0. Samples: 40595614. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:32,016][52710] Avg episode reward: [(0, '30.820'), (1, '34.600')] [2023-10-08 10:55:32,332][53885] Updated weights for policy 1, policy_version 79102 (0.0007) [2023-10-08 10:55:32,892][53852] Updated weights for policy 0, policy_version 79460 (0.0009) [2023-10-08 10:55:33,260][53852] Updated weights for policy 0, policy_version 79470 (0.0009) [2023-10-08 10:55:33,629][53852] Updated weights for policy 0, policy_version 79480 (0.0008) [2023-10-08 10:55:35,932][53885] Updated weights for policy 1, policy_version 79112 (0.0008) [2023-10-08 10:55:36,302][53885] Updated weights for policy 1, policy_version 79122 (0.0008) [2023-10-08 10:55:36,675][53885] Updated weights for policy 1, policy_version 79132 (0.0010) [2023-10-08 10:55:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162430976. Throughput: 0: 1842.5, 1: 1824.4. Samples: 40617498. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:37,016][52710] Avg episode reward: [(0, '32.110'), (1, '34.490')] [2023-10-08 10:55:37,319][53852] Updated weights for policy 0, policy_version 79490 (0.0008) [2023-10-08 10:55:37,720][53852] Updated weights for policy 0, policy_version 79500 (0.0010) [2023-10-08 10:55:38,089][53852] Updated weights for policy 0, policy_version 79510 (0.0010) [2023-10-08 10:55:38,457][53852] Updated weights for policy 0, policy_version 79520 (0.0010) [2023-10-08 10:55:40,371][53885] Updated weights for policy 1, policy_version 79142 (0.0008) [2023-10-08 10:55:40,740][53885] Updated weights for policy 1, policy_version 79152 (0.0008) [2023-10-08 10:55:41,099][53885] Updated weights for policy 1, policy_version 79162 (0.0010) [2023-10-08 10:55:41,946][53852] Updated weights for policy 0, policy_version 79530 (0.0009) [2023-10-08 10:55:42,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162496512. Throughput: 0: 1840.8, 1: 1838.1. Samples: 40628668. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:42,016][52710] Avg episode reward: [(0, '33.730'), (1, '33.860')] [2023-10-08 10:55:42,314][53852] Updated weights for policy 0, policy_version 79540 (0.0007) [2023-10-08 10:55:42,692][53852] Updated weights for policy 0, policy_version 79550 (0.0008) [2023-10-08 10:55:44,878][53885] Updated weights for policy 1, policy_version 79172 (0.0009) [2023-10-08 10:55:45,248][53885] Updated weights for policy 1, policy_version 79182 (0.0008) [2023-10-08 10:55:45,615][53885] Updated weights for policy 1, policy_version 79192 (0.0008) [2023-10-08 10:55:46,437][53852] Updated weights for policy 0, policy_version 79560 (0.0009) [2023-10-08 10:55:46,800][53852] Updated weights for policy 0, policy_version 79570 (0.0009) [2023-10-08 10:55:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162562048. Throughput: 0: 1839.8, 1: 1821.4. Samples: 40650352. Policy #0 lag: (min: 1.0, avg: 10.8, max: 33.0) [2023-10-08 10:55:47,016][52710] Avg episode reward: [(0, '32.710'), (1, '32.440')] [2023-10-08 10:55:47,174][53852] Updated weights for policy 0, policy_version 79580 (0.0007) [2023-10-08 10:55:49,314][53885] Updated weights for policy 1, policy_version 79202 (0.0008) [2023-10-08 10:55:49,683][53885] Updated weights for policy 1, policy_version 79212 (0.0010) [2023-10-08 10:55:50,057][53885] Updated weights for policy 1, policy_version 79222 (0.0008) [2023-10-08 10:55:50,418][53885] Updated weights for policy 1, policy_version 79232 (0.0007) [2023-10-08 10:55:50,779][53852] Updated weights for policy 0, policy_version 79590 (0.0009) [2023-10-08 10:55:51,164][53852] Updated weights for policy 0, policy_version 79600 (0.0009) [2023-10-08 10:55:51,533][53852] Updated weights for policy 0, policy_version 79610 (0.0009) [2023-10-08 10:55:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 162660352. Throughput: 0: 1826.9, 1: 1829.1. Samples: 40671778. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:55:52,016][52710] Avg episode reward: [(0, '34.200'), (1, '34.390')] [2023-10-08 10:55:53,987][53885] Updated weights for policy 1, policy_version 79242 (0.0010) [2023-10-08 10:55:54,361][53885] Updated weights for policy 1, policy_version 79252 (0.0009) [2023-10-08 10:55:54,721][53885] Updated weights for policy 1, policy_version 79262 (0.0007) [2023-10-08 10:55:55,195][53852] Updated weights for policy 0, policy_version 79620 (0.0008) [2023-10-08 10:55:55,560][53852] Updated weights for policy 0, policy_version 79630 (0.0008) [2023-10-08 10:55:55,922][53852] Updated weights for policy 0, policy_version 79640 (0.0008) [2023-10-08 10:55:57,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 162725888. Throughput: 0: 1841.6, 1: 1826.0. Samples: 40683454. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:55:57,015][52710] Avg episode reward: [(0, '34.000'), (1, '32.330')] [2023-10-08 10:55:58,375][53885] Updated weights for policy 1, policy_version 79272 (0.0009) [2023-10-08 10:55:58,747][53885] Updated weights for policy 1, policy_version 79282 (0.0007) [2023-10-08 10:55:59,114][53885] Updated weights for policy 1, policy_version 79292 (0.0009) [2023-10-08 10:55:59,537][53852] Updated weights for policy 0, policy_version 79650 (0.0007) [2023-10-08 10:55:59,909][53852] Updated weights for policy 0, policy_version 79660 (0.0008) [2023-10-08 10:56:00,274][53852] Updated weights for policy 0, policy_version 79670 (0.0010) [2023-10-08 10:56:00,644][53852] Updated weights for policy 0, policy_version 79680 (0.0008) [2023-10-08 10:56:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162791424. Throughput: 0: 1832.4, 1: 1832.9. Samples: 40704970. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:56:02,016][52710] Avg episode reward: [(0, '30.570'), (1, '33.730')] [2023-10-08 10:56:02,909][53885] Updated weights for policy 1, policy_version 79302 (0.0009) [2023-10-08 10:56:03,303][53885] Updated weights for policy 1, policy_version 79312 (0.0008) [2023-10-08 10:56:03,666][53885] Updated weights for policy 1, policy_version 79322 (0.0008) [2023-10-08 10:56:04,232][53852] Updated weights for policy 0, policy_version 79690 (0.0007) [2023-10-08 10:56:04,605][53852] Updated weights for policy 0, policy_version 79700 (0.0008) [2023-10-08 10:56:04,970][53852] Updated weights for policy 0, policy_version 79710 (0.0009) [2023-10-08 10:56:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162856960. Throughput: 0: 1847.8, 1: 1822.8. Samples: 40727542. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:56:07,016][52710] Avg episode reward: [(0, '32.840'), (1, '37.400')] [2023-10-08 10:56:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000079328_81231872.pth... [2023-10-08 10:56:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000079712_81625088.pth... [2023-10-08 10:56:07,063][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000077632_79495168.pth [2023-10-08 10:56:07,063][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000077984_79855616.pth [2023-10-08 10:56:07,508][53885] Updated weights for policy 1, policy_version 79332 (0.0009) [2023-10-08 10:56:07,880][53885] Updated weights for policy 1, policy_version 79342 (0.0009) [2023-10-08 10:56:08,245][53885] Updated weights for policy 1, policy_version 79352 (0.0008) [2023-10-08 10:56:08,654][53852] Updated weights for policy 0, policy_version 79720 (0.0008) [2023-10-08 10:56:09,015][53852] Updated weights for policy 0, policy_version 79730 (0.0010) [2023-10-08 10:56:09,384][53852] Updated weights for policy 0, policy_version 79740 (0.0010) [2023-10-08 10:56:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 162922496. Throughput: 0: 1832.2, 1: 1818.8. Samples: 40737462. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:56:12,015][52710] Avg episode reward: [(0, '33.770'), (1, '36.590')] [2023-10-08 10:56:12,051][53885] Updated weights for policy 1, policy_version 79362 (0.0008) [2023-10-08 10:56:12,418][53885] Updated weights for policy 1, policy_version 79372 (0.0010) [2023-10-08 10:56:12,783][53885] Updated weights for policy 1, policy_version 79382 (0.0011) [2023-10-08 10:56:13,118][53852] Updated weights for policy 0, policy_version 79750 (0.0009) [2023-10-08 10:56:13,147][53885] Updated weights for policy 1, policy_version 79392 (0.0007) [2023-10-08 10:56:13,486][53852] Updated weights for policy 0, policy_version 79760 (0.0010) [2023-10-08 10:56:13,850][53852] Updated weights for policy 0, policy_version 79770 (0.0009) [2023-10-08 10:56:16,806][53885] Updated weights for policy 1, policy_version 79402 (0.0009) [2023-10-08 10:56:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162988032. Throughput: 0: 1836.7, 1: 1819.1. Samples: 40760124. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:56:17,016][52710] Avg episode reward: [(0, '31.200'), (1, '31.760')] [2023-10-08 10:56:17,174][53885] Updated weights for policy 1, policy_version 79412 (0.0009) [2023-10-08 10:56:17,526][53852] Updated weights for policy 0, policy_version 79780 (0.0009) [2023-10-08 10:56:17,543][53885] Updated weights for policy 1, policy_version 79422 (0.0007) [2023-10-08 10:56:17,901][53852] Updated weights for policy 0, policy_version 79790 (0.0009) [2023-10-08 10:56:18,272][53852] Updated weights for policy 0, policy_version 79800 (0.0009) [2023-10-08 10:56:21,219][53885] Updated weights for policy 1, policy_version 79432 (0.0008) [2023-10-08 10:56:21,595][53885] Updated weights for policy 1, policy_version 79442 (0.0008) [2023-10-08 10:56:21,858][53852] Updated weights for policy 0, policy_version 79810 (0.0007) [2023-10-08 10:56:21,964][53885] Updated weights for policy 1, policy_version 79452 (0.0008) [2023-10-08 10:56:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 163053568. Throughput: 0: 1837.7, 1: 1827.8. Samples: 40782446. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:56:22,016][52710] Avg episode reward: [(0, '35.050'), (1, '35.490')] [2023-10-08 10:56:22,239][53852] Updated weights for policy 0, policy_version 79820 (0.0009) [2023-10-08 10:56:22,621][53852] Updated weights for policy 0, policy_version 79830 (0.0008) [2023-10-08 10:56:22,986][53852] Updated weights for policy 0, policy_version 79840 (0.0010) [2023-10-08 10:56:25,543][53885] Updated weights for policy 1, policy_version 79462 (0.0007) [2023-10-08 10:56:25,921][53885] Updated weights for policy 1, policy_version 79472 (0.0009) [2023-10-08 10:56:26,282][53885] Updated weights for policy 1, policy_version 79482 (0.0008) [2023-10-08 10:56:26,893][53852] Updated weights for policy 0, policy_version 79850 (0.0012) [2023-10-08 10:56:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163151872. Throughput: 0: 1838.9, 1: 1817.7. Samples: 40793218. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:56:27,016][52710] Avg episode reward: [(0, '32.590'), (1, '37.430')] [2023-10-08 10:56:27,257][53852] Updated weights for policy 0, policy_version 79860 (0.0007) [2023-10-08 10:56:27,635][53852] Updated weights for policy 0, policy_version 79870 (0.0007) [2023-10-08 10:56:29,766][53885] Updated weights for policy 1, policy_version 79492 (0.0009) [2023-10-08 10:56:30,140][53885] Updated weights for policy 1, policy_version 79502 (0.0009) [2023-10-08 10:56:30,505][53885] Updated weights for policy 1, policy_version 79512 (0.0011) [2023-10-08 10:56:31,348][53852] Updated weights for policy 0, policy_version 79880 (0.0008) [2023-10-08 10:56:31,727][53852] Updated weights for policy 0, policy_version 79890 (0.0008) [2023-10-08 10:56:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163217408. Throughput: 0: 1835.1, 1: 1821.9. Samples: 40814914. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:56:32,015][52710] Avg episode reward: [(0, '32.210'), (1, '32.770')] [2023-10-08 10:56:32,100][53852] Updated weights for policy 0, policy_version 79900 (0.0009) [2023-10-08 10:56:34,143][53885] Updated weights for policy 1, policy_version 79522 (0.0009) [2023-10-08 10:56:34,505][53885] Updated weights for policy 1, policy_version 79532 (0.0007) [2023-10-08 10:56:34,877][53885] Updated weights for policy 1, policy_version 79542 (0.0007) [2023-10-08 10:56:35,249][53885] Updated weights for policy 1, policy_version 79552 (0.0009) [2023-10-08 10:56:35,599][53852] Updated weights for policy 0, policy_version 79910 (0.0010) [2023-10-08 10:56:35,973][53852] Updated weights for policy 0, policy_version 79920 (0.0009) [2023-10-08 10:56:36,347][53852] Updated weights for policy 0, policy_version 79930 (0.0010) [2023-10-08 10:56:37,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 163315712. Throughput: 0: 1832.4, 1: 1830.0. Samples: 40836586. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 10:56:37,015][52710] Avg episode reward: [(0, '34.270'), (1, '34.860')] [2023-10-08 10:56:38,943][53885] Updated weights for policy 1, policy_version 79562 (0.0008) [2023-10-08 10:56:39,314][53885] Updated weights for policy 1, policy_version 79572 (0.0008) [2023-10-08 10:56:39,676][53885] Updated weights for policy 1, policy_version 79582 (0.0010) [2023-10-08 10:56:39,943][53852] Updated weights for policy 0, policy_version 79940 (0.0010) [2023-10-08 10:56:40,313][53852] Updated weights for policy 0, policy_version 79950 (0.0008) [2023-10-08 10:56:40,681][53852] Updated weights for policy 0, policy_version 79960 (0.0009) [2023-10-08 10:56:42,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163381248. Throughput: 0: 1841.6, 1: 1825.8. Samples: 40848488. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:56:42,016][52710] Avg episode reward: [(0, '32.930'), (1, '34.960')] [2023-10-08 10:56:43,319][53885] Updated weights for policy 1, policy_version 79592 (0.0009) [2023-10-08 10:56:43,681][53885] Updated weights for policy 1, policy_version 79602 (0.0008) [2023-10-08 10:56:44,047][53885] Updated weights for policy 1, policy_version 79612 (0.0010) [2023-10-08 10:56:44,102][53852] Updated weights for policy 0, policy_version 79970 (0.0010) [2023-10-08 10:56:44,466][53852] Updated weights for policy 0, policy_version 79980 (0.0010) [2023-10-08 10:56:44,837][53852] Updated weights for policy 0, policy_version 79990 (0.0009) [2023-10-08 10:56:45,207][53852] Updated weights for policy 0, policy_version 80000 (0.0008) [2023-10-08 10:56:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163446784. Throughput: 0: 1840.7, 1: 1825.7. Samples: 40869956. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:56:47,016][52710] Avg episode reward: [(0, '33.540'), (1, '35.200')] [2023-10-08 10:56:47,717][53885] Updated weights for policy 1, policy_version 79622 (0.0007) [2023-10-08 10:56:48,091][53885] Updated weights for policy 1, policy_version 79632 (0.0008) [2023-10-08 10:56:48,467][53885] Updated weights for policy 1, policy_version 79642 (0.0010) [2023-10-08 10:56:48,976][53852] Updated weights for policy 0, policy_version 80010 (0.0009) [2023-10-08 10:56:49,351][53852] Updated weights for policy 0, policy_version 80020 (0.0008) [2023-10-08 10:56:49,711][53852] Updated weights for policy 0, policy_version 80030 (0.0010) [2023-10-08 10:56:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 163512320. Throughput: 0: 1840.2, 1: 1830.9. Samples: 40892742. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:56:52,015][52710] Avg episode reward: [(0, '32.030'), (1, '30.040')] [2023-10-08 10:56:52,123][53885] Updated weights for policy 1, policy_version 79652 (0.0008) [2023-10-08 10:56:52,495][53885] Updated weights for policy 1, policy_version 79662 (0.0008) [2023-10-08 10:56:52,861][53885] Updated weights for policy 1, policy_version 79672 (0.0008) [2023-10-08 10:56:53,247][53852] Updated weights for policy 0, policy_version 80040 (0.0011) [2023-10-08 10:56:53,606][53852] Updated weights for policy 0, policy_version 80050 (0.0009) [2023-10-08 10:56:53,976][53852] Updated weights for policy 0, policy_version 80060 (0.0008) [2023-10-08 10:56:56,521][53885] Updated weights for policy 1, policy_version 79682 (0.0008) [2023-10-08 10:56:56,889][53885] Updated weights for policy 1, policy_version 79692 (0.0007) [2023-10-08 10:56:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 163577856. Throughput: 0: 1845.7, 1: 1835.1. Samples: 40903098. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:56:57,016][52710] Avg episode reward: [(0, '35.630'), (1, '34.270')] [2023-10-08 10:56:57,262][53885] Updated weights for policy 1, policy_version 79702 (0.0007) [2023-10-08 10:56:57,561][53852] Updated weights for policy 0, policy_version 80070 (0.0008) [2023-10-08 10:56:57,633][53885] Updated weights for policy 1, policy_version 79712 (0.0008) [2023-10-08 10:56:57,938][53852] Updated weights for policy 0, policy_version 80080 (0.0007) [2023-10-08 10:56:58,305][53852] Updated weights for policy 0, policy_version 80090 (0.0009) [2023-10-08 10:57:01,124][53885] Updated weights for policy 1, policy_version 79722 (0.0008) [2023-10-08 10:57:01,506][53885] Updated weights for policy 1, policy_version 79732 (0.0007) [2023-10-08 10:57:01,877][53885] Updated weights for policy 1, policy_version 79742 (0.0008) [2023-10-08 10:57:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163676160. Throughput: 0: 1845.5, 1: 1841.6. Samples: 40926042. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:57:02,016][52710] Avg episode reward: [(0, '33.210'), (1, '31.260')] [2023-10-08 10:57:02,039][53852] Updated weights for policy 0, policy_version 80100 (0.0008) [2023-10-08 10:57:02,407][53852] Updated weights for policy 0, policy_version 80110 (0.0008) [2023-10-08 10:57:02,783][53852] Updated weights for policy 0, policy_version 80120 (0.0007) [2023-10-08 10:57:05,516][53885] Updated weights for policy 1, policy_version 79752 (0.0009) [2023-10-08 10:57:05,889][53885] Updated weights for policy 1, policy_version 79762 (0.0008) [2023-10-08 10:57:06,259][53885] Updated weights for policy 1, policy_version 79772 (0.0007) [2023-10-08 10:57:06,471][53852] Updated weights for policy 0, policy_version 80130 (0.0007) [2023-10-08 10:57:06,838][53852] Updated weights for policy 0, policy_version 80140 (0.0008) [2023-10-08 10:57:07,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 163741696. Throughput: 0: 1836.1, 1: 1827.4. Samples: 40947304. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:57:07,015][52710] Avg episode reward: [(0, '32.830'), (1, '31.170')] [2023-10-08 10:57:07,209][53852] Updated weights for policy 0, policy_version 80150 (0.0008) [2023-10-08 10:57:07,585][53852] Updated weights for policy 0, policy_version 80160 (0.0010) [2023-10-08 10:57:09,919][53885] Updated weights for policy 1, policy_version 79782 (0.0008) [2023-10-08 10:57:10,281][53885] Updated weights for policy 1, policy_version 79792 (0.0010) [2023-10-08 10:57:10,642][53885] Updated weights for policy 1, policy_version 79802 (0.0010) [2023-10-08 10:57:11,147][53852] Updated weights for policy 0, policy_version 80170 (0.0010) [2023-10-08 10:57:11,520][53852] Updated weights for policy 0, policy_version 80180 (0.0008) [2023-10-08 10:57:11,896][53852] Updated weights for policy 0, policy_version 80190 (0.0010) [2023-10-08 10:57:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 163840000. Throughput: 0: 1843.1, 1: 1844.7. Samples: 40959168. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:57:12,016][52710] Avg episode reward: [(0, '34.480'), (1, '32.560')] [2023-10-08 10:57:14,293][53885] Updated weights for policy 1, policy_version 79812 (0.0008) [2023-10-08 10:57:14,658][53885] Updated weights for policy 1, policy_version 79822 (0.0009) [2023-10-08 10:57:15,037][53885] Updated weights for policy 1, policy_version 79832 (0.0008) [2023-10-08 10:57:15,595][53852] Updated weights for policy 0, policy_version 80200 (0.0009) [2023-10-08 10:57:15,957][53852] Updated weights for policy 0, policy_version 80210 (0.0007) [2023-10-08 10:57:16,328][53852] Updated weights for policy 0, policy_version 80220 (0.0007) [2023-10-08 10:57:17,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 163905536. Throughput: 0: 1841.9, 1: 1838.0. Samples: 40980514. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:57:17,016][52710] Avg episode reward: [(0, '37.170'), (1, '34.350')] [2023-10-08 10:57:17,018][53500] Saving new best policy, reward=37.170! [2023-10-08 10:57:18,724][53885] Updated weights for policy 1, policy_version 79842 (0.0008) [2023-10-08 10:57:19,095][53885] Updated weights for policy 1, policy_version 79852 (0.0007) [2023-10-08 10:57:19,461][53885] Updated weights for policy 1, policy_version 79862 (0.0008) [2023-10-08 10:57:19,818][53885] Updated weights for policy 1, policy_version 79872 (0.0008) [2023-10-08 10:57:19,992][53852] Updated weights for policy 0, policy_version 80230 (0.0008) [2023-10-08 10:57:20,365][53852] Updated weights for policy 0, policy_version 80240 (0.0010) [2023-10-08 10:57:20,738][53852] Updated weights for policy 0, policy_version 80250 (0.0009) [2023-10-08 10:57:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 163971072. Throughput: 0: 1842.8, 1: 1840.7. Samples: 41002342. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:57:22,016][52710] Avg episode reward: [(0, '32.550'), (1, '31.430')] [2023-10-08 10:57:23,432][53885] Updated weights for policy 1, policy_version 79882 (0.0010) [2023-10-08 10:57:23,799][53885] Updated weights for policy 1, policy_version 79892 (0.0007) [2023-10-08 10:57:24,161][53885] Updated weights for policy 1, policy_version 79902 (0.0007) [2023-10-08 10:57:24,207][53852] Updated weights for policy 0, policy_version 80260 (0.0009) [2023-10-08 10:57:24,586][53852] Updated weights for policy 0, policy_version 80270 (0.0007) [2023-10-08 10:57:24,960][53852] Updated weights for policy 0, policy_version 80280 (0.0008) [2023-10-08 10:57:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164036608. Throughput: 0: 1829.2, 1: 1835.6. Samples: 41013408. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:57:27,016][52710] Avg episode reward: [(0, '34.280'), (1, '34.160')] [2023-10-08 10:57:27,890][53885] Updated weights for policy 1, policy_version 79912 (0.0010) [2023-10-08 10:57:28,267][53885] Updated weights for policy 1, policy_version 79922 (0.0010) [2023-10-08 10:57:28,629][53885] Updated weights for policy 1, policy_version 79932 (0.0007) [2023-10-08 10:57:28,646][53852] Updated weights for policy 0, policy_version 80290 (0.0007) [2023-10-08 10:57:29,012][53852] Updated weights for policy 0, policy_version 80300 (0.0008) [2023-10-08 10:57:29,387][53852] Updated weights for policy 0, policy_version 80310 (0.0010) [2023-10-08 10:57:29,749][53852] Updated weights for policy 0, policy_version 80320 (0.0009) [2023-10-08 10:57:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164102144. Throughput: 0: 1832.8, 1: 1836.9. Samples: 41035096. Policy #0 lag: (min: 23.0, avg: 29.9, max: 55.0) [2023-10-08 10:57:32,015][52710] Avg episode reward: [(0, '34.080'), (1, '33.010')] [2023-10-08 10:57:32,409][53885] Updated weights for policy 1, policy_version 79942 (0.0010) [2023-10-08 10:57:32,789][53885] Updated weights for policy 1, policy_version 79952 (0.0007) [2023-10-08 10:57:33,157][53885] Updated weights for policy 1, policy_version 79962 (0.0009) [2023-10-08 10:57:33,396][53852] Updated weights for policy 0, policy_version 80330 (0.0009) [2023-10-08 10:57:33,763][53852] Updated weights for policy 0, policy_version 80340 (0.0009) [2023-10-08 10:57:34,136][53852] Updated weights for policy 0, policy_version 80350 (0.0008) [2023-10-08 10:57:37,009][53885] Updated weights for policy 1, policy_version 79972 (0.0008) [2023-10-08 10:57:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 164167680. Throughput: 0: 1844.1, 1: 1830.6. Samples: 41058106. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:57:37,016][52710] Avg episode reward: [(0, '34.290'), (1, '33.400')] [2023-10-08 10:57:37,395][53885] Updated weights for policy 1, policy_version 79982 (0.0010) [2023-10-08 10:57:37,761][53885] Updated weights for policy 1, policy_version 79992 (0.0007) [2023-10-08 10:57:37,823][53852] Updated weights for policy 0, policy_version 80360 (0.0008) [2023-10-08 10:57:38,193][53852] Updated weights for policy 0, policy_version 80370 (0.0008) [2023-10-08 10:57:38,559][53852] Updated weights for policy 0, policy_version 80380 (0.0007) [2023-10-08 10:57:41,500][53885] Updated weights for policy 1, policy_version 80002 (0.0008) [2023-10-08 10:57:41,869][53885] Updated weights for policy 1, policy_version 80012 (0.0008) [2023-10-08 10:57:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 164233216. Throughput: 0: 1837.4, 1: 1827.1. Samples: 41068000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:57:42,015][52710] Avg episode reward: [(0, '30.310'), (1, '33.220')] [2023-10-08 10:57:42,161][53852] Updated weights for policy 0, policy_version 80390 (0.0008) [2023-10-08 10:57:42,238][53885] Updated weights for policy 1, policy_version 80022 (0.0007) [2023-10-08 10:57:42,525][53852] Updated weights for policy 0, policy_version 80400 (0.0007) [2023-10-08 10:57:42,604][53885] Updated weights for policy 1, policy_version 80032 (0.0007) [2023-10-08 10:57:42,893][53852] Updated weights for policy 0, policy_version 80410 (0.0008) [2023-10-08 10:57:46,222][53885] Updated weights for policy 1, policy_version 80042 (0.0007) [2023-10-08 10:57:46,582][53885] Updated weights for policy 1, policy_version 80052 (0.0008) [2023-10-08 10:57:46,666][53852] Updated weights for policy 0, policy_version 80420 (0.0008) [2023-10-08 10:57:46,939][53885] Updated weights for policy 1, policy_version 80062 (0.0008) [2023-10-08 10:57:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164331520. Throughput: 0: 1839.0, 1: 1820.2. Samples: 41090708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:57:47,016][52710] Avg episode reward: [(0, '33.890'), (1, '33.530')] [2023-10-08 10:57:47,039][53852] Updated weights for policy 0, policy_version 80430 (0.0007) [2023-10-08 10:57:47,404][53852] Updated weights for policy 0, policy_version 80440 (0.0010) [2023-10-08 10:57:50,587][53885] Updated weights for policy 1, policy_version 80072 (0.0008) [2023-10-08 10:57:50,961][53885] Updated weights for policy 1, policy_version 80082 (0.0010) [2023-10-08 10:57:51,056][53852] Updated weights for policy 0, policy_version 80450 (0.0007) [2023-10-08 10:57:51,333][53885] Updated weights for policy 1, policy_version 80092 (0.0008) [2023-10-08 10:57:51,423][53852] Updated weights for policy 0, policy_version 80460 (0.0007) [2023-10-08 10:57:51,800][53852] Updated weights for policy 0, policy_version 80470 (0.0008) [2023-10-08 10:57:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 164397056. Throughput: 0: 1825.9, 1: 1819.3. Samples: 41111338. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:57:52,016][52710] Avg episode reward: [(0, '32.030'), (1, '33.720')] [2023-10-08 10:57:52,169][53852] Updated weights for policy 0, policy_version 80480 (0.0009) [2023-10-08 10:57:55,047][53885] Updated weights for policy 1, policy_version 80102 (0.0007) [2023-10-08 10:57:55,410][53885] Updated weights for policy 1, policy_version 80112 (0.0008) [2023-10-08 10:57:55,774][53885] Updated weights for policy 1, policy_version 80122 (0.0008) [2023-10-08 10:57:55,777][53852] Updated weights for policy 0, policy_version 80490 (0.0007) [2023-10-08 10:57:56,146][53852] Updated weights for policy 0, policy_version 80500 (0.0009) [2023-10-08 10:57:56,520][53852] Updated weights for policy 0, policy_version 80510 (0.0008) [2023-10-08 10:57:57,015][52710] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 164495360. Throughput: 0: 1836.2, 1: 1815.0. Samples: 41123474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:57:57,017][52710] Avg episode reward: [(0, '33.540'), (1, '29.700')] [2023-10-08 10:57:59,400][53885] Updated weights for policy 1, policy_version 80132 (0.0008) [2023-10-08 10:57:59,770][53885] Updated weights for policy 1, policy_version 80142 (0.0008) [2023-10-08 10:58:00,128][53885] Updated weights for policy 1, policy_version 80152 (0.0007) [2023-10-08 10:58:00,263][53852] Updated weights for policy 0, policy_version 80520 (0.0008) [2023-10-08 10:58:00,631][53852] Updated weights for policy 0, policy_version 80530 (0.0008) [2023-10-08 10:58:01,010][53852] Updated weights for policy 0, policy_version 80540 (0.0010) [2023-10-08 10:58:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164560896. Throughput: 0: 1825.8, 1: 1814.8. Samples: 41144340. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:58:02,016][52710] Avg episode reward: [(0, '30.780'), (1, '31.470')] [2023-10-08 10:58:03,854][53885] Updated weights for policy 1, policy_version 80162 (0.0009) [2023-10-08 10:58:04,216][53885] Updated weights for policy 1, policy_version 80172 (0.0008) [2023-10-08 10:58:04,588][53885] Updated weights for policy 1, policy_version 80182 (0.0009) [2023-10-08 10:58:04,631][53852] Updated weights for policy 0, policy_version 80550 (0.0008) [2023-10-08 10:58:04,957][53885] Updated weights for policy 1, policy_version 80192 (0.0007) [2023-10-08 10:58:05,001][53852] Updated weights for policy 0, policy_version 80560 (0.0008) [2023-10-08 10:58:05,371][53852] Updated weights for policy 0, policy_version 80570 (0.0008) [2023-10-08 10:58:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 164626432. Throughput: 0: 1836.3, 1: 1814.0. Samples: 41166608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:58:07,016][52710] Avg episode reward: [(0, '33.120'), (1, '34.330')] [2023-10-08 10:58:07,030][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000080576_82509824.pth... [2023-10-08 10:58:07,031][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000080192_82116608.pth... [2023-10-08 10:58:07,066][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000078496_80379904.pth [2023-10-08 10:58:07,067][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000078848_80740352.pth [2023-10-08 10:58:08,588][53885] Updated weights for policy 1, policy_version 80202 (0.0008) [2023-10-08 10:58:08,953][53885] Updated weights for policy 1, policy_version 80212 (0.0007) [2023-10-08 10:58:09,012][53852] Updated weights for policy 0, policy_version 80580 (0.0008) [2023-10-08 10:58:09,320][53885] Updated weights for policy 1, policy_version 80222 (0.0007) [2023-10-08 10:58:09,381][53852] Updated weights for policy 0, policy_version 80590 (0.0009) [2023-10-08 10:58:09,744][53852] Updated weights for policy 0, policy_version 80600 (0.0007) [2023-10-08 10:58:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 164691968. Throughput: 0: 1827.4, 1: 1814.6. Samples: 41177298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:58:12,016][52710] Avg episode reward: [(0, '30.700'), (1, '36.150')] [2023-10-08 10:58:12,890][53885] Updated weights for policy 1, policy_version 80232 (0.0009) [2023-10-08 10:58:13,263][53885] Updated weights for policy 1, policy_version 80242 (0.0010) [2023-10-08 10:58:13,418][53852] Updated weights for policy 0, policy_version 80610 (0.0009) [2023-10-08 10:58:13,622][53885] Updated weights for policy 1, policy_version 80252 (0.0008) [2023-10-08 10:58:13,784][53852] Updated weights for policy 0, policy_version 80620 (0.0008) [2023-10-08 10:58:14,152][53852] Updated weights for policy 0, policy_version 80630 (0.0010) [2023-10-08 10:58:14,521][53852] Updated weights for policy 0, policy_version 80640 (0.0010) [2023-10-08 10:58:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 164757504. Throughput: 0: 1840.6, 1: 1816.2. Samples: 41199652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:58:17,016][52710] Avg episode reward: [(0, '29.340'), (1, '35.890')] [2023-10-08 10:58:17,330][53885] Updated weights for policy 1, policy_version 80262 (0.0009) [2023-10-08 10:58:17,703][53885] Updated weights for policy 1, policy_version 80272 (0.0010) [2023-10-08 10:58:18,067][53885] Updated weights for policy 1, policy_version 80282 (0.0008) [2023-10-08 10:58:18,141][53852] Updated weights for policy 0, policy_version 80650 (0.0009) [2023-10-08 10:58:18,502][53852] Updated weights for policy 0, policy_version 80660 (0.0007) [2023-10-08 10:58:18,873][53852] Updated weights for policy 0, policy_version 80670 (0.0008) [2023-10-08 10:58:21,711][53885] Updated weights for policy 1, policy_version 80292 (0.0008) [2023-10-08 10:58:22,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 164823040. Throughput: 0: 1839.6, 1: 1825.2. Samples: 41223018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:58:22,015][52710] Avg episode reward: [(0, '29.220'), (1, '38.830')] [2023-10-08 10:58:22,102][53885] Updated weights for policy 1, policy_version 80302 (0.0009) [2023-10-08 10:58:22,276][53852] Updated weights for policy 0, policy_version 80680 (0.0008) [2023-10-08 10:58:22,470][53885] Updated weights for policy 1, policy_version 80312 (0.0008) [2023-10-08 10:58:22,644][53852] Updated weights for policy 0, policy_version 80690 (0.0009) [2023-10-08 10:58:23,019][53852] Updated weights for policy 0, policy_version 80700 (0.0007) [2023-10-08 10:58:26,136][53885] Updated weights for policy 1, policy_version 80322 (0.0008) [2023-10-08 10:58:26,504][53885] Updated weights for policy 1, policy_version 80332 (0.0009) [2023-10-08 10:58:26,626][53852] Updated weights for policy 0, policy_version 80710 (0.0008) [2023-10-08 10:58:26,873][53885] Updated weights for policy 1, policy_version 80342 (0.0007) [2023-10-08 10:58:26,989][53852] Updated weights for policy 0, policy_version 80720 (0.0008) [2023-10-08 10:58:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 164888576. Throughput: 0: 1840.3, 1: 1825.4. Samples: 41232956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 10:58:27,016][52710] Avg episode reward: [(0, '32.340'), (1, '36.250')] [2023-10-08 10:58:27,239][53885] Updated weights for policy 1, policy_version 80352 (0.0007) [2023-10-08 10:58:27,368][53852] Updated weights for policy 0, policy_version 80730 (0.0008) [2023-10-08 10:58:30,960][53852] Updated weights for policy 0, policy_version 80740 (0.0009) [2023-10-08 10:58:31,067][53885] Updated weights for policy 1, policy_version 80362 (0.0008) [2023-10-08 10:58:31,326][53852] Updated weights for policy 0, policy_version 80750 (0.0008) [2023-10-08 10:58:31,427][53885] Updated weights for policy 1, policy_version 80372 (0.0007) [2023-10-08 10:58:31,699][53852] Updated weights for policy 0, policy_version 80760 (0.0007) [2023-10-08 10:58:31,796][53885] Updated weights for policy 1, policy_version 80382 (0.0008) [2023-10-08 10:58:32,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 165019648. Throughput: 0: 1846.5, 1: 1822.6. Samples: 41255818. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:58:32,016][52710] Avg episode reward: [(0, '31.310'), (1, '35.500')] [2023-10-08 10:58:35,249][53852] Updated weights for policy 0, policy_version 80770 (0.0009) [2023-10-08 10:58:35,520][53885] Updated weights for policy 1, policy_version 80392 (0.0009) [2023-10-08 10:58:35,623][53852] Updated weights for policy 0, policy_version 80780 (0.0008) [2023-10-08 10:58:35,882][53885] Updated weights for policy 1, policy_version 80402 (0.0008) [2023-10-08 10:58:35,995][53852] Updated weights for policy 0, policy_version 80790 (0.0008) [2023-10-08 10:58:36,252][53885] Updated weights for policy 1, policy_version 80412 (0.0007) [2023-10-08 10:58:36,352][53852] Updated weights for policy 0, policy_version 80800 (0.0007) [2023-10-08 10:58:37,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 165085184. Throughput: 0: 1832.8, 1: 1823.0. Samples: 41275848. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:58:37,015][52710] Avg episode reward: [(0, '34.080'), (1, '35.930')] [2023-10-08 10:58:40,021][53885] Updated weights for policy 1, policy_version 80422 (0.0008) [2023-10-08 10:58:40,022][53852] Updated weights for policy 0, policy_version 80810 (0.0007) [2023-10-08 10:58:40,383][53885] Updated weights for policy 1, policy_version 80432 (0.0008) [2023-10-08 10:58:40,389][53852] Updated weights for policy 0, policy_version 80820 (0.0007) [2023-10-08 10:58:40,750][53885] Updated weights for policy 1, policy_version 80442 (0.0007) [2023-10-08 10:58:40,760][53852] Updated weights for policy 0, policy_version 80830 (0.0008) [2023-10-08 10:58:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 165150720. Throughput: 0: 1853.4, 1: 1827.1. Samples: 41289096. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:58:42,016][52710] Avg episode reward: [(0, '33.310'), (1, '34.520')] [2023-10-08 10:58:44,367][53885] Updated weights for policy 1, policy_version 80452 (0.0008) [2023-10-08 10:58:44,554][53852] Updated weights for policy 0, policy_version 80840 (0.0008) [2023-10-08 10:58:44,742][53885] Updated weights for policy 1, policy_version 80462 (0.0008) [2023-10-08 10:58:44,907][53852] Updated weights for policy 0, policy_version 80850 (0.0008) [2023-10-08 10:58:45,102][53885] Updated weights for policy 1, policy_version 80472 (0.0007) [2023-10-08 10:58:45,272][53852] Updated weights for policy 0, policy_version 80860 (0.0009) [2023-10-08 10:58:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165216256. Throughput: 0: 1826.9, 1: 1821.7. Samples: 41308524. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:58:47,015][52710] Avg episode reward: [(0, '33.350'), (1, '33.970')] [2023-10-08 10:58:48,860][53885] Updated weights for policy 1, policy_version 80482 (0.0008) [2023-10-08 10:58:49,043][53852] Updated weights for policy 0, policy_version 80870 (0.0009) [2023-10-08 10:58:49,230][53885] Updated weights for policy 1, policy_version 80492 (0.0007) [2023-10-08 10:58:49,419][53852] Updated weights for policy 0, policy_version 80880 (0.0008) [2023-10-08 10:58:49,600][53885] Updated weights for policy 1, policy_version 80502 (0.0009) [2023-10-08 10:58:49,784][53852] Updated weights for policy 0, policy_version 80890 (0.0008) [2023-10-08 10:58:49,964][53885] Updated weights for policy 1, policy_version 80512 (0.0008) [2023-10-08 10:58:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165281792. Throughput: 0: 1845.7, 1: 1815.7. Samples: 41331372. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:58:52,016][52710] Avg episode reward: [(0, '33.220'), (1, '37.620')] [2023-10-08 10:58:53,520][53852] Updated weights for policy 0, policy_version 80900 (0.0008) [2023-10-08 10:58:53,638][53885] Updated weights for policy 1, policy_version 80522 (0.0007) [2023-10-08 10:58:53,888][53852] Updated weights for policy 0, policy_version 80910 (0.0007) [2023-10-08 10:58:54,004][53885] Updated weights for policy 1, policy_version 80532 (0.0007) [2023-10-08 10:58:54,261][53852] Updated weights for policy 0, policy_version 80920 (0.0010) [2023-10-08 10:58:54,368][53885] Updated weights for policy 1, policy_version 80542 (0.0007) [2023-10-08 10:58:57,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 165347328. Throughput: 0: 1828.8, 1: 1815.3. Samples: 41341282. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:58:57,016][52710] Avg episode reward: [(0, '32.760'), (1, '34.210')] [2023-10-08 10:58:57,754][53852] Updated weights for policy 0, policy_version 80930 (0.0010) [2023-10-08 10:58:57,941][53885] Updated weights for policy 1, policy_version 80552 (0.0008) [2023-10-08 10:58:58,120][53852] Updated weights for policy 0, policy_version 80940 (0.0008) [2023-10-08 10:58:58,311][53885] Updated weights for policy 1, policy_version 80562 (0.0009) [2023-10-08 10:58:58,488][53852] Updated weights for policy 0, policy_version 80950 (0.0009) [2023-10-08 10:58:58,680][53885] Updated weights for policy 1, policy_version 80572 (0.0009) [2023-10-08 10:58:58,856][53852] Updated weights for policy 0, policy_version 80960 (0.0009) [2023-10-08 10:59:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 165412864. Throughput: 0: 1841.5, 1: 1821.4. Samples: 41364480. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:59:02,015][52710] Avg episode reward: [(0, '29.830'), (1, '35.090')] [2023-10-08 10:59:02,472][53885] Updated weights for policy 1, policy_version 80582 (0.0007) [2023-10-08 10:59:02,629][53852] Updated weights for policy 0, policy_version 80970 (0.0009) [2023-10-08 10:59:02,838][53885] Updated weights for policy 1, policy_version 80592 (0.0008) [2023-10-08 10:59:02,988][53852] Updated weights for policy 0, policy_version 80980 (0.0008) [2023-10-08 10:59:03,208][53885] Updated weights for policy 1, policy_version 80602 (0.0007) [2023-10-08 10:59:03,363][53852] Updated weights for policy 0, policy_version 80990 (0.0007) [2023-10-08 10:59:07,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 165478400. Throughput: 0: 1837.1, 1: 1811.8. Samples: 41387218. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:59:07,015][52710] Avg episode reward: [(0, '34.400'), (1, '36.220')] [2023-10-08 10:59:07,035][53852] Updated weights for policy 0, policy_version 81000 (0.0008) [2023-10-08 10:59:07,046][53885] Updated weights for policy 1, policy_version 80612 (0.0007) [2023-10-08 10:59:07,398][53852] Updated weights for policy 0, policy_version 81010 (0.0007) [2023-10-08 10:59:07,440][53885] Updated weights for policy 1, policy_version 80622 (0.0008) [2023-10-08 10:59:07,763][53852] Updated weights for policy 0, policy_version 81020 (0.0008) [2023-10-08 10:59:07,807][53885] Updated weights for policy 1, policy_version 80632 (0.0007) [2023-10-08 10:59:11,404][53852] Updated weights for policy 0, policy_version 81030 (0.0008) [2023-10-08 10:59:11,503][53885] Updated weights for policy 1, policy_version 80642 (0.0007) [2023-10-08 10:59:11,767][53852] Updated weights for policy 0, policy_version 81040 (0.0009) [2023-10-08 10:59:11,875][53885] Updated weights for policy 1, policy_version 80652 (0.0008) [2023-10-08 10:59:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165543936. Throughput: 0: 1835.9, 1: 1811.3. Samples: 41397078. Policy #0 lag: (min: 1.0, avg: 9.8, max: 33.0) [2023-10-08 10:59:12,016][52710] Avg episode reward: [(0, '31.810'), (1, '34.430')] [2023-10-08 10:59:12,138][53852] Updated weights for policy 0, policy_version 81050 (0.0008) [2023-10-08 10:59:12,243][53885] Updated weights for policy 1, policy_version 80662 (0.0008) [2023-10-08 10:59:12,607][53885] Updated weights for policy 1, policy_version 80672 (0.0008) [2023-10-08 10:59:15,807][53852] Updated weights for policy 0, policy_version 81060 (0.0008) [2023-10-08 10:59:16,177][53852] Updated weights for policy 0, policy_version 81070 (0.0008) [2023-10-08 10:59:16,368][53885] Updated weights for policy 1, policy_version 80682 (0.0010) [2023-10-08 10:59:16,553][53852] Updated weights for policy 0, policy_version 81080 (0.0008) [2023-10-08 10:59:16,745][53885] Updated weights for policy 1, policy_version 80692 (0.0009) [2023-10-08 10:59:17,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165642240. Throughput: 0: 1835.2, 1: 1808.9. Samples: 41419802. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:17,016][52710] Avg episode reward: [(0, '28.940'), (1, '35.230')] [2023-10-08 10:59:17,110][53885] Updated weights for policy 1, policy_version 80702 (0.0009) [2023-10-08 10:59:20,257][53852] Updated weights for policy 0, policy_version 81090 (0.0009) [2023-10-08 10:59:20,629][53852] Updated weights for policy 0, policy_version 81100 (0.0010) [2023-10-08 10:59:20,987][53885] Updated weights for policy 1, policy_version 80712 (0.0008) [2023-10-08 10:59:20,996][53852] Updated weights for policy 0, policy_version 81110 (0.0009) [2023-10-08 10:59:21,355][53885] Updated weights for policy 1, policy_version 80722 (0.0008) [2023-10-08 10:59:21,367][53852] Updated weights for policy 0, policy_version 81120 (0.0007) [2023-10-08 10:59:21,726][53885] Updated weights for policy 1, policy_version 80732 (0.0010) [2023-10-08 10:59:22,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 165740544. Throughput: 0: 1831.3, 1: 1809.6. Samples: 41439688. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:22,016][52710] Avg episode reward: [(0, '30.550'), (1, '36.990')] [2023-10-08 10:59:24,938][53852] Updated weights for policy 0, policy_version 81130 (0.0007) [2023-10-08 10:59:25,302][53852] Updated weights for policy 0, policy_version 81140 (0.0008) [2023-10-08 10:59:25,411][53885] Updated weights for policy 1, policy_version 80742 (0.0008) [2023-10-08 10:59:25,670][53852] Updated weights for policy 0, policy_version 81150 (0.0008) [2023-10-08 10:59:25,785][53885] Updated weights for policy 1, policy_version 80752 (0.0009) [2023-10-08 10:59:26,147][53885] Updated weights for policy 1, policy_version 80762 (0.0008) [2023-10-08 10:59:27,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 165806080. Throughput: 0: 1833.7, 1: 1794.7. Samples: 41452376. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:27,015][52710] Avg episode reward: [(0, '31.630'), (1, '34.910')] [2023-10-08 10:59:29,321][53852] Updated weights for policy 0, policy_version 81160 (0.0008) [2023-10-08 10:59:29,695][53852] Updated weights for policy 0, policy_version 81170 (0.0009) [2023-10-08 10:59:29,797][53885] Updated weights for policy 1, policy_version 80772 (0.0008) [2023-10-08 10:59:30,069][53852] Updated weights for policy 0, policy_version 81180 (0.0009) [2023-10-08 10:59:30,159][53885] Updated weights for policy 1, policy_version 80782 (0.0007) [2023-10-08 10:59:30,518][53885] Updated weights for policy 1, policy_version 80792 (0.0010) [2023-10-08 10:59:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 165871616. Throughput: 0: 1848.8, 1: 1805.8. Samples: 41472980. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:32,016][52710] Avg episode reward: [(0, '28.530'), (1, '34.360')] [2023-10-08 10:59:33,594][53852] Updated weights for policy 0, policy_version 81190 (0.0008) [2023-10-08 10:59:33,979][53852] Updated weights for policy 0, policy_version 81200 (0.0009) [2023-10-08 10:59:34,234][53885] Updated weights for policy 1, policy_version 80802 (0.0009) [2023-10-08 10:59:34,338][53852] Updated weights for policy 0, policy_version 81210 (0.0009) [2023-10-08 10:59:34,596][53885] Updated weights for policy 1, policy_version 80812 (0.0007) [2023-10-08 10:59:34,970][53885] Updated weights for policy 1, policy_version 80822 (0.0008) [2023-10-08 10:59:35,336][53885] Updated weights for policy 1, policy_version 80832 (0.0010) [2023-10-08 10:59:37,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 165937152. Throughput: 0: 1843.5, 1: 1798.5. Samples: 41495262. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:37,017][52710] Avg episode reward: [(0, '30.430'), (1, '35.650')] [2023-10-08 10:59:38,028][53852] Updated weights for policy 0, policy_version 81220 (0.0009) [2023-10-08 10:59:38,399][53852] Updated weights for policy 0, policy_version 81230 (0.0009) [2023-10-08 10:59:38,760][53852] Updated weights for policy 0, policy_version 81240 (0.0008) [2023-10-08 10:59:39,030][53885] Updated weights for policy 1, policy_version 80842 (0.0009) [2023-10-08 10:59:39,398][53885] Updated weights for policy 1, policy_version 80852 (0.0011) [2023-10-08 10:59:39,766][53885] Updated weights for policy 1, policy_version 80862 (0.0008) [2023-10-08 10:59:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 166002688. Throughput: 0: 1845.6, 1: 1805.2. Samples: 41505564. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:42,016][52710] Avg episode reward: [(0, '31.330'), (1, '34.110')] [2023-10-08 10:59:42,486][53852] Updated weights for policy 0, policy_version 81250 (0.0008) [2023-10-08 10:59:42,857][53852] Updated weights for policy 0, policy_version 81260 (0.0009) [2023-10-08 10:59:43,227][53852] Updated weights for policy 0, policy_version 81270 (0.0008) [2023-10-08 10:59:43,465][53885] Updated weights for policy 1, policy_version 80872 (0.0007) [2023-10-08 10:59:43,597][53852] Updated weights for policy 0, policy_version 81280 (0.0008) [2023-10-08 10:59:43,830][53885] Updated weights for policy 1, policy_version 80882 (0.0009) [2023-10-08 10:59:44,199][53885] Updated weights for policy 1, policy_version 80892 (0.0008) [2023-10-08 10:59:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 166068224. Throughput: 0: 1841.9, 1: 1788.7. Samples: 41527858. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:47,016][52710] Avg episode reward: [(0, '30.080'), (1, '35.030')] [2023-10-08 10:59:47,313][53852] Updated weights for policy 0, policy_version 81290 (0.0009) [2023-10-08 10:59:47,688][53852] Updated weights for policy 0, policy_version 81300 (0.0010) [2023-10-08 10:59:47,926][53885] Updated weights for policy 1, policy_version 80902 (0.0008) [2023-10-08 10:59:48,057][53852] Updated weights for policy 0, policy_version 81310 (0.0008) [2023-10-08 10:59:48,285][53885] Updated weights for policy 1, policy_version 80912 (0.0008) [2023-10-08 10:59:48,657][53885] Updated weights for policy 1, policy_version 80922 (0.0010) [2023-10-08 10:59:51,770][53852] Updated weights for policy 0, policy_version 81320 (0.0008) [2023-10-08 10:59:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166133760. Throughput: 0: 1835.9, 1: 1794.4. Samples: 41550578. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:52,015][52710] Avg episode reward: [(0, '30.650'), (1, '35.110')] [2023-10-08 10:59:52,148][53852] Updated weights for policy 0, policy_version 81330 (0.0009) [2023-10-08 10:59:52,421][53885] Updated weights for policy 1, policy_version 80932 (0.0009) [2023-10-08 10:59:52,505][53852] Updated weights for policy 0, policy_version 81340 (0.0009) [2023-10-08 10:59:52,820][53885] Updated weights for policy 1, policy_version 80942 (0.0009) [2023-10-08 10:59:53,187][53885] Updated weights for policy 1, policy_version 80952 (0.0008) [2023-10-08 10:59:56,123][53852] Updated weights for policy 0, policy_version 81350 (0.0009) [2023-10-08 10:59:56,494][53852] Updated weights for policy 0, policy_version 81360 (0.0009) [2023-10-08 10:59:56,850][53885] Updated weights for policy 1, policy_version 80962 (0.0009) [2023-10-08 10:59:56,867][53852] Updated weights for policy 0, policy_version 81370 (0.0007) [2023-10-08 10:59:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166199296. Throughput: 0: 1839.2, 1: 1794.6. Samples: 41560598. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 10:59:57,016][52710] Avg episode reward: [(0, '30.280'), (1, '36.420')] [2023-10-08 10:59:57,214][53885] Updated weights for policy 1, policy_version 80972 (0.0007) [2023-10-08 10:59:57,590][53885] Updated weights for policy 1, policy_version 80982 (0.0008) [2023-10-08 10:59:57,959][53885] Updated weights for policy 1, policy_version 80992 (0.0009) [2023-10-08 11:00:00,518][53852] Updated weights for policy 0, policy_version 81380 (0.0007) [2023-10-08 11:00:00,891][53852] Updated weights for policy 0, policy_version 81390 (0.0011) [2023-10-08 11:00:01,271][53852] Updated weights for policy 0, policy_version 81400 (0.0008) [2023-10-08 11:00:01,655][53885] Updated weights for policy 1, policy_version 81002 (0.0008) [2023-10-08 11:00:02,013][53885] Updated weights for policy 1, policy_version 81012 (0.0007) [2023-10-08 11:00:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 166297600. Throughput: 0: 1834.0, 1: 1801.9. Samples: 41583416. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 11:00:02,016][52710] Avg episode reward: [(0, '31.590'), (1, '36.810')] [2023-10-08 11:00:02,371][53885] Updated weights for policy 1, policy_version 81022 (0.0007) [2023-10-08 11:00:04,926][53852] Updated weights for policy 0, policy_version 81410 (0.0008) [2023-10-08 11:00:05,295][53852] Updated weights for policy 0, policy_version 81420 (0.0010) [2023-10-08 11:00:05,657][53852] Updated weights for policy 0, policy_version 81430 (0.0008) [2023-10-08 11:00:06,024][53852] Updated weights for policy 0, policy_version 81440 (0.0008) [2023-10-08 11:00:06,044][53885] Updated weights for policy 1, policy_version 81032 (0.0008) [2023-10-08 11:00:06,416][53885] Updated weights for policy 1, policy_version 81042 (0.0008) [2023-10-08 11:00:06,777][53885] Updated weights for policy 1, policy_version 81052 (0.0008) [2023-10-08 11:00:07,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 166395904. Throughput: 0: 1841.1, 1: 1816.1. Samples: 41604260. Policy #0 lag: (min: 17.0, avg: 22.3, max: 49.0) [2023-10-08 11:00:07,016][52710] Avg episode reward: [(0, '31.220'), (1, '37.540')] [2023-10-08 11:00:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000081440_83394560.pth... [2023-10-08 11:00:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000081056_83001344.pth... [2023-10-08 11:00:07,062][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000079712_81625088.pth [2023-10-08 11:00:07,071][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000079328_81231872.pth [2023-10-08 11:00:09,664][53852] Updated weights for policy 0, policy_version 81450 (0.0008) [2023-10-08 11:00:10,026][53852] Updated weights for policy 0, policy_version 81460 (0.0010) [2023-10-08 11:00:10,404][53852] Updated weights for policy 0, policy_version 81470 (0.0010) [2023-10-08 11:00:10,588][53885] Updated weights for policy 1, policy_version 81062 (0.0008) [2023-10-08 11:00:10,958][53885] Updated weights for policy 1, policy_version 81072 (0.0009) [2023-10-08 11:00:11,325][53885] Updated weights for policy 1, policy_version 81082 (0.0008) [2023-10-08 11:00:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 166461440. Throughput: 0: 1830.8, 1: 1813.4. Samples: 41616366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:12,015][52710] Avg episode reward: [(0, '32.210'), (1, '38.580')] [2023-10-08 11:00:14,024][53852] Updated weights for policy 0, policy_version 81480 (0.0007) [2023-10-08 11:00:14,394][53852] Updated weights for policy 0, policy_version 81490 (0.0008) [2023-10-08 11:00:14,755][53852] Updated weights for policy 0, policy_version 81500 (0.0010) [2023-10-08 11:00:14,981][53885] Updated weights for policy 1, policy_version 81092 (0.0008) [2023-10-08 11:00:15,355][53885] Updated weights for policy 1, policy_version 81102 (0.0010) [2023-10-08 11:00:15,712][53885] Updated weights for policy 1, policy_version 81112 (0.0007) [2023-10-08 11:00:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 166526976. Throughput: 0: 1831.6, 1: 1816.4. Samples: 41637142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:17,016][52710] Avg episode reward: [(0, '31.970'), (1, '36.260')] [2023-10-08 11:00:18,182][53852] Updated weights for policy 0, policy_version 81510 (0.0009) [2023-10-08 11:00:18,564][53852] Updated weights for policy 0, policy_version 81520 (0.0010) [2023-10-08 11:00:18,927][53852] Updated weights for policy 0, policy_version 81530 (0.0010) [2023-10-08 11:00:19,398][53885] Updated weights for policy 1, policy_version 81122 (0.0007) [2023-10-08 11:00:19,766][53885] Updated weights for policy 1, policy_version 81132 (0.0007) [2023-10-08 11:00:20,135][53885] Updated weights for policy 1, policy_version 81142 (0.0007) [2023-10-08 11:00:20,508][53885] Updated weights for policy 1, policy_version 81152 (0.0010) [2023-10-08 11:00:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 166592512. Throughput: 0: 1839.4, 1: 1821.7. Samples: 41660014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:22,016][52710] Avg episode reward: [(0, '30.020'), (1, '35.620')] [2023-10-08 11:00:22,599][53852] Updated weights for policy 0, policy_version 81540 (0.0008) [2023-10-08 11:00:22,974][53852] Updated weights for policy 0, policy_version 81550 (0.0008) [2023-10-08 11:00:23,342][53852] Updated weights for policy 0, policy_version 81560 (0.0007) [2023-10-08 11:00:23,954][53885] Updated weights for policy 1, policy_version 81162 (0.0009) [2023-10-08 11:00:24,325][53885] Updated weights for policy 1, policy_version 81172 (0.0007) [2023-10-08 11:00:24,687][53885] Updated weights for policy 1, policy_version 81182 (0.0009) [2023-10-08 11:00:26,789][53852] Updated weights for policy 0, policy_version 81570 (0.0007) [2023-10-08 11:00:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 166658048. Throughput: 0: 1842.5, 1: 1823.4. Samples: 41670532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:27,016][52710] Avg episode reward: [(0, '31.320'), (1, '38.970')] [2023-10-08 11:00:27,150][53852] Updated weights for policy 0, policy_version 81580 (0.0007) [2023-10-08 11:00:27,523][53852] Updated weights for policy 0, policy_version 81590 (0.0007) [2023-10-08 11:00:27,888][53852] Updated weights for policy 0, policy_version 81600 (0.0007) [2023-10-08 11:00:28,236][53885] Updated weights for policy 1, policy_version 81192 (0.0009) [2023-10-08 11:00:28,597][53885] Updated weights for policy 1, policy_version 81202 (0.0009) [2023-10-08 11:00:28,960][53885] Updated weights for policy 1, policy_version 81212 (0.0007) [2023-10-08 11:00:31,561][53852] Updated weights for policy 0, policy_version 81610 (0.0009) [2023-10-08 11:00:31,931][53852] Updated weights for policy 0, policy_version 81620 (0.0007) [2023-10-08 11:00:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 166723584. Throughput: 0: 1851.0, 1: 1826.8. Samples: 41693360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:32,016][52710] Avg episode reward: [(0, '33.440'), (1, '36.970')] [2023-10-08 11:00:32,298][53852] Updated weights for policy 0, policy_version 81630 (0.0008) [2023-10-08 11:00:32,807][53885] Updated weights for policy 1, policy_version 81222 (0.0009) [2023-10-08 11:00:33,183][53885] Updated weights for policy 1, policy_version 81232 (0.0008) [2023-10-08 11:00:33,562][53885] Updated weights for policy 1, policy_version 81242 (0.0008) [2023-10-08 11:00:35,831][53852] Updated weights for policy 0, policy_version 81640 (0.0010) [2023-10-08 11:00:36,203][53852] Updated weights for policy 0, policy_version 81650 (0.0007) [2023-10-08 11:00:36,576][53852] Updated weights for policy 0, policy_version 81660 (0.0009) [2023-10-08 11:00:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 166821888. Throughput: 0: 1827.3, 1: 1831.9. Samples: 41715244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:37,016][52710] Avg episode reward: [(0, '29.220'), (1, '36.010')] [2023-10-08 11:00:37,261][53885] Updated weights for policy 1, policy_version 81252 (0.0007) [2023-10-08 11:00:37,652][53885] Updated weights for policy 1, policy_version 81262 (0.0007) [2023-10-08 11:00:38,008][53885] Updated weights for policy 1, policy_version 81272 (0.0007) [2023-10-08 11:00:40,200][53852] Updated weights for policy 0, policy_version 81670 (0.0007) [2023-10-08 11:00:40,565][53852] Updated weights for policy 0, policy_version 81680 (0.0008) [2023-10-08 11:00:40,935][53852] Updated weights for policy 0, policy_version 81690 (0.0009) [2023-10-08 11:00:41,488][53885] Updated weights for policy 1, policy_version 81282 (0.0009) [2023-10-08 11:00:41,855][53885] Updated weights for policy 1, policy_version 81292 (0.0009) [2023-10-08 11:00:42,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 166887424. Throughput: 0: 1858.1, 1: 1835.9. Samples: 41726826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:42,016][52710] Avg episode reward: [(0, '31.260'), (1, '36.780')] [2023-10-08 11:00:42,222][53885] Updated weights for policy 1, policy_version 81302 (0.0008) [2023-10-08 11:00:42,579][53885] Updated weights for policy 1, policy_version 81312 (0.0008) [2023-10-08 11:00:44,571][53852] Updated weights for policy 0, policy_version 81700 (0.0007) [2023-10-08 11:00:44,937][53852] Updated weights for policy 0, policy_version 81710 (0.0007) [2023-10-08 11:00:45,314][53852] Updated weights for policy 0, policy_version 81720 (0.0008) [2023-10-08 11:00:46,345][53885] Updated weights for policy 1, policy_version 81322 (0.0009) [2023-10-08 11:00:46,713][53885] Updated weights for policy 1, policy_version 81332 (0.0008) [2023-10-08 11:00:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166952960. Throughput: 0: 1834.0, 1: 1836.1. Samples: 41748568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:47,016][52710] Avg episode reward: [(0, '32.400'), (1, '38.530')] [2023-10-08 11:00:47,073][53885] Updated weights for policy 1, policy_version 81342 (0.0007) [2023-10-08 11:00:48,921][53852] Updated weights for policy 0, policy_version 81730 (0.0008) [2023-10-08 11:00:49,282][53852] Updated weights for policy 0, policy_version 81740 (0.0009) [2023-10-08 11:00:49,651][53852] Updated weights for policy 0, policy_version 81750 (0.0008) [2023-10-08 11:00:50,020][53852] Updated weights for policy 0, policy_version 81760 (0.0010) [2023-10-08 11:00:50,792][53885] Updated weights for policy 1, policy_version 81352 (0.0008) [2023-10-08 11:00:51,160][53885] Updated weights for policy 1, policy_version 81362 (0.0011) [2023-10-08 11:00:51,525][53885] Updated weights for policy 1, policy_version 81372 (0.0008) [2023-10-08 11:00:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 167051264. Throughput: 0: 1865.2, 1: 1824.7. Samples: 41770302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:52,016][52710] Avg episode reward: [(0, '31.810'), (1, '32.770')] [2023-10-08 11:00:53,565][53852] Updated weights for policy 0, policy_version 81770 (0.0007) [2023-10-08 11:00:53,921][53852] Updated weights for policy 0, policy_version 81780 (0.0008) [2023-10-08 11:00:54,292][53852] Updated weights for policy 0, policy_version 81790 (0.0010) [2023-10-08 11:00:55,140][53885] Updated weights for policy 1, policy_version 81382 (0.0008) [2023-10-08 11:00:55,503][53885] Updated weights for policy 1, policy_version 81392 (0.0009) [2023-10-08 11:00:55,873][53885] Updated weights for policy 1, policy_version 81402 (0.0007) [2023-10-08 11:00:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 167116800. Throughput: 0: 1836.8, 1: 1840.7. Samples: 41781854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:00:57,016][52710] Avg episode reward: [(0, '30.980'), (1, '35.560')] [2023-10-08 11:00:57,943][53852] Updated weights for policy 0, policy_version 81800 (0.0007) [2023-10-08 11:00:58,304][53852] Updated weights for policy 0, policy_version 81810 (0.0010) [2023-10-08 11:00:58,668][53852] Updated weights for policy 0, policy_version 81820 (0.0008) [2023-10-08 11:00:59,452][53885] Updated weights for policy 1, policy_version 81412 (0.0007) [2023-10-08 11:00:59,831][53885] Updated weights for policy 1, policy_version 81422 (0.0011) [2023-10-08 11:01:00,188][53885] Updated weights for policy 1, policy_version 81432 (0.0009) [2023-10-08 11:01:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 167182336. Throughput: 0: 1870.5, 1: 1833.8. Samples: 41803836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:01:02,015][52710] Avg episode reward: [(0, '32.580'), (1, '41.130')] [2023-10-08 11:01:02,316][53852] Updated weights for policy 0, policy_version 81830 (0.0008) [2023-10-08 11:01:02,677][53852] Updated weights for policy 0, policy_version 81840 (0.0008) [2023-10-08 11:01:03,045][53852] Updated weights for policy 0, policy_version 81850 (0.0007) [2023-10-08 11:01:03,829][53885] Updated weights for policy 1, policy_version 81442 (0.0009) [2023-10-08 11:01:04,202][53885] Updated weights for policy 1, policy_version 81452 (0.0009) [2023-10-08 11:01:04,562][53885] Updated weights for policy 1, policy_version 81462 (0.0009) [2023-10-08 11:01:04,935][53885] Updated weights for policy 1, policy_version 81472 (0.0009) [2023-10-08 11:01:06,728][53852] Updated weights for policy 0, policy_version 81860 (0.0007) [2023-10-08 11:01:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 167247872. Throughput: 0: 1865.5, 1: 1841.6. Samples: 41826836. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:07,016][52710] Avg episode reward: [(0, '32.140'), (1, '35.300')] [2023-10-08 11:01:07,114][53852] Updated weights for policy 0, policy_version 81870 (0.0007) [2023-10-08 11:01:07,493][53852] Updated weights for policy 0, policy_version 81880 (0.0009) [2023-10-08 11:01:08,603][53885] Updated weights for policy 1, policy_version 81482 (0.0010) [2023-10-08 11:01:08,967][53885] Updated weights for policy 1, policy_version 81492 (0.0010) [2023-10-08 11:01:09,335][53885] Updated weights for policy 1, policy_version 81502 (0.0011) [2023-10-08 11:01:11,034][53852] Updated weights for policy 0, policy_version 81890 (0.0011) [2023-10-08 11:01:11,407][53852] Updated weights for policy 0, policy_version 81900 (0.0009) [2023-10-08 11:01:11,774][53852] Updated weights for policy 0, policy_version 81910 (0.0008) [2023-10-08 11:01:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 167313408. Throughput: 0: 1859.0, 1: 1833.0. Samples: 41836672. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:12,016][52710] Avg episode reward: [(0, '32.060'), (1, '37.520')] [2023-10-08 11:01:12,141][53852] Updated weights for policy 0, policy_version 81920 (0.0007) [2023-10-08 11:01:13,014][53885] Updated weights for policy 1, policy_version 81512 (0.0009) [2023-10-08 11:01:13,384][53885] Updated weights for policy 1, policy_version 81522 (0.0008) [2023-10-08 11:01:13,745][53885] Updated weights for policy 1, policy_version 81532 (0.0008) [2023-10-08 11:01:15,967][53852] Updated weights for policy 0, policy_version 81930 (0.0007) [2023-10-08 11:01:16,335][53852] Updated weights for policy 0, policy_version 81940 (0.0007) [2023-10-08 11:01:16,716][53852] Updated weights for policy 0, policy_version 81950 (0.0007) [2023-10-08 11:01:17,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 167411712. Throughput: 0: 1855.0, 1: 1840.7. Samples: 41859664. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:17,016][52710] Avg episode reward: [(0, '32.150'), (1, '36.370')] [2023-10-08 11:01:17,585][53885] Updated weights for policy 1, policy_version 81542 (0.0009) [2023-10-08 11:01:17,951][53885] Updated weights for policy 1, policy_version 81552 (0.0011) [2023-10-08 11:01:18,318][53885] Updated weights for policy 1, policy_version 81562 (0.0010) [2023-10-08 11:01:20,278][53852] Updated weights for policy 0, policy_version 81960 (0.0007) [2023-10-08 11:01:20,649][53852] Updated weights for policy 0, policy_version 81970 (0.0008) [2023-10-08 11:01:21,012][53852] Updated weights for policy 0, policy_version 81980 (0.0008) [2023-10-08 11:01:21,962][53885] Updated weights for policy 1, policy_version 81572 (0.0010) [2023-10-08 11:01:22,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 167477248. Throughput: 0: 1853.5, 1: 1841.3. Samples: 41881508. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:22,016][52710] Avg episode reward: [(0, '32.430'), (1, '40.250')] [2023-10-08 11:01:22,368][53885] Updated weights for policy 1, policy_version 81582 (0.0009) [2023-10-08 11:01:22,738][53885] Updated weights for policy 1, policy_version 81592 (0.0009) [2023-10-08 11:01:24,613][53852] Updated weights for policy 0, policy_version 81990 (0.0007) [2023-10-08 11:01:24,987][53852] Updated weights for policy 0, policy_version 82000 (0.0007) [2023-10-08 11:01:25,353][53852] Updated weights for policy 0, policy_version 82010 (0.0007) [2023-10-08 11:01:26,406][53885] Updated weights for policy 1, policy_version 81602 (0.0008) [2023-10-08 11:01:26,783][53885] Updated weights for policy 1, policy_version 81612 (0.0009) [2023-10-08 11:01:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 167542784. Throughput: 0: 1853.7, 1: 1834.7. Samples: 41892806. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:27,016][52710] Avg episode reward: [(0, '31.350'), (1, '34.940')] [2023-10-08 11:01:27,145][53885] Updated weights for policy 1, policy_version 81622 (0.0008) [2023-10-08 11:01:27,510][53885] Updated weights for policy 1, policy_version 81632 (0.0008) [2023-10-08 11:01:28,895][53852] Updated weights for policy 0, policy_version 82020 (0.0009) [2023-10-08 11:01:29,262][53852] Updated weights for policy 0, policy_version 82030 (0.0010) [2023-10-08 11:01:29,640][53852] Updated weights for policy 0, policy_version 82040 (0.0009) [2023-10-08 11:01:31,191][53885] Updated weights for policy 1, policy_version 81642 (0.0009) [2023-10-08 11:01:31,550][53885] Updated weights for policy 1, policy_version 81652 (0.0008) [2023-10-08 11:01:31,916][53885] Updated weights for policy 1, policy_version 81662 (0.0008) [2023-10-08 11:01:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 167641088. Throughput: 0: 1856.3, 1: 1835.7. Samples: 41914712. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:32,016][52710] Avg episode reward: [(0, '30.470'), (1, '35.910')] [2023-10-08 11:01:33,262][53852] Updated weights for policy 0, policy_version 82050 (0.0008) [2023-10-08 11:01:33,624][53852] Updated weights for policy 0, policy_version 82060 (0.0009) [2023-10-08 11:01:33,983][53852] Updated weights for policy 0, policy_version 82070 (0.0009) [2023-10-08 11:01:34,352][53852] Updated weights for policy 0, policy_version 82080 (0.0010) [2023-10-08 11:01:35,584][53885] Updated weights for policy 1, policy_version 81672 (0.0008) [2023-10-08 11:01:35,954][53885] Updated weights for policy 1, policy_version 81682 (0.0011) [2023-10-08 11:01:36,319][53885] Updated weights for policy 1, policy_version 81692 (0.0010) [2023-10-08 11:01:37,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 167706624. Throughput: 0: 1857.0, 1: 1829.2. Samples: 41936180. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:37,015][52710] Avg episode reward: [(0, '31.910'), (1, '39.440')] [2023-10-08 11:01:37,963][53852] Updated weights for policy 0, policy_version 82090 (0.0008) [2023-10-08 11:01:38,325][53852] Updated weights for policy 0, policy_version 82100 (0.0010) [2023-10-08 11:01:38,703][53852] Updated weights for policy 0, policy_version 82110 (0.0007) [2023-10-08 11:01:40,028][53885] Updated weights for policy 1, policy_version 81702 (0.0010) [2023-10-08 11:01:40,392][53885] Updated weights for policy 1, policy_version 81712 (0.0010) [2023-10-08 11:01:40,753][53885] Updated weights for policy 1, policy_version 81722 (0.0010) [2023-10-08 11:01:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 167772160. Throughput: 0: 1854.0, 1: 1827.8. Samples: 41947534. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:42,016][52710] Avg episode reward: [(0, '30.710'), (1, '35.130')] [2023-10-08 11:01:42,311][53852] Updated weights for policy 0, policy_version 82120 (0.0009) [2023-10-08 11:01:42,678][53852] Updated weights for policy 0, policy_version 82130 (0.0009) [2023-10-08 11:01:43,046][53852] Updated weights for policy 0, policy_version 82140 (0.0008) [2023-10-08 11:01:44,372][53885] Updated weights for policy 1, policy_version 81732 (0.0009) [2023-10-08 11:01:44,734][53885] Updated weights for policy 1, policy_version 81742 (0.0007) [2023-10-08 11:01:45,109][53885] Updated weights for policy 1, policy_version 81752 (0.0008) [2023-10-08 11:01:46,622][53852] Updated weights for policy 0, policy_version 82150 (0.0007) [2023-10-08 11:01:46,991][53852] Updated weights for policy 0, policy_version 82160 (0.0007) [2023-10-08 11:01:47,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 167837696. Throughput: 0: 1848.4, 1: 1825.4. Samples: 41969154. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:47,016][52710] Avg episode reward: [(0, '30.340'), (1, '35.880')] [2023-10-08 11:01:47,373][53852] Updated weights for policy 0, policy_version 82170 (0.0010) [2023-10-08 11:01:48,967][53885] Updated weights for policy 1, policy_version 81762 (0.0009) [2023-10-08 11:01:49,338][53885] Updated weights for policy 1, policy_version 81772 (0.0010) [2023-10-08 11:01:49,707][53885] Updated weights for policy 1, policy_version 81782 (0.0007) [2023-10-08 11:01:50,079][53885] Updated weights for policy 1, policy_version 81792 (0.0008) [2023-10-08 11:01:51,111][53852] Updated weights for policy 0, policy_version 82180 (0.0007) [2023-10-08 11:01:51,494][53852] Updated weights for policy 0, policy_version 82190 (0.0008) [2023-10-08 11:01:51,857][53852] Updated weights for policy 0, policy_version 82200 (0.0007) [2023-10-08 11:01:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 167903232. Throughput: 0: 1830.2, 1: 1816.0. Samples: 41990914. Policy #0 lag: (min: 4.0, avg: 29.3, max: 32.0) [2023-10-08 11:01:52,016][52710] Avg episode reward: [(0, '30.830'), (1, '34.970')] [2023-10-08 11:01:53,811][53885] Updated weights for policy 1, policy_version 81802 (0.0009) [2023-10-08 11:01:54,178][53885] Updated weights for policy 1, policy_version 81812 (0.0008) [2023-10-08 11:01:54,550][53885] Updated weights for policy 1, policy_version 81822 (0.0010) [2023-10-08 11:01:55,518][53852] Updated weights for policy 0, policy_version 82210 (0.0009) [2023-10-08 11:01:55,930][53852] Updated weights for policy 0, policy_version 82220 (0.0009) [2023-10-08 11:01:56,309][53852] Updated weights for policy 0, policy_version 82230 (0.0008) [2023-10-08 11:01:56,680][53852] Updated weights for policy 0, policy_version 82240 (0.0008) [2023-10-08 11:01:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168001536. Throughput: 0: 1854.8, 1: 1818.7. Samples: 42001978. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:01:57,016][52710] Avg episode reward: [(0, '32.770'), (1, '39.020')] [2023-10-08 11:01:58,194][53885] Updated weights for policy 1, policy_version 81832 (0.0009) [2023-10-08 11:01:58,567][53885] Updated weights for policy 1, policy_version 81842 (0.0009) [2023-10-08 11:01:58,942][53885] Updated weights for policy 1, policy_version 81852 (0.0009) [2023-10-08 11:02:00,224][53852] Updated weights for policy 0, policy_version 82250 (0.0009) [2023-10-08 11:02:00,599][53852] Updated weights for policy 0, policy_version 82260 (0.0010) [2023-10-08 11:02:00,957][53852] Updated weights for policy 0, policy_version 82270 (0.0007) [2023-10-08 11:02:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168067072. Throughput: 0: 1835.0, 1: 1816.4. Samples: 42023978. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:02,016][52710] Avg episode reward: [(0, '29.840'), (1, '35.060')] [2023-10-08 11:02:02,645][53885] Updated weights for policy 1, policy_version 81862 (0.0009) [2023-10-08 11:02:03,017][53885] Updated weights for policy 1, policy_version 81872 (0.0009) [2023-10-08 11:02:03,389][53885] Updated weights for policy 1, policy_version 81882 (0.0008) [2023-10-08 11:02:04,644][53852] Updated weights for policy 0, policy_version 82280 (0.0009) [2023-10-08 11:02:05,029][53852] Updated weights for policy 0, policy_version 82290 (0.0011) [2023-10-08 11:02:05,392][53852] Updated weights for policy 0, policy_version 82300 (0.0011) [2023-10-08 11:02:07,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168132608. Throughput: 0: 1846.7, 1: 1809.7. Samples: 42046044. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:07,016][52710] Avg episode reward: [(0, '32.570'), (1, '35.640')] [2023-10-08 11:02:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000082304_84279296.pth... [2023-10-08 11:02:07,055][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000080576_82509824.pth [2023-10-08 11:02:07,222][53885] Updated weights for policy 1, policy_version 81892 (0.0010) [2023-10-08 11:02:07,608][53885] Updated weights for policy 1, policy_version 81902 (0.0011) [2023-10-08 11:02:07,971][53885] Updated weights for policy 1, policy_version 81912 (0.0009) [2023-10-08 11:02:08,261][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000081920_83886080.pth... [2023-10-08 11:02:08,289][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000080192_82116608.pth [2023-10-08 11:02:09,104][53852] Updated weights for policy 0, policy_version 82310 (0.0009) [2023-10-08 11:02:09,460][53852] Updated weights for policy 0, policy_version 82320 (0.0008) [2023-10-08 11:02:09,841][53852] Updated weights for policy 0, policy_version 82330 (0.0007) [2023-10-08 11:02:11,504][53885] Updated weights for policy 1, policy_version 81922 (0.0011) [2023-10-08 11:02:11,876][53885] Updated weights for policy 1, policy_version 81932 (0.0011) [2023-10-08 11:02:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168198144. Throughput: 0: 1832.3, 1: 1812.9. Samples: 42056838. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:12,016][52710] Avg episode reward: [(0, '31.470'), (1, '37.990')] [2023-10-08 11:02:12,248][53885] Updated weights for policy 1, policy_version 81942 (0.0008) [2023-10-08 11:02:12,610][53885] Updated weights for policy 1, policy_version 81952 (0.0011) [2023-10-08 11:02:13,487][53852] Updated weights for policy 0, policy_version 82340 (0.0008) [2023-10-08 11:02:13,853][53852] Updated weights for policy 0, policy_version 82350 (0.0008) [2023-10-08 11:02:14,231][53852] Updated weights for policy 0, policy_version 82360 (0.0011) [2023-10-08 11:02:16,288][53885] Updated weights for policy 1, policy_version 81962 (0.0007) [2023-10-08 11:02:16,655][53885] Updated weights for policy 1, policy_version 81972 (0.0007) [2023-10-08 11:02:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 168263680. Throughput: 0: 1843.6, 1: 1816.6. Samples: 42079420. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:17,016][52710] Avg episode reward: [(0, '31.590'), (1, '39.800')] [2023-10-08 11:02:17,022][53885] Updated weights for policy 1, policy_version 81982 (0.0009) [2023-10-08 11:02:17,750][53852] Updated weights for policy 0, policy_version 82370 (0.0009) [2023-10-08 11:02:18,126][53852] Updated weights for policy 0, policy_version 82380 (0.0009) [2023-10-08 11:02:18,491][53852] Updated weights for policy 0, policy_version 82390 (0.0011) [2023-10-08 11:02:18,859][53852] Updated weights for policy 0, policy_version 82400 (0.0007) [2023-10-08 11:02:20,659][53885] Updated weights for policy 1, policy_version 81992 (0.0009) [2023-10-08 11:02:21,039][53885] Updated weights for policy 1, policy_version 82002 (0.0011) [2023-10-08 11:02:21,398][53885] Updated weights for policy 1, policy_version 82012 (0.0009) [2023-10-08 11:02:22,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 168361984. Throughput: 0: 1848.4, 1: 1818.6. Samples: 42101194. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:22,016][52710] Avg episode reward: [(0, '29.050'), (1, '35.070')] [2023-10-08 11:02:22,431][53852] Updated weights for policy 0, policy_version 82410 (0.0008) [2023-10-08 11:02:22,803][53852] Updated weights for policy 0, policy_version 82420 (0.0007) [2023-10-08 11:02:23,175][53852] Updated weights for policy 0, policy_version 82430 (0.0007) [2023-10-08 11:02:25,009][53885] Updated weights for policy 1, policy_version 82022 (0.0008) [2023-10-08 11:02:25,363][53885] Updated weights for policy 1, policy_version 82032 (0.0008) [2023-10-08 11:02:25,731][53885] Updated weights for policy 1, policy_version 82042 (0.0009) [2023-10-08 11:02:26,722][53852] Updated weights for policy 0, policy_version 82440 (0.0010) [2023-10-08 11:02:27,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 168427520. Throughput: 0: 1847.0, 1: 1822.8. Samples: 42112674. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:27,015][52710] Avg episode reward: [(0, '31.850'), (1, '37.520')] [2023-10-08 11:02:27,086][53852] Updated weights for policy 0, policy_version 82450 (0.0010) [2023-10-08 11:02:27,466][53852] Updated weights for policy 0, policy_version 82460 (0.0010) [2023-10-08 11:02:29,430][53885] Updated weights for policy 1, policy_version 82052 (0.0008) [2023-10-08 11:02:29,808][53885] Updated weights for policy 1, policy_version 82062 (0.0010) [2023-10-08 11:02:30,171][53885] Updated weights for policy 1, policy_version 82072 (0.0009) [2023-10-08 11:02:31,110][53852] Updated weights for policy 0, policy_version 82470 (0.0010) [2023-10-08 11:02:31,478][53852] Updated weights for policy 0, policy_version 82480 (0.0008) [2023-10-08 11:02:31,843][53852] Updated weights for policy 0, policy_version 82490 (0.0009) [2023-10-08 11:02:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 168493056. Throughput: 0: 1849.6, 1: 1821.3. Samples: 42134346. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:32,016][52710] Avg episode reward: [(0, '31.310'), (1, '35.810')] [2023-10-08 11:02:33,915][53885] Updated weights for policy 1, policy_version 82082 (0.0008) [2023-10-08 11:02:34,286][53885] Updated weights for policy 1, policy_version 82092 (0.0008) [2023-10-08 11:02:34,657][53885] Updated weights for policy 1, policy_version 82102 (0.0008) [2023-10-08 11:02:35,021][53885] Updated weights for policy 1, policy_version 82112 (0.0008) [2023-10-08 11:02:35,577][53852] Updated weights for policy 0, policy_version 82500 (0.0009) [2023-10-08 11:02:35,956][53852] Updated weights for policy 0, policy_version 82510 (0.0008) [2023-10-08 11:02:36,318][53852] Updated weights for policy 0, policy_version 82520 (0.0008) [2023-10-08 11:02:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 168591360. Throughput: 0: 1834.7, 1: 1831.0. Samples: 42155872. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:37,016][52710] Avg episode reward: [(0, '31.170'), (1, '35.280')] [2023-10-08 11:02:38,706][53885] Updated weights for policy 1, policy_version 82122 (0.0008) [2023-10-08 11:02:39,079][53885] Updated weights for policy 1, policy_version 82132 (0.0009) [2023-10-08 11:02:39,445][53885] Updated weights for policy 1, policy_version 82142 (0.0007) [2023-10-08 11:02:39,957][53852] Updated weights for policy 0, policy_version 82530 (0.0008) [2023-10-08 11:02:40,327][53852] Updated weights for policy 0, policy_version 82540 (0.0009) [2023-10-08 11:02:40,698][53852] Updated weights for policy 0, policy_version 82550 (0.0008) [2023-10-08 11:02:41,059][53852] Updated weights for policy 0, policy_version 82560 (0.0007) [2023-10-08 11:02:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168656896. Throughput: 0: 1846.0, 1: 1825.5. Samples: 42167192. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:42,016][52710] Avg episode reward: [(0, '33.710'), (1, '42.390')] [2023-10-08 11:02:42,017][53594] Saving new best policy, reward=42.390! [2023-10-08 11:02:43,196][53885] Updated weights for policy 1, policy_version 82152 (0.0008) [2023-10-08 11:02:43,560][53885] Updated weights for policy 1, policy_version 82162 (0.0007) [2023-10-08 11:02:43,926][53885] Updated weights for policy 1, policy_version 82172 (0.0008) [2023-10-08 11:02:44,945][53852] Updated weights for policy 0, policy_version 82570 (0.0009) [2023-10-08 11:02:45,313][53852] Updated weights for policy 0, policy_version 82580 (0.0008) [2023-10-08 11:02:45,685][53852] Updated weights for policy 0, policy_version 82590 (0.0009) [2023-10-08 11:02:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168722432. Throughput: 0: 1827.4, 1: 1826.3. Samples: 42188394. Policy #0 lag: (min: 29.0, avg: 31.8, max: 61.0) [2023-10-08 11:02:47,016][52710] Avg episode reward: [(0, '33.540'), (1, '36.670')] [2023-10-08 11:02:47,513][53885] Updated weights for policy 1, policy_version 82182 (0.0009) [2023-10-08 11:02:47,879][53885] Updated weights for policy 1, policy_version 82192 (0.0009) [2023-10-08 11:02:48,241][53885] Updated weights for policy 1, policy_version 82202 (0.0008) [2023-10-08 11:02:49,346][53852] Updated weights for policy 0, policy_version 82600 (0.0010) [2023-10-08 11:02:49,713][53852] Updated weights for policy 0, policy_version 82610 (0.0009) [2023-10-08 11:02:50,084][53852] Updated weights for policy 0, policy_version 82620 (0.0009) [2023-10-08 11:02:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 168787968. Throughput: 0: 1834.8, 1: 1830.5. Samples: 42210984. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:02:52,015][52710] Avg episode reward: [(0, '32.950'), (1, '35.790')] [2023-10-08 11:02:52,056][53885] Updated weights for policy 1, policy_version 82212 (0.0009) [2023-10-08 11:02:52,453][53885] Updated weights for policy 1, policy_version 82222 (0.0007) [2023-10-08 11:02:52,817][53885] Updated weights for policy 1, policy_version 82232 (0.0010) [2023-10-08 11:02:53,666][53852] Updated weights for policy 0, policy_version 82630 (0.0010) [2023-10-08 11:02:54,020][53852] Updated weights for policy 0, policy_version 82640 (0.0010) [2023-10-08 11:02:54,389][53852] Updated weights for policy 0, policy_version 82650 (0.0011) [2023-10-08 11:02:56,355][53885] Updated weights for policy 1, policy_version 82242 (0.0010) [2023-10-08 11:02:56,723][53885] Updated weights for policy 1, policy_version 82252 (0.0007) [2023-10-08 11:02:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 168853504. Throughput: 0: 1819.6, 1: 1827.6. Samples: 42220962. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:02:57,016][52710] Avg episode reward: [(0, '31.330'), (1, '37.450')] [2023-10-08 11:02:57,100][53885] Updated weights for policy 1, policy_version 82262 (0.0010) [2023-10-08 11:02:57,470][53885] Updated weights for policy 1, policy_version 82272 (0.0007) [2023-10-08 11:02:58,010][53852] Updated weights for policy 0, policy_version 82660 (0.0011) [2023-10-08 11:02:58,380][53852] Updated weights for policy 0, policy_version 82670 (0.0010) [2023-10-08 11:02:58,756][53852] Updated weights for policy 0, policy_version 82680 (0.0007) [2023-10-08 11:03:01,071][53885] Updated weights for policy 1, policy_version 82282 (0.0007) [2023-10-08 11:03:01,436][53885] Updated weights for policy 1, policy_version 82292 (0.0009) [2023-10-08 11:03:01,812][53885] Updated weights for policy 1, policy_version 82302 (0.0009) [2023-10-08 11:03:02,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168951808. Throughput: 0: 1835.3, 1: 1824.4. Samples: 42244104. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:02,016][52710] Avg episode reward: [(0, '34.190'), (1, '38.460')] [2023-10-08 11:03:02,318][53852] Updated weights for policy 0, policy_version 82690 (0.0009) [2023-10-08 11:03:02,691][53852] Updated weights for policy 0, policy_version 82700 (0.0007) [2023-10-08 11:03:03,067][53852] Updated weights for policy 0, policy_version 82710 (0.0007) [2023-10-08 11:03:03,441][53852] Updated weights for policy 0, policy_version 82720 (0.0008) [2023-10-08 11:03:05,400][53885] Updated weights for policy 1, policy_version 82312 (0.0009) [2023-10-08 11:03:05,771][53885] Updated weights for policy 1, policy_version 82322 (0.0009) [2023-10-08 11:03:06,139][53885] Updated weights for policy 1, policy_version 82332 (0.0010) [2023-10-08 11:03:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169017344. Throughput: 0: 1825.7, 1: 1826.6. Samples: 42265546. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:07,016][52710] Avg episode reward: [(0, '33.270'), (1, '32.990')] [2023-10-08 11:03:07,232][53852] Updated weights for policy 0, policy_version 82730 (0.0008) [2023-10-08 11:03:07,600][53852] Updated weights for policy 0, policy_version 82740 (0.0008) [2023-10-08 11:03:07,965][53852] Updated weights for policy 0, policy_version 82750 (0.0010) [2023-10-08 11:03:09,887][53885] Updated weights for policy 1, policy_version 82342 (0.0009) [2023-10-08 11:03:10,254][53885] Updated weights for policy 1, policy_version 82352 (0.0009) [2023-10-08 11:03:10,617][53885] Updated weights for policy 1, policy_version 82362 (0.0008) [2023-10-08 11:03:11,561][53852] Updated weights for policy 0, policy_version 82760 (0.0007) [2023-10-08 11:03:11,934][53852] Updated weights for policy 0, policy_version 82770 (0.0008) [2023-10-08 11:03:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169082880. Throughput: 0: 1828.7, 1: 1825.6. Samples: 42277120. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:12,016][52710] Avg episode reward: [(0, '32.520'), (1, '37.140')] [2023-10-08 11:03:12,315][53852] Updated weights for policy 0, policy_version 82780 (0.0009) [2023-10-08 11:03:14,244][53885] Updated weights for policy 1, policy_version 82372 (0.0008) [2023-10-08 11:03:14,620][53885] Updated weights for policy 1, policy_version 82382 (0.0010) [2023-10-08 11:03:14,978][53885] Updated weights for policy 1, policy_version 82392 (0.0010) [2023-10-08 11:03:15,849][53852] Updated weights for policy 0, policy_version 82790 (0.0009) [2023-10-08 11:03:16,218][53852] Updated weights for policy 0, policy_version 82800 (0.0008) [2023-10-08 11:03:16,595][53852] Updated weights for policy 0, policy_version 82810 (0.0011) [2023-10-08 11:03:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 169181184. Throughput: 0: 1827.9, 1: 1824.4. Samples: 42298702. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:17,016][52710] Avg episode reward: [(0, '32.850'), (1, '37.130')] [2023-10-08 11:03:18,616][53885] Updated weights for policy 1, policy_version 82402 (0.0010) [2023-10-08 11:03:18,979][53885] Updated weights for policy 1, policy_version 82412 (0.0008) [2023-10-08 11:03:19,353][53885] Updated weights for policy 1, policy_version 82422 (0.0008) [2023-10-08 11:03:19,716][53885] Updated weights for policy 1, policy_version 82432 (0.0007) [2023-10-08 11:03:20,307][53852] Updated weights for policy 0, policy_version 82820 (0.0011) [2023-10-08 11:03:20,678][53852] Updated weights for policy 0, policy_version 82830 (0.0010) [2023-10-08 11:03:21,048][53852] Updated weights for policy 0, policy_version 82840 (0.0010) [2023-10-08 11:03:22,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 169246720. Throughput: 0: 1827.1, 1: 1825.9. Samples: 42320256. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:22,015][52710] Avg episode reward: [(0, '33.870'), (1, '36.250')] [2023-10-08 11:03:23,447][53885] Updated weights for policy 1, policy_version 82442 (0.0010) [2023-10-08 11:03:23,827][53885] Updated weights for policy 1, policy_version 82452 (0.0010) [2023-10-08 11:03:24,190][53885] Updated weights for policy 1, policy_version 82462 (0.0011) [2023-10-08 11:03:24,531][53852] Updated weights for policy 0, policy_version 82850 (0.0007) [2023-10-08 11:03:24,911][53852] Updated weights for policy 0, policy_version 82860 (0.0008) [2023-10-08 11:03:25,279][53852] Updated weights for policy 0, policy_version 82870 (0.0008) [2023-10-08 11:03:25,650][53852] Updated weights for policy 0, policy_version 82880 (0.0009) [2023-10-08 11:03:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169312256. Throughput: 0: 1829.1, 1: 1826.7. Samples: 42331704. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:27,016][52710] Avg episode reward: [(0, '30.980'), (1, '35.510')] [2023-10-08 11:03:27,920][53885] Updated weights for policy 1, policy_version 82472 (0.0011) [2023-10-08 11:03:28,286][53885] Updated weights for policy 1, policy_version 82482 (0.0010) [2023-10-08 11:03:28,648][53885] Updated weights for policy 1, policy_version 82492 (0.0010) [2023-10-08 11:03:29,405][53852] Updated weights for policy 0, policy_version 82890 (0.0007) [2023-10-08 11:03:29,772][53852] Updated weights for policy 0, policy_version 82900 (0.0009) [2023-10-08 11:03:30,149][53852] Updated weights for policy 0, policy_version 82910 (0.0010) [2023-10-08 11:03:32,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169377792. Throughput: 0: 1833.4, 1: 1830.4. Samples: 42353266. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:32,016][52710] Avg episode reward: [(0, '31.330'), (1, '39.820')] [2023-10-08 11:03:32,356][53885] Updated weights for policy 1, policy_version 82502 (0.0011) [2023-10-08 11:03:32,727][53885] Updated weights for policy 1, policy_version 82512 (0.0009) [2023-10-08 11:03:33,095][53885] Updated weights for policy 1, policy_version 82522 (0.0007) [2023-10-08 11:03:33,862][53852] Updated weights for policy 0, policy_version 82920 (0.0009) [2023-10-08 11:03:34,247][53852] Updated weights for policy 0, policy_version 82930 (0.0009) [2023-10-08 11:03:34,611][53852] Updated weights for policy 0, policy_version 82940 (0.0008) [2023-10-08 11:03:36,781][53885] Updated weights for policy 1, policy_version 82532 (0.0009) [2023-10-08 11:03:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 169443328. Throughput: 0: 1841.8, 1: 1826.4. Samples: 42376054. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:37,016][52710] Avg episode reward: [(0, '34.260'), (1, '35.980')] [2023-10-08 11:03:37,176][53885] Updated weights for policy 1, policy_version 82542 (0.0010) [2023-10-08 11:03:37,561][53885] Updated weights for policy 1, policy_version 82552 (0.0011) [2023-10-08 11:03:38,244][53852] Updated weights for policy 0, policy_version 82950 (0.0007) [2023-10-08 11:03:38,626][53852] Updated weights for policy 0, policy_version 82960 (0.0007) [2023-10-08 11:03:38,995][53852] Updated weights for policy 0, policy_version 82970 (0.0009) [2023-10-08 11:03:41,247][53885] Updated weights for policy 1, policy_version 82562 (0.0009) [2023-10-08 11:03:41,613][53885] Updated weights for policy 1, policy_version 82572 (0.0008) [2023-10-08 11:03:41,984][53885] Updated weights for policy 1, policy_version 82582 (0.0008) [2023-10-08 11:03:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 169508864. Throughput: 0: 1838.4, 1: 1828.4. Samples: 42385970. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:03:42,016][52710] Avg episode reward: [(0, '31.450'), (1, '37.320')] [2023-10-08 11:03:42,349][53885] Updated weights for policy 1, policy_version 82592 (0.0009) [2023-10-08 11:03:42,620][53852] Updated weights for policy 0, policy_version 82980 (0.0009) [2023-10-08 11:03:42,990][53852] Updated weights for policy 0, policy_version 82990 (0.0008) [2023-10-08 11:03:43,363][53852] Updated weights for policy 0, policy_version 83000 (0.0007) [2023-10-08 11:03:45,944][53885] Updated weights for policy 1, policy_version 82602 (0.0008) [2023-10-08 11:03:46,317][53885] Updated weights for policy 1, policy_version 82612 (0.0008) [2023-10-08 11:03:46,688][53885] Updated weights for policy 1, policy_version 82622 (0.0011) [2023-10-08 11:03:47,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169607168. Throughput: 0: 1835.8, 1: 1823.8. Samples: 42408788. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:03:47,016][52710] Avg episode reward: [(0, '32.800'), (1, '36.510')] [2023-10-08 11:03:47,055][53852] Updated weights for policy 0, policy_version 83010 (0.0009) [2023-10-08 11:03:47,428][53852] Updated weights for policy 0, policy_version 83020 (0.0008) [2023-10-08 11:03:47,800][53852] Updated weights for policy 0, policy_version 83030 (0.0009) [2023-10-08 11:03:48,161][53852] Updated weights for policy 0, policy_version 83040 (0.0008) [2023-10-08 11:03:50,406][53885] Updated weights for policy 1, policy_version 82632 (0.0008) [2023-10-08 11:03:50,778][53885] Updated weights for policy 1, policy_version 82642 (0.0008) [2023-10-08 11:03:51,152][53885] Updated weights for policy 1, policy_version 82652 (0.0008) [2023-10-08 11:03:51,771][53852] Updated weights for policy 0, policy_version 83050 (0.0007) [2023-10-08 11:03:52,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169672704. Throughput: 0: 1836.5, 1: 1820.0. Samples: 42430088. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:03:52,016][52710] Avg episode reward: [(0, '34.310'), (1, '36.640')] [2023-10-08 11:03:52,137][53852] Updated weights for policy 0, policy_version 83060 (0.0007) [2023-10-08 11:03:52,515][53852] Updated weights for policy 0, policy_version 83070 (0.0007) [2023-10-08 11:03:54,885][53885] Updated weights for policy 1, policy_version 82662 (0.0007) [2023-10-08 11:03:55,244][53885] Updated weights for policy 1, policy_version 82672 (0.0010) [2023-10-08 11:03:55,613][53885] Updated weights for policy 1, policy_version 82682 (0.0009) [2023-10-08 11:03:56,209][53852] Updated weights for policy 0, policy_version 83080 (0.0007) [2023-10-08 11:03:56,573][53852] Updated weights for policy 0, policy_version 83090 (0.0008) [2023-10-08 11:03:56,944][53852] Updated weights for policy 0, policy_version 83100 (0.0007) [2023-10-08 11:03:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169738240. Throughput: 0: 1843.0, 1: 1817.2. Samples: 42441826. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:03:57,016][52710] Avg episode reward: [(0, '34.190'), (1, '36.640')] [2023-10-08 11:03:59,144][53885] Updated weights for policy 1, policy_version 82692 (0.0009) [2023-10-08 11:03:59,508][53885] Updated weights for policy 1, policy_version 82702 (0.0011) [2023-10-08 11:03:59,877][53885] Updated weights for policy 1, policy_version 82712 (0.0008) [2023-10-08 11:04:00,558][53852] Updated weights for policy 0, policy_version 83110 (0.0007) [2023-10-08 11:04:00,932][53852] Updated weights for policy 0, policy_version 83120 (0.0008) [2023-10-08 11:04:01,290][53852] Updated weights for policy 0, policy_version 83130 (0.0009) [2023-10-08 11:04:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 169836544. Throughput: 0: 1833.2, 1: 1822.4. Samples: 42463202. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:04:02,016][52710] Avg episode reward: [(0, '30.880'), (1, '34.420')] [2023-10-08 11:04:03,537][53885] Updated weights for policy 1, policy_version 82722 (0.0007) [2023-10-08 11:04:03,897][53885] Updated weights for policy 1, policy_version 82732 (0.0007) [2023-10-08 11:04:04,270][53885] Updated weights for policy 1, policy_version 82742 (0.0007) [2023-10-08 11:04:04,636][53885] Updated weights for policy 1, policy_version 82752 (0.0007) [2023-10-08 11:04:05,027][53852] Updated weights for policy 0, policy_version 83140 (0.0008) [2023-10-08 11:04:05,399][53852] Updated weights for policy 0, policy_version 83150 (0.0009) [2023-10-08 11:04:05,777][53852] Updated weights for policy 0, policy_version 83160 (0.0007) [2023-10-08 11:04:07,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 169902080. Throughput: 0: 1840.4, 1: 1822.3. Samples: 42485076. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:04:07,016][52710] Avg episode reward: [(0, '31.420'), (1, '36.400')] [2023-10-08 11:04:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000082752_84738048.pth... [2023-10-08 11:04:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000083168_85164032.pth... [2023-10-08 11:04:07,061][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000081440_83394560.pth [2023-10-08 11:04:07,061][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000081056_83001344.pth [2023-10-08 11:04:08,234][53885] Updated weights for policy 1, policy_version 82762 (0.0009) [2023-10-08 11:04:08,596][53885] Updated weights for policy 1, policy_version 82772 (0.0010) [2023-10-08 11:04:08,973][53885] Updated weights for policy 1, policy_version 82782 (0.0011) [2023-10-08 11:04:09,224][53852] Updated weights for policy 0, policy_version 83170 (0.0008) [2023-10-08 11:04:09,593][53852] Updated weights for policy 0, policy_version 83180 (0.0007) [2023-10-08 11:04:09,963][53852] Updated weights for policy 0, policy_version 83190 (0.0008) [2023-10-08 11:04:10,330][53852] Updated weights for policy 0, policy_version 83200 (0.0007) [2023-10-08 11:04:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 169967616. Throughput: 0: 1833.4, 1: 1824.3. Samples: 42496300. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:04:12,016][52710] Avg episode reward: [(0, '33.040'), (1, '36.390')] [2023-10-08 11:04:12,671][53885] Updated weights for policy 1, policy_version 82792 (0.0010) [2023-10-08 11:04:13,041][53885] Updated weights for policy 1, policy_version 82802 (0.0011) [2023-10-08 11:04:13,413][53885] Updated weights for policy 1, policy_version 82812 (0.0009) [2023-10-08 11:04:13,979][53852] Updated weights for policy 0, policy_version 83210 (0.0007) [2023-10-08 11:04:14,357][53852] Updated weights for policy 0, policy_version 83220 (0.0008) [2023-10-08 11:04:14,722][53852] Updated weights for policy 0, policy_version 83230 (0.0009) [2023-10-08 11:04:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170033152. Throughput: 0: 1840.8, 1: 1823.4. Samples: 42518158. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:04:17,016][52710] Avg episode reward: [(0, '32.430'), (1, '34.210')] [2023-10-08 11:04:17,118][53885] Updated weights for policy 1, policy_version 82822 (0.0008) [2023-10-08 11:04:17,486][53885] Updated weights for policy 1, policy_version 82832 (0.0007) [2023-10-08 11:04:17,849][53885] Updated weights for policy 1, policy_version 82842 (0.0007) [2023-10-08 11:04:18,442][53852] Updated weights for policy 0, policy_version 83240 (0.0007) [2023-10-08 11:04:18,813][53852] Updated weights for policy 0, policy_version 83250 (0.0009) [2023-10-08 11:04:19,183][53852] Updated weights for policy 0, policy_version 83260 (0.0008) [2023-10-08 11:04:21,419][53885] Updated weights for policy 1, policy_version 82852 (0.0009) [2023-10-08 11:04:21,787][53885] Updated weights for policy 1, policy_version 82862 (0.0010) [2023-10-08 11:04:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 170098688. Throughput: 0: 1840.4, 1: 1825.6. Samples: 42541022. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:04:22,016][52710] Avg episode reward: [(0, '31.030'), (1, '33.690')] [2023-10-08 11:04:22,155][53885] Updated weights for policy 1, policy_version 82872 (0.0007) [2023-10-08 11:04:22,803][53852] Updated weights for policy 0, policy_version 83270 (0.0008) [2023-10-08 11:04:23,167][53852] Updated weights for policy 0, policy_version 83280 (0.0007) [2023-10-08 11:04:23,546][53852] Updated weights for policy 0, policy_version 83290 (0.0010) [2023-10-08 11:04:25,948][53885] Updated weights for policy 1, policy_version 82882 (0.0008) [2023-10-08 11:04:26,325][53885] Updated weights for policy 1, policy_version 82892 (0.0007) [2023-10-08 11:04:26,697][53885] Updated weights for policy 1, policy_version 82902 (0.0009) [2023-10-08 11:04:27,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170164224. Throughput: 0: 1841.2, 1: 1836.8. Samples: 42551480. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:04:27,015][52710] Avg episode reward: [(0, '32.130'), (1, '34.430')] [2023-10-08 11:04:27,060][53885] Updated weights for policy 1, policy_version 82912 (0.0009) [2023-10-08 11:04:27,246][53852] Updated weights for policy 0, policy_version 83300 (0.0010) [2023-10-08 11:04:27,610][53852] Updated weights for policy 0, policy_version 83310 (0.0011) [2023-10-08 11:04:27,981][53852] Updated weights for policy 0, policy_version 83320 (0.0010) [2023-10-08 11:04:30,764][53885] Updated weights for policy 1, policy_version 82922 (0.0009) [2023-10-08 11:04:31,134][53885] Updated weights for policy 1, policy_version 82932 (0.0009) [2023-10-08 11:04:31,503][53885] Updated weights for policy 1, policy_version 82942 (0.0008) [2023-10-08 11:04:31,558][53852] Updated weights for policy 0, policy_version 83330 (0.0008) [2023-10-08 11:04:31,925][53852] Updated weights for policy 0, policy_version 83340 (0.0008) [2023-10-08 11:04:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170262528. Throughput: 0: 1841.8, 1: 1832.2. Samples: 42574118. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:04:32,016][52710] Avg episode reward: [(0, '29.640'), (1, '41.900')] [2023-10-08 11:04:32,301][53852] Updated weights for policy 0, policy_version 83350 (0.0009) [2023-10-08 11:04:32,664][53852] Updated weights for policy 0, policy_version 83360 (0.0010) [2023-10-08 11:04:35,199][53885] Updated weights for policy 1, policy_version 82952 (0.0009) [2023-10-08 11:04:35,556][53885] Updated weights for policy 1, policy_version 82962 (0.0011) [2023-10-08 11:04:35,928][53885] Updated weights for policy 1, policy_version 82972 (0.0008) [2023-10-08 11:04:36,287][53852] Updated weights for policy 0, policy_version 83370 (0.0008) [2023-10-08 11:04:36,652][53852] Updated weights for policy 0, policy_version 83380 (0.0009) [2023-10-08 11:04:37,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170328064. Throughput: 0: 1829.9, 1: 1839.5. Samples: 42595212. Policy #0 lag: (min: 28.0, avg: 32.7, max: 60.0) [2023-10-08 11:04:37,016][52710] Avg episode reward: [(0, '29.370'), (1, '39.480')] [2023-10-08 11:04:37,032][53852] Updated weights for policy 0, policy_version 83390 (0.0007) [2023-10-08 11:04:39,469][53885] Updated weights for policy 1, policy_version 82982 (0.0008) [2023-10-08 11:04:39,837][53885] Updated weights for policy 1, policy_version 82992 (0.0008) [2023-10-08 11:04:40,196][53885] Updated weights for policy 1, policy_version 83002 (0.0007) [2023-10-08 11:04:40,537][53852] Updated weights for policy 0, policy_version 83400 (0.0007) [2023-10-08 11:04:40,913][53852] Updated weights for policy 0, policy_version 83410 (0.0007) [2023-10-08 11:04:41,284][53852] Updated weights for policy 0, policy_version 83420 (0.0007) [2023-10-08 11:04:42,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 170426368. Throughput: 0: 1847.0, 1: 1833.8. Samples: 42607462. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:04:42,016][52710] Avg episode reward: [(0, '28.050'), (1, '37.500')] [2023-10-08 11:04:43,778][53885] Updated weights for policy 1, policy_version 83012 (0.0009) [2023-10-08 11:04:44,145][53885] Updated weights for policy 1, policy_version 83022 (0.0007) [2023-10-08 11:04:44,519][53885] Updated weights for policy 1, policy_version 83032 (0.0008) [2023-10-08 11:04:44,857][53852] Updated weights for policy 0, policy_version 83430 (0.0007) [2023-10-08 11:04:45,219][53852] Updated weights for policy 0, policy_version 83440 (0.0007) [2023-10-08 11:04:45,593][53852] Updated weights for policy 0, policy_version 83450 (0.0009) [2023-10-08 11:04:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 170491904. Throughput: 0: 1834.4, 1: 1841.9. Samples: 42628634. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:04:47,016][52710] Avg episode reward: [(0, '30.180'), (1, '41.980')] [2023-10-08 11:04:48,324][53885] Updated weights for policy 1, policy_version 83042 (0.0008) [2023-10-08 11:04:48,687][53885] Updated weights for policy 1, policy_version 83052 (0.0009) [2023-10-08 11:04:49,062][53885] Updated weights for policy 1, policy_version 83062 (0.0010) [2023-10-08 11:04:49,291][53852] Updated weights for policy 0, policy_version 83460 (0.0009) [2023-10-08 11:04:49,424][53885] Updated weights for policy 1, policy_version 83072 (0.0008) [2023-10-08 11:04:49,667][53852] Updated weights for policy 0, policy_version 83470 (0.0007) [2023-10-08 11:04:50,029][53852] Updated weights for policy 0, policy_version 83480 (0.0010) [2023-10-08 11:04:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 170557440. Throughput: 0: 1849.0, 1: 1832.0. Samples: 42650718. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:04:52,015][52710] Avg episode reward: [(0, '31.700'), (1, '40.260')] [2023-10-08 11:04:53,161][53885] Updated weights for policy 1, policy_version 83082 (0.0010) [2023-10-08 11:04:53,522][53885] Updated weights for policy 1, policy_version 83092 (0.0007) [2023-10-08 11:04:53,624][53852] Updated weights for policy 0, policy_version 83490 (0.0010) [2023-10-08 11:04:53,883][53885] Updated weights for policy 1, policy_version 83102 (0.0007) [2023-10-08 11:04:53,993][53852] Updated weights for policy 0, policy_version 83500 (0.0009) [2023-10-08 11:04:54,370][53852] Updated weights for policy 0, policy_version 83510 (0.0010) [2023-10-08 11:04:54,741][53852] Updated weights for policy 0, policy_version 83520 (0.0007) [2023-10-08 11:04:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170622976. Throughput: 0: 1831.2, 1: 1832.5. Samples: 42661166. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:04:57,016][52710] Avg episode reward: [(0, '32.540'), (1, '37.190')] [2023-10-08 11:04:57,421][53885] Updated weights for policy 1, policy_version 83112 (0.0008) [2023-10-08 11:04:57,788][53885] Updated weights for policy 1, policy_version 83122 (0.0008) [2023-10-08 11:04:58,151][53885] Updated weights for policy 1, policy_version 83132 (0.0009) [2023-10-08 11:04:58,335][53852] Updated weights for policy 0, policy_version 83530 (0.0009) [2023-10-08 11:04:58,708][53852] Updated weights for policy 0, policy_version 83540 (0.0008) [2023-10-08 11:04:59,085][53852] Updated weights for policy 0, policy_version 83550 (0.0010) [2023-10-08 11:05:01,886][53885] Updated weights for policy 1, policy_version 83142 (0.0007) [2023-10-08 11:05:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170688512. Throughput: 0: 1850.1, 1: 1837.2. Samples: 42684086. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:05:02,016][52710] Avg episode reward: [(0, '34.680'), (1, '38.530')] [2023-10-08 11:05:02,251][53885] Updated weights for policy 1, policy_version 83152 (0.0009) [2023-10-08 11:05:02,616][53885] Updated weights for policy 1, policy_version 83162 (0.0007) [2023-10-08 11:05:02,746][53852] Updated weights for policy 0, policy_version 83560 (0.0007) [2023-10-08 11:05:03,113][53852] Updated weights for policy 0, policy_version 83570 (0.0008) [2023-10-08 11:05:03,484][53852] Updated weights for policy 0, policy_version 83580 (0.0008) [2023-10-08 11:05:06,173][53885] Updated weights for policy 1, policy_version 83172 (0.0008) [2023-10-08 11:05:06,549][53885] Updated weights for policy 1, policy_version 83182 (0.0010) [2023-10-08 11:05:06,909][53885] Updated weights for policy 1, policy_version 83192 (0.0009) [2023-10-08 11:05:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 170754048. Throughput: 0: 1848.8, 1: 1825.9. Samples: 42706384. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:05:07,016][52710] Avg episode reward: [(0, '34.350'), (1, '39.040')] [2023-10-08 11:05:07,236][53852] Updated weights for policy 0, policy_version 83590 (0.0007) [2023-10-08 11:05:07,618][53852] Updated weights for policy 0, policy_version 83600 (0.0009) [2023-10-08 11:05:07,992][53852] Updated weights for policy 0, policy_version 83610 (0.0008) [2023-10-08 11:05:10,704][53885] Updated weights for policy 1, policy_version 83202 (0.0009) [2023-10-08 11:05:11,079][53885] Updated weights for policy 1, policy_version 83212 (0.0007) [2023-10-08 11:05:11,446][53885] Updated weights for policy 1, policy_version 83222 (0.0007) [2023-10-08 11:05:11,520][53852] Updated weights for policy 0, policy_version 83620 (0.0008) [2023-10-08 11:05:11,807][53885] Updated weights for policy 1, policy_version 83232 (0.0008) [2023-10-08 11:05:11,884][53852] Updated weights for policy 0, policy_version 83630 (0.0009) [2023-10-08 11:05:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 170852352. Throughput: 0: 1844.8, 1: 1831.3. Samples: 42716906. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:05:12,016][52710] Avg episode reward: [(0, '31.480'), (1, '36.910')] [2023-10-08 11:05:12,251][53852] Updated weights for policy 0, policy_version 83640 (0.0008) [2023-10-08 11:05:15,380][53885] Updated weights for policy 1, policy_version 83242 (0.0009) [2023-10-08 11:05:15,748][53885] Updated weights for policy 1, policy_version 83252 (0.0007) [2023-10-08 11:05:15,936][53852] Updated weights for policy 0, policy_version 83650 (0.0007) [2023-10-08 11:05:16,121][53885] Updated weights for policy 1, policy_version 83262 (0.0008) [2023-10-08 11:05:16,307][53852] Updated weights for policy 0, policy_version 83660 (0.0007) [2023-10-08 11:05:16,688][53852] Updated weights for policy 0, policy_version 83670 (0.0011) [2023-10-08 11:05:17,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 170917888. Throughput: 0: 1843.0, 1: 1827.2. Samples: 42739278. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:05:17,016][52710] Avg episode reward: [(0, '32.010'), (1, '38.500')] [2023-10-08 11:05:17,061][53852] Updated weights for policy 0, policy_version 83680 (0.0010) [2023-10-08 11:05:19,608][53885] Updated weights for policy 1, policy_version 83272 (0.0008) [2023-10-08 11:05:19,981][53885] Updated weights for policy 1, policy_version 83282 (0.0008) [2023-10-08 11:05:20,356][53885] Updated weights for policy 1, policy_version 83292 (0.0009) [2023-10-08 11:05:20,807][53852] Updated weights for policy 0, policy_version 83690 (0.0010) [2023-10-08 11:05:21,181][53852] Updated weights for policy 0, policy_version 83700 (0.0009) [2023-10-08 11:05:21,547][53852] Updated weights for policy 0, policy_version 83710 (0.0007) [2023-10-08 11:05:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 171016192. Throughput: 0: 1822.7, 1: 1845.9. Samples: 42760300. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:05:22,016][52710] Avg episode reward: [(0, '32.950'), (1, '39.850')] [2023-10-08 11:05:24,104][53885] Updated weights for policy 1, policy_version 83302 (0.0009) [2023-10-08 11:05:24,483][53885] Updated weights for policy 1, policy_version 83312 (0.0011) [2023-10-08 11:05:24,853][53885] Updated weights for policy 1, policy_version 83322 (0.0008) [2023-10-08 11:05:25,196][53852] Updated weights for policy 0, policy_version 83720 (0.0008) [2023-10-08 11:05:25,571][53852] Updated weights for policy 0, policy_version 83730 (0.0009) [2023-10-08 11:05:25,936][53852] Updated weights for policy 0, policy_version 83740 (0.0010) [2023-10-08 11:05:27,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 171081728. Throughput: 0: 1830.9, 1: 1827.5. Samples: 42772088. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) [2023-10-08 11:05:27,016][52710] Avg episode reward: [(0, '28.970'), (1, '40.840')] [2023-10-08 11:05:28,553][53885] Updated weights for policy 1, policy_version 83332 (0.0008) [2023-10-08 11:05:28,928][53885] Updated weights for policy 1, policy_version 83342 (0.0008) [2023-10-08 11:05:29,291][53885] Updated weights for policy 1, policy_version 83352 (0.0008) [2023-10-08 11:05:29,653][53852] Updated weights for policy 0, policy_version 83750 (0.0008) [2023-10-08 11:05:30,023][53852] Updated weights for policy 0, policy_version 83760 (0.0009) [2023-10-08 11:05:30,395][53852] Updated weights for policy 0, policy_version 83770 (0.0009) [2023-10-08 11:05:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171147264. Throughput: 0: 1821.3, 1: 1834.6. Samples: 42793150. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:05:32,016][52710] Avg episode reward: [(0, '31.200'), (1, '40.530')] [2023-10-08 11:05:32,855][53885] Updated weights for policy 1, policy_version 83362 (0.0009) [2023-10-08 11:05:33,228][53885] Updated weights for policy 1, policy_version 83372 (0.0009) [2023-10-08 11:05:33,594][53885] Updated weights for policy 1, policy_version 83382 (0.0007) [2023-10-08 11:05:33,961][53885] Updated weights for policy 1, policy_version 83392 (0.0007) [2023-10-08 11:05:34,065][53852] Updated weights for policy 0, policy_version 83780 (0.0009) [2023-10-08 11:05:34,435][53852] Updated weights for policy 0, policy_version 83790 (0.0008) [2023-10-08 11:05:34,807][53852] Updated weights for policy 0, policy_version 83800 (0.0007) [2023-10-08 11:05:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 171212800. Throughput: 0: 1828.2, 1: 1845.8. Samples: 42816050. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:05:37,015][52710] Avg episode reward: [(0, '33.310'), (1, '39.030')] [2023-10-08 11:05:37,497][53885] Updated weights for policy 1, policy_version 83402 (0.0009) [2023-10-08 11:05:37,859][53885] Updated weights for policy 1, policy_version 83412 (0.0008) [2023-10-08 11:05:38,230][53885] Updated weights for policy 1, policy_version 83422 (0.0008) [2023-10-08 11:05:38,512][53852] Updated weights for policy 0, policy_version 83810 (0.0007) [2023-10-08 11:05:38,891][53852] Updated weights for policy 0, policy_version 83820 (0.0009) [2023-10-08 11:05:39,246][53852] Updated weights for policy 0, policy_version 83830 (0.0009) [2023-10-08 11:05:39,622][53852] Updated weights for policy 0, policy_version 83840 (0.0009) [2023-10-08 11:05:41,854][53885] Updated weights for policy 1, policy_version 83432 (0.0008) [2023-10-08 11:05:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 171278336. Throughput: 0: 1822.3, 1: 1847.2. Samples: 42826292. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:05:42,016][52710] Avg episode reward: [(0, '32.370'), (1, '39.040')] [2023-10-08 11:05:42,220][53885] Updated weights for policy 1, policy_version 83442 (0.0008) [2023-10-08 11:05:42,587][53885] Updated weights for policy 1, policy_version 83452 (0.0007) [2023-10-08 11:05:43,257][53852] Updated weights for policy 0, policy_version 83850 (0.0008) [2023-10-08 11:05:43,626][53852] Updated weights for policy 0, policy_version 83860 (0.0007) [2023-10-08 11:05:44,003][53852] Updated weights for policy 0, policy_version 83870 (0.0010) [2023-10-08 11:05:46,204][53885] Updated weights for policy 1, policy_version 83462 (0.0010) [2023-10-08 11:05:46,571][53885] Updated weights for policy 1, policy_version 83472 (0.0011) [2023-10-08 11:05:46,941][53885] Updated weights for policy 1, policy_version 83482 (0.0009) [2023-10-08 11:05:47,015][52710] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 171343872. Throughput: 0: 1826.1, 1: 1848.0. Samples: 42849420. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:05:47,017][52710] Avg episode reward: [(0, '32.480'), (1, '36.940')] [2023-10-08 11:05:47,598][53852] Updated weights for policy 0, policy_version 83880 (0.0008) [2023-10-08 11:05:47,971][53852] Updated weights for policy 0, policy_version 83890 (0.0010) [2023-10-08 11:05:48,348][53852] Updated weights for policy 0, policy_version 83900 (0.0007) [2023-10-08 11:05:50,649][53885] Updated weights for policy 1, policy_version 83492 (0.0009) [2023-10-08 11:05:51,010][53885] Updated weights for policy 1, policy_version 83502 (0.0009) [2023-10-08 11:05:51,380][53885] Updated weights for policy 1, policy_version 83512 (0.0009) [2023-10-08 11:05:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171442176. Throughput: 0: 1831.4, 1: 1828.2. Samples: 42871064. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:05:52,015][52710] Avg episode reward: [(0, '33.580'), (1, '39.280')] [2023-10-08 11:05:52,120][53852] Updated weights for policy 0, policy_version 83910 (0.0007) [2023-10-08 11:05:52,508][53852] Updated weights for policy 0, policy_version 83920 (0.0007) [2023-10-08 11:05:52,883][53852] Updated weights for policy 0, policy_version 83930 (0.0007) [2023-10-08 11:05:55,148][53885] Updated weights for policy 1, policy_version 83522 (0.0008) [2023-10-08 11:05:55,518][53885] Updated weights for policy 1, policy_version 83532 (0.0008) [2023-10-08 11:05:55,888][53885] Updated weights for policy 1, policy_version 83542 (0.0008) [2023-10-08 11:05:56,257][53885] Updated weights for policy 1, policy_version 83552 (0.0009) [2023-10-08 11:05:56,458][53852] Updated weights for policy 0, policy_version 83940 (0.0008) [2023-10-08 11:05:56,836][53852] Updated weights for policy 0, policy_version 83950 (0.0009) [2023-10-08 11:05:57,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171507712. Throughput: 0: 1828.9, 1: 1840.4. Samples: 42882026. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:05:57,016][52710] Avg episode reward: [(0, '31.890'), (1, '37.180')] [2023-10-08 11:05:57,200][53852] Updated weights for policy 0, policy_version 83960 (0.0009) [2023-10-08 11:05:59,969][53885] Updated weights for policy 1, policy_version 83562 (0.0007) [2023-10-08 11:06:00,334][53885] Updated weights for policy 1, policy_version 83572 (0.0007) [2023-10-08 11:06:00,695][53885] Updated weights for policy 1, policy_version 83582 (0.0008) [2023-10-08 11:06:00,774][53852] Updated weights for policy 0, policy_version 83970 (0.0009) [2023-10-08 11:06:01,138][53852] Updated weights for policy 0, policy_version 83980 (0.0007) [2023-10-08 11:06:01,510][53852] Updated weights for policy 0, policy_version 83990 (0.0008) [2023-10-08 11:06:01,881][53852] Updated weights for policy 0, policy_version 84000 (0.0008) [2023-10-08 11:06:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 171606016. Throughput: 0: 1832.9, 1: 1825.9. Samples: 42903924. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:06:02,016][52710] Avg episode reward: [(0, '32.690'), (1, '39.330')] [2023-10-08 11:06:04,353][53885] Updated weights for policy 1, policy_version 83592 (0.0007) [2023-10-08 11:06:04,723][53885] Updated weights for policy 1, policy_version 83602 (0.0007) [2023-10-08 11:06:05,096][53885] Updated weights for policy 1, policy_version 83612 (0.0007) [2023-10-08 11:06:05,492][53852] Updated weights for policy 0, policy_version 84010 (0.0009) [2023-10-08 11:06:05,862][53852] Updated weights for policy 0, policy_version 84020 (0.0009) [2023-10-08 11:06:06,241][53852] Updated weights for policy 0, policy_version 84030 (0.0008) [2023-10-08 11:06:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 171671552. Throughput: 0: 1831.7, 1: 1828.3. Samples: 42925000. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:06:07,016][52710] Avg episode reward: [(0, '32.580'), (1, '37.720')] [2023-10-08 11:06:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000084032_86048768.pth... [2023-10-08 11:06:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000083616_85622784.pth... [2023-10-08 11:06:07,057][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000082304_84279296.pth [2023-10-08 11:06:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000081920_83886080.pth [2023-10-08 11:06:08,785][53885] Updated weights for policy 1, policy_version 83622 (0.0008) [2023-10-08 11:06:09,151][53885] Updated weights for policy 1, policy_version 83632 (0.0008) [2023-10-08 11:06:09,520][53885] Updated weights for policy 1, policy_version 83642 (0.0009) [2023-10-08 11:06:09,890][53852] Updated weights for policy 0, policy_version 84040 (0.0008) [2023-10-08 11:06:10,264][53852] Updated weights for policy 0, policy_version 84050 (0.0008) [2023-10-08 11:06:10,632][53852] Updated weights for policy 0, policy_version 84060 (0.0008) [2023-10-08 11:06:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 171737088. Throughput: 0: 1836.8, 1: 1825.1. Samples: 42936876. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:06:12,016][52710] Avg episode reward: [(0, '33.490'), (1, '38.950')] [2023-10-08 11:06:13,295][53885] Updated weights for policy 1, policy_version 83652 (0.0008) [2023-10-08 11:06:13,664][53885] Updated weights for policy 1, policy_version 83662 (0.0007) [2023-10-08 11:06:14,032][53885] Updated weights for policy 1, policy_version 83672 (0.0010) [2023-10-08 11:06:14,276][53852] Updated weights for policy 0, policy_version 84070 (0.0007) [2023-10-08 11:06:14,650][53852] Updated weights for policy 0, policy_version 84080 (0.0010) [2023-10-08 11:06:15,022][53852] Updated weights for policy 0, policy_version 84090 (0.0008) [2023-10-08 11:06:17,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 171802624. Throughput: 0: 1830.0, 1: 1832.6. Samples: 42957968. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:06:17,016][52710] Avg episode reward: [(0, '31.570'), (1, '41.220')] [2023-10-08 11:06:17,611][53885] Updated weights for policy 1, policy_version 83682 (0.0008) [2023-10-08 11:06:17,979][53885] Updated weights for policy 1, policy_version 83692 (0.0008) [2023-10-08 11:06:18,343][53885] Updated weights for policy 1, policy_version 83702 (0.0010) [2023-10-08 11:06:18,713][53885] Updated weights for policy 1, policy_version 83712 (0.0008) [2023-10-08 11:06:18,754][53852] Updated weights for policy 0, policy_version 84100 (0.0007) [2023-10-08 11:06:19,130][53852] Updated weights for policy 0, policy_version 84110 (0.0008) [2023-10-08 11:06:19,503][53852] Updated weights for policy 0, policy_version 84120 (0.0007) [2023-10-08 11:06:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 171868160. Throughput: 0: 1830.9, 1: 1829.4. Samples: 42980766. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) [2023-10-08 11:06:22,016][52710] Avg episode reward: [(0, '32.190'), (1, '38.970')] [2023-10-08 11:06:22,367][53885] Updated weights for policy 1, policy_version 83722 (0.0007) [2023-10-08 11:06:22,734][53885] Updated weights for policy 1, policy_version 83732 (0.0010) [2023-10-08 11:06:23,094][53885] Updated weights for policy 1, policy_version 83742 (0.0009) [2023-10-08 11:06:23,271][53852] Updated weights for policy 0, policy_version 84130 (0.0008) [2023-10-08 11:06:23,637][53852] Updated weights for policy 0, policy_version 84140 (0.0010) [2023-10-08 11:06:24,011][53852] Updated weights for policy 0, policy_version 84150 (0.0009) [2023-10-08 11:06:24,383][53852] Updated weights for policy 0, policy_version 84160 (0.0008) [2023-10-08 11:06:26,941][53885] Updated weights for policy 1, policy_version 83752 (0.0007) [2023-10-08 11:06:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 171933696. Throughput: 0: 1828.0, 1: 1829.4. Samples: 42990876. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:06:27,016][52710] Avg episode reward: [(0, '34.180'), (1, '38.870')] [2023-10-08 11:06:27,302][53885] Updated weights for policy 1, policy_version 83762 (0.0008) [2023-10-08 11:06:27,682][53885] Updated weights for policy 1, policy_version 83772 (0.0008) [2023-10-08 11:06:28,128][53852] Updated weights for policy 0, policy_version 84170 (0.0009) [2023-10-08 11:06:28,492][53852] Updated weights for policy 0, policy_version 84180 (0.0008) [2023-10-08 11:06:28,865][53852] Updated weights for policy 0, policy_version 84190 (0.0012) [2023-10-08 11:06:31,381][53885] Updated weights for policy 1, policy_version 83782 (0.0008) [2023-10-08 11:06:31,753][53885] Updated weights for policy 1, policy_version 83792 (0.0010) [2023-10-08 11:06:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 171999232. Throughput: 0: 1836.9, 1: 1820.1. Samples: 43013982. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:06:32,015][52710] Avg episode reward: [(0, '32.450'), (1, '43.390')] [2023-10-08 11:06:32,126][53885] Updated weights for policy 1, policy_version 83802 (0.0009) [2023-10-08 11:06:32,345][53594] Saving new best policy, reward=43.390! [2023-10-08 11:06:32,389][53852] Updated weights for policy 0, policy_version 84200 (0.0008) [2023-10-08 11:06:32,758][53852] Updated weights for policy 0, policy_version 84210 (0.0007) [2023-10-08 11:06:33,138][53852] Updated weights for policy 0, policy_version 84220 (0.0007) [2023-10-08 11:06:35,587][53885] Updated weights for policy 1, policy_version 83812 (0.0010) [2023-10-08 11:06:35,958][53885] Updated weights for policy 1, policy_version 83822 (0.0010) [2023-10-08 11:06:36,335][53885] Updated weights for policy 1, policy_version 83832 (0.0010) [2023-10-08 11:06:36,839][53852] Updated weights for policy 0, policy_version 84230 (0.0007) [2023-10-08 11:06:37,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172097536. Throughput: 0: 1833.8, 1: 1821.1. Samples: 43035536. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:06:37,016][52710] Avg episode reward: [(0, '31.240'), (1, '37.580')] [2023-10-08 11:06:37,209][53852] Updated weights for policy 0, policy_version 84240 (0.0008) [2023-10-08 11:06:37,582][53852] Updated weights for policy 0, policy_version 84250 (0.0007) [2023-10-08 11:06:39,949][53885] Updated weights for policy 1, policy_version 83842 (0.0010) [2023-10-08 11:06:40,322][53885] Updated weights for policy 1, policy_version 83852 (0.0007) [2023-10-08 11:06:40,687][53885] Updated weights for policy 1, policy_version 83862 (0.0010) [2023-10-08 11:06:41,056][53885] Updated weights for policy 1, policy_version 83872 (0.0008) [2023-10-08 11:06:41,162][53852] Updated weights for policy 0, policy_version 84260 (0.0007) [2023-10-08 11:06:41,531][53852] Updated weights for policy 0, policy_version 84270 (0.0007) [2023-10-08 11:06:41,905][53852] Updated weights for policy 0, policy_version 84280 (0.0010) [2023-10-08 11:06:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172163072. Throughput: 0: 1835.2, 1: 1828.5. Samples: 43046894. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:06:42,015][52710] Avg episode reward: [(0, '33.040'), (1, '33.800')] [2023-10-08 11:06:44,777][53885] Updated weights for policy 1, policy_version 83882 (0.0009) [2023-10-08 11:06:45,142][53885] Updated weights for policy 1, policy_version 83892 (0.0008) [2023-10-08 11:06:45,459][53852] Updated weights for policy 0, policy_version 84290 (0.0007) [2023-10-08 11:06:45,505][53885] Updated weights for policy 1, policy_version 83902 (0.0007) [2023-10-08 11:06:45,824][53852] Updated weights for policy 0, policy_version 84300 (0.0008) [2023-10-08 11:06:46,209][53852] Updated weights for policy 0, policy_version 84310 (0.0009) [2023-10-08 11:06:46,571][53852] Updated weights for policy 0, policy_version 84320 (0.0007) [2023-10-08 11:06:47,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 172261376. Throughput: 0: 1830.7, 1: 1821.2. Samples: 43068258. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:06:47,016][52710] Avg episode reward: [(0, '34.020'), (1, '35.570')] [2023-10-08 11:06:49,273][53885] Updated weights for policy 1, policy_version 83912 (0.0008) [2023-10-08 11:06:49,652][53885] Updated weights for policy 1, policy_version 83922 (0.0008) [2023-10-08 11:06:50,014][53885] Updated weights for policy 1, policy_version 83932 (0.0007) [2023-10-08 11:06:50,211][53852] Updated weights for policy 0, policy_version 84330 (0.0007) [2023-10-08 11:06:50,580][53852] Updated weights for policy 0, policy_version 84340 (0.0008) [2023-10-08 11:06:50,944][53852] Updated weights for policy 0, policy_version 84350 (0.0010) [2023-10-08 11:06:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 172326912. Throughput: 0: 1839.2, 1: 1828.6. Samples: 43090050. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:06:52,016][52710] Avg episode reward: [(0, '34.360'), (1, '39.640')] [2023-10-08 11:06:53,709][53885] Updated weights for policy 1, policy_version 83942 (0.0008) [2023-10-08 11:06:54,083][53885] Updated weights for policy 1, policy_version 83952 (0.0009) [2023-10-08 11:06:54,451][53885] Updated weights for policy 1, policy_version 83962 (0.0008) [2023-10-08 11:06:54,587][53852] Updated weights for policy 0, policy_version 84360 (0.0007) [2023-10-08 11:06:54,950][53852] Updated weights for policy 0, policy_version 84370 (0.0008) [2023-10-08 11:06:55,327][53852] Updated weights for policy 0, policy_version 84380 (0.0008) [2023-10-08 11:06:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172392448. Throughput: 0: 1829.6, 1: 1823.9. Samples: 43101286. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:06:57,015][52710] Avg episode reward: [(0, '31.760'), (1, '36.360')] [2023-10-08 11:06:58,087][53885] Updated weights for policy 1, policy_version 83972 (0.0010) [2023-10-08 11:06:58,442][53885] Updated weights for policy 1, policy_version 83982 (0.0008) [2023-10-08 11:06:58,807][53885] Updated weights for policy 1, policy_version 83992 (0.0009) [2023-10-08 11:06:59,102][53852] Updated weights for policy 0, policy_version 84390 (0.0008) [2023-10-08 11:06:59,474][53852] Updated weights for policy 0, policy_version 84400 (0.0009) [2023-10-08 11:06:59,842][53852] Updated weights for policy 0, policy_version 84410 (0.0010) [2023-10-08 11:07:02,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 172457984. Throughput: 0: 1835.5, 1: 1832.0. Samples: 43123004. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:07:02,015][52710] Avg episode reward: [(0, '31.980'), (1, '33.970')] [2023-10-08 11:07:02,508][53885] Updated weights for policy 1, policy_version 84002 (0.0008) [2023-10-08 11:07:02,882][53885] Updated weights for policy 1, policy_version 84012 (0.0009) [2023-10-08 11:07:03,245][53885] Updated weights for policy 1, policy_version 84022 (0.0008) [2023-10-08 11:07:03,453][53852] Updated weights for policy 0, policy_version 84420 (0.0009) [2023-10-08 11:07:03,615][53885] Updated weights for policy 1, policy_version 84032 (0.0009) [2023-10-08 11:07:03,816][53852] Updated weights for policy 0, policy_version 84430 (0.0008) [2023-10-08 11:07:04,190][53852] Updated weights for policy 0, policy_version 84440 (0.0010) [2023-10-08 11:07:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 172523520. Throughput: 0: 1846.2, 1: 1836.7. Samples: 43146494. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:07:07,015][52710] Avg episode reward: [(0, '33.800'), (1, '35.740')] [2023-10-08 11:07:07,150][53885] Updated weights for policy 1, policy_version 84042 (0.0007) [2023-10-08 11:07:07,513][53885] Updated weights for policy 1, policy_version 84052 (0.0010) [2023-10-08 11:07:07,801][53852] Updated weights for policy 0, policy_version 84450 (0.0007) [2023-10-08 11:07:07,885][53885] Updated weights for policy 1, policy_version 84062 (0.0008) [2023-10-08 11:07:08,168][53852] Updated weights for policy 0, policy_version 84460 (0.0009) [2023-10-08 11:07:08,536][53852] Updated weights for policy 0, policy_version 84470 (0.0008) [2023-10-08 11:07:08,907][53852] Updated weights for policy 0, policy_version 84480 (0.0007) [2023-10-08 11:07:11,556][53885] Updated weights for policy 1, policy_version 84072 (0.0008) [2023-10-08 11:07:11,927][53885] Updated weights for policy 1, policy_version 84082 (0.0009) [2023-10-08 11:07:12,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 172589056. Throughput: 0: 1847.1, 1: 1835.6. Samples: 43156594. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:07:12,016][52710] Avg episode reward: [(0, '32.880'), (1, '37.170')] [2023-10-08 11:07:12,293][53885] Updated weights for policy 1, policy_version 84092 (0.0008) [2023-10-08 11:07:12,668][53852] Updated weights for policy 0, policy_version 84490 (0.0007) [2023-10-08 11:07:13,045][53852] Updated weights for policy 0, policy_version 84500 (0.0008) [2023-10-08 11:07:13,419][53852] Updated weights for policy 0, policy_version 84510 (0.0008) [2023-10-08 11:07:15,832][53885] Updated weights for policy 1, policy_version 84102 (0.0007) [2023-10-08 11:07:16,213][53885] Updated weights for policy 1, policy_version 84112 (0.0009) [2023-10-08 11:07:16,578][53885] Updated weights for policy 1, policy_version 84122 (0.0008) [2023-10-08 11:07:17,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172687360. Throughput: 0: 1840.0, 1: 1841.0. Samples: 43179626. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) [2023-10-08 11:07:17,016][52710] Avg episode reward: [(0, '29.880'), (1, '32.020')] [2023-10-08 11:07:17,030][53852] Updated weights for policy 0, policy_version 84520 (0.0007) [2023-10-08 11:07:17,398][53852] Updated weights for policy 0, policy_version 84530 (0.0008) [2023-10-08 11:07:17,779][53852] Updated weights for policy 0, policy_version 84540 (0.0009) [2023-10-08 11:07:20,119][53885] Updated weights for policy 1, policy_version 84132 (0.0007) [2023-10-08 11:07:20,493][53885] Updated weights for policy 1, policy_version 84142 (0.0009) [2023-10-08 11:07:20,867][53885] Updated weights for policy 1, policy_version 84152 (0.0009) [2023-10-08 11:07:21,463][53852] Updated weights for policy 0, policy_version 84550 (0.0008) [2023-10-08 11:07:21,848][53852] Updated weights for policy 0, policy_version 84560 (0.0007) [2023-10-08 11:07:22,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172752896. Throughput: 0: 1831.0, 1: 1841.9. Samples: 43200816. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:07:22,016][52710] Avg episode reward: [(0, '32.600'), (1, '33.440')] [2023-10-08 11:07:22,209][53852] Updated weights for policy 0, policy_version 84570 (0.0008) [2023-10-08 11:07:24,539][53885] Updated weights for policy 1, policy_version 84162 (0.0011) [2023-10-08 11:07:24,904][53885] Updated weights for policy 1, policy_version 84172 (0.0007) [2023-10-08 11:07:25,280][53885] Updated weights for policy 1, policy_version 84182 (0.0010) [2023-10-08 11:07:25,640][53885] Updated weights for policy 1, policy_version 84192 (0.0009) [2023-10-08 11:07:25,821][53852] Updated weights for policy 0, policy_version 84580 (0.0008) [2023-10-08 11:07:26,195][53852] Updated weights for policy 0, policy_version 84590 (0.0007) [2023-10-08 11:07:26,564][53852] Updated weights for policy 0, policy_version 84600 (0.0009) [2023-10-08 11:07:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 172851200. Throughput: 0: 1841.7, 1: 1839.5. Samples: 43212548. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:07:27,016][52710] Avg episode reward: [(0, '31.810'), (1, '37.310')] [2023-10-08 11:07:29,453][53885] Updated weights for policy 1, policy_version 84202 (0.0007) [2023-10-08 11:07:29,828][53885] Updated weights for policy 1, policy_version 84212 (0.0009) [2023-10-08 11:07:30,141][53852] Updated weights for policy 0, policy_version 84610 (0.0007) [2023-10-08 11:07:30,189][53885] Updated weights for policy 1, policy_version 84222 (0.0007) [2023-10-08 11:07:30,505][53852] Updated weights for policy 0, policy_version 84620 (0.0009) [2023-10-08 11:07:30,881][53852] Updated weights for policy 0, policy_version 84630 (0.0008) [2023-10-08 11:07:31,246][53852] Updated weights for policy 0, policy_version 84640 (0.0008) [2023-10-08 11:07:32,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 172916736. Throughput: 0: 1829.1, 1: 1843.3. Samples: 43233516. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:07:32,016][52710] Avg episode reward: [(0, '32.510'), (1, '38.480')] [2023-10-08 11:07:33,991][53885] Updated weights for policy 1, policy_version 84232 (0.0009) [2023-10-08 11:07:34,360][53885] Updated weights for policy 1, policy_version 84242 (0.0008) [2023-10-08 11:07:34,726][53885] Updated weights for policy 1, policy_version 84252 (0.0008) [2023-10-08 11:07:34,955][53852] Updated weights for policy 0, policy_version 84650 (0.0011) [2023-10-08 11:07:35,328][53852] Updated weights for policy 0, policy_version 84660 (0.0009) [2023-10-08 11:07:35,693][53852] Updated weights for policy 0, policy_version 84670 (0.0008) [2023-10-08 11:07:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 172982272. Throughput: 0: 1838.8, 1: 1841.3. Samples: 43255654. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:07:37,016][52710] Avg episode reward: [(0, '33.320'), (1, '36.240')] [2023-10-08 11:07:38,287][53885] Updated weights for policy 1, policy_version 84262 (0.0010) [2023-10-08 11:07:38,653][53885] Updated weights for policy 1, policy_version 84272 (0.0008) [2023-10-08 11:07:39,024][53885] Updated weights for policy 1, policy_version 84282 (0.0008) [2023-10-08 11:07:39,307][53852] Updated weights for policy 0, policy_version 84680 (0.0008) [2023-10-08 11:07:39,675][53852] Updated weights for policy 0, policy_version 84690 (0.0007) [2023-10-08 11:07:40,046][53852] Updated weights for policy 0, policy_version 84700 (0.0008) [2023-10-08 11:07:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 173047808. Throughput: 0: 1835.6, 1: 1841.9. Samples: 43266772. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:07:42,016][52710] Avg episode reward: [(0, '31.910'), (1, '35.640')] [2023-10-08 11:07:42,773][53885] Updated weights for policy 1, policy_version 84292 (0.0009) [2023-10-08 11:07:43,141][53885] Updated weights for policy 1, policy_version 84302 (0.0008) [2023-10-08 11:07:43,513][53885] Updated weights for policy 1, policy_version 84312 (0.0009) [2023-10-08 11:07:43,723][53852] Updated weights for policy 0, policy_version 84710 (0.0008) [2023-10-08 11:07:44,091][53852] Updated weights for policy 0, policy_version 84720 (0.0007) [2023-10-08 11:07:44,466][53852] Updated weights for policy 0, policy_version 84730 (0.0007) [2023-10-08 11:07:46,990][53885] Updated weights for policy 1, policy_version 84322 (0.0007) [2023-10-08 11:07:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 173113344. Throughput: 0: 1842.8, 1: 1840.7. Samples: 43288764. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:07:47,015][52710] Avg episode reward: [(0, '28.960'), (1, '40.730')] [2023-10-08 11:07:47,360][53885] Updated weights for policy 1, policy_version 84332 (0.0008) [2023-10-08 11:07:47,733][53885] Updated weights for policy 1, policy_version 84342 (0.0008) [2023-10-08 11:07:48,099][53885] Updated weights for policy 1, policy_version 84352 (0.0007) [2023-10-08 11:07:48,115][53852] Updated weights for policy 0, policy_version 84740 (0.0008) [2023-10-08 11:07:48,488][53852] Updated weights for policy 0, policy_version 84750 (0.0007) [2023-10-08 11:07:48,864][53852] Updated weights for policy 0, policy_version 84760 (0.0007) [2023-10-08 11:07:51,772][53885] Updated weights for policy 1, policy_version 84362 (0.0008) [2023-10-08 11:07:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 173178880. Throughput: 0: 1841.3, 1: 1830.7. Samples: 43311736. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:07:52,016][52710] Avg episode reward: [(0, '30.260'), (1, '38.570')] [2023-10-08 11:07:52,144][53885] Updated weights for policy 1, policy_version 84372 (0.0007) [2023-10-08 11:07:52,408][53852] Updated weights for policy 0, policy_version 84770 (0.0009) [2023-10-08 11:07:52,519][53885] Updated weights for policy 1, policy_version 84382 (0.0009) [2023-10-08 11:07:52,769][53852] Updated weights for policy 0, policy_version 84780 (0.0008) [2023-10-08 11:07:53,144][53852] Updated weights for policy 0, policy_version 84790 (0.0007) [2023-10-08 11:07:53,520][53852] Updated weights for policy 0, policy_version 84800 (0.0008) [2023-10-08 11:07:56,150][53885] Updated weights for policy 1, policy_version 84392 (0.0008) [2023-10-08 11:07:56,512][53885] Updated weights for policy 1, policy_version 84402 (0.0009) [2023-10-08 11:07:56,881][53885] Updated weights for policy 1, policy_version 84412 (0.0007) [2023-10-08 11:07:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 173244416. Throughput: 0: 1836.7, 1: 1838.8. Samples: 43321990. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:07:57,016][52710] Avg episode reward: [(0, '30.770'), (1, '36.150')] [2023-10-08 11:07:57,229][53852] Updated weights for policy 0, policy_version 84810 (0.0008) [2023-10-08 11:07:57,598][53852] Updated weights for policy 0, policy_version 84820 (0.0008) [2023-10-08 11:07:57,968][53852] Updated weights for policy 0, policy_version 84830 (0.0007) [2023-10-08 11:08:00,568][53885] Updated weights for policy 1, policy_version 84422 (0.0009) [2023-10-08 11:08:00,938][53885] Updated weights for policy 1, policy_version 84432 (0.0007) [2023-10-08 11:08:01,311][53885] Updated weights for policy 1, policy_version 84442 (0.0007) [2023-10-08 11:08:01,574][53852] Updated weights for policy 0, policy_version 84840 (0.0008) [2023-10-08 11:08:01,935][53852] Updated weights for policy 0, policy_version 84850 (0.0009) [2023-10-08 11:08:02,015][52710] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173342720. Throughput: 0: 1837.0, 1: 1826.0. Samples: 43344460. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:08:02,015][52710] Avg episode reward: [(0, '29.990'), (1, '38.170')] [2023-10-08 11:08:02,312][53852] Updated weights for policy 0, policy_version 84860 (0.0007) [2023-10-08 11:08:05,029][53885] Updated weights for policy 1, policy_version 84452 (0.0009) [2023-10-08 11:08:05,401][53885] Updated weights for policy 1, policy_version 84462 (0.0009) [2023-10-08 11:08:05,772][53885] Updated weights for policy 1, policy_version 84472 (0.0009) [2023-10-08 11:08:05,973][53852] Updated weights for policy 0, policy_version 84870 (0.0007) [2023-10-08 11:08:06,347][53852] Updated weights for policy 0, policy_version 84880 (0.0007) [2023-10-08 11:08:06,717][53852] Updated weights for policy 0, policy_version 84890 (0.0007) [2023-10-08 11:08:07,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 173441024. Throughput: 0: 1829.8, 1: 1827.9. Samples: 43365414. Policy #0 lag: (min: 27.0, avg: 35.0, max: 59.0) [2023-10-08 11:08:07,016][52710] Avg episode reward: [(0, '31.850'), (1, '39.970')] [2023-10-08 11:08:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000084480_86507520.pth... [2023-10-08 11:08:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000084896_86933504.pth... [2023-10-08 11:08:07,060][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000083168_85164032.pth [2023-10-08 11:08:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000082752_84738048.pth [2023-10-08 11:08:09,444][53885] Updated weights for policy 1, policy_version 84482 (0.0008) [2023-10-08 11:08:09,810][53885] Updated weights for policy 1, policy_version 84492 (0.0010) [2023-10-08 11:08:10,175][53885] Updated weights for policy 1, policy_version 84502 (0.0007) [2023-10-08 11:08:10,264][53852] Updated weights for policy 0, policy_version 84900 (0.0007) [2023-10-08 11:08:10,545][53885] Updated weights for policy 1, policy_version 84512 (0.0008) [2023-10-08 11:08:10,637][53852] Updated weights for policy 0, policy_version 84910 (0.0009) [2023-10-08 11:08:11,021][53852] Updated weights for policy 0, policy_version 84920 (0.0010) [2023-10-08 11:08:12,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 173506560. Throughput: 0: 1850.4, 1: 1822.0. Samples: 43377808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:12,016][52710] Avg episode reward: [(0, '34.660'), (1, '36.560')] [2023-10-08 11:08:14,404][53885] Updated weights for policy 1, policy_version 84522 (0.0011) [2023-10-08 11:08:14,772][53885] Updated weights for policy 1, policy_version 84532 (0.0007) [2023-10-08 11:08:14,796][53852] Updated weights for policy 0, policy_version 84930 (0.0010) [2023-10-08 11:08:15,137][53885] Updated weights for policy 1, policy_version 84542 (0.0008) [2023-10-08 11:08:15,170][53852] Updated weights for policy 0, policy_version 84940 (0.0009) [2023-10-08 11:08:15,530][53852] Updated weights for policy 0, policy_version 84950 (0.0008) [2023-10-08 11:08:15,900][53852] Updated weights for policy 0, policy_version 84960 (0.0008) [2023-10-08 11:08:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173572096. Throughput: 0: 1838.4, 1: 1819.6. Samples: 43398126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:17,016][52710] Avg episode reward: [(0, '32.610'), (1, '35.670')] [2023-10-08 11:08:18,871][53885] Updated weights for policy 1, policy_version 84552 (0.0007) [2023-10-08 11:08:19,244][53885] Updated weights for policy 1, policy_version 84562 (0.0007) [2023-10-08 11:08:19,306][53852] Updated weights for policy 0, policy_version 84970 (0.0008) [2023-10-08 11:08:19,624][53885] Updated weights for policy 1, policy_version 84572 (0.0010) [2023-10-08 11:08:19,678][53852] Updated weights for policy 0, policy_version 84980 (0.0008) [2023-10-08 11:08:20,061][53852] Updated weights for policy 0, policy_version 84990 (0.0008) [2023-10-08 11:08:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173637632. Throughput: 0: 1851.2, 1: 1819.4. Samples: 43420828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:22,015][52710] Avg episode reward: [(0, '31.080'), (1, '38.800')] [2023-10-08 11:08:23,348][53885] Updated weights for policy 1, policy_version 84582 (0.0008) [2023-10-08 11:08:23,706][53885] Updated weights for policy 1, policy_version 84592 (0.0008) [2023-10-08 11:08:23,714][53852] Updated weights for policy 0, policy_version 85000 (0.0009) [2023-10-08 11:08:24,073][53885] Updated weights for policy 1, policy_version 84602 (0.0007) [2023-10-08 11:08:24,077][53852] Updated weights for policy 0, policy_version 85010 (0.0008) [2023-10-08 11:08:24,446][53852] Updated weights for policy 0, policy_version 85020 (0.0009) [2023-10-08 11:08:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 173703168. Throughput: 0: 1831.5, 1: 1817.7. Samples: 43430986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:27,016][52710] Avg episode reward: [(0, '35.470'), (1, '36.300')] [2023-10-08 11:08:27,659][53885] Updated weights for policy 1, policy_version 84612 (0.0007) [2023-10-08 11:08:28,023][53885] Updated weights for policy 1, policy_version 84622 (0.0009) [2023-10-08 11:08:28,117][53852] Updated weights for policy 0, policy_version 85030 (0.0007) [2023-10-08 11:08:28,398][53885] Updated weights for policy 1, policy_version 84632 (0.0008) [2023-10-08 11:08:28,483][53852] Updated weights for policy 0, policy_version 85040 (0.0009) [2023-10-08 11:08:28,847][53852] Updated weights for policy 0, policy_version 85050 (0.0009) [2023-10-08 11:08:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 173768704. Throughput: 0: 1851.3, 1: 1821.0. Samples: 43454018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:32,015][52710] Avg episode reward: [(0, '33.550'), (1, '35.380')] [2023-10-08 11:08:32,075][53885] Updated weights for policy 1, policy_version 84642 (0.0008) [2023-10-08 11:08:32,444][53885] Updated weights for policy 1, policy_version 84652 (0.0008) [2023-10-08 11:08:32,535][53852] Updated weights for policy 0, policy_version 85060 (0.0007) [2023-10-08 11:08:32,813][53885] Updated weights for policy 1, policy_version 84662 (0.0007) [2023-10-08 11:08:32,912][53852] Updated weights for policy 0, policy_version 85070 (0.0007) [2023-10-08 11:08:33,177][53885] Updated weights for policy 1, policy_version 84672 (0.0008) [2023-10-08 11:08:33,273][53852] Updated weights for policy 0, policy_version 85080 (0.0007) [2023-10-08 11:08:36,698][53885] Updated weights for policy 1, policy_version 84682 (0.0007) [2023-10-08 11:08:36,778][53852] Updated weights for policy 0, policy_version 85090 (0.0007) [2023-10-08 11:08:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 173834240. Throughput: 0: 1847.7, 1: 1823.5. Samples: 43476938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:37,016][52710] Avg episode reward: [(0, '35.050'), (1, '37.600')] [2023-10-08 11:08:37,064][53885] Updated weights for policy 1, policy_version 84692 (0.0007) [2023-10-08 11:08:37,142][53852] Updated weights for policy 0, policy_version 85100 (0.0007) [2023-10-08 11:08:37,427][53885] Updated weights for policy 1, policy_version 84702 (0.0008) [2023-10-08 11:08:37,508][53852] Updated weights for policy 0, policy_version 85110 (0.0007) [2023-10-08 11:08:37,889][53852] Updated weights for policy 0, policy_version 85120 (0.0009) [2023-10-08 11:08:41,171][53885] Updated weights for policy 1, policy_version 84712 (0.0007) [2023-10-08 11:08:41,539][53885] Updated weights for policy 1, policy_version 84722 (0.0007) [2023-10-08 11:08:41,626][53852] Updated weights for policy 0, policy_version 85130 (0.0008) [2023-10-08 11:08:41,906][53885] Updated weights for policy 1, policy_version 84732 (0.0009) [2023-10-08 11:08:42,006][53852] Updated weights for policy 0, policy_version 85140 (0.0009) [2023-10-08 11:08:42,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173899776. Throughput: 0: 1849.9, 1: 1821.2. Samples: 43487192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:42,016][52710] Avg episode reward: [(0, '35.190'), (1, '38.350')] [2023-10-08 11:08:42,387][53852] Updated weights for policy 0, policy_version 85150 (0.0010) [2023-10-08 11:08:45,575][53885] Updated weights for policy 1, policy_version 84742 (0.0008) [2023-10-08 11:08:45,939][53885] Updated weights for policy 1, policy_version 84752 (0.0008) [2023-10-08 11:08:46,092][53852] Updated weights for policy 0, policy_version 85160 (0.0009) [2023-10-08 11:08:46,318][53885] Updated weights for policy 1, policy_version 84762 (0.0009) [2023-10-08 11:08:46,465][53852] Updated weights for policy 0, policy_version 85170 (0.0007) [2023-10-08 11:08:46,831][53852] Updated weights for policy 0, policy_version 85180 (0.0009) [2023-10-08 11:08:47,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 174030848. Throughput: 0: 1846.4, 1: 1826.8. Samples: 43509756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:47,016][52710] Avg episode reward: [(0, '33.790'), (1, '40.940')] [2023-10-08 11:08:49,805][53885] Updated weights for policy 1, policy_version 84772 (0.0007) [2023-10-08 11:08:50,169][53885] Updated weights for policy 1, policy_version 84782 (0.0007) [2023-10-08 11:08:50,534][53852] Updated weights for policy 0, policy_version 85190 (0.0009) [2023-10-08 11:08:50,536][53885] Updated weights for policy 1, policy_version 84792 (0.0007) [2023-10-08 11:08:50,908][53852] Updated weights for policy 0, policy_version 85200 (0.0008) [2023-10-08 11:08:51,278][53852] Updated weights for policy 0, policy_version 85210 (0.0007) [2023-10-08 11:08:52,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 174096384. Throughput: 0: 1830.6, 1: 1831.3. Samples: 43530202. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:52,016][52710] Avg episode reward: [(0, '33.890'), (1, '37.050')] [2023-10-08 11:08:54,301][53885] Updated weights for policy 1, policy_version 84802 (0.0008) [2023-10-08 11:08:54,668][53885] Updated weights for policy 1, policy_version 84812 (0.0009) [2023-10-08 11:08:54,721][53852] Updated weights for policy 0, policy_version 85220 (0.0007) [2023-10-08 11:08:55,037][53885] Updated weights for policy 1, policy_version 84822 (0.0008) [2023-10-08 11:08:55,086][53852] Updated weights for policy 0, policy_version 85230 (0.0007) [2023-10-08 11:08:55,399][53885] Updated weights for policy 1, policy_version 84832 (0.0008) [2023-10-08 11:08:55,447][53852] Updated weights for policy 0, policy_version 85240 (0.0007) [2023-10-08 11:08:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 174161920. Throughput: 0: 1841.4, 1: 1827.0. Samples: 43542888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:08:57,016][52710] Avg episode reward: [(0, '32.730'), (1, '37.620')] [2023-10-08 11:08:59,061][53885] Updated weights for policy 1, policy_version 84842 (0.0007) [2023-10-08 11:08:59,194][53852] Updated weights for policy 0, policy_version 85250 (0.0008) [2023-10-08 11:08:59,439][53885] Updated weights for policy 1, policy_version 84852 (0.0008) [2023-10-08 11:08:59,570][53852] Updated weights for policy 0, policy_version 85260 (0.0008) [2023-10-08 11:08:59,809][53885] Updated weights for policy 1, policy_version 84862 (0.0010) [2023-10-08 11:08:59,946][53852] Updated weights for policy 0, policy_version 85270 (0.0009) [2023-10-08 11:09:00,323][53852] Updated weights for policy 0, policy_version 85280 (0.0007) [2023-10-08 11:09:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174227456. Throughput: 0: 1831.1, 1: 1834.9. Samples: 43563096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:09:02,015][52710] Avg episode reward: [(0, '32.350'), (1, '37.150')] [2023-10-08 11:09:03,494][53885] Updated weights for policy 1, policy_version 84872 (0.0007) [2023-10-08 11:09:03,868][53885] Updated weights for policy 1, policy_version 84882 (0.0011) [2023-10-08 11:09:04,067][53852] Updated weights for policy 0, policy_version 85290 (0.0008) [2023-10-08 11:09:04,249][53885] Updated weights for policy 1, policy_version 84892 (0.0008) [2023-10-08 11:09:04,435][53852] Updated weights for policy 0, policy_version 85300 (0.0010) [2023-10-08 11:09:04,800][53852] Updated weights for policy 0, policy_version 85310 (0.0009) [2023-10-08 11:09:07,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 174292992. Throughput: 0: 1830.8, 1: 1837.1. Samples: 43585880. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:07,016][52710] Avg episode reward: [(0, '33.410'), (1, '34.070')] [2023-10-08 11:09:07,869][53885] Updated weights for policy 1, policy_version 84902 (0.0008) [2023-10-08 11:09:08,248][53885] Updated weights for policy 1, policy_version 84912 (0.0007) [2023-10-08 11:09:08,367][53852] Updated weights for policy 0, policy_version 85320 (0.0007) [2023-10-08 11:09:08,623][53885] Updated weights for policy 1, policy_version 84922 (0.0008) [2023-10-08 11:09:08,740][53852] Updated weights for policy 0, policy_version 85330 (0.0007) [2023-10-08 11:09:09,106][53852] Updated weights for policy 0, policy_version 85340 (0.0007) [2023-10-08 11:09:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 174358528. Throughput: 0: 1829.7, 1: 1836.5. Samples: 43595962. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:12,015][52710] Avg episode reward: [(0, '31.720'), (1, '29.360')] [2023-10-08 11:09:12,335][53885] Updated weights for policy 1, policy_version 84932 (0.0009) [2023-10-08 11:09:12,699][53885] Updated weights for policy 1, policy_version 84942 (0.0007) [2023-10-08 11:09:12,775][53852] Updated weights for policy 0, policy_version 85350 (0.0008) [2023-10-08 11:09:13,064][53885] Updated weights for policy 1, policy_version 84952 (0.0008) [2023-10-08 11:09:13,147][53852] Updated weights for policy 0, policy_version 85360 (0.0008) [2023-10-08 11:09:13,522][53852] Updated weights for policy 0, policy_version 85370 (0.0007) [2023-10-08 11:09:16,742][53885] Updated weights for policy 1, policy_version 84962 (0.0008) [2023-10-08 11:09:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 174424064. Throughput: 0: 1827.3, 1: 1833.3. Samples: 43618748. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:17,016][52710] Avg episode reward: [(0, '34.080'), (1, '35.680')] [2023-10-08 11:09:17,105][53885] Updated weights for policy 1, policy_version 84972 (0.0008) [2023-10-08 11:09:17,243][53852] Updated weights for policy 0, policy_version 85380 (0.0008) [2023-10-08 11:09:17,469][53885] Updated weights for policy 1, policy_version 84982 (0.0008) [2023-10-08 11:09:17,610][53852] Updated weights for policy 0, policy_version 85390 (0.0008) [2023-10-08 11:09:17,839][53885] Updated weights for policy 1, policy_version 84992 (0.0007) [2023-10-08 11:09:17,976][53852] Updated weights for policy 0, policy_version 85400 (0.0008) [2023-10-08 11:09:21,519][53885] Updated weights for policy 1, policy_version 85002 (0.0010) [2023-10-08 11:09:21,761][53852] Updated weights for policy 0, policy_version 85410 (0.0008) [2023-10-08 11:09:21,884][53885] Updated weights for policy 1, policy_version 85012 (0.0008) [2023-10-08 11:09:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 174489600. Throughput: 0: 1823.5, 1: 1823.2. Samples: 43641036. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:22,015][52710] Avg episode reward: [(0, '32.430'), (1, '43.280')] [2023-10-08 11:09:22,133][53852] Updated weights for policy 0, policy_version 85420 (0.0007) [2023-10-08 11:09:22,255][53885] Updated weights for policy 1, policy_version 85022 (0.0008) [2023-10-08 11:09:22,508][53852] Updated weights for policy 0, policy_version 85430 (0.0007) [2023-10-08 11:09:22,862][53852] Updated weights for policy 0, policy_version 85440 (0.0007) [2023-10-08 11:09:25,859][53885] Updated weights for policy 1, policy_version 85032 (0.0009) [2023-10-08 11:09:26,223][53885] Updated weights for policy 1, policy_version 85042 (0.0008) [2023-10-08 11:09:26,411][53852] Updated weights for policy 0, policy_version 85450 (0.0007) [2023-10-08 11:09:26,587][53885] Updated weights for policy 1, policy_version 85052 (0.0010) [2023-10-08 11:09:26,787][53852] Updated weights for policy 0, policy_version 85460 (0.0008) [2023-10-08 11:09:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174587904. Throughput: 0: 1826.5, 1: 1831.2. Samples: 43651792. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:27,016][52710] Avg episode reward: [(0, '30.100'), (1, '37.310')] [2023-10-08 11:09:27,169][53852] Updated weights for policy 0, policy_version 85470 (0.0008) [2023-10-08 11:09:30,239][53885] Updated weights for policy 1, policy_version 85062 (0.0008) [2023-10-08 11:09:30,604][53885] Updated weights for policy 1, policy_version 85072 (0.0010) [2023-10-08 11:09:30,855][53852] Updated weights for policy 0, policy_version 85480 (0.0009) [2023-10-08 11:09:30,976][53885] Updated weights for policy 1, policy_version 85082 (0.0009) [2023-10-08 11:09:31,224][53852] Updated weights for policy 0, policy_version 85490 (0.0007) [2023-10-08 11:09:31,592][53852] Updated weights for policy 0, policy_version 85500 (0.0010) [2023-10-08 11:09:32,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 174686208. Throughput: 0: 1829.9, 1: 1820.4. Samples: 43674018. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:32,016][52710] Avg episode reward: [(0, '31.400'), (1, '36.400')] [2023-10-08 11:09:34,767][53885] Updated weights for policy 1, policy_version 85092 (0.0008) [2023-10-08 11:09:35,149][53885] Updated weights for policy 1, policy_version 85102 (0.0010) [2023-10-08 11:09:35,221][53852] Updated weights for policy 0, policy_version 85510 (0.0010) [2023-10-08 11:09:35,516][53885] Updated weights for policy 1, policy_version 85112 (0.0009) [2023-10-08 11:09:35,583][53852] Updated weights for policy 0, policy_version 85520 (0.0008) [2023-10-08 11:09:35,948][53852] Updated weights for policy 0, policy_version 85530 (0.0008) [2023-10-08 11:09:37,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 174751744. Throughput: 0: 1828.7, 1: 1825.2. Samples: 43694628. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:37,016][52710] Avg episode reward: [(0, '32.300'), (1, '37.880')] [2023-10-08 11:09:39,176][53885] Updated weights for policy 1, policy_version 85122 (0.0010) [2023-10-08 11:09:39,541][53885] Updated weights for policy 1, policy_version 85132 (0.0007) [2023-10-08 11:09:39,653][53852] Updated weights for policy 0, policy_version 85540 (0.0008) [2023-10-08 11:09:39,916][53885] Updated weights for policy 1, policy_version 85142 (0.0008) [2023-10-08 11:09:40,022][53852] Updated weights for policy 0, policy_version 85550 (0.0007) [2023-10-08 11:09:40,271][53885] Updated weights for policy 1, policy_version 85152 (0.0009) [2023-10-08 11:09:40,386][53852] Updated weights for policy 0, policy_version 85560 (0.0007) [2023-10-08 11:09:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 174817280. Throughput: 0: 1823.6, 1: 1826.8. Samples: 43707154. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:42,015][52710] Avg episode reward: [(0, '31.190'), (1, '34.150')] [2023-10-08 11:09:44,037][53885] Updated weights for policy 1, policy_version 85162 (0.0009) [2023-10-08 11:09:44,137][53852] Updated weights for policy 0, policy_version 85570 (0.0009) [2023-10-08 11:09:44,398][53885] Updated weights for policy 1, policy_version 85172 (0.0008) [2023-10-08 11:09:44,520][53852] Updated weights for policy 0, policy_version 85580 (0.0007) [2023-10-08 11:09:44,758][53885] Updated weights for policy 1, policy_version 85182 (0.0008) [2023-10-08 11:09:44,897][53852] Updated weights for policy 0, policy_version 85590 (0.0007) [2023-10-08 11:09:45,265][53852] Updated weights for policy 0, policy_version 85600 (0.0010) [2023-10-08 11:09:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 174882816. Throughput: 0: 1824.0, 1: 1824.3. Samples: 43727270. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:47,016][52710] Avg episode reward: [(0, '34.580'), (1, '32.990')] [2023-10-08 11:09:48,590][53885] Updated weights for policy 1, policy_version 85192 (0.0008) [2023-10-08 11:09:48,755][53852] Updated weights for policy 0, policy_version 85610 (0.0009) [2023-10-08 11:09:48,967][53885] Updated weights for policy 1, policy_version 85202 (0.0009) [2023-10-08 11:09:49,117][53852] Updated weights for policy 0, policy_version 85620 (0.0007) [2023-10-08 11:09:49,324][53885] Updated weights for policy 1, policy_version 85212 (0.0007) [2023-10-08 11:09:49,484][53852] Updated weights for policy 0, policy_version 85630 (0.0008) [2023-10-08 11:09:52,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 174948352. Throughput: 0: 1834.7, 1: 1815.6. Samples: 43750142. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:52,016][52710] Avg episode reward: [(0, '32.630'), (1, '35.400')] [2023-10-08 11:09:52,949][53885] Updated weights for policy 1, policy_version 85222 (0.0008) [2023-10-08 11:09:53,174][53852] Updated weights for policy 0, policy_version 85640 (0.0007) [2023-10-08 11:09:53,317][53885] Updated weights for policy 1, policy_version 85232 (0.0007) [2023-10-08 11:09:53,530][53852] Updated weights for policy 0, policy_version 85650 (0.0009) [2023-10-08 11:09:53,683][53885] Updated weights for policy 1, policy_version 85242 (0.0007) [2023-10-08 11:09:53,903][53852] Updated weights for policy 0, policy_version 85660 (0.0008) [2023-10-08 11:09:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 175013888. Throughput: 0: 1834.1, 1: 1813.4. Samples: 43760100. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:09:57,016][52710] Avg episode reward: [(0, '32.250'), (1, '38.290')] [2023-10-08 11:09:57,503][53885] Updated weights for policy 1, policy_version 85252 (0.0009) [2023-10-08 11:09:57,556][53852] Updated weights for policy 0, policy_version 85670 (0.0007) [2023-10-08 11:09:57,881][53885] Updated weights for policy 1, policy_version 85262 (0.0007) [2023-10-08 11:09:57,931][53852] Updated weights for policy 0, policy_version 85680 (0.0008) [2023-10-08 11:09:58,235][53885] Updated weights for policy 1, policy_version 85272 (0.0009) [2023-10-08 11:09:58,294][53852] Updated weights for policy 0, policy_version 85690 (0.0008) [2023-10-08 11:10:01,910][53852] Updated weights for policy 0, policy_version 85700 (0.0007) [2023-10-08 11:10:01,935][53885] Updated weights for policy 1, policy_version 85282 (0.0008) [2023-10-08 11:10:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 175079424. Throughput: 0: 1842.3, 1: 1809.5. Samples: 43783080. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:02,015][52710] Avg episode reward: [(0, '33.760'), (1, '34.310')] [2023-10-08 11:10:02,271][53852] Updated weights for policy 0, policy_version 85710 (0.0009) [2023-10-08 11:10:02,301][53885] Updated weights for policy 1, policy_version 85292 (0.0007) [2023-10-08 11:10:02,645][53852] Updated weights for policy 0, policy_version 85720 (0.0008) [2023-10-08 11:10:02,667][53885] Updated weights for policy 1, policy_version 85302 (0.0009) [2023-10-08 11:10:03,040][53885] Updated weights for policy 1, policy_version 85312 (0.0008) [2023-10-08 11:10:06,290][53852] Updated weights for policy 0, policy_version 85730 (0.0009) [2023-10-08 11:10:06,664][53852] Updated weights for policy 0, policy_version 85740 (0.0009) [2023-10-08 11:10:06,714][53885] Updated weights for policy 1, policy_version 85322 (0.0007) [2023-10-08 11:10:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 175144960. Throughput: 0: 1838.8, 1: 1817.5. Samples: 43805570. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:07,016][52710] Avg episode reward: [(0, '33.170'), (1, '35.090')] [2023-10-08 11:10:07,028][53852] Updated weights for policy 0, policy_version 85750 (0.0007) [2023-10-08 11:10:07,080][53885] Updated weights for policy 1, policy_version 85332 (0.0010) [2023-10-08 11:10:07,398][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000085760_87818240.pth... [2023-10-08 11:10:07,400][53852] Updated weights for policy 0, policy_version 85760 (0.0008) [2023-10-08 11:10:07,427][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000084032_86048768.pth [2023-10-08 11:10:07,431][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000085760_87818240.pth [2023-10-08 11:10:07,444][53885] Updated weights for policy 1, policy_version 85342 (0.0008) [2023-10-08 11:10:07,518][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000085344_87392256.pth... [2023-10-08 11:10:07,555][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000083616_85622784.pth [2023-10-08 11:10:07,560][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000085344_87392256.pth [2023-10-08 11:10:11,014][53852] Updated weights for policy 0, policy_version 85770 (0.0008) [2023-10-08 11:10:11,254][53885] Updated weights for policy 1, policy_version 85352 (0.0007) [2023-10-08 11:10:11,381][53852] Updated weights for policy 0, policy_version 85780 (0.0007) [2023-10-08 11:10:11,614][53885] Updated weights for policy 1, policy_version 85362 (0.0010) [2023-10-08 11:10:11,752][53852] Updated weights for policy 0, policy_version 85790 (0.0008) [2023-10-08 11:10:11,981][53885] Updated weights for policy 1, policy_version 85372 (0.0010) [2023-10-08 11:10:12,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 175243264. Throughput: 0: 1847.2, 1: 1808.4. Samples: 43816290. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:12,016][52710] Avg episode reward: [(0, '32.670'), (1, '40.830')] [2023-10-08 11:10:15,613][53852] Updated weights for policy 0, policy_version 85800 (0.0009) [2023-10-08 11:10:15,810][53885] Updated weights for policy 1, policy_version 85382 (0.0008) [2023-10-08 11:10:15,970][53852] Updated weights for policy 0, policy_version 85810 (0.0007) [2023-10-08 11:10:16,185][53885] Updated weights for policy 1, policy_version 85392 (0.0007) [2023-10-08 11:10:16,344][53852] Updated weights for policy 0, policy_version 85820 (0.0009) [2023-10-08 11:10:16,553][53885] Updated weights for policy 1, policy_version 85402 (0.0007) [2023-10-08 11:10:17,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 175341568. Throughput: 0: 1836.4, 1: 1819.7. Samples: 43838542. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:17,016][52710] Avg episode reward: [(0, '32.190'), (1, '37.170')] [2023-10-08 11:10:19,980][53852] Updated weights for policy 0, policy_version 85830 (0.0010) [2023-10-08 11:10:20,247][53885] Updated weights for policy 1, policy_version 85412 (0.0008) [2023-10-08 11:10:20,353][53852] Updated weights for policy 0, policy_version 85840 (0.0008) [2023-10-08 11:10:20,620][53885] Updated weights for policy 1, policy_version 85422 (0.0008) [2023-10-08 11:10:20,718][53852] Updated weights for policy 0, policy_version 85850 (0.0007) [2023-10-08 11:10:20,989][53885] Updated weights for policy 1, policy_version 85432 (0.0007) [2023-10-08 11:10:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 175407104. Throughput: 0: 1846.9, 1: 1803.9. Samples: 43858916. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:22,016][52710] Avg episode reward: [(0, '32.620'), (1, '38.770')] [2023-10-08 11:10:24,350][53852] Updated weights for policy 0, policy_version 85860 (0.0009) [2023-10-08 11:10:24,488][53885] Updated weights for policy 1, policy_version 85442 (0.0007) [2023-10-08 11:10:24,716][53852] Updated weights for policy 0, policy_version 85870 (0.0007) [2023-10-08 11:10:24,854][53885] Updated weights for policy 1, policy_version 85452 (0.0008) [2023-10-08 11:10:25,092][53852] Updated weights for policy 0, policy_version 85880 (0.0009) [2023-10-08 11:10:25,215][53885] Updated weights for policy 1, policy_version 85462 (0.0008) [2023-10-08 11:10:25,578][53885] Updated weights for policy 1, policy_version 85472 (0.0009) [2023-10-08 11:10:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 175472640. Throughput: 0: 1836.4, 1: 1809.2. Samples: 43871206. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:27,016][52710] Avg episode reward: [(0, '30.980'), (1, '38.180')] [2023-10-08 11:10:28,735][53852] Updated weights for policy 0, policy_version 85890 (0.0008) [2023-10-08 11:10:29,102][53852] Updated weights for policy 0, policy_version 85900 (0.0009) [2023-10-08 11:10:29,210][53885] Updated weights for policy 1, policy_version 85482 (0.0008) [2023-10-08 11:10:29,473][53852] Updated weights for policy 0, policy_version 85910 (0.0007) [2023-10-08 11:10:29,572][53885] Updated weights for policy 1, policy_version 85492 (0.0007) [2023-10-08 11:10:29,837][53852] Updated weights for policy 0, policy_version 85920 (0.0007) [2023-10-08 11:10:29,931][53885] Updated weights for policy 1, policy_version 85502 (0.0007) [2023-10-08 11:10:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 175538176. Throughput: 0: 1847.0, 1: 1802.8. Samples: 43891514. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:32,016][52710] Avg episode reward: [(0, '34.210'), (1, '38.900')] [2023-10-08 11:10:33,577][53852] Updated weights for policy 0, policy_version 85930 (0.0007) [2023-10-08 11:10:33,592][53885] Updated weights for policy 1, policy_version 85512 (0.0008) [2023-10-08 11:10:33,947][53852] Updated weights for policy 0, policy_version 85940 (0.0007) [2023-10-08 11:10:33,967][53885] Updated weights for policy 1, policy_version 85522 (0.0007) [2023-10-08 11:10:34,317][53852] Updated weights for policy 0, policy_version 85950 (0.0008) [2023-10-08 11:10:34,334][53885] Updated weights for policy 1, policy_version 85532 (0.0007) [2023-10-08 11:10:37,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 175603712. Throughput: 0: 1839.1, 1: 1813.9. Samples: 43914524. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:37,016][52710] Avg episode reward: [(0, '33.980'), (1, '35.760')] [2023-10-08 11:10:37,993][53852] Updated weights for policy 0, policy_version 85960 (0.0008) [2023-10-08 11:10:38,090][53885] Updated weights for policy 1, policy_version 85542 (0.0007) [2023-10-08 11:10:38,360][53852] Updated weights for policy 0, policy_version 85970 (0.0007) [2023-10-08 11:10:38,465][53885] Updated weights for policy 1, policy_version 85552 (0.0008) [2023-10-08 11:10:38,731][53852] Updated weights for policy 0, policy_version 85980 (0.0008) [2023-10-08 11:10:38,832][53885] Updated weights for policy 1, policy_version 85562 (0.0007) [2023-10-08 11:10:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 175669248. Throughput: 0: 1836.2, 1: 1813.6. Samples: 43924344. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:42,016][52710] Avg episode reward: [(0, '31.510'), (1, '36.350')] [2023-10-08 11:10:42,474][53852] Updated weights for policy 0, policy_version 85990 (0.0009) [2023-10-08 11:10:42,675][53885] Updated weights for policy 1, policy_version 85572 (0.0008) [2023-10-08 11:10:42,840][53852] Updated weights for policy 0, policy_version 86000 (0.0007) [2023-10-08 11:10:43,041][53885] Updated weights for policy 1, policy_version 85582 (0.0009) [2023-10-08 11:10:43,215][53852] Updated weights for policy 0, policy_version 86010 (0.0007) [2023-10-08 11:10:43,413][53885] Updated weights for policy 1, policy_version 85592 (0.0007) [2023-10-08 11:10:46,854][53852] Updated weights for policy 0, policy_version 86020 (0.0007) [2023-10-08 11:10:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175734784. Throughput: 0: 1829.3, 1: 1807.6. Samples: 43946744. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:47,016][52710] Avg episode reward: [(0, '32.090'), (1, '39.800')] [2023-10-08 11:10:47,222][53885] Updated weights for policy 1, policy_version 85602 (0.0010) [2023-10-08 11:10:47,236][53852] Updated weights for policy 0, policy_version 86030 (0.0008) [2023-10-08 11:10:47,591][53885] Updated weights for policy 1, policy_version 85612 (0.0007) [2023-10-08 11:10:47,608][53852] Updated weights for policy 0, policy_version 86040 (0.0009) [2023-10-08 11:10:47,961][53885] Updated weights for policy 1, policy_version 85622 (0.0008) [2023-10-08 11:10:48,320][53885] Updated weights for policy 1, policy_version 85632 (0.0009) [2023-10-08 11:10:51,193][53852] Updated weights for policy 0, policy_version 86050 (0.0007) [2023-10-08 11:10:51,556][53852] Updated weights for policy 0, policy_version 86060 (0.0008) [2023-10-08 11:10:51,941][53852] Updated weights for policy 0, policy_version 86070 (0.0009) [2023-10-08 11:10:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175800320. Throughput: 0: 1818.5, 1: 1808.5. Samples: 43968784. Policy #0 lag: (min: 31.0, avg: 37.9, max: 63.0) [2023-10-08 11:10:52,015][52710] Avg episode reward: [(0, '34.430'), (1, '40.180')] [2023-10-08 11:10:52,134][53885] Updated weights for policy 1, policy_version 85642 (0.0008) [2023-10-08 11:10:52,308][53852] Updated weights for policy 0, policy_version 86080 (0.0009) [2023-10-08 11:10:52,494][53885] Updated weights for policy 1, policy_version 85652 (0.0007) [2023-10-08 11:10:52,871][53885] Updated weights for policy 1, policy_version 85662 (0.0008) [2023-10-08 11:10:55,806][53852] Updated weights for policy 0, policy_version 86090 (0.0010) [2023-10-08 11:10:56,173][53852] Updated weights for policy 0, policy_version 86100 (0.0007) [2023-10-08 11:10:56,475][53885] Updated weights for policy 1, policy_version 85672 (0.0009) [2023-10-08 11:10:56,549][53852] Updated weights for policy 0, policy_version 86110 (0.0009) [2023-10-08 11:10:56,846][53885] Updated weights for policy 1, policy_version 85682 (0.0008) [2023-10-08 11:10:57,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175898624. Throughput: 0: 1824.0, 1: 1802.0. Samples: 43979460. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:10:57,016][52710] Avg episode reward: [(0, '33.130'), (1, '35.650')] [2023-10-08 11:10:57,223][53885] Updated weights for policy 1, policy_version 85692 (0.0009) [2023-10-08 11:11:00,151][53852] Updated weights for policy 0, policy_version 86120 (0.0010) [2023-10-08 11:11:00,524][53852] Updated weights for policy 0, policy_version 86130 (0.0010) [2023-10-08 11:11:00,889][53885] Updated weights for policy 1, policy_version 85702 (0.0008) [2023-10-08 11:11:00,900][53852] Updated weights for policy 0, policy_version 86140 (0.0008) [2023-10-08 11:11:01,248][53885] Updated weights for policy 1, policy_version 85712 (0.0008) [2023-10-08 11:11:01,624][53885] Updated weights for policy 1, policy_version 85722 (0.0008) [2023-10-08 11:11:02,015][52710] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 175996928. Throughput: 0: 1817.1, 1: 1805.9. Samples: 44001576. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:02,015][52710] Avg episode reward: [(0, '31.060'), (1, '38.920')] [2023-10-08 11:11:04,738][53852] Updated weights for policy 0, policy_version 86150 (0.0008) [2023-10-08 11:11:05,105][53852] Updated weights for policy 0, policy_version 86160 (0.0007) [2023-10-08 11:11:05,154][53885] Updated weights for policy 1, policy_version 85732 (0.0009) [2023-10-08 11:11:05,471][53852] Updated weights for policy 0, policy_version 86170 (0.0007) [2023-10-08 11:11:05,522][53885] Updated weights for policy 1, policy_version 85742 (0.0007) [2023-10-08 11:11:05,886][53885] Updated weights for policy 1, policy_version 85752 (0.0008) [2023-10-08 11:11:07,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 176062464. Throughput: 0: 1821.3, 1: 1809.3. Samples: 44022294. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:07,016][52710] Avg episode reward: [(0, '35.140'), (1, '39.080')] [2023-10-08 11:11:09,162][53852] Updated weights for policy 0, policy_version 86180 (0.0008) [2023-10-08 11:11:09,526][53852] Updated weights for policy 0, policy_version 86190 (0.0009) [2023-10-08 11:11:09,685][53885] Updated weights for policy 1, policy_version 85762 (0.0008) [2023-10-08 11:11:09,900][53852] Updated weights for policy 0, policy_version 86200 (0.0009) [2023-10-08 11:11:10,048][53885] Updated weights for policy 1, policy_version 85772 (0.0009) [2023-10-08 11:11:10,399][53885] Updated weights for policy 1, policy_version 85782 (0.0009) [2023-10-08 11:11:10,765][53885] Updated weights for policy 1, policy_version 85792 (0.0008) [2023-10-08 11:11:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176128000. Throughput: 0: 1815.7, 1: 1812.8. Samples: 44034488. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:12,016][52710] Avg episode reward: [(0, '35.760'), (1, '36.780')] [2023-10-08 11:11:13,603][53852] Updated weights for policy 0, policy_version 86210 (0.0008) [2023-10-08 11:11:13,971][53852] Updated weights for policy 0, policy_version 86220 (0.0009) [2023-10-08 11:11:14,339][53852] Updated weights for policy 0, policy_version 86230 (0.0008) [2023-10-08 11:11:14,499][53885] Updated weights for policy 1, policy_version 85802 (0.0007) [2023-10-08 11:11:14,716][53852] Updated weights for policy 0, policy_version 86240 (0.0009) [2023-10-08 11:11:14,873][53885] Updated weights for policy 1, policy_version 85812 (0.0008) [2023-10-08 11:11:15,239][53885] Updated weights for policy 1, policy_version 85822 (0.0007) [2023-10-08 11:11:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 176193536. Throughput: 0: 1822.1, 1: 1813.4. Samples: 44055110. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:17,016][52710] Avg episode reward: [(0, '33.440'), (1, '35.850')] [2023-10-08 11:11:18,353][53852] Updated weights for policy 0, policy_version 86250 (0.0007) [2023-10-08 11:11:18,712][53852] Updated weights for policy 0, policy_version 86260 (0.0008) [2023-10-08 11:11:18,715][53885] Updated weights for policy 1, policy_version 85832 (0.0008) [2023-10-08 11:11:19,077][53885] Updated weights for policy 1, policy_version 85842 (0.0007) [2023-10-08 11:11:19,078][53852] Updated weights for policy 0, policy_version 86270 (0.0007) [2023-10-08 11:11:19,452][53885] Updated weights for policy 1, policy_version 85852 (0.0008) [2023-10-08 11:11:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 176259072. Throughput: 0: 1824.4, 1: 1816.3. Samples: 44078352. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:22,016][52710] Avg episode reward: [(0, '36.420'), (1, '36.600')] [2023-10-08 11:11:22,761][53852] Updated weights for policy 0, policy_version 86280 (0.0008) [2023-10-08 11:11:23,133][53852] Updated weights for policy 0, policy_version 86290 (0.0007) [2023-10-08 11:11:23,136][53885] Updated weights for policy 1, policy_version 85862 (0.0009) [2023-10-08 11:11:23,493][53852] Updated weights for policy 0, policy_version 86300 (0.0008) [2023-10-08 11:11:23,517][53885] Updated weights for policy 1, policy_version 85872 (0.0008) [2023-10-08 11:11:23,888][53885] Updated weights for policy 1, policy_version 85882 (0.0008) [2023-10-08 11:11:26,994][53852] Updated weights for policy 0, policy_version 86310 (0.0007) [2023-10-08 11:11:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 176324608. Throughput: 0: 1823.2, 1: 1816.6. Samples: 44088134. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:27,016][52710] Avg episode reward: [(0, '33.670'), (1, '32.250')] [2023-10-08 11:11:27,361][53852] Updated weights for policy 0, policy_version 86320 (0.0010) [2023-10-08 11:11:27,603][53885] Updated weights for policy 1, policy_version 85892 (0.0007) [2023-10-08 11:11:27,725][53852] Updated weights for policy 0, policy_version 86330 (0.0007) [2023-10-08 11:11:27,973][53885] Updated weights for policy 1, policy_version 85902 (0.0008) [2023-10-08 11:11:28,335][53885] Updated weights for policy 1, policy_version 85912 (0.0008) [2023-10-08 11:11:31,399][53852] Updated weights for policy 0, policy_version 86340 (0.0007) [2023-10-08 11:11:31,771][53852] Updated weights for policy 0, policy_version 86350 (0.0009) [2023-10-08 11:11:31,915][53885] Updated weights for policy 1, policy_version 85922 (0.0009) [2023-10-08 11:11:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176390144. Throughput: 0: 1832.6, 1: 1828.9. Samples: 44111508. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:32,016][52710] Avg episode reward: [(0, '33.280'), (1, '34.500')] [2023-10-08 11:11:32,143][53852] Updated weights for policy 0, policy_version 86360 (0.0008) [2023-10-08 11:11:32,283][53885] Updated weights for policy 1, policy_version 85932 (0.0008) [2023-10-08 11:11:32,662][53885] Updated weights for policy 1, policy_version 85942 (0.0010) [2023-10-08 11:11:33,025][53885] Updated weights for policy 1, policy_version 85952 (0.0010) [2023-10-08 11:11:35,775][53852] Updated weights for policy 0, policy_version 86370 (0.0008) [2023-10-08 11:11:36,150][53852] Updated weights for policy 0, policy_version 86380 (0.0008) [2023-10-08 11:11:36,524][53852] Updated weights for policy 0, policy_version 86390 (0.0007) [2023-10-08 11:11:36,833][53885] Updated weights for policy 1, policy_version 85962 (0.0007) [2023-10-08 11:11:36,898][53852] Updated weights for policy 0, policy_version 86400 (0.0009) [2023-10-08 11:11:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176488448. Throughput: 0: 1826.0, 1: 1825.3. Samples: 44133094. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:37,016][52710] Avg episode reward: [(0, '34.920'), (1, '35.910')] [2023-10-08 11:11:37,196][53885] Updated weights for policy 1, policy_version 85972 (0.0011) [2023-10-08 11:11:37,562][53885] Updated weights for policy 1, policy_version 85982 (0.0010) [2023-10-08 11:11:40,605][53852] Updated weights for policy 0, policy_version 86410 (0.0008) [2023-10-08 11:11:40,971][53852] Updated weights for policy 0, policy_version 86420 (0.0010) [2023-10-08 11:11:41,297][53885] Updated weights for policy 1, policy_version 85992 (0.0009) [2023-10-08 11:11:41,333][53852] Updated weights for policy 0, policy_version 86430 (0.0008) [2023-10-08 11:11:41,659][53885] Updated weights for policy 1, policy_version 86002 (0.0009) [2023-10-08 11:11:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176553984. Throughput: 0: 1835.7, 1: 1829.2. Samples: 44144384. Policy #0 lag: (min: 30.0, avg: 35.0, max: 62.0) [2023-10-08 11:11:42,016][52710] Avg episode reward: [(0, '34.730'), (1, '34.490')] [2023-10-08 11:11:42,027][53885] Updated weights for policy 1, policy_version 86012 (0.0007) [2023-10-08 11:11:44,964][53852] Updated weights for policy 0, policy_version 86440 (0.0009) [2023-10-08 11:11:45,330][53852] Updated weights for policy 0, policy_version 86450 (0.0009) [2023-10-08 11:11:45,655][53885] Updated weights for policy 1, policy_version 86022 (0.0007) [2023-10-08 11:11:45,699][53852] Updated weights for policy 0, policy_version 86460 (0.0007) [2023-10-08 11:11:46,019][53885] Updated weights for policy 1, policy_version 86032 (0.0007) [2023-10-08 11:11:46,387][53885] Updated weights for policy 1, policy_version 86042 (0.0010) [2023-10-08 11:11:47,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 176652288. Throughput: 0: 1830.8, 1: 1831.2. Samples: 44166368. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:11:47,016][52710] Avg episode reward: [(0, '35.890'), (1, '33.980')] [2023-10-08 11:11:49,419][53852] Updated weights for policy 0, policy_version 86470 (0.0008) [2023-10-08 11:11:49,785][53852] Updated weights for policy 0, policy_version 86480 (0.0007) [2023-10-08 11:11:50,086][53885] Updated weights for policy 1, policy_version 86052 (0.0008) [2023-10-08 11:11:50,159][53852] Updated weights for policy 0, policy_version 86490 (0.0009) [2023-10-08 11:11:50,457][53885] Updated weights for policy 1, policy_version 86062 (0.0010) [2023-10-08 11:11:50,818][53885] Updated weights for policy 1, policy_version 86072 (0.0009) [2023-10-08 11:11:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 176717824. Throughput: 0: 1836.9, 1: 1833.0. Samples: 44187440. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:11:52,016][52710] Avg episode reward: [(0, '37.290'), (1, '38.710')] [2023-10-08 11:11:52,029][53500] Saving new best policy, reward=37.290! [2023-10-08 11:11:53,898][53852] Updated weights for policy 0, policy_version 86500 (0.0009) [2023-10-08 11:11:54,270][53852] Updated weights for policy 0, policy_version 86510 (0.0007) [2023-10-08 11:11:54,519][53885] Updated weights for policy 1, policy_version 86082 (0.0009) [2023-10-08 11:11:54,640][53852] Updated weights for policy 0, policy_version 86520 (0.0007) [2023-10-08 11:11:54,891][53885] Updated weights for policy 1, policy_version 86092 (0.0009) [2023-10-08 11:11:55,259][53885] Updated weights for policy 1, policy_version 86102 (0.0009) [2023-10-08 11:11:55,633][53885] Updated weights for policy 1, policy_version 86112 (0.0008) [2023-10-08 11:11:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176783360. Throughput: 0: 1830.1, 1: 1832.3. Samples: 44199300. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:11:57,016][52710] Avg episode reward: [(0, '35.620'), (1, '37.820')] [2023-10-08 11:11:58,264][53852] Updated weights for policy 0, policy_version 86530 (0.0007) [2023-10-08 11:11:58,622][53852] Updated weights for policy 0, policy_version 86540 (0.0011) [2023-10-08 11:11:59,001][53852] Updated weights for policy 0, policy_version 86550 (0.0010) [2023-10-08 11:11:59,366][53852] Updated weights for policy 0, policy_version 86560 (0.0008) [2023-10-08 11:11:59,369][53885] Updated weights for policy 1, policy_version 86122 (0.0007) [2023-10-08 11:11:59,738][53885] Updated weights for policy 1, policy_version 86132 (0.0010) [2023-10-08 11:12:00,100][53885] Updated weights for policy 1, policy_version 86142 (0.0011) [2023-10-08 11:12:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 176848896. Throughput: 0: 1841.2, 1: 1832.4. Samples: 44220420. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:12:02,016][52710] Avg episode reward: [(0, '34.100'), (1, '34.710')] [2023-10-08 11:12:03,136][53852] Updated weights for policy 0, policy_version 86570 (0.0010) [2023-10-08 11:12:03,510][53852] Updated weights for policy 0, policy_version 86580 (0.0010) [2023-10-08 11:12:03,756][53885] Updated weights for policy 1, policy_version 86152 (0.0010) [2023-10-08 11:12:03,881][53852] Updated weights for policy 0, policy_version 86590 (0.0008) [2023-10-08 11:12:04,121][53885] Updated weights for policy 1, policy_version 86162 (0.0007) [2023-10-08 11:12:04,489][53885] Updated weights for policy 1, policy_version 86172 (0.0008) [2023-10-08 11:12:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 176914432. Throughput: 0: 1841.9, 1: 1826.3. Samples: 44243420. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:12:07,016][52710] Avg episode reward: [(0, '33.780'), (1, '36.400')] [2023-10-08 11:12:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000086176_88244224.pth... [2023-10-08 11:12:07,059][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000084480_86507520.pth [2023-10-08 11:12:07,348][53852] Updated weights for policy 0, policy_version 86600 (0.0007) [2023-10-08 11:12:07,715][53852] Updated weights for policy 0, policy_version 86610 (0.0008) [2023-10-08 11:12:08,089][53852] Updated weights for policy 0, policy_version 86620 (0.0009) [2023-10-08 11:12:08,220][53885] Updated weights for policy 1, policy_version 86182 (0.0009) [2023-10-08 11:12:08,226][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000086624_88702976.pth... [2023-10-08 11:12:08,257][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000084896_86933504.pth [2023-10-08 11:12:08,586][53885] Updated weights for policy 1, policy_version 86192 (0.0010) [2023-10-08 11:12:08,960][53885] Updated weights for policy 1, policy_version 86202 (0.0010) [2023-10-08 11:12:11,555][53852] Updated weights for policy 0, policy_version 86630 (0.0008) [2023-10-08 11:12:11,920][53852] Updated weights for policy 0, policy_version 86640 (0.0008) [2023-10-08 11:12:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176979968. Throughput: 0: 1849.6, 1: 1827.0. Samples: 44253582. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:12:12,015][52710] Avg episode reward: [(0, '34.170'), (1, '35.470')] [2023-10-08 11:12:12,298][53852] Updated weights for policy 0, policy_version 86650 (0.0007) [2023-10-08 11:12:12,675][53885] Updated weights for policy 1, policy_version 86212 (0.0010) [2023-10-08 11:12:13,030][53885] Updated weights for policy 1, policy_version 86222 (0.0008) [2023-10-08 11:12:13,407][53885] Updated weights for policy 1, policy_version 86232 (0.0012) [2023-10-08 11:12:15,897][53852] Updated weights for policy 0, policy_version 86660 (0.0008) [2023-10-08 11:12:16,256][53852] Updated weights for policy 0, policy_version 86670 (0.0008) [2023-10-08 11:12:16,623][53852] Updated weights for policy 0, policy_version 86680 (0.0007) [2023-10-08 11:12:17,015][52710] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177078272. Throughput: 0: 1843.8, 1: 1819.4. Samples: 44276350. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:12:17,016][52710] Avg episode reward: [(0, '32.510'), (1, '32.380')] [2023-10-08 11:12:17,039][53885] Updated weights for policy 1, policy_version 86242 (0.0010) [2023-10-08 11:12:17,402][53885] Updated weights for policy 1, policy_version 86252 (0.0009) [2023-10-08 11:12:17,778][53885] Updated weights for policy 1, policy_version 86262 (0.0008) [2023-10-08 11:12:18,135][53885] Updated weights for policy 1, policy_version 86272 (0.0009) [2023-10-08 11:12:20,159][53852] Updated weights for policy 0, policy_version 86690 (0.0008) [2023-10-08 11:12:20,526][53852] Updated weights for policy 0, policy_version 86700 (0.0008) [2023-10-08 11:12:20,896][53852] Updated weights for policy 0, policy_version 86710 (0.0009) [2023-10-08 11:12:21,265][53852] Updated weights for policy 0, policy_version 86720 (0.0010) [2023-10-08 11:12:21,690][53885] Updated weights for policy 1, policy_version 86282 (0.0009) [2023-10-08 11:12:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 177143808. Throughput: 0: 1836.3, 1: 1821.7. Samples: 44297700. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:12:22,015][52710] Avg episode reward: [(0, '35.250'), (1, '35.000')] [2023-10-08 11:12:22,056][53885] Updated weights for policy 1, policy_version 86292 (0.0008) [2023-10-08 11:12:22,420][53885] Updated weights for policy 1, policy_version 86302 (0.0008) [2023-10-08 11:12:24,997][53852] Updated weights for policy 0, policy_version 86730 (0.0009) [2023-10-08 11:12:25,374][53852] Updated weights for policy 0, policy_version 86740 (0.0008) [2023-10-08 11:12:25,739][53852] Updated weights for policy 0, policy_version 86750 (0.0009) [2023-10-08 11:12:26,152][53885] Updated weights for policy 1, policy_version 86312 (0.0011) [2023-10-08 11:12:26,520][53885] Updated weights for policy 1, policy_version 86322 (0.0008) [2023-10-08 11:12:26,887][53885] Updated weights for policy 1, policy_version 86332 (0.0010) [2023-10-08 11:12:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177209344. Throughput: 0: 1851.0, 1: 1823.6. Samples: 44309738. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:12:27,016][52710] Avg episode reward: [(0, '34.380'), (1, '34.590')] [2023-10-08 11:12:29,322][53852] Updated weights for policy 0, policy_version 86760 (0.0008) [2023-10-08 11:12:29,688][53852] Updated weights for policy 0, policy_version 86770 (0.0008) [2023-10-08 11:12:30,049][53852] Updated weights for policy 0, policy_version 86780 (0.0008) [2023-10-08 11:12:30,411][53885] Updated weights for policy 1, policy_version 86342 (0.0008) [2023-10-08 11:12:30,783][53885] Updated weights for policy 1, policy_version 86352 (0.0009) [2023-10-08 11:12:31,156][53885] Updated weights for policy 1, policy_version 86362 (0.0009) [2023-10-08 11:12:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 177307648. Throughput: 0: 1840.9, 1: 1817.4. Samples: 44330992. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:12:32,016][52710] Avg episode reward: [(0, '33.060'), (1, '35.410')] [2023-10-08 11:12:33,714][53852] Updated weights for policy 0, policy_version 86790 (0.0009) [2023-10-08 11:12:34,077][53852] Updated weights for policy 0, policy_version 86800 (0.0009) [2023-10-08 11:12:34,452][53852] Updated weights for policy 0, policy_version 86810 (0.0008) [2023-10-08 11:12:34,817][53885] Updated weights for policy 1, policy_version 86372 (0.0009) [2023-10-08 11:12:35,184][53885] Updated weights for policy 1, policy_version 86382 (0.0010) [2023-10-08 11:12:35,557][53885] Updated weights for policy 1, policy_version 86392 (0.0009) [2023-10-08 11:12:37,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177373184. Throughput: 0: 1852.5, 1: 1822.3. Samples: 44352806. Policy #0 lag: (min: 9.0, avg: 30.1, max: 32.0) [2023-10-08 11:12:37,016][52710] Avg episode reward: [(0, '35.840'), (1, '34.190')] [2023-10-08 11:12:38,110][53852] Updated weights for policy 0, policy_version 86820 (0.0008) [2023-10-08 11:12:38,471][53852] Updated weights for policy 0, policy_version 86830 (0.0007) [2023-10-08 11:12:38,842][53852] Updated weights for policy 0, policy_version 86840 (0.0008) [2023-10-08 11:12:39,148][53885] Updated weights for policy 1, policy_version 86402 (0.0009) [2023-10-08 11:12:39,511][53885] Updated weights for policy 1, policy_version 86412 (0.0008) [2023-10-08 11:12:39,891][53885] Updated weights for policy 1, policy_version 86422 (0.0007) [2023-10-08 11:12:40,246][53885] Updated weights for policy 1, policy_version 86432 (0.0008) [2023-10-08 11:12:42,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177438720. Throughput: 0: 1843.3, 1: 1813.3. Samples: 44363850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:12:42,016][52710] Avg episode reward: [(0, '32.940'), (1, '34.620')] [2023-10-08 11:12:42,478][53852] Updated weights for policy 0, policy_version 86850 (0.0008) [2023-10-08 11:12:42,844][53852] Updated weights for policy 0, policy_version 86860 (0.0010) [2023-10-08 11:12:43,216][53852] Updated weights for policy 0, policy_version 86870 (0.0009) [2023-10-08 11:12:43,592][53852] Updated weights for policy 0, policy_version 86880 (0.0008) [2023-10-08 11:12:43,976][53885] Updated weights for policy 1, policy_version 86442 (0.0011) [2023-10-08 11:12:44,350][53885] Updated weights for policy 1, policy_version 86452 (0.0010) [2023-10-08 11:12:44,722][53885] Updated weights for policy 1, policy_version 86462 (0.0007) [2023-10-08 11:12:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 177504256. Throughput: 0: 1853.4, 1: 1823.9. Samples: 44385898. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:12:47,016][52710] Avg episode reward: [(0, '30.210'), (1, '33.420')] [2023-10-08 11:12:47,151][53852] Updated weights for policy 0, policy_version 86890 (0.0007) [2023-10-08 11:12:47,522][53852] Updated weights for policy 0, policy_version 86900 (0.0008) [2023-10-08 11:12:47,903][53852] Updated weights for policy 0, policy_version 86910 (0.0008) [2023-10-08 11:12:48,346][53885] Updated weights for policy 1, policy_version 86472 (0.0007) [2023-10-08 11:12:48,721][53885] Updated weights for policy 1, policy_version 86482 (0.0010) [2023-10-08 11:12:49,074][53885] Updated weights for policy 1, policy_version 86492 (0.0008) [2023-10-08 11:12:51,724][53852] Updated weights for policy 0, policy_version 86920 (0.0010) [2023-10-08 11:12:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 177569792. Throughput: 0: 1844.9, 1: 1828.8. Samples: 44408734. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:12:52,015][52710] Avg episode reward: [(0, '31.650'), (1, '40.830')] [2023-10-08 11:12:52,093][53852] Updated weights for policy 0, policy_version 86930 (0.0010) [2023-10-08 11:12:52,466][53852] Updated weights for policy 0, policy_version 86940 (0.0007) [2023-10-08 11:12:52,906][53885] Updated weights for policy 1, policy_version 86502 (0.0009) [2023-10-08 11:12:53,284][53885] Updated weights for policy 1, policy_version 86512 (0.0007) [2023-10-08 11:12:53,649][53885] Updated weights for policy 1, policy_version 86522 (0.0007) [2023-10-08 11:12:56,262][53852] Updated weights for policy 0, policy_version 86950 (0.0007) [2023-10-08 11:12:56,645][53852] Updated weights for policy 0, policy_version 86960 (0.0008) [2023-10-08 11:12:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177635328. Throughput: 0: 1843.1, 1: 1832.6. Samples: 44418988. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:12:57,016][52710] Avg episode reward: [(0, '27.780'), (1, '35.880')] [2023-10-08 11:12:57,020][53852] Updated weights for policy 0, policy_version 86970 (0.0010) [2023-10-08 11:12:57,282][53885] Updated weights for policy 1, policy_version 86532 (0.0008) [2023-10-08 11:12:57,646][53885] Updated weights for policy 1, policy_version 86542 (0.0008) [2023-10-08 11:12:58,011][53885] Updated weights for policy 1, policy_version 86552 (0.0010) [2023-10-08 11:13:00,575][53852] Updated weights for policy 0, policy_version 86980 (0.0008) [2023-10-08 11:13:00,940][53852] Updated weights for policy 0, policy_version 86990 (0.0007) [2023-10-08 11:13:01,316][53852] Updated weights for policy 0, policy_version 87000 (0.0007) [2023-10-08 11:13:01,667][53885] Updated weights for policy 1, policy_version 86562 (0.0010) [2023-10-08 11:13:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177733632. Throughput: 0: 1842.1, 1: 1840.5. Samples: 44442068. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:13:02,016][52710] Avg episode reward: [(0, '28.310'), (1, '37.510')] [2023-10-08 11:13:02,034][53885] Updated weights for policy 1, policy_version 86572 (0.0009) [2023-10-08 11:13:02,400][53885] Updated weights for policy 1, policy_version 86582 (0.0009) [2023-10-08 11:13:02,769][53885] Updated weights for policy 1, policy_version 86592 (0.0007) [2023-10-08 11:13:04,958][53852] Updated weights for policy 0, policy_version 87010 (0.0010) [2023-10-08 11:13:05,326][53852] Updated weights for policy 0, policy_version 87020 (0.0010) [2023-10-08 11:13:05,696][53852] Updated weights for policy 0, policy_version 87030 (0.0009) [2023-10-08 11:13:06,066][53852] Updated weights for policy 0, policy_version 87040 (0.0007) [2023-10-08 11:13:06,486][53885] Updated weights for policy 1, policy_version 86602 (0.0009) [2023-10-08 11:13:06,858][53885] Updated weights for policy 1, policy_version 86612 (0.0008) [2023-10-08 11:13:07,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 177799168. Throughput: 0: 1840.7, 1: 1836.7. Samples: 44463182. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:13:07,016][52710] Avg episode reward: [(0, '30.640'), (1, '40.190')] [2023-10-08 11:13:07,230][53885] Updated weights for policy 1, policy_version 86622 (0.0009) [2023-10-08 11:13:09,696][53852] Updated weights for policy 0, policy_version 87050 (0.0010) [2023-10-08 11:13:10,059][53852] Updated weights for policy 0, policy_version 87060 (0.0011) [2023-10-08 11:13:10,425][53852] Updated weights for policy 0, policy_version 87070 (0.0010) [2023-10-08 11:13:10,909][53885] Updated weights for policy 1, policy_version 86632 (0.0008) [2023-10-08 11:13:11,274][53885] Updated weights for policy 1, policy_version 86642 (0.0009) [2023-10-08 11:13:11,646][53885] Updated weights for policy 1, policy_version 86652 (0.0009) [2023-10-08 11:13:12,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 177897472. Throughput: 0: 1829.3, 1: 1844.4. Samples: 44475054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:13:12,016][52710] Avg episode reward: [(0, '31.350'), (1, '35.080')] [2023-10-08 11:13:14,012][53852] Updated weights for policy 0, policy_version 87080 (0.0009) [2023-10-08 11:13:14,383][53852] Updated weights for policy 0, policy_version 87090 (0.0009) [2023-10-08 11:13:14,753][53852] Updated weights for policy 0, policy_version 87100 (0.0007) [2023-10-08 11:13:15,244][53885] Updated weights for policy 1, policy_version 86662 (0.0010) [2023-10-08 11:13:15,611][53885] Updated weights for policy 1, policy_version 86672 (0.0011) [2023-10-08 11:13:15,980][53885] Updated weights for policy 1, policy_version 86682 (0.0011) [2023-10-08 11:13:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177963008. Throughput: 0: 1836.1, 1: 1840.1. Samples: 44496422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:13:17,016][52710] Avg episode reward: [(0, '32.910'), (1, '32.850')] [2023-10-08 11:13:18,264][53852] Updated weights for policy 0, policy_version 87110 (0.0010) [2023-10-08 11:13:18,634][53852] Updated weights for policy 0, policy_version 87120 (0.0008) [2023-10-08 11:13:19,008][53852] Updated weights for policy 0, policy_version 87130 (0.0007) [2023-10-08 11:13:19,571][53885] Updated weights for policy 1, policy_version 86692 (0.0009) [2023-10-08 11:13:19,946][53885] Updated weights for policy 1, policy_version 86702 (0.0007) [2023-10-08 11:13:20,312][53885] Updated weights for policy 1, policy_version 86712 (0.0009) [2023-10-08 11:13:22,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178028544. Throughput: 0: 1843.3, 1: 1846.5. Samples: 44518850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:13:22,016][52710] Avg episode reward: [(0, '35.320'), (1, '33.790')] [2023-10-08 11:13:22,542][53852] Updated weights for policy 0, policy_version 87140 (0.0008) [2023-10-08 11:13:22,901][53852] Updated weights for policy 0, policy_version 87150 (0.0007) [2023-10-08 11:13:23,275][53852] Updated weights for policy 0, policy_version 87160 (0.0008) [2023-10-08 11:13:23,984][53885] Updated weights for policy 1, policy_version 86722 (0.0010) [2023-10-08 11:13:24,343][53885] Updated weights for policy 1, policy_version 86732 (0.0009) [2023-10-08 11:13:24,714][53885] Updated weights for policy 1, policy_version 86742 (0.0009) [2023-10-08 11:13:25,081][53885] Updated weights for policy 1, policy_version 86752 (0.0009) [2023-10-08 11:13:26,854][53852] Updated weights for policy 0, policy_version 87170 (0.0008) [2023-10-08 11:13:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178094080. Throughput: 0: 1843.3, 1: 1840.3. Samples: 44529610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:13:27,016][52710] Avg episode reward: [(0, '36.260'), (1, '39.860')] [2023-10-08 11:13:27,232][53852] Updated weights for policy 0, policy_version 87180 (0.0009) [2023-10-08 11:13:27,600][53852] Updated weights for policy 0, policy_version 87190 (0.0010) [2023-10-08 11:13:27,978][53852] Updated weights for policy 0, policy_version 87200 (0.0008) [2023-10-08 11:13:28,717][53885] Updated weights for policy 1, policy_version 86762 (0.0007) [2023-10-08 11:13:29,085][53885] Updated weights for policy 1, policy_version 86772 (0.0009) [2023-10-08 11:13:29,449][53885] Updated weights for policy 1, policy_version 86782 (0.0010) [2023-10-08 11:13:31,488][53852] Updated weights for policy 0, policy_version 87210 (0.0007) [2023-10-08 11:13:31,857][53852] Updated weights for policy 0, policy_version 87220 (0.0007) [2023-10-08 11:13:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178159616. Throughput: 0: 1850.1, 1: 1851.0. Samples: 44552448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:13:32,016][52710] Avg episode reward: [(0, '36.450'), (1, '35.790')] [2023-10-08 11:13:32,227][53852] Updated weights for policy 0, policy_version 87230 (0.0011) [2023-10-08 11:13:33,087][53885] Updated weights for policy 1, policy_version 86792 (0.0010) [2023-10-08 11:13:33,459][53885] Updated weights for policy 1, policy_version 86802 (0.0007) [2023-10-08 11:13:33,819][53885] Updated weights for policy 1, policy_version 86812 (0.0008) [2023-10-08 11:13:36,006][53852] Updated weights for policy 0, policy_version 87240 (0.0008) [2023-10-08 11:13:36,378][53852] Updated weights for policy 0, policy_version 87250 (0.0008) [2023-10-08 11:13:36,762][53852] Updated weights for policy 0, policy_version 87260 (0.0007) [2023-10-08 11:13:37,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 178257920. Throughput: 0: 1832.2, 1: 1847.8. Samples: 44574334. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:13:37,016][52710] Avg episode reward: [(0, '35.720'), (1, '37.490')] [2023-10-08 11:13:37,435][53885] Updated weights for policy 1, policy_version 86822 (0.0009) [2023-10-08 11:13:37,810][53885] Updated weights for policy 1, policy_version 86832 (0.0007) [2023-10-08 11:13:38,186][53885] Updated weights for policy 1, policy_version 86842 (0.0009) [2023-10-08 11:13:40,490][53852] Updated weights for policy 0, policy_version 87270 (0.0010) [2023-10-08 11:13:40,866][53852] Updated weights for policy 0, policy_version 87280 (0.0009) [2023-10-08 11:13:41,229][53852] Updated weights for policy 0, policy_version 87290 (0.0007) [2023-10-08 11:13:41,944][53885] Updated weights for policy 1, policy_version 86852 (0.0007) [2023-10-08 11:13:42,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178323456. Throughput: 0: 1853.1, 1: 1844.9. Samples: 44585400. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:13:42,016][52710] Avg episode reward: [(0, '35.260'), (1, '34.120')] [2023-10-08 11:13:42,321][53885] Updated weights for policy 1, policy_version 86862 (0.0007) [2023-10-08 11:13:42,685][53885] Updated weights for policy 1, policy_version 86872 (0.0008) [2023-10-08 11:13:44,983][53852] Updated weights for policy 0, policy_version 87300 (0.0008) [2023-10-08 11:13:45,353][53852] Updated weights for policy 0, policy_version 87310 (0.0008) [2023-10-08 11:13:45,727][53852] Updated weights for policy 0, policy_version 87320 (0.0008) [2023-10-08 11:13:46,336][53885] Updated weights for policy 1, policy_version 86882 (0.0009) [2023-10-08 11:13:46,708][53885] Updated weights for policy 1, policy_version 86892 (0.0009) [2023-10-08 11:13:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178388992. Throughput: 0: 1831.1, 1: 1839.3. Samples: 44607236. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:13:47,016][52710] Avg episode reward: [(0, '32.960'), (1, '34.900')] [2023-10-08 11:13:47,071][53885] Updated weights for policy 1, policy_version 86902 (0.0010) [2023-10-08 11:13:47,432][53885] Updated weights for policy 1, policy_version 86912 (0.0008) [2023-10-08 11:13:49,351][53852] Updated weights for policy 0, policy_version 87330 (0.0007) [2023-10-08 11:13:49,708][53852] Updated weights for policy 0, policy_version 87340 (0.0007) [2023-10-08 11:13:50,084][53852] Updated weights for policy 0, policy_version 87350 (0.0007) [2023-10-08 11:13:50,442][53852] Updated weights for policy 0, policy_version 87360 (0.0007) [2023-10-08 11:13:51,063][53885] Updated weights for policy 1, policy_version 86922 (0.0010) [2023-10-08 11:13:51,429][53885] Updated weights for policy 1, policy_version 86932 (0.0009) [2023-10-08 11:13:51,794][53885] Updated weights for policy 1, policy_version 86942 (0.0009) [2023-10-08 11:13:52,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 178487296. Throughput: 0: 1851.8, 1: 1822.3. Samples: 44628518. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:13:52,016][52710] Avg episode reward: [(0, '34.110'), (1, '32.990')] [2023-10-08 11:13:54,051][53852] Updated weights for policy 0, policy_version 87370 (0.0011) [2023-10-08 11:13:54,423][53852] Updated weights for policy 0, policy_version 87380 (0.0009) [2023-10-08 11:13:54,802][53852] Updated weights for policy 0, policy_version 87390 (0.0007) [2023-10-08 11:13:55,405][53885] Updated weights for policy 1, policy_version 86952 (0.0009) [2023-10-08 11:13:55,779][53885] Updated weights for policy 1, policy_version 86962 (0.0010) [2023-10-08 11:13:56,142][53885] Updated weights for policy 1, policy_version 86972 (0.0008) [2023-10-08 11:13:57,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 178552832. Throughput: 0: 1836.2, 1: 1837.5. Samples: 44640370. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:13:57,016][52710] Avg episode reward: [(0, '32.270'), (1, '32.900')] [2023-10-08 11:13:58,269][53852] Updated weights for policy 0, policy_version 87400 (0.0009) [2023-10-08 11:13:58,641][53852] Updated weights for policy 0, policy_version 87410 (0.0008) [2023-10-08 11:13:59,002][53852] Updated weights for policy 0, policy_version 87420 (0.0007) [2023-10-08 11:13:59,707][53885] Updated weights for policy 1, policy_version 86982 (0.0010) [2023-10-08 11:14:00,067][53885] Updated weights for policy 1, policy_version 86992 (0.0007) [2023-10-08 11:14:00,442][53885] Updated weights for policy 1, policy_version 87002 (0.0009) [2023-10-08 11:14:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178618368. Throughput: 0: 1855.4, 1: 1819.3. Samples: 44661784. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:14:02,015][52710] Avg episode reward: [(0, '35.720'), (1, '36.400')] [2023-10-08 11:14:02,669][53852] Updated weights for policy 0, policy_version 87430 (0.0009) [2023-10-08 11:14:03,034][53852] Updated weights for policy 0, policy_version 87440 (0.0011) [2023-10-08 11:14:03,404][53852] Updated weights for policy 0, policy_version 87450 (0.0008) [2023-10-08 11:14:03,918][53885] Updated weights for policy 1, policy_version 87012 (0.0010) [2023-10-08 11:14:04,287][53885] Updated weights for policy 1, policy_version 87022 (0.0011) [2023-10-08 11:14:04,656][53885] Updated weights for policy 1, policy_version 87032 (0.0010) [2023-10-08 11:14:07,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178683904. Throughput: 0: 1849.1, 1: 1837.0. Samples: 44684726. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:14:07,015][52710] Avg episode reward: [(0, '32.470'), (1, '37.310')] [2023-10-08 11:14:07,023][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000087040_89128960.pth... [2023-10-08 11:14:07,058][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000085344_87392256.pth [2023-10-08 11:14:07,072][53852] Updated weights for policy 0, policy_version 87460 (0.0008) [2023-10-08 11:14:07,445][53852] Updated weights for policy 0, policy_version 87470 (0.0007) [2023-10-08 11:14:07,814][53852] Updated weights for policy 0, policy_version 87480 (0.0007) [2023-10-08 11:14:08,106][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000087488_89587712.pth... [2023-10-08 11:14:08,146][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000085760_87818240.pth [2023-10-08 11:14:08,386][53885] Updated weights for policy 1, policy_version 87042 (0.0008) [2023-10-08 11:14:08,756][53885] Updated weights for policy 1, policy_version 87052 (0.0007) [2023-10-08 11:14:09,131][53885] Updated weights for policy 1, policy_version 87062 (0.0008) [2023-10-08 11:14:09,496][53885] Updated weights for policy 1, policy_version 87072 (0.0010) [2023-10-08 11:14:11,428][53852] Updated weights for policy 0, policy_version 87490 (0.0007) [2023-10-08 11:14:11,813][53852] Updated weights for policy 0, policy_version 87500 (0.0007) [2023-10-08 11:14:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178749440. Throughput: 0: 1846.3, 1: 1821.0. Samples: 44694638. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:14:12,016][52710] Avg episode reward: [(0, '31.400'), (1, '35.470')] [2023-10-08 11:14:12,172][53852] Updated weights for policy 0, policy_version 87510 (0.0008) [2023-10-08 11:14:12,541][53852] Updated weights for policy 0, policy_version 87520 (0.0008) [2023-10-08 11:14:13,013][53885] Updated weights for policy 1, policy_version 87082 (0.0008) [2023-10-08 11:14:13,390][53885] Updated weights for policy 1, policy_version 87092 (0.0008) [2023-10-08 11:14:13,752][53885] Updated weights for policy 1, policy_version 87102 (0.0008) [2023-10-08 11:14:16,233][53852] Updated weights for policy 0, policy_version 87530 (0.0009) [2023-10-08 11:14:16,607][53852] Updated weights for policy 0, policy_version 87540 (0.0007) [2023-10-08 11:14:16,973][53852] Updated weights for policy 0, policy_version 87550 (0.0007) [2023-10-08 11:14:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 178814976. Throughput: 0: 1840.3, 1: 1834.6. Samples: 44717816. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:14:17,016][52710] Avg episode reward: [(0, '31.770'), (1, '35.620')] [2023-10-08 11:14:17,514][53885] Updated weights for policy 1, policy_version 87112 (0.0010) [2023-10-08 11:14:17,873][53885] Updated weights for policy 1, policy_version 87122 (0.0009) [2023-10-08 11:14:18,237][53885] Updated weights for policy 1, policy_version 87132 (0.0009) [2023-10-08 11:14:20,699][53852] Updated weights for policy 0, policy_version 87560 (0.0008) [2023-10-08 11:14:21,066][53852] Updated weights for policy 0, policy_version 87570 (0.0008) [2023-10-08 11:14:21,445][53852] Updated weights for policy 0, policy_version 87580 (0.0011) [2023-10-08 11:14:21,850][53885] Updated weights for policy 1, policy_version 87142 (0.0009) [2023-10-08 11:14:22,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178913280. Throughput: 0: 1829.9, 1: 1837.3. Samples: 44739360. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:14:22,015][52710] Avg episode reward: [(0, '32.410'), (1, '39.660')] [2023-10-08 11:14:22,214][53885] Updated weights for policy 1, policy_version 87152 (0.0010) [2023-10-08 11:14:22,582][53885] Updated weights for policy 1, policy_version 87162 (0.0007) [2023-10-08 11:14:25,078][53852] Updated weights for policy 0, policy_version 87590 (0.0009) [2023-10-08 11:14:25,449][53852] Updated weights for policy 0, policy_version 87600 (0.0008) [2023-10-08 11:14:25,817][53852] Updated weights for policy 0, policy_version 87610 (0.0009) [2023-10-08 11:14:26,383][53885] Updated weights for policy 1, policy_version 87172 (0.0008) [2023-10-08 11:14:26,741][53885] Updated weights for policy 1, policy_version 87182 (0.0007) [2023-10-08 11:14:27,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178978816. Throughput: 0: 1838.0, 1: 1837.5. Samples: 44750796. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) [2023-10-08 11:14:27,016][52710] Avg episode reward: [(0, '34.840'), (1, '38.360')] [2023-10-08 11:14:27,117][53885] Updated weights for policy 1, policy_version 87192 (0.0009) [2023-10-08 11:14:29,559][53852] Updated weights for policy 0, policy_version 87620 (0.0009) [2023-10-08 11:14:29,937][53852] Updated weights for policy 0, policy_version 87630 (0.0010) [2023-10-08 11:14:30,297][53852] Updated weights for policy 0, policy_version 87640 (0.0011) [2023-10-08 11:14:30,698][53885] Updated weights for policy 1, policy_version 87202 (0.0007) [2023-10-08 11:14:31,091][53885] Updated weights for policy 1, policy_version 87212 (0.0010) [2023-10-08 11:14:31,458][53885] Updated weights for policy 1, policy_version 87222 (0.0008) [2023-10-08 11:14:31,820][53885] Updated weights for policy 1, policy_version 87232 (0.0007) [2023-10-08 11:14:32,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 179077120. Throughput: 0: 1826.1, 1: 1843.0. Samples: 44772348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:14:32,016][52710] Avg episode reward: [(0, '33.880'), (1, '33.010')] [2023-10-08 11:14:34,013][53852] Updated weights for policy 0, policy_version 87650 (0.0009) [2023-10-08 11:14:34,381][53852] Updated weights for policy 0, policy_version 87660 (0.0008) [2023-10-08 11:14:34,750][53852] Updated weights for policy 0, policy_version 87670 (0.0007) [2023-10-08 11:14:35,118][53852] Updated weights for policy 0, policy_version 87680 (0.0009) [2023-10-08 11:14:35,330][53885] Updated weights for policy 1, policy_version 87242 (0.0007) [2023-10-08 11:14:35,700][53885] Updated weights for policy 1, policy_version 87252 (0.0008) [2023-10-08 11:14:36,070][53885] Updated weights for policy 1, policy_version 87262 (0.0009) [2023-10-08 11:14:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 179142656. Throughput: 0: 1833.0, 1: 1835.0. Samples: 44793580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:14:37,016][52710] Avg episode reward: [(0, '36.110'), (1, '37.360')] [2023-10-08 11:14:38,778][53852] Updated weights for policy 0, policy_version 87690 (0.0009) [2023-10-08 11:14:39,146][53852] Updated weights for policy 0, policy_version 87700 (0.0010) [2023-10-08 11:14:39,525][53852] Updated weights for policy 0, policy_version 87710 (0.0010) [2023-10-08 11:14:39,744][53885] Updated weights for policy 1, policy_version 87272 (0.0010) [2023-10-08 11:14:40,124][53885] Updated weights for policy 1, policy_version 87282 (0.0008) [2023-10-08 11:14:40,499][53885] Updated weights for policy 1, policy_version 87292 (0.0010) [2023-10-08 11:14:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179208192. Throughput: 0: 1822.5, 1: 1839.1. Samples: 44805142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:14:42,016][52710] Avg episode reward: [(0, '36.080'), (1, '38.140')] [2023-10-08 11:14:43,189][53852] Updated weights for policy 0, policy_version 87720 (0.0009) [2023-10-08 11:14:43,562][53852] Updated weights for policy 0, policy_version 87730 (0.0009) [2023-10-08 11:14:43,927][53852] Updated weights for policy 0, policy_version 87740 (0.0009) [2023-10-08 11:14:44,165][53885] Updated weights for policy 1, policy_version 87302 (0.0008) [2023-10-08 11:14:44,532][53885] Updated weights for policy 1, policy_version 87312 (0.0007) [2023-10-08 11:14:44,893][53885] Updated weights for policy 1, policy_version 87322 (0.0007) [2023-10-08 11:14:47,016][52710] Fps is (10 sec: 13106.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 179273728. Throughput: 0: 1823.3, 1: 1836.7. Samples: 44826488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:14:47,017][52710] Avg episode reward: [(0, '32.360'), (1, '36.210')] [2023-10-08 11:14:47,723][53852] Updated weights for policy 0, policy_version 87750 (0.0009) [2023-10-08 11:14:48,082][53852] Updated weights for policy 0, policy_version 87760 (0.0008) [2023-10-08 11:14:48,455][53852] Updated weights for policy 0, policy_version 87770 (0.0007) [2023-10-08 11:14:48,648][53885] Updated weights for policy 1, policy_version 87332 (0.0008) [2023-10-08 11:14:49,014][53885] Updated weights for policy 1, policy_version 87342 (0.0008) [2023-10-08 11:14:49,383][53885] Updated weights for policy 1, policy_version 87352 (0.0009) [2023-10-08 11:14:51,976][53852] Updated weights for policy 0, policy_version 87780 (0.0007) [2023-10-08 11:14:52,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 179339264. Throughput: 0: 1826.2, 1: 1838.8. Samples: 44849650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:14:52,016][52710] Avg episode reward: [(0, '33.010'), (1, '35.550')] [2023-10-08 11:14:52,335][53852] Updated weights for policy 0, policy_version 87790 (0.0007) [2023-10-08 11:14:52,710][53852] Updated weights for policy 0, policy_version 87800 (0.0007) [2023-10-08 11:14:53,006][53885] Updated weights for policy 1, policy_version 87362 (0.0008) [2023-10-08 11:14:53,376][53885] Updated weights for policy 1, policy_version 87372 (0.0007) [2023-10-08 11:14:53,752][53885] Updated weights for policy 1, policy_version 87382 (0.0008) [2023-10-08 11:14:54,120][53885] Updated weights for policy 1, policy_version 87392 (0.0010) [2023-10-08 11:14:56,318][53852] Updated weights for policy 0, policy_version 87810 (0.0007) [2023-10-08 11:14:56,684][53852] Updated weights for policy 0, policy_version 87820 (0.0008) [2023-10-08 11:14:57,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 179404800. Throughput: 0: 1829.5, 1: 1838.5. Samples: 44859698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:14:57,016][52710] Avg episode reward: [(0, '32.040'), (1, '35.940')] [2023-10-08 11:14:57,062][53852] Updated weights for policy 0, policy_version 87830 (0.0009) [2023-10-08 11:14:57,440][53852] Updated weights for policy 0, policy_version 87840 (0.0010) [2023-10-08 11:14:57,848][53885] Updated weights for policy 1, policy_version 87402 (0.0008) [2023-10-08 11:14:58,216][53885] Updated weights for policy 1, policy_version 87412 (0.0011) [2023-10-08 11:14:58,586][53885] Updated weights for policy 1, policy_version 87422 (0.0011) [2023-10-08 11:15:00,937][53852] Updated weights for policy 0, policy_version 87850 (0.0009) [2023-10-08 11:15:01,306][53852] Updated weights for policy 0, policy_version 87860 (0.0008) [2023-10-08 11:15:01,665][53852] Updated weights for policy 0, policy_version 87870 (0.0009) [2023-10-08 11:15:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 179503104. Throughput: 0: 1830.7, 1: 1832.7. Samples: 44882668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:02,016][52710] Avg episode reward: [(0, '31.750'), (1, '40.000')] [2023-10-08 11:15:02,379][53885] Updated weights for policy 1, policy_version 87432 (0.0008) [2023-10-08 11:15:02,753][53885] Updated weights for policy 1, policy_version 87442 (0.0008) [2023-10-08 11:15:03,130][53885] Updated weights for policy 1, policy_version 87452 (0.0008) [2023-10-08 11:15:05,236][53852] Updated weights for policy 0, policy_version 87880 (0.0009) [2023-10-08 11:15:05,595][53852] Updated weights for policy 0, policy_version 87890 (0.0010) [2023-10-08 11:15:05,960][53852] Updated weights for policy 0, policy_version 87900 (0.0009) [2023-10-08 11:15:06,740][53885] Updated weights for policy 1, policy_version 87462 (0.0009) [2023-10-08 11:15:07,016][52710] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 179568640. Throughput: 0: 1834.5, 1: 1826.4. Samples: 44904100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:07,017][52710] Avg episode reward: [(0, '33.480'), (1, '35.100')] [2023-10-08 11:15:07,106][53885] Updated weights for policy 1, policy_version 87472 (0.0007) [2023-10-08 11:15:07,472][53885] Updated weights for policy 1, policy_version 87482 (0.0009) [2023-10-08 11:15:09,653][53852] Updated weights for policy 0, policy_version 87910 (0.0009) [2023-10-08 11:15:10,024][53852] Updated weights for policy 0, policy_version 87920 (0.0010) [2023-10-08 11:15:10,397][53852] Updated weights for policy 0, policy_version 87930 (0.0010) [2023-10-08 11:15:11,086][53885] Updated weights for policy 1, policy_version 87492 (0.0010) [2023-10-08 11:15:11,457][53885] Updated weights for policy 1, policy_version 87502 (0.0008) [2023-10-08 11:15:11,835][53885] Updated weights for policy 1, policy_version 87512 (0.0009) [2023-10-08 11:15:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179634176. Throughput: 0: 1830.9, 1: 1830.1. Samples: 44915544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:12,016][52710] Avg episode reward: [(0, '33.030'), (1, '32.410')] [2023-10-08 11:15:14,105][53852] Updated weights for policy 0, policy_version 87940 (0.0009) [2023-10-08 11:15:14,466][53852] Updated weights for policy 0, policy_version 87950 (0.0008) [2023-10-08 11:15:14,838][53852] Updated weights for policy 0, policy_version 87960 (0.0008) [2023-10-08 11:15:15,663][53885] Updated weights for policy 1, policy_version 87522 (0.0009) [2023-10-08 11:15:16,067][53885] Updated weights for policy 1, policy_version 87532 (0.0007) [2023-10-08 11:15:16,442][53885] Updated weights for policy 1, policy_version 87542 (0.0009) [2023-10-08 11:15:16,799][53885] Updated weights for policy 1, policy_version 87552 (0.0007) [2023-10-08 11:15:17,015][52710] Fps is (10 sec: 16384.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 179732480. Throughput: 0: 1832.2, 1: 1825.3. Samples: 44936934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:17,016][52710] Avg episode reward: [(0, '31.170'), (1, '39.470')] [2023-10-08 11:15:18,627][53852] Updated weights for policy 0, policy_version 87970 (0.0008) [2023-10-08 11:15:19,023][53852] Updated weights for policy 0, policy_version 87980 (0.0007) [2023-10-08 11:15:19,392][53852] Updated weights for policy 0, policy_version 87990 (0.0011) [2023-10-08 11:15:19,766][53852] Updated weights for policy 0, policy_version 88000 (0.0008) [2023-10-08 11:15:20,403][53885] Updated weights for policy 1, policy_version 87562 (0.0008) [2023-10-08 11:15:20,771][53885] Updated weights for policy 1, policy_version 87572 (0.0009) [2023-10-08 11:15:21,137][53885] Updated weights for policy 1, policy_version 87582 (0.0008) [2023-10-08 11:15:22,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179798016. Throughput: 0: 1838.5, 1: 1829.9. Samples: 44958660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:22,016][52710] Avg episode reward: [(0, '30.590'), (1, '32.250')] [2023-10-08 11:15:23,385][53852] Updated weights for policy 0, policy_version 88010 (0.0009) [2023-10-08 11:15:23,761][53852] Updated weights for policy 0, policy_version 88020 (0.0009) [2023-10-08 11:15:24,139][53852] Updated weights for policy 0, policy_version 88030 (0.0007) [2023-10-08 11:15:24,609][53885] Updated weights for policy 1, policy_version 87592 (0.0009) [2023-10-08 11:15:24,970][53885] Updated weights for policy 1, policy_version 87602 (0.0009) [2023-10-08 11:15:25,336][53885] Updated weights for policy 1, policy_version 87612 (0.0009) [2023-10-08 11:15:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179863552. Throughput: 0: 1838.0, 1: 1832.6. Samples: 44970316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:27,016][52710] Avg episode reward: [(0, '34.030'), (1, '32.880')] [2023-10-08 11:15:27,667][53852] Updated weights for policy 0, policy_version 88040 (0.0007) [2023-10-08 11:15:28,029][53852] Updated weights for policy 0, policy_version 88050 (0.0007) [2023-10-08 11:15:28,399][53852] Updated weights for policy 0, policy_version 88060 (0.0009) [2023-10-08 11:15:29,076][53885] Updated weights for policy 1, policy_version 87622 (0.0009) [2023-10-08 11:15:29,444][53885] Updated weights for policy 1, policy_version 87632 (0.0010) [2023-10-08 11:15:29,815][53885] Updated weights for policy 1, policy_version 87642 (0.0010) [2023-10-08 11:15:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 179929088. Throughput: 0: 1843.9, 1: 1834.5. Samples: 44992016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:32,016][52710] Avg episode reward: [(0, '32.740'), (1, '36.340')] [2023-10-08 11:15:32,124][53852] Updated weights for policy 0, policy_version 88070 (0.0009) [2023-10-08 11:15:32,488][53852] Updated weights for policy 0, policy_version 88080 (0.0008) [2023-10-08 11:15:32,857][53852] Updated weights for policy 0, policy_version 88090 (0.0008) [2023-10-08 11:15:33,466][53885] Updated weights for policy 1, policy_version 87652 (0.0009) [2023-10-08 11:15:33,823][53885] Updated weights for policy 1, policy_version 87662 (0.0008) [2023-10-08 11:15:34,197][53885] Updated weights for policy 1, policy_version 87672 (0.0007) [2023-10-08 11:15:36,474][53852] Updated weights for policy 0, policy_version 88100 (0.0007) [2023-10-08 11:15:36,849][53852] Updated weights for policy 0, policy_version 88110 (0.0007) [2023-10-08 11:15:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 179994624. Throughput: 0: 1835.9, 1: 1834.5. Samples: 45014820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:37,015][52710] Avg episode reward: [(0, '32.300'), (1, '36.500')] [2023-10-08 11:15:37,214][53852] Updated weights for policy 0, policy_version 88120 (0.0010) [2023-10-08 11:15:37,831][53885] Updated weights for policy 1, policy_version 87682 (0.0010) [2023-10-08 11:15:38,198][53885] Updated weights for policy 1, policy_version 87692 (0.0010) [2023-10-08 11:15:38,570][53885] Updated weights for policy 1, policy_version 87702 (0.0008) [2023-10-08 11:15:38,935][53885] Updated weights for policy 1, policy_version 87712 (0.0008) [2023-10-08 11:15:40,847][53852] Updated weights for policy 0, policy_version 88130 (0.0009) [2023-10-08 11:15:41,226][53852] Updated weights for policy 0, policy_version 88140 (0.0008) [2023-10-08 11:15:41,585][53852] Updated weights for policy 0, policy_version 88150 (0.0010) [2023-10-08 11:15:41,955][53852] Updated weights for policy 0, policy_version 88160 (0.0010) [2023-10-08 11:15:42,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 180092928. Throughput: 0: 1839.9, 1: 1834.4. Samples: 45025040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:42,016][52710] Avg episode reward: [(0, '33.320'), (1, '31.690')] [2023-10-08 11:15:42,617][53885] Updated weights for policy 1, policy_version 87722 (0.0008) [2023-10-08 11:15:42,989][53885] Updated weights for policy 1, policy_version 87732 (0.0008) [2023-10-08 11:15:43,363][53885] Updated weights for policy 1, policy_version 87742 (0.0009) [2023-10-08 11:15:45,540][53852] Updated weights for policy 0, policy_version 88170 (0.0008) [2023-10-08 11:15:45,914][53852] Updated weights for policy 0, policy_version 88180 (0.0007) [2023-10-08 11:15:46,277][53852] Updated weights for policy 0, policy_version 88190 (0.0007) [2023-10-08 11:15:47,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 180158464. Throughput: 0: 1828.8, 1: 1835.5. Samples: 45047558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:47,016][52710] Avg episode reward: [(0, '35.050'), (1, '33.920')] [2023-10-08 11:15:47,034][53885] Updated weights for policy 1, policy_version 87752 (0.0012) [2023-10-08 11:15:47,400][53885] Updated weights for policy 1, policy_version 87762 (0.0011) [2023-10-08 11:15:47,766][53885] Updated weights for policy 1, policy_version 87772 (0.0010) [2023-10-08 11:15:49,950][53852] Updated weights for policy 0, policy_version 88200 (0.0007) [2023-10-08 11:15:50,307][53852] Updated weights for policy 0, policy_version 88210 (0.0008) [2023-10-08 11:15:50,680][53852] Updated weights for policy 0, policy_version 88220 (0.0008) [2023-10-08 11:15:51,382][53885] Updated weights for policy 1, policy_version 87782 (0.0008) [2023-10-08 11:15:51,742][53885] Updated weights for policy 1, policy_version 87792 (0.0007) [2023-10-08 11:15:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180224000. Throughput: 0: 1839.0, 1: 1833.2. Samples: 45069348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:52,016][52710] Avg episode reward: [(0, '33.360'), (1, '39.880')] [2023-10-08 11:15:52,108][53885] Updated weights for policy 1, policy_version 87802 (0.0007) [2023-10-08 11:15:54,224][53852] Updated weights for policy 0, policy_version 88230 (0.0007) [2023-10-08 11:15:54,598][53852] Updated weights for policy 0, policy_version 88240 (0.0007) [2023-10-08 11:15:54,976][53852] Updated weights for policy 0, policy_version 88250 (0.0007) [2023-10-08 11:15:55,672][53885] Updated weights for policy 1, policy_version 87812 (0.0008) [2023-10-08 11:15:56,038][53885] Updated weights for policy 1, policy_version 87822 (0.0009) [2023-10-08 11:15:56,404][53885] Updated weights for policy 1, policy_version 87832 (0.0007) [2023-10-08 11:15:57,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 180322304. Throughput: 0: 1831.9, 1: 1842.9. Samples: 45080910. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:15:57,016][52710] Avg episode reward: [(0, '32.060'), (1, '32.250')] [2023-10-08 11:15:58,671][53852] Updated weights for policy 0, policy_version 88260 (0.0009) [2023-10-08 11:15:59,043][53852] Updated weights for policy 0, policy_version 88270 (0.0008) [2023-10-08 11:15:59,404][53852] Updated weights for policy 0, policy_version 88280 (0.0007) [2023-10-08 11:16:00,150][53885] Updated weights for policy 1, policy_version 87842 (0.0009) [2023-10-08 11:16:00,518][53885] Updated weights for policy 1, policy_version 87852 (0.0008) [2023-10-08 11:16:00,885][53885] Updated weights for policy 1, policy_version 87862 (0.0008) [2023-10-08 11:16:01,258][53885] Updated weights for policy 1, policy_version 87872 (0.0010) [2023-10-08 11:16:02,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 180387840. Throughput: 0: 1848.8, 1: 1832.7. Samples: 45102602. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:02,016][52710] Avg episode reward: [(0, '32.710'), (1, '31.820')] [2023-10-08 11:16:02,983][53852] Updated weights for policy 0, policy_version 88290 (0.0009) [2023-10-08 11:16:03,374][53852] Updated weights for policy 0, policy_version 88300 (0.0007) [2023-10-08 11:16:03,740][53852] Updated weights for policy 0, policy_version 88310 (0.0007) [2023-10-08 11:16:04,110][53852] Updated weights for policy 0, policy_version 88320 (0.0008) [2023-10-08 11:16:04,930][53885] Updated weights for policy 1, policy_version 87882 (0.0008) [2023-10-08 11:16:05,294][53885] Updated weights for policy 1, policy_version 87892 (0.0007) [2023-10-08 11:16:05,653][53885] Updated weights for policy 1, policy_version 87902 (0.0008) [2023-10-08 11:16:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 180453376. Throughput: 0: 1849.4, 1: 1844.4. Samples: 45124882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:07,016][52710] Avg episode reward: [(0, '30.720'), (1, '36.720')] [2023-10-08 11:16:07,028][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000087904_90013696.pth... [2023-10-08 11:16:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000088320_90439680.pth... [2023-10-08 11:16:07,067][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000086176_88244224.pth [2023-10-08 11:16:07,072][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000086624_88702976.pth [2023-10-08 11:16:07,818][53852] Updated weights for policy 0, policy_version 88330 (0.0007) [2023-10-08 11:16:08,194][53852] Updated weights for policy 0, policy_version 88340 (0.0008) [2023-10-08 11:16:08,561][53852] Updated weights for policy 0, policy_version 88350 (0.0007) [2023-10-08 11:16:09,181][53885] Updated weights for policy 1, policy_version 87912 (0.0009) [2023-10-08 11:16:09,552][53885] Updated weights for policy 1, policy_version 87922 (0.0008) [2023-10-08 11:16:09,918][53885] Updated weights for policy 1, policy_version 87932 (0.0008) [2023-10-08 11:16:12,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180518912. Throughput: 0: 1842.0, 1: 1828.7. Samples: 45135494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:12,016][52710] Avg episode reward: [(0, '29.990'), (1, '34.450')] [2023-10-08 11:16:12,283][53852] Updated weights for policy 0, policy_version 88360 (0.0007) [2023-10-08 11:16:12,653][53852] Updated weights for policy 0, policy_version 88370 (0.0007) [2023-10-08 11:16:13,030][53852] Updated weights for policy 0, policy_version 88380 (0.0008) [2023-10-08 11:16:13,580][53885] Updated weights for policy 1, policy_version 87942 (0.0008) [2023-10-08 11:16:13,944][53885] Updated weights for policy 1, policy_version 87952 (0.0009) [2023-10-08 11:16:14,309][53885] Updated weights for policy 1, policy_version 87962 (0.0009) [2023-10-08 11:16:16,647][53852] Updated weights for policy 0, policy_version 88390 (0.0007) [2023-10-08 11:16:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 180584448. Throughput: 0: 1845.2, 1: 1838.8. Samples: 45157798. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:17,016][52710] Avg episode reward: [(0, '32.260'), (1, '36.320')] [2023-10-08 11:16:17,022][53852] Updated weights for policy 0, policy_version 88400 (0.0009) [2023-10-08 11:16:17,400][53852] Updated weights for policy 0, policy_version 88410 (0.0011) [2023-10-08 11:16:18,053][53885] Updated weights for policy 1, policy_version 87972 (0.0008) [2023-10-08 11:16:18,425][53885] Updated weights for policy 1, policy_version 87982 (0.0009) [2023-10-08 11:16:18,792][53885] Updated weights for policy 1, policy_version 87992 (0.0009) [2023-10-08 11:16:21,013][53852] Updated weights for policy 0, policy_version 88420 (0.0010) [2023-10-08 11:16:21,384][53852] Updated weights for policy 0, policy_version 88430 (0.0008) [2023-10-08 11:16:21,754][53852] Updated weights for policy 0, policy_version 88440 (0.0008) [2023-10-08 11:16:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 180649984. Throughput: 0: 1830.5, 1: 1835.9. Samples: 45179808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:22,016][52710] Avg episode reward: [(0, '30.270'), (1, '36.670')] [2023-10-08 11:16:22,422][53885] Updated weights for policy 1, policy_version 88002 (0.0009) [2023-10-08 11:16:22,794][53885] Updated weights for policy 1, policy_version 88012 (0.0007) [2023-10-08 11:16:23,162][53885] Updated weights for policy 1, policy_version 88022 (0.0007) [2023-10-08 11:16:23,520][53885] Updated weights for policy 1, policy_version 88032 (0.0007) [2023-10-08 11:16:25,382][53852] Updated weights for policy 0, policy_version 88450 (0.0007) [2023-10-08 11:16:25,739][53852] Updated weights for policy 0, policy_version 88460 (0.0007) [2023-10-08 11:16:26,118][53852] Updated weights for policy 0, policy_version 88470 (0.0007) [2023-10-08 11:16:26,489][53852] Updated weights for policy 0, policy_version 88480 (0.0009) [2023-10-08 11:16:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 180748288. Throughput: 0: 1847.7, 1: 1839.2. Samples: 45190950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:27,016][52710] Avg episode reward: [(0, '30.410'), (1, '34.650')] [2023-10-08 11:16:27,059][53885] Updated weights for policy 1, policy_version 88042 (0.0007) [2023-10-08 11:16:27,430][53885] Updated weights for policy 1, policy_version 88052 (0.0008) [2023-10-08 11:16:27,802][53885] Updated weights for policy 1, policy_version 88062 (0.0010) [2023-10-08 11:16:29,998][53852] Updated weights for policy 0, policy_version 88490 (0.0009) [2023-10-08 11:16:30,361][53852] Updated weights for policy 0, policy_version 88500 (0.0008) [2023-10-08 11:16:30,730][53852] Updated weights for policy 0, policy_version 88510 (0.0009) [2023-10-08 11:16:31,540][53885] Updated weights for policy 1, policy_version 88072 (0.0009) [2023-10-08 11:16:31,901][53885] Updated weights for policy 1, policy_version 88082 (0.0009) [2023-10-08 11:16:32,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180813824. Throughput: 0: 1834.8, 1: 1844.7. Samples: 45213134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:32,016][52710] Avg episode reward: [(0, '37.120'), (1, '32.700')] [2023-10-08 11:16:32,263][53885] Updated weights for policy 1, policy_version 88092 (0.0007) [2023-10-08 11:16:34,263][53852] Updated weights for policy 0, policy_version 88520 (0.0010) [2023-10-08 11:16:34,639][53852] Updated weights for policy 0, policy_version 88530 (0.0009) [2023-10-08 11:16:35,006][53852] Updated weights for policy 0, policy_version 88540 (0.0008) [2023-10-08 11:16:35,766][53885] Updated weights for policy 1, policy_version 88102 (0.0007) [2023-10-08 11:16:36,127][53885] Updated weights for policy 1, policy_version 88112 (0.0009) [2023-10-08 11:16:36,501][53885] Updated weights for policy 1, policy_version 88122 (0.0007) [2023-10-08 11:16:37,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 180912128. Throughput: 0: 1856.4, 1: 1823.3. Samples: 45234934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:37,016][52710] Avg episode reward: [(0, '34.060'), (1, '31.500')] [2023-10-08 11:16:38,685][53852] Updated weights for policy 0, policy_version 88550 (0.0010) [2023-10-08 11:16:39,048][53852] Updated weights for policy 0, policy_version 88560 (0.0009) [2023-10-08 11:16:39,418][53852] Updated weights for policy 0, policy_version 88570 (0.0008) [2023-10-08 11:16:40,117][53885] Updated weights for policy 1, policy_version 88132 (0.0007) [2023-10-08 11:16:40,478][53885] Updated weights for policy 1, policy_version 88142 (0.0008) [2023-10-08 11:16:40,851][53885] Updated weights for policy 1, policy_version 88152 (0.0008) [2023-10-08 11:16:42,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180977664. Throughput: 0: 1835.9, 1: 1841.0. Samples: 45246372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:42,016][52710] Avg episode reward: [(0, '32.320'), (1, '37.600')] [2023-10-08 11:16:43,000][53852] Updated weights for policy 0, policy_version 88580 (0.0008) [2023-10-08 11:16:43,369][53852] Updated weights for policy 0, policy_version 88590 (0.0009) [2023-10-08 11:16:43,743][53852] Updated weights for policy 0, policy_version 88600 (0.0009) [2023-10-08 11:16:44,536][53885] Updated weights for policy 1, policy_version 88162 (0.0008) [2023-10-08 11:16:44,902][53885] Updated weights for policy 1, policy_version 88172 (0.0008) [2023-10-08 11:16:45,268][53885] Updated weights for policy 1, policy_version 88182 (0.0011) [2023-10-08 11:16:45,632][53885] Updated weights for policy 1, policy_version 88192 (0.0010) [2023-10-08 11:16:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181043200. Throughput: 0: 1848.8, 1: 1826.1. Samples: 45267968. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:47,016][52710] Avg episode reward: [(0, '34.620'), (1, '35.510')] [2023-10-08 11:16:47,510][53852] Updated weights for policy 0, policy_version 88610 (0.0010) [2023-10-08 11:16:47,889][53852] Updated weights for policy 0, policy_version 88620 (0.0009) [2023-10-08 11:16:48,254][53852] Updated weights for policy 0, policy_version 88630 (0.0011) [2023-10-08 11:16:48,624][53852] Updated weights for policy 0, policy_version 88640 (0.0008) [2023-10-08 11:16:49,209][53885] Updated weights for policy 1, policy_version 88202 (0.0008) [2023-10-08 11:16:49,578][53885] Updated weights for policy 1, policy_version 88212 (0.0007) [2023-10-08 11:16:49,951][53885] Updated weights for policy 1, policy_version 88222 (0.0007) [2023-10-08 11:16:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181108736. Throughput: 0: 1844.9, 1: 1846.5. Samples: 45290992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:52,016][52710] Avg episode reward: [(0, '37.250'), (1, '34.510')] [2023-10-08 11:16:52,388][53852] Updated weights for policy 0, policy_version 88650 (0.0007) [2023-10-08 11:16:52,762][53852] Updated weights for policy 0, policy_version 88660 (0.0008) [2023-10-08 11:16:53,143][53852] Updated weights for policy 0, policy_version 88670 (0.0009) [2023-10-08 11:16:53,519][53885] Updated weights for policy 1, policy_version 88232 (0.0009) [2023-10-08 11:16:53,891][53885] Updated weights for policy 1, policy_version 88242 (0.0009) [2023-10-08 11:16:54,255][53885] Updated weights for policy 1, policy_version 88252 (0.0008) [2023-10-08 11:16:56,747][53852] Updated weights for policy 0, policy_version 88680 (0.0008) [2023-10-08 11:16:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 181174272. Throughput: 0: 1848.2, 1: 1826.3. Samples: 45300846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:16:57,015][52710] Avg episode reward: [(0, '33.800'), (1, '36.530')] [2023-10-08 11:16:57,115][53852] Updated weights for policy 0, policy_version 88690 (0.0008) [2023-10-08 11:16:57,469][53852] Updated weights for policy 0, policy_version 88700 (0.0009) [2023-10-08 11:16:57,792][53885] Updated weights for policy 1, policy_version 88262 (0.0008) [2023-10-08 11:16:58,153][53885] Updated weights for policy 1, policy_version 88272 (0.0008) [2023-10-08 11:16:58,523][53885] Updated weights for policy 1, policy_version 88282 (0.0010) [2023-10-08 11:17:01,111][53852] Updated weights for policy 0, policy_version 88710 (0.0009) [2023-10-08 11:17:01,479][53852] Updated weights for policy 0, policy_version 88720 (0.0007) [2023-10-08 11:17:01,861][53852] Updated weights for policy 0, policy_version 88730 (0.0009) [2023-10-08 11:17:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 181239808. Throughput: 0: 1843.4, 1: 1851.1. Samples: 45324048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:02,016][52710] Avg episode reward: [(0, '33.420'), (1, '36.900')] [2023-10-08 11:17:02,164][53885] Updated weights for policy 1, policy_version 88292 (0.0009) [2023-10-08 11:17:02,524][53885] Updated weights for policy 1, policy_version 88302 (0.0007) [2023-10-08 11:17:02,893][53885] Updated weights for policy 1, policy_version 88312 (0.0010) [2023-10-08 11:17:05,513][53852] Updated weights for policy 0, policy_version 88740 (0.0008) [2023-10-08 11:17:05,881][53852] Updated weights for policy 0, policy_version 88750 (0.0008) [2023-10-08 11:17:06,254][53852] Updated weights for policy 0, policy_version 88760 (0.0008) [2023-10-08 11:17:06,688][53885] Updated weights for policy 1, policy_version 88322 (0.0011) [2023-10-08 11:17:07,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 181338112. Throughput: 0: 1829.8, 1: 1855.6. Samples: 45345652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:07,016][52710] Avg episode reward: [(0, '35.260'), (1, '32.500')] [2023-10-08 11:17:07,054][53885] Updated weights for policy 1, policy_version 88332 (0.0009) [2023-10-08 11:17:07,422][53885] Updated weights for policy 1, policy_version 88342 (0.0007) [2023-10-08 11:17:07,794][53885] Updated weights for policy 1, policy_version 88352 (0.0009) [2023-10-08 11:17:09,897][53852] Updated weights for policy 0, policy_version 88770 (0.0008) [2023-10-08 11:17:10,269][53852] Updated weights for policy 0, policy_version 88780 (0.0009) [2023-10-08 11:17:10,633][53852] Updated weights for policy 0, policy_version 88790 (0.0007) [2023-10-08 11:17:10,998][53852] Updated weights for policy 0, policy_version 88800 (0.0007) [2023-10-08 11:17:11,590][53885] Updated weights for policy 1, policy_version 88362 (0.0009) [2023-10-08 11:17:11,959][53885] Updated weights for policy 1, policy_version 88372 (0.0007) [2023-10-08 11:17:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181403648. Throughput: 0: 1840.9, 1: 1851.2. Samples: 45357092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:12,016][52710] Avg episode reward: [(0, '32.410'), (1, '35.410')] [2023-10-08 11:17:12,312][53885] Updated weights for policy 1, policy_version 88382 (0.0007) [2023-10-08 11:17:14,611][53852] Updated weights for policy 0, policy_version 88810 (0.0007) [2023-10-08 11:17:14,983][53852] Updated weights for policy 0, policy_version 88820 (0.0008) [2023-10-08 11:17:15,342][53852] Updated weights for policy 0, policy_version 88830 (0.0007) [2023-10-08 11:17:16,075][53885] Updated weights for policy 1, policy_version 88392 (0.0008) [2023-10-08 11:17:16,438][53885] Updated weights for policy 1, policy_version 88402 (0.0008) [2023-10-08 11:17:16,816][53885] Updated weights for policy 1, policy_version 88412 (0.0009) [2023-10-08 11:17:17,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 181501952. Throughput: 0: 1830.6, 1: 1844.4. Samples: 45378508. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:17,016][52710] Avg episode reward: [(0, '31.130'), (1, '38.070')] [2023-10-08 11:17:18,917][53852] Updated weights for policy 0, policy_version 88840 (0.0010) [2023-10-08 11:17:19,287][53852] Updated weights for policy 0, policy_version 88850 (0.0009) [2023-10-08 11:17:19,653][53852] Updated weights for policy 0, policy_version 88860 (0.0007) [2023-10-08 11:17:20,457][53885] Updated weights for policy 1, policy_version 88422 (0.0009) [2023-10-08 11:17:20,825][53885] Updated weights for policy 1, policy_version 88432 (0.0009) [2023-10-08 11:17:21,200][53885] Updated weights for policy 1, policy_version 88442 (0.0010) [2023-10-08 11:17:22,015][52710] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 181567488. Throughput: 0: 1832.1, 1: 1837.7. Samples: 45400072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:22,015][52710] Avg episode reward: [(0, '32.350'), (1, '38.680')] [2023-10-08 11:17:23,263][53852] Updated weights for policy 0, policy_version 88870 (0.0009) [2023-10-08 11:17:23,639][53852] Updated weights for policy 0, policy_version 88880 (0.0009) [2023-10-08 11:17:24,007][53852] Updated weights for policy 0, policy_version 88890 (0.0008) [2023-10-08 11:17:24,834][53885] Updated weights for policy 1, policy_version 88452 (0.0008) [2023-10-08 11:17:25,194][53885] Updated weights for policy 1, policy_version 88462 (0.0010) [2023-10-08 11:17:25,562][53885] Updated weights for policy 1, policy_version 88472 (0.0010) [2023-10-08 11:17:27,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181633024. Throughput: 0: 1833.2, 1: 1841.6. Samples: 45411738. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:27,016][52710] Avg episode reward: [(0, '34.000'), (1, '35.260')] [2023-10-08 11:17:27,645][53852] Updated weights for policy 0, policy_version 88900 (0.0007) [2023-10-08 11:17:28,016][53852] Updated weights for policy 0, policy_version 88910 (0.0009) [2023-10-08 11:17:28,382][53852] Updated weights for policy 0, policy_version 88920 (0.0009) [2023-10-08 11:17:29,351][53885] Updated weights for policy 1, policy_version 88482 (0.0009) [2023-10-08 11:17:29,730][53885] Updated weights for policy 1, policy_version 88492 (0.0010) [2023-10-08 11:17:30,097][53885] Updated weights for policy 1, policy_version 88502 (0.0007) [2023-10-08 11:17:30,472][53885] Updated weights for policy 1, policy_version 88512 (0.0010) [2023-10-08 11:17:32,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181698560. Throughput: 0: 1838.5, 1: 1829.6. Samples: 45433030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:32,016][52710] Avg episode reward: [(0, '31.140'), (1, '35.920')] [2023-10-08 11:17:32,026][53852] Updated weights for policy 0, policy_version 88930 (0.0009) [2023-10-08 11:17:32,400][53852] Updated weights for policy 0, policy_version 88940 (0.0010) [2023-10-08 11:17:32,771][53852] Updated weights for policy 0, policy_version 88950 (0.0010) [2023-10-08 11:17:33,143][53852] Updated weights for policy 0, policy_version 88960 (0.0011) [2023-10-08 11:17:34,179][53885] Updated weights for policy 1, policy_version 88522 (0.0008) [2023-10-08 11:17:34,542][53885] Updated weights for policy 1, policy_version 88532 (0.0007) [2023-10-08 11:17:34,911][53885] Updated weights for policy 1, policy_version 88542 (0.0010) [2023-10-08 11:17:36,853][53852] Updated weights for policy 0, policy_version 88970 (0.0007) [2023-10-08 11:17:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 181764096. Throughput: 0: 1838.4, 1: 1831.9. Samples: 45456158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:37,016][52710] Avg episode reward: [(0, '32.570'), (1, '38.970')] [2023-10-08 11:17:37,222][53852] Updated weights for policy 0, policy_version 88980 (0.0009) [2023-10-08 11:17:37,592][53852] Updated weights for policy 0, policy_version 88990 (0.0007) [2023-10-08 11:17:38,553][53885] Updated weights for policy 1, policy_version 88552 (0.0009) [2023-10-08 11:17:38,937][53885] Updated weights for policy 1, policy_version 88562 (0.0009) [2023-10-08 11:17:39,307][53885] Updated weights for policy 1, policy_version 88572 (0.0008) [2023-10-08 11:17:41,251][53852] Updated weights for policy 0, policy_version 89000 (0.0008) [2023-10-08 11:17:41,627][53852] Updated weights for policy 0, policy_version 89010 (0.0008) [2023-10-08 11:17:42,005][53852] Updated weights for policy 0, policy_version 89020 (0.0009) [2023-10-08 11:17:42,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 181829632. Throughput: 0: 1845.1, 1: 1834.9. Samples: 45466446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:42,015][52710] Avg episode reward: [(0, '35.230'), (1, '33.660')] [2023-10-08 11:17:42,909][53885] Updated weights for policy 1, policy_version 88582 (0.0007) [2023-10-08 11:17:43,277][53885] Updated weights for policy 1, policy_version 88592 (0.0009) [2023-10-08 11:17:43,651][53885] Updated weights for policy 1, policy_version 88602 (0.0008) [2023-10-08 11:17:45,507][53852] Updated weights for policy 0, policy_version 89030 (0.0009) [2023-10-08 11:17:45,874][53852] Updated weights for policy 0, policy_version 89040 (0.0009) [2023-10-08 11:17:46,260][53852] Updated weights for policy 0, policy_version 89050 (0.0010) [2023-10-08 11:17:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 181927936. Throughput: 0: 1841.6, 1: 1827.4. Samples: 45489150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:47,016][52710] Avg episode reward: [(0, '33.800'), (1, '31.960')] [2023-10-08 11:17:47,220][53885] Updated weights for policy 1, policy_version 88612 (0.0010) [2023-10-08 11:17:47,596][53885] Updated weights for policy 1, policy_version 88622 (0.0007) [2023-10-08 11:17:47,955][53885] Updated weights for policy 1, policy_version 88632 (0.0011) [2023-10-08 11:17:49,847][53852] Updated weights for policy 0, policy_version 89060 (0.0010) [2023-10-08 11:17:50,224][53852] Updated weights for policy 0, policy_version 89070 (0.0008) [2023-10-08 11:17:50,592][53852] Updated weights for policy 0, policy_version 89080 (0.0009) [2023-10-08 11:17:51,657][53885] Updated weights for policy 1, policy_version 88642 (0.0010) [2023-10-08 11:17:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 181993472. Throughput: 0: 1852.1, 1: 1827.0. Samples: 45511212. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:52,016][52710] Avg episode reward: [(0, '32.280'), (1, '33.740')] [2023-10-08 11:17:52,023][53885] Updated weights for policy 1, policy_version 88652 (0.0009) [2023-10-08 11:17:52,399][53885] Updated weights for policy 1, policy_version 88662 (0.0010) [2023-10-08 11:17:52,767][53885] Updated weights for policy 1, policy_version 88672 (0.0010) [2023-10-08 11:17:54,181][53852] Updated weights for policy 0, policy_version 89090 (0.0008) [2023-10-08 11:17:54,548][53852] Updated weights for policy 0, policy_version 89100 (0.0008) [2023-10-08 11:17:54,923][53852] Updated weights for policy 0, policy_version 89110 (0.0008) [2023-10-08 11:17:55,290][53852] Updated weights for policy 0, policy_version 89120 (0.0010) [2023-10-08 11:17:56,581][53885] Updated weights for policy 1, policy_version 88682 (0.0011) [2023-10-08 11:17:56,941][53885] Updated weights for policy 1, policy_version 88692 (0.0009) [2023-10-08 11:17:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 182059008. Throughput: 0: 1841.3, 1: 1828.7. Samples: 45522242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:17:57,016][52710] Avg episode reward: [(0, '35.850'), (1, '33.390')] [2023-10-08 11:17:57,312][53885] Updated weights for policy 1, policy_version 88702 (0.0009) [2023-10-08 11:17:58,886][53852] Updated weights for policy 0, policy_version 89130 (0.0008) [2023-10-08 11:17:59,248][53852] Updated weights for policy 0, policy_version 89140 (0.0008) [2023-10-08 11:17:59,625][53852] Updated weights for policy 0, policy_version 89150 (0.0007) [2023-10-08 11:18:00,838][53885] Updated weights for policy 1, policy_version 88712 (0.0009) [2023-10-08 11:18:01,204][53885] Updated weights for policy 1, policy_version 88722 (0.0010) [2023-10-08 11:18:01,570][53885] Updated weights for policy 1, policy_version 88732 (0.0011) [2023-10-08 11:18:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 182157312. Throughput: 0: 1852.8, 1: 1831.7. Samples: 45544312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:18:02,016][52710] Avg episode reward: [(0, '34.220'), (1, '29.950')] [2023-10-08 11:18:03,311][53852] Updated weights for policy 0, policy_version 89160 (0.0007) [2023-10-08 11:18:03,681][53852] Updated weights for policy 0, policy_version 89170 (0.0007) [2023-10-08 11:18:04,053][53852] Updated weights for policy 0, policy_version 89180 (0.0007) [2023-10-08 11:18:05,188][53885] Updated weights for policy 1, policy_version 88742 (0.0009) [2023-10-08 11:18:05,563][53885] Updated weights for policy 1, policy_version 88752 (0.0009) [2023-10-08 11:18:05,928][53885] Updated weights for policy 1, policy_version 88762 (0.0007) [2023-10-08 11:18:07,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182222848. Throughput: 0: 1850.6, 1: 1832.4. Samples: 45565806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:18:07,016][52710] Avg episode reward: [(0, '32.630'), (1, '36.740')] [2023-10-08 11:18:07,026][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000088768_90898432.pth... [2023-10-08 11:18:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000089184_91324416.pth... [2023-10-08 11:18:07,063][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000087488_89587712.pth [2023-10-08 11:18:07,068][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000087040_89128960.pth [2023-10-08 11:18:07,640][53852] Updated weights for policy 0, policy_version 89190 (0.0011) [2023-10-08 11:18:08,011][53852] Updated weights for policy 0, policy_version 89200 (0.0007) [2023-10-08 11:18:08,388][53852] Updated weights for policy 0, policy_version 89210 (0.0008) [2023-10-08 11:18:09,649][53885] Updated weights for policy 1, policy_version 88772 (0.0007) [2023-10-08 11:18:10,023][53885] Updated weights for policy 1, policy_version 88782 (0.0007) [2023-10-08 11:18:10,392][53885] Updated weights for policy 1, policy_version 88792 (0.0007) [2023-10-08 11:18:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182288384. Throughput: 0: 1849.0, 1: 1828.7. Samples: 45577236. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:18:12,016][52710] Avg episode reward: [(0, '35.030'), (1, '34.010')] [2023-10-08 11:18:12,065][53852] Updated weights for policy 0, policy_version 89220 (0.0008) [2023-10-08 11:18:12,432][53852] Updated weights for policy 0, policy_version 89230 (0.0008) [2023-10-08 11:18:12,801][53852] Updated weights for policy 0, policy_version 89240 (0.0008) [2023-10-08 11:18:14,158][53885] Updated weights for policy 1, policy_version 88802 (0.0009) [2023-10-08 11:18:14,527][53885] Updated weights for policy 1, policy_version 88812 (0.0007) [2023-10-08 11:18:14,893][53885] Updated weights for policy 1, policy_version 88822 (0.0007) [2023-10-08 11:18:15,255][53885] Updated weights for policy 1, policy_version 88832 (0.0008) [2023-10-08 11:18:16,353][53852] Updated weights for policy 0, policy_version 89250 (0.0008) [2023-10-08 11:18:16,723][53852] Updated weights for policy 0, policy_version 89260 (0.0009) [2023-10-08 11:18:17,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 182353920. Throughput: 0: 1851.6, 1: 1830.4. Samples: 45598722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:18:17,016][52710] Avg episode reward: [(0, '35.240'), (1, '37.470')] [2023-10-08 11:18:17,103][53852] Updated weights for policy 0, policy_version 89270 (0.0007) [2023-10-08 11:18:17,469][53852] Updated weights for policy 0, policy_version 89280 (0.0007) [2023-10-08 11:18:18,942][53885] Updated weights for policy 1, policy_version 88842 (0.0007) [2023-10-08 11:18:19,305][53885] Updated weights for policy 1, policy_version 88852 (0.0007) [2023-10-08 11:18:19,670][53885] Updated weights for policy 1, policy_version 88862 (0.0011) [2023-10-08 11:18:21,160][53852] Updated weights for policy 0, policy_version 89290 (0.0007) [2023-10-08 11:18:21,526][53852] Updated weights for policy 0, policy_version 89300 (0.0007) [2023-10-08 11:18:21,899][53852] Updated weights for policy 0, policy_version 89310 (0.0008) [2023-10-08 11:18:22,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 182452224. Throughput: 0: 1832.5, 1: 1829.2. Samples: 45620936. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:18:22,016][52710] Avg episode reward: [(0, '30.750'), (1, '36.000')] [2023-10-08 11:18:23,463][53885] Updated weights for policy 1, policy_version 88872 (0.0008) [2023-10-08 11:18:23,828][53885] Updated weights for policy 1, policy_version 88882 (0.0007) [2023-10-08 11:18:24,198][53885] Updated weights for policy 1, policy_version 88892 (0.0007) [2023-10-08 11:18:25,531][53852] Updated weights for policy 0, policy_version 89320 (0.0009) [2023-10-08 11:18:25,900][53852] Updated weights for policy 0, policy_version 89330 (0.0008) [2023-10-08 11:18:26,278][53852] Updated weights for policy 0, policy_version 89340 (0.0009) [2023-10-08 11:18:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 182517760. Throughput: 0: 1850.8, 1: 1827.4. Samples: 45631966. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:18:27,016][52710] Avg episode reward: [(0, '33.610'), (1, '35.210')] [2023-10-08 11:18:27,955][53885] Updated weights for policy 1, policy_version 88902 (0.0009) [2023-10-08 11:18:28,317][53885] Updated weights for policy 1, policy_version 88912 (0.0011) [2023-10-08 11:18:28,691][53885] Updated weights for policy 1, policy_version 88922 (0.0009) [2023-10-08 11:18:29,916][53852] Updated weights for policy 0, policy_version 89350 (0.0007) [2023-10-08 11:18:30,280][53852] Updated weights for policy 0, policy_version 89360 (0.0007) [2023-10-08 11:18:30,655][53852] Updated weights for policy 0, policy_version 89370 (0.0008) [2023-10-08 11:18:32,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 182583296. Throughput: 0: 1832.3, 1: 1828.6. Samples: 45653890. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:18:32,015][52710] Avg episode reward: [(0, '35.640'), (1, '33.150')] [2023-10-08 11:18:32,325][53885] Updated weights for policy 1, policy_version 88932 (0.0008) [2023-10-08 11:18:32,688][53885] Updated weights for policy 1, policy_version 88942 (0.0008) [2023-10-08 11:18:33,053][53885] Updated weights for policy 1, policy_version 88952 (0.0009) [2023-10-08 11:18:34,128][53852] Updated weights for policy 0, policy_version 89380 (0.0009) [2023-10-08 11:18:34,501][53852] Updated weights for policy 0, policy_version 89390 (0.0007) [2023-10-08 11:18:34,878][53852] Updated weights for policy 0, policy_version 89400 (0.0007) [2023-10-08 11:18:36,626][53885] Updated weights for policy 1, policy_version 88962 (0.0010) [2023-10-08 11:18:36,998][53885] Updated weights for policy 1, policy_version 88972 (0.0010) [2023-10-08 11:18:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182648832. Throughput: 0: 1850.9, 1: 1827.0. Samples: 45676718. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:18:37,015][52710] Avg episode reward: [(0, '34.340'), (1, '39.190')] [2023-10-08 11:18:37,372][53885] Updated weights for policy 1, policy_version 88982 (0.0008) [2023-10-08 11:18:37,744][53885] Updated weights for policy 1, policy_version 88992 (0.0010) [2023-10-08 11:18:38,504][53852] Updated weights for policy 0, policy_version 89410 (0.0007) [2023-10-08 11:18:38,872][53852] Updated weights for policy 0, policy_version 89420 (0.0007) [2023-10-08 11:18:39,239][53852] Updated weights for policy 0, policy_version 89430 (0.0008) [2023-10-08 11:18:39,608][53852] Updated weights for policy 0, policy_version 89440 (0.0010) [2023-10-08 11:18:41,426][53885] Updated weights for policy 1, policy_version 89002 (0.0008) [2023-10-08 11:18:41,793][53885] Updated weights for policy 1, policy_version 89012 (0.0007) [2023-10-08 11:18:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182714368. Throughput: 0: 1833.8, 1: 1828.1. Samples: 45687028. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:18:42,016][52710] Avg episode reward: [(0, '32.980'), (1, '35.900')] [2023-10-08 11:18:42,159][53885] Updated weights for policy 1, policy_version 89022 (0.0008) [2023-10-08 11:18:43,394][53852] Updated weights for policy 0, policy_version 89450 (0.0007) [2023-10-08 11:18:43,769][53852] Updated weights for policy 0, policy_version 89460 (0.0008) [2023-10-08 11:18:44,134][53852] Updated weights for policy 0, policy_version 89470 (0.0010) [2023-10-08 11:18:45,679][53885] Updated weights for policy 1, policy_version 89032 (0.0009) [2023-10-08 11:18:46,049][53885] Updated weights for policy 1, policy_version 89042 (0.0010) [2023-10-08 11:18:46,429][53885] Updated weights for policy 1, policy_version 89052 (0.0009) [2023-10-08 11:18:47,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182812672. Throughput: 0: 1847.2, 1: 1826.6. Samples: 45709634. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:18:47,016][52710] Avg episode reward: [(0, '36.330'), (1, '38.180')] [2023-10-08 11:18:47,721][53852] Updated weights for policy 0, policy_version 89480 (0.0009) [2023-10-08 11:18:48,093][53852] Updated weights for policy 0, policy_version 89490 (0.0007) [2023-10-08 11:18:48,452][53852] Updated weights for policy 0, policy_version 89500 (0.0007) [2023-10-08 11:18:50,045][53885] Updated weights for policy 1, policy_version 89062 (0.0007) [2023-10-08 11:18:50,422][53885] Updated weights for policy 1, policy_version 89072 (0.0007) [2023-10-08 11:18:50,795][53885] Updated weights for policy 1, policy_version 89082 (0.0008) [2023-10-08 11:18:52,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182878208. Throughput: 0: 1853.4, 1: 1832.8. Samples: 45731686. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:18:52,015][52710] Avg episode reward: [(0, '33.310'), (1, '36.360')] [2023-10-08 11:18:52,108][53852] Updated weights for policy 0, policy_version 89510 (0.0010) [2023-10-08 11:18:52,475][53852] Updated weights for policy 0, policy_version 89520 (0.0008) [2023-10-08 11:18:52,846][53852] Updated weights for policy 0, policy_version 89530 (0.0007) [2023-10-08 11:18:54,380][53885] Updated weights for policy 1, policy_version 89092 (0.0007) [2023-10-08 11:18:54,740][53885] Updated weights for policy 1, policy_version 89102 (0.0007) [2023-10-08 11:18:55,102][53885] Updated weights for policy 1, policy_version 89112 (0.0009) [2023-10-08 11:18:56,430][53852] Updated weights for policy 0, policy_version 89540 (0.0007) [2023-10-08 11:18:56,802][53852] Updated weights for policy 0, policy_version 89550 (0.0007) [2023-10-08 11:18:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182943744. Throughput: 0: 1853.8, 1: 1829.5. Samples: 45742984. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:18:57,017][52710] Avg episode reward: [(0, '31.380'), (1, '37.320')] [2023-10-08 11:18:57,159][53852] Updated weights for policy 0, policy_version 89560 (0.0007) [2023-10-08 11:18:58,815][53885] Updated weights for policy 1, policy_version 89122 (0.0009) [2023-10-08 11:18:59,186][53885] Updated weights for policy 1, policy_version 89132 (0.0007) [2023-10-08 11:18:59,556][53885] Updated weights for policy 1, policy_version 89142 (0.0009) [2023-10-08 11:18:59,930][53885] Updated weights for policy 1, policy_version 89152 (0.0009) [2023-10-08 11:19:00,769][53852] Updated weights for policy 0, policy_version 89570 (0.0009) [2023-10-08 11:19:01,137][53852] Updated weights for policy 0, policy_version 89580 (0.0007) [2023-10-08 11:19:01,509][53852] Updated weights for policy 0, policy_version 89590 (0.0007) [2023-10-08 11:19:01,868][53852] Updated weights for policy 0, policy_version 89600 (0.0008) [2023-10-08 11:19:02,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 183042048. Throughput: 0: 1855.5, 1: 1845.3. Samples: 45765258. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:19:02,016][52710] Avg episode reward: [(0, '34.350'), (1, '37.460')] [2023-10-08 11:19:03,486][53885] Updated weights for policy 1, policy_version 89162 (0.0007) [2023-10-08 11:19:03,852][53885] Updated weights for policy 1, policy_version 89172 (0.0007) [2023-10-08 11:19:04,227][53885] Updated weights for policy 1, policy_version 89182 (0.0007) [2023-10-08 11:19:05,375][53852] Updated weights for policy 0, policy_version 89610 (0.0009) [2023-10-08 11:19:05,743][53852] Updated weights for policy 0, policy_version 89620 (0.0008) [2023-10-08 11:19:06,112][53852] Updated weights for policy 0, policy_version 89630 (0.0007) [2023-10-08 11:19:07,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 183107584. Throughput: 0: 1845.2, 1: 1846.2. Samples: 45787046. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:19:07,016][52710] Avg episode reward: [(0, '31.620'), (1, '34.350')] [2023-10-08 11:19:07,869][53885] Updated weights for policy 1, policy_version 89192 (0.0007) [2023-10-08 11:19:08,247][53885] Updated weights for policy 1, policy_version 89202 (0.0008) [2023-10-08 11:19:08,613][53885] Updated weights for policy 1, policy_version 89212 (0.0010) [2023-10-08 11:19:09,781][53852] Updated weights for policy 0, policy_version 89640 (0.0007) [2023-10-08 11:19:10,151][53852] Updated weights for policy 0, policy_version 89650 (0.0007) [2023-10-08 11:19:10,514][53852] Updated weights for policy 0, policy_version 89660 (0.0009) [2023-10-08 11:19:12,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 183173120. Throughput: 0: 1853.7, 1: 1841.1. Samples: 45798230. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:19:12,016][52710] Avg episode reward: [(0, '33.370'), (1, '33.480')] [2023-10-08 11:19:12,201][53885] Updated weights for policy 1, policy_version 89222 (0.0008) [2023-10-08 11:19:12,564][53885] Updated weights for policy 1, policy_version 89232 (0.0009) [2023-10-08 11:19:12,935][53885] Updated weights for policy 1, policy_version 89242 (0.0009) [2023-10-08 11:19:14,268][53852] Updated weights for policy 0, policy_version 89670 (0.0008) [2023-10-08 11:19:14,632][53852] Updated weights for policy 0, policy_version 89680 (0.0007) [2023-10-08 11:19:14,999][53852] Updated weights for policy 0, policy_version 89690 (0.0007) [2023-10-08 11:19:16,778][53885] Updated weights for policy 1, policy_version 89252 (0.0007) [2023-10-08 11:19:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183238656. Throughput: 0: 1842.3, 1: 1842.2. Samples: 45819692. Policy #0 lag: (min: 0.0, avg: 14.1, max: 32.0) [2023-10-08 11:19:17,016][52710] Avg episode reward: [(0, '33.130'), (1, '35.170')] [2023-10-08 11:19:17,136][53885] Updated weights for policy 1, policy_version 89262 (0.0007) [2023-10-08 11:19:17,516][53885] Updated weights for policy 1, policy_version 89272 (0.0007) [2023-10-08 11:19:18,798][53852] Updated weights for policy 0, policy_version 89700 (0.0009) [2023-10-08 11:19:19,161][53852] Updated weights for policy 0, policy_version 89710 (0.0009) [2023-10-08 11:19:19,542][53852] Updated weights for policy 0, policy_version 89720 (0.0009) [2023-10-08 11:19:21,229][53885] Updated weights for policy 1, policy_version 89282 (0.0007) [2023-10-08 11:19:21,595][53885] Updated weights for policy 1, policy_version 89292 (0.0010) [2023-10-08 11:19:21,972][53885] Updated weights for policy 1, policy_version 89302 (0.0008) [2023-10-08 11:19:22,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 183304192. Throughput: 0: 1846.4, 1: 1826.1. Samples: 45841982. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:19:22,016][52710] Avg episode reward: [(0, '33.430'), (1, '35.550')] [2023-10-08 11:19:22,338][53885] Updated weights for policy 1, policy_version 89312 (0.0007) [2023-10-08 11:19:23,188][53852] Updated weights for policy 0, policy_version 89730 (0.0008) [2023-10-08 11:19:23,566][53852] Updated weights for policy 0, policy_version 89740 (0.0009) [2023-10-08 11:19:23,928][53852] Updated weights for policy 0, policy_version 89750 (0.0007) [2023-10-08 11:19:24,297][53852] Updated weights for policy 0, policy_version 89760 (0.0007) [2023-10-08 11:19:25,796][53885] Updated weights for policy 1, policy_version 89322 (0.0011) [2023-10-08 11:19:26,164][53885] Updated weights for policy 1, policy_version 89332 (0.0009) [2023-10-08 11:19:26,528][53885] Updated weights for policy 1, policy_version 89342 (0.0010) [2023-10-08 11:19:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183402496. Throughput: 0: 1841.6, 1: 1840.6. Samples: 45852728. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:19:27,016][52710] Avg episode reward: [(0, '34.670'), (1, '36.920')] [2023-10-08 11:19:27,934][53852] Updated weights for policy 0, policy_version 89770 (0.0008) [2023-10-08 11:19:28,311][53852] Updated weights for policy 0, policy_version 89780 (0.0007) [2023-10-08 11:19:28,683][53852] Updated weights for policy 0, policy_version 89790 (0.0007) [2023-10-08 11:19:30,073][53885] Updated weights for policy 1, policy_version 89352 (0.0009) [2023-10-08 11:19:30,445][53885] Updated weights for policy 1, policy_version 89362 (0.0008) [2023-10-08 11:19:30,816][53885] Updated weights for policy 1, policy_version 89372 (0.0008) [2023-10-08 11:19:32,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183468032. Throughput: 0: 1848.4, 1: 1821.7. Samples: 45874788. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:19:32,016][52710] Avg episode reward: [(0, '34.050'), (1, '37.340')] [2023-10-08 11:19:32,383][53852] Updated weights for policy 0, policy_version 89800 (0.0008) [2023-10-08 11:19:32,752][53852] Updated weights for policy 0, policy_version 89810 (0.0007) [2023-10-08 11:19:33,128][53852] Updated weights for policy 0, policy_version 89820 (0.0007) [2023-10-08 11:19:34,458][53885] Updated weights for policy 1, policy_version 89382 (0.0007) [2023-10-08 11:19:34,834][53885] Updated weights for policy 1, policy_version 89392 (0.0007) [2023-10-08 11:19:35,201][53885] Updated weights for policy 1, policy_version 89402 (0.0008) [2023-10-08 11:19:36,763][53852] Updated weights for policy 0, policy_version 89830 (0.0007) [2023-10-08 11:19:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183533568. Throughput: 0: 1841.9, 1: 1837.3. Samples: 45897252. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:19:37,016][52710] Avg episode reward: [(0, '32.870'), (1, '38.560')] [2023-10-08 11:19:37,130][53852] Updated weights for policy 0, policy_version 89840 (0.0008) [2023-10-08 11:19:37,506][53852] Updated weights for policy 0, policy_version 89850 (0.0007) [2023-10-08 11:19:38,836][53885] Updated weights for policy 1, policy_version 89412 (0.0010) [2023-10-08 11:19:39,206][53885] Updated weights for policy 1, policy_version 89422 (0.0010) [2023-10-08 11:19:39,574][53885] Updated weights for policy 1, policy_version 89432 (0.0008) [2023-10-08 11:19:41,065][53852] Updated weights for policy 0, policy_version 89860 (0.0008) [2023-10-08 11:19:41,436][53852] Updated weights for policy 0, policy_version 89870 (0.0008) [2023-10-08 11:19:41,797][53852] Updated weights for policy 0, policy_version 89880 (0.0007) [2023-10-08 11:19:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183599104. Throughput: 0: 1840.1, 1: 1824.7. Samples: 45907896. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:19:42,015][52710] Avg episode reward: [(0, '35.280'), (1, '36.530')] [2023-10-08 11:19:43,401][53885] Updated weights for policy 1, policy_version 89442 (0.0008) [2023-10-08 11:19:43,761][53885] Updated weights for policy 1, policy_version 89452 (0.0010) [2023-10-08 11:19:44,134][53885] Updated weights for policy 1, policy_version 89462 (0.0009) [2023-10-08 11:19:44,499][53885] Updated weights for policy 1, policy_version 89472 (0.0010) [2023-10-08 11:19:45,317][53852] Updated weights for policy 0, policy_version 89890 (0.0011) [2023-10-08 11:19:45,676][53852] Updated weights for policy 0, policy_version 89900 (0.0010) [2023-10-08 11:19:46,048][53852] Updated weights for policy 0, policy_version 89910 (0.0007) [2023-10-08 11:19:46,420][53852] Updated weights for policy 0, policy_version 89920 (0.0008) [2023-10-08 11:19:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 183697408. Throughput: 0: 1829.9, 1: 1835.6. Samples: 45930202. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:19:47,016][52710] Avg episode reward: [(0, '34.830'), (1, '38.190')] [2023-10-08 11:19:48,315][53885] Updated weights for policy 1, policy_version 89482 (0.0009) [2023-10-08 11:19:48,685][53885] Updated weights for policy 1, policy_version 89492 (0.0010) [2023-10-08 11:19:49,062][53885] Updated weights for policy 1, policy_version 89502 (0.0009) [2023-10-08 11:19:50,107][53852] Updated weights for policy 0, policy_version 89930 (0.0008) [2023-10-08 11:19:50,477][53852] Updated weights for policy 0, policy_version 89940 (0.0008) [2023-10-08 11:19:50,849][53852] Updated weights for policy 0, policy_version 89950 (0.0007) [2023-10-08 11:19:52,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 183762944. Throughput: 0: 1834.2, 1: 1832.1. Samples: 45952030. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:19:52,016][52710] Avg episode reward: [(0, '34.600'), (1, '38.750')] [2023-10-08 11:19:52,599][53885] Updated weights for policy 1, policy_version 89512 (0.0008) [2023-10-08 11:19:52,970][53885] Updated weights for policy 1, policy_version 89522 (0.0008) [2023-10-08 11:19:53,335][53885] Updated weights for policy 1, policy_version 89532 (0.0007) [2023-10-08 11:19:54,386][53852] Updated weights for policy 0, policy_version 89960 (0.0007) [2023-10-08 11:19:54,763][53852] Updated weights for policy 0, policy_version 89970 (0.0007) [2023-10-08 11:19:55,133][53852] Updated weights for policy 0, policy_version 89980 (0.0009) [2023-10-08 11:19:57,011][53885] Updated weights for policy 1, policy_version 89542 (0.0009) [2023-10-08 11:19:57,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183828480. Throughput: 0: 1827.5, 1: 1836.3. Samples: 45963102. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:19:57,016][52710] Avg episode reward: [(0, '36.210'), (1, '37.350')] [2023-10-08 11:19:57,374][53885] Updated weights for policy 1, policy_version 89552 (0.0007) [2023-10-08 11:19:57,738][53885] Updated weights for policy 1, policy_version 89562 (0.0009) [2023-10-08 11:19:58,762][53852] Updated weights for policy 0, policy_version 89990 (0.0010) [2023-10-08 11:19:59,130][53852] Updated weights for policy 0, policy_version 90000 (0.0007) [2023-10-08 11:19:59,500][53852] Updated weights for policy 0, policy_version 90010 (0.0010) [2023-10-08 11:20:01,367][53885] Updated weights for policy 1, policy_version 89572 (0.0008) [2023-10-08 11:20:01,737][53885] Updated weights for policy 1, policy_version 89582 (0.0011) [2023-10-08 11:20:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 183894016. Throughput: 0: 1843.3, 1: 1835.7. Samples: 45985248. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:20:02,016][52710] Avg episode reward: [(0, '38.270'), (1, '35.440')] [2023-10-08 11:20:02,017][53500] Saving new best policy, reward=38.270! [2023-10-08 11:20:02,103][53885] Updated weights for policy 1, policy_version 89592 (0.0008) [2023-10-08 11:20:03,243][53852] Updated weights for policy 0, policy_version 90020 (0.0009) [2023-10-08 11:20:03,620][53852] Updated weights for policy 0, policy_version 90030 (0.0007) [2023-10-08 11:20:03,992][53852] Updated weights for policy 0, policy_version 90040 (0.0008) [2023-10-08 11:20:05,827][53885] Updated weights for policy 1, policy_version 89602 (0.0010) [2023-10-08 11:20:06,193][53885] Updated weights for policy 1, policy_version 89612 (0.0007) [2023-10-08 11:20:06,553][53885] Updated weights for policy 1, policy_version 89622 (0.0009) [2023-10-08 11:20:06,920][53885] Updated weights for policy 1, policy_version 89632 (0.0007) [2023-10-08 11:20:07,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 183992320. Throughput: 0: 1842.1, 1: 1827.4. Samples: 46007110. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:20:07,016][52710] Avg episode reward: [(0, '35.290'), (1, '37.420')] [2023-10-08 11:20:07,026][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000089632_91783168.pth... [2023-10-08 11:20:07,026][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000090048_92209152.pth... [2023-10-08 11:20:07,057][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000087904_90013696.pth [2023-10-08 11:20:07,061][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000088320_90439680.pth [2023-10-08 11:20:07,685][53852] Updated weights for policy 0, policy_version 90050 (0.0010) [2023-10-08 11:20:08,085][53852] Updated weights for policy 0, policy_version 90060 (0.0009) [2023-10-08 11:20:08,464][53852] Updated weights for policy 0, policy_version 90070 (0.0009) [2023-10-08 11:20:08,836][53852] Updated weights for policy 0, policy_version 90080 (0.0010) [2023-10-08 11:20:10,648][53885] Updated weights for policy 1, policy_version 89642 (0.0009) [2023-10-08 11:20:11,015][53885] Updated weights for policy 1, policy_version 89652 (0.0010) [2023-10-08 11:20:11,387][53885] Updated weights for policy 1, policy_version 89662 (0.0007) [2023-10-08 11:20:12,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184057856. Throughput: 0: 1838.6, 1: 1836.3. Samples: 46018096. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:20:12,016][52710] Avg episode reward: [(0, '35.430'), (1, '37.020')] [2023-10-08 11:20:12,521][53852] Updated weights for policy 0, policy_version 90090 (0.0008) [2023-10-08 11:20:12,892][53852] Updated weights for policy 0, policy_version 90100 (0.0008) [2023-10-08 11:20:13,266][53852] Updated weights for policy 0, policy_version 90110 (0.0009) [2023-10-08 11:20:15,043][53885] Updated weights for policy 1, policy_version 89672 (0.0008) [2023-10-08 11:20:15,402][53885] Updated weights for policy 1, policy_version 89682 (0.0007) [2023-10-08 11:20:15,774][53885] Updated weights for policy 1, policy_version 89692 (0.0007) [2023-10-08 11:20:16,901][53852] Updated weights for policy 0, policy_version 90120 (0.0009) [2023-10-08 11:20:17,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184123392. Throughput: 0: 1838.1, 1: 1831.5. Samples: 46039918. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) [2023-10-08 11:20:17,016][52710] Avg episode reward: [(0, '38.040'), (1, '34.280')] [2023-10-08 11:20:17,262][53852] Updated weights for policy 0, policy_version 90130 (0.0010) [2023-10-08 11:20:17,638][53852] Updated weights for policy 0, policy_version 90140 (0.0007) [2023-10-08 11:20:19,434][53885] Updated weights for policy 1, policy_version 89702 (0.0008) [2023-10-08 11:20:19,809][53885] Updated weights for policy 1, policy_version 89712 (0.0007) [2023-10-08 11:20:20,174][53885] Updated weights for policy 1, policy_version 89722 (0.0008) [2023-10-08 11:20:21,365][53852] Updated weights for policy 0, policy_version 90150 (0.0007) [2023-10-08 11:20:21,730][53852] Updated weights for policy 0, policy_version 90160 (0.0007) [2023-10-08 11:20:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184188928. Throughput: 0: 1824.6, 1: 1829.6. Samples: 46061692. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:20:22,016][52710] Avg episode reward: [(0, '36.100'), (1, '36.350')] [2023-10-08 11:20:22,099][53852] Updated weights for policy 0, policy_version 90170 (0.0009) [2023-10-08 11:20:23,889][53885] Updated weights for policy 1, policy_version 89732 (0.0009) [2023-10-08 11:20:24,254][53885] Updated weights for policy 1, policy_version 89742 (0.0007) [2023-10-08 11:20:24,619][53885] Updated weights for policy 1, policy_version 89752 (0.0007) [2023-10-08 11:20:25,671][53852] Updated weights for policy 0, policy_version 90180 (0.0008) [2023-10-08 11:20:26,043][53852] Updated weights for policy 0, policy_version 90190 (0.0009) [2023-10-08 11:20:26,415][53852] Updated weights for policy 0, policy_version 90200 (0.0009) [2023-10-08 11:20:27,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 184287232. Throughput: 0: 1836.9, 1: 1822.6. Samples: 46072574. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:20:27,015][52710] Avg episode reward: [(0, '34.400'), (1, '35.450')] [2023-10-08 11:20:28,351][53885] Updated weights for policy 1, policy_version 89762 (0.0007) [2023-10-08 11:20:28,731][53885] Updated weights for policy 1, policy_version 89772 (0.0009) [2023-10-08 11:20:29,098][53885] Updated weights for policy 1, policy_version 89782 (0.0008) [2023-10-08 11:20:29,464][53885] Updated weights for policy 1, policy_version 89792 (0.0009) [2023-10-08 11:20:29,919][53852] Updated weights for policy 0, policy_version 90210 (0.0010) [2023-10-08 11:20:30,280][53852] Updated weights for policy 0, policy_version 90220 (0.0007) [2023-10-08 11:20:30,650][53852] Updated weights for policy 0, policy_version 90230 (0.0007) [2023-10-08 11:20:31,022][53852] Updated weights for policy 0, policy_version 90240 (0.0009) [2023-10-08 11:20:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 184352768. Throughput: 0: 1826.9, 1: 1827.1. Samples: 46094634. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:20:32,015][52710] Avg episode reward: [(0, '35.990'), (1, '38.350')] [2023-10-08 11:20:33,134][53885] Updated weights for policy 1, policy_version 89802 (0.0008) [2023-10-08 11:20:33,507][53885] Updated weights for policy 1, policy_version 89812 (0.0008) [2023-10-08 11:20:33,879][53885] Updated weights for policy 1, policy_version 89822 (0.0009) [2023-10-08 11:20:34,755][53852] Updated weights for policy 0, policy_version 90250 (0.0009) [2023-10-08 11:20:35,123][53852] Updated weights for policy 0, policy_version 90260 (0.0007) [2023-10-08 11:20:35,496][53852] Updated weights for policy 0, policy_version 90270 (0.0010) [2023-10-08 11:20:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184418304. Throughput: 0: 1836.3, 1: 1822.1. Samples: 46116656. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:20:37,016][52710] Avg episode reward: [(0, '34.120'), (1, '35.820')] [2023-10-08 11:20:37,566][53885] Updated weights for policy 1, policy_version 89832 (0.0009) [2023-10-08 11:20:37,935][53885] Updated weights for policy 1, policy_version 89842 (0.0009) [2023-10-08 11:20:38,308][53885] Updated weights for policy 1, policy_version 89852 (0.0009) [2023-10-08 11:20:39,175][53852] Updated weights for policy 0, policy_version 90280 (0.0008) [2023-10-08 11:20:39,552][53852] Updated weights for policy 0, policy_version 90290 (0.0009) [2023-10-08 11:20:39,912][53852] Updated weights for policy 0, policy_version 90300 (0.0008) [2023-10-08 11:20:42,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184483840. Throughput: 0: 1826.1, 1: 1823.8. Samples: 46127346. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:20:42,016][52710] Avg episode reward: [(0, '32.520'), (1, '40.880')] [2023-10-08 11:20:42,026][53885] Updated weights for policy 1, policy_version 89862 (0.0007) [2023-10-08 11:20:42,407][53885] Updated weights for policy 1, policy_version 89872 (0.0008) [2023-10-08 11:20:42,776][53885] Updated weights for policy 1, policy_version 89882 (0.0008) [2023-10-08 11:20:43,641][53852] Updated weights for policy 0, policy_version 90310 (0.0007) [2023-10-08 11:20:43,997][53852] Updated weights for policy 0, policy_version 90320 (0.0008) [2023-10-08 11:20:44,369][53852] Updated weights for policy 0, policy_version 90330 (0.0010) [2023-10-08 11:20:46,222][53885] Updated weights for policy 1, policy_version 89892 (0.0008) [2023-10-08 11:20:46,597][53885] Updated weights for policy 1, policy_version 89902 (0.0009) [2023-10-08 11:20:46,970][53885] Updated weights for policy 1, policy_version 89912 (0.0009) [2023-10-08 11:20:47,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 184549376. Throughput: 0: 1827.6, 1: 1819.9. Samples: 46149384. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:20:47,015][52710] Avg episode reward: [(0, '34.380'), (1, '41.060')] [2023-10-08 11:20:48,022][53852] Updated weights for policy 0, policy_version 90340 (0.0009) [2023-10-08 11:20:48,388][53852] Updated weights for policy 0, policy_version 90350 (0.0008) [2023-10-08 11:20:48,756][53852] Updated weights for policy 0, policy_version 90360 (0.0007) [2023-10-08 11:20:50,540][53885] Updated weights for policy 1, policy_version 89922 (0.0009) [2023-10-08 11:20:50,905][53885] Updated weights for policy 1, policy_version 89932 (0.0009) [2023-10-08 11:20:51,275][53885] Updated weights for policy 1, policy_version 89942 (0.0009) [2023-10-08 11:20:51,642][53885] Updated weights for policy 1, policy_version 89952 (0.0010) [2023-10-08 11:20:52,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 184647680. Throughput: 0: 1832.8, 1: 1810.1. Samples: 46171040. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:20:52,016][52710] Avg episode reward: [(0, '34.030'), (1, '39.820')] [2023-10-08 11:20:52,449][53852] Updated weights for policy 0, policy_version 90370 (0.0008) [2023-10-08 11:20:52,845][53852] Updated weights for policy 0, policy_version 90380 (0.0009) [2023-10-08 11:20:53,211][53852] Updated weights for policy 0, policy_version 90390 (0.0008) [2023-10-08 11:20:53,581][53852] Updated weights for policy 0, policy_version 90400 (0.0007) [2023-10-08 11:20:55,366][53885] Updated weights for policy 1, policy_version 89962 (0.0008) [2023-10-08 11:20:55,728][53885] Updated weights for policy 1, policy_version 89972 (0.0007) [2023-10-08 11:20:56,092][53885] Updated weights for policy 1, policy_version 89982 (0.0008) [2023-10-08 11:20:57,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 184713216. Throughput: 0: 1829.9, 1: 1818.8. Samples: 46182286. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:20:57,015][52710] Avg episode reward: [(0, '33.550'), (1, '38.480')] [2023-10-08 11:20:57,179][53852] Updated weights for policy 0, policy_version 90410 (0.0011) [2023-10-08 11:20:57,549][53852] Updated weights for policy 0, policy_version 90420 (0.0011) [2023-10-08 11:20:57,909][53852] Updated weights for policy 0, policy_version 90430 (0.0010) [2023-10-08 11:20:59,854][53885] Updated weights for policy 1, policy_version 89992 (0.0007) [2023-10-08 11:21:00,219][53885] Updated weights for policy 1, policy_version 90002 (0.0008) [2023-10-08 11:21:00,597][53885] Updated weights for policy 1, policy_version 90012 (0.0011) [2023-10-08 11:21:01,514][53852] Updated weights for policy 0, policy_version 90440 (0.0007) [2023-10-08 11:21:01,887][53852] Updated weights for policy 0, policy_version 90450 (0.0007) [2023-10-08 11:21:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184778752. Throughput: 0: 1835.9, 1: 1814.5. Samples: 46204184. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:21:02,016][52710] Avg episode reward: [(0, '35.390'), (1, '41.610')] [2023-10-08 11:21:02,252][53852] Updated weights for policy 0, policy_version 90460 (0.0008) [2023-10-08 11:21:04,369][53885] Updated weights for policy 1, policy_version 90022 (0.0010) [2023-10-08 11:21:04,729][53885] Updated weights for policy 1, policy_version 90032 (0.0011) [2023-10-08 11:21:05,095][53885] Updated weights for policy 1, policy_version 90042 (0.0009) [2023-10-08 11:21:05,783][53852] Updated weights for policy 0, policy_version 90470 (0.0009) [2023-10-08 11:21:06,145][53852] Updated weights for policy 0, policy_version 90480 (0.0009) [2023-10-08 11:21:06,516][53852] Updated weights for policy 0, policy_version 90490 (0.0009) [2023-10-08 11:21:07,015][52710] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 184877056. Throughput: 0: 1824.9, 1: 1818.5. Samples: 46225646. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:21:07,016][52710] Avg episode reward: [(0, '36.030'), (1, '36.020')] [2023-10-08 11:21:08,892][53885] Updated weights for policy 1, policy_version 90052 (0.0011) [2023-10-08 11:21:09,260][53885] Updated weights for policy 1, policy_version 90062 (0.0011) [2023-10-08 11:21:09,632][53885] Updated weights for policy 1, policy_version 90072 (0.0010) [2023-10-08 11:21:10,090][53852] Updated weights for policy 0, policy_version 90500 (0.0008) [2023-10-08 11:21:10,469][53852] Updated weights for policy 0, policy_version 90510 (0.0010) [2023-10-08 11:21:10,842][53852] Updated weights for policy 0, policy_version 90520 (0.0007) [2023-10-08 11:21:12,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 184942592. Throughput: 0: 1844.0, 1: 1818.7. Samples: 46237394. Policy #0 lag: (min: 24.0, avg: 46.5, max: 48.0) [2023-10-08 11:21:12,016][52710] Avg episode reward: [(0, '36.300'), (1, '36.870')] [2023-10-08 11:21:13,279][53885] Updated weights for policy 1, policy_version 90082 (0.0007) [2023-10-08 11:21:13,656][53885] Updated weights for policy 1, policy_version 90092 (0.0011) [2023-10-08 11:21:14,034][53885] Updated weights for policy 1, policy_version 90102 (0.0010) [2023-10-08 11:21:14,373][53852] Updated weights for policy 0, policy_version 90530 (0.0007) [2023-10-08 11:21:14,399][53885] Updated weights for policy 1, policy_version 90112 (0.0007) [2023-10-08 11:21:14,740][53852] Updated weights for policy 0, policy_version 90540 (0.0008) [2023-10-08 11:21:15,103][53852] Updated weights for policy 0, policy_version 90550 (0.0009) [2023-10-08 11:21:15,475][53852] Updated weights for policy 0, policy_version 90560 (0.0010) [2023-10-08 11:21:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14773.4). Total num frames: 185008128. Throughput: 0: 1824.5, 1: 1814.5. Samples: 46258388. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:17,015][52710] Avg episode reward: [(0, '34.410'), (1, '37.740')] [2023-10-08 11:21:18,199][53885] Updated weights for policy 1, policy_version 90122 (0.0010) [2023-10-08 11:21:18,570][53885] Updated weights for policy 1, policy_version 90132 (0.0010) [2023-10-08 11:21:18,949][53885] Updated weights for policy 1, policy_version 90142 (0.0009) [2023-10-08 11:21:19,264][53852] Updated weights for policy 0, policy_version 90570 (0.0007) [2023-10-08 11:21:19,636][53852] Updated weights for policy 0, policy_version 90580 (0.0009) [2023-10-08 11:21:20,005][53852] Updated weights for policy 0, policy_version 90590 (0.0007) [2023-10-08 11:21:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185073664. Throughput: 0: 1837.6, 1: 1817.3. Samples: 46281128. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:22,016][52710] Avg episode reward: [(0, '32.840'), (1, '38.130')] [2023-10-08 11:21:22,606][53885] Updated weights for policy 1, policy_version 90152 (0.0008) [2023-10-08 11:21:22,983][53885] Updated weights for policy 1, policy_version 90162 (0.0008) [2023-10-08 11:21:23,342][53885] Updated weights for policy 1, policy_version 90172 (0.0007) [2023-10-08 11:21:23,570][53852] Updated weights for policy 0, policy_version 90600 (0.0007) [2023-10-08 11:21:23,938][53852] Updated weights for policy 0, policy_version 90610 (0.0008) [2023-10-08 11:21:24,303][53852] Updated weights for policy 0, policy_version 90620 (0.0008) [2023-10-08 11:21:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 185139200. Throughput: 0: 1826.5, 1: 1819.3. Samples: 46291408. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:27,016][52710] Avg episode reward: [(0, '37.990'), (1, '37.230')] [2023-10-08 11:21:27,039][53885] Updated weights for policy 1, policy_version 90182 (0.0007) [2023-10-08 11:21:27,420][53885] Updated weights for policy 1, policy_version 90192 (0.0012) [2023-10-08 11:21:27,794][53885] Updated weights for policy 1, policy_version 90202 (0.0007) [2023-10-08 11:21:27,917][53852] Updated weights for policy 0, policy_version 90630 (0.0007) [2023-10-08 11:21:28,292][53852] Updated weights for policy 0, policy_version 90640 (0.0009) [2023-10-08 11:21:28,665][53852] Updated weights for policy 0, policy_version 90650 (0.0008) [2023-10-08 11:21:31,530][53885] Updated weights for policy 1, policy_version 90212 (0.0007) [2023-10-08 11:21:31,886][53885] Updated weights for policy 1, policy_version 90222 (0.0008) [2023-10-08 11:21:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 185204736. Throughput: 0: 1850.3, 1: 1819.7. Samples: 46314536. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:32,016][52710] Avg episode reward: [(0, '38.180'), (1, '39.150')] [2023-10-08 11:21:32,258][53885] Updated weights for policy 1, policy_version 90232 (0.0009) [2023-10-08 11:21:32,381][53852] Updated weights for policy 0, policy_version 90660 (0.0007) [2023-10-08 11:21:32,751][53852] Updated weights for policy 0, policy_version 90670 (0.0008) [2023-10-08 11:21:33,121][53852] Updated weights for policy 0, policy_version 90680 (0.0008) [2023-10-08 11:21:35,892][53885] Updated weights for policy 1, policy_version 90242 (0.0009) [2023-10-08 11:21:36,265][53885] Updated weights for policy 1, policy_version 90252 (0.0008) [2023-10-08 11:21:36,628][53885] Updated weights for policy 1, policy_version 90262 (0.0009) [2023-10-08 11:21:36,682][53852] Updated weights for policy 0, policy_version 90690 (0.0009) [2023-10-08 11:21:36,993][53885] Updated weights for policy 1, policy_version 90272 (0.0008) [2023-10-08 11:21:37,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185303040. Throughput: 0: 1849.0, 1: 1828.7. Samples: 46336536. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:37,016][52710] Avg episode reward: [(0, '34.790'), (1, '36.530')] [2023-10-08 11:21:37,050][53852] Updated weights for policy 0, policy_version 90700 (0.0009) [2023-10-08 11:21:37,429][53852] Updated weights for policy 0, policy_version 90710 (0.0009) [2023-10-08 11:21:37,808][53852] Updated weights for policy 0, policy_version 90720 (0.0011) [2023-10-08 11:21:40,617][53885] Updated weights for policy 1, policy_version 90282 (0.0010) [2023-10-08 11:21:40,981][53885] Updated weights for policy 1, policy_version 90292 (0.0007) [2023-10-08 11:21:41,350][53885] Updated weights for policy 1, policy_version 90302 (0.0010) [2023-10-08 11:21:41,685][53852] Updated weights for policy 0, policy_version 90730 (0.0007) [2023-10-08 11:21:42,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 185368576. Throughput: 0: 1849.8, 1: 1819.3. Samples: 46347394. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:42,015][52710] Avg episode reward: [(0, '34.440'), (1, '33.890')] [2023-10-08 11:21:42,049][53852] Updated weights for policy 0, policy_version 90740 (0.0007) [2023-10-08 11:21:42,422][53852] Updated weights for policy 0, policy_version 90750 (0.0007) [2023-10-08 11:21:45,058][53885] Updated weights for policy 1, policy_version 90312 (0.0008) [2023-10-08 11:21:45,436][53885] Updated weights for policy 1, policy_version 90322 (0.0008) [2023-10-08 11:21:45,805][53885] Updated weights for policy 1, policy_version 90332 (0.0008) [2023-10-08 11:21:46,121][53852] Updated weights for policy 0, policy_version 90760 (0.0009) [2023-10-08 11:21:46,492][53852] Updated weights for policy 0, policy_version 90770 (0.0007) [2023-10-08 11:21:46,871][53852] Updated weights for policy 0, policy_version 90780 (0.0007) [2023-10-08 11:21:47,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185434112. Throughput: 0: 1842.8, 1: 1825.5. Samples: 46369254. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:47,016][52710] Avg episode reward: [(0, '37.810'), (1, '33.090')] [2023-10-08 11:21:49,550][53885] Updated weights for policy 1, policy_version 90342 (0.0008) [2023-10-08 11:21:49,923][53885] Updated weights for policy 1, policy_version 90352 (0.0008) [2023-10-08 11:21:50,291][53885] Updated weights for policy 1, policy_version 90362 (0.0007) [2023-10-08 11:21:50,628][53852] Updated weights for policy 0, policy_version 90790 (0.0009) [2023-10-08 11:21:50,999][53852] Updated weights for policy 0, policy_version 90800 (0.0008) [2023-10-08 11:21:51,372][53852] Updated weights for policy 0, policy_version 90810 (0.0008) [2023-10-08 11:21:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 185532416. Throughput: 0: 1831.7, 1: 1824.1. Samples: 46390154. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:52,016][52710] Avg episode reward: [(0, '39.210'), (1, '33.690')] [2023-10-08 11:21:52,028][53500] Saving new best policy, reward=39.210! [2023-10-08 11:21:53,824][53885] Updated weights for policy 1, policy_version 90372 (0.0010) [2023-10-08 11:21:54,205][53885] Updated weights for policy 1, policy_version 90382 (0.0011) [2023-10-08 11:21:54,563][53885] Updated weights for policy 1, policy_version 90392 (0.0009) [2023-10-08 11:21:55,007][53852] Updated weights for policy 0, policy_version 90820 (0.0010) [2023-10-08 11:21:55,382][53852] Updated weights for policy 0, policy_version 90830 (0.0009) [2023-10-08 11:21:55,752][53852] Updated weights for policy 0, policy_version 90840 (0.0008) [2023-10-08 11:21:57,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 185597952. Throughput: 0: 1829.3, 1: 1827.7. Samples: 46401960. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:21:57,016][52710] Avg episode reward: [(0, '31.800'), (1, '31.470')] [2023-10-08 11:21:58,113][53885] Updated weights for policy 1, policy_version 90402 (0.0008) [2023-10-08 11:21:58,476][53885] Updated weights for policy 1, policy_version 90412 (0.0007) [2023-10-08 11:21:58,855][53885] Updated weights for policy 1, policy_version 90422 (0.0008) [2023-10-08 11:21:59,210][53885] Updated weights for policy 1, policy_version 90432 (0.0010) [2023-10-08 11:21:59,282][53852] Updated weights for policy 0, policy_version 90850 (0.0008) [2023-10-08 11:21:59,649][53852] Updated weights for policy 0, policy_version 90860 (0.0008) [2023-10-08 11:22:00,017][53852] Updated weights for policy 0, policy_version 90870 (0.0007) [2023-10-08 11:22:00,389][53852] Updated weights for policy 0, policy_version 90880 (0.0008) [2023-10-08 11:22:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185663488. Throughput: 0: 1833.2, 1: 1835.0. Samples: 46423458. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:22:02,016][52710] Avg episode reward: [(0, '33.510'), (1, '36.670')] [2023-10-08 11:22:02,988][53885] Updated weights for policy 1, policy_version 90442 (0.0010) [2023-10-08 11:22:03,357][53885] Updated weights for policy 1, policy_version 90452 (0.0007) [2023-10-08 11:22:03,726][53885] Updated weights for policy 1, policy_version 90462 (0.0007) [2023-10-08 11:22:04,029][53852] Updated weights for policy 0, policy_version 90890 (0.0007) [2023-10-08 11:22:04,394][53852] Updated weights for policy 0, policy_version 90900 (0.0008) [2023-10-08 11:22:04,761][53852] Updated weights for policy 0, policy_version 90910 (0.0008) [2023-10-08 11:22:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 185729024. Throughput: 0: 1838.7, 1: 1839.2. Samples: 46446634. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:22:07,016][52710] Avg episode reward: [(0, '36.120'), (1, '35.080')] [2023-10-08 11:22:07,028][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000090912_93093888.pth... [2023-10-08 11:22:07,063][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000089184_91324416.pth [2023-10-08 11:22:07,218][53885] Updated weights for policy 1, policy_version 90472 (0.0008) [2023-10-08 11:22:07,583][53885] Updated weights for policy 1, policy_version 90482 (0.0007) [2023-10-08 11:22:07,953][53885] Updated weights for policy 1, policy_version 90492 (0.0007) [2023-10-08 11:22:08,097][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000090496_92667904.pth... [2023-10-08 11:22:08,137][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000088768_90898432.pth [2023-10-08 11:22:08,248][53852] Updated weights for policy 0, policy_version 90920 (0.0010) [2023-10-08 11:22:08,614][53852] Updated weights for policy 0, policy_version 90930 (0.0010) [2023-10-08 11:22:08,981][53852] Updated weights for policy 0, policy_version 90940 (0.0009) [2023-10-08 11:22:11,594][53885] Updated weights for policy 1, policy_version 90502 (0.0009) [2023-10-08 11:22:11,957][53885] Updated weights for policy 1, policy_version 90512 (0.0010) [2023-10-08 11:22:12,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 185794560. Throughput: 0: 1838.0, 1: 1839.7. Samples: 46456906. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) [2023-10-08 11:22:12,016][52710] Avg episode reward: [(0, '37.010'), (1, '37.430')] [2023-10-08 11:22:12,317][53885] Updated weights for policy 1, policy_version 90522 (0.0008) [2023-10-08 11:22:12,807][53852] Updated weights for policy 0, policy_version 90950 (0.0010) [2023-10-08 11:22:13,180][53852] Updated weights for policy 0, policy_version 90960 (0.0010) [2023-10-08 11:22:13,548][53852] Updated weights for policy 0, policy_version 90970 (0.0008) [2023-10-08 11:22:16,014][53885] Updated weights for policy 1, policy_version 90532 (0.0010) [2023-10-08 11:22:16,407][53885] Updated weights for policy 1, policy_version 90542 (0.0008) [2023-10-08 11:22:16,783][53885] Updated weights for policy 1, policy_version 90552 (0.0009) [2023-10-08 11:22:17,015][52710] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 185860096. Throughput: 0: 1829.2, 1: 1843.7. Samples: 46479816. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:17,016][52710] Avg episode reward: [(0, '33.890'), (1, '36.770')] [2023-10-08 11:22:17,257][53852] Updated weights for policy 0, policy_version 90980 (0.0008) [2023-10-08 11:22:17,627][53852] Updated weights for policy 0, policy_version 90990 (0.0009) [2023-10-08 11:22:17,998][53852] Updated weights for policy 0, policy_version 91000 (0.0009) [2023-10-08 11:22:20,340][53885] Updated weights for policy 1, policy_version 90562 (0.0009) [2023-10-08 11:22:20,695][53885] Updated weights for policy 1, policy_version 90572 (0.0009) [2023-10-08 11:22:21,071][53885] Updated weights for policy 1, policy_version 90582 (0.0011) [2023-10-08 11:22:21,441][53885] Updated weights for policy 1, policy_version 90592 (0.0009) [2023-10-08 11:22:21,604][53852] Updated weights for policy 0, policy_version 91010 (0.0008) [2023-10-08 11:22:21,976][53852] Updated weights for policy 0, policy_version 91020 (0.0008) [2023-10-08 11:22:22,015][52710] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 185958400. Throughput: 0: 1833.2, 1: 1832.0. Samples: 46501470. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:22,016][52710] Avg episode reward: [(0, '32.430'), (1, '34.760')] [2023-10-08 11:22:22,341][53852] Updated weights for policy 0, policy_version 91030 (0.0007) [2023-10-08 11:22:22,712][53852] Updated weights for policy 0, policy_version 91040 (0.0009) [2023-10-08 11:22:25,092][53885] Updated weights for policy 1, policy_version 90602 (0.0008) [2023-10-08 11:22:25,450][53885] Updated weights for policy 1, policy_version 90612 (0.0008) [2023-10-08 11:22:25,820][53885] Updated weights for policy 1, policy_version 90622 (0.0008) [2023-10-08 11:22:26,255][53852] Updated weights for policy 0, policy_version 91050 (0.0008) [2023-10-08 11:22:26,625][53852] Updated weights for policy 0, policy_version 91060 (0.0008) [2023-10-08 11:22:27,000][53852] Updated weights for policy 0, policy_version 91070 (0.0008) [2023-10-08 11:22:27,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186023936. Throughput: 0: 1841.6, 1: 1845.8. Samples: 46513328. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:27,015][52710] Avg episode reward: [(0, '40.890'), (1, '35.780')] [2023-10-08 11:22:27,070][53500] Saving new best policy, reward=40.890! [2023-10-08 11:22:29,523][53885] Updated weights for policy 1, policy_version 90632 (0.0007) [2023-10-08 11:22:29,891][53885] Updated weights for policy 1, policy_version 90642 (0.0007) [2023-10-08 11:22:30,260][53885] Updated weights for policy 1, policy_version 90652 (0.0008) [2023-10-08 11:22:30,704][53852] Updated weights for policy 0, policy_version 91080 (0.0008) [2023-10-08 11:22:31,076][53852] Updated weights for policy 0, policy_version 91090 (0.0008) [2023-10-08 11:22:31,439][53852] Updated weights for policy 0, policy_version 91100 (0.0007) [2023-10-08 11:22:32,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 186122240. Throughput: 0: 1840.4, 1: 1834.2. Samples: 46534614. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:32,016][52710] Avg episode reward: [(0, '37.440'), (1, '38.680')] [2023-10-08 11:22:33,886][53885] Updated weights for policy 1, policy_version 90662 (0.0009) [2023-10-08 11:22:34,254][53885] Updated weights for policy 1, policy_version 90672 (0.0009) [2023-10-08 11:22:34,609][53885] Updated weights for policy 1, policy_version 90682 (0.0008) [2023-10-08 11:22:34,980][53852] Updated weights for policy 0, policy_version 91110 (0.0009) [2023-10-08 11:22:35,339][53852] Updated weights for policy 0, policy_version 91120 (0.0009) [2023-10-08 11:22:35,712][53852] Updated weights for policy 0, policy_version 91130 (0.0009) [2023-10-08 11:22:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 186187776. Throughput: 0: 1847.8, 1: 1850.3. Samples: 46556568. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:37,016][52710] Avg episode reward: [(0, '33.420'), (1, '35.600')] [2023-10-08 11:22:38,291][53885] Updated weights for policy 1, policy_version 90692 (0.0009) [2023-10-08 11:22:38,659][53885] Updated weights for policy 1, policy_version 90702 (0.0008) [2023-10-08 11:22:39,028][53885] Updated weights for policy 1, policy_version 90712 (0.0009) [2023-10-08 11:22:39,282][53852] Updated weights for policy 0, policy_version 91140 (0.0007) [2023-10-08 11:22:39,651][53852] Updated weights for policy 0, policy_version 91150 (0.0007) [2023-10-08 11:22:40,025][53852] Updated weights for policy 0, policy_version 91160 (0.0009) [2023-10-08 11:22:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186253312. Throughput: 0: 1844.5, 1: 1835.7. Samples: 46567572. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:42,016][52710] Avg episode reward: [(0, '34.960'), (1, '36.060')] [2023-10-08 11:22:42,651][53885] Updated weights for policy 1, policy_version 90722 (0.0009) [2023-10-08 11:22:43,025][53885] Updated weights for policy 1, policy_version 90732 (0.0007) [2023-10-08 11:22:43,393][53885] Updated weights for policy 1, policy_version 90742 (0.0009) [2023-10-08 11:22:43,655][53852] Updated weights for policy 0, policy_version 91170 (0.0007) [2023-10-08 11:22:43,755][53885] Updated weights for policy 1, policy_version 90752 (0.0008) [2023-10-08 11:22:44,025][53852] Updated weights for policy 0, policy_version 91180 (0.0008) [2023-10-08 11:22:44,387][53852] Updated weights for policy 0, policy_version 91190 (0.0011) [2023-10-08 11:22:44,752][53852] Updated weights for policy 0, policy_version 91200 (0.0010) [2023-10-08 11:22:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186318848. Throughput: 0: 1849.7, 1: 1840.1. Samples: 46589500. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:47,016][52710] Avg episode reward: [(0, '38.640'), (1, '34.060')] [2023-10-08 11:22:47,365][53885] Updated weights for policy 1, policy_version 90762 (0.0008) [2023-10-08 11:22:47,732][53885] Updated weights for policy 1, policy_version 90772 (0.0011) [2023-10-08 11:22:48,094][53885] Updated weights for policy 1, policy_version 90782 (0.0010) [2023-10-08 11:22:48,471][53852] Updated weights for policy 0, policy_version 91210 (0.0011) [2023-10-08 11:22:48,841][53852] Updated weights for policy 0, policy_version 91220 (0.0011) [2023-10-08 11:22:49,213][53852] Updated weights for policy 0, policy_version 91230 (0.0010) [2023-10-08 11:22:51,717][53885] Updated weights for policy 1, policy_version 90792 (0.0008) [2023-10-08 11:22:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 186384384. Throughput: 0: 1843.3, 1: 1832.0. Samples: 46612022. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:52,016][52710] Avg episode reward: [(0, '35.700'), (1, '34.420')] [2023-10-08 11:22:52,077][53885] Updated weights for policy 1, policy_version 90802 (0.0008) [2023-10-08 11:22:52,448][53885] Updated weights for policy 1, policy_version 90812 (0.0010) [2023-10-08 11:22:52,933][53852] Updated weights for policy 0, policy_version 91240 (0.0007) [2023-10-08 11:22:53,305][53852] Updated weights for policy 0, policy_version 91250 (0.0007) [2023-10-08 11:22:53,666][53852] Updated weights for policy 0, policy_version 91260 (0.0007) [2023-10-08 11:22:56,054][53885] Updated weights for policy 1, policy_version 90822 (0.0010) [2023-10-08 11:22:56,422][53885] Updated weights for policy 1, policy_version 90832 (0.0008) [2023-10-08 11:22:56,793][53885] Updated weights for policy 1, policy_version 90842 (0.0007) [2023-10-08 11:22:57,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186482688. Throughput: 0: 1839.3, 1: 1835.3. Samples: 46622262. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:22:57,016][52710] Avg episode reward: [(0, '34.460'), (1, '38.100')] [2023-10-08 11:22:57,189][53852] Updated weights for policy 0, policy_version 91270 (0.0007) [2023-10-08 11:22:57,555][53852] Updated weights for policy 0, policy_version 91280 (0.0010) [2023-10-08 11:22:57,923][53852] Updated weights for policy 0, policy_version 91290 (0.0009) [2023-10-08 11:23:00,411][53885] Updated weights for policy 1, policy_version 90852 (0.0007) [2023-10-08 11:23:00,769][53885] Updated weights for policy 1, policy_version 90862 (0.0008) [2023-10-08 11:23:01,142][53885] Updated weights for policy 1, policy_version 90872 (0.0009) [2023-10-08 11:23:01,638][53852] Updated weights for policy 0, policy_version 91300 (0.0009) [2023-10-08 11:23:02,015][53852] Updated weights for policy 0, policy_version 91310 (0.0007) [2023-10-08 11:23:02,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 186548224. Throughput: 0: 1845.7, 1: 1827.9. Samples: 46645132. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:23:02,016][52710] Avg episode reward: [(0, '38.440'), (1, '36.500')] [2023-10-08 11:23:02,386][53852] Updated weights for policy 0, policy_version 91320 (0.0008) [2023-10-08 11:23:05,016][53885] Updated weights for policy 1, policy_version 90882 (0.0008) [2023-10-08 11:23:05,420][53885] Updated weights for policy 1, policy_version 90892 (0.0007) [2023-10-08 11:23:05,792][53885] Updated weights for policy 1, policy_version 90902 (0.0008) [2023-10-08 11:23:06,066][53852] Updated weights for policy 0, policy_version 91330 (0.0007) [2023-10-08 11:23:06,157][53885] Updated weights for policy 1, policy_version 90912 (0.0007) [2023-10-08 11:23:06,431][53852] Updated weights for policy 0, policy_version 91340 (0.0009) [2023-10-08 11:23:06,801][53852] Updated weights for policy 0, policy_version 91350 (0.0007) [2023-10-08 11:23:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 186613760. Throughput: 0: 1822.2, 1: 1838.2. Samples: 46666190. Policy #0 lag: (min: 29.0, avg: 29.0, max: 30.0) [2023-10-08 11:23:07,016][52710] Avg episode reward: [(0, '36.560'), (1, '31.290')] [2023-10-08 11:23:07,178][53852] Updated weights for policy 0, policy_version 91360 (0.0009) [2023-10-08 11:23:09,701][53885] Updated weights for policy 1, policy_version 90922 (0.0007) [2023-10-08 11:23:10,056][53885] Updated weights for policy 1, policy_version 90932 (0.0009) [2023-10-08 11:23:10,424][53885] Updated weights for policy 1, policy_version 90942 (0.0007) [2023-10-08 11:23:10,679][53852] Updated weights for policy 0, policy_version 91370 (0.0009) [2023-10-08 11:23:11,043][53852] Updated weights for policy 0, policy_version 91380 (0.0007) [2023-10-08 11:23:11,410][53852] Updated weights for policy 0, policy_version 91390 (0.0008) [2023-10-08 11:23:12,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 186712064. Throughput: 0: 1835.4, 1: 1829.1. Samples: 46678230. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:12,016][52710] Avg episode reward: [(0, '32.580'), (1, '30.100')] [2023-10-08 11:23:14,138][53885] Updated weights for policy 1, policy_version 90952 (0.0008) [2023-10-08 11:23:14,503][53885] Updated weights for policy 1, policy_version 90962 (0.0007) [2023-10-08 11:23:14,868][53885] Updated weights for policy 1, policy_version 90972 (0.0007) [2023-10-08 11:23:15,034][53852] Updated weights for policy 0, policy_version 91400 (0.0007) [2023-10-08 11:23:15,403][53852] Updated weights for policy 0, policy_version 91410 (0.0010) [2023-10-08 11:23:15,778][53852] Updated weights for policy 0, policy_version 91420 (0.0007) [2023-10-08 11:23:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 186777600. Throughput: 0: 1820.2, 1: 1836.7. Samples: 46699174. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:17,016][52710] Avg episode reward: [(0, '35.350'), (1, '36.090')] [2023-10-08 11:23:18,617][53885] Updated weights for policy 1, policy_version 90982 (0.0009) [2023-10-08 11:23:18,992][53885] Updated weights for policy 1, policy_version 90992 (0.0009) [2023-10-08 11:23:19,362][53885] Updated weights for policy 1, policy_version 91002 (0.0007) [2023-10-08 11:23:19,457][53852] Updated weights for policy 0, policy_version 91430 (0.0007) [2023-10-08 11:23:19,840][53852] Updated weights for policy 0, policy_version 91440 (0.0007) [2023-10-08 11:23:20,219][53852] Updated weights for policy 0, policy_version 91450 (0.0008) [2023-10-08 11:23:22,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186843136. Throughput: 0: 1832.1, 1: 1830.0. Samples: 46721360. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:22,016][52710] Avg episode reward: [(0, '35.800'), (1, '33.750')] [2023-10-08 11:23:22,914][53885] Updated weights for policy 1, policy_version 91012 (0.0007) [2023-10-08 11:23:23,286][53885] Updated weights for policy 1, policy_version 91022 (0.0007) [2023-10-08 11:23:23,651][53885] Updated weights for policy 1, policy_version 91032 (0.0008) [2023-10-08 11:23:23,978][53852] Updated weights for policy 0, policy_version 91460 (0.0008) [2023-10-08 11:23:24,342][53852] Updated weights for policy 0, policy_version 91470 (0.0007) [2023-10-08 11:23:24,711][53852] Updated weights for policy 0, policy_version 91480 (0.0008) [2023-10-08 11:23:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186908672. Throughput: 0: 1822.1, 1: 1836.1. Samples: 46732192. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:27,015][52710] Avg episode reward: [(0, '36.330'), (1, '30.970')] [2023-10-08 11:23:27,362][53885] Updated weights for policy 1, policy_version 91042 (0.0009) [2023-10-08 11:23:27,735][53885] Updated weights for policy 1, policy_version 91052 (0.0008) [2023-10-08 11:23:28,098][53885] Updated weights for policy 1, policy_version 91062 (0.0007) [2023-10-08 11:23:28,366][53852] Updated weights for policy 0, policy_version 91490 (0.0007) [2023-10-08 11:23:28,465][53885] Updated weights for policy 1, policy_version 91072 (0.0007) [2023-10-08 11:23:28,739][53852] Updated weights for policy 0, policy_version 91500 (0.0011) [2023-10-08 11:23:29,111][53852] Updated weights for policy 0, policy_version 91510 (0.0008) [2023-10-08 11:23:29,472][53852] Updated weights for policy 0, policy_version 91520 (0.0007) [2023-10-08 11:23:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 186974208. Throughput: 0: 1834.2, 1: 1834.8. Samples: 46754608. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:32,016][52710] Avg episode reward: [(0, '36.870'), (1, '37.110')] [2023-10-08 11:23:32,093][53885] Updated weights for policy 1, policy_version 91082 (0.0008) [2023-10-08 11:23:32,465][53885] Updated weights for policy 1, policy_version 91092 (0.0007) [2023-10-08 11:23:32,832][53885] Updated weights for policy 1, policy_version 91102 (0.0007) [2023-10-08 11:23:33,078][53852] Updated weights for policy 0, policy_version 91530 (0.0008) [2023-10-08 11:23:33,453][53852] Updated weights for policy 0, policy_version 91540 (0.0007) [2023-10-08 11:23:33,823][53852] Updated weights for policy 0, policy_version 91550 (0.0008) [2023-10-08 11:23:36,473][53885] Updated weights for policy 1, policy_version 91112 (0.0007) [2023-10-08 11:23:36,836][53885] Updated weights for policy 1, policy_version 91122 (0.0008) [2023-10-08 11:23:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 187039744. Throughput: 0: 1841.8, 1: 1828.6. Samples: 46777190. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:37,016][52710] Avg episode reward: [(0, '35.270'), (1, '37.230')] [2023-10-08 11:23:37,214][53885] Updated weights for policy 1, policy_version 91132 (0.0008) [2023-10-08 11:23:37,415][53852] Updated weights for policy 0, policy_version 91560 (0.0008) [2023-10-08 11:23:37,797][53852] Updated weights for policy 0, policy_version 91570 (0.0011) [2023-10-08 11:23:38,157][53852] Updated weights for policy 0, policy_version 91580 (0.0011) [2023-10-08 11:23:40,813][53885] Updated weights for policy 1, policy_version 91142 (0.0010) [2023-10-08 11:23:41,187][53885] Updated weights for policy 1, policy_version 91152 (0.0011) [2023-10-08 11:23:41,559][53885] Updated weights for policy 1, policy_version 91162 (0.0009) [2023-10-08 11:23:41,753][53852] Updated weights for policy 0, policy_version 91590 (0.0008) [2023-10-08 11:23:42,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187138048. Throughput: 0: 1844.3, 1: 1838.3. Samples: 46787980. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:42,016][52710] Avg episode reward: [(0, '36.650'), (1, '31.440')] [2023-10-08 11:23:42,122][53852] Updated weights for policy 0, policy_version 91600 (0.0009) [2023-10-08 11:23:42,496][53852] Updated weights for policy 0, policy_version 91610 (0.0010) [2023-10-08 11:23:45,107][53885] Updated weights for policy 1, policy_version 91172 (0.0009) [2023-10-08 11:23:45,470][53885] Updated weights for policy 1, policy_version 91182 (0.0008) [2023-10-08 11:23:45,839][53885] Updated weights for policy 1, policy_version 91192 (0.0007) [2023-10-08 11:23:46,227][53852] Updated weights for policy 0, policy_version 91620 (0.0007) [2023-10-08 11:23:46,585][53852] Updated weights for policy 0, policy_version 91630 (0.0007) [2023-10-08 11:23:46,958][53852] Updated weights for policy 0, policy_version 91640 (0.0008) [2023-10-08 11:23:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187203584. Throughput: 0: 1847.4, 1: 1830.1. Samples: 46810620. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:47,016][52710] Avg episode reward: [(0, '36.500'), (1, '32.190')] [2023-10-08 11:23:49,652][53885] Updated weights for policy 1, policy_version 91202 (0.0007) [2023-10-08 11:23:50,061][53885] Updated weights for policy 1, policy_version 91212 (0.0010) [2023-10-08 11:23:50,440][53885] Updated weights for policy 1, policy_version 91222 (0.0008) [2023-10-08 11:23:50,678][53852] Updated weights for policy 0, policy_version 91650 (0.0008) [2023-10-08 11:23:50,803][53885] Updated weights for policy 1, policy_version 91232 (0.0010) [2023-10-08 11:23:51,046][53852] Updated weights for policy 0, policy_version 91660 (0.0009) [2023-10-08 11:23:51,418][53852] Updated weights for policy 0, policy_version 91670 (0.0007) [2023-10-08 11:23:51,796][53852] Updated weights for policy 0, policy_version 91680 (0.0009) [2023-10-08 11:23:52,015][52710] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 187301888. Throughput: 0: 1835.0, 1: 1834.4. Samples: 46831312. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:52,016][52710] Avg episode reward: [(0, '35.010'), (1, '37.390')] [2023-10-08 11:23:54,517][53885] Updated weights for policy 1, policy_version 91242 (0.0007) [2023-10-08 11:23:54,884][53885] Updated weights for policy 1, policy_version 91252 (0.0008) [2023-10-08 11:23:55,247][53885] Updated weights for policy 1, policy_version 91262 (0.0008) [2023-10-08 11:23:55,333][53852] Updated weights for policy 0, policy_version 91690 (0.0008) [2023-10-08 11:23:55,705][53852] Updated weights for policy 0, policy_version 91700 (0.0009) [2023-10-08 11:23:56,073][53852] Updated weights for policy 0, policy_version 91710 (0.0010) [2023-10-08 11:23:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187367424. Throughput: 0: 1843.6, 1: 1823.5. Samples: 46843250. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:23:57,015][52710] Avg episode reward: [(0, '37.100'), (1, '34.880')] [2023-10-08 11:23:58,950][53885] Updated weights for policy 1, policy_version 91272 (0.0009) [2023-10-08 11:23:59,325][53885] Updated weights for policy 1, policy_version 91282 (0.0007) [2023-10-08 11:23:59,689][53885] Updated weights for policy 1, policy_version 91292 (0.0008) [2023-10-08 11:23:59,757][53852] Updated weights for policy 0, policy_version 91720 (0.0007) [2023-10-08 11:24:00,131][53852] Updated weights for policy 0, policy_version 91730 (0.0009) [2023-10-08 11:24:00,505][53852] Updated weights for policy 0, policy_version 91740 (0.0010) [2023-10-08 11:24:02,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187432960. Throughput: 0: 1834.6, 1: 1830.2. Samples: 46864088. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:24:02,016][52710] Avg episode reward: [(0, '34.250'), (1, '35.690')] [2023-10-08 11:24:03,439][53885] Updated weights for policy 1, policy_version 91302 (0.0008) [2023-10-08 11:24:03,816][53885] Updated weights for policy 1, policy_version 91312 (0.0007) [2023-10-08 11:24:04,177][53885] Updated weights for policy 1, policy_version 91322 (0.0007) [2023-10-08 11:24:04,212][53852] Updated weights for policy 0, policy_version 91750 (0.0008) [2023-10-08 11:24:04,592][53852] Updated weights for policy 0, policy_version 91760 (0.0007) [2023-10-08 11:24:04,956][53852] Updated weights for policy 0, policy_version 91770 (0.0007) [2023-10-08 11:24:07,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187498496. Throughput: 0: 1846.5, 1: 1827.2. Samples: 46886674. Policy #0 lag: (min: 31.0, avg: 42.6, max: 63.0) [2023-10-08 11:24:07,016][52710] Avg episode reward: [(0, '37.510'), (1, '35.660')] [2023-10-08 11:24:07,024][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000091328_93519872.pth... [2023-10-08 11:24:07,024][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000091776_93978624.pth... [2023-10-08 11:24:07,054][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000089632_91783168.pth [2023-10-08 11:24:07,057][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000090048_92209152.pth [2023-10-08 11:24:07,874][53885] Updated weights for policy 1, policy_version 91332 (0.0008) [2023-10-08 11:24:08,248][53885] Updated weights for policy 1, policy_version 91342 (0.0009) [2023-10-08 11:24:08,611][53885] Updated weights for policy 1, policy_version 91352 (0.0010) [2023-10-08 11:24:08,729][53852] Updated weights for policy 0, policy_version 91780 (0.0007) [2023-10-08 11:24:09,123][53852] Updated weights for policy 0, policy_version 91790 (0.0009) [2023-10-08 11:24:09,485][53852] Updated weights for policy 0, policy_version 91800 (0.0008) [2023-10-08 11:24:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 187564032. Throughput: 0: 1836.8, 1: 1823.3. Samples: 46896896. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:12,016][52710] Avg episode reward: [(0, '32.900'), (1, '38.330')] [2023-10-08 11:24:12,436][53885] Updated weights for policy 1, policy_version 91362 (0.0008) [2023-10-08 11:24:12,806][53885] Updated weights for policy 1, policy_version 91372 (0.0011) [2023-10-08 11:24:13,035][53852] Updated weights for policy 0, policy_version 91810 (0.0008) [2023-10-08 11:24:13,173][53885] Updated weights for policy 1, policy_version 91382 (0.0010) [2023-10-08 11:24:13,404][53852] Updated weights for policy 0, policy_version 91820 (0.0008) [2023-10-08 11:24:13,542][53885] Updated weights for policy 1, policy_version 91392 (0.0008) [2023-10-08 11:24:13,783][53852] Updated weights for policy 0, policy_version 91830 (0.0009) [2023-10-08 11:24:14,153][53852] Updated weights for policy 0, policy_version 91840 (0.0007) [2023-10-08 11:24:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 187629568. Throughput: 0: 1840.3, 1: 1814.8. Samples: 46919090. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:17,016][52710] Avg episode reward: [(0, '35.400'), (1, '40.750')] [2023-10-08 11:24:17,293][53885] Updated weights for policy 1, policy_version 91402 (0.0009) [2023-10-08 11:24:17,657][53885] Updated weights for policy 1, policy_version 91412 (0.0009) [2023-10-08 11:24:17,790][53852] Updated weights for policy 0, policy_version 91850 (0.0007) [2023-10-08 11:24:18,027][53885] Updated weights for policy 1, policy_version 91422 (0.0009) [2023-10-08 11:24:18,159][53852] Updated weights for policy 0, policy_version 91860 (0.0009) [2023-10-08 11:24:18,524][53852] Updated weights for policy 0, policy_version 91870 (0.0009) [2023-10-08 11:24:21,752][53885] Updated weights for policy 1, policy_version 91432 (0.0008) [2023-10-08 11:24:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187695104. Throughput: 0: 1833.6, 1: 1818.8. Samples: 46941550. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:22,015][52710] Avg episode reward: [(0, '32.380'), (1, '34.880')] [2023-10-08 11:24:22,114][53885] Updated weights for policy 1, policy_version 91442 (0.0008) [2023-10-08 11:24:22,356][53852] Updated weights for policy 0, policy_version 91880 (0.0008) [2023-10-08 11:24:22,485][53885] Updated weights for policy 1, policy_version 91452 (0.0009) [2023-10-08 11:24:22,720][53852] Updated weights for policy 0, policy_version 91890 (0.0008) [2023-10-08 11:24:23,094][53852] Updated weights for policy 0, policy_version 91900 (0.0007) [2023-10-08 11:24:26,291][53885] Updated weights for policy 1, policy_version 91462 (0.0007) [2023-10-08 11:24:26,660][53885] Updated weights for policy 1, policy_version 91472 (0.0007) [2023-10-08 11:24:26,888][53852] Updated weights for policy 0, policy_version 91910 (0.0007) [2023-10-08 11:24:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 187760640. Throughput: 0: 1830.9, 1: 1807.0. Samples: 46951684. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:27,016][52710] Avg episode reward: [(0, '30.740'), (1, '36.630')] [2023-10-08 11:24:27,034][53885] Updated weights for policy 1, policy_version 91482 (0.0009) [2023-10-08 11:24:27,265][53852] Updated weights for policy 0, policy_version 91920 (0.0009) [2023-10-08 11:24:27,640][53852] Updated weights for policy 0, policy_version 91930 (0.0009) [2023-10-08 11:24:30,602][53885] Updated weights for policy 1, policy_version 91492 (0.0008) [2023-10-08 11:24:30,968][53885] Updated weights for policy 1, policy_version 91502 (0.0007) [2023-10-08 11:24:31,299][53852] Updated weights for policy 0, policy_version 91940 (0.0010) [2023-10-08 11:24:31,344][53885] Updated weights for policy 1, policy_version 91512 (0.0009) [2023-10-08 11:24:31,666][53852] Updated weights for policy 0, policy_version 91950 (0.0008) [2023-10-08 11:24:32,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187858944. Throughput: 0: 1818.4, 1: 1818.6. Samples: 46974284. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:32,016][52710] Avg episode reward: [(0, '31.240'), (1, '37.980')] [2023-10-08 11:24:32,044][53852] Updated weights for policy 0, policy_version 91960 (0.0009) [2023-10-08 11:24:35,124][53885] Updated weights for policy 1, policy_version 91522 (0.0008) [2023-10-08 11:24:35,517][53885] Updated weights for policy 1, policy_version 91532 (0.0008) [2023-10-08 11:24:35,713][53852] Updated weights for policy 0, policy_version 91970 (0.0010) [2023-10-08 11:24:35,875][53885] Updated weights for policy 1, policy_version 91542 (0.0008) [2023-10-08 11:24:36,081][53852] Updated weights for policy 0, policy_version 91980 (0.0008) [2023-10-08 11:24:36,249][53885] Updated weights for policy 1, policy_version 91552 (0.0007) [2023-10-08 11:24:36,436][53852] Updated weights for policy 0, policy_version 91990 (0.0009) [2023-10-08 11:24:36,808][53852] Updated weights for policy 0, policy_version 92000 (0.0010) [2023-10-08 11:24:37,015][52710] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 187957248. Throughput: 0: 1818.5, 1: 1805.1. Samples: 46994376. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:37,016][52710] Avg episode reward: [(0, '34.350'), (1, '34.710')] [2023-10-08 11:24:39,831][53885] Updated weights for policy 1, policy_version 91562 (0.0007) [2023-10-08 11:24:40,196][53885] Updated weights for policy 1, policy_version 91572 (0.0008) [2023-10-08 11:24:40,560][53885] Updated weights for policy 1, policy_version 91582 (0.0009) [2023-10-08 11:24:40,635][53852] Updated weights for policy 0, policy_version 92010 (0.0008) [2023-10-08 11:24:41,002][53852] Updated weights for policy 0, policy_version 92020 (0.0008) [2023-10-08 11:24:41,375][53852] Updated weights for policy 0, policy_version 92030 (0.0008) [2023-10-08 11:24:42,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 188022784. Throughput: 0: 1812.2, 1: 1819.9. Samples: 47006694. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:42,016][52710] Avg episode reward: [(0, '31.820'), (1, '32.020')] [2023-10-08 11:24:44,313][53885] Updated weights for policy 1, policy_version 91592 (0.0007) [2023-10-08 11:24:44,680][53885] Updated weights for policy 1, policy_version 91602 (0.0008) [2023-10-08 11:24:44,970][53852] Updated weights for policy 0, policy_version 92040 (0.0008) [2023-10-08 11:24:45,042][53885] Updated weights for policy 1, policy_version 91612 (0.0009) [2023-10-08 11:24:45,340][53852] Updated weights for policy 0, policy_version 92050 (0.0008) [2023-10-08 11:24:45,722][53852] Updated weights for policy 0, policy_version 92060 (0.0009) [2023-10-08 11:24:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 188088320. Throughput: 0: 1814.0, 1: 1809.1. Samples: 47027130. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:47,016][52710] Avg episode reward: [(0, '33.360'), (1, '37.590')] [2023-10-08 11:24:48,713][53885] Updated weights for policy 1, policy_version 91622 (0.0008) [2023-10-08 11:24:49,075][53885] Updated weights for policy 1, policy_version 91632 (0.0010) [2023-10-08 11:24:49,351][53852] Updated weights for policy 0, policy_version 92070 (0.0007) [2023-10-08 11:24:49,450][53885] Updated weights for policy 1, policy_version 91642 (0.0008) [2023-10-08 11:24:49,721][53852] Updated weights for policy 0, policy_version 92080 (0.0007) [2023-10-08 11:24:50,094][53852] Updated weights for policy 0, policy_version 92090 (0.0007) [2023-10-08 11:24:52,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 188153856. Throughput: 0: 1812.1, 1: 1806.8. Samples: 47049526. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:52,015][52710] Avg episode reward: [(0, '33.180'), (1, '37.540')] [2023-10-08 11:24:53,324][53885] Updated weights for policy 1, policy_version 91652 (0.0009) [2023-10-08 11:24:53,689][53852] Updated weights for policy 0, policy_version 92100 (0.0009) [2023-10-08 11:24:53,694][53885] Updated weights for policy 1, policy_version 91662 (0.0009) [2023-10-08 11:24:54,045][53852] Updated weights for policy 0, policy_version 92110 (0.0007) [2023-10-08 11:24:54,050][53885] Updated weights for policy 1, policy_version 91672 (0.0009) [2023-10-08 11:24:54,414][53852] Updated weights for policy 0, policy_version 92120 (0.0007) [2023-10-08 11:24:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 188219392. Throughput: 0: 1809.8, 1: 1803.1. Samples: 47059474. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:24:57,016][52710] Avg episode reward: [(0, '33.630'), (1, '35.380')] [2023-10-08 11:24:57,535][53885] Updated weights for policy 1, policy_version 91682 (0.0007) [2023-10-08 11:24:57,908][53885] Updated weights for policy 1, policy_version 91692 (0.0007) [2023-10-08 11:24:58,055][53852] Updated weights for policy 0, policy_version 92130 (0.0007) [2023-10-08 11:24:58,266][53885] Updated weights for policy 1, policy_version 91702 (0.0008) [2023-10-08 11:24:58,425][53852] Updated weights for policy 0, policy_version 92140 (0.0008) [2023-10-08 11:24:58,629][53885] Updated weights for policy 1, policy_version 91712 (0.0007) [2023-10-08 11:24:58,795][53852] Updated weights for policy 0, policy_version 92150 (0.0009) [2023-10-08 11:24:59,167][53852] Updated weights for policy 0, policy_version 92160 (0.0007) [2023-10-08 11:25:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188284928. Throughput: 0: 1818.7, 1: 1816.6. Samples: 47082678. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:25:02,016][52710] Avg episode reward: [(0, '33.650'), (1, '36.430')] [2023-10-08 11:25:02,332][53885] Updated weights for policy 1, policy_version 91722 (0.0007) [2023-10-08 11:25:02,671][53852] Updated weights for policy 0, policy_version 92170 (0.0008) [2023-10-08 11:25:02,709][53885] Updated weights for policy 1, policy_version 91732 (0.0007) [2023-10-08 11:25:03,041][53852] Updated weights for policy 0, policy_version 92180 (0.0009) [2023-10-08 11:25:03,079][53885] Updated weights for policy 1, policy_version 91742 (0.0008) [2023-10-08 11:25:03,420][53852] Updated weights for policy 0, policy_version 92190 (0.0010) [2023-10-08 11:25:06,817][53885] Updated weights for policy 1, policy_version 91752 (0.0008) [2023-10-08 11:25:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188350464. Throughput: 0: 1824.1, 1: 1816.8. Samples: 47105388. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) [2023-10-08 11:25:07,016][52710] Avg episode reward: [(0, '34.530'), (1, '40.760')] [2023-10-08 11:25:07,178][53885] Updated weights for policy 1, policy_version 91762 (0.0008) [2023-10-08 11:25:07,200][53852] Updated weights for policy 0, policy_version 92200 (0.0007) [2023-10-08 11:25:07,546][53885] Updated weights for policy 1, policy_version 91772 (0.0008) [2023-10-08 11:25:07,563][53852] Updated weights for policy 0, policy_version 92210 (0.0007) [2023-10-08 11:25:07,940][53852] Updated weights for policy 0, policy_version 92220 (0.0008) [2023-10-08 11:25:11,346][53885] Updated weights for policy 1, policy_version 91782 (0.0007) [2023-10-08 11:25:11,477][53852] Updated weights for policy 0, policy_version 92230 (0.0007) [2023-10-08 11:25:11,719][53885] Updated weights for policy 1, policy_version 91792 (0.0007) [2023-10-08 11:25:11,852][53852] Updated weights for policy 0, policy_version 92240 (0.0007) [2023-10-08 11:25:12,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188416000. Throughput: 0: 1825.3, 1: 1813.2. Samples: 47115416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:12,016][52710] Avg episode reward: [(0, '34.490'), (1, '35.620')] [2023-10-08 11:25:12,078][53885] Updated weights for policy 1, policy_version 91802 (0.0007) [2023-10-08 11:25:12,221][53852] Updated weights for policy 0, policy_version 92250 (0.0007) [2023-10-08 11:25:15,643][53885] Updated weights for policy 1, policy_version 91812 (0.0008) [2023-10-08 11:25:15,899][53852] Updated weights for policy 0, policy_version 92260 (0.0008) [2023-10-08 11:25:16,005][53885] Updated weights for policy 1, policy_version 91822 (0.0007) [2023-10-08 11:25:16,274][53852] Updated weights for policy 0, policy_version 92270 (0.0008) [2023-10-08 11:25:16,370][53885] Updated weights for policy 1, policy_version 91832 (0.0007) [2023-10-08 11:25:16,646][53852] Updated weights for policy 0, policy_version 92280 (0.0007) [2023-10-08 11:25:17,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 188547072. Throughput: 0: 1828.2, 1: 1817.2. Samples: 47138328. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:17,016][52710] Avg episode reward: [(0, '35.920'), (1, '35.690')] [2023-10-08 11:25:20,262][53885] Updated weights for policy 1, policy_version 91842 (0.0007) [2023-10-08 11:25:20,279][53852] Updated weights for policy 0, policy_version 92290 (0.0007) [2023-10-08 11:25:20,640][53852] Updated weights for policy 0, policy_version 92300 (0.0008) [2023-10-08 11:25:20,659][53885] Updated weights for policy 1, policy_version 91852 (0.0008) [2023-10-08 11:25:21,016][53852] Updated weights for policy 0, policy_version 92310 (0.0008) [2023-10-08 11:25:21,029][53885] Updated weights for policy 1, policy_version 91862 (0.0007) [2023-10-08 11:25:21,385][53852] Updated weights for policy 0, policy_version 92320 (0.0009) [2023-10-08 11:25:21,387][53885] Updated weights for policy 1, policy_version 91872 (0.0007) [2023-10-08 11:25:22,015][52710] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 188612608. Throughput: 0: 1824.0, 1: 1813.6. Samples: 47158066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:22,016][52710] Avg episode reward: [(0, '35.590'), (1, '38.080')] [2023-10-08 11:25:24,921][53885] Updated weights for policy 1, policy_version 91882 (0.0007) [2023-10-08 11:25:25,127][53852] Updated weights for policy 0, policy_version 92330 (0.0007) [2023-10-08 11:25:25,283][53885] Updated weights for policy 1, policy_version 91892 (0.0007) [2023-10-08 11:25:25,487][53852] Updated weights for policy 0, policy_version 92340 (0.0008) [2023-10-08 11:25:25,652][53885] Updated weights for policy 1, policy_version 91902 (0.0009) [2023-10-08 11:25:25,853][53852] Updated weights for policy 0, policy_version 92350 (0.0009) [2023-10-08 11:25:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 188678144. Throughput: 0: 1834.3, 1: 1816.2. Samples: 47170966. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:27,016][52710] Avg episode reward: [(0, '33.790'), (1, '32.510')] [2023-10-08 11:25:29,300][53885] Updated weights for policy 1, policy_version 91912 (0.0009) [2023-10-08 11:25:29,471][53852] Updated weights for policy 0, policy_version 92360 (0.0007) [2023-10-08 11:25:29,674][53885] Updated weights for policy 1, policy_version 91922 (0.0009) [2023-10-08 11:25:29,837][53852] Updated weights for policy 0, policy_version 92370 (0.0007) [2023-10-08 11:25:30,042][53885] Updated weights for policy 1, policy_version 91932 (0.0008) [2023-10-08 11:25:30,208][53852] Updated weights for policy 0, policy_version 92380 (0.0007) [2023-10-08 11:25:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 188743680. Throughput: 0: 1826.4, 1: 1817.0. Samples: 47191082. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:32,016][52710] Avg episode reward: [(0, '39.470'), (1, '34.730')] [2023-10-08 11:25:33,560][53885] Updated weights for policy 1, policy_version 91942 (0.0007) [2023-10-08 11:25:33,925][53885] Updated weights for policy 1, policy_version 91952 (0.0009) [2023-10-08 11:25:34,034][53852] Updated weights for policy 0, policy_version 92390 (0.0008) [2023-10-08 11:25:34,290][53885] Updated weights for policy 1, policy_version 91962 (0.0008) [2023-10-08 11:25:34,406][53852] Updated weights for policy 0, policy_version 92400 (0.0008) [2023-10-08 11:25:34,772][53852] Updated weights for policy 0, policy_version 92410 (0.0009) [2023-10-08 11:25:37,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 188809216. Throughput: 0: 1825.8, 1: 1835.6. Samples: 47214290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:37,016][52710] Avg episode reward: [(0, '38.000'), (1, '37.800')] [2023-10-08 11:25:37,879][53885] Updated weights for policy 1, policy_version 91972 (0.0009) [2023-10-08 11:25:38,241][53885] Updated weights for policy 1, policy_version 91982 (0.0007) [2023-10-08 11:25:38,565][53852] Updated weights for policy 0, policy_version 92420 (0.0009) [2023-10-08 11:25:38,614][53885] Updated weights for policy 1, policy_version 91992 (0.0007) [2023-10-08 11:25:38,943][53852] Updated weights for policy 0, policy_version 92430 (0.0008) [2023-10-08 11:25:39,312][53852] Updated weights for policy 0, policy_version 92440 (0.0010) [2023-10-08 11:25:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 188874752. Throughput: 0: 1822.0, 1: 1840.2. Samples: 47224274. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:42,015][52710] Avg episode reward: [(0, '37.530'), (1, '32.660')] [2023-10-08 11:25:42,183][53885] Updated weights for policy 1, policy_version 92002 (0.0008) [2023-10-08 11:25:42,541][53885] Updated weights for policy 1, policy_version 92012 (0.0010) [2023-10-08 11:25:42,911][53885] Updated weights for policy 1, policy_version 92022 (0.0009) [2023-10-08 11:25:43,012][53852] Updated weights for policy 0, policy_version 92450 (0.0009) [2023-10-08 11:25:43,268][53885] Updated weights for policy 1, policy_version 92032 (0.0009) [2023-10-08 11:25:43,378][53852] Updated weights for policy 0, policy_version 92460 (0.0008) [2023-10-08 11:25:43,742][53852] Updated weights for policy 0, policy_version 92470 (0.0010) [2023-10-08 11:25:44,109][53852] Updated weights for policy 0, policy_version 92480 (0.0010) [2023-10-08 11:25:47,004][53885] Updated weights for policy 1, policy_version 92042 (0.0009) [2023-10-08 11:25:47,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188940288. Throughput: 0: 1815.6, 1: 1834.1. Samples: 47246916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:47,016][52710] Avg episode reward: [(0, '35.480'), (1, '35.360')] [2023-10-08 11:25:47,382][53885] Updated weights for policy 1, policy_version 92052 (0.0008) [2023-10-08 11:25:47,743][53885] Updated weights for policy 1, policy_version 92062 (0.0008) [2023-10-08 11:25:47,842][53852] Updated weights for policy 0, policy_version 92490 (0.0007) [2023-10-08 11:25:48,199][53852] Updated weights for policy 0, policy_version 92500 (0.0011) [2023-10-08 11:25:48,564][53852] Updated weights for policy 0, policy_version 92510 (0.0010) [2023-10-08 11:25:51,474][53885] Updated weights for policy 1, policy_version 92072 (0.0007) [2023-10-08 11:25:51,845][53885] Updated weights for policy 1, policy_version 92082 (0.0009) [2023-10-08 11:25:52,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189005824. Throughput: 0: 1818.1, 1: 1827.7. Samples: 47269448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:52,015][52710] Avg episode reward: [(0, '37.170'), (1, '39.480')] [2023-10-08 11:25:52,207][53852] Updated weights for policy 0, policy_version 92520 (0.0008) [2023-10-08 11:25:52,216][53885] Updated weights for policy 1, policy_version 92092 (0.0009) [2023-10-08 11:25:52,571][53852] Updated weights for policy 0, policy_version 92530 (0.0007) [2023-10-08 11:25:52,944][53852] Updated weights for policy 0, policy_version 92540 (0.0007) [2023-10-08 11:25:55,836][53885] Updated weights for policy 1, policy_version 92102 (0.0008) [2023-10-08 11:25:56,207][53885] Updated weights for policy 1, policy_version 92112 (0.0008) [2023-10-08 11:25:56,570][53885] Updated weights for policy 1, policy_version 92122 (0.0007) [2023-10-08 11:25:56,749][53852] Updated weights for policy 0, policy_version 92550 (0.0009) [2023-10-08 11:25:57,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189104128. Throughput: 0: 1815.1, 1: 1839.0. Samples: 47279850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:25:57,016][52710] Avg episode reward: [(0, '37.990'), (1, '34.470')] [2023-10-08 11:25:57,120][53852] Updated weights for policy 0, policy_version 92560 (0.0007) [2023-10-08 11:25:57,496][53852] Updated weights for policy 0, policy_version 92570 (0.0007) [2023-10-08 11:26:00,243][53885] Updated weights for policy 1, policy_version 92132 (0.0009) [2023-10-08 11:26:00,614][53885] Updated weights for policy 1, policy_version 92142 (0.0008) [2023-10-08 11:26:00,984][53885] Updated weights for policy 1, policy_version 92152 (0.0008) [2023-10-08 11:26:01,100][53852] Updated weights for policy 0, policy_version 92580 (0.0007) [2023-10-08 11:26:01,475][53852] Updated weights for policy 0, policy_version 92590 (0.0010) [2023-10-08 11:26:01,850][53852] Updated weights for policy 0, policy_version 92600 (0.0008) [2023-10-08 11:26:02,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189169664. Throughput: 0: 1820.1, 1: 1820.3. Samples: 47302146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:26:02,015][52710] Avg episode reward: [(0, '36.570'), (1, '35.410')] [2023-10-08 11:26:04,851][53885] Updated weights for policy 1, policy_version 92162 (0.0008) [2023-10-08 11:26:05,241][53885] Updated weights for policy 1, policy_version 92172 (0.0008) [2023-10-08 11:26:05,545][53852] Updated weights for policy 0, policy_version 92610 (0.0009) [2023-10-08 11:26:05,619][53885] Updated weights for policy 1, policy_version 92182 (0.0008) [2023-10-08 11:26:05,917][53852] Updated weights for policy 0, policy_version 92620 (0.0010) [2023-10-08 11:26:05,975][53885] Updated weights for policy 1, policy_version 92192 (0.0008) [2023-10-08 11:26:06,281][53852] Updated weights for policy 0, policy_version 92630 (0.0008) [2023-10-08 11:26:06,655][53852] Updated weights for policy 0, policy_version 92640 (0.0007) [2023-10-08 11:26:07,015][52710] Fps is (10 sec: 16383.4, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 189267968. Throughput: 0: 1821.1, 1: 1834.2. Samples: 47322552. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:07,016][52710] Avg episode reward: [(0, '37.110'), (1, '38.350')] [2023-10-08 11:26:07,027][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000092192_94404608.pth... [2023-10-08 11:26:07,027][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000092640_94863360.pth... [2023-10-08 11:26:07,079][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000090912_93093888.pth [2023-10-08 11:26:07,079][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000090496_92667904.pth [2023-10-08 11:26:09,700][53885] Updated weights for policy 1, policy_version 92202 (0.0011) [2023-10-08 11:26:10,078][53885] Updated weights for policy 1, policy_version 92212 (0.0009) [2023-10-08 11:26:10,429][53852] Updated weights for policy 0, policy_version 92650 (0.0008) [2023-10-08 11:26:10,439][53885] Updated weights for policy 1, policy_version 92222 (0.0008) [2023-10-08 11:26:10,799][53852] Updated weights for policy 0, policy_version 92660 (0.0008) [2023-10-08 11:26:11,161][53852] Updated weights for policy 0, policy_version 92670 (0.0008) [2023-10-08 11:26:12,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 189333504. Throughput: 0: 1817.4, 1: 1823.5. Samples: 47334806. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:12,016][52710] Avg episode reward: [(0, '33.340'), (1, '35.440')] [2023-10-08 11:26:14,167][53885] Updated weights for policy 1, policy_version 92232 (0.0009) [2023-10-08 11:26:14,537][53885] Updated weights for policy 1, policy_version 92242 (0.0010) [2023-10-08 11:26:14,896][53885] Updated weights for policy 1, policy_version 92252 (0.0009) [2023-10-08 11:26:14,900][53852] Updated weights for policy 0, policy_version 92680 (0.0008) [2023-10-08 11:26:15,274][53852] Updated weights for policy 0, policy_version 92690 (0.0008) [2023-10-08 11:26:15,640][53852] Updated weights for policy 0, policy_version 92700 (0.0009) [2023-10-08 11:26:17,015][52710] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 189399040. Throughput: 0: 1822.2, 1: 1826.6. Samples: 47355276. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:17,015][52710] Avg episode reward: [(0, '34.090'), (1, '34.480')] [2023-10-08 11:26:18,480][53885] Updated weights for policy 1, policy_version 92262 (0.0008) [2023-10-08 11:26:18,850][53885] Updated weights for policy 1, policy_version 92272 (0.0008) [2023-10-08 11:26:19,214][53885] Updated weights for policy 1, policy_version 92282 (0.0008) [2023-10-08 11:26:19,413][53852] Updated weights for policy 0, policy_version 92710 (0.0011) [2023-10-08 11:26:19,775][53852] Updated weights for policy 0, policy_version 92720 (0.0009) [2023-10-08 11:26:20,152][53852] Updated weights for policy 0, policy_version 92730 (0.0009) [2023-10-08 11:26:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 189464576. Throughput: 0: 1821.2, 1: 1819.9. Samples: 47378140. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:22,016][52710] Avg episode reward: [(0, '36.300'), (1, '37.270')] [2023-10-08 11:26:22,782][53885] Updated weights for policy 1, policy_version 92292 (0.0007) [2023-10-08 11:26:23,154][53885] Updated weights for policy 1, policy_version 92302 (0.0008) [2023-10-08 11:26:23,526][53885] Updated weights for policy 1, policy_version 92312 (0.0008) [2023-10-08 11:26:23,763][53852] Updated weights for policy 0, policy_version 92740 (0.0010) [2023-10-08 11:26:24,144][53852] Updated weights for policy 0, policy_version 92750 (0.0009) [2023-10-08 11:26:24,519][53852] Updated weights for policy 0, policy_version 92760 (0.0011) [2023-10-08 11:26:27,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 189530112. Throughput: 0: 1831.2, 1: 1820.4. Samples: 47388594. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:27,015][52710] Avg episode reward: [(0, '34.780'), (1, '36.560')] [2023-10-08 11:26:27,309][53885] Updated weights for policy 1, policy_version 92322 (0.0008) [2023-10-08 11:26:27,668][53885] Updated weights for policy 1, policy_version 92332 (0.0008) [2023-10-08 11:26:27,985][53852] Updated weights for policy 0, policy_version 92770 (0.0007) [2023-10-08 11:26:28,038][53885] Updated weights for policy 1, policy_version 92342 (0.0008) [2023-10-08 11:26:28,347][53852] Updated weights for policy 0, policy_version 92780 (0.0008) [2023-10-08 11:26:28,400][53885] Updated weights for policy 1, policy_version 92352 (0.0007) [2023-10-08 11:26:28,715][53852] Updated weights for policy 0, policy_version 92790 (0.0009) [2023-10-08 11:26:29,084][53852] Updated weights for policy 0, policy_version 92800 (0.0007) [2023-10-08 11:26:32,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189595648. Throughput: 0: 1831.7, 1: 1816.8. Samples: 47411100. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:32,015][52710] Avg episode reward: [(0, '35.600'), (1, '32.950')] [2023-10-08 11:26:32,161][53885] Updated weights for policy 1, policy_version 92362 (0.0007) [2023-10-08 11:26:32,522][53885] Updated weights for policy 1, policy_version 92372 (0.0009) [2023-10-08 11:26:32,692][53852] Updated weights for policy 0, policy_version 92810 (0.0008) [2023-10-08 11:26:32,887][53885] Updated weights for policy 1, policy_version 92382 (0.0007) [2023-10-08 11:26:33,066][53852] Updated weights for policy 0, policy_version 92820 (0.0008) [2023-10-08 11:26:33,439][53852] Updated weights for policy 0, policy_version 92830 (0.0010) [2023-10-08 11:26:36,600][53885] Updated weights for policy 1, policy_version 92392 (0.0008) [2023-10-08 11:26:36,960][53885] Updated weights for policy 1, policy_version 92402 (0.0007) [2023-10-08 11:26:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189661184. Throughput: 0: 1828.6, 1: 1823.3. Samples: 47433784. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:37,016][52710] Avg episode reward: [(0, '38.340'), (1, '36.030')] [2023-10-08 11:26:37,040][53852] Updated weights for policy 0, policy_version 92840 (0.0007) [2023-10-08 11:26:37,324][53885] Updated weights for policy 1, policy_version 92412 (0.0007) [2023-10-08 11:26:37,410][53852] Updated weights for policy 0, policy_version 92850 (0.0008) [2023-10-08 11:26:37,787][53852] Updated weights for policy 0, policy_version 92860 (0.0009) [2023-10-08 11:26:41,168][53885] Updated weights for policy 1, policy_version 92422 (0.0009) [2023-10-08 11:26:41,482][53852] Updated weights for policy 0, policy_version 92870 (0.0008) [2023-10-08 11:26:41,548][53885] Updated weights for policy 1, policy_version 92432 (0.0010) [2023-10-08 11:26:41,846][53852] Updated weights for policy 0, policy_version 92880 (0.0007) [2023-10-08 11:26:41,923][53885] Updated weights for policy 1, policy_version 92442 (0.0008) [2023-10-08 11:26:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189726720. Throughput: 0: 1830.0, 1: 1815.7. Samples: 47443906. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:42,015][52710] Avg episode reward: [(0, '34.240'), (1, '37.290')] [2023-10-08 11:26:42,214][53852] Updated weights for policy 0, policy_version 92890 (0.0008) [2023-10-08 11:26:45,422][53885] Updated weights for policy 1, policy_version 92452 (0.0009) [2023-10-08 11:26:45,777][53885] Updated weights for policy 1, policy_version 92462 (0.0009) [2023-10-08 11:26:46,014][53852] Updated weights for policy 0, policy_version 92900 (0.0009) [2023-10-08 11:26:46,150][53885] Updated weights for policy 1, policy_version 92472 (0.0007) [2023-10-08 11:26:46,383][53852] Updated weights for policy 0, policy_version 92910 (0.0009) [2023-10-08 11:26:46,753][53852] Updated weights for policy 0, policy_version 92920 (0.0007) [2023-10-08 11:26:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189825024. Throughput: 0: 1822.4, 1: 1821.9. Samples: 47466142. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:47,016][52710] Avg episode reward: [(0, '33.560'), (1, '29.970')] [2023-10-08 11:26:49,841][53885] Updated weights for policy 1, policy_version 92482 (0.0009) [2023-10-08 11:26:50,252][53885] Updated weights for policy 1, policy_version 92492 (0.0010) [2023-10-08 11:26:50,463][53852] Updated weights for policy 0, policy_version 92930 (0.0007) [2023-10-08 11:26:50,612][53885] Updated weights for policy 1, policy_version 92502 (0.0009) [2023-10-08 11:26:50,833][53852] Updated weights for policy 0, policy_version 92940 (0.0007) [2023-10-08 11:26:50,975][53885] Updated weights for policy 1, policy_version 92512 (0.0007) [2023-10-08 11:26:51,192][53852] Updated weights for policy 0, policy_version 92950 (0.0008) [2023-10-08 11:26:51,563][53852] Updated weights for policy 0, policy_version 92960 (0.0007) [2023-10-08 11:26:52,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 189923328. Throughput: 0: 1819.9, 1: 1818.3. Samples: 47486270. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:52,015][52710] Avg episode reward: [(0, '30.640'), (1, '33.770')] [2023-10-08 11:26:54,519][53885] Updated weights for policy 1, policy_version 92522 (0.0010) [2023-10-08 11:26:54,878][53885] Updated weights for policy 1, policy_version 92532 (0.0009) [2023-10-08 11:26:55,242][53885] Updated weights for policy 1, policy_version 92542 (0.0008) [2023-10-08 11:26:55,268][53852] Updated weights for policy 0, policy_version 92970 (0.0008) [2023-10-08 11:26:55,629][53852] Updated weights for policy 0, policy_version 92980 (0.0009) [2023-10-08 11:26:56,007][53852] Updated weights for policy 0, policy_version 92990 (0.0010) [2023-10-08 11:26:57,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189988864. Throughput: 0: 1822.9, 1: 1820.3. Samples: 47498750. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:26:57,016][52710] Avg episode reward: [(0, '34.050'), (1, '33.560')] [2023-10-08 11:26:59,027][53885] Updated weights for policy 1, policy_version 92552 (0.0009) [2023-10-08 11:26:59,403][53885] Updated weights for policy 1, policy_version 92562 (0.0010) [2023-10-08 11:26:59,761][53885] Updated weights for policy 1, policy_version 92572 (0.0008) [2023-10-08 11:26:59,820][53852] Updated weights for policy 0, policy_version 93000 (0.0007) [2023-10-08 11:27:00,179][53852] Updated weights for policy 0, policy_version 93010 (0.0007) [2023-10-08 11:27:00,549][53852] Updated weights for policy 0, policy_version 93020 (0.0008) [2023-10-08 11:27:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190054400. Throughput: 0: 1816.7, 1: 1825.5. Samples: 47519176. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) [2023-10-08 11:27:02,016][52710] Avg episode reward: [(0, '32.740'), (1, '34.680')] [2023-10-08 11:27:03,442][53885] Updated weights for policy 1, policy_version 92582 (0.0008) [2023-10-08 11:27:03,797][53885] Updated weights for policy 1, policy_version 92592 (0.0011) [2023-10-08 11:27:04,083][53852] Updated weights for policy 0, policy_version 93030 (0.0008) [2023-10-08 11:27:04,170][53885] Updated weights for policy 1, policy_version 92602 (0.0010) [2023-10-08 11:27:04,447][53852] Updated weights for policy 0, policy_version 93040 (0.0007) [2023-10-08 11:27:04,814][53852] Updated weights for policy 0, policy_version 93050 (0.0007) [2023-10-08 11:27:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190119936. Throughput: 0: 1822.0, 1: 1819.6. Samples: 47542012. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:07,016][52710] Avg episode reward: [(0, '36.480'), (1, '34.110')] [2023-10-08 11:27:07,831][53885] Updated weights for policy 1, policy_version 92612 (0.0008) [2023-10-08 11:27:08,207][53885] Updated weights for policy 1, policy_version 92622 (0.0008) [2023-10-08 11:27:08,478][53852] Updated weights for policy 0, policy_version 93060 (0.0008) [2023-10-08 11:27:08,577][53885] Updated weights for policy 1, policy_version 92632 (0.0008) [2023-10-08 11:27:08,857][53852] Updated weights for policy 0, policy_version 93070 (0.0008) [2023-10-08 11:27:09,225][53852] Updated weights for policy 0, policy_version 93080 (0.0009) [2023-10-08 11:27:12,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190185472. Throughput: 0: 1811.1, 1: 1821.0. Samples: 47552040. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:12,016][52710] Avg episode reward: [(0, '37.640'), (1, '30.680')] [2023-10-08 11:27:12,083][53885] Updated weights for policy 1, policy_version 92642 (0.0007) [2023-10-08 11:27:12,463][53885] Updated weights for policy 1, policy_version 92652 (0.0009) [2023-10-08 11:27:12,833][53885] Updated weights for policy 1, policy_version 92662 (0.0009) [2023-10-08 11:27:12,855][53852] Updated weights for policy 0, policy_version 93090 (0.0009) [2023-10-08 11:27:13,197][53885] Updated weights for policy 1, policy_version 92672 (0.0009) [2023-10-08 11:27:13,230][53852] Updated weights for policy 0, policy_version 93100 (0.0008) [2023-10-08 11:27:13,590][53852] Updated weights for policy 0, policy_version 93110 (0.0008) [2023-10-08 11:27:13,961][53852] Updated weights for policy 0, policy_version 93120 (0.0009) [2023-10-08 11:27:16,926][53885] Updated weights for policy 1, policy_version 92682 (0.0008) [2023-10-08 11:27:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 190251008. Throughput: 0: 1817.0, 1: 1828.5. Samples: 47575146. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:17,016][52710] Avg episode reward: [(0, '36.790'), (1, '29.620')] [2023-10-08 11:27:17,287][53885] Updated weights for policy 1, policy_version 92692 (0.0007) [2023-10-08 11:27:17,624][53852] Updated weights for policy 0, policy_version 93130 (0.0007) [2023-10-08 11:27:17,667][53885] Updated weights for policy 1, policy_version 92702 (0.0009) [2023-10-08 11:27:17,991][53852] Updated weights for policy 0, policy_version 93140 (0.0007) [2023-10-08 11:27:18,354][53852] Updated weights for policy 0, policy_version 93150 (0.0007) [2023-10-08 11:27:21,257][53885] Updated weights for policy 1, policy_version 92712 (0.0011) [2023-10-08 11:27:21,630][53885] Updated weights for policy 1, policy_version 92722 (0.0008) [2023-10-08 11:27:21,993][53885] Updated weights for policy 1, policy_version 92732 (0.0008) [2023-10-08 11:27:22,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 190316544. Throughput: 0: 1817.2, 1: 1815.0. Samples: 47597232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:22,016][52710] Avg episode reward: [(0, '35.780'), (1, '33.410')] [2023-10-08 11:27:22,063][53852] Updated weights for policy 0, policy_version 93160 (0.0009) [2023-10-08 11:27:22,434][53852] Updated weights for policy 0, policy_version 93170 (0.0008) [2023-10-08 11:27:22,806][53852] Updated weights for policy 0, policy_version 93180 (0.0008) [2023-10-08 11:27:25,721][53885] Updated weights for policy 1, policy_version 92742 (0.0010) [2023-10-08 11:27:26,092][53885] Updated weights for policy 1, policy_version 92752 (0.0007) [2023-10-08 11:27:26,304][53852] Updated weights for policy 0, policy_version 93190 (0.0008) [2023-10-08 11:27:26,463][53885] Updated weights for policy 1, policy_version 92762 (0.0008) [2023-10-08 11:27:26,674][53852] Updated weights for policy 0, policy_version 93200 (0.0008) [2023-10-08 11:27:27,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 190414848. Throughput: 0: 1816.9, 1: 1829.1. Samples: 47607974. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:27,016][52710] Avg episode reward: [(0, '39.000'), (1, '34.970')] [2023-10-08 11:27:27,058][53852] Updated weights for policy 0, policy_version 93210 (0.0010) [2023-10-08 11:27:30,317][53885] Updated weights for policy 1, policy_version 92772 (0.0010) [2023-10-08 11:27:30,682][53885] Updated weights for policy 1, policy_version 92782 (0.0007) [2023-10-08 11:27:30,762][53852] Updated weights for policy 0, policy_version 93220 (0.0009) [2023-10-08 11:27:31,043][53885] Updated weights for policy 1, policy_version 92792 (0.0007) [2023-10-08 11:27:31,134][53852] Updated weights for policy 0, policy_version 93230 (0.0007) [2023-10-08 11:27:31,503][53852] Updated weights for policy 0, policy_version 93240 (0.0007) [2023-10-08 11:27:32,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 190513152. Throughput: 0: 1823.7, 1: 1825.7. Samples: 47630366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:32,016][52710] Avg episode reward: [(0, '37.400'), (1, '34.530')] [2023-10-08 11:27:34,821][53885] Updated weights for policy 1, policy_version 92802 (0.0008) [2023-10-08 11:27:35,119][53852] Updated weights for policy 0, policy_version 93250 (0.0008) [2023-10-08 11:27:35,226][53885] Updated weights for policy 1, policy_version 92812 (0.0008) [2023-10-08 11:27:35,481][53852] Updated weights for policy 0, policy_version 93260 (0.0008) [2023-10-08 11:27:35,595][53885] Updated weights for policy 1, policy_version 92822 (0.0007) [2023-10-08 11:27:35,855][53852] Updated weights for policy 0, policy_version 93270 (0.0008) [2023-10-08 11:27:35,963][53885] Updated weights for policy 1, policy_version 92832 (0.0009) [2023-10-08 11:27:36,209][53852] Updated weights for policy 0, policy_version 93280 (0.0007) [2023-10-08 11:27:37,015][52710] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 190578688. Throughput: 0: 1826.4, 1: 1829.7. Samples: 47650794. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:37,016][52710] Avg episode reward: [(0, '33.260'), (1, '34.830')] [2023-10-08 11:27:39,893][53885] Updated weights for policy 1, policy_version 92842 (0.0007) [2023-10-08 11:27:40,020][53852] Updated weights for policy 0, policy_version 93290 (0.0007) [2023-10-08 11:27:40,265][53885] Updated weights for policy 1, policy_version 92852 (0.0009) [2023-10-08 11:27:40,396][53852] Updated weights for policy 0, policy_version 93300 (0.0009) [2023-10-08 11:27:40,636][53885] Updated weights for policy 1, policy_version 92862 (0.0009) [2023-10-08 11:27:40,766][53852] Updated weights for policy 0, policy_version 93310 (0.0007) [2023-10-08 11:27:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 190644224. Throughput: 0: 1828.4, 1: 1826.8. Samples: 47663230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:42,016][52710] Avg episode reward: [(0, '35.750'), (1, '39.390')] [2023-10-08 11:27:44,319][53885] Updated weights for policy 1, policy_version 92872 (0.0007) [2023-10-08 11:27:44,393][53852] Updated weights for policy 0, policy_version 93320 (0.0009) [2023-10-08 11:27:44,692][53885] Updated weights for policy 1, policy_version 92882 (0.0007) [2023-10-08 11:27:44,767][53852] Updated weights for policy 0, policy_version 93330 (0.0009) [2023-10-08 11:27:45,051][53885] Updated weights for policy 1, policy_version 92892 (0.0007) [2023-10-08 11:27:45,131][53852] Updated weights for policy 0, policy_version 93340 (0.0007) [2023-10-08 11:27:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190709760. Throughput: 0: 1826.9, 1: 1811.0. Samples: 47682882. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:47,016][52710] Avg episode reward: [(0, '38.060'), (1, '36.850')] [2023-10-08 11:27:48,711][53885] Updated weights for policy 1, policy_version 92902 (0.0008) [2023-10-08 11:27:48,896][53852] Updated weights for policy 0, policy_version 93350 (0.0007) [2023-10-08 11:27:49,074][53885] Updated weights for policy 1, policy_version 92912 (0.0009) [2023-10-08 11:27:49,270][53852] Updated weights for policy 0, policy_version 93360 (0.0008) [2023-10-08 11:27:49,442][53885] Updated weights for policy 1, policy_version 92922 (0.0009) [2023-10-08 11:27:49,639][53852] Updated weights for policy 0, policy_version 93370 (0.0008) [2023-10-08 11:27:52,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 190775296. Throughput: 0: 1828.9, 1: 1805.2. Samples: 47705546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:52,016][52710] Avg episode reward: [(0, '31.460'), (1, '34.600')] [2023-10-08 11:27:53,246][53885] Updated weights for policy 1, policy_version 92932 (0.0008) [2023-10-08 11:27:53,374][53852] Updated weights for policy 0, policy_version 93380 (0.0009) [2023-10-08 11:27:53,607][53885] Updated weights for policy 1, policy_version 92942 (0.0008) [2023-10-08 11:27:53,759][53852] Updated weights for policy 0, policy_version 93390 (0.0008) [2023-10-08 11:27:53,973][53885] Updated weights for policy 1, policy_version 92952 (0.0008) [2023-10-08 11:27:54,132][53852] Updated weights for policy 0, policy_version 93400 (0.0009) [2023-10-08 11:27:57,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 190840832. Throughput: 0: 1831.8, 1: 1798.5. Samples: 47715404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:27:57,016][52710] Avg episode reward: [(0, '37.340'), (1, '37.940')] [2023-10-08 11:27:57,705][53885] Updated weights for policy 1, policy_version 92962 (0.0007) [2023-10-08 11:27:57,800][53852] Updated weights for policy 0, policy_version 93410 (0.0008) [2023-10-08 11:27:58,072][53885] Updated weights for policy 1, policy_version 92972 (0.0007) [2023-10-08 11:27:58,165][53852] Updated weights for policy 0, policy_version 93420 (0.0008) [2023-10-08 11:27:58,445][53885] Updated weights for policy 1, policy_version 92982 (0.0008) [2023-10-08 11:27:58,537][53852] Updated weights for policy 0, policy_version 93430 (0.0008) [2023-10-08 11:27:58,810][53885] Updated weights for policy 1, policy_version 92992 (0.0007) [2023-10-08 11:27:58,908][53852] Updated weights for policy 0, policy_version 93440 (0.0008) [2023-10-08 11:28:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 190906368. Throughput: 0: 1833.7, 1: 1792.6. Samples: 47738330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:28:02,016][52710] Avg episode reward: [(0, '37.970'), (1, '33.710')] [2023-10-08 11:28:02,489][53885] Updated weights for policy 1, policy_version 93002 (0.0010) [2023-10-08 11:28:02,547][53852] Updated weights for policy 0, policy_version 93450 (0.0007) [2023-10-08 11:28:02,845][53885] Updated weights for policy 1, policy_version 93012 (0.0010) [2023-10-08 11:28:02,927][53852] Updated weights for policy 0, policy_version 93460 (0.0007) [2023-10-08 11:28:03,208][53885] Updated weights for policy 1, policy_version 93022 (0.0010) [2023-10-08 11:28:03,293][53852] Updated weights for policy 0, policy_version 93470 (0.0008) [2023-10-08 11:28:06,868][53885] Updated weights for policy 1, policy_version 93032 (0.0007) [2023-10-08 11:28:07,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 190971904. Throughput: 0: 1825.5, 1: 1813.9. Samples: 47761004. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:07,016][52710] Avg episode reward: [(0, '37.720'), (1, '32.520')] [2023-10-08 11:28:07,048][53852] Updated weights for policy 0, policy_version 93480 (0.0008) [2023-10-08 11:28:07,238][53885] Updated weights for policy 1, policy_version 93042 (0.0007) [2023-10-08 11:28:07,415][53852] Updated weights for policy 0, policy_version 93490 (0.0007) [2023-10-08 11:28:07,601][53885] Updated weights for policy 1, policy_version 93052 (0.0007) [2023-10-08 11:28:07,744][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000093056_95289344.pth... [2023-10-08 11:28:07,773][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000091328_93519872.pth [2023-10-08 11:28:07,785][53852] Updated weights for policy 0, policy_version 93500 (0.0008) [2023-10-08 11:28:07,931][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000093504_95748096.pth... [2023-10-08 11:28:07,962][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000091776_93978624.pth [2023-10-08 11:28:11,260][53885] Updated weights for policy 1, policy_version 93062 (0.0009) [2023-10-08 11:28:11,499][53852] Updated weights for policy 0, policy_version 93510 (0.0008) [2023-10-08 11:28:11,629][53885] Updated weights for policy 1, policy_version 93072 (0.0009) [2023-10-08 11:28:11,872][53852] Updated weights for policy 0, policy_version 93520 (0.0008) [2023-10-08 11:28:11,993][53885] Updated weights for policy 1, policy_version 93082 (0.0007) [2023-10-08 11:28:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191037440. Throughput: 0: 1825.4, 1: 1794.2. Samples: 47770856. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:12,016][52710] Avg episode reward: [(0, '34.050'), (1, '37.440')] [2023-10-08 11:28:12,242][53852] Updated weights for policy 0, policy_version 93530 (0.0008) [2023-10-08 11:28:15,738][53885] Updated weights for policy 1, policy_version 93092 (0.0009) [2023-10-08 11:28:16,053][53852] Updated weights for policy 0, policy_version 93540 (0.0008) [2023-10-08 11:28:16,106][53885] Updated weights for policy 1, policy_version 93102 (0.0008) [2023-10-08 11:28:16,427][53852] Updated weights for policy 0, policy_version 93550 (0.0007) [2023-10-08 11:28:16,467][53885] Updated weights for policy 1, policy_version 93112 (0.0007) [2023-10-08 11:28:16,797][53852] Updated weights for policy 0, policy_version 93560 (0.0007) [2023-10-08 11:28:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191135744. Throughput: 0: 1821.1, 1: 1810.2. Samples: 47793774. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:17,016][52710] Avg episode reward: [(0, '33.260'), (1, '34.300')] [2023-10-08 11:28:20,133][53885] Updated weights for policy 1, policy_version 93122 (0.0007) [2023-10-08 11:28:20,458][53852] Updated weights for policy 0, policy_version 93570 (0.0008) [2023-10-08 11:28:20,509][53885] Updated weights for policy 1, policy_version 93132 (0.0008) [2023-10-08 11:28:20,823][53852] Updated weights for policy 0, policy_version 93580 (0.0011) [2023-10-08 11:28:20,883][53885] Updated weights for policy 1, policy_version 93142 (0.0008) [2023-10-08 11:28:21,192][53852] Updated weights for policy 0, policy_version 93590 (0.0009) [2023-10-08 11:28:21,256][53885] Updated weights for policy 1, policy_version 93152 (0.0008) [2023-10-08 11:28:21,557][53852] Updated weights for policy 0, policy_version 93600 (0.0007) [2023-10-08 11:28:22,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 191234048. Throughput: 0: 1820.2, 1: 1798.9. Samples: 47813654. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:22,016][52710] Avg episode reward: [(0, '31.010'), (1, '33.810')] [2023-10-08 11:28:25,034][53885] Updated weights for policy 1, policy_version 93162 (0.0009) [2023-10-08 11:28:25,204][53852] Updated weights for policy 0, policy_version 93610 (0.0008) [2023-10-08 11:28:25,400][53885] Updated weights for policy 1, policy_version 93172 (0.0009) [2023-10-08 11:28:25,579][53852] Updated weights for policy 0, policy_version 93620 (0.0008) [2023-10-08 11:28:25,772][53885] Updated weights for policy 1, policy_version 93182 (0.0009) [2023-10-08 11:28:25,945][53852] Updated weights for policy 0, policy_version 93630 (0.0007) [2023-10-08 11:28:27,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191299584. Throughput: 0: 1822.0, 1: 1806.8. Samples: 47826528. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:27,016][52710] Avg episode reward: [(0, '31.480'), (1, '36.370')] [2023-10-08 11:28:29,478][53885] Updated weights for policy 1, policy_version 93192 (0.0007) [2023-10-08 11:28:29,528][53852] Updated weights for policy 0, policy_version 93640 (0.0007) [2023-10-08 11:28:29,833][53885] Updated weights for policy 1, policy_version 93202 (0.0008) [2023-10-08 11:28:29,902][53852] Updated weights for policy 0, policy_version 93650 (0.0008) [2023-10-08 11:28:30,204][53885] Updated weights for policy 1, policy_version 93212 (0.0008) [2023-10-08 11:28:30,264][53852] Updated weights for policy 0, policy_version 93660 (0.0007) [2023-10-08 11:28:32,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 191365120. Throughput: 0: 1824.7, 1: 1812.1. Samples: 47846538. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:32,016][52710] Avg episode reward: [(0, '30.740'), (1, '37.340')] [2023-10-08 11:28:33,881][53885] Updated weights for policy 1, policy_version 93222 (0.0007) [2023-10-08 11:28:33,962][53852] Updated weights for policy 0, policy_version 93670 (0.0009) [2023-10-08 11:28:34,249][53885] Updated weights for policy 1, policy_version 93232 (0.0008) [2023-10-08 11:28:34,326][53852] Updated weights for policy 0, policy_version 93680 (0.0007) [2023-10-08 11:28:34,611][53885] Updated weights for policy 1, policy_version 93242 (0.0008) [2023-10-08 11:28:34,701][53852] Updated weights for policy 0, policy_version 93690 (0.0007) [2023-10-08 11:28:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191430656. Throughput: 0: 1823.6, 1: 1813.7. Samples: 47869220. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:37,016][52710] Avg episode reward: [(0, '35.950'), (1, '34.720')] [2023-10-08 11:28:38,313][53885] Updated weights for policy 1, policy_version 93252 (0.0009) [2023-10-08 11:28:38,493][53852] Updated weights for policy 0, policy_version 93700 (0.0008) [2023-10-08 11:28:38,683][53885] Updated weights for policy 1, policy_version 93262 (0.0008) [2023-10-08 11:28:38,866][53852] Updated weights for policy 0, policy_version 93710 (0.0009) [2023-10-08 11:28:39,047][53885] Updated weights for policy 1, policy_version 93272 (0.0008) [2023-10-08 11:28:39,230][53852] Updated weights for policy 0, policy_version 93720 (0.0007) [2023-10-08 11:28:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191496192. Throughput: 0: 1816.1, 1: 1816.8. Samples: 47878884. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:42,015][52710] Avg episode reward: [(0, '32.520'), (1, '34.940')] [2023-10-08 11:28:42,770][53885] Updated weights for policy 1, policy_version 93282 (0.0007) [2023-10-08 11:28:42,868][53852] Updated weights for policy 0, policy_version 93730 (0.0008) [2023-10-08 11:28:43,132][53885] Updated weights for policy 1, policy_version 93292 (0.0007) [2023-10-08 11:28:43,240][53852] Updated weights for policy 0, policy_version 93740 (0.0007) [2023-10-08 11:28:43,499][53885] Updated weights for policy 1, policy_version 93302 (0.0007) [2023-10-08 11:28:43,608][53852] Updated weights for policy 0, policy_version 93750 (0.0007) [2023-10-08 11:28:43,867][53885] Updated weights for policy 1, policy_version 93312 (0.0007) [2023-10-08 11:28:43,970][53852] Updated weights for policy 0, policy_version 93760 (0.0008) [2023-10-08 11:28:47,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191561728. Throughput: 0: 1816.0, 1: 1814.4. Samples: 47901702. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:47,016][52710] Avg episode reward: [(0, '34.370'), (1, '35.950')] [2023-10-08 11:28:47,637][53885] Updated weights for policy 1, policy_version 93322 (0.0007) [2023-10-08 11:28:47,687][53852] Updated weights for policy 0, policy_version 93770 (0.0008) [2023-10-08 11:28:48,005][53885] Updated weights for policy 1, policy_version 93332 (0.0007) [2023-10-08 11:28:48,054][53852] Updated weights for policy 0, policy_version 93780 (0.0008) [2023-10-08 11:28:48,369][53885] Updated weights for policy 1, policy_version 93342 (0.0008) [2023-10-08 11:28:48,424][53852] Updated weights for policy 0, policy_version 93790 (0.0009) [2023-10-08 11:28:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191627264. Throughput: 0: 1824.0, 1: 1812.9. Samples: 47924664. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:52,016][52710] Avg episode reward: [(0, '34.360'), (1, '33.590')] [2023-10-08 11:28:52,029][53852] Updated weights for policy 0, policy_version 93800 (0.0008) [2023-10-08 11:28:52,064][53885] Updated weights for policy 1, policy_version 93352 (0.0008) [2023-10-08 11:28:52,393][53852] Updated weights for policy 0, policy_version 93810 (0.0007) [2023-10-08 11:28:52,427][53885] Updated weights for policy 1, policy_version 93362 (0.0008) [2023-10-08 11:28:52,764][53852] Updated weights for policy 0, policy_version 93820 (0.0008) [2023-10-08 11:28:52,798][53885] Updated weights for policy 1, policy_version 93372 (0.0008) [2023-10-08 11:28:56,377][53852] Updated weights for policy 0, policy_version 93830 (0.0008) [2023-10-08 11:28:56,530][53885] Updated weights for policy 1, policy_version 93382 (0.0008) [2023-10-08 11:28:56,742][53852] Updated weights for policy 0, policy_version 93840 (0.0008) [2023-10-08 11:28:56,897][53885] Updated weights for policy 1, policy_version 93392 (0.0007) [2023-10-08 11:28:57,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 191692800. Throughput: 0: 1825.6, 1: 1811.0. Samples: 47934502. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) [2023-10-08 11:28:57,016][52710] Avg episode reward: [(0, '36.280'), (1, '32.260')] [2023-10-08 11:28:57,107][53852] Updated weights for policy 0, policy_version 93850 (0.0007) [2023-10-08 11:28:57,269][53885] Updated weights for policy 1, policy_version 93402 (0.0007) [2023-10-08 11:29:00,691][53852] Updated weights for policy 0, policy_version 93860 (0.0007) [2023-10-08 11:29:01,032][53885] Updated weights for policy 1, policy_version 93412 (0.0008) [2023-10-08 11:29:01,062][53852] Updated weights for policy 0, policy_version 93870 (0.0009) [2023-10-08 11:29:01,402][53885] Updated weights for policy 1, policy_version 93422 (0.0008) [2023-10-08 11:29:01,426][53852] Updated weights for policy 0, policy_version 93880 (0.0009) [2023-10-08 11:29:01,778][53885] Updated weights for policy 1, policy_version 93432 (0.0010) [2023-10-08 11:29:02,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191791104. Throughput: 0: 1827.7, 1: 1802.2. Samples: 47957120. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:02,015][52710] Avg episode reward: [(0, '32.570'), (1, '32.760')] [2023-10-08 11:29:05,012][53852] Updated weights for policy 0, policy_version 93890 (0.0008) [2023-10-08 11:29:05,390][53852] Updated weights for policy 0, policy_version 93900 (0.0008) [2023-10-08 11:29:05,459][53885] Updated weights for policy 1, policy_version 93442 (0.0009) [2023-10-08 11:29:05,748][53852] Updated weights for policy 0, policy_version 93910 (0.0008) [2023-10-08 11:29:05,861][53885] Updated weights for policy 1, policy_version 93452 (0.0007) [2023-10-08 11:29:06,113][53852] Updated weights for policy 0, policy_version 93920 (0.0009) [2023-10-08 11:29:06,225][53885] Updated weights for policy 1, policy_version 93462 (0.0007) [2023-10-08 11:29:06,600][53885] Updated weights for policy 1, policy_version 93472 (0.0008) [2023-10-08 11:29:07,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 191889408. Throughput: 0: 1829.5, 1: 1802.2. Samples: 47977080. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:07,016][52710] Avg episode reward: [(0, '34.050'), (1, '36.050')] [2023-10-08 11:29:09,705][53852] Updated weights for policy 0, policy_version 93930 (0.0007) [2023-10-08 11:29:10,077][53852] Updated weights for policy 0, policy_version 93940 (0.0007) [2023-10-08 11:29:10,314][53885] Updated weights for policy 1, policy_version 93482 (0.0008) [2023-10-08 11:29:10,440][53852] Updated weights for policy 0, policy_version 93950 (0.0008) [2023-10-08 11:29:10,681][53885] Updated weights for policy 1, policy_version 93492 (0.0007) [2023-10-08 11:29:11,053][53885] Updated weights for policy 1, policy_version 93502 (0.0009) [2023-10-08 11:29:12,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 191954944. Throughput: 0: 1830.7, 1: 1803.1. Samples: 47990048. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:12,016][52710] Avg episode reward: [(0, '34.000'), (1, '33.550')] [2023-10-08 11:29:14,144][53852] Updated weights for policy 0, policy_version 93960 (0.0007) [2023-10-08 11:29:14,515][53852] Updated weights for policy 0, policy_version 93970 (0.0007) [2023-10-08 11:29:14,668][53885] Updated weights for policy 1, policy_version 93512 (0.0008) [2023-10-08 11:29:14,891][53852] Updated weights for policy 0, policy_version 93980 (0.0011) [2023-10-08 11:29:15,040][53885] Updated weights for policy 1, policy_version 93522 (0.0007) [2023-10-08 11:29:15,418][53885] Updated weights for policy 1, policy_version 93532 (0.0010) [2023-10-08 11:29:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 192020480. Throughput: 0: 1833.3, 1: 1806.3. Samples: 48010320. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:17,015][52710] Avg episode reward: [(0, '34.480'), (1, '32.830')] [2023-10-08 11:29:18,386][53852] Updated weights for policy 0, policy_version 93990 (0.0010) [2023-10-08 11:29:18,749][53852] Updated weights for policy 0, policy_version 94000 (0.0009) [2023-10-08 11:29:19,113][53852] Updated weights for policy 0, policy_version 94010 (0.0007) [2023-10-08 11:29:19,156][53885] Updated weights for policy 1, policy_version 93542 (0.0009) [2023-10-08 11:29:19,524][53885] Updated weights for policy 1, policy_version 93552 (0.0008) [2023-10-08 11:29:19,890][53885] Updated weights for policy 1, policy_version 93562 (0.0008) [2023-10-08 11:29:22,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 192086016. Throughput: 0: 1842.3, 1: 1811.2. Samples: 48033628. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:22,016][52710] Avg episode reward: [(0, '36.330'), (1, '36.790')] [2023-10-08 11:29:22,852][53852] Updated weights for policy 0, policy_version 94020 (0.0008) [2023-10-08 11:29:23,220][53852] Updated weights for policy 0, policy_version 94030 (0.0007) [2023-10-08 11:29:23,553][53885] Updated weights for policy 1, policy_version 93572 (0.0010) [2023-10-08 11:29:23,592][53852] Updated weights for policy 0, policy_version 94040 (0.0009) [2023-10-08 11:29:23,918][53885] Updated weights for policy 1, policy_version 93582 (0.0007) [2023-10-08 11:29:24,280][53885] Updated weights for policy 1, policy_version 93592 (0.0008) [2023-10-08 11:29:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192151552. Throughput: 0: 1848.4, 1: 1814.2. Samples: 48043704. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:27,016][52710] Avg episode reward: [(0, '35.390'), (1, '36.100')] [2023-10-08 11:29:27,474][53852] Updated weights for policy 0, policy_version 94050 (0.0009) [2023-10-08 11:29:27,865][53852] Updated weights for policy 0, policy_version 94060 (0.0008) [2023-10-08 11:29:27,954][53885] Updated weights for policy 1, policy_version 93602 (0.0008) [2023-10-08 11:29:28,231][53852] Updated weights for policy 0, policy_version 94070 (0.0008) [2023-10-08 11:29:28,321][53885] Updated weights for policy 1, policy_version 93612 (0.0008) [2023-10-08 11:29:28,598][53852] Updated weights for policy 0, policy_version 94080 (0.0008) [2023-10-08 11:29:28,679][53885] Updated weights for policy 1, policy_version 93622 (0.0008) [2023-10-08 11:29:29,056][53885] Updated weights for policy 1, policy_version 93632 (0.0011) [2023-10-08 11:29:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192217088. Throughput: 0: 1837.8, 1: 1813.6. Samples: 48066012. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:32,016][52710] Avg episode reward: [(0, '36.600'), (1, '33.800')] [2023-10-08 11:29:32,213][53852] Updated weights for policy 0, policy_version 94090 (0.0008) [2023-10-08 11:29:32,579][53852] Updated weights for policy 0, policy_version 94100 (0.0008) [2023-10-08 11:29:32,899][53885] Updated weights for policy 1, policy_version 93642 (0.0007) [2023-10-08 11:29:32,954][53852] Updated weights for policy 0, policy_version 94110 (0.0007) [2023-10-08 11:29:33,262][53885] Updated weights for policy 1, policy_version 93652 (0.0007) [2023-10-08 11:29:33,634][53885] Updated weights for policy 1, policy_version 93662 (0.0007) [2023-10-08 11:29:36,575][53852] Updated weights for policy 0, policy_version 94120 (0.0010) [2023-10-08 11:29:36,949][53852] Updated weights for policy 0, policy_version 94130 (0.0009) [2023-10-08 11:29:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192282624. Throughput: 0: 1829.5, 1: 1818.8. Samples: 48088840. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:37,016][52710] Avg episode reward: [(0, '36.680'), (1, '35.780')] [2023-10-08 11:29:37,095][53885] Updated weights for policy 1, policy_version 93672 (0.0008) [2023-10-08 11:29:37,330][53852] Updated weights for policy 0, policy_version 94140 (0.0008) [2023-10-08 11:29:37,455][53885] Updated weights for policy 1, policy_version 93682 (0.0008) [2023-10-08 11:29:37,825][53885] Updated weights for policy 1, policy_version 93692 (0.0007) [2023-10-08 11:29:41,074][53852] Updated weights for policy 0, policy_version 94150 (0.0008) [2023-10-08 11:29:41,434][53852] Updated weights for policy 0, policy_version 94160 (0.0008) [2023-10-08 11:29:41,594][53885] Updated weights for policy 1, policy_version 93702 (0.0008) [2023-10-08 11:29:41,812][53852] Updated weights for policy 0, policy_version 94170 (0.0009) [2023-10-08 11:29:41,960][53885] Updated weights for policy 1, policy_version 93712 (0.0007) [2023-10-08 11:29:42,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192348160. Throughput: 0: 1836.8, 1: 1818.9. Samples: 48099008. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:42,015][52710] Avg episode reward: [(0, '37.010'), (1, '38.400')] [2023-10-08 11:29:42,331][53885] Updated weights for policy 1, policy_version 93722 (0.0007) [2023-10-08 11:29:45,514][53852] Updated weights for policy 0, policy_version 94180 (0.0007) [2023-10-08 11:29:45,886][53852] Updated weights for policy 0, policy_version 94190 (0.0007) [2023-10-08 11:29:45,953][53885] Updated weights for policy 1, policy_version 93732 (0.0008) [2023-10-08 11:29:46,253][53852] Updated weights for policy 0, policy_version 94200 (0.0008) [2023-10-08 11:29:46,321][53885] Updated weights for policy 1, policy_version 93742 (0.0007) [2023-10-08 11:29:46,690][53885] Updated weights for policy 1, policy_version 93752 (0.0007) [2023-10-08 11:29:47,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 192479232. Throughput: 0: 1829.4, 1: 1824.2. Samples: 48121530. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:47,015][52710] Avg episode reward: [(0, '37.110'), (1, '36.120')] [2023-10-08 11:29:49,779][53852] Updated weights for policy 0, policy_version 94210 (0.0008) [2023-10-08 11:29:50,151][53852] Updated weights for policy 0, policy_version 94220 (0.0008) [2023-10-08 11:29:50,431][53885] Updated weights for policy 1, policy_version 93762 (0.0010) [2023-10-08 11:29:50,522][53852] Updated weights for policy 0, policy_version 94230 (0.0007) [2023-10-08 11:29:50,842][53885] Updated weights for policy 1, policy_version 93772 (0.0008) [2023-10-08 11:29:50,885][53852] Updated weights for policy 0, policy_version 94240 (0.0009) [2023-10-08 11:29:51,212][53885] Updated weights for policy 1, policy_version 93782 (0.0008) [2023-10-08 11:29:51,572][53885] Updated weights for policy 1, policy_version 93792 (0.0009) [2023-10-08 11:29:52,015][52710] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 192544768. Throughput: 0: 1832.4, 1: 1825.2. Samples: 48141672. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:52,016][52710] Avg episode reward: [(0, '33.960'), (1, '36.780')] [2023-10-08 11:29:54,571][53852] Updated weights for policy 0, policy_version 94250 (0.0008) [2023-10-08 11:29:54,945][53852] Updated weights for policy 0, policy_version 94260 (0.0007) [2023-10-08 11:29:55,192][53885] Updated weights for policy 1, policy_version 93802 (0.0009) [2023-10-08 11:29:55,317][53852] Updated weights for policy 0, policy_version 94270 (0.0007) [2023-10-08 11:29:55,565][53885] Updated weights for policy 1, policy_version 93812 (0.0010) [2023-10-08 11:29:55,924][53885] Updated weights for policy 1, policy_version 93822 (0.0009) [2023-10-08 11:29:57,015][52710] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 192610304. Throughput: 0: 1823.7, 1: 1821.9. Samples: 48154102. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) [2023-10-08 11:29:57,016][52710] Avg episode reward: [(0, '35.710'), (1, '38.030')] [2023-10-08 11:29:59,029][53852] Updated weights for policy 0, policy_version 94280 (0.0008) [2023-10-08 11:29:59,395][53852] Updated weights for policy 0, policy_version 94290 (0.0008) [2023-10-08 11:29:59,636][53885] Updated weights for policy 1, policy_version 93832 (0.0010) [2023-10-08 11:29:59,768][53852] Updated weights for policy 0, policy_version 94300 (0.0008) [2023-10-08 11:30:00,006][53885] Updated weights for policy 1, policy_version 93842 (0.0007) [2023-10-08 11:30:00,377][53885] Updated weights for policy 1, policy_version 93852 (0.0009) [2023-10-08 11:30:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192675840. Throughput: 0: 1828.3, 1: 1817.2. Samples: 48174368. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:02,016][52710] Avg episode reward: [(0, '34.160'), (1, '35.260')] [2023-10-08 11:30:03,462][53852] Updated weights for policy 0, policy_version 94310 (0.0007) [2023-10-08 11:30:03,830][53852] Updated weights for policy 0, policy_version 94320 (0.0008) [2023-10-08 11:30:04,062][53885] Updated weights for policy 1, policy_version 93862 (0.0008) [2023-10-08 11:30:04,198][53852] Updated weights for policy 0, policy_version 94330 (0.0008) [2023-10-08 11:30:04,430][53885] Updated weights for policy 1, policy_version 93872 (0.0007) [2023-10-08 11:30:04,795][53885] Updated weights for policy 1, policy_version 93882 (0.0008) [2023-10-08 11:30:07,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 192741376. Throughput: 0: 1818.3, 1: 1813.6. Samples: 48197064. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:07,016][52710] Avg episode reward: [(0, '35.980'), (1, '36.990')] [2023-10-08 11:30:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000094336_96600064.pth... [2023-10-08 11:30:07,025][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000093888_96141312.pth... [2023-10-08 11:30:07,054][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000092640_94863360.pth [2023-10-08 11:30:07,058][53500] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p0/milestones/checkpoint_000094336_96600064.pth [2023-10-08 11:30:07,066][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000092192_94404608.pth [2023-10-08 11:30:07,071][53594] Saving a milestone ./train_atari/atari_asterix_APPO/checkpoint_p1/milestones/checkpoint_000093888_96141312.pth [2023-10-08 11:30:07,910][53852] Updated weights for policy 0, policy_version 94340 (0.0009) [2023-10-08 11:30:08,281][53852] Updated weights for policy 0, policy_version 94350 (0.0010) [2023-10-08 11:30:08,612][53885] Updated weights for policy 1, policy_version 93892 (0.0008) [2023-10-08 11:30:08,641][53852] Updated weights for policy 0, policy_version 94360 (0.0008) [2023-10-08 11:30:08,976][53885] Updated weights for policy 1, policy_version 93902 (0.0009) [2023-10-08 11:30:09,339][53885] Updated weights for policy 1, policy_version 93912 (0.0009) [2023-10-08 11:30:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192806912. Throughput: 0: 1815.2, 1: 1812.3. Samples: 48206942. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:12,016][52710] Avg episode reward: [(0, '34.470'), (1, '34.590')] [2023-10-08 11:30:12,487][53852] Updated weights for policy 0, policy_version 94370 (0.0008) [2023-10-08 11:30:12,855][53852] Updated weights for policy 0, policy_version 94380 (0.0011) [2023-10-08 11:30:13,071][53885] Updated weights for policy 1, policy_version 93922 (0.0010) [2023-10-08 11:30:13,226][53852] Updated weights for policy 0, policy_version 94390 (0.0009) [2023-10-08 11:30:13,437][53885] Updated weights for policy 1, policy_version 93932 (0.0007) [2023-10-08 11:30:13,585][53852] Updated weights for policy 0, policy_version 94400 (0.0007) [2023-10-08 11:30:13,800][53885] Updated weights for policy 1, policy_version 93942 (0.0008) [2023-10-08 11:30:14,162][53885] Updated weights for policy 1, policy_version 93952 (0.0008) [2023-10-08 11:30:17,015][52710] Fps is (10 sec: 13107.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 192872448. Throughput: 0: 1813.5, 1: 1805.0. Samples: 48228846. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:17,015][52710] Avg episode reward: [(0, '36.690'), (1, '34.960')] [2023-10-08 11:30:17,482][53852] Updated weights for policy 0, policy_version 94410 (0.0007) [2023-10-08 11:30:17,850][53852] Updated weights for policy 0, policy_version 94420 (0.0011) [2023-10-08 11:30:18,054][53885] Updated weights for policy 1, policy_version 93962 (0.0009) [2023-10-08 11:30:18,220][53852] Updated weights for policy 0, policy_version 94430 (0.0008) [2023-10-08 11:30:18,423][53885] Updated weights for policy 1, policy_version 93972 (0.0009) [2023-10-08 11:30:18,793][53885] Updated weights for policy 1, policy_version 93982 (0.0008) [2023-10-08 11:30:21,870][53852] Updated weights for policy 0, policy_version 94440 (0.0008) [2023-10-08 11:30:22,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 192937984. Throughput: 0: 1819.6, 1: 1802.4. Samples: 48251832. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:22,016][52710] Avg episode reward: [(0, '36.970'), (1, '36.510')] [2023-10-08 11:30:22,242][53852] Updated weights for policy 0, policy_version 94450 (0.0009) [2023-10-08 11:30:22,434][53885] Updated weights for policy 1, policy_version 93992 (0.0007) [2023-10-08 11:30:22,605][53852] Updated weights for policy 0, policy_version 94460 (0.0007) [2023-10-08 11:30:22,804][53885] Updated weights for policy 1, policy_version 94002 (0.0008) [2023-10-08 11:30:23,161][53885] Updated weights for policy 1, policy_version 94012 (0.0009) [2023-10-08 11:30:26,458][53852] Updated weights for policy 0, policy_version 94470 (0.0010) [2023-10-08 11:30:26,730][53885] Updated weights for policy 1, policy_version 94022 (0.0009) [2023-10-08 11:30:26,836][53852] Updated weights for policy 0, policy_version 94480 (0.0007) [2023-10-08 11:30:27,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193003520. Throughput: 0: 1811.1, 1: 1803.7. Samples: 48261676. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:27,016][52710] Avg episode reward: [(0, '36.380'), (1, '37.150')] [2023-10-08 11:30:27,095][53885] Updated weights for policy 1, policy_version 94032 (0.0009) [2023-10-08 11:30:27,198][53852] Updated weights for policy 0, policy_version 94490 (0.0007) [2023-10-08 11:30:27,466][53885] Updated weights for policy 1, policy_version 94042 (0.0008) [2023-10-08 11:30:30,844][53852] Updated weights for policy 0, policy_version 94500 (0.0008) [2023-10-08 11:30:31,213][53852] Updated weights for policy 0, policy_version 94510 (0.0008) [2023-10-08 11:30:31,235][53885] Updated weights for policy 1, policy_version 94052 (0.0009) [2023-10-08 11:30:31,577][53852] Updated weights for policy 0, policy_version 94520 (0.0008) [2023-10-08 11:30:31,601][53885] Updated weights for policy 1, policy_version 94062 (0.0009) [2023-10-08 11:30:31,978][53885] Updated weights for policy 1, policy_version 94072 (0.0009) [2023-10-08 11:30:32,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193101824. Throughput: 0: 1813.9, 1: 1809.1. Samples: 48284566. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:32,016][52710] Avg episode reward: [(0, '38.260'), (1, '33.720')] [2023-10-08 11:30:35,181][53852] Updated weights for policy 0, policy_version 94530 (0.0008) [2023-10-08 11:30:35,549][53852] Updated weights for policy 0, policy_version 94540 (0.0008) [2023-10-08 11:30:35,734][53885] Updated weights for policy 1, policy_version 94082 (0.0008) [2023-10-08 11:30:35,920][53852] Updated weights for policy 0, policy_version 94550 (0.0009) [2023-10-08 11:30:36,103][53885] Updated weights for policy 1, policy_version 94092 (0.0007) [2023-10-08 11:30:36,290][53852] Updated weights for policy 0, policy_version 94560 (0.0008) [2023-10-08 11:30:36,483][53885] Updated weights for policy 1, policy_version 94102 (0.0010) [2023-10-08 11:30:36,837][53885] Updated weights for policy 1, policy_version 94112 (0.0009) [2023-10-08 11:30:37,015][52710] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 193200128. Throughput: 0: 1810.2, 1: 1818.0. Samples: 48304940. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:37,015][52710] Avg episode reward: [(0, '37.880'), (1, '32.680')] [2023-10-08 11:30:40,014][53852] Updated weights for policy 0, policy_version 94570 (0.0007) [2023-10-08 11:30:40,386][53852] Updated weights for policy 0, policy_version 94580 (0.0009) [2023-10-08 11:30:40,512][53885] Updated weights for policy 1, policy_version 94122 (0.0007) [2023-10-08 11:30:40,750][53852] Updated weights for policy 0, policy_version 94590 (0.0009) [2023-10-08 11:30:40,878][53885] Updated weights for policy 1, policy_version 94132 (0.0009) [2023-10-08 11:30:41,237][53885] Updated weights for policy 1, policy_version 94142 (0.0010) [2023-10-08 11:30:42,015][52710] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 193265664. Throughput: 0: 1815.0, 1: 1815.1. Samples: 48317456. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:42,015][52710] Avg episode reward: [(0, '35.930'), (1, '39.710')] [2023-10-08 11:30:44,479][53852] Updated weights for policy 0, policy_version 94600 (0.0008) [2023-10-08 11:30:44,797][53885] Updated weights for policy 1, policy_version 94152 (0.0008) [2023-10-08 11:30:44,851][53852] Updated weights for policy 0, policy_version 94610 (0.0008) [2023-10-08 11:30:45,156][53885] Updated weights for policy 1, policy_version 94162 (0.0007) [2023-10-08 11:30:45,218][53852] Updated weights for policy 0, policy_version 94620 (0.0007) [2023-10-08 11:30:45,518][53885] Updated weights for policy 1, policy_version 94172 (0.0010) [2023-10-08 11:30:47,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 193331200. Throughput: 0: 1804.7, 1: 1823.3. Samples: 48337628. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:47,016][52710] Avg episode reward: [(0, '32.480'), (1, '35.810')] [2023-10-08 11:30:48,779][53852] Updated weights for policy 0, policy_version 94630 (0.0008) [2023-10-08 11:30:49,146][53852] Updated weights for policy 0, policy_version 94640 (0.0009) [2023-10-08 11:30:49,220][53885] Updated weights for policy 1, policy_version 94182 (0.0008) [2023-10-08 11:30:49,511][53852] Updated weights for policy 0, policy_version 94650 (0.0008) [2023-10-08 11:30:49,587][53885] Updated weights for policy 1, policy_version 94192 (0.0008) [2023-10-08 11:30:49,960][53885] Updated weights for policy 1, policy_version 94202 (0.0009) [2023-10-08 11:30:52,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 193396736. Throughput: 0: 1808.6, 1: 1818.1. Samples: 48360268. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:52,016][52710] Avg episode reward: [(0, '33.280'), (1, '36.980')] [2023-10-08 11:30:53,214][53852] Updated weights for policy 0, policy_version 94660 (0.0010) [2023-10-08 11:30:53,578][53852] Updated weights for policy 0, policy_version 94670 (0.0010) [2023-10-08 11:30:53,751][53885] Updated weights for policy 1, policy_version 94212 (0.0009) [2023-10-08 11:30:53,944][53852] Updated weights for policy 0, policy_version 94680 (0.0008) [2023-10-08 11:30:54,116][53885] Updated weights for policy 1, policy_version 94222 (0.0009) [2023-10-08 11:30:54,482][53885] Updated weights for policy 1, policy_version 94232 (0.0008) [2023-10-08 11:30:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 193462272. Throughput: 0: 1811.3, 1: 1824.2. Samples: 48370540. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) [2023-10-08 11:30:57,016][52710] Avg episode reward: [(0, '36.870'), (1, '36.570')] [2023-10-08 11:30:57,567][53852] Updated weights for policy 0, policy_version 94690 (0.0008) [2023-10-08 11:30:57,934][53852] Updated weights for policy 0, policy_version 94700 (0.0007) [2023-10-08 11:30:58,310][53852] Updated weights for policy 0, policy_version 94710 (0.0008) [2023-10-08 11:30:58,347][53885] Updated weights for policy 1, policy_version 94242 (0.0007) [2023-10-08 11:30:58,669][53852] Updated weights for policy 0, policy_version 94720 (0.0007) [2023-10-08 11:30:58,711][53885] Updated weights for policy 1, policy_version 94252 (0.0007) [2023-10-08 11:30:59,073][53885] Updated weights for policy 1, policy_version 94262 (0.0007) [2023-10-08 11:30:59,436][53885] Updated weights for policy 1, policy_version 94272 (0.0007) [2023-10-08 11:31:02,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 193527808. Throughput: 0: 1824.4, 1: 1830.4. Samples: 48393308. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:02,016][52710] Avg episode reward: [(0, '32.670'), (1, '39.260')] [2023-10-08 11:31:02,538][53852] Updated weights for policy 0, policy_version 94730 (0.0007) [2023-10-08 11:31:02,909][53852] Updated weights for policy 0, policy_version 94740 (0.0007) [2023-10-08 11:31:03,114][53885] Updated weights for policy 1, policy_version 94282 (0.0007) [2023-10-08 11:31:03,284][53852] Updated weights for policy 0, policy_version 94750 (0.0007) [2023-10-08 11:31:03,487][53885] Updated weights for policy 1, policy_version 94292 (0.0009) [2023-10-08 11:31:03,847][53885] Updated weights for policy 1, policy_version 94302 (0.0009) [2023-10-08 11:31:06,975][53852] Updated weights for policy 0, policy_version 94760 (0.0008) [2023-10-08 11:31:07,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193593344. Throughput: 0: 1817.4, 1: 1829.9. Samples: 48415962. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:07,016][52710] Avg episode reward: [(0, '33.840'), (1, '34.610')] [2023-10-08 11:31:07,347][53852] Updated weights for policy 0, policy_version 94770 (0.0010) [2023-10-08 11:31:07,480][53885] Updated weights for policy 1, policy_version 94312 (0.0007) [2023-10-08 11:31:07,714][53852] Updated weights for policy 0, policy_version 94780 (0.0009) [2023-10-08 11:31:07,846][53885] Updated weights for policy 1, policy_version 94322 (0.0008) [2023-10-08 11:31:08,213][53885] Updated weights for policy 1, policy_version 94332 (0.0008) [2023-10-08 11:31:11,384][53852] Updated weights for policy 0, policy_version 94790 (0.0010) [2023-10-08 11:31:11,753][53852] Updated weights for policy 0, policy_version 94800 (0.0009) [2023-10-08 11:31:11,848][53885] Updated weights for policy 1, policy_version 94342 (0.0008) [2023-10-08 11:31:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 193658880. Throughput: 0: 1816.7, 1: 1833.1. Samples: 48425914. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:12,015][52710] Avg episode reward: [(0, '32.590'), (1, '37.310')] [2023-10-08 11:31:12,118][53852] Updated weights for policy 0, policy_version 94810 (0.0010) [2023-10-08 11:31:12,219][53885] Updated weights for policy 1, policy_version 94352 (0.0007) [2023-10-08 11:31:12,585][53885] Updated weights for policy 1, policy_version 94362 (0.0008) [2023-10-08 11:31:15,761][53852] Updated weights for policy 0, policy_version 94820 (0.0008) [2023-10-08 11:31:16,130][53852] Updated weights for policy 0, policy_version 94830 (0.0007) [2023-10-08 11:31:16,289][53885] Updated weights for policy 1, policy_version 94372 (0.0009) [2023-10-08 11:31:16,508][53852] Updated weights for policy 0, policy_version 94840 (0.0007) [2023-10-08 11:31:16,652][53885] Updated weights for policy 1, policy_version 94382 (0.0007) [2023-10-08 11:31:17,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193757184. Throughput: 0: 1821.1, 1: 1828.1. Samples: 48448776. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:17,015][52710] Avg episode reward: [(0, '35.510'), (1, '38.430')] [2023-10-08 11:31:17,024][53885] Updated weights for policy 1, policy_version 94392 (0.0009) [2023-10-08 11:31:20,242][53852] Updated weights for policy 0, policy_version 94850 (0.0008) [2023-10-08 11:31:20,609][53852] Updated weights for policy 0, policy_version 94860 (0.0007) [2023-10-08 11:31:20,673][53885] Updated weights for policy 1, policy_version 94402 (0.0007) [2023-10-08 11:31:20,974][53852] Updated weights for policy 0, policy_version 94870 (0.0008) [2023-10-08 11:31:21,089][53885] Updated weights for policy 1, policy_version 94412 (0.0008) [2023-10-08 11:31:21,343][53852] Updated weights for policy 0, policy_version 94880 (0.0008) [2023-10-08 11:31:21,451][53885] Updated weights for policy 1, policy_version 94422 (0.0008) [2023-10-08 11:31:21,816][53885] Updated weights for policy 1, policy_version 94432 (0.0008) [2023-10-08 11:31:22,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 193855488. Throughput: 0: 1816.9, 1: 1824.8. Samples: 48468818. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:22,016][52710] Avg episode reward: [(0, '32.390'), (1, '33.560')] [2023-10-08 11:31:25,028][53852] Updated weights for policy 0, policy_version 94890 (0.0007) [2023-10-08 11:31:25,367][53885] Updated weights for policy 1, policy_version 94442 (0.0007) [2023-10-08 11:31:25,398][53852] Updated weights for policy 0, policy_version 94900 (0.0007) [2023-10-08 11:31:25,733][53885] Updated weights for policy 1, policy_version 94452 (0.0008) [2023-10-08 11:31:25,763][53852] Updated weights for policy 0, policy_version 94910 (0.0009) [2023-10-08 11:31:26,107][53885] Updated weights for policy 1, policy_version 94462 (0.0010) [2023-10-08 11:31:27,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 193921024. Throughput: 0: 1822.0, 1: 1826.4. Samples: 48481636. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:27,016][52710] Avg episode reward: [(0, '34.170'), (1, '36.750')] [2023-10-08 11:31:29,418][53852] Updated weights for policy 0, policy_version 94920 (0.0008) [2023-10-08 11:31:29,789][53852] Updated weights for policy 0, policy_version 94930 (0.0008) [2023-10-08 11:31:29,819][53885] Updated weights for policy 1, policy_version 94472 (0.0009) [2023-10-08 11:31:30,159][53852] Updated weights for policy 0, policy_version 94940 (0.0008) [2023-10-08 11:31:30,180][53885] Updated weights for policy 1, policy_version 94482 (0.0007) [2023-10-08 11:31:30,545][53885] Updated weights for policy 1, policy_version 94492 (0.0007) [2023-10-08 11:31:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 193986560. Throughput: 0: 1824.5, 1: 1822.8. Samples: 48501758. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:32,015][52710] Avg episode reward: [(0, '33.100'), (1, '36.310')] [2023-10-08 11:31:33,808][53852] Updated weights for policy 0, policy_version 94950 (0.0007) [2023-10-08 11:31:34,145][53885] Updated weights for policy 1, policy_version 94502 (0.0008) [2023-10-08 11:31:34,184][53852] Updated weights for policy 0, policy_version 94960 (0.0007) [2023-10-08 11:31:34,520][53885] Updated weights for policy 1, policy_version 94512 (0.0009) [2023-10-08 11:31:34,554][53852] Updated weights for policy 0, policy_version 94970 (0.0007) [2023-10-08 11:31:34,884][53885] Updated weights for policy 1, policy_version 94522 (0.0008) [2023-10-08 11:31:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 194052096. Throughput: 0: 1820.8, 1: 1827.5. Samples: 48524442. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:37,016][52710] Avg episode reward: [(0, '35.610'), (1, '38.390')] [2023-10-08 11:31:38,319][53852] Updated weights for policy 0, policy_version 94980 (0.0008) [2023-10-08 11:31:38,626][53885] Updated weights for policy 1, policy_version 94532 (0.0008) [2023-10-08 11:31:38,677][53852] Updated weights for policy 0, policy_version 94990 (0.0007) [2023-10-08 11:31:38,989][53885] Updated weights for policy 1, policy_version 94542 (0.0008) [2023-10-08 11:31:39,050][53852] Updated weights for policy 0, policy_version 95000 (0.0007) [2023-10-08 11:31:39,354][53885] Updated weights for policy 1, policy_version 94552 (0.0007) [2023-10-08 11:31:42,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 194117632. Throughput: 0: 1818.0, 1: 1823.6. Samples: 48534410. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:42,016][52710] Avg episode reward: [(0, '34.390'), (1, '35.640')] [2023-10-08 11:31:42,753][53852] Updated weights for policy 0, policy_version 95010 (0.0008) [2023-10-08 11:31:43,111][53885] Updated weights for policy 1, policy_version 94562 (0.0007) [2023-10-08 11:31:43,122][53852] Updated weights for policy 0, policy_version 95020 (0.0007) [2023-10-08 11:31:43,476][53885] Updated weights for policy 1, policy_version 94572 (0.0007) [2023-10-08 11:31:43,492][53852] Updated weights for policy 0, policy_version 95030 (0.0008) [2023-10-08 11:31:43,842][53885] Updated weights for policy 1, policy_version 94582 (0.0008) [2023-10-08 11:31:43,860][53852] Updated weights for policy 0, policy_version 95040 (0.0008) [2023-10-08 11:31:44,208][53885] Updated weights for policy 1, policy_version 94592 (0.0010) [2023-10-08 11:31:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194183168. Throughput: 0: 1815.4, 1: 1824.3. Samples: 48557096. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:47,015][52710] Avg episode reward: [(0, '32.850'), (1, '35.700')] [2023-10-08 11:31:47,609][53852] Updated weights for policy 0, policy_version 95050 (0.0008) [2023-10-08 11:31:47,766][53885] Updated weights for policy 1, policy_version 94602 (0.0009) [2023-10-08 11:31:47,982][53852] Updated weights for policy 0, policy_version 95060 (0.0008) [2023-10-08 11:31:48,123][53885] Updated weights for policy 1, policy_version 94612 (0.0009) [2023-10-08 11:31:48,343][53852] Updated weights for policy 0, policy_version 95070 (0.0007) [2023-10-08 11:31:48,483][53885] Updated weights for policy 1, policy_version 94622 (0.0009) [2023-10-08 11:31:52,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194248704. Throughput: 0: 1822.1, 1: 1821.6. Samples: 48579930. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:52,016][52710] Avg episode reward: [(0, '34.640'), (1, '36.060')] [2023-10-08 11:31:52,056][53852] Updated weights for policy 0, policy_version 95080 (0.0007) [2023-10-08 11:31:52,306][53885] Updated weights for policy 1, policy_version 94632 (0.0008) [2023-10-08 11:31:52,423][53852] Updated weights for policy 0, policy_version 95090 (0.0009) [2023-10-08 11:31:52,668][53885] Updated weights for policy 1, policy_version 94642 (0.0007) [2023-10-08 11:31:52,800][53852] Updated weights for policy 0, policy_version 95100 (0.0008) [2023-10-08 11:31:53,045][53885] Updated weights for policy 1, policy_version 94652 (0.0007) [2023-10-08 11:31:56,558][53852] Updated weights for policy 0, policy_version 95110 (0.0007) [2023-10-08 11:31:56,616][53885] Updated weights for policy 1, policy_version 94662 (0.0008) [2023-10-08 11:31:56,931][53852] Updated weights for policy 0, policy_version 95120 (0.0008) [2023-10-08 11:31:56,982][53885] Updated weights for policy 1, policy_version 94672 (0.0007) [2023-10-08 11:31:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194314240. Throughput: 0: 1818.6, 1: 1816.2. Samples: 48589480. Policy #0 lag: (min: 8.0, avg: 31.8, max: 40.0) [2023-10-08 11:31:57,016][52710] Avg episode reward: [(0, '35.790'), (1, '33.680')] [2023-10-08 11:31:57,298][53852] Updated weights for policy 0, policy_version 95130 (0.0008) [2023-10-08 11:31:57,343][53885] Updated weights for policy 1, policy_version 94682 (0.0007) [2023-10-08 11:32:00,987][53852] Updated weights for policy 0, policy_version 95140 (0.0009) [2023-10-08 11:32:01,106][53885] Updated weights for policy 1, policy_version 94692 (0.0007) [2023-10-08 11:32:01,352][53852] Updated weights for policy 0, policy_version 95150 (0.0007) [2023-10-08 11:32:01,473][53885] Updated weights for policy 1, policy_version 94702 (0.0008) [2023-10-08 11:32:01,726][53852] Updated weights for policy 0, policy_version 95160 (0.0007) [2023-10-08 11:32:01,847][53885] Updated weights for policy 1, policy_version 94712 (0.0008) [2023-10-08 11:32:02,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 194379776. Throughput: 0: 1816.8, 1: 1816.0. Samples: 48612254. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:02,016][52710] Avg episode reward: [(0, '33.700'), (1, '36.280')] [2023-10-08 11:32:05,354][53852] Updated weights for policy 0, policy_version 95170 (0.0007) [2023-10-08 11:32:05,499][53885] Updated weights for policy 1, policy_version 94722 (0.0009) [2023-10-08 11:32:05,722][53852] Updated weights for policy 0, policy_version 95180 (0.0008) [2023-10-08 11:32:05,869][53885] Updated weights for policy 1, policy_version 94732 (0.0008) [2023-10-08 11:32:06,089][53852] Updated weights for policy 0, policy_version 95190 (0.0007) [2023-10-08 11:32:06,236][53885] Updated weights for policy 1, policy_version 94742 (0.0008) [2023-10-08 11:32:06,452][53852] Updated weights for policy 0, policy_version 95200 (0.0008) [2023-10-08 11:32:06,599][53885] Updated weights for policy 1, policy_version 94752 (0.0007) [2023-10-08 11:32:07,015][52710] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 194510848. Throughput: 0: 1820.5, 1: 1814.1. Samples: 48632374. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:07,016][52710] Avg episode reward: [(0, '32.680'), (1, '34.470')] [2023-10-08 11:32:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000095200_97484800.pth... [2023-10-08 11:32:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000094752_97026048.pth... [2023-10-08 11:32:07,060][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000093504_95748096.pth [2023-10-08 11:32:07,064][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000093056_95289344.pth [2023-10-08 11:32:10,107][53852] Updated weights for policy 0, policy_version 95210 (0.0008) [2023-10-08 11:32:10,444][53885] Updated weights for policy 1, policy_version 94762 (0.0008) [2023-10-08 11:32:10,477][53852] Updated weights for policy 0, policy_version 95220 (0.0008) [2023-10-08 11:32:10,820][53885] Updated weights for policy 1, policy_version 94772 (0.0008) [2023-10-08 11:32:10,845][53852] Updated weights for policy 0, policy_version 95230 (0.0009) [2023-10-08 11:32:11,180][53885] Updated weights for policy 1, policy_version 94782 (0.0008) [2023-10-08 11:32:12,015][52710] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 194576384. Throughput: 0: 1822.1, 1: 1814.1. Samples: 48645266. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:12,016][52710] Avg episode reward: [(0, '34.400'), (1, '36.110')] [2023-10-08 11:32:14,310][53852] Updated weights for policy 0, policy_version 95240 (0.0007) [2023-10-08 11:32:14,684][53852] Updated weights for policy 0, policy_version 95250 (0.0007) [2023-10-08 11:32:14,845][53885] Updated weights for policy 1, policy_version 94792 (0.0008) [2023-10-08 11:32:15,040][53852] Updated weights for policy 0, policy_version 95260 (0.0007) [2023-10-08 11:32:15,213][53885] Updated weights for policy 1, policy_version 94802 (0.0008) [2023-10-08 11:32:15,568][53885] Updated weights for policy 1, policy_version 94812 (0.0008) [2023-10-08 11:32:17,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 194641920. Throughput: 0: 1822.2, 1: 1814.2. Samples: 48665398. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:17,016][52710] Avg episode reward: [(0, '35.600'), (1, '34.130')] [2023-10-08 11:32:18,632][53852] Updated weights for policy 0, policy_version 95270 (0.0008) [2023-10-08 11:32:19,005][53852] Updated weights for policy 0, policy_version 95280 (0.0010) [2023-10-08 11:32:19,321][53885] Updated weights for policy 1, policy_version 94822 (0.0008) [2023-10-08 11:32:19,362][53852] Updated weights for policy 0, policy_version 95290 (0.0007) [2023-10-08 11:32:19,684][53885] Updated weights for policy 1, policy_version 94832 (0.0008) [2023-10-08 11:32:20,053][53885] Updated weights for policy 1, policy_version 94842 (0.0008) [2023-10-08 11:32:22,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194707456. Throughput: 0: 1822.8, 1: 1811.5. Samples: 48687986. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:22,015][52710] Avg episode reward: [(0, '30.720'), (1, '36.280')] [2023-10-08 11:32:23,028][53852] Updated weights for policy 0, policy_version 95300 (0.0008) [2023-10-08 11:32:23,401][53852] Updated weights for policy 0, policy_version 95310 (0.0009) [2023-10-08 11:32:23,771][53852] Updated weights for policy 0, policy_version 95320 (0.0009) [2023-10-08 11:32:23,812][53885] Updated weights for policy 1, policy_version 94852 (0.0008) [2023-10-08 11:32:24,182][53885] Updated weights for policy 1, policy_version 94862 (0.0007) [2023-10-08 11:32:24,545][53885] Updated weights for policy 1, policy_version 94872 (0.0008) [2023-10-08 11:32:27,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194772992. Throughput: 0: 1823.2, 1: 1816.2. Samples: 48698182. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:27,016][52710] Avg episode reward: [(0, '30.360'), (1, '38.340')] [2023-10-08 11:32:27,479][53852] Updated weights for policy 0, policy_version 95330 (0.0009) [2023-10-08 11:32:27,842][53852] Updated weights for policy 0, policy_version 95340 (0.0008) [2023-10-08 11:32:28,212][53852] Updated weights for policy 0, policy_version 95350 (0.0007) [2023-10-08 11:32:28,246][53885] Updated weights for policy 1, policy_version 94882 (0.0007) [2023-10-08 11:32:28,579][53852] Updated weights for policy 0, policy_version 95360 (0.0009) [2023-10-08 11:32:28,603][53885] Updated weights for policy 1, policy_version 94892 (0.0008) [2023-10-08 11:32:28,973][53885] Updated weights for policy 1, policy_version 94902 (0.0009) [2023-10-08 11:32:29,344][53885] Updated weights for policy 1, policy_version 94912 (0.0009) [2023-10-08 11:32:32,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 194838528. Throughput: 0: 1824.9, 1: 1811.4. Samples: 48720728. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:32,016][52710] Avg episode reward: [(0, '31.940'), (1, '35.060')] [2023-10-08 11:32:32,343][53852] Updated weights for policy 0, policy_version 95370 (0.0009) [2023-10-08 11:32:32,717][53852] Updated weights for policy 0, policy_version 95380 (0.0008) [2023-10-08 11:32:32,980][53885] Updated weights for policy 1, policy_version 94922 (0.0009) [2023-10-08 11:32:33,088][53852] Updated weights for policy 0, policy_version 95390 (0.0007) [2023-10-08 11:32:33,348][53885] Updated weights for policy 1, policy_version 94932 (0.0008) [2023-10-08 11:32:33,707][53885] Updated weights for policy 1, policy_version 94942 (0.0010) [2023-10-08 11:32:36,947][53852] Updated weights for policy 0, policy_version 95400 (0.0008) [2023-10-08 11:32:37,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194904064. Throughput: 0: 1819.2, 1: 1813.2. Samples: 48743386. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:37,016][52710] Avg episode reward: [(0, '33.440'), (1, '36.570')] [2023-10-08 11:32:37,318][53852] Updated weights for policy 0, policy_version 95410 (0.0008) [2023-10-08 11:32:37,464][53885] Updated weights for policy 1, policy_version 94952 (0.0009) [2023-10-08 11:32:37,696][53852] Updated weights for policy 0, policy_version 95420 (0.0008) [2023-10-08 11:32:37,830][53885] Updated weights for policy 1, policy_version 94962 (0.0008) [2023-10-08 11:32:38,195][53885] Updated weights for policy 1, policy_version 94972 (0.0009) [2023-10-08 11:32:41,393][53852] Updated weights for policy 0, policy_version 95430 (0.0008) [2023-10-08 11:32:41,728][53885] Updated weights for policy 1, policy_version 94982 (0.0009) [2023-10-08 11:32:41,758][53852] Updated weights for policy 0, policy_version 95440 (0.0009) [2023-10-08 11:32:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 194969600. Throughput: 0: 1825.1, 1: 1820.1. Samples: 48753514. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:42,016][52710] Avg episode reward: [(0, '31.890'), (1, '38.260')] [2023-10-08 11:32:42,099][53885] Updated weights for policy 1, policy_version 94992 (0.0007) [2023-10-08 11:32:42,122][53852] Updated weights for policy 0, policy_version 95450 (0.0008) [2023-10-08 11:32:42,463][53885] Updated weights for policy 1, policy_version 95002 (0.0008) [2023-10-08 11:32:45,670][53852] Updated weights for policy 0, policy_version 95460 (0.0007) [2023-10-08 11:32:46,038][53852] Updated weights for policy 0, policy_version 95470 (0.0007) [2023-10-08 11:32:46,208][53885] Updated weights for policy 1, policy_version 95012 (0.0009) [2023-10-08 11:32:46,406][53852] Updated weights for policy 0, policy_version 95480 (0.0007) [2023-10-08 11:32:46,574][53885] Updated weights for policy 1, policy_version 95022 (0.0007) [2023-10-08 11:32:46,936][53885] Updated weights for policy 1, policy_version 95032 (0.0007) [2023-10-08 11:32:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 195067904. Throughput: 0: 1831.5, 1: 1821.5. Samples: 48776636. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:47,016][52710] Avg episode reward: [(0, '31.390'), (1, '35.060')] [2023-10-08 11:32:49,923][53852] Updated weights for policy 0, policy_version 95490 (0.0008) [2023-10-08 11:32:50,283][53852] Updated weights for policy 0, policy_version 95500 (0.0009) [2023-10-08 11:32:50,473][53885] Updated weights for policy 1, policy_version 95042 (0.0007) [2023-10-08 11:32:50,652][53852] Updated weights for policy 0, policy_version 95510 (0.0009) [2023-10-08 11:32:50,842][53885] Updated weights for policy 1, policy_version 95052 (0.0008) [2023-10-08 11:32:51,021][53852] Updated weights for policy 0, policy_version 95520 (0.0009) [2023-10-08 11:32:51,207][53885] Updated weights for policy 1, policy_version 95062 (0.0010) [2023-10-08 11:32:51,582][53885] Updated weights for policy 1, policy_version 95072 (0.0010) [2023-10-08 11:32:52,015][52710] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 195166208. Throughput: 0: 1834.4, 1: 1825.0. Samples: 48797044. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:52,016][52710] Avg episode reward: [(0, '34.330'), (1, '35.940')] [2023-10-08 11:32:54,630][53852] Updated weights for policy 0, policy_version 95530 (0.0009) [2023-10-08 11:32:54,997][53852] Updated weights for policy 0, policy_version 95540 (0.0009) [2023-10-08 11:32:55,283][53885] Updated weights for policy 1, policy_version 95082 (0.0009) [2023-10-08 11:32:55,357][53852] Updated weights for policy 0, policy_version 95550 (0.0008) [2023-10-08 11:32:55,643][53885] Updated weights for policy 1, policy_version 95092 (0.0010) [2023-10-08 11:32:56,013][53885] Updated weights for policy 1, policy_version 95102 (0.0009) [2023-10-08 11:32:57,015][52710] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 195231744. Throughput: 0: 1824.2, 1: 1831.8. Samples: 48809788. Policy #0 lag: (min: 19.0, avg: 24.2, max: 51.0) [2023-10-08 11:32:57,015][52710] Avg episode reward: [(0, '35.060'), (1, '40.130')] [2023-10-08 11:32:59,009][53852] Updated weights for policy 0, policy_version 95560 (0.0007) [2023-10-08 11:32:59,377][53852] Updated weights for policy 0, policy_version 95570 (0.0007) [2023-10-08 11:32:59,683][53885] Updated weights for policy 1, policy_version 95112 (0.0008) [2023-10-08 11:32:59,739][53852] Updated weights for policy 0, policy_version 95580 (0.0007) [2023-10-08 11:33:00,043][53885] Updated weights for policy 1, policy_version 95122 (0.0007) [2023-10-08 11:33:00,414][53885] Updated weights for policy 1, policy_version 95132 (0.0009) [2023-10-08 11:33:02,015][52710] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 195297280. Throughput: 0: 1836.3, 1: 1821.7. Samples: 48830008. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:02,016][52710] Avg episode reward: [(0, '32.040'), (1, '35.780')] [2023-10-08 11:33:03,277][53852] Updated weights for policy 0, policy_version 95590 (0.0008) [2023-10-08 11:33:03,654][53852] Updated weights for policy 0, policy_version 95600 (0.0009) [2023-10-08 11:33:04,022][53852] Updated weights for policy 0, policy_version 95610 (0.0010) [2023-10-08 11:33:04,166][53885] Updated weights for policy 1, policy_version 95142 (0.0009) [2023-10-08 11:33:04,522][53885] Updated weights for policy 1, policy_version 95152 (0.0008) [2023-10-08 11:33:04,890][53885] Updated weights for policy 1, policy_version 95162 (0.0010) [2023-10-08 11:33:07,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 195362816. Throughput: 0: 1841.6, 1: 1826.5. Samples: 48853052. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:07,015][52710] Avg episode reward: [(0, '32.450'), (1, '36.240')] [2023-10-08 11:33:07,669][53852] Updated weights for policy 0, policy_version 95620 (0.0007) [2023-10-08 11:33:08,045][53852] Updated weights for policy 0, policy_version 95630 (0.0010) [2023-10-08 11:33:08,420][53852] Updated weights for policy 0, policy_version 95640 (0.0012) [2023-10-08 11:33:08,667][53885] Updated weights for policy 1, policy_version 95172 (0.0009) [2023-10-08 11:33:09,029][53885] Updated weights for policy 1, policy_version 95182 (0.0008) [2023-10-08 11:33:09,401][53885] Updated weights for policy 1, policy_version 95192 (0.0008) [2023-10-08 11:33:12,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195428352. Throughput: 0: 1841.4, 1: 1825.7. Samples: 48863202. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:12,015][52710] Avg episode reward: [(0, '34.850'), (1, '36.630')] [2023-10-08 11:33:12,147][53852] Updated weights for policy 0, policy_version 95650 (0.0008) [2023-10-08 11:33:12,519][53852] Updated weights for policy 0, policy_version 95660 (0.0007) [2023-10-08 11:33:12,892][53852] Updated weights for policy 0, policy_version 95670 (0.0007) [2023-10-08 11:33:13,106][53885] Updated weights for policy 1, policy_version 95202 (0.0009) [2023-10-08 11:33:13,263][53852] Updated weights for policy 0, policy_version 95680 (0.0007) [2023-10-08 11:33:13,471][53885] Updated weights for policy 1, policy_version 95212 (0.0010) [2023-10-08 11:33:13,845][53885] Updated weights for policy 1, policy_version 95222 (0.0008) [2023-10-08 11:33:14,205][53885] Updated weights for policy 1, policy_version 95232 (0.0009) [2023-10-08 11:33:17,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 195493888. Throughput: 0: 1841.4, 1: 1832.0. Samples: 48886032. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:17,016][52710] Avg episode reward: [(0, '32.110'), (1, '39.460')] [2023-10-08 11:33:17,071][53852] Updated weights for policy 0, policy_version 95690 (0.0010) [2023-10-08 11:33:17,444][53852] Updated weights for policy 0, policy_version 95700 (0.0008) [2023-10-08 11:33:17,809][53852] Updated weights for policy 0, policy_version 95710 (0.0009) [2023-10-08 11:33:17,858][53885] Updated weights for policy 1, policy_version 95242 (0.0008) [2023-10-08 11:33:18,230][53885] Updated weights for policy 1, policy_version 95252 (0.0009) [2023-10-08 11:33:18,600][53885] Updated weights for policy 1, policy_version 95262 (0.0009) [2023-10-08 11:33:21,378][53852] Updated weights for policy 0, policy_version 95720 (0.0009) [2023-10-08 11:33:21,752][53852] Updated weights for policy 0, policy_version 95730 (0.0009) [2023-10-08 11:33:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 195559424. Throughput: 0: 1834.0, 1: 1829.0. Samples: 48908224. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:22,015][52710] Avg episode reward: [(0, '32.040'), (1, '32.870')] [2023-10-08 11:33:22,118][53852] Updated weights for policy 0, policy_version 95740 (0.0010) [2023-10-08 11:33:22,306][53885] Updated weights for policy 1, policy_version 95272 (0.0008) [2023-10-08 11:33:22,682][53885] Updated weights for policy 1, policy_version 95282 (0.0010) [2023-10-08 11:33:23,039][53885] Updated weights for policy 1, policy_version 95292 (0.0009) [2023-10-08 11:33:25,765][53852] Updated weights for policy 0, policy_version 95750 (0.0008) [2023-10-08 11:33:26,133][53852] Updated weights for policy 0, policy_version 95760 (0.0008) [2023-10-08 11:33:26,504][53852] Updated weights for policy 0, policy_version 95770 (0.0007) [2023-10-08 11:33:26,514][53885] Updated weights for policy 1, policy_version 95302 (0.0008) [2023-10-08 11:33:26,873][53885] Updated weights for policy 1, policy_version 95312 (0.0007) [2023-10-08 11:33:27,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195657728. Throughput: 0: 1845.7, 1: 1829.3. Samples: 48918892. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:27,016][52710] Avg episode reward: [(0, '33.530'), (1, '34.760')] [2023-10-08 11:33:27,243][53885] Updated weights for policy 1, policy_version 95322 (0.0007) [2023-10-08 11:33:29,995][53852] Updated weights for policy 0, policy_version 95780 (0.0007) [2023-10-08 11:33:30,360][53852] Updated weights for policy 0, policy_version 95790 (0.0008) [2023-10-08 11:33:30,728][53852] Updated weights for policy 0, policy_version 95800 (0.0010) [2023-10-08 11:33:30,964][53885] Updated weights for policy 1, policy_version 95332 (0.0007) [2023-10-08 11:33:31,335][53885] Updated weights for policy 1, policy_version 95342 (0.0007) [2023-10-08 11:33:31,703][53885] Updated weights for policy 1, policy_version 95352 (0.0007) [2023-10-08 11:33:32,015][52710] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 195756032. Throughput: 0: 1827.6, 1: 1834.9. Samples: 48941448. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:32,016][52710] Avg episode reward: [(0, '33.350'), (1, '37.430')] [2023-10-08 11:33:34,218][53852] Updated weights for policy 0, policy_version 95810 (0.0007) [2023-10-08 11:33:34,581][53852] Updated weights for policy 0, policy_version 95820 (0.0009) [2023-10-08 11:33:34,956][53852] Updated weights for policy 0, policy_version 95830 (0.0008) [2023-10-08 11:33:35,193][53885] Updated weights for policy 1, policy_version 95362 (0.0007) [2023-10-08 11:33:35,322][53852] Updated weights for policy 0, policy_version 95840 (0.0008) [2023-10-08 11:33:35,566][53885] Updated weights for policy 1, policy_version 95372 (0.0008) [2023-10-08 11:33:35,938][53885] Updated weights for policy 1, policy_version 95382 (0.0008) [2023-10-08 11:33:36,298][53885] Updated weights for policy 1, policy_version 95392 (0.0008) [2023-10-08 11:33:37,015][52710] Fps is (10 sec: 16383.6, 60 sec: 15291.6, 300 sec: 14662.3). Total num frames: 195821568. Throughput: 0: 1851.3, 1: 1832.4. Samples: 48962812. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:37,016][52710] Avg episode reward: [(0, '33.690'), (1, '33.200')] [2023-10-08 11:33:38,907][53852] Updated weights for policy 0, policy_version 95850 (0.0009) [2023-10-08 11:33:39,277][53852] Updated weights for policy 0, policy_version 95860 (0.0009) [2023-10-08 11:33:39,657][53852] Updated weights for policy 0, policy_version 95870 (0.0009) [2023-10-08 11:33:40,040][53885] Updated weights for policy 1, policy_version 95402 (0.0009) [2023-10-08 11:33:40,406][53885] Updated weights for policy 1, policy_version 95412 (0.0008) [2023-10-08 11:33:40,778][53885] Updated weights for policy 1, policy_version 95422 (0.0008) [2023-10-08 11:33:42,015][52710] Fps is (10 sec: 13107.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 195887104. Throughput: 0: 1832.0, 1: 1830.8. Samples: 48974614. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:42,015][52710] Avg episode reward: [(0, '36.170'), (1, '31.100')] [2023-10-08 11:33:43,385][53852] Updated weights for policy 0, policy_version 95880 (0.0008) [2023-10-08 11:33:43,748][53852] Updated weights for policy 0, policy_version 95890 (0.0009) [2023-10-08 11:33:44,118][53852] Updated weights for policy 0, policy_version 95900 (0.0010) [2023-10-08 11:33:44,355][53885] Updated weights for policy 1, policy_version 95432 (0.0008) [2023-10-08 11:33:44,719][53885] Updated weights for policy 1, policy_version 95442 (0.0007) [2023-10-08 11:33:45,093][53885] Updated weights for policy 1, policy_version 95452 (0.0008) [2023-10-08 11:33:47,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195952640. Throughput: 0: 1844.0, 1: 1835.8. Samples: 48995600. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:47,016][52710] Avg episode reward: [(0, '30.730'), (1, '40.000')] [2023-10-08 11:33:47,663][53852] Updated weights for policy 0, policy_version 95910 (0.0008) [2023-10-08 11:33:48,026][53852] Updated weights for policy 0, policy_version 95920 (0.0007) [2023-10-08 11:33:48,384][53852] Updated weights for policy 0, policy_version 95930 (0.0007) [2023-10-08 11:33:48,758][53885] Updated weights for policy 1, policy_version 95462 (0.0010) [2023-10-08 11:33:49,121][53885] Updated weights for policy 1, policy_version 95472 (0.0007) [2023-10-08 11:33:49,496][53885] Updated weights for policy 1, policy_version 95482 (0.0007) [2023-10-08 11:33:52,015][52710] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 196018176. Throughput: 0: 1841.4, 1: 1834.9. Samples: 49018486. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:52,016][52710] Avg episode reward: [(0, '33.890'), (1, '35.740')] [2023-10-08 11:33:52,239][53852] Updated weights for policy 0, policy_version 95940 (0.0007) [2023-10-08 11:33:52,607][53852] Updated weights for policy 0, policy_version 95950 (0.0007) [2023-10-08 11:33:52,972][53852] Updated weights for policy 0, policy_version 95960 (0.0007) [2023-10-08 11:33:53,168][53885] Updated weights for policy 1, policy_version 95492 (0.0008) [2023-10-08 11:33:53,531][53885] Updated weights for policy 1, policy_version 95502 (0.0009) [2023-10-08 11:33:53,901][53885] Updated weights for policy 1, policy_version 95512 (0.0010) [2023-10-08 11:33:56,582][53852] Updated weights for policy 0, policy_version 95970 (0.0007) [2023-10-08 11:33:56,970][53852] Updated weights for policy 0, policy_version 95980 (0.0010) [2023-10-08 11:33:57,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196083712. Throughput: 0: 1842.4, 1: 1830.7. Samples: 49028490. Policy #0 lag: (min: 2.0, avg: 2.9, max: 23.0) [2023-10-08 11:33:57,016][52710] Avg episode reward: [(0, '36.960'), (1, '34.080')] [2023-10-08 11:33:57,337][53852] Updated weights for policy 0, policy_version 95990 (0.0010) [2023-10-08 11:33:57,656][53885] Updated weights for policy 1, policy_version 95522 (0.0009) [2023-10-08 11:33:57,705][53852] Updated weights for policy 0, policy_version 96000 (0.0008) [2023-10-08 11:33:58,020][53885] Updated weights for policy 1, policy_version 95532 (0.0009) [2023-10-08 11:33:58,390][53885] Updated weights for policy 1, policy_version 95542 (0.0007) [2023-10-08 11:33:58,760][53885] Updated weights for policy 1, policy_version 95552 (0.0008) [2023-10-08 11:34:01,446][53852] Updated weights for policy 0, policy_version 96010 (0.0008) [2023-10-08 11:34:01,810][53852] Updated weights for policy 0, policy_version 96020 (0.0009) [2023-10-08 11:34:02,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196149248. Throughput: 0: 1841.6, 1: 1834.1. Samples: 49051440. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:02,016][52710] Avg episode reward: [(0, '33.760'), (1, '37.010')] [2023-10-08 11:34:02,180][53852] Updated weights for policy 0, policy_version 96030 (0.0008) [2023-10-08 11:34:02,459][53885] Updated weights for policy 1, policy_version 95562 (0.0008) [2023-10-08 11:34:02,833][53885] Updated weights for policy 1, policy_version 95572 (0.0010) [2023-10-08 11:34:03,199][53885] Updated weights for policy 1, policy_version 95582 (0.0009) [2023-10-08 11:34:06,012][53852] Updated weights for policy 0, policy_version 96040 (0.0008) [2023-10-08 11:34:06,395][53852] Updated weights for policy 0, policy_version 96050 (0.0007) [2023-10-08 11:34:06,764][53852] Updated weights for policy 0, policy_version 96060 (0.0007) [2023-10-08 11:34:06,819][53885] Updated weights for policy 1, policy_version 95592 (0.0007) [2023-10-08 11:34:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196247552. Throughput: 0: 1827.9, 1: 1832.0. Samples: 49072920. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:07,016][52710] Avg episode reward: [(0, '32.270'), (1, '33.150')] [2023-10-08 11:34:07,025][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000096064_98369536.pth... [2023-10-08 11:34:07,058][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000094336_96600064.pth [2023-10-08 11:34:07,174][53885] Updated weights for policy 1, policy_version 95602 (0.0010) [2023-10-08 11:34:07,546][53885] Updated weights for policy 1, policy_version 95612 (0.0010) [2023-10-08 11:34:07,692][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000095616_97910784.pth... [2023-10-08 11:34:07,722][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000093888_96141312.pth [2023-10-08 11:34:10,400][53852] Updated weights for policy 0, policy_version 96070 (0.0007) [2023-10-08 11:34:10,768][53852] Updated weights for policy 0, policy_version 96080 (0.0009) [2023-10-08 11:34:11,140][53852] Updated weights for policy 0, policy_version 96090 (0.0008) [2023-10-08 11:34:11,306][53885] Updated weights for policy 1, policy_version 95622 (0.0008) [2023-10-08 11:34:11,669][53885] Updated weights for policy 1, policy_version 95632 (0.0008) [2023-10-08 11:34:12,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196313088. Throughput: 0: 1843.2, 1: 1827.7. Samples: 49084084. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:12,015][52710] Avg episode reward: [(0, '37.680'), (1, '33.790')] [2023-10-08 11:34:12,037][53885] Updated weights for policy 1, policy_version 95642 (0.0008) [2023-10-08 11:34:14,745][53852] Updated weights for policy 0, policy_version 96100 (0.0007) [2023-10-08 11:34:15,115][53852] Updated weights for policy 0, policy_version 96110 (0.0007) [2023-10-08 11:34:15,496][53852] Updated weights for policy 0, policy_version 96120 (0.0009) [2023-10-08 11:34:15,558][53885] Updated weights for policy 1, policy_version 95652 (0.0007) [2023-10-08 11:34:15,915][53885] Updated weights for policy 1, policy_version 95662 (0.0007) [2023-10-08 11:34:16,287][53885] Updated weights for policy 1, policy_version 95672 (0.0010) [2023-10-08 11:34:17,015][52710] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 196411392. Throughput: 0: 1831.9, 1: 1823.2. Samples: 49105926. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:17,016][52710] Avg episode reward: [(0, '38.030'), (1, '36.590')] [2023-10-08 11:34:18,905][53852] Updated weights for policy 0, policy_version 96130 (0.0009) [2023-10-08 11:34:19,278][53852] Updated weights for policy 0, policy_version 96140 (0.0010) [2023-10-08 11:34:19,640][53852] Updated weights for policy 0, policy_version 96150 (0.0007) [2023-10-08 11:34:20,004][53852] Updated weights for policy 0, policy_version 96160 (0.0007) [2023-10-08 11:34:20,059][53885] Updated weights for policy 1, policy_version 95682 (0.0010) [2023-10-08 11:34:20,422][53885] Updated weights for policy 1, policy_version 95692 (0.0008) [2023-10-08 11:34:20,796][53885] Updated weights for policy 1, policy_version 95702 (0.0007) [2023-10-08 11:34:21,160][53885] Updated weights for policy 1, policy_version 95712 (0.0007) [2023-10-08 11:34:22,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 196476928. Throughput: 0: 1836.7, 1: 1821.5. Samples: 49127428. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:22,016][52710] Avg episode reward: [(0, '37.040'), (1, '38.490')] [2023-10-08 11:34:23,570][53852] Updated weights for policy 0, policy_version 96170 (0.0007) [2023-10-08 11:34:23,930][53852] Updated weights for policy 0, policy_version 96180 (0.0009) [2023-10-08 11:34:24,296][53852] Updated weights for policy 0, policy_version 96190 (0.0009) [2023-10-08 11:34:24,819][53885] Updated weights for policy 1, policy_version 95722 (0.0009) [2023-10-08 11:34:25,181][53885] Updated weights for policy 1, policy_version 95732 (0.0008) [2023-10-08 11:34:25,546][53885] Updated weights for policy 1, policy_version 95742 (0.0009) [2023-10-08 11:34:27,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196542464. Throughput: 0: 1829.7, 1: 1817.0. Samples: 49138716. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:27,016][52710] Avg episode reward: [(0, '38.750'), (1, '37.100')] [2023-10-08 11:34:28,111][53852] Updated weights for policy 0, policy_version 96200 (0.0009) [2023-10-08 11:34:28,484][53852] Updated weights for policy 0, policy_version 96210 (0.0007) [2023-10-08 11:34:28,855][53852] Updated weights for policy 0, policy_version 96220 (0.0007) [2023-10-08 11:34:29,327][53885] Updated weights for policy 1, policy_version 95752 (0.0008) [2023-10-08 11:34:29,700][53885] Updated weights for policy 1, policy_version 95762 (0.0008) [2023-10-08 11:34:30,070][53885] Updated weights for policy 1, policy_version 95772 (0.0009) [2023-10-08 11:34:32,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 196608000. Throughput: 0: 1838.3, 1: 1821.6. Samples: 49160296. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:32,016][52710] Avg episode reward: [(0, '38.240'), (1, '37.080')] [2023-10-08 11:34:32,553][53852] Updated weights for policy 0, policy_version 96230 (0.0007) [2023-10-08 11:34:32,920][53852] Updated weights for policy 0, policy_version 96240 (0.0007) [2023-10-08 11:34:33,291][53852] Updated weights for policy 0, policy_version 96250 (0.0007) [2023-10-08 11:34:33,770][53885] Updated weights for policy 1, policy_version 95782 (0.0008) [2023-10-08 11:34:34,144][53885] Updated weights for policy 1, policy_version 95792 (0.0010) [2023-10-08 11:34:34,514][53885] Updated weights for policy 1, policy_version 95802 (0.0009) [2023-10-08 11:34:36,752][53852] Updated weights for policy 0, policy_version 96260 (0.0007) [2023-10-08 11:34:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14662.3). Total num frames: 196673536. Throughput: 0: 1846.9, 1: 1824.9. Samples: 49183720. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:37,016][52710] Avg episode reward: [(0, '36.340'), (1, '35.990')] [2023-10-08 11:34:37,119][53852] Updated weights for policy 0, policy_version 96270 (0.0008) [2023-10-08 11:34:37,496][53852] Updated weights for policy 0, policy_version 96280 (0.0009) [2023-10-08 11:34:38,128][53885] Updated weights for policy 1, policy_version 95812 (0.0008) [2023-10-08 11:34:38,496][53885] Updated weights for policy 1, policy_version 95822 (0.0008) [2023-10-08 11:34:38,872][53885] Updated weights for policy 1, policy_version 95832 (0.0007) [2023-10-08 11:34:41,140][53852] Updated weights for policy 0, policy_version 96290 (0.0009) [2023-10-08 11:34:41,511][53852] Updated weights for policy 0, policy_version 96300 (0.0007) [2023-10-08 11:34:41,868][53852] Updated weights for policy 0, policy_version 96310 (0.0008) [2023-10-08 11:34:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 196739072. Throughput: 0: 1849.0, 1: 1825.2. Samples: 49193826. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:42,016][52710] Avg episode reward: [(0, '36.020'), (1, '37.380')] [2023-10-08 11:34:42,249][53852] Updated weights for policy 0, policy_version 96320 (0.0008) [2023-10-08 11:34:42,403][53885] Updated weights for policy 1, policy_version 95842 (0.0010) [2023-10-08 11:34:42,775][53885] Updated weights for policy 1, policy_version 95852 (0.0007) [2023-10-08 11:34:43,144][53885] Updated weights for policy 1, policy_version 95862 (0.0008) [2023-10-08 11:34:43,507][53885] Updated weights for policy 1, policy_version 95872 (0.0009) [2023-10-08 11:34:45,907][53852] Updated weights for policy 0, policy_version 96330 (0.0010) [2023-10-08 11:34:46,270][53852] Updated weights for policy 0, policy_version 96340 (0.0009) [2023-10-08 11:34:46,639][53852] Updated weights for policy 0, policy_version 96350 (0.0008) [2023-10-08 11:34:47,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 196837376. Throughput: 0: 1847.6, 1: 1825.9. Samples: 49216750. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:47,016][52710] Avg episode reward: [(0, '38.330'), (1, '34.740')] [2023-10-08 11:34:47,279][53885] Updated weights for policy 1, policy_version 95882 (0.0008) [2023-10-08 11:34:47,646][53885] Updated weights for policy 1, policy_version 95892 (0.0009) [2023-10-08 11:34:48,013][53885] Updated weights for policy 1, policy_version 95902 (0.0008) [2023-10-08 11:34:50,120][53852] Updated weights for policy 0, policy_version 96360 (0.0007) [2023-10-08 11:34:50,487][53852] Updated weights for policy 0, policy_version 96370 (0.0008) [2023-10-08 11:34:50,861][53852] Updated weights for policy 0, policy_version 96380 (0.0011) [2023-10-08 11:34:51,731][53885] Updated weights for policy 1, policy_version 95912 (0.0009) [2023-10-08 11:34:52,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 196902912. Throughput: 0: 1846.0, 1: 1826.8. Samples: 49238198. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:52,015][52710] Avg episode reward: [(0, '36.470'), (1, '34.990')] [2023-10-08 11:34:52,104][53885] Updated weights for policy 1, policy_version 95922 (0.0009) [2023-10-08 11:34:52,474][53885] Updated weights for policy 1, policy_version 95932 (0.0008) [2023-10-08 11:34:54,491][53852] Updated weights for policy 0, policy_version 96390 (0.0009) [2023-10-08 11:34:54,867][53852] Updated weights for policy 0, policy_version 96400 (0.0007) [2023-10-08 11:34:55,235][53852] Updated weights for policy 0, policy_version 96410 (0.0007) [2023-10-08 11:34:56,126][53885] Updated weights for policy 1, policy_version 95942 (0.0008) [2023-10-08 11:34:56,494][53885] Updated weights for policy 1, policy_version 95952 (0.0007) [2023-10-08 11:34:56,858][53885] Updated weights for policy 1, policy_version 95962 (0.0007) [2023-10-08 11:34:57,015][52710] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 196968448. Throughput: 0: 1850.6, 1: 1832.2. Samples: 49249810. Policy #0 lag: (min: 25.0, avg: 33.6, max: 57.0) [2023-10-08 11:34:57,016][52710] Avg episode reward: [(0, '34.970'), (1, '35.220')] [2023-10-08 11:34:58,968][53852] Updated weights for policy 0, policy_version 96420 (0.0009) [2023-10-08 11:34:59,328][53852] Updated weights for policy 0, policy_version 96430 (0.0010) [2023-10-08 11:34:59,699][53852] Updated weights for policy 0, policy_version 96440 (0.0010) [2023-10-08 11:35:00,578][53885] Updated weights for policy 1, policy_version 95972 (0.0007) [2023-10-08 11:35:00,936][53885] Updated weights for policy 1, policy_version 95982 (0.0007) [2023-10-08 11:35:01,306][53885] Updated weights for policy 1, policy_version 95992 (0.0007) [2023-10-08 11:35:02,015][52710] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 197066752. Throughput: 0: 1847.5, 1: 1825.4. Samples: 49271206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:02,016][52710] Avg episode reward: [(0, '38.590'), (1, '35.000')] [2023-10-08 11:35:03,490][53852] Updated weights for policy 0, policy_version 96450 (0.0010) [2023-10-08 11:35:03,866][53852] Updated weights for policy 0, policy_version 96460 (0.0007) [2023-10-08 11:35:04,228][53852] Updated weights for policy 0, policy_version 96470 (0.0007) [2023-10-08 11:35:04,598][53852] Updated weights for policy 0, policy_version 96480 (0.0008) [2023-10-08 11:35:05,009][53885] Updated weights for policy 1, policy_version 96002 (0.0008) [2023-10-08 11:35:05,381][53885] Updated weights for policy 1, policy_version 96012 (0.0009) [2023-10-08 11:35:05,741][53885] Updated weights for policy 1, policy_version 96022 (0.0007) [2023-10-08 11:35:06,115][53885] Updated weights for policy 1, policy_version 96032 (0.0007) [2023-10-08 11:35:07,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197132288. Throughput: 0: 1845.8, 1: 1827.6. Samples: 49292730. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:07,016][52710] Avg episode reward: [(0, '36.680'), (1, '38.170')] [2023-10-08 11:35:08,235][53852] Updated weights for policy 0, policy_version 96490 (0.0007) [2023-10-08 11:35:08,607][53852] Updated weights for policy 0, policy_version 96500 (0.0008) [2023-10-08 11:35:08,980][53852] Updated weights for policy 0, policy_version 96510 (0.0007) [2023-10-08 11:35:09,848][53885] Updated weights for policy 1, policy_version 96042 (0.0010) [2023-10-08 11:35:10,210][53885] Updated weights for policy 1, policy_version 96052 (0.0008) [2023-10-08 11:35:10,570][53885] Updated weights for policy 1, policy_version 96062 (0.0008) [2023-10-08 11:35:12,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197197824. Throughput: 0: 1843.5, 1: 1828.5. Samples: 49303956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:12,016][52710] Avg episode reward: [(0, '35.410'), (1, '37.870')] [2023-10-08 11:35:12,575][53852] Updated weights for policy 0, policy_version 96520 (0.0007) [2023-10-08 11:35:12,952][53852] Updated weights for policy 0, policy_version 96530 (0.0007) [2023-10-08 11:35:13,327][53852] Updated weights for policy 0, policy_version 96540 (0.0008) [2023-10-08 11:35:14,220][53885] Updated weights for policy 1, policy_version 96072 (0.0011) [2023-10-08 11:35:14,584][53885] Updated weights for policy 1, policy_version 96082 (0.0008) [2023-10-08 11:35:14,957][53885] Updated weights for policy 1, policy_version 96092 (0.0009) [2023-10-08 11:35:16,970][53852] Updated weights for policy 0, policy_version 96550 (0.0009) [2023-10-08 11:35:17,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 197263360. Throughput: 0: 1846.3, 1: 1830.3. Samples: 49325746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:17,016][52710] Avg episode reward: [(0, '33.840'), (1, '39.970')] [2023-10-08 11:35:17,341][53852] Updated weights for policy 0, policy_version 96560 (0.0007) [2023-10-08 11:35:17,709][53852] Updated weights for policy 0, policy_version 96570 (0.0007) [2023-10-08 11:35:18,771][53885] Updated weights for policy 1, policy_version 96102 (0.0008) [2023-10-08 11:35:19,154][53885] Updated weights for policy 1, policy_version 96112 (0.0008) [2023-10-08 11:35:19,524][53885] Updated weights for policy 1, policy_version 96122 (0.0008) [2023-10-08 11:35:21,264][53852] Updated weights for policy 0, policy_version 96580 (0.0008) [2023-10-08 11:35:21,626][53852] Updated weights for policy 0, policy_version 96590 (0.0010) [2023-10-08 11:35:22,006][53852] Updated weights for policy 0, policy_version 96600 (0.0009) [2023-10-08 11:35:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 197328896. Throughput: 0: 1825.8, 1: 1828.4. Samples: 49348158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:22,016][52710] Avg episode reward: [(0, '36.000'), (1, '40.240')] [2023-10-08 11:35:23,170][53885] Updated weights for policy 1, policy_version 96132 (0.0007) [2023-10-08 11:35:23,547][53885] Updated weights for policy 1, policy_version 96142 (0.0008) [2023-10-08 11:35:23,928][53885] Updated weights for policy 1, policy_version 96152 (0.0009) [2023-10-08 11:35:25,553][53852] Updated weights for policy 0, policy_version 96610 (0.0008) [2023-10-08 11:35:25,921][53852] Updated weights for policy 0, policy_version 96620 (0.0008) [2023-10-08 11:35:26,297][53852] Updated weights for policy 0, policy_version 96630 (0.0007) [2023-10-08 11:35:26,673][53852] Updated weights for policy 0, policy_version 96640 (0.0008) [2023-10-08 11:35:27,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197427200. Throughput: 0: 1842.7, 1: 1824.3. Samples: 49358838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:27,016][52710] Avg episode reward: [(0, '32.610'), (1, '38.540')] [2023-10-08 11:35:27,696][53885] Updated weights for policy 1, policy_version 96162 (0.0009) [2023-10-08 11:35:28,061][53885] Updated weights for policy 1, policy_version 96172 (0.0007) [2023-10-08 11:35:28,433][53885] Updated weights for policy 1, policy_version 96182 (0.0007) [2023-10-08 11:35:28,806][53885] Updated weights for policy 1, policy_version 96192 (0.0008) [2023-10-08 11:35:30,264][53852] Updated weights for policy 0, policy_version 96650 (0.0010) [2023-10-08 11:35:30,635][53852] Updated weights for policy 0, policy_version 96660 (0.0011) [2023-10-08 11:35:31,004][53852] Updated weights for policy 0, policy_version 96670 (0.0011) [2023-10-08 11:35:32,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197492736. Throughput: 0: 1832.0, 1: 1826.2. Samples: 49381370. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:32,016][52710] Avg episode reward: [(0, '33.050'), (1, '36.980')] [2023-10-08 11:35:32,407][53885] Updated weights for policy 1, policy_version 96202 (0.0008) [2023-10-08 11:35:32,783][53885] Updated weights for policy 1, policy_version 96212 (0.0007) [2023-10-08 11:35:33,146][53885] Updated weights for policy 1, policy_version 96222 (0.0007) [2023-10-08 11:35:34,701][53852] Updated weights for policy 0, policy_version 96680 (0.0011) [2023-10-08 11:35:35,082][53852] Updated weights for policy 0, policy_version 96690 (0.0011) [2023-10-08 11:35:35,446][53852] Updated weights for policy 0, policy_version 96700 (0.0011) [2023-10-08 11:35:36,694][53885] Updated weights for policy 1, policy_version 96232 (0.0010) [2023-10-08 11:35:37,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197558272. Throughput: 0: 1843.4, 1: 1825.9. Samples: 49403316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:37,016][52710] Avg episode reward: [(0, '37.610'), (1, '41.590')] [2023-10-08 11:35:37,061][53885] Updated weights for policy 1, policy_version 96242 (0.0009) [2023-10-08 11:35:37,429][53885] Updated weights for policy 1, policy_version 96252 (0.0010) [2023-10-08 11:35:38,979][53852] Updated weights for policy 0, policy_version 96710 (0.0010) [2023-10-08 11:35:39,337][53852] Updated weights for policy 0, policy_version 96720 (0.0007) [2023-10-08 11:35:39,712][53852] Updated weights for policy 0, policy_version 96730 (0.0008) [2023-10-08 11:35:41,123][53885] Updated weights for policy 1, policy_version 96262 (0.0009) [2023-10-08 11:35:41,491][53885] Updated weights for policy 1, policy_version 96272 (0.0009) [2023-10-08 11:35:41,864][53885] Updated weights for policy 1, policy_version 96282 (0.0008) [2023-10-08 11:35:42,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197623808. Throughput: 0: 1829.0, 1: 1822.6. Samples: 49414130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:42,016][52710] Avg episode reward: [(0, '38.770'), (1, '38.160')] [2023-10-08 11:35:43,266][53852] Updated weights for policy 0, policy_version 96740 (0.0008) [2023-10-08 11:35:43,637][53852] Updated weights for policy 0, policy_version 96750 (0.0007) [2023-10-08 11:35:44,001][53852] Updated weights for policy 0, policy_version 96760 (0.0008) [2023-10-08 11:35:45,495][53885] Updated weights for policy 1, policy_version 96292 (0.0008) [2023-10-08 11:35:45,858][53885] Updated weights for policy 1, policy_version 96302 (0.0007) [2023-10-08 11:35:46,235][53885] Updated weights for policy 1, policy_version 96312 (0.0008) [2023-10-08 11:35:47,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197722112. Throughput: 0: 1852.4, 1: 1820.6. Samples: 49436492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:47,016][52710] Avg episode reward: [(0, '35.780'), (1, '34.940')] [2023-10-08 11:35:47,670][53852] Updated weights for policy 0, policy_version 96770 (0.0010) [2023-10-08 11:35:48,042][53852] Updated weights for policy 0, policy_version 96780 (0.0009) [2023-10-08 11:35:48,407][53852] Updated weights for policy 0, policy_version 96790 (0.0007) [2023-10-08 11:35:48,776][53852] Updated weights for policy 0, policy_version 96800 (0.0008) [2023-10-08 11:35:49,980][53885] Updated weights for policy 1, policy_version 96322 (0.0010) [2023-10-08 11:35:50,347][53885] Updated weights for policy 1, policy_version 96332 (0.0009) [2023-10-08 11:35:50,723][53885] Updated weights for policy 1, policy_version 96342 (0.0008) [2023-10-08 11:35:51,093][53885] Updated weights for policy 1, policy_version 96352 (0.0008) [2023-10-08 11:35:52,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197787648. Throughput: 0: 1859.2, 1: 1827.2. Samples: 49458618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:52,015][52710] Avg episode reward: [(0, '36.840'), (1, '37.090')] [2023-10-08 11:35:52,380][53852] Updated weights for policy 0, policy_version 96810 (0.0008) [2023-10-08 11:35:52,743][53852] Updated weights for policy 0, policy_version 96820 (0.0007) [2023-10-08 11:35:53,118][53852] Updated weights for policy 0, policy_version 96830 (0.0007) [2023-10-08 11:35:54,766][53885] Updated weights for policy 1, policy_version 96362 (0.0008) [2023-10-08 11:35:55,130][53885] Updated weights for policy 1, policy_version 96372 (0.0008) [2023-10-08 11:35:55,498][53885] Updated weights for policy 1, policy_version 96382 (0.0008) [2023-10-08 11:35:56,681][53852] Updated weights for policy 0, policy_version 96840 (0.0007) [2023-10-08 11:35:57,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197853184. Throughput: 0: 1861.5, 1: 1826.7. Samples: 49469922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:35:57,016][52710] Avg episode reward: [(0, '36.010'), (1, '40.460')] [2023-10-08 11:35:57,044][53852] Updated weights for policy 0, policy_version 96850 (0.0008) [2023-10-08 11:35:57,413][53852] Updated weights for policy 0, policy_version 96860 (0.0008) [2023-10-08 11:35:59,191][53885] Updated weights for policy 1, policy_version 96392 (0.0010) [2023-10-08 11:35:59,574][53885] Updated weights for policy 1, policy_version 96402 (0.0007) [2023-10-08 11:35:59,942][53885] Updated weights for policy 1, policy_version 96412 (0.0008) [2023-10-08 11:36:00,973][53852] Updated weights for policy 0, policy_version 96870 (0.0010) [2023-10-08 11:36:01,335][53852] Updated weights for policy 0, policy_version 96880 (0.0007) [2023-10-08 11:36:01,713][53852] Updated weights for policy 0, policy_version 96890 (0.0008) [2023-10-08 11:36:02,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 197951488. Throughput: 0: 1867.6, 1: 1828.0. Samples: 49492044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:02,016][52710] Avg episode reward: [(0, '33.030'), (1, '33.860')] [2023-10-08 11:36:03,705][53885] Updated weights for policy 1, policy_version 96422 (0.0008) [2023-10-08 11:36:04,072][53885] Updated weights for policy 1, policy_version 96432 (0.0007) [2023-10-08 11:36:04,437][53885] Updated weights for policy 1, policy_version 96442 (0.0007) [2023-10-08 11:36:05,447][53852] Updated weights for policy 0, policy_version 96900 (0.0008) [2023-10-08 11:36:05,806][53852] Updated weights for policy 0, policy_version 96910 (0.0008) [2023-10-08 11:36:06,175][53852] Updated weights for policy 0, policy_version 96920 (0.0008) [2023-10-08 11:36:07,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 198017024. Throughput: 0: 1842.3, 1: 1824.7. Samples: 49513174. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:07,016][52710] Avg episode reward: [(0, '30.430'), (1, '35.090')] [2023-10-08 11:36:07,031][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000096928_99254272.pth... [2023-10-08 11:36:07,031][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000096448_98762752.pth... [2023-10-08 11:36:07,065][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000094752_97026048.pth [2023-10-08 11:36:07,070][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000095200_97484800.pth [2023-10-08 11:36:08,174][53885] Updated weights for policy 1, policy_version 96452 (0.0007) [2023-10-08 11:36:08,545][53885] Updated weights for policy 1, policy_version 96462 (0.0007) [2023-10-08 11:36:08,905][53885] Updated weights for policy 1, policy_version 96472 (0.0008) [2023-10-08 11:36:09,916][53852] Updated weights for policy 0, policy_version 96930 (0.0009) [2023-10-08 11:36:10,289][53852] Updated weights for policy 0, policy_version 96940 (0.0010) [2023-10-08 11:36:10,646][53852] Updated weights for policy 0, policy_version 96950 (0.0010) [2023-10-08 11:36:11,021][53852] Updated weights for policy 0, policy_version 96960 (0.0011) [2023-10-08 11:36:12,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198082560. Throughput: 0: 1855.4, 1: 1820.6. Samples: 49524256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:12,016][52710] Avg episode reward: [(0, '35.050'), (1, '39.990')] [2023-10-08 11:36:12,498][53885] Updated weights for policy 1, policy_version 96482 (0.0010) [2023-10-08 11:36:12,874][53885] Updated weights for policy 1, policy_version 96492 (0.0007) [2023-10-08 11:36:13,256][53885] Updated weights for policy 1, policy_version 96502 (0.0009) [2023-10-08 11:36:13,608][53885] Updated weights for policy 1, policy_version 96512 (0.0010) [2023-10-08 11:36:14,677][53852] Updated weights for policy 0, policy_version 96970 (0.0009) [2023-10-08 11:36:15,046][53852] Updated weights for policy 0, policy_version 96980 (0.0008) [2023-10-08 11:36:15,418][53852] Updated weights for policy 0, policy_version 96990 (0.0010) [2023-10-08 11:36:17,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198148096. Throughput: 0: 1830.7, 1: 1820.4. Samples: 49545670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:17,016][52710] Avg episode reward: [(0, '34.770'), (1, '34.870')] [2023-10-08 11:36:17,357][53885] Updated weights for policy 1, policy_version 96522 (0.0009) [2023-10-08 11:36:17,727][53885] Updated weights for policy 1, policy_version 96532 (0.0009) [2023-10-08 11:36:18,097][53885] Updated weights for policy 1, policy_version 96542 (0.0009) [2023-10-08 11:36:19,017][53852] Updated weights for policy 0, policy_version 97000 (0.0010) [2023-10-08 11:36:19,386][53852] Updated weights for policy 0, policy_version 97010 (0.0011) [2023-10-08 11:36:19,755][53852] Updated weights for policy 0, policy_version 97020 (0.0009) [2023-10-08 11:36:21,530][53885] Updated weights for policy 1, policy_version 96552 (0.0011) [2023-10-08 11:36:21,894][53885] Updated weights for policy 1, policy_version 96562 (0.0010) [2023-10-08 11:36:22,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198213632. Throughput: 0: 1846.4, 1: 1816.6. Samples: 49568154. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:22,016][52710] Avg episode reward: [(0, '32.340'), (1, '34.090')] [2023-10-08 11:36:22,259][53885] Updated weights for policy 1, policy_version 96572 (0.0010) [2023-10-08 11:36:23,602][53852] Updated weights for policy 0, policy_version 97030 (0.0011) [2023-10-08 11:36:23,969][53852] Updated weights for policy 0, policy_version 97040 (0.0010) [2023-10-08 11:36:24,336][53852] Updated weights for policy 0, policy_version 97050 (0.0010) [2023-10-08 11:36:26,082][53885] Updated weights for policy 1, policy_version 96582 (0.0010) [2023-10-08 11:36:26,444][53885] Updated weights for policy 1, policy_version 96592 (0.0009) [2023-10-08 11:36:26,806][53885] Updated weights for policy 1, policy_version 96602 (0.0007) [2023-10-08 11:36:27,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198279168. Throughput: 0: 1831.2, 1: 1825.6. Samples: 49578690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:27,016][52710] Avg episode reward: [(0, '37.810'), (1, '36.450')] [2023-10-08 11:36:27,945][53852] Updated weights for policy 0, policy_version 97060 (0.0009) [2023-10-08 11:36:28,310][53852] Updated weights for policy 0, policy_version 97070 (0.0010) [2023-10-08 11:36:28,686][53852] Updated weights for policy 0, policy_version 97080 (0.0011) [2023-10-08 11:36:30,448][53885] Updated weights for policy 1, policy_version 96612 (0.0009) [2023-10-08 11:36:30,812][53885] Updated weights for policy 1, policy_version 96622 (0.0009) [2023-10-08 11:36:31,182][53885] Updated weights for policy 1, policy_version 96632 (0.0009) [2023-10-08 11:36:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198377472. Throughput: 0: 1839.9, 1: 1824.0. Samples: 49601368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:32,016][52710] Avg episode reward: [(0, '35.450'), (1, '37.520')] [2023-10-08 11:36:32,416][53852] Updated weights for policy 0, policy_version 97090 (0.0008) [2023-10-08 11:36:32,812][53852] Updated weights for policy 0, policy_version 97100 (0.0009) [2023-10-08 11:36:33,181][53852] Updated weights for policy 0, policy_version 97110 (0.0009) [2023-10-08 11:36:33,555][53852] Updated weights for policy 0, policy_version 97120 (0.0009) [2023-10-08 11:36:34,905][53885] Updated weights for policy 1, policy_version 96642 (0.0008) [2023-10-08 11:36:35,270][53885] Updated weights for policy 1, policy_version 96652 (0.0007) [2023-10-08 11:36:35,640][53885] Updated weights for policy 1, policy_version 96662 (0.0007) [2023-10-08 11:36:36,003][53885] Updated weights for policy 1, policy_version 96672 (0.0007) [2023-10-08 11:36:37,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198443008. Throughput: 0: 1843.0, 1: 1827.0. Samples: 49623768. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:37,016][52710] Avg episode reward: [(0, '34.780'), (1, '37.050')] [2023-10-08 11:36:37,098][53852] Updated weights for policy 0, policy_version 97130 (0.0007) [2023-10-08 11:36:37,468][53852] Updated weights for policy 0, policy_version 97140 (0.0008) [2023-10-08 11:36:37,845][53852] Updated weights for policy 0, policy_version 97150 (0.0009) [2023-10-08 11:36:39,383][53885] Updated weights for policy 1, policy_version 96682 (0.0008) [2023-10-08 11:36:39,755][53885] Updated weights for policy 1, policy_version 96692 (0.0007) [2023-10-08 11:36:40,125][53885] Updated weights for policy 1, policy_version 96702 (0.0010) [2023-10-08 11:36:41,368][53852] Updated weights for policy 0, policy_version 97160 (0.0008) [2023-10-08 11:36:41,738][53852] Updated weights for policy 0, policy_version 97170 (0.0008) [2023-10-08 11:36:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198508544. Throughput: 0: 1841.9, 1: 1825.4. Samples: 49634950. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:42,016][52710] Avg episode reward: [(0, '37.430'), (1, '36.450')] [2023-10-08 11:36:42,114][53852] Updated weights for policy 0, policy_version 97180 (0.0008) [2023-10-08 11:36:43,734][53885] Updated weights for policy 1, policy_version 96712 (0.0009) [2023-10-08 11:36:44,102][53885] Updated weights for policy 1, policy_version 96722 (0.0009) [2023-10-08 11:36:44,462][53885] Updated weights for policy 1, policy_version 96732 (0.0009) [2023-10-08 11:36:45,595][53852] Updated weights for policy 0, policy_version 97190 (0.0008) [2023-10-08 11:36:45,963][53852] Updated weights for policy 0, policy_version 97200 (0.0008) [2023-10-08 11:36:46,342][53852] Updated weights for policy 0, policy_version 97210 (0.0008) [2023-10-08 11:36:47,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 198606848. Throughput: 0: 1828.9, 1: 1835.5. Samples: 49656942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:47,015][52710] Avg episode reward: [(0, '37.430'), (1, '36.480')] [2023-10-08 11:36:48,098][53885] Updated weights for policy 1, policy_version 96742 (0.0008) [2023-10-08 11:36:48,475][53885] Updated weights for policy 1, policy_version 96752 (0.0009) [2023-10-08 11:36:48,849][53885] Updated weights for policy 1, policy_version 96762 (0.0008) [2023-10-08 11:36:49,880][53852] Updated weights for policy 0, policy_version 97220 (0.0009) [2023-10-08 11:36:50,249][53852] Updated weights for policy 0, policy_version 97230 (0.0008) [2023-10-08 11:36:50,616][53852] Updated weights for policy 0, policy_version 97240 (0.0007) [2023-10-08 11:36:52,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 198672384. Throughput: 0: 1839.0, 1: 1845.9. Samples: 49678996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:52,016][52710] Avg episode reward: [(0, '31.550'), (1, '35.290')] [2023-10-08 11:36:52,376][53885] Updated weights for policy 1, policy_version 96772 (0.0008) [2023-10-08 11:36:52,747][53885] Updated weights for policy 1, policy_version 96782 (0.0007) [2023-10-08 11:36:53,115][53885] Updated weights for policy 1, policy_version 96792 (0.0007) [2023-10-08 11:36:54,107][53852] Updated weights for policy 0, policy_version 97250 (0.0010) [2023-10-08 11:36:54,466][53852] Updated weights for policy 0, policy_version 97260 (0.0007) [2023-10-08 11:36:54,842][53852] Updated weights for policy 0, policy_version 97270 (0.0008) [2023-10-08 11:36:55,210][53852] Updated weights for policy 0, policy_version 97280 (0.0010) [2023-10-08 11:36:56,721][53885] Updated weights for policy 1, policy_version 96802 (0.0007) [2023-10-08 11:36:57,015][52710] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 198737920. Throughput: 0: 1835.7, 1: 1852.3. Samples: 49690214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:36:57,016][52710] Avg episode reward: [(0, '33.470'), (1, '36.370')] [2023-10-08 11:36:57,090][53885] Updated weights for policy 1, policy_version 96812 (0.0007) [2023-10-08 11:36:57,463][53885] Updated weights for policy 1, policy_version 96822 (0.0009) [2023-10-08 11:36:57,834][53885] Updated weights for policy 1, policy_version 96832 (0.0008) [2023-10-08 11:36:58,981][53852] Updated weights for policy 0, policy_version 97290 (0.0007) [2023-10-08 11:36:59,342][53852] Updated weights for policy 0, policy_version 97300 (0.0007) [2023-10-08 11:36:59,711][53852] Updated weights for policy 0, policy_version 97310 (0.0007) [2023-10-08 11:37:01,583][53885] Updated weights for policy 1, policy_version 96842 (0.0007) [2023-10-08 11:37:01,961][53885] Updated weights for policy 1, policy_version 96852 (0.0009) [2023-10-08 11:37:02,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198803456. Throughput: 0: 1845.5, 1: 1853.2. Samples: 49712108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) [2023-10-08 11:37:02,016][52710] Avg episode reward: [(0, '34.660'), (1, '38.940')] [2023-10-08 11:37:02,329][53885] Updated weights for policy 1, policy_version 96862 (0.0007) [2023-10-08 11:37:03,508][53852] Updated weights for policy 0, policy_version 97320 (0.0010) [2023-10-08 11:37:03,872][53852] Updated weights for policy 0, policy_version 97330 (0.0010) [2023-10-08 11:37:04,251][53852] Updated weights for policy 0, policy_version 97340 (0.0008) [2023-10-08 11:37:05,900][53885] Updated weights for policy 1, policy_version 96872 (0.0009) [2023-10-08 11:37:06,277][53885] Updated weights for policy 1, policy_version 96882 (0.0009) [2023-10-08 11:37:06,645][53885] Updated weights for policy 1, policy_version 96892 (0.0007) [2023-10-08 11:37:07,015][52710] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198901760. Throughput: 0: 1851.5, 1: 1834.7. Samples: 49734034. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:07,016][52710] Avg episode reward: [(0, '31.270'), (1, '38.130')] [2023-10-08 11:37:08,108][53852] Updated weights for policy 0, policy_version 97350 (0.0009) [2023-10-08 11:37:08,486][53852] Updated weights for policy 0, policy_version 97360 (0.0008) [2023-10-08 11:37:08,851][53852] Updated weights for policy 0, policy_version 97370 (0.0008) [2023-10-08 11:37:10,231][53885] Updated weights for policy 1, policy_version 96902 (0.0009) [2023-10-08 11:37:10,595][53885] Updated weights for policy 1, policy_version 96912 (0.0009) [2023-10-08 11:37:10,957][53885] Updated weights for policy 1, policy_version 96922 (0.0011) [2023-10-08 11:37:12,015][52710] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 198967296. Throughput: 0: 1845.7, 1: 1857.9. Samples: 49745350. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:12,015][52710] Avg episode reward: [(0, '33.600'), (1, '36.150')] [2023-10-08 11:37:12,392][53852] Updated weights for policy 0, policy_version 97380 (0.0007) [2023-10-08 11:37:12,759][53852] Updated weights for policy 0, policy_version 97390 (0.0007) [2023-10-08 11:37:13,126][53852] Updated weights for policy 0, policy_version 97400 (0.0007) [2023-10-08 11:37:14,675][53885] Updated weights for policy 1, policy_version 96932 (0.0009) [2023-10-08 11:37:15,053][53885] Updated weights for policy 1, policy_version 96942 (0.0008) [2023-10-08 11:37:15,410][53885] Updated weights for policy 1, policy_version 96952 (0.0010) [2023-10-08 11:37:16,665][53852] Updated weights for policy 0, policy_version 97410 (0.0008) [2023-10-08 11:37:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199032832. Throughput: 0: 1853.4, 1: 1837.0. Samples: 49767436. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:17,016][52710] Avg episode reward: [(0, '36.710'), (1, '38.530')] [2023-10-08 11:37:17,036][53852] Updated weights for policy 0, policy_version 97420 (0.0008) [2023-10-08 11:37:17,401][53852] Updated weights for policy 0, policy_version 97430 (0.0007) [2023-10-08 11:37:17,762][53852] Updated weights for policy 0, policy_version 97440 (0.0009) [2023-10-08 11:37:18,884][53885] Updated weights for policy 1, policy_version 96962 (0.0010) [2023-10-08 11:37:19,253][53885] Updated weights for policy 1, policy_version 96972 (0.0008) [2023-10-08 11:37:19,616][53885] Updated weights for policy 1, policy_version 96982 (0.0008) [2023-10-08 11:37:19,987][53885] Updated weights for policy 1, policy_version 96992 (0.0007) [2023-10-08 11:37:21,511][53852] Updated weights for policy 0, policy_version 97450 (0.0009) [2023-10-08 11:37:21,882][53852] Updated weights for policy 0, policy_version 97460 (0.0008) [2023-10-08 11:37:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199098368. Throughput: 0: 1831.1, 1: 1859.4. Samples: 49789842. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:22,015][52710] Avg episode reward: [(0, '32.790'), (1, '37.550')] [2023-10-08 11:37:22,259][53852] Updated weights for policy 0, policy_version 97470 (0.0008) [2023-10-08 11:37:23,699][53885] Updated weights for policy 1, policy_version 97002 (0.0007) [2023-10-08 11:37:24,063][53885] Updated weights for policy 1, policy_version 97012 (0.0008) [2023-10-08 11:37:24,443][53885] Updated weights for policy 1, policy_version 97022 (0.0008) [2023-10-08 11:37:25,737][53852] Updated weights for policy 0, policy_version 97480 (0.0008) [2023-10-08 11:37:26,102][53852] Updated weights for policy 0, policy_version 97490 (0.0010) [2023-10-08 11:37:26,473][53852] Updated weights for policy 0, policy_version 97500 (0.0010) [2023-10-08 11:37:27,015][52710] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 199196672. Throughput: 0: 1845.3, 1: 1830.7. Samples: 49800370. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:27,016][52710] Avg episode reward: [(0, '33.640'), (1, '37.160')] [2023-10-08 11:37:28,059][53885] Updated weights for policy 1, policy_version 97032 (0.0008) [2023-10-08 11:37:28,417][53885] Updated weights for policy 1, policy_version 97042 (0.0009) [2023-10-08 11:37:28,781][53885] Updated weights for policy 1, policy_version 97052 (0.0010) [2023-10-08 11:37:29,981][53852] Updated weights for policy 0, policy_version 97510 (0.0008) [2023-10-08 11:37:30,348][53852] Updated weights for policy 0, policy_version 97520 (0.0008) [2023-10-08 11:37:30,727][53852] Updated weights for policy 0, policy_version 97530 (0.0008) [2023-10-08 11:37:32,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 199262208. Throughput: 0: 1823.9, 1: 1847.1. Samples: 49822134. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:32,016][52710] Avg episode reward: [(0, '34.850'), (1, '34.720')] [2023-10-08 11:37:32,488][53885] Updated weights for policy 1, policy_version 97062 (0.0008) [2023-10-08 11:37:32,854][53885] Updated weights for policy 1, policy_version 97072 (0.0007) [2023-10-08 11:37:33,221][53885] Updated weights for policy 1, policy_version 97082 (0.0007) [2023-10-08 11:37:34,418][53852] Updated weights for policy 0, policy_version 97540 (0.0008) [2023-10-08 11:37:34,786][53852] Updated weights for policy 0, policy_version 97550 (0.0007) [2023-10-08 11:37:35,160][53852] Updated weights for policy 0, policy_version 97560 (0.0008) [2023-10-08 11:37:36,908][53885] Updated weights for policy 1, policy_version 97092 (0.0008) [2023-10-08 11:37:37,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 199327744. Throughput: 0: 1840.7, 1: 1845.0. Samples: 49844854. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:37,015][52710] Avg episode reward: [(0, '34.860'), (1, '35.680')] [2023-10-08 11:37:37,293][53885] Updated weights for policy 1, policy_version 97102 (0.0007) [2023-10-08 11:37:37,667][53885] Updated weights for policy 1, policy_version 97112 (0.0008) [2023-10-08 11:37:38,749][53852] Updated weights for policy 0, policy_version 97570 (0.0008) [2023-10-08 11:37:39,119][53852] Updated weights for policy 0, policy_version 97580 (0.0008) [2023-10-08 11:37:39,483][53852] Updated weights for policy 0, policy_version 97590 (0.0008) [2023-10-08 11:37:39,863][53852] Updated weights for policy 0, policy_version 97600 (0.0009) [2023-10-08 11:37:41,333][53885] Updated weights for policy 1, policy_version 97122 (0.0008) [2023-10-08 11:37:41,694][53885] Updated weights for policy 1, policy_version 97132 (0.0011) [2023-10-08 11:37:42,015][52710] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199393280. Throughput: 0: 1829.9, 1: 1841.0. Samples: 49855404. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:42,016][52710] Avg episode reward: [(0, '36.650'), (1, '33.920')] [2023-10-08 11:37:42,073][53885] Updated weights for policy 1, policy_version 97142 (0.0008) [2023-10-08 11:37:42,437][53885] Updated weights for policy 1, policy_version 97152 (0.0007) [2023-10-08 11:37:43,560][53852] Updated weights for policy 0, policy_version 97610 (0.0008) [2023-10-08 11:37:43,933][53852] Updated weights for policy 0, policy_version 97620 (0.0010) [2023-10-08 11:37:44,307][53852] Updated weights for policy 0, policy_version 97630 (0.0010) [2023-10-08 11:37:46,154][53885] Updated weights for policy 1, policy_version 97162 (0.0010) [2023-10-08 11:37:46,535][53885] Updated weights for policy 1, policy_version 97172 (0.0010) [2023-10-08 11:37:46,901][53885] Updated weights for policy 1, policy_version 97182 (0.0007) [2023-10-08 11:37:47,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199491584. Throughput: 0: 1848.0, 1: 1837.2. Samples: 49877938. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:47,016][52710] Avg episode reward: [(0, '35.060'), (1, '38.290')] [2023-10-08 11:37:48,018][53852] Updated weights for policy 0, policy_version 97640 (0.0009) [2023-10-08 11:37:48,392][53852] Updated weights for policy 0, policy_version 97650 (0.0010) [2023-10-08 11:37:48,753][53852] Updated weights for policy 0, policy_version 97660 (0.0010) [2023-10-08 11:37:50,565][53885] Updated weights for policy 1, policy_version 97192 (0.0007) [2023-10-08 11:37:50,926][53885] Updated weights for policy 1, policy_version 97202 (0.0009) [2023-10-08 11:37:51,308][53885] Updated weights for policy 1, policy_version 97212 (0.0010) [2023-10-08 11:37:52,015][52710] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199557120. Throughput: 0: 1851.9, 1: 1829.8. Samples: 49899708. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:52,016][52710] Avg episode reward: [(0, '37.790'), (1, '36.470')] [2023-10-08 11:37:52,381][53852] Updated weights for policy 0, policy_version 97670 (0.0010) [2023-10-08 11:37:52,756][53852] Updated weights for policy 0, policy_version 97680 (0.0010) [2023-10-08 11:37:53,125][53852] Updated weights for policy 0, policy_version 97690 (0.0009) [2023-10-08 11:37:54,924][53885] Updated weights for policy 1, policy_version 97222 (0.0008) [2023-10-08 11:37:55,290][53885] Updated weights for policy 1, policy_version 97232 (0.0007) [2023-10-08 11:37:55,658][53885] Updated weights for policy 1, policy_version 97242 (0.0009) [2023-10-08 11:37:56,743][53852] Updated weights for policy 0, policy_version 97700 (0.0010) [2023-10-08 11:37:57,015][52710] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199622656. Throughput: 0: 1854.0, 1: 1835.7. Samples: 49911386. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:37:57,016][52710] Avg episode reward: [(0, '36.970'), (1, '41.050')] [2023-10-08 11:37:57,113][53852] Updated weights for policy 0, policy_version 97710 (0.0007) [2023-10-08 11:37:57,479][53852] Updated weights for policy 0, policy_version 97720 (0.0007) [2023-10-08 11:37:59,217][53885] Updated weights for policy 1, policy_version 97252 (0.0008) [2023-10-08 11:37:59,593][53885] Updated weights for policy 1, policy_version 97262 (0.0007) [2023-10-08 11:37:59,956][53885] Updated weights for policy 1, policy_version 97272 (0.0009) [2023-10-08 11:38:01,017][53852] Updated weights for policy 0, policy_version 97730 (0.0008) [2023-10-08 11:38:01,378][53852] Updated weights for policy 0, policy_version 97740 (0.0007) [2023-10-08 11:38:01,747][53852] Updated weights for policy 0, policy_version 97750 (0.0010) [2023-10-08 11:38:02,015][52710] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199688192. Throughput: 0: 1849.8, 1: 1837.2. Samples: 49933350. Policy #0 lag: (min: 1.0, avg: 8.0, max: 33.0) [2023-10-08 11:38:02,016][52710] Avg episode reward: [(0, '34.540'), (1, '44.650')] [2023-10-08 11:38:02,017][53594] Saving new best policy, reward=44.650! [2023-10-08 11:38:02,122][53852] Updated weights for policy 0, policy_version 97760 (0.0011) [2023-10-08 11:38:03,723][53885] Updated weights for policy 1, policy_version 97282 (0.0009) [2023-10-08 11:38:04,080][53885] Updated weights for policy 1, policy_version 97292 (0.0007) [2023-10-08 11:38:04,446][53885] Updated weights for policy 1, policy_version 97302 (0.0009) [2023-10-08 11:38:04,820][53885] Updated weights for policy 1, policy_version 97312 (0.0009) [2023-10-08 11:38:05,725][53852] Updated weights for policy 0, policy_version 97770 (0.0009) [2023-10-08 11:38:06,103][53852] Updated weights for policy 0, policy_version 97780 (0.0007) [2023-10-08 11:38:06,470][53852] Updated weights for policy 0, policy_version 97790 (0.0008) [2023-10-08 11:38:07,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 199786496. Throughput: 0: 1834.7, 1: 1833.3. Samples: 49954904. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:07,016][52710] Avg episode reward: [(0, '36.890'), (1, '37.240')] [2023-10-08 11:38:07,029][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000097312_99647488.pth... [2023-10-08 11:38:07,029][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000097792_100139008.pth... [2023-10-08 11:38:07,067][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000095616_97910784.pth [2023-10-08 11:38:07,071][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000096064_98369536.pth [2023-10-08 11:38:08,373][53885] Updated weights for policy 1, policy_version 97322 (0.0010) [2023-10-08 11:38:08,744][53885] Updated weights for policy 1, policy_version 97332 (0.0009) [2023-10-08 11:38:09,104][53885] Updated weights for policy 1, policy_version 97342 (0.0008) [2023-10-08 11:38:10,101][53852] Updated weights for policy 0, policy_version 97800 (0.0008) [2023-10-08 11:38:10,467][53852] Updated weights for policy 0, policy_version 97810 (0.0009) [2023-10-08 11:38:10,837][53852] Updated weights for policy 0, policy_version 97820 (0.0009) [2023-10-08 11:38:12,015][52710] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 199852032. Throughput: 0: 1852.7, 1: 1837.4. Samples: 49966424. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:12,016][52710] Avg episode reward: [(0, '35.330'), (1, '36.230')] [2023-10-08 11:38:12,704][53885] Updated weights for policy 1, policy_version 97352 (0.0007) [2023-10-08 11:38:13,072][53885] Updated weights for policy 1, policy_version 97362 (0.0009) [2023-10-08 11:38:13,439][53885] Updated weights for policy 1, policy_version 97372 (0.0009) [2023-10-08 11:38:14,428][53852] Updated weights for policy 0, policy_version 97830 (0.0011) [2023-10-08 11:38:14,791][53852] Updated weights for policy 0, policy_version 97840 (0.0010) [2023-10-08 11:38:15,157][53852] Updated weights for policy 0, policy_version 97850 (0.0009) [2023-10-08 11:38:16,986][53885] Updated weights for policy 1, policy_version 97382 (0.0010) [2023-10-08 11:38:17,015][52710] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 199917568. Throughput: 0: 1839.5, 1: 1841.9. Samples: 49987796. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:17,016][52710] Avg episode reward: [(0, '36.000'), (1, '35.980')] [2023-10-08 11:38:17,358][53885] Updated weights for policy 1, policy_version 97392 (0.0007) [2023-10-08 11:38:17,720][53885] Updated weights for policy 1, policy_version 97402 (0.0008) [2023-10-08 11:38:18,755][53852] Updated weights for policy 0, policy_version 97860 (0.0010) [2023-10-08 11:38:19,122][53852] Updated weights for policy 0, policy_version 97870 (0.0009) [2023-10-08 11:38:19,493][53852] Updated weights for policy 0, policy_version 97880 (0.0010) [2023-10-08 11:38:21,371][53885] Updated weights for policy 1, policy_version 97412 (0.0009) [2023-10-08 11:38:21,729][53885] Updated weights for policy 1, policy_version 97422 (0.0009) [2023-10-08 11:38:22,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 199983104. Throughput: 0: 1848.8, 1: 1831.8. Samples: 50010484. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:22,016][52710] Avg episode reward: [(0, '35.950'), (1, '38.530')] [2023-10-08 11:38:22,110][53885] Updated weights for policy 1, policy_version 97432 (0.0008) [2023-10-08 11:38:23,273][53852] Updated weights for policy 0, policy_version 97890 (0.0010) [2023-10-08 11:38:23,641][53852] Updated weights for policy 0, policy_version 97900 (0.0008) [2023-10-08 11:38:24,010][53852] Updated weights for policy 0, policy_version 97910 (0.0009) [2023-10-08 11:38:24,378][53852] Updated weights for policy 0, policy_version 97920 (0.0007) [2023-10-08 11:38:25,854][53885] Updated weights for policy 1, policy_version 97442 (0.0009) [2023-10-08 11:38:26,234][53885] Updated weights for policy 1, policy_version 97452 (0.0008) [2023-10-08 11:38:26,604][53885] Updated weights for policy 1, policy_version 97462 (0.0007) [2023-10-08 11:38:26,968][53885] Updated weights for policy 1, policy_version 97472 (0.0008) [2023-10-08 11:38:27,015][52710] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 200081408. Throughput: 0: 1832.1, 1: 1843.8. Samples: 50020818. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:27,015][52710] Avg episode reward: [(0, '39.050'), (1, '33.930')] [2023-10-08 11:38:28,171][53852] Updated weights for policy 0, policy_version 97930 (0.0010) [2023-10-08 11:38:28,551][53852] Updated weights for policy 0, policy_version 97940 (0.0008) [2023-10-08 11:38:28,910][53852] Updated weights for policy 0, policy_version 97950 (0.0007) [2023-10-08 11:38:30,511][53885] Updated weights for policy 1, policy_version 97482 (0.0007) [2023-10-08 11:38:30,877][53885] Updated weights for policy 1, policy_version 97492 (0.0008) [2023-10-08 11:38:31,241][53885] Updated weights for policy 1, policy_version 97502 (0.0012) [2023-10-08 11:38:32,015][52710] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 200146944. Throughput: 0: 1840.0, 1: 1836.6. Samples: 50043384. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:32,016][52710] Avg episode reward: [(0, '37.400'), (1, '34.700')] [2023-10-08 11:38:32,570][53852] Updated weights for policy 0, policy_version 97960 (0.0009) [2023-10-08 11:38:32,935][53852] Updated weights for policy 0, policy_version 97970 (0.0007) [2023-10-08 11:38:33,299][53852] Updated weights for policy 0, policy_version 97980 (0.0008) [2023-10-08 11:38:34,843][53885] Updated weights for policy 1, policy_version 97512 (0.0007) [2023-10-08 11:38:35,217][53885] Updated weights for policy 1, policy_version 97522 (0.0007) [2023-10-08 11:38:35,577][53885] Updated weights for policy 1, policy_version 97532 (0.0008) [2023-10-08 11:38:36,991][53852] Updated weights for policy 0, policy_version 97990 (0.0007) [2023-10-08 11:38:37,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 200212480. Throughput: 0: 1831.3, 1: 1848.6. Samples: 50065304. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:37,015][52710] Avg episode reward: [(0, '36.040'), (1, '38.940')] [2023-10-08 11:38:37,356][53852] Updated weights for policy 0, policy_version 98000 (0.0008) [2023-10-08 11:38:37,731][53852] Updated weights for policy 0, policy_version 98010 (0.0010) [2023-10-08 11:38:39,430][53885] Updated weights for policy 1, policy_version 97542 (0.0008) [2023-10-08 11:38:39,795][53885] Updated weights for policy 1, policy_version 97552 (0.0009) [2023-10-08 11:38:40,169][53885] Updated weights for policy 1, policy_version 97562 (0.0008) [2023-10-08 11:38:41,398][53852] Updated weights for policy 0, policy_version 98020 (0.0009) [2023-10-08 11:38:41,773][53852] Updated weights for policy 0, policy_version 98030 (0.0007) [2023-10-08 11:38:42,015][52710] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 200278016. Throughput: 0: 1830.1, 1: 1829.7. Samples: 50076078. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:42,015][52710] Avg episode reward: [(0, '37.860'), (1, '39.930')] [2023-10-08 11:38:42,131][53852] Updated weights for policy 0, policy_version 98040 (0.0007) [2023-10-08 11:38:44,013][53885] Updated weights for policy 1, policy_version 97572 (0.0009) [2023-10-08 11:38:44,371][53885] Updated weights for policy 1, policy_version 97582 (0.0008) [2023-10-08 11:38:44,745][53885] Updated weights for policy 1, policy_version 97592 (0.0008) [2023-10-08 11:38:45,869][53852] Updated weights for policy 0, policy_version 98050 (0.0007) [2023-10-08 11:38:46,242][53852] Updated weights for policy 0, policy_version 98060 (0.0008) [2023-10-08 11:38:46,619][53852] Updated weights for policy 0, policy_version 98070 (0.0008) [2023-10-08 11:38:46,986][53852] Updated weights for policy 0, policy_version 98080 (0.0007) [2023-10-08 11:38:47,015][52710] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14773.4). Total num frames: 200376320. Throughput: 0: 1821.1, 1: 1830.1. Samples: 50097656. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:47,016][52710] Avg episode reward: [(0, '37.180'), (1, '34.840')] [2023-10-08 11:38:48,438][53885] Updated weights for policy 1, policy_version 97602 (0.0008) [2023-10-08 11:38:48,814][53885] Updated weights for policy 1, policy_version 97612 (0.0008) [2023-10-08 11:38:49,183][53885] Updated weights for policy 1, policy_version 97622 (0.0007) [2023-10-08 11:38:49,545][53885] Updated weights for policy 1, policy_version 97632 (0.0008) [2023-10-08 11:38:50,723][53852] Updated weights for policy 0, policy_version 98090 (0.0007) [2023-10-08 11:38:51,101][53852] Updated weights for policy 0, policy_version 98100 (0.0008) [2023-10-08 11:38:51,460][53852] Updated weights for policy 0, policy_version 98110 (0.0009) [2023-10-08 11:38:52,015][52710] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 200441856. Throughput: 0: 1815.6, 1: 1838.2. Samples: 50119324. Policy #0 lag: (min: 13.0, avg: 21.0, max: 45.0) [2023-10-08 11:38:52,016][52710] Avg episode reward: [(0, '34.100'), (1, '37.190')] [2023-10-08 11:38:53,094][53885] Updated weights for policy 1, policy_version 97642 (0.0010) [2023-10-08 11:38:53,476][53885] Updated weights for policy 1, policy_version 97652 (0.0011) [2023-10-08 11:38:53,839][53885] Updated weights for policy 1, policy_version 97662 (0.0011) [2023-10-08 11:38:54,991][53852] Updated weights for policy 0, policy_version 98120 (0.0007) [2023-10-08 11:38:55,370][53852] Updated weights for policy 0, policy_version 98130 (0.0009) [2023-10-08 11:38:55,738][53852] Updated weights for policy 0, policy_version 98140 (0.0007) [2023-10-08 11:38:55,878][53898] Stopping RolloutWorker_w11... [2023-10-08 11:38:55,878][53898] Loop rollout_proc11_evt_loop terminating... [2023-10-08 11:38:55,878][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000098144_100499456.pth... [2023-10-08 11:38:55,878][52710] Component RolloutWorker_w11 stopped! [2023-10-08 11:38:55,878][53594] Stopping Batcher_1... [2023-10-08 11:38:55,879][53895] Stopping RolloutWorker_w7... [2023-10-08 11:38:55,879][53594] Loop batcher_evt_loop terminating... [2023-10-08 11:38:55,879][52710] Component Batcher_1 stopped! [2023-10-08 11:38:55,879][53895] Loop rollout_proc7_evt_loop terminating... [2023-10-08 11:38:55,879][52710] Component Batcher_0 stopped! [2023-10-08 11:38:55,880][53888] Stopping RolloutWorker_w2... [2023-10-08 11:38:55,879][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000097664_100007936.pth... [2023-10-08 11:38:55,880][53886] Stopping RolloutWorker_w0... [2023-10-08 11:38:55,880][52710] Component RolloutWorker_w7 stopped! [2023-10-08 11:38:55,880][53888] Loop rollout_proc2_evt_loop terminating... [2023-10-08 11:38:55,880][53889] Stopping RolloutWorker_w1... [2023-10-08 11:38:55,880][53897] Stopping RolloutWorker_w9... [2023-10-08 11:38:55,880][53900] Stopping RolloutWorker_w12... [2023-10-08 11:38:55,880][53886] Loop rollout_proc0_evt_loop terminating... [2023-10-08 11:38:55,880][52710] Component RolloutWorker_w2 stopped! [2023-10-08 11:38:55,880][53889] Loop rollout_proc1_evt_loop terminating... [2023-10-08 11:38:55,880][53897] Loop rollout_proc9_evt_loop terminating... [2023-10-08 11:38:55,880][53900] Loop rollout_proc12_evt_loop terminating... [2023-10-08 11:38:55,881][52710] Component RolloutWorker_w0 stopped! [2023-10-08 11:38:55,881][53901] Stopping RolloutWorker_w13... [2023-10-08 11:38:55,881][52710] Component RolloutWorker_w1 stopped! [2023-10-08 11:38:55,881][53901] Loop rollout_proc13_evt_loop terminating... [2023-10-08 11:38:55,881][52710] Component RolloutWorker_w9 stopped! [2023-10-08 11:38:55,882][52710] Component RolloutWorker_w12 stopped! [2023-10-08 11:38:55,881][54537] Stopping RolloutWorker_w15... [2023-10-08 11:38:55,882][53890] Stopping RolloutWorker_w3... [2023-10-08 11:38:55,882][53890] Loop rollout_proc3_evt_loop terminating... [2023-10-08 11:38:55,882][54537] Loop rollout_proc15_evt_loop terminating... [2023-10-08 11:38:55,882][52710] Component RolloutWorker_w13 stopped! [2023-10-08 11:38:55,882][53893] Stopping RolloutWorker_w5... [2023-10-08 11:38:55,882][52710] Component RolloutWorker_w15 stopped! [2023-10-08 11:38:55,883][53893] Loop rollout_proc5_evt_loop terminating... [2023-10-08 11:38:55,883][52710] Component RolloutWorker_w3 stopped! [2023-10-08 11:38:55,883][53894] Stopping RolloutWorker_w6... [2023-10-08 11:38:55,883][54536] Stopping RolloutWorker_w14... [2023-10-08 11:38:55,883][52710] Component RolloutWorker_w5 stopped! [2023-10-08 11:38:55,883][54536] Loop rollout_proc14_evt_loop terminating... [2023-10-08 11:38:55,883][53894] Loop rollout_proc6_evt_loop terminating... [2023-10-08 11:38:55,883][53899] Stopping RolloutWorker_w10... [2023-10-08 11:38:55,883][52710] Component RolloutWorker_w6 stopped! [2023-10-08 11:38:55,884][53899] Loop rollout_proc10_evt_loop terminating... [2023-10-08 11:38:55,884][52710] Component RolloutWorker_w14 stopped! [2023-10-08 11:38:55,884][53896] Stopping RolloutWorker_w8... [2023-10-08 11:38:55,884][52710] Component RolloutWorker_w10 stopped! [2023-10-08 11:38:55,884][53896] Loop rollout_proc8_evt_loop terminating... [2023-10-08 11:38:55,884][52710] Component RolloutWorker_w8 stopped! [2023-10-08 11:38:55,884][53891] Stopping RolloutWorker_w4... [2023-10-08 11:38:55,884][52710] Component RolloutWorker_w4 stopped! [2023-10-08 11:38:55,885][53891] Loop rollout_proc4_evt_loop terminating... [2023-10-08 11:38:55,878][53500] Stopping Batcher_0... [2023-10-08 11:38:55,908][53885] Weights refcount: 2 0 [2023-10-08 11:38:55,910][53885] Stopping InferenceWorker_p1-w0... [2023-10-08 11:38:55,910][52710] Component InferenceWorker_p1-w0 stopped! [2023-10-08 11:38:55,910][53885] Loop inference_proc1-0_evt_loop terminating... [2023-10-08 11:38:55,902][53500] Loop batcher_evt_loop terminating... [2023-10-08 11:38:55,912][53500] Removing ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000096928_99254272.pth [2023-10-08 11:38:55,917][53500] Saving ./train_atari/atari_asterix_APPO/checkpoint_p0/checkpoint_000098144_100499456.pth... [2023-10-08 11:38:55,919][53852] Weights refcount: 2 0 [2023-10-08 11:38:55,921][53852] Stopping InferenceWorker_p0-w0... [2023-10-08 11:38:55,921][52710] Component InferenceWorker_p0-w0 stopped! [2023-10-08 11:38:55,921][53852] Loop inference_proc0-0_evt_loop terminating... [2023-10-08 11:38:55,933][53594] Removing ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000096448_98762752.pth [2023-10-08 11:38:55,939][53594] Saving ./train_atari/atari_asterix_APPO/checkpoint_p1/checkpoint_000097664_100007936.pth... [2023-10-08 11:38:55,957][53500] Stopping LearnerWorker_p0... [2023-10-08 11:38:55,957][53500] Loop learner_proc0_evt_loop terminating... [2023-10-08 11:38:55,957][52710] Component LearnerWorker_p0 stopped! [2023-10-08 11:38:55,992][53594] Stopping LearnerWorker_p1... [2023-10-08 11:38:55,992][52710] Component LearnerWorker_p1 stopped! [2023-10-08 11:38:55,992][53594] Loop learner_proc1_evt_loop terminating... [2023-10-08 11:38:55,993][52710] Waiting for process learner_proc0 to stop... [2023-10-08 11:38:56,730][52710] Waiting for process learner_proc1 to stop... [2023-10-08 11:38:56,895][52710] Waiting for process inference_proc0-0 to join... [2023-10-08 11:38:56,896][52710] Waiting for process inference_proc1-0 to join... [2023-10-08 11:38:56,896][52710] Waiting for process rollout_proc0 to join... [2023-10-08 11:38:56,897][52710] Waiting for process rollout_proc1 to join... [2023-10-08 11:38:56,898][52710] Waiting for process rollout_proc2 to join... [2023-10-08 11:38:56,898][52710] Waiting for process rollout_proc3 to join... [2023-10-08 11:38:56,899][52710] Waiting for process rollout_proc4 to join... [2023-10-08 11:38:56,899][52710] Waiting for process rollout_proc5 to join... [2023-10-08 11:38:56,900][52710] Waiting for process rollout_proc6 to join... [2023-10-08 11:38:56,901][52710] Waiting for process rollout_proc7 to join... [2023-10-08 11:38:56,901][52710] Waiting for process rollout_proc8 to join... [2023-10-08 11:38:56,902][52710] Waiting for process rollout_proc9 to join... [2023-10-08 11:38:56,903][52710] Waiting for process rollout_proc10 to join... [2023-10-08 11:38:56,903][52710] Waiting for process rollout_proc11 to join... [2023-10-08 11:38:56,903][52710] Waiting for process rollout_proc12 to join... [2023-10-08 11:38:56,904][52710] Waiting for process rollout_proc13 to join... [2023-10-08 11:38:56,904][52710] Waiting for process rollout_proc14 to join... [2023-10-08 11:38:56,904][52710] Waiting for process rollout_proc15 to join... [2023-10-08 11:38:56,905][52710] Batcher 0 profile tree view: batching: 171.8990, releasing_batches: 0.0905 [2023-10-08 11:38:56,905][52710] Batcher 1 profile tree view: batching: 171.9787, releasing_batches: 0.0921 [2023-10-08 11:38:56,905][52710] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0002 wait_policy_total: 1701.7078 update_model: 199.6811 weight_update: 0.0009 one_step: 0.0053 handle_policy_step: 11146.3952 deserialize: 62.5900, stack: 191.1937, obs_to_device_normalize: 2473.5888, forward: 5037.3043, prepare_outputs: 2453.0399, send_messages: 454.6092 [2023-10-08 11:38:56,906][52710] InferenceWorker_p1-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 1760.9635 update_model: 195.8664 weight_update: 0.0010 one_step: 0.0025 handle_policy_step: 11095.4281 deserialize: 63.0523, stack: 190.6251, obs_to_device_normalize: 2469.8648, forward: 5007.2355, prepare_outputs: 2430.5146, send_messages: 454.0096 [2023-10-08 11:38:56,906][52710] Learner 0 profile tree view: misc: 0.0181, prepare_batch: 269.5858 train: 3644.0779 epoch_init: 0.1910, minibatch_init: 12.9688, losses_postprocess: 897.0256, kl_divergence: 31.3988, update: 390.2990, after_optimizer: 2128.9486 calculate_losses: 166.4176 losses_init: 0.3919, forward_head: 55.8778, bptt_initial: 1.4228, bptt: 1.9933, tail: 38.3055, advantages_returns: 10.9857, losses: 44.0117 [2023-10-08 11:38:56,906][52710] Learner 1 profile tree view: misc: 0.0184, prepare_batch: 268.5345 train: 3606.0910 epoch_init: 0.1911, minibatch_init: 12.9866, losses_postprocess: 891.1729, kl_divergence: 30.7918, update: 389.1824, after_optimizer: 2098.2499 calculate_losses: 166.3702 losses_init: 0.4711, forward_head: 55.6402, bptt_initial: 1.4338, bptt: 1.9823, tail: 38.1973, advantages_returns: 11.0857, losses: 43.5756 [2023-10-08 11:38:56,906][52710] RolloutWorker_w0 profile tree view: wait_for_trajectories: 1.2568, enqueue_policy_requests: 416.8141, process_policy_outputs: 192.3271, env_step: 6353.0684, finalize_trajectories: 3.5818, complete_rollouts: 3.0126 post_env_step: 380.9417 process_env_step: 85.6083 [2023-10-08 11:38:56,907][52710] RolloutWorker_w15 profile tree view: wait_for_trajectories: 1.2368, enqueue_policy_requests: 403.6991, process_policy_outputs: 192.3459, env_step: 6361.8468, finalize_trajectories: 3.4442, complete_rollouts: 2.9816 post_env_step: 374.3615 process_env_step: 84.9863 [2023-10-08 11:38:56,907][52710] Loop Runner_EvtLoop terminating... [2023-10-08 11:38:56,908][52710] Runner profile tree view: main_loop: 13725.2745 [2023-10-08 11:38:56,908][52710] Collected {0: 100499456, 1: 100007936}, FPS: 14608.6