[2023-02-26 10:10:18,458][00304] Saving configuration to /content/train_dir/default_experiment/config.json... [2023-02-26 10:10:18,462][00304] Rollout worker 0 uses device cpu [2023-02-26 10:10:18,464][00304] Rollout worker 1 uses device cpu [2023-02-26 10:10:18,467][00304] Rollout worker 2 uses device cpu [2023-02-26 10:10:18,468][00304] Rollout worker 3 uses device cpu [2023-02-26 10:10:18,469][00304] Rollout worker 4 uses device cpu [2023-02-26 10:10:18,472][00304] Rollout worker 5 uses device cpu [2023-02-26 10:10:18,473][00304] Rollout worker 6 uses device cpu [2023-02-26 10:10:18,475][00304] Rollout worker 7 uses device cpu [2023-02-26 10:10:18,704][00304] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-26 10:10:18,707][00304] InferenceWorker_p0-w0: min num requests: 2 [2023-02-26 10:10:18,751][00304] Starting all processes... [2023-02-26 10:10:18,754][00304] Starting process learner_proc0 [2023-02-26 10:10:18,840][00304] Starting all processes... [2023-02-26 10:10:18,854][00304] Starting process inference_proc0-0 [2023-02-26 10:10:18,870][00304] Starting process rollout_proc0 [2023-02-26 10:10:18,871][00304] Starting process rollout_proc1 [2023-02-26 10:10:18,871][00304] Starting process rollout_proc2 [2023-02-26 10:10:18,871][00304] Starting process rollout_proc3 [2023-02-26 10:10:18,871][00304] Starting process rollout_proc4 [2023-02-26 10:10:18,871][00304] Starting process rollout_proc5 [2023-02-26 10:10:18,871][00304] Starting process rollout_proc6 [2023-02-26 10:10:18,871][00304] Starting process rollout_proc7 [2023-02-26 10:10:32,790][10798] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-26 10:10:32,792][10798] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-02-26 10:10:33,007][10818] Worker 5 uses CPU cores [1] [2023-02-26 10:10:33,023][10813] Worker 1 uses CPU cores [1] [2023-02-26 10:10:33,128][10820] Worker 7 uses CPU cores [1] [2023-02-26 10:10:33,180][10814] Worker 2 uses CPU cores [0] [2023-02-26 10:10:33,339][10817] Worker 4 uses CPU cores [0] [2023-02-26 10:10:33,362][10816] Worker 3 uses CPU cores [1] [2023-02-26 10:10:33,506][10815] Worker 0 uses CPU cores [0] [2023-02-26 10:10:33,647][10811] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-26 10:10:33,652][10811] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-02-26 10:10:33,721][10819] Worker 6 uses CPU cores [0] [2023-02-26 10:10:33,860][10811] Num visible devices: 1 [2023-02-26 10:10:33,865][10798] Num visible devices: 1 [2023-02-26 10:10:33,866][10798] Starting seed is not provided [2023-02-26 10:10:33,866][10798] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-26 10:10:33,867][10798] Initializing actor-critic model on device cuda:0 [2023-02-26 10:10:33,867][10798] RunningMeanStd input shape: (3, 72, 128) [2023-02-26 10:10:33,869][10798] RunningMeanStd input shape: (1,) [2023-02-26 10:10:33,896][10798] ConvEncoder: input_channels=3 [2023-02-26 10:10:34,468][10798] Conv encoder output size: 512 [2023-02-26 10:10:34,469][10798] Policy head output size: 512 [2023-02-26 10:10:34,548][10798] Created Actor Critic model with architecture: [2023-02-26 10:10:34,549][10798] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2023-02-26 10:10:38,695][00304] Heartbeat connected on Batcher_0 [2023-02-26 10:10:38,705][00304] Heartbeat connected on InferenceWorker_p0-w0 [2023-02-26 10:10:38,718][00304] Heartbeat connected on RolloutWorker_w0 [2023-02-26 10:10:38,722][00304] Heartbeat connected on RolloutWorker_w1 [2023-02-26 10:10:38,728][00304] Heartbeat connected on RolloutWorker_w2 [2023-02-26 10:10:38,733][00304] Heartbeat connected on RolloutWorker_w3 [2023-02-26 10:10:38,737][00304] Heartbeat connected on RolloutWorker_w4 [2023-02-26 10:10:38,743][00304] Heartbeat connected on RolloutWorker_w5 [2023-02-26 10:10:38,748][00304] Heartbeat connected on RolloutWorker_w6 [2023-02-26 10:10:38,751][00304] Heartbeat connected on RolloutWorker_w7 [2023-02-26 10:10:42,390][10798] Using optimizer [2023-02-26 10:10:42,391][10798] No checkpoints found [2023-02-26 10:10:42,391][10798] Did not load from checkpoint, starting from scratch! [2023-02-26 10:10:42,391][10798] Initialized policy 0 weights for model version 0 [2023-02-26 10:10:42,395][10798] LearnerWorker_p0 finished initialization! [2023-02-26 10:10:42,396][00304] Heartbeat connected on LearnerWorker_p0 [2023-02-26 10:10:42,403][10798] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-26 10:10:42,597][10811] RunningMeanStd input shape: (3, 72, 128) [2023-02-26 10:10:42,598][10811] RunningMeanStd input shape: (1,) [2023-02-26 10:10:42,611][10811] ConvEncoder: input_channels=3 [2023-02-26 10:10:42,716][10811] Conv encoder output size: 512 [2023-02-26 10:10:42,716][10811] Policy head output size: 512 [2023-02-26 10:10:43,705][00304] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-26 10:10:45,011][00304] Inference worker 0-0 is ready! [2023-02-26 10:10:45,012][00304] All inference workers are ready! Signal rollout workers to start! [2023-02-26 10:10:45,105][10814] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:10:45,108][10819] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:10:45,141][10815] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:10:45,147][10817] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:10:45,168][10816] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:10:45,166][10820] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:10:45,182][10818] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:10:45,203][10813] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:10:46,013][10820] Decorrelating experience for 0 frames... [2023-02-26 10:10:46,011][10813] Decorrelating experience for 0 frames... [2023-02-26 10:10:46,256][10819] Decorrelating experience for 0 frames... [2023-02-26 10:10:46,260][10817] Decorrelating experience for 0 frames... [2023-02-26 10:10:46,266][10814] Decorrelating experience for 0 frames... [2023-02-26 10:10:46,651][10819] Decorrelating experience for 32 frames... [2023-02-26 10:10:47,006][10818] Decorrelating experience for 0 frames... [2023-02-26 10:10:47,077][10813] Decorrelating experience for 32 frames... [2023-02-26 10:10:47,165][10819] Decorrelating experience for 64 frames... [2023-02-26 10:10:47,643][10820] Decorrelating experience for 32 frames... [2023-02-26 10:10:47,674][10816] Decorrelating experience for 0 frames... [2023-02-26 10:10:48,367][10817] Decorrelating experience for 32 frames... [2023-02-26 10:10:48,553][10814] Decorrelating experience for 32 frames... [2023-02-26 10:10:48,705][00304] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-26 10:10:48,822][10819] Decorrelating experience for 96 frames... [2023-02-26 10:10:48,915][10818] Decorrelating experience for 32 frames... [2023-02-26 10:10:49,361][10815] Decorrelating experience for 0 frames... [2023-02-26 10:10:49,415][10813] Decorrelating experience for 64 frames... [2023-02-26 10:10:50,037][10817] Decorrelating experience for 64 frames... [2023-02-26 10:10:50,202][10814] Decorrelating experience for 64 frames... [2023-02-26 10:10:50,546][10815] Decorrelating experience for 32 frames... [2023-02-26 10:10:50,921][10820] Decorrelating experience for 64 frames... [2023-02-26 10:10:51,695][10817] Decorrelating experience for 96 frames... [2023-02-26 10:10:51,989][10818] Decorrelating experience for 64 frames... [2023-02-26 10:10:52,139][10813] Decorrelating experience for 96 frames... [2023-02-26 10:10:52,315][10816] Decorrelating experience for 32 frames... [2023-02-26 10:10:52,529][10820] Decorrelating experience for 96 frames... [2023-02-26 10:10:53,036][10816] Decorrelating experience for 64 frames... [2023-02-26 10:10:53,426][10814] Decorrelating experience for 96 frames... [2023-02-26 10:10:53,705][00304] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-26 10:10:53,836][10816] Decorrelating experience for 96 frames... [2023-02-26 10:10:54,048][10815] Decorrelating experience for 64 frames... [2023-02-26 10:10:54,340][10818] Decorrelating experience for 96 frames... [2023-02-26 10:10:54,884][10815] Decorrelating experience for 96 frames... [2023-02-26 10:10:58,063][10798] Signal inference workers to stop experience collection... [2023-02-26 10:10:58,076][10811] InferenceWorker_p0-w0: stopping experience collection [2023-02-26 10:10:58,705][00304] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 101.6. Samples: 1524. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-26 10:10:58,706][00304] Avg episode reward: [(0, '1.840')] [2023-02-26 10:11:00,697][10798] Signal inference workers to resume experience collection... [2023-02-26 10:11:00,697][10811] InferenceWorker_p0-w0: resuming experience collection [2023-02-26 10:11:03,705][00304] Fps is (10 sec: 1638.4, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 16384. Throughput: 0: 200.5. Samples: 4010. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:11:03,712][00304] Avg episode reward: [(0, '3.125')] [2023-02-26 10:11:08,705][00304] Fps is (10 sec: 3686.4, 60 sec: 1474.6, 300 sec: 1474.6). Total num frames: 36864. Throughput: 0: 298.8. Samples: 7470. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-26 10:11:08,714][00304] Avg episode reward: [(0, '3.767')] [2023-02-26 10:11:09,903][10811] Updated weights for policy 0, policy_version 10 (0.0019) [2023-02-26 10:11:13,705][00304] Fps is (10 sec: 3276.8, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 49152. Throughput: 0: 404.4. Samples: 12132. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-26 10:11:13,710][00304] Avg episode reward: [(0, '4.354')] [2023-02-26 10:11:18,705][00304] Fps is (10 sec: 3276.8, 60 sec: 1989.5, 300 sec: 1989.5). Total num frames: 69632. Throughput: 0: 496.4. Samples: 17374. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:11:18,708][00304] Avg episode reward: [(0, '4.419')] [2023-02-26 10:11:21,006][10811] Updated weights for policy 0, policy_version 20 (0.0014) [2023-02-26 10:11:23,705][00304] Fps is (10 sec: 4505.5, 60 sec: 2355.2, 300 sec: 2355.2). Total num frames: 94208. Throughput: 0: 522.3. Samples: 20892. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:11:23,707][00304] Avg episode reward: [(0, '4.410')] [2023-02-26 10:11:28,705][00304] Fps is (10 sec: 4096.0, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 110592. Throughput: 0: 617.9. Samples: 27804. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:11:28,710][00304] Avg episode reward: [(0, '4.198')] [2023-02-26 10:11:28,714][10798] Saving new best policy, reward=4.198! [2023-02-26 10:11:31,700][10811] Updated weights for policy 0, policy_version 30 (0.0016) [2023-02-26 10:11:33,705][00304] Fps is (10 sec: 3276.8, 60 sec: 2539.5, 300 sec: 2539.5). Total num frames: 126976. Throughput: 0: 716.5. Samples: 32244. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:11:33,713][00304] Avg episode reward: [(0, '4.307')] [2023-02-26 10:11:33,722][10798] Saving new best policy, reward=4.307! [2023-02-26 10:11:38,705][00304] Fps is (10 sec: 3276.8, 60 sec: 2606.5, 300 sec: 2606.5). Total num frames: 143360. Throughput: 0: 762.7. Samples: 34320. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:11:38,708][00304] Avg episode reward: [(0, '4.424')] [2023-02-26 10:11:38,710][10798] Saving new best policy, reward=4.424! [2023-02-26 10:11:42,765][10811] Updated weights for policy 0, policy_version 40 (0.0021) [2023-02-26 10:11:43,705][00304] Fps is (10 sec: 4095.9, 60 sec: 2798.9, 300 sec: 2798.9). Total num frames: 167936. Throughput: 0: 871.0. Samples: 40720. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:11:43,711][00304] Avg episode reward: [(0, '4.387')] [2023-02-26 10:11:48,705][00304] Fps is (10 sec: 4505.7, 60 sec: 3140.3, 300 sec: 2898.7). Total num frames: 188416. Throughput: 0: 959.5. Samples: 47188. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:11:48,708][00304] Avg episode reward: [(0, '4.307')] [2023-02-26 10:11:53,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 2867.2). Total num frames: 200704. Throughput: 0: 932.0. Samples: 49408. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:11:53,712][00304] Avg episode reward: [(0, '4.442')] [2023-02-26 10:11:53,727][10798] Saving new best policy, reward=4.442! [2023-02-26 10:11:54,028][10811] Updated weights for policy 0, policy_version 50 (0.0023) [2023-02-26 10:11:58,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 2949.1). Total num frames: 221184. Throughput: 0: 931.2. Samples: 54038. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-26 10:11:58,712][00304] Avg episode reward: [(0, '4.491')] [2023-02-26 10:11:58,718][10798] Saving new best policy, reward=4.491! [2023-02-26 10:12:03,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3754.7, 300 sec: 3020.8). Total num frames: 241664. Throughput: 0: 970.9. Samples: 61064. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:12:03,713][00304] Avg episode reward: [(0, '4.602')] [2023-02-26 10:12:03,730][10798] Saving new best policy, reward=4.602! [2023-02-26 10:12:03,964][10811] Updated weights for policy 0, policy_version 60 (0.0029) [2023-02-26 10:12:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3084.1). Total num frames: 262144. Throughput: 0: 969.3. Samples: 64510. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-26 10:12:08,709][00304] Avg episode reward: [(0, '4.708')] [2023-02-26 10:12:08,714][10798] Saving new best policy, reward=4.708! [2023-02-26 10:12:13,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3094.8). Total num frames: 278528. Throughput: 0: 917.1. Samples: 69072. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:12:13,711][00304] Avg episode reward: [(0, '4.701')] [2023-02-26 10:12:13,722][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000068_278528.pth... [2023-02-26 10:12:16,684][10811] Updated weights for policy 0, policy_version 70 (0.0042) [2023-02-26 10:12:18,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3104.3). Total num frames: 294912. Throughput: 0: 927.2. Samples: 73966. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:12:18,712][00304] Avg episode reward: [(0, '4.519')] [2023-02-26 10:12:23,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3754.7, 300 sec: 3194.9). Total num frames: 319488. Throughput: 0: 959.2. Samples: 77482. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:12:23,711][00304] Avg episode reward: [(0, '4.522')] [2023-02-26 10:12:25,092][10811] Updated weights for policy 0, policy_version 80 (0.0024) [2023-02-26 10:12:28,710][00304] Fps is (10 sec: 4503.1, 60 sec: 3822.6, 300 sec: 3237.6). Total num frames: 339968. Throughput: 0: 970.9. Samples: 84416. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:12:28,713][00304] Avg episode reward: [(0, '4.482')] [2023-02-26 10:12:33,712][00304] Fps is (10 sec: 3274.4, 60 sec: 3754.2, 300 sec: 3202.1). Total num frames: 352256. Throughput: 0: 923.7. Samples: 88762. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:12:33,715][00304] Avg episode reward: [(0, '4.444')] [2023-02-26 10:12:37,829][10811] Updated weights for policy 0, policy_version 90 (0.0014) [2023-02-26 10:12:38,705][00304] Fps is (10 sec: 3278.6, 60 sec: 3822.9, 300 sec: 3241.2). Total num frames: 372736. Throughput: 0: 921.5. Samples: 90874. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:12:38,707][00304] Avg episode reward: [(0, '4.521')] [2023-02-26 10:12:43,705][00304] Fps is (10 sec: 4099.1, 60 sec: 3754.7, 300 sec: 3276.8). Total num frames: 393216. Throughput: 0: 971.2. Samples: 97742. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:12:43,711][00304] Avg episode reward: [(0, '4.738')] [2023-02-26 10:12:43,718][10798] Saving new best policy, reward=4.738! [2023-02-26 10:12:46,389][10811] Updated weights for policy 0, policy_version 100 (0.0017) [2023-02-26 10:12:48,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3309.6). Total num frames: 413696. Throughput: 0: 954.7. Samples: 104024. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:12:48,708][00304] Avg episode reward: [(0, '4.659')] [2023-02-26 10:12:53,706][00304] Fps is (10 sec: 3685.8, 60 sec: 3822.8, 300 sec: 3308.3). Total num frames: 430080. Throughput: 0: 925.9. Samples: 106176. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-26 10:12:53,711][00304] Avg episode reward: [(0, '4.789')] [2023-02-26 10:12:53,721][10798] Saving new best policy, reward=4.789! [2023-02-26 10:12:58,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3307.1). Total num frames: 446464. Throughput: 0: 927.1. Samples: 110792. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-26 10:12:58,707][00304] Avg episode reward: [(0, '4.871')] [2023-02-26 10:12:58,712][10798] Saving new best policy, reward=4.871! [2023-02-26 10:12:59,215][10811] Updated weights for policy 0, policy_version 110 (0.0014) [2023-02-26 10:13:03,705][00304] Fps is (10 sec: 4096.6, 60 sec: 3822.9, 300 sec: 3364.6). Total num frames: 471040. Throughput: 0: 971.8. Samples: 117698. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:13:03,711][00304] Avg episode reward: [(0, '4.540')] [2023-02-26 10:13:08,711][00304] Fps is (10 sec: 4093.3, 60 sec: 3754.3, 300 sec: 3361.4). Total num frames: 487424. Throughput: 0: 972.0. Samples: 121230. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:13:08,714][00304] Avg episode reward: [(0, '4.564')] [2023-02-26 10:13:08,776][10811] Updated weights for policy 0, policy_version 120 (0.0020) [2023-02-26 10:13:13,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3358.7). Total num frames: 503808. Throughput: 0: 921.0. Samples: 125854. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:13:13,710][00304] Avg episode reward: [(0, '4.649')] [2023-02-26 10:13:18,705][00304] Fps is (10 sec: 3688.8, 60 sec: 3822.9, 300 sec: 3382.5). Total num frames: 524288. Throughput: 0: 936.7. Samples: 130908. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-26 10:13:18,712][00304] Avg episode reward: [(0, '4.832')] [2023-02-26 10:13:20,254][10811] Updated weights for policy 0, policy_version 130 (0.0012) [2023-02-26 10:13:23,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3404.8). Total num frames: 544768. Throughput: 0: 969.2. Samples: 134488. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:13:23,717][00304] Avg episode reward: [(0, '4.935')] [2023-02-26 10:13:23,725][10798] Saving new best policy, reward=4.935! [2023-02-26 10:13:28,705][00304] Fps is (10 sec: 3686.2, 60 sec: 3686.7, 300 sec: 3400.9). Total num frames: 561152. Throughput: 0: 953.0. Samples: 140626. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:13:28,711][00304] Avg episode reward: [(0, '4.875')] [2023-02-26 10:13:32,596][10811] Updated weights for policy 0, policy_version 140 (0.0012) [2023-02-26 10:13:33,709][00304] Fps is (10 sec: 2865.9, 60 sec: 3686.6, 300 sec: 3373.1). Total num frames: 573440. Throughput: 0: 892.1. Samples: 144174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:13:33,714][00304] Avg episode reward: [(0, '4.991')] [2023-02-26 10:13:33,726][10798] Saving new best policy, reward=4.991! [2023-02-26 10:13:38,705][00304] Fps is (10 sec: 2457.7, 60 sec: 3549.9, 300 sec: 3347.0). Total num frames: 585728. Throughput: 0: 882.2. Samples: 145872. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:13:38,710][00304] Avg episode reward: [(0, '4.768')] [2023-02-26 10:13:43,705][00304] Fps is (10 sec: 3278.3, 60 sec: 3549.9, 300 sec: 3367.8). Total num frames: 606208. Throughput: 0: 880.3. Samples: 150404. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:13:43,707][00304] Avg episode reward: [(0, '4.839')] [2023-02-26 10:13:45,280][10811] Updated weights for policy 0, policy_version 150 (0.0016) [2023-02-26 10:13:48,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3387.5). Total num frames: 626688. Throughput: 0: 884.7. Samples: 157508. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:13:48,707][00304] Avg episode reward: [(0, '5.079')] [2023-02-26 10:13:48,733][10798] Saving new best policy, reward=5.079! [2023-02-26 10:13:53,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3618.2, 300 sec: 3406.1). Total num frames: 647168. Throughput: 0: 884.2. Samples: 161014. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:13:53,708][00304] Avg episode reward: [(0, '5.043')] [2023-02-26 10:13:55,776][10811] Updated weights for policy 0, policy_version 160 (0.0038) [2023-02-26 10:13:58,705][00304] Fps is (10 sec: 3686.2, 60 sec: 3618.1, 300 sec: 3402.8). Total num frames: 663552. Throughput: 0: 881.5. Samples: 165524. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:13:58,710][00304] Avg episode reward: [(0, '5.149')] [2023-02-26 10:13:58,715][10798] Saving new best policy, reward=5.149! [2023-02-26 10:14:03,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3420.2). Total num frames: 684032. Throughput: 0: 889.1. Samples: 170916. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:14:03,710][00304] Avg episode reward: [(0, '5.004')] [2023-02-26 10:14:06,285][10811] Updated weights for policy 0, policy_version 170 (0.0021) [2023-02-26 10:14:08,705][00304] Fps is (10 sec: 4096.2, 60 sec: 3618.5, 300 sec: 3436.6). Total num frames: 704512. Throughput: 0: 886.8. Samples: 174392. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:14:08,708][00304] Avg episode reward: [(0, '5.318')] [2023-02-26 10:14:08,712][10798] Saving new best policy, reward=5.318! [2023-02-26 10:14:13,706][00304] Fps is (10 sec: 4095.8, 60 sec: 3686.3, 300 sec: 3452.3). Total num frames: 724992. Throughput: 0: 894.9. Samples: 180898. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:14:13,712][00304] Avg episode reward: [(0, '5.346')] [2023-02-26 10:14:13,723][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000177_724992.pth... [2023-02-26 10:14:13,854][10798] Saving new best policy, reward=5.346! [2023-02-26 10:14:17,388][10811] Updated weights for policy 0, policy_version 180 (0.0017) [2023-02-26 10:14:18,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3429.2). Total num frames: 737280. Throughput: 0: 912.6. Samples: 185238. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:14:18,707][00304] Avg episode reward: [(0, '5.452')] [2023-02-26 10:14:18,787][10798] Saving new best policy, reward=5.452! [2023-02-26 10:14:23,705][00304] Fps is (10 sec: 3277.1, 60 sec: 3549.9, 300 sec: 3444.4). Total num frames: 757760. Throughput: 0: 929.3. Samples: 187690. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:14:23,708][00304] Avg episode reward: [(0, '5.361')] [2023-02-26 10:14:27,353][10811] Updated weights for policy 0, policy_version 190 (0.0026) [2023-02-26 10:14:28,705][00304] Fps is (10 sec: 4505.5, 60 sec: 3686.4, 300 sec: 3477.0). Total num frames: 782336. Throughput: 0: 987.4. Samples: 194838. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:14:28,713][00304] Avg episode reward: [(0, '5.738')] [2023-02-26 10:14:28,718][10798] Saving new best policy, reward=5.738! [2023-02-26 10:14:33,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3754.9, 300 sec: 3472.7). Total num frames: 798720. Throughput: 0: 960.5. Samples: 200730. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:14:33,709][00304] Avg episode reward: [(0, '5.963')] [2023-02-26 10:14:33,720][10798] Saving new best policy, reward=5.963! [2023-02-26 10:14:38,705][00304] Fps is (10 sec: 3276.6, 60 sec: 3822.9, 300 sec: 3468.5). Total num frames: 815104. Throughput: 0: 929.7. Samples: 202850. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:14:38,711][00304] Avg episode reward: [(0, '5.805')] [2023-02-26 10:14:39,664][10811] Updated weights for policy 0, policy_version 200 (0.0012) [2023-02-26 10:14:43,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3481.6). Total num frames: 835584. Throughput: 0: 943.1. Samples: 207962. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:14:43,710][00304] Avg episode reward: [(0, '5.583')] [2023-02-26 10:14:48,560][10811] Updated weights for policy 0, policy_version 210 (0.0028) [2023-02-26 10:14:48,705][00304] Fps is (10 sec: 4505.9, 60 sec: 3891.2, 300 sec: 3510.9). Total num frames: 860160. Throughput: 0: 982.2. Samples: 215114. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:14:48,711][00304] Avg episode reward: [(0, '5.683')] [2023-02-26 10:14:53,709][00304] Fps is (10 sec: 4094.3, 60 sec: 3822.7, 300 sec: 3506.1). Total num frames: 876544. Throughput: 0: 982.2. Samples: 218594. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:14:53,711][00304] Avg episode reward: [(0, '6.138')] [2023-02-26 10:14:53,724][10798] Saving new best policy, reward=6.138! [2023-02-26 10:14:58,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3823.0, 300 sec: 3501.7). Total num frames: 892928. Throughput: 0: 934.4. Samples: 222946. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:14:58,708][00304] Avg episode reward: [(0, '6.480')] [2023-02-26 10:14:58,710][10798] Saving new best policy, reward=6.480! [2023-02-26 10:15:00,950][10811] Updated weights for policy 0, policy_version 220 (0.0021) [2023-02-26 10:15:03,705][00304] Fps is (10 sec: 3687.9, 60 sec: 3822.9, 300 sec: 3513.1). Total num frames: 913408. Throughput: 0: 961.2. Samples: 228490. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-26 10:15:03,708][00304] Avg episode reward: [(0, '6.730')] [2023-02-26 10:15:03,716][10798] Saving new best policy, reward=6.730! [2023-02-26 10:15:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3524.1). Total num frames: 933888. Throughput: 0: 985.1. Samples: 232018. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:15:08,707][00304] Avg episode reward: [(0, '6.977')] [2023-02-26 10:15:08,714][10798] Saving new best policy, reward=6.977! [2023-02-26 10:15:09,745][10811] Updated weights for policy 0, policy_version 230 (0.0014) [2023-02-26 10:15:13,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3823.0, 300 sec: 3534.7). Total num frames: 954368. Throughput: 0: 965.6. Samples: 238292. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:15:13,707][00304] Avg episode reward: [(0, '7.558')] [2023-02-26 10:15:13,719][10798] Saving new best policy, reward=7.558! [2023-02-26 10:15:18,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3515.1). Total num frames: 966656. Throughput: 0: 932.7. Samples: 242700. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:15:18,707][00304] Avg episode reward: [(0, '7.857')] [2023-02-26 10:15:18,714][10798] Saving new best policy, reward=7.873! [2023-02-26 10:15:22,041][10811] Updated weights for policy 0, policy_version 240 (0.0016) [2023-02-26 10:15:23,707][00304] Fps is (10 sec: 3275.9, 60 sec: 3822.8, 300 sec: 3525.5). Total num frames: 987136. Throughput: 0: 941.9. Samples: 245236. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:15:23,713][00304] Avg episode reward: [(0, '7.205')] [2023-02-26 10:15:28,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3549.9). Total num frames: 1011712. Throughput: 0: 986.4. Samples: 252348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:15:28,707][00304] Avg episode reward: [(0, '6.740')] [2023-02-26 10:15:30,544][10811] Updated weights for policy 0, policy_version 250 (0.0025) [2023-02-26 10:15:33,705][00304] Fps is (10 sec: 4506.6, 60 sec: 3891.2, 300 sec: 3559.3). Total num frames: 1032192. Throughput: 0: 960.8. Samples: 258350. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:15:33,708][00304] Avg episode reward: [(0, '7.093')] [2023-02-26 10:15:38,708][00304] Fps is (10 sec: 3275.7, 60 sec: 3822.8, 300 sec: 3540.6). Total num frames: 1044480. Throughput: 0: 931.9. Samples: 260530. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:15:38,713][00304] Avg episode reward: [(0, '7.351')] [2023-02-26 10:15:42,805][10811] Updated weights for policy 0, policy_version 260 (0.0045) [2023-02-26 10:15:43,705][00304] Fps is (10 sec: 3277.0, 60 sec: 3822.9, 300 sec: 3610.0). Total num frames: 1064960. Throughput: 0: 951.2. Samples: 265748. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:15:43,708][00304] Avg episode reward: [(0, '7.397')] [2023-02-26 10:15:48,705][00304] Fps is (10 sec: 4507.1, 60 sec: 3822.9, 300 sec: 3693.3). Total num frames: 1089536. Throughput: 0: 985.1. Samples: 272820. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:15:48,711][00304] Avg episode reward: [(0, '7.952')] [2023-02-26 10:15:48,715][10798] Saving new best policy, reward=7.952! [2023-02-26 10:15:52,180][10811] Updated weights for policy 0, policy_version 270 (0.0019) [2023-02-26 10:15:53,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3891.5, 300 sec: 3762.8). Total num frames: 1110016. Throughput: 0: 978.0. Samples: 276028. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:15:53,710][00304] Avg episode reward: [(0, '7.779')] [2023-02-26 10:15:58,705][00304] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1122304. Throughput: 0: 939.8. Samples: 280582. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:15:58,709][00304] Avg episode reward: [(0, '8.659')] [2023-02-26 10:15:58,714][10798] Saving new best policy, reward=8.659! [2023-02-26 10:16:03,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1142784. Throughput: 0: 965.9. Samples: 286166. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:16:03,710][00304] Avg episode reward: [(0, '8.576')] [2023-02-26 10:16:04,077][10811] Updated weights for policy 0, policy_version 280 (0.0019) [2023-02-26 10:16:08,705][00304] Fps is (10 sec: 4505.7, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 1167360. Throughput: 0: 986.7. Samples: 289634. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:16:08,708][00304] Avg episode reward: [(0, '9.058')] [2023-02-26 10:16:08,713][10798] Saving new best policy, reward=9.058! [2023-02-26 10:16:13,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1183744. Throughput: 0: 968.1. Samples: 295914. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:16:13,710][00304] Avg episode reward: [(0, '8.894')] [2023-02-26 10:16:13,724][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000289_1183744.pth... [2023-02-26 10:16:13,862][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000068_278528.pth [2023-02-26 10:16:14,128][10811] Updated weights for policy 0, policy_version 290 (0.0022) [2023-02-26 10:16:18,705][00304] Fps is (10 sec: 3276.9, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 1200128. Throughput: 0: 934.2. Samples: 300388. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:16:18,716][00304] Avg episode reward: [(0, '9.432')] [2023-02-26 10:16:18,720][10798] Saving new best policy, reward=9.432! [2023-02-26 10:16:23,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.4, 300 sec: 3762.8). Total num frames: 1220608. Throughput: 0: 940.0. Samples: 302826. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:16:23,709][00304] Avg episode reward: [(0, '9.769')] [2023-02-26 10:16:23,723][10798] Saving new best policy, reward=9.769! [2023-02-26 10:16:25,514][10811] Updated weights for policy 0, policy_version 300 (0.0040) [2023-02-26 10:16:28,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3762.8). Total num frames: 1236992. Throughput: 0: 954.4. Samples: 308696. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:16:28,708][00304] Avg episode reward: [(0, '10.407')] [2023-02-26 10:16:28,715][10798] Saving new best policy, reward=10.407! [2023-02-26 10:16:33,706][00304] Fps is (10 sec: 2866.9, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 1249280. Throughput: 0: 888.8. Samples: 312818. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-26 10:16:33,708][00304] Avg episode reward: [(0, '10.981')] [2023-02-26 10:16:33,720][10798] Saving new best policy, reward=10.981! [2023-02-26 10:16:38,705][00304] Fps is (10 sec: 2457.6, 60 sec: 3618.3, 300 sec: 3707.2). Total num frames: 1261568. Throughput: 0: 858.6. Samples: 314666. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:16:38,716][00304] Avg episode reward: [(0, '10.511')] [2023-02-26 10:16:40,799][10811] Updated weights for policy 0, policy_version 310 (0.0033) [2023-02-26 10:16:43,705][00304] Fps is (10 sec: 2867.5, 60 sec: 3549.9, 300 sec: 3693.3). Total num frames: 1277952. Throughput: 0: 854.9. Samples: 319054. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:16:43,707][00304] Avg episode reward: [(0, '10.395')] [2023-02-26 10:16:48,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3735.0). Total num frames: 1302528. Throughput: 0: 883.5. Samples: 325924. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:16:48,711][00304] Avg episode reward: [(0, '10.319')] [2023-02-26 10:16:49,971][10811] Updated weights for policy 0, policy_version 320 (0.0021) [2023-02-26 10:16:53,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 1327104. Throughput: 0: 883.3. Samples: 329384. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-26 10:16:53,710][00304] Avg episode reward: [(0, '10.794')] [2023-02-26 10:16:58,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3721.1). Total num frames: 1339392. Throughput: 0: 862.8. Samples: 334740. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:16:58,709][00304] Avg episode reward: [(0, '10.550')] [2023-02-26 10:17:01,924][10811] Updated weights for policy 0, policy_version 330 (0.0018) [2023-02-26 10:17:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3707.2). Total num frames: 1355776. Throughput: 0: 866.1. Samples: 339362. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:17:03,714][00304] Avg episode reward: [(0, '11.330')] [2023-02-26 10:17:03,725][10798] Saving new best policy, reward=11.330! [2023-02-26 10:17:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3735.0). Total num frames: 1380352. Throughput: 0: 888.9. Samples: 342826. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:17:08,716][00304] Avg episode reward: [(0, '12.349')] [2023-02-26 10:17:08,721][10798] Saving new best policy, reward=12.349! [2023-02-26 10:17:11,257][10811] Updated weights for policy 0, policy_version 340 (0.0021) [2023-02-26 10:17:13,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 1400832. Throughput: 0: 913.6. Samples: 349808. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:17:13,709][00304] Avg episode reward: [(0, '13.176')] [2023-02-26 10:17:13,729][10798] Saving new best policy, reward=13.176! [2023-02-26 10:17:18,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3721.1). Total num frames: 1417216. Throughput: 0: 927.3. Samples: 354546. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:17:18,707][00304] Avg episode reward: [(0, '13.347')] [2023-02-26 10:17:18,710][10798] Saving new best policy, reward=13.347! [2023-02-26 10:17:23,570][10811] Updated weights for policy 0, policy_version 350 (0.0030) [2023-02-26 10:17:23,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3707.3). Total num frames: 1433600. Throughput: 0: 935.0. Samples: 356740. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:17:23,707][00304] Avg episode reward: [(0, '13.270')] [2023-02-26 10:17:28,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3749.0). Total num frames: 1458176. Throughput: 0: 979.4. Samples: 363128. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:17:28,707][00304] Avg episode reward: [(0, '13.893')] [2023-02-26 10:17:28,713][10798] Saving new best policy, reward=13.893! [2023-02-26 10:17:32,226][10811] Updated weights for policy 0, policy_version 360 (0.0026) [2023-02-26 10:17:33,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3823.0, 300 sec: 3748.9). Total num frames: 1478656. Throughput: 0: 982.0. Samples: 370112. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:17:33,709][00304] Avg episode reward: [(0, '14.632')] [2023-02-26 10:17:33,722][10798] Saving new best policy, reward=14.632! [2023-02-26 10:17:38,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 1490944. Throughput: 0: 952.1. Samples: 372230. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:17:38,707][00304] Avg episode reward: [(0, '15.106')] [2023-02-26 10:17:38,786][10798] Saving new best policy, reward=15.106! [2023-02-26 10:17:43,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 1507328. Throughput: 0: 931.5. Samples: 376656. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:17:43,710][00304] Avg episode reward: [(0, '15.887')] [2023-02-26 10:17:43,726][10798] Saving new best policy, reward=15.887! [2023-02-26 10:17:44,764][10811] Updated weights for policy 0, policy_version 370 (0.0016) [2023-02-26 10:17:48,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 1531904. Throughput: 0: 978.3. Samples: 383386. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:17:48,712][00304] Avg episode reward: [(0, '16.659')] [2023-02-26 10:17:48,719][10798] Saving new best policy, reward=16.659! [2023-02-26 10:17:53,669][10811] Updated weights for policy 0, policy_version 380 (0.0012) [2023-02-26 10:17:53,705][00304] Fps is (10 sec: 4915.0, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 1556480. Throughput: 0: 979.0. Samples: 386882. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:17:53,708][00304] Avg episode reward: [(0, '15.999')] [2023-02-26 10:17:58,706][00304] Fps is (10 sec: 3685.9, 60 sec: 3822.8, 300 sec: 3721.1). Total num frames: 1568768. Throughput: 0: 940.0. Samples: 392108. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:17:58,710][00304] Avg episode reward: [(0, '16.391')] [2023-02-26 10:18:03,705][00304] Fps is (10 sec: 2867.3, 60 sec: 3822.9, 300 sec: 3721.2). Total num frames: 1585152. Throughput: 0: 935.5. Samples: 396642. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:18:03,708][00304] Avg episode reward: [(0, '17.330')] [2023-02-26 10:18:03,718][10798] Saving new best policy, reward=17.330! [2023-02-26 10:18:05,841][10811] Updated weights for policy 0, policy_version 390 (0.0024) [2023-02-26 10:18:08,705][00304] Fps is (10 sec: 4096.6, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1609728. Throughput: 0: 963.2. Samples: 400086. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:18:08,711][00304] Avg episode reward: [(0, '17.604')] [2023-02-26 10:18:08,716][10798] Saving new best policy, reward=17.604! [2023-02-26 10:18:13,707][00304] Fps is (10 sec: 4504.5, 60 sec: 3822.8, 300 sec: 3748.9). Total num frames: 1630208. Throughput: 0: 976.9. Samples: 407090. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:18:13,711][00304] Avg episode reward: [(0, '17.843')] [2023-02-26 10:18:13,723][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000398_1630208.pth... [2023-02-26 10:18:13,917][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000177_724992.pth [2023-02-26 10:18:13,934][10798] Saving new best policy, reward=17.843! [2023-02-26 10:18:15,806][10811] Updated weights for policy 0, policy_version 400 (0.0026) [2023-02-26 10:18:18,705][00304] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 1646592. Throughput: 0: 926.6. Samples: 411810. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:18:18,708][00304] Avg episode reward: [(0, '18.108')] [2023-02-26 10:18:18,710][10798] Saving new best policy, reward=18.108! [2023-02-26 10:18:23,705][00304] Fps is (10 sec: 2867.9, 60 sec: 3754.7, 300 sec: 3721.1). Total num frames: 1658880. Throughput: 0: 928.9. Samples: 414030. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:18:23,708][00304] Avg episode reward: [(0, '17.318')] [2023-02-26 10:18:27,147][10811] Updated weights for policy 0, policy_version 410 (0.0014) [2023-02-26 10:18:28,707][00304] Fps is (10 sec: 3685.8, 60 sec: 3754.5, 300 sec: 3762.8). Total num frames: 1683456. Throughput: 0: 972.9. Samples: 420438. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:18:28,709][00304] Avg episode reward: [(0, '17.299')] [2023-02-26 10:18:33,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 1708032. Throughput: 0: 980.8. Samples: 427524. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:18:33,711][00304] Avg episode reward: [(0, '15.535')] [2023-02-26 10:18:36,963][10811] Updated weights for policy 0, policy_version 420 (0.0014) [2023-02-26 10:18:38,709][00304] Fps is (10 sec: 4095.0, 60 sec: 3890.9, 300 sec: 3790.5). Total num frames: 1724416. Throughput: 0: 955.2. Samples: 429870. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:18:38,714][00304] Avg episode reward: [(0, '15.266')] [2023-02-26 10:18:43,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 1740800. Throughput: 0: 939.3. Samples: 434376. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:18:43,707][00304] Avg episode reward: [(0, '14.433')] [2023-02-26 10:18:48,075][10811] Updated weights for policy 0, policy_version 430 (0.0033) [2023-02-26 10:18:48,705][00304] Fps is (10 sec: 3688.1, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1761280. Throughput: 0: 984.4. Samples: 440938. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:18:48,708][00304] Avg episode reward: [(0, '14.989')] [2023-02-26 10:18:53,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3823.0, 300 sec: 3804.4). Total num frames: 1785856. Throughput: 0: 987.5. Samples: 444524. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:18:53,708][00304] Avg episode reward: [(0, '14.899')] [2023-02-26 10:18:58,512][10811] Updated weights for policy 0, policy_version 440 (0.0015) [2023-02-26 10:18:58,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3891.3, 300 sec: 3790.5). Total num frames: 1802240. Throughput: 0: 955.7. Samples: 450096. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:18:58,709][00304] Avg episode reward: [(0, '15.640')] [2023-02-26 10:19:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 1814528. Throughput: 0: 954.2. Samples: 454748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:19:03,712][00304] Avg episode reward: [(0, '15.480')] [2023-02-26 10:19:08,705][00304] Fps is (10 sec: 3686.5, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1839104. Throughput: 0: 980.1. Samples: 458136. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:19:08,706][00304] Avg episode reward: [(0, '15.769')] [2023-02-26 10:19:08,859][10811] Updated weights for policy 0, policy_version 450 (0.0030) [2023-02-26 10:19:13,705][00304] Fps is (10 sec: 4915.0, 60 sec: 3891.3, 300 sec: 3818.3). Total num frames: 1863680. Throughput: 0: 995.9. Samples: 465250. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:19:13,713][00304] Avg episode reward: [(0, '16.614')] [2023-02-26 10:19:18,705][00304] Fps is (10 sec: 4095.8, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 1880064. Throughput: 0: 949.9. Samples: 470272. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:19:18,710][00304] Avg episode reward: [(0, '16.401')] [2023-02-26 10:19:19,698][10811] Updated weights for policy 0, policy_version 460 (0.0046) [2023-02-26 10:19:23,705][00304] Fps is (10 sec: 2867.3, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 1892352. Throughput: 0: 948.0. Samples: 472524. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:19:23,712][00304] Avg episode reward: [(0, '16.627')] [2023-02-26 10:19:28,706][00304] Fps is (10 sec: 2866.9, 60 sec: 3754.7, 300 sec: 3762.7). Total num frames: 1908736. Throughput: 0: 949.5. Samples: 477104. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:19:28,709][00304] Avg episode reward: [(0, '18.535')] [2023-02-26 10:19:28,714][10798] Saving new best policy, reward=18.535! [2023-02-26 10:19:33,275][10811] Updated weights for policy 0, policy_version 470 (0.0050) [2023-02-26 10:19:33,708][00304] Fps is (10 sec: 3275.7, 60 sec: 3617.9, 300 sec: 3762.7). Total num frames: 1925120. Throughput: 0: 905.6. Samples: 481692. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:19:33,711][00304] Avg episode reward: [(0, '18.495')] [2023-02-26 10:19:38,705][00304] Fps is (10 sec: 3277.3, 60 sec: 3618.4, 300 sec: 3748.9). Total num frames: 1941504. Throughput: 0: 874.1. Samples: 483858. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:19:38,707][00304] Avg episode reward: [(0, '19.608')] [2023-02-26 10:19:38,715][10798] Saving new best policy, reward=19.608! [2023-02-26 10:19:43,706][00304] Fps is (10 sec: 2867.9, 60 sec: 3549.8, 300 sec: 3707.2). Total num frames: 1953792. Throughput: 0: 850.9. Samples: 488386. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:19:43,711][00304] Avg episode reward: [(0, '19.577')] [2023-02-26 10:19:45,806][10811] Updated weights for policy 0, policy_version 480 (0.0012) [2023-02-26 10:19:48,705][00304] Fps is (10 sec: 3686.2, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 1978368. Throughput: 0: 890.6. Samples: 494824. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:19:48,710][00304] Avg episode reward: [(0, '18.656')] [2023-02-26 10:19:53,705][00304] Fps is (10 sec: 4915.6, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 2002944. Throughput: 0: 894.9. Samples: 498406. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:19:53,712][00304] Avg episode reward: [(0, '17.933')] [2023-02-26 10:19:54,674][10811] Updated weights for policy 0, policy_version 490 (0.0039) [2023-02-26 10:19:58,705][00304] Fps is (10 sec: 4096.2, 60 sec: 3618.2, 300 sec: 3748.9). Total num frames: 2019328. Throughput: 0: 863.2. Samples: 504092. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:19:58,709][00304] Avg episode reward: [(0, '16.509')] [2023-02-26 10:20:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3721.1). Total num frames: 2031616. Throughput: 0: 849.9. Samples: 508516. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:20:03,709][00304] Avg episode reward: [(0, '14.949')] [2023-02-26 10:20:06,682][10811] Updated weights for policy 0, policy_version 500 (0.0051) [2023-02-26 10:20:08,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 2056192. Throughput: 0: 872.4. Samples: 511782. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:20:08,709][00304] Avg episode reward: [(0, '14.981')] [2023-02-26 10:20:13,705][00304] Fps is (10 sec: 4915.1, 60 sec: 3618.2, 300 sec: 3776.6). Total num frames: 2080768. Throughput: 0: 931.0. Samples: 518996. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:20:13,713][00304] Avg episode reward: [(0, '14.595')] [2023-02-26 10:20:13,728][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000508_2080768.pth... [2023-02-26 10:20:13,910][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000289_1183744.pth [2023-02-26 10:20:16,193][10811] Updated weights for policy 0, policy_version 510 (0.0028) [2023-02-26 10:20:18,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 2093056. Throughput: 0: 943.2. Samples: 524132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:20:18,713][00304] Avg episode reward: [(0, '15.244')] [2023-02-26 10:20:23,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3721.1). Total num frames: 2109440. Throughput: 0: 946.7. Samples: 526458. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:20:23,715][00304] Avg episode reward: [(0, '15.341')] [2023-02-26 10:20:27,442][10811] Updated weights for policy 0, policy_version 520 (0.0022) [2023-02-26 10:20:28,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3754.8, 300 sec: 3735.0). Total num frames: 2134016. Throughput: 0: 980.8. Samples: 532522. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:20:28,706][00304] Avg episode reward: [(0, '15.672')] [2023-02-26 10:20:33,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3891.4, 300 sec: 3776.7). Total num frames: 2158592. Throughput: 0: 998.3. Samples: 539748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:20:33,707][00304] Avg episode reward: [(0, '15.373')] [2023-02-26 10:20:37,396][10811] Updated weights for policy 0, policy_version 530 (0.0017) [2023-02-26 10:20:38,706][00304] Fps is (10 sec: 3686.0, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2170880. Throughput: 0: 974.7. Samples: 542270. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:20:38,710][00304] Avg episode reward: [(0, '15.410')] [2023-02-26 10:20:43,705][00304] Fps is (10 sec: 2867.1, 60 sec: 3891.2, 300 sec: 3721.1). Total num frames: 2187264. Throughput: 0: 949.3. Samples: 546810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:20:43,714][00304] Avg episode reward: [(0, '15.977')] [2023-02-26 10:20:48,436][10811] Updated weights for policy 0, policy_version 540 (0.0019) [2023-02-26 10:20:48,705][00304] Fps is (10 sec: 4096.4, 60 sec: 3891.2, 300 sec: 3735.0). Total num frames: 2211840. Throughput: 0: 990.6. Samples: 553094. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:20:48,707][00304] Avg episode reward: [(0, '17.205')] [2023-02-26 10:20:53,706][00304] Fps is (10 sec: 4914.6, 60 sec: 3891.1, 300 sec: 3776.6). Total num frames: 2236416. Throughput: 0: 996.8. Samples: 556640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:20:53,709][00304] Avg episode reward: [(0, '17.741')] [2023-02-26 10:20:58,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2248704. Throughput: 0: 964.1. Samples: 562380. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:20:58,711][00304] Avg episode reward: [(0, '18.728')] [2023-02-26 10:20:58,892][10811] Updated weights for policy 0, policy_version 550 (0.0023) [2023-02-26 10:21:03,705][00304] Fps is (10 sec: 2867.6, 60 sec: 3891.2, 300 sec: 3721.1). Total num frames: 2265088. Throughput: 0: 950.0. Samples: 566880. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:21:03,709][00304] Avg episode reward: [(0, '19.662')] [2023-02-26 10:21:03,729][10798] Saving new best policy, reward=19.662! [2023-02-26 10:21:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2289664. Throughput: 0: 969.8. Samples: 570098. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:21:08,707][00304] Avg episode reward: [(0, '18.747')] [2023-02-26 10:21:09,425][10811] Updated weights for policy 0, policy_version 560 (0.0016) [2023-02-26 10:21:13,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 2310144. Throughput: 0: 992.3. Samples: 577174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:21:13,711][00304] Avg episode reward: [(0, '19.773')] [2023-02-26 10:21:13,775][10798] Saving new best policy, reward=19.773! [2023-02-26 10:21:18,706][00304] Fps is (10 sec: 3685.8, 60 sec: 3891.1, 300 sec: 3748.9). Total num frames: 2326528. Throughput: 0: 945.5. Samples: 582296. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:21:18,711][00304] Avg episode reward: [(0, '20.030')] [2023-02-26 10:21:18,717][10798] Saving new best policy, reward=20.030! [2023-02-26 10:21:20,685][10811] Updated weights for policy 0, policy_version 570 (0.0029) [2023-02-26 10:21:23,705][00304] Fps is (10 sec: 3276.6, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2342912. Throughput: 0: 937.5. Samples: 584456. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:21:23,708][00304] Avg episode reward: [(0, '20.282')] [2023-02-26 10:21:23,724][10798] Saving new best policy, reward=20.282! [2023-02-26 10:21:28,705][00304] Fps is (10 sec: 3686.9, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2363392. Throughput: 0: 969.5. Samples: 590438. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:21:28,714][00304] Avg episode reward: [(0, '20.098')] [2023-02-26 10:21:30,506][10811] Updated weights for policy 0, policy_version 580 (0.0027) [2023-02-26 10:21:33,705][00304] Fps is (10 sec: 4505.9, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2387968. Throughput: 0: 989.3. Samples: 597614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:21:33,714][00304] Avg episode reward: [(0, '21.631')] [2023-02-26 10:21:33,831][10798] Saving new best policy, reward=21.631! [2023-02-26 10:21:38,708][00304] Fps is (10 sec: 4094.5, 60 sec: 3891.0, 300 sec: 3818.3). Total num frames: 2404352. Throughput: 0: 968.0. Samples: 600202. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:21:38,715][00304] Avg episode reward: [(0, '20.796')] [2023-02-26 10:21:42,157][10811] Updated weights for policy 0, policy_version 590 (0.0027) [2023-02-26 10:21:43,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2420736. Throughput: 0: 941.8. Samples: 604760. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:21:43,713][00304] Avg episode reward: [(0, '20.107')] [2023-02-26 10:21:48,705][00304] Fps is (10 sec: 3687.7, 60 sec: 3822.9, 300 sec: 3776.6). Total num frames: 2441216. Throughput: 0: 979.5. Samples: 610956. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:21:48,711][00304] Avg episode reward: [(0, '21.372')] [2023-02-26 10:21:51,392][10811] Updated weights for policy 0, policy_version 600 (0.0013) [2023-02-26 10:21:53,705][00304] Fps is (10 sec: 4505.7, 60 sec: 3823.0, 300 sec: 3818.3). Total num frames: 2465792. Throughput: 0: 988.4. Samples: 614578. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:21:53,707][00304] Avg episode reward: [(0, '20.301')] [2023-02-26 10:21:58,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3818.3). Total num frames: 2482176. Throughput: 0: 963.0. Samples: 620508. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:21:58,709][00304] Avg episode reward: [(0, '19.867')] [2023-02-26 10:22:03,061][10811] Updated weights for policy 0, policy_version 610 (0.0016) [2023-02-26 10:22:03,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2498560. Throughput: 0: 951.3. Samples: 625104. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:22:03,715][00304] Avg episode reward: [(0, '19.721')] [2023-02-26 10:22:08,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 2523136. Throughput: 0: 971.8. Samples: 628186. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:22:08,707][00304] Avg episode reward: [(0, '18.784')] [2023-02-26 10:22:11,923][10811] Updated weights for policy 0, policy_version 620 (0.0030) [2023-02-26 10:22:13,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3959.5, 300 sec: 3832.2). Total num frames: 2547712. Throughput: 0: 1000.0. Samples: 635440. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:22:13,712][00304] Avg episode reward: [(0, '16.969')] [2023-02-26 10:22:13,724][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000622_2547712.pth... [2023-02-26 10:22:13,844][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000398_1630208.pth [2023-02-26 10:22:18,711][00304] Fps is (10 sec: 3684.1, 60 sec: 3890.9, 300 sec: 3818.2). Total num frames: 2560000. Throughput: 0: 959.8. Samples: 640812. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:22:18,713][00304] Avg episode reward: [(0, '17.219')] [2023-02-26 10:22:23,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2576384. Throughput: 0: 952.2. Samples: 643046. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:22:23,708][00304] Avg episode reward: [(0, '17.718')] [2023-02-26 10:22:24,253][10811] Updated weights for policy 0, policy_version 630 (0.0025) [2023-02-26 10:22:28,705][00304] Fps is (10 sec: 3278.8, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2592768. Throughput: 0: 953.6. Samples: 647670. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:22:28,710][00304] Avg episode reward: [(0, '19.042')] [2023-02-26 10:22:33,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3776.6). Total num frames: 2605056. Throughput: 0: 915.6. Samples: 652158. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:22:33,713][00304] Avg episode reward: [(0, '20.517')] [2023-02-26 10:22:38,015][10811] Updated weights for policy 0, policy_version 640 (0.0019) [2023-02-26 10:22:38,709][00304] Fps is (10 sec: 2865.9, 60 sec: 3618.1, 300 sec: 3776.6). Total num frames: 2621440. Throughput: 0: 885.5. Samples: 654430. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:22:38,717][00304] Avg episode reward: [(0, '19.093')] [2023-02-26 10:22:43,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 2637824. Throughput: 0: 852.2. Samples: 658858. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:22:43,707][00304] Avg episode reward: [(0, '19.732')] [2023-02-26 10:22:48,705][00304] Fps is (10 sec: 3688.0, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 2658304. Throughput: 0: 884.1. Samples: 664888. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:22:48,713][00304] Avg episode reward: [(0, '20.346')] [2023-02-26 10:22:49,146][10811] Updated weights for policy 0, policy_version 650 (0.0014) [2023-02-26 10:22:53,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3776.7). Total num frames: 2682880. Throughput: 0: 894.4. Samples: 668436. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:22:53,714][00304] Avg episode reward: [(0, '21.043')] [2023-02-26 10:22:58,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3776.7). Total num frames: 2699264. Throughput: 0: 866.4. Samples: 674428. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:22:58,708][00304] Avg episode reward: [(0, '20.892')] [2023-02-26 10:22:59,741][10811] Updated weights for policy 0, policy_version 660 (0.0018) [2023-02-26 10:23:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3735.0). Total num frames: 2711552. Throughput: 0: 848.5. Samples: 678988. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:23:03,712][00304] Avg episode reward: [(0, '21.428')] [2023-02-26 10:23:08,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 2736128. Throughput: 0: 865.1. Samples: 681976. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:23:08,707][00304] Avg episode reward: [(0, '22.448')] [2023-02-26 10:23:08,714][10798] Saving new best policy, reward=22.448! [2023-02-26 10:23:10,105][10811] Updated weights for policy 0, policy_version 670 (0.0031) [2023-02-26 10:23:13,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3549.9, 300 sec: 3776.7). Total num frames: 2760704. Throughput: 0: 919.6. Samples: 689054. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:23:13,714][00304] Avg episode reward: [(0, '21.231')] [2023-02-26 10:23:18,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3618.5, 300 sec: 3790.5). Total num frames: 2777088. Throughput: 0: 941.0. Samples: 694504. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:23:18,712][00304] Avg episode reward: [(0, '20.246')] [2023-02-26 10:23:21,404][10811] Updated weights for policy 0, policy_version 680 (0.0019) [2023-02-26 10:23:23,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3748.9). Total num frames: 2789376. Throughput: 0: 941.3. Samples: 696786. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:23:23,710][00304] Avg episode reward: [(0, '22.131')] [2023-02-26 10:23:28,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3748.9). Total num frames: 2813952. Throughput: 0: 973.8. Samples: 702680. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-26 10:23:28,713][00304] Avg episode reward: [(0, '21.294')] [2023-02-26 10:23:30,854][10811] Updated weights for policy 0, policy_version 690 (0.0029) [2023-02-26 10:23:33,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 2838528. Throughput: 0: 999.7. Samples: 709874. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:23:33,710][00304] Avg episode reward: [(0, '20.813')] [2023-02-26 10:23:38,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.5, 300 sec: 3776.7). Total num frames: 2854912. Throughput: 0: 982.5. Samples: 712650. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-26 10:23:38,711][00304] Avg episode reward: [(0, '21.234')] [2023-02-26 10:23:42,581][10811] Updated weights for policy 0, policy_version 700 (0.0013) [2023-02-26 10:23:43,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2867200. Throughput: 0: 951.3. Samples: 717236. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-26 10:23:43,708][00304] Avg episode reward: [(0, '20.695')] [2023-02-26 10:23:48,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2891776. Throughput: 0: 990.4. Samples: 723556. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-02-26 10:23:48,707][00304] Avg episode reward: [(0, '18.929')] [2023-02-26 10:23:51,616][10811] Updated weights for policy 0, policy_version 710 (0.0020) [2023-02-26 10:23:53,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 2916352. Throughput: 0: 1001.4. Samples: 727040. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:23:53,708][00304] Avg episode reward: [(0, '19.262')] [2023-02-26 10:23:58,706][00304] Fps is (10 sec: 4095.6, 60 sec: 3891.1, 300 sec: 3790.5). Total num frames: 2932736. Throughput: 0: 973.2. Samples: 732850. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:23:58,712][00304] Avg episode reward: [(0, '19.302')] [2023-02-26 10:24:03,523][10811] Updated weights for policy 0, policy_version 720 (0.0030) [2023-02-26 10:24:03,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3762.8). Total num frames: 2949120. Throughput: 0: 951.6. Samples: 737328. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:24:03,707][00304] Avg episode reward: [(0, '19.912')] [2023-02-26 10:24:08,705][00304] Fps is (10 sec: 3686.7, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2969600. Throughput: 0: 970.8. Samples: 740474. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:24:08,708][00304] Avg episode reward: [(0, '20.725')] [2023-02-26 10:24:12,491][10811] Updated weights for policy 0, policy_version 730 (0.0034) [2023-02-26 10:24:13,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 2994176. Throughput: 0: 999.5. Samples: 747656. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:24:13,708][00304] Avg episode reward: [(0, '21.823')] [2023-02-26 10:24:13,720][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000731_2994176.pth... [2023-02-26 10:24:13,841][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000508_2080768.pth [2023-02-26 10:24:18,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 3010560. Throughput: 0: 959.7. Samples: 753062. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:24:18,715][00304] Avg episode reward: [(0, '21.658')] [2023-02-26 10:24:23,707][00304] Fps is (10 sec: 3276.2, 60 sec: 3959.3, 300 sec: 3790.5). Total num frames: 3026944. Throughput: 0: 947.6. Samples: 755296. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:24:23,720][00304] Avg episode reward: [(0, '20.520')] [2023-02-26 10:24:24,666][10811] Updated weights for policy 0, policy_version 740 (0.0028) [2023-02-26 10:24:28,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3804.5). Total num frames: 3047424. Throughput: 0: 977.6. Samples: 761226. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:24:28,712][00304] Avg episode reward: [(0, '20.635')] [2023-02-26 10:24:33,127][10811] Updated weights for policy 0, policy_version 750 (0.0035) [2023-02-26 10:24:33,705][00304] Fps is (10 sec: 4506.5, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3072000. Throughput: 0: 996.0. Samples: 768374. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:24:33,708][00304] Avg episode reward: [(0, '19.186')] [2023-02-26 10:24:38,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3088384. Throughput: 0: 980.3. Samples: 771152. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:24:38,711][00304] Avg episode reward: [(0, '17.652')] [2023-02-26 10:24:43,708][00304] Fps is (10 sec: 3275.8, 60 sec: 3959.3, 300 sec: 3818.3). Total num frames: 3104768. Throughput: 0: 952.4. Samples: 775708. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:24:43,711][00304] Avg episode reward: [(0, '19.014')] [2023-02-26 10:24:45,407][10811] Updated weights for policy 0, policy_version 760 (0.0012) [2023-02-26 10:24:48,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 3125248. Throughput: 0: 992.1. Samples: 781972. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:24:48,708][00304] Avg episode reward: [(0, '20.495')] [2023-02-26 10:24:53,705][00304] Fps is (10 sec: 4507.0, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3149824. Throughput: 0: 1002.0. Samples: 785562. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:24:53,708][00304] Avg episode reward: [(0, '21.666')] [2023-02-26 10:24:54,016][10811] Updated weights for policy 0, policy_version 770 (0.0017) [2023-02-26 10:24:58,706][00304] Fps is (10 sec: 4095.5, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3166208. Throughput: 0: 972.7. Samples: 791428. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:24:58,709][00304] Avg episode reward: [(0, '21.770')] [2023-02-26 10:25:03,705][00304] Fps is (10 sec: 3276.7, 60 sec: 3891.2, 300 sec: 3818.3). Total num frames: 3182592. Throughput: 0: 952.6. Samples: 795930. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-26 10:25:03,713][00304] Avg episode reward: [(0, '23.762')] [2023-02-26 10:25:03,729][10798] Saving new best policy, reward=23.762! [2023-02-26 10:25:06,383][10811] Updated weights for policy 0, policy_version 780 (0.0018) [2023-02-26 10:25:08,705][00304] Fps is (10 sec: 3686.8, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 3203072. Throughput: 0: 971.5. Samples: 799012. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-26 10:25:08,710][00304] Avg episode reward: [(0, '23.497')] [2023-02-26 10:25:13,705][00304] Fps is (10 sec: 4505.8, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3227648. Throughput: 0: 999.9. Samples: 806222. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:25:13,706][00304] Avg episode reward: [(0, '22.452')] [2023-02-26 10:25:15,237][10811] Updated weights for policy 0, policy_version 790 (0.0014) [2023-02-26 10:25:18,705][00304] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3244032. Throughput: 0: 958.5. Samples: 811508. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:25:18,708][00304] Avg episode reward: [(0, '21.802')] [2023-02-26 10:25:23,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.3, 300 sec: 3818.3). Total num frames: 3260416. Throughput: 0: 946.7. Samples: 813754. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:25:23,711][00304] Avg episode reward: [(0, '22.312')] [2023-02-26 10:25:28,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 3272704. Throughput: 0: 948.0. Samples: 818366. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:25:28,708][00304] Avg episode reward: [(0, '21.747')] [2023-02-26 10:25:28,784][10811] Updated weights for policy 0, policy_version 800 (0.0023) [2023-02-26 10:25:33,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3790.5). Total num frames: 3289088. Throughput: 0: 909.2. Samples: 822884. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:25:33,709][00304] Avg episode reward: [(0, '22.344')] [2023-02-26 10:25:38,706][00304] Fps is (10 sec: 3276.3, 60 sec: 3618.0, 300 sec: 3790.5). Total num frames: 3305472. Throughput: 0: 881.3. Samples: 825222. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:25:38,709][00304] Avg episode reward: [(0, '22.870')] [2023-02-26 10:25:42,287][10811] Updated weights for policy 0, policy_version 810 (0.0059) [2023-02-26 10:25:43,708][00304] Fps is (10 sec: 3275.8, 60 sec: 3618.1, 300 sec: 3762.7). Total num frames: 3321856. Throughput: 0: 852.8. Samples: 829806. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:25:43,710][00304] Avg episode reward: [(0, '23.196')] [2023-02-26 10:25:48,705][00304] Fps is (10 sec: 3686.9, 60 sec: 3618.1, 300 sec: 3748.9). Total num frames: 3342336. Throughput: 0: 893.4. Samples: 836132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:25:48,708][00304] Avg episode reward: [(0, '24.225')] [2023-02-26 10:25:48,711][10798] Saving new best policy, reward=24.225! [2023-02-26 10:25:51,535][10811] Updated weights for policy 0, policy_version 820 (0.0017) [2023-02-26 10:25:53,705][00304] Fps is (10 sec: 4506.9, 60 sec: 3618.1, 300 sec: 3790.5). Total num frames: 3366912. Throughput: 0: 903.2. Samples: 839654. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:25:53,711][00304] Avg episode reward: [(0, '23.863')] [2023-02-26 10:25:58,707][00304] Fps is (10 sec: 4095.0, 60 sec: 3618.1, 300 sec: 3790.5). Total num frames: 3383296. Throughput: 0: 875.0. Samples: 845598. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:25:58,710][00304] Avg episode reward: [(0, '24.273')] [2023-02-26 10:25:58,722][10798] Saving new best policy, reward=24.273! [2023-02-26 10:26:03,584][10811] Updated weights for policy 0, policy_version 830 (0.0015) [2023-02-26 10:26:03,706][00304] Fps is (10 sec: 3276.5, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 3399680. Throughput: 0: 858.5. Samples: 850142. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:26:03,712][00304] Avg episode reward: [(0, '23.528')] [2023-02-26 10:26:08,705][00304] Fps is (10 sec: 3687.3, 60 sec: 3618.1, 300 sec: 3762.8). Total num frames: 3420160. Throughput: 0: 878.6. Samples: 853292. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:26:08,707][00304] Avg episode reward: [(0, '21.981')] [2023-02-26 10:26:12,079][10811] Updated weights for policy 0, policy_version 840 (0.0021) [2023-02-26 10:26:13,705][00304] Fps is (10 sec: 4506.0, 60 sec: 3618.1, 300 sec: 3790.6). Total num frames: 3444736. Throughput: 0: 938.5. Samples: 860600. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:26:13,707][00304] Avg episode reward: [(0, '21.487')] [2023-02-26 10:26:13,785][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000842_3448832.pth... [2023-02-26 10:26:13,904][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000622_2547712.pth [2023-02-26 10:26:18,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3618.1, 300 sec: 3790.5). Total num frames: 3461120. Throughput: 0: 956.2. Samples: 865914. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:26:18,712][00304] Avg episode reward: [(0, '22.784')] [2023-02-26 10:26:23,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3776.7). Total num frames: 3477504. Throughput: 0: 953.6. Samples: 868132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:26:23,709][00304] Avg episode reward: [(0, '22.340')] [2023-02-26 10:26:24,486][10811] Updated weights for policy 0, policy_version 850 (0.0030) [2023-02-26 10:26:28,705][00304] Fps is (10 sec: 4096.1, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 3502080. Throughput: 0: 984.4. Samples: 874100. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:26:28,708][00304] Avg episode reward: [(0, '22.856')] [2023-02-26 10:26:32,906][10811] Updated weights for policy 0, policy_version 860 (0.0026) [2023-02-26 10:26:33,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3790.6). Total num frames: 3522560. Throughput: 0: 1003.4. Samples: 881286. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:26:33,708][00304] Avg episode reward: [(0, '23.450')] [2023-02-26 10:26:38,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.3, 300 sec: 3790.5). Total num frames: 3538944. Throughput: 0: 984.8. Samples: 883972. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:26:38,712][00304] Avg episode reward: [(0, '21.838')] [2023-02-26 10:26:43,707][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.4, 300 sec: 3776.7). Total num frames: 3555328. Throughput: 0: 952.4. Samples: 888452. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:26:43,710][00304] Avg episode reward: [(0, '20.925')] [2023-02-26 10:26:45,277][10811] Updated weights for policy 0, policy_version 870 (0.0029) [2023-02-26 10:26:48,707][00304] Fps is (10 sec: 4095.0, 60 sec: 3959.3, 300 sec: 3776.6). Total num frames: 3579904. Throughput: 0: 994.5. Samples: 894898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:26:48,709][00304] Avg episode reward: [(0, '21.275')] [2023-02-26 10:26:53,705][00304] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 3600384. Throughput: 0: 1004.6. Samples: 898498. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:26:53,710][00304] Avg episode reward: [(0, '20.815')] [2023-02-26 10:26:53,844][10811] Updated weights for policy 0, policy_version 880 (0.0014) [2023-02-26 10:26:58,705][00304] Fps is (10 sec: 3687.3, 60 sec: 3891.4, 300 sec: 3790.5). Total num frames: 3616768. Throughput: 0: 968.7. Samples: 904192. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:26:58,714][00304] Avg episode reward: [(0, '20.567')] [2023-02-26 10:27:03,706][00304] Fps is (10 sec: 3276.6, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 3633152. Throughput: 0: 952.8. Samples: 908790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:27:03,711][00304] Avg episode reward: [(0, '21.615')] [2023-02-26 10:27:06,140][10811] Updated weights for policy 0, policy_version 890 (0.0025) [2023-02-26 10:27:08,709][00304] Fps is (10 sec: 4094.0, 60 sec: 3959.2, 300 sec: 3762.7). Total num frames: 3657728. Throughput: 0: 974.7. Samples: 911996. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:27:08,714][00304] Avg episode reward: [(0, '22.592')] [2023-02-26 10:27:13,705][00304] Fps is (10 sec: 4506.0, 60 sec: 3891.2, 300 sec: 3790.6). Total num frames: 3678208. Throughput: 0: 1001.4. Samples: 919164. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:27:13,708][00304] Avg episode reward: [(0, '23.654')] [2023-02-26 10:27:15,148][10811] Updated weights for policy 0, policy_version 900 (0.0013) [2023-02-26 10:27:18,707][00304] Fps is (10 sec: 3687.2, 60 sec: 3891.1, 300 sec: 3790.5). Total num frames: 3694592. Throughput: 0: 961.6. Samples: 924560. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:27:18,713][00304] Avg episode reward: [(0, '23.070')] [2023-02-26 10:27:23,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 3710976. Throughput: 0: 950.7. Samples: 926752. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:27:23,710][00304] Avg episode reward: [(0, '23.794')] [2023-02-26 10:27:26,893][10811] Updated weights for policy 0, policy_version 910 (0.0016) [2023-02-26 10:27:28,705][00304] Fps is (10 sec: 4097.0, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3735552. Throughput: 0: 982.7. Samples: 932674. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:27:28,713][00304] Avg episode reward: [(0, '24.272')] [2023-02-26 10:27:33,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 3760128. Throughput: 0: 999.6. Samples: 939878. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:27:33,707][00304] Avg episode reward: [(0, '24.563')] [2023-02-26 10:27:33,716][10798] Saving new best policy, reward=24.563! [2023-02-26 10:27:36,126][10811] Updated weights for policy 0, policy_version 920 (0.0013) [2023-02-26 10:27:38,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3772416. Throughput: 0: 977.3. Samples: 942478. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-26 10:27:38,707][00304] Avg episode reward: [(0, '23.800')] [2023-02-26 10:27:43,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3788800. Throughput: 0: 952.3. Samples: 947046. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:27:43,713][00304] Avg episode reward: [(0, '24.264')] [2023-02-26 10:27:47,684][10811] Updated weights for policy 0, policy_version 930 (0.0014) [2023-02-26 10:27:48,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3891.4, 300 sec: 3832.2). Total num frames: 3813376. Throughput: 0: 990.2. Samples: 953348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:27:48,708][00304] Avg episode reward: [(0, '23.326')] [2023-02-26 10:27:53,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 3837952. Throughput: 0: 999.2. Samples: 956954. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-26 10:27:53,714][00304] Avg episode reward: [(0, '23.495')] [2023-02-26 10:27:57,652][10811] Updated weights for policy 0, policy_version 940 (0.0017) [2023-02-26 10:27:58,705][00304] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3860.0). Total num frames: 3850240. Throughput: 0: 965.6. Samples: 962616. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:27:58,708][00304] Avg episode reward: [(0, '24.289')] [2023-02-26 10:28:03,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3866624. Throughput: 0: 946.7. Samples: 967158. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:28:03,712][00304] Avg episode reward: [(0, '25.362')] [2023-02-26 10:28:03,723][10798] Saving new best policy, reward=25.362! [2023-02-26 10:28:08,620][10811] Updated weights for policy 0, policy_version 950 (0.0019) [2023-02-26 10:28:08,705][00304] Fps is (10 sec: 4095.9, 60 sec: 3891.5, 300 sec: 3832.2). Total num frames: 3891200. Throughput: 0: 965.6. Samples: 970206. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:28:08,713][00304] Avg episode reward: [(0, '24.897')] [2023-02-26 10:28:13,705][00304] Fps is (10 sec: 4915.2, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 3915776. Throughput: 0: 995.4. Samples: 977466. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-26 10:28:13,707][00304] Avg episode reward: [(0, '23.066')] [2023-02-26 10:28:13,724][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000956_3915776.pth... [2023-02-26 10:28:13,843][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000731_2994176.pth [2023-02-26 10:28:18,705][00304] Fps is (10 sec: 3686.5, 60 sec: 3891.4, 300 sec: 3860.0). Total num frames: 3928064. Throughput: 0: 957.2. Samples: 982950. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-26 10:28:18,708][00304] Avg episode reward: [(0, '21.745')] [2023-02-26 10:28:18,815][10811] Updated weights for policy 0, policy_version 960 (0.0022) [2023-02-26 10:28:23,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3944448. Throughput: 0: 949.4. Samples: 985200. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:28:23,708][00304] Avg episode reward: [(0, '20.301')] [2023-02-26 10:28:28,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 3960832. Throughput: 0: 948.6. Samples: 989732. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-26 10:28:28,711][00304] Avg episode reward: [(0, '19.722')] [2023-02-26 10:28:32,462][10811] Updated weights for policy 0, policy_version 970 (0.0045) [2023-02-26 10:28:33,705][00304] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3804.4). Total num frames: 3977216. Throughput: 0: 910.0. Samples: 994300. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-26 10:28:33,709][00304] Avg episode reward: [(0, '19.654')] [2023-02-26 10:28:38,705][00304] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3804.4). Total num frames: 3989504. Throughput: 0: 879.4. Samples: 996528. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-26 10:28:38,707][00304] Avg episode reward: [(0, '19.283')] [2023-02-26 10:28:43,262][10798] Stopping Batcher_0... [2023-02-26 10:28:43,265][10798] Loop batcher_evt_loop terminating... [2023-02-26 10:28:43,263][00304] Component Batcher_0 stopped! [2023-02-26 10:28:43,272][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2023-02-26 10:28:43,345][00304] Component RolloutWorker_w5 stopped! [2023-02-26 10:28:43,352][10818] Stopping RolloutWorker_w5... [2023-02-26 10:28:43,353][10818] Loop rollout_proc5_evt_loop terminating... [2023-02-26 10:28:43,375][10817] Stopping RolloutWorker_w4... [2023-02-26 10:28:43,379][10817] Loop rollout_proc4_evt_loop terminating... [2023-02-26 10:28:43,373][00304] Component RolloutWorker_w1 stopped! [2023-02-26 10:28:43,380][00304] Component RolloutWorker_w4 stopped! [2023-02-26 10:28:43,383][10813] Stopping RolloutWorker_w1... [2023-02-26 10:28:43,385][10813] Loop rollout_proc1_evt_loop terminating... [2023-02-26 10:28:43,393][10811] Weights refcount: 2 0 [2023-02-26 10:28:43,411][10815] Stopping RolloutWorker_w0... [2023-02-26 10:28:43,411][10815] Loop rollout_proc0_evt_loop terminating... [2023-02-26 10:28:43,412][00304] Component RolloutWorker_w0 stopped! [2023-02-26 10:28:43,417][10811] Stopping InferenceWorker_p0-w0... [2023-02-26 10:28:43,418][10811] Loop inference_proc0-0_evt_loop terminating... [2023-02-26 10:28:43,415][00304] Component RolloutWorker_w3 stopped! [2023-02-26 10:28:43,411][10816] Stopping RolloutWorker_w3... [2023-02-26 10:28:43,421][10816] Loop rollout_proc3_evt_loop terminating... [2023-02-26 10:28:43,420][00304] Component InferenceWorker_p0-w0 stopped! [2023-02-26 10:28:43,439][00304] Component RolloutWorker_w7 stopped! [2023-02-26 10:28:43,444][10820] Stopping RolloutWorker_w7... [2023-02-26 10:28:43,445][10820] Loop rollout_proc7_evt_loop terminating... [2023-02-26 10:28:43,450][10814] Stopping RolloutWorker_w2... [2023-02-26 10:28:43,450][10814] Loop rollout_proc2_evt_loop terminating... [2023-02-26 10:28:43,455][10819] Stopping RolloutWorker_w6... [2023-02-26 10:28:43,456][10819] Loop rollout_proc6_evt_loop terminating... [2023-02-26 10:28:43,450][00304] Component RolloutWorker_w2 stopped! [2023-02-26 10:28:43,456][00304] Component RolloutWorker_w6 stopped! [2023-02-26 10:28:43,504][10798] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000842_3448832.pth [2023-02-26 10:28:43,525][10798] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2023-02-26 10:28:43,815][00304] Component LearnerWorker_p0 stopped! [2023-02-26 10:28:43,822][00304] Waiting for process learner_proc0 to stop... [2023-02-26 10:28:43,826][10798] Stopping LearnerWorker_p0... [2023-02-26 10:28:43,827][10798] Loop learner_proc0_evt_loop terminating... [2023-02-26 10:28:45,715][00304] Waiting for process inference_proc0-0 to join... [2023-02-26 10:28:46,135][00304] Waiting for process rollout_proc0 to join... [2023-02-26 10:28:46,462][00304] Waiting for process rollout_proc1 to join... [2023-02-26 10:28:46,464][00304] Waiting for process rollout_proc2 to join... [2023-02-26 10:28:46,465][00304] Waiting for process rollout_proc3 to join... [2023-02-26 10:28:46,466][00304] Waiting for process rollout_proc4 to join... [2023-02-26 10:28:46,467][00304] Waiting for process rollout_proc5 to join... [2023-02-26 10:28:46,469][00304] Waiting for process rollout_proc6 to join... [2023-02-26 10:28:46,470][00304] Waiting for process rollout_proc7 to join... [2023-02-26 10:28:46,471][00304] Batcher 0 profile tree view: batching: 25.6426, releasing_batches: 0.0233 [2023-02-26 10:28:46,473][00304] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 515.4273 update_model: 7.7167 weight_update: 0.0020 one_step: 0.0160 handle_policy_step: 512.5966 deserialize: 14.5257, stack: 3.1535, obs_to_device_normalize: 113.7392, forward: 246.9027, send_messages: 26.2634 prepare_outputs: 82.6558 to_cpu: 51.8260 [2023-02-26 10:28:46,474][00304] Learner 0 profile tree view: misc: 0.0054, prepare_batch: 15.6405 train: 75.6909 epoch_init: 0.0126, minibatch_init: 0.0087, losses_postprocess: 0.6636, kl_divergence: 0.5817, after_optimizer: 33.0682 calculate_losses: 26.6977 losses_init: 0.0033, forward_head: 1.7413, bptt_initial: 17.7023, tail: 1.0487, advantages_returns: 0.3586, losses: 3.4147 bptt: 2.1564 bptt_forward_core: 2.0685 update: 14.0975 clip: 1.3532 [2023-02-26 10:28:46,475][00304] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.2947, enqueue_policy_requests: 138.2589, env_step: 812.3938, overhead: 20.3042, complete_rollouts: 6.7712 save_policy_outputs: 19.4772 split_output_tensors: 9.6457 [2023-02-26 10:28:46,479][00304] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.3469, enqueue_policy_requests: 137.8682, env_step: 812.1064, overhead: 19.4119, complete_rollouts: 6.7750 save_policy_outputs: 19.7513 split_output_tensors: 9.5300 [2023-02-26 10:28:46,480][00304] Loop Runner_EvtLoop terminating... [2023-02-26 10:28:46,482][00304] Runner profile tree view: main_loop: 1107.7311 [2023-02-26 10:28:46,485][00304] Collected {0: 4005888}, FPS: 3616.3 [2023-02-26 10:28:46,639][00304] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-02-26 10:28:46,645][00304] Overriding arg 'num_workers' with value 1 passed from command line [2023-02-26 10:28:46,647][00304] Adding new argument 'no_render'=True that is not in the saved config file! [2023-02-26 10:28:46,648][00304] Adding new argument 'save_video'=True that is not in the saved config file! [2023-02-26 10:28:46,650][00304] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-02-26 10:28:46,651][00304] Adding new argument 'video_name'=None that is not in the saved config file! [2023-02-26 10:28:46,652][00304] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2023-02-26 10:28:46,653][00304] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-02-26 10:28:46,656][00304] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2023-02-26 10:28:46,658][00304] Adding new argument 'hf_repository'=None that is not in the saved config file! [2023-02-26 10:28:46,659][00304] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-02-26 10:28:46,661][00304] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-02-26 10:28:46,662][00304] Adding new argument 'train_script'=None that is not in the saved config file! [2023-02-26 10:28:46,664][00304] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-02-26 10:28:46,666][00304] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-02-26 10:28:46,698][00304] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-26 10:28:46,701][00304] RunningMeanStd input shape: (3, 72, 128) [2023-02-26 10:28:46,703][00304] RunningMeanStd input shape: (1,) [2023-02-26 10:28:46,720][00304] ConvEncoder: input_channels=3 [2023-02-26 10:28:47,377][00304] Conv encoder output size: 512 [2023-02-26 10:28:47,379][00304] Policy head output size: 512 [2023-02-26 10:28:49,785][00304] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2023-02-26 10:28:51,033][00304] Num frames 100... [2023-02-26 10:28:51,153][00304] Num frames 200... [2023-02-26 10:28:51,276][00304] Num frames 300... [2023-02-26 10:28:51,388][00304] Num frames 400... [2023-02-26 10:28:51,502][00304] Num frames 500... [2023-02-26 10:28:51,621][00304] Num frames 600... [2023-02-26 10:28:51,734][00304] Num frames 700... [2023-02-26 10:28:51,853][00304] Num frames 800... [2023-02-26 10:28:51,967][00304] Num frames 900... [2023-02-26 10:28:52,128][00304] Avg episode rewards: #0: 24.920, true rewards: #0: 9.920 [2023-02-26 10:28:52,130][00304] Avg episode reward: 24.920, avg true_objective: 9.920 [2023-02-26 10:28:52,143][00304] Num frames 1000... [2023-02-26 10:28:52,261][00304] Num frames 1100... [2023-02-26 10:28:52,385][00304] Num frames 1200... [2023-02-26 10:28:52,496][00304] Num frames 1300... [2023-02-26 10:28:52,621][00304] Num frames 1400... [2023-02-26 10:28:52,736][00304] Num frames 1500... [2023-02-26 10:28:52,798][00304] Avg episode rewards: #0: 17.520, true rewards: #0: 7.520 [2023-02-26 10:28:52,800][00304] Avg episode reward: 17.520, avg true_objective: 7.520 [2023-02-26 10:28:52,908][00304] Num frames 1600... [2023-02-26 10:28:53,016][00304] Num frames 1700... [2023-02-26 10:28:53,135][00304] Num frames 1800... [2023-02-26 10:28:53,256][00304] Num frames 1900... [2023-02-26 10:28:53,373][00304] Num frames 2000... [2023-02-26 10:28:53,488][00304] Num frames 2100... [2023-02-26 10:28:53,608][00304] Num frames 2200... [2023-02-26 10:28:53,723][00304] Num frames 2300... [2023-02-26 10:28:53,846][00304] Num frames 2400... [2023-02-26 10:28:53,959][00304] Num frames 2500... [2023-02-26 10:28:54,082][00304] Num frames 2600... [2023-02-26 10:28:54,247][00304] Num frames 2700... [2023-02-26 10:28:54,446][00304] Avg episode rewards: #0: 23.280, true rewards: #0: 9.280 [2023-02-26 10:28:54,448][00304] Avg episode reward: 23.280, avg true_objective: 9.280 [2023-02-26 10:28:54,475][00304] Num frames 2800... [2023-02-26 10:28:54,637][00304] Num frames 2900... [2023-02-26 10:28:54,797][00304] Num frames 3000... [2023-02-26 10:28:54,954][00304] Num frames 3100... [2023-02-26 10:28:55,115][00304] Num frames 3200... [2023-02-26 10:28:55,279][00304] Num frames 3300... [2023-02-26 10:28:55,455][00304] Num frames 3400... [2023-02-26 10:28:55,620][00304] Num frames 3500... [2023-02-26 10:28:55,821][00304] Avg episode rewards: #0: 21.960, true rewards: #0: 8.960 [2023-02-26 10:28:55,825][00304] Avg episode reward: 21.960, avg true_objective: 8.960 [2023-02-26 10:28:55,864][00304] Num frames 3600... [2023-02-26 10:28:56,036][00304] Num frames 3700... [2023-02-26 10:28:56,206][00304] Num frames 3800... [2023-02-26 10:28:56,366][00304] Num frames 3900... [2023-02-26 10:28:56,529][00304] Num frames 4000... [2023-02-26 10:28:56,691][00304] Num frames 4100... [2023-02-26 10:28:56,851][00304] Num frames 4200... [2023-02-26 10:28:57,006][00304] Num frames 4300... [2023-02-26 10:28:57,174][00304] Num frames 4400... [2023-02-26 10:28:57,335][00304] Num frames 4500... [2023-02-26 10:28:57,499][00304] Num frames 4600... [2023-02-26 10:28:57,667][00304] Num frames 4700... [2023-02-26 10:28:57,783][00304] Num frames 4800... [2023-02-26 10:28:57,892][00304] Avg episode rewards: #0: 23.076, true rewards: #0: 9.676 [2023-02-26 10:28:57,895][00304] Avg episode reward: 23.076, avg true_objective: 9.676 [2023-02-26 10:28:57,964][00304] Num frames 4900... [2023-02-26 10:28:58,082][00304] Num frames 5000... [2023-02-26 10:28:58,210][00304] Num frames 5100... [2023-02-26 10:28:58,333][00304] Num frames 5200... [2023-02-26 10:28:58,450][00304] Avg episode rewards: #0: 20.257, true rewards: #0: 8.757 [2023-02-26 10:28:58,452][00304] Avg episode reward: 20.257, avg true_objective: 8.757 [2023-02-26 10:28:58,508][00304] Num frames 5300... [2023-02-26 10:28:58,629][00304] Num frames 5400... [2023-02-26 10:28:58,745][00304] Num frames 5500... [2023-02-26 10:28:58,864][00304] Num frames 5600... [2023-02-26 10:28:58,977][00304] Num frames 5700... [2023-02-26 10:28:59,071][00304] Avg episode rewards: #0: 18.334, true rewards: #0: 8.191 [2023-02-26 10:28:59,072][00304] Avg episode reward: 18.334, avg true_objective: 8.191 [2023-02-26 10:28:59,149][00304] Num frames 5800... [2023-02-26 10:28:59,269][00304] Num frames 5900... [2023-02-26 10:28:59,427][00304] Avg episode rewards: #0: 16.363, true rewards: #0: 7.487 [2023-02-26 10:28:59,429][00304] Avg episode reward: 16.363, avg true_objective: 7.487 [2023-02-26 10:28:59,449][00304] Num frames 6000... [2023-02-26 10:28:59,563][00304] Num frames 6100... [2023-02-26 10:28:59,684][00304] Num frames 6200... [2023-02-26 10:28:59,800][00304] Num frames 6300... [2023-02-26 10:28:59,919][00304] Num frames 6400... [2023-02-26 10:29:00,033][00304] Num frames 6500... [2023-02-26 10:29:00,152][00304] Num frames 6600... [2023-02-26 10:29:00,267][00304] Num frames 6700... [2023-02-26 10:29:00,384][00304] Num frames 6800... [2023-02-26 10:29:00,504][00304] Num frames 6900... [2023-02-26 10:29:00,622][00304] Num frames 7000... [2023-02-26 10:29:00,738][00304] Num frames 7100... [2023-02-26 10:29:00,851][00304] Num frames 7200... [2023-02-26 10:29:00,969][00304] Num frames 7300... [2023-02-26 10:29:01,082][00304] Num frames 7400... [2023-02-26 10:29:01,203][00304] Num frames 7500... [2023-02-26 10:29:01,319][00304] Num frames 7600... [2023-02-26 10:29:01,436][00304] Num frames 7700... [2023-02-26 10:29:01,555][00304] Num frames 7800... [2023-02-26 10:29:01,679][00304] Num frames 7900... [2023-02-26 10:29:01,769][00304] Avg episode rewards: #0: 19.477, true rewards: #0: 8.810 [2023-02-26 10:29:01,771][00304] Avg episode reward: 19.477, avg true_objective: 8.810 [2023-02-26 10:29:01,854][00304] Num frames 8000... [2023-02-26 10:29:01,973][00304] Num frames 8100... [2023-02-26 10:29:02,086][00304] Num frames 8200... [2023-02-26 10:29:02,208][00304] Num frames 8300... [2023-02-26 10:29:02,325][00304] Num frames 8400... [2023-02-26 10:29:02,441][00304] Num frames 8500... [2023-02-26 10:29:02,563][00304] Num frames 8600... [2023-02-26 10:29:02,684][00304] Num frames 8700... [2023-02-26 10:29:02,802][00304] Num frames 8800... [2023-02-26 10:29:02,924][00304] Num frames 8900... [2023-02-26 10:29:03,042][00304] Num frames 9000... [2023-02-26 10:29:03,120][00304] Avg episode rewards: #0: 19.717, true rewards: #0: 9.017 [2023-02-26 10:29:03,122][00304] Avg episode reward: 19.717, avg true_objective: 9.017 [2023-02-26 10:29:56,992][00304] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2023-02-26 10:29:57,292][00304] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-02-26 10:29:57,299][00304] Overriding arg 'num_workers' with value 1 passed from command line [2023-02-26 10:29:57,300][00304] Adding new argument 'no_render'=True that is not in the saved config file! [2023-02-26 10:29:57,301][00304] Adding new argument 'save_video'=True that is not in the saved config file! [2023-02-26 10:29:57,302][00304] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-02-26 10:29:57,304][00304] Adding new argument 'video_name'=None that is not in the saved config file! [2023-02-26 10:29:57,305][00304] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2023-02-26 10:29:57,306][00304] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-02-26 10:29:57,307][00304] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2023-02-26 10:29:57,308][00304] Adding new argument 'hf_repository'='RegisGraptin/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2023-02-26 10:29:57,310][00304] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-02-26 10:29:57,311][00304] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-02-26 10:29:57,312][00304] Adding new argument 'train_script'=None that is not in the saved config file! [2023-02-26 10:29:57,313][00304] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-02-26 10:29:57,314][00304] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-02-26 10:29:57,341][00304] RunningMeanStd input shape: (3, 72, 128) [2023-02-26 10:29:57,350][00304] RunningMeanStd input shape: (1,) [2023-02-26 10:29:57,368][00304] ConvEncoder: input_channels=3 [2023-02-26 10:29:57,425][00304] Conv encoder output size: 512 [2023-02-26 10:29:57,427][00304] Policy head output size: 512 [2023-02-26 10:29:57,457][00304] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2023-02-26 10:29:58,143][00304] Num frames 100... [2023-02-26 10:29:58,302][00304] Num frames 200... [2023-02-26 10:29:58,453][00304] Num frames 300... [2023-02-26 10:29:58,601][00304] Num frames 400... [2023-02-26 10:29:58,750][00304] Num frames 500... [2023-02-26 10:29:58,898][00304] Num frames 600... [2023-02-26 10:29:59,061][00304] Num frames 700... [2023-02-26 10:29:59,232][00304] Num frames 800... [2023-02-26 10:29:59,382][00304] Num frames 900... [2023-02-26 10:29:59,531][00304] Num frames 1000... [2023-02-26 10:29:59,689][00304] Num frames 1100... [2023-02-26 10:29:59,832][00304] Avg episode rewards: #0: 26.520, true rewards: #0: 11.520 [2023-02-26 10:29:59,834][00304] Avg episode reward: 26.520, avg true_objective: 11.520 [2023-02-26 10:29:59,907][00304] Num frames 1200... [2023-02-26 10:30:00,079][00304] Num frames 1300... [2023-02-26 10:30:00,242][00304] Num frames 1400... [2023-02-26 10:30:00,405][00304] Num frames 1500... [2023-02-26 10:30:00,579][00304] Num frames 1600... [2023-02-26 10:30:00,772][00304] Num frames 1700... [2023-02-26 10:30:00,954][00304] Num frames 1800... [2023-02-26 10:30:01,134][00304] Num frames 1900... [2023-02-26 10:30:01,303][00304] Num frames 2000... [2023-02-26 10:30:01,463][00304] Num frames 2100... [2023-02-26 10:30:01,630][00304] Num frames 2200... [2023-02-26 10:30:01,802][00304] Num frames 2300... [2023-02-26 10:30:01,957][00304] Num frames 2400... [2023-02-26 10:30:02,116][00304] Num frames 2500... [2023-02-26 10:30:02,272][00304] Num frames 2600... [2023-02-26 10:30:02,410][00304] Avg episode rewards: #0: 30.365, true rewards: #0: 13.365 [2023-02-26 10:30:02,412][00304] Avg episode reward: 30.365, avg true_objective: 13.365 [2023-02-26 10:30:02,445][00304] Num frames 2700... [2023-02-26 10:30:02,560][00304] Num frames 2800... [2023-02-26 10:30:02,672][00304] Num frames 2900... [2023-02-26 10:30:02,783][00304] Num frames 3000... [2023-02-26 10:30:02,904][00304] Num frames 3100... [2023-02-26 10:30:03,015][00304] Num frames 3200... [2023-02-26 10:30:03,134][00304] Num frames 3300... [2023-02-26 10:30:03,256][00304] Num frames 3400... [2023-02-26 10:30:03,371][00304] Num frames 3500... [2023-02-26 10:30:03,483][00304] Num frames 3600... [2023-02-26 10:30:03,603][00304] Num frames 3700... [2023-02-26 10:30:03,721][00304] Num frames 3800... [2023-02-26 10:30:03,831][00304] Num frames 3900... [2023-02-26 10:30:03,992][00304] Num frames 4000... [2023-02-26 10:30:04,156][00304] Num frames 4100... [2023-02-26 10:30:04,324][00304] Num frames 4200... [2023-02-26 10:30:04,493][00304] Num frames 4300... [2023-02-26 10:30:04,659][00304] Num frames 4400... [2023-02-26 10:30:04,819][00304] Num frames 4500... [2023-02-26 10:30:04,992][00304] Num frames 4600... [2023-02-26 10:30:05,169][00304] Num frames 4700... [2023-02-26 10:30:05,282][00304] Avg episode rewards: #0: 37.436, true rewards: #0: 15.770 [2023-02-26 10:30:05,284][00304] Avg episode reward: 37.436, avg true_objective: 15.770 [2023-02-26 10:30:05,392][00304] Num frames 4800... [2023-02-26 10:30:05,556][00304] Num frames 4900... [2023-02-26 10:30:05,718][00304] Num frames 5000... [2023-02-26 10:30:05,874][00304] Num frames 5100... [2023-02-26 10:30:06,034][00304] Num frames 5200... [2023-02-26 10:30:06,202][00304] Num frames 5300... [2023-02-26 10:30:06,380][00304] Num frames 5400... [2023-02-26 10:30:06,550][00304] Num frames 5500... [2023-02-26 10:30:06,714][00304] Num frames 5600... [2023-02-26 10:30:06,888][00304] Num frames 5700... [2023-02-26 10:30:07,052][00304] Num frames 5800... [2023-02-26 10:30:07,216][00304] Num frames 5900... [2023-02-26 10:30:07,390][00304] Num frames 6000... [2023-02-26 10:30:07,506][00304] Num frames 6100... [2023-02-26 10:30:07,629][00304] Num frames 6200... [2023-02-26 10:30:07,744][00304] Num frames 6300... [2023-02-26 10:30:07,858][00304] Num frames 6400... [2023-02-26 10:30:07,972][00304] Num frames 6500... [2023-02-26 10:30:08,089][00304] Num frames 6600... [2023-02-26 10:30:08,210][00304] Num frames 6700... [2023-02-26 10:30:08,329][00304] Num frames 6800... [2023-02-26 10:30:08,421][00304] Avg episode rewards: #0: 41.577, true rewards: #0: 17.078 [2023-02-26 10:30:08,423][00304] Avg episode reward: 41.577, avg true_objective: 17.078 [2023-02-26 10:30:08,504][00304] Num frames 6900... [2023-02-26 10:30:08,619][00304] Num frames 7000... [2023-02-26 10:30:08,732][00304] Num frames 7100... [2023-02-26 10:30:08,844][00304] Num frames 7200... [2023-02-26 10:30:08,957][00304] Num frames 7300... [2023-02-26 10:30:09,070][00304] Num frames 7400... [2023-02-26 10:30:09,185][00304] Num frames 7500... [2023-02-26 10:30:09,297][00304] Num frames 7600... [2023-02-26 10:30:09,414][00304] Num frames 7700... [2023-02-26 10:30:09,525][00304] Num frames 7800... [2023-02-26 10:30:09,679][00304] Avg episode rewards: #0: 37.974, true rewards: #0: 15.774 [2023-02-26 10:30:09,681][00304] Avg episode reward: 37.974, avg true_objective: 15.774 [2023-02-26 10:30:09,699][00304] Num frames 7900... [2023-02-26 10:30:09,808][00304] Num frames 8000... [2023-02-26 10:30:09,918][00304] Num frames 8100... [2023-02-26 10:30:10,030][00304] Num frames 8200... [2023-02-26 10:30:10,143][00304] Num frames 8300... [2023-02-26 10:30:10,256][00304] Num frames 8400... [2023-02-26 10:30:10,376][00304] Num frames 8500... [2023-02-26 10:30:10,488][00304] Num frames 8600... [2023-02-26 10:30:10,601][00304] Num frames 8700... [2023-02-26 10:30:10,727][00304] Avg episode rewards: #0: 34.936, true rewards: #0: 14.603 [2023-02-26 10:30:10,729][00304] Avg episode reward: 34.936, avg true_objective: 14.603 [2023-02-26 10:30:10,772][00304] Num frames 8800... [2023-02-26 10:30:10,886][00304] Num frames 8900... [2023-02-26 10:30:10,998][00304] Num frames 9000... [2023-02-26 10:30:11,111][00304] Num frames 9100... [2023-02-26 10:30:11,229][00304] Num frames 9200... [2023-02-26 10:30:11,340][00304] Num frames 9300... [2023-02-26 10:30:11,457][00304] Num frames 9400... [2023-02-26 10:30:11,569][00304] Num frames 9500... [2023-02-26 10:30:11,685][00304] Num frames 9600... [2023-02-26 10:30:11,798][00304] Num frames 9700... [2023-02-26 10:30:11,910][00304] Num frames 9800... [2023-02-26 10:30:12,020][00304] Num frames 9900... [2023-02-26 10:30:12,135][00304] Num frames 10000... [2023-02-26 10:30:12,250][00304] Num frames 10100... [2023-02-26 10:30:12,364][00304] Num frames 10200... [2023-02-26 10:30:12,485][00304] Num frames 10300... [2023-02-26 10:30:12,607][00304] Avg episode rewards: #0: 35.510, true rewards: #0: 14.796 [2023-02-26 10:30:12,609][00304] Avg episode reward: 35.510, avg true_objective: 14.796 [2023-02-26 10:30:12,659][00304] Num frames 10400... [2023-02-26 10:30:12,770][00304] Num frames 10500... [2023-02-26 10:30:12,882][00304] Num frames 10600... [2023-02-26 10:30:12,993][00304] Num frames 10700... [2023-02-26 10:30:13,104][00304] Num frames 10800... [2023-02-26 10:30:13,220][00304] Num frames 10900... [2023-02-26 10:30:13,337][00304] Num frames 11000... [2023-02-26 10:30:13,456][00304] Num frames 11100... [2023-02-26 10:30:13,567][00304] Num frames 11200... [2023-02-26 10:30:13,680][00304] Num frames 11300... [2023-02-26 10:30:13,791][00304] Num frames 11400... [2023-02-26 10:30:13,912][00304] Num frames 11500... [2023-02-26 10:30:14,023][00304] Num frames 11600... [2023-02-26 10:30:14,135][00304] Num frames 11700... [2023-02-26 10:30:14,194][00304] Avg episode rewards: #0: 34.501, true rewards: #0: 14.626 [2023-02-26 10:30:14,196][00304] Avg episode reward: 34.501, avg true_objective: 14.626 [2023-02-26 10:30:14,313][00304] Num frames 11800... [2023-02-26 10:30:14,434][00304] Num frames 11900... [2023-02-26 10:30:14,547][00304] Num frames 12000... [2023-02-26 10:30:14,663][00304] Num frames 12100... [2023-02-26 10:30:14,775][00304] Num frames 12200... [2023-02-26 10:30:14,888][00304] Num frames 12300... [2023-02-26 10:30:14,999][00304] Num frames 12400... [2023-02-26 10:30:15,062][00304] Avg episode rewards: #0: 32.561, true rewards: #0: 13.783 [2023-02-26 10:30:15,063][00304] Avg episode reward: 32.561, avg true_objective: 13.783 [2023-02-26 10:30:15,180][00304] Num frames 12500... [2023-02-26 10:30:15,296][00304] Num frames 12600... [2023-02-26 10:30:15,408][00304] Num frames 12700... [2023-02-26 10:30:15,528][00304] Num frames 12800... [2023-02-26 10:30:15,643][00304] Num frames 12900... [2023-02-26 10:30:15,754][00304] Num frames 13000... [2023-02-26 10:30:15,864][00304] Num frames 13100... [2023-02-26 10:30:15,975][00304] Num frames 13200... [2023-02-26 10:30:16,087][00304] Num frames 13300... [2023-02-26 10:30:16,206][00304] Num frames 13400... [2023-02-26 10:30:16,300][00304] Avg episode rewards: #0: 31.229, true rewards: #0: 13.429 [2023-02-26 10:30:16,301][00304] Avg episode reward: 31.229, avg true_objective: 13.429 [2023-02-26 10:31:38,407][00304] Replay video saved to /content/train_dir/default_experiment/replay.mp4!