diff --git "a/sf_log.txt" "b/sf_log.txt" --- "a/sf_log.txt" +++ "b/sf_log.txt" @@ -1,50 +1,50 @@ -[2023-02-22 19:21:17,455][00804] Saving configuration to /content/train_dir/default_experiment/config.json... -[2023-02-22 19:21:17,462][00804] Rollout worker 0 uses device cpu -[2023-02-22 19:21:17,463][00804] Rollout worker 1 uses device cpu -[2023-02-22 19:21:17,465][00804] Rollout worker 2 uses device cpu -[2023-02-22 19:21:17,469][00804] Rollout worker 3 uses device cpu -[2023-02-22 19:21:17,472][00804] Rollout worker 4 uses device cpu -[2023-02-22 19:21:17,476][00804] Rollout worker 5 uses device cpu -[2023-02-22 19:21:17,477][00804] Rollout worker 6 uses device cpu -[2023-02-22 19:21:17,480][00804] Rollout worker 7 uses device cpu -[2023-02-22 19:21:17,947][00804] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-22 19:21:17,958][00804] InferenceWorker_p0-w0: min num requests: 2 -[2023-02-22 19:21:18,081][00804] Starting all processes... -[2023-02-22 19:21:18,096][00804] Starting process learner_proc0 -[2023-02-22 19:21:18,266][00804] Starting all processes... -[2023-02-22 19:21:18,307][00804] Starting process inference_proc0-0 -[2023-02-22 19:21:18,310][00804] Starting process rollout_proc0 -[2023-02-22 19:21:18,310][00804] Starting process rollout_proc1 -[2023-02-22 19:21:18,310][00804] Starting process rollout_proc2 -[2023-02-22 19:21:18,310][00804] Starting process rollout_proc3 -[2023-02-22 19:21:18,310][00804] Starting process rollout_proc4 -[2023-02-22 19:21:18,348][00804] Starting process rollout_proc5 -[2023-02-22 19:21:18,348][00804] Starting process rollout_proc6 -[2023-02-22 19:21:18,348][00804] Starting process rollout_proc7 -[2023-02-22 19:21:30,535][11058] Worker 3 uses CPU cores [1] -[2023-02-22 19:21:30,604][11041] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-22 19:21:30,605][11041] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2023-02-22 19:21:30,829][11059] Worker 2 uses CPU cores [0] -[2023-02-22 19:21:30,848][11063] Worker 7 uses CPU cores [1] -[2023-02-22 19:21:31,203][11061] Worker 5 uses CPU cores [1] -[2023-02-22 19:21:31,278][11056] Worker 0 uses CPU cores [0] -[2023-02-22 19:21:31,296][11057] Worker 1 uses CPU cores [1] -[2023-02-22 19:21:31,390][11055] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-22 19:21:31,392][11055] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2023-02-22 19:21:31,532][11062] Worker 6 uses CPU cores [0] -[2023-02-22 19:21:31,533][11060] Worker 4 uses CPU cores [0] -[2023-02-22 19:21:31,682][11055] Num visible devices: 1 -[2023-02-22 19:21:31,686][11041] Num visible devices: 1 -[2023-02-22 19:21:31,698][11041] Starting seed is not provided -[2023-02-22 19:21:31,698][11041] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-22 19:21:31,699][11041] Initializing actor-critic model on device cuda:0 -[2023-02-22 19:21:31,700][11041] RunningMeanStd input shape: (3, 72, 128) -[2023-02-22 19:21:31,702][11041] RunningMeanStd input shape: (1,) -[2023-02-22 19:21:31,721][11041] ConvEncoder: input_channels=3 -[2023-02-22 19:21:32,133][11041] Conv encoder output size: 512 -[2023-02-22 19:21:32,135][11041] Policy head output size: 512 -[2023-02-22 19:21:32,255][11041] Created Actor Critic model with architecture: -[2023-02-22 19:21:32,257][11041] ActorCriticSharedWeights( +[2023-02-23 20:07:09,899][00631] Saving configuration to /content/train_dir/default_experiment/config.json... +[2023-02-23 20:07:09,904][00631] Rollout worker 0 uses device cpu +[2023-02-23 20:07:09,905][00631] Rollout worker 1 uses device cpu +[2023-02-23 20:07:09,909][00631] Rollout worker 2 uses device cpu +[2023-02-23 20:07:09,910][00631] Rollout worker 3 uses device cpu +[2023-02-23 20:07:09,914][00631] Rollout worker 4 uses device cpu +[2023-02-23 20:07:09,917][00631] Rollout worker 5 uses device cpu +[2023-02-23 20:07:09,918][00631] Rollout worker 6 uses device cpu +[2023-02-23 20:07:09,922][00631] Rollout worker 7 uses device cpu +[2023-02-23 20:07:10,107][00631] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-23 20:07:10,109][00631] InferenceWorker_p0-w0: min num requests: 2 +[2023-02-23 20:07:10,141][00631] Starting all processes... +[2023-02-23 20:07:10,143][00631] Starting process learner_proc0 +[2023-02-23 20:07:10,205][00631] Starting all processes... +[2023-02-23 20:07:10,214][00631] Starting process inference_proc0-0 +[2023-02-23 20:07:10,215][00631] Starting process rollout_proc0 +[2023-02-23 20:07:10,215][00631] Starting process rollout_proc1 +[2023-02-23 20:07:10,215][00631] Starting process rollout_proc2 +[2023-02-23 20:07:10,215][00631] Starting process rollout_proc3 +[2023-02-23 20:07:10,215][00631] Starting process rollout_proc4 +[2023-02-23 20:07:10,215][00631] Starting process rollout_proc5 +[2023-02-23 20:07:10,215][00631] Starting process rollout_proc6 +[2023-02-23 20:07:10,215][00631] Starting process rollout_proc7 +[2023-02-23 20:07:23,376][10905] Worker 6 uses CPU cores [0] +[2023-02-23 20:07:23,521][10899] Worker 0 uses CPU cores [0] +[2023-02-23 20:07:23,659][10900] Worker 1 uses CPU cores [1] +[2023-02-23 20:07:23,824][10906] Worker 7 uses CPU cores [1] +[2023-02-23 20:07:23,854][10884] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-23 20:07:23,854][10884] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 +[2023-02-23 20:07:23,867][10904] Worker 5 uses CPU cores [1] +[2023-02-23 20:07:24,010][10902] Worker 3 uses CPU cores [1] +[2023-02-23 20:07:24,014][10898] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-23 20:07:24,015][10898] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 +[2023-02-23 20:07:24,056][10901] Worker 2 uses CPU cores [0] +[2023-02-23 20:07:24,128][10903] Worker 4 uses CPU cores [0] +[2023-02-23 20:07:24,580][10898] Num visible devices: 1 +[2023-02-23 20:07:24,580][10884] Num visible devices: 1 +[2023-02-23 20:07:24,596][10884] Starting seed is not provided +[2023-02-23 20:07:24,598][10884] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-23 20:07:24,599][10884] Initializing actor-critic model on device cuda:0 +[2023-02-23 20:07:24,600][10884] RunningMeanStd input shape: (3, 72, 128) +[2023-02-23 20:07:24,603][10884] RunningMeanStd input shape: (1,) +[2023-02-23 20:07:24,649][10884] ConvEncoder: input_channels=3 +[2023-02-23 20:07:25,151][10884] Conv encoder output size: 512 +[2023-02-23 20:07:25,151][10884] Policy head output size: 512 +[2023-02-23 20:07:25,227][10884] Created Actor Critic model with architecture: +[2023-02-23 20:07:25,228][10884] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( @@ -85,1127 +85,1095 @@ (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) -[2023-02-22 19:21:37,919][00804] Heartbeat connected on Batcher_0 -[2023-02-22 19:21:37,948][00804] Heartbeat connected on InferenceWorker_p0-w0 -[2023-02-22 19:21:37,965][00804] Heartbeat connected on RolloutWorker_w0 -[2023-02-22 19:21:37,991][00804] Heartbeat connected on RolloutWorker_w1 -[2023-02-22 19:21:37,999][00804] Heartbeat connected on RolloutWorker_w2 -[2023-02-22 19:21:38,019][00804] Heartbeat connected on RolloutWorker_w3 -[2023-02-22 19:21:38,023][00804] Heartbeat connected on RolloutWorker_w4 -[2023-02-22 19:21:38,052][00804] Heartbeat connected on RolloutWorker_w5 -[2023-02-22 19:21:38,058][00804] Heartbeat connected on RolloutWorker_w6 -[2023-02-22 19:21:38,079][00804] Heartbeat connected on RolloutWorker_w7 -[2023-02-22 19:21:40,651][11041] Using optimizer -[2023-02-22 19:21:40,652][11041] No checkpoints found -[2023-02-22 19:21:40,652][11041] Did not load from checkpoint, starting from scratch! -[2023-02-22 19:21:40,652][11041] Initialized policy 0 weights for model version 0 -[2023-02-22 19:21:40,656][11041] LearnerWorker_p0 finished initialization! -[2023-02-22 19:21:40,660][11041] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-02-22 19:21:40,658][00804] Heartbeat connected on LearnerWorker_p0 -[2023-02-22 19:21:40,876][11055] RunningMeanStd input shape: (3, 72, 128) -[2023-02-22 19:21:40,878][11055] RunningMeanStd input shape: (1,) -[2023-02-22 19:21:40,890][11055] ConvEncoder: input_channels=3 -[2023-02-22 19:21:40,990][11055] Conv encoder output size: 512 -[2023-02-22 19:21:40,990][11055] Policy head output size: 512 -[2023-02-22 19:21:42,718][00804] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-22 19:21:43,270][00804] Inference worker 0-0 is ready! -[2023-02-22 19:21:43,273][00804] All inference workers are ready! Signal rollout workers to start! -[2023-02-22 19:21:43,413][11063] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:21:43,425][11060] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:21:43,439][11056] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:21:43,447][11061] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:21:43,442][11062] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:21:43,452][11057] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:21:43,445][11059] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:21:43,465][11058] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:21:43,672][11056] VizDoom game.init() threw an exception ViZDoomUnexpectedExitException('Controlled ViZDoom instance exited unexpectedly.'). Terminate process... -[2023-02-22 19:21:43,675][11056] EvtLoop [rollout_proc0_evt_loop, process=rollout_proc0] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init - self.game.init() -vizdoom.vizdoom.ViZDoomUnexpectedExitException: Controlled ViZDoom instance exited unexpectedly. - -During handling of the above exception, another exception occurred: - -Traceback (most recent call last): - File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal - slot_callable(*args) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init - env_runner.init(self.timing) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init - self._reset() - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset - observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset - obs, info = self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset - return self.env.reset(**kwargs) - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset - self._ensure_initialized() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized - self.initialize() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize - self._game_init() - File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init - raise EnvCriticalError() -sample_factory.envs.env_utils.EnvCriticalError -[2023-02-22 19:21:43,681][11056] Unhandled exception in evt loop rollout_proc0_evt_loop -[2023-02-22 19:21:44,662][11060] Decorrelating experience for 0 frames... -[2023-02-22 19:21:44,662][11059] Decorrelating experience for 0 frames... -[2023-02-22 19:21:44,834][11063] Decorrelating experience for 0 frames... -[2023-02-22 19:21:44,836][11057] Decorrelating experience for 0 frames... -[2023-02-22 19:21:44,847][11058] Decorrelating experience for 0 frames... -[2023-02-22 19:21:44,851][11061] Decorrelating experience for 0 frames... -[2023-02-22 19:21:45,531][11063] Decorrelating experience for 32 frames... -[2023-02-22 19:21:45,540][11058] Decorrelating experience for 32 frames... -[2023-02-22 19:21:45,912][11059] Decorrelating experience for 32 frames... -[2023-02-22 19:21:45,918][11060] Decorrelating experience for 32 frames... -[2023-02-22 19:21:45,981][11062] Decorrelating experience for 0 frames... -[2023-02-22 19:21:46,466][11061] Decorrelating experience for 32 frames... -[2023-02-22 19:21:46,588][11063] Decorrelating experience for 64 frames... -[2023-02-22 19:21:46,830][11057] Decorrelating experience for 32 frames... -[2023-02-22 19:21:47,106][11062] Decorrelating experience for 32 frames... -[2023-02-22 19:21:47,255][11059] Decorrelating experience for 64 frames... -[2023-02-22 19:21:47,270][11060] Decorrelating experience for 64 frames... -[2023-02-22 19:21:47,718][00804] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-22 19:21:48,453][11058] Decorrelating experience for 64 frames... -[2023-02-22 19:21:48,631][11063] Decorrelating experience for 96 frames... -[2023-02-22 19:21:48,747][11057] Decorrelating experience for 64 frames... -[2023-02-22 19:21:49,318][11062] Decorrelating experience for 64 frames... -[2023-02-22 19:21:49,324][11059] Decorrelating experience for 96 frames... -[2023-02-22 19:21:49,344][11060] Decorrelating experience for 96 frames... -[2023-02-22 19:21:50,331][11061] Decorrelating experience for 64 frames... -[2023-02-22 19:21:50,437][11058] Decorrelating experience for 96 frames... -[2023-02-22 19:21:50,754][11057] Decorrelating experience for 96 frames... -[2023-02-22 19:21:51,160][11061] Decorrelating experience for 96 frames... -[2023-02-22 19:21:51,497][11062] Decorrelating experience for 96 frames... -[2023-02-22 19:21:52,718][00804] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-22 19:21:56,435][11041] Signal inference workers to stop experience collection... -[2023-02-22 19:21:56,448][11055] InferenceWorker_p0-w0: stopping experience collection -[2023-02-22 19:21:57,718][00804] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 122.9. Samples: 1844. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-02-22 19:21:57,720][00804] Avg episode reward: [(0, '2.625')] -[2023-02-22 19:21:58,911][11041] Signal inference workers to resume experience collection... -[2023-02-22 19:21:58,913][11055] InferenceWorker_p0-w0: resuming experience collection -[2023-02-22 19:22:02,718][00804] Fps is (10 sec: 2048.0, 60 sec: 1024.0, 300 sec: 1024.0). Total num frames: 20480. Throughput: 0: 227.8. Samples: 4556. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) -[2023-02-22 19:22:02,723][00804] Avg episode reward: [(0, '3.596')] -[2023-02-22 19:22:07,568][11055] Updated weights for policy 0, policy_version 10 (0.0014) -[2023-02-22 19:22:07,718][00804] Fps is (10 sec: 4096.0, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 40960. Throughput: 0: 408.6. Samples: 10214. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:22:07,724][00804] Avg episode reward: [(0, '3.896')] -[2023-02-22 19:22:12,718][00804] Fps is (10 sec: 3276.8, 60 sec: 1774.9, 300 sec: 1774.9). Total num frames: 53248. Throughput: 0: 412.6. Samples: 12378. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:22:12,720][00804] Avg episode reward: [(0, '4.335')] -[2023-02-22 19:22:17,718][00804] Fps is (10 sec: 3276.8, 60 sec: 2106.5, 300 sec: 2106.5). Total num frames: 73728. Throughput: 0: 511.7. Samples: 17908. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:22:17,721][00804] Avg episode reward: [(0, '4.368')] -[2023-02-22 19:22:18,961][11055] Updated weights for policy 0, policy_version 20 (0.0027) -[2023-02-22 19:22:22,718][00804] Fps is (10 sec: 4505.8, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 98304. Throughput: 0: 623.8. Samples: 24952. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:22:22,721][00804] Avg episode reward: [(0, '4.363')] -[2023-02-22 19:22:27,721][00804] Fps is (10 sec: 4094.7, 60 sec: 2548.4, 300 sec: 2548.4). Total num frames: 114688. Throughput: 0: 618.8. Samples: 27850. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:22:27,727][00804] Avg episode reward: [(0, '4.449')] -[2023-02-22 19:22:27,732][11041] Saving new best policy, reward=4.449! -[2023-02-22 19:22:29,789][11055] Updated weights for policy 0, policy_version 30 (0.0022) -[2023-02-22 19:22:32,718][00804] Fps is (10 sec: 3276.8, 60 sec: 2621.4, 300 sec: 2621.4). Total num frames: 131072. Throughput: 0: 716.4. Samples: 32240. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:22:32,720][00804] Avg episode reward: [(0, '4.389')] -[2023-02-22 19:22:37,718][00804] Fps is (10 sec: 3687.6, 60 sec: 2755.5, 300 sec: 2755.5). Total num frames: 151552. Throughput: 0: 860.3. Samples: 38712. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:22:37,724][00804] Avg episode reward: [(0, '4.423')] -[2023-02-22 19:22:39,566][11055] Updated weights for policy 0, policy_version 40 (0.0016) -[2023-02-22 19:22:42,718][00804] Fps is (10 sec: 4505.6, 60 sec: 2935.5, 300 sec: 2935.5). Total num frames: 176128. Throughput: 0: 897.0. Samples: 42210. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:22:42,720][00804] Avg episode reward: [(0, '4.594')] -[2023-02-22 19:22:42,731][11041] Saving new best policy, reward=4.594! -[2023-02-22 19:22:47,747][00804] Fps is (10 sec: 4084.0, 60 sec: 3207.0, 300 sec: 2960.4). Total num frames: 192512. Throughput: 0: 956.1. Samples: 47608. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:22:47,758][00804] Avg episode reward: [(0, '4.427')] -[2023-02-22 19:22:52,400][11055] Updated weights for policy 0, policy_version 50 (0.0018) -[2023-02-22 19:22:52,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 2925.7). Total num frames: 204800. Throughput: 0: 918.0. Samples: 51522. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:22:52,720][00804] Avg episode reward: [(0, '4.574')] -[2023-02-22 19:22:57,720][00804] Fps is (10 sec: 3285.6, 60 sec: 3754.5, 300 sec: 3003.6). Total num frames: 225280. Throughput: 0: 943.6. Samples: 54840. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:22:57,723][00804] Avg episode reward: [(0, '4.457')] -[2023-02-22 19:23:01,744][11055] Updated weights for policy 0, policy_version 60 (0.0017) -[2023-02-22 19:23:02,726][00804] Fps is (10 sec: 4092.8, 60 sec: 3754.2, 300 sec: 3071.7). Total num frames: 245760. Throughput: 0: 968.8. Samples: 61510. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:23:02,733][00804] Avg episode reward: [(0, '4.308')] -[2023-02-22 19:23:07,718][00804] Fps is (10 sec: 3687.2, 60 sec: 3686.4, 300 sec: 3084.1). Total num frames: 262144. Throughput: 0: 916.0. Samples: 66172. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:23:07,726][00804] Avg episode reward: [(0, '4.389')] -[2023-02-22 19:23:12,718][00804] Fps is (10 sec: 3279.3, 60 sec: 3754.7, 300 sec: 3094.8). Total num frames: 278528. Throughput: 0: 898.5. Samples: 68278. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:23:12,724][00804] Avg episode reward: [(0, '4.723')] -[2023-02-22 19:23:12,735][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000068_278528.pth... -[2023-02-22 19:23:12,841][11041] Saving new best policy, reward=4.723! -[2023-02-22 19:23:14,532][11055] Updated weights for policy 0, policy_version 70 (0.0018) -[2023-02-22 19:23:17,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3147.5). Total num frames: 299008. Throughput: 0: 932.9. Samples: 74220. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:23:17,726][00804] Avg episode reward: [(0, '4.610')] -[2023-02-22 19:23:22,723][00804] Fps is (10 sec: 4093.9, 60 sec: 3686.1, 300 sec: 3194.7). Total num frames: 319488. Throughput: 0: 930.9. Samples: 80606. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:23:22,726][00804] Avg episode reward: [(0, '4.292')] -[2023-02-22 19:23:25,091][11055] Updated weights for policy 0, policy_version 80 (0.0011) -[2023-02-22 19:23:27,722][00804] Fps is (10 sec: 3275.4, 60 sec: 3618.1, 300 sec: 3159.6). Total num frames: 331776. Throughput: 0: 897.8. Samples: 82616. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:23:27,725][00804] Avg episode reward: [(0, '4.220')] -[2023-02-22 19:23:32,718][00804] Fps is (10 sec: 2868.7, 60 sec: 3618.1, 300 sec: 3165.1). Total num frames: 348160. Throughput: 0: 875.4. Samples: 86976. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:23:32,726][00804] Avg episode reward: [(0, '4.378')] -[2023-02-22 19:23:36,298][11055] Updated weights for policy 0, policy_version 90 (0.0023) -[2023-02-22 19:23:37,718][00804] Fps is (10 sec: 4097.8, 60 sec: 3686.4, 300 sec: 3241.2). Total num frames: 372736. Throughput: 0: 937.4. Samples: 93704. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:23:37,721][00804] Avg episode reward: [(0, '4.501')] -[2023-02-22 19:23:42,718][00804] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3276.8). Total num frames: 393216. Throughput: 0: 936.1. Samples: 96962. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:23:42,720][00804] Avg episode reward: [(0, '4.553')] -[2023-02-22 19:23:47,721][00804] Fps is (10 sec: 3275.7, 60 sec: 3551.4, 300 sec: 3243.9). Total num frames: 405504. Throughput: 0: 890.5. Samples: 101578. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:23:47,726][00804] Avg episode reward: [(0, '4.411')] -[2023-02-22 19:23:48,114][11055] Updated weights for policy 0, policy_version 100 (0.0026) -[2023-02-22 19:23:52,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3276.8). Total num frames: 425984. Throughput: 0: 901.2. Samples: 106724. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:23:52,722][00804] Avg episode reward: [(0, '4.603')] -[2023-02-22 19:23:57,718][00804] Fps is (10 sec: 4097.3, 60 sec: 3686.5, 300 sec: 3307.1). Total num frames: 446464. Throughput: 0: 926.2. Samples: 109956. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:23:57,726][00804] Avg episode reward: [(0, '4.636')] -[2023-02-22 19:23:58,184][11055] Updated weights for policy 0, policy_version 110 (0.0014) -[2023-02-22 19:24:02,718][00804] Fps is (10 sec: 3686.3, 60 sec: 3618.6, 300 sec: 3306.1). Total num frames: 462848. Throughput: 0: 927.3. Samples: 115948. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:24:02,724][00804] Avg episode reward: [(0, '4.512')] -[2023-02-22 19:24:07,719][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3305.1). Total num frames: 479232. Throughput: 0: 878.3. Samples: 120124. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:24:07,727][00804] Avg episode reward: [(0, '4.464')] -[2023-02-22 19:24:10,757][11055] Updated weights for policy 0, policy_version 120 (0.0027) -[2023-02-22 19:24:12,718][00804] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3331.4). Total num frames: 499712. Throughput: 0: 890.8. Samples: 122698. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) -[2023-02-22 19:24:12,720][00804] Avg episode reward: [(0, '4.510')] -[2023-02-22 19:24:17,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3356.1). Total num frames: 520192. Throughput: 0: 943.5. Samples: 129432. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:24:17,720][00804] Avg episode reward: [(0, '4.453')] -[2023-02-22 19:24:20,621][11055] Updated weights for policy 0, policy_version 130 (0.0015) -[2023-02-22 19:24:22,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3618.5, 300 sec: 3353.6). Total num frames: 536576. Throughput: 0: 915.1. Samples: 134884. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:24:22,725][00804] Avg episode reward: [(0, '4.259')] -[2023-02-22 19:24:27,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3618.4, 300 sec: 3326.5). Total num frames: 548864. Throughput: 0: 889.9. Samples: 137006. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:24:27,726][00804] Avg episode reward: [(0, '4.429')] -[2023-02-22 19:24:32,722][00804] Fps is (10 sec: 2866.0, 60 sec: 3617.9, 300 sec: 3324.9). Total num frames: 565248. Throughput: 0: 884.3. Samples: 141370. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:24:32,727][00804] Avg episode reward: [(0, '4.521')] -[2023-02-22 19:24:35,012][11055] Updated weights for policy 0, policy_version 140 (0.0013) -[2023-02-22 19:24:37,718][00804] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3323.6). Total num frames: 581632. Throughput: 0: 862.5. Samples: 145536. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:24:37,726][00804] Avg episode reward: [(0, '4.442')] -[2023-02-22 19:24:42,718][00804] Fps is (10 sec: 2868.4, 60 sec: 3345.1, 300 sec: 3299.6). Total num frames: 593920. Throughput: 0: 844.1. Samples: 147942. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:24:42,722][00804] Avg episode reward: [(0, '4.454')] -[2023-02-22 19:24:47,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3413.5, 300 sec: 3298.9). Total num frames: 610304. Throughput: 0: 803.2. Samples: 152094. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:24:47,721][00804] Avg episode reward: [(0, '4.490')] -[2023-02-22 19:24:48,521][11055] Updated weights for policy 0, policy_version 150 (0.0038) -[2023-02-22 19:24:52,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3319.9). Total num frames: 630784. Throughput: 0: 849.4. Samples: 158348. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:24:52,724][00804] Avg episode reward: [(0, '4.440')] -[2023-02-22 19:24:57,718][00804] Fps is (10 sec: 4096.1, 60 sec: 3413.3, 300 sec: 3339.8). Total num frames: 651264. Throughput: 0: 863.8. Samples: 161568. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:24:57,720][00804] Avg episode reward: [(0, '4.530')] -[2023-02-22 19:24:57,914][11055] Updated weights for policy 0, policy_version 160 (0.0018) -[2023-02-22 19:25:02,720][00804] Fps is (10 sec: 3685.5, 60 sec: 3413.2, 300 sec: 3338.2). Total num frames: 667648. Throughput: 0: 828.5. Samples: 166716. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:25:02,723][00804] Avg episode reward: [(0, '4.591')] -[2023-02-22 19:25:07,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3336.7). Total num frames: 684032. Throughput: 0: 805.0. Samples: 171110. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:25:07,720][00804] Avg episode reward: [(0, '4.819')] -[2023-02-22 19:25:07,724][11041] Saving new best policy, reward=4.819! -[2023-02-22 19:25:10,595][11055] Updated weights for policy 0, policy_version 170 (0.0021) -[2023-02-22 19:25:12,718][00804] Fps is (10 sec: 3687.3, 60 sec: 3413.3, 300 sec: 3354.8). Total num frames: 704512. Throughput: 0: 829.2. Samples: 174320. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:25:12,722][00804] Avg episode reward: [(0, '4.941')] -[2023-02-22 19:25:12,738][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000172_704512.pth... -[2023-02-22 19:25:12,860][11041] Saving new best policy, reward=4.941! -[2023-02-22 19:25:17,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3372.1). Total num frames: 724992. Throughput: 0: 876.7. Samples: 180818. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:25:17,727][00804] Avg episode reward: [(0, '4.797')] -[2023-02-22 19:25:21,559][11055] Updated weights for policy 0, policy_version 180 (0.0013) -[2023-02-22 19:25:22,721][00804] Fps is (10 sec: 3275.7, 60 sec: 3344.9, 300 sec: 3351.2). Total num frames: 737280. Throughput: 0: 884.1. Samples: 185322. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:25:22,724][00804] Avg episode reward: [(0, '4.698')] -[2023-02-22 19:25:27,718][00804] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3367.8). Total num frames: 757760. Throughput: 0: 879.2. Samples: 187506. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:25:27,720][00804] Avg episode reward: [(0, '4.809')] -[2023-02-22 19:25:32,359][11055] Updated weights for policy 0, policy_version 190 (0.0019) -[2023-02-22 19:25:32,718][00804] Fps is (10 sec: 4097.2, 60 sec: 3550.1, 300 sec: 3383.6). Total num frames: 778240. Throughput: 0: 928.2. Samples: 193862. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-22 19:25:32,721][00804] Avg episode reward: [(0, '4.925')] -[2023-02-22 19:25:37,718][00804] Fps is (10 sec: 4096.1, 60 sec: 3618.2, 300 sec: 3398.8). Total num frames: 798720. Throughput: 0: 927.5. Samples: 200084. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:25:37,720][00804] Avg episode reward: [(0, '4.883')] -[2023-02-22 19:25:42,718][00804] Fps is (10 sec: 3276.9, 60 sec: 3618.1, 300 sec: 3379.2). Total num frames: 811008. Throughput: 0: 902.1. Samples: 202164. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:25:42,727][00804] Avg episode reward: [(0, '4.864')] -[2023-02-22 19:25:44,668][11055] Updated weights for policy 0, policy_version 200 (0.0015) -[2023-02-22 19:25:47,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3393.8). Total num frames: 831488. Throughput: 0: 890.7. Samples: 206794. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:25:47,725][00804] Avg episode reward: [(0, '4.769')] -[2023-02-22 19:25:52,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3407.9). Total num frames: 851968. Throughput: 0: 942.1. Samples: 213504. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:25:52,726][00804] Avg episode reward: [(0, '4.907')] -[2023-02-22 19:25:54,293][11055] Updated weights for policy 0, policy_version 210 (0.0011) -[2023-02-22 19:25:57,724][00804] Fps is (10 sec: 4093.4, 60 sec: 3686.0, 300 sec: 3421.3). Total num frames: 872448. Throughput: 0: 944.4. Samples: 216824. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:25:57,727][00804] Avg episode reward: [(0, '4.921')] -[2023-02-22 19:26:02,718][00804] Fps is (10 sec: 3276.7, 60 sec: 3618.3, 300 sec: 3402.8). Total num frames: 884736. Throughput: 0: 894.1. Samples: 221052. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:26:02,726][00804] Avg episode reward: [(0, '5.000')] -[2023-02-22 19:26:02,736][11041] Saving new best policy, reward=5.000! -[2023-02-22 19:26:06,777][11055] Updated weights for policy 0, policy_version 220 (0.0020) -[2023-02-22 19:26:07,718][00804] Fps is (10 sec: 2869.0, 60 sec: 3618.1, 300 sec: 3400.5). Total num frames: 901120. Throughput: 0: 912.2. Samples: 226366. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:26:07,726][00804] Avg episode reward: [(0, '5.132')] -[2023-02-22 19:26:07,783][11041] Saving new best policy, reward=5.132! -[2023-02-22 19:26:12,718][00804] Fps is (10 sec: 4096.1, 60 sec: 3686.4, 300 sec: 3428.5). Total num frames: 925696. Throughput: 0: 937.1. Samples: 229676. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-22 19:26:12,721][00804] Avg episode reward: [(0, '5.179')] -[2023-02-22 19:26:12,734][11041] Saving new best policy, reward=5.179! -[2023-02-22 19:26:16,392][11055] Updated weights for policy 0, policy_version 230 (0.0017) -[2023-02-22 19:26:17,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3425.7). Total num frames: 942080. Throughput: 0: 928.6. Samples: 235650. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:26:17,722][00804] Avg episode reward: [(0, '5.020')] -[2023-02-22 19:26:22,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3618.3, 300 sec: 3408.5). Total num frames: 954368. Throughput: 0: 881.5. Samples: 239750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:26:22,721][00804] Avg episode reward: [(0, '4.932')] -[2023-02-22 19:26:27,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3420.5). Total num frames: 974848. Throughput: 0: 897.6. Samples: 242558. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:26:27,719][00804] Avg episode reward: [(0, '5.076')] -[2023-02-22 19:26:28,582][11055] Updated weights for policy 0, policy_version 240 (0.0015) -[2023-02-22 19:26:32,718][00804] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3446.3). Total num frames: 999424. Throughput: 0: 944.8. Samples: 249310. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:26:32,725][00804] Avg episode reward: [(0, '5.075')] -[2023-02-22 19:26:37,722][00804] Fps is (10 sec: 4094.5, 60 sec: 3617.9, 300 sec: 3443.4). Total num frames: 1015808. Throughput: 0: 909.8. Samples: 254448. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:26:37,724][00804] Avg episode reward: [(0, '5.073')] -[2023-02-22 19:26:40,482][11055] Updated weights for policy 0, policy_version 250 (0.0023) -[2023-02-22 19:26:42,718][00804] Fps is (10 sec: 2867.1, 60 sec: 3618.1, 300 sec: 3485.1). Total num frames: 1028096. Throughput: 0: 883.0. Samples: 256554. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) -[2023-02-22 19:26:42,725][00804] Avg episode reward: [(0, '5.208')] -[2023-02-22 19:26:42,736][11041] Saving new best policy, reward=5.208! -[2023-02-22 19:26:47,718][00804] Fps is (10 sec: 3278.0, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 1048576. Throughput: 0: 911.5. Samples: 262070. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) -[2023-02-22 19:26:47,724][00804] Avg episode reward: [(0, '5.105')] -[2023-02-22 19:26:50,695][11055] Updated weights for policy 0, policy_version 260 (0.0034) -[2023-02-22 19:26:52,718][00804] Fps is (10 sec: 4505.7, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 1073152. Throughput: 0: 942.3. Samples: 268768. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:26:52,724][00804] Avg episode reward: [(0, '5.245')] -[2023-02-22 19:26:52,736][11041] Saving new best policy, reward=5.245! -[2023-02-22 19:26:57,719][00804] Fps is (10 sec: 3685.9, 60 sec: 3550.2, 300 sec: 3610.0). Total num frames: 1085440. Throughput: 0: 916.5. Samples: 270918. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:26:57,724][00804] Avg episode reward: [(0, '5.450')] -[2023-02-22 19:26:57,728][11041] Saving new best policy, reward=5.450! -[2023-02-22 19:27:02,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3618.2, 300 sec: 3596.1). Total num frames: 1101824. Throughput: 0: 874.4. Samples: 274996. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:27:02,720][00804] Avg episode reward: [(0, '5.733')] -[2023-02-22 19:27:02,731][11041] Saving new best policy, reward=5.733! -[2023-02-22 19:27:03,477][11055] Updated weights for policy 0, policy_version 270 (0.0015) -[2023-02-22 19:27:07,718][00804] Fps is (10 sec: 3686.9, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 1122304. Throughput: 0: 920.3. Samples: 281164. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) -[2023-02-22 19:27:07,720][00804] Avg episode reward: [(0, '5.729')] -[2023-02-22 19:27:12,723][00804] Fps is (10 sec: 4093.8, 60 sec: 3617.8, 300 sec: 3623.9). Total num frames: 1142784. Throughput: 0: 931.7. Samples: 284490. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:27:12,726][00804] Avg episode reward: [(0, '5.989')] -[2023-02-22 19:27:12,743][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000279_1142784.pth... -[2023-02-22 19:27:12,878][11041] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000068_278528.pth -[2023-02-22 19:27:12,900][11041] Saving new best policy, reward=5.989! -[2023-02-22 19:27:13,246][11055] Updated weights for policy 0, policy_version 280 (0.0013) -[2023-02-22 19:27:17,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3596.1). Total num frames: 1159168. Throughput: 0: 889.5. Samples: 289336. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:27:17,724][00804] Avg episode reward: [(0, '6.205')] -[2023-02-22 19:27:17,728][11041] Saving new best policy, reward=6.205! -[2023-02-22 19:27:22,718][00804] Fps is (10 sec: 2868.7, 60 sec: 3618.1, 300 sec: 3582.3). Total num frames: 1171456. Throughput: 0: 876.0. Samples: 293864. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) -[2023-02-22 19:27:22,726][00804] Avg episode reward: [(0, '6.092')] -[2023-02-22 19:27:25,426][11055] Updated weights for policy 0, policy_version 290 (0.0012) -[2023-02-22 19:27:27,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 1196032. Throughput: 0: 902.9. Samples: 297186. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:27:27,725][00804] Avg episode reward: [(0, '6.117')] -[2023-02-22 19:27:32,722][00804] Fps is (10 sec: 4503.7, 60 sec: 3617.9, 300 sec: 3610.0). Total num frames: 1216512. Throughput: 0: 925.5. Samples: 303720. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:27:32,730][00804] Avg episode reward: [(0, '5.917')] -[2023-02-22 19:27:36,642][11055] Updated weights for policy 0, policy_version 300 (0.0015) -[2023-02-22 19:27:37,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3550.1, 300 sec: 3568.4). Total num frames: 1228800. Throughput: 0: 871.3. Samples: 307976. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:27:37,724][00804] Avg episode reward: [(0, '5.900')] -[2023-02-22 19:27:42,718][00804] Fps is (10 sec: 2868.4, 60 sec: 3618.1, 300 sec: 3568.7). Total num frames: 1245184. Throughput: 0: 871.1. Samples: 310118. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:27:42,720][00804] Avg episode reward: [(0, '5.726')] -[2023-02-22 19:27:47,469][11055] Updated weights for policy 0, policy_version 310 (0.0015) -[2023-02-22 19:27:47,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 1269760. Throughput: 0: 924.7. Samples: 316608. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:27:47,720][00804] Avg episode reward: [(0, '6.345')] -[2023-02-22 19:27:47,728][11041] Saving new best policy, reward=6.345! -[2023-02-22 19:27:52,718][00804] Fps is (10 sec: 4505.7, 60 sec: 3618.1, 300 sec: 3610.1). Total num frames: 1290240. Throughput: 0: 922.1. Samples: 322658. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:27:52,720][00804] Avg episode reward: [(0, '6.413')] -[2023-02-22 19:27:52,732][11041] Saving new best policy, reward=6.413! -[2023-02-22 19:27:57,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3582.4). Total num frames: 1302528. Throughput: 0: 894.2. Samples: 324724. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:27:57,721][00804] Avg episode reward: [(0, '6.560')] -[2023-02-22 19:27:57,723][11041] Saving new best policy, reward=6.560! -[2023-02-22 19:28:00,060][11055] Updated weights for policy 0, policy_version 320 (0.0014) -[2023-02-22 19:28:02,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3582.3). Total num frames: 1318912. Throughput: 0: 890.7. Samples: 329418. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:28:02,720][00804] Avg episode reward: [(0, '6.468')] -[2023-02-22 19:28:07,718][00804] Fps is (10 sec: 4095.9, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 1343488. Throughput: 0: 940.5. Samples: 336186. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:28:07,721][00804] Avg episode reward: [(0, '7.199')] -[2023-02-22 19:28:07,724][11041] Saving new best policy, reward=7.199! -[2023-02-22 19:28:09,184][11055] Updated weights for policy 0, policy_version 330 (0.0021) -[2023-02-22 19:28:12,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.5, 300 sec: 3596.1). Total num frames: 1359872. Throughput: 0: 938.7. Samples: 339428. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:28:12,720][00804] Avg episode reward: [(0, '7.439')] -[2023-02-22 19:28:12,729][11041] Saving new best policy, reward=7.439! -[2023-02-22 19:28:17,720][00804] Fps is (10 sec: 3276.1, 60 sec: 3618.0, 300 sec: 3582.3). Total num frames: 1376256. Throughput: 0: 889.9. Samples: 343762. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:28:17,723][00804] Avg episode reward: [(0, '7.509')] -[2023-02-22 19:28:17,728][11041] Saving new best policy, reward=7.509! -[2023-02-22 19:28:21,730][11055] Updated weights for policy 0, policy_version 340 (0.0022) -[2023-02-22 19:28:22,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3610.1). Total num frames: 1396736. Throughput: 0: 921.2. Samples: 349430. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:28:22,725][00804] Avg episode reward: [(0, '7.434')] -[2023-02-22 19:28:27,718][00804] Fps is (10 sec: 4097.0, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 1417216. Throughput: 0: 950.5. Samples: 352892. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:28:27,721][00804] Avg episode reward: [(0, '7.914')] -[2023-02-22 19:28:27,760][11041] Saving new best policy, reward=7.914! -[2023-02-22 19:28:31,204][11055] Updated weights for policy 0, policy_version 350 (0.0014) -[2023-02-22 19:28:32,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3686.7, 300 sec: 3610.0). Total num frames: 1437696. Throughput: 0: 943.1. Samples: 359046. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:28:32,725][00804] Avg episode reward: [(0, '7.964')] -[2023-02-22 19:28:32,739][11041] Saving new best policy, reward=7.964! -[2023-02-22 19:28:37,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 1449984. Throughput: 0: 906.8. Samples: 363464. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:28:37,725][00804] Avg episode reward: [(0, '8.360')] -[2023-02-22 19:28:37,727][11041] Saving new best policy, reward=8.360! -[2023-02-22 19:28:42,483][11055] Updated weights for policy 0, policy_version 360 (0.0019) -[2023-02-22 19:28:42,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3624.0). Total num frames: 1474560. Throughput: 0: 927.0. Samples: 366438. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:28:42,720][00804] Avg episode reward: [(0, '8.779')] -[2023-02-22 19:28:42,733][11041] Saving new best policy, reward=8.779! -[2023-02-22 19:28:47,718][00804] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3623.9). Total num frames: 1495040. Throughput: 0: 975.6. Samples: 373318. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:28:47,721][00804] Avg episode reward: [(0, '10.356')] -[2023-02-22 19:28:47,724][11041] Saving new best policy, reward=10.356! -[2023-02-22 19:28:52,718][00804] Fps is (10 sec: 3686.3, 60 sec: 3686.4, 300 sec: 3610.0). Total num frames: 1511424. Throughput: 0: 943.0. Samples: 378620. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:28:52,725][00804] Avg episode reward: [(0, '10.666')] -[2023-02-22 19:28:52,736][11041] Saving new best policy, reward=10.666! -[2023-02-22 19:28:53,829][11055] Updated weights for policy 0, policy_version 370 (0.0011) -[2023-02-22 19:28:57,720][00804] Fps is (10 sec: 2866.7, 60 sec: 3686.3, 300 sec: 3596.1). Total num frames: 1523712. Throughput: 0: 909.4. Samples: 380352. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:28:57,726][00804] Avg episode reward: [(0, '11.593')] -[2023-02-22 19:28:57,729][11041] Saving new best policy, reward=11.593! -[2023-02-22 19:29:02,720][00804] Fps is (10 sec: 2457.1, 60 sec: 3618.0, 300 sec: 3582.2). Total num frames: 1536000. Throughput: 0: 887.9. Samples: 383718. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:29:02,725][00804] Avg episode reward: [(0, '11.584')] -[2023-02-22 19:29:07,592][11055] Updated weights for policy 0, policy_version 380 (0.0012) -[2023-02-22 19:29:07,718][00804] Fps is (10 sec: 3277.4, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 1556480. Throughput: 0: 887.8. Samples: 389380. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:29:07,721][00804] Avg episode reward: [(0, '11.088')] -[2023-02-22 19:29:12,720][00804] Fps is (10 sec: 4095.9, 60 sec: 3618.0, 300 sec: 3582.2). Total num frames: 1576960. Throughput: 0: 886.3. Samples: 392778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:29:12,727][00804] Avg episode reward: [(0, '10.786')] -[2023-02-22 19:29:12,742][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000385_1576960.pth... -[2023-02-22 19:29:12,883][11041] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000172_704512.pth -[2023-02-22 19:29:17,719][00804] Fps is (10 sec: 3276.4, 60 sec: 3549.9, 300 sec: 3568.4). Total num frames: 1589248. Throughput: 0: 853.9. Samples: 397474. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:29:17,726][00804] Avg episode reward: [(0, '10.547')] -[2023-02-22 19:29:19,603][11055] Updated weights for policy 0, policy_version 390 (0.0021) -[2023-02-22 19:29:22,718][00804] Fps is (10 sec: 3277.6, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 1609728. Throughput: 0: 866.7. Samples: 402466. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:29:22,721][00804] Avg episode reward: [(0, '10.128')] -[2023-02-22 19:29:27,718][00804] Fps is (10 sec: 4096.5, 60 sec: 3549.9, 300 sec: 3610.1). Total num frames: 1630208. Throughput: 0: 877.3. Samples: 405918. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:29:27,726][00804] Avg episode reward: [(0, '10.701')] -[2023-02-22 19:29:29,034][11055] Updated weights for policy 0, policy_version 400 (0.0019) -[2023-02-22 19:29:32,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3623.9). Total num frames: 1650688. Throughput: 0: 868.4. Samples: 412394. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:29:32,720][00804] Avg episode reward: [(0, '10.413')] -[2023-02-22 19:29:37,725][00804] Fps is (10 sec: 3274.4, 60 sec: 3549.4, 300 sec: 3623.8). Total num frames: 1662976. Throughput: 0: 847.0. Samples: 416742. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:29:37,729][00804] Avg episode reward: [(0, '10.769')] -[2023-02-22 19:29:41,544][11055] Updated weights for policy 0, policy_version 410 (0.0025) -[2023-02-22 19:29:42,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3637.8). Total num frames: 1683456. Throughput: 0: 859.3. Samples: 419020. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:29:42,726][00804] Avg episode reward: [(0, '12.010')] -[2023-02-22 19:29:42,735][11041] Saving new best policy, reward=12.010! -[2023-02-22 19:29:47,718][00804] Fps is (10 sec: 4099.0, 60 sec: 3481.6, 300 sec: 3637.8). Total num frames: 1703936. Throughput: 0: 932.4. Samples: 425676. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:29:47,720][00804] Avg episode reward: [(0, '13.293')] -[2023-02-22 19:29:47,727][11041] Saving new best policy, reward=13.293! -[2023-02-22 19:29:51,050][11055] Updated weights for policy 0, policy_version 420 (0.0012) -[2023-02-22 19:29:52,718][00804] Fps is (10 sec: 4095.8, 60 sec: 3549.9, 300 sec: 3637.8). Total num frames: 1724416. Throughput: 0: 933.5. Samples: 431388. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:29:52,721][00804] Avg episode reward: [(0, '13.583')] -[2023-02-22 19:29:52,733][11041] Saving new best policy, reward=13.583! -[2023-02-22 19:29:57,718][00804] Fps is (10 sec: 3276.6, 60 sec: 3549.9, 300 sec: 3623.9). Total num frames: 1736704. Throughput: 0: 903.0. Samples: 433410. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:29:57,721][00804] Avg episode reward: [(0, '13.816')] -[2023-02-22 19:29:57,726][11041] Saving new best policy, reward=13.816! -[2023-02-22 19:30:02,718][00804] Fps is (10 sec: 3276.9, 60 sec: 3686.5, 300 sec: 3637.8). Total num frames: 1757184. Throughput: 0: 907.0. Samples: 438288. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:30:02,725][00804] Avg episode reward: [(0, '13.814')] -[2023-02-22 19:30:03,693][11055] Updated weights for policy 0, policy_version 430 (0.0038) -[2023-02-22 19:30:07,718][00804] Fps is (10 sec: 4096.2, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 1777664. Throughput: 0: 942.3. Samples: 444868. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:30:07,720][00804] Avg episode reward: [(0, '14.664')] -[2023-02-22 19:30:07,725][11041] Saving new best policy, reward=14.664! -[2023-02-22 19:30:12,718][00804] Fps is (10 sec: 3686.5, 60 sec: 3618.3, 300 sec: 3623.9). Total num frames: 1794048. Throughput: 0: 930.6. Samples: 447794. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:30:12,721][00804] Avg episode reward: [(0, '15.159')] -[2023-02-22 19:30:12,735][11041] Saving new best policy, reward=15.159! -[2023-02-22 19:30:15,161][11055] Updated weights for policy 0, policy_version 440 (0.0015) -[2023-02-22 19:30:17,721][00804] Fps is (10 sec: 2866.3, 60 sec: 3618.0, 300 sec: 3623.9). Total num frames: 1806336. Throughput: 0: 876.5. Samples: 451840. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:30:17,724][00804] Avg episode reward: [(0, '15.761')] -[2023-02-22 19:30:17,728][11041] Saving new best policy, reward=15.761! -[2023-02-22 19:30:22,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 1826816. Throughput: 0: 904.3. Samples: 457428. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:30:22,720][00804] Avg episode reward: [(0, '16.319')] -[2023-02-22 19:30:22,733][11041] Saving new best policy, reward=16.319! -[2023-02-22 19:30:25,746][11055] Updated weights for policy 0, policy_version 450 (0.0025) -[2023-02-22 19:30:27,718][00804] Fps is (10 sec: 4507.0, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 1851392. Throughput: 0: 926.0. Samples: 460690. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:30:27,721][00804] Avg episode reward: [(0, '16.021')] -[2023-02-22 19:30:32,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 1863680. Throughput: 0: 902.9. Samples: 466308. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:30:32,725][00804] Avg episode reward: [(0, '15.107')] -[2023-02-22 19:30:37,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3618.6, 300 sec: 3623.9). Total num frames: 1880064. Throughput: 0: 865.9. Samples: 470354. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:30:37,723][00804] Avg episode reward: [(0, '14.583')] -[2023-02-22 19:30:38,424][11055] Updated weights for policy 0, policy_version 460 (0.0040) -[2023-02-22 19:30:42,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 1900544. Throughput: 0: 890.3. Samples: 473474. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:30:42,720][00804] Avg episode reward: [(0, '14.217')] -[2023-02-22 19:30:47,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 1921024. Throughput: 0: 929.0. Samples: 480094. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:30:47,723][00804] Avg episode reward: [(0, '15.128')] -[2023-02-22 19:30:47,913][11055] Updated weights for policy 0, policy_version 470 (0.0023) -[2023-02-22 19:30:52,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3610.1). Total num frames: 1937408. Throughput: 0: 888.2. Samples: 484836. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:30:52,724][00804] Avg episode reward: [(0, '15.161')] -[2023-02-22 19:30:57,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3623.9). Total num frames: 1953792. Throughput: 0: 868.2. Samples: 486862. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:30:57,721][00804] Avg episode reward: [(0, '15.737')] -[2023-02-22 19:31:00,566][11055] Updated weights for policy 0, policy_version 480 (0.0027) -[2023-02-22 19:31:02,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 1974272. Throughput: 0: 912.1. Samples: 492880. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) -[2023-02-22 19:31:02,725][00804] Avg episode reward: [(0, '16.568')] -[2023-02-22 19:31:02,735][11041] Saving new best policy, reward=16.568! -[2023-02-22 19:31:07,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 1994752. Throughput: 0: 927.5. Samples: 499164. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:31:07,722][00804] Avg episode reward: [(0, '17.030')] -[2023-02-22 19:31:07,727][11041] Saving new best policy, reward=17.030! -[2023-02-22 19:31:11,858][11055] Updated weights for policy 0, policy_version 490 (0.0014) -[2023-02-22 19:31:12,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 2007040. Throughput: 0: 899.9. Samples: 501186. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:31:12,724][00804] Avg episode reward: [(0, '17.010')] -[2023-02-22 19:31:12,736][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000490_2007040.pth... -[2023-02-22 19:31:12,870][11041] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000279_1142784.pth -[2023-02-22 19:31:17,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3618.3, 300 sec: 3623.9). Total num frames: 2023424. Throughput: 0: 869.9. Samples: 505452. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:31:17,721][00804] Avg episode reward: [(0, '16.790')] -[2023-02-22 19:31:22,548][11055] Updated weights for policy 0, policy_version 500 (0.0024) -[2023-02-22 19:31:22,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 2048000. Throughput: 0: 929.5. Samples: 512182. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:31:22,720][00804] Avg episode reward: [(0, '16.067')] -[2023-02-22 19:31:27,718][00804] Fps is (10 sec: 4505.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2068480. Throughput: 0: 933.5. Samples: 515480. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:31:27,726][00804] Avg episode reward: [(0, '15.065')] -[2023-02-22 19:31:32,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3610.1). Total num frames: 2080768. Throughput: 0: 887.5. Samples: 520032. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:31:32,726][00804] Avg episode reward: [(0, '13.873')] -[2023-02-22 19:31:35,158][11055] Updated weights for policy 0, policy_version 510 (0.0013) -[2023-02-22 19:31:37,718][00804] Fps is (10 sec: 2867.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2097152. Throughput: 0: 892.4. Samples: 524994. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:31:37,726][00804] Avg episode reward: [(0, '13.726')] -[2023-02-22 19:31:42,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 2121728. Throughput: 0: 923.2. Samples: 528406. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:31:42,720][00804] Avg episode reward: [(0, '14.147')] -[2023-02-22 19:31:44,628][11055] Updated weights for policy 0, policy_version 520 (0.0015) -[2023-02-22 19:31:47,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3610.0). Total num frames: 2138112. Throughput: 0: 925.8. Samples: 534540. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:31:47,719][00804] Avg episode reward: [(0, '15.159')] -[2023-02-22 19:31:52,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3610.1). Total num frames: 2150400. Throughput: 0: 879.4. Samples: 538736. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:31:52,721][00804] Avg episode reward: [(0, '16.804')] -[2023-02-22 19:31:57,172][11055] Updated weights for policy 0, policy_version 530 (0.0031) -[2023-02-22 19:31:57,719][00804] Fps is (10 sec: 3276.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2170880. Throughput: 0: 886.4. Samples: 541074. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:31:57,722][00804] Avg episode reward: [(0, '17.633')] -[2023-02-22 19:31:57,728][11041] Saving new best policy, reward=17.633! -[2023-02-22 19:32:02,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2191360. Throughput: 0: 934.7. Samples: 547514. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:32:02,724][00804] Avg episode reward: [(0, '18.108')] -[2023-02-22 19:32:02,736][11041] Saving new best policy, reward=18.108! -[2023-02-22 19:32:07,718][00804] Fps is (10 sec: 3686.8, 60 sec: 3549.9, 300 sec: 3610.1). Total num frames: 2207744. Throughput: 0: 904.8. Samples: 552900. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:32:07,722][00804] Avg episode reward: [(0, '18.383')] -[2023-02-22 19:32:07,728][11041] Saving new best policy, reward=18.383! -[2023-02-22 19:32:08,185][11055] Updated weights for policy 0, policy_version 540 (0.0021) -[2023-02-22 19:32:12,719][00804] Fps is (10 sec: 3276.4, 60 sec: 3618.1, 300 sec: 3610.0). Total num frames: 2224128. Throughput: 0: 877.0. Samples: 554944. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:32:12,727][00804] Avg episode reward: [(0, '17.231')] -[2023-02-22 19:32:17,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2240512. Throughput: 0: 889.4. Samples: 560054. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:32:17,721][00804] Avg episode reward: [(0, '15.837')] -[2023-02-22 19:32:19,671][11055] Updated weights for policy 0, policy_version 550 (0.0017) -[2023-02-22 19:32:22,718][00804] Fps is (10 sec: 4096.4, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2265088. Throughput: 0: 925.5. Samples: 566642. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:32:22,720][00804] Avg episode reward: [(0, '15.372')] -[2023-02-22 19:32:27,720][00804] Fps is (10 sec: 4095.0, 60 sec: 3549.8, 300 sec: 3610.1). Total num frames: 2281472. Throughput: 0: 910.7. Samples: 569390. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:32:27,723][00804] Avg episode reward: [(0, '14.448')] -[2023-02-22 19:32:32,117][11055] Updated weights for policy 0, policy_version 560 (0.0027) -[2023-02-22 19:32:32,718][00804] Fps is (10 sec: 2867.1, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 2293760. Throughput: 0: 864.0. Samples: 573422. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:32:32,728][00804] Avg episode reward: [(0, '14.982')] -[2023-02-22 19:32:37,718][00804] Fps is (10 sec: 3277.6, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2314240. Throughput: 0: 897.7. Samples: 579134. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:32:37,726][00804] Avg episode reward: [(0, '14.537')] -[2023-02-22 19:32:41,798][11055] Updated weights for policy 0, policy_version 570 (0.0015) -[2023-02-22 19:32:42,718][00804] Fps is (10 sec: 4096.1, 60 sec: 3549.9, 300 sec: 3610.0). Total num frames: 2334720. Throughput: 0: 919.0. Samples: 582430. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:32:42,720][00804] Avg episode reward: [(0, '13.628')] -[2023-02-22 19:32:47,718][00804] Fps is (10 sec: 3686.2, 60 sec: 3549.8, 300 sec: 3596.1). Total num frames: 2351104. Throughput: 0: 893.8. Samples: 587736. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:32:47,723][00804] Avg episode reward: [(0, '13.846')] -[2023-02-22 19:32:52,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 2363392. Throughput: 0: 869.7. Samples: 592036. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:32:52,724][00804] Avg episode reward: [(0, '13.971')] -[2023-02-22 19:32:54,566][11055] Updated weights for policy 0, policy_version 580 (0.0012) -[2023-02-22 19:32:57,718][00804] Fps is (10 sec: 3686.6, 60 sec: 3618.2, 300 sec: 3623.9). Total num frames: 2387968. Throughput: 0: 893.8. Samples: 595166. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:32:57,721][00804] Avg episode reward: [(0, '14.018')] -[2023-02-22 19:33:02,718][00804] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3610.0). Total num frames: 2408448. Throughput: 0: 923.6. Samples: 601618. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:33:02,720][00804] Avg episode reward: [(0, '14.242')] -[2023-02-22 19:33:04,876][11055] Updated weights for policy 0, policy_version 590 (0.0014) -[2023-02-22 19:33:07,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 2420736. Throughput: 0: 878.4. Samples: 606170. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:33:07,721][00804] Avg episode reward: [(0, '14.564')] -[2023-02-22 19:33:12,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3596.2). Total num frames: 2437120. Throughput: 0: 863.3. Samples: 608238. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:33:12,725][00804] Avg episode reward: [(0, '14.844')] -[2023-02-22 19:33:12,739][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000595_2437120.pth... -[2023-02-22 19:33:12,843][11041] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000385_1576960.pth -[2023-02-22 19:33:17,719][00804] Fps is (10 sec: 3276.4, 60 sec: 3549.8, 300 sec: 3582.2). Total num frames: 2453504. Throughput: 0: 889.4. Samples: 613448. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:33:17,724][00804] Avg episode reward: [(0, '15.366')] -[2023-02-22 19:33:18,316][11055] Updated weights for policy 0, policy_version 600 (0.0024) -[2023-02-22 19:33:22,719][00804] Fps is (10 sec: 2866.9, 60 sec: 3345.0, 300 sec: 3554.5). Total num frames: 2465792. Throughput: 0: 850.7. Samples: 617416. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:33:22,723][00804] Avg episode reward: [(0, '16.173')] -[2023-02-22 19:33:27,718][00804] Fps is (10 sec: 2457.9, 60 sec: 3276.9, 300 sec: 3526.7). Total num frames: 2478080. Throughput: 0: 814.8. Samples: 619094. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:33:27,720][00804] Avg episode reward: [(0, '16.870')] -[2023-02-22 19:33:32,718][00804] Fps is (10 sec: 2867.5, 60 sec: 3345.1, 300 sec: 3540.6). Total num frames: 2494464. Throughput: 0: 786.1. Samples: 623110. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:33:32,724][00804] Avg episode reward: [(0, '18.095')] -[2023-02-22 19:33:33,424][11055] Updated weights for policy 0, policy_version 610 (0.0022) -[2023-02-22 19:33:37,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3526.7). Total num frames: 2514944. Throughput: 0: 829.5. Samples: 629364. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:33:37,725][00804] Avg episode reward: [(0, '19.023')] -[2023-02-22 19:33:37,732][11041] Saving new best policy, reward=19.023! -[2023-02-22 19:33:42,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3526.7). Total num frames: 2535424. Throughput: 0: 832.0. Samples: 632604. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:33:42,721][00804] Avg episode reward: [(0, '19.522')] -[2023-02-22 19:33:42,736][11041] Saving new best policy, reward=19.522! -[2023-02-22 19:33:43,404][11055] Updated weights for policy 0, policy_version 620 (0.0011) -[2023-02-22 19:33:47,720][00804] Fps is (10 sec: 3276.1, 60 sec: 3276.7, 300 sec: 3512.8). Total num frames: 2547712. Throughput: 0: 793.1. Samples: 637310. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:33:47,727][00804] Avg episode reward: [(0, '19.712')] -[2023-02-22 19:33:47,732][11041] Saving new best policy, reward=19.712! -[2023-02-22 19:33:52,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3526.7). Total num frames: 2564096. Throughput: 0: 791.2. Samples: 641772. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:33:52,724][00804] Avg episode reward: [(0, '19.523')] -[2023-02-22 19:33:55,866][11055] Updated weights for policy 0, policy_version 630 (0.0014) -[2023-02-22 19:33:57,718][00804] Fps is (10 sec: 3687.2, 60 sec: 3276.8, 300 sec: 3554.5). Total num frames: 2584576. Throughput: 0: 815.1. Samples: 644918. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:33:57,722][00804] Avg episode reward: [(0, '19.646')] -[2023-02-22 19:34:02,718][00804] Fps is (10 sec: 4096.1, 60 sec: 3276.8, 300 sec: 3554.5). Total num frames: 2605056. Throughput: 0: 841.0. Samples: 651294. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:34:02,720][00804] Avg episode reward: [(0, '19.420')] -[2023-02-22 19:34:07,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3526.8). Total num frames: 2617344. Throughput: 0: 840.9. Samples: 655254. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:34:07,724][00804] Avg episode reward: [(0, '20.185')] -[2023-02-22 19:34:07,726][11041] Saving new best policy, reward=20.185! -[2023-02-22 19:34:08,300][11055] Updated weights for policy 0, policy_version 640 (0.0025) -[2023-02-22 19:34:12,719][00804] Fps is (10 sec: 2867.0, 60 sec: 3276.8, 300 sec: 3540.6). Total num frames: 2633728. Throughput: 0: 846.1. Samples: 657170. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:34:12,731][00804] Avg episode reward: [(0, '20.738')] -[2023-02-22 19:34:12,771][11041] Saving new best policy, reward=20.738! -[2023-02-22 19:34:17,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3413.4, 300 sec: 3554.5). Total num frames: 2658304. Throughput: 0: 900.1. Samples: 663616. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:34:17,726][00804] Avg episode reward: [(0, '20.954')] -[2023-02-22 19:34:17,729][11041] Saving new best policy, reward=20.954! -[2023-02-22 19:34:18,744][11055] Updated weights for policy 0, policy_version 650 (0.0031) -[2023-02-22 19:34:22,718][00804] Fps is (10 sec: 4096.3, 60 sec: 3481.7, 300 sec: 3540.6). Total num frames: 2674688. Throughput: 0: 890.2. Samples: 669424. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:34:22,724][00804] Avg episode reward: [(0, '19.897')] -[2023-02-22 19:34:27,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 2686976. Throughput: 0: 864.9. Samples: 671526. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:34:27,720][00804] Avg episode reward: [(0, '18.935')] -[2023-02-22 19:34:31,238][11055] Updated weights for policy 0, policy_version 660 (0.0013) -[2023-02-22 19:34:32,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.7). Total num frames: 2707456. Throughput: 0: 867.8. Samples: 676360. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:34:32,726][00804] Avg episode reward: [(0, '19.296')] -[2023-02-22 19:34:37,718][00804] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 2732032. Throughput: 0: 918.0. Samples: 683084. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:34:37,726][00804] Avg episode reward: [(0, '19.664')] -[2023-02-22 19:34:40,678][11055] Updated weights for policy 0, policy_version 670 (0.0012) -[2023-02-22 19:34:42,722][00804] Fps is (10 sec: 4094.4, 60 sec: 3549.6, 300 sec: 3540.6). Total num frames: 2748416. Throughput: 0: 917.5. Samples: 686210. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:34:42,729][00804] Avg episode reward: [(0, '19.804')] -[2023-02-22 19:34:47,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3550.0, 300 sec: 3512.8). Total num frames: 2760704. Throughput: 0: 866.6. Samples: 690290. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:34:47,720][00804] Avg episode reward: [(0, '20.264')] -[2023-02-22 19:34:52,718][00804] Fps is (10 sec: 3278.1, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 2781184. Throughput: 0: 900.4. Samples: 695774. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:34:52,726][00804] Avg episode reward: [(0, '20.557')] -[2023-02-22 19:34:53,235][11055] Updated weights for policy 0, policy_version 680 (0.0031) -[2023-02-22 19:34:57,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 2801664. Throughput: 0: 930.7. Samples: 699050. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:34:57,720][00804] Avg episode reward: [(0, '20.915')] -[2023-02-22 19:35:02,719][00804] Fps is (10 sec: 3686.1, 60 sec: 3549.8, 300 sec: 3526.7). Total num frames: 2818048. Throughput: 0: 913.7. Samples: 704732. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:35:02,732][00804] Avg episode reward: [(0, '19.946')] -[2023-02-22 19:35:04,778][11055] Updated weights for policy 0, policy_version 690 (0.0014) -[2023-02-22 19:35:07,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 2834432. Throughput: 0: 875.1. Samples: 708802. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:35:07,731][00804] Avg episode reward: [(0, '19.905')] -[2023-02-22 19:35:12,718][00804] Fps is (10 sec: 3686.7, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 2854912. Throughput: 0: 891.6. Samples: 711648. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:35:12,721][00804] Avg episode reward: [(0, '20.442')] -[2023-02-22 19:35:12,739][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000697_2854912.pth... -[2023-02-22 19:35:12,864][11041] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000490_2007040.pth -[2023-02-22 19:35:15,486][11055] Updated weights for policy 0, policy_version 700 (0.0026) -[2023-02-22 19:35:17,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 2875392. Throughput: 0: 930.2. Samples: 718220. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:35:17,721][00804] Avg episode reward: [(0, '21.393')] -[2023-02-22 19:35:17,729][11041] Saving new best policy, reward=21.393! -[2023-02-22 19:35:22,718][00804] Fps is (10 sec: 3686.2, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 2891776. Throughput: 0: 887.5. Samples: 723024. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:35:22,731][00804] Avg episode reward: [(0, '20.745')] -[2023-02-22 19:35:27,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 2904064. Throughput: 0: 863.4. Samples: 725060. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:35:27,724][00804] Avg episode reward: [(0, '20.015')] -[2023-02-22 19:35:28,478][11055] Updated weights for policy 0, policy_version 710 (0.0016) -[2023-02-22 19:35:32,718][00804] Fps is (10 sec: 3277.0, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 2924544. Throughput: 0: 897.9. Samples: 730694. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:35:32,723][00804] Avg episode reward: [(0, '20.294')] -[2023-02-22 19:35:37,719][00804] Fps is (10 sec: 4095.4, 60 sec: 3549.8, 300 sec: 3540.6). Total num frames: 2945024. Throughput: 0: 924.3. Samples: 737370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:35:37,724][00804] Avg episode reward: [(0, '20.140')] -[2023-02-22 19:35:37,760][11055] Updated weights for policy 0, policy_version 720 (0.0015) -[2023-02-22 19:35:42,721][00804] Fps is (10 sec: 3685.2, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 2961408. Throughput: 0: 899.4. Samples: 739526. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:35:42,723][00804] Avg episode reward: [(0, '21.348')] -[2023-02-22 19:35:47,718][00804] Fps is (10 sec: 3277.3, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 2977792. Throughput: 0: 866.8. Samples: 743736. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:35:47,730][00804] Avg episode reward: [(0, '20.515')] -[2023-02-22 19:35:50,412][11055] Updated weights for policy 0, policy_version 730 (0.0024) -[2023-02-22 19:35:52,718][00804] Fps is (10 sec: 3687.5, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 2998272. Throughput: 0: 917.7. Samples: 750098. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:35:52,726][00804] Avg episode reward: [(0, '21.592')] -[2023-02-22 19:35:52,743][11041] Saving new best policy, reward=21.592! -[2023-02-22 19:35:57,724][00804] Fps is (10 sec: 4093.6, 60 sec: 3617.8, 300 sec: 3540.5). Total num frames: 3018752. Throughput: 0: 922.0. Samples: 753142. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:35:57,729][00804] Avg episode reward: [(0, '22.688')] -[2023-02-22 19:35:57,735][11041] Saving new best policy, reward=22.688! -[2023-02-22 19:36:02,148][11055] Updated weights for policy 0, policy_version 740 (0.0012) -[2023-02-22 19:36:02,718][00804] Fps is (10 sec: 3276.9, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3031040. Throughput: 0: 875.7. Samples: 757628. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:36:02,722][00804] Avg episode reward: [(0, '21.838')] -[2023-02-22 19:36:07,718][00804] Fps is (10 sec: 2868.9, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 3047424. Throughput: 0: 870.8. Samples: 762208. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:36:07,721][00804] Avg episode reward: [(0, '20.676')] -[2023-02-22 19:36:12,718][00804] Fps is (10 sec: 3686.3, 60 sec: 3549.8, 300 sec: 3540.6). Total num frames: 3067904. Throughput: 0: 899.0. Samples: 765516. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:36:12,721][00804] Avg episode reward: [(0, '19.683')] -[2023-02-22 19:36:12,982][11055] Updated weights for policy 0, policy_version 750 (0.0045) -[2023-02-22 19:36:17,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 3088384. Throughput: 0: 919.1. Samples: 772054. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:36:17,721][00804] Avg episode reward: [(0, '19.689')] -[2023-02-22 19:36:22,719][00804] Fps is (10 sec: 3276.6, 60 sec: 3481.6, 300 sec: 3498.9). Total num frames: 3100672. Throughput: 0: 861.7. Samples: 776146. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:36:22,736][00804] Avg episode reward: [(0, '18.274')] -[2023-02-22 19:36:25,774][11055] Updated weights for policy 0, policy_version 760 (0.0012) -[2023-02-22 19:36:27,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 3121152. Throughput: 0: 862.6. Samples: 778342. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:36:27,725][00804] Avg episode reward: [(0, '18.567')] -[2023-02-22 19:36:32,718][00804] Fps is (10 sec: 4096.4, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 3141632. Throughput: 0: 912.9. Samples: 784816. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:36:32,721][00804] Avg episode reward: [(0, '18.209')] -[2023-02-22 19:36:35,540][11055] Updated weights for policy 0, policy_version 770 (0.0013) -[2023-02-22 19:36:37,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3158016. Throughput: 0: 892.8. Samples: 790272. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:36:37,723][00804] Avg episode reward: [(0, '20.174')] -[2023-02-22 19:36:42,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3550.1, 300 sec: 3512.8). Total num frames: 3174400. Throughput: 0: 871.8. Samples: 792368. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:36:42,721][00804] Avg episode reward: [(0, '21.874')] -[2023-02-22 19:36:47,718][00804] Fps is (10 sec: 3276.7, 60 sec: 3549.8, 300 sec: 3526.7). Total num frames: 3190784. Throughput: 0: 882.5. Samples: 797342. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:36:47,728][00804] Avg episode reward: [(0, '21.858')] -[2023-02-22 19:36:47,977][11055] Updated weights for policy 0, policy_version 780 (0.0022) -[2023-02-22 19:36:52,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.2, 300 sec: 3540.6). Total num frames: 3215360. Throughput: 0: 931.3. Samples: 804116. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:36:52,728][00804] Avg episode reward: [(0, '24.463')] -[2023-02-22 19:36:52,740][11041] Saving new best policy, reward=24.463! -[2023-02-22 19:36:57,718][00804] Fps is (10 sec: 4096.1, 60 sec: 3550.2, 300 sec: 3526.7). Total num frames: 3231744. Throughput: 0: 922.5. Samples: 807028. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:36:57,727][00804] Avg episode reward: [(0, '25.598')] -[2023-02-22 19:36:57,733][11041] Saving new best policy, reward=25.598! -[2023-02-22 19:36:58,838][11055] Updated weights for policy 0, policy_version 790 (0.0012) -[2023-02-22 19:37:02,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3244032. Throughput: 0: 868.6. Samples: 811142. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:37:02,725][00804] Avg episode reward: [(0, '26.121')] -[2023-02-22 19:37:02,736][11041] Saving new best policy, reward=26.121! -[2023-02-22 19:37:07,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 3264512. Throughput: 0: 900.0. Samples: 816644. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:37:07,726][00804] Avg episode reward: [(0, '26.207')] -[2023-02-22 19:37:07,730][11041] Saving new best policy, reward=26.207! -[2023-02-22 19:37:10,167][11055] Updated weights for policy 0, policy_version 800 (0.0016) -[2023-02-22 19:37:12,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3618.2, 300 sec: 3540.6). Total num frames: 3284992. Throughput: 0: 920.5. Samples: 819764. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:37:12,721][00804] Avg episode reward: [(0, '24.942')] -[2023-02-22 19:37:12,728][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000802_3284992.pth... -[2023-02-22 19:37:12,839][11041] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000595_2437120.pth -[2023-02-22 19:37:17,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 3301376. Throughput: 0: 898.3. Samples: 825240. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:37:17,720][00804] Avg episode reward: [(0, '25.108')] -[2023-02-22 19:37:22,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 3313664. Throughput: 0: 871.2. Samples: 829478. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:37:22,723][00804] Avg episode reward: [(0, '25.359')] -[2023-02-22 19:37:22,941][11055] Updated weights for policy 0, policy_version 810 (0.0018) -[2023-02-22 19:37:27,720][00804] Fps is (10 sec: 3685.5, 60 sec: 3618.0, 300 sec: 3540.6). Total num frames: 3338240. Throughput: 0: 891.6. Samples: 832490. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:37:27,726][00804] Avg episode reward: [(0, '25.081')] -[2023-02-22 19:37:32,233][11055] Updated weights for policy 0, policy_version 820 (0.0011) -[2023-02-22 19:37:32,718][00804] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 3358720. Throughput: 0: 928.5. Samples: 839124. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:37:32,723][00804] Avg episode reward: [(0, '24.504')] -[2023-02-22 19:37:37,718][00804] Fps is (10 sec: 3277.5, 60 sec: 3549.8, 300 sec: 3512.8). Total num frames: 3371008. Throughput: 0: 881.2. Samples: 843772. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:37:37,723][00804] Avg episode reward: [(0, '24.388')] -[2023-02-22 19:37:42,718][00804] Fps is (10 sec: 2457.5, 60 sec: 3481.6, 300 sec: 3499.0). Total num frames: 3383296. Throughput: 0: 852.0. Samples: 845368. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:37:42,725][00804] Avg episode reward: [(0, '24.836')] -[2023-02-22 19:37:47,718][00804] Fps is (10 sec: 2457.7, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 3395584. Throughput: 0: 833.2. Samples: 848638. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:37:47,724][00804] Avg episode reward: [(0, '25.277')] -[2023-02-22 19:37:48,421][11055] Updated weights for policy 0, policy_version 830 (0.0042) -[2023-02-22 19:37:52,718][00804] Fps is (10 sec: 3276.9, 60 sec: 3345.1, 300 sec: 3485.1). Total num frames: 3416064. Throughput: 0: 846.2. Samples: 854722. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:37:52,721][00804] Avg episode reward: [(0, '23.068')] -[2023-02-22 19:37:57,721][00804] Fps is (10 sec: 4095.2, 60 sec: 3413.2, 300 sec: 3485.0). Total num frames: 3436544. Throughput: 0: 849.3. Samples: 857982. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:37:57,723][00804] Avg episode reward: [(0, '23.583')] -[2023-02-22 19:37:59,044][11055] Updated weights for policy 0, policy_version 840 (0.0025) -[2023-02-22 19:38:02,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 3448832. Throughput: 0: 823.4. Samples: 862294. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:38:02,722][00804] Avg episode reward: [(0, '23.858')] -[2023-02-22 19:38:07,718][00804] Fps is (10 sec: 3277.5, 60 sec: 3413.3, 300 sec: 3499.0). Total num frames: 3469312. Throughput: 0: 841.3. Samples: 867336. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:38:07,725][00804] Avg episode reward: [(0, '24.572')] -[2023-02-22 19:38:10,666][11055] Updated weights for policy 0, policy_version 850 (0.0015) -[2023-02-22 19:38:12,718][00804] Fps is (10 sec: 4096.1, 60 sec: 3413.3, 300 sec: 3512.9). Total num frames: 3489792. Throughput: 0: 847.6. Samples: 870630. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:38:12,721][00804] Avg episode reward: [(0, '24.184')] -[2023-02-22 19:38:17,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 3506176. Throughput: 0: 835.7. Samples: 876732. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:38:17,725][00804] Avg episode reward: [(0, '23.544')] -[2023-02-22 19:38:22,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3526.7). Total num frames: 3518464. Throughput: 0: 822.0. Samples: 880762. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:38:22,728][00804] Avg episode reward: [(0, '23.787')] -[2023-02-22 19:38:22,968][11055] Updated weights for policy 0, policy_version 860 (0.0012) -[2023-02-22 19:38:27,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3345.2, 300 sec: 3540.6). Total num frames: 3538944. Throughput: 0: 844.8. Samples: 883384. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:38:27,727][00804] Avg episode reward: [(0, '23.567')] -[2023-02-22 19:38:32,479][11055] Updated weights for policy 0, policy_version 870 (0.0020) -[2023-02-22 19:38:32,718][00804] Fps is (10 sec: 4505.6, 60 sec: 3413.3, 300 sec: 3554.5). Total num frames: 3563520. Throughput: 0: 919.7. Samples: 890024. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) -[2023-02-22 19:38:32,723][00804] Avg episode reward: [(0, '23.328')] -[2023-02-22 19:38:37,718][00804] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 3579904. Throughput: 0: 902.4. Samples: 895332. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:38:37,722][00804] Avg episode reward: [(0, '22.489')] -[2023-02-22 19:38:42,718][00804] Fps is (10 sec: 2867.1, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 3592192. Throughput: 0: 877.2. Samples: 897454. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:38:42,727][00804] Avg episode reward: [(0, '21.861')] -[2023-02-22 19:38:45,228][11055] Updated weights for policy 0, policy_version 880 (0.0023) -[2023-02-22 19:38:47,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3612672. Throughput: 0: 901.9. Samples: 902880. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:38:47,726][00804] Avg episode reward: [(0, '22.385')] -[2023-02-22 19:38:52,718][00804] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 3637248. Throughput: 0: 939.7. Samples: 909622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:38:52,726][00804] Avg episode reward: [(0, '23.322')] -[2023-02-22 19:38:54,915][11055] Updated weights for policy 0, policy_version 890 (0.0019) -[2023-02-22 19:38:57,718][00804] Fps is (10 sec: 4095.9, 60 sec: 3618.2, 300 sec: 3554.5). Total num frames: 3653632. Throughput: 0: 923.8. Samples: 912202. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:38:57,722][00804] Avg episode reward: [(0, '23.527')] -[2023-02-22 19:39:02,720][00804] Fps is (10 sec: 2866.6, 60 sec: 3618.0, 300 sec: 3554.5). Total num frames: 3665920. Throughput: 0: 885.4. Samples: 916576. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:39:02,723][00804] Avg episode reward: [(0, '22.808')] -[2023-02-22 19:39:06,753][11055] Updated weights for policy 0, policy_version 900 (0.0025) -[2023-02-22 19:39:07,718][00804] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 3690496. Throughput: 0: 929.5. Samples: 922588. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:39:07,720][00804] Avg episode reward: [(0, '23.872')] -[2023-02-22 19:39:12,718][00804] Fps is (10 sec: 4506.6, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 3710976. Throughput: 0: 946.4. Samples: 925974. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:39:12,726][00804] Avg episode reward: [(0, '24.689')] -[2023-02-22 19:39:12,739][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000906_3710976.pth... -[2023-02-22 19:39:12,852][11041] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000697_2854912.pth -[2023-02-22 19:39:17,718][00804] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 3723264. Throughput: 0: 917.6. Samples: 931316. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:39:17,725][00804] Avg episode reward: [(0, '24.809')] -[2023-02-22 19:39:17,743][11055] Updated weights for policy 0, policy_version 910 (0.0013) -[2023-02-22 19:39:22,718][00804] Fps is (10 sec: 2867.2, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 3739648. Throughput: 0: 892.9. Samples: 935514. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:39:22,720][00804] Avg episode reward: [(0, '24.071')] -[2023-02-22 19:39:27,720][00804] Fps is (10 sec: 4094.9, 60 sec: 3754.5, 300 sec: 3582.2). Total num frames: 3764224. Throughput: 0: 919.8. Samples: 938848. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:39:27,729][00804] Avg episode reward: [(0, '24.038')] -[2023-02-22 19:39:28,619][11055] Updated weights for policy 0, policy_version 920 (0.0015) -[2023-02-22 19:39:32,721][00804] Fps is (10 sec: 4504.1, 60 sec: 3686.2, 300 sec: 3568.3). Total num frames: 3784704. Throughput: 0: 944.1. Samples: 945368. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:39:32,723][00804] Avg episode reward: [(0, '25.252')] -[2023-02-22 19:39:37,720][00804] Fps is (10 sec: 3276.9, 60 sec: 3618.0, 300 sec: 3554.5). Total num frames: 3796992. Throughput: 0: 896.8. Samples: 949978. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:39:37,726][00804] Avg episode reward: [(0, '24.801')] -[2023-02-22 19:39:41,551][11055] Updated weights for policy 0, policy_version 930 (0.0018) -[2023-02-22 19:39:42,718][00804] Fps is (10 sec: 2868.2, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 3813376. Throughput: 0: 885.7. Samples: 952058. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:39:42,728][00804] Avg episode reward: [(0, '25.037')] -[2023-02-22 19:39:47,718][00804] Fps is (10 sec: 3687.3, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 3833856. Throughput: 0: 922.3. Samples: 958076. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:39:47,720][00804] Avg episode reward: [(0, '25.544')] -[2023-02-22 19:39:50,742][11055] Updated weights for policy 0, policy_version 940 (0.0017) -[2023-02-22 19:39:52,722][00804] Fps is (10 sec: 4094.5, 60 sec: 3617.9, 300 sec: 3568.3). Total num frames: 3854336. Throughput: 0: 930.7. Samples: 964472. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:39:52,724][00804] Avg episode reward: [(0, '25.286')] -[2023-02-22 19:39:57,718][00804] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 3870720. Throughput: 0: 902.1. Samples: 966570. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) -[2023-02-22 19:39:57,726][00804] Avg episode reward: [(0, '26.110')] -[2023-02-22 19:40:02,718][00804] Fps is (10 sec: 3277.9, 60 sec: 3686.5, 300 sec: 3568.4). Total num frames: 3887104. Throughput: 0: 878.7. Samples: 970856. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:40:02,721][00804] Avg episode reward: [(0, '25.651')] -[2023-02-22 19:40:03,696][11055] Updated weights for policy 0, policy_version 950 (0.0018) -[2023-02-22 19:40:07,718][00804] Fps is (10 sec: 3686.5, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 3907584. Throughput: 0: 928.9. Samples: 977316. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) -[2023-02-22 19:40:07,723][00804] Avg episode reward: [(0, '24.852')] -[2023-02-22 19:40:12,719][00804] Fps is (10 sec: 4095.8, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 3928064. Throughput: 0: 928.9. Samples: 980648. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) -[2023-02-22 19:40:12,723][00804] Avg episode reward: [(0, '22.974')] -[2023-02-22 19:40:13,775][11055] Updated weights for policy 0, policy_version 960 (0.0011) -[2023-02-22 19:40:17,723][00804] Fps is (10 sec: 3275.0, 60 sec: 3617.8, 300 sec: 3554.4). Total num frames: 3940352. Throughput: 0: 884.1. Samples: 985154. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) -[2023-02-22 19:40:17,730][00804] Avg episode reward: [(0, '21.882')] -[2023-02-22 19:40:22,718][00804] Fps is (10 sec: 2867.5, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 3956736. Throughput: 0: 889.9. Samples: 990020. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) -[2023-02-22 19:40:22,721][00804] Avg episode reward: [(0, '21.508')] -[2023-02-22 19:40:25,834][11055] Updated weights for policy 0, policy_version 970 (0.0034) -[2023-02-22 19:40:27,718][00804] Fps is (10 sec: 4098.2, 60 sec: 3618.3, 300 sec: 3582.3). Total num frames: 3981312. Throughput: 0: 915.2. Samples: 993244. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) -[2023-02-22 19:40:27,721][00804] Avg episode reward: [(0, '20.654')] -[2023-02-22 19:40:32,720][00804] Fps is (10 sec: 4095.1, 60 sec: 3549.9, 300 sec: 3568.4). Total num frames: 3997696. Throughput: 0: 917.9. Samples: 999384. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) -[2023-02-22 19:40:32,723][00804] Avg episode reward: [(0, '21.686')] -[2023-02-22 19:40:34,982][11041] Stopping Batcher_0... -[2023-02-22 19:40:34,983][11041] Loop batcher_evt_loop terminating... -[2023-02-22 19:40:34,984][00804] Component Batcher_0 stopped! -[2023-02-22 19:40:34,986][00804] Component RolloutWorker_w0 process died already! Don't wait for it. -[2023-02-22 19:40:34,995][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2023-02-22 19:40:35,062][11055] Weights refcount: 2 0 -[2023-02-22 19:40:35,065][11055] Stopping InferenceWorker_p0-w0... -[2023-02-22 19:40:35,066][00804] Component InferenceWorker_p0-w0 stopped! -[2023-02-22 19:40:35,078][00804] Component RolloutWorker_w1 stopped! -[2023-02-22 19:40:35,082][11057] Stopping RolloutWorker_w1... -[2023-02-22 19:40:35,082][11057] Loop rollout_proc1_evt_loop terminating... -[2023-02-22 19:40:35,093][00804] Component RolloutWorker_w5 stopped! -[2023-02-22 19:40:35,095][11061] Stopping RolloutWorker_w5... -[2023-02-22 19:40:35,065][11055] Loop inference_proc0-0_evt_loop terminating... -[2023-02-22 19:40:35,102][11061] Loop rollout_proc5_evt_loop terminating... -[2023-02-22 19:40:35,124][00804] Component RolloutWorker_w3 stopped! -[2023-02-22 19:40:35,127][11058] Stopping RolloutWorker_w3... -[2023-02-22 19:40:35,140][11060] Stopping RolloutWorker_w4... -[2023-02-22 19:40:35,140][00804] Component RolloutWorker_w4 stopped! -[2023-02-22 19:40:35,146][00804] Component RolloutWorker_w7 stopped! -[2023-02-22 19:40:35,150][11063] Stopping RolloutWorker_w7... -[2023-02-22 19:40:35,151][11063] Loop rollout_proc7_evt_loop terminating... -[2023-02-22 19:40:35,136][11058] Loop rollout_proc3_evt_loop terminating... -[2023-02-22 19:40:35,162][11060] Loop rollout_proc4_evt_loop terminating... -[2023-02-22 19:40:35,192][11041] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000802_3284992.pth -[2023-02-22 19:40:35,201][11041] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2023-02-22 19:40:35,227][00804] Component RolloutWorker_w2 stopped! -[2023-02-22 19:40:35,233][11059] Stopping RolloutWorker_w2... -[2023-02-22 19:40:35,233][11059] Loop rollout_proc2_evt_loop terminating... -[2023-02-22 19:40:35,239][00804] Component RolloutWorker_w6 stopped! -[2023-02-22 19:40:35,241][11062] Stopping RolloutWorker_w6... -[2023-02-22 19:40:35,249][11062] Loop rollout_proc6_evt_loop terminating... -[2023-02-22 19:40:35,435][00804] Component LearnerWorker_p0 stopped! -[2023-02-22 19:40:35,439][00804] Waiting for process learner_proc0 to stop... -[2023-02-22 19:40:35,445][11041] Stopping LearnerWorker_p0... -[2023-02-22 19:40:35,446][11041] Loop learner_proc0_evt_loop terminating... -[2023-02-22 19:40:37,695][00804] Waiting for process inference_proc0-0 to join... -[2023-02-22 19:40:38,526][00804] Waiting for process rollout_proc0 to join... -[2023-02-22 19:40:38,533][00804] Waiting for process rollout_proc1 to join... -[2023-02-22 19:40:38,783][00804] Waiting for process rollout_proc2 to join... -[2023-02-22 19:40:38,911][00804] Waiting for process rollout_proc3 to join... -[2023-02-22 19:40:38,913][00804] Waiting for process rollout_proc4 to join... -[2023-02-22 19:40:38,915][00804] Waiting for process rollout_proc5 to join... -[2023-02-22 19:40:38,916][00804] Waiting for process rollout_proc6 to join... -[2023-02-22 19:40:38,917][00804] Waiting for process rollout_proc7 to join... -[2023-02-22 19:40:38,920][00804] Batcher 0 profile tree view: -batching: 26.2346, releasing_batches: 0.0318 -[2023-02-22 19:40:38,922][00804] InferenceWorker_p0-w0 profile tree view: -wait_policy: 0.0000 - wait_policy_total: 544.7672 -update_model: 8.1558 - weight_update: 0.0017 -one_step: 0.0169 - handle_policy_step: 535.7754 - deserialize: 15.6744, stack: 3.1573, obs_to_device_normalize: 119.9518, forward: 260.5496, send_messages: 25.1291 - prepare_outputs: 84.2802 - to_cpu: 52.2570 -[2023-02-22 19:40:38,923][00804] Learner 0 profile tree view: -misc: 0.0062, prepare_batch: 15.1086 -train: 74.6749 - epoch_init: 0.0159, minibatch_init: 0.0163, losses_postprocess: 0.5405, kl_divergence: 0.5935, after_optimizer: 32.6658 - calculate_losses: 26.1922 - losses_init: 0.0076, forward_head: 1.7194, bptt_initial: 17.2524, tail: 0.9984, advantages_returns: 0.3143, losses: 3.4254 - bptt: 2.1589 - bptt_forward_core: 2.0823 - update: 13.9871 - clip: 1.4154 -[2023-02-22 19:40:38,927][00804] RolloutWorker_w7 profile tree view: -wait_for_trajectories: 0.3476, enqueue_policy_requests: 146.1553, env_step: 855.0100, overhead: 23.1727, complete_rollouts: 8.0434 -save_policy_outputs: 21.3814 - split_output_tensors: 10.2768 -[2023-02-22 19:40:38,928][00804] Loop Runner_EvtLoop terminating... -[2023-02-22 19:40:38,930][00804] Runner profile tree view: -main_loop: 1160.8495 -[2023-02-22 19:40:38,933][00804] Collected {0: 4005888}, FPS: 3450.8 -[2023-02-22 19:40:39,101][00804] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2023-02-22 19:40:39,103][00804] Overriding arg 'num_workers' with value 1 passed from command line -[2023-02-22 19:40:39,106][00804] Adding new argument 'no_render'=True that is not in the saved config file! -[2023-02-22 19:40:39,108][00804] Adding new argument 'save_video'=True that is not in the saved config file! -[2023-02-22 19:40:39,110][00804] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2023-02-22 19:40:39,112][00804] Adding new argument 'video_name'=None that is not in the saved config file! -[2023-02-22 19:40:39,114][00804] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! -[2023-02-22 19:40:39,115][00804] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2023-02-22 19:40:39,117][00804] Adding new argument 'push_to_hub'=False that is not in the saved config file! -[2023-02-22 19:40:39,118][00804] Adding new argument 'hf_repository'=None that is not in the saved config file! -[2023-02-22 19:40:39,119][00804] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2023-02-22 19:40:39,121][00804] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2023-02-22 19:40:39,122][00804] Adding new argument 'train_script'=None that is not in the saved config file! -[2023-02-22 19:40:39,123][00804] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2023-02-22 19:40:39,125][00804] Using frameskip 1 and render_action_repeat=4 for evaluation -[2023-02-22 19:40:39,153][00804] Doom resolution: 160x120, resize resolution: (128, 72) -[2023-02-22 19:40:39,156][00804] RunningMeanStd input shape: (3, 72, 128) -[2023-02-22 19:40:39,157][00804] RunningMeanStd input shape: (1,) -[2023-02-22 19:40:39,174][00804] ConvEncoder: input_channels=3 -[2023-02-22 19:40:39,889][00804] Conv encoder output size: 512 -[2023-02-22 19:40:39,892][00804] Policy head output size: 512 -[2023-02-22 19:40:42,361][00804] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2023-02-22 19:40:43,689][00804] Num frames 100... -[2023-02-22 19:40:43,809][00804] Num frames 200... -[2023-02-22 19:40:43,930][00804] Num frames 300... -[2023-02-22 19:40:44,045][00804] Num frames 400... -[2023-02-22 19:40:44,160][00804] Num frames 500... -[2023-02-22 19:40:44,272][00804] Avg episode rewards: #0: 9.440, true rewards: #0: 5.440 -[2023-02-22 19:40:44,274][00804] Avg episode reward: 9.440, avg true_objective: 5.440 -[2023-02-22 19:40:44,341][00804] Num frames 600... -[2023-02-22 19:40:44,460][00804] Num frames 700... -[2023-02-22 19:40:44,574][00804] Num frames 800... -[2023-02-22 19:40:44,698][00804] Num frames 900... -[2023-02-22 19:40:44,821][00804] Num frames 1000... -[2023-02-22 19:40:44,948][00804] Num frames 1100... -[2023-02-22 19:40:45,068][00804] Num frames 1200... -[2023-02-22 19:40:45,184][00804] Num frames 1300... -[2023-02-22 19:40:45,296][00804] Num frames 1400... -[2023-02-22 19:40:45,423][00804] Num frames 1500... -[2023-02-22 19:40:45,537][00804] Num frames 1600... -[2023-02-22 19:40:45,656][00804] Num frames 1700... -[2023-02-22 19:40:45,784][00804] Num frames 1800... -[2023-02-22 19:40:45,902][00804] Avg episode rewards: #0: 21.280, true rewards: #0: 9.280 -[2023-02-22 19:40:45,904][00804] Avg episode reward: 21.280, avg true_objective: 9.280 -[2023-02-22 19:40:45,957][00804] Num frames 1900... -[2023-02-22 19:40:46,082][00804] Num frames 2000... -[2023-02-22 19:40:46,202][00804] Num frames 2100... -[2023-02-22 19:40:46,324][00804] Num frames 2200... -[2023-02-22 19:40:46,438][00804] Num frames 2300... -[2023-02-22 19:40:46,559][00804] Num frames 2400... -[2023-02-22 19:40:46,673][00804] Num frames 2500... -[2023-02-22 19:40:46,800][00804] Num frames 2600... -[2023-02-22 19:40:46,922][00804] Num frames 2700... -[2023-02-22 19:40:47,037][00804] Num frames 2800... -[2023-02-22 19:40:47,154][00804] Num frames 2900... -[2023-02-22 19:40:47,258][00804] Avg episode rewards: #0: 22.480, true rewards: #0: 9.813 -[2023-02-22 19:40:47,260][00804] Avg episode reward: 22.480, avg true_objective: 9.813 -[2023-02-22 19:40:47,333][00804] Num frames 3000... -[2023-02-22 19:40:47,447][00804] Num frames 3100... -[2023-02-22 19:40:47,569][00804] Num frames 3200... -[2023-02-22 19:40:47,679][00804] Num frames 3300... -[2023-02-22 19:40:47,799][00804] Num frames 3400... -[2023-02-22 19:40:47,915][00804] Num frames 3500... -[2023-02-22 19:40:48,036][00804] Avg episode rewards: #0: 19.880, true rewards: #0: 8.880 -[2023-02-22 19:40:48,038][00804] Avg episode reward: 19.880, avg true_objective: 8.880 -[2023-02-22 19:40:48,097][00804] Num frames 3600... -[2023-02-22 19:40:48,219][00804] Num frames 3700... -[2023-02-22 19:40:48,332][00804] Num frames 3800... -[2023-02-22 19:40:48,447][00804] Num frames 3900... -[2023-02-22 19:40:48,569][00804] Num frames 4000... -[2023-02-22 19:40:48,686][00804] Num frames 4100... -[2023-02-22 19:40:48,815][00804] Num frames 4200... -[2023-02-22 19:40:48,929][00804] Num frames 4300... -[2023-02-22 19:40:49,048][00804] Num frames 4400... -[2023-02-22 19:40:49,197][00804] Num frames 4500... -[2023-02-22 19:40:49,281][00804] Avg episode rewards: #0: 20.624, true rewards: #0: 9.024 -[2023-02-22 19:40:49,283][00804] Avg episode reward: 20.624, avg true_objective: 9.024 -[2023-02-22 19:40:49,428][00804] Num frames 4600... -[2023-02-22 19:40:49,590][00804] Num frames 4700... -[2023-02-22 19:40:49,766][00804] Num frames 4800... -[2023-02-22 19:40:49,951][00804] Num frames 4900... -[2023-02-22 19:40:50,134][00804] Num frames 5000... -[2023-02-22 19:40:50,300][00804] Num frames 5100... -[2023-02-22 19:40:50,464][00804] Num frames 5200... -[2023-02-22 19:40:50,619][00804] Num frames 5300... -[2023-02-22 19:40:50,791][00804] Num frames 5400... -[2023-02-22 19:40:50,960][00804] Num frames 5500... -[2023-02-22 19:40:51,133][00804] Num frames 5600... -[2023-02-22 19:40:51,296][00804] Num frames 5700... -[2023-02-22 19:40:51,471][00804] Num frames 5800... -[2023-02-22 19:40:51,636][00804] Num frames 5900... -[2023-02-22 19:40:51,728][00804] Avg episode rewards: #0: 22.700, true rewards: #0: 9.867 -[2023-02-22 19:40:51,730][00804] Avg episode reward: 22.700, avg true_objective: 9.867 -[2023-02-22 19:40:51,862][00804] Num frames 6000... -[2023-02-22 19:40:52,040][00804] Num frames 6100... -[2023-02-22 19:40:52,197][00804] Num frames 6200... -[2023-02-22 19:40:52,363][00804] Num frames 6300... -[2023-02-22 19:40:52,524][00804] Num frames 6400... -[2023-02-22 19:40:52,677][00804] Num frames 6500... -[2023-02-22 19:40:52,798][00804] Num frames 6600... -[2023-02-22 19:40:52,915][00804] Num frames 6700... -[2023-02-22 19:40:53,050][00804] Num frames 6800... -[2023-02-22 19:40:53,172][00804] Num frames 6900... -[2023-02-22 19:40:53,293][00804] Num frames 7000... -[2023-02-22 19:40:53,405][00804] Num frames 7100... -[2023-02-22 19:40:53,525][00804] Num frames 7200... -[2023-02-22 19:40:53,637][00804] Num frames 7300... -[2023-02-22 19:40:53,751][00804] Num frames 7400... -[2023-02-22 19:40:53,875][00804] Num frames 7500... -[2023-02-22 19:40:54,009][00804] Num frames 7600... -[2023-02-22 19:40:54,126][00804] Num frames 7700... -[2023-02-22 19:40:54,241][00804] Num frames 7800... -[2023-02-22 19:40:54,357][00804] Num frames 7900... -[2023-02-22 19:40:54,479][00804] Num frames 8000... -[2023-02-22 19:40:54,558][00804] Avg episode rewards: #0: 27.600, true rewards: #0: 11.457 -[2023-02-22 19:40:54,560][00804] Avg episode reward: 27.600, avg true_objective: 11.457 -[2023-02-22 19:40:54,654][00804] Num frames 8100... -[2023-02-22 19:40:54,771][00804] Num frames 8200... -[2023-02-22 19:40:54,884][00804] Num frames 8300... -[2023-02-22 19:40:55,007][00804] Num frames 8400... -[2023-02-22 19:40:55,122][00804] Num frames 8500... -[2023-02-22 19:40:55,245][00804] Num frames 8600... -[2023-02-22 19:40:55,362][00804] Num frames 8700... -[2023-02-22 19:40:55,482][00804] Num frames 8800... -[2023-02-22 19:40:55,594][00804] Num frames 8900... -[2023-02-22 19:40:55,711][00804] Num frames 9000... -[2023-02-22 19:40:55,825][00804] Num frames 9100... -[2023-02-22 19:40:55,950][00804] Num frames 9200... -[2023-02-22 19:40:56,089][00804] Avg episode rewards: #0: 27.710, true rewards: #0: 11.585 -[2023-02-22 19:40:56,090][00804] Avg episode reward: 27.710, avg true_objective: 11.585 -[2023-02-22 19:40:56,134][00804] Num frames 9300... -[2023-02-22 19:40:56,252][00804] Num frames 9400... -[2023-02-22 19:40:56,368][00804] Num frames 9500... -[2023-02-22 19:40:56,490][00804] Num frames 9600... -[2023-02-22 19:40:56,606][00804] Num frames 9700... -[2023-02-22 19:40:56,725][00804] Num frames 9800... -[2023-02-22 19:40:56,838][00804] Num frames 9900... -[2023-02-22 19:40:56,951][00804] Num frames 10000... -[2023-02-22 19:40:57,078][00804] Num frames 10100... -[2023-02-22 19:40:57,205][00804] Num frames 10200... -[2023-02-22 19:40:57,317][00804] Num frames 10300... -[2023-02-22 19:40:57,436][00804] Num frames 10400... -[2023-02-22 19:40:57,552][00804] Num frames 10500... -[2023-02-22 19:40:57,674][00804] Num frames 10600... -[2023-02-22 19:40:57,797][00804] Num frames 10700... -[2023-02-22 19:40:57,918][00804] Num frames 10800... -[2023-02-22 19:40:58,031][00804] Num frames 10900... -[2023-02-22 19:40:58,166][00804] Num frames 11000... -[2023-02-22 19:40:58,285][00804] Num frames 11100... -[2023-02-22 19:40:58,404][00804] Num frames 11200... -[2023-02-22 19:40:58,523][00804] Num frames 11300... -[2023-02-22 19:40:58,660][00804] Avg episode rewards: #0: 31.187, true rewards: #0: 12.631 -[2023-02-22 19:40:58,661][00804] Avg episode reward: 31.187, avg true_objective: 12.631 -[2023-02-22 19:40:58,701][00804] Num frames 11400... -[2023-02-22 19:40:58,822][00804] Num frames 11500... -[2023-02-22 19:40:58,933][00804] Num frames 11600... -[2023-02-22 19:40:59,049][00804] Num frames 11700... -[2023-02-22 19:40:59,178][00804] Num frames 11800... -[2023-02-22 19:40:59,271][00804] Avg episode rewards: #0: 28.931, true rewards: #0: 11.831 -[2023-02-22 19:40:59,272][00804] Avg episode reward: 28.931, avg true_objective: 11.831 -[2023-02-22 19:42:12,432][00804] Replay video saved to /content/train_dir/default_experiment/replay.mp4! -[2023-02-22 20:15:31,100][00804] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json -[2023-02-22 20:15:31,102][00804] Overriding arg 'num_workers' with value 1 passed from command line -[2023-02-22 20:15:31,105][00804] Adding new argument 'no_render'=True that is not in the saved config file! -[2023-02-22 20:15:31,106][00804] Adding new argument 'save_video'=True that is not in the saved config file! -[2023-02-22 20:15:31,108][00804] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! -[2023-02-22 20:15:31,111][00804] Adding new argument 'video_name'=None that is not in the saved config file! -[2023-02-22 20:15:31,112][00804] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! -[2023-02-22 20:15:31,113][00804] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! -[2023-02-22 20:15:31,115][00804] Adding new argument 'push_to_hub'=True that is not in the saved config file! -[2023-02-22 20:15:31,116][00804] Adding new argument 'hf_repository'='albertqueralto/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! -[2023-02-22 20:15:31,122][00804] Adding new argument 'policy_index'=0 that is not in the saved config file! -[2023-02-22 20:15:31,124][00804] Adding new argument 'eval_deterministic'=False that is not in the saved config file! -[2023-02-22 20:15:31,125][00804] Adding new argument 'train_script'=None that is not in the saved config file! -[2023-02-22 20:15:31,126][00804] Adding new argument 'enjoy_script'=None that is not in the saved config file! -[2023-02-22 20:15:31,128][00804] Using frameskip 1 and render_action_repeat=4 for evaluation -[2023-02-22 20:15:31,154][00804] RunningMeanStd input shape: (3, 72, 128) -[2023-02-22 20:15:31,159][00804] RunningMeanStd input shape: (1,) -[2023-02-22 20:15:31,173][00804] ConvEncoder: input_channels=3 -[2023-02-22 20:15:31,211][00804] Conv encoder output size: 512 -[2023-02-22 20:15:31,213][00804] Policy head output size: 512 -[2023-02-22 20:15:31,233][00804] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... -[2023-02-22 20:15:31,678][00804] Num frames 100... -[2023-02-22 20:15:31,789][00804] Num frames 200... -[2023-02-22 20:15:31,916][00804] Num frames 300... -[2023-02-22 20:15:32,027][00804] Num frames 400... -[2023-02-22 20:15:32,163][00804] Num frames 500... -[2023-02-22 20:15:32,279][00804] Num frames 600... -[2023-02-22 20:15:32,401][00804] Num frames 700... -[2023-02-22 20:15:32,524][00804] Num frames 800... -[2023-02-22 20:15:32,612][00804] Avg episode rewards: #0: 16.280, true rewards: #0: 8.280 -[2023-02-22 20:15:32,617][00804] Avg episode reward: 16.280, avg true_objective: 8.280 -[2023-02-22 20:15:32,701][00804] Num frames 900... -[2023-02-22 20:15:32,813][00804] Num frames 1000... -[2023-02-22 20:15:32,934][00804] Num frames 1100... -[2023-02-22 20:15:33,050][00804] Num frames 1200... -[2023-02-22 20:15:33,183][00804] Num frames 1300... -[2023-02-22 20:15:33,296][00804] Num frames 1400... -[2023-02-22 20:15:33,411][00804] Num frames 1500... -[2023-02-22 20:15:33,530][00804] Num frames 1600... -[2023-02-22 20:15:33,644][00804] Num frames 1700... -[2023-02-22 20:15:33,759][00804] Num frames 1800... -[2023-02-22 20:15:33,877][00804] Num frames 1900... -[2023-02-22 20:15:33,987][00804] Num frames 2000... -[2023-02-22 20:15:34,101][00804] Num frames 2100... -[2023-02-22 20:15:34,217][00804] Num frames 2200... -[2023-02-22 20:15:34,310][00804] Avg episode rewards: #0: 26.160, true rewards: #0: 11.160 -[2023-02-22 20:15:34,312][00804] Avg episode reward: 26.160, avg true_objective: 11.160 -[2023-02-22 20:15:34,397][00804] Num frames 2300... -[2023-02-22 20:15:34,508][00804] Num frames 2400... -[2023-02-22 20:15:34,643][00804] Num frames 2500... -[2023-02-22 20:15:34,756][00804] Num frames 2600... -[2023-02-22 20:15:34,874][00804] Num frames 2700... -[2023-02-22 20:15:34,992][00804] Num frames 2800... -[2023-02-22 20:15:35,107][00804] Num frames 2900... -[2023-02-22 20:15:35,226][00804] Num frames 3000... -[2023-02-22 20:15:35,290][00804] Avg episode rewards: #0: 23.010, true rewards: #0: 10.010 -[2023-02-22 20:15:35,292][00804] Avg episode reward: 23.010, avg true_objective: 10.010 -[2023-02-22 20:15:35,452][00804] Num frames 3100... -[2023-02-22 20:15:35,646][00804] Num frames 3200... -[2023-02-22 20:15:35,797][00804] Num frames 3300... -[2023-02-22 20:15:35,952][00804] Num frames 3400... -[2023-02-22 20:15:36,105][00804] Num frames 3500... -[2023-02-22 20:15:36,266][00804] Num frames 3600... -[2023-02-22 20:15:36,430][00804] Num frames 3700... -[2023-02-22 20:15:36,587][00804] Num frames 3800... -[2023-02-22 20:15:36,763][00804] Num frames 3900... -[2023-02-22 20:15:36,927][00804] Num frames 4000... -[2023-02-22 20:15:37,084][00804] Num frames 4100... -[2023-02-22 20:15:37,254][00804] Num frames 4200... -[2023-02-22 20:15:37,418][00804] Num frames 4300... -[2023-02-22 20:15:37,586][00804] Num frames 4400... -[2023-02-22 20:15:37,754][00804] Num frames 4500... -[2023-02-22 20:15:37,919][00804] Num frames 4600... -[2023-02-22 20:15:38,090][00804] Num frames 4700... -[2023-02-22 20:15:38,256][00804] Num frames 4800... -[2023-02-22 20:15:38,423][00804] Num frames 4900... -[2023-02-22 20:15:38,594][00804] Num frames 5000... -[2023-02-22 20:15:38,762][00804] Num frames 5100... -[2023-02-22 20:15:38,829][00804] Avg episode rewards: #0: 31.257, true rewards: #0: 12.757 -[2023-02-22 20:15:38,831][00804] Avg episode reward: 31.257, avg true_objective: 12.757 -[2023-02-22 20:15:38,969][00804] Num frames 5200... -[2023-02-22 20:15:39,090][00804] Num frames 5300... -[2023-02-22 20:15:39,208][00804] Num frames 5400... -[2023-02-22 20:15:39,327][00804] Num frames 5500... -[2023-02-22 20:15:39,451][00804] Num frames 5600... -[2023-02-22 20:15:39,573][00804] Num frames 5700... -[2023-02-22 20:15:39,684][00804] Num frames 5800... -[2023-02-22 20:15:39,813][00804] Num frames 5900... -[2023-02-22 20:15:39,874][00804] Avg episode rewards: #0: 28.006, true rewards: #0: 11.806 -[2023-02-22 20:15:39,876][00804] Avg episode reward: 28.006, avg true_objective: 11.806 -[2023-02-22 20:15:39,989][00804] Num frames 6000... -[2023-02-22 20:15:40,112][00804] Num frames 6100... -[2023-02-22 20:15:40,228][00804] Num frames 6200... -[2023-02-22 20:15:40,339][00804] Num frames 6300... -[2023-02-22 20:15:40,462][00804] Num frames 6400... -[2023-02-22 20:15:40,583][00804] Num frames 6500... -[2023-02-22 20:15:40,729][00804] Avg episode rewards: #0: 25.800, true rewards: #0: 10.967 -[2023-02-22 20:15:40,730][00804] Avg episode reward: 25.800, avg true_objective: 10.967 -[2023-02-22 20:15:40,760][00804] Num frames 6600... -[2023-02-22 20:15:40,881][00804] Num frames 6700... -[2023-02-22 20:15:40,992][00804] Num frames 6800... -[2023-02-22 20:15:41,109][00804] Num frames 6900... -[2023-02-22 20:15:41,219][00804] Num frames 7000... -[2023-02-22 20:15:41,343][00804] Num frames 7100... -[2023-02-22 20:15:41,467][00804] Num frames 7200... -[2023-02-22 20:15:41,581][00804] Num frames 7300... -[2023-02-22 20:15:41,700][00804] Num frames 7400... -[2023-02-22 20:15:41,822][00804] Num frames 7500... -[2023-02-22 20:15:41,937][00804] Num frames 7600... -[2023-02-22 20:15:42,063][00804] Num frames 7700... -[2023-02-22 20:15:42,185][00804] Num frames 7800... -[2023-02-22 20:15:42,300][00804] Num frames 7900... -[2023-02-22 20:15:42,417][00804] Num frames 8000... -[2023-02-22 20:15:42,538][00804] Num frames 8100... -[2023-02-22 20:15:42,668][00804] Num frames 8200... -[2023-02-22 20:15:42,740][00804] Avg episode rewards: #0: 28.017, true rewards: #0: 11.731 -[2023-02-22 20:15:42,743][00804] Avg episode reward: 28.017, avg true_objective: 11.731 -[2023-02-22 20:15:42,849][00804] Num frames 8300... -[2023-02-22 20:15:42,971][00804] Num frames 8400... -[2023-02-22 20:15:43,089][00804] Num frames 8500... -[2023-02-22 20:15:43,211][00804] Num frames 8600... -[2023-02-22 20:15:43,325][00804] Num frames 8700... -[2023-02-22 20:15:43,442][00804] Num frames 8800... -[2023-02-22 20:15:43,554][00804] Num frames 8900... -[2023-02-22 20:15:43,676][00804] Num frames 9000... -[2023-02-22 20:15:43,747][00804] Avg episode rewards: #0: 26.390, true rewards: #0: 11.265 -[2023-02-22 20:15:43,752][00804] Avg episode reward: 26.390, avg true_objective: 11.265 -[2023-02-22 20:15:43,863][00804] Num frames 9100... -[2023-02-22 20:15:43,982][00804] Num frames 9200... -[2023-02-22 20:15:44,104][00804] Num frames 9300... -[2023-02-22 20:15:44,216][00804] Num frames 9400... -[2023-02-22 20:15:44,332][00804] Num frames 9500... -[2023-02-22 20:15:44,460][00804] Num frames 9600... -[2023-02-22 20:15:44,572][00804] Num frames 9700... -[2023-02-22 20:15:44,687][00804] Num frames 9800... -[2023-02-22 20:15:44,796][00804] Num frames 9900... -[2023-02-22 20:15:44,918][00804] Num frames 10000... -[2023-02-22 20:15:45,028][00804] Num frames 10100... -[2023-02-22 20:15:45,145][00804] Num frames 10200... -[2023-02-22 20:15:45,266][00804] Num frames 10300... -[2023-02-22 20:15:45,382][00804] Num frames 10400... -[2023-02-22 20:15:45,501][00804] Num frames 10500... -[2023-02-22 20:15:45,624][00804] Num frames 10600... -[2023-02-22 20:15:45,738][00804] Num frames 10700... -[2023-02-22 20:15:45,857][00804] Num frames 10800... -[2023-02-22 20:15:45,976][00804] Num frames 10900... -[2023-02-22 20:15:46,094][00804] Num frames 11000... -[2023-02-22 20:15:46,211][00804] Num frames 11100... -[2023-02-22 20:15:46,281][00804] Avg episode rewards: #0: 30.013, true rewards: #0: 12.347 -[2023-02-22 20:15:46,282][00804] Avg episode reward: 30.013, avg true_objective: 12.347 -[2023-02-22 20:15:46,401][00804] Num frames 11200... -[2023-02-22 20:15:46,521][00804] Num frames 11300... -[2023-02-22 20:15:46,641][00804] Num frames 11400... -[2023-02-22 20:15:46,757][00804] Num frames 11500... -[2023-02-22 20:15:46,881][00804] Num frames 11600... -[2023-02-22 20:15:47,001][00804] Num frames 11700... -[2023-02-22 20:15:47,130][00804] Num frames 11800... -[2023-02-22 20:15:47,244][00804] Num frames 11900... -[2023-02-22 20:15:47,358][00804] Num frames 12000... -[2023-02-22 20:15:47,483][00804] Num frames 12100... -[2023-02-22 20:15:47,621][00804] Avg episode rewards: #0: 29.268, true rewards: #0: 12.168 -[2023-02-22 20:15:47,623][00804] Avg episode reward: 29.268, avg true_objective: 12.168 -[2023-02-22 20:17:03,872][00804] Replay video saved to /content/train_dir/default_experiment/replay.mp4! +[2023-02-23 20:07:30,100][00631] Heartbeat connected on Batcher_0 +[2023-02-23 20:07:30,108][00631] Heartbeat connected on InferenceWorker_p0-w0 +[2023-02-23 20:07:30,117][00631] Heartbeat connected on RolloutWorker_w0 +[2023-02-23 20:07:30,122][00631] Heartbeat connected on RolloutWorker_w1 +[2023-02-23 20:07:30,125][00631] Heartbeat connected on RolloutWorker_w2 +[2023-02-23 20:07:30,129][00631] Heartbeat connected on RolloutWorker_w3 +[2023-02-23 20:07:30,133][00631] Heartbeat connected on RolloutWorker_w4 +[2023-02-23 20:07:30,134][00631] Heartbeat connected on RolloutWorker_w5 +[2023-02-23 20:07:30,137][00631] Heartbeat connected on RolloutWorker_w6 +[2023-02-23 20:07:30,141][00631] Heartbeat connected on RolloutWorker_w7 +[2023-02-23 20:07:32,608][10884] Using optimizer +[2023-02-23 20:07:32,609][10884] No checkpoints found +[2023-02-23 20:07:32,609][10884] Did not load from checkpoint, starting from scratch! +[2023-02-23 20:07:32,610][10884] Initialized policy 0 weights for model version 0 +[2023-02-23 20:07:32,614][10884] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-02-23 20:07:32,622][10884] LearnerWorker_p0 finished initialization! +[2023-02-23 20:07:32,623][00631] Heartbeat connected on LearnerWorker_p0 +[2023-02-23 20:07:32,731][10898] RunningMeanStd input shape: (3, 72, 128) +[2023-02-23 20:07:32,734][10898] RunningMeanStd input shape: (1,) +[2023-02-23 20:07:32,747][10898] ConvEncoder: input_channels=3 +[2023-02-23 20:07:32,855][10898] Conv encoder output size: 512 +[2023-02-23 20:07:32,856][10898] Policy head output size: 512 +[2023-02-23 20:07:35,197][00631] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-02-23 20:07:35,879][00631] Inference worker 0-0 is ready! +[2023-02-23 20:07:35,883][00631] All inference workers are ready! Signal rollout workers to start! +[2023-02-23 20:07:36,064][10905] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:07:36,079][10902] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:07:36,102][10906] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:07:36,108][10900] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:07:36,111][10904] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:07:36,108][10899] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:07:36,153][10903] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:07:36,173][10901] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:07:37,213][10902] Decorrelating experience for 0 frames... +[2023-02-23 20:07:37,212][10906] Decorrelating experience for 0 frames... +[2023-02-23 20:07:38,051][10905] Decorrelating experience for 0 frames... +[2023-02-23 20:07:38,063][10899] Decorrelating experience for 0 frames... +[2023-02-23 20:07:38,065][10903] Decorrelating experience for 0 frames... +[2023-02-23 20:07:39,245][10900] Decorrelating experience for 0 frames... +[2023-02-23 20:07:39,270][10906] Decorrelating experience for 32 frames... +[2023-02-23 20:07:39,275][10902] Decorrelating experience for 32 frames... +[2023-02-23 20:07:39,733][10901] Decorrelating experience for 0 frames... +[2023-02-23 20:07:40,197][00631] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-02-23 20:07:40,350][10903] Decorrelating experience for 32 frames... +[2023-02-23 20:07:40,361][10905] Decorrelating experience for 32 frames... +[2023-02-23 20:07:40,477][10900] Decorrelating experience for 32 frames... +[2023-02-23 20:07:40,746][10902] Decorrelating experience for 64 frames... +[2023-02-23 20:07:40,770][10906] Decorrelating experience for 64 frames... +[2023-02-23 20:07:41,084][10901] Decorrelating experience for 32 frames... +[2023-02-23 20:07:41,103][10899] Decorrelating experience for 32 frames... +[2023-02-23 20:07:41,623][10904] Decorrelating experience for 0 frames... +[2023-02-23 20:07:41,643][10903] Decorrelating experience for 64 frames... +[2023-02-23 20:07:41,893][10901] Decorrelating experience for 64 frames... +[2023-02-23 20:07:41,937][10902] Decorrelating experience for 96 frames... +[2023-02-23 20:07:42,383][10899] Decorrelating experience for 64 frames... +[2023-02-23 20:07:42,409][10906] Decorrelating experience for 96 frames... +[2023-02-23 20:07:43,085][10904] Decorrelating experience for 32 frames... +[2023-02-23 20:07:43,165][10900] Decorrelating experience for 64 frames... +[2023-02-23 20:07:43,223][10905] Decorrelating experience for 64 frames... +[2023-02-23 20:07:43,287][10899] Decorrelating experience for 96 frames... +[2023-02-23 20:07:44,376][10901] Decorrelating experience for 96 frames... +[2023-02-23 20:07:44,539][10903] Decorrelating experience for 96 frames... +[2023-02-23 20:07:44,593][10900] Decorrelating experience for 96 frames... +[2023-02-23 20:07:44,638][10905] Decorrelating experience for 96 frames... +[2023-02-23 20:07:44,702][10904] Decorrelating experience for 64 frames... +[2023-02-23 20:07:45,140][10904] Decorrelating experience for 96 frames... +[2023-02-23 20:07:45,197][00631] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-02-23 20:07:48,472][10884] Signal inference workers to stop experience collection... +[2023-02-23 20:07:48,496][10898] InferenceWorker_p0-w0: stopping experience collection +[2023-02-23 20:07:50,197][00631] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 121.1. Samples: 1816. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-02-23 20:07:50,199][00631] Avg episode reward: [(0, '1.838')] +[2023-02-23 20:07:51,154][10884] Signal inference workers to resume experience collection... +[2023-02-23 20:07:51,155][10898] InferenceWorker_p0-w0: resuming experience collection +[2023-02-23 20:07:55,197][00631] Fps is (10 sec: 1228.8, 60 sec: 614.4, 300 sec: 614.4). Total num frames: 12288. Throughput: 0: 157.2. Samples: 3144. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) +[2023-02-23 20:07:55,207][00631] Avg episode reward: [(0, '2.943')] +[2023-02-23 20:08:00,197][00631] Fps is (10 sec: 2867.1, 60 sec: 1146.9, 300 sec: 1146.9). Total num frames: 28672. Throughput: 0: 298.8. Samples: 7470. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) +[2023-02-23 20:08:00,199][00631] Avg episode reward: [(0, '3.638')] +[2023-02-23 20:08:03,235][10898] Updated weights for policy 0, policy_version 10 (0.0013) +[2023-02-23 20:08:05,197][00631] Fps is (10 sec: 3686.4, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 49152. Throughput: 0: 351.5. Samples: 10546. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:08:05,203][00631] Avg episode reward: [(0, '4.272')] +[2023-02-23 20:08:10,197][00631] Fps is (10 sec: 3686.4, 60 sec: 1872.4, 300 sec: 1872.4). Total num frames: 65536. Throughput: 0: 475.9. Samples: 16658. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:08:10,204][00631] Avg episode reward: [(0, '4.529')] +[2023-02-23 20:08:14,880][10898] Updated weights for policy 0, policy_version 20 (0.0015) +[2023-02-23 20:08:15,200][00631] Fps is (10 sec: 3275.7, 60 sec: 2047.8, 300 sec: 2047.8). Total num frames: 81920. Throughput: 0: 516.6. Samples: 20664. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) +[2023-02-23 20:08:15,205][00631] Avg episode reward: [(0, '4.487')] +[2023-02-23 20:08:20,197][00631] Fps is (10 sec: 2867.3, 60 sec: 2093.5, 300 sec: 2093.5). Total num frames: 94208. Throughput: 0: 496.9. Samples: 22360. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) +[2023-02-23 20:08:20,200][00631] Avg episode reward: [(0, '4.356')] +[2023-02-23 20:08:25,197][00631] Fps is (10 sec: 3277.9, 60 sec: 2293.8, 300 sec: 2293.8). Total num frames: 114688. Throughput: 0: 621.7. Samples: 27978. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) +[2023-02-23 20:08:25,200][00631] Avg episode reward: [(0, '4.441')] +[2023-02-23 20:08:25,214][10884] Saving new best policy, reward=4.441! +[2023-02-23 20:08:26,759][10898] Updated weights for policy 0, policy_version 30 (0.0048) +[2023-02-23 20:08:30,197][00631] Fps is (10 sec: 4095.9, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 135168. Throughput: 0: 762.4. Samples: 34310. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-23 20:08:30,208][00631] Avg episode reward: [(0, '4.576')] +[2023-02-23 20:08:30,211][10884] Saving new best policy, reward=4.576! +[2023-02-23 20:08:35,199][00631] Fps is (10 sec: 3276.1, 60 sec: 2457.5, 300 sec: 2457.5). Total num frames: 147456. Throughput: 0: 765.1. Samples: 36248. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:08:35,206][00631] Avg episode reward: [(0, '4.420')] +[2023-02-23 20:08:40,169][10898] Updated weights for policy 0, policy_version 40 (0.0032) +[2023-02-23 20:08:40,198][00631] Fps is (10 sec: 2867.2, 60 sec: 2730.6, 300 sec: 2520.6). Total num frames: 163840. Throughput: 0: 825.3. Samples: 40284. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-23 20:08:40,201][00631] Avg episode reward: [(0, '4.371')] +[2023-02-23 20:08:45,197][00631] Fps is (10 sec: 3277.4, 60 sec: 3003.7, 300 sec: 2574.6). Total num frames: 180224. Throughput: 0: 854.0. Samples: 45900. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:08:45,203][00631] Avg episode reward: [(0, '4.508')] +[2023-02-23 20:08:50,197][00631] Fps is (10 sec: 3686.6, 60 sec: 3345.1, 300 sec: 2676.1). Total num frames: 200704. Throughput: 0: 856.6. Samples: 49092. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-23 20:08:50,200][00631] Avg episode reward: [(0, '4.616')] +[2023-02-23 20:08:50,202][10884] Saving new best policy, reward=4.616! +[2023-02-23 20:08:50,460][10898] Updated weights for policy 0, policy_version 50 (0.0019) +[2023-02-23 20:08:55,199][00631] Fps is (10 sec: 3685.6, 60 sec: 3413.2, 300 sec: 2713.5). Total num frames: 217088. Throughput: 0: 836.6. Samples: 54308. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:08:55,203][00631] Avg episode reward: [(0, '4.502')] +[2023-02-23 20:09:00,198][00631] Fps is (10 sec: 2867.0, 60 sec: 3345.1, 300 sec: 2698.5). Total num frames: 229376. Throughput: 0: 839.4. Samples: 58436. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:09:00,207][00631] Avg episode reward: [(0, '4.571')] +[2023-02-23 20:09:04,138][10898] Updated weights for policy 0, policy_version 60 (0.0016) +[2023-02-23 20:09:05,197][00631] Fps is (10 sec: 3277.5, 60 sec: 3345.1, 300 sec: 2776.2). Total num frames: 249856. Throughput: 0: 853.1. Samples: 60748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:09:05,203][00631] Avg episode reward: [(0, '4.345')] +[2023-02-23 20:09:05,216][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000061_249856.pth... +[2023-02-23 20:09:10,197][00631] Fps is (10 sec: 4096.2, 60 sec: 3413.3, 300 sec: 2845.6). Total num frames: 270336. Throughput: 0: 871.8. Samples: 67208. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:09:10,203][00631] Avg episode reward: [(0, '4.530')] +[2023-02-23 20:09:14,370][10898] Updated weights for policy 0, policy_version 70 (0.0022) +[2023-02-23 20:09:15,200][00631] Fps is (10 sec: 3685.2, 60 sec: 3413.3, 300 sec: 2867.1). Total num frames: 286720. Throughput: 0: 850.1. Samples: 72568. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:09:15,206][00631] Avg episode reward: [(0, '4.589')] +[2023-02-23 20:09:20,202][00631] Fps is (10 sec: 3275.1, 60 sec: 3481.3, 300 sec: 2886.6). Total num frames: 303104. Throughput: 0: 852.0. Samples: 74592. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:09:20,211][00631] Avg episode reward: [(0, '4.404')] +[2023-02-23 20:09:25,197][00631] Fps is (10 sec: 3277.9, 60 sec: 3413.3, 300 sec: 2904.4). Total num frames: 319488. Throughput: 0: 860.7. Samples: 79016. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:09:25,200][00631] Avg episode reward: [(0, '4.316')] +[2023-02-23 20:09:27,001][10898] Updated weights for policy 0, policy_version 80 (0.0036) +[2023-02-23 20:09:30,197][00631] Fps is (10 sec: 3688.3, 60 sec: 3413.3, 300 sec: 2956.2). Total num frames: 339968. Throughput: 0: 876.9. Samples: 85360. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:09:30,205][00631] Avg episode reward: [(0, '4.337')] +[2023-02-23 20:09:35,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.7, 300 sec: 2969.6). Total num frames: 356352. Throughput: 0: 874.4. Samples: 88440. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:09:35,207][00631] Avg episode reward: [(0, '4.448')] +[2023-02-23 20:09:38,850][10898] Updated weights for policy 0, policy_version 90 (0.0017) +[2023-02-23 20:09:40,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.4, 300 sec: 2949.1). Total num frames: 368640. Throughput: 0: 852.4. Samples: 92662. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:09:40,203][00631] Avg episode reward: [(0, '4.534')] +[2023-02-23 20:09:45,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 2961.7). Total num frames: 385024. Throughput: 0: 855.7. Samples: 96944. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:09:45,205][00631] Avg episode reward: [(0, '4.616')] +[2023-02-23 20:09:50,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3003.7). Total num frames: 405504. Throughput: 0: 873.1. Samples: 100038. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:09:50,200][00631] Avg episode reward: [(0, '4.444')] +[2023-02-23 20:09:50,756][10898] Updated weights for policy 0, policy_version 100 (0.0015) +[2023-02-23 20:09:55,197][00631] Fps is (10 sec: 4096.1, 60 sec: 3481.7, 300 sec: 3042.7). Total num frames: 425984. Throughput: 0: 872.4. Samples: 106464. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:09:55,199][00631] Avg episode reward: [(0, '4.345')] +[2023-02-23 20:10:00,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3050.8). Total num frames: 442368. Throughput: 0: 849.7. Samples: 110802. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:10:00,204][00631] Avg episode reward: [(0, '4.354')] +[2023-02-23 20:10:03,245][10898] Updated weights for policy 0, policy_version 110 (0.0027) +[2023-02-23 20:10:05,198][00631] Fps is (10 sec: 2866.9, 60 sec: 3413.3, 300 sec: 3031.0). Total num frames: 454656. Throughput: 0: 850.3. Samples: 112852. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:10:05,208][00631] Avg episode reward: [(0, '4.361')] +[2023-02-23 20:10:10,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3065.4). Total num frames: 475136. Throughput: 0: 875.2. Samples: 118398. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:10:10,200][00631] Avg episode reward: [(0, '4.567')] +[2023-02-23 20:10:13,829][10898] Updated weights for policy 0, policy_version 120 (0.0013) +[2023-02-23 20:10:15,197][00631] Fps is (10 sec: 4096.4, 60 sec: 3481.8, 300 sec: 3097.6). Total num frames: 495616. Throughput: 0: 872.6. Samples: 124628. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:10:15,200][00631] Avg episode reward: [(0, '4.658')] +[2023-02-23 20:10:15,212][10884] Saving new best policy, reward=4.658! +[2023-02-23 20:10:20,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.6, 300 sec: 3078.2). Total num frames: 507904. Throughput: 0: 850.3. Samples: 126704. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:10:20,205][00631] Avg episode reward: [(0, '4.765')] +[2023-02-23 20:10:20,214][10884] Saving new best policy, reward=4.765! +[2023-02-23 20:10:25,197][00631] Fps is (10 sec: 2457.5, 60 sec: 3345.1, 300 sec: 3059.9). Total num frames: 520192. Throughput: 0: 844.5. Samples: 130664. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:10:25,201][00631] Avg episode reward: [(0, '4.671')] +[2023-02-23 20:10:27,821][10898] Updated weights for policy 0, policy_version 130 (0.0022) +[2023-02-23 20:10:30,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3089.6). Total num frames: 540672. Throughput: 0: 868.4. Samples: 136020. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:10:30,201][00631] Avg episode reward: [(0, '4.472')] +[2023-02-23 20:10:35,201][00631] Fps is (10 sec: 3275.6, 60 sec: 3276.6, 300 sec: 3071.9). Total num frames: 552960. Throughput: 0: 845.4. Samples: 138084. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:10:35,215][00631] Avg episode reward: [(0, '4.431')] +[2023-02-23 20:10:40,198][00631] Fps is (10 sec: 2457.5, 60 sec: 3276.8, 300 sec: 3055.4). Total num frames: 565248. Throughput: 0: 784.3. Samples: 141758. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:10:40,201][00631] Avg episode reward: [(0, '4.522')] +[2023-02-23 20:10:42,980][10898] Updated weights for policy 0, policy_version 140 (0.0016) +[2023-02-23 20:10:45,197][00631] Fps is (10 sec: 2458.5, 60 sec: 3208.5, 300 sec: 3039.7). Total num frames: 577536. Throughput: 0: 763.6. Samples: 145162. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:10:45,199][00631] Avg episode reward: [(0, '4.514')] +[2023-02-23 20:10:50,203][00631] Fps is (10 sec: 2456.2, 60 sec: 3071.7, 300 sec: 3024.6). Total num frames: 589824. Throughput: 0: 763.0. Samples: 147192. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:10:50,209][00631] Avg episode reward: [(0, '4.530')] +[2023-02-23 20:10:55,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3072.0, 300 sec: 3051.5). Total num frames: 610304. Throughput: 0: 768.0. Samples: 152958. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:10:55,203][00631] Avg episode reward: [(0, '4.299')] +[2023-02-23 20:10:55,221][10898] Updated weights for policy 0, policy_version 150 (0.0014) +[2023-02-23 20:11:00,197][00631] Fps is (10 sec: 4508.5, 60 sec: 3208.5, 300 sec: 3097.0). Total num frames: 634880. Throughput: 0: 770.0. Samples: 159280. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:11:00,202][00631] Avg episode reward: [(0, '4.356')] +[2023-02-23 20:11:05,202][00631] Fps is (10 sec: 3684.6, 60 sec: 3208.3, 300 sec: 3081.7). Total num frames: 647168. Throughput: 0: 768.7. Samples: 161298. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:11:05,207][00631] Avg episode reward: [(0, '4.429')] +[2023-02-23 20:11:05,225][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000158_647168.pth... +[2023-02-23 20:11:07,728][10898] Updated weights for policy 0, policy_version 160 (0.0017) +[2023-02-23 20:11:10,197][00631] Fps is (10 sec: 2457.5, 60 sec: 3072.0, 300 sec: 3067.2). Total num frames: 659456. Throughput: 0: 768.6. Samples: 165252. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:11:10,203][00631] Avg episode reward: [(0, '4.523')] +[2023-02-23 20:11:15,197][00631] Fps is (10 sec: 3278.4, 60 sec: 3072.0, 300 sec: 3090.6). Total num frames: 679936. Throughput: 0: 776.7. Samples: 170970. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:11:15,202][00631] Avg episode reward: [(0, '4.324')] +[2023-02-23 20:11:18,130][10898] Updated weights for policy 0, policy_version 170 (0.0018) +[2023-02-23 20:11:20,197][00631] Fps is (10 sec: 4505.7, 60 sec: 3276.8, 300 sec: 3131.2). Total num frames: 704512. Throughput: 0: 804.4. Samples: 174278. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:11:20,199][00631] Avg episode reward: [(0, '4.448')] +[2023-02-23 20:11:25,197][00631] Fps is (10 sec: 3686.3, 60 sec: 3276.8, 300 sec: 3116.5). Total num frames: 716800. Throughput: 0: 840.7. Samples: 179588. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:11:25,201][00631] Avg episode reward: [(0, '4.455')] +[2023-02-23 20:11:30,200][00631] Fps is (10 sec: 2456.9, 60 sec: 3140.1, 300 sec: 3102.5). Total num frames: 729088. Throughput: 0: 856.6. Samples: 183712. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:11:30,203][00631] Avg episode reward: [(0, '4.417')] +[2023-02-23 20:11:31,780][10898] Updated weights for policy 0, policy_version 180 (0.0019) +[2023-02-23 20:11:35,197][00631] Fps is (10 sec: 3276.9, 60 sec: 3277.0, 300 sec: 3123.2). Total num frames: 749568. Throughput: 0: 866.4. Samples: 186174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:11:35,205][00631] Avg episode reward: [(0, '4.640')] +[2023-02-23 20:11:40,197][00631] Fps is (10 sec: 4097.1, 60 sec: 3413.4, 300 sec: 3143.1). Total num frames: 770048. Throughput: 0: 879.9. Samples: 192552. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:11:40,203][00631] Avg episode reward: [(0, '4.894')] +[2023-02-23 20:11:40,277][10884] Saving new best policy, reward=4.894! +[2023-02-23 20:11:41,344][10898] Updated weights for policy 0, policy_version 190 (0.0023) +[2023-02-23 20:11:45,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3145.7). Total num frames: 786432. Throughput: 0: 853.8. Samples: 197702. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:11:45,204][00631] Avg episode reward: [(0, '4.931')] +[2023-02-23 20:11:45,213][10884] Saving new best policy, reward=4.931! +[2023-02-23 20:11:50,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3550.2, 300 sec: 3148.3). Total num frames: 802816. Throughput: 0: 851.9. Samples: 199628. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:11:50,204][00631] Avg episode reward: [(0, '4.898')] +[2023-02-23 20:11:54,983][10898] Updated weights for policy 0, policy_version 200 (0.0012) +[2023-02-23 20:11:55,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3150.8). Total num frames: 819200. Throughput: 0: 865.0. Samples: 204176. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:11:55,205][00631] Avg episode reward: [(0, '4.655')] +[2023-02-23 20:12:00,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3168.6). Total num frames: 839680. Throughput: 0: 881.2. Samples: 210626. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:12:00,199][00631] Avg episode reward: [(0, '4.388')] +[2023-02-23 20:12:05,200][00631] Fps is (10 sec: 3685.2, 60 sec: 3481.7, 300 sec: 3170.6). Total num frames: 856064. Throughput: 0: 878.8. Samples: 213826. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:12:05,203][00631] Avg episode reward: [(0, '4.440')] +[2023-02-23 20:12:05,646][10898] Updated weights for policy 0, policy_version 210 (0.0017) +[2023-02-23 20:12:10,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3172.5). Total num frames: 872448. Throughput: 0: 852.6. Samples: 217954. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:12:10,207][00631] Avg episode reward: [(0, '4.498')] +[2023-02-23 20:12:15,197][00631] Fps is (10 sec: 2868.1, 60 sec: 3413.3, 300 sec: 3159.8). Total num frames: 884736. Throughput: 0: 858.1. Samples: 222326. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:12:15,205][00631] Avg episode reward: [(0, '4.423')] +[2023-02-23 20:12:18,321][10898] Updated weights for policy 0, policy_version 220 (0.0021) +[2023-02-23 20:12:20,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3190.6). Total num frames: 909312. Throughput: 0: 875.2. Samples: 225558. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:12:20,200][00631] Avg episode reward: [(0, '4.270')] +[2023-02-23 20:12:25,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3192.1). Total num frames: 925696. Throughput: 0: 876.8. Samples: 232008. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:12:25,208][00631] Avg episode reward: [(0, '4.430')] +[2023-02-23 20:12:29,701][10898] Updated weights for policy 0, policy_version 230 (0.0018) +[2023-02-23 20:12:30,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3550.0, 300 sec: 3193.5). Total num frames: 942080. Throughput: 0: 853.4. Samples: 236106. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:12:30,200][00631] Avg episode reward: [(0, '4.580')] +[2023-02-23 20:12:35,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3235.1). Total num frames: 954368. Throughput: 0: 854.5. Samples: 238080. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:12:35,200][00631] Avg episode reward: [(0, '4.619')] +[2023-02-23 20:12:40,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3304.6). Total num frames: 974848. Throughput: 0: 878.0. Samples: 243684. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:12:40,199][00631] Avg episode reward: [(0, '4.610')] +[2023-02-23 20:12:41,351][10898] Updated weights for policy 0, policy_version 240 (0.0015) +[2023-02-23 20:12:45,202][00631] Fps is (10 sec: 4093.8, 60 sec: 3481.3, 300 sec: 3373.9). Total num frames: 995328. Throughput: 0: 875.1. Samples: 250010. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:12:45,208][00631] Avg episode reward: [(0, '4.291')] +[2023-02-23 20:12:50,199][00631] Fps is (10 sec: 3685.8, 60 sec: 3481.5, 300 sec: 3387.9). Total num frames: 1011712. Throughput: 0: 852.0. Samples: 252164. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:12:50,201][00631] Avg episode reward: [(0, '4.396')] +[2023-02-23 20:12:54,572][10898] Updated weights for policy 0, policy_version 250 (0.0036) +[2023-02-23 20:12:55,197][00631] Fps is (10 sec: 2868.7, 60 sec: 3413.3, 300 sec: 3374.0). Total num frames: 1024000. Throughput: 0: 850.5. Samples: 256226. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:12:55,200][00631] Avg episode reward: [(0, '4.492')] +[2023-02-23 20:13:00,197][00631] Fps is (10 sec: 3277.3, 60 sec: 3413.3, 300 sec: 3374.0). Total num frames: 1044480. Throughput: 0: 874.1. Samples: 261662. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:13:00,200][00631] Avg episode reward: [(0, '4.709')] +[2023-02-23 20:13:04,689][10898] Updated weights for policy 0, policy_version 260 (0.0019) +[2023-02-23 20:13:05,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.8, 300 sec: 3387.9). Total num frames: 1064960. Throughput: 0: 871.6. Samples: 264778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:13:05,202][00631] Avg episode reward: [(0, '4.515')] +[2023-02-23 20:13:05,225][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000260_1064960.pth... +[2023-02-23 20:13:05,375][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000061_249856.pth +[2023-02-23 20:13:10,199][00631] Fps is (10 sec: 3685.7, 60 sec: 3481.5, 300 sec: 3387.9). Total num frames: 1081344. Throughput: 0: 845.4. Samples: 270052. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:13:10,211][00631] Avg episode reward: [(0, '4.467')] +[2023-02-23 20:13:15,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3387.9). Total num frames: 1093632. Throughput: 0: 845.5. Samples: 274154. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:13:15,204][00631] Avg episode reward: [(0, '4.494')] +[2023-02-23 20:13:18,419][10898] Updated weights for policy 0, policy_version 270 (0.0014) +[2023-02-23 20:13:20,197][00631] Fps is (10 sec: 3277.5, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1114112. Throughput: 0: 856.3. Samples: 276614. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:13:20,199][00631] Avg episode reward: [(0, '4.476')] +[2023-02-23 20:13:25,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3387.9). Total num frames: 1134592. Throughput: 0: 877.5. Samples: 283172. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:13:25,200][00631] Avg episode reward: [(0, '4.752')] +[2023-02-23 20:13:27,924][10898] Updated weights for policy 0, policy_version 280 (0.0014) +[2023-02-23 20:13:30,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1150976. Throughput: 0: 857.2. Samples: 288578. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:13:30,202][00631] Avg episode reward: [(0, '4.749')] +[2023-02-23 20:13:35,197][00631] Fps is (10 sec: 2867.1, 60 sec: 3481.6, 300 sec: 3387.9). Total num frames: 1163264. Throughput: 0: 855.2. Samples: 290648. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:13:35,200][00631] Avg episode reward: [(0, '4.833')] +[2023-02-23 20:13:40,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1183744. Throughput: 0: 866.5. Samples: 295220. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:13:40,203][00631] Avg episode reward: [(0, '5.153')] +[2023-02-23 20:13:40,208][10884] Saving new best policy, reward=5.153! +[2023-02-23 20:13:41,094][10898] Updated weights for policy 0, policy_version 290 (0.0037) +[2023-02-23 20:13:45,197][00631] Fps is (10 sec: 4096.1, 60 sec: 3481.9, 300 sec: 3401.8). Total num frames: 1204224. Throughput: 0: 885.9. Samples: 301526. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:13:45,200][00631] Avg episode reward: [(0, '4.909')] +[2023-02-23 20:13:50,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.7, 300 sec: 3401.8). Total num frames: 1220608. Throughput: 0: 889.6. Samples: 304812. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:13:50,200][00631] Avg episode reward: [(0, '4.702')] +[2023-02-23 20:13:51,928][10898] Updated weights for policy 0, policy_version 300 (0.0020) +[2023-02-23 20:13:55,202][00631] Fps is (10 sec: 3275.1, 60 sec: 3549.6, 300 sec: 3415.6). Total num frames: 1236992. Throughput: 0: 866.0. Samples: 309026. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:13:55,213][00631] Avg episode reward: [(0, '4.725')] +[2023-02-23 20:14:00,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1253376. Throughput: 0: 876.8. Samples: 313610. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:14:00,204][00631] Avg episode reward: [(0, '4.865')] +[2023-02-23 20:14:03,885][10898] Updated weights for policy 0, policy_version 310 (0.0016) +[2023-02-23 20:14:05,197][00631] Fps is (10 sec: 3688.3, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1273856. Throughput: 0: 895.0. Samples: 316888. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:14:05,204][00631] Avg episode reward: [(0, '5.020')] +[2023-02-23 20:14:10,197][00631] Fps is (10 sec: 4096.1, 60 sec: 3550.0, 300 sec: 3415.7). Total num frames: 1294336. Throughput: 0: 893.0. Samples: 323358. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:14:10,201][00631] Avg episode reward: [(0, '4.917')] +[2023-02-23 20:14:15,198][00631] Fps is (10 sec: 3276.5, 60 sec: 3549.8, 300 sec: 3401.8). Total num frames: 1306624. Throughput: 0: 863.1. Samples: 327418. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:14:15,208][00631] Avg episode reward: [(0, '4.619')] +[2023-02-23 20:14:16,248][10898] Updated weights for policy 0, policy_version 320 (0.0018) +[2023-02-23 20:14:20,197][00631] Fps is (10 sec: 2457.6, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1318912. Throughput: 0: 862.2. Samples: 329448. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:14:20,200][00631] Avg episode reward: [(0, '5.008')] +[2023-02-23 20:14:25,197][00631] Fps is (10 sec: 3277.1, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1339392. Throughput: 0: 886.8. Samples: 335124. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:14:25,200][00631] Avg episode reward: [(0, '5.163')] +[2023-02-23 20:14:25,258][10884] Saving new best policy, reward=5.163! +[2023-02-23 20:14:27,147][10898] Updated weights for policy 0, policy_version 330 (0.0015) +[2023-02-23 20:14:30,197][00631] Fps is (10 sec: 4505.6, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 1363968. Throughput: 0: 889.3. Samples: 341544. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:14:30,200][00631] Avg episode reward: [(0, '5.168')] +[2023-02-23 20:14:30,205][10884] Saving new best policy, reward=5.168! +[2023-02-23 20:14:35,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 1376256. Throughput: 0: 864.7. Samples: 343722. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:14:35,202][00631] Avg episode reward: [(0, '5.217')] +[2023-02-23 20:14:35,225][10884] Saving new best policy, reward=5.217! +[2023-02-23 20:14:40,200][00631] Fps is (10 sec: 2456.8, 60 sec: 3413.2, 300 sec: 3401.7). Total num frames: 1388544. Throughput: 0: 860.5. Samples: 347748. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:14:40,202][00631] Avg episode reward: [(0, '5.291')] +[2023-02-23 20:14:40,252][10884] Saving new best policy, reward=5.291! +[2023-02-23 20:14:40,273][10898] Updated weights for policy 0, policy_version 340 (0.0023) +[2023-02-23 20:14:45,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 1409024. Throughput: 0: 878.7. Samples: 353152. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:14:45,200][00631] Avg episode reward: [(0, '5.197')] +[2023-02-23 20:14:50,197][00631] Fps is (10 sec: 4097.3, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1429504. Throughput: 0: 876.9. Samples: 356348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:14:50,200][00631] Avg episode reward: [(0, '5.056')] +[2023-02-23 20:14:50,357][10898] Updated weights for policy 0, policy_version 350 (0.0016) +[2023-02-23 20:14:55,198][00631] Fps is (10 sec: 3686.1, 60 sec: 3481.9, 300 sec: 3401.8). Total num frames: 1445888. Throughput: 0: 855.1. Samples: 361836. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:14:55,207][00631] Avg episode reward: [(0, '5.167')] +[2023-02-23 20:15:00,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3415.7). Total num frames: 1462272. Throughput: 0: 856.5. Samples: 365960. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:15:00,200][00631] Avg episode reward: [(0, '4.956')] +[2023-02-23 20:15:03,900][10898] Updated weights for policy 0, policy_version 360 (0.0024) +[2023-02-23 20:15:05,197][00631] Fps is (10 sec: 3277.0, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 1478656. Throughput: 0: 862.0. Samples: 368240. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:15:05,205][00631] Avg episode reward: [(0, '5.097')] +[2023-02-23 20:15:05,225][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000361_1478656.pth... +[2023-02-23 20:15:05,364][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000158_647168.pth +[2023-02-23 20:15:10,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 1499136. Throughput: 0: 878.5. Samples: 374656. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:15:10,200][00631] Avg episode reward: [(0, '5.143')] +[2023-02-23 20:15:14,035][10898] Updated weights for policy 0, policy_version 370 (0.0014) +[2023-02-23 20:15:15,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.7, 300 sec: 3415.6). Total num frames: 1515520. Throughput: 0: 858.7. Samples: 380184. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:15:15,202][00631] Avg episode reward: [(0, '5.081')] +[2023-02-23 20:15:20,197][00631] Fps is (10 sec: 3276.7, 60 sec: 3549.9, 300 sec: 3429.5). Total num frames: 1531904. Throughput: 0: 854.8. Samples: 382186. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:15:20,204][00631] Avg episode reward: [(0, '5.154')] +[2023-02-23 20:15:25,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 1548288. Throughput: 0: 861.1. Samples: 386494. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:15:25,199][00631] Avg episode reward: [(0, '5.350')] +[2023-02-23 20:15:25,215][10884] Saving new best policy, reward=5.350! +[2023-02-23 20:15:27,003][10898] Updated weights for policy 0, policy_version 380 (0.0035) +[2023-02-23 20:15:30,197][00631] Fps is (10 sec: 3686.5, 60 sec: 3413.3, 300 sec: 3443.5). Total num frames: 1568768. Throughput: 0: 881.5. Samples: 392820. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:15:30,204][00631] Avg episode reward: [(0, '5.653')] +[2023-02-23 20:15:30,206][10884] Saving new best policy, reward=5.653! +[2023-02-23 20:15:35,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 1589248. Throughput: 0: 879.4. Samples: 395920. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:15:35,200][00631] Avg episode reward: [(0, '5.533')] +[2023-02-23 20:15:38,106][10898] Updated weights for policy 0, policy_version 390 (0.0017) +[2023-02-23 20:15:40,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3550.1, 300 sec: 3471.2). Total num frames: 1601536. Throughput: 0: 854.3. Samples: 400280. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:15:40,200][00631] Avg episode reward: [(0, '5.266')] +[2023-02-23 20:15:45,197][00631] Fps is (10 sec: 2457.6, 60 sec: 3413.3, 300 sec: 3471.3). Total num frames: 1613824. Throughput: 0: 854.9. Samples: 404432. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) +[2023-02-23 20:15:45,205][00631] Avg episode reward: [(0, '5.129')] +[2023-02-23 20:15:50,151][10898] Updated weights for policy 0, policy_version 400 (0.0023) +[2023-02-23 20:15:50,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 1638400. Throughput: 0: 874.9. Samples: 407612. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:15:50,202][00631] Avg episode reward: [(0, '5.390')] +[2023-02-23 20:15:55,202][00631] Fps is (10 sec: 4503.6, 60 sec: 3549.6, 300 sec: 3471.1). Total num frames: 1658880. Throughput: 0: 879.6. Samples: 414244. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:15:55,204][00631] Avg episode reward: [(0, '5.415')] +[2023-02-23 20:16:00,200][00631] Fps is (10 sec: 3275.7, 60 sec: 3481.4, 300 sec: 3471.2). Total num frames: 1671168. Throughput: 0: 854.4. Samples: 418634. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:16:00,203][00631] Avg episode reward: [(0, '5.618')] +[2023-02-23 20:16:02,511][10898] Updated weights for policy 0, policy_version 410 (0.0023) +[2023-02-23 20:16:05,198][00631] Fps is (10 sec: 2458.4, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 1683456. Throughput: 0: 853.9. Samples: 420614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:16:05,201][00631] Avg episode reward: [(0, '5.386')] +[2023-02-23 20:16:10,199][00631] Fps is (10 sec: 2458.0, 60 sec: 3276.7, 300 sec: 3443.4). Total num frames: 1695744. Throughput: 0: 833.8. Samples: 424018. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:16:10,202][00631] Avg episode reward: [(0, '5.548')] +[2023-02-23 20:16:15,197][00631] Fps is (10 sec: 2457.9, 60 sec: 3208.5, 300 sec: 3401.8). Total num frames: 1708032. Throughput: 0: 785.2. Samples: 428156. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:16:15,200][00631] Avg episode reward: [(0, '5.754')] +[2023-02-23 20:16:15,213][10884] Saving new best policy, reward=5.754! +[2023-02-23 20:16:17,357][10898] Updated weights for policy 0, policy_version 420 (0.0038) +[2023-02-23 20:16:20,198][00631] Fps is (10 sec: 2867.3, 60 sec: 3208.5, 300 sec: 3415.6). Total num frames: 1724416. Throughput: 0: 783.9. Samples: 431198. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:16:20,203][00631] Avg episode reward: [(0, '5.921')] +[2023-02-23 20:16:20,260][10884] Saving new best policy, reward=5.921! +[2023-02-23 20:16:25,203][00631] Fps is (10 sec: 3275.1, 60 sec: 3208.2, 300 sec: 3429.5). Total num frames: 1740800. Throughput: 0: 777.7. Samples: 435280. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:16:25,205][00631] Avg episode reward: [(0, '5.646')] +[2023-02-23 20:16:30,197][00631] Fps is (10 sec: 3277.2, 60 sec: 3140.3, 300 sec: 3415.6). Total num frames: 1757184. Throughput: 0: 786.1. Samples: 439808. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-23 20:16:30,200][00631] Avg episode reward: [(0, '5.704')] +[2023-02-23 20:16:31,005][10898] Updated weights for policy 0, policy_version 430 (0.0039) +[2023-02-23 20:16:35,197][00631] Fps is (10 sec: 3688.3, 60 sec: 3140.3, 300 sec: 3415.6). Total num frames: 1777664. Throughput: 0: 788.0. Samples: 443074. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:16:35,203][00631] Avg episode reward: [(0, '5.820')] +[2023-02-23 20:16:40,199][00631] Fps is (10 sec: 4095.0, 60 sec: 3276.7, 300 sec: 3429.5). Total num frames: 1798144. Throughput: 0: 784.9. Samples: 449562. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:16:40,202][00631] Avg episode reward: [(0, '6.686')] +[2023-02-23 20:16:40,204][10884] Saving new best policy, reward=6.686! +[2023-02-23 20:16:41,533][10898] Updated weights for policy 0, policy_version 440 (0.0023) +[2023-02-23 20:16:45,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3415.6). Total num frames: 1810432. Throughput: 0: 773.6. Samples: 453444. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-23 20:16:45,200][00631] Avg episode reward: [(0, '6.589')] +[2023-02-23 20:16:50,197][00631] Fps is (10 sec: 2867.9, 60 sec: 3140.3, 300 sec: 3415.6). Total num frames: 1826816. Throughput: 0: 776.1. Samples: 455538. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:16:50,199][00631] Avg episode reward: [(0, '7.056')] +[2023-02-23 20:16:50,207][10884] Saving new best policy, reward=7.056! +[2023-02-23 20:16:54,063][10898] Updated weights for policy 0, policy_version 450 (0.0031) +[2023-02-23 20:16:55,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3140.5, 300 sec: 3415.6). Total num frames: 1847296. Throughput: 0: 828.6. Samples: 461304. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:16:55,200][00631] Avg episode reward: [(0, '6.928')] +[2023-02-23 20:17:00,197][00631] Fps is (10 sec: 4095.9, 60 sec: 3277.0, 300 sec: 3429.6). Total num frames: 1867776. Throughput: 0: 877.6. Samples: 467646. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:17:00,203][00631] Avg episode reward: [(0, '6.895')] +[2023-02-23 20:17:05,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3276.9, 300 sec: 3415.6). Total num frames: 1880064. Throughput: 0: 854.9. Samples: 469668. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:17:05,203][00631] Avg episode reward: [(0, '6.679')] +[2023-02-23 20:17:05,223][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000459_1880064.pth... +[2023-02-23 20:17:05,380][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000260_1064960.pth +[2023-02-23 20:17:05,613][10898] Updated weights for policy 0, policy_version 460 (0.0025) +[2023-02-23 20:17:10,197][00631] Fps is (10 sec: 2867.3, 60 sec: 3345.1, 300 sec: 3429.5). Total num frames: 1896448. Throughput: 0: 854.7. Samples: 473738. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:17:10,209][00631] Avg episode reward: [(0, '6.361')] +[2023-02-23 20:17:15,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 1916928. Throughput: 0: 880.8. Samples: 479446. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:17:15,200][00631] Avg episode reward: [(0, '6.224')] +[2023-02-23 20:17:17,031][10898] Updated weights for policy 0, policy_version 470 (0.0036) +[2023-02-23 20:17:20,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3429.5). Total num frames: 1937408. Throughput: 0: 880.4. Samples: 482692. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:17:20,200][00631] Avg episode reward: [(0, '6.567')] +[2023-02-23 20:17:25,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.9, 300 sec: 3415.6). Total num frames: 1949696. Throughput: 0: 852.9. Samples: 487940. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:17:25,206][00631] Avg episode reward: [(0, '6.631')] +[2023-02-23 20:17:30,107][10898] Updated weights for policy 0, policy_version 480 (0.0026) +[2023-02-23 20:17:30,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 1966080. Throughput: 0: 858.1. Samples: 492058. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:17:30,201][00631] Avg episode reward: [(0, '6.598')] +[2023-02-23 20:17:35,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 1982464. Throughput: 0: 864.1. Samples: 494424. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:17:35,201][00631] Avg episode reward: [(0, '7.240')] +[2023-02-23 20:17:35,218][10884] Saving new best policy, reward=7.240! +[2023-02-23 20:17:40,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.5, 300 sec: 3415.7). Total num frames: 2002944. Throughput: 0: 875.8. Samples: 500716. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:17:40,204][00631] Avg episode reward: [(0, '7.084')] +[2023-02-23 20:17:40,498][10898] Updated weights for policy 0, policy_version 490 (0.0024) +[2023-02-23 20:17:45,204][00631] Fps is (10 sec: 3683.7, 60 sec: 3481.2, 300 sec: 3415.6). Total num frames: 2019328. Throughput: 0: 851.6. Samples: 505974. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:17:45,210][00631] Avg episode reward: [(0, '7.505')] +[2023-02-23 20:17:45,219][10884] Saving new best policy, reward=7.505! +[2023-02-23 20:17:50,198][00631] Fps is (10 sec: 2866.9, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 2031616. Throughput: 0: 850.2. Samples: 507926. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:17:50,201][00631] Avg episode reward: [(0, '7.473')] +[2023-02-23 20:17:53,994][10898] Updated weights for policy 0, policy_version 500 (0.0017) +[2023-02-23 20:17:55,197][00631] Fps is (10 sec: 3279.2, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 2052096. Throughput: 0: 857.4. Samples: 512322. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:17:55,200][00631] Avg episode reward: [(0, '7.523')] +[2023-02-23 20:17:55,207][10884] Saving new best policy, reward=7.523! +[2023-02-23 20:18:00,197][00631] Fps is (10 sec: 4096.5, 60 sec: 3413.4, 300 sec: 3415.6). Total num frames: 2072576. Throughput: 0: 871.5. Samples: 518664. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:18:00,200][00631] Avg episode reward: [(0, '6.969')] +[2023-02-23 20:18:04,006][10898] Updated weights for policy 0, policy_version 510 (0.0019) +[2023-02-23 20:18:05,203][00631] Fps is (10 sec: 3684.1, 60 sec: 3481.2, 300 sec: 3415.6). Total num frames: 2088960. Throughput: 0: 868.7. Samples: 521790. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:18:05,206][00631] Avg episode reward: [(0, '7.577')] +[2023-02-23 20:18:05,219][10884] Saving new best policy, reward=7.577! +[2023-02-23 20:18:10,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 2101248. Throughput: 0: 842.7. Samples: 525862. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:18:10,199][00631] Avg episode reward: [(0, '7.828')] +[2023-02-23 20:18:10,225][10884] Saving new best policy, reward=7.828! +[2023-02-23 20:18:15,197][00631] Fps is (10 sec: 2869.0, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 2117632. Throughput: 0: 844.0. Samples: 530036. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:18:15,203][00631] Avg episode reward: [(0, '7.907')] +[2023-02-23 20:18:15,221][10884] Saving new best policy, reward=7.907! +[2023-02-23 20:18:17,647][10898] Updated weights for policy 0, policy_version 520 (0.0013) +[2023-02-23 20:18:20,201][00631] Fps is (10 sec: 3685.0, 60 sec: 3344.9, 300 sec: 3401.7). Total num frames: 2138112. Throughput: 0: 862.2. Samples: 533224. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:18:20,204][00631] Avg episode reward: [(0, '7.665')] +[2023-02-23 20:18:25,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 2158592. Throughput: 0: 866.1. Samples: 539690. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:18:25,202][00631] Avg episode reward: [(0, '7.608')] +[2023-02-23 20:18:28,881][10898] Updated weights for policy 0, policy_version 530 (0.0016) +[2023-02-23 20:18:30,197][00631] Fps is (10 sec: 3278.0, 60 sec: 3413.3, 300 sec: 3415.7). Total num frames: 2170880. Throughput: 0: 843.2. Samples: 543912. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:18:30,200][00631] Avg episode reward: [(0, '8.311')] +[2023-02-23 20:18:30,206][10884] Saving new best policy, reward=8.311! +[2023-02-23 20:18:35,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 2187264. Throughput: 0: 843.1. Samples: 545866. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:18:35,205][00631] Avg episode reward: [(0, '8.714')] +[2023-02-23 20:18:35,216][10884] Saving new best policy, reward=8.714! +[2023-02-23 20:18:40,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 2207744. Throughput: 0: 868.2. Samples: 551392. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:18:40,203][00631] Avg episode reward: [(0, '8.961')] +[2023-02-23 20:18:40,207][10884] Saving new best policy, reward=8.961! +[2023-02-23 20:18:40,895][10898] Updated weights for policy 0, policy_version 540 (0.0035) +[2023-02-23 20:18:45,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3482.0, 300 sec: 3415.6). Total num frames: 2228224. Throughput: 0: 867.2. Samples: 557688. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:18:45,201][00631] Avg episode reward: [(0, '8.587')] +[2023-02-23 20:18:50,199][00631] Fps is (10 sec: 3276.1, 60 sec: 3481.5, 300 sec: 3401.8). Total num frames: 2240512. Throughput: 0: 847.1. Samples: 559906. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:18:50,206][00631] Avg episode reward: [(0, '8.543')] +[2023-02-23 20:18:53,554][10898] Updated weights for policy 0, policy_version 550 (0.0015) +[2023-02-23 20:18:55,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 2256896. Throughput: 0: 847.9. Samples: 564018. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:18:55,204][00631] Avg episode reward: [(0, '8.362')] +[2023-02-23 20:19:00,197][00631] Fps is (10 sec: 3277.5, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 2273280. Throughput: 0: 876.4. Samples: 569474. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:19:00,206][00631] Avg episode reward: [(0, '8.083')] +[2023-02-23 20:19:04,094][10898] Updated weights for policy 0, policy_version 560 (0.0013) +[2023-02-23 20:19:05,197][00631] Fps is (10 sec: 4096.1, 60 sec: 3482.0, 300 sec: 3401.8). Total num frames: 2297856. Throughput: 0: 878.2. Samples: 572738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:19:05,204][00631] Avg episode reward: [(0, '8.450')] +[2023-02-23 20:19:05,215][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000561_2297856.pth... +[2023-02-23 20:19:05,358][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000361_1478656.pth +[2023-02-23 20:19:10,197][00631] Fps is (10 sec: 3686.3, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2310144. Throughput: 0: 854.0. Samples: 578120. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:19:10,204][00631] Avg episode reward: [(0, '8.656')] +[2023-02-23 20:19:15,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 2326528. Throughput: 0: 850.0. Samples: 582160. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:19:15,201][00631] Avg episode reward: [(0, '9.083')] +[2023-02-23 20:19:15,211][10884] Saving new best policy, reward=9.083! +[2023-02-23 20:19:17,827][10898] Updated weights for policy 0, policy_version 570 (0.0049) +[2023-02-23 20:19:20,197][00631] Fps is (10 sec: 3276.9, 60 sec: 3413.6, 300 sec: 3401.8). Total num frames: 2342912. Throughput: 0: 856.6. Samples: 584412. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:19:20,204][00631] Avg episode reward: [(0, '8.578')] +[2023-02-23 20:19:25,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2363392. Throughput: 0: 878.8. Samples: 590940. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:19:25,205][00631] Avg episode reward: [(0, '9.034')] +[2023-02-23 20:19:27,237][10898] Updated weights for policy 0, policy_version 580 (0.0014) +[2023-02-23 20:19:30,203][00631] Fps is (10 sec: 3684.2, 60 sec: 3481.3, 300 sec: 3401.7). Total num frames: 2379776. Throughput: 0: 856.6. Samples: 596238. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:19:30,207][00631] Avg episode reward: [(0, '8.816')] +[2023-02-23 20:19:35,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3415.7). Total num frames: 2396160. Throughput: 0: 853.0. Samples: 598290. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:19:35,202][00631] Avg episode reward: [(0, '8.581')] +[2023-02-23 20:19:40,198][00631] Fps is (10 sec: 3278.6, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 2412544. Throughput: 0: 858.7. Samples: 602660. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:19:40,201][00631] Avg episode reward: [(0, '9.021')] +[2023-02-23 20:19:40,998][10898] Updated weights for policy 0, policy_version 590 (0.0013) +[2023-02-23 20:19:45,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 2433024. Throughput: 0: 874.3. Samples: 608816. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:19:45,200][00631] Avg episode reward: [(0, '9.892')] +[2023-02-23 20:19:45,212][10884] Saving new best policy, reward=9.892! +[2023-02-23 20:19:50,198][00631] Fps is (10 sec: 3686.3, 60 sec: 3481.7, 300 sec: 3401.8). Total num frames: 2449408. Throughput: 0: 869.1. Samples: 611850. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:19:50,203][00631] Avg episode reward: [(0, '9.855')] +[2023-02-23 20:19:52,216][10898] Updated weights for policy 0, policy_version 600 (0.0021) +[2023-02-23 20:19:55,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2461696. Throughput: 0: 844.8. Samples: 616136. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:19:55,199][00631] Avg episode reward: [(0, '10.865')] +[2023-02-23 20:19:55,217][10884] Saving new best policy, reward=10.865! +[2023-02-23 20:20:00,197][00631] Fps is (10 sec: 2867.4, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2478080. Throughput: 0: 847.5. Samples: 620296. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:20:00,204][00631] Avg episode reward: [(0, '9.850')] +[2023-02-23 20:20:04,430][10898] Updated weights for policy 0, policy_version 610 (0.0015) +[2023-02-23 20:20:05,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 2498560. Throughput: 0: 870.4. Samples: 623580. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:20:05,200][00631] Avg episode reward: [(0, '10.555')] +[2023-02-23 20:20:10,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2519040. Throughput: 0: 868.5. Samples: 630024. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:20:10,200][00631] Avg episode reward: [(0, '10.528')] +[2023-02-23 20:20:15,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2531328. Throughput: 0: 839.2. Samples: 633996. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:20:15,199][00631] Avg episode reward: [(0, '10.914')] +[2023-02-23 20:20:15,228][10884] Saving new best policy, reward=10.914! +[2023-02-23 20:20:17,326][10898] Updated weights for policy 0, policy_version 620 (0.0026) +[2023-02-23 20:20:20,197][00631] Fps is (10 sec: 2457.5, 60 sec: 3345.0, 300 sec: 3374.0). Total num frames: 2543616. Throughput: 0: 836.2. Samples: 635920. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:20:20,204][00631] Avg episode reward: [(0, '10.304')] +[2023-02-23 20:20:25,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2568192. Throughput: 0: 862.1. Samples: 641454. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:20:25,205][00631] Avg episode reward: [(0, '10.527')] +[2023-02-23 20:20:27,898][10898] Updated weights for policy 0, policy_version 630 (0.0018) +[2023-02-23 20:20:30,197][00631] Fps is (10 sec: 4505.6, 60 sec: 3481.9, 300 sec: 3387.9). Total num frames: 2588672. Throughput: 0: 863.9. Samples: 647690. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:20:30,204][00631] Avg episode reward: [(0, '11.505')] +[2023-02-23 20:20:30,208][10884] Saving new best policy, reward=11.505! +[2023-02-23 20:20:35,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2600960. Throughput: 0: 844.1. Samples: 649836. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:20:35,204][00631] Avg episode reward: [(0, '11.784')] +[2023-02-23 20:20:35,216][10884] Saving new best policy, reward=11.784! +[2023-02-23 20:20:40,198][00631] Fps is (10 sec: 2457.6, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 2613248. Throughput: 0: 837.7. Samples: 653832. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:20:40,210][00631] Avg episode reward: [(0, '12.264')] +[2023-02-23 20:20:40,213][10884] Saving new best policy, reward=12.264! +[2023-02-23 20:20:41,805][10898] Updated weights for policy 0, policy_version 640 (0.0032) +[2023-02-23 20:20:45,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 2633728. Throughput: 0: 866.8. Samples: 659302. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:20:45,205][00631] Avg episode reward: [(0, '11.640')] +[2023-02-23 20:20:50,197][00631] Fps is (10 sec: 4096.2, 60 sec: 3413.4, 300 sec: 3374.0). Total num frames: 2654208. Throughput: 0: 865.6. Samples: 662530. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:20:50,204][00631] Avg episode reward: [(0, '11.628')] +[2023-02-23 20:20:51,374][10898] Updated weights for policy 0, policy_version 650 (0.0015) +[2023-02-23 20:20:55,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3387.9). Total num frames: 2670592. Throughput: 0: 842.7. Samples: 667944. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:20:55,202][00631] Avg episode reward: [(0, '11.410')] +[2023-02-23 20:21:00,197][00631] Fps is (10 sec: 2867.1, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2682880. Throughput: 0: 845.6. Samples: 672046. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:21:00,201][00631] Avg episode reward: [(0, '11.291')] +[2023-02-23 20:21:04,799][10898] Updated weights for policy 0, policy_version 660 (0.0030) +[2023-02-23 20:21:05,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3415.7). Total num frames: 2703360. Throughput: 0: 854.4. Samples: 674370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:21:05,205][00631] Avg episode reward: [(0, '11.255')] +[2023-02-23 20:21:05,220][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000660_2703360.pth... +[2023-02-23 20:21:05,342][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000459_1880064.pth +[2023-02-23 20:21:10,197][00631] Fps is (10 sec: 4096.1, 60 sec: 3413.3, 300 sec: 3443.4). Total num frames: 2723840. Throughput: 0: 873.2. Samples: 680750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:21:10,204][00631] Avg episode reward: [(0, '11.247')] +[2023-02-23 20:21:15,202][00631] Fps is (10 sec: 3684.7, 60 sec: 3481.3, 300 sec: 3443.4). Total num frames: 2740224. Throughput: 0: 850.8. Samples: 685978. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:21:15,205][00631] Avg episode reward: [(0, '11.546')] +[2023-02-23 20:21:15,967][10898] Updated weights for policy 0, policy_version 670 (0.0026) +[2023-02-23 20:21:20,197][00631] Fps is (10 sec: 2867.1, 60 sec: 3481.6, 300 sec: 3429.6). Total num frames: 2752512. Throughput: 0: 848.4. Samples: 688016. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:21:20,202][00631] Avg episode reward: [(0, '12.145')] +[2023-02-23 20:21:25,197][00631] Fps is (10 sec: 2868.5, 60 sec: 3345.1, 300 sec: 3429.5). Total num frames: 2768896. Throughput: 0: 855.5. Samples: 692330. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:21:25,200][00631] Avg episode reward: [(0, '12.932')] +[2023-02-23 20:21:25,208][10884] Saving new best policy, reward=12.932! +[2023-02-23 20:21:28,273][10898] Updated weights for policy 0, policy_version 680 (0.0039) +[2023-02-23 20:21:30,205][00631] Fps is (10 sec: 3683.7, 60 sec: 3344.7, 300 sec: 3429.4). Total num frames: 2789376. Throughput: 0: 866.7. Samples: 698312. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:21:30,207][00631] Avg episode reward: [(0, '12.863')] +[2023-02-23 20:21:35,200][00631] Fps is (10 sec: 3275.8, 60 sec: 3344.9, 300 sec: 3401.8). Total num frames: 2801664. Throughput: 0: 838.9. Samples: 700282. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:21:35,202][00631] Avg episode reward: [(0, '12.977')] +[2023-02-23 20:21:35,222][10884] Saving new best policy, reward=12.977! +[2023-02-23 20:21:40,201][00631] Fps is (10 sec: 2458.5, 60 sec: 3344.9, 300 sec: 3401.7). Total num frames: 2813952. Throughput: 0: 790.6. Samples: 703522. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:21:40,208][00631] Avg episode reward: [(0, '12.004')] +[2023-02-23 20:21:45,085][10898] Updated weights for policy 0, policy_version 690 (0.0015) +[2023-02-23 20:21:45,197][00631] Fps is (10 sec: 2458.4, 60 sec: 3208.5, 300 sec: 3387.9). Total num frames: 2826240. Throughput: 0: 778.9. Samples: 707094. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:21:45,203][00631] Avg episode reward: [(0, '11.558')] +[2023-02-23 20:21:50,197][00631] Fps is (10 sec: 2868.3, 60 sec: 3140.3, 300 sec: 3374.0). Total num frames: 2842624. Throughput: 0: 777.7. Samples: 709366. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:21:50,202][00631] Avg episode reward: [(0, '11.112')] +[2023-02-23 20:21:55,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3208.5, 300 sec: 3374.0). Total num frames: 2863104. Throughput: 0: 780.9. Samples: 715890. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:21:55,200][00631] Avg episode reward: [(0, '12.171')] +[2023-02-23 20:21:55,303][10898] Updated weights for policy 0, policy_version 700 (0.0018) +[2023-02-23 20:22:00,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 2883584. Throughput: 0: 788.0. Samples: 721432. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:22:00,205][00631] Avg episode reward: [(0, '12.791')] +[2023-02-23 20:22:05,200][00631] Fps is (10 sec: 3275.8, 60 sec: 3208.4, 300 sec: 3387.8). Total num frames: 2895872. Throughput: 0: 787.2. Samples: 723442. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:22:05,204][00631] Avg episode reward: [(0, '13.207')] +[2023-02-23 20:22:05,225][10884] Saving new best policy, reward=13.207! +[2023-02-23 20:22:08,745][10898] Updated weights for policy 0, policy_version 710 (0.0015) +[2023-02-23 20:22:10,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3140.3, 300 sec: 3374.0). Total num frames: 2912256. Throughput: 0: 787.7. Samples: 727778. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:22:10,200][00631] Avg episode reward: [(0, '13.409')] +[2023-02-23 20:22:10,203][10884] Saving new best policy, reward=13.409! +[2023-02-23 20:22:15,197][00631] Fps is (10 sec: 3687.5, 60 sec: 3208.8, 300 sec: 3374.0). Total num frames: 2932736. Throughput: 0: 797.6. Samples: 734196. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:22:15,204][00631] Avg episode reward: [(0, '13.174')] +[2023-02-23 20:22:18,311][10898] Updated weights for policy 0, policy_version 720 (0.0031) +[2023-02-23 20:22:20,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 2953216. Throughput: 0: 825.6. Samples: 737432. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:22:20,199][00631] Avg episode reward: [(0, '13.895')] +[2023-02-23 20:22:20,202][10884] Saving new best policy, reward=13.895! +[2023-02-23 20:22:25,198][00631] Fps is (10 sec: 3276.5, 60 sec: 3276.7, 300 sec: 3387.9). Total num frames: 2965504. Throughput: 0: 844.9. Samples: 741540. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:22:25,204][00631] Avg episode reward: [(0, '13.690')] +[2023-02-23 20:22:30,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3208.9, 300 sec: 3387.9). Total num frames: 2981888. Throughput: 0: 861.5. Samples: 745862. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:22:30,205][00631] Avg episode reward: [(0, '14.377')] +[2023-02-23 20:22:30,208][10884] Saving new best policy, reward=14.377! +[2023-02-23 20:22:32,140][10898] Updated weights for policy 0, policy_version 730 (0.0022) +[2023-02-23 20:22:35,197][00631] Fps is (10 sec: 3686.7, 60 sec: 3345.2, 300 sec: 3387.9). Total num frames: 3002368. Throughput: 0: 879.9. Samples: 748964. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:22:35,205][00631] Avg episode reward: [(0, '13.019')] +[2023-02-23 20:22:40,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.8, 300 sec: 3401.8). Total num frames: 3022848. Throughput: 0: 881.2. Samples: 755546. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:22:40,201][00631] Avg episode reward: [(0, '12.255')] +[2023-02-23 20:22:42,687][10898] Updated weights for policy 0, policy_version 740 (0.0013) +[2023-02-23 20:22:45,197][00631] Fps is (10 sec: 3276.9, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 3035136. Throughput: 0: 851.1. Samples: 759730. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:22:45,204][00631] Avg episode reward: [(0, '12.477')] +[2023-02-23 20:22:50,197][00631] Fps is (10 sec: 2457.6, 60 sec: 3413.3, 300 sec: 3374.0). Total num frames: 3047424. Throughput: 0: 850.7. Samples: 761722. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:22:50,200][00631] Avg episode reward: [(0, '12.962')] +[2023-02-23 20:22:55,134][10898] Updated weights for policy 0, policy_version 750 (0.0018) +[2023-02-23 20:22:55,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3387.9). Total num frames: 3072000. Throughput: 0: 877.5. Samples: 767264. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:22:55,199][00631] Avg episode reward: [(0, '14.248')] +[2023-02-23 20:23:00,197][00631] Fps is (10 sec: 4505.6, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 3092480. Throughput: 0: 878.4. Samples: 773726. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-23 20:23:00,205][00631] Avg episode reward: [(0, '15.095')] +[2023-02-23 20:23:00,209][10884] Saving new best policy, reward=15.095! +[2023-02-23 20:23:05,197][00631] Fps is (10 sec: 3276.7, 60 sec: 3481.8, 300 sec: 3401.8). Total num frames: 3104768. Throughput: 0: 853.1. Samples: 775820. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:23:05,202][00631] Avg episode reward: [(0, '14.526')] +[2023-02-23 20:23:05,217][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000758_3104768.pth... +[2023-02-23 20:23:05,427][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000561_2297856.pth +[2023-02-23 20:23:07,533][10898] Updated weights for policy 0, policy_version 760 (0.0019) +[2023-02-23 20:23:10,197][00631] Fps is (10 sec: 2457.6, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 3117056. Throughput: 0: 849.4. Samples: 779762. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:23:10,204][00631] Avg episode reward: [(0, '15.004')] +[2023-02-23 20:23:15,198][00631] Fps is (10 sec: 3276.7, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 3137536. Throughput: 0: 874.8. Samples: 785228. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:23:15,201][00631] Avg episode reward: [(0, '15.152')] +[2023-02-23 20:23:15,217][10884] Saving new best policy, reward=15.152! +[2023-02-23 20:23:18,398][10898] Updated weights for policy 0, policy_version 770 (0.0033) +[2023-02-23 20:23:20,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 3158016. Throughput: 0: 876.7. Samples: 788416. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:23:20,200][00631] Avg episode reward: [(0, '14.772')] +[2023-02-23 20:23:25,197][00631] Fps is (10 sec: 3686.6, 60 sec: 3481.7, 300 sec: 3401.8). Total num frames: 3174400. Throughput: 0: 854.1. Samples: 793980. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:23:25,203][00631] Avg episode reward: [(0, '14.043')] +[2023-02-23 20:23:30,199][00631] Fps is (10 sec: 2866.7, 60 sec: 3413.2, 300 sec: 3387.9). Total num frames: 3186688. Throughput: 0: 851.5. Samples: 798048. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:23:30,209][00631] Avg episode reward: [(0, '14.067')] +[2023-02-23 20:23:31,778][10898] Updated weights for policy 0, policy_version 780 (0.0013) +[2023-02-23 20:23:35,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 3207168. Throughput: 0: 858.6. Samples: 800360. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:23:35,203][00631] Avg episode reward: [(0, '14.713')] +[2023-02-23 20:23:40,197][00631] Fps is (10 sec: 4096.7, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 3227648. Throughput: 0: 881.8. Samples: 806944. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:23:40,200][00631] Avg episode reward: [(0, '16.030')] +[2023-02-23 20:23:40,207][10884] Saving new best policy, reward=16.030! +[2023-02-23 20:23:41,511][10898] Updated weights for policy 0, policy_version 790 (0.0014) +[2023-02-23 20:23:45,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 3244032. Throughput: 0: 857.7. Samples: 812322. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:23:45,201][00631] Avg episode reward: [(0, '15.609')] +[2023-02-23 20:23:50,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3401.8). Total num frames: 3260416. Throughput: 0: 857.3. Samples: 814398. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:23:50,200][00631] Avg episode reward: [(0, '16.033')] +[2023-02-23 20:23:50,202][10884] Saving new best policy, reward=16.033! +[2023-02-23 20:23:54,949][10898] Updated weights for policy 0, policy_version 800 (0.0026) +[2023-02-23 20:23:55,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 3276800. Throughput: 0: 863.8. Samples: 818632. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:23:55,200][00631] Avg episode reward: [(0, '16.357')] +[2023-02-23 20:23:55,208][10884] Saving new best policy, reward=16.357! +[2023-02-23 20:24:00,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 3297280. Throughput: 0: 882.1. Samples: 824924. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:24:00,200][00631] Avg episode reward: [(0, '15.760')] +[2023-02-23 20:24:05,144][10898] Updated weights for policy 0, policy_version 810 (0.0032) +[2023-02-23 20:24:05,200][00631] Fps is (10 sec: 4094.7, 60 sec: 3549.7, 300 sec: 3415.6). Total num frames: 3317760. Throughput: 0: 885.4. Samples: 828260. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:24:05,203][00631] Avg episode reward: [(0, '16.946')] +[2023-02-23 20:24:05,222][10884] Saving new best policy, reward=16.946! +[2023-02-23 20:24:10,199][00631] Fps is (10 sec: 3276.3, 60 sec: 3549.8, 300 sec: 3401.7). Total num frames: 3330048. Throughput: 0: 859.5. Samples: 832658. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:24:10,213][00631] Avg episode reward: [(0, '17.842')] +[2023-02-23 20:24:10,223][10884] Saving new best policy, reward=17.842! +[2023-02-23 20:24:15,197][00631] Fps is (10 sec: 2458.4, 60 sec: 3413.4, 300 sec: 3387.9). Total num frames: 3342336. Throughput: 0: 861.2. Samples: 836800. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:24:15,199][00631] Avg episode reward: [(0, '17.808')] +[2023-02-23 20:24:18,119][10898] Updated weights for policy 0, policy_version 820 (0.0032) +[2023-02-23 20:24:20,197][00631] Fps is (10 sec: 3686.9, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 3366912. Throughput: 0: 884.0. Samples: 840138. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:24:20,203][00631] Avg episode reward: [(0, '16.501')] +[2023-02-23 20:24:25,197][00631] Fps is (10 sec: 4505.5, 60 sec: 3549.9, 300 sec: 3415.7). Total num frames: 3387392. Throughput: 0: 879.2. Samples: 846510. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-23 20:24:25,200][00631] Avg episode reward: [(0, '15.105')] +[2023-02-23 20:24:29,173][10898] Updated weights for policy 0, policy_version 830 (0.0012) +[2023-02-23 20:24:30,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3550.0, 300 sec: 3401.8). Total num frames: 3399680. Throughput: 0: 856.3. Samples: 850854. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:24:30,205][00631] Avg episode reward: [(0, '14.501')] +[2023-02-23 20:24:35,197][00631] Fps is (10 sec: 2457.6, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 3411968. Throughput: 0: 856.1. Samples: 852922. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:24:35,202][00631] Avg episode reward: [(0, '14.072')] +[2023-02-23 20:24:40,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 3436544. Throughput: 0: 886.3. Samples: 858514. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:24:40,204][00631] Avg episode reward: [(0, '14.796')] +[2023-02-23 20:24:41,030][10898] Updated weights for policy 0, policy_version 840 (0.0016) +[2023-02-23 20:24:45,197][00631] Fps is (10 sec: 4505.6, 60 sec: 3549.9, 300 sec: 3415.7). Total num frames: 3457024. Throughput: 0: 890.0. Samples: 864976. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:24:45,207][00631] Avg episode reward: [(0, '15.032')] +[2023-02-23 20:24:50,203][00631] Fps is (10 sec: 3274.8, 60 sec: 3481.2, 300 sec: 3415.6). Total num frames: 3469312. Throughput: 0: 866.1. Samples: 867236. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:24:50,212][00631] Avg episode reward: [(0, '14.688')] +[2023-02-23 20:24:53,575][10898] Updated weights for policy 0, policy_version 850 (0.0027) +[2023-02-23 20:24:55,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 3485696. Throughput: 0: 857.5. Samples: 871244. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:24:55,203][00631] Avg episode reward: [(0, '16.377')] +[2023-02-23 20:25:00,197][00631] Fps is (10 sec: 3688.7, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 3506176. Throughput: 0: 889.9. Samples: 876846. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:25:00,200][00631] Avg episode reward: [(0, '18.595')] +[2023-02-23 20:25:00,205][10884] Saving new best policy, reward=18.595! +[2023-02-23 20:25:03,944][10898] Updated weights for policy 0, policy_version 860 (0.0027) +[2023-02-23 20:25:05,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.8, 300 sec: 3415.6). Total num frames: 3526656. Throughput: 0: 885.9. Samples: 880004. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:25:05,199][00631] Avg episode reward: [(0, '18.896')] +[2023-02-23 20:25:05,214][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000861_3526656.pth... +[2023-02-23 20:25:05,366][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000660_2703360.pth +[2023-02-23 20:25:05,394][10884] Saving new best policy, reward=18.896! +[2023-02-23 20:25:10,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.7, 300 sec: 3415.6). Total num frames: 3538944. Throughput: 0: 860.0. Samples: 885210. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) +[2023-02-23 20:25:10,204][00631] Avg episode reward: [(0, '19.572')] +[2023-02-23 20:25:10,226][10884] Saving new best policy, reward=19.572! +[2023-02-23 20:25:15,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3429.5). Total num frames: 3555328. Throughput: 0: 852.2. Samples: 889202. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:25:15,204][00631] Avg episode reward: [(0, '20.910')] +[2023-02-23 20:25:15,222][10884] Saving new best policy, reward=20.910! +[2023-02-23 20:25:17,752][10898] Updated weights for policy 0, policy_version 870 (0.0017) +[2023-02-23 20:25:20,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 3571712. Throughput: 0: 858.7. Samples: 891562. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:25:20,200][00631] Avg episode reward: [(0, '21.470')] +[2023-02-23 20:25:20,205][10884] Saving new best policy, reward=21.470! +[2023-02-23 20:25:25,197][00631] Fps is (10 sec: 3686.3, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 3592192. Throughput: 0: 876.0. Samples: 897934. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:25:25,200][00631] Avg episode reward: [(0, '21.191')] +[2023-02-23 20:25:27,606][10898] Updated weights for policy 0, policy_version 880 (0.0024) +[2023-02-23 20:25:30,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 3608576. Throughput: 0: 851.2. Samples: 903282. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:25:30,201][00631] Avg episode reward: [(0, '21.903')] +[2023-02-23 20:25:30,206][10884] Saving new best policy, reward=21.903! +[2023-02-23 20:25:35,197][00631] Fps is (10 sec: 3276.9, 60 sec: 3549.9, 300 sec: 3429.5). Total num frames: 3624960. Throughput: 0: 845.0. Samples: 905256. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:25:35,206][00631] Avg episode reward: [(0, '23.333')] +[2023-02-23 20:25:35,218][10884] Saving new best policy, reward=23.333! +[2023-02-23 20:25:40,198][00631] Fps is (10 sec: 3276.6, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 3641344. Throughput: 0: 854.0. Samples: 909674. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:25:40,208][00631] Avg episode reward: [(0, '23.403')] +[2023-02-23 20:25:40,211][10884] Saving new best policy, reward=23.403! +[2023-02-23 20:25:41,099][10898] Updated weights for policy 0, policy_version 890 (0.0013) +[2023-02-23 20:25:45,197][00631] Fps is (10 sec: 3686.3, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 3661824. Throughput: 0: 869.1. Samples: 915954. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:25:45,200][00631] Avg episode reward: [(0, '23.378')] +[2023-02-23 20:25:50,201][00631] Fps is (10 sec: 3685.1, 60 sec: 3481.7, 300 sec: 3415.6). Total num frames: 3678208. Throughput: 0: 871.2. Samples: 919212. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:25:50,205][00631] Avg episode reward: [(0, '24.062')] +[2023-02-23 20:25:50,207][10884] Saving new best policy, reward=24.062! +[2023-02-23 20:25:52,341][10898] Updated weights for policy 0, policy_version 900 (0.0016) +[2023-02-23 20:25:55,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 3694592. Throughput: 0: 847.6. Samples: 923350. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) +[2023-02-23 20:25:55,202][00631] Avg episode reward: [(0, '23.451')] +[2023-02-23 20:26:00,197][00631] Fps is (10 sec: 2868.3, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 3706880. Throughput: 0: 859.9. Samples: 927898. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:26:00,200][00631] Avg episode reward: [(0, '23.280')] +[2023-02-23 20:26:03,952][10898] Updated weights for policy 0, policy_version 910 (0.0031) +[2023-02-23 20:26:05,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 3731456. Throughput: 0: 881.3. Samples: 931220. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:26:05,203][00631] Avg episode reward: [(0, '22.549')] +[2023-02-23 20:26:10,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3415.7). Total num frames: 3747840. Throughput: 0: 880.3. Samples: 937546. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:26:10,199][00631] Avg episode reward: [(0, '23.821')] +[2023-02-23 20:26:15,198][00631] Fps is (10 sec: 3276.5, 60 sec: 3481.5, 300 sec: 3429.5). Total num frames: 3764224. Throughput: 0: 852.3. Samples: 941638. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:26:15,201][00631] Avg episode reward: [(0, '23.852')] +[2023-02-23 20:26:16,581][10898] Updated weights for policy 0, policy_version 920 (0.0021) +[2023-02-23 20:26:20,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 3776512. Throughput: 0: 852.7. Samples: 943628. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) +[2023-02-23 20:26:20,200][00631] Avg episode reward: [(0, '25.148')] +[2023-02-23 20:26:20,205][10884] Saving new best policy, reward=25.148! +[2023-02-23 20:26:25,197][00631] Fps is (10 sec: 3277.1, 60 sec: 3413.3, 300 sec: 3415.7). Total num frames: 3796992. Throughput: 0: 879.0. Samples: 949230. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) +[2023-02-23 20:26:25,202][00631] Avg episode reward: [(0, '24.122')] +[2023-02-23 20:26:27,505][10898] Updated weights for policy 0, policy_version 930 (0.0015) +[2023-02-23 20:26:30,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3443.5). Total num frames: 3817472. Throughput: 0: 880.2. Samples: 955562. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:26:30,202][00631] Avg episode reward: [(0, '23.277')] +[2023-02-23 20:26:35,199][00631] Fps is (10 sec: 3685.9, 60 sec: 3481.5, 300 sec: 3457.3). Total num frames: 3833856. Throughput: 0: 854.0. Samples: 957642. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:26:35,203][00631] Avg episode reward: [(0, '22.759')] +[2023-02-23 20:26:40,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3457.3). Total num frames: 3846144. Throughput: 0: 851.7. Samples: 961676. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:26:40,202][00631] Avg episode reward: [(0, '22.627')] +[2023-02-23 20:26:41,056][10898] Updated weights for policy 0, policy_version 940 (0.0024) +[2023-02-23 20:26:45,197][00631] Fps is (10 sec: 3277.3, 60 sec: 3413.3, 300 sec: 3471.2). Total num frames: 3866624. Throughput: 0: 875.6. Samples: 967302. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) +[2023-02-23 20:26:45,204][00631] Avg episode reward: [(0, '21.283')] +[2023-02-23 20:26:50,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3413.6, 300 sec: 3457.3). Total num frames: 3883008. Throughput: 0: 874.4. Samples: 970568. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:26:50,200][00631] Avg episode reward: [(0, '19.885')] +[2023-02-23 20:26:52,110][10898] Updated weights for policy 0, policy_version 950 (0.0032) +[2023-02-23 20:26:55,197][00631] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3429.5). Total num frames: 3895296. Throughput: 0: 818.3. Samples: 974370. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:26:55,200][00631] Avg episode reward: [(0, '19.712')] +[2023-02-23 20:27:00,197][00631] Fps is (10 sec: 2457.6, 60 sec: 3345.1, 300 sec: 3429.6). Total num frames: 3907584. Throughput: 0: 798.8. Samples: 977582. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:27:00,202][00631] Avg episode reward: [(0, '20.625')] +[2023-02-23 20:27:05,197][00631] Fps is (10 sec: 2457.6, 60 sec: 3140.3, 300 sec: 3415.6). Total num frames: 3919872. Throughput: 0: 794.0. Samples: 979356. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:27:05,206][00631] Avg episode reward: [(0, '21.015')] +[2023-02-23 20:27:05,222][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000957_3919872.pth... +[2023-02-23 20:27:05,398][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000758_3104768.pth +[2023-02-23 20:27:07,807][10898] Updated weights for policy 0, policy_version 960 (0.0017) +[2023-02-23 20:27:10,197][00631] Fps is (10 sec: 3276.8, 60 sec: 3208.5, 300 sec: 3415.6). Total num frames: 3940352. Throughput: 0: 789.1. Samples: 984738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:27:10,205][00631] Avg episode reward: [(0, '20.055')] +[2023-02-23 20:27:15,197][00631] Fps is (10 sec: 4096.0, 60 sec: 3276.9, 300 sec: 3415.6). Total num frames: 3960832. Throughput: 0: 790.4. Samples: 991132. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) +[2023-02-23 20:27:15,200][00631] Avg episode reward: [(0, '20.939')] +[2023-02-23 20:27:18,274][10898] Updated weights for policy 0, policy_version 970 (0.0019) +[2023-02-23 20:27:20,197][00631] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3429.5). Total num frames: 3977216. Throughput: 0: 795.6. Samples: 993442. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) +[2023-02-23 20:27:20,201][00631] Avg episode reward: [(0, '21.019')] +[2023-02-23 20:27:25,197][00631] Fps is (10 sec: 2867.1, 60 sec: 3208.5, 300 sec: 3415.6). Total num frames: 3989504. Throughput: 0: 796.6. Samples: 997524. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) +[2023-02-23 20:27:25,210][00631] Avg episode reward: [(0, '21.804')] +[2023-02-23 20:27:28,918][10884] Stopping Batcher_0... +[2023-02-23 20:27:28,919][10884] Loop batcher_evt_loop terminating... +[2023-02-23 20:27:28,920][00631] Component Batcher_0 stopped! +[2023-02-23 20:27:28,925][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2023-02-23 20:27:28,967][10904] Stopping RolloutWorker_w5... +[2023-02-23 20:27:28,968][00631] Component RolloutWorker_w5 stopped! +[2023-02-23 20:27:28,979][10900] Stopping RolloutWorker_w1... +[2023-02-23 20:27:28,969][10904] Loop rollout_proc5_evt_loop terminating... +[2023-02-23 20:27:28,979][00631] Component RolloutWorker_w1 stopped! +[2023-02-23 20:27:28,987][10906] Stopping RolloutWorker_w7... +[2023-02-23 20:27:28,990][10902] Stopping RolloutWorker_w3... +[2023-02-23 20:27:28,988][00631] Component RolloutWorker_w7 stopped! +[2023-02-23 20:27:28,992][00631] Component RolloutWorker_w3 stopped! +[2023-02-23 20:27:28,980][10900] Loop rollout_proc1_evt_loop terminating... +[2023-02-23 20:27:28,988][10906] Loop rollout_proc7_evt_loop terminating... +[2023-02-23 20:27:29,001][10902] Loop rollout_proc3_evt_loop terminating... +[2023-02-23 20:27:29,015][10898] Weights refcount: 2 0 +[2023-02-23 20:27:29,022][00631] Component RolloutWorker_w6 stopped! +[2023-02-23 20:27:29,026][10905] Stopping RolloutWorker_w6... +[2023-02-23 20:27:29,027][10905] Loop rollout_proc6_evt_loop terminating... +[2023-02-23 20:27:29,034][00631] Component InferenceWorker_p0-w0 stopped! +[2023-02-23 20:27:29,034][10898] Stopping InferenceWorker_p0-w0... +[2023-02-23 20:27:29,040][10898] Loop inference_proc0-0_evt_loop terminating... +[2023-02-23 20:27:29,046][00631] Component RolloutWorker_w4 stopped! +[2023-02-23 20:27:29,046][10903] Stopping RolloutWorker_w4... +[2023-02-23 20:27:29,062][10903] Loop rollout_proc4_evt_loop terminating... +[2023-02-23 20:27:29,076][10899] Stopping RolloutWorker_w0... +[2023-02-23 20:27:29,077][10899] Loop rollout_proc0_evt_loop terminating... +[2023-02-23 20:27:29,082][00631] Component RolloutWorker_w0 stopped! +[2023-02-23 20:27:29,087][10901] Stopping RolloutWorker_w2... +[2023-02-23 20:27:29,089][00631] Component RolloutWorker_w2 stopped! +[2023-02-23 20:27:29,088][10901] Loop rollout_proc2_evt_loop terminating... +[2023-02-23 20:27:29,140][10884] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000861_3526656.pth +[2023-02-23 20:27:29,149][10884] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2023-02-23 20:27:29,310][00631] Component LearnerWorker_p0 stopped! +[2023-02-23 20:27:29,310][10884] Stopping LearnerWorker_p0... +[2023-02-23 20:27:29,312][00631] Waiting for process learner_proc0 to stop... +[2023-02-23 20:27:29,312][10884] Loop learner_proc0_evt_loop terminating... +[2023-02-23 20:27:31,206][00631] Waiting for process inference_proc0-0 to join... +[2023-02-23 20:27:31,589][00631] Waiting for process rollout_proc0 to join... +[2023-02-23 20:27:32,015][00631] Waiting for process rollout_proc1 to join... +[2023-02-23 20:27:32,016][00631] Waiting for process rollout_proc2 to join... +[2023-02-23 20:27:32,028][00631] Waiting for process rollout_proc3 to join... +[2023-02-23 20:27:32,029][00631] Waiting for process rollout_proc4 to join... +[2023-02-23 20:27:32,032][00631] Waiting for process rollout_proc5 to join... +[2023-02-23 20:27:32,035][00631] Waiting for process rollout_proc6 to join... +[2023-02-23 20:27:32,038][00631] Waiting for process rollout_proc7 to join... +[2023-02-23 20:27:32,045][00631] Batcher 0 profile tree view: +batching: 27.7991, releasing_batches: 0.0274 +[2023-02-23 20:27:32,052][00631] InferenceWorker_p0-w0 profile tree view: +wait_policy: 0.0001 + wait_policy_total: 571.8816 +update_model: 8.3421 + weight_update: 0.0026 +one_step: 0.0110 + handle_policy_step: 566.0898 + deserialize: 15.8038, stack: 3.1940, obs_to_device_normalize: 121.8924, forward: 277.7904, send_messages: 27.5475 + prepare_outputs: 90.8741 + to_cpu: 56.2697 +[2023-02-23 20:27:32,054][00631] Learner 0 profile tree view: +misc: 0.0064, prepare_batch: 17.4821 +train: 77.7810 + epoch_init: 0.0106, minibatch_init: 0.0064, losses_postprocess: 0.6149, kl_divergence: 0.6251, after_optimizer: 32.6310 + calculate_losses: 27.8452 + losses_init: 0.0039, forward_head: 1.8086, bptt_initial: 18.1649, tail: 1.3877, advantages_returns: 0.2862, losses: 3.4075 + bptt: 2.4140 + bptt_forward_core: 2.3457 + update: 15.4625 + clip: 1.5144 +[2023-02-23 20:27:32,055][00631] RolloutWorker_w0 profile tree view: +wait_for_trajectories: 0.4067, enqueue_policy_requests: 161.8164, env_step: 890.7249, overhead: 24.0875, complete_rollouts: 7.8281 +save_policy_outputs: 22.5890 + split_output_tensors: 11.2162 +[2023-02-23 20:27:32,057][00631] RolloutWorker_w7 profile tree view: +wait_for_trajectories: 0.3422, enqueue_policy_requests: 165.6100, env_step: 886.8571, overhead: 23.7131, complete_rollouts: 7.5647 +save_policy_outputs: 22.4796 + split_output_tensors: 10.8264 +[2023-02-23 20:27:32,058][00631] Loop Runner_EvtLoop terminating... +[2023-02-23 20:27:32,060][00631] Runner profile tree view: +main_loop: 1221.9194 +[2023-02-23 20:27:32,062][00631] Collected {0: 4005888}, FPS: 3278.4 +[2023-02-23 20:27:32,214][00631] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2023-02-23 20:27:32,217][00631] Overriding arg 'num_workers' with value 1 passed from command line +[2023-02-23 20:27:32,221][00631] Adding new argument 'no_render'=True that is not in the saved config file! +[2023-02-23 20:27:32,224][00631] Adding new argument 'save_video'=True that is not in the saved config file! +[2023-02-23 20:27:32,227][00631] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2023-02-23 20:27:32,230][00631] Adding new argument 'video_name'=None that is not in the saved config file! +[2023-02-23 20:27:32,231][00631] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! +[2023-02-23 20:27:32,236][00631] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2023-02-23 20:27:32,238][00631] Adding new argument 'push_to_hub'=False that is not in the saved config file! +[2023-02-23 20:27:32,240][00631] Adding new argument 'hf_repository'=None that is not in the saved config file! +[2023-02-23 20:27:32,244][00631] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2023-02-23 20:27:32,248][00631] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2023-02-23 20:27:32,249][00631] Adding new argument 'train_script'=None that is not in the saved config file! +[2023-02-23 20:27:32,252][00631] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2023-02-23 20:27:32,253][00631] Using frameskip 1 and render_action_repeat=4 for evaluation +[2023-02-23 20:27:32,274][00631] Doom resolution: 160x120, resize resolution: (128, 72) +[2023-02-23 20:27:32,278][00631] RunningMeanStd input shape: (3, 72, 128) +[2023-02-23 20:27:32,282][00631] RunningMeanStd input shape: (1,) +[2023-02-23 20:27:32,300][00631] ConvEncoder: input_channels=3 +[2023-02-23 20:27:32,999][00631] Conv encoder output size: 512 +[2023-02-23 20:27:33,001][00631] Policy head output size: 512 +[2023-02-23 20:27:35,369][00631] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2023-02-23 20:27:36,809][00631] Num frames 100... +[2023-02-23 20:27:36,967][00631] Num frames 200... +[2023-02-23 20:27:37,127][00631] Num frames 300... +[2023-02-23 20:27:37,297][00631] Num frames 400... +[2023-02-23 20:27:37,462][00631] Num frames 500... +[2023-02-23 20:27:37,627][00631] Num frames 600... +[2023-02-23 20:27:37,797][00631] Num frames 700... +[2023-02-23 20:27:37,953][00631] Num frames 800... +[2023-02-23 20:27:38,119][00631] Num frames 900... +[2023-02-23 20:27:38,289][00631] Num frames 1000... +[2023-02-23 20:27:38,387][00631] Avg episode rewards: #0: 20.240, true rewards: #0: 10.240 +[2023-02-23 20:27:38,390][00631] Avg episode reward: 20.240, avg true_objective: 10.240 +[2023-02-23 20:27:38,518][00631] Num frames 1100... +[2023-02-23 20:27:38,680][00631] Num frames 1200... +[2023-02-23 20:27:38,842][00631] Num frames 1300... +[2023-02-23 20:27:38,999][00631] Num frames 1400... +[2023-02-23 20:27:39,158][00631] Num frames 1500... +[2023-02-23 20:27:39,235][00631] Avg episode rewards: #0: 13.560, true rewards: #0: 7.560 +[2023-02-23 20:27:39,238][00631] Avg episode reward: 13.560, avg true_objective: 7.560 +[2023-02-23 20:27:39,410][00631] Num frames 1600... +[2023-02-23 20:27:39,573][00631] Num frames 1700... +[2023-02-23 20:27:39,742][00631] Num frames 1800... +[2023-02-23 20:27:39,904][00631] Num frames 1900... +[2023-02-23 20:27:40,054][00631] Num frames 2000... +[2023-02-23 20:27:40,179][00631] Num frames 2100... +[2023-02-23 20:27:40,339][00631] Avg episode rewards: #0: 12.947, true rewards: #0: 7.280 +[2023-02-23 20:27:40,341][00631] Avg episode reward: 12.947, avg true_objective: 7.280 +[2023-02-23 20:27:40,368][00631] Num frames 2200... +[2023-02-23 20:27:40,491][00631] Num frames 2300... +[2023-02-23 20:27:40,620][00631] Num frames 2400... +[2023-02-23 20:27:40,744][00631] Num frames 2500... +[2023-02-23 20:27:40,870][00631] Num frames 2600... +[2023-02-23 20:27:40,983][00631] Num frames 2700... +[2023-02-23 20:27:41,093][00631] Num frames 2800... +[2023-02-23 20:27:41,217][00631] Num frames 2900... +[2023-02-23 20:27:41,332][00631] Num frames 3000... +[2023-02-23 20:27:41,426][00631] Avg episode rewards: #0: 14.330, true rewards: #0: 7.580 +[2023-02-23 20:27:41,428][00631] Avg episode reward: 14.330, avg true_objective: 7.580 +[2023-02-23 20:27:41,517][00631] Num frames 3100... +[2023-02-23 20:27:41,633][00631] Num frames 3200... +[2023-02-23 20:27:41,750][00631] Num frames 3300... +[2023-02-23 20:27:41,887][00631] Avg episode rewards: #0: 12.930, true rewards: #0: 6.730 +[2023-02-23 20:27:41,888][00631] Avg episode reward: 12.930, avg true_objective: 6.730 +[2023-02-23 20:27:41,937][00631] Num frames 3400... +[2023-02-23 20:27:42,050][00631] Num frames 3500... +[2023-02-23 20:27:42,167][00631] Num frames 3600... +[2023-02-23 20:27:42,290][00631] Num frames 3700... +[2023-02-23 20:27:42,405][00631] Num frames 3800... +[2023-02-23 20:27:42,519][00631] Num frames 3900... +[2023-02-23 20:27:42,633][00631] Num frames 4000... +[2023-02-23 20:27:42,696][00631] Avg episode rewards: #0: 12.842, true rewards: #0: 6.675 +[2023-02-23 20:27:42,698][00631] Avg episode reward: 12.842, avg true_objective: 6.675 +[2023-02-23 20:27:42,813][00631] Num frames 4100... +[2023-02-23 20:27:42,941][00631] Num frames 4200... +[2023-02-23 20:27:43,051][00631] Num frames 4300... +[2023-02-23 20:27:43,160][00631] Num frames 4400... +[2023-02-23 20:27:43,278][00631] Num frames 4500... +[2023-02-23 20:27:43,393][00631] Num frames 4600... +[2023-02-23 20:27:43,509][00631] Num frames 4700... +[2023-02-23 20:27:43,634][00631] Num frames 4800... +[2023-02-23 20:27:43,703][00631] Avg episode rewards: #0: 13.872, true rewards: #0: 6.871 +[2023-02-23 20:27:43,708][00631] Avg episode reward: 13.872, avg true_objective: 6.871 +[2023-02-23 20:27:43,817][00631] Num frames 4900... +[2023-02-23 20:27:43,946][00631] Num frames 5000... +[2023-02-23 20:27:44,058][00631] Num frames 5100... +[2023-02-23 20:27:44,172][00631] Num frames 5200... +[2023-02-23 20:27:44,297][00631] Num frames 5300... +[2023-02-23 20:27:44,417][00631] Num frames 5400... +[2023-02-23 20:27:44,535][00631] Num frames 5500... +[2023-02-23 20:27:44,668][00631] Num frames 5600... +[2023-02-23 20:27:44,794][00631] Num frames 5700... +[2023-02-23 20:27:44,921][00631] Num frames 5800... +[2023-02-23 20:27:45,044][00631] Num frames 5900... +[2023-02-23 20:27:45,160][00631] Num frames 6000... +[2023-02-23 20:27:45,275][00631] Num frames 6100... +[2023-02-23 20:27:45,398][00631] Num frames 6200... +[2023-02-23 20:27:45,515][00631] Num frames 6300... +[2023-02-23 20:27:45,632][00631] Num frames 6400... +[2023-02-23 20:27:45,746][00631] Num frames 6500... +[2023-02-23 20:27:45,864][00631] Num frames 6600... +[2023-02-23 20:27:45,984][00631] Num frames 6700... +[2023-02-23 20:27:46,104][00631] Num frames 6800... +[2023-02-23 20:27:46,228][00631] Avg episode rewards: #0: 19.072, true rewards: #0: 8.572 +[2023-02-23 20:27:46,230][00631] Avg episode reward: 19.072, avg true_objective: 8.572 +[2023-02-23 20:27:46,285][00631] Num frames 6900... +[2023-02-23 20:27:46,414][00631] Num frames 7000... +[2023-02-23 20:27:46,540][00631] Num frames 7100... +[2023-02-23 20:27:46,653][00631] Num frames 7200... +[2023-02-23 20:27:46,765][00631] Num frames 7300... +[2023-02-23 20:27:46,880][00631] Num frames 7400... +[2023-02-23 20:27:47,000][00631] Num frames 7500... +[2023-02-23 20:27:47,116][00631] Num frames 7600... +[2023-02-23 20:27:47,233][00631] Num frames 7700... +[2023-02-23 20:27:47,356][00631] Num frames 7800... +[2023-02-23 20:27:47,471][00631] Num frames 7900... +[2023-02-23 20:27:47,587][00631] Num frames 8000... +[2023-02-23 20:27:47,657][00631] Avg episode rewards: #0: 20.011, true rewards: #0: 8.900 +[2023-02-23 20:27:47,658][00631] Avg episode reward: 20.011, avg true_objective: 8.900 +[2023-02-23 20:27:47,771][00631] Num frames 8100... +[2023-02-23 20:27:47,889][00631] Num frames 8200... +[2023-02-23 20:27:48,005][00631] Num frames 8300... +[2023-02-23 20:27:48,120][00631] Num frames 8400... +[2023-02-23 20:27:48,240][00631] Num frames 8500... +[2023-02-23 20:27:48,356][00631] Num frames 8600... +[2023-02-23 20:27:48,474][00631] Num frames 8700... +[2023-02-23 20:27:48,589][00631] Num frames 8800... +[2023-02-23 20:27:48,707][00631] Num frames 8900... +[2023-02-23 20:27:48,819][00631] Num frames 9000... +[2023-02-23 20:27:48,933][00631] Num frames 9100... +[2023-02-23 20:27:49,103][00631] Avg episode rewards: #0: 20.794, true rewards: #0: 9.194 +[2023-02-23 20:27:49,105][00631] Avg episode reward: 20.794, avg true_objective: 9.194 +[2023-02-23 20:28:50,274][00631] Replay video saved to /content/train_dir/default_experiment/replay.mp4! +[2023-02-23 20:39:39,759][00631] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json +[2023-02-23 20:39:39,763][00631] Overriding arg 'num_workers' with value 1 passed from command line +[2023-02-23 20:39:39,766][00631] Adding new argument 'no_render'=True that is not in the saved config file! +[2023-02-23 20:39:39,767][00631] Adding new argument 'save_video'=True that is not in the saved config file! +[2023-02-23 20:39:39,768][00631] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! +[2023-02-23 20:39:39,770][00631] Adding new argument 'video_name'=None that is not in the saved config file! +[2023-02-23 20:39:39,772][00631] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! +[2023-02-23 20:39:39,778][00631] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! +[2023-02-23 20:39:39,783][00631] Adding new argument 'push_to_hub'=True that is not in the saved config file! +[2023-02-23 20:39:39,784][00631] Adding new argument 'hf_repository'='albertqueralto/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! +[2023-02-23 20:39:39,786][00631] Adding new argument 'policy_index'=0 that is not in the saved config file! +[2023-02-23 20:39:39,787][00631] Adding new argument 'eval_deterministic'=False that is not in the saved config file! +[2023-02-23 20:39:39,788][00631] Adding new argument 'train_script'=None that is not in the saved config file! +[2023-02-23 20:39:39,789][00631] Adding new argument 'enjoy_script'=None that is not in the saved config file! +[2023-02-23 20:39:39,790][00631] Using frameskip 1 and render_action_repeat=4 for evaluation +[2023-02-23 20:39:39,815][00631] RunningMeanStd input shape: (3, 72, 128) +[2023-02-23 20:39:39,820][00631] RunningMeanStd input shape: (1,) +[2023-02-23 20:39:39,833][00631] ConvEncoder: input_channels=3 +[2023-02-23 20:39:39,873][00631] Conv encoder output size: 512 +[2023-02-23 20:39:39,874][00631] Policy head output size: 512 +[2023-02-23 20:39:39,897][00631] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... +[2023-02-23 20:39:40,340][00631] Num frames 100... +[2023-02-23 20:39:40,472][00631] Num frames 200... +[2023-02-23 20:39:40,596][00631] Num frames 300... +[2023-02-23 20:39:40,713][00631] Num frames 400... +[2023-02-23 20:39:40,836][00631] Num frames 500... +[2023-02-23 20:39:40,953][00631] Num frames 600... +[2023-02-23 20:39:41,066][00631] Num frames 700... +[2023-02-23 20:39:41,186][00631] Num frames 800... +[2023-02-23 20:39:41,308][00631] Num frames 900... +[2023-02-23 20:39:41,434][00631] Num frames 1000... +[2023-02-23 20:39:41,566][00631] Num frames 1100... +[2023-02-23 20:39:41,720][00631] Avg episode rewards: #0: 28.840, true rewards: #0: 11.840 +[2023-02-23 20:39:41,722][00631] Avg episode reward: 28.840, avg true_objective: 11.840 +[2023-02-23 20:39:41,744][00631] Num frames 1200... +[2023-02-23 20:39:41,856][00631] Num frames 1300... +[2023-02-23 20:39:41,975][00631] Num frames 1400... +[2023-02-23 20:39:42,097][00631] Num frames 1500... +[2023-02-23 20:39:42,212][00631] Num frames 1600... +[2023-02-23 20:39:42,327][00631] Num frames 1700... +[2023-02-23 20:39:42,441][00631] Num frames 1800... +[2023-02-23 20:39:42,563][00631] Num frames 1900... +[2023-02-23 20:39:42,693][00631] Num frames 2000... +[2023-02-23 20:39:42,812][00631] Num frames 2100... +[2023-02-23 20:39:42,930][00631] Num frames 2200... +[2023-02-23 20:39:43,049][00631] Num frames 2300... +[2023-02-23 20:39:43,162][00631] Num frames 2400... +[2023-02-23 20:39:43,277][00631] Num frames 2500... +[2023-02-23 20:39:43,387][00631] Num frames 2600... +[2023-02-23 20:39:43,505][00631] Num frames 2700... +[2023-02-23 20:39:43,629][00631] Num frames 2800... +[2023-02-23 20:39:43,754][00631] Num frames 2900... +[2023-02-23 20:39:43,877][00631] Num frames 3000... +[2023-02-23 20:39:44,003][00631] Num frames 3100... +[2023-02-23 20:39:44,122][00631] Num frames 3200... +[2023-02-23 20:39:44,278][00631] Avg episode rewards: #0: 42.885, true rewards: #0: 16.385 +[2023-02-23 20:39:44,280][00631] Avg episode reward: 42.885, avg true_objective: 16.385 +[2023-02-23 20:39:44,312][00631] Num frames 3300... +[2023-02-23 20:39:44,443][00631] Num frames 3400... +[2023-02-23 20:39:44,587][00631] Num frames 3500... +[2023-02-23 20:39:44,713][00631] Num frames 3600... +[2023-02-23 20:39:44,838][00631] Num frames 3700... +[2023-02-23 20:39:44,962][00631] Num frames 3800... +[2023-02-23 20:39:45,084][00631] Num frames 3900... +[2023-02-23 20:39:45,204][00631] Num frames 4000... +[2023-02-23 20:39:45,319][00631] Num frames 4100... +[2023-02-23 20:39:45,434][00631] Num frames 4200... +[2023-02-23 20:39:45,551][00631] Num frames 4300... +[2023-02-23 20:39:45,676][00631] Num frames 4400... +[2023-02-23 20:39:45,803][00631] Num frames 4500... +[2023-02-23 20:39:45,855][00631] Avg episode rewards: #0: 37.666, true rewards: #0: 15.000 +[2023-02-23 20:39:45,857][00631] Avg episode reward: 37.666, avg true_objective: 15.000 +[2023-02-23 20:39:45,980][00631] Num frames 4600... +[2023-02-23 20:39:46,099][00631] Num frames 4700... +[2023-02-23 20:39:46,214][00631] Num frames 4800... +[2023-02-23 20:39:46,331][00631] Num frames 4900... +[2023-02-23 20:39:46,463][00631] Num frames 5000... +[2023-02-23 20:39:46,584][00631] Num frames 5100... +[2023-02-23 20:39:46,709][00631] Num frames 5200... +[2023-02-23 20:39:46,827][00631] Num frames 5300... +[2023-02-23 20:39:46,946][00631] Num frames 5400... +[2023-02-23 20:39:47,042][00631] Avg episode rewards: #0: 34.340, true rewards: #0: 13.590 +[2023-02-23 20:39:47,044][00631] Avg episode reward: 34.340, avg true_objective: 13.590 +[2023-02-23 20:39:47,128][00631] Num frames 5500... +[2023-02-23 20:39:47,267][00631] Num frames 5600... +[2023-02-23 20:39:47,399][00631] Num frames 5700... +[2023-02-23 20:39:47,527][00631] Num frames 5800... +[2023-02-23 20:39:47,662][00631] Num frames 5900... +[2023-02-23 20:39:47,787][00631] Num frames 6000... +[2023-02-23 20:39:47,913][00631] Num frames 6100... +[2023-02-23 20:39:48,028][00631] Num frames 6200... +[2023-02-23 20:39:48,145][00631] Num frames 6300... +[2023-02-23 20:39:48,263][00631] Num frames 6400... +[2023-02-23 20:39:48,382][00631] Num frames 6500... +[2023-02-23 20:39:48,501][00631] Avg episode rewards: #0: 32.912, true rewards: #0: 13.112 +[2023-02-23 20:39:48,503][00631] Avg episode reward: 32.912, avg true_objective: 13.112 +[2023-02-23 20:39:48,559][00631] Num frames 6600... +[2023-02-23 20:39:48,693][00631] Num frames 6700... +[2023-02-23 20:39:48,812][00631] Num frames 6800... +[2023-02-23 20:39:48,932][00631] Num frames 6900... +[2023-02-23 20:39:49,075][00631] Num frames 7000... +[2023-02-23 20:39:49,239][00631] Num frames 7100... +[2023-02-23 20:39:49,397][00631] Num frames 7200... +[2023-02-23 20:39:49,560][00631] Num frames 7300... +[2023-02-23 20:39:49,786][00631] Avg episode rewards: #0: 30.825, true rewards: #0: 12.325 +[2023-02-23 20:39:49,788][00631] Avg episode reward: 30.825, avg true_objective: 12.325 +[2023-02-23 20:39:49,803][00631] Num frames 7400... +[2023-02-23 20:39:49,965][00631] Num frames 7500... +[2023-02-23 20:39:50,141][00631] Num frames 7600... +[2023-02-23 20:39:50,299][00631] Num frames 7700... +[2023-02-23 20:39:50,467][00631] Num frames 7800... +[2023-02-23 20:39:50,635][00631] Num frames 7900... +[2023-02-23 20:39:50,810][00631] Num frames 8000... +[2023-02-23 20:39:50,979][00631] Num frames 8100... +[2023-02-23 20:39:51,143][00631] Num frames 8200... +[2023-02-23 20:39:51,307][00631] Num frames 8300... +[2023-02-23 20:39:51,472][00631] Num frames 8400... +[2023-02-23 20:39:51,641][00631] Num frames 8500... +[2023-02-23 20:39:51,812][00631] Num frames 8600... +[2023-02-23 20:39:51,981][00631] Num frames 8700... +[2023-02-23 20:39:52,147][00631] Num frames 8800... +[2023-02-23 20:39:52,318][00631] Num frames 8900... +[2023-02-23 20:39:52,487][00631] Num frames 9000... +[2023-02-23 20:39:52,653][00631] Num frames 9100... +[2023-02-23 20:39:52,770][00631] Num frames 9200... +[2023-02-23 20:39:52,903][00631] Num frames 9300... +[2023-02-23 20:39:53,032][00631] Num frames 9400... +[2023-02-23 20:39:53,199][00631] Avg episode rewards: #0: 34.707, true rewards: #0: 13.564 +[2023-02-23 20:39:53,202][00631] Avg episode reward: 34.707, avg true_objective: 13.564 +[2023-02-23 20:39:53,217][00631] Num frames 9500... +[2023-02-23 20:39:53,346][00631] Num frames 9600... +[2023-02-23 20:39:53,473][00631] Num frames 9700... +[2023-02-23 20:39:53,611][00631] Num frames 9800... +[2023-02-23 20:39:53,737][00631] Num frames 9900... +[2023-02-23 20:39:53,861][00631] Num frames 10000... +[2023-02-23 20:39:54,001][00631] Avg episode rewards: #0: 31.839, true rewards: #0: 12.589 +[2023-02-23 20:39:54,003][00631] Avg episode reward: 31.839, avg true_objective: 12.589 +[2023-02-23 20:39:54,041][00631] Num frames 10100... +[2023-02-23 20:39:54,159][00631] Num frames 10200... +[2023-02-23 20:39:54,274][00631] Num frames 10300... +[2023-02-23 20:39:54,387][00631] Num frames 10400... +[2023-02-23 20:39:54,512][00631] Num frames 10500... +[2023-02-23 20:39:54,626][00631] Num frames 10600... +[2023-02-23 20:39:54,739][00631] Num frames 10700... +[2023-02-23 20:39:54,861][00631] Num frames 10800... +[2023-02-23 20:39:54,984][00631] Num frames 10900... +[2023-02-23 20:39:55,100][00631] Num frames 11000... +[2023-02-23 20:39:55,224][00631] Num frames 11100... +[2023-02-23 20:39:55,290][00631] Avg episode rewards: #0: 31.007, true rewards: #0: 12.340 +[2023-02-23 20:39:55,292][00631] Avg episode reward: 31.007, avg true_objective: 12.340 +[2023-02-23 20:39:55,412][00631] Num frames 11200... +[2023-02-23 20:39:55,531][00631] Num frames 11300... +[2023-02-23 20:39:55,653][00631] Num frames 11400... +[2023-02-23 20:39:55,776][00631] Num frames 11500... +[2023-02-23 20:39:55,907][00631] Num frames 11600... +[2023-02-23 20:39:56,020][00631] Num frames 11700... +[2023-02-23 20:39:56,131][00631] Num frames 11800... +[2023-02-23 20:39:56,245][00631] Num frames 11900... +[2023-02-23 20:39:56,362][00631] Num frames 12000... +[2023-02-23 20:39:56,485][00631] Avg episode rewards: #0: 29.957, true rewards: #0: 12.057 +[2023-02-23 20:39:56,487][00631] Avg episode reward: 29.957, avg true_objective: 12.057 +[2023-02-23 20:41:14,867][00631] Replay video saved to /content/train_dir/default_experiment/replay.mp4!