[2023-02-23 23:09:59,263][02125] Saving configuration to /content/train_dir/default_experiment/config.json... [2023-02-23 23:09:59,269][02125] Rollout worker 0 uses device cpu [2023-02-23 23:09:59,271][02125] Rollout worker 1 uses device cpu [2023-02-23 23:09:59,273][02125] Rollout worker 2 uses device cpu [2023-02-23 23:09:59,275][02125] Rollout worker 3 uses device cpu [2023-02-23 23:09:59,277][02125] Rollout worker 4 uses device cpu [2023-02-23 23:09:59,279][02125] Rollout worker 5 uses device cpu [2023-02-23 23:09:59,281][02125] Rollout worker 6 uses device cpu [2023-02-23 23:09:59,283][02125] Rollout worker 7 uses device cpu [2023-02-23 23:09:59,495][02125] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:09:59,498][02125] InferenceWorker_p0-w0: min num requests: 2 [2023-02-23 23:09:59,540][02125] Starting all processes... [2023-02-23 23:09:59,542][02125] Starting process learner_proc0 [2023-02-23 23:09:59,618][02125] Starting all processes... [2023-02-23 23:09:59,636][02125] Starting process inference_proc0-0 [2023-02-23 23:09:59,637][02125] Starting process rollout_proc0 [2023-02-23 23:09:59,638][02125] Starting process rollout_proc1 [2023-02-23 23:09:59,638][02125] Starting process rollout_proc2 [2023-02-23 23:09:59,638][02125] Starting process rollout_proc3 [2023-02-23 23:09:59,639][02125] Starting process rollout_proc4 [2023-02-23 23:09:59,639][02125] Starting process rollout_proc5 [2023-02-23 23:09:59,639][02125] Starting process rollout_proc6 [2023-02-23 23:09:59,639][02125] Starting process rollout_proc7 [2023-02-23 23:10:10,184][12294] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:10:10,190][12294] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-02-23 23:10:10,458][12309] Worker 0 uses CPU cores [0] [2023-02-23 23:10:10,525][12310] Worker 1 uses CPU cores [1] [2023-02-23 23:10:10,593][12315] Worker 6 uses CPU cores [0] [2023-02-23 23:10:10,667][12313] Worker 5 uses CPU cores [1] [2023-02-23 23:10:10,736][12307] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:10:10,745][12307] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-02-23 23:10:10,790][12316] Worker 7 uses CPU cores [1] [2023-02-23 23:10:10,851][12311] Worker 3 uses CPU cores [1] [2023-02-23 23:10:10,874][12312] Worker 4 uses CPU cores [0] [2023-02-23 23:10:10,973][12314] Worker 2 uses CPU cores [0] [2023-02-23 23:10:11,386][12307] Num visible devices: 1 [2023-02-23 23:10:11,390][12294] Num visible devices: 1 [2023-02-23 23:10:11,403][12294] Starting seed is not provided [2023-02-23 23:10:11,409][12294] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:10:11,409][12294] Initializing actor-critic model on device cuda:0 [2023-02-23 23:10:11,410][12294] RunningMeanStd input shape: (3, 72, 128) [2023-02-23 23:10:11,411][12294] RunningMeanStd input shape: (1,) [2023-02-23 23:10:11,465][12294] ConvEncoder: input_channels=3 [2023-02-23 23:10:12,097][12294] Conv encoder output size: 512 [2023-02-23 23:10:12,098][12294] Policy head output size: 512 [2023-02-23 23:10:12,171][12294] Created Actor Critic model with architecture: [2023-02-23 23:10:12,172][12294] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2023-02-23 23:10:19,486][02125] Heartbeat connected on Batcher_0 [2023-02-23 23:10:19,496][02125] Heartbeat connected on InferenceWorker_p0-w0 [2023-02-23 23:10:19,505][02125] Heartbeat connected on RolloutWorker_w0 [2023-02-23 23:10:19,511][02125] Heartbeat connected on RolloutWorker_w1 [2023-02-23 23:10:19,514][02125] Heartbeat connected on RolloutWorker_w2 [2023-02-23 23:10:19,520][02125] Heartbeat connected on RolloutWorker_w3 [2023-02-23 23:10:19,524][02125] Heartbeat connected on RolloutWorker_w4 [2023-02-23 23:10:19,529][02125] Heartbeat connected on RolloutWorker_w5 [2023-02-23 23:10:19,536][02125] Heartbeat connected on RolloutWorker_w6 [2023-02-23 23:10:19,538][02125] Heartbeat connected on RolloutWorker_w7 [2023-02-23 23:10:20,568][12294] Using optimizer [2023-02-23 23:10:20,569][12294] No checkpoints found [2023-02-23 23:10:20,569][12294] Did not load from checkpoint, starting from scratch! [2023-02-23 23:10:20,570][12294] Initialized policy 0 weights for model version 0 [2023-02-23 23:10:20,573][12294] LearnerWorker_p0 finished initialization! [2023-02-23 23:10:20,577][12294] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:10:20,574][02125] Heartbeat connected on LearnerWorker_p0 [2023-02-23 23:10:20,692][12307] RunningMeanStd input shape: (3, 72, 128) [2023-02-23 23:10:20,693][12307] RunningMeanStd input shape: (1,) [2023-02-23 23:10:20,706][12307] ConvEncoder: input_channels=3 [2023-02-23 23:10:20,809][12307] Conv encoder output size: 512 [2023-02-23 23:10:20,809][12307] Policy head output size: 512 [2023-02-23 23:10:23,035][02125] Inference worker 0-0 is ready! [2023-02-23 23:10:23,036][02125] All inference workers are ready! Signal rollout workers to start! [2023-02-23 23:10:23,170][12315] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:10:23,192][12316] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:10:23,193][12314] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:10:23,190][12309] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:10:23,195][12310] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:10:23,219][12312] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:10:23,226][12313] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:10:23,241][12311] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:10:23,409][12315] VizDoom game.init() threw an exception ViZDoomUnexpectedExitException('Controlled ViZDoom instance exited unexpectedly.'). Terminate process... [2023-02-23 23:10:23,410][12312] VizDoom game.init() threw an exception ViZDoomUnexpectedExitException('Controlled ViZDoom instance exited unexpectedly.'). Terminate process... [2023-02-23 23:10:23,409][12310] VizDoom game.init() threw an exception ViZDoomUnexpectedExitException('Controlled ViZDoom instance exited unexpectedly.'). Terminate process... [2023-02-23 23:10:23,411][12316] VizDoom game.init() threw an exception ViZDoomUnexpectedExitException('Controlled ViZDoom instance exited unexpectedly.'). Terminate process... [2023-02-23 23:10:23,413][12315] EvtLoop [rollout_proc6_evt_loop, process=rollout_proc6] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init self.game.init() vizdoom.vizdoom.ViZDoomUnexpectedExitException: Controlled ViZDoom instance exited unexpectedly. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init env_runner.init(self.timing) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init self._reset() File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset self._ensure_initialized() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized self.initialize() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize self._game_init() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init raise EnvCriticalError() sample_factory.envs.env_utils.EnvCriticalError [2023-02-23 23:10:23,412][12312] EvtLoop [rollout_proc4_evt_loop, process=rollout_proc4] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init self.game.init() vizdoom.vizdoom.ViZDoomUnexpectedExitException: Controlled ViZDoom instance exited unexpectedly. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init env_runner.init(self.timing) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init self._reset() File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset self._ensure_initialized() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized self.initialize() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize self._game_init() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init raise EnvCriticalError() sample_factory.envs.env_utils.EnvCriticalError [2023-02-23 23:10:23,418][12315] Unhandled exception in evt loop rollout_proc6_evt_loop [2023-02-23 23:10:23,420][12312] Unhandled exception in evt loop rollout_proc4_evt_loop [2023-02-23 23:10:23,418][12310] EvtLoop [rollout_proc1_evt_loop, process=rollout_proc1] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init self.game.init() vizdoom.vizdoom.ViZDoomUnexpectedExitException: Controlled ViZDoom instance exited unexpectedly. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init env_runner.init(self.timing) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init self._reset() File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset self._ensure_initialized() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized self.initialize() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize self._game_init() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init raise EnvCriticalError() sample_factory.envs.env_utils.EnvCriticalError [2023-02-23 23:10:23,425][12316] EvtLoop [rollout_proc7_evt_loop, process=rollout_proc7] unhandled exception in slot='init' connected to emitter=Emitter(object_id='Sampler', signal_name='_inference_workers_initialized'), args=() Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 228, in _game_init self.game.init() vizdoom.vizdoom.ViZDoomUnexpectedExitException: Controlled ViZDoom instance exited unexpectedly. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 150, in init env_runner.init(self.timing) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 418, in init self._reset() File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 430, in _reset observations, info = e.reset(seed=seed) # new way of doing seeding since Gym 0.26.0 File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 125, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/algo/utils/make_env.py", line 110, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/scenario_wrappers/gathering_reward_shaping.py", line 30, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 379, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sample_factory/envs/env_wrappers.py", line 84, in reset obs, info = self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/gym/core.py", line 323, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 51, in reset return self.env.reset(**kwargs) File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 323, in reset self._ensure_initialized() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 274, in _ensure_initialized self.initialize() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 269, in initialize self._game_init() File "/usr/local/lib/python3.8/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 244, in _game_init raise EnvCriticalError() sample_factory.envs.env_utils.EnvCriticalError [2023-02-23 23:10:23,434][12316] Unhandled exception in evt loop rollout_proc7_evt_loop [2023-02-23 23:10:23,434][12310] Unhandled exception in evt loop rollout_proc1_evt_loop [2023-02-23 23:10:24,232][02125] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-23 23:10:24,383][12309] Decorrelating experience for 0 frames... [2023-02-23 23:10:24,709][12311] Decorrelating experience for 0 frames... [2023-02-23 23:10:24,815][12309] Decorrelating experience for 32 frames... [2023-02-23 23:10:24,843][12313] Decorrelating experience for 0 frames... [2023-02-23 23:10:25,584][12309] Decorrelating experience for 64 frames... [2023-02-23 23:10:25,603][12311] Decorrelating experience for 32 frames... [2023-02-23 23:10:25,606][12314] Decorrelating experience for 0 frames... [2023-02-23 23:10:25,686][12313] Decorrelating experience for 32 frames... [2023-02-23 23:10:25,984][12314] Decorrelating experience for 32 frames... [2023-02-23 23:10:26,210][12313] Decorrelating experience for 64 frames... [2023-02-23 23:10:26,394][12314] Decorrelating experience for 64 frames... [2023-02-23 23:10:26,474][12311] Decorrelating experience for 64 frames... [2023-02-23 23:10:26,869][12309] Decorrelating experience for 96 frames... [2023-02-23 23:10:27,384][12311] Decorrelating experience for 96 frames... [2023-02-23 23:10:27,704][12313] Decorrelating experience for 96 frames... [2023-02-23 23:10:28,242][12314] Decorrelating experience for 96 frames... [2023-02-23 23:10:29,232][02125] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-23 23:10:34,065][12294] Signal inference workers to stop experience collection... [2023-02-23 23:10:34,079][12307] InferenceWorker_p0-w0: stopping experience collection [2023-02-23 23:10:34,232][02125] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 100.8. Samples: 1008. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-23 23:10:34,238][02125] Avg episode reward: [(0, '2.777')] [2023-02-23 23:10:36,030][12294] Signal inference workers to resume experience collection... [2023-02-23 23:10:36,032][12307] InferenceWorker_p0-w0: resuming experience collection [2023-02-23 23:10:39,232][02125] Fps is (10 sec: 1638.4, 60 sec: 1092.3, 300 sec: 1092.3). Total num frames: 16384. Throughput: 0: 226.9. Samples: 3404. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:10:39,234][02125] Avg episode reward: [(0, '3.700')] [2023-02-23 23:10:44,232][02125] Fps is (10 sec: 3276.7, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 32768. Throughput: 0: 442.9. Samples: 8858. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:10:44,237][02125] Avg episode reward: [(0, '4.112')] [2023-02-23 23:10:46,567][12307] Updated weights for policy 0, policy_version 10 (0.0348) [2023-02-23 23:10:49,235][02125] Fps is (10 sec: 2866.2, 60 sec: 1802.0, 300 sec: 1802.0). Total num frames: 45056. Throughput: 0: 430.6. Samples: 10766. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:10:49,238][02125] Avg episode reward: [(0, '4.379')] [2023-02-23 23:10:54,232][02125] Fps is (10 sec: 3276.8, 60 sec: 2184.5, 300 sec: 2184.5). Total num frames: 65536. Throughput: 0: 522.9. Samples: 15688. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:10:54,235][02125] Avg episode reward: [(0, '4.386')] [2023-02-23 23:10:58,102][12307] Updated weights for policy 0, policy_version 20 (0.0023) [2023-02-23 23:10:59,232][02125] Fps is (10 sec: 4097.4, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 86016. Throughput: 0: 619.8. Samples: 21692. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:10:59,234][02125] Avg episode reward: [(0, '4.349')] [2023-02-23 23:11:04,236][02125] Fps is (10 sec: 3275.6, 60 sec: 2457.4, 300 sec: 2457.4). Total num frames: 98304. Throughput: 0: 603.4. Samples: 24140. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:11:04,238][02125] Avg episode reward: [(0, '4.291')] [2023-02-23 23:11:09,235][02125] Fps is (10 sec: 2456.8, 60 sec: 2457.4, 300 sec: 2457.4). Total num frames: 110592. Throughput: 0: 615.9. Samples: 27718. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:11:09,238][02125] Avg episode reward: [(0, '4.313')] [2023-02-23 23:11:09,247][12294] Saving new best policy, reward=4.313! [2023-02-23 23:11:13,533][12307] Updated weights for policy 0, policy_version 30 (0.0019) [2023-02-23 23:11:14,232][02125] Fps is (10 sec: 2458.5, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 122880. Throughput: 0: 693.1. Samples: 31190. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:11:14,235][02125] Avg episode reward: [(0, '4.294')] [2023-02-23 23:11:19,232][02125] Fps is (10 sec: 3278.0, 60 sec: 2606.6, 300 sec: 2606.6). Total num frames: 143360. Throughput: 0: 735.5. Samples: 34106. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:11:19,235][02125] Avg episode reward: [(0, '4.337')] [2023-02-23 23:11:19,237][12294] Saving new best policy, reward=4.337! [2023-02-23 23:11:24,233][02125] Fps is (10 sec: 3276.5, 60 sec: 2594.1, 300 sec: 2594.1). Total num frames: 155648. Throughput: 0: 795.6. Samples: 39206. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:11:24,241][02125] Avg episode reward: [(0, '4.255')] [2023-02-23 23:11:26,839][12307] Updated weights for policy 0, policy_version 40 (0.0013) [2023-02-23 23:11:29,232][02125] Fps is (10 sec: 2457.6, 60 sec: 2798.9, 300 sec: 2583.6). Total num frames: 167936. Throughput: 0: 752.4. Samples: 42718. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:11:29,239][02125] Avg episode reward: [(0, '4.344')] [2023-02-23 23:11:29,245][12294] Saving new best policy, reward=4.344! [2023-02-23 23:11:34,232][02125] Fps is (10 sec: 3277.1, 60 sec: 3140.3, 300 sec: 2691.7). Total num frames: 188416. Throughput: 0: 775.5. Samples: 45660. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:11:34,239][02125] Avg episode reward: [(0, '4.518')] [2023-02-23 23:11:34,249][12294] Saving new best policy, reward=4.518! [2023-02-23 23:11:37,891][12307] Updated weights for policy 0, policy_version 50 (0.0015) [2023-02-23 23:11:39,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3208.5, 300 sec: 2785.3). Total num frames: 208896. Throughput: 0: 798.0. Samples: 51596. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:11:39,235][02125] Avg episode reward: [(0, '4.627')] [2023-02-23 23:11:39,241][12294] Saving new best policy, reward=4.627! [2023-02-23 23:11:44,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3140.3, 300 sec: 2764.8). Total num frames: 221184. Throughput: 0: 756.9. Samples: 55752. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:11:44,237][02125] Avg episode reward: [(0, '4.544')] [2023-02-23 23:11:49,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3208.7, 300 sec: 2794.9). Total num frames: 237568. Throughput: 0: 748.8. Samples: 57832. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:11:49,234][02125] Avg episode reward: [(0, '4.543')] [2023-02-23 23:11:51,032][12307] Updated weights for policy 0, policy_version 60 (0.0012) [2023-02-23 23:11:54,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3208.5, 300 sec: 2867.2). Total num frames: 258048. Throughput: 0: 802.2. Samples: 63816. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:11:54,238][02125] Avg episode reward: [(0, '4.524')] [2023-02-23 23:11:54,247][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000063_258048.pth... [2023-02-23 23:11:59,234][02125] Fps is (10 sec: 3685.7, 60 sec: 3140.2, 300 sec: 2888.7). Total num frames: 274432. Throughput: 0: 837.6. Samples: 68882. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:11:59,238][02125] Avg episode reward: [(0, '4.653')] [2023-02-23 23:11:59,242][12294] Saving new best policy, reward=4.653! [2023-02-23 23:12:03,874][12307] Updated weights for policy 0, policy_version 70 (0.0015) [2023-02-23 23:12:04,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3140.5, 300 sec: 2867.2). Total num frames: 286720. Throughput: 0: 813.9. Samples: 70732. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:12:04,242][02125] Avg episode reward: [(0, '4.555')] [2023-02-23 23:12:09,232][02125] Fps is (10 sec: 2867.8, 60 sec: 3208.7, 300 sec: 2886.7). Total num frames: 303104. Throughput: 0: 812.9. Samples: 75786. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:12:09,234][02125] Avg episode reward: [(0, '4.592')] [2023-02-23 23:12:14,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 2941.7). Total num frames: 323584. Throughput: 0: 871.2. Samples: 81924. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:12:14,234][02125] Avg episode reward: [(0, '4.661')] [2023-02-23 23:12:14,247][12294] Saving new best policy, reward=4.661! [2023-02-23 23:12:14,462][12307] Updated weights for policy 0, policy_version 80 (0.0012) [2023-02-23 23:12:19,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 2956.2). Total num frames: 339968. Throughput: 0: 856.2. Samples: 84190. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:12:19,234][02125] Avg episode reward: [(0, '4.716')] [2023-02-23 23:12:19,242][12294] Saving new best policy, reward=4.716! [2023-02-23 23:12:24,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 2969.6). Total num frames: 356352. Throughput: 0: 815.4. Samples: 88290. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:12:24,235][02125] Avg episode reward: [(0, '4.567')] [2023-02-23 23:12:27,148][12307] Updated weights for policy 0, policy_version 90 (0.0012) [2023-02-23 23:12:29,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3014.7). Total num frames: 376832. Throughput: 0: 858.9. Samples: 94402. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:12:29,234][02125] Avg episode reward: [(0, '4.319')] [2023-02-23 23:12:34,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3024.7). Total num frames: 393216. Throughput: 0: 878.9. Samples: 97384. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:12:34,234][02125] Avg episode reward: [(0, '4.615')] [2023-02-23 23:12:39,233][02125] Fps is (10 sec: 2866.8, 60 sec: 3276.7, 300 sec: 3003.7). Total num frames: 405504. Throughput: 0: 837.6. Samples: 101510. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:12:39,236][02125] Avg episode reward: [(0, '5.022')] [2023-02-23 23:12:39,239][12294] Saving new best policy, reward=5.022! [2023-02-23 23:12:39,661][12307] Updated weights for policy 0, policy_version 100 (0.0017) [2023-02-23 23:12:44,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3042.7). Total num frames: 425984. Throughput: 0: 843.3. Samples: 106828. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:12:44,240][02125] Avg episode reward: [(0, '5.110')] [2023-02-23 23:12:44,250][12294] Saving new best policy, reward=5.110! [2023-02-23 23:12:49,232][02125] Fps is (10 sec: 4096.6, 60 sec: 3481.6, 300 sec: 3079.1). Total num frames: 446464. Throughput: 0: 870.4. Samples: 109898. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:12:49,234][02125] Avg episode reward: [(0, '4.682')] [2023-02-23 23:12:49,890][12307] Updated weights for policy 0, policy_version 110 (0.0013) [2023-02-23 23:12:54,236][02125] Fps is (10 sec: 3275.3, 60 sec: 3344.8, 300 sec: 3058.3). Total num frames: 458752. Throughput: 0: 870.2. Samples: 114950. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:12:54,238][02125] Avg episode reward: [(0, '4.492')] [2023-02-23 23:12:59,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.2, 300 sec: 3065.4). Total num frames: 475136. Throughput: 0: 825.2. Samples: 119058. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:12:59,239][02125] Avg episode reward: [(0, '4.621')] [2023-02-23 23:13:03,330][12307] Updated weights for policy 0, policy_version 120 (0.0015) [2023-02-23 23:13:04,232][02125] Fps is (10 sec: 3278.2, 60 sec: 3413.3, 300 sec: 3072.0). Total num frames: 491520. Throughput: 0: 839.4. Samples: 121964. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:13:04,234][02125] Avg episode reward: [(0, '4.703')] [2023-02-23 23:13:09,238][02125] Fps is (10 sec: 3684.1, 60 sec: 3481.2, 300 sec: 3102.9). Total num frames: 512000. Throughput: 0: 883.5. Samples: 128054. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:13:09,241][02125] Avg episode reward: [(0, '4.593')] [2023-02-23 23:13:14,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3084.0). Total num frames: 524288. Throughput: 0: 838.2. Samples: 132122. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:13:14,237][02125] Avg episode reward: [(0, '4.551')] [2023-02-23 23:13:16,147][12307] Updated weights for policy 0, policy_version 130 (0.0014) [2023-02-23 23:13:19,232][02125] Fps is (10 sec: 3278.9, 60 sec: 3413.3, 300 sec: 3113.0). Total num frames: 544768. Throughput: 0: 823.7. Samples: 134450. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:13:19,239][02125] Avg episode reward: [(0, '4.684')] [2023-02-23 23:13:24,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3140.3). Total num frames: 565248. Throughput: 0: 870.5. Samples: 140680. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:13:24,240][02125] Avg episode reward: [(0, '5.086')] [2023-02-23 23:13:26,028][12307] Updated weights for policy 0, policy_version 140 (0.0013) [2023-02-23 23:13:29,235][02125] Fps is (10 sec: 3685.2, 60 sec: 3413.1, 300 sec: 3143.9). Total num frames: 581632. Throughput: 0: 863.3. Samples: 145680. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:13:29,238][02125] Avg episode reward: [(0, '5.290')] [2023-02-23 23:13:29,240][12294] Saving new best policy, reward=5.290! [2023-02-23 23:13:34,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3125.9). Total num frames: 593920. Throughput: 0: 837.2. Samples: 147572. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:13:34,239][02125] Avg episode reward: [(0, '5.271')] [2023-02-23 23:13:38,927][12307] Updated weights for policy 0, policy_version 150 (0.0015) [2023-02-23 23:13:39,232][02125] Fps is (10 sec: 3277.9, 60 sec: 3481.7, 300 sec: 3150.8). Total num frames: 614400. Throughput: 0: 844.7. Samples: 152956. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:13:39,237][02125] Avg episode reward: [(0, '5.327')] [2023-02-23 23:13:39,242][12294] Saving new best policy, reward=5.327! [2023-02-23 23:13:44,232][02125] Fps is (10 sec: 4095.8, 60 sec: 3481.6, 300 sec: 3174.4). Total num frames: 634880. Throughput: 0: 887.9. Samples: 159012. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:13:44,238][02125] Avg episode reward: [(0, '5.410')] [2023-02-23 23:13:44,256][12294] Saving new best policy, reward=5.410! [2023-02-23 23:13:49,234][02125] Fps is (10 sec: 3276.2, 60 sec: 3345.0, 300 sec: 3156.9). Total num frames: 647168. Throughput: 0: 866.0. Samples: 160936. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:13:49,240][02125] Avg episode reward: [(0, '5.164')] [2023-02-23 23:13:51,925][12307] Updated weights for policy 0, policy_version 160 (0.0016) [2023-02-23 23:13:54,232][02125] Fps is (10 sec: 2867.3, 60 sec: 3413.6, 300 sec: 3159.8). Total num frames: 663552. Throughput: 0: 826.5. Samples: 165242. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:13:54,239][02125] Avg episode reward: [(0, '4.902')] [2023-02-23 23:13:54,254][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000162_663552.pth... [2023-02-23 23:13:59,232][02125] Fps is (10 sec: 3687.1, 60 sec: 3481.6, 300 sec: 3181.5). Total num frames: 684032. Throughput: 0: 869.2. Samples: 171236. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:13:59,239][02125] Avg episode reward: [(0, '4.942')] [2023-02-23 23:14:02,421][12307] Updated weights for policy 0, policy_version 170 (0.0013) [2023-02-23 23:14:04,234][02125] Fps is (10 sec: 3685.5, 60 sec: 3481.5, 300 sec: 3183.7). Total num frames: 700416. Throughput: 0: 879.4. Samples: 174024. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:04,245][02125] Avg episode reward: [(0, '4.897')] [2023-02-23 23:14:09,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.4, 300 sec: 3167.6). Total num frames: 712704. Throughput: 0: 828.1. Samples: 177944. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:09,241][02125] Avg episode reward: [(0, '5.164')] [2023-02-23 23:14:14,232][02125] Fps is (10 sec: 3277.6, 60 sec: 3481.6, 300 sec: 3187.8). Total num frames: 733184. Throughput: 0: 840.4. Samples: 183496. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:14,234][02125] Avg episode reward: [(0, '5.054')] [2023-02-23 23:14:15,160][12307] Updated weights for policy 0, policy_version 180 (0.0012) [2023-02-23 23:14:19,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3207.1). Total num frames: 753664. Throughput: 0: 865.3. Samples: 186510. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:19,239][02125] Avg episode reward: [(0, '5.379')] [2023-02-23 23:14:24,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3191.5). Total num frames: 765952. Throughput: 0: 854.8. Samples: 191422. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:24,241][02125] Avg episode reward: [(0, '5.300')] [2023-02-23 23:14:27,881][12307] Updated weights for policy 0, policy_version 190 (0.0021) [2023-02-23 23:14:29,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.3, 300 sec: 3193.2). Total num frames: 782336. Throughput: 0: 822.3. Samples: 196016. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:29,235][02125] Avg episode reward: [(0, '4.976')] [2023-02-23 23:14:34,232][02125] Fps is (10 sec: 3686.2, 60 sec: 3481.6, 300 sec: 3211.3). Total num frames: 802816. Throughput: 0: 848.5. Samples: 199118. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:14:34,240][02125] Avg episode reward: [(0, '5.055')] [2023-02-23 23:14:38,506][12307] Updated weights for policy 0, policy_version 200 (0.0013) [2023-02-23 23:14:39,232][02125] Fps is (10 sec: 3686.3, 60 sec: 3413.3, 300 sec: 3212.5). Total num frames: 819200. Throughput: 0: 879.8. Samples: 204834. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:14:39,238][02125] Avg episode reward: [(0, '5.180')] [2023-02-23 23:14:44,232][02125] Fps is (10 sec: 2867.3, 60 sec: 3276.8, 300 sec: 3198.0). Total num frames: 831488. Throughput: 0: 833.4. Samples: 208740. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:44,241][02125] Avg episode reward: [(0, '5.215')] [2023-02-23 23:14:49,232][02125] Fps is (10 sec: 3276.9, 60 sec: 3413.4, 300 sec: 3215.0). Total num frames: 851968. Throughput: 0: 829.4. Samples: 211346. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:49,239][02125] Avg episode reward: [(0, '5.602')] [2023-02-23 23:14:49,244][12294] Saving new best policy, reward=5.602! [2023-02-23 23:14:50,955][12307] Updated weights for policy 0, policy_version 210 (0.0016) [2023-02-23 23:14:54,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3231.3). Total num frames: 872448. Throughput: 0: 877.0. Samples: 217408. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:14:54,240][02125] Avg episode reward: [(0, '5.711')] [2023-02-23 23:14:54,252][12294] Saving new best policy, reward=5.711! [2023-02-23 23:14:59,234][02125] Fps is (10 sec: 3276.0, 60 sec: 3344.9, 300 sec: 3217.2). Total num frames: 884736. Throughput: 0: 845.4. Samples: 221540. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:14:59,238][02125] Avg episode reward: [(0, '5.949')] [2023-02-23 23:14:59,243][12294] Saving new best policy, reward=5.949! [2023-02-23 23:15:04,233][02125] Fps is (10 sec: 2047.7, 60 sec: 3208.6, 300 sec: 3189.0). Total num frames: 892928. Throughput: 0: 812.5. Samples: 223074. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:15:04,237][02125] Avg episode reward: [(0, '6.019')] [2023-02-23 23:15:04,248][12294] Saving new best policy, reward=6.019! [2023-02-23 23:15:06,556][12307] Updated weights for policy 0, policy_version 220 (0.0016) [2023-02-23 23:15:09,232][02125] Fps is (10 sec: 2458.2, 60 sec: 3276.8, 300 sec: 3190.6). Total num frames: 909312. Throughput: 0: 780.6. Samples: 226548. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:15:09,234][02125] Avg episode reward: [(0, '6.219')] [2023-02-23 23:15:09,243][12294] Saving new best policy, reward=6.219! [2023-02-23 23:15:14,232][02125] Fps is (10 sec: 3686.9, 60 sec: 3276.8, 300 sec: 3206.2). Total num frames: 929792. Throughput: 0: 815.0. Samples: 232690. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:15:14,234][02125] Avg episode reward: [(0, '6.452')] [2023-02-23 23:15:14,248][12294] Saving new best policy, reward=6.452! [2023-02-23 23:15:17,047][12307] Updated weights for policy 0, policy_version 230 (0.0012) [2023-02-23 23:15:19,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3208.5, 300 sec: 3207.4). Total num frames: 946176. Throughput: 0: 814.9. Samples: 235790. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:15:19,238][02125] Avg episode reward: [(0, '6.821')] [2023-02-23 23:15:19,240][12294] Saving new best policy, reward=6.821! [2023-02-23 23:15:24,232][02125] Fps is (10 sec: 3276.6, 60 sec: 3276.8, 300 sec: 3262.9). Total num frames: 962560. Throughput: 0: 781.8. Samples: 240016. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:15:24,240][02125] Avg episode reward: [(0, '6.847')] [2023-02-23 23:15:24,254][12294] Saving new best policy, reward=6.847! [2023-02-23 23:15:29,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3318.5). Total num frames: 978944. Throughput: 0: 811.2. Samples: 245242. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:15:29,237][02125] Avg episode reward: [(0, '7.014')] [2023-02-23 23:15:29,240][12294] Saving new best policy, reward=7.014! [2023-02-23 23:15:29,798][12307] Updated weights for policy 0, policy_version 240 (0.0018) [2023-02-23 23:15:34,232][02125] Fps is (10 sec: 3686.6, 60 sec: 3276.8, 300 sec: 3332.3). Total num frames: 999424. Throughput: 0: 820.8. Samples: 248282. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:15:34,234][02125] Avg episode reward: [(0, '6.503')] [2023-02-23 23:15:39,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 3332.3). Total num frames: 1015808. Throughput: 0: 804.0. Samples: 253590. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:15:39,235][02125] Avg episode reward: [(0, '6.513')] [2023-02-23 23:15:41,910][12307] Updated weights for policy 0, policy_version 250 (0.0020) [2023-02-23 23:15:44,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3332.4). Total num frames: 1028096. Throughput: 0: 807.8. Samples: 257888. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:15:44,234][02125] Avg episode reward: [(0, '6.583')] [2023-02-23 23:15:49,232][02125] Fps is (10 sec: 3276.7, 60 sec: 3276.8, 300 sec: 3332.3). Total num frames: 1048576. Throughput: 0: 842.1. Samples: 260966. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:15:49,235][02125] Avg episode reward: [(0, '7.241')] [2023-02-23 23:15:49,239][12294] Saving new best policy, reward=7.241! [2023-02-23 23:15:52,620][12307] Updated weights for policy 0, policy_version 260 (0.0015) [2023-02-23 23:15:54,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3276.8, 300 sec: 3332.3). Total num frames: 1069056. Throughput: 0: 901.3. Samples: 267106. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:15:54,236][02125] Avg episode reward: [(0, '7.960')] [2023-02-23 23:15:54,253][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000261_1069056.pth... [2023-02-23 23:15:54,395][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000063_258048.pth [2023-02-23 23:15:54,411][12294] Saving new best policy, reward=7.960! [2023-02-23 23:15:59,232][02125] Fps is (10 sec: 3276.9, 60 sec: 3276.9, 300 sec: 3332.4). Total num frames: 1081344. Throughput: 0: 853.2. Samples: 271084. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:15:59,237][02125] Avg episode reward: [(0, '8.260')] [2023-02-23 23:15:59,244][12294] Saving new best policy, reward=8.260! [2023-02-23 23:16:04,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.4, 300 sec: 3346.3). Total num frames: 1097728. Throughput: 0: 835.8. Samples: 273400. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:16:04,234][02125] Avg episode reward: [(0, '8.411')] [2023-02-23 23:16:04,290][12294] Saving new best policy, reward=8.411! [2023-02-23 23:16:05,352][12307] Updated weights for policy 0, policy_version 270 (0.0019) [2023-02-23 23:16:09,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3374.0). Total num frames: 1118208. Throughput: 0: 877.1. Samples: 279484. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:16:09,235][02125] Avg episode reward: [(0, '8.140')] [2023-02-23 23:16:14,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3360.1). Total num frames: 1134592. Throughput: 0: 872.8. Samples: 284516. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:16:14,234][02125] Avg episode reward: [(0, '7.580')] [2023-02-23 23:16:17,899][12307] Updated weights for policy 0, policy_version 280 (0.0019) [2023-02-23 23:16:19,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3360.1). Total num frames: 1146880. Throughput: 0: 849.3. Samples: 286502. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:16:19,234][02125] Avg episode reward: [(0, '7.500')] [2023-02-23 23:16:24,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1171456. Throughput: 0: 853.1. Samples: 291978. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:16:24,239][02125] Avg episode reward: [(0, '7.280')] [2023-02-23 23:16:28,195][12307] Updated weights for policy 0, policy_version 290 (0.0013) [2023-02-23 23:16:29,232][02125] Fps is (10 sec: 4505.6, 60 sec: 3549.9, 300 sec: 3401.8). Total num frames: 1191936. Throughput: 0: 895.0. Samples: 298164. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:16:29,239][02125] Avg episode reward: [(0, '7.849')] [2023-02-23 23:16:34,232][02125] Fps is (10 sec: 3276.7, 60 sec: 3413.3, 300 sec: 3374.0). Total num frames: 1204224. Throughput: 0: 871.2. Samples: 300168. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:16:34,235][02125] Avg episode reward: [(0, '8.146')] [2023-02-23 23:16:39,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1220608. Throughput: 0: 831.2. Samples: 304510. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:16:39,239][02125] Avg episode reward: [(0, '8.454')] [2023-02-23 23:16:39,243][12294] Saving new best policy, reward=8.454! [2023-02-23 23:16:41,032][12307] Updated weights for policy 0, policy_version 300 (0.0012) [2023-02-23 23:16:44,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3401.8). Total num frames: 1241088. Throughput: 0: 877.9. Samples: 310588. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:16:44,239][02125] Avg episode reward: [(0, '9.230')] [2023-02-23 23:16:44,249][12294] Saving new best policy, reward=9.230! [2023-02-23 23:16:49,234][02125] Fps is (10 sec: 3685.5, 60 sec: 3481.5, 300 sec: 3387.9). Total num frames: 1257472. Throughput: 0: 893.3. Samples: 313600. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:16:49,237][02125] Avg episode reward: [(0, '9.640')] [2023-02-23 23:16:49,239][12294] Saving new best policy, reward=9.640! [2023-02-23 23:16:53,355][12307] Updated weights for policy 0, policy_version 310 (0.0013) [2023-02-23 23:16:54,232][02125] Fps is (10 sec: 2867.1, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 1269760. Throughput: 0: 845.3. Samples: 317522. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:16:54,239][02125] Avg episode reward: [(0, '9.586')] [2023-02-23 23:16:59,232][02125] Fps is (10 sec: 3277.6, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1290240. Throughput: 0: 851.6. Samples: 322838. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:16:59,235][02125] Avg episode reward: [(0, '9.275')] [2023-02-23 23:17:03,973][12307] Updated weights for policy 0, policy_version 320 (0.0015) [2023-02-23 23:17:04,232][02125] Fps is (10 sec: 4096.1, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 1310720. Throughput: 0: 876.0. Samples: 325920. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:17:04,240][02125] Avg episode reward: [(0, '9.957')] [2023-02-23 23:17:04,253][12294] Saving new best policy, reward=9.957! [2023-02-23 23:17:09,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1323008. Throughput: 0: 866.0. Samples: 330946. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:17:09,234][02125] Avg episode reward: [(0, '9.722')] [2023-02-23 23:17:14,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1339392. Throughput: 0: 823.4. Samples: 335218. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:17:14,234][02125] Avg episode reward: [(0, '10.248')] [2023-02-23 23:17:14,245][12294] Saving new best policy, reward=10.248! [2023-02-23 23:17:16,937][12307] Updated weights for policy 0, policy_version 330 (0.0021) [2023-02-23 23:17:19,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3401.8). Total num frames: 1359872. Throughput: 0: 846.4. Samples: 338258. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:17:19,234][02125] Avg episode reward: [(0, '10.445')] [2023-02-23 23:17:19,238][12294] Saving new best policy, reward=10.445! [2023-02-23 23:17:24,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1376256. Throughput: 0: 887.2. Samples: 344434. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:17:24,243][02125] Avg episode reward: [(0, '10.175')] [2023-02-23 23:17:29,086][12307] Updated weights for policy 0, policy_version 340 (0.0027) [2023-02-23 23:17:29,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 1392640. Throughput: 0: 841.0. Samples: 348434. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:17:29,242][02125] Avg episode reward: [(0, '10.265')] [2023-02-23 23:17:34,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 1409024. Throughput: 0: 828.1. Samples: 350864. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:17:34,234][02125] Avg episode reward: [(0, '10.918')] [2023-02-23 23:17:34,248][12294] Saving new best policy, reward=10.918! [2023-02-23 23:17:39,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1429504. Throughput: 0: 876.8. Samples: 356976. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:17:39,240][02125] Avg episode reward: [(0, '11.276')] [2023-02-23 23:17:39,245][12294] Saving new best policy, reward=11.276! [2023-02-23 23:17:39,687][12307] Updated weights for policy 0, policy_version 350 (0.0013) [2023-02-23 23:17:44,233][02125] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1445888. Throughput: 0: 870.6. Samples: 362014. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:17:44,238][02125] Avg episode reward: [(0, '12.371')] [2023-02-23 23:17:44,256][12294] Saving new best policy, reward=12.371! [2023-02-23 23:17:49,232][02125] Fps is (10 sec: 2867.1, 60 sec: 3345.2, 300 sec: 3387.9). Total num frames: 1458176. Throughput: 0: 844.9. Samples: 363942. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:17:49,241][02125] Avg episode reward: [(0, '12.355')] [2023-02-23 23:17:52,527][12307] Updated weights for policy 0, policy_version 360 (0.0015) [2023-02-23 23:17:54,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1478656. Throughput: 0: 857.4. Samples: 369528. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:17:54,241][02125] Avg episode reward: [(0, '12.240')] [2023-02-23 23:17:54,251][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000361_1478656.pth... [2023-02-23 23:17:54,348][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000162_663552.pth [2023-02-23 23:17:59,235][02125] Fps is (10 sec: 4094.7, 60 sec: 3481.4, 300 sec: 3415.6). Total num frames: 1499136. Throughput: 0: 891.4. Samples: 375336. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:17:59,238][02125] Avg episode reward: [(0, '11.601')] [2023-02-23 23:18:04,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3388.0). Total num frames: 1511424. Throughput: 0: 866.8. Samples: 377264. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:18:04,237][02125] Avg episode reward: [(0, '11.791')] [2023-02-23 23:18:05,038][12307] Updated weights for policy 0, policy_version 370 (0.0016) [2023-02-23 23:18:09,232][02125] Fps is (10 sec: 2868.2, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 1527808. Throughput: 0: 828.1. Samples: 381698. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:18:09,240][02125] Avg episode reward: [(0, '12.060')] [2023-02-23 23:18:14,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1548288. Throughput: 0: 874.6. Samples: 387792. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:18:14,239][02125] Avg episode reward: [(0, '12.378')] [2023-02-23 23:18:14,255][12294] Saving new best policy, reward=12.378! [2023-02-23 23:18:15,670][12307] Updated weights for policy 0, policy_version 380 (0.0013) [2023-02-23 23:18:19,234][02125] Fps is (10 sec: 3685.6, 60 sec: 3413.2, 300 sec: 3387.9). Total num frames: 1564672. Throughput: 0: 880.8. Samples: 390504. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:18:19,237][02125] Avg episode reward: [(0, '12.674')] [2023-02-23 23:18:19,240][12294] Saving new best policy, reward=12.674! [2023-02-23 23:18:24,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 1576960. Throughput: 0: 833.2. Samples: 394468. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:18:24,239][02125] Avg episode reward: [(0, '13.170')] [2023-02-23 23:18:24,249][12294] Saving new best policy, reward=13.170! [2023-02-23 23:18:28,531][12307] Updated weights for policy 0, policy_version 390 (0.0016) [2023-02-23 23:18:29,232][02125] Fps is (10 sec: 3277.4, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 1597440. Throughput: 0: 848.5. Samples: 400198. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:18:29,241][02125] Avg episode reward: [(0, '13.370')] [2023-02-23 23:18:29,244][12294] Saving new best policy, reward=13.370! [2023-02-23 23:18:34,233][02125] Fps is (10 sec: 4095.5, 60 sec: 3481.5, 300 sec: 3401.7). Total num frames: 1617920. Throughput: 0: 872.5. Samples: 403206. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:18:34,235][02125] Avg episode reward: [(0, '12.979')] [2023-02-23 23:18:39,232][02125] Fps is (10 sec: 3276.9, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 1630208. Throughput: 0: 852.7. Samples: 407900. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:18:39,234][02125] Avg episode reward: [(0, '13.458')] [2023-02-23 23:18:39,236][12294] Saving new best policy, reward=13.458! [2023-02-23 23:18:41,141][12307] Updated weights for policy 0, policy_version 400 (0.0015) [2023-02-23 23:18:44,232][02125] Fps is (10 sec: 2867.6, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 1646592. Throughput: 0: 829.4. Samples: 412654. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:18:44,235][02125] Avg episode reward: [(0, '12.285')] [2023-02-23 23:18:49,232][02125] Fps is (10 sec: 3276.6, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 1662976. Throughput: 0: 841.1. Samples: 415116. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:18:49,237][02125] Avg episode reward: [(0, '12.612')] [2023-02-23 23:18:54,232][02125] Fps is (10 sec: 2867.1, 60 sec: 3276.8, 300 sec: 3360.1). Total num frames: 1675264. Throughput: 0: 828.9. Samples: 419000. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:18:54,239][02125] Avg episode reward: [(0, '12.816')] [2023-02-23 23:18:54,818][12307] Updated weights for policy 0, policy_version 410 (0.0019) [2023-02-23 23:18:59,232][02125] Fps is (10 sec: 2457.7, 60 sec: 3140.4, 300 sec: 3346.3). Total num frames: 1687552. Throughput: 0: 781.1. Samples: 422942. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:18:59,239][02125] Avg episode reward: [(0, '12.837')] [2023-02-23 23:19:04,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3374.0). Total num frames: 1708032. Throughput: 0: 781.4. Samples: 425664. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:19:04,237][02125] Avg episode reward: [(0, '13.644')] [2023-02-23 23:19:04,253][12294] Saving new best policy, reward=13.644! [2023-02-23 23:19:06,703][12307] Updated weights for policy 0, policy_version 420 (0.0022) [2023-02-23 23:19:09,232][02125] Fps is (10 sec: 4096.1, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 1728512. Throughput: 0: 830.0. Samples: 431820. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:19:09,237][02125] Avg episode reward: [(0, '15.017')] [2023-02-23 23:19:09,244][12294] Saving new best policy, reward=15.017! [2023-02-23 23:19:14,233][02125] Fps is (10 sec: 3685.9, 60 sec: 3276.7, 300 sec: 3360.1). Total num frames: 1744896. Throughput: 0: 803.4. Samples: 436352. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:19:14,236][02125] Avg episode reward: [(0, '15.642')] [2023-02-23 23:19:14,261][12294] Saving new best policy, reward=15.642! [2023-02-23 23:19:19,232][02125] Fps is (10 sec: 2867.1, 60 sec: 3208.6, 300 sec: 3360.1). Total num frames: 1757184. Throughput: 0: 781.6. Samples: 438378. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:19:19,237][02125] Avg episode reward: [(0, '16.376')] [2023-02-23 23:19:19,242][12294] Saving new best policy, reward=16.376! [2023-02-23 23:19:19,689][12307] Updated weights for policy 0, policy_version 430 (0.0014) [2023-02-23 23:19:24,232][02125] Fps is (10 sec: 3277.3, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 1777664. Throughput: 0: 806.1. Samples: 444176. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:19:24,238][02125] Avg episode reward: [(0, '16.769')] [2023-02-23 23:19:24,253][12294] Saving new best policy, reward=16.769! [2023-02-23 23:19:29,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 1798144. Throughput: 0: 824.6. Samples: 449762. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:19:29,234][02125] Avg episode reward: [(0, '17.576')] [2023-02-23 23:19:29,237][12294] Saving new best policy, reward=17.576! [2023-02-23 23:19:30,813][12307] Updated weights for policy 0, policy_version 440 (0.0013) [2023-02-23 23:19:34,233][02125] Fps is (10 sec: 3276.8, 60 sec: 3208.6, 300 sec: 3360.1). Total num frames: 1810432. Throughput: 0: 810.9. Samples: 451604. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:19:34,236][02125] Avg episode reward: [(0, '18.118')] [2023-02-23 23:19:34,248][12294] Saving new best policy, reward=18.118! [2023-02-23 23:19:39,232][02125] Fps is (10 sec: 2867.3, 60 sec: 3276.8, 300 sec: 3374.0). Total num frames: 1826816. Throughput: 0: 832.3. Samples: 456452. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:19:39,237][02125] Avg episode reward: [(0, '16.470')] [2023-02-23 23:19:42,561][12307] Updated weights for policy 0, policy_version 450 (0.0018) [2023-02-23 23:19:44,232][02125] Fps is (10 sec: 3686.3, 60 sec: 3345.0, 300 sec: 3374.0). Total num frames: 1847296. Throughput: 0: 882.4. Samples: 462650. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:19:44,236][02125] Avg episode reward: [(0, '16.661')] [2023-02-23 23:19:49,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3360.1). Total num frames: 1863680. Throughput: 0: 878.8. Samples: 465208. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:19:49,237][02125] Avg episode reward: [(0, '16.374')] [2023-02-23 23:19:54,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3360.1). Total num frames: 1875968. Throughput: 0: 829.3. Samples: 469138. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:19:54,234][02125] Avg episode reward: [(0, '16.562')] [2023-02-23 23:19:54,249][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000459_1880064.pth... [2023-02-23 23:19:54,352][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000261_1069056.pth [2023-02-23 23:19:55,297][12307] Updated weights for policy 0, policy_version 460 (0.0022) [2023-02-23 23:19:59,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 1896448. Throughput: 0: 861.5. Samples: 475118. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:19:59,233][02125] Avg episode reward: [(0, '16.646')] [2023-02-23 23:20:04,236][02125] Fps is (10 sec: 4094.3, 60 sec: 3481.4, 300 sec: 3415.6). Total num frames: 1916928. Throughput: 0: 884.4. Samples: 478178. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:20:04,239][02125] Avg episode reward: [(0, '18.330')] [2023-02-23 23:20:04,253][12294] Saving new best policy, reward=18.330! [2023-02-23 23:20:06,518][12307] Updated weights for policy 0, policy_version 470 (0.0016) [2023-02-23 23:20:09,235][02125] Fps is (10 sec: 3275.8, 60 sec: 3344.9, 300 sec: 3387.8). Total num frames: 1929216. Throughput: 0: 853.3. Samples: 482578. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:20:09,240][02125] Avg episode reward: [(0, '18.257')] [2023-02-23 23:20:14,232][02125] Fps is (10 sec: 3278.2, 60 sec: 3413.4, 300 sec: 3401.8). Total num frames: 1949696. Throughput: 0: 839.4. Samples: 487536. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:20:14,237][02125] Avg episode reward: [(0, '17.691')] [2023-02-23 23:20:18,229][12307] Updated weights for policy 0, policy_version 480 (0.0016) [2023-02-23 23:20:19,232][02125] Fps is (10 sec: 4097.3, 60 sec: 3549.9, 300 sec: 3415.7). Total num frames: 1970176. Throughput: 0: 867.6. Samples: 490644. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:20:19,237][02125] Avg episode reward: [(0, '19.150')] [2023-02-23 23:20:19,241][12294] Saving new best policy, reward=19.150! [2023-02-23 23:20:24,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 1982464. Throughput: 0: 878.0. Samples: 495962. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:20:24,235][02125] Avg episode reward: [(0, '19.272')] [2023-02-23 23:20:24,254][12294] Saving new best policy, reward=19.272! [2023-02-23 23:20:29,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 1998848. Throughput: 0: 827.7. Samples: 499896. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:20:29,240][02125] Avg episode reward: [(0, '18.989')] [2023-02-23 23:20:31,111][12307] Updated weights for policy 0, policy_version 490 (0.0013) [2023-02-23 23:20:34,232][02125] Fps is (10 sec: 3686.3, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2019328. Throughput: 0: 838.0. Samples: 502916. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:20:34,234][02125] Avg episode reward: [(0, '19.150')] [2023-02-23 23:20:39,234][02125] Fps is (10 sec: 4095.0, 60 sec: 3549.7, 300 sec: 3429.5). Total num frames: 2039808. Throughput: 0: 890.3. Samples: 509202. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:20:39,238][02125] Avg episode reward: [(0, '19.561')] [2023-02-23 23:20:39,245][12294] Saving new best policy, reward=19.561! [2023-02-23 23:20:42,361][12307] Updated weights for policy 0, policy_version 500 (0.0027) [2023-02-23 23:20:44,232][02125] Fps is (10 sec: 3276.9, 60 sec: 3413.4, 300 sec: 3401.8). Total num frames: 2052096. Throughput: 0: 850.0. Samples: 513370. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:20:44,240][02125] Avg episode reward: [(0, '19.268')] [2023-02-23 23:20:49,232][02125] Fps is (10 sec: 2867.9, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2068480. Throughput: 0: 830.0. Samples: 515524. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:20:49,235][02125] Avg episode reward: [(0, '18.548')] [2023-02-23 23:20:53,816][12307] Updated weights for policy 0, policy_version 510 (0.0013) [2023-02-23 23:20:54,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 2088960. Throughput: 0: 868.2. Samples: 521644. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:20:54,234][02125] Avg episode reward: [(0, '19.587')] [2023-02-23 23:20:54,243][12294] Saving new best policy, reward=19.587! [2023-02-23 23:20:59,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 2105344. Throughput: 0: 872.2. Samples: 526786. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:20:59,236][02125] Avg episode reward: [(0, '20.021')] [2023-02-23 23:20:59,238][12294] Saving new best policy, reward=20.021! [2023-02-23 23:21:04,232][02125] Fps is (10 sec: 2867.1, 60 sec: 3345.3, 300 sec: 3387.9). Total num frames: 2117632. Throughput: 0: 844.5. Samples: 528648. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:21:04,236][02125] Avg episode reward: [(0, '20.809')] [2023-02-23 23:21:04,246][12294] Saving new best policy, reward=20.809! [2023-02-23 23:21:06,954][12307] Updated weights for policy 0, policy_version 520 (0.0014) [2023-02-23 23:21:09,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.8, 300 sec: 3401.8). Total num frames: 2138112. Throughput: 0: 840.2. Samples: 533772. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:21:09,238][02125] Avg episode reward: [(0, '19.883')] [2023-02-23 23:21:14,232][02125] Fps is (10 sec: 4096.1, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 2158592. Throughput: 0: 888.6. Samples: 539882. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:21:14,239][02125] Avg episode reward: [(0, '21.067')] [2023-02-23 23:21:14,251][12294] Saving new best policy, reward=21.067! [2023-02-23 23:21:18,558][12307] Updated weights for policy 0, policy_version 530 (0.0023) [2023-02-23 23:21:19,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 2170880. Throughput: 0: 867.0. Samples: 541932. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:21:19,236][02125] Avg episode reward: [(0, '21.166')] [2023-02-23 23:21:19,243][12294] Saving new best policy, reward=21.166! [2023-02-23 23:21:24,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3374.0). Total num frames: 2187264. Throughput: 0: 822.1. Samples: 546196. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:21:24,238][02125] Avg episode reward: [(0, '21.798')] [2023-02-23 23:21:24,249][12294] Saving new best policy, reward=21.798! [2023-02-23 23:21:29,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2207744. Throughput: 0: 865.0. Samples: 552296. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:21:29,240][02125] Avg episode reward: [(0, '21.406')] [2023-02-23 23:21:29,908][12307] Updated weights for policy 0, policy_version 540 (0.0021) [2023-02-23 23:21:34,233][02125] Fps is (10 sec: 3685.9, 60 sec: 3413.3, 300 sec: 3401.7). Total num frames: 2224128. Throughput: 0: 885.6. Samples: 555378. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:21:34,237][02125] Avg episode reward: [(0, '22.429')] [2023-02-23 23:21:34,251][12294] Saving new best policy, reward=22.429! [2023-02-23 23:21:39,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3276.9, 300 sec: 3374.0). Total num frames: 2236416. Throughput: 0: 837.9. Samples: 559348. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:21:39,235][02125] Avg episode reward: [(0, '22.768')] [2023-02-23 23:21:39,245][12294] Saving new best policy, reward=22.768! [2023-02-23 23:21:42,743][12307] Updated weights for policy 0, policy_version 550 (0.0013) [2023-02-23 23:21:44,232][02125] Fps is (10 sec: 3277.2, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2256896. Throughput: 0: 845.2. Samples: 564820. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:21:44,234][02125] Avg episode reward: [(0, '21.207')] [2023-02-23 23:21:49,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3415.7). Total num frames: 2277376. Throughput: 0: 872.1. Samples: 567894. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:21:49,240][02125] Avg episode reward: [(0, '21.510')] [2023-02-23 23:21:54,238][02125] Fps is (10 sec: 3274.7, 60 sec: 3344.7, 300 sec: 3387.8). Total num frames: 2289664. Throughput: 0: 866.2. Samples: 572758. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:21:54,241][02125] Avg episode reward: [(0, '22.609')] [2023-02-23 23:21:54,260][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000559_2289664.pth... [2023-02-23 23:21:54,400][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000361_1478656.pth [2023-02-23 23:21:54,573][12307] Updated weights for policy 0, policy_version 560 (0.0012) [2023-02-23 23:21:59,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 2306048. Throughput: 0: 830.8. Samples: 577270. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:21:59,234][02125] Avg episode reward: [(0, '22.773')] [2023-02-23 23:21:59,241][12294] Saving new best policy, reward=22.773! [2023-02-23 23:22:04,232][02125] Fps is (10 sec: 3688.8, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2326528. Throughput: 0: 852.8. Samples: 580310. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:22:04,239][02125] Avg episode reward: [(0, '23.002')] [2023-02-23 23:22:04,250][12294] Saving new best policy, reward=23.002! [2023-02-23 23:22:05,664][12307] Updated weights for policy 0, policy_version 570 (0.0014) [2023-02-23 23:22:09,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 2342912. Throughput: 0: 889.6. Samples: 586230. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:22:09,233][02125] Avg episode reward: [(0, '22.498')] [2023-02-23 23:22:14,232][02125] Fps is (10 sec: 3276.7, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 2359296. Throughput: 0: 840.4. Samples: 590114. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:22:14,238][02125] Avg episode reward: [(0, '23.808')] [2023-02-23 23:22:14,253][12294] Saving new best policy, reward=23.808! [2023-02-23 23:22:18,423][12307] Updated weights for policy 0, policy_version 580 (0.0018) [2023-02-23 23:22:19,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2375680. Throughput: 0: 831.2. Samples: 592780. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:22:19,239][02125] Avg episode reward: [(0, '23.693')] [2023-02-23 23:22:24,232][02125] Fps is (10 sec: 3686.5, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2396160. Throughput: 0: 879.1. Samples: 598906. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:22:24,238][02125] Avg episode reward: [(0, '23.308')] [2023-02-23 23:22:29,232][02125] Fps is (10 sec: 3686.5, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 2412544. Throughput: 0: 860.9. Samples: 603562. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:22:29,238][02125] Avg episode reward: [(0, '22.099')] [2023-02-23 23:22:30,364][12307] Updated weights for policy 0, policy_version 590 (0.0012) [2023-02-23 23:22:34,234][02125] Fps is (10 sec: 2866.5, 60 sec: 3345.0, 300 sec: 3374.0). Total num frames: 2424832. Throughput: 0: 837.3. Samples: 605574. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:22:34,238][02125] Avg episode reward: [(0, '22.512')] [2023-02-23 23:22:39,232][02125] Fps is (10 sec: 2457.5, 60 sec: 3345.0, 300 sec: 3360.1). Total num frames: 2437120. Throughput: 0: 823.5. Samples: 609810. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:22:39,238][02125] Avg episode reward: [(0, '21.977')] [2023-02-23 23:22:44,232][02125] Fps is (10 sec: 2867.9, 60 sec: 3276.8, 300 sec: 3374.0). Total num frames: 2453504. Throughput: 0: 825.9. Samples: 614436. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:22:44,238][02125] Avg episode reward: [(0, '20.372')] [2023-02-23 23:22:44,506][12307] Updated weights for policy 0, policy_version 600 (0.0020) [2023-02-23 23:22:49,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3208.5, 300 sec: 3360.1). Total num frames: 2469888. Throughput: 0: 802.8. Samples: 616436. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:22:49,235][02125] Avg episode reward: [(0, '20.030')] [2023-02-23 23:22:54,232][02125] Fps is (10 sec: 3276.7, 60 sec: 3277.1, 300 sec: 3346.3). Total num frames: 2486272. Throughput: 0: 781.4. Samples: 621392. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:22:54,235][02125] Avg episode reward: [(0, '20.676')] [2023-02-23 23:22:56,510][12307] Updated weights for policy 0, policy_version 610 (0.0012) [2023-02-23 23:22:59,232][02125] Fps is (10 sec: 3686.5, 60 sec: 3345.1, 300 sec: 3374.0). Total num frames: 2506752. Throughput: 0: 831.7. Samples: 627540. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:22:59,234][02125] Avg episode reward: [(0, '21.236')] [2023-02-23 23:23:04,232][02125] Fps is (10 sec: 3686.5, 60 sec: 3276.8, 300 sec: 3374.0). Total num frames: 2523136. Throughput: 0: 826.0. Samples: 629950. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:23:04,234][02125] Avg episode reward: [(0, '20.964')] [2023-02-23 23:23:09,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3208.5, 300 sec: 3346.2). Total num frames: 2535424. Throughput: 0: 778.9. Samples: 633956. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:23:09,234][02125] Avg episode reward: [(0, '22.917')] [2023-02-23 23:23:09,304][12307] Updated weights for policy 0, policy_version 620 (0.0012) [2023-02-23 23:23:14,242][02125] Fps is (10 sec: 3682.7, 60 sec: 3344.5, 300 sec: 3373.9). Total num frames: 2560000. Throughput: 0: 809.5. Samples: 639998. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:23:14,248][02125] Avg episode reward: [(0, '24.540')] [2023-02-23 23:23:14,261][12294] Saving new best policy, reward=24.540! [2023-02-23 23:23:19,235][02125] Fps is (10 sec: 4094.6, 60 sec: 3344.9, 300 sec: 3387.8). Total num frames: 2576384. Throughput: 0: 831.5. Samples: 642994. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:23:19,237][02125] Avg episode reward: [(0, '24.075')] [2023-02-23 23:23:19,924][12307] Updated weights for policy 0, policy_version 630 (0.0014) [2023-02-23 23:23:24,232][02125] Fps is (10 sec: 2870.0, 60 sec: 3208.5, 300 sec: 3360.1). Total num frames: 2588672. Throughput: 0: 834.1. Samples: 647344. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:23:24,239][02125] Avg episode reward: [(0, '24.417')] [2023-02-23 23:23:29,232][02125] Fps is (10 sec: 3277.9, 60 sec: 3276.8, 300 sec: 3360.1). Total num frames: 2609152. Throughput: 0: 844.8. Samples: 652452. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:23:29,240][02125] Avg episode reward: [(0, '23.049')] [2023-02-23 23:23:32,233][12307] Updated weights for policy 0, policy_version 640 (0.0013) [2023-02-23 23:23:34,232][02125] Fps is (10 sec: 4096.1, 60 sec: 3413.5, 300 sec: 3387.9). Total num frames: 2629632. Throughput: 0: 867.7. Samples: 655482. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:23:34,238][02125] Avg episode reward: [(0, '22.905')] [2023-02-23 23:23:39,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.4, 300 sec: 3374.0). Total num frames: 2641920. Throughput: 0: 876.9. Samples: 660854. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:23:39,238][02125] Avg episode reward: [(0, '21.270')] [2023-02-23 23:23:44,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3374.0). Total num frames: 2658304. Throughput: 0: 829.9. Samples: 664886. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:23:44,238][02125] Avg episode reward: [(0, '20.070')] [2023-02-23 23:23:44,886][12307] Updated weights for policy 0, policy_version 650 (0.0015) [2023-02-23 23:23:49,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2678784. Throughput: 0: 845.4. Samples: 667994. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:23:49,235][02125] Avg episode reward: [(0, '18.390')] [2023-02-23 23:23:54,232][02125] Fps is (10 sec: 4095.9, 60 sec: 3549.9, 300 sec: 3429.5). Total num frames: 2699264. Throughput: 0: 894.3. Samples: 674198. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:23:54,234][02125] Avg episode reward: [(0, '18.753')] [2023-02-23 23:23:54,248][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000659_2699264.pth... [2023-02-23 23:23:54,368][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000459_1880064.pth [2023-02-23 23:23:55,721][12307] Updated weights for policy 0, policy_version 660 (0.0013) [2023-02-23 23:23:59,232][02125] Fps is (10 sec: 3276.6, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 2711552. Throughput: 0: 849.9. Samples: 678236. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:23:59,235][02125] Avg episode reward: [(0, '17.959')] [2023-02-23 23:24:04,232][02125] Fps is (10 sec: 2867.3, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2727936. Throughput: 0: 834.1. Samples: 680526. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:24:04,238][02125] Avg episode reward: [(0, '16.792')] [2023-02-23 23:24:07,615][12307] Updated weights for policy 0, policy_version 670 (0.0012) [2023-02-23 23:24:09,232][02125] Fps is (10 sec: 3686.6, 60 sec: 3549.9, 300 sec: 3401.8). Total num frames: 2748416. Throughput: 0: 876.2. Samples: 686774. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:24:09,234][02125] Avg episode reward: [(0, '17.111')] [2023-02-23 23:24:14,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3413.9, 300 sec: 3415.7). Total num frames: 2764800. Throughput: 0: 876.9. Samples: 691912. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:24:14,240][02125] Avg episode reward: [(0, '17.979')] [2023-02-23 23:24:19,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.3, 300 sec: 3387.9). Total num frames: 2777088. Throughput: 0: 853.0. Samples: 693868. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:24:19,237][02125] Avg episode reward: [(0, '17.519')] [2023-02-23 23:24:20,412][12307] Updated weights for policy 0, policy_version 680 (0.0027) [2023-02-23 23:24:24,232][02125] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3387.9). Total num frames: 2797568. Throughput: 0: 855.7. Samples: 699362. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:24:24,236][02125] Avg episode reward: [(0, '18.511')] [2023-02-23 23:24:29,232][02125] Fps is (10 sec: 4095.9, 60 sec: 3481.6, 300 sec: 3415.6). Total num frames: 2818048. Throughput: 0: 902.1. Samples: 705480. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:24:29,238][02125] Avg episode reward: [(0, '19.899')] [2023-02-23 23:24:30,942][12307] Updated weights for policy 0, policy_version 690 (0.0013) [2023-02-23 23:24:34,232][02125] Fps is (10 sec: 3686.5, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 2834432. Throughput: 0: 877.1. Samples: 707464. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:24:34,234][02125] Avg episode reward: [(0, '20.034')] [2023-02-23 23:24:39,232][02125] Fps is (10 sec: 3276.9, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2850816. Throughput: 0: 838.5. Samples: 711932. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:24:39,235][02125] Avg episode reward: [(0, '21.932')] [2023-02-23 23:24:43,190][12307] Updated weights for policy 0, policy_version 700 (0.0014) [2023-02-23 23:24:44,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 2871296. Throughput: 0: 883.7. Samples: 718002. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:24:44,240][02125] Avg episode reward: [(0, '22.204')] [2023-02-23 23:24:49,232][02125] Fps is (10 sec: 3686.2, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 2887680. Throughput: 0: 897.7. Samples: 720922. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:24:49,237][02125] Avg episode reward: [(0, '23.818')] [2023-02-23 23:24:54,233][02125] Fps is (10 sec: 2866.9, 60 sec: 3345.0, 300 sec: 3401.8). Total num frames: 2899968. Throughput: 0: 844.1. Samples: 724758. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:24:54,237][02125] Avg episode reward: [(0, '24.055')] [2023-02-23 23:24:56,181][12307] Updated weights for policy 0, policy_version 710 (0.0012) [2023-02-23 23:24:59,232][02125] Fps is (10 sec: 3277.0, 60 sec: 3481.6, 300 sec: 3401.8). Total num frames: 2920448. Throughput: 0: 853.3. Samples: 730310. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:24:59,234][02125] Avg episode reward: [(0, '24.410')] [2023-02-23 23:25:04,232][02125] Fps is (10 sec: 4096.4, 60 sec: 3549.9, 300 sec: 3429.6). Total num frames: 2940928. Throughput: 0: 878.3. Samples: 733390. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:25:04,235][02125] Avg episode reward: [(0, '22.878')] [2023-02-23 23:25:06,965][12307] Updated weights for policy 0, policy_version 720 (0.0013) [2023-02-23 23:25:09,234][02125] Fps is (10 sec: 3276.0, 60 sec: 3413.2, 300 sec: 3401.7). Total num frames: 2953216. Throughput: 0: 866.4. Samples: 738354. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:25:09,237][02125] Avg episode reward: [(0, '22.196')] [2023-02-23 23:25:14,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 2969600. Throughput: 0: 832.0. Samples: 742922. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:25:14,240][02125] Avg episode reward: [(0, '21.545')] [2023-02-23 23:25:18,884][12307] Updated weights for policy 0, policy_version 730 (0.0015) [2023-02-23 23:25:19,232][02125] Fps is (10 sec: 3687.3, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 2990080. Throughput: 0: 854.9. Samples: 745934. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:25:19,234][02125] Avg episode reward: [(0, '20.965')] [2023-02-23 23:25:24,238][02125] Fps is (10 sec: 3684.0, 60 sec: 3481.2, 300 sec: 3415.6). Total num frames: 3006464. Throughput: 0: 888.2. Samples: 751908. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:25:24,241][02125] Avg episode reward: [(0, '20.524')] [2023-02-23 23:25:29,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 3022848. Throughput: 0: 843.7. Samples: 755970. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:25:29,235][02125] Avg episode reward: [(0, '20.774')] [2023-02-23 23:25:31,340][12307] Updated weights for policy 0, policy_version 740 (0.0017) [2023-02-23 23:25:34,232][02125] Fps is (10 sec: 3278.9, 60 sec: 3413.3, 300 sec: 3387.9). Total num frames: 3039232. Throughput: 0: 839.6. Samples: 758702. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:25:34,234][02125] Avg episode reward: [(0, '22.174')] [2023-02-23 23:25:39,239][02125] Fps is (10 sec: 4092.9, 60 sec: 3549.4, 300 sec: 3429.4). Total num frames: 3063808. Throughput: 0: 893.9. Samples: 764988. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:25:39,243][02125] Avg episode reward: [(0, '21.285')] [2023-02-23 23:25:41,912][12307] Updated weights for policy 0, policy_version 750 (0.0012) [2023-02-23 23:25:44,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 3076096. Throughput: 0: 877.6. Samples: 769800. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:25:44,243][02125] Avg episode reward: [(0, '20.854')] [2023-02-23 23:25:49,232][02125] Fps is (10 sec: 2869.4, 60 sec: 3413.4, 300 sec: 3401.8). Total num frames: 3092480. Throughput: 0: 853.2. Samples: 771784. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:25:49,234][02125] Avg episode reward: [(0, '20.146')] [2023-02-23 23:25:54,016][12307] Updated weights for policy 0, policy_version 760 (0.0014) [2023-02-23 23:25:54,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3415.6). Total num frames: 3112960. Throughput: 0: 870.8. Samples: 777540. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:25:54,240][02125] Avg episode reward: [(0, '19.959')] [2023-02-23 23:25:54,249][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000760_3112960.pth... [2023-02-23 23:25:54,348][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000559_2289664.pth [2023-02-23 23:25:59,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 3129344. Throughput: 0: 896.2. Samples: 783250. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:25:59,234][02125] Avg episode reward: [(0, '19.234')] [2023-02-23 23:26:04,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 3141632. Throughput: 0: 872.6. Samples: 785200. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:26:04,239][02125] Avg episode reward: [(0, '20.507')] [2023-02-23 23:26:06,957][12307] Updated weights for policy 0, policy_version 770 (0.0013) [2023-02-23 23:26:09,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.7, 300 sec: 3401.8). Total num frames: 3162112. Throughput: 0: 843.4. Samples: 789856. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:26:09,237][02125] Avg episode reward: [(0, '20.662')] [2023-02-23 23:26:14,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3429.5). Total num frames: 3182592. Throughput: 0: 890.4. Samples: 796040. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:26:14,241][02125] Avg episode reward: [(0, '20.589')] [2023-02-23 23:26:17,578][12307] Updated weights for policy 0, policy_version 780 (0.0013) [2023-02-23 23:26:19,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 3198976. Throughput: 0: 888.8. Samples: 798700. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:26:19,243][02125] Avg episode reward: [(0, '20.399')] [2023-02-23 23:26:24,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.7, 300 sec: 3401.8). Total num frames: 3211264. Throughput: 0: 833.8. Samples: 802502. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:26:24,235][02125] Avg episode reward: [(0, '21.578')] [2023-02-23 23:26:29,232][02125] Fps is (10 sec: 2457.5, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 3223552. Throughput: 0: 808.9. Samples: 806202. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:26:29,238][02125] Avg episode reward: [(0, '21.539')] [2023-02-23 23:26:32,365][12307] Updated weights for policy 0, policy_version 790 (0.0029) [2023-02-23 23:26:34,232][02125] Fps is (10 sec: 2867.1, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 3239936. Throughput: 0: 823.3. Samples: 808834. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:26:34,238][02125] Avg episode reward: [(0, '20.598')] [2023-02-23 23:26:39,234][02125] Fps is (10 sec: 3276.2, 60 sec: 3208.8, 300 sec: 3387.9). Total num frames: 3256320. Throughput: 0: 809.0. Samples: 813948. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:26:39,237][02125] Avg episode reward: [(0, '21.693')] [2023-02-23 23:26:44,232][02125] Fps is (10 sec: 3276.9, 60 sec: 3276.8, 300 sec: 3374.0). Total num frames: 3272704. Throughput: 0: 783.9. Samples: 818524. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:26:44,234][02125] Avg episode reward: [(0, '20.559')] [2023-02-23 23:26:44,953][12307] Updated weights for policy 0, policy_version 800 (0.0014) [2023-02-23 23:26:49,232][02125] Fps is (10 sec: 3687.2, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 3293184. Throughput: 0: 810.6. Samples: 821676. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:26:49,235][02125] Avg episode reward: [(0, '22.017')] [2023-02-23 23:26:54,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 3401.8). Total num frames: 3309568. Throughput: 0: 839.3. Samples: 827626. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:26:54,239][02125] Avg episode reward: [(0, '21.471')] [2023-02-23 23:26:56,185][12307] Updated weights for policy 0, policy_version 810 (0.0012) [2023-02-23 23:26:59,233][02125] Fps is (10 sec: 2867.0, 60 sec: 3208.5, 300 sec: 3374.0). Total num frames: 3321856. Throughput: 0: 790.9. Samples: 831630. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:26:59,240][02125] Avg episode reward: [(0, '22.321')] [2023-02-23 23:27:04,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3387.9). Total num frames: 3342336. Throughput: 0: 789.2. Samples: 834212. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:27:04,234][02125] Avg episode reward: [(0, '21.425')] [2023-02-23 23:27:07,465][12307] Updated weights for policy 0, policy_version 820 (0.0027) [2023-02-23 23:27:09,232][02125] Fps is (10 sec: 4096.3, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 3362816. Throughput: 0: 843.7. Samples: 840468. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:27:09,234][02125] Avg episode reward: [(0, '21.902')] [2023-02-23 23:27:14,234][02125] Fps is (10 sec: 3685.6, 60 sec: 3276.7, 300 sec: 3401.7). Total num frames: 3379200. Throughput: 0: 870.7. Samples: 845384. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:27:14,238][02125] Avg episode reward: [(0, '21.736')] [2023-02-23 23:27:19,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3208.5, 300 sec: 3374.0). Total num frames: 3391488. Throughput: 0: 857.5. Samples: 847420. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:27:19,236][02125] Avg episode reward: [(0, '22.345')] [2023-02-23 23:27:20,219][12307] Updated weights for policy 0, policy_version 830 (0.0015) [2023-02-23 23:27:24,232][02125] Fps is (10 sec: 3687.2, 60 sec: 3413.3, 300 sec: 3401.8). Total num frames: 3416064. Throughput: 0: 869.1. Samples: 853056. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:27:24,234][02125] Avg episode reward: [(0, '21.375')] [2023-02-23 23:27:29,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3415.7). Total num frames: 3432448. Throughput: 0: 896.4. Samples: 858860. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:27:29,234][02125] Avg episode reward: [(0, '21.328')] [2023-02-23 23:27:31,365][12307] Updated weights for policy 0, policy_version 840 (0.0020) [2023-02-23 23:27:34,234][02125] Fps is (10 sec: 2866.6, 60 sec: 3413.2, 300 sec: 3415.6). Total num frames: 3444736. Throughput: 0: 868.8. Samples: 860776. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:27:34,243][02125] Avg episode reward: [(0, '20.618')] [2023-02-23 23:27:39,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.7, 300 sec: 3429.5). Total num frames: 3465216. Throughput: 0: 840.4. Samples: 865446. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:27:39,240][02125] Avg episode reward: [(0, '21.324')] [2023-02-23 23:27:43,092][12307] Updated weights for policy 0, policy_version 850 (0.0013) [2023-02-23 23:27:44,232][02125] Fps is (10 sec: 4096.9, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 3485696. Throughput: 0: 887.9. Samples: 871586. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:27:44,235][02125] Avg episode reward: [(0, '21.030')] [2023-02-23 23:27:49,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3429.5). Total num frames: 3497984. Throughput: 0: 893.0. Samples: 874398. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:27:49,239][02125] Avg episode reward: [(0, '20.966')] [2023-02-23 23:27:54,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 3514368. Throughput: 0: 841.2. Samples: 878320. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:27:54,239][02125] Avg episode reward: [(0, '21.134')] [2023-02-23 23:27:54,254][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000858_3514368.pth... [2023-02-23 23:27:54,351][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000659_2699264.pth [2023-02-23 23:27:56,062][12307] Updated weights for policy 0, policy_version 860 (0.0014) [2023-02-23 23:27:59,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3429.5). Total num frames: 3534848. Throughput: 0: 859.5. Samples: 884058. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:27:59,240][02125] Avg episode reward: [(0, '22.000')] [2023-02-23 23:28:04,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3457.3). Total num frames: 3555328. Throughput: 0: 883.9. Samples: 887196. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:28:04,239][02125] Avg episode reward: [(0, '22.400')] [2023-02-23 23:28:06,853][12307] Updated weights for policy 0, policy_version 870 (0.0014) [2023-02-23 23:28:09,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3415.8). Total num frames: 3567616. Throughput: 0: 866.3. Samples: 892038. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:28:09,235][02125] Avg episode reward: [(0, '22.056')] [2023-02-23 23:28:14,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.5, 300 sec: 3415.7). Total num frames: 3584000. Throughput: 0: 844.5. Samples: 896862. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:28:14,241][02125] Avg episode reward: [(0, '23.198')] [2023-02-23 23:28:18,588][12307] Updated weights for policy 0, policy_version 880 (0.0013) [2023-02-23 23:28:19,232][02125] Fps is (10 sec: 3686.3, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 3604480. Throughput: 0: 870.8. Samples: 899962. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:28:19,235][02125] Avg episode reward: [(0, '23.704')] [2023-02-23 23:28:24,232][02125] Fps is (10 sec: 3686.3, 60 sec: 3413.3, 300 sec: 3429.5). Total num frames: 3620864. Throughput: 0: 892.0. Samples: 905584. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:28:24,237][02125] Avg episode reward: [(0, '23.637')] [2023-02-23 23:28:29,232][02125] Fps is (10 sec: 3276.9, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 3637248. Throughput: 0: 846.2. Samples: 909664. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:28:29,235][02125] Avg episode reward: [(0, '24.271')] [2023-02-23 23:28:31,169][12307] Updated weights for policy 0, policy_version 890 (0.0035) [2023-02-23 23:28:34,232][02125] Fps is (10 sec: 3686.5, 60 sec: 3550.0, 300 sec: 3443.4). Total num frames: 3657728. Throughput: 0: 847.6. Samples: 912540. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:28:34,238][02125] Avg episode reward: [(0, '23.296')] [2023-02-23 23:28:39,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3457.3). Total num frames: 3678208. Throughput: 0: 899.1. Samples: 918778. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:28:39,235][02125] Avg episode reward: [(0, '24.645')] [2023-02-23 23:28:39,237][12294] Saving new best policy, reward=24.645! [2023-02-23 23:28:41,998][12307] Updated weights for policy 0, policy_version 900 (0.0017) [2023-02-23 23:28:44,234][02125] Fps is (10 sec: 3276.0, 60 sec: 3413.2, 300 sec: 3429.5). Total num frames: 3690496. Throughput: 0: 870.6. Samples: 923236. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:28:44,243][02125] Avg episode reward: [(0, '23.206')] [2023-02-23 23:28:49,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3415.7). Total num frames: 3706880. Throughput: 0: 844.1. Samples: 925180. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:28:49,234][02125] Avg episode reward: [(0, '23.188')] [2023-02-23 23:28:54,035][12307] Updated weights for policy 0, policy_version 910 (0.0024) [2023-02-23 23:28:54,232][02125] Fps is (10 sec: 3687.3, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 3727360. Throughput: 0: 868.5. Samples: 931120. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:28:54,239][02125] Avg episode reward: [(0, '23.550')] [2023-02-23 23:28:59,235][02125] Fps is (10 sec: 3685.1, 60 sec: 3481.4, 300 sec: 3443.4). Total num frames: 3743744. Throughput: 0: 883.2. Samples: 936608. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:28:59,241][02125] Avg episode reward: [(0, '23.164')] [2023-02-23 23:29:04,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 3415.6). Total num frames: 3756032. Throughput: 0: 858.4. Samples: 938590. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:29:04,235][02125] Avg episode reward: [(0, '24.841')] [2023-02-23 23:29:04,253][12294] Saving new best policy, reward=24.841! [2023-02-23 23:29:06,781][12307] Updated weights for policy 0, policy_version 920 (0.0013) [2023-02-23 23:29:09,232][02125] Fps is (10 sec: 3277.9, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 3776512. Throughput: 0: 845.8. Samples: 943644. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:29:09,237][02125] Avg episode reward: [(0, '23.043')] [2023-02-23 23:29:14,232][02125] Fps is (10 sec: 4096.1, 60 sec: 3549.9, 300 sec: 3457.3). Total num frames: 3796992. Throughput: 0: 895.2. Samples: 949946. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:29:14,234][02125] Avg episode reward: [(0, '23.168')] [2023-02-23 23:29:17,248][12307] Updated weights for policy 0, policy_version 930 (0.0012) [2023-02-23 23:29:19,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3443.4). Total num frames: 3813376. Throughput: 0: 885.5. Samples: 952388. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:29:19,234][02125] Avg episode reward: [(0, '21.697')] [2023-02-23 23:29:24,232][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3415.6). Total num frames: 3825664. Throughput: 0: 835.5. Samples: 956374. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:29:24,235][02125] Avg episode reward: [(0, '22.752')] [2023-02-23 23:29:29,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3429.5). Total num frames: 3846144. Throughput: 0: 874.2. Samples: 962572. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:29:29,234][02125] Avg episode reward: [(0, '21.589')] [2023-02-23 23:29:29,299][12307] Updated weights for policy 0, policy_version 940 (0.0020) [2023-02-23 23:29:34,235][02125] Fps is (10 sec: 4094.6, 60 sec: 3481.4, 300 sec: 3443.4). Total num frames: 3866624. Throughput: 0: 900.4. Samples: 965700. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:29:34,248][02125] Avg episode reward: [(0, '21.163')] [2023-02-23 23:29:39,234][02125] Fps is (10 sec: 3276.0, 60 sec: 3344.9, 300 sec: 3415.6). Total num frames: 3878912. Throughput: 0: 868.0. Samples: 970180. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:29:39,240][02125] Avg episode reward: [(0, '21.327')] [2023-02-23 23:29:41,848][12307] Updated weights for policy 0, policy_version 950 (0.0014) [2023-02-23 23:29:44,232][02125] Fps is (10 sec: 3277.9, 60 sec: 3481.7, 300 sec: 3429.5). Total num frames: 3899392. Throughput: 0: 860.9. Samples: 975346. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:29:44,235][02125] Avg episode reward: [(0, '21.380')] [2023-02-23 23:29:49,234][02125] Fps is (10 sec: 4096.0, 60 sec: 3549.7, 300 sec: 3457.3). Total num frames: 3919872. Throughput: 0: 886.2. Samples: 978472. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:29:49,236][02125] Avg episode reward: [(0, '21.547')] [2023-02-23 23:29:51,893][12307] Updated weights for policy 0, policy_version 960 (0.0012) [2023-02-23 23:29:54,232][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3443.4). Total num frames: 3936256. Throughput: 0: 897.5. Samples: 984030. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:29:54,237][02125] Avg episode reward: [(0, '21.660')] [2023-02-23 23:29:54,249][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000961_3936256.pth... [2023-02-23 23:29:54,248][02125] No heartbeat for components: RolloutWorker_w1 (1174 seconds), RolloutWorker_w4 (1174 seconds), RolloutWorker_w6 (1174 seconds), RolloutWorker_w7 (1174 seconds) [2023-02-23 23:29:54,389][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000760_3112960.pth [2023-02-23 23:29:59,232][02125] Fps is (10 sec: 2867.9, 60 sec: 3413.5, 300 sec: 3415.6). Total num frames: 3948544. Throughput: 0: 844.4. Samples: 987942. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) [2023-02-23 23:29:59,235][02125] Avg episode reward: [(0, '21.373')] [2023-02-23 23:30:04,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 3969024. Throughput: 0: 859.1. Samples: 991048. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:30:04,239][02125] Avg episode reward: [(0, '20.815')] [2023-02-23 23:30:04,467][12307] Updated weights for policy 0, policy_version 970 (0.0015) [2023-02-23 23:30:09,232][02125] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3457.3). Total num frames: 3989504. Throughput: 0: 908.9. Samples: 997276. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:30:09,235][02125] Avg episode reward: [(0, '21.241')] [2023-02-23 23:30:14,232][02125] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3429.5). Total num frames: 4001792. Throughput: 0: 853.4. Samples: 1000974. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) [2023-02-23 23:30:14,234][02125] Avg episode reward: [(0, '21.625')] [2023-02-23 23:30:15,295][12294] Stopping Batcher_0... [2023-02-23 23:30:15,296][12294] Loop batcher_evt_loop terminating... [2023-02-23 23:30:15,299][02125] Component Batcher_0 stopped! [2023-02-23 23:30:15,304][02125] Component RolloutWorker_w1 process died already! Don't wait for it. [2023-02-23 23:30:15,307][02125] Component RolloutWorker_w4 process died already! Don't wait for it. [2023-02-23 23:30:15,308][02125] Component RolloutWorker_w6 process died already! Don't wait for it. [2023-02-23 23:30:15,309][02125] Component RolloutWorker_w7 process died already! Don't wait for it. [2023-02-23 23:30:15,316][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2023-02-23 23:30:15,397][12307] Weights refcount: 2 0 [2023-02-23 23:30:15,402][02125] Component InferenceWorker_p0-w0 stopped! [2023-02-23 23:30:15,402][12307] Stopping InferenceWorker_p0-w0... [2023-02-23 23:30:15,411][12307] Loop inference_proc0-0_evt_loop terminating... [2023-02-23 23:30:15,418][12313] Stopping RolloutWorker_w5... [2023-02-23 23:30:15,420][02125] Component RolloutWorker_w5 stopped! [2023-02-23 23:30:15,421][12313] Loop rollout_proc5_evt_loop terminating... [2023-02-23 23:30:15,443][02125] Component RolloutWorker_w2 stopped! [2023-02-23 23:30:15,445][12314] Stopping RolloutWorker_w2... [2023-02-23 23:30:15,458][02125] Component RolloutWorker_w0 stopped! [2023-02-23 23:30:15,460][12309] Stopping RolloutWorker_w0... [2023-02-23 23:30:15,460][12309] Loop rollout_proc0_evt_loop terminating... [2023-02-23 23:30:15,471][12311] Stopping RolloutWorker_w3... [2023-02-23 23:30:15,471][12311] Loop rollout_proc3_evt_loop terminating... [2023-02-23 23:30:15,477][12314] Loop rollout_proc2_evt_loop terminating... [2023-02-23 23:30:15,470][02125] Component RolloutWorker_w3 stopped! [2023-02-23 23:30:15,536][12294] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000858_3514368.pth [2023-02-23 23:30:15,553][12294] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2023-02-23 23:30:15,787][12294] Stopping LearnerWorker_p0... [2023-02-23 23:30:15,787][02125] Component LearnerWorker_p0 stopped! [2023-02-23 23:30:15,788][12294] Loop learner_proc0_evt_loop terminating... [2023-02-23 23:30:15,789][02125] Waiting for process learner_proc0 to stop... [2023-02-23 23:30:18,284][02125] Waiting for process inference_proc0-0 to join... [2023-02-23 23:30:18,691][02125] Waiting for process rollout_proc0 to join... [2023-02-23 23:30:18,992][02125] Waiting for process rollout_proc1 to join... [2023-02-23 23:30:19,000][02125] Waiting for process rollout_proc2 to join... [2023-02-23 23:30:19,002][02125] Waiting for process rollout_proc3 to join... [2023-02-23 23:30:19,004][02125] Waiting for process rollout_proc4 to join... [2023-02-23 23:30:19,006][02125] Waiting for process rollout_proc5 to join... [2023-02-23 23:30:19,009][02125] Waiting for process rollout_proc6 to join... [2023-02-23 23:30:19,013][02125] Waiting for process rollout_proc7 to join... [2023-02-23 23:30:19,017][02125] Batcher 0 profile tree view: batching: 22.9575, releasing_batches: 0.0341 [2023-02-23 23:30:19,018][02125] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0054 wait_policy_total: 536.6956 update_model: 9.1187 weight_update: 0.0013 one_step: 0.0027 handle_policy_step: 599.8122 deserialize: 16.1768, stack: 3.8218, obs_to_device_normalize: 136.5735, forward: 296.9912, send_messages: 22.8998 prepare_outputs: 92.7067 to_cpu: 58.9255 [2023-02-23 23:30:19,020][02125] Learner 0 profile tree view: misc: 0.0051, prepare_batch: 15.1418 train: 71.1327 epoch_init: 0.0086, minibatch_init: 0.0063, losses_postprocess: 0.5766, kl_divergence: 0.4703, after_optimizer: 31.9969 calculate_losses: 24.3897 losses_init: 0.0035, forward_head: 1.6100, bptt_initial: 16.7412, tail: 0.9957, advantages_returns: 0.2535, losses: 2.6267 bptt: 1.8936 bptt_forward_core: 1.7835 update: 13.1385 clip: 1.3696 [2023-02-23 23:30:19,022][02125] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.5513, enqueue_policy_requests: 213.2908, env_step: 805.7359, overhead: 34.5545, complete_rollouts: 7.0017 save_policy_outputs: 29.9151 split_output_tensors: 14.2990 [2023-02-23 23:30:19,023][02125] Loop Runner_EvtLoop terminating... [2023-02-23 23:30:19,025][02125] Runner profile tree view: main_loop: 1219.4857 [2023-02-23 23:30:19,027][02125] Collected {0: 4005888}, FPS: 3284.9 [2023-02-23 23:30:19,047][02125] Environment doom_basic already registered, overwriting... [2023-02-23 23:30:19,050][02125] Environment doom_two_colors_easy already registered, overwriting... [2023-02-23 23:30:19,051][02125] Environment doom_two_colors_hard already registered, overwriting... [2023-02-23 23:30:19,053][02125] Environment doom_dm already registered, overwriting... [2023-02-23 23:30:19,054][02125] Environment doom_dwango5 already registered, overwriting... [2023-02-23 23:30:19,056][02125] Environment doom_my_way_home_flat_actions already registered, overwriting... [2023-02-23 23:30:19,058][02125] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2023-02-23 23:30:19,060][02125] Environment doom_my_way_home already registered, overwriting... [2023-02-23 23:30:19,063][02125] Environment doom_deadly_corridor already registered, overwriting... [2023-02-23 23:30:19,064][02125] Environment doom_defend_the_center already registered, overwriting... [2023-02-23 23:30:19,066][02125] Environment doom_defend_the_line already registered, overwriting... [2023-02-23 23:30:19,068][02125] Environment doom_health_gathering already registered, overwriting... [2023-02-23 23:30:19,070][02125] Environment doom_health_gathering_supreme already registered, overwriting... [2023-02-23 23:30:19,072][02125] Environment doom_battle already registered, overwriting... [2023-02-23 23:30:19,073][02125] Environment doom_battle2 already registered, overwriting... [2023-02-23 23:30:19,088][02125] Environment doom_duel_bots already registered, overwriting... [2023-02-23 23:30:19,089][02125] Environment doom_deathmatch_bots already registered, overwriting... [2023-02-23 23:30:19,091][02125] Environment doom_duel already registered, overwriting... [2023-02-23 23:30:19,093][02125] Environment doom_deathmatch_full already registered, overwriting... [2023-02-23 23:30:19,095][02125] Environment doom_benchmark already registered, overwriting... [2023-02-23 23:30:19,097][02125] register_encoder_factory: [2023-02-23 23:30:19,125][02125] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-02-23 23:30:19,128][02125] Overriding arg 'train_for_env_steps' with value 8000000 passed from command line [2023-02-23 23:30:19,135][02125] Experiment dir /content/train_dir/default_experiment already exists! [2023-02-23 23:30:19,137][02125] Resuming existing experiment from /content/train_dir/default_experiment... [2023-02-23 23:30:19,139][02125] Weights and Biases integration disabled [2023-02-23 23:30:19,141][02125] Environment var CUDA_VISIBLE_DEVICES is 0 [2023-02-23 23:30:21,173][02125] Starting experiment with the following configuration: help=False algo=APPO env=doom_health_gathering_supreme experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=gpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=4 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=8000000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=1 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} git_hash=unknown git_repo_name=not a git repository [2023-02-23 23:30:21,176][02125] Saving configuration to /content/train_dir/default_experiment/config.json... [2023-02-23 23:30:21,179][02125] Rollout worker 0 uses device cpu [2023-02-23 23:30:21,180][02125] Rollout worker 1 uses device cpu [2023-02-23 23:30:21,182][02125] Rollout worker 2 uses device cpu [2023-02-23 23:30:21,184][02125] Rollout worker 3 uses device cpu [2023-02-23 23:30:21,185][02125] Rollout worker 4 uses device cpu [2023-02-23 23:30:21,187][02125] Rollout worker 5 uses device cpu [2023-02-23 23:30:21,188][02125] Rollout worker 6 uses device cpu [2023-02-23 23:30:21,190][02125] Rollout worker 7 uses device cpu [2023-02-23 23:30:21,331][02125] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:30:21,333][02125] InferenceWorker_p0-w0: min num requests: 2 [2023-02-23 23:30:21,367][02125] Starting all processes... [2023-02-23 23:30:21,371][02125] Starting process learner_proc0 [2023-02-23 23:30:21,427][02125] Starting all processes... [2023-02-23 23:30:21,436][02125] Starting process inference_proc0-0 [2023-02-23 23:30:21,436][02125] Starting process rollout_proc0 [2023-02-23 23:30:21,438][02125] Starting process rollout_proc1 [2023-02-23 23:30:21,438][02125] Starting process rollout_proc2 [2023-02-23 23:30:21,438][02125] Starting process rollout_proc3 [2023-02-23 23:30:21,440][02125] Starting process rollout_proc5 [2023-02-23 23:30:21,440][02125] Starting process rollout_proc6 [2023-02-23 23:30:21,440][02125] Starting process rollout_proc7 [2023-02-23 23:30:21,440][02125] Starting process rollout_proc4 [2023-02-23 23:30:29,945][18153] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:30:29,947][18153] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-02-23 23:30:30,710][18170] Worker 2 uses CPU cores [0] [2023-02-23 23:30:30,914][18179] Worker 4 uses CPU cores [0] [2023-02-23 23:30:31,203][18169] Worker 1 uses CPU cores [1] [2023-02-23 23:30:31,330][18178] Worker 7 uses CPU cores [1] [2023-02-23 23:30:31,360][18168] Worker 0 uses CPU cores [0] [2023-02-23 23:30:31,400][18167] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:30:31,400][18167] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-02-23 23:30:31,414][18173] Worker 6 uses CPU cores [0] [2023-02-23 23:30:31,422][18171] Worker 3 uses CPU cores [1] [2023-02-23 23:30:31,444][18172] Worker 5 uses CPU cores [1] [2023-02-23 23:30:31,544][18167] Num visible devices: 1 [2023-02-23 23:30:31,547][18153] Num visible devices: 1 [2023-02-23 23:30:31,548][18153] Starting seed is not provided [2023-02-23 23:30:31,549][18153] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:30:31,550][18153] Initializing actor-critic model on device cuda:0 [2023-02-23 23:30:31,551][18153] RunningMeanStd input shape: (3, 72, 128) [2023-02-23 23:30:31,552][18153] RunningMeanStd input shape: (1,) [2023-02-23 23:30:31,571][18153] ConvEncoder: input_channels=3 [2023-02-23 23:30:31,761][18153] Conv encoder output size: 512 [2023-02-23 23:30:31,763][18153] Policy head output size: 512 [2023-02-23 23:30:31,787][18153] Created Actor Critic model with architecture: [2023-02-23 23:30:31,788][18153] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2023-02-23 23:30:34,802][18153] Using optimizer [2023-02-23 23:30:34,804][18153] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2023-02-23 23:30:34,844][18153] Loading model from checkpoint [2023-02-23 23:30:34,852][18153] Loaded experiment state at self.train_step=978, self.env_steps=4005888 [2023-02-23 23:30:34,853][18153] Initialized policy 0 weights for model version 978 [2023-02-23 23:30:34,857][18153] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-02-23 23:30:34,868][18153] LearnerWorker_p0 finished initialization! [2023-02-23 23:30:35,011][18167] RunningMeanStd input shape: (3, 72, 128) [2023-02-23 23:30:35,012][18167] RunningMeanStd input shape: (1,) [2023-02-23 23:30:35,024][18167] ConvEncoder: input_channels=3 [2023-02-23 23:30:35,127][18167] Conv encoder output size: 512 [2023-02-23 23:30:35,127][18167] Policy head output size: 512 [2023-02-23 23:30:37,271][02125] Inference worker 0-0 is ready! [2023-02-23 23:30:37,272][02125] All inference workers are ready! Signal rollout workers to start! [2023-02-23 23:30:37,369][18179] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:30:37,372][18170] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:30:37,373][18178] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:30:37,376][18169] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:30:37,376][18172] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:30:37,376][18171] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:30:37,375][18173] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:30:37,374][18168] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:30:38,844][18170] Decorrelating experience for 0 frames... [2023-02-23 23:30:38,847][18173] Decorrelating experience for 0 frames... [2023-02-23 23:30:38,851][18168] Decorrelating experience for 0 frames... [2023-02-23 23:30:38,850][18179] Decorrelating experience for 0 frames... [2023-02-23 23:30:38,875][18178] Decorrelating experience for 0 frames... [2023-02-23 23:30:38,880][18169] Decorrelating experience for 0 frames... [2023-02-23 23:30:38,886][18172] Decorrelating experience for 0 frames... [2023-02-23 23:30:38,888][18171] Decorrelating experience for 0 frames... [2023-02-23 23:30:39,142][02125] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 4005888. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-23 23:30:39,865][18168] Decorrelating experience for 32 frames... [2023-02-23 23:30:39,868][18179] Decorrelating experience for 32 frames... [2023-02-23 23:30:39,983][18170] Decorrelating experience for 32 frames... [2023-02-23 23:30:40,012][18169] Decorrelating experience for 32 frames... [2023-02-23 23:30:40,020][18178] Decorrelating experience for 32 frames... [2023-02-23 23:30:40,029][18172] Decorrelating experience for 32 frames... [2023-02-23 23:30:40,777][18173] Decorrelating experience for 32 frames... [2023-02-23 23:30:40,921][18170] Decorrelating experience for 64 frames... [2023-02-23 23:30:41,097][18171] Decorrelating experience for 32 frames... [2023-02-23 23:30:41,326][02125] Heartbeat connected on Batcher_0 [2023-02-23 23:30:41,331][02125] Heartbeat connected on LearnerWorker_p0 [2023-02-23 23:30:41,359][18169] Decorrelating experience for 64 frames... [2023-02-23 23:30:41,362][18172] Decorrelating experience for 64 frames... [2023-02-23 23:30:41,366][02125] Heartbeat connected on InferenceWorker_p0-w0 [2023-02-23 23:30:41,552][18179] Decorrelating experience for 64 frames... [2023-02-23 23:30:41,908][18173] Decorrelating experience for 64 frames... [2023-02-23 23:30:42,225][18170] Decorrelating experience for 96 frames... [2023-02-23 23:30:42,413][02125] Heartbeat connected on RolloutWorker_w2 [2023-02-23 23:30:42,425][18169] Decorrelating experience for 96 frames... [2023-02-23 23:30:42,427][18172] Decorrelating experience for 96 frames... [2023-02-23 23:30:42,643][02125] Heartbeat connected on RolloutWorker_w1 [2023-02-23 23:30:42,649][02125] Heartbeat connected on RolloutWorker_w5 [2023-02-23 23:30:42,671][18178] Decorrelating experience for 64 frames... [2023-02-23 23:30:43,075][18179] Decorrelating experience for 96 frames... [2023-02-23 23:30:43,284][02125] Heartbeat connected on RolloutWorker_w4 [2023-02-23 23:30:43,363][18173] Decorrelating experience for 96 frames... [2023-02-23 23:30:43,557][02125] Heartbeat connected on RolloutWorker_w6 [2023-02-23 23:30:43,640][18171] Decorrelating experience for 64 frames... [2023-02-23 23:30:43,984][18178] Decorrelating experience for 96 frames... [2023-02-23 23:30:44,142][02125] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 4.8. Samples: 24. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-23 23:30:44,273][02125] Heartbeat connected on RolloutWorker_w7 [2023-02-23 23:30:45,601][18171] Decorrelating experience for 96 frames... [2023-02-23 23:30:46,264][02125] Heartbeat connected on RolloutWorker_w3 [2023-02-23 23:30:47,601][18168] Decorrelating experience for 64 frames... [2023-02-23 23:30:48,597][18153] Signal inference workers to stop experience collection... [2023-02-23 23:30:48,631][18167] InferenceWorker_p0-w0: stopping experience collection [2023-02-23 23:30:49,142][02125] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 159.2. Samples: 1592. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-23 23:30:49,148][02125] Avg episode reward: [(0, '2.148')] [2023-02-23 23:30:49,835][18168] Decorrelating experience for 96 frames... [2023-02-23 23:30:50,060][02125] Heartbeat connected on RolloutWorker_w0 [2023-02-23 23:30:50,165][18153] Signal inference workers to resume experience collection... [2023-02-23 23:30:50,166][18167] InferenceWorker_p0-w0: resuming experience collection [2023-02-23 23:30:54,142][02125] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 4018176. Throughput: 0: 218.0. Samples: 3270. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-02-23 23:30:54,146][02125] Avg episode reward: [(0, '4.635')] [2023-02-23 23:30:59,142][02125] Fps is (10 sec: 3276.8, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 4038656. Throughput: 0: 431.2. Samples: 8624. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:30:59,147][02125] Avg episode reward: [(0, '11.111')] [2023-02-23 23:31:00,292][18167] Updated weights for policy 0, policy_version 988 (0.0020) [2023-02-23 23:31:04,142][02125] Fps is (10 sec: 4505.5, 60 sec: 2293.7, 300 sec: 2293.7). Total num frames: 4063232. Throughput: 0: 474.1. Samples: 11852. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:31:04,144][02125] Avg episode reward: [(0, '18.787')] [2023-02-23 23:31:09,142][02125] Fps is (10 sec: 3686.4, 60 sec: 2321.1, 300 sec: 2321.1). Total num frames: 4075520. Throughput: 0: 581.5. Samples: 17444. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:31:09,144][02125] Avg episode reward: [(0, '20.154')] [2023-02-23 23:31:12,466][18167] Updated weights for policy 0, policy_version 998 (0.0033) [2023-02-23 23:31:14,143][02125] Fps is (10 sec: 2867.2, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 4091904. Throughput: 0: 619.9. Samples: 21696. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:31:14,150][02125] Avg episode reward: [(0, '21.654')] [2023-02-23 23:31:19,142][02125] Fps is (10 sec: 3686.4, 60 sec: 2662.4, 300 sec: 2662.4). Total num frames: 4112384. Throughput: 0: 605.7. Samples: 24228. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:31:19,149][02125] Avg episode reward: [(0, '23.781')] [2023-02-23 23:31:22,882][18167] Updated weights for policy 0, policy_version 1008 (0.0020) [2023-02-23 23:31:24,142][02125] Fps is (10 sec: 4096.1, 60 sec: 2821.7, 300 sec: 2821.7). Total num frames: 4132864. Throughput: 0: 685.2. Samples: 30836. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:31:24,150][02125] Avg episode reward: [(0, '25.895')] [2023-02-23 23:31:24,153][18153] Saving new best policy, reward=25.895! [2023-02-23 23:31:29,142][02125] Fps is (10 sec: 3686.4, 60 sec: 2867.2, 300 sec: 2867.2). Total num frames: 4149248. Throughput: 0: 798.6. Samples: 35962. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:31:29,146][02125] Avg episode reward: [(0, '27.468')] [2023-02-23 23:31:29,156][18153] Saving new best policy, reward=27.468! [2023-02-23 23:31:34,142][02125] Fps is (10 sec: 2867.1, 60 sec: 2830.0, 300 sec: 2830.0). Total num frames: 4161536. Throughput: 0: 808.3. Samples: 37966. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:31:34,149][02125] Avg episode reward: [(0, '27.884')] [2023-02-23 23:31:34,151][18153] Saving new best policy, reward=27.884! [2023-02-23 23:31:36,307][18167] Updated weights for policy 0, policy_version 1018 (0.0021) [2023-02-23 23:31:39,142][02125] Fps is (10 sec: 3276.8, 60 sec: 2935.5, 300 sec: 2935.5). Total num frames: 4182016. Throughput: 0: 878.3. Samples: 42794. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:31:39,145][02125] Avg episode reward: [(0, '27.518')] [2023-02-23 23:31:44,143][02125] Fps is (10 sec: 4095.6, 60 sec: 3276.7, 300 sec: 3024.7). Total num frames: 4202496. Throughput: 0: 900.6. Samples: 49150. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:31:44,150][02125] Avg episode reward: [(0, '25.727')] [2023-02-23 23:31:45,713][18167] Updated weights for policy 0, policy_version 1028 (0.0025) [2023-02-23 23:31:49,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3042.7). Total num frames: 4218880. Throughput: 0: 897.9. Samples: 52258. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:31:49,146][02125] Avg episode reward: [(0, '24.786')] [2023-02-23 23:31:54,149][02125] Fps is (10 sec: 2867.0, 60 sec: 3549.7, 300 sec: 3003.7). Total num frames: 4231168. Throughput: 0: 865.9. Samples: 56410. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:31:54,152][02125] Avg episode reward: [(0, '23.894')] [2023-02-23 23:31:58,937][18167] Updated weights for policy 0, policy_version 1038 (0.0028) [2023-02-23 23:31:59,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3072.0). Total num frames: 4251648. Throughput: 0: 881.1. Samples: 61346. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:31:59,147][02125] Avg episode reward: [(0, '25.495')] [2023-02-23 23:32:04,144][02125] Fps is (10 sec: 4095.8, 60 sec: 3481.5, 300 sec: 3132.1). Total num frames: 4272128. Throughput: 0: 899.5. Samples: 64706. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:32:04,147][02125] Avg episode reward: [(0, '25.563')] [2023-02-23 23:32:09,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3140.3). Total num frames: 4288512. Throughput: 0: 888.6. Samples: 70822. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:32:09,150][02125] Avg episode reward: [(0, '24.106')] [2023-02-23 23:32:09,513][18167] Updated weights for policy 0, policy_version 1048 (0.0031) [2023-02-23 23:32:14,146][02125] Fps is (10 sec: 3276.1, 60 sec: 3549.6, 300 sec: 3147.3). Total num frames: 4304896. Throughput: 0: 869.2. Samples: 75082. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:32:14,149][02125] Avg episode reward: [(0, '24.553')] [2023-02-23 23:32:19,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3153.9). Total num frames: 4321280. Throughput: 0: 871.8. Samples: 77198. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:32:19,145][02125] Avg episode reward: [(0, '26.732')] [2023-02-23 23:32:19,154][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001055_4321280.pth... [2023-02-23 23:32:19,275][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000961_3936256.pth [2023-02-23 23:32:21,279][18167] Updated weights for policy 0, policy_version 1058 (0.0030) [2023-02-23 23:32:24,142][02125] Fps is (10 sec: 4097.8, 60 sec: 3549.9, 300 sec: 3237.8). Total num frames: 4345856. Throughput: 0: 906.9. Samples: 83604. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:32:24,145][02125] Avg episode reward: [(0, '27.468')] [2023-02-23 23:32:29,142][02125] Fps is (10 sec: 4095.9, 60 sec: 3549.8, 300 sec: 3239.5). Total num frames: 4362240. Throughput: 0: 900.5. Samples: 89672. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:32:29,149][02125] Avg episode reward: [(0, '25.875')] [2023-02-23 23:32:32,665][18167] Updated weights for policy 0, policy_version 1068 (0.0018) [2023-02-23 23:32:34,146][02125] Fps is (10 sec: 3275.3, 60 sec: 3617.9, 300 sec: 3241.1). Total num frames: 4378624. Throughput: 0: 876.6. Samples: 91710. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:32:34,154][02125] Avg episode reward: [(0, '26.193')] [2023-02-23 23:32:39,142][02125] Fps is (10 sec: 3276.9, 60 sec: 3549.9, 300 sec: 3242.7). Total num frames: 4395008. Throughput: 0: 879.6. Samples: 95990. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:32:39,148][02125] Avg episode reward: [(0, '26.811')] [2023-02-23 23:32:43,529][18167] Updated weights for policy 0, policy_version 1078 (0.0024) [2023-02-23 23:32:44,142][02125] Fps is (10 sec: 3687.9, 60 sec: 3549.9, 300 sec: 3276.8). Total num frames: 4415488. Throughput: 0: 917.1. Samples: 102616. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:32:44,148][02125] Avg episode reward: [(0, '27.549')] [2023-02-23 23:32:49,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3308.3). Total num frames: 4435968. Throughput: 0: 916.7. Samples: 105956. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:32:49,148][02125] Avg episode reward: [(0, '25.795')] [2023-02-23 23:32:54,142][02125] Fps is (10 sec: 3276.9, 60 sec: 3618.3, 300 sec: 3276.8). Total num frames: 4448256. Throughput: 0: 879.5. Samples: 110400. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:32:54,146][02125] Avg episode reward: [(0, '24.172')] [2023-02-23 23:32:56,473][18167] Updated weights for policy 0, policy_version 1088 (0.0012) [2023-02-23 23:32:59,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3276.8). Total num frames: 4464640. Throughput: 0: 882.3. Samples: 114780. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:32:59,145][02125] Avg episode reward: [(0, '24.728')] [2023-02-23 23:33:04,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3550.0, 300 sec: 3305.0). Total num frames: 4485120. Throughput: 0: 909.2. Samples: 118112. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:33:04,144][02125] Avg episode reward: [(0, '23.371')] [2023-02-23 23:33:06,013][18167] Updated weights for policy 0, policy_version 1098 (0.0022) [2023-02-23 23:33:09,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3331.4). Total num frames: 4505600. Throughput: 0: 913.3. Samples: 124704. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:33:09,147][02125] Avg episode reward: [(0, '23.195')] [2023-02-23 23:33:14,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3618.4, 300 sec: 3329.7). Total num frames: 4521984. Throughput: 0: 877.3. Samples: 129152. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:33:14,146][02125] Avg episode reward: [(0, '22.385')] [2023-02-23 23:33:19,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3302.4). Total num frames: 4534272. Throughput: 0: 880.8. Samples: 131344. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:33:19,144][02125] Avg episode reward: [(0, '23.667')] [2023-02-23 23:33:19,288][18167] Updated weights for policy 0, policy_version 1108 (0.0042) [2023-02-23 23:33:24,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3351.3). Total num frames: 4558848. Throughput: 0: 916.3. Samples: 137224. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:33:24,145][02125] Avg episode reward: [(0, '23.739')] [2023-02-23 23:33:28,318][18167] Updated weights for policy 0, policy_version 1118 (0.0014) [2023-02-23 23:33:29,151][02125] Fps is (10 sec: 4501.4, 60 sec: 3617.6, 300 sec: 3373.0). Total num frames: 4579328. Throughput: 0: 917.0. Samples: 143890. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:33:29,160][02125] Avg episode reward: [(0, '23.877')] [2023-02-23 23:33:34,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3618.4, 300 sec: 3370.4). Total num frames: 4595712. Throughput: 0: 890.4. Samples: 146026. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:33:34,144][02125] Avg episode reward: [(0, '23.566')] [2023-02-23 23:33:39,142][02125] Fps is (10 sec: 2869.9, 60 sec: 3549.9, 300 sec: 3345.1). Total num frames: 4608000. Throughput: 0: 886.8. Samples: 150306. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:33:39,147][02125] Avg episode reward: [(0, '24.403')] [2023-02-23 23:33:41,148][18167] Updated weights for policy 0, policy_version 1128 (0.0033) [2023-02-23 23:33:44,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3387.5). Total num frames: 4632576. Throughput: 0: 928.4. Samples: 156558. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:33:44,147][02125] Avg episode reward: [(0, '23.079')] [2023-02-23 23:33:49,143][02125] Fps is (10 sec: 4504.9, 60 sec: 3618.0, 300 sec: 3406.1). Total num frames: 4653056. Throughput: 0: 930.2. Samples: 159972. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:33:49,150][02125] Avg episode reward: [(0, '24.209')] [2023-02-23 23:33:51,274][18167] Updated weights for policy 0, policy_version 1138 (0.0020) [2023-02-23 23:33:54,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3381.8). Total num frames: 4665344. Throughput: 0: 897.9. Samples: 165108. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:33:54,150][02125] Avg episode reward: [(0, '24.632')] [2023-02-23 23:33:59,143][02125] Fps is (10 sec: 2457.6, 60 sec: 3549.8, 300 sec: 3358.7). Total num frames: 4677632. Throughput: 0: 880.1. Samples: 168760. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:33:59,145][02125] Avg episode reward: [(0, '25.874')] [2023-02-23 23:34:04,145][02125] Fps is (10 sec: 2456.7, 60 sec: 3413.1, 300 sec: 3336.7). Total num frames: 4689920. Throughput: 0: 870.8. Samples: 170532. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:34:04,148][02125] Avg episode reward: [(0, '26.103')] [2023-02-23 23:34:07,103][18167] Updated weights for policy 0, policy_version 1148 (0.0023) [2023-02-23 23:34:09,145][02125] Fps is (10 sec: 3276.2, 60 sec: 3413.1, 300 sec: 3354.8). Total num frames: 4710400. Throughput: 0: 834.7. Samples: 174788. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:34:09,148][02125] Avg episode reward: [(0, '26.817')] [2023-02-23 23:34:14,142][02125] Fps is (10 sec: 3687.7, 60 sec: 3413.3, 300 sec: 3353.0). Total num frames: 4726784. Throughput: 0: 812.6. Samples: 180450. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:34:14,149][02125] Avg episode reward: [(0, '25.297')] [2023-02-23 23:34:19,142][02125] Fps is (10 sec: 2868.2, 60 sec: 3413.3, 300 sec: 3332.7). Total num frames: 4739072. Throughput: 0: 811.3. Samples: 182534. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:34:19,147][02125] Avg episode reward: [(0, '27.236')] [2023-02-23 23:34:19,163][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001157_4739072.pth... [2023-02-23 23:34:19,303][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth [2023-02-23 23:34:19,679][18167] Updated weights for policy 0, policy_version 1158 (0.0030) [2023-02-23 23:34:24,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3349.6). Total num frames: 4759552. Throughput: 0: 818.4. Samples: 187134. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:34:24,148][02125] Avg episode reward: [(0, '26.952')] [2023-02-23 23:34:29,144][02125] Fps is (10 sec: 4095.0, 60 sec: 3345.5, 300 sec: 3365.8). Total num frames: 4780032. Throughput: 0: 826.5. Samples: 193752. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:34:29,150][02125] Avg episode reward: [(0, '28.224')] [2023-02-23 23:34:29,161][18153] Saving new best policy, reward=28.224! [2023-02-23 23:34:29,600][18167] Updated weights for policy 0, policy_version 1168 (0.0029) [2023-02-23 23:34:34,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 3363.9). Total num frames: 4796416. Throughput: 0: 821.3. Samples: 196928. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:34:34,150][02125] Avg episode reward: [(0, '27.197')] [2023-02-23 23:34:39,142][02125] Fps is (10 sec: 3277.6, 60 sec: 3413.3, 300 sec: 3362.1). Total num frames: 4812800. Throughput: 0: 802.3. Samples: 201210. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:34:39,147][02125] Avg episode reward: [(0, '26.378')] [2023-02-23 23:34:42,574][18167] Updated weights for policy 0, policy_version 1178 (0.0031) [2023-02-23 23:34:44,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3360.4). Total num frames: 4829184. Throughput: 0: 829.5. Samples: 206086. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:34:44,150][02125] Avg episode reward: [(0, '26.666')] [2023-02-23 23:34:49,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3276.9, 300 sec: 3375.1). Total num frames: 4849664. Throughput: 0: 862.1. Samples: 209322. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:34:49,145][02125] Avg episode reward: [(0, '26.627')] [2023-02-23 23:34:51,920][18167] Updated weights for policy 0, policy_version 1188 (0.0016) [2023-02-23 23:34:54,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3389.2). Total num frames: 4870144. Throughput: 0: 912.2. Samples: 215834. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:34:54,144][02125] Avg episode reward: [(0, '28.020')] [2023-02-23 23:34:59,142][02125] Fps is (10 sec: 3276.7, 60 sec: 3413.4, 300 sec: 3371.3). Total num frames: 4882432. Throughput: 0: 877.2. Samples: 219926. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-23 23:34:59,146][02125] Avg episode reward: [(0, '27.590')] [2023-02-23 23:35:04,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3550.1, 300 sec: 3385.0). Total num frames: 4902912. Throughput: 0: 877.6. Samples: 222028. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-23 23:35:04,144][02125] Avg episode reward: [(0, '27.201')] [2023-02-23 23:35:04,979][18167] Updated weights for policy 0, policy_version 1198 (0.0019) [2023-02-23 23:35:09,144][02125] Fps is (10 sec: 4095.4, 60 sec: 3550.0, 300 sec: 3398.1). Total num frames: 4923392. Throughput: 0: 915.7. Samples: 228342. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:35:09,148][02125] Avg episode reward: [(0, '27.582')] [2023-02-23 23:35:14,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3410.9). Total num frames: 4943872. Throughput: 0: 910.3. Samples: 234712. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:35:14,152][02125] Avg episode reward: [(0, '28.671')] [2023-02-23 23:35:14,156][18153] Saving new best policy, reward=28.671! [2023-02-23 23:35:15,494][18167] Updated weights for policy 0, policy_version 1208 (0.0012) [2023-02-23 23:35:19,142][02125] Fps is (10 sec: 3277.4, 60 sec: 3618.1, 300 sec: 3393.8). Total num frames: 4956160. Throughput: 0: 884.6. Samples: 236734. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:35:19,145][02125] Avg episode reward: [(0, '27.608')] [2023-02-23 23:35:24,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3391.8). Total num frames: 4972544. Throughput: 0: 885.0. Samples: 241034. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:35:24,148][02125] Avg episode reward: [(0, '27.178')] [2023-02-23 23:35:27,236][18167] Updated weights for policy 0, policy_version 1218 (0.0012) [2023-02-23 23:35:29,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3618.3, 300 sec: 3418.0). Total num frames: 4997120. Throughput: 0: 922.5. Samples: 247600. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:35:29,149][02125] Avg episode reward: [(0, '25.800')] [2023-02-23 23:35:34,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3415.6). Total num frames: 5013504. Throughput: 0: 920.2. Samples: 250730. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:35:34,145][02125] Avg episode reward: [(0, '25.657')] [2023-02-23 23:35:38,692][18167] Updated weights for policy 0, policy_version 1228 (0.0025) [2023-02-23 23:35:39,144][02125] Fps is (10 sec: 3276.0, 60 sec: 3618.0, 300 sec: 3471.2). Total num frames: 5029888. Throughput: 0: 884.3. Samples: 255630. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:35:39,151][02125] Avg episode reward: [(0, '23.802')] [2023-02-23 23:35:44,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 5046272. Throughput: 0: 888.7. Samples: 259916. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:35:44,150][02125] Avg episode reward: [(0, '23.468')] [2023-02-23 23:35:49,142][02125] Fps is (10 sec: 3687.3, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 5066752. Throughput: 0: 917.1. Samples: 263296. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:35:49,150][02125] Avg episode reward: [(0, '23.201')] [2023-02-23 23:35:49,575][18167] Updated weights for policy 0, policy_version 1238 (0.0026) [2023-02-23 23:35:54,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 5087232. Throughput: 0: 923.7. Samples: 269908. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:35:54,148][02125] Avg episode reward: [(0, '25.362')] [2023-02-23 23:35:59,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3512.8). Total num frames: 5099520. Throughput: 0: 882.0. Samples: 274402. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:35:59,144][02125] Avg episode reward: [(0, '25.951')] [2023-02-23 23:36:02,436][18167] Updated weights for policy 0, policy_version 1248 (0.0017) [2023-02-23 23:36:04,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 5115904. Throughput: 0: 881.8. Samples: 276414. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:36:04,149][02125] Avg episode reward: [(0, '26.124')] [2023-02-23 23:36:09,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3550.0, 300 sec: 3540.6). Total num frames: 5136384. Throughput: 0: 915.2. Samples: 282220. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:36:09,144][02125] Avg episode reward: [(0, '25.259')] [2023-02-23 23:36:11,802][18167] Updated weights for policy 0, policy_version 1258 (0.0025) [2023-02-23 23:36:14,142][02125] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 5160960. Throughput: 0: 918.9. Samples: 288952. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:36:14,148][02125] Avg episode reward: [(0, '26.109')] [2023-02-23 23:36:19,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 5173248. Throughput: 0: 897.3. Samples: 291108. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:36:19,144][02125] Avg episode reward: [(0, '25.513')] [2023-02-23 23:36:19,163][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001263_5173248.pth... [2023-02-23 23:36:19,328][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001055_4321280.pth [2023-02-23 23:36:24,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 5189632. Throughput: 0: 883.8. Samples: 295400. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:36:24,143][02125] Avg episode reward: [(0, '24.462')] [2023-02-23 23:36:24,905][18167] Updated weights for policy 0, policy_version 1268 (0.0017) [2023-02-23 23:36:29,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 5210112. Throughput: 0: 921.2. Samples: 301370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:36:29,145][02125] Avg episode reward: [(0, '23.571')] [2023-02-23 23:36:34,145][02125] Fps is (10 sec: 4094.5, 60 sec: 3617.9, 300 sec: 3554.5). Total num frames: 5230592. Throughput: 0: 916.0. Samples: 304520. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:36:34,148][02125] Avg episode reward: [(0, '24.973')] [2023-02-23 23:36:34,319][18167] Updated weights for policy 0, policy_version 1278 (0.0012) [2023-02-23 23:36:39,142][02125] Fps is (10 sec: 3686.1, 60 sec: 3618.2, 300 sec: 3540.6). Total num frames: 5246976. Throughput: 0: 887.3. Samples: 309836. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:36:39,148][02125] Avg episode reward: [(0, '26.638')] [2023-02-23 23:36:44,142][02125] Fps is (10 sec: 2868.2, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 5259264. Throughput: 0: 882.4. Samples: 314112. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:36:44,145][02125] Avg episode reward: [(0, '27.659')] [2023-02-23 23:36:47,009][18167] Updated weights for policy 0, policy_version 1288 (0.0015) [2023-02-23 23:36:49,142][02125] Fps is (10 sec: 3686.6, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 5283840. Throughput: 0: 904.7. Samples: 317126. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:36:49,148][02125] Avg episode reward: [(0, '28.554')] [2023-02-23 23:36:54,142][02125] Fps is (10 sec: 4505.5, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 5304320. Throughput: 0: 922.5. Samples: 323734. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:36:54,145][02125] Avg episode reward: [(0, '30.061')] [2023-02-23 23:36:54,149][18153] Saving new best policy, reward=30.061! [2023-02-23 23:36:57,585][18167] Updated weights for policy 0, policy_version 1298 (0.0019) [2023-02-23 23:36:59,142][02125] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3554.5). Total num frames: 5320704. Throughput: 0: 882.5. Samples: 328664. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:36:59,151][02125] Avg episode reward: [(0, '29.967')] [2023-02-23 23:37:04,142][02125] Fps is (10 sec: 2867.3, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 5332992. Throughput: 0: 878.6. Samples: 330644. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:37:04,149][02125] Avg episode reward: [(0, '30.933')] [2023-02-23 23:37:04,152][18153] Saving new best policy, reward=30.933! [2023-02-23 23:37:09,148][02125] Fps is (10 sec: 3274.6, 60 sec: 3617.7, 300 sec: 3554.5). Total num frames: 5353472. Throughput: 0: 890.5. Samples: 335478. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:37:09,152][02125] Avg episode reward: [(0, '30.930')] [2023-02-23 23:37:10,045][18167] Updated weights for policy 0, policy_version 1308 (0.0022) [2023-02-23 23:37:14,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3568.4). Total num frames: 5373952. Throughput: 0: 902.6. Samples: 341986. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:37:14,148][02125] Avg episode reward: [(0, '30.281')] [2023-02-23 23:37:19,142][02125] Fps is (10 sec: 3688.5, 60 sec: 3618.1, 300 sec: 3540.6). Total num frames: 5390336. Throughput: 0: 897.8. Samples: 344920. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:37:19,146][02125] Avg episode reward: [(0, '29.723')] [2023-02-23 23:37:21,784][18167] Updated weights for policy 0, policy_version 1318 (0.0021) [2023-02-23 23:37:24,142][02125] Fps is (10 sec: 2867.0, 60 sec: 3549.8, 300 sec: 3526.7). Total num frames: 5402624. Throughput: 0: 874.4. Samples: 349184. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:37:24,147][02125] Avg episode reward: [(0, '28.538')] [2023-02-23 23:37:29,142][02125] Fps is (10 sec: 3277.1, 60 sec: 3549.9, 300 sec: 3540.7). Total num frames: 5423104. Throughput: 0: 900.8. Samples: 354650. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:37:29,149][02125] Avg episode reward: [(0, '28.827')] [2023-02-23 23:37:32,318][18167] Updated weights for policy 0, policy_version 1328 (0.0018) [2023-02-23 23:37:34,142][02125] Fps is (10 sec: 4096.3, 60 sec: 3550.1, 300 sec: 3554.5). Total num frames: 5443584. Throughput: 0: 907.4. Samples: 357958. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:37:34,144][02125] Avg episode reward: [(0, '28.398')] [2023-02-23 23:37:39,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3618.2, 300 sec: 3554.5). Total num frames: 5464064. Throughput: 0: 892.9. Samples: 363914. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:37:39,145][02125] Avg episode reward: [(0, '28.159')] [2023-02-23 23:37:44,142][02125] Fps is (10 sec: 3276.6, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 5476352. Throughput: 0: 869.8. Samples: 367804. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:37:44,147][02125] Avg episode reward: [(0, '26.212')] [2023-02-23 23:37:45,862][18167] Updated weights for policy 0, policy_version 1338 (0.0012) [2023-02-23 23:37:49,142][02125] Fps is (10 sec: 2457.6, 60 sec: 3413.4, 300 sec: 3526.7). Total num frames: 5488640. Throughput: 0: 861.7. Samples: 369420. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:37:49,146][02125] Avg episode reward: [(0, '26.840')] [2023-02-23 23:37:54,142][02125] Fps is (10 sec: 2457.7, 60 sec: 3276.8, 300 sec: 3512.8). Total num frames: 5500928. Throughput: 0: 841.4. Samples: 373336. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:37:54,148][02125] Avg episode reward: [(0, '27.102')] [2023-02-23 23:37:58,906][18167] Updated weights for policy 0, policy_version 1348 (0.0041) [2023-02-23 23:37:59,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3512.8). Total num frames: 5521408. Throughput: 0: 820.9. Samples: 378928. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:37:59,144][02125] Avg episode reward: [(0, '26.424')] [2023-02-23 23:38:04,145][02125] Fps is (10 sec: 3275.7, 60 sec: 3344.9, 300 sec: 3485.0). Total num frames: 5533696. Throughput: 0: 801.0. Samples: 380966. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:38:04,148][02125] Avg episode reward: [(0, '25.824')] [2023-02-23 23:38:09,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3277.2, 300 sec: 3485.1). Total num frames: 5550080. Throughput: 0: 799.3. Samples: 385154. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:38:09,145][02125] Avg episode reward: [(0, '26.424')] [2023-02-23 23:38:11,947][18167] Updated weights for policy 0, policy_version 1358 (0.0049) [2023-02-23 23:38:14,142][02125] Fps is (10 sec: 3687.7, 60 sec: 3276.8, 300 sec: 3512.8). Total num frames: 5570560. Throughput: 0: 810.7. Samples: 391132. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:38:14,144][02125] Avg episode reward: [(0, '26.447')] [2023-02-23 23:38:19,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3499.0). Total num frames: 5591040. Throughput: 0: 811.8. Samples: 394488. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:38:19,144][02125] Avg episode reward: [(0, '27.415')] [2023-02-23 23:38:19,165][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001365_5591040.pth... [2023-02-23 23:38:19,315][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001157_4739072.pth [2023-02-23 23:38:22,707][18167] Updated weights for policy 0, policy_version 1368 (0.0012) [2023-02-23 23:38:24,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3471.3). Total num frames: 5603328. Throughput: 0: 793.7. Samples: 399630. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:38:24,148][02125] Avg episode reward: [(0, '27.745')] [2023-02-23 23:38:29,146][02125] Fps is (10 sec: 2865.9, 60 sec: 3276.6, 300 sec: 3471.1). Total num frames: 5619712. Throughput: 0: 799.0. Samples: 403764. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:38:29,149][02125] Avg episode reward: [(0, '27.615')] [2023-02-23 23:38:34,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 3499.0). Total num frames: 5640192. Throughput: 0: 825.0. Samples: 406546. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:38:34,149][02125] Avg episode reward: [(0, '26.802')] [2023-02-23 23:38:34,569][18167] Updated weights for policy 0, policy_version 1378 (0.0022) [2023-02-23 23:38:39,142][02125] Fps is (10 sec: 4507.6, 60 sec: 3345.1, 300 sec: 3499.0). Total num frames: 5664768. Throughput: 0: 888.0. Samples: 413296. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:38:39,148][02125] Avg episode reward: [(0, '26.811')] [2023-02-23 23:38:44,142][02125] Fps is (10 sec: 3686.3, 60 sec: 3345.1, 300 sec: 3471.2). Total num frames: 5677056. Throughput: 0: 876.5. Samples: 418372. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:38:44,147][02125] Avg episode reward: [(0, '26.006')] [2023-02-23 23:38:46,226][18167] Updated weights for policy 0, policy_version 1388 (0.0014) [2023-02-23 23:38:49,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3485.1). Total num frames: 5693440. Throughput: 0: 877.8. Samples: 420462. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:38:49,145][02125] Avg episode reward: [(0, '25.021')] [2023-02-23 23:38:54,142][02125] Fps is (10 sec: 3686.5, 60 sec: 3549.9, 300 sec: 3512.9). Total num frames: 5713920. Throughput: 0: 901.9. Samples: 425738. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:38:54,145][02125] Avg episode reward: [(0, '22.855')] [2023-02-23 23:38:56,663][18167] Updated weights for policy 0, policy_version 1398 (0.0031) [2023-02-23 23:38:59,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3540.7). Total num frames: 5734400. Throughput: 0: 916.5. Samples: 432374. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:38:59,145][02125] Avg episode reward: [(0, '22.903')] [2023-02-23 23:39:04,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3618.3, 300 sec: 3526.8). Total num frames: 5750784. Throughput: 0: 900.0. Samples: 434990. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:39:04,144][02125] Avg episode reward: [(0, '24.574')] [2023-02-23 23:39:09,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 5763072. Throughput: 0: 877.6. Samples: 439124. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:39:09,146][02125] Avg episode reward: [(0, '24.050')] [2023-02-23 23:39:09,821][18167] Updated weights for policy 0, policy_version 1408 (0.0030) [2023-02-23 23:39:14,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3540.6). Total num frames: 5783552. Throughput: 0: 905.2. Samples: 444492. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:39:14,149][02125] Avg episode reward: [(0, '23.931')] [2023-02-23 23:39:19,142][02125] Fps is (10 sec: 4095.9, 60 sec: 3549.8, 300 sec: 3540.6). Total num frames: 5804032. Throughput: 0: 917.5. Samples: 447834. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:39:19,150][02125] Avg episode reward: [(0, '25.471')] [2023-02-23 23:39:19,310][18167] Updated weights for policy 0, policy_version 1418 (0.0020) [2023-02-23 23:39:24,148][02125] Fps is (10 sec: 3684.1, 60 sec: 3617.8, 300 sec: 3526.7). Total num frames: 5820416. Throughput: 0: 897.2. Samples: 453676. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:39:24,157][02125] Avg episode reward: [(0, '26.676')] [2023-02-23 23:39:29,142][02125] Fps is (10 sec: 2867.3, 60 sec: 3550.1, 300 sec: 3512.8). Total num frames: 5832704. Throughput: 0: 873.0. Samples: 457658. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:39:29,150][02125] Avg episode reward: [(0, '26.761')] [2023-02-23 23:39:32,840][18167] Updated weights for policy 0, policy_version 1428 (0.0012) [2023-02-23 23:39:34,142][02125] Fps is (10 sec: 3278.9, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 5853184. Throughput: 0: 874.0. Samples: 459790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:39:34,145][02125] Avg episode reward: [(0, '27.481')] [2023-02-23 23:39:39,142][02125] Fps is (10 sec: 4095.9, 60 sec: 3481.6, 300 sec: 3540.6). Total num frames: 5873664. Throughput: 0: 901.0. Samples: 466282. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:39:39,150][02125] Avg episode reward: [(0, '26.462')] [2023-02-23 23:39:42,734][18167] Updated weights for policy 0, policy_version 1438 (0.0012) [2023-02-23 23:39:44,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 5890048. Throughput: 0: 878.2. Samples: 471894. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:39:44,145][02125] Avg episode reward: [(0, '27.524')] [2023-02-23 23:39:49,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 5906432. Throughput: 0: 866.2. Samples: 473970. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:39:49,145][02125] Avg episode reward: [(0, '29.040')] [2023-02-23 23:39:54,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 5922816. Throughput: 0: 869.7. Samples: 478262. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:39:54,147][02125] Avg episode reward: [(0, '28.509')] [2023-02-23 23:39:55,810][18167] Updated weights for policy 0, policy_version 1448 (0.0041) [2023-02-23 23:39:59,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 5943296. Throughput: 0: 895.0. Samples: 484766. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:39:59,145][02125] Avg episode reward: [(0, '29.074')] [2023-02-23 23:40:04,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 5963776. Throughput: 0: 893.0. Samples: 488020. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:40:04,148][02125] Avg episode reward: [(0, '30.146')] [2023-02-23 23:40:06,883][18167] Updated weights for policy 0, policy_version 1458 (0.0012) [2023-02-23 23:40:09,148][02125] Fps is (10 sec: 3274.7, 60 sec: 3549.5, 300 sec: 3498.9). Total num frames: 5976064. Throughput: 0: 859.2. Samples: 492342. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:40:09,155][02125] Avg episode reward: [(0, '30.067')] [2023-02-23 23:40:14,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 5992448. Throughput: 0: 872.0. Samples: 496898. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:40:14,149][02125] Avg episode reward: [(0, '30.325')] [2023-02-23 23:40:18,273][18167] Updated weights for policy 0, policy_version 1468 (0.0023) [2023-02-23 23:40:19,142][02125] Fps is (10 sec: 3688.8, 60 sec: 3481.6, 300 sec: 3526.7). Total num frames: 6012928. Throughput: 0: 896.8. Samples: 500146. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:40:19,156][02125] Avg episode reward: [(0, '30.098')] [2023-02-23 23:40:19,187][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001469_6017024.pth... [2023-02-23 23:40:19,329][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001263_5173248.pth [2023-02-23 23:40:24,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3550.2, 300 sec: 3512.8). Total num frames: 6033408. Throughput: 0: 893.0. Samples: 506468. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:40:24,144][02125] Avg episode reward: [(0, '29.214')] [2023-02-23 23:40:29,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 6045696. Throughput: 0: 863.2. Samples: 510738. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:40:29,146][02125] Avg episode reward: [(0, '28.229')] [2023-02-23 23:40:30,899][18167] Updated weights for policy 0, policy_version 1478 (0.0025) [2023-02-23 23:40:34,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3499.0). Total num frames: 6062080. Throughput: 0: 862.6. Samples: 512786. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:40:34,150][02125] Avg episode reward: [(0, '28.707')] [2023-02-23 23:40:39,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3512.8). Total num frames: 6082560. Throughput: 0: 894.9. Samples: 518534. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:40:39,147][02125] Avg episode reward: [(0, '28.185')] [2023-02-23 23:40:41,143][18167] Updated weights for policy 0, policy_version 1488 (0.0013) [2023-02-23 23:40:44,143][02125] Fps is (10 sec: 4095.6, 60 sec: 3549.8, 300 sec: 3512.8). Total num frames: 6103040. Throughput: 0: 898.3. Samples: 525188. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:40:44,151][02125] Avg episode reward: [(0, '28.373')] [2023-02-23 23:40:49,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3499.0). Total num frames: 6119424. Throughput: 0: 872.4. Samples: 527280. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:40:49,144][02125] Avg episode reward: [(0, '28.543')] [2023-02-23 23:40:54,142][02125] Fps is (10 sec: 2867.5, 60 sec: 3481.6, 300 sec: 3499.0). Total num frames: 6131712. Throughput: 0: 868.3. Samples: 531412. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:40:54,148][02125] Avg episode reward: [(0, '29.588')] [2023-02-23 23:40:54,350][18167] Updated weights for policy 0, policy_version 1498 (0.0013) [2023-02-23 23:40:59,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3526.7). Total num frames: 6156288. Throughput: 0: 898.3. Samples: 537320. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:40:59,147][02125] Avg episode reward: [(0, '29.649')] [2023-02-23 23:41:03,823][18167] Updated weights for policy 0, policy_version 1508 (0.0023) [2023-02-23 23:41:04,158][02125] Fps is (10 sec: 4498.4, 60 sec: 3548.9, 300 sec: 3526.5). Total num frames: 6176768. Throughput: 0: 897.0. Samples: 540526. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:41:04,160][02125] Avg episode reward: [(0, '29.843')] [2023-02-23 23:41:09,142][02125] Fps is (10 sec: 3276.7, 60 sec: 3550.2, 300 sec: 3485.1). Total num frames: 6189056. Throughput: 0: 869.1. Samples: 545576. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:41:09,149][02125] Avg episode reward: [(0, '29.303')] [2023-02-23 23:41:14,142][02125] Fps is (10 sec: 2461.5, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6201344. Throughput: 0: 865.4. Samples: 549680. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:41:14,146][02125] Avg episode reward: [(0, '29.479')] [2023-02-23 23:41:17,360][18167] Updated weights for policy 0, policy_version 1518 (0.0038) [2023-02-23 23:41:19,142][02125] Fps is (10 sec: 3276.9, 60 sec: 3481.6, 300 sec: 3499.0). Total num frames: 6221824. Throughput: 0: 881.0. Samples: 552430. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:41:19,149][02125] Avg episode reward: [(0, '28.675')] [2023-02-23 23:41:24,142][02125] Fps is (10 sec: 4505.6, 60 sec: 3549.8, 300 sec: 3512.8). Total num frames: 6246400. Throughput: 0: 899.5. Samples: 559010. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:41:24,145][02125] Avg episode reward: [(0, '25.325')] [2023-02-23 23:41:28,820][18167] Updated weights for policy 0, policy_version 1528 (0.0024) [2023-02-23 23:41:29,143][02125] Fps is (10 sec: 3685.8, 60 sec: 3549.8, 300 sec: 3485.1). Total num frames: 6258688. Throughput: 0: 852.6. Samples: 563554. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:41:29,147][02125] Avg episode reward: [(0, '24.648')] [2023-02-23 23:41:34,145][02125] Fps is (10 sec: 2047.4, 60 sec: 3413.1, 300 sec: 3457.3). Total num frames: 6266880. Throughput: 0: 841.6. Samples: 565156. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:41:34,148][02125] Avg episode reward: [(0, '23.824')] [2023-02-23 23:41:39,142][02125] Fps is (10 sec: 2048.3, 60 sec: 3276.8, 300 sec: 3457.3). Total num frames: 6279168. Throughput: 0: 822.4. Samples: 568420. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:41:39,147][02125] Avg episode reward: [(0, '23.031')] [2023-02-23 23:41:43,822][18167] Updated weights for policy 0, policy_version 1538 (0.0014) [2023-02-23 23:41:44,142][02125] Fps is (10 sec: 3277.9, 60 sec: 3276.8, 300 sec: 3443.4). Total num frames: 6299648. Throughput: 0: 801.6. Samples: 573390. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:41:44,148][02125] Avg episode reward: [(0, '23.523')] [2023-02-23 23:41:49,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3345.1, 300 sec: 3443.4). Total num frames: 6320128. Throughput: 0: 804.1. Samples: 576696. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:41:49,150][02125] Avg episode reward: [(0, '23.994')] [2023-02-23 23:41:54,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3443.4). Total num frames: 6336512. Throughput: 0: 821.4. Samples: 582540. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:41:54,146][02125] Avg episode reward: [(0, '24.719')] [2023-02-23 23:41:54,749][18167] Updated weights for policy 0, policy_version 1548 (0.0014) [2023-02-23 23:41:59,142][02125] Fps is (10 sec: 3276.7, 60 sec: 3276.8, 300 sec: 3457.3). Total num frames: 6352896. Throughput: 0: 822.1. Samples: 586676. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:41:59,145][02125] Avg episode reward: [(0, '24.856')] [2023-02-23 23:42:04,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3209.4, 300 sec: 3443.5). Total num frames: 6369280. Throughput: 0: 806.5. Samples: 588724. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:42:04,148][02125] Avg episode reward: [(0, '25.528')] [2023-02-23 23:42:06,709][18167] Updated weights for policy 0, policy_version 1558 (0.0016) [2023-02-23 23:42:09,142][02125] Fps is (10 sec: 3686.5, 60 sec: 3345.1, 300 sec: 3443.4). Total num frames: 6389760. Throughput: 0: 803.7. Samples: 595178. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:42:09,147][02125] Avg episode reward: [(0, '25.679')] [2023-02-23 23:42:14,151][02125] Fps is (10 sec: 3685.3, 60 sec: 3413.2, 300 sec: 3443.4). Total num frames: 6406144. Throughput: 0: 827.8. Samples: 600806. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:42:14,158][02125] Avg episode reward: [(0, '25.066')] [2023-02-23 23:42:18,848][18167] Updated weights for policy 0, policy_version 1568 (0.0017) [2023-02-23 23:42:19,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3457.3). Total num frames: 6422528. Throughput: 0: 837.3. Samples: 602830. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:42:19,145][02125] Avg episode reward: [(0, '24.935')] [2023-02-23 23:42:19,159][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001568_6422528.pth... [2023-02-23 23:42:19,311][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001365_5591040.pth [2023-02-23 23:42:24,142][02125] Fps is (10 sec: 3277.8, 60 sec: 3208.6, 300 sec: 3443.4). Total num frames: 6438912. Throughput: 0: 859.0. Samples: 607076. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:42:24,144][02125] Avg episode reward: [(0, '23.356')] [2023-02-23 23:42:29,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3345.2, 300 sec: 3443.4). Total num frames: 6459392. Throughput: 0: 895.0. Samples: 613664. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:42:29,145][02125] Avg episode reward: [(0, '23.845')] [2023-02-23 23:42:29,665][18167] Updated weights for policy 0, policy_version 1578 (0.0018) [2023-02-23 23:42:34,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3550.1, 300 sec: 3443.4). Total num frames: 6479872. Throughput: 0: 891.8. Samples: 616826. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:42:34,147][02125] Avg episode reward: [(0, '23.367')] [2023-02-23 23:42:39,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 6492160. Throughput: 0: 857.5. Samples: 621126. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:42:39,146][02125] Avg episode reward: [(0, '22.717')] [2023-02-23 23:42:42,982][18167] Updated weights for policy 0, policy_version 1588 (0.0014) [2023-02-23 23:42:44,142][02125] Fps is (10 sec: 2867.1, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 6508544. Throughput: 0: 866.1. Samples: 625650. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:42:44,144][02125] Avg episode reward: [(0, '22.553')] [2023-02-23 23:42:49,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6529024. Throughput: 0: 895.2. Samples: 629008. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:42:49,153][02125] Avg episode reward: [(0, '23.354')] [2023-02-23 23:42:52,252][18167] Updated weights for policy 0, policy_version 1598 (0.0017) [2023-02-23 23:42:54,142][02125] Fps is (10 sec: 4096.1, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6549504. Throughput: 0: 896.5. Samples: 635522. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:42:54,145][02125] Avg episode reward: [(0, '23.385')] [2023-02-23 23:42:59,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6561792. Throughput: 0: 866.1. Samples: 639780. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:42:59,150][02125] Avg episode reward: [(0, '22.580')] [2023-02-23 23:43:04,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6578176. Throughput: 0: 864.3. Samples: 641722. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:43:04,146][02125] Avg episode reward: [(0, '22.189')] [2023-02-23 23:43:05,796][18167] Updated weights for policy 0, policy_version 1608 (0.0014) [2023-02-23 23:43:09,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6598656. Throughput: 0: 894.1. Samples: 647312. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:43:09,146][02125] Avg episode reward: [(0, '23.841')] [2023-02-23 23:43:14,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3550.0, 300 sec: 3485.1). Total num frames: 6619136. Throughput: 0: 893.3. Samples: 653864. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:43:14,153][02125] Avg episode reward: [(0, '23.978')] [2023-02-23 23:43:16,262][18167] Updated weights for policy 0, policy_version 1618 (0.0013) [2023-02-23 23:43:19,147][02125] Fps is (10 sec: 3684.4, 60 sec: 3549.5, 300 sec: 3498.9). Total num frames: 6635520. Throughput: 0: 868.6. Samples: 655920. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:43:19,153][02125] Avg episode reward: [(0, '22.723')] [2023-02-23 23:43:24,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6647808. Throughput: 0: 865.0. Samples: 660050. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) [2023-02-23 23:43:24,148][02125] Avg episode reward: [(0, '23.026')] [2023-02-23 23:43:28,515][18167] Updated weights for policy 0, policy_version 1628 (0.0016) [2023-02-23 23:43:29,142][02125] Fps is (10 sec: 3278.6, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6668288. Throughput: 0: 896.1. Samples: 665974. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:43:29,150][02125] Avg episode reward: [(0, '23.913')] [2023-02-23 23:43:34,142][02125] Fps is (10 sec: 4505.5, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6692864. Throughput: 0: 892.2. Samples: 669158. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) [2023-02-23 23:43:34,150][02125] Avg episode reward: [(0, '23.491')] [2023-02-23 23:43:39,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6705152. Throughput: 0: 865.3. Samples: 674460. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) [2023-02-23 23:43:39,147][02125] Avg episode reward: [(0, '22.505')] [2023-02-23 23:43:39,883][18167] Updated weights for policy 0, policy_version 1638 (0.0019) [2023-02-23 23:43:44,142][02125] Fps is (10 sec: 2457.7, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 6717440. Throughput: 0: 862.5. Samples: 678592. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:43:44,145][02125] Avg episode reward: [(0, '23.344')] [2023-02-23 23:43:49,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 6737920. Throughput: 0: 880.8. Samples: 681356. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:43:49,144][02125] Avg episode reward: [(0, '23.742')] [2023-02-23 23:43:51,160][18167] Updated weights for policy 0, policy_version 1648 (0.0027) [2023-02-23 23:43:54,142][02125] Fps is (10 sec: 4505.6, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6762496. Throughput: 0: 901.4. Samples: 687876. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:43:54,144][02125] Avg episode reward: [(0, '24.220')] [2023-02-23 23:43:59,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 6774784. Throughput: 0: 870.2. Samples: 693024. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:43:59,147][02125] Avg episode reward: [(0, '25.506')] [2023-02-23 23:44:03,881][18167] Updated weights for policy 0, policy_version 1658 (0.0013) [2023-02-23 23:44:04,143][02125] Fps is (10 sec: 2866.8, 60 sec: 3549.8, 300 sec: 3485.1). Total num frames: 6791168. Throughput: 0: 870.0. Samples: 695066. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:44:04,148][02125] Avg episode reward: [(0, '25.531')] [2023-02-23 23:44:09,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 6807552. Throughput: 0: 884.9. Samples: 699872. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:44:09,150][02125] Avg episode reward: [(0, '26.447')] [2023-02-23 23:44:14,025][18167] Updated weights for policy 0, policy_version 1668 (0.0017) [2023-02-23 23:44:14,142][02125] Fps is (10 sec: 4096.6, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6832128. Throughput: 0: 898.0. Samples: 706386. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:44:14,149][02125] Avg episode reward: [(0, '24.656')] [2023-02-23 23:44:19,142][02125] Fps is (10 sec: 4095.9, 60 sec: 3550.2, 300 sec: 3485.1). Total num frames: 6848512. Throughput: 0: 893.3. Samples: 709356. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:44:19,150][02125] Avg episode reward: [(0, '24.501')] [2023-02-23 23:44:19,161][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001672_6848512.pth... [2023-02-23 23:44:19,298][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001469_6017024.pth [2023-02-23 23:44:24,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6860800. Throughput: 0: 865.4. Samples: 713402. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:44:24,149][02125] Avg episode reward: [(0, '22.981')] [2023-02-23 23:44:27,182][18167] Updated weights for policy 0, policy_version 1678 (0.0024) [2023-02-23 23:44:29,142][02125] Fps is (10 sec: 3276.9, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6881280. Throughput: 0: 884.5. Samples: 718396. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:44:29,152][02125] Avg episode reward: [(0, '23.627')] [2023-02-23 23:44:34,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6901760. Throughput: 0: 895.3. Samples: 721646. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:44:34,149][02125] Avg episode reward: [(0, '23.696')] [2023-02-23 23:44:36,917][18167] Updated weights for policy 0, policy_version 1688 (0.0017) [2023-02-23 23:44:39,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6918144. Throughput: 0: 886.9. Samples: 727786. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:44:39,148][02125] Avg episode reward: [(0, '23.315')] [2023-02-23 23:44:44,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 6930432. Throughput: 0: 862.6. Samples: 731842. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:44:44,144][02125] Avg episode reward: [(0, '24.187')] [2023-02-23 23:44:49,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 6950912. Throughput: 0: 864.2. Samples: 733954. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-23 23:44:49,145][02125] Avg episode reward: [(0, '25.501')] [2023-02-23 23:44:49,948][18167] Updated weights for policy 0, policy_version 1698 (0.0013) [2023-02-23 23:44:54,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3485.1). Total num frames: 6971392. Throughput: 0: 896.9. Samples: 740232. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:44:54,145][02125] Avg episode reward: [(0, '27.295')] [2023-02-23 23:44:59,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 6987776. Throughput: 0: 883.0. Samples: 746120. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-23 23:44:59,146][02125] Avg episode reward: [(0, '27.388')] [2023-02-23 23:45:01,268][18167] Updated weights for policy 0, policy_version 1708 (0.0024) [2023-02-23 23:45:04,144][02125] Fps is (10 sec: 2866.6, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7000064. Throughput: 0: 861.7. Samples: 748132. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:45:04,149][02125] Avg episode reward: [(0, '27.119')] [2023-02-23 23:45:09,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7016448. Throughput: 0: 864.8. Samples: 752316. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:45:09,147][02125] Avg episode reward: [(0, '26.671')] [2023-02-23 23:45:14,144][02125] Fps is (10 sec: 3276.7, 60 sec: 3344.9, 300 sec: 3457.3). Total num frames: 7032832. Throughput: 0: 855.9. Samples: 756912. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:45:14,149][02125] Avg episode reward: [(0, '27.533')] [2023-02-23 23:45:15,593][18167] Updated weights for policy 0, policy_version 1718 (0.0051) [2023-02-23 23:45:19,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3429.5). Total num frames: 7045120. Throughput: 0: 827.1. Samples: 758866. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:45:19,150][02125] Avg episode reward: [(0, '27.787')] [2023-02-23 23:45:24,147][02125] Fps is (10 sec: 2457.0, 60 sec: 3276.5, 300 sec: 3429.5). Total num frames: 7057408. Throughput: 0: 769.9. Samples: 762434. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:45:24,152][02125] Avg episode reward: [(0, '28.811')] [2023-02-23 23:45:29,142][02125] Fps is (10 sec: 2457.6, 60 sec: 3140.3, 300 sec: 3415.6). Total num frames: 7069696. Throughput: 0: 772.4. Samples: 766600. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:45:29,144][02125] Avg episode reward: [(0, '28.328')] [2023-02-23 23:45:30,377][18167] Updated weights for policy 0, policy_version 1728 (0.0034) [2023-02-23 23:45:34,142][02125] Fps is (10 sec: 3688.2, 60 sec: 3208.5, 300 sec: 3429.5). Total num frames: 7094272. Throughput: 0: 794.8. Samples: 769718. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:45:34,144][02125] Avg episode reward: [(0, '27.828')] [2023-02-23 23:45:39,147][02125] Fps is (10 sec: 4503.4, 60 sec: 3276.5, 300 sec: 3429.5). Total num frames: 7114752. Throughput: 0: 802.6. Samples: 776354. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:45:39,151][02125] Avg episode reward: [(0, '28.132')] [2023-02-23 23:45:39,842][18167] Updated weights for policy 0, policy_version 1738 (0.0012) [2023-02-23 23:45:44,144][02125] Fps is (10 sec: 3276.2, 60 sec: 3276.7, 300 sec: 3415.6). Total num frames: 7127040. Throughput: 0: 772.9. Samples: 780900. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:45:44,154][02125] Avg episode reward: [(0, '28.917')] [2023-02-23 23:45:49,142][02125] Fps is (10 sec: 2868.6, 60 sec: 3208.5, 300 sec: 3429.5). Total num frames: 7143424. Throughput: 0: 774.9. Samples: 783002. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:45:49,147][02125] Avg episode reward: [(0, '29.690')] [2023-02-23 23:45:52,956][18167] Updated weights for policy 0, policy_version 1748 (0.0030) [2023-02-23 23:45:54,142][02125] Fps is (10 sec: 3687.1, 60 sec: 3208.5, 300 sec: 3415.6). Total num frames: 7163904. Throughput: 0: 799.5. Samples: 788292. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:45:54,144][02125] Avg episode reward: [(0, '28.905')] [2023-02-23 23:45:59,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3276.8, 300 sec: 3415.8). Total num frames: 7184384. Throughput: 0: 843.9. Samples: 794884. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:45:59,147][02125] Avg episode reward: [(0, '28.340')] [2023-02-23 23:46:04,016][18167] Updated weights for policy 0, policy_version 1758 (0.0014) [2023-02-23 23:46:04,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3345.2, 300 sec: 3429.5). Total num frames: 7200768. Throughput: 0: 856.5. Samples: 797410. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:46:04,148][02125] Avg episode reward: [(0, '27.558')] [2023-02-23 23:46:09,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3429.5). Total num frames: 7213056. Throughput: 0: 869.8. Samples: 801570. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:46:09,149][02125] Avg episode reward: [(0, '28.168')] [2023-02-23 23:46:14,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.2, 300 sec: 3429.5). Total num frames: 7233536. Throughput: 0: 889.9. Samples: 806646. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:46:14,149][02125] Avg episode reward: [(0, '27.335')] [2023-02-23 23:46:15,896][18167] Updated weights for policy 0, policy_version 1768 (0.0020) [2023-02-23 23:46:19,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3415.7). Total num frames: 7254016. Throughput: 0: 894.8. Samples: 809986. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:46:19,144][02125] Avg episode reward: [(0, '28.219')] [2023-02-23 23:46:19,162][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001771_7254016.pth... [2023-02-23 23:46:19,291][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001568_6422528.pth [2023-02-23 23:46:24,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3550.2, 300 sec: 3429.6). Total num frames: 7270400. Throughput: 0: 875.1. Samples: 815730. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:46:24,154][02125] Avg episode reward: [(0, '28.750')] [2023-02-23 23:46:28,080][18167] Updated weights for policy 0, policy_version 1778 (0.0020) [2023-02-23 23:46:29,144][02125] Fps is (10 sec: 2866.5, 60 sec: 3549.7, 300 sec: 3443.4). Total num frames: 7282688. Throughput: 0: 863.4. Samples: 819754. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:46:29,149][02125] Avg episode reward: [(0, '26.749')] [2023-02-23 23:46:34,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7303168. Throughput: 0: 865.6. Samples: 821956. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:46:34,145][02125] Avg episode reward: [(0, '26.574')] [2023-02-23 23:46:38,649][18167] Updated weights for policy 0, policy_version 1788 (0.0031) [2023-02-23 23:46:39,142][02125] Fps is (10 sec: 4097.0, 60 sec: 3481.9, 300 sec: 3471.2). Total num frames: 7323648. Throughput: 0: 892.8. Samples: 828468. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:46:39,145][02125] Avg episode reward: [(0, '26.833')] [2023-02-23 23:46:44,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3550.0, 300 sec: 3457.3). Total num frames: 7340032. Throughput: 0: 870.8. Samples: 834070. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:46:44,146][02125] Avg episode reward: [(0, '26.691')] [2023-02-23 23:46:49,143][02125] Fps is (10 sec: 2866.8, 60 sec: 3481.5, 300 sec: 3443.4). Total num frames: 7352320. Throughput: 0: 861.1. Samples: 836160. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:46:49,146][02125] Avg episode reward: [(0, '24.689')] [2023-02-23 23:46:51,938][18167] Updated weights for policy 0, policy_version 1798 (0.0021) [2023-02-23 23:46:54,142][02125] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 7372800. Throughput: 0: 867.9. Samples: 840624. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:46:54,150][02125] Avg episode reward: [(0, '25.263')] [2023-02-23 23:46:59,142][02125] Fps is (10 sec: 4096.6, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7393280. Throughput: 0: 904.0. Samples: 847328. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) [2023-02-23 23:46:59,150][02125] Avg episode reward: [(0, '25.067')] [2023-02-23 23:47:01,254][18167] Updated weights for policy 0, policy_version 1808 (0.0021) [2023-02-23 23:47:04,142][02125] Fps is (10 sec: 4096.2, 60 sec: 3549.9, 300 sec: 3471.2). Total num frames: 7413760. Throughput: 0: 901.8. Samples: 850566. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:47:04,146][02125] Avg episode reward: [(0, '24.257')] [2023-02-23 23:47:09,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3457.3). Total num frames: 7426048. Throughput: 0: 869.5. Samples: 854856. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:47:09,149][02125] Avg episode reward: [(0, '24.200')] [2023-02-23 23:47:14,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 7442432. Throughput: 0: 878.9. Samples: 859304. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:47:14,144][02125] Avg episode reward: [(0, '23.836')] [2023-02-23 23:47:14,601][18167] Updated weights for policy 0, policy_version 1818 (0.0025) [2023-02-23 23:47:19,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7462912. Throughput: 0: 901.3. Samples: 862514. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:47:19,149][02125] Avg episode reward: [(0, '25.155')] [2023-02-23 23:47:24,144][02125] Fps is (10 sec: 4095.0, 60 sec: 3549.7, 300 sec: 3471.2). Total num frames: 7483392. Throughput: 0: 896.9. Samples: 868830. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:47:24,147][02125] Avg episode reward: [(0, '24.986')] [2023-02-23 23:47:25,274][18167] Updated weights for policy 0, policy_version 1828 (0.0018) [2023-02-23 23:47:29,142][02125] Fps is (10 sec: 3276.7, 60 sec: 3550.0, 300 sec: 3443.4). Total num frames: 7495680. Throughput: 0: 864.6. Samples: 872978. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:47:29,148][02125] Avg episode reward: [(0, '25.901')] [2023-02-23 23:47:34,142][02125] Fps is (10 sec: 2867.9, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 7512064. Throughput: 0: 863.2. Samples: 875004. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:47:34,149][02125] Avg episode reward: [(0, '25.730')] [2023-02-23 23:47:37,577][18167] Updated weights for policy 0, policy_version 1838 (0.0038) [2023-02-23 23:47:39,142][02125] Fps is (10 sec: 3686.5, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7532544. Throughput: 0: 894.8. Samples: 880890. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:47:39,149][02125] Avg episode reward: [(0, '26.022')] [2023-02-23 23:47:44,143][02125] Fps is (10 sec: 4095.5, 60 sec: 3549.8, 300 sec: 3471.2). Total num frames: 7553024. Throughput: 0: 888.4. Samples: 887308. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:47:44,153][02125] Avg episode reward: [(0, '26.658')] [2023-02-23 23:47:49,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3550.0, 300 sec: 3443.4). Total num frames: 7565312. Throughput: 0: 862.0. Samples: 889358. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:47:49,147][02125] Avg episode reward: [(0, '26.156')] [2023-02-23 23:47:49,344][18167] Updated weights for policy 0, policy_version 1848 (0.0014) [2023-02-23 23:47:54,142][02125] Fps is (10 sec: 2867.6, 60 sec: 3481.6, 300 sec: 3457.3). Total num frames: 7581696. Throughput: 0: 857.7. Samples: 893452. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:47:54,147][02125] Avg episode reward: [(0, '25.006')] [2023-02-23 23:47:59,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7602176. Throughput: 0: 886.5. Samples: 899198. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:47:59,149][02125] Avg episode reward: [(0, '25.091')] [2023-02-23 23:48:00,645][18167] Updated weights for policy 0, policy_version 1858 (0.0013) [2023-02-23 23:48:04,142][02125] Fps is (10 sec: 4096.0, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7622656. Throughput: 0: 886.8. Samples: 902418. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:48:04,148][02125] Avg episode reward: [(0, '24.985')] [2023-02-23 23:48:09,142][02125] Fps is (10 sec: 3686.3, 60 sec: 3549.8, 300 sec: 3457.3). Total num frames: 7639040. Throughput: 0: 863.5. Samples: 907684. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:48:09,152][02125] Avg episode reward: [(0, '25.321')] [2023-02-23 23:48:13,875][18167] Updated weights for policy 0, policy_version 1868 (0.0018) [2023-02-23 23:48:14,142][02125] Fps is (10 sec: 2867.1, 60 sec: 3481.6, 300 sec: 3443.5). Total num frames: 7651328. Throughput: 0: 860.4. Samples: 911698. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-23 23:48:14,149][02125] Avg episode reward: [(0, '25.613')] [2023-02-23 23:48:19,142][02125] Fps is (10 sec: 3277.0, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7671808. Throughput: 0: 874.0. Samples: 914336. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:48:19,150][02125] Avg episode reward: [(0, '24.470')] [2023-02-23 23:48:19,160][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001873_7671808.pth... [2023-02-23 23:48:19,321][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001672_6848512.pth [2023-02-23 23:48:23,522][18167] Updated weights for policy 0, policy_version 1878 (0.0023) [2023-02-23 23:48:24,142][02125] Fps is (10 sec: 4096.1, 60 sec: 3481.7, 300 sec: 3471.2). Total num frames: 7692288. Throughput: 0: 887.5. Samples: 920826. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:48:24,144][02125] Avg episode reward: [(0, '23.913')] [2023-02-23 23:48:29,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 7708672. Throughput: 0: 861.9. Samples: 926094. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:48:29,146][02125] Avg episode reward: [(0, '24.570')] [2023-02-23 23:48:34,143][02125] Fps is (10 sec: 2866.8, 60 sec: 3481.5, 300 sec: 3443.4). Total num frames: 7720960. Throughput: 0: 861.7. Samples: 928136. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-23 23:48:34,147][02125] Avg episode reward: [(0, '24.651')] [2023-02-23 23:48:36,683][18167] Updated weights for policy 0, policy_version 1888 (0.0036) [2023-02-23 23:48:39,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3471.2). Total num frames: 7741440. Throughput: 0: 880.7. Samples: 933084. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:48:39,148][02125] Avg episode reward: [(0, '25.701')] [2023-02-23 23:48:44,142][02125] Fps is (10 sec: 4506.3, 60 sec: 3549.9, 300 sec: 3485.1). Total num frames: 7766016. Throughput: 0: 900.0. Samples: 939696. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:48:44,149][02125] Avg episode reward: [(0, '25.026')] [2023-02-23 23:48:46,278][18167] Updated weights for policy 0, policy_version 1898 (0.0013) [2023-02-23 23:48:49,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3443.4). Total num frames: 7778304. Throughput: 0: 893.2. Samples: 942610. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:48:49,149][02125] Avg episode reward: [(0, '25.741')] [2023-02-23 23:48:54,143][02125] Fps is (10 sec: 2866.8, 60 sec: 3549.8, 300 sec: 3457.3). Total num frames: 7794688. Throughput: 0: 867.0. Samples: 946700. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:48:54,148][02125] Avg episode reward: [(0, '26.279')] [2023-02-23 23:48:59,142][02125] Fps is (10 sec: 2457.6, 60 sec: 3345.1, 300 sec: 3429.5). Total num frames: 7802880. Throughput: 0: 851.8. Samples: 950028. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) [2023-02-23 23:48:59,148][02125] Avg episode reward: [(0, '26.497')] [2023-02-23 23:49:02,532][18167] Updated weights for policy 0, policy_version 1908 (0.0022) [2023-02-23 23:49:04,142][02125] Fps is (10 sec: 2458.0, 60 sec: 3276.8, 300 sec: 3429.5). Total num frames: 7819264. Throughput: 0: 836.4. Samples: 951976. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:49:04,145][02125] Avg episode reward: [(0, '26.984')] [2023-02-23 23:49:09,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3401.8). Total num frames: 7835648. Throughput: 0: 811.1. Samples: 957324. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:49:09,149][02125] Avg episode reward: [(0, '26.728')] [2023-02-23 23:49:14,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3401.8). Total num frames: 7852032. Throughput: 0: 794.0. Samples: 961822. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:49:14,150][02125] Avg episode reward: [(0, '26.015')] [2023-02-23 23:49:15,319][18167] Updated weights for policy 0, policy_version 1918 (0.0014) [2023-02-23 23:49:19,142][02125] Fps is (10 sec: 2867.2, 60 sec: 3208.5, 300 sec: 3401.8). Total num frames: 7864320. Throughput: 0: 793.5. Samples: 963842. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:49:19,149][02125] Avg episode reward: [(0, '26.400')] [2023-02-23 23:49:24,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3276.8, 300 sec: 3415.6). Total num frames: 7888896. Throughput: 0: 806.1. Samples: 969360. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:49:24,151][02125] Avg episode reward: [(0, '27.497')] [2023-02-23 23:49:25,958][18167] Updated weights for policy 0, policy_version 1928 (0.0021) [2023-02-23 23:49:29,142][02125] Fps is (10 sec: 4505.3, 60 sec: 3345.0, 300 sec: 3415.6). Total num frames: 7909376. Throughput: 0: 806.4. Samples: 975984. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-02-23 23:49:29,151][02125] Avg episode reward: [(0, '26.715')] [2023-02-23 23:49:34,142][02125] Fps is (10 sec: 3276.8, 60 sec: 3345.2, 300 sec: 3401.8). Total num frames: 7921664. Throughput: 0: 793.7. Samples: 978326. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-02-23 23:49:34,148][02125] Avg episode reward: [(0, '27.344')] [2023-02-23 23:49:38,991][18167] Updated weights for policy 0, policy_version 1938 (0.0017) [2023-02-23 23:49:39,142][02125] Fps is (10 sec: 2867.4, 60 sec: 3276.8, 300 sec: 3415.6). Total num frames: 7938048. Throughput: 0: 792.6. Samples: 982364. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) [2023-02-23 23:49:39,151][02125] Avg episode reward: [(0, '27.386')] [2023-02-23 23:49:44,142][02125] Fps is (10 sec: 3686.4, 60 sec: 3208.5, 300 sec: 3415.6). Total num frames: 7958528. Throughput: 0: 841.2. Samples: 987880. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:49:44,145][02125] Avg episode reward: [(0, '26.435')] [2023-02-23 23:49:48,847][18167] Updated weights for policy 0, policy_version 1948 (0.0028) [2023-02-23 23:49:49,142][02125] Fps is (10 sec: 4096.1, 60 sec: 3345.1, 300 sec: 3415.6). Total num frames: 7979008. Throughput: 0: 869.1. Samples: 991086. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:49:49,143][02125] Avg episode reward: [(0, '27.340')] [2023-02-23 23:49:54,142][02125] Fps is (10 sec: 3276.6, 60 sec: 3276.9, 300 sec: 3401.8). Total num frames: 7991296. Throughput: 0: 874.3. Samples: 996668. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-23 23:49:54,151][02125] Avg episode reward: [(0, '25.456')] [2023-02-23 23:49:58,667][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... [2023-02-23 23:49:58,676][18153] Stopping Batcher_0... [2023-02-23 23:49:58,676][18153] Loop batcher_evt_loop terminating... [2023-02-23 23:49:58,677][02125] Component Batcher_0 stopped! [2023-02-23 23:49:58,742][18173] Stopping RolloutWorker_w6... [2023-02-23 23:49:58,742][18173] Loop rollout_proc6_evt_loop terminating... [2023-02-23 23:49:58,744][02125] Component RolloutWorker_w6 stopped! [2023-02-23 23:49:58,757][02125] Component RolloutWorker_w0 stopped! [2023-02-23 23:49:58,757][18168] Stopping RolloutWorker_w0... [2023-02-23 23:49:58,765][18168] Loop rollout_proc0_evt_loop terminating... [2023-02-23 23:49:58,790][18167] Weights refcount: 2 0 [2023-02-23 23:49:58,792][02125] Component InferenceWorker_p0-w0 stopped! [2023-02-23 23:49:58,794][18167] Stopping InferenceWorker_p0-w0... [2023-02-23 23:49:58,797][02125] Component RolloutWorker_w1 stopped! [2023-02-23 23:49:58,809][18167] Loop inference_proc0-0_evt_loop terminating... [2023-02-23 23:49:58,809][18170] Stopping RolloutWorker_w2... [2023-02-23 23:49:58,809][02125] Component RolloutWorker_w2 stopped! [2023-02-23 23:49:58,800][18169] Stopping RolloutWorker_w1... [2023-02-23 23:49:58,813][18169] Loop rollout_proc1_evt_loop terminating... [2023-02-23 23:49:58,817][18179] Stopping RolloutWorker_w4... [2023-02-23 23:49:58,818][18179] Loop rollout_proc4_evt_loop terminating... [2023-02-23 23:49:58,817][02125] Component RolloutWorker_w4 stopped! [2023-02-23 23:49:58,810][18170] Loop rollout_proc2_evt_loop terminating... [2023-02-23 23:49:58,851][02125] Component RolloutWorker_w5 stopped! [2023-02-23 23:49:58,854][18172] Stopping RolloutWorker_w5... [2023-02-23 23:49:58,854][18172] Loop rollout_proc5_evt_loop terminating... [2023-02-23 23:49:58,869][02125] Component RolloutWorker_w7 stopped! [2023-02-23 23:49:58,874][18178] Stopping RolloutWorker_w7... [2023-02-23 23:49:58,876][18171] Stopping RolloutWorker_w3... [2023-02-23 23:49:58,876][18171] Loop rollout_proc3_evt_loop terminating... [2023-02-23 23:49:58,877][02125] Component RolloutWorker_w3 stopped! [2023-02-23 23:49:58,888][18178] Loop rollout_proc7_evt_loop terminating... [2023-02-23 23:49:58,946][18153] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001771_7254016.pth [2023-02-23 23:49:58,967][18153] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... [2023-02-23 23:49:59,310][18153] Stopping LearnerWorker_p0... [2023-02-23 23:49:59,311][18153] Loop learner_proc0_evt_loop terminating... [2023-02-23 23:49:59,310][02125] Component LearnerWorker_p0 stopped! [2023-02-23 23:49:59,314][02125] Waiting for process learner_proc0 to stop... [2023-02-23 23:50:01,601][02125] Waiting for process inference_proc0-0 to join... [2023-02-23 23:50:01,604][02125] Waiting for process rollout_proc0 to join... [2023-02-23 23:50:02,142][02125] Waiting for process rollout_proc1 to join... [2023-02-23 23:50:02,144][02125] Waiting for process rollout_proc2 to join... [2023-02-23 23:50:02,152][02125] Waiting for process rollout_proc3 to join... [2023-02-23 23:50:02,156][02125] Waiting for process rollout_proc4 to join... [2023-02-23 23:50:02,157][02125] Waiting for process rollout_proc5 to join... [2023-02-23 23:50:02,158][02125] Waiting for process rollout_proc6 to join... [2023-02-23 23:50:02,159][02125] Waiting for process rollout_proc7 to join... [2023-02-23 23:50:02,160][02125] Batcher 0 profile tree view: batching: 27.1303, releasing_batches: 0.0263 [2023-02-23 23:50:02,162][02125] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0000 wait_policy_total: 549.4836 update_model: 8.3087 weight_update: 0.0039 one_step: 0.0091 handle_policy_step: 558.9114 deserialize: 15.8802, stack: 3.3282, obs_to_device_normalize: 122.2237, forward: 271.4568, send_messages: 28.2144 prepare_outputs: 89.2860 to_cpu: 55.8985 [2023-02-23 23:50:02,163][02125] Learner 0 profile tree view: misc: 0.0069, prepare_batch: 18.0709 train: 80.3500 epoch_init: 0.0219, minibatch_init: 0.0110, losses_postprocess: 0.5556, kl_divergence: 0.5831, after_optimizer: 3.6153 calculate_losses: 26.6007 losses_init: 0.0114, forward_head: 1.9399, bptt_initial: 17.1480, tail: 1.1973, advantages_returns: 0.3688, losses: 3.3331 bptt: 2.2368 bptt_forward_core: 2.1367 update: 48.3058 clip: 1.4333 [2023-02-23 23:50:02,165][02125] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.3886, enqueue_policy_requests: 155.7346, env_step: 865.9364, overhead: 22.6093, complete_rollouts: 7.6219 save_policy_outputs: 22.7169 split_output_tensors: 10.9917 [2023-02-23 23:50:02,166][02125] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.3749, enqueue_policy_requests: 153.8391, env_step: 870.7873, overhead: 23.5506, complete_rollouts: 7.9157 save_policy_outputs: 22.3478 split_output_tensors: 10.9760 [2023-02-23 23:50:02,168][02125] Loop Runner_EvtLoop terminating... [2023-02-23 23:50:02,169][02125] Runner profile tree view: main_loop: 1180.8023 [2023-02-23 23:50:02,171][02125] Collected {0: 8007680}, FPS: 3389.0 [2023-02-23 23:50:02,285][02125] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-02-23 23:50:02,287][02125] Overriding arg 'num_workers' with value 1 passed from command line [2023-02-23 23:50:02,289][02125] Adding new argument 'no_render'=True that is not in the saved config file! [2023-02-23 23:50:02,291][02125] Adding new argument 'save_video'=True that is not in the saved config file! [2023-02-23 23:50:02,294][02125] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-02-23 23:50:02,296][02125] Adding new argument 'video_name'=None that is not in the saved config file! [2023-02-23 23:50:02,298][02125] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2023-02-23 23:50:02,299][02125] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-02-23 23:50:02,300][02125] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2023-02-23 23:50:02,301][02125] Adding new argument 'hf_repository'=None that is not in the saved config file! [2023-02-23 23:50:02,303][02125] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-02-23 23:50:02,304][02125] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-02-23 23:50:02,305][02125] Adding new argument 'train_script'=None that is not in the saved config file! [2023-02-23 23:50:02,306][02125] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-02-23 23:50:02,307][02125] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-02-23 23:50:02,334][02125] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-23 23:50:02,336][02125] RunningMeanStd input shape: (3, 72, 128) [2023-02-23 23:50:02,339][02125] RunningMeanStd input shape: (1,) [2023-02-23 23:50:02,356][02125] ConvEncoder: input_channels=3 [2023-02-23 23:50:03,056][02125] Conv encoder output size: 512 [2023-02-23 23:50:03,059][02125] Policy head output size: 512 [2023-02-23 23:50:05,338][02125] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... [2023-02-23 23:50:06,622][02125] Num frames 100... [2023-02-23 23:50:06,739][02125] Num frames 200... [2023-02-23 23:50:06,854][02125] Num frames 300... [2023-02-23 23:50:07,003][02125] Avg episode rewards: #0: 7.840, true rewards: #0: 3.840 [2023-02-23 23:50:07,006][02125] Avg episode reward: 7.840, avg true_objective: 3.840 [2023-02-23 23:50:07,028][02125] Num frames 400... [2023-02-23 23:50:07,139][02125] Num frames 500... [2023-02-23 23:50:07,253][02125] Num frames 600... [2023-02-23 23:50:07,369][02125] Num frames 700... [2023-02-23 23:50:07,490][02125] Num frames 800... [2023-02-23 23:50:07,616][02125] Avg episode rewards: #0: 7.820, true rewards: #0: 4.320 [2023-02-23 23:50:07,619][02125] Avg episode reward: 7.820, avg true_objective: 4.320 [2023-02-23 23:50:07,666][02125] Num frames 900... [2023-02-23 23:50:07,784][02125] Num frames 1000... [2023-02-23 23:50:07,907][02125] Num frames 1100... [2023-02-23 23:50:08,028][02125] Num frames 1200... [2023-02-23 23:50:08,139][02125] Num frames 1300... [2023-02-23 23:50:08,257][02125] Num frames 1400... [2023-02-23 23:50:08,372][02125] Num frames 1500... [2023-02-23 23:50:08,491][02125] Num frames 1600... [2023-02-23 23:50:08,614][02125] Num frames 1700... [2023-02-23 23:50:08,729][02125] Num frames 1800... [2023-02-23 23:50:08,847][02125] Num frames 1900... [2023-02-23 23:50:08,970][02125] Num frames 2000... [2023-02-23 23:50:09,085][02125] Num frames 2100... [2023-02-23 23:50:09,204][02125] Num frames 2200... [2023-02-23 23:50:09,320][02125] Num frames 2300... [2023-02-23 23:50:09,436][02125] Num frames 2400... [2023-02-23 23:50:09,548][02125] Num frames 2500... [2023-02-23 23:50:09,675][02125] Num frames 2600... [2023-02-23 23:50:09,790][02125] Num frames 2700... [2023-02-23 23:50:09,909][02125] Num frames 2800... [2023-02-23 23:50:10,029][02125] Num frames 2900... [2023-02-23 23:50:10,157][02125] Avg episode rewards: #0: 24.213, true rewards: #0: 9.880 [2023-02-23 23:50:10,160][02125] Avg episode reward: 24.213, avg true_objective: 9.880 [2023-02-23 23:50:10,212][02125] Num frames 3000... [2023-02-23 23:50:10,367][02125] Num frames 3100... [2023-02-23 23:50:10,529][02125] Num frames 3200... [2023-02-23 23:50:10,705][02125] Num frames 3300... [2023-02-23 23:50:10,865][02125] Num frames 3400... [2023-02-23 23:50:10,994][02125] Avg episode rewards: #0: 20.110, true rewards: #0: 8.610 [2023-02-23 23:50:10,996][02125] Avg episode reward: 20.110, avg true_objective: 8.610 [2023-02-23 23:50:11,092][02125] Num frames 3500... [2023-02-23 23:50:11,252][02125] Num frames 3600... [2023-02-23 23:50:11,409][02125] Num frames 3700... [2023-02-23 23:50:11,567][02125] Num frames 3800... [2023-02-23 23:50:11,723][02125] Num frames 3900... [2023-02-23 23:50:11,886][02125] Num frames 4000... [2023-02-23 23:50:12,047][02125] Num frames 4100... [2023-02-23 23:50:12,184][02125] Avg episode rewards: #0: 18.896, true rewards: #0: 8.296 [2023-02-23 23:50:12,186][02125] Avg episode reward: 18.896, avg true_objective: 8.296 [2023-02-23 23:50:12,284][02125] Num frames 4200... [2023-02-23 23:50:12,454][02125] Num frames 4300... [2023-02-23 23:50:12,615][02125] Num frames 4400... [2023-02-23 23:50:12,782][02125] Num frames 4500... [2023-02-23 23:50:12,951][02125] Num frames 4600... [2023-02-23 23:50:13,117][02125] Num frames 4700... [2023-02-23 23:50:13,283][02125] Num frames 4800... [2023-02-23 23:50:13,453][02125] Num frames 4900... [2023-02-23 23:50:13,628][02125] Num frames 5000... [2023-02-23 23:50:13,764][02125] Num frames 5100... [2023-02-23 23:50:13,830][02125] Avg episode rewards: #0: 19.847, true rewards: #0: 8.513 [2023-02-23 23:50:13,831][02125] Avg episode reward: 19.847, avg true_objective: 8.513 [2023-02-23 23:50:13,947][02125] Num frames 5200... [2023-02-23 23:50:14,061][02125] Num frames 5300... [2023-02-23 23:50:14,174][02125] Num frames 5400... [2023-02-23 23:50:14,304][02125] Avg episode rewards: #0: 18.093, true rewards: #0: 7.807 [2023-02-23 23:50:14,306][02125] Avg episode reward: 18.093, avg true_objective: 7.807 [2023-02-23 23:50:14,353][02125] Num frames 5500... [2023-02-23 23:50:14,481][02125] Num frames 5600... [2023-02-23 23:50:14,605][02125] Num frames 5700... [2023-02-23 23:50:14,727][02125] Num frames 5800... [2023-02-23 23:50:14,856][02125] Num frames 5900... [2023-02-23 23:50:14,977][02125] Num frames 6000... [2023-02-23 23:50:15,085][02125] Avg episode rewards: #0: 17.426, true rewards: #0: 7.551 [2023-02-23 23:50:15,086][02125] Avg episode reward: 17.426, avg true_objective: 7.551 [2023-02-23 23:50:15,166][02125] Num frames 6100... [2023-02-23 23:50:15,292][02125] Num frames 6200... [2023-02-23 23:50:15,412][02125] Num frames 6300... [2023-02-23 23:50:15,531][02125] Num frames 6400... [2023-02-23 23:50:15,652][02125] Num frames 6500... [2023-02-23 23:50:15,766][02125] Num frames 6600... [2023-02-23 23:50:15,893][02125] Num frames 6700... [2023-02-23 23:50:16,011][02125] Num frames 6800... [2023-02-23 23:50:16,132][02125] Num frames 6900... [2023-02-23 23:50:16,245][02125] Num frames 7000... [2023-02-23 23:50:16,358][02125] Num frames 7100... [2023-02-23 23:50:16,476][02125] Num frames 7200... [2023-02-23 23:50:16,589][02125] Num frames 7300... [2023-02-23 23:50:16,708][02125] Num frames 7400... [2023-02-23 23:50:16,819][02125] Num frames 7500... [2023-02-23 23:50:16,941][02125] Num frames 7600... [2023-02-23 23:50:17,056][02125] Num frames 7700... [2023-02-23 23:50:17,174][02125] Num frames 7800... [2023-02-23 23:50:17,292][02125] Num frames 7900... [2023-02-23 23:50:17,405][02125] Num frames 8000... [2023-02-23 23:50:17,492][02125] Avg episode rewards: #0: 21.472, true rewards: #0: 8.917 [2023-02-23 23:50:17,495][02125] Avg episode reward: 21.472, avg true_objective: 8.917 [2023-02-23 23:50:17,591][02125] Num frames 8100... [2023-02-23 23:50:17,720][02125] Num frames 8200... [2023-02-23 23:50:17,843][02125] Num frames 8300... [2023-02-23 23:50:17,970][02125] Num frames 8400... [2023-02-23 23:50:18,083][02125] Num frames 8500... [2023-02-23 23:50:18,202][02125] Num frames 8600... [2023-02-23 23:50:18,324][02125] Num frames 8700... [2023-02-23 23:50:18,442][02125] Num frames 8800... [2023-02-23 23:50:18,557][02125] Num frames 8900... [2023-02-23 23:50:18,671][02125] Num frames 9000... [2023-02-23 23:50:18,784][02125] Num frames 9100... [2023-02-23 23:50:18,908][02125] Num frames 9200... [2023-02-23 23:50:19,038][02125] Num frames 9300... [2023-02-23 23:50:19,164][02125] Num frames 9400... [2023-02-23 23:50:19,285][02125] Avg episode rewards: #0: 22.859, true rewards: #0: 9.459 [2023-02-23 23:50:19,288][02125] Avg episode reward: 22.859, avg true_objective: 9.459 [2023-02-23 23:51:19,564][02125] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2023-02-23 23:55:36,771][02125] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-02-23 23:55:36,782][02125] Overriding arg 'num_workers' with value 1 passed from command line [2023-02-23 23:55:36,789][02125] Adding new argument 'no_render'=True that is not in the saved config file! [2023-02-23 23:55:36,795][02125] Adding new argument 'save_video'=True that is not in the saved config file! [2023-02-23 23:55:36,798][02125] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-02-23 23:55:36,806][02125] Adding new argument 'video_name'=None that is not in the saved config file! [2023-02-23 23:55:36,809][02125] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2023-02-23 23:55:36,815][02125] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-02-23 23:55:36,817][02125] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2023-02-23 23:55:36,823][02125] Adding new argument 'hf_repository'='dn-gh/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2023-02-23 23:55:36,824][02125] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-02-23 23:55:36,825][02125] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-02-23 23:55:36,827][02125] Adding new argument 'train_script'=None that is not in the saved config file! [2023-02-23 23:55:36,829][02125] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-02-23 23:55:36,831][02125] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-02-23 23:55:36,870][02125] RunningMeanStd input shape: (3, 72, 128) [2023-02-23 23:55:36,872][02125] RunningMeanStd input shape: (1,) [2023-02-23 23:55:36,886][02125] ConvEncoder: input_channels=3 [2023-02-23 23:55:36,924][02125] Conv encoder output size: 512 [2023-02-23 23:55:36,925][02125] Policy head output size: 512 [2023-02-23 23:55:36,945][02125] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... [2023-02-23 23:55:37,392][02125] Num frames 100... [2023-02-23 23:55:37,505][02125] Num frames 200... [2023-02-23 23:55:37,624][02125] Num frames 300... [2023-02-23 23:55:37,743][02125] Avg episode rewards: #0: 7.520, true rewards: #0: 3.520 [2023-02-23 23:55:37,745][02125] Avg episode reward: 7.520, avg true_objective: 3.520 [2023-02-23 23:55:37,803][02125] Num frames 400... [2023-02-23 23:55:37,913][02125] Num frames 500... [2023-02-23 23:55:38,026][02125] Num frames 600... [2023-02-23 23:55:38,143][02125] Num frames 700... [2023-02-23 23:55:38,260][02125] Num frames 800... [2023-02-23 23:55:38,373][02125] Num frames 900... [2023-02-23 23:55:38,486][02125] Num frames 1000... [2023-02-23 23:55:38,599][02125] Num frames 1100... [2023-02-23 23:55:38,716][02125] Num frames 1200... [2023-02-23 23:55:38,838][02125] Num frames 1300... [2023-02-23 23:55:38,959][02125] Num frames 1400... [2023-02-23 23:55:39,063][02125] Avg episode rewards: #0: 14.710, true rewards: #0: 7.210 [2023-02-23 23:55:39,067][02125] Avg episode reward: 14.710, avg true_objective: 7.210 [2023-02-23 23:55:39,143][02125] Num frames 1500... [2023-02-23 23:55:39,257][02125] Num frames 1600... [2023-02-23 23:55:39,373][02125] Num frames 1700... [2023-02-23 23:55:39,492][02125] Num frames 1800... [2023-02-23 23:55:39,605][02125] Num frames 1900... [2023-02-23 23:55:39,725][02125] Avg episode rewards: #0: 12.847, true rewards: #0: 6.513 [2023-02-23 23:55:39,728][02125] Avg episode reward: 12.847, avg true_objective: 6.513 [2023-02-23 23:55:39,784][02125] Num frames 2000... [2023-02-23 23:55:39,904][02125] Num frames 2100... [2023-02-23 23:55:40,029][02125] Num frames 2200... [2023-02-23 23:55:40,160][02125] Num frames 2300... [2023-02-23 23:55:40,278][02125] Num frames 2400... [2023-02-23 23:55:40,396][02125] Num frames 2500... [2023-02-23 23:55:40,520][02125] Num frames 2600... [2023-02-23 23:55:40,643][02125] Num frames 2700... [2023-02-23 23:55:40,766][02125] Num frames 2800... [2023-02-23 23:55:40,898][02125] Num frames 2900... [2023-02-23 23:55:41,027][02125] Num frames 3000... [2023-02-23 23:55:41,179][02125] Num frames 3100... [2023-02-23 23:55:41,321][02125] Num frames 3200... [2023-02-23 23:55:41,474][02125] Num frames 3300... [2023-02-23 23:55:41,623][02125] Num frames 3400... [2023-02-23 23:55:41,749][02125] Num frames 3500... [2023-02-23 23:55:41,861][02125] Num frames 3600... [2023-02-23 23:55:41,979][02125] Num frames 3700... [2023-02-23 23:55:42,095][02125] Num frames 3800... [2023-02-23 23:55:42,176][02125] Avg episode rewards: #0: 22.552, true rewards: #0: 9.552 [2023-02-23 23:55:42,178][02125] Avg episode reward: 22.552, avg true_objective: 9.552 [2023-02-23 23:55:42,273][02125] Num frames 3900... [2023-02-23 23:55:42,389][02125] Num frames 4000... [2023-02-23 23:55:42,519][02125] Num frames 4100... [2023-02-23 23:55:42,643][02125] Num frames 4200... [2023-02-23 23:55:42,769][02125] Num frames 4300... [2023-02-23 23:55:42,893][02125] Num frames 4400... [2023-02-23 23:55:43,062][02125] Num frames 4500... [2023-02-23 23:55:43,231][02125] Num frames 4600... [2023-02-23 23:55:43,324][02125] Avg episode rewards: #0: 21.042, true rewards: #0: 9.242 [2023-02-23 23:55:43,327][02125] Avg episode reward: 21.042, avg true_objective: 9.242 [2023-02-23 23:55:43,463][02125] Num frames 4700... [2023-02-23 23:55:43,624][02125] Num frames 4800... [2023-02-23 23:55:43,777][02125] Num frames 4900... [2023-02-23 23:55:43,929][02125] Num frames 5000... [2023-02-23 23:55:44,128][02125] Num frames 5100... [2023-02-23 23:55:44,291][02125] Num frames 5200... [2023-02-23 23:55:44,445][02125] Num frames 5300... [2023-02-23 23:55:44,603][02125] Num frames 5400... [2023-02-23 23:55:44,797][02125] Num frames 5500... [2023-02-23 23:55:44,961][02125] Num frames 5600... [2023-02-23 23:55:45,127][02125] Num frames 5700... [2023-02-23 23:55:45,300][02125] Num frames 5800... [2023-02-23 23:55:45,465][02125] Num frames 5900... [2023-02-23 23:55:45,629][02125] Num frames 6000... [2023-02-23 23:55:45,798][02125] Num frames 6100... [2023-02-23 23:55:46,004][02125] Avg episode rewards: #0: 23.815, true rewards: #0: 10.315 [2023-02-23 23:55:46,007][02125] Avg episode reward: 23.815, avg true_objective: 10.315 [2023-02-23 23:55:46,031][02125] Num frames 6200... [2023-02-23 23:55:46,200][02125] Num frames 6300... [2023-02-23 23:55:46,363][02125] Num frames 6400... [2023-02-23 23:55:46,524][02125] Num frames 6500... [2023-02-23 23:55:46,647][02125] Num frames 6600... [2023-02-23 23:55:46,758][02125] Num frames 6700... [2023-02-23 23:55:46,870][02125] Num frames 6800... [2023-02-23 23:55:47,033][02125] Avg episode rewards: #0: 22.561, true rewards: #0: 9.847 [2023-02-23 23:55:47,036][02125] Avg episode reward: 22.561, avg true_objective: 9.847 [2023-02-23 23:55:47,048][02125] Num frames 6900... [2023-02-23 23:55:47,165][02125] Num frames 7000... [2023-02-23 23:55:47,282][02125] Num frames 7100... [2023-02-23 23:55:47,392][02125] Num frames 7200... [2023-02-23 23:55:47,506][02125] Num frames 7300... [2023-02-23 23:55:47,618][02125] Num frames 7400... [2023-02-23 23:55:47,734][02125] Num frames 7500... [2023-02-23 23:55:47,850][02125] Num frames 7600... [2023-02-23 23:55:47,963][02125] Num frames 7700... [2023-02-23 23:55:48,074][02125] Num frames 7800... [2023-02-23 23:55:48,187][02125] Num frames 7900... [2023-02-23 23:55:48,310][02125] Num frames 8000... [2023-02-23 23:55:48,434][02125] Num frames 8100... [2023-02-23 23:55:48,549][02125] Num frames 8200... [2023-02-23 23:55:48,663][02125] Num frames 8300... [2023-02-23 23:55:48,775][02125] Num frames 8400... [2023-02-23 23:55:48,899][02125] Avg episode rewards: #0: 24.576, true rewards: #0: 10.576 [2023-02-23 23:55:48,901][02125] Avg episode reward: 24.576, avg true_objective: 10.576 [2023-02-23 23:55:48,949][02125] Num frames 8500... [2023-02-23 23:55:49,063][02125] Num frames 8600... [2023-02-23 23:55:49,176][02125] Num frames 8700... [2023-02-23 23:55:49,285][02125] Num frames 8800... [2023-02-23 23:55:49,401][02125] Num frames 8900... [2023-02-23 23:55:49,513][02125] Num frames 9000... [2023-02-23 23:55:49,629][02125] Num frames 9100... [2023-02-23 23:55:49,746][02125] Num frames 9200... [2023-02-23 23:55:49,859][02125] Num frames 9300... [2023-02-23 23:55:49,978][02125] Num frames 9400... [2023-02-23 23:55:50,095][02125] Avg episode rewards: #0: 23.948, true rewards: #0: 10.503 [2023-02-23 23:55:50,096][02125] Avg episode reward: 23.948, avg true_objective: 10.503 [2023-02-23 23:55:50,158][02125] Num frames 9500... [2023-02-23 23:55:50,277][02125] Num frames 9600... [2023-02-23 23:55:50,398][02125] Num frames 9700... [2023-02-23 23:55:50,535][02125] Num frames 9800... [2023-02-23 23:55:50,676][02125] Avg episode rewards: #0: 22.169, true rewards: #0: 9.869 [2023-02-23 23:55:50,678][02125] Avg episode reward: 22.169, avg true_objective: 9.869 [2023-02-23 23:56:55,620][02125] Replay video saved to /content/train_dir/default_experiment/replay.mp4!