[2024-08-11 09:39:25,968][05236] Saving configuration to /content/train_dir/default_experiment/config.json...
[2024-08-11 09:39:25,972][05236] Rollout worker 0 uses device cpu
[2024-08-11 09:39:25,973][05236] Rollout worker 1 uses device cpu
[2024-08-11 09:39:25,974][05236] Rollout worker 2 uses device cpu
[2024-08-11 09:39:25,976][05236] Rollout worker 3 uses device cpu
[2024-08-11 09:39:25,977][05236] Rollout worker 4 uses device cpu
[2024-08-11 09:39:25,979][05236] Rollout worker 5 uses device cpu
[2024-08-11 09:39:25,980][05236] Rollout worker 6 uses device cpu
[2024-08-11 09:39:25,981][05236] Rollout worker 7 uses device cpu
[2024-08-11 09:39:26,147][05236] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-08-11 09:39:26,148][05236] InferenceWorker_p0-w0: min num requests: 2
[2024-08-11 09:39:26,182][05236] Starting all processes...
[2024-08-11 09:39:26,184][05236] Starting process learner_proc0
[2024-08-11 09:39:27,452][05236] Starting all processes...
[2024-08-11 09:39:27,463][05236] Starting process inference_proc0-0
[2024-08-11 09:39:27,464][05236] Starting process rollout_proc0
[2024-08-11 09:39:27,464][05236] Starting process rollout_proc1
[2024-08-11 09:39:27,464][05236] Starting process rollout_proc2
[2024-08-11 09:39:27,464][05236] Starting process rollout_proc3
[2024-08-11 09:39:27,464][05236] Starting process rollout_proc4
[2024-08-11 09:39:27,464][05236] Starting process rollout_proc5
[2024-08-11 09:39:27,464][05236] Starting process rollout_proc6
[2024-08-11 09:39:27,464][05236] Starting process rollout_proc7
[2024-08-11 09:39:41,930][08106] Worker 1 uses CPU cores [1]
[2024-08-11 09:39:42,196][08111] Worker 3 uses CPU cores [1]
[2024-08-11 09:39:42,202][08104] Worker 0 uses CPU cores [0]
[2024-08-11 09:39:42,216][08091] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-08-11 09:39:42,220][08112] Worker 7 uses CPU cores [1]
[2024-08-11 09:39:42,217][08091] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2024-08-11 09:39:42,232][08108] Worker 5 uses CPU cores [1]
[2024-08-11 09:39:42,269][08091] Num visible devices: 1
[2024-08-11 09:39:42,269][08109] Worker 6 uses CPU cores [0]
[2024-08-11 09:39:42,297][08091] Starting seed is not provided
[2024-08-11 09:39:42,298][08091] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-08-11 09:39:42,298][08091] Initializing actor-critic model on device cuda:0
[2024-08-11 09:39:42,299][08091] RunningMeanStd input shape: (3, 72, 128)
[2024-08-11 09:39:42,302][08091] RunningMeanStd input shape: (1,)
[2024-08-11 09:39:42,337][08091] ConvEncoder: input_channels=3
[2024-08-11 09:39:42,358][08107] Worker 2 uses CPU cores [0]
[2024-08-11 09:39:42,388][08110] Worker 4 uses CPU cores [0]
[2024-08-11 09:39:42,435][08105] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-08-11 09:39:42,435][08105] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2024-08-11 09:39:42,454][08105] Num visible devices: 1
[2024-08-11 09:39:42,607][08091] Conv encoder output size: 512
[2024-08-11 09:39:42,607][08091] Policy head output size: 512
[2024-08-11 09:39:42,670][08091] Created Actor Critic model with architecture:
[2024-08-11 09:39:42,671][08091] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ELU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ELU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ELU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ELU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): GRU(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2024-08-11 09:39:43,168][08091] Using optimizer <class 'torch.optim.adam.Adam'>
[2024-08-11 09:39:44,274][08091] No checkpoints found
[2024-08-11 09:39:44,276][08091] Did not load from checkpoint, starting from scratch!
[2024-08-11 09:39:44,276][08091] Initialized policy 0 weights for model version 0
[2024-08-11 09:39:44,285][08091] LearnerWorker_p0 finished initialization!
[2024-08-11 09:39:44,288][08091] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-08-11 09:39:44,568][08105] RunningMeanStd input shape: (3, 72, 128)
[2024-08-11 09:39:44,570][08105] RunningMeanStd input shape: (1,)
[2024-08-11 09:39:44,590][08105] ConvEncoder: input_channels=3
[2024-08-11 09:39:44,759][08105] Conv encoder output size: 512
[2024-08-11 09:39:44,759][08105] Policy head output size: 512
[2024-08-11 09:39:44,838][05236] Inference worker 0-0 is ready!
[2024-08-11 09:39:44,839][05236] All inference workers are ready! Signal rollout workers to start!
[2024-08-11 09:39:45,217][08110] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:39:45,226][08107] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:39:45,223][08109] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:39:45,225][08104] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:39:45,248][08112] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:39:45,249][08106] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:39:45,277][08111] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:39:45,281][08108] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:39:46,141][05236] Heartbeat connected on Batcher_0
[2024-08-11 09:39:46,144][05236] Heartbeat connected on LearnerWorker_p0
[2024-08-11 09:39:46,189][05236] Heartbeat connected on InferenceWorker_p0-w0
[2024-08-11 09:39:46,859][05236] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2024-08-11 09:39:47,558][08111] Decorrelating experience for 0 frames...
[2024-08-11 09:39:47,558][08112] Decorrelating experience for 0 frames...
[2024-08-11 09:39:47,555][08108] Decorrelating experience for 0 frames...
[2024-08-11 09:39:47,558][08109] Decorrelating experience for 0 frames...
[2024-08-11 09:39:47,557][08107] Decorrelating experience for 0 frames...
[2024-08-11 09:39:47,560][08110] Decorrelating experience for 0 frames...
[2024-08-11 09:39:48,694][08106] Decorrelating experience for 0 frames...
[2024-08-11 09:39:48,696][08111] Decorrelating experience for 32 frames...
[2024-08-11 09:39:48,698][08108] Decorrelating experience for 32 frames...
[2024-08-11 09:39:48,699][08104] Decorrelating experience for 0 frames...
[2024-08-11 09:39:48,729][08107] Decorrelating experience for 32 frames...
[2024-08-11 09:39:48,745][08110] Decorrelating experience for 32 frames...
[2024-08-11 09:39:49,883][08104] Decorrelating experience for 32 frames...
[2024-08-11 09:39:50,237][08106] Decorrelating experience for 32 frames...
[2024-08-11 09:39:50,250][08112] Decorrelating experience for 32 frames...
[2024-08-11 09:39:50,256][08107] Decorrelating experience for 64 frames...
[2024-08-11 09:39:50,369][08110] Decorrelating experience for 64 frames...
[2024-08-11 09:39:50,706][08108] Decorrelating experience for 64 frames...
[2024-08-11 09:39:50,708][08111] Decorrelating experience for 64 frames...
[2024-08-11 09:39:51,855][05236] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2024-08-11 09:39:51,864][08104] Decorrelating experience for 64 frames...
[2024-08-11 09:39:51,866][08109] Decorrelating experience for 32 frames...
[2024-08-11 09:39:51,905][08106] Decorrelating experience for 64 frames...
[2024-08-11 09:39:51,934][08112] Decorrelating experience for 64 frames...
[2024-08-11 09:39:52,001][08107] Decorrelating experience for 96 frames...
[2024-08-11 09:39:52,041][08111] Decorrelating experience for 96 frames...
[2024-08-11 09:39:52,230][05236] Heartbeat connected on RolloutWorker_w2
[2024-08-11 09:39:52,283][05236] Heartbeat connected on RolloutWorker_w3
[2024-08-11 09:39:52,400][08110] Decorrelating experience for 96 frames...
[2024-08-11 09:39:52,764][05236] Heartbeat connected on RolloutWorker_w4
[2024-08-11 09:39:53,446][08108] Decorrelating experience for 96 frames...
[2024-08-11 09:39:53,471][08106] Decorrelating experience for 96 frames...
[2024-08-11 09:39:53,498][08112] Decorrelating experience for 96 frames...
[2024-08-11 09:39:53,709][08104] Decorrelating experience for 96 frames...
[2024-08-11 09:39:53,796][05236] Heartbeat connected on RolloutWorker_w5
[2024-08-11 09:39:53,844][05236] Heartbeat connected on RolloutWorker_w1
[2024-08-11 09:39:53,884][05236] Heartbeat connected on RolloutWorker_w7
[2024-08-11 09:39:53,916][05236] Heartbeat connected on RolloutWorker_w0
[2024-08-11 09:39:53,939][08109] Decorrelating experience for 64 frames...
[2024-08-11 09:39:55,561][08109] Decorrelating experience for 96 frames...
[2024-08-11 09:39:56,061][05236] Heartbeat connected on RolloutWorker_w6
[2024-08-11 09:39:56,796][08091] Signal inference workers to stop experience collection...
[2024-08-11 09:39:56,815][08105] InferenceWorker_p0-w0: stopping experience collection
[2024-08-11 09:39:56,855][05236] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 235.7. Samples: 2356. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2024-08-11 09:39:56,858][05236] Avg episode reward: [(0, '2.337')]
[2024-08-11 09:40:00,805][08091] Signal inference workers to resume experience collection...
[2024-08-11 09:40:00,806][08105] InferenceWorker_p0-w0: resuming experience collection
[2024-08-11 09:40:01,855][05236] Fps is (10 sec: 409.6, 60 sec: 273.1, 300 sec: 273.1). Total num frames: 4096. Throughput: 0: 157.1. Samples: 2356. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
[2024-08-11 09:40:01,858][05236] Avg episode reward: [(0, '2.574')]
[2024-08-11 09:40:06,855][05236] Fps is (10 sec: 2457.6, 60 sec: 1229.0, 300 sec: 1229.0). Total num frames: 24576. Throughput: 0: 320.1. Samples: 6400. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0)
[2024-08-11 09:40:06,859][05236] Avg episode reward: [(0, '3.676')]
[2024-08-11 09:40:09,744][08105] Updated weights for policy 0, policy_version 10 (0.0023)
[2024-08-11 09:40:11,855][05236] Fps is (10 sec: 4505.5, 60 sec: 1966.4, 300 sec: 1966.4). Total num frames: 49152. Throughput: 0: 525.4. Samples: 13132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:40:11,858][05236] Avg episode reward: [(0, '4.341')]
[2024-08-11 09:40:16,856][05236] Fps is (10 sec: 3686.1, 60 sec: 2048.2, 300 sec: 2048.2). Total num frames: 61440. Throughput: 0: 522.6. Samples: 15676. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0)
[2024-08-11 09:40:16,858][05236] Avg episode reward: [(0, '4.364')]
[2024-08-11 09:40:21,855][05236] Fps is (10 sec: 2867.3, 60 sec: 2223.8, 300 sec: 2223.8). Total num frames: 77824. Throughput: 0: 559.7. Samples: 19588. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:40:21,858][05236] Avg episode reward: [(0, '4.305')]
[2024-08-11 09:40:22,423][08105] Updated weights for policy 0, policy_version 20 (0.0047)
[2024-08-11 09:40:26,855][05236] Fps is (10 sec: 3686.7, 60 sec: 2457.8, 300 sec: 2457.8). Total num frames: 98304. Throughput: 0: 640.9. Samples: 25632. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:40:26,859][05236] Avg episode reward: [(0, '4.362')]
[2024-08-11 09:40:31,855][05236] Fps is (10 sec: 4095.8, 60 sec: 2639.8, 300 sec: 2639.8). Total num frames: 118784. Throughput: 0: 646.1. Samples: 29072. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:40:31,858][05236] Avg episode reward: [(0, '4.385')]
[2024-08-11 09:40:31,868][08091] Saving new best policy, reward=4.385!
[2024-08-11 09:40:32,147][08105] Updated weights for policy 0, policy_version 30 (0.0027)
[2024-08-11 09:40:36,855][05236] Fps is (10 sec: 3686.4, 60 sec: 2703.6, 300 sec: 2703.6). Total num frames: 135168. Throughput: 0: 753.6. Samples: 33914. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:40:36,859][05236] Avg episode reward: [(0, '4.461')]
[2024-08-11 09:40:36,864][08091] Saving new best policy, reward=4.461!
[2024-08-11 09:40:41,855][05236] Fps is (10 sec: 3277.0, 60 sec: 2755.7, 300 sec: 2755.7). Total num frames: 151552. Throughput: 0: 821.6. Samples: 39330. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:40:41,860][05236] Avg episode reward: [(0, '4.455')]
[2024-08-11 09:40:43,990][08105] Updated weights for policy 0, policy_version 40 (0.0041)
[2024-08-11 09:40:46,855][05236] Fps is (10 sec: 4096.0, 60 sec: 2935.7, 300 sec: 2935.7). Total num frames: 176128. Throughput: 0: 894.8. Samples: 42624. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:40:46,857][05236] Avg episode reward: [(0, '4.469')]
[2024-08-11 09:40:46,861][08091] Saving new best policy, reward=4.469!
[2024-08-11 09:40:51,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3072.0, 300 sec: 2835.9). Total num frames: 184320. Throughput: 0: 909.0. Samples: 47306. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:40:51,859][05236] Avg episode reward: [(0, '4.372')]
[2024-08-11 09:40:56,855][05236] Fps is (10 sec: 2457.6, 60 sec: 3345.1, 300 sec: 2867.4). Total num frames: 200704. Throughput: 0: 786.0. Samples: 48500. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:40:56,859][05236] Avg episode reward: [(0, '4.515')]
[2024-08-11 09:40:56,862][08091] Saving new best policy, reward=4.515!
[2024-08-11 09:40:57,934][08105] Updated weights for policy 0, policy_version 50 (0.0035)
[2024-08-11 09:41:01,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 2949.3). Total num frames: 221184. Throughput: 0: 848.8. Samples: 53870. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:41:01,865][05236] Avg episode reward: [(0, '4.404')]
[2024-08-11 09:41:06,855][05236] Fps is (10 sec: 4095.8, 60 sec: 3618.1, 300 sec: 3020.9). Total num frames: 241664. Throughput: 0: 906.7. Samples: 60392. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:41:06,860][05236] Avg episode reward: [(0, '4.344')]
[2024-08-11 09:41:08,002][08105] Updated weights for policy 0, policy_version 60 (0.0038)
[2024-08-11 09:41:11,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 2987.8). Total num frames: 253952. Throughput: 0: 819.4. Samples: 62506. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:41:11,859][05236] Avg episode reward: [(0, '4.337')]
[2024-08-11 09:41:16,855][05236] Fps is (10 sec: 2048.1, 60 sec: 3345.1, 300 sec: 2912.8). Total num frames: 262144. Throughput: 0: 809.5. Samples: 65498. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:41:16,861][05236] Avg episode reward: [(0, '4.436')]
[2024-08-11 09:41:21,860][05236] Fps is (10 sec: 2865.7, 60 sec: 3413.0, 300 sec: 2974.9). Total num frames: 282624. Throughput: 0: 813.5. Samples: 70528. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
[2024-08-11 09:41:21,863][05236] Avg episode reward: [(0, '4.457')]
[2024-08-11 09:41:21,871][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000069_282624.pth...
[2024-08-11 09:41:22,541][08105] Updated weights for policy 0, policy_version 70 (0.0022)
[2024-08-11 09:41:26,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3345.1, 300 sec: 2990.2). Total num frames: 299008. Throughput: 0: 803.5. Samples: 75486. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:41:26,861][05236] Avg episode reward: [(0, '4.543')]
[2024-08-11 09:41:26,868][08091] Saving new best policy, reward=4.543!
[2024-08-11 09:41:31,855][05236] Fps is (10 sec: 2868.7, 60 sec: 3208.6, 300 sec: 2964.8). Total num frames: 311296. Throughput: 0: 773.2. Samples: 77420. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:41:31,858][05236] Avg episode reward: [(0, '4.364')]
[2024-08-11 09:41:35,071][08105] Updated weights for policy 0, policy_version 80 (0.0037)
[2024-08-11 09:41:36,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3016.2). Total num frames: 331776. Throughput: 0: 799.2. Samples: 83272. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:41:36,859][05236] Avg episode reward: [(0, '4.300')]
[2024-08-11 09:41:41,856][05236] Fps is (10 sec: 4095.7, 60 sec: 3345.0, 300 sec: 3063.2). Total num frames: 352256. Throughput: 0: 902.3. Samples: 89102. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:41:41,857][05236] Avg episode reward: [(0, '4.326')]
[2024-08-11 09:41:46,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3140.3, 300 sec: 3038.0). Total num frames: 364544. Throughput: 0: 823.4. Samples: 90922. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:41:46,878][05236] Avg episode reward: [(0, '4.392')]
[2024-08-11 09:41:47,544][08105] Updated weights for policy 0, policy_version 90 (0.0037)
[2024-08-11 09:41:51,855][05236] Fps is (10 sec: 3277.0, 60 sec: 3345.1, 300 sec: 3080.3). Total num frames: 385024. Throughput: 0: 792.8. Samples: 96066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:41:51,858][05236] Avg episode reward: [(0, '4.346')]
[2024-08-11 09:41:56,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3413.3, 300 sec: 3119.4). Total num frames: 405504. Throughput: 0: 891.2. Samples: 102610. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:41:56,862][05236] Avg episode reward: [(0, '4.477')]
[2024-08-11 09:41:57,100][08105] Updated weights for policy 0, policy_version 100 (0.0031)
[2024-08-11 09:42:01,857][05236] Fps is (10 sec: 3685.7, 60 sec: 3344.9, 300 sec: 3125.1). Total num frames: 421888. Throughput: 0: 879.2. Samples: 105064. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:42:01,859][05236] Avg episode reward: [(0, '4.611')]
[2024-08-11 09:42:01,871][08091] Saving new best policy, reward=4.611!
[2024-08-11 09:42:06,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3276.8, 300 sec: 3130.6). Total num frames: 438272. Throughput: 0: 857.3. Samples: 109100. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:42:06,859][05236] Avg episode reward: [(0, '4.748')]
[2024-08-11 09:42:06,862][08091] Saving new best policy, reward=4.748!
[2024-08-11 09:42:09,699][08105] Updated weights for policy 0, policy_version 110 (0.0024)
[2024-08-11 09:42:11,855][05236] Fps is (10 sec: 3687.2, 60 sec: 3413.3, 300 sec: 3163.9). Total num frames: 458752. Throughput: 0: 891.4. Samples: 115598. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:42:11,857][05236] Avg episode reward: [(0, '4.592')]
[2024-08-11 09:42:16,855][05236] Fps is (10 sec: 3686.2, 60 sec: 3549.8, 300 sec: 3167.6). Total num frames: 475136. Throughput: 0: 914.8. Samples: 118586. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:42:16,858][05236] Avg episode reward: [(0, '4.435')]
[2024-08-11 09:42:21,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3413.6, 300 sec: 3144.7). Total num frames: 487424. Throughput: 0: 864.2. Samples: 122162. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:42:21,861][05236] Avg episode reward: [(0, '4.405')]
[2024-08-11 09:42:22,752][08105] Updated weights for policy 0, policy_version 120 (0.0030)
[2024-08-11 09:42:26,857][05236] Fps is (10 sec: 3276.4, 60 sec: 3481.5, 300 sec: 3174.4). Total num frames: 507904. Throughput: 0: 862.3. Samples: 127908. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:42:26,859][05236] Avg episode reward: [(0, '4.756')]
[2024-08-11 09:42:26,863][08091] Saving new best policy, reward=4.756!
[2024-08-11 09:42:31,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3202.4). Total num frames: 528384. Throughput: 0: 892.5. Samples: 131084. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:42:31,861][05236] Avg episode reward: [(0, '4.788')]
[2024-08-11 09:42:31,869][08091] Saving new best policy, reward=4.788!
[2024-08-11 09:42:32,249][08105] Updated weights for policy 0, policy_version 130 (0.0029)
[2024-08-11 09:42:36,855][05236] Fps is (10 sec: 3277.3, 60 sec: 3481.6, 300 sec: 3180.5). Total num frames: 540672. Throughput: 0: 886.0. Samples: 135936. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:42:36,857][05236] Avg episode reward: [(0, '4.787')]
[2024-08-11 09:42:41,858][05236] Fps is (10 sec: 2866.3, 60 sec: 3413.2, 300 sec: 3183.2). Total num frames: 557056. Throughput: 0: 844.3. Samples: 140604. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:42:41,860][05236] Avg episode reward: [(0, '4.680')]
[2024-08-11 09:42:45,108][08105] Updated weights for policy 0, policy_version 140 (0.0029)
[2024-08-11 09:42:46,855][05236] Fps is (10 sec: 3686.5, 60 sec: 3549.9, 300 sec: 3208.6). Total num frames: 577536. Throughput: 0: 859.0. Samples: 143718. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:42:46,857][05236] Avg episode reward: [(0, '4.540')]
[2024-08-11 09:42:51,859][05236] Fps is (10 sec: 4095.7, 60 sec: 3549.7, 300 sec: 3232.5). Total num frames: 598016. Throughput: 0: 897.3. Samples: 149484. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:42:51,862][05236] Avg episode reward: [(0, '4.558')]
[2024-08-11 09:42:56,859][05236] Fps is (10 sec: 3275.3, 60 sec: 3413.1, 300 sec: 3212.1). Total num frames: 610304. Throughput: 0: 795.2. Samples: 151386. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:42:56,871][05236] Avg episode reward: [(0, '4.441')]
[2024-08-11 09:42:57,534][08105] Updated weights for policy 0, policy_version 150 (0.0039)
[2024-08-11 09:43:01,855][05236] Fps is (10 sec: 3277.8, 60 sec: 3481.7, 300 sec: 3234.8). Total num frames: 630784. Throughput: 0: 846.6. Samples: 156682. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:43:01,858][05236] Avg episode reward: [(0, '4.695')]
[2024-08-11 09:43:06,812][08105] Updated weights for policy 0, policy_version 160 (0.0030)
[2024-08-11 09:43:06,855][05236] Fps is (10 sec: 4507.6, 60 sec: 3618.1, 300 sec: 3276.9). Total num frames: 655360. Throughput: 0: 916.2. Samples: 163390. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:43:06,859][05236] Avg episode reward: [(0, '4.754')]
[2024-08-11 09:43:11,857][05236] Fps is (10 sec: 3686.0, 60 sec: 3481.5, 300 sec: 3256.9). Total num frames: 667648. Throughput: 0: 883.6. Samples: 167670. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:43:11,859][05236] Avg episode reward: [(0, '4.711')]
[2024-08-11 09:43:16,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3257.4). Total num frames: 684032. Throughput: 0: 861.1. Samples: 169832. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:43:16,863][05236] Avg episode reward: [(0, '4.492')]
[2024-08-11 09:43:19,433][08105] Updated weights for policy 0, policy_version 170 (0.0020)
[2024-08-11 09:43:21,855][05236] Fps is (10 sec: 3687.0, 60 sec: 3618.1, 300 sec: 3276.9). Total num frames: 704512. Throughput: 0: 896.4. Samples: 176276. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:43:21,863][05236] Avg episode reward: [(0, '4.439')]
[2024-08-11 09:43:21,873][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000172_704512.pth...
[2024-08-11 09:43:26,859][05236] Fps is (10 sec: 3685.0, 60 sec: 3549.7, 300 sec: 3276.8). Total num frames: 720896. Throughput: 0: 910.2. Samples: 181562. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:43:26,861][05236] Avg episode reward: [(0, '4.510')]
[2024-08-11 09:43:31,860][05236] Fps is (10 sec: 2865.6, 60 sec: 3413.0, 300 sec: 3258.6). Total num frames: 733184. Throughput: 0: 882.0. Samples: 183414. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:43:31,863][05236] Avg episode reward: [(0, '4.801')]
[2024-08-11 09:43:31,872][08091] Saving new best policy, reward=4.801!
[2024-08-11 09:43:32,133][08105] Updated weights for policy 0, policy_version 180 (0.0023)
[2024-08-11 09:43:36,855][05236] Fps is (10 sec: 3687.8, 60 sec: 3618.1, 300 sec: 3294.7). Total num frames: 757760. Throughput: 0: 880.6. Samples: 189106. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:43:36,857][05236] Avg episode reward: [(0, '4.958')]
[2024-08-11 09:43:36,862][08091] Saving new best policy, reward=4.958!
[2024-08-11 09:43:41,502][08105] Updated weights for policy 0, policy_version 190 (0.0023)
[2024-08-11 09:43:41,855][05236] Fps is (10 sec: 4508.1, 60 sec: 3686.6, 300 sec: 3311.7). Total num frames: 778240. Throughput: 0: 908.8. Samples: 192280. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:43:41,858][05236] Avg episode reward: [(0, '5.012')]
[2024-08-11 09:43:41,870][08091] Saving new best policy, reward=5.012!
[2024-08-11 09:43:46,857][05236] Fps is (10 sec: 3275.9, 60 sec: 3549.7, 300 sec: 3293.9). Total num frames: 790528. Throughput: 0: 900.6. Samples: 197210. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:43:46,860][05236] Avg episode reward: [(0, '5.163')]
[2024-08-11 09:43:46,863][08091] Saving new best policy, reward=5.163!
[2024-08-11 09:43:51,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3481.8, 300 sec: 3293.6). Total num frames: 806912. Throughput: 0: 856.8. Samples: 201946. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:43:51,859][05236] Avg episode reward: [(0, '5.085')]
[2024-08-11 09:43:53,860][08105] Updated weights for policy 0, policy_version 200 (0.0031)
[2024-08-11 09:43:56,855][05236] Fps is (10 sec: 3687.4, 60 sec: 3618.4, 300 sec: 3309.6). Total num frames: 827392. Throughput: 0: 834.1. Samples: 205204. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:43:56,861][05236] Avg episode reward: [(0, '4.883')]
[2024-08-11 09:44:01,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3618.2, 300 sec: 3325.0). Total num frames: 847872. Throughput: 0: 927.6. Samples: 211572. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:44:01,862][05236] Avg episode reward: [(0, '4.875')]
[2024-08-11 09:44:05,862][08105] Updated weights for policy 0, policy_version 210 (0.0048)
[2024-08-11 09:44:06,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3413.3, 300 sec: 3308.4). Total num frames: 860160. Throughput: 0: 871.9. Samples: 215512. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:44:06,861][05236] Avg episode reward: [(0, '4.980')]
[2024-08-11 09:44:11,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3338.7). Total num frames: 884736. Throughput: 0: 899.4. Samples: 222030. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:44:11,857][05236] Avg episode reward: [(0, '4.973')]
[2024-08-11 09:44:15,280][08105] Updated weights for policy 0, policy_version 220 (0.0027)
[2024-08-11 09:44:16,859][05236] Fps is (10 sec: 4503.5, 60 sec: 3686.1, 300 sec: 3352.6). Total num frames: 905216. Throughput: 0: 932.0. Samples: 225352. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:44:16,862][05236] Avg episode reward: [(0, '4.847')]
[2024-08-11 09:44:21,857][05236] Fps is (10 sec: 3275.9, 60 sec: 3549.7, 300 sec: 3336.4). Total num frames: 917504. Throughput: 0: 901.9. Samples: 229694. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:44:21,862][05236] Avg episode reward: [(0, '4.733')]
[2024-08-11 09:44:26,855][05236] Fps is (10 sec: 3278.3, 60 sec: 3618.4, 300 sec: 3350.0). Total num frames: 937984. Throughput: 0: 953.4. Samples: 235184. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:44:26,860][05236] Avg episode reward: [(0, '4.973')]
[2024-08-11 09:44:27,704][08105] Updated weights for policy 0, policy_version 230 (0.0028)
[2024-08-11 09:44:31,855][05236] Fps is (10 sec: 4097.1, 60 sec: 3755.0, 300 sec: 3363.1). Total num frames: 958464. Throughput: 0: 916.7. Samples: 238458. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:44:31,861][05236] Avg episode reward: [(0, '4.959')]
[2024-08-11 09:44:36,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3361.6). Total num frames: 974848. Throughput: 0: 933.8. Samples: 243968. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:44:36,859][05236] Avg episode reward: [(0, '4.863')]
[2024-08-11 09:44:39,507][08105] Updated weights for policy 0, policy_version 240 (0.0031)
[2024-08-11 09:44:41,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3360.2). Total num frames: 991232. Throughput: 0: 961.5. Samples: 248472. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:44:41,857][05236] Avg episode reward: [(0, '5.019')]
[2024-08-11 09:44:46,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.6, 300 sec: 3429.5). Total num frames: 1011712. Throughput: 0: 892.7. Samples: 251742. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:44:46,862][05236] Avg episode reward: [(0, '4.972')]
[2024-08-11 09:44:49,477][08105] Updated weights for policy 0, policy_version 250 (0.0021)
[2024-08-11 09:44:51,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3499.0). Total num frames: 1032192. Throughput: 0: 942.4. Samples: 257922. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:44:51,862][05236] Avg episode reward: [(0, '4.951')]
[2024-08-11 09:44:56,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 1044480. Throughput: 0: 884.0. Samples: 261810. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:44:56,862][05236] Avg episode reward: [(0, '5.006')]
[2024-08-11 09:45:01,647][08105] Updated weights for policy 0, policy_version 260 (0.0020)
[2024-08-11 09:45:01,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 1064960. Throughput: 0: 875.9. Samples: 264764. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:45:01,859][05236] Avg episode reward: [(0, '5.046')]
[2024-08-11 09:45:06,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3512.8). Total num frames: 1085440. Throughput: 0: 922.3. Samples: 271194. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:45:06,857][05236] Avg episode reward: [(0, '5.120')]
[2024-08-11 09:45:11,855][05236] Fps is (10 sec: 3276.6, 60 sec: 3549.8, 300 sec: 3512.8). Total num frames: 1097728. Throughput: 0: 901.5. Samples: 275752. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:45:11,861][05236] Avg episode reward: [(0, '5.393')]
[2024-08-11 09:45:11,872][08091] Saving new best policy, reward=5.393!
[2024-08-11 09:45:13,877][08105] Updated weights for policy 0, policy_version 270 (0.0022)
[2024-08-11 09:45:16,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3481.9, 300 sec: 3512.8). Total num frames: 1114112. Throughput: 0: 872.2. Samples: 277708. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:45:16,861][05236] Avg episode reward: [(0, '5.457')]
[2024-08-11 09:45:16,864][08091] Saving new best policy, reward=5.457!
[2024-08-11 09:45:21,855][05236] Fps is (10 sec: 3686.6, 60 sec: 3618.3, 300 sec: 3512.8). Total num frames: 1134592. Throughput: 0: 889.9. Samples: 284014. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:45:21,859][05236] Avg episode reward: [(0, '5.401')]
[2024-08-11 09:45:21,869][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000277_1134592.pth...
[2024-08-11 09:45:22,006][08091] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000069_282624.pth
[2024-08-11 09:45:24,018][08105] Updated weights for policy 0, policy_version 280 (0.0022)
[2024-08-11 09:45:26,858][05236] Fps is (10 sec: 4094.5, 60 sec: 3617.9, 300 sec: 3512.8). Total num frames: 1155072. Throughput: 0: 858.8. Samples: 287122. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:45:26,866][05236] Avg episode reward: [(0, '5.525')]
[2024-08-11 09:45:26,870][08091] Saving new best policy, reward=5.525!
[2024-08-11 09:45:31,857][05236] Fps is (10 sec: 3276.1, 60 sec: 3481.5, 300 sec: 3498.9). Total num frames: 1167360. Throughput: 0: 879.0. Samples: 291300. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:45:31,865][05236] Avg episode reward: [(0, '5.369')]
[2024-08-11 09:45:36,444][08105] Updated weights for policy 0, policy_version 290 (0.0034)
[2024-08-11 09:45:36,855][05236] Fps is (10 sec: 3278.0, 60 sec: 3549.9, 300 sec: 3512.8). Total num frames: 1187840. Throughput: 0: 864.9. Samples: 296844. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:45:36,860][05236] Avg episode reward: [(0, '5.598')]
[2024-08-11 09:45:36,863][08091] Saving new best policy, reward=5.598!
[2024-08-11 09:45:41,855][05236] Fps is (10 sec: 4096.9, 60 sec: 3618.1, 300 sec: 3499.0). Total num frames: 1208320. Throughput: 0: 920.1. Samples: 303216. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:45:41,858][05236] Avg episode reward: [(0, '5.680')]
[2024-08-11 09:45:41,869][08091] Saving new best policy, reward=5.680!
[2024-08-11 09:45:46,857][05236] Fps is (10 sec: 3276.2, 60 sec: 3481.5, 300 sec: 3512.8). Total num frames: 1220608. Throughput: 0: 895.8. Samples: 305076. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:45:46,860][05236] Avg episode reward: [(0, '5.644')]
[2024-08-11 09:45:49,017][08105] Updated weights for policy 0, policy_version 300 (0.0026)
[2024-08-11 09:45:51,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3512.8). Total num frames: 1236992. Throughput: 0: 853.5. Samples: 309600. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:45:51,862][05236] Avg episode reward: [(0, '6.028')]
[2024-08-11 09:45:51,872][08091] Saving new best policy, reward=6.028!
[2024-08-11 09:45:56,855][05236] Fps is (10 sec: 4096.8, 60 sec: 3618.1, 300 sec: 3526.7). Total num frames: 1261568. Throughput: 0: 896.1. Samples: 316076. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:45:56,857][05236] Avg episode reward: [(0, '6.197')]
[2024-08-11 09:45:56,862][08091] Saving new best policy, reward=6.197!
[2024-08-11 09:45:58,725][08105] Updated weights for policy 0, policy_version 310 (0.0020)
[2024-08-11 09:46:01,855][05236] Fps is (10 sec: 4095.9, 60 sec: 3549.8, 300 sec: 3512.8). Total num frames: 1277952. Throughput: 0: 917.4. Samples: 318992. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:46:01,859][05236] Avg episode reward: [(0, '5.917')]
[2024-08-11 09:46:06,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3413.3, 300 sec: 3512.8). Total num frames: 1290240. Throughput: 0: 866.2. Samples: 322994. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:46:06,860][05236] Avg episode reward: [(0, '5.835')]
[2024-08-11 09:46:11,159][08105] Updated weights for policy 0, policy_version 320 (0.0020)
[2024-08-11 09:46:11,855][05236] Fps is (10 sec: 3276.9, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1310720. Throughput: 0: 933.0. Samples: 329102. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:46:11,858][05236] Avg episode reward: [(0, '5.984')]
[2024-08-11 09:46:16,858][05236] Fps is (10 sec: 4094.5, 60 sec: 3617.9, 300 sec: 3554.5). Total num frames: 1331200. Throughput: 0: 910.5. Samples: 332274. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:46:16,860][05236] Avg episode reward: [(0, '5.829')]
[2024-08-11 09:46:21,857][05236] Fps is (10 sec: 3685.7, 60 sec: 3549.8, 300 sec: 3554.5). Total num frames: 1347584. Throughput: 0: 886.4. Samples: 336732. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:46:21,859][05236] Avg episode reward: [(0, '5.651')]
[2024-08-11 09:46:23,444][08105] Updated weights for policy 0, policy_version 330 (0.0059)
[2024-08-11 09:46:26,855][05236] Fps is (10 sec: 3278.0, 60 sec: 3481.8, 300 sec: 3568.4). Total num frames: 1363968. Throughput: 0: 859.3. Samples: 341886. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:46:26,857][05236] Avg episode reward: [(0, '5.732')]
[2024-08-11 09:46:31,858][05236] Fps is (10 sec: 3686.1, 60 sec: 3618.1, 300 sec: 3568.3). Total num frames: 1384448. Throughput: 0: 888.2. Samples: 345046. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:46:31,860][05236] Avg episode reward: [(0, '5.673')]
[2024-08-11 09:46:33,224][08105] Updated weights for policy 0, policy_version 340 (0.0030)
[2024-08-11 09:46:36,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1400832. Throughput: 0: 913.7. Samples: 350718. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:46:36,857][05236] Avg episode reward: [(0, '5.623')]
[2024-08-11 09:46:41,855][05236] Fps is (10 sec: 3277.7, 60 sec: 3481.6, 300 sec: 3568.4). Total num frames: 1417216. Throughput: 0: 862.7. Samples: 354896. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:46:41,857][05236] Avg episode reward: [(0, '5.951')]
[2024-08-11 09:46:45,598][08105] Updated weights for policy 0, policy_version 350 (0.0040)
[2024-08-11 09:46:46,855][05236] Fps is (10 sec: 3686.3, 60 sec: 3618.2, 300 sec: 3568.4). Total num frames: 1437696. Throughput: 0: 870.3. Samples: 358154. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:46:46,861][05236] Avg episode reward: [(0, '6.164')]
[2024-08-11 09:46:51,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3554.5). Total num frames: 1454080. Throughput: 0: 920.7. Samples: 364424. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:46:51,859][05236] Avg episode reward: [(0, '6.251')]
[2024-08-11 09:46:51,890][08091] Saving new best policy, reward=6.251!
[2024-08-11 09:46:56,855][05236] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1470464. Throughput: 0: 869.8. Samples: 368242. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:46:56,860][05236] Avg episode reward: [(0, '6.343')]
[2024-08-11 09:46:56,862][08091] Saving new best policy, reward=6.343!
[2024-08-11 09:46:58,385][08105] Updated weights for policy 0, policy_version 360 (0.0055)
[2024-08-11 09:47:01,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3554.5). Total num frames: 1486848. Throughput: 0: 860.4. Samples: 370988. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:47:01,858][05236] Avg episode reward: [(0, '6.910')]
[2024-08-11 09:47:01,865][08091] Saving new best policy, reward=6.910!
[2024-08-11 09:47:06,855][05236] Fps is (10 sec: 4096.2, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 1511424. Throughput: 0: 905.2. Samples: 377466. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:47:06,862][05236] Avg episode reward: [(0, '6.934')]
[2024-08-11 09:47:06,865][08091] Saving new best policy, reward=6.934!
[2024-08-11 09:47:07,863][08105] Updated weights for policy 0, policy_version 370 (0.0019)
[2024-08-11 09:47:11,855][05236] Fps is (10 sec: 3686.3, 60 sec: 3549.9, 300 sec: 3554.5). Total num frames: 1523712. Throughput: 0: 896.1. Samples: 382212. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:47:11,857][05236] Avg episode reward: [(0, '6.629')]
[2024-08-11 09:47:16,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3481.8, 300 sec: 3568.4). Total num frames: 1540096. Throughput: 0: 868.4. Samples: 384122. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:47:16,857][05236] Avg episode reward: [(0, '6.753')]
[2024-08-11 09:47:20,142][08105] Updated weights for policy 0, policy_version 380 (0.0041)
[2024-08-11 09:47:21,855][05236] Fps is (10 sec: 3686.5, 60 sec: 3550.0, 300 sec: 3568.4). Total num frames: 1560576. Throughput: 0: 882.9. Samples: 390450. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:47:21,857][05236] Avg episode reward: [(0, '7.441')]
[2024-08-11 09:47:21,868][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000381_1560576.pth...
[2024-08-11 09:47:22,018][08091] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000172_704512.pth
[2024-08-11 09:47:22,036][08091] Saving new best policy, reward=7.441!
[2024-08-11 09:47:26,855][05236] Fps is (10 sec: 4096.1, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 1581056. Throughput: 0: 917.3. Samples: 396174. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:47:26,859][05236] Avg episode reward: [(0, '7.408')]
[2024-08-11 09:47:31,855][05236] Fps is (10 sec: 3276.7, 60 sec: 3481.7, 300 sec: 3568.4). Total num frames: 1593344. Throughput: 0: 888.7. Samples: 398146. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:47:31,861][05236] Avg episode reward: [(0, '7.551')]
[2024-08-11 09:47:31,878][08091] Saving new best policy, reward=7.551!
[2024-08-11 09:47:32,513][08105] Updated weights for policy 0, policy_version 390 (0.0015)
[2024-08-11 09:47:36,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 1613824. Throughput: 0: 871.2. Samples: 403628. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:47:36,862][05236] Avg episode reward: [(0, '6.684')]
[2024-08-11 09:47:41,765][08105] Updated weights for policy 0, policy_version 400 (0.0046)
[2024-08-11 09:47:41,855][05236] Fps is (10 sec: 4505.7, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 1638400. Throughput: 0: 932.9. Samples: 410220. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:47:41,864][05236] Avg episode reward: [(0, '7.134')]
[2024-08-11 09:47:46,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3568.4). Total num frames: 1650688. Throughput: 0: 921.2. Samples: 412440. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:47:46,857][05236] Avg episode reward: [(0, '7.469')]
[2024-08-11 09:47:51,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 1667072. Throughput: 0: 881.5. Samples: 417134. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:47:51,857][05236] Avg episode reward: [(0, '8.318')]
[2024-08-11 09:47:51,867][08091] Saving new best policy, reward=8.318!
[2024-08-11 09:47:54,232][08105] Updated weights for policy 0, policy_version 410 (0.0025)
[2024-08-11 09:47:56,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3596.2). Total num frames: 1691648. Throughput: 0: 920.5. Samples: 423636. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:47:56,862][05236] Avg episode reward: [(0, '8.303')]
[2024-08-11 09:48:01,855][05236] Fps is (10 sec: 4095.9, 60 sec: 3686.4, 300 sec: 3568.4). Total num frames: 1708032. Throughput: 0: 948.7. Samples: 426814. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:48:01,858][05236] Avg episode reward: [(0, '8.648')]
[2024-08-11 09:48:01,872][08091] Saving new best policy, reward=8.648!
[2024-08-11 09:48:05,764][08105] Updated weights for policy 0, policy_version 420 (0.0020)
[2024-08-11 09:48:06,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3481.6, 300 sec: 3568.4). Total num frames: 1720320. Throughput: 0: 895.3. Samples: 430738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:48:06,860][05236] Avg episode reward: [(0, '8.875')]
[2024-08-11 09:48:06,864][08091] Saving new best policy, reward=8.875!
[2024-08-11 09:48:11,855][05236] Fps is (10 sec: 3686.3, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 1744896. Throughput: 0: 908.7. Samples: 437066. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:48:11,858][05236] Avg episode reward: [(0, '9.806')]
[2024-08-11 09:48:11,867][08091] Saving new best policy, reward=9.806!
[2024-08-11 09:48:15,448][08105] Updated weights for policy 0, policy_version 430 (0.0065)
[2024-08-11 09:48:16,855][05236] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3596.2). Total num frames: 1765376. Throughput: 0: 937.7. Samples: 440344. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
[2024-08-11 09:48:16,857][05236] Avg episode reward: [(0, '11.269')]
[2024-08-11 09:48:16,863][08091] Saving new best policy, reward=11.269!
[2024-08-11 09:48:21,855][05236] Fps is (10 sec: 3276.9, 60 sec: 3618.1, 300 sec: 3582.3). Total num frames: 1777664. Throughput: 0: 920.0. Samples: 445030. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:48:21,857][05236] Avg episode reward: [(0, '10.969')]
[2024-08-11 09:48:26,855][05236] Fps is (10 sec: 3276.7, 60 sec: 3618.1, 300 sec: 3610.1). Total num frames: 1798144. Throughput: 0: 886.5. Samples: 450112. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:48:26,862][05236] Avg episode reward: [(0, '11.537')]
[2024-08-11 09:48:26,865][08091] Saving new best policy, reward=11.537!
[2024-08-11 09:48:27,814][08105] Updated weights for policy 0, policy_version 440 (0.0016)
[2024-08-11 09:48:31,855][05236] Fps is (10 sec: 4095.9, 60 sec: 3754.7, 300 sec: 3596.1). Total num frames: 1818624. Throughput: 0: 905.7. Samples: 453196. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:48:31,858][05236] Avg episode reward: [(0, '12.102')]
[2024-08-11 09:48:31,869][08091] Saving new best policy, reward=12.102!
[2024-08-11 09:48:36,855][05236] Fps is (10 sec: 3686.3, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 1835008. Throughput: 0: 926.9. Samples: 458844. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:48:36,858][05236] Avg episode reward: [(0, '12.766')]
[2024-08-11 09:48:36,862][08091] Saving new best policy, reward=12.766!
[2024-08-11 09:48:39,781][08105] Updated weights for policy 0, policy_version 450 (0.0016)
[2024-08-11 09:48:41,855][05236] Fps is (10 sec: 2867.3, 60 sec: 3481.6, 300 sec: 3582.3). Total num frames: 1847296. Throughput: 0: 876.4. Samples: 463074. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:48:41,862][05236] Avg episode reward: [(0, '13.040')]
[2024-08-11 09:48:41,878][08091] Saving new best policy, reward=13.040!
[2024-08-11 09:48:46,855][05236] Fps is (10 sec: 3277.0, 60 sec: 3618.1, 300 sec: 3596.1). Total num frames: 1867776. Throughput: 0: 874.9. Samples: 466186. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:48:46,862][05236] Avg episode reward: [(0, '13.769')]
[2024-08-11 09:48:46,874][08091] Saving new best policy, reward=13.769!
[2024-08-11 09:48:50,064][08105] Updated weights for policy 0, policy_version 460 (0.0020)
[2024-08-11 09:48:51,858][05236] Fps is (10 sec: 4094.9, 60 sec: 3686.2, 300 sec: 3596.1). Total num frames: 1888256. Throughput: 0: 931.1. Samples: 472640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:48:51,860][05236] Avg episode reward: [(0, '14.199')]
[2024-08-11 09:48:51,872][08091] Saving new best policy, reward=14.199!
[2024-08-11 09:48:56,855][05236] Fps is (10 sec: 3276.7, 60 sec: 3481.6, 300 sec: 3568.4). Total num frames: 1900544. Throughput: 0: 874.1. Samples: 476400. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:48:56,857][05236] Avg episode reward: [(0, '14.170')]
[2024-08-11 09:49:01,855][05236] Fps is (10 sec: 3277.7, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 1921024. Throughput: 0: 857.5. Samples: 478932. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:49:01,863][05236] Avg episode reward: [(0, '14.394')]
[2024-08-11 09:49:01,874][08091] Saving new best policy, reward=14.394!
[2024-08-11 09:49:02,582][08105] Updated weights for policy 0, policy_version 470 (0.0025)
[2024-08-11 09:49:06,855][05236] Fps is (10 sec: 4096.1, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 1941504. Throughput: 0: 897.1. Samples: 485398. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:49:06,864][05236] Avg episode reward: [(0, '16.633')]
[2024-08-11 09:49:06,870][08091] Saving new best policy, reward=16.633!
[2024-08-11 09:49:11,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3568.4). Total num frames: 1957888. Throughput: 0: 891.3. Samples: 490222. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:49:11,859][05236] Avg episode reward: [(0, '16.807')]
[2024-08-11 09:49:11,865][08091] Saving new best policy, reward=16.807!
[2024-08-11 09:49:14,625][08105] Updated weights for policy 0, policy_version 480 (0.0021)
[2024-08-11 09:49:16,855][05236] Fps is (10 sec: 2867.1, 60 sec: 3413.3, 300 sec: 3568.4). Total num frames: 1970176. Throughput: 0: 865.3. Samples: 492134. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:49:16,857][05236] Avg episode reward: [(0, '17.574')]
[2024-08-11 09:49:16,865][08091] Saving new best policy, reward=17.574!
[2024-08-11 09:49:21,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3582.3). Total num frames: 1994752. Throughput: 0: 879.1. Samples: 498404. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:49:21,862][05236] Avg episode reward: [(0, '16.528')]
[2024-08-11 09:49:21,871][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000487_1994752.pth...
[2024-08-11 09:49:21,991][08091] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000277_1134592.pth
[2024-08-11 09:49:24,675][08105] Updated weights for policy 0, policy_version 490 (0.0043)
[2024-08-11 09:49:26,857][05236] Fps is (10 sec: 4095.4, 60 sec: 3549.8, 300 sec: 3568.4). Total num frames: 2011136. Throughput: 0: 917.6. Samples: 504366. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:49:26,864][05236] Avg episode reward: [(0, '14.760')]
[2024-08-11 09:49:31,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3568.4). Total num frames: 2027520. Throughput: 0: 890.9. Samples: 506276. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:49:31,857][05236] Avg episode reward: [(0, '14.951')]
[2024-08-11 09:49:36,675][08105] Updated weights for policy 0, policy_version 500 (0.0023)
[2024-08-11 09:49:36,855][05236] Fps is (10 sec: 3687.1, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 2048000. Throughput: 0: 867.5. Samples: 511676. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:49:36,857][05236] Avg episode reward: [(0, '14.054')]
[2024-08-11 09:49:41,855][05236] Fps is (10 sec: 4095.9, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 2068480. Throughput: 0: 934.1. Samples: 518436. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:49:41,861][05236] Avg episode reward: [(0, '14.185')]
[2024-08-11 09:49:46,855][05236] Fps is (10 sec: 3686.3, 60 sec: 3618.1, 300 sec: 3568.4). Total num frames: 2084864. Throughput: 0: 931.6. Samples: 520856. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:49:46,860][05236] Avg episode reward: [(0, '16.061')]
[2024-08-11 09:49:47,809][08105] Updated weights for policy 0, policy_version 510 (0.0030)
[2024-08-11 09:49:51,855][05236] Fps is (10 sec: 3276.9, 60 sec: 3550.0, 300 sec: 3582.3). Total num frames: 2101248. Throughput: 0: 888.1. Samples: 525362. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:49:51,860][05236] Avg episode reward: [(0, '16.464')]
[2024-08-11 09:49:56,855][05236] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 2121728. Throughput: 0: 928.7. Samples: 532014. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:49:56,857][05236] Avg episode reward: [(0, '15.888')]
[2024-08-11 09:49:57,761][08105] Updated weights for policy 0, policy_version 520 (0.0017)
[2024-08-11 09:50:01,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3582.3). Total num frames: 2142208. Throughput: 0: 959.4. Samples: 535308. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:50:01,859][05236] Avg episode reward: [(0, '17.390')]
[2024-08-11 09:50:06,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3582.3). Total num frames: 2154496. Throughput: 0: 908.7. Samples: 539294. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:50:06,857][05236] Avg episode reward: [(0, '17.533')]
[2024-08-11 09:50:09,983][08105] Updated weights for policy 0, policy_version 530 (0.0019)
[2024-08-11 09:50:11,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3596.1). Total num frames: 2174976. Throughput: 0: 910.1. Samples: 545318. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:50:11,863][05236] Avg episode reward: [(0, '17.228')]
[2024-08-11 09:50:16,855][05236] Fps is (10 sec: 4505.6, 60 sec: 3823.0, 300 sec: 3610.0). Total num frames: 2199552. Throughput: 0: 939.7. Samples: 548564. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:50:16,860][05236] Avg episode reward: [(0, '16.949')]
[2024-08-11 09:50:20,852][08105] Updated weights for policy 0, policy_version 540 (0.0025)
[2024-08-11 09:50:21,860][05236] Fps is (10 sec: 3684.3, 60 sec: 3617.8, 300 sec: 3582.2). Total num frames: 2211840. Throughput: 0: 930.8. Samples: 553568. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:50:21,863][05236] Avg episode reward: [(0, '18.135')]
[2024-08-11 09:50:21,883][08091] Saving new best policy, reward=18.135!
[2024-08-11 09:50:26,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3618.2, 300 sec: 3596.2). Total num frames: 2228224. Throughput: 0: 890.1. Samples: 558492. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
[2024-08-11 09:50:26,857][05236] Avg episode reward: [(0, '17.910')]
[2024-08-11 09:50:31,533][08105] Updated weights for policy 0, policy_version 550 (0.0028)
[2024-08-11 09:50:31,855][05236] Fps is (10 sec: 4098.3, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 2252800. Throughput: 0: 909.8. Samples: 561798. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:50:31,862][05236] Avg episode reward: [(0, '18.100')]
[2024-08-11 09:50:36,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3596.1). Total num frames: 2269184. Throughput: 0: 945.2. Samples: 567896. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:50:36,862][05236] Avg episode reward: [(0, '16.779')]
[2024-08-11 09:50:41,856][05236] Fps is (10 sec: 2867.0, 60 sec: 3549.9, 300 sec: 3596.2). Total num frames: 2281472. Throughput: 0: 886.8. Samples: 571920. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:50:41,862][05236] Avg episode reward: [(0, '17.699')]
[2024-08-11 09:50:43,881][08105] Updated weights for policy 0, policy_version 560 (0.0023)
[2024-08-11 09:50:46,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3623.9). Total num frames: 2306048. Throughput: 0: 886.5. Samples: 575200. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:50:46,863][05236] Avg episode reward: [(0, '17.485')]
[2024-08-11 09:50:51,855][05236] Fps is (10 sec: 4505.9, 60 sec: 3754.7, 300 sec: 3610.0). Total num frames: 2326528. Throughput: 0: 948.4. Samples: 581972. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:50:51,863][05236] Avg episode reward: [(0, '17.912')]
[2024-08-11 09:50:54,136][08105] Updated weights for policy 0, policy_version 570 (0.0032)
[2024-08-11 09:50:56,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3596.2). Total num frames: 2338816. Throughput: 0: 865.2. Samples: 584254. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:50:56,857][05236] Avg episode reward: [(0, '18.002')]
[2024-08-11 09:51:01,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2359296. Throughput: 0: 889.6. Samples: 588598. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:51:01,860][05236] Avg episode reward: [(0, '17.691')]
[2024-08-11 09:51:04,972][08105] Updated weights for policy 0, policy_version 580 (0.0019)
[2024-08-11 09:51:06,855][05236] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3637.8). Total num frames: 2383872. Throughput: 0: 931.2. Samples: 595466. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:51:06,857][05236] Avg episode reward: [(0, '17.364')]
[2024-08-11 09:51:11,855][05236] Fps is (10 sec: 4095.8, 60 sec: 3754.6, 300 sec: 3624.0). Total num frames: 2400256. Throughput: 0: 944.5. Samples: 600996. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:51:11,858][05236] Avg episode reward: [(0, '17.680')]
[2024-08-11 09:51:16,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3610.1). Total num frames: 2412544. Throughput: 0: 914.8. Samples: 602966. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
[2024-08-11 09:51:16,857][05236] Avg episode reward: [(0, '16.901')]
[2024-08-11 09:51:17,027][08105] Updated weights for policy 0, policy_version 590 (0.0019)
[2024-08-11 09:51:21,855][05236] Fps is (10 sec: 3686.6, 60 sec: 3755.0, 300 sec: 3637.8). Total num frames: 2437120. Throughput: 0: 919.1. Samples: 609254. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:51:21,860][05236] Avg episode reward: [(0, '18.290')]
[2024-08-11 09:51:21,877][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000595_2437120.pth...
[2024-08-11 09:51:22,012][08091] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000381_1560576.pth
[2024-08-11 09:51:22,031][08091] Saving new best policy, reward=18.290!
[2024-08-11 09:51:26,439][08105] Updated weights for policy 0, policy_version 600 (0.0022)
[2024-08-11 09:51:26,855][05236] Fps is (10 sec: 4505.7, 60 sec: 3822.9, 300 sec: 3637.8). Total num frames: 2457600. Throughput: 0: 964.5. Samples: 615320. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:51:26,859][05236] Avg episode reward: [(0, '17.710')]
[2024-08-11 09:51:31,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3623.9). Total num frames: 2469888. Throughput: 0: 934.3. Samples: 617242. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:51:31,859][05236] Avg episode reward: [(0, '18.250')]
[2024-08-11 09:51:36,855][05236] Fps is (10 sec: 3276.7, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 2490368. Throughput: 0: 895.9. Samples: 622286. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:51:36,858][05236] Avg episode reward: [(0, '18.754')]
[2024-08-11 09:51:36,862][08091] Saving new best policy, reward=18.754!
[2024-08-11 09:51:38,616][08105] Updated weights for policy 0, policy_version 610 (0.0030)
[2024-08-11 09:51:41,855][05236] Fps is (10 sec: 4095.9, 60 sec: 3823.0, 300 sec: 3637.8). Total num frames: 2510848. Throughput: 0: 993.4. Samples: 628958. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:51:41,860][05236] Avg episode reward: [(0, '19.211')]
[2024-08-11 09:51:41,867][08091] Saving new best policy, reward=19.211!
[2024-08-11 09:51:46,855][05236] Fps is (10 sec: 3686.5, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 2527232. Throughput: 0: 951.2. Samples: 631400. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:51:46,857][05236] Avg episode reward: [(0, '19.961')]
[2024-08-11 09:51:46,859][08091] Saving new best policy, reward=19.961!
[2024-08-11 09:51:50,728][08105] Updated weights for policy 0, policy_version 620 (0.0024)
[2024-08-11 09:51:51,855][05236] Fps is (10 sec: 3276.9, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 2543616. Throughput: 0: 890.7. Samples: 635548. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:51:51,862][05236] Avg episode reward: [(0, '19.408')]
[2024-08-11 09:51:56,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 2564096. Throughput: 0: 914.4. Samples: 642142. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:51:56,857][05236] Avg episode reward: [(0, '21.072')]
[2024-08-11 09:51:56,860][08091] Saving new best policy, reward=21.072!
[2024-08-11 09:52:00,092][08105] Updated weights for policy 0, policy_version 630 (0.0027)
[2024-08-11 09:52:01,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3637.8). Total num frames: 2584576. Throughput: 0: 942.9. Samples: 645396. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:52:01,863][05236] Avg episode reward: [(0, '22.326')]
[2024-08-11 09:52:01,879][08091] Saving new best policy, reward=22.326!
[2024-08-11 09:52:06,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3637.8). Total num frames: 2596864. Throughput: 0: 894.7. Samples: 649516. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:52:06,860][05236] Avg episode reward: [(0, '23.337')]
[2024-08-11 09:52:06,863][08091] Saving new best policy, reward=23.337!
[2024-08-11 09:52:11,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3651.7). Total num frames: 2617344. Throughput: 0: 890.0. Samples: 655370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:52:11,858][05236] Avg episode reward: [(0, '23.585')]
[2024-08-11 09:52:11,866][08091] Saving new best policy, reward=23.585!
[2024-08-11 09:52:12,305][08105] Updated weights for policy 0, policy_version 640 (0.0019)
[2024-08-11 09:52:16,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 2637824. Throughput: 0: 919.5. Samples: 658618. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:52:16,857][05236] Avg episode reward: [(0, '22.632')]
[2024-08-11 09:52:21,855][05236] Fps is (10 sec: 3686.3, 60 sec: 3618.1, 300 sec: 3637.8). Total num frames: 2654208. Throughput: 0: 924.8. Samples: 663902. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2024-08-11 09:52:21,858][05236] Avg episode reward: [(0, '21.696')]
[2024-08-11 09:52:24,217][08105] Updated weights for policy 0, policy_version 650 (0.0024)
[2024-08-11 09:52:26,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3651.7). Total num frames: 2670592. Throughput: 0: 882.0. Samples: 668650. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:52:26,861][05236] Avg episode reward: [(0, '21.922')]
[2024-08-11 09:52:31,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3651.7). Total num frames: 2691072. Throughput: 0: 901.3. Samples: 671958. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:52:31,861][05236] Avg episode reward: [(0, '21.387')]
[2024-08-11 09:52:33,848][08105] Updated weights for policy 0, policy_version 660 (0.0015)
[2024-08-11 09:52:36,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3637.8). Total num frames: 2711552. Throughput: 0: 950.2. Samples: 678308. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:52:36,858][05236] Avg episode reward: [(0, '21.377')]
[2024-08-11 09:52:41,857][05236] Fps is (10 sec: 3276.2, 60 sec: 3549.8, 300 sec: 3637.8). Total num frames: 2723840. Throughput: 0: 892.3. Samples: 682298. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:52:41,860][05236] Avg episode reward: [(0, '21.486')]
[2024-08-11 09:52:45,912][08105] Updated weights for policy 0, policy_version 670 (0.0035)
[2024-08-11 09:52:46,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 2748416. Throughput: 0: 893.1. Samples: 685586. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:52:46,862][05236] Avg episode reward: [(0, '21.644')]
[2024-08-11 09:52:51,855][05236] Fps is (10 sec: 4506.4, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 2768896. Throughput: 0: 951.2. Samples: 692320. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:52:51,858][05236] Avg episode reward: [(0, '21.852')]
[2024-08-11 09:52:56,812][08105] Updated weights for policy 0, policy_version 680 (0.0032)
[2024-08-11 09:52:56,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3651.7). Total num frames: 2785280. Throughput: 0: 922.0. Samples: 696862. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:52:56,861][05236] Avg episode reward: [(0, '22.635')]
[2024-08-11 09:53:01,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3665.6). Total num frames: 2801664. Throughput: 0: 896.1. Samples: 698942. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:53:01,863][05236] Avg episode reward: [(0, '22.717')]
[2024-08-11 09:53:06,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 2822144. Throughput: 0: 930.2. Samples: 705762. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:53:06,860][05236] Avg episode reward: [(0, '22.165')]
[2024-08-11 09:53:07,083][08105] Updated weights for policy 0, policy_version 690 (0.0019)
[2024-08-11 09:53:11,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3651.7). Total num frames: 2842624. Throughput: 0: 952.8. Samples: 711526. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:53:11,858][05236] Avg episode reward: [(0, '23.080')]
[2024-08-11 09:53:16,855][05236] Fps is (10 sec: 3276.6, 60 sec: 3618.1, 300 sec: 3651.7). Total num frames: 2854912. Throughput: 0: 923.1. Samples: 713498. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:53:16,857][05236] Avg episode reward: [(0, '24.883')]
[2024-08-11 09:53:16,862][08091] Saving new best policy, reward=24.883!
[2024-08-11 09:53:19,160][08105] Updated weights for policy 0, policy_version 700 (0.0035)
[2024-08-11 09:53:21,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3665.6). Total num frames: 2879488. Throughput: 0: 912.6. Samples: 719374. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:53:21,860][05236] Avg episode reward: [(0, '24.438')]
[2024-08-11 09:53:21,877][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000703_2879488.pth...
[2024-08-11 09:53:22,017][08091] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000487_1994752.pth
[2024-08-11 09:53:26,855][05236] Fps is (10 sec: 4505.9, 60 sec: 3822.9, 300 sec: 3665.6). Total num frames: 2899968. Throughput: 0: 969.5. Samples: 725926. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:53:26,860][05236] Avg episode reward: [(0, '22.592')]
[2024-08-11 09:53:29,395][08105] Updated weights for policy 0, policy_version 710 (0.0013)
[2024-08-11 09:53:31,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3651.7). Total num frames: 2912256. Throughput: 0: 940.2. Samples: 727894. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:53:31,858][05236] Avg episode reward: [(0, '22.725')]
[2024-08-11 09:53:36,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 2932736. Throughput: 0: 897.9. Samples: 732726. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:53:36,857][05236] Avg episode reward: [(0, '21.628')]
[2024-08-11 09:53:40,489][08105] Updated weights for policy 0, policy_version 720 (0.0049)
[2024-08-11 09:53:41,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3823.0, 300 sec: 3679.5). Total num frames: 2953216. Throughput: 0: 946.9. Samples: 739474. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:53:41,861][05236] Avg episode reward: [(0, '20.418')]
[2024-08-11 09:53:46,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3665.6). Total num frames: 2969600. Throughput: 0: 968.7. Samples: 742532. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:53:46,863][05236] Avg episode reward: [(0, '19.766')]
[2024-08-11 09:53:51,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3679.5). Total num frames: 2985984. Throughput: 0: 907.7. Samples: 746610. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:53:51,862][05236] Avg episode reward: [(0, '20.555')]
[2024-08-11 09:53:52,259][08105] Updated weights for policy 0, policy_version 730 (0.0022)
[2024-08-11 09:53:56,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 3006464. Throughput: 0: 924.9. Samples: 753148. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:53:56,858][05236] Avg episode reward: [(0, '20.580')]
[2024-08-11 09:54:01,786][08105] Updated weights for policy 0, policy_version 740 (0.0020)
[2024-08-11 09:54:01,855][05236] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3693.3). Total num frames: 3031040. Throughput: 0: 954.9. Samples: 756466. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:54:01,860][05236] Avg episode reward: [(0, '21.101')]
[2024-08-11 09:54:06,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3679.5). Total num frames: 3043328. Throughput: 0: 926.9. Samples: 761084. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
[2024-08-11 09:54:06,859][05236] Avg episode reward: [(0, '20.356')]
[2024-08-11 09:54:11,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3693.3). Total num frames: 3059712. Throughput: 0: 833.1. Samples: 763416. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:54:11,860][05236] Avg episode reward: [(0, '21.309')]
[2024-08-11 09:54:13,649][08105] Updated weights for policy 0, policy_version 750 (0.0037)
[2024-08-11 09:54:16,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3823.0, 300 sec: 3693.3). Total num frames: 3084288. Throughput: 0: 935.6. Samples: 769998. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:54:16,860][05236] Avg episode reward: [(0, '20.463')]
[2024-08-11 09:54:21,855][05236] Fps is (10 sec: 4095.9, 60 sec: 3686.4, 300 sec: 3693.4). Total num frames: 3100672. Throughput: 0: 952.6. Samples: 775594. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:54:21,858][05236] Avg episode reward: [(0, '20.880')]
[2024-08-11 09:54:25,561][08105] Updated weights for policy 0, policy_version 760 (0.0019)
[2024-08-11 09:54:26,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3693.3). Total num frames: 3117056. Throughput: 0: 903.2. Samples: 780116. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:54:26,862][05236] Avg episode reward: [(0, '22.001')]
[2024-08-11 09:54:31,855][05236] Fps is (10 sec: 3686.5, 60 sec: 3754.7, 300 sec: 3693.3). Total num frames: 3137536. Throughput: 0: 912.2. Samples: 783580. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:54:31,858][05236] Avg episode reward: [(0, '21.525')]
[2024-08-11 09:54:34,735][08105] Updated weights for policy 0, policy_version 770 (0.0022)
[2024-08-11 09:54:36,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3693.3). Total num frames: 3158016. Throughput: 0: 965.1. Samples: 790040. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:54:36,858][05236] Avg episode reward: [(0, '21.629')]
[2024-08-11 09:54:41,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3693.3). Total num frames: 3174400. Throughput: 0: 863.6. Samples: 792008. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:54:41,861][05236] Avg episode reward: [(0, '22.670')]
[2024-08-11 09:54:46,643][08105] Updated weights for policy 0, policy_version 780 (0.0021)
[2024-08-11 09:54:46,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 3194880. Throughput: 0: 905.7. Samples: 797222. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:54:46,857][05236] Avg episode reward: [(0, '24.076')]
[2024-08-11 09:54:51,855][05236] Fps is (10 sec: 4095.8, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 3215360. Throughput: 0: 953.3. Samples: 803982. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:54:51,859][05236] Avg episode reward: [(0, '22.541')]
[2024-08-11 09:54:56,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3693.3). Total num frames: 3231744. Throughput: 0: 1004.4. Samples: 808612. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:54:56,861][05236] Avg episode reward: [(0, '23.258')]
[2024-08-11 09:54:58,177][08105] Updated weights for policy 0, policy_version 790 (0.0031)
[2024-08-11 09:55:01,855][05236] Fps is (10 sec: 3276.9, 60 sec: 3618.1, 300 sec: 3707.2). Total num frames: 3248128. Throughput: 0: 904.4. Samples: 810698. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:55:01,862][05236] Avg episode reward: [(0, '22.726')]
[2024-08-11 09:55:06,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 3268608. Throughput: 0: 930.9. Samples: 817484. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:55:06,857][05236] Avg episode reward: [(0, '23.498')]
[2024-08-11 09:55:07,774][08105] Updated weights for policy 0, policy_version 800 (0.0039)
[2024-08-11 09:55:11,861][05236] Fps is (10 sec: 4093.3, 60 sec: 3822.5, 300 sec: 3693.3). Total num frames: 3289088. Throughput: 0: 905.8. Samples: 820882. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:55:11,864][05236] Avg episode reward: [(0, '24.755')]
[2024-08-11 09:55:16,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3693.4). Total num frames: 3301376. Throughput: 0: 926.3. Samples: 825264. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:55:16,857][05236] Avg episode reward: [(0, '23.615')]
[2024-08-11 09:55:19,829][08105] Updated weights for policy 0, policy_version 810 (0.0030)
[2024-08-11 09:55:21,855][05236] Fps is (10 sec: 3688.8, 60 sec: 3754.7, 300 sec: 3721.1). Total num frames: 3325952. Throughput: 0: 915.8. Samples: 831252. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:55:21,857][05236] Avg episode reward: [(0, '24.763')]
[2024-08-11 09:55:21,872][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000812_3325952.pth...
[2024-08-11 09:55:22,022][08091] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000595_2437120.pth
[2024-08-11 09:55:26,857][05236] Fps is (10 sec: 4504.8, 60 sec: 3822.8, 300 sec: 3707.2). Total num frames: 3346432. Throughput: 0: 944.7. Samples: 834520. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:55:26,859][05236] Avg episode reward: [(0, '25.237')]
[2024-08-11 09:55:26,863][08091] Saving new best policy, reward=25.237!
[2024-08-11 09:55:30,919][08105] Updated weights for policy 0, policy_version 820 (0.0029)
[2024-08-11 09:55:31,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3693.3). Total num frames: 3358720. Throughput: 0: 941.5. Samples: 839588. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:55:31,857][05236] Avg episode reward: [(0, '25.082')]
[2024-08-11 09:55:36,855][05236] Fps is (10 sec: 3277.4, 60 sec: 3686.4, 300 sec: 3721.1). Total num frames: 3379200. Throughput: 0: 899.9. Samples: 844478. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:55:36,857][05236] Avg episode reward: [(0, '24.801')]
[2024-08-11 09:55:41,187][08105] Updated weights for policy 0, policy_version 830 (0.0035)
[2024-08-11 09:55:41,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 3399680. Throughput: 0: 949.1. Samples: 851320. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:55:41,858][05236] Avg episode reward: [(0, '24.333')]
[2024-08-11 09:55:46,859][05236] Fps is (10 sec: 3684.7, 60 sec: 3686.1, 300 sec: 3693.3). Total num frames: 3416064. Throughput: 0: 967.2. Samples: 854226. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:55:46,861][05236] Avg episode reward: [(0, '24.784')]
[2024-08-11 09:55:51,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3707.2). Total num frames: 3432448. Throughput: 0: 904.3. Samples: 858176. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-08-11 09:55:51,864][05236] Avg episode reward: [(0, '24.234')]
[2024-08-11 09:55:53,195][08105] Updated weights for policy 0, policy_version 840 (0.0033)
[2024-08-11 09:55:56,855][05236] Fps is (10 sec: 3688.1, 60 sec: 3686.4, 300 sec: 3707.2). Total num frames: 3452928. Throughput: 0: 901.4. Samples: 861440. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:55:56,857][05236] Avg episode reward: [(0, '23.909')]
[2024-08-11 09:56:01,856][05236] Fps is (10 sec: 4095.4, 60 sec: 3754.6, 300 sec: 3693.3). Total num frames: 3473408. Throughput: 0: 950.3. Samples: 868030. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:56:01,865][05236] Avg episode reward: [(0, '23.145')]
[2024-08-11 09:56:03,511][08105] Updated weights for policy 0, policy_version 850 (0.0036)
[2024-08-11 09:56:06,857][05236] Fps is (10 sec: 3685.5, 60 sec: 3686.2, 300 sec: 3693.3). Total num frames: 3489792. Throughput: 0: 916.1. Samples: 872480. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:56:06,861][05236] Avg episode reward: [(0, '24.651')]
[2024-08-11 09:56:11,855][05236] Fps is (10 sec: 3687.0, 60 sec: 3686.8, 300 sec: 3721.1). Total num frames: 3510272. Throughput: 0: 897.0. Samples: 874882. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:56:11,863][05236] Avg episode reward: [(0, '24.049')]
[2024-08-11 09:56:14,472][08105] Updated weights for policy 0, policy_version 860 (0.0044)
[2024-08-11 09:56:16,855][05236] Fps is (10 sec: 4097.1, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 3530752. Throughput: 0: 934.1. Samples: 881622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:56:16,863][05236] Avg episode reward: [(0, '24.526')]
[2024-08-11 09:56:21,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3693.3). Total num frames: 3547136. Throughput: 0: 945.6. Samples: 887028. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:56:21,857][05236] Avg episode reward: [(0, '24.336')]
[2024-08-11 09:56:26,582][08105] Updated weights for policy 0, policy_version 870 (0.0025)
[2024-08-11 09:56:26,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3707.2). Total num frames: 3563520. Throughput: 0: 900.4. Samples: 891838. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:56:26,862][05236] Avg episode reward: [(0, '25.331')]
[2024-08-11 09:56:26,865][08091] Saving new best policy, reward=25.331!
[2024-08-11 09:56:31,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 3584000. Throughput: 0: 904.8. Samples: 894940. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0)
[2024-08-11 09:56:31,863][05236] Avg episode reward: [(0, '25.532')]
[2024-08-11 09:56:31,874][08091] Saving new best policy, reward=25.532!
[2024-08-11 09:56:36,482][08105] Updated weights for policy 0, policy_version 880 (0.0024)
[2024-08-11 09:56:36,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 3604480. Throughput: 0: 959.2. Samples: 901338. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:56:36,860][05236] Avg episode reward: [(0, '23.689')]
[2024-08-11 09:56:41,856][05236] Fps is (10 sec: 3276.6, 60 sec: 3618.1, 300 sec: 3693.3). Total num frames: 3616768. Throughput: 0: 978.2. Samples: 905458. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:56:41,859][05236] Avg episode reward: [(0, '24.494')]
[2024-08-11 09:56:46,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3686.7, 300 sec: 3707.2). Total num frames: 3637248. Throughput: 0: 901.4. Samples: 908590. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:56:46,857][05236] Avg episode reward: [(0, '23.483')]
[2024-08-11 09:56:47,842][08105] Updated weights for policy 0, policy_version 890 (0.0039)
[2024-08-11 09:56:51,855][05236] Fps is (10 sec: 4505.9, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 3661824. Throughput: 0: 952.4. Samples: 915336. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:56:51,862][05236] Avg episode reward: [(0, '22.167')]
[2024-08-11 09:56:56,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3693.3). Total num frames: 3674112. Throughput: 0: 958.8. Samples: 918028. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:56:56,857][05236] Avg episode reward: [(0, '22.526')]
[2024-08-11 09:56:59,923][08105] Updated weights for policy 0, policy_version 900 (0.0027)
[2024-08-11 09:57:01,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3686.5, 300 sec: 3721.1). Total num frames: 3694592. Throughput: 0: 896.7. Samples: 921972. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2024-08-11 09:57:01,860][05236] Avg episode reward: [(0, '22.560')]
[2024-08-11 09:57:06,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.8, 300 sec: 3721.1). Total num frames: 3715072. Throughput: 0: 928.1. Samples: 928792. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:57:06,859][05236] Avg episode reward: [(0, '21.435')]
[2024-08-11 09:57:09,005][08105] Updated weights for policy 0, policy_version 910 (0.0030)
[2024-08-11 09:57:11,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3721.1). Total num frames: 3735552. Throughput: 0: 950.3. Samples: 934600. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:57:11,857][05236] Avg episode reward: [(0, '22.533')]
[2024-08-11 09:57:16,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3707.2). Total num frames: 3747840. Throughput: 0: 925.4. Samples: 936582. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:57:16,860][05236] Avg episode reward: [(0, '23.173')]
[2024-08-11 09:57:20,911][08105] Updated weights for policy 0, policy_version 920 (0.0040)
[2024-08-11 09:57:21,855][05236] Fps is (10 sec: 3686.3, 60 sec: 3754.7, 300 sec: 3735.0). Total num frames: 3772416. Throughput: 0: 914.8. Samples: 942502. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:57:21,863][05236] Avg episode reward: [(0, '22.748')]
[2024-08-11 09:57:21,874][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000921_3772416.pth...
[2024-08-11 09:57:21,998][08091] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000703_2879488.pth
[2024-08-11 09:57:26,855][05236] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 3792896. Throughput: 0: 895.5. Samples: 945756. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:57:26,859][05236] Avg episode reward: [(0, '22.195')]
[2024-08-11 09:57:31,857][05236] Fps is (10 sec: 3276.0, 60 sec: 3686.2, 300 sec: 3707.2). Total num frames: 3805184. Throughput: 0: 940.6. Samples: 950920. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:57:31,863][05236] Avg episode reward: [(0, '22.547')]
[2024-08-11 09:57:32,914][08105] Updated weights for policy 0, policy_version 930 (0.0025)
[2024-08-11 09:57:36,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3721.1). Total num frames: 3821568. Throughput: 0: 895.0. Samples: 955610. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-08-11 09:57:36,857][05236] Avg episode reward: [(0, '23.260')]
[2024-08-11 09:57:41,855][05236] Fps is (10 sec: 4097.1, 60 sec: 3823.0, 300 sec: 3721.1). Total num frames: 3846144. Throughput: 0: 983.1. Samples: 962266. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:57:41,857][05236] Avg episode reward: [(0, '22.051')]
[2024-08-11 09:57:42,727][08105] Updated weights for policy 0, policy_version 940 (0.0018)
[2024-08-11 09:57:46,855][05236] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3707.2). Total num frames: 3862528. Throughput: 0: 960.9. Samples: 965214. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:57:46,861][05236] Avg episode reward: [(0, '23.130')]
[2024-08-11 09:57:51,855][05236] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3693.3). Total num frames: 3874816. Throughput: 0: 896.5. Samples: 969136. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:57:51,858][05236] Avg episode reward: [(0, '23.080')]
[2024-08-11 09:57:54,814][08105] Updated weights for policy 0, policy_version 950 (0.0029)
[2024-08-11 09:57:56,855][05236] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3721.1). Total num frames: 3899392. Throughput: 0: 908.7. Samples: 975490. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-08-11 09:57:56,857][05236] Avg episode reward: [(0, '22.393')]
[2024-08-11 09:58:01,856][05236] Fps is (10 sec: 4505.2, 60 sec: 3754.6, 300 sec: 3721.1). Total num frames: 3919872. Throughput: 0: 933.6. Samples: 978596. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-08-11 09:58:01,858][05236] Avg episode reward: [(0, '22.102')]
[2024-08-11 09:58:06,337][08105] Updated weights for policy 0, policy_version 960 (0.0029)
[2024-08-11 09:58:06,855][05236] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3693.3). Total num frames: 3932160. Throughput: 0: 905.1. Samples: 983230. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:58:06,866][05236] Avg episode reward: [(0, '21.972')]
[2024-08-11 09:58:11,855][05236] Fps is (10 sec: 2867.5, 60 sec: 3549.9, 300 sec: 3707.2). Total num frames: 3948544. Throughput: 0: 952.8. Samples: 988634. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-08-11 09:58:11,862][05236] Avg episode reward: [(0, '20.611')]
[2024-08-11 09:58:16,599][08105] Updated weights for policy 0, policy_version 970 (0.0022)
[2024-08-11 09:58:16,855][05236] Fps is (10 sec: 4095.9, 60 sec: 3754.6, 300 sec: 3707.2). Total num frames: 3973120. Throughput: 0: 907.8. Samples: 991768. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0)
[2024-08-11 09:58:16,862][05236] Avg episode reward: [(0, '21.454')]
[2024-08-11 09:58:21,857][05236] Fps is (10 sec: 4094.9, 60 sec: 3618.0, 300 sec: 3693.3). Total num frames: 3989504. Throughput: 0: 928.8. Samples: 997408. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-08-11 09:58:21,861][05236] Avg episode reward: [(0, '21.213')]
[2024-08-11 09:58:26,855][05236] Fps is (10 sec: 2867.3, 60 sec: 3481.6, 300 sec: 3693.3). Total num frames: 4001792. Throughput: 0: 879.0. Samples: 1001820. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2024-08-11 09:58:26,862][05236] Avg episode reward: [(0, '22.819')]
[2024-08-11 09:58:27,019][08091] Stopping Batcher_0...
[2024-08-11 09:58:27,021][08091] Loop batcher_evt_loop terminating...
[2024-08-11 09:58:27,022][05236] Component Batcher_0 stopped!
[2024-08-11 09:58:27,027][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2024-08-11 09:58:27,079][05236] Component RolloutWorker_w7 stopped!
[2024-08-11 09:58:27,079][08112] Stopping RolloutWorker_w7...
[2024-08-11 09:58:27,090][08108] Stopping RolloutWorker_w5...
[2024-08-11 09:58:27,090][05236] Component RolloutWorker_w5 stopped!
[2024-08-11 09:58:27,086][08112] Loop rollout_proc7_evt_loop terminating...
[2024-08-11 09:58:27,092][08108] Loop rollout_proc5_evt_loop terminating...
[2024-08-11 09:58:27,103][08106] Stopping RolloutWorker_w1...
[2024-08-11 09:58:27,103][05236] Component RolloutWorker_w1 stopped!
[2024-08-11 09:58:27,104][08106] Loop rollout_proc1_evt_loop terminating...
[2024-08-11 09:58:27,115][08105] Weights refcount: 2 0
[2024-08-11 09:58:27,118][08105] Stopping InferenceWorker_p0-w0...
[2024-08-11 09:58:27,117][05236] Component RolloutWorker_w2 stopped!
[2024-08-11 09:58:27,121][05236] Component InferenceWorker_p0-w0 stopped!
[2024-08-11 09:58:27,124][08109] Stopping RolloutWorker_w6...
[2024-08-11 09:58:27,124][08109] Loop rollout_proc6_evt_loop terminating...
[2024-08-11 09:58:27,125][05236] Component RolloutWorker_w6 stopped!
[2024-08-11 09:58:27,129][08107] Stopping RolloutWorker_w2...
[2024-08-11 09:58:27,119][08105] Loop inference_proc0-0_evt_loop terminating...
[2024-08-11 09:58:27,140][08107] Loop rollout_proc2_evt_loop terminating...
[2024-08-11 09:58:27,149][05236] Component RolloutWorker_w0 stopped!
[2024-08-11 09:58:27,153][08104] Stopping RolloutWorker_w0...
[2024-08-11 09:58:27,153][08104] Loop rollout_proc0_evt_loop terminating...
[2024-08-11 09:58:27,166][08111] Stopping RolloutWorker_w3...
[2024-08-11 09:58:27,167][08111] Loop rollout_proc3_evt_loop terminating...
[2024-08-11 09:58:27,166][05236] Component RolloutWorker_w3 stopped!
[2024-08-11 09:58:27,191][05236] Component RolloutWorker_w4 stopped!
[2024-08-11 09:58:27,195][08110] Stopping RolloutWorker_w4...
[2024-08-11 09:58:27,195][08110] Loop rollout_proc4_evt_loop terminating...
[2024-08-11 09:58:27,224][08091] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000812_3325952.pth
[2024-08-11 09:58:27,237][08091] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2024-08-11 09:58:27,409][05236] Component LearnerWorker_p0 stopped!
[2024-08-11 09:58:27,413][05236] Waiting for process learner_proc0 to stop...
[2024-08-11 09:58:27,415][08091] Stopping LearnerWorker_p0...
[2024-08-11 09:58:27,416][08091] Loop learner_proc0_evt_loop terminating...
[2024-08-11 09:58:28,944][05236] Waiting for process inference_proc0-0 to join...
[2024-08-11 09:58:28,948][05236] Waiting for process rollout_proc0 to join...
[2024-08-11 09:58:30,726][05236] Waiting for process rollout_proc1 to join...
[2024-08-11 09:58:30,993][05236] Waiting for process rollout_proc2 to join...
[2024-08-11 09:58:30,997][05236] Waiting for process rollout_proc3 to join...
[2024-08-11 09:58:31,000][05236] Waiting for process rollout_proc4 to join...
[2024-08-11 09:58:31,003][05236] Waiting for process rollout_proc5 to join...
[2024-08-11 09:58:31,007][05236] Waiting for process rollout_proc6 to join...
[2024-08-11 09:58:31,011][05236] Waiting for process rollout_proc7 to join...
[2024-08-11 09:58:31,015][05236] Batcher 0 profile tree view:
batching: 27.4283, releasing_batches: 0.0318
[2024-08-11 09:58:31,017][05236] InferenceWorker_p0-w0 profile tree view:
wait_policy: 0.0000
  wait_policy_total: 431.2351
update_model: 9.6799
  weight_update: 0.0025
one_step: 0.0141
  handle_policy_step: 632.2925
    deserialize: 17.0362, stack: 3.3466, obs_to_device_normalize: 128.0264, forward: 338.4033, send_messages: 30.4862
    prepare_outputs: 84.6601
      to_cpu: 48.3400
[2024-08-11 09:58:31,020][05236] Learner 0 profile tree view:
misc: 0.0063, prepare_batch: 14.3756
train: 76.3544
  epoch_init: 0.0063, minibatch_init: 0.0079, losses_postprocess: 0.6559, kl_divergence: 0.7429, after_optimizer: 34.3057
  calculate_losses: 28.2091
    losses_init: 0.0053, forward_head: 1.3292, bptt_initial: 18.8123, tail: 1.1588, advantages_returns: 0.3118, losses: 4.0987
    bptt: 2.1331
      bptt_forward_core: 2.0444
  update: 11.7183
    clip: 0.9720
[2024-08-11 09:58:31,022][05236] RolloutWorker_w0 profile tree view:
wait_for_trajectories: 0.3509, enqueue_policy_requests: 107.9632, env_step: 867.1321, overhead: 15.9382, complete_rollouts: 7.8033
save_policy_outputs: 22.3398
  split_output_tensors: 8.9557
[2024-08-11 09:58:31,024][05236] RolloutWorker_w7 profile tree view:
wait_for_trajectories: 0.4346, enqueue_policy_requests: 110.4874, env_step: 865.0737, overhead: 15.8075, complete_rollouts: 6.8289
save_policy_outputs: 22.0537
  split_output_tensors: 8.7828
[2024-08-11 09:58:31,025][05236] Loop Runner_EvtLoop terminating...
[2024-08-11 09:58:31,026][05236] Runner profile tree view:
main_loop: 1144.8442
[2024-08-11 09:58:31,027][05236] Collected {0: 4005888}, FPS: 3499.1
[2024-08-11 09:58:31,439][05236] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json
[2024-08-11 09:58:31,440][05236] Overriding arg 'num_workers' with value 1 passed from command line
[2024-08-11 09:58:31,444][05236] Adding new argument 'no_render'=True that is not in the saved config file!
[2024-08-11 09:58:31,446][05236] Adding new argument 'save_video'=True that is not in the saved config file!
[2024-08-11 09:58:31,447][05236] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
[2024-08-11 09:58:31,450][05236] Adding new argument 'video_name'=None that is not in the saved config file!
[2024-08-11 09:58:31,452][05236] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
[2024-08-11 09:58:31,454][05236] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
[2024-08-11 09:58:31,455][05236] Adding new argument 'push_to_hub'=False that is not in the saved config file!
[2024-08-11 09:58:31,456][05236] Adding new argument 'hf_repository'=None that is not in the saved config file!
[2024-08-11 09:58:31,457][05236] Adding new argument 'policy_index'=0 that is not in the saved config file!
[2024-08-11 09:58:31,458][05236] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
[2024-08-11 09:58:31,459][05236] Adding new argument 'train_script'=None that is not in the saved config file!
[2024-08-11 09:58:31,461][05236] Adding new argument 'enjoy_script'=None that is not in the saved config file!
[2024-08-11 09:58:31,462][05236] Using frameskip 1 and render_action_repeat=4 for evaluation
[2024-08-11 09:58:31,493][05236] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-08-11 09:58:31,496][05236] RunningMeanStd input shape: (3, 72, 128)
[2024-08-11 09:58:31,497][05236] RunningMeanStd input shape: (1,)
[2024-08-11 09:58:31,515][05236] ConvEncoder: input_channels=3
[2024-08-11 09:58:31,620][05236] Conv encoder output size: 512
[2024-08-11 09:58:31,621][05236] Policy head output size: 512
[2024-08-11 09:58:31,918][05236] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2024-08-11 09:58:32,691][05236] Num frames 100...
[2024-08-11 09:58:32,815][05236] Num frames 200...
[2024-08-11 09:58:32,943][05236] Num frames 300...
[2024-08-11 09:58:33,076][05236] Num frames 400...
[2024-08-11 09:58:33,198][05236] Num frames 500...
[2024-08-11 09:58:33,320][05236] Num frames 600...
[2024-08-11 09:58:33,447][05236] Num frames 700...
[2024-08-11 09:58:33,572][05236] Num frames 800...
[2024-08-11 09:58:33,706][05236] Num frames 900...
[2024-08-11 09:58:33,830][05236] Num frames 1000...
[2024-08-11 09:58:33,915][05236] Avg episode rewards: #0: 18.240, true rewards: #0: 10.240
[2024-08-11 09:58:33,917][05236] Avg episode reward: 18.240, avg true_objective: 10.240
[2024-08-11 09:58:34,026][05236] Num frames 1100...
[2024-08-11 09:58:34,145][05236] Num frames 1200...
[2024-08-11 09:58:34,268][05236] Num frames 1300...
[2024-08-11 09:58:34,389][05236] Num frames 1400...
[2024-08-11 09:58:34,513][05236] Num frames 1500...
[2024-08-11 09:58:34,636][05236] Num frames 1600...
[2024-08-11 09:58:34,768][05236] Num frames 1700...
[2024-08-11 09:58:34,906][05236] Num frames 1800...
[2024-08-11 09:58:35,039][05236] Num frames 1900...
[2024-08-11 09:58:35,165][05236] Num frames 2000...
[2024-08-11 09:58:35,290][05236] Num frames 2100...
[2024-08-11 09:58:35,415][05236] Num frames 2200...
[2024-08-11 09:58:35,481][05236] Avg episode rewards: #0: 23.040, true rewards: #0: 11.040
[2024-08-11 09:58:35,483][05236] Avg episode reward: 23.040, avg true_objective: 11.040
[2024-08-11 09:58:35,600][05236] Num frames 2300...
[2024-08-11 09:58:35,730][05236] Num frames 2400...
[2024-08-11 09:58:35,902][05236] Num frames 2500...
[2024-08-11 09:58:36,082][05236] Num frames 2600...
[2024-08-11 09:58:36,256][05236] Num frames 2700...
[2024-08-11 09:58:36,423][05236] Num frames 2800...
[2024-08-11 09:58:36,603][05236] Num frames 2900...
[2024-08-11 09:58:36,772][05236] Num frames 3000...
[2024-08-11 09:58:36,947][05236] Num frames 3100...
[2024-08-11 09:58:37,121][05236] Num frames 3200...
[2024-08-11 09:58:37,295][05236] Num frames 3300...
[2024-08-11 09:58:37,473][05236] Num frames 3400...
[2024-08-11 09:58:37,652][05236] Num frames 3500...
[2024-08-11 09:58:37,847][05236] Num frames 3600...
[2024-08-11 09:58:38,021][05236] Num frames 3700...
[2024-08-11 09:58:38,204][05236] Num frames 3800...
[2024-08-11 09:58:38,384][05236] Num frames 3900...
[2024-08-11 09:58:38,484][05236] Avg episode rewards: #0: 27.453, true rewards: #0: 13.120
[2024-08-11 09:58:38,486][05236] Avg episode reward: 27.453, avg true_objective: 13.120
[2024-08-11 09:58:38,567][05236] Num frames 4000...
[2024-08-11 09:58:38,693][05236] Num frames 4100...
[2024-08-11 09:58:38,815][05236] Num frames 4200...
[2024-08-11 09:58:38,939][05236] Num frames 4300...
[2024-08-11 09:58:39,060][05236] Num frames 4400...
[2024-08-11 09:58:39,186][05236] Num frames 4500...
[2024-08-11 09:58:39,307][05236] Num frames 4600...
[2024-08-11 09:58:39,427][05236] Num frames 4700...
[2024-08-11 09:58:39,553][05236] Num frames 4800...
[2024-08-11 09:58:39,605][05236] Avg episode rewards: #0: 25.250, true rewards: #0: 12.000
[2024-08-11 09:58:39,607][05236] Avg episode reward: 25.250, avg true_objective: 12.000
[2024-08-11 09:58:39,737][05236] Num frames 4900...
[2024-08-11 09:58:39,863][05236] Num frames 5000...
[2024-08-11 09:58:39,986][05236] Num frames 5100...
[2024-08-11 09:58:40,110][05236] Num frames 5200...
[2024-08-11 09:58:40,243][05236] Num frames 5300...
[2024-08-11 09:58:40,330][05236] Avg episode rewards: #0: 21.848, true rewards: #0: 10.648
[2024-08-11 09:58:40,331][05236] Avg episode reward: 21.848, avg true_objective: 10.648
[2024-08-11 09:58:40,429][05236] Num frames 5400...
[2024-08-11 09:58:40,551][05236] Num frames 5500...
[2024-08-11 09:58:40,676][05236] Num frames 5600...
[2024-08-11 09:58:40,809][05236] Num frames 5700...
[2024-08-11 09:58:40,956][05236] Avg episode rewards: #0: 19.953, true rewards: #0: 9.620
[2024-08-11 09:58:40,958][05236] Avg episode reward: 19.953, avg true_objective: 9.620
[2024-08-11 09:58:40,996][05236] Num frames 5800...
[2024-08-11 09:58:41,118][05236] Num frames 5900...
[2024-08-11 09:58:41,251][05236] Num frames 6000...
[2024-08-11 09:58:41,375][05236] Num frames 6100...
[2024-08-11 09:58:41,498][05236] Num frames 6200...
[2024-08-11 09:58:41,620][05236] Num frames 6300...
[2024-08-11 09:58:41,750][05236] Num frames 6400...
[2024-08-11 09:58:41,873][05236] Num frames 6500...
[2024-08-11 09:58:41,995][05236] Num frames 6600...
[2024-08-11 09:58:42,114][05236] Num frames 6700...
[2024-08-11 09:58:42,240][05236] Num frames 6800...
[2024-08-11 09:58:42,367][05236] Num frames 6900...
[2024-08-11 09:58:42,491][05236] Num frames 7000...
[2024-08-11 09:58:42,615][05236] Num frames 7100...
[2024-08-11 09:58:42,691][05236] Avg episode rewards: #0: 22.166, true rewards: #0: 10.166
[2024-08-11 09:58:42,693][05236] Avg episode reward: 22.166, avg true_objective: 10.166
[2024-08-11 09:58:42,799][05236] Num frames 7200...
[2024-08-11 09:58:42,922][05236] Num frames 7300...
[2024-08-11 09:58:43,045][05236] Num frames 7400...
[2024-08-11 09:58:43,165][05236] Num frames 7500...
[2024-08-11 09:58:43,296][05236] Num frames 7600...
[2024-08-11 09:58:43,422][05236] Num frames 7700...
[2024-08-11 09:58:43,546][05236] Num frames 7800...
[2024-08-11 09:58:43,672][05236] Num frames 7900...
[2024-08-11 09:58:43,801][05236] Num frames 8000...
[2024-08-11 09:58:43,873][05236] Avg episode rewards: #0: 21.640, true rewards: #0: 10.015
[2024-08-11 09:58:43,875][05236] Avg episode reward: 21.640, avg true_objective: 10.015
[2024-08-11 09:58:43,990][05236] Num frames 8100...
[2024-08-11 09:58:44,113][05236] Num frames 8200...
[2024-08-11 09:58:44,240][05236] Num frames 8300...
[2024-08-11 09:58:44,374][05236] Num frames 8400...
[2024-08-11 09:58:44,500][05236] Num frames 8500...
[2024-08-11 09:58:44,625][05236] Num frames 8600...
[2024-08-11 09:58:44,758][05236] Num frames 8700...
[2024-08-11 09:58:44,885][05236] Num frames 8800...
[2024-08-11 09:58:45,009][05236] Num frames 8900...
[2024-08-11 09:58:45,136][05236] Num frames 9000...
[2024-08-11 09:58:45,261][05236] Num frames 9100...
[2024-08-11 09:58:45,395][05236] Num frames 9200...
[2024-08-11 09:58:45,482][05236] Avg episode rewards: #0: 22.916, true rewards: #0: 10.249
[2024-08-11 09:58:45,484][05236] Avg episode reward: 22.916, avg true_objective: 10.249
[2024-08-11 09:58:45,580][05236] Num frames 9300...
[2024-08-11 09:58:45,711][05236] Num frames 9400...
[2024-08-11 09:58:45,835][05236] Num frames 9500...
[2024-08-11 09:58:45,958][05236] Num frames 9600...
[2024-08-11 09:58:46,079][05236] Num frames 9700...
[2024-08-11 09:58:46,201][05236] Num frames 9800...
[2024-08-11 09:58:46,254][05236] Avg episode rewards: #0: 21.600, true rewards: #0: 9.800
[2024-08-11 09:58:46,256][05236] Avg episode reward: 21.600, avg true_objective: 9.800
[2024-08-11 09:59:41,706][05236] Replay video saved to /content/train_dir/default_experiment/replay.mp4!
[2024-08-11 09:59:42,457][05236] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json
[2024-08-11 09:59:42,459][05236] Overriding arg 'num_workers' with value 1 passed from command line
[2024-08-11 09:59:42,460][05236] Adding new argument 'no_render'=True that is not in the saved config file!
[2024-08-11 09:59:42,461][05236] Adding new argument 'save_video'=True that is not in the saved config file!
[2024-08-11 09:59:42,463][05236] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
[2024-08-11 09:59:42,464][05236] Adding new argument 'video_name'=None that is not in the saved config file!
[2024-08-11 09:59:42,465][05236] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
[2024-08-11 09:59:42,467][05236] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
[2024-08-11 09:59:42,468][05236] Adding new argument 'push_to_hub'=True that is not in the saved config file!
[2024-08-11 09:59:42,470][05236] Adding new argument 'hf_repository'='ArunAIML/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
[2024-08-11 09:59:42,471][05236] Adding new argument 'policy_index'=0 that is not in the saved config file!
[2024-08-11 09:59:42,473][05236] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
[2024-08-11 09:59:42,475][05236] Adding new argument 'train_script'=None that is not in the saved config file!
[2024-08-11 09:59:42,476][05236] Adding new argument 'enjoy_script'=None that is not in the saved config file!
[2024-08-11 09:59:42,478][05236] Using frameskip 1 and render_action_repeat=4 for evaluation
[2024-08-11 09:59:42,538][05236] RunningMeanStd input shape: (3, 72, 128)
[2024-08-11 09:59:42,543][05236] RunningMeanStd input shape: (1,)
[2024-08-11 09:59:42,572][05236] ConvEncoder: input_channels=3
[2024-08-11 09:59:42,639][05236] Conv encoder output size: 512
[2024-08-11 09:59:42,644][05236] Policy head output size: 512
[2024-08-11 09:59:42,679][05236] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2024-08-11 09:59:43,451][05236] Num frames 100...
[2024-08-11 09:59:43,616][05236] Num frames 200...
[2024-08-11 09:59:43,785][05236] Num frames 300...
[2024-08-11 09:59:43,946][05236] Num frames 400...
[2024-08-11 09:59:44,107][05236] Num frames 500...
[2024-08-11 09:59:44,272][05236] Num frames 600...
[2024-08-11 09:59:44,437][05236] Num frames 700...
[2024-08-11 09:59:44,614][05236] Avg episode rewards: #0: 18.680, true rewards: #0: 7.680
[2024-08-11 09:59:44,616][05236] Avg episode reward: 18.680, avg true_objective: 7.680
[2024-08-11 09:59:44,680][05236] Num frames 800...
[2024-08-11 09:59:44,847][05236] Num frames 900...
[2024-08-11 09:59:45,009][05236] Num frames 1000...
[2024-08-11 09:59:45,176][05236] Num frames 1100...
[2024-08-11 09:59:45,344][05236] Num frames 1200...
[2024-08-11 09:59:45,524][05236] Num frames 1300...
[2024-08-11 09:59:45,713][05236] Num frames 1400...
[2024-08-11 09:59:45,884][05236] Num frames 1500...
[2024-08-11 09:59:46,047][05236] Num frames 1600...
[2024-08-11 09:59:46,219][05236] Num frames 1700...
[2024-08-11 09:59:46,407][05236] Num frames 1800...
[2024-08-11 09:59:46,626][05236] Num frames 1900...
[2024-08-11 09:59:46,836][05236] Num frames 2000...
[2024-08-11 09:59:47,036][05236] Num frames 2100...
[2024-08-11 09:59:47,221][05236] Num frames 2200...
[2024-08-11 09:59:47,406][05236] Num frames 2300...
[2024-08-11 09:59:47,589][05236] Num frames 2400...
[2024-08-11 09:59:47,801][05236] Num frames 2500...
[2024-08-11 09:59:47,984][05236] Num frames 2600...
[2024-08-11 09:59:48,153][05236] Num frames 2700...
[2024-08-11 09:59:48,322][05236] Num frames 2800...
[2024-08-11 09:59:48,498][05236] Avg episode rewards: #0: 33.840, true rewards: #0: 14.340
[2024-08-11 09:59:48,501][05236] Avg episode reward: 33.840, avg true_objective: 14.340
[2024-08-11 09:59:48,564][05236] Num frames 2900...
[2024-08-11 09:59:48,747][05236] Num frames 3000...
[2024-08-11 09:59:48,871][05236] Num frames 3100...
[2024-08-11 09:59:48,992][05236] Num frames 3200...
[2024-08-11 09:59:49,111][05236] Num frames 3300...
[2024-08-11 09:59:49,232][05236] Num frames 3400...
[2024-08-11 09:59:49,353][05236] Num frames 3500...
[2024-08-11 09:59:49,482][05236] Num frames 3600...
[2024-08-11 09:59:49,602][05236] Num frames 3700...
[2024-08-11 09:59:49,732][05236] Num frames 3800...
[2024-08-11 09:59:49,864][05236] Num frames 3900...
[2024-08-11 09:59:49,986][05236] Num frames 4000...
[2024-08-11 09:59:50,105][05236] Num frames 4100...
[2024-08-11 09:59:50,226][05236] Num frames 4200...
[2024-08-11 09:59:50,349][05236] Num frames 4300...
[2024-08-11 09:59:50,478][05236] Num frames 4400...
[2024-08-11 09:59:50,601][05236] Num frames 4500...
[2024-08-11 09:59:50,731][05236] Num frames 4600...
[2024-08-11 09:59:50,866][05236] Num frames 4700...
[2024-08-11 09:59:50,989][05236] Num frames 4800...
[2024-08-11 09:59:51,113][05236] Num frames 4900...
[2024-08-11 09:59:51,252][05236] Avg episode rewards: #0: 42.226, true rewards: #0: 16.560
[2024-08-11 09:59:51,253][05236] Avg episode reward: 42.226, avg true_objective: 16.560
[2024-08-11 09:59:51,297][05236] Num frames 5000...
[2024-08-11 09:59:51,422][05236] Num frames 5100...
[2024-08-11 09:59:51,549][05236] Num frames 5200...
[2024-08-11 09:59:51,651][05236] Avg episode rewards: #0: 32.840, true rewards: #0: 13.090
[2024-08-11 09:59:51,652][05236] Avg episode reward: 32.840, avg true_objective: 13.090
[2024-08-11 09:59:51,744][05236] Num frames 5300...
[2024-08-11 09:59:51,875][05236] Num frames 5400...
[2024-08-11 09:59:51,996][05236] Num frames 5500...
[2024-08-11 09:59:52,116][05236] Num frames 5600...
[2024-08-11 09:59:52,238][05236] Num frames 5700...
[2024-08-11 09:59:52,327][05236] Avg episode rewards: #0: 28.056, true rewards: #0: 11.456
[2024-08-11 09:59:52,330][05236] Avg episode reward: 28.056, avg true_objective: 11.456
[2024-08-11 09:59:52,422][05236] Num frames 5800...
[2024-08-11 09:59:52,548][05236] Num frames 5900...
[2024-08-11 09:59:52,673][05236] Num frames 6000...
[2024-08-11 09:59:52,804][05236] Num frames 6100...
[2024-08-11 09:59:52,937][05236] Num frames 6200...
[2024-08-11 09:59:53,061][05236] Num frames 6300...
[2024-08-11 09:59:53,182][05236] Num frames 6400...
[2024-08-11 09:59:53,307][05236] Num frames 6500...
[2024-08-11 09:59:53,480][05236] Num frames 6600...
[2024-08-11 09:59:53,654][05236] Num frames 6700...
[2024-08-11 09:59:53,817][05236] Num frames 6800...
[2024-08-11 09:59:53,984][05236] Num frames 6900...
[2024-08-11 09:59:54,147][05236] Num frames 7000...
[2024-08-11 09:59:54,305][05236] Num frames 7100...
[2024-08-11 09:59:54,493][05236] Avg episode rewards: #0: 29.466, true rewards: #0: 11.967
[2024-08-11 09:59:54,499][05236] Avg episode reward: 29.466, avg true_objective: 11.967
[2024-08-11 09:59:54,541][05236] Num frames 7200...
[2024-08-11 09:59:54,718][05236] Num frames 7300...
[2024-08-11 09:59:54,899][05236] Num frames 7400...
[2024-08-11 09:59:55,069][05236] Num frames 7500...
[2024-08-11 09:59:55,240][05236] Num frames 7600...
[2024-08-11 09:59:55,416][05236] Num frames 7700...
[2024-08-11 09:59:55,589][05236] Num frames 7800...
[2024-08-11 09:59:55,760][05236] Num frames 7900...
[2024-08-11 09:59:55,949][05236] Avg episode rewards: #0: 27.400, true rewards: #0: 11.400
[2024-08-11 09:59:55,951][05236] Avg episode reward: 27.400, avg true_objective: 11.400
[2024-08-11 09:59:55,979][05236] Num frames 8000...
[2024-08-11 09:59:56,100][05236] Num frames 8100...
[2024-08-11 09:59:56,231][05236] Num frames 8200...
[2024-08-11 09:59:56,368][05236] Avg episode rewards: #0: 24.585, true rewards: #0: 10.335
[2024-08-11 09:59:56,370][05236] Avg episode reward: 24.585, avg true_objective: 10.335
[2024-08-11 09:59:56,416][05236] Num frames 8300...
[2024-08-11 09:59:56,538][05236] Num frames 8400...
[2024-08-11 09:59:56,661][05236] Num frames 8500...
[2024-08-11 09:59:56,798][05236] Num frames 8600...
[2024-08-11 09:59:56,925][05236] Num frames 8700...
[2024-08-11 09:59:57,053][05236] Num frames 8800...
[2024-08-11 09:59:57,177][05236] Num frames 8900...
[2024-08-11 09:59:57,300][05236] Num frames 9000...
[2024-08-11 09:59:57,434][05236] Num frames 9100...
[2024-08-11 09:59:57,561][05236] Num frames 9200...
[2024-08-11 09:59:57,694][05236] Num frames 9300...
[2024-08-11 09:59:57,815][05236] Num frames 9400...
[2024-08-11 09:59:57,938][05236] Num frames 9500...
[2024-08-11 09:59:58,056][05236] Avg episode rewards: #0: 25.271, true rewards: #0: 10.604
[2024-08-11 09:59:58,058][05236] Avg episode reward: 25.271, avg true_objective: 10.604
[2024-08-11 09:59:58,132][05236] Num frames 9600...
[2024-08-11 09:59:58,256][05236] Num frames 9700...
[2024-08-11 09:59:58,376][05236] Num frames 9800...
[2024-08-11 09:59:58,503][05236] Num frames 9900...
[2024-08-11 09:59:58,623][05236] Num frames 10000...
[2024-08-11 09:59:58,754][05236] Num frames 10100...
[2024-08-11 09:59:58,875][05236] Num frames 10200...
[2024-08-11 09:59:58,995][05236] Num frames 10300...
[2024-08-11 09:59:59,138][05236] Num frames 10400...
[2024-08-11 09:59:59,302][05236] Num frames 10500...
[2024-08-11 09:59:59,427][05236] Num frames 10600...
[2024-08-11 09:59:59,550][05236] Num frames 10700...
[2024-08-11 09:59:59,677][05236] Num frames 10800...
[2024-08-11 09:59:59,807][05236] Num frames 10900...
[2024-08-11 09:59:59,890][05236] Avg episode rewards: #0: 26.020, true rewards: #0: 10.920
[2024-08-11 09:59:59,893][05236] Avg episode reward: 26.020, avg true_objective: 10.920
[2024-08-11 10:01:00,801][05236] Replay video saved to /content/train_dir/default_experiment/replay.mp4!
[2024-08-11 10:02:36,973][05236] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json
[2024-08-11 10:02:36,975][05236] Overriding arg 'num_workers' with value 1 passed from command line
[2024-08-11 10:02:36,977][05236] Adding new argument 'no_render'=True that is not in the saved config file!
[2024-08-11 10:02:36,978][05236] Adding new argument 'save_video'=True that is not in the saved config file!
[2024-08-11 10:02:36,980][05236] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
[2024-08-11 10:02:36,982][05236] Adding new argument 'video_name'=None that is not in the saved config file!
[2024-08-11 10:02:36,984][05236] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
[2024-08-11 10:02:36,985][05236] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
[2024-08-11 10:02:36,986][05236] Adding new argument 'push_to_hub'=True that is not in the saved config file!
[2024-08-11 10:02:36,987][05236] Adding new argument 'hf_repository'='ArunAIML/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
[2024-08-11 10:02:36,988][05236] Adding new argument 'policy_index'=0 that is not in the saved config file!
[2024-08-11 10:02:36,989][05236] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
[2024-08-11 10:02:36,990][05236] Adding new argument 'train_script'=None that is not in the saved config file!
[2024-08-11 10:02:36,991][05236] Adding new argument 'enjoy_script'=None that is not in the saved config file!
[2024-08-11 10:02:36,992][05236] Using frameskip 1 and render_action_repeat=4 for evaluation
[2024-08-11 10:02:37,026][05236] RunningMeanStd input shape: (3, 72, 128)
[2024-08-11 10:02:37,027][05236] RunningMeanStd input shape: (1,)
[2024-08-11 10:02:37,041][05236] ConvEncoder: input_channels=3
[2024-08-11 10:02:37,077][05236] Conv encoder output size: 512
[2024-08-11 10:02:37,078][05236] Policy head output size: 512
[2024-08-11 10:02:37,097][05236] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
[2024-08-11 10:02:37,517][05236] Num frames 100...
[2024-08-11 10:02:37,636][05236] Num frames 200...
[2024-08-11 10:02:37,778][05236] Num frames 300...
[2024-08-11 10:02:37,920][05236] Num frames 400...
[2024-08-11 10:02:38,040][05236] Num frames 500...
[2024-08-11 10:02:38,166][05236] Num frames 600...
[2024-08-11 10:02:38,290][05236] Num frames 700...
[2024-08-11 10:02:38,416][05236] Num frames 800...
[2024-08-11 10:02:38,541][05236] Num frames 900...
[2024-08-11 10:02:38,668][05236] Num frames 1000...
[2024-08-11 10:02:38,797][05236] Num frames 1100...
[2024-08-11 10:02:38,930][05236] Num frames 1200...
[2024-08-11 10:02:39,057][05236] Num frames 1300...
[2024-08-11 10:02:39,183][05236] Num frames 1400...
[2024-08-11 10:02:39,308][05236] Num frames 1500...
[2024-08-11 10:02:39,433][05236] Num frames 1600...
[2024-08-11 10:02:39,557][05236] Num frames 1700...
[2024-08-11 10:02:39,693][05236] Num frames 1800...
[2024-08-11 10:02:39,819][05236] Num frames 1900...
[2024-08-11 10:02:39,951][05236] Num frames 2000...
[2024-08-11 10:02:40,083][05236] Num frames 2100...
[2024-08-11 10:02:40,137][05236] Avg episode rewards: #0: 55.999, true rewards: #0: 21.000
[2024-08-11 10:02:40,139][05236] Avg episode reward: 55.999, avg true_objective: 21.000
[2024-08-11 10:02:40,264][05236] Num frames 2200...
[2024-08-11 10:02:40,387][05236] Num frames 2300...
[2024-08-11 10:02:40,513][05236] Num frames 2400...
[2024-08-11 10:02:40,640][05236] Num frames 2500...
[2024-08-11 10:02:40,774][05236] Num frames 2600...
[2024-08-11 10:02:40,900][05236] Num frames 2700...
[2024-08-11 10:02:41,035][05236] Num frames 2800...
[2024-08-11 10:02:41,161][05236] Num frames 2900...
[2024-08-11 10:02:41,286][05236] Num frames 3000...
[2024-08-11 10:02:41,410][05236] Num frames 3100...
[2024-08-11 10:02:41,533][05236] Num frames 3200...
[2024-08-11 10:02:41,660][05236] Num frames 3300...
[2024-08-11 10:02:41,790][05236] Num frames 3400...
[2024-08-11 10:02:41,919][05236] Num frames 3500...
[2024-08-11 10:02:42,025][05236] Avg episode rewards: #0: 42.700, true rewards: #0: 17.700
[2024-08-11 10:02:42,027][05236] Avg episode reward: 42.700, avg true_objective: 17.700
[2024-08-11 10:02:42,103][05236] Num frames 3600...
[2024-08-11 10:02:42,224][05236] Num frames 3700...
[2024-08-11 10:02:42,345][05236] Num frames 3800...
[2024-08-11 10:02:42,470][05236] Num frames 3900...
[2024-08-11 10:02:42,594][05236] Num frames 4000...
[2024-08-11 10:02:42,745][05236] Num frames 4100...
[2024-08-11 10:02:42,914][05236] Num frames 4200...
[2024-08-11 10:02:42,994][05236] Avg episode rewards: #0: 32.373, true rewards: #0: 14.040
[2024-08-11 10:02:42,996][05236] Avg episode reward: 32.373, avg true_objective: 14.040
[2024-08-11 10:02:43,149][05236] Num frames 4300...
[2024-08-11 10:02:43,311][05236] Num frames 4400...
[2024-08-11 10:02:43,477][05236] Num frames 4500...
[2024-08-11 10:02:43,644][05236] Num frames 4600...
[2024-08-11 10:02:43,816][05236] Num frames 4700...
[2024-08-11 10:02:43,987][05236] Num frames 4800...
[2024-08-11 10:02:44,178][05236] Num frames 4900...
[2024-08-11 10:02:44,348][05236] Num frames 5000...
[2024-08-11 10:02:44,526][05236] Num frames 5100...
[2024-08-11 10:02:44,707][05236] Num frames 5200...
[2024-08-11 10:02:44,879][05236] Num frames 5300...
[2024-08-11 10:02:45,049][05236] Num frames 5400...
[2024-08-11 10:02:45,240][05236] Num frames 5500...
[2024-08-11 10:02:45,392][05236] Num frames 5600...
[2024-08-11 10:02:45,523][05236] Num frames 5700...
[2024-08-11 10:02:45,646][05236] Num frames 5800...
[2024-08-11 10:02:45,781][05236] Num frames 5900...
[2024-08-11 10:02:45,906][05236] Num frames 6000...
[2024-08-11 10:02:46,027][05236] Num frames 6100...
[2024-08-11 10:02:46,164][05236] Num frames 6200...
[2024-08-11 10:02:46,275][05236] Avg episode rewards: #0: 37.857, true rewards: #0: 15.608
[2024-08-11 10:02:46,277][05236] Avg episode reward: 37.857, avg true_objective: 15.608
[2024-08-11 10:02:46,349][05236] Num frames 6300...
[2024-08-11 10:02:46,474][05236] Num frames 6400...
[2024-08-11 10:02:46,600][05236] Num frames 6500...
[2024-08-11 10:02:46,742][05236] Num frames 6600...
[2024-08-11 10:02:46,870][05236] Num frames 6700...
[2024-08-11 10:02:46,995][05236] Num frames 6800...
[2024-08-11 10:02:47,117][05236] Num frames 6900...
[2024-08-11 10:02:47,250][05236] Num frames 7000...
[2024-08-11 10:02:47,373][05236] Num frames 7100...
[2024-08-11 10:02:47,528][05236] Avg episode rewards: #0: 33.942, true rewards: #0: 14.342
[2024-08-11 10:02:47,530][05236] Avg episode reward: 33.942, avg true_objective: 14.342
[2024-08-11 10:02:47,570][05236] Num frames 7200...
[2024-08-11 10:02:47,704][05236] Num frames 7300...
[2024-08-11 10:02:47,830][05236] Num frames 7400...
[2024-08-11 10:02:47,957][05236] Avg episode rewards: #0: 29.098, true rewards: #0: 12.432
[2024-08-11 10:02:47,959][05236] Avg episode reward: 29.098, avg true_objective: 12.432
[2024-08-11 10:02:48,010][05236] Num frames 7500...
[2024-08-11 10:02:48,136][05236] Num frames 7600...
[2024-08-11 10:02:48,268][05236] Num frames 7700...
[2024-08-11 10:02:48,391][05236] Num frames 7800...
[2024-08-11 10:02:48,512][05236] Num frames 7900...
[2024-08-11 10:02:48,630][05236] Num frames 8000...
[2024-08-11 10:02:48,760][05236] Num frames 8100...
[2024-08-11 10:02:48,880][05236] Num frames 8200...
[2024-08-11 10:02:48,999][05236] Num frames 8300...
[2024-08-11 10:02:49,125][05236] Num frames 8400...
[2024-08-11 10:02:49,257][05236] Num frames 8500...
[2024-08-11 10:02:49,380][05236] Num frames 8600...
[2024-08-11 10:02:49,505][05236] Num frames 8700...
[2024-08-11 10:02:49,628][05236] Num frames 8800...
[2024-08-11 10:02:49,761][05236] Num frames 8900...
[2024-08-11 10:02:49,886][05236] Num frames 9000...
[2024-08-11 10:02:50,010][05236] Num frames 9100...
[2024-08-11 10:02:50,131][05236] Num frames 9200...
[2024-08-11 10:02:50,261][05236] Num frames 9300...
[2024-08-11 10:02:50,421][05236] Avg episode rewards: #0: 31.405, true rewards: #0: 13.406
[2024-08-11 10:02:50,423][05236] Avg episode reward: 31.405, avg true_objective: 13.406
[2024-08-11 10:02:50,447][05236] Num frames 9400...
[2024-08-11 10:02:50,572][05236] Num frames 9500...
[2024-08-11 10:02:50,703][05236] Num frames 9600...
[2024-08-11 10:02:50,828][05236] Num frames 9700...
[2024-08-11 10:02:50,951][05236] Num frames 9800...
[2024-08-11 10:02:51,072][05236] Num frames 9900...
[2024-08-11 10:02:51,193][05236] Num frames 10000...
[2024-08-11 10:02:51,324][05236] Num frames 10100...
[2024-08-11 10:02:51,448][05236] Num frames 10200...
[2024-08-11 10:02:51,570][05236] Num frames 10300...
[2024-08-11 10:02:51,672][05236] Avg episode rewards: #0: 29.671, true rewards: #0: 12.921
[2024-08-11 10:02:51,674][05236] Avg episode reward: 29.671, avg true_objective: 12.921
[2024-08-11 10:02:51,756][05236] Num frames 10400...
[2024-08-11 10:02:51,881][05236] Num frames 10500...
[2024-08-11 10:02:52,003][05236] Num frames 10600...
[2024-08-11 10:02:52,124][05236] Num frames 10700...
[2024-08-11 10:02:52,244][05236] Num frames 10800...
[2024-08-11 10:02:52,373][05236] Num frames 10900...
[2024-08-11 10:02:52,496][05236] Num frames 11000...
[2024-08-11 10:02:52,619][05236] Num frames 11100...
[2024-08-11 10:02:52,750][05236] Num frames 11200...
[2024-08-11 10:02:52,894][05236] Avg episode rewards: #0: 28.410, true rewards: #0: 12.521
[2024-08-11 10:02:52,896][05236] Avg episode reward: 28.410, avg true_objective: 12.521
[2024-08-11 10:02:52,936][05236] Num frames 11300...
[2024-08-11 10:02:53,057][05236] Num frames 11400...
[2024-08-11 10:02:53,186][05236] Num frames 11500...
[2024-08-11 10:02:53,316][05236] Num frames 11600...
[2024-08-11 10:02:53,444][05236] Num frames 11700...
[2024-08-11 10:02:53,569][05236] Num frames 11800...
[2024-08-11 10:02:53,697][05236] Num frames 11900...
[2024-08-11 10:02:53,817][05236] Num frames 12000...
[2024-08-11 10:02:53,939][05236] Num frames 12100...
[2024-08-11 10:02:54,060][05236] Num frames 12200...
[2024-08-11 10:02:54,157][05236] Avg episode rewards: #0: 28.135, true rewards: #0: 12.235
[2024-08-11 10:02:54,160][05236] Avg episode reward: 28.135, avg true_objective: 12.235
[2024-08-11 10:04:02,945][05236] Replay video saved to /content/train_dir/default_experiment/replay.mp4!