| [2024-09-06 07:54:51,174][01070] Saving configuration to /content/train_dir/default_experiment/config.json... | |
| [2024-09-06 07:54:51,178][01070] Rollout worker 0 uses device cpu | |
| [2024-09-06 07:54:51,179][01070] Rollout worker 1 uses device cpu | |
| [2024-09-06 07:54:51,182][01070] Rollout worker 2 uses device cpu | |
| [2024-09-06 07:54:51,183][01070] Rollout worker 3 uses device cpu | |
| [2024-09-06 07:54:51,184][01070] Rollout worker 4 uses device cpu | |
| [2024-09-06 07:54:51,185][01070] Rollout worker 5 uses device cpu | |
| [2024-09-06 07:54:51,186][01070] Rollout worker 6 uses device cpu | |
| [2024-09-06 07:54:51,187][01070] Rollout worker 7 uses device cpu | |
| [2024-09-06 07:54:51,345][01070] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 07:54:51,348][01070] InferenceWorker_p0-w0: min num requests: 2 | |
| [2024-09-06 07:54:51,381][01070] Starting all processes... | |
| [2024-09-06 07:54:51,382][01070] Starting process learner_proc0 | |
| [2024-09-06 07:54:52,097][01070] Starting all processes... | |
| [2024-09-06 07:54:52,107][01070] Starting process inference_proc0-0 | |
| [2024-09-06 07:54:52,108][01070] Starting process rollout_proc0 | |
| [2024-09-06 07:54:52,108][01070] Starting process rollout_proc1 | |
| [2024-09-06 07:54:52,108][01070] Starting process rollout_proc2 | |
| [2024-09-06 07:54:52,108][01070] Starting process rollout_proc3 | |
| [2024-09-06 07:54:52,108][01070] Starting process rollout_proc4 | |
| [2024-09-06 07:54:52,108][01070] Starting process rollout_proc5 | |
| [2024-09-06 07:54:52,108][01070] Starting process rollout_proc6 | |
| [2024-09-06 07:54:52,108][01070] Starting process rollout_proc7 | |
| [2024-09-06 07:55:08,667][06068] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 07:55:08,668][06068] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2024-09-06 07:55:08,754][06069] Worker 0 uses CPU cores [0] | |
| [2024-09-06 07:55:08,849][06068] Num visible devices: 1 | |
| [2024-09-06 07:55:08,857][06055] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 07:55:08,862][06055] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2024-09-06 07:55:08,865][06072] Worker 3 uses CPU cores [1] | |
| [2024-09-06 07:55:08,930][06073] Worker 4 uses CPU cores [0] | |
| [2024-09-06 07:55:08,955][06055] Num visible devices: 1 | |
| [2024-09-06 07:55:08,995][06055] Starting seed is not provided | |
| [2024-09-06 07:55:08,996][06055] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 07:55:08,997][06055] Initializing actor-critic model on device cuda:0 | |
| [2024-09-06 07:55:08,998][06055] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 07:55:09,001][06055] RunningMeanStd input shape: (1,) | |
| [2024-09-06 07:55:09,045][06075] Worker 6 uses CPU cores [0] | |
| [2024-09-06 07:55:09,054][06074] Worker 5 uses CPU cores [1] | |
| [2024-09-06 07:55:09,100][06055] ConvEncoder: input_channels=3 | |
| [2024-09-06 07:55:09,170][06071] Worker 2 uses CPU cores [0] | |
| [2024-09-06 07:55:09,188][06076] Worker 7 uses CPU cores [1] | |
| [2024-09-06 07:55:09,206][06070] Worker 1 uses CPU cores [1] | |
| [2024-09-06 07:55:09,434][06055] Conv encoder output size: 512 | |
| [2024-09-06 07:55:09,434][06055] Policy head output size: 512 | |
| [2024-09-06 07:55:09,504][06055] Created Actor Critic model with architecture: | |
| [2024-09-06 07:55:09,505][06055] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2024-09-06 07:55:09,884][06055] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2024-09-06 07:55:10,611][06055] No checkpoints found | |
| [2024-09-06 07:55:10,612][06055] Did not load from checkpoint, starting from scratch! | |
| [2024-09-06 07:55:10,612][06055] Initialized policy 0 weights for model version 0 | |
| [2024-09-06 07:55:10,618][06055] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 07:55:10,624][06055] LearnerWorker_p0 finished initialization! | |
| [2024-09-06 07:55:10,711][06068] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 07:55:10,713][06068] RunningMeanStd input shape: (1,) | |
| [2024-09-06 07:55:10,725][06068] ConvEncoder: input_channels=3 | |
| [2024-09-06 07:55:10,825][06068] Conv encoder output size: 512 | |
| [2024-09-06 07:55:10,826][06068] Policy head output size: 512 | |
| [2024-09-06 07:55:10,875][01070] Inference worker 0-0 is ready! | |
| [2024-09-06 07:55:10,877][01070] All inference workers are ready! Signal rollout workers to start! | |
| [2024-09-06 07:55:11,084][06071] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 07:55:11,088][06069] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 07:55:11,090][06075] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 07:55:11,099][06076] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 07:55:11,093][06073] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 07:55:11,101][06074] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 07:55:11,097][06072] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 07:55:11,109][06070] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 07:55:11,317][01070] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 07:55:11,338][01070] Heartbeat connected on Batcher_0 | |
| [2024-09-06 07:55:11,342][01070] Heartbeat connected on LearnerWorker_p0 | |
| [2024-09-06 07:55:11,375][01070] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2024-09-06 07:55:12,668][06071] Decorrelating experience for 0 frames... | |
| [2024-09-06 07:55:12,666][06069] Decorrelating experience for 0 frames... | |
| [2024-09-06 07:55:12,669][06075] Decorrelating experience for 0 frames... | |
| [2024-09-06 07:55:12,943][06076] Decorrelating experience for 0 frames... | |
| [2024-09-06 07:55:12,952][06072] Decorrelating experience for 0 frames... | |
| [2024-09-06 07:55:12,955][06074] Decorrelating experience for 0 frames... | |
| [2024-09-06 07:55:12,961][06070] Decorrelating experience for 0 frames... | |
| [2024-09-06 07:55:13,724][06076] Decorrelating experience for 32 frames... | |
| [2024-09-06 07:55:13,727][06074] Decorrelating experience for 32 frames... | |
| [2024-09-06 07:55:14,072][06071] Decorrelating experience for 32 frames... | |
| [2024-09-06 07:55:14,074][06069] Decorrelating experience for 32 frames... | |
| [2024-09-06 07:55:14,077][06075] Decorrelating experience for 32 frames... | |
| [2024-09-06 07:55:14,540][06073] Decorrelating experience for 0 frames... | |
| [2024-09-06 07:55:14,891][06070] Decorrelating experience for 32 frames... | |
| [2024-09-06 07:55:15,287][06074] Decorrelating experience for 64 frames... | |
| [2024-09-06 07:55:15,383][06072] Decorrelating experience for 32 frames... | |
| [2024-09-06 07:55:15,566][06071] Decorrelating experience for 64 frames... | |
| [2024-09-06 07:55:15,585][06069] Decorrelating experience for 64 frames... | |
| [2024-09-06 07:55:16,059][06075] Decorrelating experience for 64 frames... | |
| [2024-09-06 07:55:16,321][01070] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 07:55:16,546][06071] Decorrelating experience for 96 frames... | |
| [2024-09-06 07:55:16,581][06070] Decorrelating experience for 64 frames... | |
| [2024-09-06 07:55:16,677][06076] Decorrelating experience for 64 frames... | |
| [2024-09-06 07:55:16,724][06074] Decorrelating experience for 96 frames... | |
| [2024-09-06 07:55:16,739][01070] Heartbeat connected on RolloutWorker_w2 | |
| [2024-09-06 07:55:16,988][01070] Heartbeat connected on RolloutWorker_w5 | |
| [2024-09-06 07:55:17,186][06075] Decorrelating experience for 96 frames... | |
| [2024-09-06 07:55:17,328][01070] Heartbeat connected on RolloutWorker_w6 | |
| [2024-09-06 07:55:17,605][06069] Decorrelating experience for 96 frames... | |
| [2024-09-06 07:55:17,810][01070] Heartbeat connected on RolloutWorker_w0 | |
| [2024-09-06 07:55:17,989][06072] Decorrelating experience for 64 frames... | |
| [2024-09-06 07:55:17,992][06070] Decorrelating experience for 96 frames... | |
| [2024-09-06 07:55:18,111][06076] Decorrelating experience for 96 frames... | |
| [2024-09-06 07:55:18,212][01070] Heartbeat connected on RolloutWorker_w1 | |
| [2024-09-06 07:55:18,296][01070] Heartbeat connected on RolloutWorker_w7 | |
| [2024-09-06 07:55:18,667][06073] Decorrelating experience for 32 frames... | |
| [2024-09-06 07:55:18,736][06072] Decorrelating experience for 96 frames... | |
| [2024-09-06 07:55:18,820][01070] Heartbeat connected on RolloutWorker_w3 | |
| [2024-09-06 07:55:20,114][06073] Decorrelating experience for 64 frames... | |
| [2024-09-06 07:55:21,318][01070] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 91.4. Samples: 914. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 07:55:21,321][01070] Avg episode reward: [(0, '1.348')] | |
| [2024-09-06 07:55:22,967][06055] Signal inference workers to stop experience collection... | |
| [2024-09-06 07:55:22,981][06068] InferenceWorker_p0-w0: stopping experience collection | |
| [2024-09-06 07:55:23,292][06073] Decorrelating experience for 96 frames... | |
| [2024-09-06 07:55:23,475][01070] Heartbeat connected on RolloutWorker_w4 | |
| [2024-09-06 07:55:26,317][01070] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 152.5. Samples: 2288. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 07:55:26,322][01070] Avg episode reward: [(0, '2.368')] | |
| [2024-09-06 07:55:26,933][06055] Signal inference workers to resume experience collection... | |
| [2024-09-06 07:55:26,935][06068] InferenceWorker_p0-w0: resuming experience collection | |
| [2024-09-06 07:55:31,317][01070] Fps is (10 sec: 2458.0, 60 sec: 1228.8, 300 sec: 1228.8). Total num frames: 24576. Throughput: 0: 209.5. Samples: 4190. Policy #0 lag: (min: 0.0, avg: 0.4, max: 3.0) | |
| [2024-09-06 07:55:31,322][01070] Avg episode reward: [(0, '3.545')] | |
| [2024-09-06 07:55:34,552][06068] Updated weights for policy 0, policy_version 10 (0.0173) | |
| [2024-09-06 07:55:36,317][01070] Fps is (10 sec: 4505.4, 60 sec: 1802.2, 300 sec: 1802.2). Total num frames: 45056. Throughput: 0: 445.8. Samples: 11144. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 07:55:36,323][01070] Avg episode reward: [(0, '4.241')] | |
| [2024-09-06 07:55:41,319][01070] Fps is (10 sec: 3276.0, 60 sec: 1911.3, 300 sec: 1911.3). Total num frames: 57344. Throughput: 0: 511.6. Samples: 15350. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 07:55:41,324][01070] Avg episode reward: [(0, '4.401')] | |
| [2024-09-06 07:55:46,317][01070] Fps is (10 sec: 3276.8, 60 sec: 2223.5, 300 sec: 2223.5). Total num frames: 77824. Throughput: 0: 500.7. Samples: 17526. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:55:46,319][01070] Avg episode reward: [(0, '4.396')] | |
| [2024-09-06 07:55:47,170][06068] Updated weights for policy 0, policy_version 20 (0.0030) | |
| [2024-09-06 07:55:51,317][01070] Fps is (10 sec: 4097.0, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 98304. Throughput: 0: 607.7. Samples: 24308. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 07:55:51,319][01070] Avg episode reward: [(0, '4.360')] | |
| [2024-09-06 07:55:56,317][01070] Fps is (10 sec: 4096.0, 60 sec: 2639.6, 300 sec: 2639.6). Total num frames: 118784. Throughput: 0: 684.4. Samples: 30798. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 07:55:56,321][01070] Avg episode reward: [(0, '4.546')] | |
| [2024-09-06 07:55:56,332][06055] Saving new best policy, reward=4.546! | |
| [2024-09-06 07:55:56,696][06068] Updated weights for policy 0, policy_version 30 (0.0020) | |
| [2024-09-06 07:56:01,317][01070] Fps is (10 sec: 3686.4, 60 sec: 2703.4, 300 sec: 2703.4). Total num frames: 135168. Throughput: 0: 728.7. Samples: 32786. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:56:01,321][01070] Avg episode reward: [(0, '4.463')] | |
| [2024-09-06 07:56:06,316][01070] Fps is (10 sec: 4096.2, 60 sec: 2904.4, 300 sec: 2904.4). Total num frames: 159744. Throughput: 0: 840.3. Samples: 38726. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 07:56:06,319][01070] Avg episode reward: [(0, '4.541')] | |
| [2024-09-06 07:56:07,255][06068] Updated weights for policy 0, policy_version 40 (0.0021) | |
| [2024-09-06 07:56:11,317][01070] Fps is (10 sec: 3686.4, 60 sec: 2867.2, 300 sec: 2867.2). Total num frames: 172032. Throughput: 0: 933.5. Samples: 44294. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:56:11,325][01070] Avg episode reward: [(0, '4.557')] | |
| [2024-09-06 07:56:11,327][06055] Saving new best policy, reward=4.557! | |
| [2024-09-06 07:56:16,317][01070] Fps is (10 sec: 2457.4, 60 sec: 3072.2, 300 sec: 2835.7). Total num frames: 184320. Throughput: 0: 930.2. Samples: 46048. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:56:16,322][01070] Avg episode reward: [(0, '4.495')] | |
| [2024-09-06 07:56:21,317][01070] Fps is (10 sec: 2867.2, 60 sec: 3345.1, 300 sec: 2867.2). Total num frames: 200704. Throughput: 0: 859.7. Samples: 49832. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 07:56:21,319][01070] Avg episode reward: [(0, '4.274')] | |
| [2024-09-06 07:56:21,856][06068] Updated weights for policy 0, policy_version 50 (0.0042) | |
| [2024-09-06 07:56:26,320][01070] Fps is (10 sec: 4094.9, 60 sec: 3754.4, 300 sec: 3003.6). Total num frames: 225280. Throughput: 0: 917.1. Samples: 56622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:56:26,326][01070] Avg episode reward: [(0, '4.304')] | |
| [2024-09-06 07:56:30,559][06068] Updated weights for policy 0, policy_version 60 (0.0031) | |
| [2024-09-06 07:56:31,316][01070] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3072.0). Total num frames: 245760. Throughput: 0: 946.4. Samples: 60114. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 07:56:31,322][01070] Avg episode reward: [(0, '4.369')] | |
| [2024-09-06 07:56:36,317][01070] Fps is (10 sec: 3687.7, 60 sec: 3618.2, 300 sec: 3084.0). Total num frames: 262144. Throughput: 0: 907.4. Samples: 65140. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:56:36,319][01070] Avg episode reward: [(0, '4.407')] | |
| [2024-09-06 07:56:41,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3754.8, 300 sec: 3140.3). Total num frames: 282624. Throughput: 0: 891.6. Samples: 70920. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:56:41,318][01070] Avg episode reward: [(0, '4.396')] | |
| [2024-09-06 07:56:42,046][06068] Updated weights for policy 0, policy_version 70 (0.0046) | |
| [2024-09-06 07:56:46,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3233.7). Total num frames: 307200. Throughput: 0: 924.6. Samples: 74394. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 07:56:46,323][01070] Avg episode reward: [(0, '4.273')] | |
| [2024-09-06 07:56:46,333][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000075_307200.pth... | |
| [2024-09-06 07:56:51,317][01070] Fps is (10 sec: 3686.3, 60 sec: 3686.4, 300 sec: 3194.9). Total num frames: 319488. Throughput: 0: 923.6. Samples: 80288. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 07:56:51,321][01070] Avg episode reward: [(0, '4.403')] | |
| [2024-09-06 07:56:53,243][06068] Updated weights for policy 0, policy_version 80 (0.0022) | |
| [2024-09-06 07:56:56,317][01070] Fps is (10 sec: 2867.2, 60 sec: 3618.2, 300 sec: 3198.8). Total num frames: 335872. Throughput: 0: 904.1. Samples: 84980. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:56:56,321][01070] Avg episode reward: [(0, '4.573')] | |
| [2024-09-06 07:56:56,329][06055] Saving new best policy, reward=4.573! | |
| [2024-09-06 07:57:01,316][01070] Fps is (10 sec: 4096.1, 60 sec: 3754.7, 300 sec: 3276.8). Total num frames: 360448. Throughput: 0: 940.5. Samples: 88368. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 07:57:01,325][01070] Avg episode reward: [(0, '4.464')] | |
| [2024-09-06 07:57:02,726][06068] Updated weights for policy 0, policy_version 90 (0.0031) | |
| [2024-09-06 07:57:06,316][01070] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3312.4). Total num frames: 380928. Throughput: 0: 1012.4. Samples: 95392. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 07:57:06,319][01070] Avg episode reward: [(0, '4.686')] | |
| [2024-09-06 07:57:06,327][06055] Saving new best policy, reward=4.686! | |
| [2024-09-06 07:57:11,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3276.8). Total num frames: 393216. Throughput: 0: 953.2. Samples: 99512. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 07:57:11,320][01070] Avg episode reward: [(0, '4.739')] | |
| [2024-09-06 07:57:11,330][06055] Saving new best policy, reward=4.739! | |
| [2024-09-06 07:57:14,413][06068] Updated weights for policy 0, policy_version 100 (0.0021) | |
| [2024-09-06 07:57:16,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.3, 300 sec: 3342.3). Total num frames: 417792. Throughput: 0: 941.3. Samples: 102472. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 07:57:16,324][01070] Avg episode reward: [(0, '4.631')] | |
| [2024-09-06 07:57:21,317][01070] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3402.8). Total num frames: 442368. Throughput: 0: 983.4. Samples: 109392. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 07:57:21,324][01070] Avg episode reward: [(0, '4.517')] | |
| [2024-09-06 07:57:23,713][06068] Updated weights for policy 0, policy_version 110 (0.0022) | |
| [2024-09-06 07:57:26,318][01070] Fps is (10 sec: 3685.8, 60 sec: 3823.1, 300 sec: 3367.8). Total num frames: 454656. Throughput: 0: 971.4. Samples: 114634. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:57:26,321][01070] Avg episode reward: [(0, '4.538')] | |
| [2024-09-06 07:57:31,316][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3393.8). Total num frames: 475136. Throughput: 0: 942.6. Samples: 116810. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:57:31,323][01070] Avg episode reward: [(0, '4.571')] | |
| [2024-09-06 07:57:34,630][06068] Updated weights for policy 0, policy_version 120 (0.0029) | |
| [2024-09-06 07:57:36,317][01070] Fps is (10 sec: 4096.7, 60 sec: 3891.2, 300 sec: 3418.0). Total num frames: 495616. Throughput: 0: 962.0. Samples: 123580. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 07:57:36,324][01070] Avg episode reward: [(0, '4.726')] | |
| [2024-09-06 07:57:41,317][01070] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3440.6). Total num frames: 516096. Throughput: 0: 999.0. Samples: 129936. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 07:57:41,323][01070] Avg episode reward: [(0, '4.768')] | |
| [2024-09-06 07:57:41,328][06055] Saving new best policy, reward=4.768! | |
| [2024-09-06 07:57:46,181][06068] Updated weights for policy 0, policy_version 130 (0.0043) | |
| [2024-09-06 07:57:46,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3435.4). Total num frames: 532480. Throughput: 0: 968.0. Samples: 131926. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 07:57:46,324][01070] Avg episode reward: [(0, '4.814')] | |
| [2024-09-06 07:57:46,339][06055] Saving new best policy, reward=4.814! | |
| [2024-09-06 07:57:51,316][01070] Fps is (10 sec: 3686.5, 60 sec: 3891.2, 300 sec: 3456.0). Total num frames: 552960. Throughput: 0: 937.5. Samples: 137578. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 07:57:51,319][01070] Avg episode reward: [(0, '4.845')] | |
| [2024-09-06 07:57:51,323][06055] Saving new best policy, reward=4.845! | |
| [2024-09-06 07:57:55,369][06068] Updated weights for policy 0, policy_version 140 (0.0029) | |
| [2024-09-06 07:57:56,317][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3500.2). Total num frames: 577536. Throughput: 0: 1000.7. Samples: 144544. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:57:56,319][01070] Avg episode reward: [(0, '4.741')] | |
| [2024-09-06 07:58:01,317][01070] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3469.6). Total num frames: 589824. Throughput: 0: 990.9. Samples: 147064. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 07:58:01,322][01070] Avg episode reward: [(0, '4.681')] | |
| [2024-09-06 07:58:06,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3487.5). Total num frames: 610304. Throughput: 0: 940.4. Samples: 151710. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:58:06,321][01070] Avg episode reward: [(0, '4.354')] | |
| [2024-09-06 07:58:06,904][06068] Updated weights for policy 0, policy_version 150 (0.0034) | |
| [2024-09-06 07:58:11,317][01070] Fps is (10 sec: 4096.1, 60 sec: 3959.5, 300 sec: 3504.4). Total num frames: 630784. Throughput: 0: 978.3. Samples: 158654. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:58:11,319][01070] Avg episode reward: [(0, '4.484')] | |
| [2024-09-06 07:58:16,317][01070] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3520.3). Total num frames: 651264. Throughput: 0: 1007.8. Samples: 162160. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 07:58:16,322][01070] Avg episode reward: [(0, '4.499')] | |
| [2024-09-06 07:58:17,037][06068] Updated weights for policy 0, policy_version 160 (0.0034) | |
| [2024-09-06 07:58:21,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3492.4). Total num frames: 663552. Throughput: 0: 953.2. Samples: 166476. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:58:21,322][01070] Avg episode reward: [(0, '4.346')] | |
| [2024-09-06 07:58:26,317][01070] Fps is (10 sec: 3686.5, 60 sec: 3891.3, 300 sec: 3528.9). Total num frames: 688128. Throughput: 0: 952.7. Samples: 172806. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:58:26,319][01070] Avg episode reward: [(0, '4.378')] | |
| [2024-09-06 07:58:27,560][06068] Updated weights for policy 0, policy_version 170 (0.0040) | |
| [2024-09-06 07:58:31,317][01070] Fps is (10 sec: 4915.1, 60 sec: 3959.4, 300 sec: 3563.5). Total num frames: 712704. Throughput: 0: 985.9. Samples: 176290. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 07:58:31,324][01070] Avg episode reward: [(0, '4.546')] | |
| [2024-09-06 07:58:36,322][01070] Fps is (10 sec: 3684.3, 60 sec: 3822.6, 300 sec: 3536.4). Total num frames: 724992. Throughput: 0: 981.4. Samples: 181748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:58:36,325][01070] Avg episode reward: [(0, '4.479')] | |
| [2024-09-06 07:58:39,036][06068] Updated weights for policy 0, policy_version 180 (0.0054) | |
| [2024-09-06 07:58:41,317][01070] Fps is (10 sec: 3276.9, 60 sec: 3822.9, 300 sec: 3549.9). Total num frames: 745472. Throughput: 0: 945.0. Samples: 187070. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 07:58:41,323][01070] Avg episode reward: [(0, '4.487')] | |
| [2024-09-06 07:58:46,317][01070] Fps is (10 sec: 4508.1, 60 sec: 3959.5, 300 sec: 3581.6). Total num frames: 770048. Throughput: 0: 963.1. Samples: 190404. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 07:58:46,319][01070] Avg episode reward: [(0, '4.599')] | |
| [2024-09-06 07:58:46,332][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000188_770048.pth... | |
| [2024-09-06 07:58:48,051][06068] Updated weights for policy 0, policy_version 190 (0.0033) | |
| [2024-09-06 07:58:51,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3574.7). Total num frames: 786432. Throughput: 0: 1001.3. Samples: 196768. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 07:58:51,319][01070] Avg episode reward: [(0, '4.716')] | |
| [2024-09-06 07:58:56,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3568.1). Total num frames: 802816. Throughput: 0: 945.9. Samples: 201218. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:58:56,324][01070] Avg episode reward: [(0, '4.677')] | |
| [2024-09-06 07:58:59,542][06068] Updated weights for policy 0, policy_version 200 (0.0036) | |
| [2024-09-06 07:59:01,316][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3579.5). Total num frames: 823296. Throughput: 0: 944.4. Samples: 204656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:59:01,319][01070] Avg episode reward: [(0, '4.666')] | |
| [2024-09-06 07:59:06,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3608.0). Total num frames: 847872. Throughput: 0: 1001.9. Samples: 211562. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 07:59:06,323][01070] Avg episode reward: [(0, '4.550')] | |
| [2024-09-06 07:59:10,208][06068] Updated weights for policy 0, policy_version 210 (0.0028) | |
| [2024-09-06 07:59:11,316][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3584.0). Total num frames: 860160. Throughput: 0: 960.0. Samples: 216004. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 07:59:11,327][01070] Avg episode reward: [(0, '4.533')] | |
| [2024-09-06 07:59:16,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3823.0, 300 sec: 3594.4). Total num frames: 880640. Throughput: 0: 939.2. Samples: 218556. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 07:59:16,323][01070] Avg episode reward: [(0, '4.805')] | |
| [2024-09-06 07:59:20,094][06068] Updated weights for policy 0, policy_version 220 (0.0030) | |
| [2024-09-06 07:59:21,316][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3620.9). Total num frames: 905216. Throughput: 0: 972.9. Samples: 225522. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 07:59:21,319][01070] Avg episode reward: [(0, '4.579')] | |
| [2024-09-06 07:59:26,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3614.1). Total num frames: 921600. Throughput: 0: 980.2. Samples: 231180. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:59:26,322][01070] Avg episode reward: [(0, '4.617')] | |
| [2024-09-06 07:59:31,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3607.6). Total num frames: 937984. Throughput: 0: 952.4. Samples: 233264. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:59:31,321][01070] Avg episode reward: [(0, '4.698')] | |
| [2024-09-06 07:59:31,746][06068] Updated weights for policy 0, policy_version 230 (0.0037) | |
| [2024-09-06 07:59:36,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.8, 300 sec: 3632.3). Total num frames: 962560. Throughput: 0: 960.2. Samples: 239978. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:59:36,324][01070] Avg episode reward: [(0, '4.640')] | |
| [2024-09-06 07:59:40,641][06068] Updated weights for policy 0, policy_version 240 (0.0019) | |
| [2024-09-06 07:59:41,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3640.9). Total num frames: 983040. Throughput: 0: 1006.4. Samples: 246506. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 07:59:41,319][01070] Avg episode reward: [(0, '4.662')] | |
| [2024-09-06 07:59:46,321][01070] Fps is (10 sec: 3684.9, 60 sec: 3822.7, 300 sec: 3634.2). Total num frames: 999424. Throughput: 0: 975.4. Samples: 248554. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:59:46,324][01070] Avg episode reward: [(0, '4.517')] | |
| [2024-09-06 07:59:51,316][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3642.5). Total num frames: 1019904. Throughput: 0: 945.4. Samples: 254104. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 07:59:51,319][01070] Avg episode reward: [(0, '4.585')] | |
| [2024-09-06 07:59:52,074][06068] Updated weights for policy 0, policy_version 250 (0.0044) | |
| [2024-09-06 07:59:56,317][01070] Fps is (10 sec: 3687.9, 60 sec: 3891.2, 300 sec: 3636.1). Total num frames: 1036288. Throughput: 0: 969.2. Samples: 259616. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 07:59:56,319][01070] Avg episode reward: [(0, '4.734')] | |
| [2024-09-06 08:00:01,317][01070] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3615.8). Total num frames: 1048576. Throughput: 0: 955.6. Samples: 261556. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:00:01,321][01070] Avg episode reward: [(0, '4.762')] | |
| [2024-09-06 08:00:06,317][01070] Fps is (10 sec: 2457.6, 60 sec: 3549.9, 300 sec: 3596.1). Total num frames: 1060864. Throughput: 0: 888.8. Samples: 265516. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:00:06,318][01070] Avg episode reward: [(0, '4.686')] | |
| [2024-09-06 08:00:06,345][06068] Updated weights for policy 0, policy_version 260 (0.0046) | |
| [2024-09-06 08:00:11,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3679.5). Total num frames: 1085440. Throughput: 0: 911.2. Samples: 272186. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:00:11,319][01070] Avg episode reward: [(0, '4.470')] | |
| [2024-09-06 08:00:15,254][06068] Updated weights for policy 0, policy_version 270 (0.0039) | |
| [2024-09-06 08:00:16,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3748.9). Total num frames: 1105920. Throughput: 0: 939.1. Samples: 275524. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:00:16,318][01070] Avg episode reward: [(0, '4.535')] | |
| [2024-09-06 08:00:21,328][01070] Fps is (10 sec: 3682.2, 60 sec: 3617.4, 300 sec: 3804.3). Total num frames: 1122304. Throughput: 0: 901.1. Samples: 280540. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:00:21,339][01070] Avg episode reward: [(0, '4.759')] | |
| [2024-09-06 08:00:26,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3790.5). Total num frames: 1142784. Throughput: 0: 885.6. Samples: 286356. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:00:26,321][01070] Avg episode reward: [(0, '4.852')] | |
| [2024-09-06 08:00:26,332][06055] Saving new best policy, reward=4.852! | |
| [2024-09-06 08:00:26,855][06068] Updated weights for policy 0, policy_version 280 (0.0043) | |
| [2024-09-06 08:00:31,317][01070] Fps is (10 sec: 4510.8, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 1167360. Throughput: 0: 914.9. Samples: 289722. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:00:31,318][01070] Avg episode reward: [(0, '4.581')] | |
| [2024-09-06 08:00:36,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3818.3). Total num frames: 1183744. Throughput: 0: 929.6. Samples: 295936. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:00:36,323][01070] Avg episode reward: [(0, '4.600')] | |
| [2024-09-06 08:00:37,327][06068] Updated weights for policy 0, policy_version 290 (0.0022) | |
| [2024-09-06 08:00:41,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3804.4). Total num frames: 1200128. Throughput: 0: 911.8. Samples: 300646. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:00:41,323][01070] Avg episode reward: [(0, '4.564')] | |
| [2024-09-06 08:00:46,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3686.6, 300 sec: 3804.4). Total num frames: 1220608. Throughput: 0: 943.5. Samples: 304012. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:00:46,321][01070] Avg episode reward: [(0, '4.603')] | |
| [2024-09-06 08:00:46,372][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000299_1224704.pth... | |
| [2024-09-06 08:00:46,507][06055] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000075_307200.pth | |
| [2024-09-06 08:00:47,243][06068] Updated weights for policy 0, policy_version 300 (0.0033) | |
| [2024-09-06 08:00:51,316][01070] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3804.4). Total num frames: 1241088. Throughput: 0: 1005.5. Samples: 310764. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:00:51,321][01070] Avg episode reward: [(0, '4.905')] | |
| [2024-09-06 08:00:51,344][06055] Saving new best policy, reward=4.905! | |
| [2024-09-06 08:00:56,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3804.4). Total num frames: 1257472. Throughput: 0: 950.9. Samples: 314976. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:00:56,324][01070] Avg episode reward: [(0, '4.923')] | |
| [2024-09-06 08:00:56,342][06055] Saving new best policy, reward=4.923! | |
| [2024-09-06 08:00:58,832][06068] Updated weights for policy 0, policy_version 310 (0.0029) | |
| [2024-09-06 08:01:01,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 1277952. Throughput: 0: 942.3. Samples: 317926. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:01:01,322][01070] Avg episode reward: [(0, '4.709')] | |
| [2024-09-06 08:01:06,317][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3832.2). Total num frames: 1302528. Throughput: 0: 984.8. Samples: 324844. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:01:06,319][01070] Avg episode reward: [(0, '4.749')] | |
| [2024-09-06 08:01:08,155][06068] Updated weights for policy 0, policy_version 320 (0.0016) | |
| [2024-09-06 08:01:11,316][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 1318912. Throughput: 0: 972.0. Samples: 330098. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:01:11,320][01070] Avg episode reward: [(0, '4.757')] | |
| [2024-09-06 08:01:16,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 1335296. Throughput: 0: 944.2. Samples: 332210. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 08:01:16,324][01070] Avg episode reward: [(0, '4.768')] | |
| [2024-09-06 08:01:19,327][06068] Updated weights for policy 0, policy_version 330 (0.0026) | |
| [2024-09-06 08:01:21,316][01070] Fps is (10 sec: 4096.0, 60 sec: 3960.2, 300 sec: 3846.1). Total num frames: 1359872. Throughput: 0: 959.8. Samples: 339126. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:01:21,320][01070] Avg episode reward: [(0, '4.852')] | |
| [2024-09-06 08:01:26,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 1380352. Throughput: 0: 994.4. Samples: 345396. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:01:26,321][01070] Avg episode reward: [(0, '4.673')] | |
| [2024-09-06 08:01:30,374][06068] Updated weights for policy 0, policy_version 340 (0.0044) | |
| [2024-09-06 08:01:31,316][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3832.2). Total num frames: 1392640. Throughput: 0: 965.5. Samples: 347460. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 08:01:31,320][01070] Avg episode reward: [(0, '4.731')] | |
| [2024-09-06 08:01:36,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 1417216. Throughput: 0: 949.2. Samples: 353480. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 08:01:36,318][01070] Avg episode reward: [(0, '4.856')] | |
| [2024-09-06 08:01:39,566][06068] Updated weights for policy 0, policy_version 350 (0.0025) | |
| [2024-09-06 08:01:41,316][01070] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3846.1). Total num frames: 1441792. Throughput: 0: 1011.6. Samples: 360496. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:01:41,323][01070] Avg episode reward: [(0, '4.687')] | |
| [2024-09-06 08:01:46,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 1454080. Throughput: 0: 996.0. Samples: 362748. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:01:46,323][01070] Avg episode reward: [(0, '4.580')] | |
| [2024-09-06 08:01:51,317][01070] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 1470464. Throughput: 0: 945.5. Samples: 367392. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 08:01:51,321][01070] Avg episode reward: [(0, '4.688')] | |
| [2024-09-06 08:01:51,501][06068] Updated weights for policy 0, policy_version 360 (0.0037) | |
| [2024-09-06 08:01:56,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 1495040. Throughput: 0: 984.5. Samples: 374400. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:01:56,324][01070] Avg episode reward: [(0, '4.809')] | |
| [2024-09-06 08:02:01,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 1511424. Throughput: 0: 1012.0. Samples: 377750. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:01,325][01070] Avg episode reward: [(0, '5.081')] | |
| [2024-09-06 08:02:01,334][06055] Saving new best policy, reward=5.081! | |
| [2024-09-06 08:02:01,346][06068] Updated weights for policy 0, policy_version 370 (0.0044) | |
| [2024-09-06 08:02:06,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3846.1). Total num frames: 1527808. Throughput: 0: 953.3. Samples: 382024. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:06,324][01070] Avg episode reward: [(0, '5.109')] | |
| [2024-09-06 08:02:06,335][06055] Saving new best policy, reward=5.109! | |
| [2024-09-06 08:02:11,317][01070] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 1552384. Throughput: 0: 958.0. Samples: 388506. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:11,327][01070] Avg episode reward: [(0, '4.909')] | |
| [2024-09-06 08:02:12,040][06068] Updated weights for policy 0, policy_version 380 (0.0043) | |
| [2024-09-06 08:02:16,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3832.2). Total num frames: 1572864. Throughput: 0: 989.5. Samples: 391988. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:16,319][01070] Avg episode reward: [(0, '4.791')] | |
| [2024-09-06 08:02:21,316][01070] Fps is (10 sec: 3686.5, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 1589248. Throughput: 0: 966.2. Samples: 396960. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:02:21,321][01070] Avg episode reward: [(0, '4.771')] | |
| [2024-09-06 08:02:23,656][06068] Updated weights for policy 0, policy_version 390 (0.0062) | |
| [2024-09-06 08:02:26,317][01070] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 1609728. Throughput: 0: 936.7. Samples: 402648. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:26,322][01070] Avg episode reward: [(0, '4.651')] | |
| [2024-09-06 08:02:31,316][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 1630208. Throughput: 0: 964.6. Samples: 406156. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:31,322][01070] Avg episode reward: [(0, '4.673')] | |
| [2024-09-06 08:02:32,348][06068] Updated weights for policy 0, policy_version 400 (0.0035) | |
| [2024-09-06 08:02:36,317][01070] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 1650688. Throughput: 0: 999.3. Samples: 412360. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:36,318][01070] Avg episode reward: [(0, '4.851')] | |
| [2024-09-06 08:02:41,317][01070] Fps is (10 sec: 3276.7, 60 sec: 3686.4, 300 sec: 3832.2). Total num frames: 1662976. Throughput: 0: 946.6. Samples: 416998. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:41,322][01070] Avg episode reward: [(0, '5.188')] | |
| [2024-09-06 08:02:41,329][06055] Saving new best policy, reward=5.188! | |
| [2024-09-06 08:02:43,939][06068] Updated weights for policy 0, policy_version 410 (0.0044) | |
| [2024-09-06 08:02:46,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 1687552. Throughput: 0: 948.0. Samples: 420410. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:02:46,322][01070] Avg episode reward: [(0, '5.138')] | |
| [2024-09-06 08:02:46,333][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000412_1687552.pth... | |
| [2024-09-06 08:02:46,505][06055] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000188_770048.pth | |
| [2024-09-06 08:02:51,318][01070] Fps is (10 sec: 4504.9, 60 sec: 3959.3, 300 sec: 3832.2). Total num frames: 1708032. Throughput: 0: 1003.2. Samples: 427168. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:02:51,321][01070] Avg episode reward: [(0, '4.904')] | |
| [2024-09-06 08:02:54,708][06068] Updated weights for policy 0, policy_version 420 (0.0014) | |
| [2024-09-06 08:02:56,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 1724416. Throughput: 0: 952.8. Samples: 431380. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:02:56,319][01070] Avg episode reward: [(0, '4.841')] | |
| [2024-09-06 08:03:01,317][01070] Fps is (10 sec: 3687.1, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 1744896. Throughput: 0: 938.4. Samples: 434216. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:03:01,319][01070] Avg episode reward: [(0, '4.777')] | |
| [2024-09-06 08:03:04,504][06068] Updated weights for policy 0, policy_version 430 (0.0014) | |
| [2024-09-06 08:03:06,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 1765376. Throughput: 0: 983.9. Samples: 441236. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:03:06,323][01070] Avg episode reward: [(0, '4.848')] | |
| [2024-09-06 08:03:11,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3823.0, 300 sec: 3832.2). Total num frames: 1781760. Throughput: 0: 978.1. Samples: 446662. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:03:11,322][01070] Avg episode reward: [(0, '4.875')] | |
| [2024-09-06 08:03:16,317][01070] Fps is (10 sec: 3276.7, 60 sec: 3754.6, 300 sec: 3846.1). Total num frames: 1798144. Throughput: 0: 947.8. Samples: 448806. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:03:16,319][01070] Avg episode reward: [(0, '5.049')] | |
| [2024-09-06 08:03:16,335][06068] Updated weights for policy 0, policy_version 440 (0.0023) | |
| [2024-09-06 08:03:21,316][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 1822720. Throughput: 0: 954.9. Samples: 455330. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:03:21,320][01070] Avg episode reward: [(0, '5.155')] | |
| [2024-09-06 08:03:25,412][06068] Updated weights for policy 0, policy_version 450 (0.0032) | |
| [2024-09-06 08:03:26,317][01070] Fps is (10 sec: 4505.7, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 1843200. Throughput: 0: 993.1. Samples: 461688. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:03:26,318][01070] Avg episode reward: [(0, '5.165')] | |
| [2024-09-06 08:03:31,322][01070] Fps is (10 sec: 3684.2, 60 sec: 3822.6, 300 sec: 3846.1). Total num frames: 1859584. Throughput: 0: 961.7. Samples: 463694. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:03:31,325][01070] Avg episode reward: [(0, '5.015')] | |
| [2024-09-06 08:03:36,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 1880064. Throughput: 0: 941.8. Samples: 469546. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:03:36,319][01070] Avg episode reward: [(0, '4.801')] | |
| [2024-09-06 08:03:36,683][06068] Updated weights for policy 0, policy_version 460 (0.0046) | |
| [2024-09-06 08:03:41,317][01070] Fps is (10 sec: 4098.3, 60 sec: 3959.5, 300 sec: 3832.2). Total num frames: 1900544. Throughput: 0: 995.4. Samples: 476172. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:03:41,324][01070] Avg episode reward: [(0, '5.126')] | |
| [2024-09-06 08:03:46,323][01070] Fps is (10 sec: 3274.6, 60 sec: 3754.2, 300 sec: 3818.2). Total num frames: 1912832. Throughput: 0: 972.6. Samples: 477988. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:03:46,326][01070] Avg episode reward: [(0, '5.144')] | |
| [2024-09-06 08:03:50,595][06068] Updated weights for policy 0, policy_version 470 (0.0043) | |
| [2024-09-06 08:03:51,317][01070] Fps is (10 sec: 2457.7, 60 sec: 3618.2, 300 sec: 3804.4). Total num frames: 1925120. Throughput: 0: 891.7. Samples: 481364. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:03:51,320][01070] Avg episode reward: [(0, '5.180')] | |
| [2024-09-06 08:03:56,317][01070] Fps is (10 sec: 3688.9, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 1949696. Throughput: 0: 907.3. Samples: 487490. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:03:56,320][01070] Avg episode reward: [(0, '4.899')] | |
| [2024-09-06 08:03:59,818][06068] Updated weights for policy 0, policy_version 480 (0.0031) | |
| [2024-09-06 08:04:01,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 1970176. Throughput: 0: 936.2. Samples: 490936. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:04:01,319][01070] Avg episode reward: [(0, '4.708')] | |
| [2024-09-06 08:04:06,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3818.3). Total num frames: 1986560. Throughput: 0: 912.2. Samples: 496378. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:04:06,319][01070] Avg episode reward: [(0, '4.858')] | |
| [2024-09-06 08:04:11,317][01070] Fps is (10 sec: 3276.7, 60 sec: 3686.4, 300 sec: 3804.4). Total num frames: 2002944. Throughput: 0: 889.3. Samples: 501706. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:04:11,323][01070] Avg episode reward: [(0, '5.187')] | |
| [2024-09-06 08:04:11,461][06068] Updated weights for policy 0, policy_version 490 (0.0044) | |
| [2024-09-06 08:04:16,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3823.0, 300 sec: 3804.4). Total num frames: 2027520. Throughput: 0: 922.6. Samples: 505206. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:04:16,321][01070] Avg episode reward: [(0, '5.216')] | |
| [2024-09-06 08:04:16,333][06055] Saving new best policy, reward=5.216! | |
| [2024-09-06 08:04:21,319][01070] Fps is (10 sec: 4094.9, 60 sec: 3686.2, 300 sec: 3804.4). Total num frames: 2043904. Throughput: 0: 931.9. Samples: 511486. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:04:21,322][01070] Avg episode reward: [(0, '5.013')] | |
| [2024-09-06 08:04:21,394][06068] Updated weights for policy 0, policy_version 500 (0.0030) | |
| [2024-09-06 08:04:26,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3804.4). Total num frames: 2060288. Throughput: 0: 878.6. Samples: 515708. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:04:26,324][01070] Avg episode reward: [(0, '4.618')] | |
| [2024-09-06 08:04:31,317][01070] Fps is (10 sec: 4097.1, 60 sec: 3755.0, 300 sec: 3804.4). Total num frames: 2084864. Throughput: 0: 916.1. Samples: 519206. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:04:31,319][01070] Avg episode reward: [(0, '4.710')] | |
| [2024-09-06 08:04:32,059][06068] Updated weights for policy 0, policy_version 510 (0.0024) | |
| [2024-09-06 08:04:36,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 2105344. Throughput: 0: 995.1. Samples: 526144. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:04:36,319][01070] Avg episode reward: [(0, '5.116')] | |
| [2024-09-06 08:04:41,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3804.5). Total num frames: 2121728. Throughput: 0: 964.6. Samples: 530896. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:04:41,319][01070] Avg episode reward: [(0, '5.008')] | |
| [2024-09-06 08:04:43,583][06068] Updated weights for policy 0, policy_version 520 (0.0023) | |
| [2024-09-06 08:04:46,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3823.4, 300 sec: 3804.4). Total num frames: 2142208. Throughput: 0: 945.4. Samples: 533480. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:04:46,320][01070] Avg episode reward: [(0, '5.078')] | |
| [2024-09-06 08:04:46,333][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000523_2142208.pth... | |
| [2024-09-06 08:04:46,489][06055] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000299_1224704.pth | |
| [2024-09-06 08:04:51,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3818.3). Total num frames: 2162688. Throughput: 0: 975.2. Samples: 540264. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:04:51,321][01070] Avg episode reward: [(0, '5.193')] | |
| [2024-09-06 08:04:52,623][06068] Updated weights for policy 0, policy_version 530 (0.0036) | |
| [2024-09-06 08:04:56,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3832.2). Total num frames: 2179072. Throughput: 0: 981.4. Samples: 545870. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:04:56,323][01070] Avg episode reward: [(0, '4.939')] | |
| [2024-09-06 08:05:01,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3846.1). Total num frames: 2195456. Throughput: 0: 949.8. Samples: 547946. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:05:01,323][01070] Avg episode reward: [(0, '5.156')] | |
| [2024-09-06 08:05:04,000][06068] Updated weights for policy 0, policy_version 540 (0.0028) | |
| [2024-09-06 08:05:06,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2220032. Throughput: 0: 959.5. Samples: 554662. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:05:06,324][01070] Avg episode reward: [(0, '5.088')] | |
| [2024-09-06 08:05:11,316][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 2240512. Throughput: 0: 1010.8. Samples: 561196. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:05:11,320][01070] Avg episode reward: [(0, '4.885')] | |
| [2024-09-06 08:05:14,445][06068] Updated weights for policy 0, policy_version 550 (0.0037) | |
| [2024-09-06 08:05:16,318][01070] Fps is (10 sec: 3685.7, 60 sec: 3822.8, 300 sec: 3846.2). Total num frames: 2256896. Throughput: 0: 978.9. Samples: 563260. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:05:16,321][01070] Avg episode reward: [(0, '5.189')] | |
| [2024-09-06 08:05:21,317][01070] Fps is (10 sec: 3686.3, 60 sec: 3891.4, 300 sec: 3846.1). Total num frames: 2277376. Throughput: 0: 948.0. Samples: 568804. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:05:21,319][01070] Avg episode reward: [(0, '5.640')] | |
| [2024-09-06 08:05:21,324][06055] Saving new best policy, reward=5.640! | |
| [2024-09-06 08:05:24,437][06068] Updated weights for policy 0, policy_version 560 (0.0040) | |
| [2024-09-06 08:05:26,317][01070] Fps is (10 sec: 4506.5, 60 sec: 4027.7, 300 sec: 3846.1). Total num frames: 2301952. Throughput: 0: 998.1. Samples: 575810. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:05:26,321][01070] Avg episode reward: [(0, '5.759')] | |
| [2024-09-06 08:05:26,331][06055] Saving new best policy, reward=5.759! | |
| [2024-09-06 08:05:31,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2318336. Throughput: 0: 999.7. Samples: 578466. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:05:31,322][01070] Avg episode reward: [(0, '5.847')] | |
| [2024-09-06 08:05:31,326][06055] Saving new best policy, reward=5.847! | |
| [2024-09-06 08:05:35,920][06068] Updated weights for policy 0, policy_version 570 (0.0024) | |
| [2024-09-06 08:05:36,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 2334720. Throughput: 0: 952.0. Samples: 583102. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:05:36,319][01070] Avg episode reward: [(0, '5.772')] | |
| [2024-09-06 08:05:41,317][01070] Fps is (10 sec: 4096.1, 60 sec: 3959.5, 300 sec: 3860.0). Total num frames: 2359296. Throughput: 0: 984.4. Samples: 590166. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:05:41,319][01070] Avg episode reward: [(0, '5.772')] | |
| [2024-09-06 08:05:44,966][06068] Updated weights for policy 0, policy_version 580 (0.0031) | |
| [2024-09-06 08:05:46,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2375680. Throughput: 0: 1015.8. Samples: 593656. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:05:46,326][01070] Avg episode reward: [(0, '5.835')] | |
| [2024-09-06 08:05:51,317][01070] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 2392064. Throughput: 0: 961.4. Samples: 597926. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:05:51,325][01070] Avg episode reward: [(0, '5.707')] | |
| [2024-09-06 08:05:56,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2412544. Throughput: 0: 956.4. Samples: 604232. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:05:56,325][01070] Avg episode reward: [(0, '5.539')] | |
| [2024-09-06 08:05:56,404][06068] Updated weights for policy 0, policy_version 590 (0.0035) | |
| [2024-09-06 08:06:01,317][01070] Fps is (10 sec: 4505.7, 60 sec: 4027.7, 300 sec: 3846.1). Total num frames: 2437120. Throughput: 0: 987.2. Samples: 607680. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:06:01,321][01070] Avg episode reward: [(0, '5.823')] | |
| [2024-09-06 08:06:06,317][01070] Fps is (10 sec: 4095.7, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2453504. Throughput: 0: 982.3. Samples: 613010. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:06:06,323][01070] Avg episode reward: [(0, '5.596')] | |
| [2024-09-06 08:06:07,575][06068] Updated weights for policy 0, policy_version 600 (0.0034) | |
| [2024-09-06 08:06:11,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 2469888. Throughput: 0: 945.9. Samples: 618374. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:06:11,324][01070] Avg episode reward: [(0, '5.336')] | |
| [2024-09-06 08:06:16,317][01070] Fps is (10 sec: 4096.2, 60 sec: 3959.6, 300 sec: 3846.1). Total num frames: 2494464. Throughput: 0: 964.4. Samples: 621862. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:06:16,324][01070] Avg episode reward: [(0, '5.749')] | |
| [2024-09-06 08:06:16,804][06068] Updated weights for policy 0, policy_version 610 (0.0036) | |
| [2024-09-06 08:06:21,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 2510848. Throughput: 0: 1000.3. Samples: 628114. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:06:21,322][01070] Avg episode reward: [(0, '6.247')] | |
| [2024-09-06 08:06:21,328][06055] Saving new best policy, reward=6.247! | |
| [2024-09-06 08:06:26,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3846.1). Total num frames: 2527232. Throughput: 0: 943.9. Samples: 632640. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:06:26,324][01070] Avg episode reward: [(0, '6.326')] | |
| [2024-09-06 08:06:26,335][06055] Saving new best policy, reward=6.326! | |
| [2024-09-06 08:06:28,439][06068] Updated weights for policy 0, policy_version 620 (0.0026) | |
| [2024-09-06 08:06:31,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2551808. Throughput: 0: 943.6. Samples: 636116. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:06:31,323][01070] Avg episode reward: [(0, '5.950')] | |
| [2024-09-06 08:06:36,323][01070] Fps is (10 sec: 4502.5, 60 sec: 3959.0, 300 sec: 3832.1). Total num frames: 2572288. Throughput: 0: 1005.1. Samples: 643160. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:06:36,326][01070] Avg episode reward: [(0, '5.960')] | |
| [2024-09-06 08:06:38,101][06068] Updated weights for policy 0, policy_version 630 (0.0023) | |
| [2024-09-06 08:06:41,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 2588672. Throughput: 0: 964.2. Samples: 647620. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:06:41,319][01070] Avg episode reward: [(0, '5.987')] | |
| [2024-09-06 08:06:46,317][01070] Fps is (10 sec: 3688.9, 60 sec: 3891.2, 300 sec: 3860.0). Total num frames: 2609152. Throughput: 0: 950.4. Samples: 650448. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:06:46,323][01070] Avg episode reward: [(0, '6.091')] | |
| [2024-09-06 08:06:46,333][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000637_2609152.pth... | |
| [2024-09-06 08:06:46,464][06055] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000412_1687552.pth | |
| [2024-09-06 08:06:48,887][06068] Updated weights for policy 0, policy_version 640 (0.0049) | |
| [2024-09-06 08:06:51,316][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 2629632. Throughput: 0: 982.1. Samples: 657202. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:06:51,323][01070] Avg episode reward: [(0, '6.072')] | |
| [2024-09-06 08:06:56,317][01070] Fps is (10 sec: 3686.5, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2646016. Throughput: 0: 980.7. Samples: 662504. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:06:56,319][01070] Avg episode reward: [(0, '5.924')] | |
| [2024-09-06 08:07:00,529][06068] Updated weights for policy 0, policy_version 650 (0.0028) | |
| [2024-09-06 08:07:01,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3846.1). Total num frames: 2662400. Throughput: 0: 950.1. Samples: 664618. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:07:01,319][01070] Avg episode reward: [(0, '5.764')] | |
| [2024-09-06 08:07:06,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 2686976. Throughput: 0: 963.3. Samples: 671462. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2024-09-06 08:07:06,318][01070] Avg episode reward: [(0, '5.901')] | |
| [2024-09-06 08:07:09,346][06068] Updated weights for policy 0, policy_version 660 (0.0045) | |
| [2024-09-06 08:07:11,317][01070] Fps is (10 sec: 4505.2, 60 sec: 3959.4, 300 sec: 3846.1). Total num frames: 2707456. Throughput: 0: 1000.9. Samples: 677680. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:07:11,320][01070] Avg episode reward: [(0, '6.609')] | |
| [2024-09-06 08:07:11,322][06055] Saving new best policy, reward=6.609! | |
| [2024-09-06 08:07:16,318][01070] Fps is (10 sec: 3276.4, 60 sec: 3754.6, 300 sec: 3832.2). Total num frames: 2719744. Throughput: 0: 967.8. Samples: 679670. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:07:16,324][01070] Avg episode reward: [(0, '6.832')] | |
| [2024-09-06 08:07:16,336][06055] Saving new best policy, reward=6.832! | |
| [2024-09-06 08:07:21,160][06068] Updated weights for policy 0, policy_version 670 (0.0030) | |
| [2024-09-06 08:07:21,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.1, 300 sec: 3846.1). Total num frames: 2744320. Throughput: 0: 935.1. Samples: 685232. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:07:21,321][01070] Avg episode reward: [(0, '6.827')] | |
| [2024-09-06 08:07:26,317][01070] Fps is (10 sec: 4096.4, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 2760704. Throughput: 0: 966.4. Samples: 691106. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:07:26,323][01070] Avg episode reward: [(0, '6.439')] | |
| [2024-09-06 08:07:31,319][01070] Fps is (10 sec: 2866.6, 60 sec: 3686.2, 300 sec: 3804.4). Total num frames: 2772992. Throughput: 0: 942.7. Samples: 692870. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:07:31,325][01070] Avg episode reward: [(0, '6.339')] | |
| [2024-09-06 08:07:35,290][06068] Updated weights for policy 0, policy_version 680 (0.0034) | |
| [2024-09-06 08:07:36,317][01070] Fps is (10 sec: 2457.6, 60 sec: 3550.3, 300 sec: 3804.4). Total num frames: 2785280. Throughput: 0: 878.4. Samples: 696728. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2024-09-06 08:07:36,321][01070] Avg episode reward: [(0, '6.444')] | |
| [2024-09-06 08:07:41,317][01070] Fps is (10 sec: 3687.5, 60 sec: 3686.4, 300 sec: 3804.4). Total num frames: 2809856. Throughput: 0: 908.9. Samples: 703404. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:07:41,319][01070] Avg episode reward: [(0, '6.654')] | |
| [2024-09-06 08:07:44,151][06068] Updated weights for policy 0, policy_version 690 (0.0028) | |
| [2024-09-06 08:07:46,317][01070] Fps is (10 sec: 4915.2, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 2834432. Throughput: 0: 940.5. Samples: 706940. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:07:46,319][01070] Avg episode reward: [(0, '6.718')] | |
| [2024-09-06 08:07:51,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3804.4). Total num frames: 2846720. Throughput: 0: 901.1. Samples: 712010. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:07:51,319][01070] Avg episode reward: [(0, '6.882')] | |
| [2024-09-06 08:07:51,324][06055] Saving new best policy, reward=6.882! | |
| [2024-09-06 08:07:55,843][06068] Updated weights for policy 0, policy_version 700 (0.0039) | |
| [2024-09-06 08:07:56,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3804.4). Total num frames: 2867200. Throughput: 0: 884.7. Samples: 717490. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:07:56,325][01070] Avg episode reward: [(0, '7.244')] | |
| [2024-09-06 08:07:56,336][06055] Saving new best policy, reward=7.244! | |
| [2024-09-06 08:08:01,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3818.3). Total num frames: 2891776. Throughput: 0: 917.8. Samples: 720972. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:08:01,324][01070] Avg episode reward: [(0, '7.391')] | |
| [2024-09-06 08:08:01,326][06055] Saving new best policy, reward=7.391! | |
| [2024-09-06 08:08:05,806][06068] Updated weights for policy 0, policy_version 710 (0.0027) | |
| [2024-09-06 08:08:06,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3818.3). Total num frames: 2908160. Throughput: 0: 930.1. Samples: 727084. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:08:06,325][01070] Avg episode reward: [(0, '6.836')] | |
| [2024-09-06 08:08:11,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3818.3). Total num frames: 2924544. Throughput: 0: 901.6. Samples: 731678. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:08:11,319][01070] Avg episode reward: [(0, '6.692')] | |
| [2024-09-06 08:08:16,037][06068] Updated weights for policy 0, policy_version 720 (0.0022) | |
| [2024-09-06 08:08:16,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3823.0, 300 sec: 3818.3). Total num frames: 2949120. Throughput: 0: 940.1. Samples: 735172. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:08:16,319][01070] Avg episode reward: [(0, '6.850')] | |
| [2024-09-06 08:08:21,321][01070] Fps is (10 sec: 4503.7, 60 sec: 3754.5, 300 sec: 3818.2). Total num frames: 2969600. Throughput: 0: 1006.2. Samples: 742010. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:08:21,323][01070] Avg episode reward: [(0, '7.485')] | |
| [2024-09-06 08:08:21,325][06055] Saving new best policy, reward=7.485! | |
| [2024-09-06 08:08:26,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3804.5). Total num frames: 2981888. Throughput: 0: 953.0. Samples: 746290. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:08:26,319][01070] Avg episode reward: [(0, '7.434')] | |
| [2024-09-06 08:08:27,902][06068] Updated weights for policy 0, policy_version 730 (0.0032) | |
| [2024-09-06 08:08:31,317][01070] Fps is (10 sec: 3278.2, 60 sec: 3823.1, 300 sec: 3804.4). Total num frames: 3002368. Throughput: 0: 939.8. Samples: 749232. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:08:31,319][01070] Avg episode reward: [(0, '7.716')] | |
| [2024-09-06 08:08:31,354][06055] Saving new best policy, reward=7.716! | |
| [2024-09-06 08:08:36,317][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3818.3). Total num frames: 3026944. Throughput: 0: 982.7. Samples: 756230. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:08:36,319][01070] Avg episode reward: [(0, '7.928')] | |
| [2024-09-06 08:08:36,335][06055] Saving new best policy, reward=7.928! | |
| [2024-09-06 08:08:36,786][06068] Updated weights for policy 0, policy_version 740 (0.0040) | |
| [2024-09-06 08:08:41,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3832.3). Total num frames: 3043328. Throughput: 0: 975.6. Samples: 761394. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:08:41,319][01070] Avg episode reward: [(0, '7.704')] | |
| [2024-09-06 08:08:46,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3846.1). Total num frames: 3059712. Throughput: 0: 946.2. Samples: 763550. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:08:46,318][01070] Avg episode reward: [(0, '7.583')] | |
| [2024-09-06 08:08:46,332][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000747_3059712.pth... | |
| [2024-09-06 08:08:46,453][06055] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000523_2142208.pth | |
| [2024-09-06 08:08:48,365][06068] Updated weights for policy 0, policy_version 750 (0.0018) | |
| [2024-09-06 08:08:51,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 3084288. Throughput: 0: 960.6. Samples: 770310. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:08:51,322][01070] Avg episode reward: [(0, '7.840')] | |
| [2024-09-06 08:08:56,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 3100672. Throughput: 0: 992.8. Samples: 776356. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:08:56,320][01070] Avg episode reward: [(0, '7.925')] | |
| [2024-09-06 08:08:59,558][06068] Updated weights for policy 0, policy_version 760 (0.0058) | |
| [2024-09-06 08:09:01,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3832.2). Total num frames: 3117056. Throughput: 0: 961.3. Samples: 778430. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:09:01,323][01070] Avg episode reward: [(0, '7.469')] | |
| [2024-09-06 08:09:06,318][01070] Fps is (10 sec: 4095.5, 60 sec: 3891.1, 300 sec: 3859.9). Total num frames: 3141632. Throughput: 0: 942.7. Samples: 784430. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2024-09-06 08:09:06,322][01070] Avg episode reward: [(0, '7.036')] | |
| [2024-09-06 08:09:08,713][06068] Updated weights for policy 0, policy_version 770 (0.0036) | |
| [2024-09-06 08:09:11,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 3162112. Throughput: 0: 1005.6. Samples: 791544. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:09:11,324][01070] Avg episode reward: [(0, '7.387')] | |
| [2024-09-06 08:09:16,322][01070] Fps is (10 sec: 3684.7, 60 sec: 3822.6, 300 sec: 3846.0). Total num frames: 3178496. Throughput: 0: 991.1. Samples: 793836. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:09:16,325][01070] Avg episode reward: [(0, '7.663')] | |
| [2024-09-06 08:09:20,382][06068] Updated weights for policy 0, policy_version 780 (0.0022) | |
| [2024-09-06 08:09:21,316][01070] Fps is (10 sec: 3686.4, 60 sec: 3823.2, 300 sec: 3860.0). Total num frames: 3198976. Throughput: 0: 946.0. Samples: 798798. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:09:21,325][01070] Avg episode reward: [(0, '7.722')] | |
| [2024-09-06 08:09:26,317][01070] Fps is (10 sec: 4508.2, 60 sec: 4027.7, 300 sec: 3860.0). Total num frames: 3223552. Throughput: 0: 988.2. Samples: 805864. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:09:26,320][01070] Avg episode reward: [(0, '7.744')] | |
| [2024-09-06 08:09:29,518][06068] Updated weights for policy 0, policy_version 790 (0.0014) | |
| [2024-09-06 08:09:31,319][01070] Fps is (10 sec: 4095.0, 60 sec: 3959.3, 300 sec: 3846.0). Total num frames: 3239936. Throughput: 0: 1014.1. Samples: 809186. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:09:31,322][01070] Avg episode reward: [(0, '7.755')] | |
| [2024-09-06 08:09:36,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 3256320. Throughput: 0: 960.5. Samples: 813534. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:09:36,318][01070] Avg episode reward: [(0, '8.140')] | |
| [2024-09-06 08:09:36,331][06055] Saving new best policy, reward=8.140! | |
| [2024-09-06 08:09:40,585][06068] Updated weights for policy 0, policy_version 800 (0.0042) | |
| [2024-09-06 08:09:41,317][01070] Fps is (10 sec: 3687.3, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3276800. Throughput: 0: 974.7. Samples: 820218. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:09:41,323][01070] Avg episode reward: [(0, '8.704')] | |
| [2024-09-06 08:09:41,332][06055] Saving new best policy, reward=8.704! | |
| [2024-09-06 08:09:46,317][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3860.0). Total num frames: 3301376. Throughput: 0: 1006.8. Samples: 823736. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:09:46,322][01070] Avg episode reward: [(0, '8.566')] | |
| [2024-09-06 08:09:51,321][01070] Fps is (10 sec: 3684.6, 60 sec: 3822.6, 300 sec: 3846.0). Total num frames: 3313664. Throughput: 0: 986.4. Samples: 828822. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:09:51,327][01070] Avg episode reward: [(0, '8.672')] | |
| [2024-09-06 08:09:51,764][06068] Updated weights for policy 0, policy_version 810 (0.0032) | |
| [2024-09-06 08:09:56,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3860.0). Total num frames: 3334144. Throughput: 0: 955.2. Samples: 834528. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:09:56,323][01070] Avg episode reward: [(0, '8.667')] | |
| [2024-09-06 08:10:00,867][06068] Updated weights for policy 0, policy_version 820 (0.0032) | |
| [2024-09-06 08:10:01,317][01070] Fps is (10 sec: 4507.8, 60 sec: 4027.7, 300 sec: 3860.0). Total num frames: 3358720. Throughput: 0: 982.2. Samples: 838030. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:10:01,319][01070] Avg episode reward: [(0, '9.323')] | |
| [2024-09-06 08:10:01,327][06055] Saving new best policy, reward=9.323! | |
| [2024-09-06 08:10:06,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.3, 300 sec: 3846.1). Total num frames: 3375104. Throughput: 0: 1002.8. Samples: 843922. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:10:06,322][01070] Avg episode reward: [(0, '9.281')] | |
| [2024-09-06 08:10:11,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 3391488. Throughput: 0: 951.8. Samples: 848696. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:10:11,319][01070] Avg episode reward: [(0, '9.212')] | |
| [2024-09-06 08:10:12,461][06068] Updated weights for policy 0, policy_version 830 (0.0025) | |
| [2024-09-06 08:10:16,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.9, 300 sec: 3860.0). Total num frames: 3416064. Throughput: 0: 956.3. Samples: 852216. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:10:16,319][01070] Avg episode reward: [(0, '9.185')] | |
| [2024-09-06 08:10:21,317][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 3436544. Throughput: 0: 1017.1. Samples: 859302. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:10:21,320][01070] Avg episode reward: [(0, '9.434')] | |
| [2024-09-06 08:10:21,324][06055] Saving new best policy, reward=9.434! | |
| [2024-09-06 08:10:22,066][06068] Updated weights for policy 0, policy_version 840 (0.0030) | |
| [2024-09-06 08:10:26,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 3452928. Throughput: 0: 961.2. Samples: 863472. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:10:26,322][01070] Avg episode reward: [(0, '9.987')] | |
| [2024-09-06 08:10:26,338][06055] Saving new best policy, reward=9.987! | |
| [2024-09-06 08:10:31,316][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.4, 300 sec: 3860.0). Total num frames: 3473408. Throughput: 0: 951.3. Samples: 866546. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:10:31,322][01070] Avg episode reward: [(0, '10.544')] | |
| [2024-09-06 08:10:31,325][06055] Saving new best policy, reward=10.544! | |
| [2024-09-06 08:10:32,772][06068] Updated weights for policy 0, policy_version 850 (0.0028) | |
| [2024-09-06 08:10:36,317][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3860.0). Total num frames: 3497984. Throughput: 0: 993.9. Samples: 873542. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:10:36,319][01070] Avg episode reward: [(0, '11.038')] | |
| [2024-09-06 08:10:36,329][06055] Saving new best policy, reward=11.038! | |
| [2024-09-06 08:10:41,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3846.1). Total num frames: 3510272. Throughput: 0: 980.8. Samples: 878664. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:10:41,325][01070] Avg episode reward: [(0, '10.332')] | |
| [2024-09-06 08:10:44,181][06068] Updated weights for policy 0, policy_version 860 (0.0030) | |
| [2024-09-06 08:10:46,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3860.0). Total num frames: 3530752. Throughput: 0: 953.3. Samples: 880928. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:10:46,318][01070] Avg episode reward: [(0, '10.049')] | |
| [2024-09-06 08:10:46,335][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000862_3530752.pth... | |
| [2024-09-06 08:10:46,466][06055] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000637_2609152.pth | |
| [2024-09-06 08:10:51,317][01070] Fps is (10 sec: 4505.6, 60 sec: 4028.1, 300 sec: 3873.8). Total num frames: 3555328. Throughput: 0: 981.0. Samples: 888068. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:10:51,325][01070] Avg episode reward: [(0, '9.881')] | |
| [2024-09-06 08:10:52,867][06068] Updated weights for policy 0, policy_version 870 (0.0024) | |
| [2024-09-06 08:10:56,318][01070] Fps is (10 sec: 4095.3, 60 sec: 3959.3, 300 sec: 3846.1). Total num frames: 3571712. Throughput: 0: 1010.2. Samples: 894156. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:10:56,321][01070] Avg episode reward: [(0, '10.324')] | |
| [2024-09-06 08:11:01,317][01070] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 3588096. Throughput: 0: 978.2. Samples: 896234. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:11:01,324][01070] Avg episode reward: [(0, '10.751')] | |
| [2024-09-06 08:11:04,130][06068] Updated weights for policy 0, policy_version 880 (0.0019) | |
| [2024-09-06 08:11:06,317][01070] Fps is (10 sec: 4096.7, 60 sec: 3959.5, 300 sec: 3873.8). Total num frames: 3612672. Throughput: 0: 960.8. Samples: 902540. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:11:06,321][01070] Avg episode reward: [(0, '11.121')] | |
| [2024-09-06 08:11:06,332][06055] Saving new best policy, reward=11.121! | |
| [2024-09-06 08:11:11,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.4, 300 sec: 3846.1). Total num frames: 3629056. Throughput: 0: 994.5. Samples: 908226. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:11:11,319][01070] Avg episode reward: [(0, '11.751')] | |
| [2024-09-06 08:11:11,326][06055] Saving new best policy, reward=11.751! | |
| [2024-09-06 08:11:16,317][01070] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3832.2). Total num frames: 3641344. Throughput: 0: 964.6. Samples: 909952. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:11:16,319][01070] Avg episode reward: [(0, '12.049')] | |
| [2024-09-06 08:11:16,333][06055] Saving new best policy, reward=12.049! | |
| [2024-09-06 08:11:17,256][06068] Updated weights for policy 0, policy_version 890 (0.0045) | |
| [2024-09-06 08:11:21,316][01070] Fps is (10 sec: 2867.3, 60 sec: 3686.4, 300 sec: 3832.2). Total num frames: 3657728. Throughput: 0: 893.7. Samples: 913758. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:11:21,318][01070] Avg episode reward: [(0, '13.812')] | |
| [2024-09-06 08:11:21,326][06055] Saving new best policy, reward=13.812! | |
| [2024-09-06 08:11:26,317][01070] Fps is (10 sec: 3686.3, 60 sec: 3754.6, 300 sec: 3818.3). Total num frames: 3678208. Throughput: 0: 932.5. Samples: 920626. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:11:26,325][01070] Avg episode reward: [(0, '15.231')] | |
| [2024-09-06 08:11:26,390][06055] Saving new best policy, reward=15.231! | |
| [2024-09-06 08:11:27,393][06068] Updated weights for policy 0, policy_version 900 (0.0024) | |
| [2024-09-06 08:11:31,316][01070] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3832.3). Total num frames: 3702784. Throughput: 0: 960.0. Samples: 924128. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:11:31,322][01070] Avg episode reward: [(0, '15.872')] | |
| [2024-09-06 08:11:31,328][06055] Saving new best policy, reward=15.872! | |
| [2024-09-06 08:11:36,317][01070] Fps is (10 sec: 3686.6, 60 sec: 3618.1, 300 sec: 3818.3). Total num frames: 3715072. Throughput: 0: 907.6. Samples: 928912. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:11:36,320][01070] Avg episode reward: [(0, '16.561')] | |
| [2024-09-06 08:11:36,333][06055] Saving new best policy, reward=16.561! | |
| [2024-09-06 08:11:38,716][06068] Updated weights for policy 0, policy_version 910 (0.0025) | |
| [2024-09-06 08:11:41,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3832.2). Total num frames: 3739648. Throughput: 0: 909.8. Samples: 935094. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:11:41,318][01070] Avg episode reward: [(0, '14.523')] | |
| [2024-09-06 08:11:46,317][01070] Fps is (10 sec: 4505.4, 60 sec: 3822.9, 300 sec: 3832.2). Total num frames: 3760128. Throughput: 0: 942.4. Samples: 938640. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:11:46,320][01070] Avg episode reward: [(0, '14.427')] | |
| [2024-09-06 08:11:47,204][06068] Updated weights for policy 0, policy_version 920 (0.0035) | |
| [2024-09-06 08:11:51,316][01070] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3832.2). Total num frames: 3776512. Throughput: 0: 934.4. Samples: 944586. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:11:51,321][01070] Avg episode reward: [(0, '14.547')] | |
| [2024-09-06 08:11:56,317][01070] Fps is (10 sec: 3686.6, 60 sec: 3754.8, 300 sec: 3846.1). Total num frames: 3796992. Throughput: 0: 915.3. Samples: 949416. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:11:56,319][01070] Avg episode reward: [(0, '15.007')] | |
| [2024-09-06 08:11:58,865][06068] Updated weights for policy 0, policy_version 930 (0.0040) | |
| [2024-09-06 08:12:01,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3832.2). Total num frames: 3817472. Throughput: 0: 955.3. Samples: 952942. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:12:01,319][01070] Avg episode reward: [(0, '16.278')] | |
| [2024-09-06 08:12:06,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3832.2). Total num frames: 3837952. Throughput: 0: 1027.6. Samples: 959998. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:12:06,326][01070] Avg episode reward: [(0, '16.304')] | |
| [2024-09-06 08:12:09,321][06068] Updated weights for policy 0, policy_version 940 (0.0024) | |
| [2024-09-06 08:12:11,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3846.1). Total num frames: 3854336. Throughput: 0: 969.6. Samples: 964258. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:12:11,320][01070] Avg episode reward: [(0, '16.576')] | |
| [2024-09-06 08:12:11,326][06055] Saving new best policy, reward=16.576! | |
| [2024-09-06 08:12:16,317][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3846.1). Total num frames: 3878912. Throughput: 0: 961.4. Samples: 967390. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:12:16,322][01070] Avg episode reward: [(0, '16.934')] | |
| [2024-09-06 08:12:16,332][06055] Saving new best policy, reward=16.934! | |
| [2024-09-06 08:12:18,962][06068] Updated weights for policy 0, policy_version 950 (0.0061) | |
| [2024-09-06 08:12:21,317][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3860.0). Total num frames: 3899392. Throughput: 0: 1011.4. Samples: 974424. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:12:21,321][01070] Avg episode reward: [(0, '16.941')] | |
| [2024-09-06 08:12:21,326][06055] Saving new best policy, reward=16.941! | |
| [2024-09-06 08:12:26,317][01070] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3873.9). Total num frames: 3915776. Throughput: 0: 986.0. Samples: 979462. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:12:26,321][01070] Avg episode reward: [(0, '17.171')] | |
| [2024-09-06 08:12:26,332][06055] Saving new best policy, reward=17.171! | |
| [2024-09-06 08:12:30,483][06068] Updated weights for policy 0, policy_version 960 (0.0038) | |
| [2024-09-06 08:12:31,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3887.7). Total num frames: 3932160. Throughput: 0: 955.2. Samples: 981622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:12:31,318][01070] Avg episode reward: [(0, '17.914')] | |
| [2024-09-06 08:12:31,330][06055] Saving new best policy, reward=17.914! | |
| [2024-09-06 08:12:36,317][01070] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3887.7). Total num frames: 3956736. Throughput: 0: 980.8. Samples: 988720. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:12:36,319][01070] Avg episode reward: [(0, '17.778')] | |
| [2024-09-06 08:12:39,405][06068] Updated weights for policy 0, policy_version 970 (0.0031) | |
| [2024-09-06 08:12:41,318][01070] Fps is (10 sec: 4504.8, 60 sec: 3959.3, 300 sec: 3873.8). Total num frames: 3977216. Throughput: 0: 1011.3. Samples: 994928. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:12:41,321][01070] Avg episode reward: [(0, '17.411')] | |
| [2024-09-06 08:12:46,317][01070] Fps is (10 sec: 3276.8, 60 sec: 3823.0, 300 sec: 3873.8). Total num frames: 3989504. Throughput: 0: 979.5. Samples: 997020. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:12:46,319][01070] Avg episode reward: [(0, '16.564')] | |
| [2024-09-06 08:12:46,386][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000975_3993600.pth... | |
| [2024-09-06 08:12:46,505][06055] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000747_3059712.pth | |
| [2024-09-06 08:12:49,066][06055] Stopping Batcher_0... | |
| [2024-09-06 08:12:49,066][06055] Loop batcher_evt_loop terminating... | |
| [2024-09-06 08:12:49,068][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-09-06 08:12:49,066][01070] Component Batcher_0 stopped! | |
| [2024-09-06 08:12:49,172][06068] Weights refcount: 2 0 | |
| [2024-09-06 08:12:49,178][06068] Stopping InferenceWorker_p0-w0... | |
| [2024-09-06 08:12:49,180][01070] Component InferenceWorker_p0-w0 stopped! | |
| [2024-09-06 08:12:49,179][06068] Loop inference_proc0-0_evt_loop terminating... | |
| [2024-09-06 08:12:49,262][06055] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000862_3530752.pth | |
| [2024-09-06 08:12:49,282][06055] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-09-06 08:12:49,452][01070] Component LearnerWorker_p0 stopped! | |
| [2024-09-06 08:12:49,458][06055] Stopping LearnerWorker_p0... | |
| [2024-09-06 08:12:49,458][06055] Loop learner_proc0_evt_loop terminating... | |
| [2024-09-06 08:12:49,517][06070] Stopping RolloutWorker_w1... | |
| [2024-09-06 08:12:49,518][06070] Loop rollout_proc1_evt_loop terminating... | |
| [2024-09-06 08:12:49,518][01070] Component RolloutWorker_w1 stopped! | |
| [2024-09-06 08:12:49,528][01070] Component RolloutWorker_w7 stopped! | |
| [2024-09-06 08:12:49,528][06076] Stopping RolloutWorker_w7... | |
| [2024-09-06 08:12:49,534][06076] Loop rollout_proc7_evt_loop terminating... | |
| [2024-09-06 08:12:49,583][06072] Stopping RolloutWorker_w3... | |
| [2024-09-06 08:12:49,587][06074] Stopping RolloutWorker_w5... | |
| [2024-09-06 08:12:49,584][01070] Component RolloutWorker_w3 stopped! | |
| [2024-09-06 08:12:49,589][06072] Loop rollout_proc3_evt_loop terminating... | |
| [2024-09-06 08:12:49,589][01070] Component RolloutWorker_w5 stopped! | |
| [2024-09-06 08:12:49,589][06074] Loop rollout_proc5_evt_loop terminating... | |
| [2024-09-06 08:12:49,624][01070] Component RolloutWorker_w0 stopped! | |
| [2024-09-06 08:12:49,629][06069] Stopping RolloutWorker_w0... | |
| [2024-09-06 08:12:49,636][06069] Loop rollout_proc0_evt_loop terminating... | |
| [2024-09-06 08:12:49,659][01070] Component RolloutWorker_w6 stopped! | |
| [2024-09-06 08:12:49,662][06075] Stopping RolloutWorker_w6... | |
| [2024-09-06 08:12:49,663][06075] Loop rollout_proc6_evt_loop terminating... | |
| [2024-09-06 08:12:49,674][01070] Component RolloutWorker_w2 stopped! | |
| [2024-09-06 08:12:49,677][06071] Stopping RolloutWorker_w2... | |
| [2024-09-06 08:12:49,677][06071] Loop rollout_proc2_evt_loop terminating... | |
| [2024-09-06 08:12:49,711][01070] Component RolloutWorker_w4 stopped! | |
| [2024-09-06 08:12:49,714][01070] Waiting for process learner_proc0 to stop... | |
| [2024-09-06 08:12:49,717][06073] Stopping RolloutWorker_w4... | |
| [2024-09-06 08:12:49,718][06073] Loop rollout_proc4_evt_loop terminating... | |
| [2024-09-06 08:12:51,010][01070] Waiting for process inference_proc0-0 to join... | |
| [2024-09-06 08:12:51,015][01070] Waiting for process rollout_proc0 to join... | |
| [2024-09-06 08:12:53,043][01070] Waiting for process rollout_proc1 to join... | |
| [2024-09-06 08:12:53,046][01070] Waiting for process rollout_proc2 to join... | |
| [2024-09-06 08:12:53,049][01070] Waiting for process rollout_proc3 to join... | |
| [2024-09-06 08:12:53,051][01070] Waiting for process rollout_proc4 to join... | |
| [2024-09-06 08:12:53,052][01070] Waiting for process rollout_proc5 to join... | |
| [2024-09-06 08:12:53,054][01070] Waiting for process rollout_proc6 to join... | |
| [2024-09-06 08:12:53,057][01070] Waiting for process rollout_proc7 to join... | |
| [2024-09-06 08:12:53,059][01070] Batcher 0 profile tree view: | |
| batching: 28.0352, releasing_batches: 0.0261 | |
| [2024-09-06 08:12:53,060][01070] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0001 | |
| wait_policy_total: 396.0637 | |
| update_model: 9.3731 | |
| weight_update: 0.0039 | |
| one_step: 0.0115 | |
| handle_policy_step: 605.9671 | |
| deserialize: 14.6060, stack: 3.1718, obs_to_device_normalize: 122.4392, forward: 322.4806, send_messages: 28.9576 | |
| prepare_outputs: 84.3394 | |
| to_cpu: 49.4062 | |
| [2024-09-06 08:12:53,062][01070] Learner 0 profile tree view: | |
| misc: 0.0070, prepare_batch: 14.0241 | |
| train: 74.3266 | |
| epoch_init: 0.0157, minibatch_init: 0.0064, losses_postprocess: 0.6825, kl_divergence: 0.6975, after_optimizer: 33.2803 | |
| calculate_losses: 26.8972 | |
| losses_init: 0.0105, forward_head: 1.2733, bptt_initial: 18.0432, tail: 1.1022, advantages_returns: 0.2551, losses: 3.8922 | |
| bptt: 1.9881 | |
| bptt_forward_core: 1.8812 | |
| update: 12.1225 | |
| clip: 0.8825 | |
| [2024-09-06 08:12:53,063][01070] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.3955, enqueue_policy_requests: 94.3963, env_step: 821.5439, overhead: 13.5076, complete_rollouts: 7.0298 | |
| save_policy_outputs: 20.8305 | |
| split_output_tensors: 8.5837 | |
| [2024-09-06 08:12:53,065][01070] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.3496, enqueue_policy_requests: 97.5017, env_step: 821.5330, overhead: 13.0956, complete_rollouts: 6.8883 | |
| save_policy_outputs: 20.0181 | |
| split_output_tensors: 7.8801 | |
| [2024-09-06 08:12:53,066][01070] Loop Runner_EvtLoop terminating... | |
| [2024-09-06 08:12:53,068][01070] Runner profile tree view: | |
| main_loop: 1081.6875 | |
| [2024-09-06 08:12:53,069][01070] Collected {0: 4005888}, FPS: 3703.4 | |
| [2024-09-06 08:26:47,354][01070] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2024-09-06 08:26:47,356][01070] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-09-06 08:26:47,359][01070] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-09-06 08:26:47,361][01070] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-09-06 08:26:47,363][01070] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 08:26:47,365][01070] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-09-06 08:26:47,366][01070] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 08:26:47,367][01070] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-09-06 08:26:47,368][01070] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2024-09-06 08:26:47,369][01070] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2024-09-06 08:26:47,370][01070] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-09-06 08:26:47,371][01070] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-09-06 08:26:47,372][01070] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-09-06 08:26:47,373][01070] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-09-06 08:26:47,374][01070] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-09-06 08:26:47,410][01070] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:26:47,413][01070] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 08:26:47,415][01070] RunningMeanStd input shape: (1,) | |
| [2024-09-06 08:26:47,432][01070] ConvEncoder: input_channels=3 | |
| [2024-09-06 08:26:47,594][01070] Conv encoder output size: 512 | |
| [2024-09-06 08:26:47,596][01070] Policy head output size: 512 | |
| [2024-09-06 08:26:47,888][01070] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-09-06 08:26:48,754][01070] Num frames 100... | |
| [2024-09-06 08:26:48,875][01070] Num frames 200... | |
| [2024-09-06 08:26:48,994][01070] Num frames 300... | |
| [2024-09-06 08:26:49,113][01070] Num frames 400... | |
| [2024-09-06 08:26:49,254][01070] Num frames 500... | |
| [2024-09-06 08:26:49,388][01070] Avg episode rewards: #0: 11.440, true rewards: #0: 5.440 | |
| [2024-09-06 08:26:49,391][01070] Avg episode reward: 11.440, avg true_objective: 5.440 | |
| [2024-09-06 08:26:49,511][01070] Num frames 600... | |
| [2024-09-06 08:26:49,678][01070] Num frames 700... | |
| [2024-09-06 08:26:49,846][01070] Num frames 800... | |
| [2024-09-06 08:26:50,007][01070] Num frames 900... | |
| [2024-09-06 08:26:50,168][01070] Num frames 1000... | |
| [2024-09-06 08:26:50,331][01070] Num frames 1100... | |
| [2024-09-06 08:26:50,503][01070] Num frames 1200... | |
| [2024-09-06 08:26:50,681][01070] Num frames 1300... | |
| [2024-09-06 08:26:50,858][01070] Num frames 1400... | |
| [2024-09-06 08:26:51,030][01070] Num frames 1500... | |
| [2024-09-06 08:26:51,200][01070] Num frames 1600... | |
| [2024-09-06 08:26:51,364][01070] Avg episode rewards: #0: 19.795, true rewards: #0: 8.295 | |
| [2024-09-06 08:26:51,366][01070] Avg episode reward: 19.795, avg true_objective: 8.295 | |
| [2024-09-06 08:26:51,440][01070] Num frames 1700... | |
| [2024-09-06 08:26:51,597][01070] Num frames 1800... | |
| [2024-09-06 08:26:51,716][01070] Num frames 1900... | |
| [2024-09-06 08:26:51,843][01070] Num frames 2000... | |
| [2024-09-06 08:26:51,964][01070] Num frames 2100... | |
| [2024-09-06 08:26:52,083][01070] Num frames 2200... | |
| [2024-09-06 08:26:52,203][01070] Num frames 2300... | |
| [2024-09-06 08:26:52,366][01070] Avg episode rewards: #0: 18.633, true rewards: #0: 7.967 | |
| [2024-09-06 08:26:52,367][01070] Avg episode reward: 18.633, avg true_objective: 7.967 | |
| [2024-09-06 08:26:52,382][01070] Num frames 2400... | |
| [2024-09-06 08:26:52,509][01070] Num frames 2500... | |
| [2024-09-06 08:26:52,629][01070] Num frames 2600... | |
| [2024-09-06 08:26:52,748][01070] Num frames 2700... | |
| [2024-09-06 08:26:52,878][01070] Num frames 2800... | |
| [2024-09-06 08:26:52,997][01070] Num frames 2900... | |
| [2024-09-06 08:26:53,117][01070] Num frames 3000... | |
| [2024-09-06 08:26:53,239][01070] Num frames 3100... | |
| [2024-09-06 08:26:53,401][01070] Avg episode rewards: #0: 18.478, true rewards: #0: 7.977 | |
| [2024-09-06 08:26:53,402][01070] Avg episode reward: 18.478, avg true_objective: 7.977 | |
| [2024-09-06 08:26:53,416][01070] Num frames 3200... | |
| [2024-09-06 08:26:53,543][01070] Num frames 3300... | |
| [2024-09-06 08:26:53,662][01070] Num frames 3400... | |
| [2024-09-06 08:26:53,779][01070] Num frames 3500... | |
| [2024-09-06 08:26:53,906][01070] Num frames 3600... | |
| [2024-09-06 08:26:54,025][01070] Num frames 3700... | |
| [2024-09-06 08:26:54,144][01070] Num frames 3800... | |
| [2024-09-06 08:26:54,263][01070] Num frames 3900... | |
| [2024-09-06 08:26:54,327][01070] Avg episode rewards: #0: 18.210, true rewards: #0: 7.810 | |
| [2024-09-06 08:26:54,328][01070] Avg episode reward: 18.210, avg true_objective: 7.810 | |
| [2024-09-06 08:26:54,438][01070] Num frames 4000... | |
| [2024-09-06 08:26:54,568][01070] Num frames 4100... | |
| [2024-09-06 08:26:54,685][01070] Num frames 4200... | |
| [2024-09-06 08:26:54,803][01070] Num frames 4300... | |
| [2024-09-06 08:26:54,928][01070] Num frames 4400... | |
| [2024-09-06 08:26:55,056][01070] Num frames 4500... | |
| [2024-09-06 08:26:55,190][01070] Num frames 4600... | |
| [2024-09-06 08:26:55,326][01070] Num frames 4700... | |
| [2024-09-06 08:26:55,391][01070] Avg episode rewards: #0: 17.675, true rewards: #0: 7.842 | |
| [2024-09-06 08:26:55,392][01070] Avg episode reward: 17.675, avg true_objective: 7.842 | |
| [2024-09-06 08:26:55,513][01070] Num frames 4800... | |
| [2024-09-06 08:26:55,637][01070] Num frames 4900... | |
| [2024-09-06 08:26:55,758][01070] Num frames 5000... | |
| [2024-09-06 08:26:55,827][01070] Avg episode rewards: #0: 15.870, true rewards: #0: 7.156 | |
| [2024-09-06 08:26:55,828][01070] Avg episode reward: 15.870, avg true_objective: 7.156 | |
| [2024-09-06 08:26:55,944][01070] Num frames 5100... | |
| [2024-09-06 08:26:56,065][01070] Num frames 5200... | |
| [2024-09-06 08:26:56,186][01070] Num frames 5300... | |
| [2024-09-06 08:26:56,305][01070] Num frames 5400... | |
| [2024-09-06 08:26:56,425][01070] Num frames 5500... | |
| [2024-09-06 08:26:56,557][01070] Num frames 5600... | |
| [2024-09-06 08:26:56,681][01070] Num frames 5700... | |
| [2024-09-06 08:26:56,803][01070] Num frames 5800... | |
| [2024-09-06 08:26:56,931][01070] Num frames 5900... | |
| [2024-09-06 08:26:57,053][01070] Num frames 6000... | |
| [2024-09-06 08:26:57,173][01070] Num frames 6100... | |
| [2024-09-06 08:26:57,294][01070] Num frames 6200... | |
| [2024-09-06 08:26:57,418][01070] Num frames 6300... | |
| [2024-09-06 08:26:57,547][01070] Num frames 6400... | |
| [2024-09-06 08:26:57,675][01070] Num frames 6500... | |
| [2024-09-06 08:26:57,795][01070] Num frames 6600... | |
| [2024-09-06 08:26:57,916][01070] Num frames 6700... | |
| [2024-09-06 08:26:58,045][01070] Num frames 6800... | |
| [2024-09-06 08:26:58,169][01070] Num frames 6900... | |
| [2024-09-06 08:26:58,260][01070] Avg episode rewards: #0: 19.911, true rewards: #0: 8.661 | |
| [2024-09-06 08:26:58,261][01070] Avg episode reward: 19.911, avg true_objective: 8.661 | |
| [2024-09-06 08:26:58,349][01070] Num frames 7000... | |
| [2024-09-06 08:26:58,470][01070] Num frames 7100... | |
| [2024-09-06 08:26:58,599][01070] Num frames 7200... | |
| [2024-09-06 08:26:58,720][01070] Num frames 7300... | |
| [2024-09-06 08:26:58,842][01070] Num frames 7400... | |
| [2024-09-06 08:26:58,969][01070] Num frames 7500... | |
| [2024-09-06 08:26:59,094][01070] Num frames 7600... | |
| [2024-09-06 08:26:59,214][01070] Num frames 7700... | |
| [2024-09-06 08:26:59,332][01070] Num frames 7800... | |
| [2024-09-06 08:26:59,457][01070] Num frames 7900... | |
| [2024-09-06 08:26:59,545][01070] Avg episode rewards: #0: 19.801, true rewards: #0: 8.801 | |
| [2024-09-06 08:26:59,546][01070] Avg episode reward: 19.801, avg true_objective: 8.801 | |
| [2024-09-06 08:26:59,649][01070] Num frames 8000... | |
| [2024-09-06 08:26:59,793][01070] Num frames 8100... | |
| [2024-09-06 08:26:59,916][01070] Num frames 8200... | |
| [2024-09-06 08:27:00,042][01070] Num frames 8300... | |
| [2024-09-06 08:27:00,161][01070] Num frames 8400... | |
| [2024-09-06 08:27:00,280][01070] Num frames 8500... | |
| [2024-09-06 08:27:00,411][01070] Avg episode rewards: #0: 18.961, true rewards: #0: 8.561 | |
| [2024-09-06 08:27:00,412][01070] Avg episode reward: 18.961, avg true_objective: 8.561 | |
| [2024-09-06 08:27:55,072][01070] Replay video saved to /content/train_dir/default_experiment/replay.mp4! | |
| [2024-09-06 08:29:44,806][01070] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2024-09-06 08:29:44,808][01070] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-09-06 08:29:44,810][01070] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-09-06 08:29:44,811][01070] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-09-06 08:29:44,813][01070] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 08:29:44,814][01070] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-09-06 08:29:44,817][01070] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2024-09-06 08:29:44,818][01070] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-09-06 08:29:44,820][01070] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2024-09-06 08:29:44,822][01070] Adding new argument 'hf_repository'='Re-Re/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2024-09-06 08:29:44,823][01070] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-09-06 08:29:44,826][01070] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-09-06 08:29:44,827][01070] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-09-06 08:29:44,828][01070] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-09-06 08:29:44,829][01070] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-09-06 08:29:44,858][01070] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 08:29:44,861][01070] RunningMeanStd input shape: (1,) | |
| [2024-09-06 08:29:44,875][01070] ConvEncoder: input_channels=3 | |
| [2024-09-06 08:29:44,911][01070] Conv encoder output size: 512 | |
| [2024-09-06 08:29:44,912][01070] Policy head output size: 512 | |
| [2024-09-06 08:29:44,931][01070] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-09-06 08:29:45,358][01070] Num frames 100... | |
| [2024-09-06 08:29:45,488][01070] Num frames 200... | |
| [2024-09-06 08:29:45,624][01070] Num frames 300... | |
| [2024-09-06 08:29:45,744][01070] Num frames 400... | |
| [2024-09-06 08:29:45,863][01070] Num frames 500... | |
| [2024-09-06 08:29:45,987][01070] Num frames 600... | |
| [2024-09-06 08:29:46,108][01070] Num frames 700... | |
| [2024-09-06 08:29:46,235][01070] Num frames 800... | |
| [2024-09-06 08:29:46,300][01070] Avg episode rewards: #0: 17.070, true rewards: #0: 8.070 | |
| [2024-09-06 08:29:46,302][01070] Avg episode reward: 17.070, avg true_objective: 8.070 | |
| [2024-09-06 08:29:46,416][01070] Num frames 900... | |
| [2024-09-06 08:29:46,542][01070] Num frames 1000... | |
| [2024-09-06 08:29:46,669][01070] Num frames 1100... | |
| [2024-09-06 08:29:46,794][01070] Num frames 1200... | |
| [2024-09-06 08:29:46,917][01070] Num frames 1300... | |
| [2024-09-06 08:29:47,039][01070] Num frames 1400... | |
| [2024-09-06 08:29:47,189][01070] Avg episode rewards: #0: 14.895, true rewards: #0: 7.395 | |
| [2024-09-06 08:29:47,191][01070] Avg episode reward: 14.895, avg true_objective: 7.395 | |
| [2024-09-06 08:29:47,218][01070] Num frames 1500... | |
| [2024-09-06 08:29:47,338][01070] Num frames 1600... | |
| [2024-09-06 08:29:47,461][01070] Num frames 1700... | |
| [2024-09-06 08:29:47,587][01070] Num frames 1800... | |
| [2024-09-06 08:29:47,708][01070] Num frames 1900... | |
| [2024-09-06 08:29:47,826][01070] Num frames 2000... | |
| [2024-09-06 08:29:47,910][01070] Avg episode rewards: #0: 12.410, true rewards: #0: 6.743 | |
| [2024-09-06 08:29:47,913][01070] Avg episode reward: 12.410, avg true_objective: 6.743 | |
| [2024-09-06 08:29:48,007][01070] Num frames 2100... | |
| [2024-09-06 08:29:48,128][01070] Num frames 2200... | |
| [2024-09-06 08:29:48,257][01070] Num frames 2300... | |
| [2024-09-06 08:29:48,380][01070] Num frames 2400... | |
| [2024-09-06 08:29:48,524][01070] Avg episode rewards: #0: 11.178, true rewards: #0: 6.177 | |
| [2024-09-06 08:29:48,526][01070] Avg episode reward: 11.178, avg true_objective: 6.177 | |
| [2024-09-06 08:29:48,562][01070] Num frames 2500... | |
| [2024-09-06 08:29:48,679][01070] Num frames 2600... | |
| [2024-09-06 08:29:48,802][01070] Num frames 2700... | |
| [2024-09-06 08:29:48,926][01070] Num frames 2800... | |
| [2024-09-06 08:29:49,047][01070] Num frames 2900... | |
| [2024-09-06 08:29:49,168][01070] Num frames 3000... | |
| [2024-09-06 08:29:49,296][01070] Num frames 3100... | |
| [2024-09-06 08:29:49,421][01070] Num frames 3200... | |
| [2024-09-06 08:29:49,551][01070] Num frames 3300... | |
| [2024-09-06 08:29:49,677][01070] Num frames 3400... | |
| [2024-09-06 08:29:49,816][01070] Avg episode rewards: #0: 12.736, true rewards: #0: 6.936 | |
| [2024-09-06 08:29:49,819][01070] Avg episode reward: 12.736, avg true_objective: 6.936 | |
| [2024-09-06 08:29:49,860][01070] Num frames 3500... | |
| [2024-09-06 08:29:49,981][01070] Num frames 3600... | |
| [2024-09-06 08:29:50,103][01070] Num frames 3700... | |
| [2024-09-06 08:29:50,228][01070] Num frames 3800... | |
| [2024-09-06 08:29:50,360][01070] Num frames 3900... | |
| [2024-09-06 08:29:50,439][01070] Avg episode rewards: #0: 11.862, true rewards: #0: 6.528 | |
| [2024-09-06 08:29:50,441][01070] Avg episode reward: 11.862, avg true_objective: 6.528 | |
| [2024-09-06 08:29:50,548][01070] Num frames 4000... | |
| [2024-09-06 08:29:50,713][01070] Num frames 4100... | |
| [2024-09-06 08:29:50,880][01070] Num frames 4200... | |
| [2024-09-06 08:29:51,043][01070] Num frames 4300... | |
| [2024-09-06 08:29:51,205][01070] Avg episode rewards: #0: 10.950, true rewards: #0: 6.236 | |
| [2024-09-06 08:29:51,208][01070] Avg episode reward: 10.950, avg true_objective: 6.236 | |
| [2024-09-06 08:29:51,274][01070] Num frames 4400... | |
| [2024-09-06 08:29:51,443][01070] Num frames 4500... | |
| [2024-09-06 08:29:51,608][01070] Num frames 4600... | |
| [2024-09-06 08:29:51,767][01070] Num frames 4700... | |
| [2024-09-06 08:29:51,938][01070] Num frames 4800... | |
| [2024-09-06 08:29:52,113][01070] Num frames 4900... | |
| [2024-09-06 08:29:52,291][01070] Avg episode rewards: #0: 10.716, true rewards: #0: 6.216 | |
| [2024-09-06 08:29:52,294][01070] Avg episode reward: 10.716, avg true_objective: 6.216 | |
| [2024-09-06 08:29:52,349][01070] Num frames 5000... | |
| [2024-09-06 08:29:52,549][01070] Num frames 5100... | |
| [2024-09-06 08:29:52,723][01070] Num frames 5200... | |
| [2024-09-06 08:29:52,895][01070] Num frames 5300... | |
| [2024-09-06 08:29:53,063][01070] Num frames 5400... | |
| [2024-09-06 08:29:53,183][01070] Num frames 5500... | |
| [2024-09-06 08:29:53,304][01070] Num frames 5600... | |
| [2024-09-06 08:29:53,437][01070] Num frames 5700... | |
| [2024-09-06 08:29:53,565][01070] Num frames 5800... | |
| [2024-09-06 08:29:53,688][01070] Num frames 5900... | |
| [2024-09-06 08:29:53,810][01070] Num frames 6000... | |
| [2024-09-06 08:29:53,930][01070] Num frames 6100... | |
| [2024-09-06 08:29:54,050][01070] Num frames 6200... | |
| [2024-09-06 08:29:54,174][01070] Num frames 6300... | |
| [2024-09-06 08:29:54,254][01070] Avg episode rewards: #0: 13.130, true rewards: #0: 7.019 | |
| [2024-09-06 08:29:54,255][01070] Avg episode reward: 13.130, avg true_objective: 7.019 | |
| [2024-09-06 08:29:54,356][01070] Num frames 6400... | |
| [2024-09-06 08:29:54,490][01070] Num frames 6500... | |
| [2024-09-06 08:29:54,609][01070] Num frames 6600... | |
| [2024-09-06 08:29:54,726][01070] Num frames 6700... | |
| [2024-09-06 08:29:54,842][01070] Num frames 6800... | |
| [2024-09-06 08:29:54,961][01070] Num frames 6900... | |
| [2024-09-06 08:29:55,080][01070] Num frames 7000... | |
| [2024-09-06 08:29:55,202][01070] Num frames 7100... | |
| [2024-09-06 08:29:55,323][01070] Num frames 7200... | |
| [2024-09-06 08:29:55,437][01070] Avg episode rewards: #0: 13.847, true rewards: #0: 7.247 | |
| [2024-09-06 08:29:55,439][01070] Avg episode reward: 13.847, avg true_objective: 7.247 | |
| [2024-09-06 08:30:39,348][01070] Replay video saved to /content/train_dir/default_experiment/replay.mp4! | |
| [2024-09-06 08:30:46,004][01070] The model has been pushed to https://huggingface.co/Re-Re/rl_course_vizdoom_health_gathering_supreme | |
| [2024-09-06 08:35:27,232][01070] Environment doom_basic already registered, overwriting... | |
| [2024-09-06 08:35:27,234][01070] Environment doom_two_colors_easy already registered, overwriting... | |
| [2024-09-06 08:35:27,237][01070] Environment doom_two_colors_hard already registered, overwriting... | |
| [2024-09-06 08:35:27,238][01070] Environment doom_dm already registered, overwriting... | |
| [2024-09-06 08:35:27,240][01070] Environment doom_dwango5 already registered, overwriting... | |
| [2024-09-06 08:35:27,241][01070] Environment doom_my_way_home_flat_actions already registered, overwriting... | |
| [2024-09-06 08:35:27,242][01070] Environment doom_defend_the_center_flat_actions already registered, overwriting... | |
| [2024-09-06 08:35:27,243][01070] Environment doom_my_way_home already registered, overwriting... | |
| [2024-09-06 08:35:27,244][01070] Environment doom_deadly_corridor already registered, overwriting... | |
| [2024-09-06 08:35:27,245][01070] Environment doom_defend_the_center already registered, overwriting... | |
| [2024-09-06 08:35:27,246][01070] Environment doom_defend_the_line already registered, overwriting... | |
| [2024-09-06 08:35:27,247][01070] Environment doom_health_gathering already registered, overwriting... | |
| [2024-09-06 08:35:27,248][01070] Environment doom_health_gathering_supreme already registered, overwriting... | |
| [2024-09-06 08:35:27,250][01070] Environment doom_battle already registered, overwriting... | |
| [2024-09-06 08:35:27,251][01070] Environment doom_battle2 already registered, overwriting... | |
| [2024-09-06 08:35:27,252][01070] Environment doom_duel_bots already registered, overwriting... | |
| [2024-09-06 08:35:27,253][01070] Environment doom_deathmatch_bots already registered, overwriting... | |
| [2024-09-06 08:35:27,254][01070] Environment doom_duel already registered, overwriting... | |
| [2024-09-06 08:35:27,255][01070] Environment doom_deathmatch_full already registered, overwriting... | |
| [2024-09-06 08:35:27,256][01070] Environment doom_benchmark already registered, overwriting... | |
| [2024-09-06 08:35:27,257][01070] register_encoder_factory: <function make_vizdoom_encoder at 0x78dc5537e170> | |
| [2024-09-06 08:35:27,282][01070] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2024-09-06 08:35:27,284][01070] Overriding arg 'train_for_env_steps' with value 5000000 passed from command line | |
| [2024-09-06 08:35:27,291][01070] Experiment dir /content/train_dir/default_experiment already exists! | |
| [2024-09-06 08:35:27,293][01070] Resuming existing experiment from /content/train_dir/default_experiment... | |
| [2024-09-06 08:35:27,295][01070] Weights and Biases integration disabled | |
| [2024-09-06 08:35:27,298][01070] Environment var CUDA_VISIBLE_DEVICES is 0 | |
| [2024-09-06 08:35:29,357][01070] Starting experiment with the following configuration: | |
| help=False | |
| algo=APPO | |
| env=doom_health_gathering_supreme | |
| experiment=default_experiment | |
| train_dir=/content/train_dir | |
| restart_behavior=resume | |
| device=gpu | |
| seed=None | |
| num_policies=1 | |
| async_rl=True | |
| serial_mode=False | |
| batched_sampling=False | |
| num_batches_to_accumulate=2 | |
| worker_num_splits=2 | |
| policy_workers_per_policy=1 | |
| max_policy_lag=1000 | |
| num_workers=8 | |
| num_envs_per_worker=4 | |
| batch_size=1024 | |
| num_batches_per_epoch=1 | |
| num_epochs=1 | |
| rollout=32 | |
| recurrence=32 | |
| shuffle_minibatches=False | |
| gamma=0.99 | |
| reward_scale=1.0 | |
| reward_clip=1000.0 | |
| value_bootstrap=False | |
| normalize_returns=True | |
| exploration_loss_coeff=0.001 | |
| value_loss_coeff=0.5 | |
| kl_loss_coeff=0.0 | |
| exploration_loss=symmetric_kl | |
| gae_lambda=0.95 | |
| ppo_clip_ratio=0.1 | |
| ppo_clip_value=0.2 | |
| with_vtrace=False | |
| vtrace_rho=1.0 | |
| vtrace_c=1.0 | |
| optimizer=adam | |
| adam_eps=1e-06 | |
| adam_beta1=0.9 | |
| adam_beta2=0.999 | |
| max_grad_norm=4.0 | |
| learning_rate=0.0001 | |
| lr_schedule=constant | |
| lr_schedule_kl_threshold=0.008 | |
| lr_adaptive_min=1e-06 | |
| lr_adaptive_max=0.01 | |
| obs_subtract_mean=0.0 | |
| obs_scale=255.0 | |
| normalize_input=True | |
| normalize_input_keys=None | |
| decorrelate_experience_max_seconds=0 | |
| decorrelate_envs_on_one_worker=True | |
| actor_worker_gpus=[] | |
| set_workers_cpu_affinity=True | |
| force_envs_single_thread=False | |
| default_niceness=0 | |
| log_to_file=True | |
| experiment_summaries_interval=10 | |
| flush_summaries_interval=30 | |
| stats_avg=100 | |
| summaries_use_frameskip=True | |
| heartbeat_interval=20 | |
| heartbeat_reporting_interval=600 | |
| train_for_env_steps=5000000 | |
| train_for_seconds=10000000000 | |
| save_every_sec=120 | |
| keep_checkpoints=2 | |
| load_checkpoint_kind=latest | |
| save_milestones_sec=-1 | |
| save_best_every_sec=5 | |
| save_best_metric=reward | |
| save_best_after=100000 | |
| benchmark=False | |
| encoder_mlp_layers=[512, 512] | |
| encoder_conv_architecture=convnet_simple | |
| encoder_conv_mlp_layers=[512] | |
| use_rnn=True | |
| rnn_size=512 | |
| rnn_type=gru | |
| rnn_num_layers=1 | |
| decoder_mlp_layers=[] | |
| nonlinearity=elu | |
| policy_initialization=orthogonal | |
| policy_init_gain=1.0 | |
| actor_critic_share_weights=True | |
| adaptive_stddev=True | |
| continuous_tanh_scale=0.0 | |
| initial_stddev=1.0 | |
| use_env_info_cache=False | |
| env_gpu_actions=False | |
| env_gpu_observations=True | |
| env_frameskip=4 | |
| env_framestack=1 | |
| pixel_format=CHW | |
| use_record_episode_statistics=False | |
| with_wandb=False | |
| wandb_user=None | |
| wandb_project=sample_factory | |
| wandb_group=None | |
| wandb_job_type=SF | |
| wandb_tags=[] | |
| with_pbt=False | |
| pbt_mix_policies_in_one_env=True | |
| pbt_period_env_steps=5000000 | |
| pbt_start_mutation=20000000 | |
| pbt_replace_fraction=0.3 | |
| pbt_mutation_rate=0.15 | |
| pbt_replace_reward_gap=0.1 | |
| pbt_replace_reward_gap_absolute=1e-06 | |
| pbt_optimize_gamma=False | |
| pbt_target_objective=true_objective | |
| pbt_perturb_min=1.1 | |
| pbt_perturb_max=1.5 | |
| num_agents=-1 | |
| num_humans=0 | |
| num_bots=-1 | |
| start_bot_difficulty=None | |
| timelimit=None | |
| res_w=128 | |
| res_h=72 | |
| wide_aspect_ratio=False | |
| eval_env_frameskip=1 | |
| fps=35 | |
| command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 | |
| cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} | |
| git_hash=unknown | |
| git_repo_name=not a git repository | |
| [2024-09-06 08:35:29,359][01070] Saving configuration to /content/train_dir/default_experiment/config.json... | |
| [2024-09-06 08:35:29,362][01070] Rollout worker 0 uses device cpu | |
| [2024-09-06 08:35:29,364][01070] Rollout worker 1 uses device cpu | |
| [2024-09-06 08:35:29,365][01070] Rollout worker 2 uses device cpu | |
| [2024-09-06 08:35:29,366][01070] Rollout worker 3 uses device cpu | |
| [2024-09-06 08:35:29,368][01070] Rollout worker 4 uses device cpu | |
| [2024-09-06 08:35:29,369][01070] Rollout worker 5 uses device cpu | |
| [2024-09-06 08:35:29,370][01070] Rollout worker 6 uses device cpu | |
| [2024-09-06 08:35:29,372][01070] Rollout worker 7 uses device cpu | |
| [2024-09-06 08:35:29,446][01070] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 08:35:29,447][01070] InferenceWorker_p0-w0: min num requests: 2 | |
| [2024-09-06 08:35:29,485][01070] Starting all processes... | |
| [2024-09-06 08:35:29,486][01070] Starting process learner_proc0 | |
| [2024-09-06 08:35:29,535][01070] Starting all processes... | |
| [2024-09-06 08:35:29,540][01070] Starting process inference_proc0-0 | |
| [2024-09-06 08:35:29,541][01070] Starting process rollout_proc0 | |
| [2024-09-06 08:35:29,541][01070] Starting process rollout_proc1 | |
| [2024-09-06 08:35:29,541][01070] Starting process rollout_proc2 | |
| [2024-09-06 08:35:29,541][01070] Starting process rollout_proc3 | |
| [2024-09-06 08:35:29,541][01070] Starting process rollout_proc4 | |
| [2024-09-06 08:35:29,541][01070] Starting process rollout_proc5 | |
| [2024-09-06 08:35:29,736][01070] Starting process rollout_proc7 | |
| [2024-09-06 08:35:29,753][01070] Starting process rollout_proc6 | |
| [2024-09-06 08:35:43,326][19093] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 08:35:43,327][19093] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2024-09-06 08:35:43,391][19093] Num visible devices: 1 | |
| [2024-09-06 08:35:43,432][19093] Starting seed is not provided | |
| [2024-09-06 08:35:43,433][19093] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 08:35:43,434][19093] Initializing actor-critic model on device cuda:0 | |
| [2024-09-06 08:35:43,434][19093] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 08:35:43,436][19093] RunningMeanStd input shape: (1,) | |
| [2024-09-06 08:35:43,522][19093] ConvEncoder: input_channels=3 | |
| [2024-09-06 08:35:44,545][19093] Conv encoder output size: 512 | |
| [2024-09-06 08:35:44,548][19093] Policy head output size: 512 | |
| [2024-09-06 08:35:44,679][19093] Created Actor Critic model with architecture: | |
| [2024-09-06 08:35:44,680][19093] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2024-09-06 08:35:45,399][19110] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 08:35:45,404][19110] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2024-09-06 08:35:45,627][19116] Worker 5 uses CPU cores [1] | |
| [2024-09-06 08:35:45,628][19113] Worker 2 uses CPU cores [0] | |
| [2024-09-06 08:35:45,638][19110] Num visible devices: 1 | |
| [2024-09-06 08:35:45,693][19114] Worker 4 uses CPU cores [0] | |
| [2024-09-06 08:35:45,750][19093] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2024-09-06 08:35:45,925][19111] Worker 1 uses CPU cores [1] | |
| [2024-09-06 08:35:45,964][19117] Worker 7 uses CPU cores [1] | |
| [2024-09-06 08:35:46,011][19118] Worker 6 uses CPU cores [0] | |
| [2024-09-06 08:35:46,116][19112] Worker 0 uses CPU cores [0] | |
| [2024-09-06 08:35:46,119][19115] Worker 3 uses CPU cores [1] | |
| [2024-09-06 08:35:46,846][19093] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2024-09-06 08:35:46,893][19093] Loading model from checkpoint | |
| [2024-09-06 08:35:46,895][19093] Loaded experiment state at self.train_step=978, self.env_steps=4005888 | |
| [2024-09-06 08:35:46,896][19093] Initialized policy 0 weights for model version 978 | |
| [2024-09-06 08:35:46,906][19093] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 08:35:46,914][19093] LearnerWorker_p0 finished initialization! | |
| [2024-09-06 08:35:47,083][19110] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 08:35:47,085][19110] RunningMeanStd input shape: (1,) | |
| [2024-09-06 08:35:47,103][19110] ConvEncoder: input_channels=3 | |
| [2024-09-06 08:35:47,256][19110] Conv encoder output size: 512 | |
| [2024-09-06 08:35:47,257][19110] Policy head output size: 512 | |
| [2024-09-06 08:35:47,299][01070] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 4005888. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 08:35:47,343][01070] Inference worker 0-0 is ready! | |
| [2024-09-06 08:35:47,345][01070] All inference workers are ready! Signal rollout workers to start! | |
| [2024-09-06 08:35:47,666][19115] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:35:47,759][19117] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:35:47,841][19111] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:35:47,847][19116] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:35:47,859][19118] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:35:47,902][19114] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:35:47,911][19113] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:35:47,923][19112] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 08:35:49,437][01070] Heartbeat connected on Batcher_0 | |
| [2024-09-06 08:35:49,445][01070] Heartbeat connected on LearnerWorker_p0 | |
| [2024-09-06 08:35:49,476][01070] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2024-09-06 08:35:49,689][19112] Decorrelating experience for 0 frames... | |
| [2024-09-06 08:35:49,691][19113] Decorrelating experience for 0 frames... | |
| [2024-09-06 08:35:49,916][19115] Decorrelating experience for 0 frames... | |
| [2024-09-06 08:35:49,947][19117] Decorrelating experience for 0 frames... | |
| [2024-09-06 08:35:50,051][19111] Decorrelating experience for 0 frames... | |
| [2024-09-06 08:35:50,057][19116] Decorrelating experience for 0 frames... | |
| [2024-09-06 08:35:50,424][19112] Decorrelating experience for 32 frames... | |
| [2024-09-06 08:35:51,332][19117] Decorrelating experience for 32 frames... | |
| [2024-09-06 08:35:51,378][19115] Decorrelating experience for 32 frames... | |
| [2024-09-06 08:35:51,423][19111] Decorrelating experience for 32 frames... | |
| [2024-09-06 08:35:51,456][19114] Decorrelating experience for 0 frames... | |
| [2024-09-06 08:35:51,798][19113] Decorrelating experience for 32 frames... | |
| [2024-09-06 08:35:52,103][19112] Decorrelating experience for 64 frames... | |
| [2024-09-06 08:35:52,299][01070] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 08:35:52,385][19118] Decorrelating experience for 0 frames... | |
| [2024-09-06 08:35:52,773][19116] Decorrelating experience for 32 frames... | |
| [2024-09-06 08:35:53,059][19117] Decorrelating experience for 64 frames... | |
| [2024-09-06 08:35:53,088][19115] Decorrelating experience for 64 frames... | |
| [2024-09-06 08:35:53,163][19111] Decorrelating experience for 64 frames... | |
| [2024-09-06 08:35:53,227][19114] Decorrelating experience for 32 frames... | |
| [2024-09-06 08:35:54,004][19116] Decorrelating experience for 64 frames... | |
| [2024-09-06 08:35:54,080][19111] Decorrelating experience for 96 frames... | |
| [2024-09-06 08:35:54,169][01070] Heartbeat connected on RolloutWorker_w1 | |
| [2024-09-06 08:35:54,232][19113] Decorrelating experience for 64 frames... | |
| [2024-09-06 08:35:54,438][19118] Decorrelating experience for 32 frames... | |
| [2024-09-06 08:35:54,884][19114] Decorrelating experience for 64 frames... | |
| [2024-09-06 08:35:55,752][19112] Decorrelating experience for 96 frames... | |
| [2024-09-06 08:35:55,972][01070] Heartbeat connected on RolloutWorker_w0 | |
| [2024-09-06 08:35:56,002][19113] Decorrelating experience for 96 frames... | |
| [2024-09-06 08:35:56,247][01070] Heartbeat connected on RolloutWorker_w2 | |
| [2024-09-06 08:35:56,489][19118] Decorrelating experience for 64 frames... | |
| [2024-09-06 08:35:56,773][19114] Decorrelating experience for 96 frames... | |
| [2024-09-06 08:35:57,299][01070] Heartbeat connected on RolloutWorker_w4 | |
| [2024-09-06 08:35:57,303][01070] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 4005888. Throughput: 0: 71.2. Samples: 712. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 08:35:57,307][01070] Avg episode reward: [(0, '5.702')] | |
| [2024-09-06 08:35:57,398][19116] Decorrelating experience for 96 frames... | |
| [2024-09-06 08:35:57,706][01070] Heartbeat connected on RolloutWorker_w5 | |
| [2024-09-06 08:35:57,841][19117] Decorrelating experience for 96 frames... | |
| [2024-09-06 08:35:58,244][01070] Heartbeat connected on RolloutWorker_w7 | |
| [2024-09-06 08:36:00,233][19093] Signal inference workers to stop experience collection... | |
| [2024-09-06 08:36:00,248][19110] InferenceWorker_p0-w0: stopping experience collection | |
| [2024-09-06 08:36:00,307][19118] Decorrelating experience for 96 frames... | |
| [2024-09-06 08:36:00,409][01070] Heartbeat connected on RolloutWorker_w6 | |
| [2024-09-06 08:36:00,811][19115] Decorrelating experience for 96 frames... | |
| [2024-09-06 08:36:00,904][01070] Heartbeat connected on RolloutWorker_w3 | |
| [2024-09-06 08:36:02,107][19093] Signal inference workers to resume experience collection... | |
| [2024-09-06 08:36:02,108][19110] InferenceWorker_p0-w0: resuming experience collection | |
| [2024-09-06 08:36:02,300][01070] Fps is (10 sec: 409.6, 60 sec: 273.0, 300 sec: 273.0). Total num frames: 4009984. Throughput: 0: 147.9. Samples: 2218. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2024-09-06 08:36:02,302][01070] Avg episode reward: [(0, '4.701')] | |
| [2024-09-06 08:36:07,302][01070] Fps is (10 sec: 1638.4, 60 sec: 819.0, 300 sec: 819.0). Total num frames: 4022272. Throughput: 0: 228.8. Samples: 4576. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2024-09-06 08:36:07,305][01070] Avg episode reward: [(0, '6.521')] | |
| [2024-09-06 08:36:12,073][19110] Updated weights for policy 0, policy_version 988 (0.0020) | |
| [2024-09-06 08:36:12,300][01070] Fps is (10 sec: 3686.1, 60 sec: 1638.3, 300 sec: 1638.3). Total num frames: 4046848. Throughput: 0: 391.6. Samples: 9790. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:36:12,306][01070] Avg episode reward: [(0, '11.236')] | |
| [2024-09-06 08:36:17,299][01070] Fps is (10 sec: 4507.3, 60 sec: 2048.0, 300 sec: 2048.0). Total num frames: 4067328. Throughput: 0: 445.7. Samples: 13370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:36:17,305][01070] Avg episode reward: [(0, '12.296')] | |
| [2024-09-06 08:36:22,299][01070] Fps is (10 sec: 3687.1, 60 sec: 2223.5, 300 sec: 2223.5). Total num frames: 4083712. Throughput: 0: 556.9. Samples: 19492. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:36:22,301][01070] Avg episode reward: [(0, '14.420')] | |
| [2024-09-06 08:36:22,439][19110] Updated weights for policy 0, policy_version 998 (0.0030) | |
| [2024-09-06 08:36:27,299][01070] Fps is (10 sec: 3686.3, 60 sec: 2457.6, 300 sec: 2457.6). Total num frames: 4104192. Throughput: 0: 602.8. Samples: 24114. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) | |
| [2024-09-06 08:36:27,301][01070] Avg episode reward: [(0, '16.055')] | |
| [2024-09-06 08:36:32,188][19110] Updated weights for policy 0, policy_version 1008 (0.0021) | |
| [2024-09-06 08:36:32,299][01070] Fps is (10 sec: 4505.6, 60 sec: 2730.7, 300 sec: 2730.7). Total num frames: 4128768. Throughput: 0: 615.9. Samples: 27716. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) | |
| [2024-09-06 08:36:32,305][01070] Avg episode reward: [(0, '18.102')] | |
| [2024-09-06 08:36:32,309][19093] Saving new best policy, reward=18.102! | |
| [2024-09-06 08:36:37,300][01070] Fps is (10 sec: 4505.2, 60 sec: 2867.1, 300 sec: 2867.1). Total num frames: 4149248. Throughput: 0: 773.5. Samples: 34808. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:36:37,305][01070] Avg episode reward: [(0, '19.063')] | |
| [2024-09-06 08:36:37,312][19093] Saving new best policy, reward=19.063! | |
| [2024-09-06 08:36:42,299][01070] Fps is (10 sec: 3276.7, 60 sec: 2829.9, 300 sec: 2829.9). Total num frames: 4161536. Throughput: 0: 852.4. Samples: 39066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:36:42,303][01070] Avg episode reward: [(0, '20.446')] | |
| [2024-09-06 08:36:42,306][19093] Saving new best policy, reward=20.446! | |
| [2024-09-06 08:36:43,965][19110] Updated weights for policy 0, policy_version 1018 (0.0034) | |
| [2024-09-06 08:36:47,299][01070] Fps is (10 sec: 3277.1, 60 sec: 2935.5, 300 sec: 2935.5). Total num frames: 4182016. Throughput: 0: 879.8. Samples: 41808. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:36:47,305][01070] Avg episode reward: [(0, '21.840')] | |
| [2024-09-06 08:36:47,316][19093] Saving new best policy, reward=21.840! | |
| [2024-09-06 08:36:52,299][01070] Fps is (10 sec: 4505.7, 60 sec: 3345.1, 300 sec: 3087.8). Total num frames: 4206592. Throughput: 0: 982.8. Samples: 48798. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) | |
| [2024-09-06 08:36:52,304][01070] Avg episode reward: [(0, '21.047')] | |
| [2024-09-06 08:36:52,709][19110] Updated weights for policy 0, policy_version 1028 (0.0017) | |
| [2024-09-06 08:36:57,304][01070] Fps is (10 sec: 4094.0, 60 sec: 3618.1, 300 sec: 3101.0). Total num frames: 4222976. Throughput: 0: 987.4. Samples: 54226. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:36:57,306][01070] Avg episode reward: [(0, '20.552')] | |
| [2024-09-06 08:37:02,299][01070] Fps is (10 sec: 2457.6, 60 sec: 3686.5, 300 sec: 3003.7). Total num frames: 4231168. Throughput: 0: 944.6. Samples: 55878. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:37:02,301][01070] Avg episode reward: [(0, '21.311')] | |
| [2024-09-06 08:37:07,058][19110] Updated weights for policy 0, policy_version 1038 (0.0038) | |
| [2024-09-06 08:37:07,299][01070] Fps is (10 sec: 2868.7, 60 sec: 3823.2, 300 sec: 3072.0). Total num frames: 4251648. Throughput: 0: 897.5. Samples: 59880. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:37:07,301][01070] Avg episode reward: [(0, '20.332')] | |
| [2024-09-06 08:37:12,299][01070] Fps is (10 sec: 4096.0, 60 sec: 3754.8, 300 sec: 3132.2). Total num frames: 4272128. Throughput: 0: 950.4. Samples: 66884. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:37:12,301][01070] Avg episode reward: [(0, '19.661')] | |
| [2024-09-06 08:37:17,299][01070] Fps is (10 sec: 3686.4, 60 sec: 3686.4, 300 sec: 3140.3). Total num frames: 4288512. Throughput: 0: 928.7. Samples: 69506. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:37:17,302][01070] Avg episode reward: [(0, '20.777')] | |
| [2024-09-06 08:37:18,168][19110] Updated weights for policy 0, policy_version 1048 (0.0027) | |
| [2024-09-06 08:37:22,299][01070] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3190.6). Total num frames: 4308992. Throughput: 0: 873.3. Samples: 74104. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:37:22,304][01070] Avg episode reward: [(0, '21.018')] | |
| [2024-09-06 08:37:27,299][01070] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3276.8). Total num frames: 4333568. Throughput: 0: 938.8. Samples: 81312. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:37:27,300][19110] Updated weights for policy 0, policy_version 1058 (0.0031) | |
| [2024-09-06 08:37:27,301][01070] Avg episode reward: [(0, '21.381')] | |
| [2024-09-06 08:37:27,314][19093] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001058_4333568.pth... | |
| [2024-09-06 08:37:27,438][19093] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000975_3993600.pth | |
| [2024-09-06 08:37:32,300][01070] Fps is (10 sec: 4095.5, 60 sec: 3686.3, 300 sec: 3276.8). Total num frames: 4349952. Throughput: 0: 954.6. Samples: 84768. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:37:32,306][01070] Avg episode reward: [(0, '20.498')] | |
| [2024-09-06 08:37:37,299][01070] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3276.8). Total num frames: 4366336. Throughput: 0: 899.8. Samples: 89288. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:37:37,302][01070] Avg episode reward: [(0, '20.356')] | |
| [2024-09-06 08:37:38,951][19110] Updated weights for policy 0, policy_version 1068 (0.0030) | |
| [2024-09-06 08:37:42,302][01070] Fps is (10 sec: 3685.8, 60 sec: 3754.5, 300 sec: 3312.3). Total num frames: 4386816. Throughput: 0: 918.0. Samples: 95532. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:37:42,306][01070] Avg episode reward: [(0, '18.966')] | |
| [2024-09-06 08:37:47,299][01070] Fps is (10 sec: 4505.6, 60 sec: 3823.0, 300 sec: 3379.2). Total num frames: 4411392. Throughput: 0: 960.9. Samples: 99118. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:37:47,303][01070] Avg episode reward: [(0, '19.345')] | |
| [2024-09-06 08:37:48,240][19110] Updated weights for policy 0, policy_version 1078 (0.0013) | |
| [2024-09-06 08:37:52,299][01070] Fps is (10 sec: 3687.5, 60 sec: 3618.1, 300 sec: 3342.3). Total num frames: 4423680. Throughput: 0: 989.1. Samples: 104388. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:37:52,301][01070] Avg episode reward: [(0, '19.443')] | |
| [2024-09-06 08:37:57,299][01070] Fps is (10 sec: 3686.4, 60 sec: 3755.0, 300 sec: 3402.8). Total num frames: 4448256. Throughput: 0: 957.7. Samples: 109982. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:37:57,301][01070] Avg episode reward: [(0, '20.099')] | |
| [2024-09-06 08:37:59,151][19110] Updated weights for policy 0, policy_version 1088 (0.0024) | |
| [2024-09-06 08:38:02,299][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3428.5). Total num frames: 4468736. Throughput: 0: 978.0. Samples: 113514. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:38:02,306][01070] Avg episode reward: [(0, '20.360')] | |
| [2024-09-06 08:38:07,299][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3423.1). Total num frames: 4485120. Throughput: 0: 1019.1. Samples: 119964. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:38:07,302][01070] Avg episode reward: [(0, '20.240')] | |
| [2024-09-06 08:38:10,312][19110] Updated weights for policy 0, policy_version 1098 (0.0022) | |
| [2024-09-06 08:38:12,299][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3418.0). Total num frames: 4501504. Throughput: 0: 953.8. Samples: 124232. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:38:12,304][01070] Avg episode reward: [(0, '19.557')] | |
| [2024-09-06 08:38:17,299][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3467.9). Total num frames: 4526080. Throughput: 0: 954.6. Samples: 127726. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:38:17,301][01070] Avg episode reward: [(0, '20.239')] | |
| [2024-09-06 08:38:19,439][19110] Updated weights for policy 0, policy_version 1108 (0.0041) | |
| [2024-09-06 08:38:22,301][01070] Fps is (10 sec: 4504.4, 60 sec: 3959.3, 300 sec: 3488.1). Total num frames: 4546560. Throughput: 0: 1011.7. Samples: 134818. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:38:22,304][01070] Avg episode reward: [(0, '20.506')] | |
| [2024-09-06 08:38:27,299][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3481.6). Total num frames: 4562944. Throughput: 0: 977.9. Samples: 139534. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:38:27,303][01070] Avg episode reward: [(0, '20.585')] | |
| [2024-09-06 08:38:31,030][19110] Updated weights for policy 0, policy_version 1118 (0.0046) | |
| [2024-09-06 08:38:32,299][01070] Fps is (10 sec: 3687.4, 60 sec: 3891.3, 300 sec: 3500.2). Total num frames: 4583424. Throughput: 0: 953.9. Samples: 142044. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 08:38:32,306][01070] Avg episode reward: [(0, '20.308')] | |
| [2024-09-06 08:38:37,299][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3541.8). Total num frames: 4608000. Throughput: 0: 993.6. Samples: 149102. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:38:37,304][01070] Avg episode reward: [(0, '21.252')] | |
| [2024-09-06 08:38:40,626][19110] Updated weights for policy 0, policy_version 1128 (0.0030) | |
| [2024-09-06 08:38:42,299][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.7, 300 sec: 3534.3). Total num frames: 4624384. Throughput: 0: 994.4. Samples: 154730. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:38:42,303][01070] Avg episode reward: [(0, '20.914')] | |
| [2024-09-06 08:38:47,299][01070] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3527.1). Total num frames: 4640768. Throughput: 0: 963.1. Samples: 156854. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:38:47,307][01070] Avg episode reward: [(0, '20.844')] | |
| [2024-09-06 08:38:51,335][19110] Updated weights for policy 0, policy_version 1138 (0.0028) | |
| [2024-09-06 08:38:52,299][01070] Fps is (10 sec: 4095.9, 60 sec: 4027.7, 300 sec: 3564.6). Total num frames: 4665344. Throughput: 0: 963.3. Samples: 163312. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:38:52,301][01070] Avg episode reward: [(0, '22.493')] | |
| [2024-09-06 08:38:52,304][19093] Saving new best policy, reward=22.493! | |
| [2024-09-06 08:38:57,299][01070] Fps is (10 sec: 4505.7, 60 sec: 3959.5, 300 sec: 3578.6). Total num frames: 4685824. Throughput: 0: 1017.3. Samples: 170010. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:38:57,305][01070] Avg episode reward: [(0, '21.806')] | |
| [2024-09-06 08:39:02,299][01070] Fps is (10 sec: 3276.9, 60 sec: 3822.9, 300 sec: 3549.9). Total num frames: 4698112. Throughput: 0: 985.0. Samples: 172050. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:39:02,300][01070] Avg episode reward: [(0, '22.885')] | |
| [2024-09-06 08:39:02,305][19093] Saving new best policy, reward=22.885! | |
| [2024-09-06 08:39:02,950][19110] Updated weights for policy 0, policy_version 1148 (0.0035) | |
| [2024-09-06 08:39:07,299][01070] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3563.5). Total num frames: 4718592. Throughput: 0: 948.3. Samples: 177488. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:39:07,301][01070] Avg episode reward: [(0, '23.190')] | |
| [2024-09-06 08:39:07,318][19093] Saving new best policy, reward=23.190! | |
| [2024-09-06 08:39:11,864][19110] Updated weights for policy 0, policy_version 1158 (0.0024) | |
| [2024-09-06 08:39:12,299][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3596.5). Total num frames: 4743168. Throughput: 0: 994.3. Samples: 184276. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 08:39:12,303][01070] Avg episode reward: [(0, '23.069')] | |
| [2024-09-06 08:39:17,299][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3588.9). Total num frames: 4759552. Throughput: 0: 1001.5. Samples: 187112. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 08:39:17,301][01070] Avg episode reward: [(0, '22.125')] | |
| [2024-09-06 08:39:22,299][01070] Fps is (10 sec: 3276.8, 60 sec: 3823.1, 300 sec: 3581.6). Total num frames: 4775936. Throughput: 0: 941.3. Samples: 191462. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 08:39:22,303][01070] Avg episode reward: [(0, '21.027')] | |
| [2024-09-06 08:39:23,456][19110] Updated weights for policy 0, policy_version 1168 (0.0022) | |
| [2024-09-06 08:39:27,299][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3611.9). Total num frames: 4800512. Throughput: 0: 973.6. Samples: 198540. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:39:27,301][01070] Avg episode reward: [(0, '19.180')] | |
| [2024-09-06 08:39:27,314][19093] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001172_4800512.pth... | |
| [2024-09-06 08:39:27,463][19093] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth | |
| [2024-09-06 08:39:32,300][01070] Fps is (10 sec: 4504.9, 60 sec: 3959.4, 300 sec: 3622.7). Total num frames: 4820992. Throughput: 0: 1000.5. Samples: 201878. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 08:39:32,303][01070] Avg episode reward: [(0, '20.069')] | |
| [2024-09-06 08:39:33,797][19110] Updated weights for policy 0, policy_version 1178 (0.0024) | |
| [2024-09-06 08:39:37,299][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3597.4). Total num frames: 4833280. Throughput: 0: 959.6. Samples: 206492. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 08:39:37,301][01070] Avg episode reward: [(0, '19.170')] | |
| [2024-09-06 08:39:42,299][01070] Fps is (10 sec: 3687.0, 60 sec: 3891.2, 300 sec: 3625.4). Total num frames: 4857856. Throughput: 0: 947.1. Samples: 212628. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 08:39:42,306][01070] Avg episode reward: [(0, '20.382')] | |
| [2024-09-06 08:39:43,732][19110] Updated weights for policy 0, policy_version 1188 (0.0024) | |
| [2024-09-06 08:39:47,299][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3635.2). Total num frames: 4878336. Throughput: 0: 980.8. Samples: 216188. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:39:47,313][01070] Avg episode reward: [(0, '21.641')] | |
| [2024-09-06 08:39:52,299][01070] Fps is (10 sec: 3686.4, 60 sec: 3823.0, 300 sec: 3627.9). Total num frames: 4894720. Throughput: 0: 984.4. Samples: 221786. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:39:52,301][01070] Avg episode reward: [(0, '21.803')] | |
| [2024-09-06 08:39:55,248][19110] Updated weights for policy 0, policy_version 1198 (0.0031) | |
| [2024-09-06 08:39:57,299][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3637.2). Total num frames: 4915200. Throughput: 0: 951.6. Samples: 227098. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:39:57,304][01070] Avg episode reward: [(0, '22.439')] | |
| [2024-09-06 08:40:02,299][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3662.3). Total num frames: 4939776. Throughput: 0: 968.4. Samples: 230692. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:40:02,305][01070] Avg episode reward: [(0, '23.387')] | |
| [2024-09-06 08:40:02,307][19093] Saving new best policy, reward=23.387! | |
| [2024-09-06 08:40:04,048][19110] Updated weights for policy 0, policy_version 1208 (0.0019) | |
| [2024-09-06 08:40:07,299][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3654.9). Total num frames: 4956160. Throughput: 0: 1015.8. Samples: 237174. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:40:07,304][01070] Avg episode reward: [(0, '25.478')] | |
| [2024-09-06 08:40:07,312][19093] Saving new best policy, reward=25.478! | |
| [2024-09-06 08:40:12,299][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3647.8). Total num frames: 4972544. Throughput: 0: 950.4. Samples: 241310. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 08:40:12,301][01070] Avg episode reward: [(0, '26.573')] | |
| [2024-09-06 08:40:12,304][19093] Saving new best policy, reward=26.573! | |
| [2024-09-06 08:40:15,918][19110] Updated weights for policy 0, policy_version 1218 (0.0034) | |
| [2024-09-06 08:40:17,299][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3656.1). Total num frames: 4993024. Throughput: 0: 947.8. Samples: 244528. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 08:40:17,301][01070] Avg episode reward: [(0, '26.655')] | |
| [2024-09-06 08:40:17,310][19093] Saving new best policy, reward=26.655! | |
| [2024-09-06 08:40:19,413][19093] Stopping Batcher_0... | |
| [2024-09-06 08:40:19,413][01070] Component Batcher_0 stopped! | |
| [2024-09-06 08:40:19,415][19093] Loop batcher_evt_loop terminating... | |
| [2024-09-06 08:40:19,420][19093] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth... | |
| [2024-09-06 08:40:19,470][19110] Weights refcount: 2 0 | |
| [2024-09-06 08:40:19,478][01070] Component InferenceWorker_p0-w0 stopped! | |
| [2024-09-06 08:40:19,481][19110] Stopping InferenceWorker_p0-w0... | |
| [2024-09-06 08:40:19,481][19110] Loop inference_proc0-0_evt_loop terminating... | |
| [2024-09-06 08:40:19,543][19093] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001058_4333568.pth | |
| [2024-09-06 08:40:19,557][19093] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth... | |
| [2024-09-06 08:40:19,761][01070] Component LearnerWorker_p0 stopped! | |
| [2024-09-06 08:40:19,761][19093] Stopping LearnerWorker_p0... | |
| [2024-09-06 08:40:19,766][19093] Loop learner_proc0_evt_loop terminating... | |
| [2024-09-06 08:40:19,774][01070] Component RolloutWorker_w3 stopped! | |
| [2024-09-06 08:40:19,777][19115] Stopping RolloutWorker_w3... | |
| [2024-09-06 08:40:19,779][19115] Loop rollout_proc3_evt_loop terminating... | |
| [2024-09-06 08:40:19,804][01070] Component RolloutWorker_w1 stopped! | |
| [2024-09-06 08:40:19,806][19111] Stopping RolloutWorker_w1... | |
| [2024-09-06 08:40:19,813][19111] Loop rollout_proc1_evt_loop terminating... | |
| [2024-09-06 08:40:19,874][01070] Component RolloutWorker_w7 stopped! | |
| [2024-09-06 08:40:19,880][19117] Stopping RolloutWorker_w7... | |
| [2024-09-06 08:40:19,888][19117] Loop rollout_proc7_evt_loop terminating... | |
| [2024-09-06 08:40:19,902][01070] Component RolloutWorker_w5 stopped! | |
| [2024-09-06 08:40:19,904][19116] Stopping RolloutWorker_w5... | |
| [2024-09-06 08:40:19,909][19116] Loop rollout_proc5_evt_loop terminating... | |
| [2024-09-06 08:40:20,003][19118] Stopping RolloutWorker_w6... | |
| [2024-09-06 08:40:20,003][19118] Loop rollout_proc6_evt_loop terminating... | |
| [2024-09-06 08:40:20,003][01070] Component RolloutWorker_w6 stopped! | |
| [2024-09-06 08:40:20,035][19113] Stopping RolloutWorker_w2... | |
| [2024-09-06 08:40:20,037][19113] Loop rollout_proc2_evt_loop terminating... | |
| [2024-09-06 08:40:20,035][01070] Component RolloutWorker_w2 stopped! | |
| [2024-09-06 08:40:20,058][19114] Stopping RolloutWorker_w4... | |
| [2024-09-06 08:40:20,057][01070] Component RolloutWorker_w4 stopped! | |
| [2024-09-06 08:40:20,062][19114] Loop rollout_proc4_evt_loop terminating... | |
| [2024-09-06 08:40:20,066][19112] Stopping RolloutWorker_w0... | |
| [2024-09-06 08:40:20,066][01070] Component RolloutWorker_w0 stopped! | |
| [2024-09-06 08:40:20,068][01070] Waiting for process learner_proc0 to stop... | |
| [2024-09-06 08:40:20,078][19112] Loop rollout_proc0_evt_loop terminating... | |
| [2024-09-06 08:40:21,230][01070] Waiting for process inference_proc0-0 to join... | |
| [2024-09-06 08:40:21,233][01070] Waiting for process rollout_proc0 to join... | |
| [2024-09-06 08:40:24,143][01070] Waiting for process rollout_proc1 to join... | |
| [2024-09-06 08:40:24,148][01070] Waiting for process rollout_proc2 to join... | |
| [2024-09-06 08:40:24,150][01070] Waiting for process rollout_proc3 to join... | |
| [2024-09-06 08:40:24,153][01070] Waiting for process rollout_proc4 to join... | |
| [2024-09-06 08:40:24,159][01070] Waiting for process rollout_proc5 to join... | |
| [2024-09-06 08:40:24,162][01070] Waiting for process rollout_proc6 to join... | |
| [2024-09-06 08:40:24,165][01070] Waiting for process rollout_proc7 to join... | |
| [2024-09-06 08:40:24,168][01070] Batcher 0 profile tree view: | |
| batching: 7.2186, releasing_batches: 0.0064 | |
| [2024-09-06 08:40:24,170][01070] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0054 | |
| wait_policy_total: 102.6532 | |
| update_model: 2.2548 | |
| weight_update: 0.0030 | |
| one_step: 0.0026 | |
| handle_policy_step: 154.4121 | |
| deserialize: 3.6770, stack: 0.8213, obs_to_device_normalize: 31.1067, forward: 82.7946, send_messages: 7.5294 | |
| prepare_outputs: 21.0675 | |
| to_cpu: 12.5076 | |
| [2024-09-06 08:40:24,171][01070] Learner 0 profile tree view: | |
| misc: 0.0012, prepare_batch: 5.2823 | |
| train: 21.8730 | |
| epoch_init: 0.0014, minibatch_init: 0.0016, losses_postprocess: 0.1603, kl_divergence: 0.2048, after_optimizer: 0.9110 | |
| calculate_losses: 8.7013 | |
| losses_init: 0.0013, forward_head: 0.6601, bptt_initial: 6.1101, tail: 0.3325, advantages_returns: 0.0894, losses: 0.9715 | |
| bptt: 0.4678 | |
| bptt_forward_core: 0.4488 | |
| update: 11.7699 | |
| clip: 0.2453 | |
| [2024-09-06 08:40:24,173][01070] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.0619, enqueue_policy_requests: 23.3356, env_step: 206.1955, overhead: 3.3833, complete_rollouts: 1.8563 | |
| save_policy_outputs: 5.2619 | |
| split_output_tensors: 2.1953 | |
| [2024-09-06 08:40:24,175][01070] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.0745, enqueue_policy_requests: 24.4494, env_step: 203.3563, overhead: 3.1667, complete_rollouts: 1.6788 | |
| save_policy_outputs: 4.9515 | |
| split_output_tensors: 2.0046 | |
| [2024-09-06 08:40:24,177][01070] Loop Runner_EvtLoop terminating... | |
| [2024-09-06 08:40:24,179][01070] Runner profile tree view: | |
| main_loop: 294.6944 | |
| [2024-09-06 08:40:24,180][01070] Collected {0: 5005312}, FPS: 3391.4 | |
| [2024-09-06 08:56:44,318][01070] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2024-09-06 08:56:44,320][01070] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-09-06 08:56:44,322][01070] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-09-06 08:56:44,324][01070] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-09-06 08:56:44,326][01070] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 08:56:44,327][01070] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-09-06 08:56:44,328][01070] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 08:56:44,329][01070] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-09-06 08:56:44,331][01070] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2024-09-06 08:56:44,332][01070] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2024-09-06 08:56:44,333][01070] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-09-06 08:56:44,334][01070] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-09-06 08:56:44,339][01070] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-09-06 08:56:44,340][01070] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-09-06 08:56:44,341][01070] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-09-06 08:56:44,368][01070] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 08:56:44,370][01070] RunningMeanStd input shape: (1,) | |
| [2024-09-06 08:56:44,390][01070] ConvEncoder: input_channels=3 | |
| [2024-09-06 08:56:44,435][01070] Conv encoder output size: 512 | |
| [2024-09-06 08:56:44,436][01070] Policy head output size: 512 | |
| [2024-09-06 08:56:44,457][01070] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth... | |
| [2024-09-06 08:56:44,883][01070] Num frames 100... | |
| [2024-09-06 08:56:45,006][01070] Num frames 200... | |
| [2024-09-06 08:56:45,128][01070] Num frames 300... | |
| [2024-09-06 08:56:45,247][01070] Num frames 400... | |
| [2024-09-06 08:56:45,367][01070] Num frames 500... | |
| [2024-09-06 08:56:45,497][01070] Num frames 600... | |
| [2024-09-06 08:56:45,618][01070] Num frames 700... | |
| [2024-09-06 08:56:45,741][01070] Num frames 800... | |
| [2024-09-06 08:56:45,860][01070] Num frames 900... | |
| [2024-09-06 08:56:45,978][01070] Num frames 1000... | |
| [2024-09-06 08:56:46,102][01070] Num frames 1100... | |
| [2024-09-06 08:56:46,224][01070] Num frames 1200... | |
| [2024-09-06 08:56:46,348][01070] Num frames 1300... | |
| [2024-09-06 08:56:46,479][01070] Num frames 1400... | |
| [2024-09-06 08:56:46,606][01070] Num frames 1500... | |
| [2024-09-06 08:56:46,726][01070] Num frames 1600... | |
| [2024-09-06 08:56:46,845][01070] Num frames 1700... | |
| [2024-09-06 08:56:46,967][01070] Num frames 1800... | |
| [2024-09-06 08:56:47,110][01070] Avg episode rewards: #0: 46.719, true rewards: #0: 18.720 | |
| [2024-09-06 08:56:47,113][01070] Avg episode reward: 46.719, avg true_objective: 18.720 | |
| [2024-09-06 08:56:47,150][01070] Num frames 1900... | |
| [2024-09-06 08:56:47,268][01070] Num frames 2000... | |
| [2024-09-06 08:56:47,388][01070] Num frames 2100... | |
| [2024-09-06 08:56:47,523][01070] Num frames 2200... | |
| [2024-09-06 08:56:47,671][01070] Num frames 2300... | |
| [2024-09-06 08:56:47,751][01070] Avg episode rewards: #0: 27.100, true rewards: #0: 11.600 | |
| [2024-09-06 08:56:47,752][01070] Avg episode reward: 27.100, avg true_objective: 11.600 | |
| [2024-09-06 08:56:47,854][01070] Num frames 2400... | |
| [2024-09-06 08:56:47,975][01070] Num frames 2500... | |
| [2024-09-06 08:56:48,094][01070] Num frames 2600... | |
| [2024-09-06 08:56:48,217][01070] Num frames 2700... | |
| [2024-09-06 08:56:48,342][01070] Num frames 2800... | |
| [2024-09-06 08:56:48,467][01070] Num frames 2900... | |
| [2024-09-06 08:56:48,602][01070] Num frames 3000... | |
| [2024-09-06 08:56:48,749][01070] Num frames 3100... | |
| [2024-09-06 08:56:48,919][01070] Num frames 3200... | |
| [2024-09-06 08:56:49,093][01070] Num frames 3300... | |
| [2024-09-06 08:56:49,273][01070] Avg episode rewards: #0: 26.253, true rewards: #0: 11.253 | |
| [2024-09-06 08:56:49,277][01070] Avg episode reward: 26.253, avg true_objective: 11.253 | |
| [2024-09-06 08:56:49,319][01070] Num frames 3400... | |
| [2024-09-06 08:56:49,496][01070] Num frames 3500... | |
| [2024-09-06 08:56:49,661][01070] Num frames 3600... | |
| [2024-09-06 08:56:49,822][01070] Num frames 3700... | |
| [2024-09-06 08:56:50,036][01070] Avg episode rewards: #0: 21.230, true rewards: #0: 9.480 | |
| [2024-09-06 08:56:50,038][01070] Avg episode reward: 21.230, avg true_objective: 9.480 | |
| [2024-09-06 08:56:50,057][01070] Num frames 3800... | |
| [2024-09-06 08:56:50,224][01070] Num frames 3900... | |
| [2024-09-06 08:56:50,397][01070] Num frames 4000... | |
| [2024-09-06 08:56:50,585][01070] Num frames 4100... | |
| [2024-09-06 08:56:50,755][01070] Num frames 4200... | |
| [2024-09-06 08:56:50,944][01070] Num frames 4300... | |
| [2024-09-06 08:56:51,125][01070] Num frames 4400... | |
| [2024-09-06 08:56:51,259][01070] Num frames 4500... | |
| [2024-09-06 08:56:51,382][01070] Num frames 4600... | |
| [2024-09-06 08:56:51,504][01070] Num frames 4700... | |
| [2024-09-06 08:56:51,623][01070] Num frames 4800... | |
| [2024-09-06 08:56:51,752][01070] Num frames 4900... | |
| [2024-09-06 08:56:51,870][01070] Num frames 5000... | |
| [2024-09-06 08:56:51,988][01070] Num frames 5100... | |
| [2024-09-06 08:56:52,111][01070] Num frames 5200... | |
| [2024-09-06 08:56:52,228][01070] Num frames 5300... | |
| [2024-09-06 08:56:52,393][01070] Avg episode rewards: #0: 24.784, true rewards: #0: 10.784 | |
| [2024-09-06 08:56:52,394][01070] Avg episode reward: 24.784, avg true_objective: 10.784 | |
| [2024-09-06 08:56:52,408][01070] Num frames 5400... | |
| [2024-09-06 08:56:52,534][01070] Num frames 5500... | |
| [2024-09-06 08:56:52,657][01070] Num frames 5600... | |
| [2024-09-06 08:56:52,788][01070] Num frames 5700... | |
| [2024-09-06 08:56:52,907][01070] Num frames 5800... | |
| [2024-09-06 08:56:53,028][01070] Num frames 5900... | |
| [2024-09-06 08:56:53,148][01070] Num frames 6000... | |
| [2024-09-06 08:56:53,268][01070] Num frames 6100... | |
| [2024-09-06 08:56:53,388][01070] Num frames 6200... | |
| [2024-09-06 08:56:53,515][01070] Num frames 6300... | |
| [2024-09-06 08:56:53,638][01070] Num frames 6400... | |
| [2024-09-06 08:56:53,779][01070] Num frames 6500... | |
| [2024-09-06 08:56:53,901][01070] Num frames 6600... | |
| [2024-09-06 08:56:54,025][01070] Num frames 6700... | |
| [2024-09-06 08:56:54,146][01070] Num frames 6800... | |
| [2024-09-06 08:56:54,268][01070] Num frames 6900... | |
| [2024-09-06 08:56:54,390][01070] Num frames 7000... | |
| [2024-09-06 08:56:54,560][01070] Avg episode rewards: #0: 28.313, true rewards: #0: 11.813 | |
| [2024-09-06 08:56:54,562][01070] Avg episode reward: 28.313, avg true_objective: 11.813 | |
| [2024-09-06 08:56:54,581][01070] Num frames 7100... | |
| [2024-09-06 08:56:54,699][01070] Num frames 7200... | |
| [2024-09-06 08:56:54,827][01070] Num frames 7300... | |
| [2024-09-06 08:56:54,945][01070] Num frames 7400... | |
| [2024-09-06 08:56:55,064][01070] Num frames 7500... | |
| [2024-09-06 08:56:55,188][01070] Num frames 7600... | |
| [2024-09-06 08:56:55,281][01070] Avg episode rewards: #0: 25.760, true rewards: #0: 10.903 | |
| [2024-09-06 08:56:55,283][01070] Avg episode reward: 25.760, avg true_objective: 10.903 | |
| [2024-09-06 08:56:55,366][01070] Num frames 7700... | |
| [2024-09-06 08:56:55,489][01070] Num frames 7800... | |
| [2024-09-06 08:56:55,612][01070] Num frames 7900... | |
| [2024-09-06 08:56:55,732][01070] Num frames 8000... | |
| [2024-09-06 08:56:55,892][01070] Avg episode rewards: #0: 23.600, true rewards: #0: 10.100 | |
| [2024-09-06 08:56:55,894][01070] Avg episode reward: 23.600, avg true_objective: 10.100 | |
| [2024-09-06 08:56:55,921][01070] Num frames 8100... | |
| [2024-09-06 08:56:56,039][01070] Num frames 8200... | |
| [2024-09-06 08:56:56,159][01070] Num frames 8300... | |
| [2024-09-06 08:56:56,280][01070] Num frames 8400... | |
| [2024-09-06 08:56:56,402][01070] Num frames 8500... | |
| [2024-09-06 08:56:56,532][01070] Num frames 8600... | |
| [2024-09-06 08:56:56,656][01070] Num frames 8700... | |
| [2024-09-06 08:56:56,782][01070] Num frames 8800... | |
| [2024-09-06 08:56:56,911][01070] Num frames 8900... | |
| [2024-09-06 08:56:57,036][01070] Num frames 9000... | |
| [2024-09-06 08:56:57,162][01070] Num frames 9100... | |
| [2024-09-06 08:56:57,286][01070] Num frames 9200... | |
| [2024-09-06 08:56:57,409][01070] Num frames 9300... | |
| [2024-09-06 08:56:57,498][01070] Avg episode rewards: #0: 24.253, true rewards: #0: 10.364 | |
| [2024-09-06 08:56:57,500][01070] Avg episode reward: 24.253, avg true_objective: 10.364 | |
| [2024-09-06 08:56:57,588][01070] Num frames 9400... | |
| [2024-09-06 08:56:57,710][01070] Num frames 9500... | |
| [2024-09-06 08:56:57,848][01070] Num frames 9600... | |
| [2024-09-06 08:56:57,973][01070] Num frames 9700... | |
| [2024-09-06 08:56:58,097][01070] Num frames 9800... | |
| [2024-09-06 08:56:58,209][01070] Avg episode rewards: #0: 22.746, true rewards: #0: 9.846 | |
| [2024-09-06 08:56:58,211][01070] Avg episode reward: 22.746, avg true_objective: 9.846 | |
| [2024-09-06 08:57:59,802][01070] Replay video saved to /content/train_dir/default_experiment/replay.mp4! | |
| [2024-09-06 09:00:55,778][01070] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2024-09-06 09:00:55,780][01070] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-09-06 09:00:55,782][01070] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-09-06 09:00:55,784][01070] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-09-06 09:00:55,786][01070] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 09:00:55,789][01070] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-09-06 09:00:55,791][01070] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2024-09-06 09:00:55,792][01070] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-09-06 09:00:55,793][01070] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2024-09-06 09:00:55,794][01070] Adding new argument 'hf_repository'='Re-Re/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2024-09-06 09:00:55,795][01070] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-09-06 09:00:55,796][01070] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-09-06 09:00:55,797][01070] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-09-06 09:00:55,798][01070] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-09-06 09:00:55,799][01070] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-09-06 09:00:55,830][01070] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 09:00:55,832][01070] RunningMeanStd input shape: (1,) | |
| [2024-09-06 09:00:55,846][01070] ConvEncoder: input_channels=3 | |
| [2024-09-06 09:00:55,882][01070] Conv encoder output size: 512 | |
| [2024-09-06 09:00:55,884][01070] Policy head output size: 512 | |
| [2024-09-06 09:00:55,904][01070] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth... | |
| [2024-09-06 09:00:56,320][01070] Num frames 100... | |
| [2024-09-06 09:00:56,441][01070] Num frames 200... | |
| [2024-09-06 09:00:56,597][01070] Num frames 300... | |
| [2024-09-06 09:00:56,715][01070] Num frames 400... | |
| [2024-09-06 09:00:56,840][01070] Num frames 500... | |
| [2024-09-06 09:00:56,960][01070] Num frames 600... | |
| [2024-09-06 09:00:57,080][01070] Num frames 700... | |
| [2024-09-06 09:00:57,182][01070] Avg episode rewards: #0: 16.380, true rewards: #0: 7.380 | |
| [2024-09-06 09:00:57,184][01070] Avg episode reward: 16.380, avg true_objective: 7.380 | |
| [2024-09-06 09:00:57,264][01070] Num frames 800... | |
| [2024-09-06 09:00:57,399][01070] Num frames 900... | |
| [2024-09-06 09:00:57,529][01070] Num frames 1000... | |
| [2024-09-06 09:00:57,650][01070] Num frames 1100... | |
| [2024-09-06 09:00:57,775][01070] Num frames 1200... | |
| [2024-09-06 09:00:57,898][01070] Num frames 1300... | |
| [2024-09-06 09:00:58,016][01070] Num frames 1400... | |
| [2024-09-06 09:00:58,136][01070] Num frames 1500... | |
| [2024-09-06 09:00:58,255][01070] Num frames 1600... | |
| [2024-09-06 09:00:58,314][01070] Avg episode rewards: #0: 18.010, true rewards: #0: 8.010 | |
| [2024-09-06 09:00:58,316][01070] Avg episode reward: 18.010, avg true_objective: 8.010 | |
| [2024-09-06 09:00:58,433][01070] Num frames 1700... | |
| [2024-09-06 09:00:58,562][01070] Num frames 1800... | |
| [2024-09-06 09:00:58,686][01070] Num frames 1900... | |
| [2024-09-06 09:00:58,804][01070] Num frames 2000... | |
| [2024-09-06 09:00:58,927][01070] Num frames 2100... | |
| [2024-09-06 09:00:59,047][01070] Num frames 2200... | |
| [2024-09-06 09:00:59,165][01070] Num frames 2300... | |
| [2024-09-06 09:00:59,285][01070] Num frames 2400... | |
| [2024-09-06 09:00:59,413][01070] Num frames 2500... | |
| [2024-09-06 09:00:59,544][01070] Num frames 2600... | |
| [2024-09-06 09:00:59,669][01070] Num frames 2700... | |
| [2024-09-06 09:00:59,789][01070] Num frames 2800... | |
| [2024-09-06 09:00:59,937][01070] Num frames 2900... | |
| [2024-09-06 09:01:00,112][01070] Num frames 3000... | |
| [2024-09-06 09:01:00,278][01070] Num frames 3100... | |
| [2024-09-06 09:01:00,453][01070] Num frames 3200... | |
| [2024-09-06 09:01:00,623][01070] Num frames 3300... | |
| [2024-09-06 09:01:00,786][01070] Num frames 3400... | |
| [2024-09-06 09:01:00,952][01070] Num frames 3500... | |
| [2024-09-06 09:01:01,119][01070] Num frames 3600... | |
| [2024-09-06 09:01:01,294][01070] Num frames 3700... | |
| [2024-09-06 09:01:01,356][01070] Avg episode rewards: #0: 30.673, true rewards: #0: 12.340 | |
| [2024-09-06 09:01:01,357][01070] Avg episode reward: 30.673, avg true_objective: 12.340 | |
| [2024-09-06 09:01:01,535][01070] Num frames 3800... | |
| [2024-09-06 09:01:01,711][01070] Num frames 3900... | |
| [2024-09-06 09:01:01,881][01070] Num frames 4000... | |
| [2024-09-06 09:01:02,049][01070] Num frames 4100... | |
| [2024-09-06 09:01:02,222][01070] Num frames 4200... | |
| [2024-09-06 09:01:02,396][01070] Num frames 4300... | |
| [2024-09-06 09:01:02,534][01070] Num frames 4400... | |
| [2024-09-06 09:01:02,655][01070] Num frames 4500... | |
| [2024-09-06 09:01:02,776][01070] Num frames 4600... | |
| [2024-09-06 09:01:02,898][01070] Num frames 4700... | |
| [2024-09-06 09:01:03,020][01070] Num frames 4800... | |
| [2024-09-06 09:01:03,140][01070] Num frames 4900... | |
| [2024-09-06 09:01:03,261][01070] Num frames 5000... | |
| [2024-09-06 09:01:03,383][01070] Num frames 5100... | |
| [2024-09-06 09:01:03,519][01070] Num frames 5200... | |
| [2024-09-06 09:01:03,641][01070] Num frames 5300... | |
| [2024-09-06 09:01:03,817][01070] Avg episode rewards: #0: 34.225, true rewards: #0: 13.475 | |
| [2024-09-06 09:01:03,818][01070] Avg episode reward: 34.225, avg true_objective: 13.475 | |
| [2024-09-06 09:01:03,835][01070] Num frames 5400... | |
| [2024-09-06 09:01:03,956][01070] Num frames 5500... | |
| [2024-09-06 09:01:04,074][01070] Num frames 5600... | |
| [2024-09-06 09:01:04,194][01070] Num frames 5700... | |
| [2024-09-06 09:01:04,314][01070] Num frames 5800... | |
| [2024-09-06 09:01:04,436][01070] Num frames 5900... | |
| [2024-09-06 09:01:04,574][01070] Num frames 6000... | |
| [2024-09-06 09:01:04,693][01070] Num frames 6100... | |
| [2024-09-06 09:01:04,810][01070] Num frames 6200... | |
| [2024-09-06 09:01:04,934][01070] Num frames 6300... | |
| [2024-09-06 09:01:05,057][01070] Num frames 6400... | |
| [2024-09-06 09:01:05,177][01070] Num frames 6500... | |
| [2024-09-06 09:01:05,296][01070] Num frames 6600... | |
| [2024-09-06 09:01:05,424][01070] Num frames 6700... | |
| [2024-09-06 09:01:05,562][01070] Num frames 6800... | |
| [2024-09-06 09:01:05,698][01070] Num frames 6900... | |
| [2024-09-06 09:01:05,816][01070] Num frames 7000... | |
| [2024-09-06 09:01:05,936][01070] Num frames 7100... | |
| [2024-09-06 09:01:06,057][01070] Num frames 7200... | |
| [2024-09-06 09:01:06,179][01070] Num frames 7300... | |
| [2024-09-06 09:01:06,303][01070] Num frames 7400... | |
| [2024-09-06 09:01:06,469][01070] Avg episode rewards: #0: 38.779, true rewards: #0: 14.980 | |
| [2024-09-06 09:01:06,474][01070] Avg episode reward: 38.779, avg true_objective: 14.980 | |
| [2024-09-06 09:01:06,493][01070] Num frames 7500... | |
| [2024-09-06 09:01:06,629][01070] Num frames 7600... | |
| [2024-09-06 09:01:06,753][01070] Num frames 7700... | |
| [2024-09-06 09:01:06,875][01070] Num frames 7800... | |
| [2024-09-06 09:01:06,995][01070] Num frames 7900... | |
| [2024-09-06 09:01:07,118][01070] Num frames 8000... | |
| [2024-09-06 09:01:07,240][01070] Num frames 8100... | |
| [2024-09-06 09:01:07,359][01070] Num frames 8200... | |
| [2024-09-06 09:01:07,476][01070] Avg episode rewards: #0: 34.920, true rewards: #0: 13.753 | |
| [2024-09-06 09:01:07,478][01070] Avg episode reward: 34.920, avg true_objective: 13.753 | |
| [2024-09-06 09:01:07,541][01070] Num frames 8300... | |
| [2024-09-06 09:01:07,670][01070] Num frames 8400... | |
| [2024-09-06 09:01:07,789][01070] Num frames 8500... | |
| [2024-09-06 09:01:07,912][01070] Num frames 8600... | |
| [2024-09-06 09:01:08,030][01070] Num frames 8700... | |
| [2024-09-06 09:01:08,150][01070] Num frames 8800... | |
| [2024-09-06 09:01:08,275][01070] Num frames 8900... | |
| [2024-09-06 09:01:08,400][01070] Avg episode rewards: #0: 31.794, true rewards: #0: 12.794 | |
| [2024-09-06 09:01:08,402][01070] Avg episode reward: 31.794, avg true_objective: 12.794 | |
| [2024-09-06 09:01:08,457][01070] Num frames 9000... | |
| [2024-09-06 09:01:08,587][01070] Num frames 9100... | |
| [2024-09-06 09:01:08,723][01070] Num frames 9200... | |
| [2024-09-06 09:01:08,851][01070] Num frames 9300... | |
| [2024-09-06 09:01:08,979][01070] Num frames 9400... | |
| [2024-09-06 09:01:09,106][01070] Num frames 9500... | |
| [2024-09-06 09:01:09,230][01070] Num frames 9600... | |
| [2024-09-06 09:01:09,356][01070] Num frames 9700... | |
| [2024-09-06 09:01:09,487][01070] Num frames 9800... | |
| [2024-09-06 09:01:09,614][01070] Num frames 9900... | |
| [2024-09-06 09:01:09,745][01070] Num frames 10000... | |
| [2024-09-06 09:01:09,870][01070] Num frames 10100... | |
| [2024-09-06 09:01:09,995][01070] Num frames 10200... | |
| [2024-09-06 09:01:10,118][01070] Num frames 10300... | |
| [2024-09-06 09:01:10,238][01070] Num frames 10400... | |
| [2024-09-06 09:01:10,360][01070] Num frames 10500... | |
| [2024-09-06 09:01:10,493][01070] Num frames 10600... | |
| [2024-09-06 09:01:10,614][01070] Num frames 10700... | |
| [2024-09-06 09:01:10,741][01070] Num frames 10800... | |
| [2024-09-06 09:01:10,866][01070] Num frames 10900... | |
| [2024-09-06 09:01:10,993][01070] Num frames 11000... | |
| [2024-09-06 09:01:11,116][01070] Avg episode rewards: #0: 34.820, true rewards: #0: 13.820 | |
| [2024-09-06 09:01:11,118][01070] Avg episode reward: 34.820, avg true_objective: 13.820 | |
| [2024-09-06 09:01:11,175][01070] Num frames 11100... | |
| [2024-09-06 09:01:11,294][01070] Num frames 11200... | |
| [2024-09-06 09:01:11,414][01070] Num frames 11300... | |
| [2024-09-06 09:01:11,549][01070] Num frames 11400... | |
| [2024-09-06 09:01:11,615][01070] Avg episode rewards: #0: 31.453, true rewards: #0: 12.676 | |
| [2024-09-06 09:01:11,617][01070] Avg episode reward: 31.453, avg true_objective: 12.676 | |
| [2024-09-06 09:01:11,740][01070] Num frames 11500... | |
| [2024-09-06 09:01:11,869][01070] Num frames 11600... | |
| [2024-09-06 09:01:11,989][01070] Num frames 11700... | |
| [2024-09-06 09:01:12,106][01070] Num frames 11800... | |
| [2024-09-06 09:01:12,231][01070] Avg episode rewards: #0: 29.156, true rewards: #0: 11.856 | |
| [2024-09-06 09:01:12,233][01070] Avg episode reward: 29.156, avg true_objective: 11.856 | |
| [2024-09-06 09:02:25,893][01070] Replay video saved to /content/train_dir/default_experiment/replay.mp4! | |
| [2024-09-06 09:02:31,093][01070] The model has been pushed to https://huggingface.co/Re-Re/rl_course_vizdoom_health_gathering_supreme | |
| [2024-09-06 09:06:01,283][01070] Environment doom_basic already registered, overwriting... | |
| [2024-09-06 09:06:01,285][01070] Environment doom_two_colors_easy already registered, overwriting... | |
| [2024-09-06 09:06:01,287][01070] Environment doom_two_colors_hard already registered, overwriting... | |
| [2024-09-06 09:06:01,290][01070] Environment doom_dm already registered, overwriting... | |
| [2024-09-06 09:06:01,293][01070] Environment doom_dwango5 already registered, overwriting... | |
| [2024-09-06 09:06:01,295][01070] Environment doom_my_way_home_flat_actions already registered, overwriting... | |
| [2024-09-06 09:06:01,296][01070] Environment doom_defend_the_center_flat_actions already registered, overwriting... | |
| [2024-09-06 09:06:01,298][01070] Environment doom_my_way_home already registered, overwriting... | |
| [2024-09-06 09:06:01,300][01070] Environment doom_deadly_corridor already registered, overwriting... | |
| [2024-09-06 09:06:01,302][01070] Environment doom_defend_the_center already registered, overwriting... | |
| [2024-09-06 09:06:01,304][01070] Environment doom_defend_the_line already registered, overwriting... | |
| [2024-09-06 09:06:01,306][01070] Environment doom_health_gathering already registered, overwriting... | |
| [2024-09-06 09:06:01,308][01070] Environment doom_health_gathering_supreme already registered, overwriting... | |
| [2024-09-06 09:06:01,310][01070] Environment doom_battle already registered, overwriting... | |
| [2024-09-06 09:06:01,313][01070] Environment doom_battle2 already registered, overwriting... | |
| [2024-09-06 09:06:01,315][01070] Environment doom_duel_bots already registered, overwriting... | |
| [2024-09-06 09:06:01,316][01070] Environment doom_deathmatch_bots already registered, overwriting... | |
| [2024-09-06 09:06:01,318][01070] Environment doom_duel already registered, overwriting... | |
| [2024-09-06 09:06:01,320][01070] Environment doom_deathmatch_full already registered, overwriting... | |
| [2024-09-06 09:06:01,322][01070] Environment doom_benchmark already registered, overwriting... | |
| [2024-09-06 09:06:01,323][01070] register_encoder_factory: <function make_vizdoom_encoder at 0x78dc5537e170> | |
| [2024-09-06 09:06:01,341][01070] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2024-09-06 09:06:01,342][01070] Overriding arg 'train_for_env_steps' with value 7500000 passed from command line | |
| [2024-09-06 09:06:01,349][01070] Experiment dir /content/train_dir/default_experiment already exists! | |
| [2024-09-06 09:06:01,350][01070] Resuming existing experiment from /content/train_dir/default_experiment... | |
| [2024-09-06 09:06:01,352][01070] Weights and Biases integration disabled | |
| [2024-09-06 09:06:01,355][01070] Environment var CUDA_VISIBLE_DEVICES is 0 | |
| [2024-09-06 09:06:03,516][01070] Starting experiment with the following configuration: | |
| help=False | |
| algo=APPO | |
| env=doom_health_gathering_supreme | |
| experiment=default_experiment | |
| train_dir=/content/train_dir | |
| restart_behavior=resume | |
| device=gpu | |
| seed=None | |
| num_policies=1 | |
| async_rl=True | |
| serial_mode=False | |
| batched_sampling=False | |
| num_batches_to_accumulate=2 | |
| worker_num_splits=2 | |
| policy_workers_per_policy=1 | |
| max_policy_lag=1000 | |
| num_workers=8 | |
| num_envs_per_worker=4 | |
| batch_size=1024 | |
| num_batches_per_epoch=1 | |
| num_epochs=1 | |
| rollout=32 | |
| recurrence=32 | |
| shuffle_minibatches=False | |
| gamma=0.99 | |
| reward_scale=1.0 | |
| reward_clip=1000.0 | |
| value_bootstrap=False | |
| normalize_returns=True | |
| exploration_loss_coeff=0.001 | |
| value_loss_coeff=0.5 | |
| kl_loss_coeff=0.0 | |
| exploration_loss=symmetric_kl | |
| gae_lambda=0.95 | |
| ppo_clip_ratio=0.1 | |
| ppo_clip_value=0.2 | |
| with_vtrace=False | |
| vtrace_rho=1.0 | |
| vtrace_c=1.0 | |
| optimizer=adam | |
| adam_eps=1e-06 | |
| adam_beta1=0.9 | |
| adam_beta2=0.999 | |
| max_grad_norm=4.0 | |
| learning_rate=0.0001 | |
| lr_schedule=constant | |
| lr_schedule_kl_threshold=0.008 | |
| lr_adaptive_min=1e-06 | |
| lr_adaptive_max=0.01 | |
| obs_subtract_mean=0.0 | |
| obs_scale=255.0 | |
| normalize_input=True | |
| normalize_input_keys=None | |
| decorrelate_experience_max_seconds=0 | |
| decorrelate_envs_on_one_worker=True | |
| actor_worker_gpus=[] | |
| set_workers_cpu_affinity=True | |
| force_envs_single_thread=False | |
| default_niceness=0 | |
| log_to_file=True | |
| experiment_summaries_interval=10 | |
| flush_summaries_interval=30 | |
| stats_avg=100 | |
| summaries_use_frameskip=True | |
| heartbeat_interval=20 | |
| heartbeat_reporting_interval=600 | |
| train_for_env_steps=7500000 | |
| train_for_seconds=10000000000 | |
| save_every_sec=120 | |
| keep_checkpoints=2 | |
| load_checkpoint_kind=latest | |
| save_milestones_sec=-1 | |
| save_best_every_sec=5 | |
| save_best_metric=reward | |
| save_best_after=100000 | |
| benchmark=False | |
| encoder_mlp_layers=[512, 512] | |
| encoder_conv_architecture=convnet_simple | |
| encoder_conv_mlp_layers=[512] | |
| use_rnn=True | |
| rnn_size=512 | |
| rnn_type=gru | |
| rnn_num_layers=1 | |
| decoder_mlp_layers=[] | |
| nonlinearity=elu | |
| policy_initialization=orthogonal | |
| policy_init_gain=1.0 | |
| actor_critic_share_weights=True | |
| adaptive_stddev=True | |
| continuous_tanh_scale=0.0 | |
| initial_stddev=1.0 | |
| use_env_info_cache=False | |
| env_gpu_actions=False | |
| env_gpu_observations=True | |
| env_frameskip=4 | |
| env_framestack=1 | |
| pixel_format=CHW | |
| use_record_episode_statistics=False | |
| with_wandb=False | |
| wandb_user=None | |
| wandb_project=sample_factory | |
| wandb_group=None | |
| wandb_job_type=SF | |
| wandb_tags=[] | |
| with_pbt=False | |
| pbt_mix_policies_in_one_env=True | |
| pbt_period_env_steps=5000000 | |
| pbt_start_mutation=20000000 | |
| pbt_replace_fraction=0.3 | |
| pbt_mutation_rate=0.15 | |
| pbt_replace_reward_gap=0.1 | |
| pbt_replace_reward_gap_absolute=1e-06 | |
| pbt_optimize_gamma=False | |
| pbt_target_objective=true_objective | |
| pbt_perturb_min=1.1 | |
| pbt_perturb_max=1.5 | |
| num_agents=-1 | |
| num_humans=0 | |
| num_bots=-1 | |
| start_bot_difficulty=None | |
| timelimit=None | |
| res_w=128 | |
| res_h=72 | |
| wide_aspect_ratio=False | |
| eval_env_frameskip=1 | |
| fps=35 | |
| command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 | |
| cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} | |
| git_hash=unknown | |
| git_repo_name=not a git repository | |
| [2024-09-06 09:06:03,519][01070] Saving configuration to /content/train_dir/default_experiment/config.json... | |
| [2024-09-06 09:06:03,524][01070] Rollout worker 0 uses device cpu | |
| [2024-09-06 09:06:03,526][01070] Rollout worker 1 uses device cpu | |
| [2024-09-06 09:06:03,527][01070] Rollout worker 2 uses device cpu | |
| [2024-09-06 09:06:03,529][01070] Rollout worker 3 uses device cpu | |
| [2024-09-06 09:06:03,530][01070] Rollout worker 4 uses device cpu | |
| [2024-09-06 09:06:03,531][01070] Rollout worker 5 uses device cpu | |
| [2024-09-06 09:06:03,532][01070] Rollout worker 6 uses device cpu | |
| [2024-09-06 09:06:03,534][01070] Rollout worker 7 uses device cpu | |
| [2024-09-06 09:06:03,608][01070] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 09:06:03,610][01070] InferenceWorker_p0-w0: min num requests: 2 | |
| [2024-09-06 09:06:03,643][01070] Starting all processes... | |
| [2024-09-06 09:06:03,645][01070] Starting process learner_proc0 | |
| [2024-09-06 09:06:03,693][01070] Starting all processes... | |
| [2024-09-06 09:06:03,699][01070] Starting process inference_proc0-0 | |
| [2024-09-06 09:06:03,700][01070] Starting process rollout_proc0 | |
| [2024-09-06 09:06:03,701][01070] Starting process rollout_proc1 | |
| [2024-09-06 09:06:03,701][01070] Starting process rollout_proc2 | |
| [2024-09-06 09:06:03,702][01070] Starting process rollout_proc3 | |
| [2024-09-06 09:06:03,702][01070] Starting process rollout_proc4 | |
| [2024-09-06 09:06:03,702][01070] Starting process rollout_proc5 | |
| [2024-09-06 09:06:03,702][01070] Starting process rollout_proc6 | |
| [2024-09-06 09:06:03,702][01070] Starting process rollout_proc7 | |
| [2024-09-06 09:06:20,401][26905] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 09:06:20,401][26905] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2024-09-06 09:06:20,451][26905] Num visible devices: 1 | |
| [2024-09-06 09:06:20,487][26905] Starting seed is not provided | |
| [2024-09-06 09:06:20,488][26905] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 09:06:20,489][26905] Initializing actor-critic model on device cuda:0 | |
| [2024-09-06 09:06:20,489][26905] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 09:06:20,491][26905] RunningMeanStd input shape: (1,) | |
| [2024-09-06 09:06:20,545][26905] ConvEncoder: input_channels=3 | |
| [2024-09-06 09:06:20,881][26918] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 09:06:20,882][26918] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2024-09-06 09:06:20,909][26926] Worker 7 uses CPU cores [1] | |
| [2024-09-06 09:06:20,990][26918] Num visible devices: 1 | |
| [2024-09-06 09:06:21,196][26923] Worker 4 uses CPU cores [0] | |
| [2024-09-06 09:06:21,222][26924] Worker 5 uses CPU cores [1] | |
| [2024-09-06 09:06:21,235][26919] Worker 0 uses CPU cores [0] | |
| [2024-09-06 09:06:21,331][26920] Worker 1 uses CPU cores [1] | |
| [2024-09-06 09:06:21,331][26921] Worker 2 uses CPU cores [0] | |
| [2024-09-06 09:06:21,344][26905] Conv encoder output size: 512 | |
| [2024-09-06 09:06:21,345][26905] Policy head output size: 512 | |
| [2024-09-06 09:06:21,393][26922] Worker 3 uses CPU cores [1] | |
| [2024-09-06 09:06:21,395][26905] Created Actor Critic model with architecture: | |
| [2024-09-06 09:06:21,395][26905] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2024-09-06 09:06:21,429][26925] Worker 6 uses CPU cores [0] | |
| [2024-09-06 09:06:21,538][26905] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2024-09-06 09:06:22,136][26905] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth... | |
| [2024-09-06 09:06:22,176][26905] Loading model from checkpoint | |
| [2024-09-06 09:06:22,178][26905] Loaded experiment state at self.train_step=1222, self.env_steps=5005312 | |
| [2024-09-06 09:06:22,178][26905] Initialized policy 0 weights for model version 1222 | |
| [2024-09-06 09:06:22,183][26905] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2024-09-06 09:06:22,190][26905] LearnerWorker_p0 finished initialization! | |
| [2024-09-06 09:06:22,275][26918] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 09:06:22,276][26918] RunningMeanStd input shape: (1,) | |
| [2024-09-06 09:06:22,288][26918] ConvEncoder: input_channels=3 | |
| [2024-09-06 09:06:22,391][26918] Conv encoder output size: 512 | |
| [2024-09-06 09:06:22,391][26918] Policy head output size: 512 | |
| [2024-09-06 09:06:22,443][01070] Inference worker 0-0 is ready! | |
| [2024-09-06 09:06:22,445][01070] All inference workers are ready! Signal rollout workers to start! | |
| [2024-09-06 09:06:22,677][26920] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 09:06:22,689][26922] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 09:06:22,692][26924] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 09:06:22,683][26926] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 09:06:22,720][26919] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 09:06:22,725][26925] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 09:06:22,727][26921] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 09:06:22,715][26923] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2024-09-06 09:06:23,532][26919] Decorrelating experience for 0 frames... | |
| [2024-09-06 09:06:23,600][01070] Heartbeat connected on Batcher_0 | |
| [2024-09-06 09:06:23,605][01070] Heartbeat connected on LearnerWorker_p0 | |
| [2024-09-06 09:06:23,634][01070] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2024-09-06 09:06:23,927][26923] Decorrelating experience for 0 frames... | |
| [2024-09-06 09:06:24,411][26922] Decorrelating experience for 0 frames... | |
| [2024-09-06 09:06:24,421][26924] Decorrelating experience for 0 frames... | |
| [2024-09-06 09:06:24,414][26920] Decorrelating experience for 0 frames... | |
| [2024-09-06 09:06:24,428][26926] Decorrelating experience for 0 frames... | |
| [2024-09-06 09:06:26,027][26919] Decorrelating experience for 32 frames... | |
| [2024-09-06 09:06:26,059][26921] Decorrelating experience for 0 frames... | |
| [2024-09-06 09:06:26,129][26922] Decorrelating experience for 32 frames... | |
| [2024-09-06 09:06:26,130][26920] Decorrelating experience for 32 frames... | |
| [2024-09-06 09:06:26,135][26924] Decorrelating experience for 32 frames... | |
| [2024-09-06 09:06:26,142][26926] Decorrelating experience for 32 frames... | |
| [2024-09-06 09:06:26,359][01070] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 5005312. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 09:06:27,542][26921] Decorrelating experience for 32 frames... | |
| [2024-09-06 09:06:27,689][26923] Decorrelating experience for 32 frames... | |
| [2024-09-06 09:06:27,841][26920] Decorrelating experience for 64 frames... | |
| [2024-09-06 09:06:28,170][26919] Decorrelating experience for 64 frames... | |
| [2024-09-06 09:06:29,208][26925] Decorrelating experience for 0 frames... | |
| [2024-09-06 09:06:29,215][26922] Decorrelating experience for 64 frames... | |
| [2024-09-06 09:06:29,356][26921] Decorrelating experience for 64 frames... | |
| [2024-09-06 09:06:29,802][26924] Decorrelating experience for 64 frames... | |
| [2024-09-06 09:06:29,918][26920] Decorrelating experience for 96 frames... | |
| [2024-09-06 09:06:30,160][01070] Heartbeat connected on RolloutWorker_w1 | |
| [2024-09-06 09:06:30,247][26919] Decorrelating experience for 96 frames... | |
| [2024-09-06 09:06:30,413][01070] Heartbeat connected on RolloutWorker_w0 | |
| [2024-09-06 09:06:30,668][26921] Decorrelating experience for 96 frames... | |
| [2024-09-06 09:06:30,864][01070] Heartbeat connected on RolloutWorker_w2 | |
| [2024-09-06 09:06:30,935][26926] Decorrelating experience for 64 frames... | |
| [2024-09-06 09:06:31,356][01070] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 5005312. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 09:06:31,362][01070] Avg episode reward: [(0, '0.380')] | |
| [2024-09-06 09:06:31,506][26924] Decorrelating experience for 96 frames... | |
| [2024-09-06 09:06:31,519][26923] Decorrelating experience for 64 frames... | |
| [2024-09-06 09:06:31,875][01070] Heartbeat connected on RolloutWorker_w5 | |
| [2024-09-06 09:06:33,162][26926] Decorrelating experience for 96 frames... | |
| [2024-09-06 09:06:33,550][01070] Heartbeat connected on RolloutWorker_w7 | |
| [2024-09-06 09:06:33,915][26922] Decorrelating experience for 96 frames... | |
| [2024-09-06 09:06:34,378][01070] Heartbeat connected on RolloutWorker_w3 | |
| [2024-09-06 09:06:34,636][26905] Signal inference workers to stop experience collection... | |
| [2024-09-06 09:06:34,665][26918] InferenceWorker_p0-w0: stopping experience collection | |
| [2024-09-06 09:06:34,723][26923] Decorrelating experience for 96 frames... | |
| [2024-09-06 09:06:34,840][01070] Heartbeat connected on RolloutWorker_w4 | |
| [2024-09-06 09:06:35,103][26925] Decorrelating experience for 32 frames... | |
| [2024-09-06 09:06:35,634][26925] Decorrelating experience for 64 frames... | |
| [2024-09-06 09:06:36,059][26925] Decorrelating experience for 96 frames... | |
| [2024-09-06 09:06:36,142][01070] Heartbeat connected on RolloutWorker_w6 | |
| [2024-09-06 09:06:36,355][01070] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 5005312. Throughput: 0: 218.7. Samples: 2186. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2024-09-06 09:06:36,358][01070] Avg episode reward: [(0, '5.207')] | |
| [2024-09-06 09:06:37,319][26905] Signal inference workers to resume experience collection... | |
| [2024-09-06 09:06:37,321][26918] InferenceWorker_p0-w0: resuming experience collection | |
| [2024-09-06 09:06:41,356][01070] Fps is (10 sec: 2048.0, 60 sec: 1365.6, 300 sec: 1365.6). Total num frames: 5025792. Throughput: 0: 364.1. Samples: 5460. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:06:41,358][01070] Avg episode reward: [(0, '6.203')] | |
| [2024-09-06 09:06:46,356][01070] Fps is (10 sec: 3276.8, 60 sec: 1638.6, 300 sec: 1638.6). Total num frames: 5038080. Throughput: 0: 379.1. Samples: 7580. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 09:06:46,366][01070] Avg episode reward: [(0, '11.223')] | |
| [2024-09-06 09:06:47,757][26918] Updated weights for policy 0, policy_version 1232 (0.0226) | |
| [2024-09-06 09:06:51,356][01070] Fps is (10 sec: 3686.3, 60 sec: 2294.0, 300 sec: 2294.0). Total num frames: 5062656. Throughput: 0: 512.5. Samples: 12812. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:06:51,362][01070] Avg episode reward: [(0, '17.027')] | |
| [2024-09-06 09:06:56,356][01070] Fps is (10 sec: 4096.0, 60 sec: 2457.8, 300 sec: 2457.8). Total num frames: 5079040. Throughput: 0: 637.0. Samples: 19108. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:06:56,358][01070] Avg episode reward: [(0, '18.689')] | |
| [2024-09-06 09:06:57,557][26918] Updated weights for policy 0, policy_version 1242 (0.0039) | |
| [2024-09-06 09:07:01,360][01070] Fps is (10 sec: 3275.6, 60 sec: 2574.6, 300 sec: 2574.6). Total num frames: 5095424. Throughput: 0: 621.9. Samples: 21768. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:07:01,363][01070] Avg episode reward: [(0, '20.324')] | |
| [2024-09-06 09:07:06,356][01070] Fps is (10 sec: 3686.4, 60 sec: 2765.0, 300 sec: 2765.0). Total num frames: 5115904. Throughput: 0: 649.8. Samples: 25990. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:07:06,357][01070] Avg episode reward: [(0, '21.032')] | |
| [2024-09-06 09:07:08,828][26918] Updated weights for policy 0, policy_version 1252 (0.0020) | |
| [2024-09-06 09:07:11,356][01070] Fps is (10 sec: 4097.5, 60 sec: 2912.9, 300 sec: 2912.9). Total num frames: 5136384. Throughput: 0: 737.4. Samples: 33182. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:07:11,364][01070] Avg episode reward: [(0, '25.508')] | |
| [2024-09-06 09:07:16,356][01070] Fps is (10 sec: 4095.7, 60 sec: 3031.2, 300 sec: 3031.2). Total num frames: 5156864. Throughput: 0: 816.6. Samples: 36746. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:07:16,362][01070] Avg episode reward: [(0, '26.378')] | |
| [2024-09-06 09:07:19,470][26918] Updated weights for policy 0, policy_version 1262 (0.0038) | |
| [2024-09-06 09:07:21,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3053.5, 300 sec: 3053.5). Total num frames: 5173248. Throughput: 0: 874.2. Samples: 41524. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:07:21,358][01070] Avg episode reward: [(0, '25.084')] | |
| [2024-09-06 09:07:26,356][01070] Fps is (10 sec: 3686.6, 60 sec: 3140.4, 300 sec: 3140.4). Total num frames: 5193728. Throughput: 0: 937.4. Samples: 47642. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:07:26,358][01070] Avg episode reward: [(0, '24.867')] | |
| [2024-09-06 09:07:28,996][26918] Updated weights for policy 0, policy_version 1272 (0.0035) | |
| [2024-09-06 09:07:31,356][01070] Fps is (10 sec: 4505.7, 60 sec: 3549.9, 300 sec: 3277.0). Total num frames: 5218304. Throughput: 0: 967.2. Samples: 51102. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:07:31,358][01070] Avg episode reward: [(0, '24.314')] | |
| [2024-09-06 09:07:36,356][01070] Fps is (10 sec: 4096.1, 60 sec: 3822.9, 300 sec: 3276.9). Total num frames: 5234688. Throughput: 0: 981.9. Samples: 56998. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:07:36,360][01070] Avg episode reward: [(0, '22.614')] | |
| [2024-09-06 09:07:40,320][26918] Updated weights for policy 0, policy_version 1282 (0.0028) | |
| [2024-09-06 09:07:41,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3331.5). Total num frames: 5255168. Throughput: 0: 955.6. Samples: 62112. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:07:41,358][01070] Avg episode reward: [(0, '21.784')] | |
| [2024-09-06 09:07:46,356][01070] Fps is (10 sec: 4505.5, 60 sec: 4027.7, 300 sec: 3430.5). Total num frames: 5279744. Throughput: 0: 978.5. Samples: 65798. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:07:46,358][01070] Avg episode reward: [(0, '22.907')] | |
| [2024-09-06 09:07:48,978][26918] Updated weights for policy 0, policy_version 1292 (0.0023) | |
| [2024-09-06 09:07:51,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3421.5). Total num frames: 5296128. Throughput: 0: 1038.5. Samples: 72724. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 09:07:51,363][01070] Avg episode reward: [(0, '23.940')] | |
| [2024-09-06 09:07:56,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3413.4). Total num frames: 5312512. Throughput: 0: 974.1. Samples: 77018. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:07:56,358][01070] Avg episode reward: [(0, '23.407')] | |
| [2024-09-06 09:08:00,195][26918] Updated weights for policy 0, policy_version 1302 (0.0029) | |
| [2024-09-06 09:08:01,356][01070] Fps is (10 sec: 4096.0, 60 sec: 4028.0, 300 sec: 3492.5). Total num frames: 5337088. Throughput: 0: 964.8. Samples: 80160. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 09:08:01,362][01070] Avg episode reward: [(0, '23.887')] | |
| [2024-09-06 09:08:01,369][26905] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001303_5337088.pth... | |
| [2024-09-06 09:08:01,513][26905] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001172_4800512.pth | |
| [2024-09-06 09:08:06,356][01070] Fps is (10 sec: 4915.2, 60 sec: 4096.0, 300 sec: 3563.6). Total num frames: 5361664. Throughput: 0: 1017.4. Samples: 87308. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:08:06,363][01070] Avg episode reward: [(0, '25.013')] | |
| [2024-09-06 09:08:10,733][26918] Updated weights for policy 0, policy_version 1312 (0.0019) | |
| [2024-09-06 09:08:11,358][01070] Fps is (10 sec: 3685.5, 60 sec: 3959.3, 300 sec: 3510.9). Total num frames: 5373952. Throughput: 0: 993.7. Samples: 92362. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2024-09-06 09:08:11,364][01070] Avg episode reward: [(0, '24.217')] | |
| [2024-09-06 09:08:16,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3537.5). Total num frames: 5394432. Throughput: 0: 965.2. Samples: 94536. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:08:16,363][01070] Avg episode reward: [(0, '23.685')] | |
| [2024-09-06 09:08:20,626][26918] Updated weights for policy 0, policy_version 1322 (0.0015) | |
| [2024-09-06 09:08:21,356][01070] Fps is (10 sec: 4096.9, 60 sec: 4027.8, 300 sec: 3561.8). Total num frames: 5414912. Throughput: 0: 991.2. Samples: 101604. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:08:21,365][01070] Avg episode reward: [(0, '23.700')] | |
| [2024-09-06 09:08:26,356][01070] Fps is (10 sec: 4095.9, 60 sec: 4027.7, 300 sec: 3584.1). Total num frames: 5435392. Throughput: 0: 1018.9. Samples: 107962. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:08:26,358][01070] Avg episode reward: [(0, '22.939')] | |
| [2024-09-06 09:08:31,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3571.8). Total num frames: 5451776. Throughput: 0: 982.6. Samples: 110016. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:08:31,362][01070] Avg episode reward: [(0, '21.481')] | |
| [2024-09-06 09:08:32,058][26918] Updated weights for policy 0, policy_version 1332 (0.0022) | |
| [2024-09-06 09:08:36,356][01070] Fps is (10 sec: 4096.1, 60 sec: 4027.7, 300 sec: 3623.5). Total num frames: 5476352. Throughput: 0: 964.4. Samples: 116120. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:08:36,358][01070] Avg episode reward: [(0, '21.845')] | |
| [2024-09-06 09:08:40,559][26918] Updated weights for policy 0, policy_version 1342 (0.0036) | |
| [2024-09-06 09:08:41,357][01070] Fps is (10 sec: 4504.8, 60 sec: 4027.6, 300 sec: 3640.9). Total num frames: 5496832. Throughput: 0: 1029.1. Samples: 123328. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:08:41,362][01070] Avg episode reward: [(0, '22.820')] | |
| [2024-09-06 09:08:46,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3628.0). Total num frames: 5513216. Throughput: 0: 1012.0. Samples: 125700. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:08:46,358][01070] Avg episode reward: [(0, '24.266')] | |
| [2024-09-06 09:08:51,356][01070] Fps is (10 sec: 3687.1, 60 sec: 3959.5, 300 sec: 3644.1). Total num frames: 5533696. Throughput: 0: 961.3. Samples: 130566. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 09:08:51,359][01070] Avg episode reward: [(0, '26.457')] | |
| [2024-09-06 09:08:52,046][26918] Updated weights for policy 0, policy_version 1352 (0.0038) | |
| [2024-09-06 09:08:56,355][01070] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3686.5). Total num frames: 5558272. Throughput: 0: 1008.9. Samples: 137758. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:08:56,363][01070] Avg episode reward: [(0, '26.386')] | |
| [2024-09-06 09:09:01,358][01070] Fps is (10 sec: 4095.1, 60 sec: 3959.3, 300 sec: 3673.2). Total num frames: 5574656. Throughput: 0: 1032.2. Samples: 140988. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:09:01,363][01070] Avg episode reward: [(0, '26.784')] | |
| [2024-09-06 09:09:01,373][26905] Saving new best policy, reward=26.784! | |
| [2024-09-06 09:09:02,724][26918] Updated weights for policy 0, policy_version 1362 (0.0027) | |
| [2024-09-06 09:09:06,356][01070] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3635.3). Total num frames: 5586944. Throughput: 0: 968.6. Samples: 145192. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:09:06,359][01070] Avg episode reward: [(0, '26.871')] | |
| [2024-09-06 09:09:06,422][26905] Saving new best policy, reward=26.871! | |
| [2024-09-06 09:09:11,356][01070] Fps is (10 sec: 3687.2, 60 sec: 3959.6, 300 sec: 3674.1). Total num frames: 5611520. Throughput: 0: 970.1. Samples: 151618. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:09:11,362][01070] Avg episode reward: [(0, '26.221')] | |
| [2024-09-06 09:09:12,605][26918] Updated weights for policy 0, policy_version 1372 (0.0024) | |
| [2024-09-06 09:09:16,361][01070] Fps is (10 sec: 4912.6, 60 sec: 4027.4, 300 sec: 3710.4). Total num frames: 5636096. Throughput: 0: 1003.0. Samples: 155156. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:09:16,368][01070] Avg episode reward: [(0, '25.283')] | |
| [2024-09-06 09:09:21,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3674.8). Total num frames: 5648384. Throughput: 0: 987.0. Samples: 160534. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:09:21,361][01070] Avg episode reward: [(0, '25.122')] | |
| [2024-09-06 09:09:24,141][26918] Updated weights for policy 0, policy_version 1382 (0.0031) | |
| [2024-09-06 09:09:26,356][01070] Fps is (10 sec: 3278.5, 60 sec: 3891.2, 300 sec: 3686.5). Total num frames: 5668864. Throughput: 0: 951.7. Samples: 166152. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 09:09:26,358][01070] Avg episode reward: [(0, '25.810')] | |
| [2024-09-06 09:09:31,356][01070] Fps is (10 sec: 4505.6, 60 sec: 4027.7, 300 sec: 3719.7). Total num frames: 5693440. Throughput: 0: 974.4. Samples: 169550. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:09:31,362][01070] Avg episode reward: [(0, '24.912')] | |
| [2024-09-06 09:09:32,863][26918] Updated weights for policy 0, policy_version 1392 (0.0021) | |
| [2024-09-06 09:09:36,357][01070] Fps is (10 sec: 4095.7, 60 sec: 3891.1, 300 sec: 3708.0). Total num frames: 5709824. Throughput: 0: 1008.7. Samples: 175958. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:09:36,359][01070] Avg episode reward: [(0, '25.858')] | |
| [2024-09-06 09:09:41,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3823.1, 300 sec: 3697.0). Total num frames: 5726208. Throughput: 0: 948.8. Samples: 180456. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:09:41,361][01070] Avg episode reward: [(0, '25.156')] | |
| [2024-09-06 09:09:44,121][26918] Updated weights for policy 0, policy_version 1402 (0.0019) | |
| [2024-09-06 09:09:46,356][01070] Fps is (10 sec: 4096.4, 60 sec: 3959.4, 300 sec: 3727.4). Total num frames: 5750784. Throughput: 0: 955.1. Samples: 183966. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 09:09:46,362][01070] Avg episode reward: [(0, '24.774')] | |
| [2024-09-06 09:09:51,356][01070] Fps is (10 sec: 4505.5, 60 sec: 3959.4, 300 sec: 3736.4). Total num frames: 5771264. Throughput: 0: 1022.3. Samples: 191196. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 09:09:51,358][01070] Avg episode reward: [(0, '25.942')] | |
| [2024-09-06 09:09:54,349][26918] Updated weights for policy 0, policy_version 1412 (0.0027) | |
| [2024-09-06 09:09:56,356][01070] Fps is (10 sec: 3686.2, 60 sec: 3822.9, 300 sec: 3725.4). Total num frames: 5787648. Throughput: 0: 982.8. Samples: 195844. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:09:56,359][01070] Avg episode reward: [(0, '25.874')] | |
| [2024-09-06 09:10:01,356][01070] Fps is (10 sec: 2867.3, 60 sec: 3754.8, 300 sec: 3696.0). Total num frames: 5799936. Throughput: 0: 943.3. Samples: 197598. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:10:01,363][01070] Avg episode reward: [(0, '26.130')] | |
| [2024-09-06 09:10:01,378][26905] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001416_5799936.pth... | |
| [2024-09-06 09:10:01,573][26905] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth | |
| [2024-09-06 09:10:06,356][01070] Fps is (10 sec: 2867.4, 60 sec: 3822.9, 300 sec: 3686.5). Total num frames: 5816320. Throughput: 0: 924.6. Samples: 202142. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:10:06,357][01070] Avg episode reward: [(0, '25.358')] | |
| [2024-09-06 09:10:07,253][26918] Updated weights for policy 0, policy_version 1422 (0.0021) | |
| [2024-09-06 09:10:11,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3695.6). Total num frames: 5836800. Throughput: 0: 948.5. Samples: 208836. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:10:11,366][01070] Avg episode reward: [(0, '26.176')] | |
| [2024-09-06 09:10:16,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3618.5, 300 sec: 3686.4). Total num frames: 5853184. Throughput: 0: 920.9. Samples: 210992. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:10:16,358][01070] Avg episode reward: [(0, '25.004')] | |
| [2024-09-06 09:10:18,768][26918] Updated weights for policy 0, policy_version 1432 (0.0026) | |
| [2024-09-06 09:10:21,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3712.6). Total num frames: 5877760. Throughput: 0: 901.2. Samples: 216510. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:10:21,363][01070] Avg episode reward: [(0, '25.456')] | |
| [2024-09-06 09:10:26,355][01070] Fps is (10 sec: 4915.2, 60 sec: 3891.2, 300 sec: 3737.6). Total num frames: 5902336. Throughput: 0: 961.4. Samples: 223718. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:10:26,360][01070] Avg episode reward: [(0, '24.545')] | |
| [2024-09-06 09:10:27,436][26918] Updated weights for policy 0, policy_version 1442 (0.0016) | |
| [2024-09-06 09:10:31,360][01070] Fps is (10 sec: 3684.9, 60 sec: 3686.2, 300 sec: 3711.5). Total num frames: 5914624. Throughput: 0: 949.5. Samples: 226696. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:10:31,370][01070] Avg episode reward: [(0, '25.006')] | |
| [2024-09-06 09:10:36,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3719.2). Total num frames: 5935104. Throughput: 0: 885.6. Samples: 231046. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:10:36,358][01070] Avg episode reward: [(0, '25.665')] | |
| [2024-09-06 09:10:38,547][26918] Updated weights for policy 0, policy_version 1452 (0.0016) | |
| [2024-09-06 09:10:41,356][01070] Fps is (10 sec: 4507.4, 60 sec: 3891.2, 300 sec: 3742.7). Total num frames: 5959680. Throughput: 0: 940.1. Samples: 238146. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 09:10:41,357][01070] Avg episode reward: [(0, '25.912')] | |
| [2024-09-06 09:10:46,356][01070] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3749.5). Total num frames: 5980160. Throughput: 0: 981.6. Samples: 241772. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) | |
| [2024-09-06 09:10:46,362][01070] Avg episode reward: [(0, '25.585')] | |
| [2024-09-06 09:10:49,005][26918] Updated weights for policy 0, policy_version 1462 (0.0028) | |
| [2024-09-06 09:10:51,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3725.1). Total num frames: 5992448. Throughput: 0: 986.2. Samples: 246522. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:10:51,364][01070] Avg episode reward: [(0, '25.632')] | |
| [2024-09-06 09:10:56,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3823.0, 300 sec: 3747.1). Total num frames: 6017024. Throughput: 0: 971.8. Samples: 252568. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:10:56,363][01070] Avg episode reward: [(0, '27.421')] | |
| [2024-09-06 09:10:56,365][26905] Saving new best policy, reward=27.421! | |
| [2024-09-06 09:10:59,150][26918] Updated weights for policy 0, policy_version 1472 (0.0039) | |
| [2024-09-06 09:11:01,356][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3753.5). Total num frames: 6037504. Throughput: 0: 998.2. Samples: 255912. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:11:01,358][01070] Avg episode reward: [(0, '26.535')] | |
| [2024-09-06 09:11:06,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3745.0). Total num frames: 6053888. Throughput: 0: 997.8. Samples: 261410. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:11:06,359][01070] Avg episode reward: [(0, '25.905')] | |
| [2024-09-06 09:11:10,964][26918] Updated weights for policy 0, policy_version 1482 (0.0037) | |
| [2024-09-06 09:11:11,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3736.7). Total num frames: 6070272. Throughput: 0: 944.5. Samples: 266222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:11:11,358][01070] Avg episode reward: [(0, '26.962')] | |
| [2024-09-06 09:11:16,356][01070] Fps is (10 sec: 4095.7, 60 sec: 4027.7, 300 sec: 3757.1). Total num frames: 6094848. Throughput: 0: 958.3. Samples: 269818. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:11:16,360][01070] Avg episode reward: [(0, '26.222')] | |
| [2024-09-06 09:11:19,722][26918] Updated weights for policy 0, policy_version 1492 (0.0021) | |
| [2024-09-06 09:11:21,356][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3762.8). Total num frames: 6115328. Throughput: 0: 1021.4. Samples: 277008. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 09:11:21,363][01070] Avg episode reward: [(0, '26.579')] | |
| [2024-09-06 09:11:26,356][01070] Fps is (10 sec: 3277.0, 60 sec: 3754.7, 300 sec: 3804.4). Total num frames: 6127616. Throughput: 0: 959.7. Samples: 281332. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:11:26,360][01070] Avg episode reward: [(0, '26.111')] | |
| [2024-09-06 09:11:30,986][26918] Updated weights for policy 0, policy_version 1502 (0.0029) | |
| [2024-09-06 09:11:31,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3959.7, 300 sec: 3887.7). Total num frames: 6152192. Throughput: 0: 948.0. Samples: 284432. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:11:31,358][01070] Avg episode reward: [(0, '27.333')] | |
| [2024-09-06 09:11:36,356][01070] Fps is (10 sec: 4914.9, 60 sec: 4027.7, 300 sec: 3901.6). Total num frames: 6176768. Throughput: 0: 998.4. Samples: 291450. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:11:36,358][01070] Avg episode reward: [(0, '27.316')] | |
| [2024-09-06 09:11:41,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 6189056. Throughput: 0: 983.0. Samples: 296802. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:11:41,362][01070] Avg episode reward: [(0, '27.328')] | |
| [2024-09-06 09:11:41,444][26918] Updated weights for policy 0, policy_version 1512 (0.0034) | |
| [2024-09-06 09:11:46,356][01070] Fps is (10 sec: 3277.0, 60 sec: 3822.9, 300 sec: 3887.7). Total num frames: 6209536. Throughput: 0: 956.0. Samples: 298930. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 09:11:46,363][01070] Avg episode reward: [(0, '27.148')] | |
| [2024-09-06 09:11:51,072][26918] Updated weights for policy 0, policy_version 1522 (0.0025) | |
| [2024-09-06 09:11:51,356][01070] Fps is (10 sec: 4505.5, 60 sec: 4027.7, 300 sec: 3915.5). Total num frames: 6234112. Throughput: 0: 990.5. Samples: 305984. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:11:51,358][01070] Avg episode reward: [(0, '27.216')] | |
| [2024-09-06 09:11:56,356][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3929.4). Total num frames: 6254592. Throughput: 0: 1028.2. Samples: 312490. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 09:11:56,361][01070] Avg episode reward: [(0, '26.377')] | |
| [2024-09-06 09:12:01,358][01070] Fps is (10 sec: 3276.0, 60 sec: 3822.8, 300 sec: 3901.6). Total num frames: 6266880. Throughput: 0: 995.6. Samples: 314624. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:12:01,361][01070] Avg episode reward: [(0, '26.319')] | |
| [2024-09-06 09:12:01,378][26905] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001530_6266880.pth... | |
| [2024-09-06 09:12:01,525][26905] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001303_5337088.pth | |
| [2024-09-06 09:12:02,686][26918] Updated weights for policy 0, policy_version 1532 (0.0038) | |
| [2024-09-06 09:12:06,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 6291456. Throughput: 0: 960.0. Samples: 320206. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:12:06,363][01070] Avg episode reward: [(0, '26.331')] | |
| [2024-09-06 09:12:11,270][26918] Updated weights for policy 0, policy_version 1542 (0.0038) | |
| [2024-09-06 09:12:11,356][01070] Fps is (10 sec: 4916.4, 60 sec: 4096.0, 300 sec: 3929.4). Total num frames: 6316032. Throughput: 0: 1024.0. Samples: 327410. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:12:11,362][01070] Avg episode reward: [(0, '25.599')] | |
| [2024-09-06 09:12:16,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 6328320. Throughput: 0: 1012.3. Samples: 329984. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 09:12:16,364][01070] Avg episode reward: [(0, '26.978')] | |
| [2024-09-06 09:12:21,356][01070] Fps is (10 sec: 3276.7, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 6348800. Throughput: 0: 960.3. Samples: 334664. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:12:21,360][01070] Avg episode reward: [(0, '27.058')] | |
| [2024-09-06 09:12:22,708][26918] Updated weights for policy 0, policy_version 1552 (0.0022) | |
| [2024-09-06 09:12:26,356][01070] Fps is (10 sec: 4505.6, 60 sec: 4096.0, 300 sec: 3915.5). Total num frames: 6373376. Throughput: 0: 1002.0. Samples: 341894. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:12:26,360][01070] Avg episode reward: [(0, '26.297')] | |
| [2024-09-06 09:12:31,356][01070] Fps is (10 sec: 4096.1, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 6389760. Throughput: 0: 1035.0. Samples: 345504. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:12:31,362][01070] Avg episode reward: [(0, '25.448')] | |
| [2024-09-06 09:12:32,950][26918] Updated weights for policy 0, policy_version 1562 (0.0029) | |
| [2024-09-06 09:12:36,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3823.0, 300 sec: 3901.6). Total num frames: 6406144. Throughput: 0: 975.6. Samples: 349884. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:12:36,358][01070] Avg episode reward: [(0, '24.471')] | |
| [2024-09-06 09:12:41,356][01070] Fps is (10 sec: 4096.0, 60 sec: 4027.7, 300 sec: 3901.6). Total num frames: 6430720. Throughput: 0: 974.3. Samples: 356334. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:12:41,358][01070] Avg episode reward: [(0, '24.485')] | |
| [2024-09-06 09:12:42,679][26918] Updated weights for policy 0, policy_version 1572 (0.0032) | |
| [2024-09-06 09:12:46,359][01070] Fps is (10 sec: 4913.6, 60 sec: 4095.8, 300 sec: 3929.3). Total num frames: 6455296. Throughput: 0: 1007.1. Samples: 359942. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:12:46,361][01070] Avg episode reward: [(0, '23.821')] | |
| [2024-09-06 09:12:51,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3915.5). Total num frames: 6467584. Throughput: 0: 1006.8. Samples: 365512. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:12:51,359][01070] Avg episode reward: [(0, '24.404')] | |
| [2024-09-06 09:12:54,080][26918] Updated weights for policy 0, policy_version 1582 (0.0035) | |
| [2024-09-06 09:12:56,356][01070] Fps is (10 sec: 3277.8, 60 sec: 3891.2, 300 sec: 3901.6). Total num frames: 6488064. Throughput: 0: 965.9. Samples: 370876. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:12:56,358][01070] Avg episode reward: [(0, '25.540')] | |
| [2024-09-06 09:13:01,356][01070] Fps is (10 sec: 4505.6, 60 sec: 4096.2, 300 sec: 3901.6). Total num frames: 6512640. Throughput: 0: 987.1. Samples: 374404. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:13:01,360][01070] Avg episode reward: [(0, '25.883')] | |
| [2024-09-06 09:13:02,760][26918] Updated weights for policy 0, policy_version 1592 (0.0021) | |
| [2024-09-06 09:13:06,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3915.5). Total num frames: 6529024. Throughput: 0: 1028.7. Samples: 380956. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:13:06,359][01070] Avg episode reward: [(0, '27.463')] | |
| [2024-09-06 09:13:06,363][26905] Saving new best policy, reward=27.463! | |
| [2024-09-06 09:13:11,356][01070] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3887.7). Total num frames: 6541312. Throughput: 0: 953.2. Samples: 384790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:13:11,359][01070] Avg episode reward: [(0, '26.136')] | |
| [2024-09-06 09:13:15,232][26918] Updated weights for policy 0, policy_version 1602 (0.0024) | |
| [2024-09-06 09:13:16,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3901.6). Total num frames: 6565888. Throughput: 0: 938.2. Samples: 387722. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:13:16,358][01070] Avg episode reward: [(0, '26.648')] | |
| [2024-09-06 09:13:21,356][01070] Fps is (10 sec: 4505.6, 60 sec: 3959.5, 300 sec: 3901.6). Total num frames: 6586368. Throughput: 0: 990.0. Samples: 394432. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:13:21,362][01070] Avg episode reward: [(0, '26.222')] | |
| [2024-09-06 09:13:26,258][26918] Updated weights for policy 0, policy_version 1612 (0.0029) | |
| [2024-09-06 09:13:26,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3901.6). Total num frames: 6602752. Throughput: 0: 956.8. Samples: 399388. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:13:26,362][01070] Avg episode reward: [(0, '25.453')] | |
| [2024-09-06 09:13:31,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3873.8). Total num frames: 6619136. Throughput: 0: 922.8. Samples: 401464. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:13:31,363][01070] Avg episode reward: [(0, '25.128')] | |
| [2024-09-06 09:13:36,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3873.9). Total num frames: 6639616. Throughput: 0: 946.2. Samples: 408090. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:13:36,362][01070] Avg episode reward: [(0, '24.206')] | |
| [2024-09-06 09:13:36,380][26918] Updated weights for policy 0, policy_version 1622 (0.0021) | |
| [2024-09-06 09:13:41,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3887.7). Total num frames: 6660096. Throughput: 0: 955.2. Samples: 413862. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2024-09-06 09:13:41,358][01070] Avg episode reward: [(0, '26.013')] | |
| [2024-09-06 09:13:46,359][01070] Fps is (10 sec: 2866.1, 60 sec: 3549.8, 300 sec: 3846.0). Total num frames: 6668288. Throughput: 0: 912.7. Samples: 415480. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:13:46,363][01070] Avg episode reward: [(0, '26.088')] | |
| [2024-09-06 09:13:51,356][01070] Fps is (10 sec: 2048.0, 60 sec: 3549.9, 300 sec: 3804.4). Total num frames: 6680576. Throughput: 0: 838.4. Samples: 418684. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:13:51,364][01070] Avg episode reward: [(0, '26.938')] | |
| [2024-09-06 09:13:51,474][26918] Updated weights for policy 0, policy_version 1632 (0.0031) | |
| [2024-09-06 09:13:56,356][01070] Fps is (10 sec: 3687.8, 60 sec: 3618.1, 300 sec: 3832.2). Total num frames: 6705152. Throughput: 0: 892.4. Samples: 424946. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:13:56,361][01070] Avg episode reward: [(0, '24.673')] | |
| [2024-09-06 09:14:00,965][26918] Updated weights for policy 0, policy_version 1642 (0.0021) | |
| [2024-09-06 09:14:01,356][01070] Fps is (10 sec: 4505.7, 60 sec: 3549.9, 300 sec: 3860.0). Total num frames: 6725632. Throughput: 0: 902.8. Samples: 428348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:14:01,359][01070] Avg episode reward: [(0, '26.432')] | |
| [2024-09-06 09:14:01,378][26905] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001642_6725632.pth... | |
| [2024-09-06 09:14:01,547][26905] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001416_5799936.pth | |
| [2024-09-06 09:14:06,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3481.6, 300 sec: 3818.3). Total num frames: 6737920. Throughput: 0: 856.3. Samples: 432966. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:14:06,358][01070] Avg episode reward: [(0, '26.729')] | |
| [2024-09-06 09:14:11,356][01070] Fps is (10 sec: 3276.7, 60 sec: 3618.1, 300 sec: 3804.5). Total num frames: 6758400. Throughput: 0: 871.9. Samples: 438622. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:14:11,360][01070] Avg episode reward: [(0, '26.781')] | |
| [2024-09-06 09:14:12,518][26918] Updated weights for policy 0, policy_version 1652 (0.0031) | |
| [2024-09-06 09:14:16,356][01070] Fps is (10 sec: 4505.6, 60 sec: 3618.1, 300 sec: 3846.1). Total num frames: 6782976. Throughput: 0: 901.9. Samples: 442050. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:14:16,361][01070] Avg episode reward: [(0, '24.850')] | |
| [2024-09-06 09:14:21,356][01070] Fps is (10 sec: 4096.1, 60 sec: 3549.9, 300 sec: 3832.2). Total num frames: 6799360. Throughput: 0: 893.2. Samples: 448282. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:14:21,363][01070] Avg episode reward: [(0, '24.715')] | |
| [2024-09-06 09:14:23,497][26918] Updated weights for policy 0, policy_version 1662 (0.0015) | |
| [2024-09-06 09:14:26,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3804.4). Total num frames: 6815744. Throughput: 0: 869.5. Samples: 452988. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:14:26,363][01070] Avg episode reward: [(0, '24.597')] | |
| [2024-09-06 09:14:31,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3686.4, 300 sec: 3832.2). Total num frames: 6840320. Throughput: 0: 910.8. Samples: 456464. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:14:31,362][01070] Avg episode reward: [(0, '25.878')] | |
| [2024-09-06 09:14:32,781][26918] Updated weights for policy 0, policy_version 1672 (0.0018) | |
| [2024-09-06 09:14:36,356][01070] Fps is (10 sec: 4505.4, 60 sec: 3686.4, 300 sec: 3846.1). Total num frames: 6860800. Throughput: 0: 995.2. Samples: 463466. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:14:36,359][01070] Avg episode reward: [(0, '26.414')] | |
| [2024-09-06 09:14:41,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3804.4). Total num frames: 6873088. Throughput: 0: 954.0. Samples: 467874. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:14:41,360][01070] Avg episode reward: [(0, '24.276')] | |
| [2024-09-06 09:14:44,286][26918] Updated weights for policy 0, policy_version 1682 (0.0023) | |
| [2024-09-06 09:14:46,356][01070] Fps is (10 sec: 3686.5, 60 sec: 3823.2, 300 sec: 3818.3). Total num frames: 6897664. Throughput: 0: 940.6. Samples: 470674. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:14:46,358][01070] Avg episode reward: [(0, '24.187')] | |
| [2024-09-06 09:14:51,356][01070] Fps is (10 sec: 4915.2, 60 sec: 4027.7, 300 sec: 3846.1). Total num frames: 6922240. Throughput: 0: 997.2. Samples: 477840. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:14:51,358][01070] Avg episode reward: [(0, '24.909')] | |
| [2024-09-06 09:14:53,391][26918] Updated weights for policy 0, policy_version 1692 (0.0019) | |
| [2024-09-06 09:14:56,356][01070] Fps is (10 sec: 4095.8, 60 sec: 3891.2, 300 sec: 3860.0). Total num frames: 6938624. Throughput: 0: 992.7. Samples: 483292. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:14:56,363][01070] Avg episode reward: [(0, '25.437')] | |
| [2024-09-06 09:15:01,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3860.0). Total num frames: 6955008. Throughput: 0: 962.7. Samples: 485372. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:15:01,362][01070] Avg episode reward: [(0, '24.558')] | |
| [2024-09-06 09:15:04,835][26918] Updated weights for policy 0, policy_version 1702 (0.0034) | |
| [2024-09-06 09:15:06,356][01070] Fps is (10 sec: 3686.5, 60 sec: 3959.4, 300 sec: 3860.0). Total num frames: 6975488. Throughput: 0: 964.8. Samples: 491700. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:15:06,359][01070] Avg episode reward: [(0, '25.932')] | |
| [2024-09-06 09:15:11,359][01070] Fps is (10 sec: 4094.7, 60 sec: 3959.3, 300 sec: 3873.8). Total num frames: 6995968. Throughput: 0: 999.5. Samples: 497968. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:15:11,361][01070] Avg episode reward: [(0, '26.598')] | |
| [2024-09-06 09:15:16,356][01070] Fps is (10 sec: 3276.9, 60 sec: 3754.7, 300 sec: 3832.2). Total num frames: 7008256. Throughput: 0: 966.7. Samples: 499964. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 09:15:16,360][01070] Avg episode reward: [(0, '26.669')] | |
| [2024-09-06 09:15:16,629][26918] Updated weights for policy 0, policy_version 1712 (0.0029) | |
| [2024-09-06 09:15:21,356][01070] Fps is (10 sec: 3687.5, 60 sec: 3891.2, 300 sec: 3832.2). Total num frames: 7032832. Throughput: 0: 934.1. Samples: 505502. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:15:21,361][01070] Avg episode reward: [(0, '28.340')] | |
| [2024-09-06 09:15:21,369][26905] Saving new best policy, reward=28.340! | |
| [2024-09-06 09:15:25,736][26918] Updated weights for policy 0, policy_version 1722 (0.0017) | |
| [2024-09-06 09:15:26,356][01070] Fps is (10 sec: 4505.5, 60 sec: 3959.4, 300 sec: 3860.0). Total num frames: 7053312. Throughput: 0: 988.7. Samples: 512366. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:15:26,361][01070] Avg episode reward: [(0, '26.164')] | |
| [2024-09-06 09:15:31,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 7069696. Throughput: 0: 981.7. Samples: 514852. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2024-09-06 09:15:31,358][01070] Avg episode reward: [(0, '25.977')] | |
| [2024-09-06 09:15:36,356][01070] Fps is (10 sec: 3276.9, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 7086080. Throughput: 0: 917.2. Samples: 519116. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2024-09-06 09:15:36,358][01070] Avg episode reward: [(0, '24.462')] | |
| [2024-09-06 09:15:37,677][26918] Updated weights for policy 0, policy_version 1732 (0.0020) | |
| [2024-09-06 09:15:41,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3832.2). Total num frames: 7110656. Throughput: 0: 950.7. Samples: 526072. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:15:41,363][01070] Avg episode reward: [(0, '26.157')] | |
| [2024-09-06 09:15:46,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 7127040. Throughput: 0: 981.0. Samples: 529516. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:15:46,359][01070] Avg episode reward: [(0, '26.246')] | |
| [2024-09-06 09:15:48,160][26918] Updated weights for policy 0, policy_version 1742 (0.0027) | |
| [2024-09-06 09:15:51,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3818.3). Total num frames: 7143424. Throughput: 0: 937.5. Samples: 533888. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:15:51,361][01070] Avg episode reward: [(0, '26.209')] | |
| [2024-09-06 09:15:56,355][01070] Fps is (10 sec: 4096.0, 60 sec: 3823.0, 300 sec: 3832.2). Total num frames: 7168000. Throughput: 0: 938.6. Samples: 540204. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:15:56,361][01070] Avg episode reward: [(0, '25.364')] | |
| [2024-09-06 09:15:58,022][26918] Updated weights for policy 0, policy_version 1752 (0.0021) | |
| [2024-09-06 09:16:01,362][01070] Fps is (10 sec: 4502.7, 60 sec: 3890.8, 300 sec: 3846.0). Total num frames: 7188480. Throughput: 0: 974.7. Samples: 543832. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:16:01,368][01070] Avg episode reward: [(0, '25.809')] | |
| [2024-09-06 09:16:01,384][26905] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001755_7188480.pth... | |
| [2024-09-06 09:16:01,576][26905] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001530_6266880.pth | |
| [2024-09-06 09:16:06,356][01070] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 7204864. Throughput: 0: 972.9. Samples: 549282. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:16:06,361][01070] Avg episode reward: [(0, '26.285')] | |
| [2024-09-06 09:16:09,912][26918] Updated weights for policy 0, policy_version 1762 (0.0018) | |
| [2024-09-06 09:16:11,356][01070] Fps is (10 sec: 3278.9, 60 sec: 3754.9, 300 sec: 3818.3). Total num frames: 7221248. Throughput: 0: 929.5. Samples: 554194. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:16:11,362][01070] Avg episode reward: [(0, '25.498')] | |
| [2024-09-06 09:16:16,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3832.2). Total num frames: 7245824. Throughput: 0: 949.5. Samples: 557580. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:16:16,358][01070] Avg episode reward: [(0, '25.721')] | |
| [2024-09-06 09:16:19,014][26918] Updated weights for policy 0, policy_version 1772 (0.0031) | |
| [2024-09-06 09:16:21,356][01070] Fps is (10 sec: 4095.9, 60 sec: 3822.9, 300 sec: 3846.1). Total num frames: 7262208. Throughput: 0: 997.7. Samples: 564012. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:16:21,363][01070] Avg episode reward: [(0, '26.446')] | |
| [2024-09-06 09:16:26,356][01070] Fps is (10 sec: 2867.2, 60 sec: 3686.4, 300 sec: 3804.4). Total num frames: 7274496. Throughput: 0: 934.9. Samples: 568142. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:16:26,361][01070] Avg episode reward: [(0, '26.027')] | |
| [2024-09-06 09:16:30,972][26918] Updated weights for policy 0, policy_version 1782 (0.0023) | |
| [2024-09-06 09:16:31,357][01070] Fps is (10 sec: 3686.2, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 7299072. Throughput: 0: 925.0. Samples: 571140. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:16:31,361][01070] Avg episode reward: [(0, '27.410')] | |
| [2024-09-06 09:16:36,357][01070] Fps is (10 sec: 4914.6, 60 sec: 3959.4, 300 sec: 3846.1). Total num frames: 7323648. Throughput: 0: 976.6. Samples: 577838. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:16:36,365][01070] Avg episode reward: [(0, '26.279')] | |
| [2024-09-06 09:16:41,356][01070] Fps is (10 sec: 3686.8, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 7335936. Throughput: 0: 945.0. Samples: 582728. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2024-09-06 09:16:41,361][01070] Avg episode reward: [(0, '26.079')] | |
| [2024-09-06 09:16:42,668][26918] Updated weights for policy 0, policy_version 1792 (0.0023) | |
| [2024-09-06 09:16:46,356][01070] Fps is (10 sec: 2867.6, 60 sec: 3754.7, 300 sec: 3790.5). Total num frames: 7352320. Throughput: 0: 909.9. Samples: 584772. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:16:46,362][01070] Avg episode reward: [(0, '24.573')] | |
| [2024-09-06 09:16:51,356][01070] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 7376896. Throughput: 0: 937.6. Samples: 591472. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:16:51,358][01070] Avg episode reward: [(0, '24.745')] | |
| [2024-09-06 09:16:51,968][26918] Updated weights for policy 0, policy_version 1802 (0.0024) | |
| [2024-09-06 09:16:56,356][01070] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3818.3). Total num frames: 7393280. Throughput: 0: 963.2. Samples: 597536. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:16:56,363][01070] Avg episode reward: [(0, '24.638')] | |
| [2024-09-06 09:17:01,355][01070] Fps is (10 sec: 3276.9, 60 sec: 3686.8, 300 sec: 3790.5). Total num frames: 7409664. Throughput: 0: 931.6. Samples: 599500. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:17:01,361][01070] Avg episode reward: [(0, '23.608')] | |
| [2024-09-06 09:17:04,174][26918] Updated weights for policy 0, policy_version 1812 (0.0038) | |
| [2024-09-06 09:17:06,356][01070] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 7430144. Throughput: 0: 909.0. Samples: 604916. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2024-09-06 09:17:06,361][01070] Avg episode reward: [(0, '24.117')] | |
| [2024-09-06 09:17:11,356][01070] Fps is (10 sec: 4095.9, 60 sec: 3822.9, 300 sec: 3804.4). Total num frames: 7450624. Throughput: 0: 967.2. Samples: 611664. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:17:11,360][01070] Avg episode reward: [(0, '25.639')] | |
| [2024-09-06 09:17:15,155][26918] Updated weights for policy 0, policy_version 1822 (0.0022) | |
| [2024-09-06 09:17:16,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3776.7). Total num frames: 7462912. Throughput: 0: 950.2. Samples: 613900. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:17:16,360][01070] Avg episode reward: [(0, '24.625')] | |
| [2024-09-06 09:17:21,356][01070] Fps is (10 sec: 3276.8, 60 sec: 3686.4, 300 sec: 3762.8). Total num frames: 7483392. Throughput: 0: 902.2. Samples: 618438. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2024-09-06 09:17:21,363][01070] Avg episode reward: [(0, '23.896')] | |
| [2024-09-06 09:17:26,357][01070] Fps is (10 sec: 3685.9, 60 sec: 3754.6, 300 sec: 3762.7). Total num frames: 7499776. Throughput: 0: 928.5. Samples: 624510. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2024-09-06 09:17:26,362][01070] Avg episode reward: [(0, '23.156')] | |
| [2024-09-06 09:17:26,513][26918] Updated weights for policy 0, policy_version 1832 (0.0033) | |
| [2024-09-06 09:17:28,093][26905] Stopping Batcher_0... | |
| [2024-09-06 09:17:28,095][26905] Loop batcher_evt_loop terminating... | |
| [2024-09-06 09:17:28,094][01070] Component Batcher_0 stopped! | |
| [2024-09-06 09:17:28,112][26905] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001833_7507968.pth... | |
| [2024-09-06 09:17:28,214][26918] Weights refcount: 2 0 | |
| [2024-09-06 09:17:28,227][01070] Component InferenceWorker_p0-w0 stopped! | |
| [2024-09-06 09:17:28,231][26918] Stopping InferenceWorker_p0-w0... | |
| [2024-09-06 09:17:28,232][26918] Loop inference_proc0-0_evt_loop terminating... | |
| [2024-09-06 09:17:28,257][26905] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001642_6725632.pth | |
| [2024-09-06 09:17:28,275][26905] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001833_7507968.pth... | |
| [2024-09-06 09:17:28,542][01070] Component LearnerWorker_p0 stopped! | |
| [2024-09-06 09:17:28,549][26905] Stopping LearnerWorker_p0... | |
| [2024-09-06 09:17:28,550][26905] Loop learner_proc0_evt_loop terminating... | |
| [2024-09-06 09:17:28,936][26924] Stopping RolloutWorker_w5... | |
| [2024-09-06 09:17:28,931][01070] Component RolloutWorker_w5 stopped! | |
| [2024-09-06 09:17:28,944][01070] Component RolloutWorker_w2 stopped! | |
| [2024-09-06 09:17:28,946][26921] Stopping RolloutWorker_w2... | |
| [2024-09-06 09:17:28,954][01070] Component RolloutWorker_w0 stopped! | |
| [2024-09-06 09:17:28,958][26919] Stopping RolloutWorker_w0... | |
| [2024-09-06 09:17:28,958][26919] Loop rollout_proc0_evt_loop terminating... | |
| [2024-09-06 09:17:28,963][26922] Stopping RolloutWorker_w3... | |
| [2024-09-06 09:17:28,963][26922] Loop rollout_proc3_evt_loop terminating... | |
| [2024-09-06 09:17:28,963][01070] Component RolloutWorker_w3 stopped! | |
| [2024-09-06 09:17:28,947][26921] Loop rollout_proc2_evt_loop terminating... | |
| [2024-09-06 09:17:28,976][26924] Loop rollout_proc5_evt_loop terminating... | |
| [2024-09-06 09:17:28,980][01070] Component RolloutWorker_w4 stopped! | |
| [2024-09-06 09:17:28,982][26923] Stopping RolloutWorker_w4... | |
| [2024-09-06 09:17:28,983][26923] Loop rollout_proc4_evt_loop terminating... | |
| [2024-09-06 09:17:28,999][26920] Stopping RolloutWorker_w1... | |
| [2024-09-06 09:17:29,000][26920] Loop rollout_proc1_evt_loop terminating... | |
| [2024-09-06 09:17:28,999][01070] Component RolloutWorker_w1 stopped! | |
| [2024-09-06 09:17:29,023][01070] Component RolloutWorker_w6 stopped! | |
| [2024-09-06 09:17:29,025][26925] Stopping RolloutWorker_w6... | |
| [2024-09-06 09:17:29,026][26925] Loop rollout_proc6_evt_loop terminating... | |
| [2024-09-06 09:17:29,026][26926] Stopping RolloutWorker_w7... | |
| [2024-09-06 09:17:29,034][26926] Loop rollout_proc7_evt_loop terminating... | |
| [2024-09-06 09:17:29,035][01070] Component RolloutWorker_w7 stopped! | |
| [2024-09-06 09:17:29,037][01070] Waiting for process learner_proc0 to stop... | |
| [2024-09-06 09:17:31,285][01070] Waiting for process inference_proc0-0 to join... | |
| [2024-09-06 09:17:31,293][01070] Waiting for process rollout_proc0 to join... | |
| [2024-09-06 09:17:34,584][01070] Waiting for process rollout_proc1 to join... | |
| [2024-09-06 09:17:34,588][01070] Waiting for process rollout_proc2 to join... | |
| [2024-09-06 09:17:34,592][01070] Waiting for process rollout_proc3 to join... | |
| [2024-09-06 09:17:34,598][01070] Waiting for process rollout_proc4 to join... | |
| [2024-09-06 09:17:34,600][01070] Waiting for process rollout_proc5 to join... | |
| [2024-09-06 09:17:34,604][01070] Waiting for process rollout_proc6 to join... | |
| [2024-09-06 09:17:34,609][01070] Waiting for process rollout_proc7 to join... | |
| [2024-09-06 09:17:34,612][01070] Batcher 0 profile tree view: | |
| batching: 17.9925, releasing_batches: 0.0162 | |
| [2024-09-06 09:17:34,615][01070] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0010 | |
| wait_policy_total: 248.3928 | |
| update_model: 5.8404 | |
| weight_update: 0.0033 | |
| one_step: 0.0117 | |
| handle_policy_step: 381.6419 | |
| deserialize: 9.7064, stack: 1.8896, obs_to_device_normalize: 77.3906, forward: 201.8424, send_messages: 18.9870 | |
| prepare_outputs: 52.8040 | |
| to_cpu: 30.3389 | |
| [2024-09-06 09:17:34,617][01070] Learner 0 profile tree view: | |
| misc: 0.0038, prepare_batch: 8.6835 | |
| train: 47.1106 | |
| epoch_init: 0.0036, minibatch_init: 0.0064, losses_postprocess: 0.4114, kl_divergence: 0.4119, after_optimizer: 2.1648 | |
| calculate_losses: 16.5354 | |
| losses_init: 0.0030, forward_head: 1.0620, bptt_initial: 10.6952, tail: 0.7308, advantages_returns: 0.2059, losses: 2.4252 | |
| bptt: 1.2532 | |
| bptt_forward_core: 1.1527 | |
| update: 27.2086 | |
| clip: 0.5411 | |
| [2024-09-06 09:17:34,618][01070] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.1890, enqueue_policy_requests: 59.3218, env_step: 514.9058, overhead: 8.3718, complete_rollouts: 4.3760 | |
| save_policy_outputs: 13.4348 | |
| split_output_tensors: 5.2857 | |
| [2024-09-06 09:17:34,623][01070] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.2244, enqueue_policy_requests: 61.7752, env_step: 509.9034, overhead: 8.3385, complete_rollouts: 4.7589 | |
| save_policy_outputs: 12.9599 | |
| split_output_tensors: 5.3185 | |
| [2024-09-06 09:17:34,624][01070] Loop Runner_EvtLoop terminating... | |
| [2024-09-06 09:17:34,626][01070] Runner profile tree view: | |
| main_loop: 690.9829 | |
| [2024-09-06 09:17:34,629][01070] Collected {0: 7507968}, FPS: 3621.9 | |
| [2024-09-06 09:17:45,779][01070] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2024-09-06 09:17:45,780][01070] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-09-06 09:17:45,781][01070] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-09-06 09:17:45,782][01070] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-09-06 09:17:45,784][01070] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 09:17:45,785][01070] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-09-06 09:17:45,786][01070] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 09:17:45,787][01070] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-09-06 09:17:45,788][01070] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2024-09-06 09:17:45,789][01070] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2024-09-06 09:17:45,790][01070] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-09-06 09:17:45,791][01070] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-09-06 09:17:45,793][01070] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-09-06 09:17:45,794][01070] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-09-06 09:17:45,795][01070] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-09-06 09:17:45,830][01070] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 09:17:45,831][01070] RunningMeanStd input shape: (1,) | |
| [2024-09-06 09:17:45,845][01070] ConvEncoder: input_channels=3 | |
| [2024-09-06 09:17:45,883][01070] Conv encoder output size: 512 | |
| [2024-09-06 09:17:45,885][01070] Policy head output size: 512 | |
| [2024-09-06 09:17:45,905][01070] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001833_7507968.pth... | |
| [2024-09-06 09:17:46,321][01070] Num frames 100... | |
| [2024-09-06 09:17:46,441][01070] Num frames 200... | |
| [2024-09-06 09:17:46,582][01070] Num frames 300... | |
| [2024-09-06 09:17:46,701][01070] Num frames 400... | |
| [2024-09-06 09:17:46,820][01070] Num frames 500... | |
| [2024-09-06 09:17:46,938][01070] Num frames 600... | |
| [2024-09-06 09:17:47,062][01070] Num frames 700... | |
| [2024-09-06 09:17:47,185][01070] Num frames 800... | |
| [2024-09-06 09:17:47,317][01070] Num frames 900... | |
| [2024-09-06 09:17:47,445][01070] Avg episode rewards: #0: 19.600, true rewards: #0: 9.600 | |
| [2024-09-06 09:17:47,447][01070] Avg episode reward: 19.600, avg true_objective: 9.600 | |
| [2024-09-06 09:17:47,505][01070] Num frames 1000... | |
| [2024-09-06 09:17:47,628][01070] Num frames 1100... | |
| [2024-09-06 09:17:47,749][01070] Num frames 1200... | |
| [2024-09-06 09:17:47,870][01070] Num frames 1300... | |
| [2024-09-06 09:17:47,994][01070] Num frames 1400... | |
| [2024-09-06 09:17:48,115][01070] Num frames 1500... | |
| [2024-09-06 09:17:48,237][01070] Num frames 1600... | |
| [2024-09-06 09:17:48,367][01070] Num frames 1700... | |
| [2024-09-06 09:17:48,500][01070] Num frames 1800... | |
| [2024-09-06 09:17:48,628][01070] Num frames 1900... | |
| [2024-09-06 09:17:48,753][01070] Num frames 2000... | |
| [2024-09-06 09:17:48,873][01070] Num frames 2100... | |
| [2024-09-06 09:17:48,996][01070] Num frames 2200... | |
| [2024-09-06 09:17:49,120][01070] Num frames 2300... | |
| [2024-09-06 09:17:49,242][01070] Num frames 2400... | |
| [2024-09-06 09:17:49,375][01070] Num frames 2500... | |
| [2024-09-06 09:17:49,505][01070] Num frames 2600... | |
| [2024-09-06 09:17:49,630][01070] Num frames 2700... | |
| [2024-09-06 09:17:49,753][01070] Num frames 2800... | |
| [2024-09-06 09:17:49,829][01070] Avg episode rewards: #0: 33.080, true rewards: #0: 14.080 | |
| [2024-09-06 09:17:49,832][01070] Avg episode reward: 33.080, avg true_objective: 14.080 | |
| [2024-09-06 09:17:49,934][01070] Num frames 2900... | |
| [2024-09-06 09:17:50,057][01070] Num frames 3000... | |
| [2024-09-06 09:17:50,185][01070] Num frames 3100... | |
| [2024-09-06 09:17:50,309][01070] Num frames 3200... | |
| [2024-09-06 09:17:50,439][01070] Num frames 3300... | |
| [2024-09-06 09:17:50,572][01070] Num frames 3400... | |
| [2024-09-06 09:17:50,696][01070] Num frames 3500... | |
| [2024-09-06 09:17:50,818][01070] Num frames 3600... | |
| [2024-09-06 09:17:50,943][01070] Num frames 3700... | |
| [2024-09-06 09:17:51,066][01070] Num frames 3800... | |
| [2024-09-06 09:17:51,193][01070] Num frames 3900... | |
| [2024-09-06 09:17:51,319][01070] Num frames 4000... | |
| [2024-09-06 09:17:51,454][01070] Num frames 4100... | |
| [2024-09-06 09:17:51,590][01070] Num frames 4200... | |
| [2024-09-06 09:17:51,712][01070] Num frames 4300... | |
| [2024-09-06 09:17:51,829][01070] Avg episode rewards: #0: 34.170, true rewards: #0: 14.503 | |
| [2024-09-06 09:17:51,831][01070] Avg episode reward: 34.170, avg true_objective: 14.503 | |
| [2024-09-06 09:17:51,892][01070] Num frames 4400... | |
| [2024-09-06 09:17:52,013][01070] Num frames 4500... | |
| [2024-09-06 09:17:52,134][01070] Num frames 4600... | |
| [2024-09-06 09:17:52,254][01070] Num frames 4700... | |
| [2024-09-06 09:17:52,392][01070] Num frames 4800... | |
| [2024-09-06 09:17:52,519][01070] Num frames 4900... | |
| [2024-09-06 09:17:52,643][01070] Num frames 5000... | |
| [2024-09-06 09:17:52,767][01070] Num frames 5100... | |
| [2024-09-06 09:17:52,887][01070] Num frames 5200... | |
| [2024-09-06 09:17:53,047][01070] Avg episode rewards: #0: 31.220, true rewards: #0: 13.220 | |
| [2024-09-06 09:17:53,050][01070] Avg episode reward: 31.220, avg true_objective: 13.220 | |
| [2024-09-06 09:17:53,067][01070] Num frames 5300... | |
| [2024-09-06 09:17:53,188][01070] Num frames 5400... | |
| [2024-09-06 09:17:53,311][01070] Num frames 5500... | |
| [2024-09-06 09:17:53,439][01070] Num frames 5600... | |
| [2024-09-06 09:17:53,569][01070] Num frames 5700... | |
| [2024-09-06 09:17:53,688][01070] Num frames 5800... | |
| [2024-09-06 09:17:53,820][01070] Num frames 5900... | |
| [2024-09-06 09:17:53,942][01070] Num frames 6000... | |
| [2024-09-06 09:17:54,064][01070] Num frames 6100... | |
| [2024-09-06 09:17:54,186][01070] Num frames 6200... | |
| [2024-09-06 09:17:54,305][01070] Num frames 6300... | |
| [2024-09-06 09:17:54,429][01070] Num frames 6400... | |
| [2024-09-06 09:17:54,564][01070] Num frames 6500... | |
| [2024-09-06 09:17:54,684][01070] Num frames 6600... | |
| [2024-09-06 09:17:54,803][01070] Num frames 6700... | |
| [2024-09-06 09:17:54,921][01070] Num frames 6800... | |
| [2024-09-06 09:17:55,042][01070] Num frames 6900... | |
| [2024-09-06 09:17:55,161][01070] Num frames 7000... | |
| [2024-09-06 09:17:55,283][01070] Num frames 7100... | |
| [2024-09-06 09:17:55,406][01070] Num frames 7200... | |
| [2024-09-06 09:17:55,568][01070] Avg episode rewards: #0: 36.544, true rewards: #0: 14.544 | |
| [2024-09-06 09:17:55,570][01070] Avg episode reward: 36.544, avg true_objective: 14.544 | |
| [2024-09-06 09:17:55,620][01070] Num frames 7300... | |
| [2024-09-06 09:17:55,785][01070] Num frames 7400... | |
| [2024-09-06 09:17:55,948][01070] Num frames 7500... | |
| [2024-09-06 09:17:56,111][01070] Num frames 7600... | |
| [2024-09-06 09:17:56,313][01070] Avg episode rewards: #0: 31.980, true rewards: #0: 12.813 | |
| [2024-09-06 09:17:56,316][01070] Avg episode reward: 31.980, avg true_objective: 12.813 | |
| [2024-09-06 09:17:56,341][01070] Num frames 7700... | |
| [2024-09-06 09:17:56,516][01070] Num frames 7800... | |
| [2024-09-06 09:17:56,682][01070] Num frames 7900... | |
| [2024-09-06 09:17:56,852][01070] Num frames 8000... | |
| [2024-09-06 09:17:57,025][01070] Num frames 8100... | |
| [2024-09-06 09:17:57,187][01070] Num frames 8200... | |
| [2024-09-06 09:17:57,363][01070] Num frames 8300... | |
| [2024-09-06 09:17:57,543][01070] Num frames 8400... | |
| [2024-09-06 09:17:57,725][01070] Num frames 8500... | |
| [2024-09-06 09:17:57,897][01070] Num frames 8600... | |
| [2024-09-06 09:17:58,074][01070] Num frames 8700... | |
| [2024-09-06 09:17:58,236][01070] Avg episode rewards: #0: 30.823, true rewards: #0: 12.537 | |
| [2024-09-06 09:17:58,239][01070] Avg episode reward: 30.823, avg true_objective: 12.537 | |
| [2024-09-06 09:17:58,269][01070] Num frames 8800... | |
| [2024-09-06 09:17:58,395][01070] Num frames 8900... | |
| [2024-09-06 09:17:58,527][01070] Num frames 9000... | |
| [2024-09-06 09:17:58,658][01070] Num frames 9100... | |
| [2024-09-06 09:17:58,782][01070] Num frames 9200... | |
| [2024-09-06 09:17:58,903][01070] Num frames 9300... | |
| [2024-09-06 09:17:59,024][01070] Num frames 9400... | |
| [2024-09-06 09:17:59,146][01070] Num frames 9500... | |
| [2024-09-06 09:17:59,272][01070] Num frames 9600... | |
| [2024-09-06 09:17:59,395][01070] Num frames 9700... | |
| [2024-09-06 09:17:59,524][01070] Num frames 9800... | |
| [2024-09-06 09:17:59,647][01070] Num frames 9900... | |
| [2024-09-06 09:17:59,745][01070] Avg episode rewards: #0: 30.285, true rewards: #0: 12.410 | |
| [2024-09-06 09:17:59,746][01070] Avg episode reward: 30.285, avg true_objective: 12.410 | |
| [2024-09-06 09:17:59,839][01070] Num frames 10000... | |
| [2024-09-06 09:17:59,962][01070] Num frames 10100... | |
| [2024-09-06 09:18:00,084][01070] Num frames 10200... | |
| [2024-09-06 09:18:00,206][01070] Num frames 10300... | |
| [2024-09-06 09:18:00,365][01070] Avg episode rewards: #0: 27.871, true rewards: #0: 11.538 | |
| [2024-09-06 09:18:00,366][01070] Avg episode reward: 27.871, avg true_objective: 11.538 | |
| [2024-09-06 09:18:00,389][01070] Num frames 10400... | |
| [2024-09-06 09:18:00,522][01070] Num frames 10500... | |
| [2024-09-06 09:18:00,660][01070] Num frames 10600... | |
| [2024-09-06 09:18:00,798][01070] Num frames 10700... | |
| [2024-09-06 09:18:00,918][01070] Num frames 10800... | |
| [2024-09-06 09:18:01,038][01070] Num frames 10900... | |
| [2024-09-06 09:18:01,163][01070] Num frames 11000... | |
| [2024-09-06 09:18:01,284][01070] Num frames 11100... | |
| [2024-09-06 09:18:01,405][01070] Num frames 11200... | |
| [2024-09-06 09:18:01,533][01070] Num frames 11300... | |
| [2024-09-06 09:18:01,658][01070] Num frames 11400... | |
| [2024-09-06 09:18:01,790][01070] Num frames 11500... | |
| [2024-09-06 09:18:01,915][01070] Num frames 11600... | |
| [2024-09-06 09:18:02,004][01070] Avg episode rewards: #0: 27.925, true rewards: #0: 11.625 | |
| [2024-09-06 09:18:02,006][01070] Avg episode reward: 27.925, avg true_objective: 11.625 | |
| [2024-09-06 09:19:12,012][01070] Replay video saved to /content/train_dir/default_experiment/replay.mp4! | |
| [2024-09-06 09:19:36,347][01070] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2024-09-06 09:19:36,349][01070] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2024-09-06 09:19:36,350][01070] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2024-09-06 09:19:36,351][01070] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2024-09-06 09:19:36,355][01070] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2024-09-06 09:19:36,356][01070] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2024-09-06 09:19:36,359][01070] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2024-09-06 09:19:36,359][01070] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2024-09-06 09:19:36,361][01070] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2024-09-06 09:19:36,362][01070] Adding new argument 'hf_repository'='Re-Re/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2024-09-06 09:19:36,364][01070] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2024-09-06 09:19:36,365][01070] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2024-09-06 09:19:36,367][01070] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2024-09-06 09:19:36,368][01070] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2024-09-06 09:19:36,369][01070] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2024-09-06 09:19:36,399][01070] RunningMeanStd input shape: (3, 72, 128) | |
| [2024-09-06 09:19:36,400][01070] RunningMeanStd input shape: (1,) | |
| [2024-09-06 09:19:36,414][01070] ConvEncoder: input_channels=3 | |
| [2024-09-06 09:19:36,451][01070] Conv encoder output size: 512 | |
| [2024-09-06 09:19:36,452][01070] Policy head output size: 512 | |
| [2024-09-06 09:19:36,477][01070] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001833_7507968.pth... | |
| [2024-09-06 09:19:36,955][01070] Num frames 100... | |
| [2024-09-06 09:19:37,079][01070] Num frames 200... | |
| [2024-09-06 09:19:37,209][01070] Num frames 300... | |
| [2024-09-06 09:19:37,385][01070] Num frames 400... | |
| [2024-09-06 09:19:37,559][01070] Num frames 500... | |
| [2024-09-06 09:19:37,729][01070] Num frames 600... | |
| [2024-09-06 09:19:37,920][01070] Num frames 700... | |
| [2024-09-06 09:19:38,094][01070] Num frames 800... | |
| [2024-09-06 09:19:38,257][01070] Num frames 900... | |
| [2024-09-06 09:19:38,425][01070] Num frames 1000... | |
| [2024-09-06 09:19:38,607][01070] Num frames 1100... | |
| [2024-09-06 09:19:38,779][01070] Num frames 1200... | |
| [2024-09-06 09:19:38,970][01070] Num frames 1300... | |
| [2024-09-06 09:19:39,151][01070] Num frames 1400... | |
| [2024-09-06 09:19:39,333][01070] Num frames 1500... | |
| [2024-09-06 09:19:39,514][01070] Num frames 1600... | |
| [2024-09-06 09:19:39,697][01070] Num frames 1700... | |
| [2024-09-06 09:19:39,863][01070] Num frames 1800... | |
| [2024-09-06 09:19:39,995][01070] Num frames 1900... | |
| [2024-09-06 09:19:40,121][01070] Num frames 2000... | |
| [2024-09-06 09:19:40,238][01070] Avg episode rewards: #0: 51.479, true rewards: #0: 20.480 | |
| [2024-09-06 09:19:40,240][01070] Avg episode reward: 51.479, avg true_objective: 20.480 | |
| [2024-09-06 09:19:40,307][01070] Num frames 2100... | |
| [2024-09-06 09:19:40,432][01070] Num frames 2200... | |
| [2024-09-06 09:19:40,565][01070] Num frames 2300... | |
| [2024-09-06 09:19:40,689][01070] Num frames 2400... | |
| [2024-09-06 09:19:40,851][01070] Avg episode rewards: #0: 28.429, true rewards: #0: 12.430 | |
| [2024-09-06 09:19:40,853][01070] Avg episode reward: 28.429, avg true_objective: 12.430 | |
| [2024-09-06 09:19:40,874][01070] Num frames 2500... | |
| [2024-09-06 09:19:41,006][01070] Num frames 2600... | |
| [2024-09-06 09:19:41,130][01070] Num frames 2700... | |
| [2024-09-06 09:19:41,253][01070] Num frames 2800... | |
| [2024-09-06 09:19:41,420][01070] Avg episode rewards: #0: 21.636, true rewards: #0: 9.637 | |
| [2024-09-06 09:19:41,422][01070] Avg episode reward: 21.636, avg true_objective: 9.637 | |
| [2024-09-06 09:19:41,438][01070] Num frames 2900... | |
| [2024-09-06 09:19:41,568][01070] Num frames 3000... | |
| [2024-09-06 09:19:41,692][01070] Num frames 3100... | |
| [2024-09-06 09:19:41,815][01070] Num frames 3200... | |
| [2024-09-06 09:19:41,942][01070] Num frames 3300... | |
| [2024-09-06 09:19:42,072][01070] Num frames 3400... | |
| [2024-09-06 09:19:42,225][01070] Avg episode rewards: #0: 19.450, true rewards: #0: 8.700 | |
| [2024-09-06 09:19:42,228][01070] Avg episode reward: 19.450, avg true_objective: 8.700 | |
| [2024-09-06 09:19:42,256][01070] Num frames 3500... | |
| [2024-09-06 09:19:42,378][01070] Num frames 3600... | |
| [2024-09-06 09:19:42,511][01070] Num frames 3700... | |
| [2024-09-06 09:19:42,635][01070] Num frames 3800... | |
| [2024-09-06 09:19:42,758][01070] Num frames 3900... | |
| [2024-09-06 09:19:42,882][01070] Num frames 4000... | |
| [2024-09-06 09:19:43,006][01070] Num frames 4100... | |
| [2024-09-06 09:19:43,138][01070] Num frames 4200... | |
| [2024-09-06 09:19:43,263][01070] Num frames 4300... | |
| [2024-09-06 09:19:43,384][01070] Num frames 4400... | |
| [2024-09-06 09:19:43,510][01070] Num frames 4500... | |
| [2024-09-06 09:19:43,637][01070] Num frames 4600... | |
| [2024-09-06 09:19:43,807][01070] Avg episode rewards: #0: 20.992, true rewards: #0: 9.392 | |
| [2024-09-06 09:19:43,808][01070] Avg episode reward: 20.992, avg true_objective: 9.392 | |
| [2024-09-06 09:19:43,817][01070] Num frames 4700... | |
| [2024-09-06 09:19:43,941][01070] Num frames 4800... | |
| [2024-09-06 09:19:44,073][01070] Num frames 4900... | |
| [2024-09-06 09:19:44,194][01070] Num frames 5000... | |
| [2024-09-06 09:19:44,313][01070] Num frames 5100... | |
| [2024-09-06 09:19:44,458][01070] Avg episode rewards: #0: 19.460, true rewards: #0: 8.627 | |
| [2024-09-06 09:19:44,461][01070] Avg episode reward: 19.460, avg true_objective: 8.627 | |
| [2024-09-06 09:19:44,499][01070] Num frames 5200... | |
| [2024-09-06 09:19:44,619][01070] Num frames 5300... | |
| [2024-09-06 09:19:44,747][01070] Num frames 5400... | |
| [2024-09-06 09:19:44,868][01070] Num frames 5500... | |
| [2024-09-06 09:19:44,985][01070] Num frames 5600... | |
| [2024-09-06 09:19:45,160][01070] Avg episode rewards: #0: 18.424, true rewards: #0: 8.139 | |
| [2024-09-06 09:19:45,162][01070] Avg episode reward: 18.424, avg true_objective: 8.139 | |
| [2024-09-06 09:19:45,169][01070] Num frames 5700... | |
| [2024-09-06 09:19:45,291][01070] Num frames 5800... | |
| [2024-09-06 09:19:45,409][01070] Num frames 5900... | |
| [2024-09-06 09:19:45,542][01070] Num frames 6000... | |
| [2024-09-06 09:19:45,659][01070] Num frames 6100... | |
| [2024-09-06 09:19:45,778][01070] Num frames 6200... | |
| [2024-09-06 09:19:45,899][01070] Num frames 6300... | |
| [2024-09-06 09:19:46,016][01070] Num frames 6400... | |
| [2024-09-06 09:19:46,141][01070] Num frames 6500... | |
| [2024-09-06 09:19:46,260][01070] Num frames 6600... | |
| [2024-09-06 09:19:46,384][01070] Num frames 6700... | |
| [2024-09-06 09:19:46,514][01070] Num frames 6800... | |
| [2024-09-06 09:19:46,639][01070] Num frames 6900... | |
| [2024-09-06 09:19:46,763][01070] Num frames 7000... | |
| [2024-09-06 09:19:46,887][01070] Num frames 7100... | |
| [2024-09-06 09:19:47,010][01070] Num frames 7200... | |
| [2024-09-06 09:19:47,139][01070] Num frames 7300... | |
| [2024-09-06 09:19:47,260][01070] Num frames 7400... | |
| [2024-09-06 09:19:47,388][01070] Num frames 7500... | |
| [2024-09-06 09:19:47,517][01070] Num frames 7600... | |
| [2024-09-06 09:19:47,639][01070] Num frames 7700... | |
| [2024-09-06 09:19:47,813][01070] Avg episode rewards: #0: 23.371, true rewards: #0: 9.746 | |
| [2024-09-06 09:19:47,815][01070] Avg episode reward: 23.371, avg true_objective: 9.746 | |
| [2024-09-06 09:19:47,821][01070] Num frames 7800... | |
| [2024-09-06 09:19:47,942][01070] Num frames 7900... | |
| [2024-09-06 09:19:48,066][01070] Num frames 8000... | |
| [2024-09-06 09:19:48,202][01070] Num frames 8100... | |
| [2024-09-06 09:19:48,326][01070] Num frames 8200... | |
| [2024-09-06 09:19:48,450][01070] Num frames 8300... | |
| [2024-09-06 09:19:48,583][01070] Num frames 8400... | |
| [2024-09-06 09:19:48,708][01070] Num frames 8500... | |
| [2024-09-06 09:19:48,838][01070] Num frames 8600... | |
| [2024-09-06 09:19:48,963][01070] Num frames 8700... | |
| [2024-09-06 09:19:49,088][01070] Num frames 8800... | |
| [2024-09-06 09:19:49,227][01070] Num frames 8900... | |
| [2024-09-06 09:19:49,353][01070] Num frames 9000... | |
| [2024-09-06 09:19:49,485][01070] Num frames 9100... | |
| [2024-09-06 09:19:49,612][01070] Num frames 9200... | |
| [2024-09-06 09:19:49,738][01070] Num frames 9300... | |
| [2024-09-06 09:19:49,893][01070] Num frames 9400... | |
| [2024-09-06 09:19:50,071][01070] Num frames 9500... | |
| [2024-09-06 09:19:50,251][01070] Num frames 9600... | |
| [2024-09-06 09:19:50,422][01070] Num frames 9700... | |
| [2024-09-06 09:19:50,629][01070] Avg episode rewards: #0: 26.986, true rewards: #0: 10.876 | |
| [2024-09-06 09:19:50,631][01070] Avg episode reward: 26.986, avg true_objective: 10.876 | |
| [2024-09-06 09:19:50,658][01070] Num frames 9800... | |
| [2024-09-06 09:19:50,826][01070] Num frames 9900... | |
| [2024-09-06 09:19:50,989][01070] Num frames 10000... | |
| [2024-09-06 09:19:51,162][01070] Num frames 10100... | |
| [2024-09-06 09:19:51,348][01070] Num frames 10200... | |
| [2024-09-06 09:19:51,531][01070] Num frames 10300... | |
| [2024-09-06 09:19:51,711][01070] Num frames 10400... | |
| [2024-09-06 09:19:51,893][01070] Num frames 10500... | |
| [2024-09-06 09:19:52,069][01070] Num frames 10600... | |
| [2024-09-06 09:19:52,177][01070] Avg episode rewards: #0: 26.327, true rewards: #0: 10.627 | |
| [2024-09-06 09:19:52,179][01070] Avg episode reward: 26.327, avg true_objective: 10.627 | |
| [2024-09-06 09:20:57,694][01070] Replay video saved to /content/train_dir/default_experiment/replay.mp4! | |