| [2023-02-22 18:30:14,984][01245] Saving configuration to /content/train_dir/default_experiment/config.json... | |
| [2023-02-22 18:30:14,987][01245] Rollout worker 0 uses device cpu | |
| [2023-02-22 18:30:14,989][01245] Rollout worker 1 uses device cpu | |
| [2023-02-22 18:30:14,992][01245] Rollout worker 2 uses device cpu | |
| [2023-02-22 18:30:14,994][01245] Rollout worker 3 uses device cpu | |
| [2023-02-22 18:30:14,997][01245] Rollout worker 4 uses device cpu | |
| [2023-02-22 18:30:14,999][01245] Rollout worker 5 uses device cpu | |
| [2023-02-22 18:30:15,000][01245] Rollout worker 6 uses device cpu | |
| [2023-02-22 18:30:15,002][01245] Rollout worker 7 uses device cpu | |
| [2023-02-22 18:30:15,191][01245] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-02-22 18:30:15,192][01245] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-02-22 18:30:15,224][01245] Starting all processes... | |
| [2023-02-22 18:30:15,226][01245] Starting process learner_proc0 | |
| [2023-02-22 18:30:15,278][01245] Starting all processes... | |
| [2023-02-22 18:30:15,290][01245] Starting process inference_proc0-0 | |
| [2023-02-22 18:30:15,290][01245] Starting process rollout_proc0 | |
| [2023-02-22 18:30:15,292][01245] Starting process rollout_proc1 | |
| [2023-02-22 18:30:15,292][01245] Starting process rollout_proc2 | |
| [2023-02-22 18:30:15,292][01245] Starting process rollout_proc3 | |
| [2023-02-22 18:30:15,292][01245] Starting process rollout_proc4 | |
| [2023-02-22 18:30:15,292][01245] Starting process rollout_proc5 | |
| [2023-02-22 18:30:15,292][01245] Starting process rollout_proc6 | |
| [2023-02-22 18:30:15,292][01245] Starting process rollout_proc7 | |
| [2023-02-22 18:30:26,318][15062] Worker 4 uses CPU cores [0] | |
| [2023-02-22 18:30:26,608][15039] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-02-22 18:30:26,612][15039] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-02-22 18:30:26,860][15059] Worker 1 uses CPU cores [1] | |
| [2023-02-22 18:30:26,893][15058] Worker 0 uses CPU cores [0] | |
| [2023-02-22 18:30:27,034][15064] Worker 6 uses CPU cores [0] | |
| [2023-02-22 18:30:26,984][15063] Worker 3 uses CPU cores [1] | |
| [2023-02-22 18:30:27,039][15060] Worker 2 uses CPU cores [0] | |
| [2023-02-22 18:30:27,144][15057] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-02-22 18:30:27,146][15057] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-02-22 18:30:27,200][15065] Worker 7 uses CPU cores [1] | |
| [2023-02-22 18:30:27,201][15061] Worker 5 uses CPU cores [1] | |
| [2023-02-22 18:30:27,592][15039] Num visible devices: 1 | |
| [2023-02-22 18:30:27,592][15057] Num visible devices: 1 | |
| [2023-02-22 18:30:27,608][15039] Starting seed is not provided | |
| [2023-02-22 18:30:27,608][15039] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-02-22 18:30:27,608][15039] Initializing actor-critic model on device cuda:0 | |
| [2023-02-22 18:30:27,608][15039] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-22 18:30:27,610][15039] RunningMeanStd input shape: (1,) | |
| [2023-02-22 18:30:27,623][15039] ConvEncoder: input_channels=3 | |
| [2023-02-22 18:30:27,906][15039] Conv encoder output size: 512 | |
| [2023-02-22 18:30:27,906][15039] Policy head output size: 512 | |
| [2023-02-22 18:30:27,958][15039] Created Actor Critic model with architecture: | |
| [2023-02-22 18:30:27,958][15039] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-02-22 18:30:35,015][15039] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-02-22 18:30:35,016][15039] No checkpoints found | |
| [2023-02-22 18:30:35,017][15039] Did not load from checkpoint, starting from scratch! | |
| [2023-02-22 18:30:35,017][15039] Initialized policy 0 weights for model version 0 | |
| [2023-02-22 18:30:35,020][15039] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-02-22 18:30:35,031][15039] LearnerWorker_p0 finished initialization! | |
| [2023-02-22 18:30:35,187][01245] Heartbeat connected on LearnerWorker_p0 | |
| [2023-02-22 18:30:35,204][01245] Heartbeat connected on RolloutWorker_w0 | |
| [2023-02-22 18:30:35,209][01245] Heartbeat connected on RolloutWorker_w1 | |
| [2023-02-22 18:30:35,211][01245] Heartbeat connected on RolloutWorker_w2 | |
| [2023-02-22 18:30:35,213][01245] Heartbeat connected on RolloutWorker_w3 | |
| [2023-02-22 18:30:35,215][01245] Heartbeat connected on RolloutWorker_w4 | |
| [2023-02-22 18:30:35,217][01245] Heartbeat connected on RolloutWorker_w5 | |
| [2023-02-22 18:30:35,223][01245] Heartbeat connected on RolloutWorker_w7 | |
| [2023-02-22 18:30:35,226][01245] Heartbeat connected on RolloutWorker_w6 | |
| [2023-02-22 18:30:35,239][01245] Heartbeat connected on Batcher_0 | |
| [2023-02-22 18:30:35,255][15057] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-22 18:30:35,256][15057] RunningMeanStd input shape: (1,) | |
| [2023-02-22 18:30:35,268][15057] ConvEncoder: input_channels=3 | |
| [2023-02-22 18:30:35,365][15057] Conv encoder output size: 512 | |
| [2023-02-22 18:30:35,365][15057] Policy head output size: 512 | |
| [2023-02-22 18:30:35,594][01245] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-22 18:30:38,287][01245] Inference worker 0-0 is ready! | |
| [2023-02-22 18:30:38,293][01245] All inference workers are ready! Signal rollout workers to start! | |
| [2023-02-22 18:30:38,297][01245] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-02-22 18:30:38,454][15065] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:30:38,468][15063] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:30:38,478][15061] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:30:38,498][15059] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:30:38,521][15060] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:30:38,562][15062] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:30:38,597][15064] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:30:38,605][15058] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:30:40,128][15064] Decorrelating experience for 0 frames... | |
| [2023-02-22 18:30:40,132][15062] Decorrelating experience for 0 frames... | |
| [2023-02-22 18:30:40,242][15061] Decorrelating experience for 0 frames... | |
| [2023-02-22 18:30:40,248][15065] Decorrelating experience for 0 frames... | |
| [2023-02-22 18:30:40,250][15063] Decorrelating experience for 0 frames... | |
| [2023-02-22 18:30:40,258][15059] Decorrelating experience for 0 frames... | |
| [2023-02-22 18:30:40,594][01245] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-22 18:30:41,384][15062] Decorrelating experience for 32 frames... | |
| [2023-02-22 18:30:41,632][15064] Decorrelating experience for 32 frames... | |
| [2023-02-22 18:30:41,637][15060] Decorrelating experience for 0 frames... | |
| [2023-02-22 18:30:41,697][15061] Decorrelating experience for 32 frames... | |
| [2023-02-22 18:30:41,699][15065] Decorrelating experience for 32 frames... | |
| [2023-02-22 18:30:41,703][15063] Decorrelating experience for 32 frames... | |
| [2023-02-22 18:30:42,708][15058] Decorrelating experience for 0 frames... | |
| [2023-02-22 18:30:42,715][15062] Decorrelating experience for 64 frames... | |
| [2023-02-22 18:30:42,864][15064] Decorrelating experience for 64 frames... | |
| [2023-02-22 18:30:42,881][15059] Decorrelating experience for 32 frames... | |
| [2023-02-22 18:30:43,050][15061] Decorrelating experience for 64 frames... | |
| [2023-02-22 18:30:43,096][15063] Decorrelating experience for 64 frames... | |
| [2023-02-22 18:30:43,781][15058] Decorrelating experience for 32 frames... | |
| [2023-02-22 18:30:43,887][15062] Decorrelating experience for 96 frames... | |
| [2023-02-22 18:30:44,038][15060] Decorrelating experience for 32 frames... | |
| [2023-02-22 18:30:44,356][15065] Decorrelating experience for 64 frames... | |
| [2023-02-22 18:30:44,525][15061] Decorrelating experience for 96 frames... | |
| [2023-02-22 18:30:44,611][15063] Decorrelating experience for 96 frames... | |
| [2023-02-22 18:30:45,118][15058] Decorrelating experience for 64 frames... | |
| [2023-02-22 18:30:45,135][15059] Decorrelating experience for 64 frames... | |
| [2023-02-22 18:30:45,165][15064] Decorrelating experience for 96 frames... | |
| [2023-02-22 18:30:45,483][15065] Decorrelating experience for 96 frames... | |
| [2023-02-22 18:30:45,594][01245] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-22 18:30:45,796][15059] Decorrelating experience for 96 frames... | |
| [2023-02-22 18:30:46,132][15060] Decorrelating experience for 64 frames... | |
| [2023-02-22 18:30:46,163][15058] Decorrelating experience for 96 frames... | |
| [2023-02-22 18:30:46,546][15060] Decorrelating experience for 96 frames... | |
| [2023-02-22 18:30:50,549][15039] Signal inference workers to stop experience collection... | |
| [2023-02-22 18:30:50,556][15057] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-02-22 18:30:50,594][01245] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 138.5. Samples: 2078. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-22 18:30:50,600][01245] Avg episode reward: [(0, '1.678')] | |
| [2023-02-22 18:30:53,343][15039] Signal inference workers to resume experience collection... | |
| [2023-02-22 18:30:53,344][15057] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-02-22 18:30:55,594][01245] Fps is (10 sec: 409.6, 60 sec: 204.8, 300 sec: 204.8). Total num frames: 4096. Throughput: 0: 113.9. Samples: 2278. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2023-02-22 18:30:55,596][01245] Avg episode reward: [(0, '2.358')] | |
| [2023-02-22 18:31:00,594][01245] Fps is (10 sec: 2457.6, 60 sec: 983.0, 300 sec: 983.0). Total num frames: 24576. Throughput: 0: 226.6. Samples: 5664. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) | |
| [2023-02-22 18:31:00,599][01245] Avg episode reward: [(0, '3.545')] | |
| [2023-02-22 18:31:05,088][15057] Updated weights for policy 0, policy_version 10 (0.0030) | |
| [2023-02-22 18:31:05,598][01245] Fps is (10 sec: 3684.7, 60 sec: 1365.1, 300 sec: 1365.1). Total num frames: 40960. Throughput: 0: 356.0. Samples: 10682. Policy #0 lag: (min: 0.0, avg: 0.1, max: 1.0) | |
| [2023-02-22 18:31:05,601][01245] Avg episode reward: [(0, '3.873')] | |
| [2023-02-22 18:31:10,597][01245] Fps is (10 sec: 2866.2, 60 sec: 1521.2, 300 sec: 1521.2). Total num frames: 53248. Throughput: 0: 370.6. Samples: 12974. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:31:10,601][01245] Avg episode reward: [(0, '4.477')] | |
| [2023-02-22 18:31:15,594][01245] Fps is (10 sec: 2868.6, 60 sec: 1740.8, 300 sec: 1740.8). Total num frames: 69632. Throughput: 0: 425.4. Samples: 17014. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) | |
| [2023-02-22 18:31:15,596][01245] Avg episode reward: [(0, '4.623')] | |
| [2023-02-22 18:31:19,600][15057] Updated weights for policy 0, policy_version 20 (0.0059) | |
| [2023-02-22 18:31:20,594][01245] Fps is (10 sec: 2868.2, 60 sec: 1820.4, 300 sec: 1820.4). Total num frames: 81920. Throughput: 0: 474.1. Samples: 21334. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0) | |
| [2023-02-22 18:31:20,596][01245] Avg episode reward: [(0, '4.567')] | |
| [2023-02-22 18:31:25,594][01245] Fps is (10 sec: 3686.4, 60 sec: 2129.9, 300 sec: 2129.9). Total num frames: 106496. Throughput: 0: 553.5. Samples: 24906. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:31:25,597][01245] Avg episode reward: [(0, '4.389')] | |
| [2023-02-22 18:31:25,606][15039] Saving new best policy, reward=4.389! | |
| [2023-02-22 18:31:28,621][15057] Updated weights for policy 0, policy_version 30 (0.0028) | |
| [2023-02-22 18:31:30,594][01245] Fps is (10 sec: 4915.2, 60 sec: 2383.1, 300 sec: 2383.1). Total num frames: 131072. Throughput: 0: 712.3. Samples: 32052. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:31:30,596][01245] Avg episode reward: [(0, '4.475')] | |
| [2023-02-22 18:31:30,601][15039] Saving new best policy, reward=4.475! | |
| [2023-02-22 18:31:35,594][01245] Fps is (10 sec: 3686.4, 60 sec: 2389.3, 300 sec: 2389.3). Total num frames: 143360. Throughput: 0: 771.9. Samples: 36812. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:31:35,596][01245] Avg episode reward: [(0, '4.472')] | |
| [2023-02-22 18:31:40,594][01245] Fps is (10 sec: 2867.2, 60 sec: 2662.4, 300 sec: 2457.6). Total num frames: 159744. Throughput: 0: 814.5. Samples: 38930. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:31:40,600][01245] Avg episode reward: [(0, '4.479')] | |
| [2023-02-22 18:31:40,609][15039] Saving new best policy, reward=4.479! | |
| [2023-02-22 18:31:41,134][15057] Updated weights for policy 0, policy_version 40 (0.0028) | |
| [2023-02-22 18:31:45,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3072.0, 300 sec: 2633.1). Total num frames: 184320. Throughput: 0: 879.2. Samples: 45226. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:31:45,600][01245] Avg episode reward: [(0, '4.337')] | |
| [2023-02-22 18:31:49,875][15057] Updated weights for policy 0, policy_version 50 (0.0022) | |
| [2023-02-22 18:31:50,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3413.3, 300 sec: 2730.7). Total num frames: 204800. Throughput: 0: 920.2. Samples: 52088. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:31:50,597][01245] Avg episode reward: [(0, '4.334')] | |
| [2023-02-22 18:31:55,594][01245] Fps is (10 sec: 3686.3, 60 sec: 3618.1, 300 sec: 2764.8). Total num frames: 221184. Throughput: 0: 919.0. Samples: 54328. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:31:55,600][01245] Avg episode reward: [(0, '4.377')] | |
| [2023-02-22 18:32:00,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 2794.9). Total num frames: 237568. Throughput: 0: 930.0. Samples: 58864. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:32:00,599][01245] Avg episode reward: [(0, '4.376')] | |
| [2023-02-22 18:32:02,319][15057] Updated weights for policy 0, policy_version 60 (0.0014) | |
| [2023-02-22 18:32:05,594][01245] Fps is (10 sec: 3686.5, 60 sec: 3618.4, 300 sec: 2867.2). Total num frames: 258048. Throughput: 0: 984.8. Samples: 65648. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:32:05,600][01245] Avg episode reward: [(0, '4.569')] | |
| [2023-02-22 18:32:05,607][15039] Saving new best policy, reward=4.569! | |
| [2023-02-22 18:32:10,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3823.2, 300 sec: 2975.0). Total num frames: 282624. Throughput: 0: 982.3. Samples: 69110. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:32:10,596][01245] Avg episode reward: [(0, '4.601')] | |
| [2023-02-22 18:32:10,612][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000069_282624.pth... | |
| [2023-02-22 18:32:10,730][15039] Saving new best policy, reward=4.601! | |
| [2023-02-22 18:32:11,711][15057] Updated weights for policy 0, policy_version 70 (0.0024) | |
| [2023-02-22 18:32:15,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3754.7, 300 sec: 2949.1). Total num frames: 294912. Throughput: 0: 935.1. Samples: 74132. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:32:15,597][01245] Avg episode reward: [(0, '4.507')] | |
| [2023-02-22 18:32:20,594][01245] Fps is (10 sec: 2457.6, 60 sec: 3754.7, 300 sec: 2925.7). Total num frames: 307200. Throughput: 0: 905.7. Samples: 77568. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:32:20,597][01245] Avg episode reward: [(0, '4.375')] | |
| [2023-02-22 18:32:24,548][15057] Updated weights for policy 0, policy_version 80 (0.0038) | |
| [2023-02-22 18:32:25,595][01245] Fps is (10 sec: 3685.9, 60 sec: 3754.6, 300 sec: 3016.1). Total num frames: 331776. Throughput: 0: 936.3. Samples: 81066. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:32:25,601][01245] Avg episode reward: [(0, '4.316')] | |
| [2023-02-22 18:32:30,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3063.1). Total num frames: 352256. Throughput: 0: 954.2. Samples: 88166. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:32:30,598][01245] Avg episode reward: [(0, '4.466')] | |
| [2023-02-22 18:32:34,650][15057] Updated weights for policy 0, policy_version 90 (0.0040) | |
| [2023-02-22 18:32:35,594][01245] Fps is (10 sec: 3686.9, 60 sec: 3754.7, 300 sec: 3072.0). Total num frames: 368640. Throughput: 0: 909.6. Samples: 93020. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:32:35,598][01245] Avg episode reward: [(0, '4.400')] | |
| [2023-02-22 18:32:40,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3080.2). Total num frames: 385024. Throughput: 0: 910.1. Samples: 95280. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:32:40,597][01245] Avg episode reward: [(0, '4.593')] | |
| [2023-02-22 18:32:45,526][15057] Updated weights for policy 0, policy_version 100 (0.0028) | |
| [2023-02-22 18:32:45,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3754.7, 300 sec: 3150.8). Total num frames: 409600. Throughput: 0: 948.0. Samples: 101526. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:32:45,596][01245] Avg episode reward: [(0, '4.463')] | |
| [2023-02-22 18:32:50,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3754.7, 300 sec: 3185.8). Total num frames: 430080. Throughput: 0: 948.0. Samples: 108306. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:32:50,603][01245] Avg episode reward: [(0, '4.448')] | |
| [2023-02-22 18:32:55,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3618.2, 300 sec: 3130.5). Total num frames: 438272. Throughput: 0: 897.1. Samples: 109480. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:32:55,601][01245] Avg episode reward: [(0, '4.494')] | |
| [2023-02-22 18:32:59,439][15057] Updated weights for policy 0, policy_version 110 (0.0022) | |
| [2023-02-22 18:33:00,594][01245] Fps is (10 sec: 2457.6, 60 sec: 3618.1, 300 sec: 3135.6). Total num frames: 454656. Throughput: 0: 865.1. Samples: 113062. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:33:00,599][01245] Avg episode reward: [(0, '4.739')] | |
| [2023-02-22 18:33:00,610][15039] Saving new best policy, reward=4.739! | |
| [2023-02-22 18:33:05,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3167.6). Total num frames: 475136. Throughput: 0: 941.8. Samples: 119950. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:33:05,599][01245] Avg episode reward: [(0, '4.502')] | |
| [2023-02-22 18:33:08,215][15057] Updated weights for policy 0, policy_version 120 (0.0019) | |
| [2023-02-22 18:33:10,597][01245] Fps is (10 sec: 4504.2, 60 sec: 3618.0, 300 sec: 3223.9). Total num frames: 499712. Throughput: 0: 944.2. Samples: 123556. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0) | |
| [2023-02-22 18:33:10,600][01245] Avg episode reward: [(0, '4.562')] | |
| [2023-02-22 18:33:15,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3618.1, 300 sec: 3200.0). Total num frames: 512000. Throughput: 0: 895.0. Samples: 128442. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:33:15,598][01245] Avg episode reward: [(0, '4.671')] | |
| [2023-02-22 18:33:20,594][01245] Fps is (10 sec: 2868.0, 60 sec: 3686.4, 300 sec: 3202.3). Total num frames: 528384. Throughput: 0: 889.9. Samples: 133066. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:33:20,597][01245] Avg episode reward: [(0, '4.528')] | |
| [2023-02-22 18:33:20,906][15057] Updated weights for policy 0, policy_version 130 (0.0021) | |
| [2023-02-22 18:33:25,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3686.5, 300 sec: 3252.7). Total num frames: 552960. Throughput: 0: 913.7. Samples: 136396. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:33:25,596][01245] Avg episode reward: [(0, '4.443')] | |
| [2023-02-22 18:33:30,144][15057] Updated weights for policy 0, policy_version 140 (0.0025) | |
| [2023-02-22 18:33:30,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3686.4, 300 sec: 3276.8). Total num frames: 573440. Throughput: 0: 928.0. Samples: 143284. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:33:30,598][01245] Avg episode reward: [(0, '4.480')] | |
| [2023-02-22 18:33:35,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3618.1, 300 sec: 3254.0). Total num frames: 585728. Throughput: 0: 877.6. Samples: 147798. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:33:35,602][01245] Avg episode reward: [(0, '4.427')] | |
| [2023-02-22 18:33:40,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3618.1, 300 sec: 3254.7). Total num frames: 602112. Throughput: 0: 900.2. Samples: 149990. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:33:40,599][01245] Avg episode reward: [(0, '4.443')] | |
| [2023-02-22 18:33:42,182][15057] Updated weights for policy 0, policy_version 150 (0.0013) | |
| [2023-02-22 18:33:45,597][01245] Fps is (10 sec: 4094.8, 60 sec: 3618.0, 300 sec: 3298.3). Total num frames: 626688. Throughput: 0: 967.0. Samples: 156582. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:33:45,602][01245] Avg episode reward: [(0, '4.514')] | |
| [2023-02-22 18:33:50,594][01245] Fps is (10 sec: 4915.2, 60 sec: 3686.4, 300 sec: 3339.8). Total num frames: 651264. Throughput: 0: 960.2. Samples: 163158. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:33:50,597][01245] Avg episode reward: [(0, '4.537')] | |
| [2023-02-22 18:33:51,999][15057] Updated weights for policy 0, policy_version 160 (0.0011) | |
| [2023-02-22 18:33:55,594][01245] Fps is (10 sec: 3687.5, 60 sec: 3754.7, 300 sec: 3317.8). Total num frames: 663552. Throughput: 0: 927.7. Samples: 165298. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:33:55,599][01245] Avg episode reward: [(0, '4.488')] | |
| [2023-02-22 18:34:00,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3316.8). Total num frames: 679936. Throughput: 0: 916.6. Samples: 169688. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:34:00,599][01245] Avg episode reward: [(0, '4.451')] | |
| [2023-02-22 18:34:03,692][15057] Updated weights for policy 0, policy_version 170 (0.0058) | |
| [2023-02-22 18:34:05,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3354.8). Total num frames: 704512. Throughput: 0: 966.6. Samples: 176564. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:34:05,596][01245] Avg episode reward: [(0, '4.552')] | |
| [2023-02-22 18:34:10,600][01245] Fps is (10 sec: 4503.0, 60 sec: 3754.5, 300 sec: 3372.0). Total num frames: 724992. Throughput: 0: 971.9. Samples: 180138. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:34:10,602][01245] Avg episode reward: [(0, '4.513')] | |
| [2023-02-22 18:34:10,620][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000177_724992.pth... | |
| [2023-02-22 18:34:14,103][15057] Updated weights for policy 0, policy_version 180 (0.0013) | |
| [2023-02-22 18:34:15,596][01245] Fps is (10 sec: 3685.7, 60 sec: 3822.8, 300 sec: 3369.9). Total num frames: 741376. Throughput: 0: 925.2. Samples: 184918. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:34:15,599][01245] Avg episode reward: [(0, '4.557')] | |
| [2023-02-22 18:34:20,598][01245] Fps is (10 sec: 3277.4, 60 sec: 3822.7, 300 sec: 3367.8). Total num frames: 757760. Throughput: 0: 929.0. Samples: 189608. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:34:20,608][01245] Avg episode reward: [(0, '4.737')] | |
| [2023-02-22 18:34:25,030][15057] Updated weights for policy 0, policy_version 190 (0.0023) | |
| [2023-02-22 18:34:25,594][01245] Fps is (10 sec: 3687.1, 60 sec: 3754.7, 300 sec: 3383.7). Total num frames: 778240. Throughput: 0: 956.3. Samples: 193024. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:34:25,597][01245] Avg episode reward: [(0, '4.669')] | |
| [2023-02-22 18:34:30,594][01245] Fps is (10 sec: 4097.6, 60 sec: 3754.7, 300 sec: 3398.8). Total num frames: 798720. Throughput: 0: 968.4. Samples: 200158. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:34:30,600][01245] Avg episode reward: [(0, '4.449')] | |
| [2023-02-22 18:34:35,598][01245] Fps is (10 sec: 3684.7, 60 sec: 3822.6, 300 sec: 3396.2). Total num frames: 815104. Throughput: 0: 923.5. Samples: 204720. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:34:35,601][01245] Avg episode reward: [(0, '4.517')] | |
| [2023-02-22 18:34:36,456][15057] Updated weights for policy 0, policy_version 200 (0.0012) | |
| [2023-02-22 18:34:40,597][01245] Fps is (10 sec: 3275.8, 60 sec: 3822.7, 300 sec: 3393.8). Total num frames: 831488. Throughput: 0: 923.3. Samples: 206850. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:34:40,604][01245] Avg episode reward: [(0, '4.527')] | |
| [2023-02-22 18:34:45,596][01245] Fps is (10 sec: 4096.9, 60 sec: 3823.0, 300 sec: 3424.2). Total num frames: 856064. Throughput: 0: 974.6. Samples: 213546. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:34:45,600][01245] Avg episode reward: [(0, '4.420')] | |
| [2023-02-22 18:34:46,110][15057] Updated weights for policy 0, policy_version 210 (0.0015) | |
| [2023-02-22 18:34:50,598][01245] Fps is (10 sec: 4505.1, 60 sec: 3754.4, 300 sec: 3437.4). Total num frames: 876544. Throughput: 0: 969.5. Samples: 220194. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:34:50,605][01245] Avg episode reward: [(0, '4.510')] | |
| [2023-02-22 18:34:55,594][01245] Fps is (10 sec: 3687.3, 60 sec: 3822.9, 300 sec: 3434.3). Total num frames: 892928. Throughput: 0: 939.1. Samples: 222392. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:34:55,601][01245] Avg episode reward: [(0, '4.594')] | |
| [2023-02-22 18:34:58,150][15057] Updated weights for policy 0, policy_version 220 (0.0013) | |
| [2023-02-22 18:35:00,594][01245] Fps is (10 sec: 3278.2, 60 sec: 3822.9, 300 sec: 3431.4). Total num frames: 909312. Throughput: 0: 934.6. Samples: 226972. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:35:00,596][01245] Avg episode reward: [(0, '4.515')] | |
| [2023-02-22 18:35:05,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3458.8). Total num frames: 933888. Throughput: 0: 988.4. Samples: 234080. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:35:05,602][01245] Avg episode reward: [(0, '4.595')] | |
| [2023-02-22 18:35:06,927][15057] Updated weights for policy 0, policy_version 230 (0.0024) | |
| [2023-02-22 18:35:10,597][01245] Fps is (10 sec: 4503.9, 60 sec: 3823.1, 300 sec: 3470.4). Total num frames: 954368. Throughput: 0: 991.4. Samples: 237640. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:35:10,600][01245] Avg episode reward: [(0, '4.649')] | |
| [2023-02-22 18:35:15,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3823.1, 300 sec: 3467.0). Total num frames: 970752. Throughput: 0: 939.4. Samples: 242432. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:35:15,599][01245] Avg episode reward: [(0, '4.570')] | |
| [2023-02-22 18:35:19,280][15057] Updated weights for policy 0, policy_version 240 (0.0020) | |
| [2023-02-22 18:35:20,594][01245] Fps is (10 sec: 3278.0, 60 sec: 3823.2, 300 sec: 3463.6). Total num frames: 987136. Throughput: 0: 947.7. Samples: 247364. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:35:20,596][01245] Avg episode reward: [(0, '4.721')] | |
| [2023-02-22 18:35:25,594][01245] Fps is (10 sec: 4095.9, 60 sec: 3891.2, 300 sec: 3488.7). Total num frames: 1011712. Throughput: 0: 978.6. Samples: 250884. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:35:25,596][01245] Avg episode reward: [(0, '4.573')] | |
| [2023-02-22 18:35:28,126][15057] Updated weights for policy 0, policy_version 250 (0.0013) | |
| [2023-02-22 18:35:30,598][01245] Fps is (10 sec: 4503.5, 60 sec: 3890.9, 300 sec: 3498.9). Total num frames: 1032192. Throughput: 0: 989.5. Samples: 258076. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:35:30,606][01245] Avg episode reward: [(0, '4.444')] | |
| [2023-02-22 18:35:35,594][01245] Fps is (10 sec: 3276.9, 60 sec: 3823.2, 300 sec: 3540.6). Total num frames: 1044480. Throughput: 0: 941.6. Samples: 262564. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:35:35,600][01245] Avg episode reward: [(0, '4.547')] | |
| [2023-02-22 18:35:40,557][15057] Updated weights for policy 0, policy_version 260 (0.0023) | |
| [2023-02-22 18:35:40,594][01245] Fps is (10 sec: 3278.1, 60 sec: 3891.4, 300 sec: 3610.0). Total num frames: 1064960. Throughput: 0: 942.5. Samples: 264806. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:35:40,598][01245] Avg episode reward: [(0, '4.563')] | |
| [2023-02-22 18:35:45,599][01245] Fps is (10 sec: 3275.0, 60 sec: 3686.2, 300 sec: 3651.6). Total num frames: 1077248. Throughput: 0: 950.0. Samples: 269728. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:35:45,605][01245] Avg episode reward: [(0, '4.542')] | |
| [2023-02-22 18:35:50,594][01245] Fps is (10 sec: 2867.4, 60 sec: 3618.4, 300 sec: 3693.3). Total num frames: 1093632. Throughput: 0: 888.8. Samples: 274078. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:35:50,599][01245] Avg episode reward: [(0, '4.434')] | |
| [2023-02-22 18:35:54,788][15057] Updated weights for policy 0, policy_version 270 (0.0015) | |
| [2023-02-22 18:35:55,594][01245] Fps is (10 sec: 2868.8, 60 sec: 3549.9, 300 sec: 3665.6). Total num frames: 1105920. Throughput: 0: 853.8. Samples: 276060. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:35:55,596][01245] Avg episode reward: [(0, '4.483')] | |
| [2023-02-22 18:36:00,594][01245] Fps is (10 sec: 2867.3, 60 sec: 3549.9, 300 sec: 3665.6). Total num frames: 1122304. Throughput: 0: 844.9. Samples: 280452. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:36:00,600][01245] Avg episode reward: [(0, '4.508')] | |
| [2023-02-22 18:36:05,264][15057] Updated weights for policy 0, policy_version 280 (0.0014) | |
| [2023-02-22 18:36:05,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3549.9, 300 sec: 3707.3). Total num frames: 1146880. Throughput: 0: 888.8. Samples: 287362. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:36:05,596][01245] Avg episode reward: [(0, '4.542')] | |
| [2023-02-22 18:36:10,600][01245] Fps is (10 sec: 4502.6, 60 sec: 3549.7, 300 sec: 3721.0). Total num frames: 1167360. Throughput: 0: 890.1. Samples: 290946. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:36:10,603][01245] Avg episode reward: [(0, '4.764')] | |
| [2023-02-22 18:36:10,619][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000285_1167360.pth... | |
| [2023-02-22 18:36:10,827][15039] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000069_282624.pth | |
| [2023-02-22 18:36:10,851][15039] Saving new best policy, reward=4.764! | |
| [2023-02-22 18:36:15,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3549.9, 300 sec: 3735.0). Total num frames: 1183744. Throughput: 0: 838.8. Samples: 295820. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:36:15,598][01245] Avg episode reward: [(0, '4.643')] | |
| [2023-02-22 18:36:16,587][15057] Updated weights for policy 0, policy_version 290 (0.0013) | |
| [2023-02-22 18:36:20,594][01245] Fps is (10 sec: 3279.0, 60 sec: 3549.9, 300 sec: 3707.2). Total num frames: 1200128. Throughput: 0: 843.3. Samples: 300514. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:36:20,596][01245] Avg episode reward: [(0, '4.469')] | |
| [2023-02-22 18:36:25,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3693.3). Total num frames: 1220608. Throughput: 0: 871.1. Samples: 304006. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:36:25,596][01245] Avg episode reward: [(0, '4.550')] | |
| [2023-02-22 18:36:26,431][15057] Updated weights for policy 0, policy_version 300 (0.0012) | |
| [2023-02-22 18:36:30,594][01245] Fps is (10 sec: 4505.5, 60 sec: 3550.1, 300 sec: 3735.0). Total num frames: 1245184. Throughput: 0: 922.6. Samples: 311238. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:36:30,603][01245] Avg episode reward: [(0, '4.623')] | |
| [2023-02-22 18:36:35,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3618.1, 300 sec: 3735.0). Total num frames: 1261568. Throughput: 0: 930.2. Samples: 315938. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:36:35,601][01245] Avg episode reward: [(0, '4.560')] | |
| [2023-02-22 18:36:38,075][15057] Updated weights for policy 0, policy_version 310 (0.0019) | |
| [2023-02-22 18:36:40,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3707.2). Total num frames: 1277952. Throughput: 0: 935.2. Samples: 318146. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:36:40,602][01245] Avg episode reward: [(0, '4.493')] | |
| [2023-02-22 18:36:45,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3686.7, 300 sec: 3707.2). Total num frames: 1298432. Throughput: 0: 981.7. Samples: 324628. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:36:45,601][01245] Avg episode reward: [(0, '4.590')] | |
| [2023-02-22 18:36:47,373][15057] Updated weights for policy 0, policy_version 320 (0.0018) | |
| [2023-02-22 18:36:50,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 1323008. Throughput: 0: 979.3. Samples: 331430. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:36:50,596][01245] Avg episode reward: [(0, '4.483')] | |
| [2023-02-22 18:36:55,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3735.0). Total num frames: 1339392. Throughput: 0: 949.2. Samples: 333652. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:36:55,597][01245] Avg episode reward: [(0, '4.561')] | |
| [2023-02-22 18:36:59,596][15057] Updated weights for policy 0, policy_version 330 (0.0013) | |
| [2023-02-22 18:37:00,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3721.1). Total num frames: 1355776. Throughput: 0: 942.3. Samples: 338224. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:37:00,598][01245] Avg episode reward: [(0, '4.679')] | |
| [2023-02-22 18:37:05,594][01245] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 1376256. Throughput: 0: 991.9. Samples: 345152. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:37:05,597][01245] Avg episode reward: [(0, '4.513')] | |
| [2023-02-22 18:37:08,330][15057] Updated weights for policy 0, policy_version 340 (0.0025) | |
| [2023-02-22 18:37:10,597][01245] Fps is (10 sec: 4503.9, 60 sec: 3891.4, 300 sec: 3748.8). Total num frames: 1400832. Throughput: 0: 991.6. Samples: 348630. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:37:10,600][01245] Avg episode reward: [(0, '4.526')] | |
| [2023-02-22 18:37:15,594][01245] Fps is (10 sec: 3686.5, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1413120. Throughput: 0: 942.7. Samples: 353658. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:37:15,596][01245] Avg episode reward: [(0, '4.554')] | |
| [2023-02-22 18:37:20,594][01245] Fps is (10 sec: 2868.3, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 1429504. Throughput: 0: 942.7. Samples: 358360. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:37:20,596][01245] Avg episode reward: [(0, '4.451')] | |
| [2023-02-22 18:37:20,833][15057] Updated weights for policy 0, policy_version 350 (0.0027) | |
| [2023-02-22 18:37:25,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3735.0). Total num frames: 1454080. Throughput: 0: 971.0. Samples: 361840. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:37:25,598][01245] Avg episode reward: [(0, '4.522')] | |
| [2023-02-22 18:37:29,466][15057] Updated weights for policy 0, policy_version 360 (0.0024) | |
| [2023-02-22 18:37:30,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1474560. Throughput: 0: 985.0. Samples: 368954. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:37:30,604][01245] Avg episode reward: [(0, '4.621')] | |
| [2023-02-22 18:37:35,594][01245] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 1490944. Throughput: 0: 936.2. Samples: 373558. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:37:35,598][01245] Avg episode reward: [(0, '4.635')] | |
| [2023-02-22 18:37:40,594][01245] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 1507328. Throughput: 0: 937.5. Samples: 375842. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:37:40,597][01245] Avg episode reward: [(0, '4.643')] | |
| [2023-02-22 18:37:41,721][15057] Updated weights for policy 0, policy_version 370 (0.0016) | |
| [2023-02-22 18:37:45,594][01245] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3735.0). Total num frames: 1531904. Throughput: 0: 982.1. Samples: 382418. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:37:45,596][01245] Avg episode reward: [(0, '4.661')] | |
| [2023-02-22 18:37:50,594][01245] Fps is (10 sec: 4505.7, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1552384. Throughput: 0: 978.2. Samples: 389170. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:37:50,596][01245] Avg episode reward: [(0, '4.567')] | |
| [2023-02-22 18:37:50,880][15057] Updated weights for policy 0, policy_version 380 (0.0017) | |
| [2023-02-22 18:37:55,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1568768. Throughput: 0: 950.1. Samples: 391382. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:37:55,598][01245] Avg episode reward: [(0, '4.430')] | |
| [2023-02-22 18:38:00,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 1585152. Throughput: 0: 937.3. Samples: 395836. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:38:00,601][01245] Avg episode reward: [(0, '4.453')] | |
| [2023-02-22 18:38:02,784][15057] Updated weights for policy 0, policy_version 390 (0.0035) | |
| [2023-02-22 18:38:05,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 1609728. Throughput: 0: 989.0. Samples: 402864. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-02-22 18:38:05,599][01245] Avg episode reward: [(0, '4.747')] | |
| [2023-02-22 18:38:10,598][01245] Fps is (10 sec: 4503.5, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 1630208. Throughput: 0: 990.3. Samples: 406406. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:38:10,601][01245] Avg episode reward: [(0, '4.403')] | |
| [2023-02-22 18:38:10,616][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000398_1630208.pth... | |
| [2023-02-22 18:38:10,797][15039] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000177_724992.pth | |
| [2023-02-22 18:38:12,790][15057] Updated weights for policy 0, policy_version 400 (0.0021) | |
| [2023-02-22 18:38:15,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 1646592. Throughput: 0: 940.2. Samples: 411262. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:38:15,596][01245] Avg episode reward: [(0, '4.397')] | |
| [2023-02-22 18:38:20,594][01245] Fps is (10 sec: 3278.3, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 1662976. Throughput: 0: 946.9. Samples: 416168. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:38:20,600][01245] Avg episode reward: [(0, '4.760')] | |
| [2023-02-22 18:38:23,839][15057] Updated weights for policy 0, policy_version 410 (0.0017) | |
| [2023-02-22 18:38:25,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 1683456. Throughput: 0: 974.8. Samples: 419706. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:38:25,596][01245] Avg episode reward: [(0, '4.992')] | |
| [2023-02-22 18:38:25,684][15039] Saving new best policy, reward=4.992! | |
| [2023-02-22 18:38:30,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3804.4). Total num frames: 1708032. Throughput: 0: 982.5. Samples: 426630. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:38:30,598][01245] Avg episode reward: [(0, '4.989')] | |
| [2023-02-22 18:38:34,685][15057] Updated weights for policy 0, policy_version 420 (0.0027) | |
| [2023-02-22 18:38:35,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 1720320. Throughput: 0: 933.6. Samples: 431180. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:38:35,601][01245] Avg episode reward: [(0, '4.941')] | |
| [2023-02-22 18:38:40,597][01245] Fps is (10 sec: 2866.1, 60 sec: 3822.7, 300 sec: 3762.8). Total num frames: 1736704. Throughput: 0: 933.6. Samples: 433398. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:38:40,605][01245] Avg episode reward: [(0, '4.785')] | |
| [2023-02-22 18:38:45,038][15057] Updated weights for policy 0, policy_version 430 (0.0030) | |
| [2023-02-22 18:38:45,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 1761280. Throughput: 0: 982.8. Samples: 440064. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:38:45,601][01245] Avg episode reward: [(0, '4.828')] | |
| [2023-02-22 18:38:50,594][01245] Fps is (10 sec: 4507.3, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 1781760. Throughput: 0: 973.1. Samples: 446654. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:38:50,597][01245] Avg episode reward: [(0, '4.828')] | |
| [2023-02-22 18:38:55,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 1798144. Throughput: 0: 944.5. Samples: 448902. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:38:55,597][01245] Avg episode reward: [(0, '5.184')] | |
| [2023-02-22 18:38:55,603][15039] Saving new best policy, reward=5.184! | |
| [2023-02-22 18:38:56,526][15057] Updated weights for policy 0, policy_version 440 (0.0015) | |
| [2023-02-22 18:39:00,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 1814528. Throughput: 0: 936.9. Samples: 453422. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:39:00,602][01245] Avg episode reward: [(0, '5.113')] | |
| [2023-02-22 18:39:05,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1839104. Throughput: 0: 986.0. Samples: 460536. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:39:05,597][01245] Avg episode reward: [(0, '4.682')] | |
| [2023-02-22 18:39:06,031][15057] Updated weights for policy 0, policy_version 450 (0.0022) | |
| [2023-02-22 18:39:10,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3823.2, 300 sec: 3790.6). Total num frames: 1859584. Throughput: 0: 986.8. Samples: 464114. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:39:10,600][01245] Avg episode reward: [(0, '4.479')] | |
| [2023-02-22 18:39:15,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3790.6). Total num frames: 1875968. Throughput: 0: 936.9. Samples: 468790. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:39:15,598][01245] Avg episode reward: [(0, '4.610')] | |
| [2023-02-22 18:39:18,199][15057] Updated weights for policy 0, policy_version 460 (0.0018) | |
| [2023-02-22 18:39:20,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1892352. Throughput: 0: 946.7. Samples: 473780. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:39:20,597][01245] Avg episode reward: [(0, '4.776')] | |
| [2023-02-22 18:39:25,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 1916928. Throughput: 0: 974.8. Samples: 477260. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:39:25,597][01245] Avg episode reward: [(0, '4.976')] | |
| [2023-02-22 18:39:27,361][15057] Updated weights for policy 0, policy_version 470 (0.0024) | |
| [2023-02-22 18:39:30,595][01245] Fps is (10 sec: 4505.1, 60 sec: 3822.9, 300 sec: 3804.5). Total num frames: 1937408. Throughput: 0: 979.4. Samples: 484136. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:39:30,602][01245] Avg episode reward: [(0, '4.735')] | |
| [2023-02-22 18:39:35,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3790.6). Total num frames: 1949696. Throughput: 0: 926.8. Samples: 488358. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:39:35,601][01245] Avg episode reward: [(0, '4.805')] | |
| [2023-02-22 18:39:39,846][15057] Updated weights for policy 0, policy_version 480 (0.0022) | |
| [2023-02-22 18:39:40,594][01245] Fps is (10 sec: 2867.5, 60 sec: 3823.2, 300 sec: 3762.8). Total num frames: 1966080. Throughput: 0: 926.3. Samples: 490586. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:39:40,600][01245] Avg episode reward: [(0, '4.965')] | |
| [2023-02-22 18:39:45,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 1990656. Throughput: 0: 980.9. Samples: 497564. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:39:45,604][01245] Avg episode reward: [(0, '4.781')] | |
| [2023-02-22 18:39:48,641][15057] Updated weights for policy 0, policy_version 490 (0.0015) | |
| [2023-02-22 18:39:50,601][01245] Fps is (10 sec: 4502.2, 60 sec: 3822.4, 300 sec: 3790.4). Total num frames: 2011136. Throughput: 0: 960.7. Samples: 503776. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:39:50,606][01245] Avg episode reward: [(0, '4.883')] | |
| [2023-02-22 18:39:55,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 2027520. Throughput: 0: 931.3. Samples: 506024. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:39:55,600][01245] Avg episode reward: [(0, '4.950')] | |
| [2023-02-22 18:40:00,594][01245] Fps is (10 sec: 3279.3, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 2043904. Throughput: 0: 931.0. Samples: 510686. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:40:00,601][01245] Avg episode reward: [(0, '5.082')] | |
| [2023-02-22 18:40:01,072][15057] Updated weights for policy 0, policy_version 500 (0.0021) | |
| [2023-02-22 18:40:05,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2068480. Throughput: 0: 974.2. Samples: 517620. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:40:05,600][01245] Avg episode reward: [(0, '4.937')] | |
| [2023-02-22 18:40:10,212][15057] Updated weights for policy 0, policy_version 510 (0.0011) | |
| [2023-02-22 18:40:10,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 2088960. Throughput: 0: 973.7. Samples: 521078. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:40:10,595][01245] Avg episode reward: [(0, '4.708')] | |
| [2023-02-22 18:40:10,611][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000510_2088960.pth... | |
| [2023-02-22 18:40:10,758][15039] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000285_1167360.pth | |
| [2023-02-22 18:40:15,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3754.7, 300 sec: 3776.7). Total num frames: 2101248. Throughput: 0: 922.1. Samples: 525628. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:40:15,596][01245] Avg episode reward: [(0, '4.711')] | |
| [2023-02-22 18:40:20,595][01245] Fps is (10 sec: 2457.2, 60 sec: 3686.3, 300 sec: 3735.0). Total num frames: 2113536. Throughput: 0: 912.2. Samples: 529408. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:40:20,609][01245] Avg episode reward: [(0, '4.459')] | |
| [2023-02-22 18:40:25,399][15057] Updated weights for policy 0, policy_version 520 (0.0012) | |
| [2023-02-22 18:40:25,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3549.9, 300 sec: 3721.2). Total num frames: 2129920. Throughput: 0: 909.2. Samples: 531498. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:40:25,599][01245] Avg episode reward: [(0, '4.419')] | |
| [2023-02-22 18:40:30,594][01245] Fps is (10 sec: 3277.4, 60 sec: 3481.7, 300 sec: 3735.0). Total num frames: 2146304. Throughput: 0: 870.5. Samples: 536736. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:40:30,601][01245] Avg episode reward: [(0, '4.662')] | |
| [2023-02-22 18:40:35,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3721.1). Total num frames: 2162688. Throughput: 0: 837.7. Samples: 541468. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:40:35,598][01245] Avg episode reward: [(0, '4.908')] | |
| [2023-02-22 18:40:37,803][15057] Updated weights for policy 0, policy_version 530 (0.0012) | |
| [2023-02-22 18:40:40,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3549.9, 300 sec: 3735.1). Total num frames: 2179072. Throughput: 0: 837.1. Samples: 543692. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:40:40,600][01245] Avg episode reward: [(0, '4.898')] | |
| [2023-02-22 18:40:45,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3748.9). Total num frames: 2199552. Throughput: 0: 874.0. Samples: 550016. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:40:45,598][01245] Avg episode reward: [(0, '4.726')] | |
| [2023-02-22 18:40:47,385][15057] Updated weights for policy 0, policy_version 540 (0.0031) | |
| [2023-02-22 18:40:50,598][01245] Fps is (10 sec: 4503.5, 60 sec: 3550.0, 300 sec: 3790.5). Total num frames: 2224128. Throughput: 0: 871.6. Samples: 556848. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:40:50,600][01245] Avg episode reward: [(0, '4.896')] | |
| [2023-02-22 18:40:55,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3481.6, 300 sec: 3776.7). Total num frames: 2236416. Throughput: 0: 844.1. Samples: 559062. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:40:55,598][01245] Avg episode reward: [(0, '4.866')] | |
| [2023-02-22 18:40:59,887][15057] Updated weights for policy 0, policy_version 550 (0.0024) | |
| [2023-02-22 18:41:00,594][01245] Fps is (10 sec: 2868.5, 60 sec: 3481.6, 300 sec: 3748.9). Total num frames: 2252800. Throughput: 0: 839.4. Samples: 563402. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:41:00,600][01245] Avg episode reward: [(0, '5.080')] | |
| [2023-02-22 18:41:05,594][01245] Fps is (10 sec: 4095.9, 60 sec: 3481.6, 300 sec: 3762.8). Total num frames: 2277376. Throughput: 0: 903.5. Samples: 570062. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:41:05,597][01245] Avg episode reward: [(0, '5.148')] | |
| [2023-02-22 18:41:08,651][15057] Updated weights for policy 0, policy_version 560 (0.0013) | |
| [2023-02-22 18:41:10,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3481.6, 300 sec: 3776.7). Total num frames: 2297856. Throughput: 0: 936.2. Samples: 573626. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:41:10,604][01245] Avg episode reward: [(0, '5.176')] | |
| [2023-02-22 18:41:15,597][01245] Fps is (10 sec: 3685.1, 60 sec: 3549.6, 300 sec: 3776.6). Total num frames: 2314240. Throughput: 0: 934.3. Samples: 578784. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:41:15,599][01245] Avg episode reward: [(0, '5.084')] | |
| [2023-02-22 18:41:20,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3618.2, 300 sec: 3762.8). Total num frames: 2330624. Throughput: 0: 928.5. Samples: 583252. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:41:20,596][01245] Avg episode reward: [(0, '5.064')] | |
| [2023-02-22 18:41:21,302][15057] Updated weights for policy 0, policy_version 570 (0.0012) | |
| [2023-02-22 18:41:25,594][01245] Fps is (10 sec: 4097.5, 60 sec: 3754.7, 300 sec: 3762.8). Total num frames: 2355200. Throughput: 0: 955.3. Samples: 586680. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:41:25,596][01245] Avg episode reward: [(0, '4.888')] | |
| [2023-02-22 18:41:30,003][15057] Updated weights for policy 0, policy_version 580 (0.0018) | |
| [2023-02-22 18:41:30,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2375680. Throughput: 0: 969.7. Samples: 593652. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:41:30,596][01245] Avg episode reward: [(0, '4.805')] | |
| [2023-02-22 18:41:35,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2392064. Throughput: 0: 925.1. Samples: 598472. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:41:35,601][01245] Avg episode reward: [(0, '4.819')] | |
| [2023-02-22 18:41:40,594][01245] Fps is (10 sec: 2867.0, 60 sec: 3754.6, 300 sec: 3748.9). Total num frames: 2404352. Throughput: 0: 924.3. Samples: 600658. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:41:40,597][01245] Avg episode reward: [(0, '4.841')] | |
| [2023-02-22 18:41:42,443][15057] Updated weights for policy 0, policy_version 590 (0.0013) | |
| [2023-02-22 18:41:45,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2428928. Throughput: 0: 966.8. Samples: 606908. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:41:45,602][01245] Avg episode reward: [(0, '4.926')] | |
| [2023-02-22 18:41:50,600][01245] Fps is (10 sec: 4912.3, 60 sec: 3822.8, 300 sec: 3776.6). Total num frames: 2453504. Throughput: 0: 973.8. Samples: 613888. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:41:50,610][01245] Avg episode reward: [(0, '4.961')] | |
| [2023-02-22 18:41:51,940][15057] Updated weights for policy 0, policy_version 600 (0.0014) | |
| [2023-02-22 18:41:55,598][01245] Fps is (10 sec: 3684.6, 60 sec: 3822.6, 300 sec: 3762.7). Total num frames: 2465792. Throughput: 0: 942.7. Samples: 616054. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:41:55,607][01245] Avg episode reward: [(0, '5.074')] | |
| [2023-02-22 18:42:00,594][01245] Fps is (10 sec: 2869.1, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2482176. Throughput: 0: 925.8. Samples: 620442. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:42:00,600][01245] Avg episode reward: [(0, '5.041')] | |
| [2023-02-22 18:42:03,585][15057] Updated weights for policy 0, policy_version 610 (0.0018) | |
| [2023-02-22 18:42:05,594][01245] Fps is (10 sec: 4098.0, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2506752. Throughput: 0: 977.1. Samples: 627222. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:42:05,599][01245] Avg episode reward: [(0, '4.935')] | |
| [2023-02-22 18:42:10,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2527232. Throughput: 0: 977.7. Samples: 630678. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:42:10,601][01245] Avg episode reward: [(0, '4.765')] | |
| [2023-02-22 18:42:10,618][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000617_2527232.pth... | |
| [2023-02-22 18:42:10,784][15039] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000398_1630208.pth | |
| [2023-02-22 18:42:13,694][15057] Updated weights for policy 0, policy_version 620 (0.0018) | |
| [2023-02-22 18:42:15,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3823.2, 300 sec: 3776.7). Total num frames: 2543616. Throughput: 0: 938.5. Samples: 635884. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:42:15,604][01245] Avg episode reward: [(0, '4.801')] | |
| [2023-02-22 18:42:20,594][01245] Fps is (10 sec: 3276.7, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2560000. Throughput: 0: 934.6. Samples: 640528. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:42:20,597][01245] Avg episode reward: [(0, '5.031')] | |
| [2023-02-22 18:42:24,615][15057] Updated weights for policy 0, policy_version 630 (0.0021) | |
| [2023-02-22 18:42:25,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 2584576. Throughput: 0: 964.9. Samples: 644080. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:42:25,596][01245] Avg episode reward: [(0, '5.379')] | |
| [2023-02-22 18:42:25,600][15039] Saving new best policy, reward=5.379! | |
| [2023-02-22 18:42:30,594][01245] Fps is (10 sec: 4505.7, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2605056. Throughput: 0: 982.5. Samples: 651122. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:42:30,601][01245] Avg episode reward: [(0, '5.374')] | |
| [2023-02-22 18:42:35,366][15057] Updated weights for policy 0, policy_version 640 (0.0024) | |
| [2023-02-22 18:42:35,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2621440. Throughput: 0: 935.6. Samples: 655984. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:42:35,599][01245] Avg episode reward: [(0, '5.186')] | |
| [2023-02-22 18:42:40,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 2637824. Throughput: 0: 937.0. Samples: 658214. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:42:40,596][01245] Avg episode reward: [(0, '4.934')] | |
| [2023-02-22 18:42:45,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2658304. Throughput: 0: 980.2. Samples: 664550. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:42:45,601][01245] Avg episode reward: [(0, '4.900')] | |
| [2023-02-22 18:42:45,698][15057] Updated weights for policy 0, policy_version 650 (0.0014) | |
| [2023-02-22 18:42:50,597][01245] Fps is (10 sec: 4503.9, 60 sec: 3823.1, 300 sec: 3776.6). Total num frames: 2682880. Throughput: 0: 982.7. Samples: 671446. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:42:50,601][01245] Avg episode reward: [(0, '5.090')] | |
| [2023-02-22 18:42:55,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3823.2, 300 sec: 3762.8). Total num frames: 2695168. Throughput: 0: 955.1. Samples: 673656. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:42:55,601][01245] Avg episode reward: [(0, '5.365')] | |
| [2023-02-22 18:42:57,173][15057] Updated weights for policy 0, policy_version 660 (0.0020) | |
| [2023-02-22 18:43:00,594][01245] Fps is (10 sec: 2868.3, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 2711552. Throughput: 0: 940.4. Samples: 678200. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:43:00,599][01245] Avg episode reward: [(0, '5.134')] | |
| [2023-02-22 18:43:05,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2736128. Throughput: 0: 991.8. Samples: 685160. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:43:05,599][01245] Avg episode reward: [(0, '5.119')] | |
| [2023-02-22 18:43:06,652][15057] Updated weights for policy 0, policy_version 670 (0.0018) | |
| [2023-02-22 18:43:10,594][01245] Fps is (10 sec: 4915.2, 60 sec: 3891.2, 300 sec: 3776.7). Total num frames: 2760704. Throughput: 0: 992.4. Samples: 688738. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:43:10,604][01245] Avg episode reward: [(0, '5.301')] | |
| [2023-02-22 18:43:15,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3762.8). Total num frames: 2772992. Throughput: 0: 949.2. Samples: 693834. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:43:15,601][01245] Avg episode reward: [(0, '5.671')] | |
| [2023-02-22 18:43:15,638][15039] Saving new best policy, reward=5.671! | |
| [2023-02-22 18:43:18,415][15057] Updated weights for policy 0, policy_version 680 (0.0023) | |
| [2023-02-22 18:43:20,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2789376. Throughput: 0: 944.9. Samples: 698504. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:43:20,601][01245] Avg episode reward: [(0, '5.757')] | |
| [2023-02-22 18:43:20,615][15039] Saving new best policy, reward=5.757! | |
| [2023-02-22 18:43:25,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 2813952. Throughput: 0: 974.1. Samples: 702050. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:43:25,596][01245] Avg episode reward: [(0, '5.167')] | |
| [2023-02-22 18:43:27,477][15057] Updated weights for policy 0, policy_version 690 (0.0032) | |
| [2023-02-22 18:43:30,594][01245] Fps is (10 sec: 4915.2, 60 sec: 3891.2, 300 sec: 3790.5). Total num frames: 2838528. Throughput: 0: 996.4. Samples: 709386. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:43:30,597][01245] Avg episode reward: [(0, '5.187')] | |
| [2023-02-22 18:43:35,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3790.6). Total num frames: 2854912. Throughput: 0: 951.0. Samples: 714236. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:43:35,602][01245] Avg episode reward: [(0, '5.155')] | |
| [2023-02-22 18:43:39,403][15057] Updated weights for policy 0, policy_version 700 (0.0014) | |
| [2023-02-22 18:43:40,594][01245] Fps is (10 sec: 3276.7, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 2871296. Throughput: 0: 952.5. Samples: 716518. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:43:40,596][01245] Avg episode reward: [(0, '4.910')] | |
| [2023-02-22 18:43:45,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3959.5, 300 sec: 3776.7). Total num frames: 2895872. Throughput: 0: 997.6. Samples: 723092. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:43:45,598][01245] Avg episode reward: [(0, '4.841')] | |
| [2023-02-22 18:43:48,115][15057] Updated weights for policy 0, policy_version 710 (0.0034) | |
| [2023-02-22 18:43:50,594][01245] Fps is (10 sec: 4505.8, 60 sec: 3891.4, 300 sec: 3790.5). Total num frames: 2916352. Throughput: 0: 993.4. Samples: 729862. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:43:50,596][01245] Avg episode reward: [(0, '5.354')] | |
| [2023-02-22 18:43:55,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3959.5, 300 sec: 3790.5). Total num frames: 2932736. Throughput: 0: 965.0. Samples: 732164. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:43:55,602][01245] Avg episode reward: [(0, '5.491')] | |
| [2023-02-22 18:44:00,215][15057] Updated weights for policy 0, policy_version 720 (0.0025) | |
| [2023-02-22 18:44:00,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3959.5, 300 sec: 3762.8). Total num frames: 2949120. Throughput: 0: 956.5. Samples: 736878. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:44:00,598][01245] Avg episode reward: [(0, '5.624')] | |
| [2023-02-22 18:44:05,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 2969600. Throughput: 0: 996.2. Samples: 743334. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:44:05,596][01245] Avg episode reward: [(0, '5.642')] | |
| [2023-02-22 18:44:09,325][15057] Updated weights for policy 0, policy_version 730 (0.0015) | |
| [2023-02-22 18:44:10,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3776.7). Total num frames: 2990080. Throughput: 0: 995.0. Samples: 746826. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:44:10,596][01245] Avg episode reward: [(0, '6.050')] | |
| [2023-02-22 18:44:10,665][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000731_2994176.pth... | |
| [2023-02-22 18:44:10,806][15039] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000510_2088960.pth | |
| [2023-02-22 18:44:10,825][15039] Saving new best policy, reward=6.050! | |
| [2023-02-22 18:44:15,597][01245] Fps is (10 sec: 3275.6, 60 sec: 3822.7, 300 sec: 3762.7). Total num frames: 3002368. Throughput: 0: 928.1. Samples: 751152. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:44:15,604][01245] Avg episode reward: [(0, '6.124')] | |
| [2023-02-22 18:44:15,656][15039] Saving new best policy, reward=6.124! | |
| [2023-02-22 18:44:20,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 3018752. Throughput: 0: 921.9. Samples: 755720. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:44:20,602][01245] Avg episode reward: [(0, '6.291')] | |
| [2023-02-22 18:44:20,612][15039] Saving new best policy, reward=6.291! | |
| [2023-02-22 18:44:22,629][15057] Updated weights for policy 0, policy_version 740 (0.0017) | |
| [2023-02-22 18:44:25,594][01245] Fps is (10 sec: 4097.5, 60 sec: 3822.9, 300 sec: 3748.9). Total num frames: 3043328. Throughput: 0: 948.1. Samples: 759180. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:44:25,598][01245] Avg episode reward: [(0, '6.194')] | |
| [2023-02-22 18:44:30,594][01245] Fps is (10 sec: 4915.2, 60 sec: 3822.9, 300 sec: 3790.5). Total num frames: 3067904. Throughput: 0: 960.9. Samples: 766334. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:44:30,599][01245] Avg episode reward: [(0, '6.597')] | |
| [2023-02-22 18:44:30,609][15039] Saving new best policy, reward=6.597! | |
| [2023-02-22 18:44:32,069][15057] Updated weights for policy 0, policy_version 750 (0.0024) | |
| [2023-02-22 18:44:35,597][01245] Fps is (10 sec: 3685.0, 60 sec: 3754.4, 300 sec: 3776.6). Total num frames: 3080192. Throughput: 0: 913.9. Samples: 770992. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:44:35,600][01245] Avg episode reward: [(0, '6.842')] | |
| [2023-02-22 18:44:35,609][15039] Saving new best policy, reward=6.842! | |
| [2023-02-22 18:44:40,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3754.7, 300 sec: 3748.9). Total num frames: 3096576. Throughput: 0: 908.2. Samples: 773032. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:44:40,601][01245] Avg episode reward: [(0, '6.904')] | |
| [2023-02-22 18:44:40,613][15039] Saving new best policy, reward=6.904! | |
| [2023-02-22 18:44:44,160][15057] Updated weights for policy 0, policy_version 760 (0.0024) | |
| [2023-02-22 18:44:45,594][01245] Fps is (10 sec: 3687.6, 60 sec: 3686.4, 300 sec: 3749.0). Total num frames: 3117056. Throughput: 0: 936.3. Samples: 779010. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:44:45,603][01245] Avg episode reward: [(0, '7.115')] | |
| [2023-02-22 18:44:45,608][15039] Saving new best policy, reward=7.115! | |
| [2023-02-22 18:44:50,596][01245] Fps is (10 sec: 4094.9, 60 sec: 3686.2, 300 sec: 3762.7). Total num frames: 3137536. Throughput: 0: 937.9. Samples: 785540. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:44:50,600][01245] Avg episode reward: [(0, '6.755')] | |
| [2023-02-22 18:44:55,595][01245] Fps is (10 sec: 3276.3, 60 sec: 3618.0, 300 sec: 3748.9). Total num frames: 3149824. Throughput: 0: 902.6. Samples: 787444. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:44:55,600][01245] Avg episode reward: [(0, '7.164')] | |
| [2023-02-22 18:44:55,607][15039] Saving new best policy, reward=7.164! | |
| [2023-02-22 18:44:56,755][15057] Updated weights for policy 0, policy_version 770 (0.0023) | |
| [2023-02-22 18:45:00,595][01245] Fps is (10 sec: 2048.3, 60 sec: 3481.5, 300 sec: 3693.3). Total num frames: 3158016. Throughput: 0: 872.8. Samples: 790428. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:45:00,603][01245] Avg episode reward: [(0, '7.473')] | |
| [2023-02-22 18:45:00,614][15039] Saving new best policy, reward=7.473! | |
| [2023-02-22 18:45:05,594][01245] Fps is (10 sec: 2048.3, 60 sec: 3345.1, 300 sec: 3665.6). Total num frames: 3170304. Throughput: 0: 836.5. Samples: 793362. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:45:05,597][01245] Avg episode reward: [(0, '7.367')] | |
| [2023-02-22 18:45:10,594][01245] Fps is (10 sec: 2867.6, 60 sec: 3276.8, 300 sec: 3679.5). Total num frames: 3186688. Throughput: 0: 810.3. Samples: 795644. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:45:10,597][01245] Avg episode reward: [(0, '8.313')] | |
| [2023-02-22 18:45:10,699][15039] Saving new best policy, reward=8.313! | |
| [2023-02-22 18:45:11,689][15057] Updated weights for policy 0, policy_version 780 (0.0031) | |
| [2023-02-22 18:45:15,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3481.8, 300 sec: 3721.1). Total num frames: 3211264. Throughput: 0: 798.0. Samples: 802244. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:45:15,602][01245] Avg episode reward: [(0, '8.678')] | |
| [2023-02-22 18:45:15,604][15039] Saving new best policy, reward=8.678! | |
| [2023-02-22 18:45:20,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3707.2). Total num frames: 3223552. Throughput: 0: 797.6. Samples: 806882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:45:20,601][01245] Avg episode reward: [(0, '8.683')] | |
| [2023-02-22 18:45:20,611][15039] Saving new best policy, reward=8.683! | |
| [2023-02-22 18:45:23,863][15057] Updated weights for policy 0, policy_version 790 (0.0026) | |
| [2023-02-22 18:45:25,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3276.8, 300 sec: 3707.2). Total num frames: 3239936. Throughput: 0: 800.7. Samples: 809062. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:45:25,596][01245] Avg episode reward: [(0, '8.073')] | |
| [2023-02-22 18:45:30,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3276.8, 300 sec: 3735.0). Total num frames: 3264512. Throughput: 0: 807.9. Samples: 815366. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:45:30,602][01245] Avg episode reward: [(0, '7.897')] | |
| [2023-02-22 18:45:33,194][15057] Updated weights for policy 0, policy_version 800 (0.0012) | |
| [2023-02-22 18:45:35,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3413.5, 300 sec: 3748.9). Total num frames: 3284992. Throughput: 0: 814.8. Samples: 822202. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:45:35,598][01245] Avg episode reward: [(0, '8.396')] | |
| [2023-02-22 18:45:40,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3413.3, 300 sec: 3735.0). Total num frames: 3301376. Throughput: 0: 821.4. Samples: 824404. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:45:40,600][01245] Avg episode reward: [(0, '9.292')] | |
| [2023-02-22 18:45:40,616][15039] Saving new best policy, reward=9.292! | |
| [2023-02-22 18:45:45,551][15057] Updated weights for policy 0, policy_version 810 (0.0025) | |
| [2023-02-22 18:45:45,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3345.1, 300 sec: 3707.3). Total num frames: 3317760. Throughput: 0: 851.5. Samples: 828744. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:45:45,597][01245] Avg episode reward: [(0, '9.482')] | |
| [2023-02-22 18:45:45,604][15039] Saving new best policy, reward=9.482! | |
| [2023-02-22 18:45:50,594][01245] Fps is (10 sec: 3686.3, 60 sec: 3345.2, 300 sec: 3735.0). Total num frames: 3338240. Throughput: 0: 938.5. Samples: 835594. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:45:50,599][01245] Avg episode reward: [(0, '9.601')] | |
| [2023-02-22 18:45:50,613][15039] Saving new best policy, reward=9.601! | |
| [2023-02-22 18:45:54,327][15057] Updated weights for policy 0, policy_version 820 (0.0012) | |
| [2023-02-22 18:45:55,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3550.0, 300 sec: 3762.8). Total num frames: 3362816. Throughput: 0: 964.2. Samples: 839034. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:45:55,596][01245] Avg episode reward: [(0, '8.585')] | |
| [2023-02-22 18:46:00,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3618.2, 300 sec: 3721.1). Total num frames: 3375104. Throughput: 0: 933.2. Samples: 844238. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:46:00,603][01245] Avg episode reward: [(0, '8.321')] | |
| [2023-02-22 18:46:05,594][01245] Fps is (10 sec: 2867.2, 60 sec: 3686.4, 300 sec: 3707.2). Total num frames: 3391488. Throughput: 0: 929.8. Samples: 848724. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:46:05,596][01245] Avg episode reward: [(0, '8.667')] | |
| [2023-02-22 18:46:06,875][15057] Updated weights for policy 0, policy_version 830 (0.0016) | |
| [2023-02-22 18:46:10,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 3416064. Throughput: 0: 960.0. Samples: 852262. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:46:10,596][01245] Avg episode reward: [(0, '8.954')] | |
| [2023-02-22 18:46:10,607][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000834_3416064.pth... | |
| [2023-02-22 18:46:10,740][15039] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000617_2527232.pth | |
| [2023-02-22 18:46:15,596][01245] Fps is (10 sec: 4504.4, 60 sec: 3754.5, 300 sec: 3748.8). Total num frames: 3436544. Throughput: 0: 974.3. Samples: 859212. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:46:15,601][01245] Avg episode reward: [(0, '9.443')] | |
| [2023-02-22 18:46:15,839][15057] Updated weights for policy 0, policy_version 840 (0.0023) | |
| [2023-02-22 18:46:20,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 3452928. Throughput: 0: 928.0. Samples: 863960. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:46:20,596][01245] Avg episode reward: [(0, '9.179')] | |
| [2023-02-22 18:46:25,594][01245] Fps is (10 sec: 3277.7, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 3469312. Throughput: 0: 928.6. Samples: 866190. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:46:25,596][01245] Avg episode reward: [(0, '8.944')] | |
| [2023-02-22 18:46:27,872][15057] Updated weights for policy 0, policy_version 850 (0.0019) | |
| [2023-02-22 18:46:30,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 3493888. Throughput: 0: 976.0. Samples: 872664. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-02-22 18:46:30,597][01245] Avg episode reward: [(0, '8.765')] | |
| [2023-02-22 18:46:35,598][01245] Fps is (10 sec: 4503.5, 60 sec: 3822.6, 300 sec: 3762.7). Total num frames: 3514368. Throughput: 0: 977.1. Samples: 879570. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:46:35,603][01245] Avg episode reward: [(0, '9.319')] | |
| [2023-02-22 18:46:37,367][15057] Updated weights for policy 0, policy_version 860 (0.0025) | |
| [2023-02-22 18:46:40,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 3530752. Throughput: 0: 951.1. Samples: 881832. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:46:40,603][01245] Avg episode reward: [(0, '9.505')] | |
| [2023-02-22 18:46:45,594][01245] Fps is (10 sec: 3278.3, 60 sec: 3822.9, 300 sec: 3707.3). Total num frames: 3547136. Throughput: 0: 932.5. Samples: 886202. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:46:45,600][01245] Avg episode reward: [(0, '10.274')] | |
| [2023-02-22 18:46:45,607][15039] Saving new best policy, reward=10.274! | |
| [2023-02-22 18:46:49,069][15057] Updated weights for policy 0, policy_version 870 (0.0018) | |
| [2023-02-22 18:46:50,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3735.1). Total num frames: 3567616. Throughput: 0: 979.1. Samples: 892784. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:46:50,602][01245] Avg episode reward: [(0, '10.256')] | |
| [2023-02-22 18:46:55,595][01245] Fps is (10 sec: 4504.8, 60 sec: 3822.8, 300 sec: 3762.7). Total num frames: 3592192. Throughput: 0: 978.7. Samples: 896304. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:46:55,602][01245] Avg episode reward: [(0, '10.309')] | |
| [2023-02-22 18:46:55,606][15039] Saving new best policy, reward=10.309! | |
| [2023-02-22 18:46:59,481][15057] Updated weights for policy 0, policy_version 880 (0.0026) | |
| [2023-02-22 18:47:00,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 3604480. Throughput: 0: 943.6. Samples: 901670. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-02-22 18:47:00,598][01245] Avg episode reward: [(0, '10.353')] | |
| [2023-02-22 18:47:00,612][15039] Saving new best policy, reward=10.353! | |
| [2023-02-22 18:47:05,594][01245] Fps is (10 sec: 2867.7, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 3620864. Throughput: 0: 938.4. Samples: 906190. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:47:05,601][01245] Avg episode reward: [(0, '10.590')] | |
| [2023-02-22 18:47:05,607][15039] Saving new best policy, reward=10.590! | |
| [2023-02-22 18:47:10,218][15057] Updated weights for policy 0, policy_version 890 (0.0025) | |
| [2023-02-22 18:47:10,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 3645440. Throughput: 0: 967.8. Samples: 909740. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:47:10,596][01245] Avg episode reward: [(0, '11.236')] | |
| [2023-02-22 18:47:10,604][15039] Saving new best policy, reward=11.236! | |
| [2023-02-22 18:47:15,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3823.1, 300 sec: 3748.9). Total num frames: 3665920. Throughput: 0: 980.9. Samples: 916806. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:47:15,606][01245] Avg episode reward: [(0, '11.866')] | |
| [2023-02-22 18:47:15,610][15039] Saving new best policy, reward=11.866! | |
| [2023-02-22 18:47:20,594][01245] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 3682304. Throughput: 0: 932.8. Samples: 921540. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-02-22 18:47:20,599][01245] Avg episode reward: [(0, '12.075')] | |
| [2023-02-22 18:47:20,611][15039] Saving new best policy, reward=12.075! | |
| [2023-02-22 18:47:21,354][15057] Updated weights for policy 0, policy_version 900 (0.0013) | |
| [2023-02-22 18:47:25,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3822.9, 300 sec: 3707.2). Total num frames: 3698688. Throughput: 0: 931.6. Samples: 923752. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:47:25,600][01245] Avg episode reward: [(0, '11.108')] | |
| [2023-02-22 18:47:30,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 3723264. Throughput: 0: 977.6. Samples: 930194. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:47:30,602][01245] Avg episode reward: [(0, '11.198')] | |
| [2023-02-22 18:47:31,125][15057] Updated weights for policy 0, policy_version 910 (0.0020) | |
| [2023-02-22 18:47:35,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3823.2, 300 sec: 3748.9). Total num frames: 3743744. Throughput: 0: 988.9. Samples: 937284. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:47:35,599][01245] Avg episode reward: [(0, '10.621')] | |
| [2023-02-22 18:47:40,594][01245] Fps is (10 sec: 3686.3, 60 sec: 3822.9, 300 sec: 3735.0). Total num frames: 3760128. Throughput: 0: 960.5. Samples: 939526. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-22 18:47:40,599][01245] Avg episode reward: [(0, '10.889')] | |
| [2023-02-22 18:47:42,561][15057] Updated weights for policy 0, policy_version 920 (0.0022) | |
| [2023-02-22 18:47:45,595][01245] Fps is (10 sec: 3276.3, 60 sec: 3822.8, 300 sec: 3707.3). Total num frames: 3776512. Throughput: 0: 941.7. Samples: 944046. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:47:45,602][01245] Avg episode reward: [(0, '11.553')] | |
| [2023-02-22 18:47:50,594][01245] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 3801088. Throughput: 0: 993.6. Samples: 950900. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:47:50,596][01245] Avg episode reward: [(0, '11.358')] | |
| [2023-02-22 18:47:52,061][15057] Updated weights for policy 0, policy_version 930 (0.0034) | |
| [2023-02-22 18:47:55,598][01245] Fps is (10 sec: 4913.7, 60 sec: 3891.0, 300 sec: 3776.6). Total num frames: 3825664. Throughput: 0: 993.9. Samples: 954472. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:47:55,602][01245] Avg episode reward: [(0, '11.286')] | |
| [2023-02-22 18:48:00,594][01245] Fps is (10 sec: 3686.3, 60 sec: 3891.2, 300 sec: 3735.0). Total num frames: 3837952. Throughput: 0: 956.5. Samples: 959850. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:48:00,596][01245] Avg episode reward: [(0, '11.653')] | |
| [2023-02-22 18:48:03,797][15057] Updated weights for policy 0, policy_version 940 (0.0021) | |
| [2023-02-22 18:48:05,594][01245] Fps is (10 sec: 2868.5, 60 sec: 3891.2, 300 sec: 3707.2). Total num frames: 3854336. Throughput: 0: 951.3. Samples: 964348. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2023-02-22 18:48:05,600][01245] Avg episode reward: [(0, '11.339')] | |
| [2023-02-22 18:48:10,594][01245] Fps is (10 sec: 4096.1, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 3878912. Throughput: 0: 980.8. Samples: 967890. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-02-22 18:48:10,596][01245] Avg episode reward: [(0, '11.033')] | |
| [2023-02-22 18:48:10,604][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000947_3878912.pth... | |
| [2023-02-22 18:48:10,736][15039] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000731_2994176.pth | |
| [2023-02-22 18:48:13,020][15057] Updated weights for policy 0, policy_version 950 (0.0014) | |
| [2023-02-22 18:48:15,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3762.8). Total num frames: 3899392. Throughput: 0: 991.7. Samples: 974822. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:48:15,599][01245] Avg episode reward: [(0, '11.520')] | |
| [2023-02-22 18:48:20,596][01245] Fps is (10 sec: 3685.4, 60 sec: 3891.0, 300 sec: 3735.0). Total num frames: 3915776. Throughput: 0: 940.2. Samples: 979594. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-02-22 18:48:20,601][01245] Avg episode reward: [(0, '11.486')] | |
| [2023-02-22 18:48:25,573][15057] Updated weights for policy 0, policy_version 960 (0.0024) | |
| [2023-02-22 18:48:25,594][01245] Fps is (10 sec: 3276.8, 60 sec: 3891.2, 300 sec: 3707.2). Total num frames: 3932160. Throughput: 0: 940.5. Samples: 981850. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:48:25,596][01245] Avg episode reward: [(0, '11.949')] | |
| [2023-02-22 18:48:30,594][01245] Fps is (10 sec: 3687.4, 60 sec: 3822.9, 300 sec: 3721.1). Total num frames: 3952640. Throughput: 0: 982.3. Samples: 988248. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:48:30,597][01245] Avg episode reward: [(0, '12.353')] | |
| [2023-02-22 18:48:30,667][15039] Saving new best policy, reward=12.353! | |
| [2023-02-22 18:48:34,027][15057] Updated weights for policy 0, policy_version 970 (0.0012) | |
| [2023-02-22 18:48:35,594][01245] Fps is (10 sec: 4505.6, 60 sec: 3891.2, 300 sec: 3748.9). Total num frames: 3977216. Throughput: 0: 987.7. Samples: 995348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:48:35,605][01245] Avg episode reward: [(0, '13.230')] | |
| [2023-02-22 18:48:35,613][15039] Saving new best policy, reward=13.230! | |
| [2023-02-22 18:48:40,594][01245] Fps is (10 sec: 4096.0, 60 sec: 3891.2, 300 sec: 3721.1). Total num frames: 3993600. Throughput: 0: 955.7. Samples: 997472. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-22 18:48:40,596][01245] Avg episode reward: [(0, '13.192')] | |
| [2023-02-22 18:48:44,303][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-02-22 18:48:44,311][01245] Component Batcher_0 stopped! | |
| [2023-02-22 18:48:44,310][15039] Stopping Batcher_0... | |
| [2023-02-22 18:48:44,319][15039] Loop batcher_evt_loop terminating... | |
| [2023-02-22 18:48:44,371][15057] Weights refcount: 2 0 | |
| [2023-02-22 18:48:44,383][01245] Component InferenceWorker_p0-w0 stopped! | |
| [2023-02-22 18:48:44,390][15057] Stopping InferenceWorker_p0-w0... | |
| [2023-02-22 18:48:44,390][15057] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-02-22 18:48:44,394][01245] Component RolloutWorker_w6 stopped! | |
| [2023-02-22 18:48:44,398][15064] Stopping RolloutWorker_w6... | |
| [2023-02-22 18:48:44,399][15064] Loop rollout_proc6_evt_loop terminating... | |
| [2023-02-22 18:48:44,412][15060] Stopping RolloutWorker_w2... | |
| [2023-02-22 18:48:44,413][15060] Loop rollout_proc2_evt_loop terminating... | |
| [2023-02-22 18:48:44,409][01245] Component RolloutWorker_w2 stopped! | |
| [2023-02-22 18:48:44,457][15063] Stopping RolloutWorker_w3... | |
| [2023-02-22 18:48:44,458][01245] Component RolloutWorker_w3 stopped! | |
| [2023-02-22 18:48:44,469][15061] Stopping RolloutWorker_w5... | |
| [2023-02-22 18:48:44,470][01245] Component RolloutWorker_w5 stopped! | |
| [2023-02-22 18:48:44,476][01245] Component RolloutWorker_w4 stopped! | |
| [2023-02-22 18:48:44,480][15062] Stopping RolloutWorker_w4... | |
| [2023-02-22 18:48:44,480][15062] Loop rollout_proc4_evt_loop terminating... | |
| [2023-02-22 18:48:44,479][15059] Stopping RolloutWorker_w1... | |
| [2023-02-22 18:48:44,485][15059] Loop rollout_proc1_evt_loop terminating... | |
| [2023-02-22 18:48:44,484][01245] Component RolloutWorker_w1 stopped! | |
| [2023-02-22 18:48:44,470][15061] Loop rollout_proc5_evt_loop terminating... | |
| [2023-02-22 18:48:44,500][15058] Stopping RolloutWorker_w0... | |
| [2023-02-22 18:48:44,500][15058] Loop rollout_proc0_evt_loop terminating... | |
| [2023-02-22 18:48:44,458][15063] Loop rollout_proc3_evt_loop terminating... | |
| [2023-02-22 18:48:44,503][01245] Component RolloutWorker_w0 stopped! | |
| [2023-02-22 18:48:44,552][15039] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000834_3416064.pth | |
| [2023-02-22 18:48:44,573][15039] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-02-22 18:48:44,604][01245] Component RolloutWorker_w7 stopped! | |
| [2023-02-22 18:48:44,613][15065] Stopping RolloutWorker_w7... | |
| [2023-02-22 18:48:44,620][15065] Loop rollout_proc7_evt_loop terminating... | |
| [2023-02-22 18:48:44,795][01245] Component LearnerWorker_p0 stopped! | |
| [2023-02-22 18:48:44,799][01245] Waiting for process learner_proc0 to stop... | |
| [2023-02-22 18:48:44,802][15039] Stopping LearnerWorker_p0... | |
| [2023-02-22 18:48:44,803][15039] Loop learner_proc0_evt_loop terminating... | |
| [2023-02-22 18:48:46,695][01245] Waiting for process inference_proc0-0 to join... | |
| [2023-02-22 18:48:47,062][01245] Waiting for process rollout_proc0 to join... | |
| [2023-02-22 18:48:47,498][01245] Waiting for process rollout_proc1 to join... | |
| [2023-02-22 18:48:47,500][01245] Waiting for process rollout_proc2 to join... | |
| [2023-02-22 18:48:47,501][01245] Waiting for process rollout_proc3 to join... | |
| [2023-02-22 18:48:47,503][01245] Waiting for process rollout_proc4 to join... | |
| [2023-02-22 18:48:47,504][01245] Waiting for process rollout_proc5 to join... | |
| [2023-02-22 18:48:47,505][01245] Waiting for process rollout_proc6 to join... | |
| [2023-02-22 18:48:47,511][01245] Waiting for process rollout_proc7 to join... | |
| [2023-02-22 18:48:47,512][01245] Batcher 0 profile tree view: | |
| batching: 24.8797, releasing_batches: 0.0222 | |
| [2023-02-22 18:48:47,514][01245] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0138 | |
| wait_policy_total: 524.3641 | |
| update_model: 7.7211 | |
| weight_update: 0.0016 | |
| one_step: 0.0040 | |
| handle_policy_step: 511.1373 | |
| deserialize: 14.4846, stack: 2.9721, obs_to_device_normalize: 114.4620, forward: 244.7572, send_messages: 26.4607 | |
| prepare_outputs: 81.9385 | |
| to_cpu: 50.7951 | |
| [2023-02-22 18:48:47,515][01245] Learner 0 profile tree view: | |
| misc: 0.0060, prepare_batch: 16.4633 | |
| train: 76.3437 | |
| epoch_init: 0.0106, minibatch_init: 0.0184, losses_postprocess: 0.5721, kl_divergence: 0.6301, after_optimizer: 32.9371 | |
| calculate_losses: 27.4845 | |
| losses_init: 0.0065, forward_head: 1.7713, bptt_initial: 18.0713, tail: 1.0499, advantages_returns: 0.2806, losses: 3.6938 | |
| bptt: 2.3116 | |
| bptt_forward_core: 2.1994 | |
| update: 14.0724 | |
| clip: 1.3864 | |
| [2023-02-22 18:48:47,516][01245] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.3414, enqueue_policy_requests: 140.1474, env_step: 818.3112, overhead: 20.0871, complete_rollouts: 6.6500 | |
| save_policy_outputs: 19.8322 | |
| split_output_tensors: 9.6844 | |
| [2023-02-22 18:48:47,518][01245] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.3328, enqueue_policy_requests: 140.2637, env_step: 817.0888, overhead: 20.3039, complete_rollouts: 7.3417 | |
| save_policy_outputs: 19.3221 | |
| split_output_tensors: 9.2830 | |
| [2023-02-22 18:48:47,520][01245] Loop Runner_EvtLoop terminating... | |
| [2023-02-22 18:48:47,522][01245] Runner profile tree view: | |
| main_loop: 1112.2975 | |
| [2023-02-22 18:48:47,523][01245] Collected {0: 4005888}, FPS: 3601.5 | |
| [2023-02-22 18:48:47,656][01245] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2023-02-22 18:48:47,658][01245] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-02-22 18:48:47,659][01245] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-02-22 18:48:47,661][01245] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-02-22 18:48:47,662][01245] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-22 18:48:47,663][01245] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-02-22 18:48:47,664][01245] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-22 18:48:47,666][01245] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-02-22 18:48:47,667][01245] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2023-02-22 18:48:47,669][01245] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2023-02-22 18:48:47,670][01245] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-02-22 18:48:47,671][01245] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-02-22 18:48:47,672][01245] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-02-22 18:48:47,673][01245] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-02-22 18:48:47,674][01245] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-02-22 18:48:47,705][01245] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-22 18:48:47,711][01245] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-22 18:48:47,713][01245] RunningMeanStd input shape: (1,) | |
| [2023-02-22 18:48:47,730][01245] ConvEncoder: input_channels=3 | |
| [2023-02-22 18:48:48,421][01245] Conv encoder output size: 512 | |
| [2023-02-22 18:48:48,422][01245] Policy head output size: 512 | |
| [2023-02-22 18:48:50,843][01245] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-02-22 18:48:52,110][01245] Num frames 100... | |
| [2023-02-22 18:48:52,223][01245] Num frames 200... | |
| [2023-02-22 18:48:52,337][01245] Num frames 300... | |
| [2023-02-22 18:48:52,458][01245] Num frames 400... | |
| [2023-02-22 18:48:52,578][01245] Num frames 500... | |
| [2023-02-22 18:48:52,693][01245] Num frames 600... | |
| [2023-02-22 18:48:52,812][01245] Num frames 700... | |
| [2023-02-22 18:48:52,931][01245] Num frames 800... | |
| [2023-02-22 18:48:53,094][01245] Avg episode rewards: #0: 16.960, true rewards: #0: 8.960 | |
| [2023-02-22 18:48:53,097][01245] Avg episode reward: 16.960, avg true_objective: 8.960 | |
| [2023-02-22 18:48:53,106][01245] Num frames 900... | |
| [2023-02-22 18:48:53,217][01245] Num frames 1000... | |
| [2023-02-22 18:48:53,331][01245] Num frames 1100... | |
| [2023-02-22 18:48:53,450][01245] Num frames 1200... | |
| [2023-02-22 18:48:53,569][01245] Num frames 1300... | |
| [2023-02-22 18:48:53,682][01245] Num frames 1400... | |
| [2023-02-22 18:48:53,796][01245] Num frames 1500... | |
| [2023-02-22 18:48:53,914][01245] Num frames 1600... | |
| [2023-02-22 18:48:54,031][01245] Num frames 1700... | |
| [2023-02-22 18:48:54,118][01245] Avg episode rewards: #0: 17.140, true rewards: #0: 8.640 | |
| [2023-02-22 18:48:54,119][01245] Avg episode reward: 17.140, avg true_objective: 8.640 | |
| [2023-02-22 18:48:54,202][01245] Num frames 1800... | |
| [2023-02-22 18:48:54,314][01245] Num frames 1900... | |
| [2023-02-22 18:48:54,431][01245] Num frames 2000... | |
| [2023-02-22 18:48:54,548][01245] Num frames 2100... | |
| [2023-02-22 18:48:54,664][01245] Num frames 2200... | |
| [2023-02-22 18:48:54,737][01245] Avg episode rewards: #0: 14.710, true rewards: #0: 7.377 | |
| [2023-02-22 18:48:54,738][01245] Avg episode reward: 14.710, avg true_objective: 7.377 | |
| [2023-02-22 18:48:54,846][01245] Num frames 2300... | |
| [2023-02-22 18:48:54,961][01245] Num frames 2400... | |
| [2023-02-22 18:48:55,071][01245] Num frames 2500... | |
| [2023-02-22 18:48:55,233][01245] Num frames 2600... | |
| [2023-02-22 18:48:55,412][01245] Avg episode rewards: #0: 13.438, true rewards: #0: 6.687 | |
| [2023-02-22 18:48:55,415][01245] Avg episode reward: 13.438, avg true_objective: 6.687 | |
| [2023-02-22 18:48:55,458][01245] Num frames 2700... | |
| [2023-02-22 18:48:55,631][01245] Num frames 2800... | |
| [2023-02-22 18:48:55,790][01245] Num frames 2900... | |
| [2023-02-22 18:48:55,948][01245] Num frames 3000... | |
| [2023-02-22 18:48:56,111][01245] Num frames 3100... | |
| [2023-02-22 18:48:56,272][01245] Num frames 3200... | |
| [2023-02-22 18:48:56,427][01245] Num frames 3300... | |
| [2023-02-22 18:48:56,588][01245] Num frames 3400... | |
| [2023-02-22 18:48:56,761][01245] Num frames 3500... | |
| [2023-02-22 18:48:56,883][01245] Avg episode rewards: #0: 14.278, true rewards: #0: 7.078 | |
| [2023-02-22 18:48:56,885][01245] Avg episode reward: 14.278, avg true_objective: 7.078 | |
| [2023-02-22 18:48:56,989][01245] Num frames 3600... | |
| [2023-02-22 18:48:57,151][01245] Num frames 3700... | |
| [2023-02-22 18:48:57,325][01245] Num frames 3800... | |
| [2023-02-22 18:48:57,488][01245] Num frames 3900... | |
| [2023-02-22 18:48:57,669][01245] Num frames 4000... | |
| [2023-02-22 18:48:57,837][01245] Num frames 4100... | |
| [2023-02-22 18:48:58,003][01245] Num frames 4200... | |
| [2023-02-22 18:48:58,168][01245] Num frames 4300... | |
| [2023-02-22 18:48:58,336][01245] Num frames 4400... | |
| [2023-02-22 18:48:58,506][01245] Num frames 4500... | |
| [2023-02-22 18:48:58,661][01245] Avg episode rewards: #0: 14.938, true rewards: #0: 7.605 | |
| [2023-02-22 18:48:58,663][01245] Avg episode reward: 14.938, avg true_objective: 7.605 | |
| [2023-02-22 18:48:58,713][01245] Num frames 4600... | |
| [2023-02-22 18:48:58,824][01245] Num frames 4700... | |
| [2023-02-22 18:48:58,939][01245] Num frames 4800... | |
| [2023-02-22 18:48:59,060][01245] Num frames 4900... | |
| [2023-02-22 18:48:59,175][01245] Num frames 5000... | |
| [2023-02-22 18:48:59,288][01245] Num frames 5100... | |
| [2023-02-22 18:48:59,403][01245] Num frames 5200... | |
| [2023-02-22 18:48:59,517][01245] Avg episode rewards: #0: 14.350, true rewards: #0: 7.493 | |
| [2023-02-22 18:48:59,519][01245] Avg episode reward: 14.350, avg true_objective: 7.493 | |
| [2023-02-22 18:48:59,586][01245] Num frames 5300... | |
| [2023-02-22 18:48:59,711][01245] Num frames 5400... | |
| [2023-02-22 18:48:59,826][01245] Num frames 5500... | |
| [2023-02-22 18:48:59,956][01245] Avg episode rewards: #0: 13.331, true rewards: #0: 6.956 | |
| [2023-02-22 18:48:59,958][01245] Avg episode reward: 13.331, avg true_objective: 6.956 | |
| [2023-02-22 18:49:00,004][01245] Num frames 5600... | |
| [2023-02-22 18:49:00,119][01245] Num frames 5700... | |
| [2023-02-22 18:49:00,232][01245] Num frames 5800... | |
| [2023-02-22 18:49:00,347][01245] Num frames 5900... | |
| [2023-02-22 18:49:00,469][01245] Num frames 6000... | |
| [2023-02-22 18:49:00,585][01245] Num frames 6100... | |
| [2023-02-22 18:49:00,700][01245] Num frames 6200... | |
| [2023-02-22 18:49:00,821][01245] Num frames 6300... | |
| [2023-02-22 18:49:00,939][01245] Num frames 6400... | |
| [2023-02-22 18:49:01,054][01245] Num frames 6500... | |
| [2023-02-22 18:49:01,181][01245] Num frames 6600... | |
| [2023-02-22 18:49:01,306][01245] Num frames 6700... | |
| [2023-02-22 18:49:01,422][01245] Num frames 6800... | |
| [2023-02-22 18:49:01,537][01245] Num frames 6900... | |
| [2023-02-22 18:49:01,655][01245] Num frames 7000... | |
| [2023-02-22 18:49:01,776][01245] Num frames 7100... | |
| [2023-02-22 18:49:01,866][01245] Avg episode rewards: #0: 15.588, true rewards: #0: 7.921 | |
| [2023-02-22 18:49:01,868][01245] Avg episode reward: 15.588, avg true_objective: 7.921 | |
| [2023-02-22 18:49:01,950][01245] Num frames 7200... | |
| [2023-02-22 18:49:02,064][01245] Num frames 7300... | |
| [2023-02-22 18:49:02,183][01245] Num frames 7400... | |
| [2023-02-22 18:49:02,297][01245] Num frames 7500... | |
| [2023-02-22 18:49:02,436][01245] Avg episode rewards: #0: 14.577, true rewards: #0: 7.577 | |
| [2023-02-22 18:49:02,438][01245] Avg episode reward: 14.577, avg true_objective: 7.577 | |
| [2023-02-22 18:49:49,879][01245] Replay video saved to /content/train_dir/default_experiment/replay.mp4! | |
| [2023-02-22 18:50:45,670][01245] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json | |
| [2023-02-22 18:50:45,672][01245] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-02-22 18:50:45,674][01245] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-02-22 18:50:45,676][01245] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-02-22 18:50:45,679][01245] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-22 18:50:45,681][01245] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-02-22 18:50:45,682][01245] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-02-22 18:50:45,684][01245] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-02-22 18:50:45,687][01245] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-02-22 18:50:45,688][01245] Adding new argument 'hf_repository'='NoNameFound/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-02-22 18:50:45,690][01245] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-02-22 18:50:45,691][01245] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-02-22 18:50:45,692][01245] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-02-22 18:50:45,694][01245] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-02-22 18:50:45,695][01245] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-02-22 18:50:45,725][01245] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-22 18:50:45,727][01245] RunningMeanStd input shape: (1,) | |
| [2023-02-22 18:50:45,742][01245] ConvEncoder: input_channels=3 | |
| [2023-02-22 18:50:45,781][01245] Conv encoder output size: 512 | |
| [2023-02-22 18:50:45,783][01245] Policy head output size: 512 | |
| [2023-02-22 18:50:45,804][01245] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-02-22 18:50:46,261][01245] Num frames 100... | |
| [2023-02-22 18:50:46,379][01245] Num frames 200... | |
| [2023-02-22 18:50:46,507][01245] Num frames 300... | |
| [2023-02-22 18:50:46,630][01245] Num frames 400... | |
| [2023-02-22 18:50:46,753][01245] Num frames 500... | |
| [2023-02-22 18:50:46,877][01245] Num frames 600... | |
| [2023-02-22 18:50:47,000][01245] Num frames 700... | |
| [2023-02-22 18:50:47,062][01245] Avg episode rewards: #0: 13.040, true rewards: #0: 7.040 | |
| [2023-02-22 18:50:47,063][01245] Avg episode reward: 13.040, avg true_objective: 7.040 | |
| [2023-02-22 18:50:47,176][01245] Num frames 800... | |
| [2023-02-22 18:50:47,292][01245] Num frames 900... | |
| [2023-02-22 18:50:47,450][01245] Avg episode rewards: #0: 8.960, true rewards: #0: 4.960 | |
| [2023-02-22 18:50:47,452][01245] Avg episode reward: 8.960, avg true_objective: 4.960 | |
| [2023-02-22 18:50:47,466][01245] Num frames 1000... | |
| [2023-02-22 18:50:47,584][01245] Num frames 1100... | |
| [2023-02-22 18:50:47,718][01245] Num frames 1200... | |
| [2023-02-22 18:50:47,840][01245] Num frames 1300... | |
| [2023-02-22 18:50:47,978][01245] Num frames 1400... | |
| [2023-02-22 18:50:48,107][01245] Num frames 1500... | |
| [2023-02-22 18:50:48,205][01245] Avg episode rewards: #0: 9.120, true rewards: #0: 5.120 | |
| [2023-02-22 18:50:48,207][01245] Avg episode reward: 9.120, avg true_objective: 5.120 | |
| [2023-02-22 18:50:48,282][01245] Num frames 1600... | |
| [2023-02-22 18:50:48,399][01245] Num frames 1700... | |
| [2023-02-22 18:50:48,514][01245] Num frames 1800... | |
| [2023-02-22 18:50:48,636][01245] Num frames 1900... | |
| [2023-02-22 18:50:48,751][01245] Num frames 2000... | |
| [2023-02-22 18:50:48,865][01245] Num frames 2100... | |
| [2023-02-22 18:50:48,946][01245] Avg episode rewards: #0: 10.053, true rewards: #0: 5.302 | |
| [2023-02-22 18:50:48,947][01245] Avg episode reward: 10.053, avg true_objective: 5.302 | |
| [2023-02-22 18:50:49,045][01245] Num frames 2200... | |
| [2023-02-22 18:50:49,161][01245] Num frames 2300... | |
| [2023-02-22 18:50:49,277][01245] Num frames 2400... | |
| [2023-02-22 18:50:49,391][01245] Num frames 2500... | |
| [2023-02-22 18:50:49,510][01245] Num frames 2600... | |
| [2023-02-22 18:50:49,624][01245] Num frames 2700... | |
| [2023-02-22 18:50:49,735][01245] Num frames 2800... | |
| [2023-02-22 18:50:49,820][01245] Avg episode rewards: #0: 10.250, true rewards: #0: 5.650 | |
| [2023-02-22 18:50:49,821][01245] Avg episode reward: 10.250, avg true_objective: 5.650 | |
| [2023-02-22 18:50:49,911][01245] Num frames 2900... | |
| [2023-02-22 18:50:50,031][01245] Num frames 3000... | |
| [2023-02-22 18:50:50,153][01245] Num frames 3100... | |
| [2023-02-22 18:50:50,266][01245] Num frames 3200... | |
| [2023-02-22 18:50:50,390][01245] Num frames 3300... | |
| [2023-02-22 18:50:50,507][01245] Num frames 3400... | |
| [2023-02-22 18:50:50,621][01245] Num frames 3500... | |
| [2023-02-22 18:50:50,733][01245] Num frames 3600... | |
| [2023-02-22 18:50:50,857][01245] Num frames 3700... | |
| [2023-02-22 18:50:50,980][01245] Num frames 3800... | |
| [2023-02-22 18:50:51,098][01245] Num frames 3900... | |
| [2023-02-22 18:50:51,220][01245] Num frames 4000... | |
| [2023-02-22 18:50:51,297][01245] Avg episode rewards: #0: 12.687, true rewards: #0: 6.687 | |
| [2023-02-22 18:50:51,299][01245] Avg episode reward: 12.687, avg true_objective: 6.687 | |
| [2023-02-22 18:50:51,409][01245] Num frames 4100... | |
| [2023-02-22 18:50:51,520][01245] Num frames 4200... | |
| [2023-02-22 18:50:51,634][01245] Num frames 4300... | |
| [2023-02-22 18:50:51,747][01245] Num frames 4400... | |
| [2023-02-22 18:50:51,861][01245] Num frames 4500... | |
| [2023-02-22 18:50:51,978][01245] Num frames 4600... | |
| [2023-02-22 18:50:52,102][01245] Num frames 4700... | |
| [2023-02-22 18:50:52,222][01245] Num frames 4800... | |
| [2023-02-22 18:50:52,329][01245] Avg episode rewards: #0: 13.063, true rewards: #0: 6.920 | |
| [2023-02-22 18:50:52,331][01245] Avg episode reward: 13.063, avg true_objective: 6.920 | |
| [2023-02-22 18:50:52,398][01245] Num frames 4900... | |
| [2023-02-22 18:50:52,512][01245] Num frames 5000... | |
| [2023-02-22 18:50:52,627][01245] Num frames 5100... | |
| [2023-02-22 18:50:52,748][01245] Num frames 5200... | |
| [2023-02-22 18:50:52,861][01245] Num frames 5300... | |
| [2023-02-22 18:50:52,976][01245] Num frames 5400... | |
| [2023-02-22 18:50:53,096][01245] Avg episode rewards: #0: 12.565, true rewards: #0: 6.815 | |
| [2023-02-22 18:50:53,098][01245] Avg episode reward: 12.565, avg true_objective: 6.815 | |
| [2023-02-22 18:50:53,154][01245] Num frames 5500... | |
| [2023-02-22 18:50:53,266][01245] Num frames 5600... | |
| [2023-02-22 18:50:53,382][01245] Num frames 5700... | |
| [2023-02-22 18:50:53,500][01245] Num frames 5800... | |
| [2023-02-22 18:50:53,620][01245] Num frames 5900... | |
| [2023-02-22 18:50:53,739][01245] Num frames 6000... | |
| [2023-02-22 18:50:53,861][01245] Num frames 6100... | |
| [2023-02-22 18:50:53,980][01245] Num frames 6200... | |
| [2023-02-22 18:50:54,111][01245] Num frames 6300... | |
| [2023-02-22 18:50:54,223][01245] Avg episode rewards: #0: 13.498, true rewards: #0: 7.053 | |
| [2023-02-22 18:50:54,225][01245] Avg episode reward: 13.498, avg true_objective: 7.053 | |
| [2023-02-22 18:50:54,300][01245] Num frames 6400... | |
| [2023-02-22 18:50:54,474][01245] Num frames 6500... | |
| [2023-02-22 18:50:54,648][01245] Num frames 6600... | |
| [2023-02-22 18:50:54,810][01245] Num frames 6700... | |
| [2023-02-22 18:50:54,978][01245] Num frames 6800... | |
| [2023-02-22 18:50:55,145][01245] Num frames 6900... | |
| [2023-02-22 18:50:55,311][01245] Num frames 7000... | |
| [2023-02-22 18:50:55,491][01245] Num frames 7100... | |
| [2023-02-22 18:50:55,667][01245] Num frames 7200... | |
| [2023-02-22 18:50:55,745][01245] Avg episode rewards: #0: 13.812, true rewards: #0: 7.212 | |
| [2023-02-22 18:50:55,751][01245] Avg episode reward: 13.812, avg true_objective: 7.212 | |
| [2023-02-22 18:51:41,114][01245] Replay video saved to /content/train_dir/default_experiment/replay.mp4! | |