| [2023-02-24 07:55:59,139][784615] Saving configuration to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json... | |
| [2023-02-24 07:55:59,305][784615] Rollout worker 0 uses device cpu | |
| [2023-02-24 07:55:59,305][784615] Rollout worker 1 uses device cpu | |
| [2023-02-24 07:55:59,306][784615] Rollout worker 2 uses device cpu | |
| [2023-02-24 07:55:59,306][784615] Rollout worker 3 uses device cpu | |
| [2023-02-24 07:55:59,306][784615] Rollout worker 4 uses device cpu | |
| [2023-02-24 07:55:59,307][784615] Rollout worker 5 uses device cpu | |
| [2023-02-24 07:55:59,307][784615] Rollout worker 6 uses device cpu | |
| [2023-02-24 07:55:59,308][784615] Rollout worker 7 uses device cpu | |
| [2023-02-24 07:55:59,357][784615] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:55:59,358][784615] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-02-24 07:55:59,378][784615] Starting all processes... | |
| [2023-02-24 07:55:59,378][784615] Starting process learner_proc0 | |
| [2023-02-24 07:55:59,428][784615] Starting all processes... | |
| [2023-02-24 07:55:59,438][784615] Starting process inference_proc0-0 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc0 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc1 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc2 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc3 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc4 | |
| [2023-02-24 07:55:59,439][784615] Starting process rollout_proc5 | |
| [2023-02-24 07:55:59,440][784615] Starting process rollout_proc6 | |
| [2023-02-24 07:55:59,441][784615] Starting process rollout_proc7 | |
| [2023-02-24 07:56:00,842][794035] Worker 3 uses CPU cores [3] | |
| [2023-02-24 07:56:00,862][794036] Worker 5 uses CPU cores [5] | |
| [2023-02-24 07:56:00,978][794037] Worker 4 uses CPU cores [4] | |
| [2023-02-24 07:56:00,998][794038] Worker 2 uses CPU cores [2] | |
| [2023-02-24 07:56:00,999][794019] Low niceness requires sudo! | |
| [2023-02-24 07:56:00,999][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:56:01,000][794019] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for learning process 0 | |
| [2023-02-24 07:56:01,022][794032] Low niceness requires sudo! | |
| [2023-02-24 07:56:01,022][794032] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:56:01,022][794032] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for inference process 0 | |
| [2023-02-24 07:56:01,025][794019] Num visible devices: 1 | |
| [2023-02-24 07:56:01,040][794032] Num visible devices: 1 | |
| [2023-02-24 07:56:01,050][794019] Starting seed is not provided | |
| [2023-02-24 07:56:01,050][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:56:01,050][794019] Initializing actor-critic model on device cuda:0 | |
| [2023-02-24 07:56:01,051][794019] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 07:56:01,051][794019] RunningMeanStd input shape: (1,) | |
| [2023-02-24 07:56:01,058][794040] Worker 7 uses CPU cores [7] | |
| [2023-02-24 07:56:01,065][794019] ConvEncoder: input_channels=3 | |
| [2023-02-24 07:56:01,154][794034] Worker 1 uses CPU cores [1] | |
| [2023-02-24 07:56:01,186][794039] Worker 6 uses CPU cores [6] | |
| [2023-02-24 07:56:01,194][794033] Worker 0 uses CPU cores [0] | |
| [2023-02-24 07:56:01,198][794019] Conv encoder output size: 512 | |
| [2023-02-24 07:56:01,198][794019] Policy head output size: 512 | |
| [2023-02-24 07:56:01,216][794019] Created Actor Critic model with architecture: | |
| [2023-02-24 07:56:01,216][794019] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-02-24 07:56:04,122][794019] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-02-24 07:56:04,122][794019] No checkpoints found | |
| [2023-02-24 07:56:04,122][794019] Did not load from checkpoint, starting from scratch! | |
| [2023-02-24 07:56:04,122][794019] Initialized policy 0 weights for model version 0 | |
| [2023-02-24 07:56:04,124][794019] LearnerWorker_p0 finished initialization! | |
| [2023-02-24 07:56:04,124][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) | |
| [2023-02-24 07:56:05,229][794032] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 07:56:05,229][794032] RunningMeanStd input shape: (1,) | |
| [2023-02-24 07:56:05,237][794032] ConvEncoder: input_channels=3 | |
| [2023-02-24 07:56:05,307][794032] Conv encoder output size: 512 | |
| [2023-02-24 07:56:05,307][794032] Policy head output size: 512 | |
| [2023-02-24 07:56:06,350][784615] Inference worker 0-0 is ready! | |
| [2023-02-24 07:56:06,350][784615] All inference workers are ready! Signal rollout workers to start! | |
| [2023-02-24 07:56:06,367][794036] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,368][794039] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,368][794034] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,368][794035] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,369][794038] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,374][794040] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,390][794037] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,393][794033] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:56:06,669][794035] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,672][794039] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,678][794036] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,681][794037] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,683][794040] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,929][794035] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:06,946][794040] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:06,958][794036] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:06,977][794037] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:06,986][794038] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:06,997][794039] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:07,033][794033] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:07,227][794038] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:07,268][794034] Decorrelating experience for 0 frames... | |
| [2023-02-24 07:56:07,269][794036] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,278][794037] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,293][794033] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:07,502][794038] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,545][794039] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,584][794036] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,591][794033] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,594][794040] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:07,600][794037] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,628][794034] Decorrelating experience for 32 frames... | |
| [2023-02-24 07:56:07,822][794033] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,846][794039] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,890][794038] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:07,895][794040] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:08,065][794035] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:08,153][784615] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-02-24 07:56:08,319][794034] Decorrelating experience for 64 frames... | |
| [2023-02-24 07:56:08,572][794035] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:08,577][794034] Decorrelating experience for 96 frames... | |
| [2023-02-24 07:56:10,130][794019] Signal inference workers to stop experience collection... | |
| [2023-02-24 07:56:10,137][794032] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-02-24 07:56:12,636][794019] Signal inference workers to resume experience collection... | |
| [2023-02-24 07:56:12,637][794032] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-02-24 07:56:13,153][784615] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 4096. Throughput: 0: 494.8. Samples: 2474. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2023-02-24 07:56:13,154][784615] Avg episode reward: [(0, '2.494')] | |
| [2023-02-24 07:56:15,144][794032] Updated weights for policy 0, policy_version 10 (0.0249) | |
| [2023-02-24 07:56:17,664][794032] Updated weights for policy 0, policy_version 20 (0.0006) | |
| [2023-02-24 07:56:18,153][784615] Fps is (10 sec: 8601.6, 60 sec: 8601.6, 300 sec: 8601.6). Total num frames: 86016. Throughput: 0: 1166.6. Samples: 11666. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-24 07:56:18,154][784615] Avg episode reward: [(0, '4.437')] | |
| [2023-02-24 07:56:19,351][784615] Heartbeat connected on Batcher_0 | |
| [2023-02-24 07:56:19,354][784615] Heartbeat connected on LearnerWorker_p0 | |
| [2023-02-24 07:56:19,362][784615] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-02-24 07:56:19,364][784615] Heartbeat connected on RolloutWorker_w0 | |
| [2023-02-24 07:56:19,365][784615] Heartbeat connected on RolloutWorker_w1 | |
| [2023-02-24 07:56:19,368][784615] Heartbeat connected on RolloutWorker_w3 | |
| [2023-02-24 07:56:19,373][784615] Heartbeat connected on RolloutWorker_w4 | |
| [2023-02-24 07:56:19,374][784615] Heartbeat connected on RolloutWorker_w5 | |
| [2023-02-24 07:56:19,375][784615] Heartbeat connected on RolloutWorker_w2 | |
| [2023-02-24 07:56:19,377][784615] Heartbeat connected on RolloutWorker_w7 | |
| [2023-02-24 07:56:19,378][784615] Heartbeat connected on RolloutWorker_w6 | |
| [2023-02-24 07:56:20,182][794032] Updated weights for policy 0, policy_version 30 (0.0006) | |
| [2023-02-24 07:56:22,465][794032] Updated weights for policy 0, policy_version 40 (0.0007) | |
| [2023-02-24 07:56:23,153][784615] Fps is (10 sec: 16793.7, 60 sec: 11468.8, 300 sec: 11468.8). Total num frames: 172032. Throughput: 0: 2462.0. Samples: 36930. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 07:56:23,154][784615] Avg episode reward: [(0, '4.501')] | |
| [2023-02-24 07:56:23,155][794019] Saving new best policy, reward=4.501! | |
| [2023-02-24 07:56:25,053][794032] Updated weights for policy 0, policy_version 50 (0.0007) | |
| [2023-02-24 07:56:27,490][794032] Updated weights for policy 0, policy_version 60 (0.0007) | |
| [2023-02-24 07:56:28,153][784615] Fps is (10 sec: 16793.6, 60 sec: 12697.6, 300 sec: 12697.6). Total num frames: 253952. Throughput: 0: 3086.0. Samples: 61720. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:28,154][784615] Avg episode reward: [(0, '4.402')] | |
| [2023-02-24 07:56:30,062][794032] Updated weights for policy 0, policy_version 70 (0.0007) | |
| [2023-02-24 07:56:32,640][794032] Updated weights for policy 0, policy_version 80 (0.0007) | |
| [2023-02-24 07:56:33,153][784615] Fps is (10 sec: 15974.4, 60 sec: 13271.1, 300 sec: 13271.1). Total num frames: 331776. Throughput: 0: 2951.7. Samples: 73792. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:33,154][784615] Avg episode reward: [(0, '4.289')] | |
| [2023-02-24 07:56:35,246][794032] Updated weights for policy 0, policy_version 90 (0.0007) | |
| [2023-02-24 07:56:37,855][794032] Updated weights for policy 0, policy_version 100 (0.0008) | |
| [2023-02-24 07:56:38,153][784615] Fps is (10 sec: 15974.5, 60 sec: 13789.9, 300 sec: 13789.9). Total num frames: 413696. Throughput: 0: 3249.9. Samples: 97496. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:38,154][784615] Avg episode reward: [(0, '4.457')] | |
| [2023-02-24 07:56:40,454][794032] Updated weights for policy 0, policy_version 110 (0.0007) | |
| [2023-02-24 07:56:43,061][794032] Updated weights for policy 0, policy_version 120 (0.0006) | |
| [2023-02-24 07:56:43,153][784615] Fps is (10 sec: 15973.9, 60 sec: 14043.3, 300 sec: 14043.3). Total num frames: 491520. Throughput: 0: 3461.9. Samples: 121166. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:43,154][784615] Avg episode reward: [(0, '4.401')] | |
| [2023-02-24 07:56:45,699][794032] Updated weights for policy 0, policy_version 130 (0.0007) | |
| [2023-02-24 07:56:48,153][784615] Fps is (10 sec: 15564.7, 60 sec: 14233.6, 300 sec: 14233.6). Total num frames: 569344. Throughput: 0: 3324.0. Samples: 132960. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 07:56:48,154][784615] Avg episode reward: [(0, '4.667')] | |
| [2023-02-24 07:56:48,157][794019] Saving new best policy, reward=4.667! | |
| [2023-02-24 07:56:48,334][794032] Updated weights for policy 0, policy_version 140 (0.0007) | |
| [2023-02-24 07:56:50,891][794032] Updated weights for policy 0, policy_version 150 (0.0006) | |
| [2023-02-24 07:56:53,153][784615] Fps is (10 sec: 15565.3, 60 sec: 14381.5, 300 sec: 14381.5). Total num frames: 647168. Throughput: 0: 3474.6. Samples: 156358. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:56:53,154][784615] Avg episode reward: [(0, '4.622')] | |
| [2023-02-24 07:56:53,523][794032] Updated weights for policy 0, policy_version 160 (0.0008) | |
| [2023-02-24 07:56:56,158][794032] Updated weights for policy 0, policy_version 170 (0.0007) | |
| [2023-02-24 07:56:58,153][784615] Fps is (10 sec: 15565.0, 60 sec: 14499.9, 300 sec: 14499.9). Total num frames: 724992. Throughput: 0: 3940.0. Samples: 179774. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-02-24 07:56:58,153][784615] Avg episode reward: [(0, '4.603')] | |
| [2023-02-24 07:56:58,738][794032] Updated weights for policy 0, policy_version 180 (0.0008) | |
| [2023-02-24 07:57:01,298][794032] Updated weights for policy 0, policy_version 190 (0.0007) | |
| [2023-02-24 07:57:03,153][784615] Fps is (10 sec: 15974.3, 60 sec: 14671.1, 300 sec: 14671.1). Total num frames: 806912. Throughput: 0: 3998.0. Samples: 191576. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-02-24 07:57:03,154][784615] Avg episode reward: [(0, '4.850')] | |
| [2023-02-24 07:57:03,155][794019] Saving new best policy, reward=4.850! | |
| [2023-02-24 07:57:03,931][794032] Updated weights for policy 0, policy_version 200 (0.0007) | |
| [2023-02-24 07:57:06,517][794032] Updated weights for policy 0, policy_version 210 (0.0007) | |
| [2023-02-24 07:57:08,153][784615] Fps is (10 sec: 15974.2, 60 sec: 14745.6, 300 sec: 14745.6). Total num frames: 884736. Throughput: 0: 3958.6. Samples: 215068. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-02-24 07:57:08,154][784615] Avg episode reward: [(0, '5.203')] | |
| [2023-02-24 07:57:08,156][794019] Saving new best policy, reward=5.203! | |
| [2023-02-24 07:57:09,207][794032] Updated weights for policy 0, policy_version 220 (0.0006) | |
| [2023-02-24 07:57:11,788][794032] Updated weights for policy 0, policy_version 230 (0.0006) | |
| [2023-02-24 07:57:13,153][784615] Fps is (10 sec: 15564.9, 60 sec: 15974.4, 300 sec: 14808.6). Total num frames: 962560. Throughput: 0: 3932.8. Samples: 238698. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:57:13,154][784615] Avg episode reward: [(0, '4.831')] | |
| [2023-02-24 07:57:14,400][794032] Updated weights for policy 0, policy_version 240 (0.0006) | |
| [2023-02-24 07:57:17,007][794032] Updated weights for policy 0, policy_version 250 (0.0008) | |
| [2023-02-24 07:57:18,153][784615] Fps is (10 sec: 15564.8, 60 sec: 15906.1, 300 sec: 14862.6). Total num frames: 1040384. Throughput: 0: 3928.0. Samples: 250550. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2023-02-24 07:57:18,154][784615] Avg episode reward: [(0, '6.010')] | |
| [2023-02-24 07:57:18,157][794019] Saving new best policy, reward=6.010! | |
| [2023-02-24 07:57:19,591][794032] Updated weights for policy 0, policy_version 260 (0.0006) | |
| [2023-02-24 07:57:21,628][784615] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 784615], exiting... | |
| [2023-02-24 07:57:21,629][794019] Stopping Batcher_0... | |
| [2023-02-24 07:57:21,629][794019] Loop batcher_evt_loop terminating... | |
| [2023-02-24 07:57:21,628][784615] Runner profile tree view: | |
| main_loop: 82.2510 | |
| [2023-02-24 07:57:21,630][784615] Collected {0: 1093632}, FPS: 13296.3 | |
| [2023-02-24 07:57:21,638][794039] Stopping RolloutWorker_w6... | |
| [2023-02-24 07:57:21,638][794035] Stopping RolloutWorker_w3... | |
| [2023-02-24 07:57:21,638][794035] Loop rollout_proc3_evt_loop terminating... | |
| [2023-02-24 07:57:21,638][794034] Stopping RolloutWorker_w1... | |
| [2023-02-24 07:57:21,639][794039] Loop rollout_proc6_evt_loop terminating... | |
| [2023-02-24 07:57:21,639][794034] Loop rollout_proc1_evt_loop terminating... | |
| [2023-02-24 07:57:21,645][794033] Stopping RolloutWorker_w0... | |
| [2023-02-24 07:57:21,646][794033] Loop rollout_proc0_evt_loop terminating... | |
| [2023-02-24 07:57:21,646][794037] Stopping RolloutWorker_w4... | |
| [2023-02-24 07:57:21,647][794037] Loop rollout_proc4_evt_loop terminating... | |
| [2023-02-24 07:57:21,647][794038] Stopping RolloutWorker_w2... | |
| [2023-02-24 07:57:21,648][794038] Loop rollout_proc2_evt_loop terminating... | |
| [2023-02-24 07:57:21,651][794036] Stopping RolloutWorker_w5... | |
| [2023-02-24 07:57:21,651][794036] Loop rollout_proc5_evt_loop terminating... | |
| [2023-02-24 07:57:21,652][794040] Stopping RolloutWorker_w7... | |
| [2023-02-24 07:57:21,653][794040] Loop rollout_proc7_evt_loop terminating... | |
| [2023-02-24 07:57:21,667][794019] Saving /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... | |
| [2023-02-24 07:57:21,688][794032] Weights refcount: 2 0 | |
| [2023-02-24 07:57:21,695][794032] Stopping InferenceWorker_p0-w0... | |
| [2023-02-24 07:57:21,696][794032] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-02-24 07:57:21,825][794019] Stopping LearnerWorker_p0... | |
| [2023-02-24 07:57:21,826][794019] Loop learner_proc0_evt_loop terminating... | |
| [2023-02-24 07:57:37,644][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 07:57:37,644][784615] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-02-24 07:57:37,645][784615] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-02-24 07:57:37,645][784615] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-02-24 07:57:37,645][784615] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-24 07:57:37,646][784615] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-02-24 07:57:37,646][784615] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-02-24 07:57:37,646][784615] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-02-24 07:57:37,647][784615] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-02-24 07:57:37,647][784615] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-02-24 07:57:37,647][784615] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-02-24 07:57:37,648][784615] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-02-24 07:57:37,648][784615] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-02-24 07:57:37,648][784615] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-02-24 07:57:37,649][784615] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-02-24 07:57:37,655][784615] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-02-24 07:57:37,656][784615] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 07:57:37,657][784615] RunningMeanStd input shape: (1,) | |
| [2023-02-24 07:57:37,665][784615] ConvEncoder: input_channels=3 | |
| [2023-02-24 07:57:37,755][784615] Conv encoder output size: 512 | |
| [2023-02-24 07:57:37,755][784615] Policy head output size: 512 | |
| [2023-02-24 07:57:40,369][784615] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... | |
| [2023-02-24 07:57:42,531][784615] Num frames 100... | |
| [2023-02-24 07:57:42,595][784615] Num frames 200... | |
| [2023-02-24 07:57:42,663][784615] Num frames 300... | |
| [2023-02-24 07:57:42,732][784615] Num frames 400... | |
| [2023-02-24 07:57:42,832][784615] Avg episode rewards: #0: 6.800, true rewards: #0: 4.800 | |
| [2023-02-24 07:57:42,833][784615] Avg episode reward: 6.800, avg true_objective: 4.800 | |
| [2023-02-24 07:57:42,848][784615] Num frames 500... | |
| [2023-02-24 07:57:42,911][784615] Num frames 600... | |
| [2023-02-24 07:57:42,969][784615] Num frames 700... | |
| [2023-02-24 07:57:43,043][784615] Num frames 800... | |
| [2023-02-24 07:57:43,113][784615] Avg episode rewards: #0: 5.630, true rewards: #0: 4.130 | |
| [2023-02-24 07:57:43,113][784615] Avg episode reward: 5.630, avg true_objective: 4.130 | |
| [2023-02-24 07:57:43,175][784615] Num frames 900... | |
| [2023-02-24 07:57:43,244][784615] Num frames 1000... | |
| [2023-02-24 07:57:43,309][784615] Num frames 1100... | |
| [2023-02-24 07:57:43,371][784615] Num frames 1200... | |
| [2023-02-24 07:57:43,470][784615] Avg episode rewards: #0: 5.580, true rewards: #0: 4.247 | |
| [2023-02-24 07:57:43,471][784615] Avg episode reward: 5.580, avg true_objective: 4.247 | |
| [2023-02-24 07:57:43,487][784615] Num frames 1300... | |
| [2023-02-24 07:57:43,564][784615] Num frames 1400... | |
| [2023-02-24 07:57:43,628][784615] Num frames 1500... | |
| [2023-02-24 07:57:43,689][784615] Num frames 1600... | |
| [2023-02-24 07:57:43,781][784615] Avg episode rewards: #0: 5.145, true rewards: #0: 4.145 | |
| [2023-02-24 07:57:43,781][784615] Avg episode reward: 5.145, avg true_objective: 4.145 | |
| [2023-02-24 07:57:43,817][784615] Num frames 1700... | |
| [2023-02-24 07:57:43,887][784615] Num frames 1800... | |
| [2023-02-24 07:57:43,956][784615] Num frames 1900... | |
| [2023-02-24 07:57:44,024][784615] Num frames 2000... | |
| [2023-02-24 07:57:44,089][784615] Num frames 2100... | |
| [2023-02-24 07:57:44,147][784615] Avg episode rewards: #0: 5.612, true rewards: #0: 4.212 | |
| [2023-02-24 07:57:44,148][784615] Avg episode reward: 5.612, avg true_objective: 4.212 | |
| [2023-02-24 07:57:44,208][784615] Num frames 2200... | |
| [2023-02-24 07:57:44,288][784615] Num frames 2300... | |
| [2023-02-24 07:57:44,350][784615] Num frames 2400... | |
| [2023-02-24 07:57:44,463][784615] Avg episode rewards: #0: 5.317, true rewards: #0: 4.150 | |
| [2023-02-24 07:57:44,463][784615] Avg episode reward: 5.317, avg true_objective: 4.150 | |
| [2023-02-24 07:57:44,471][784615] Num frames 2500... | |
| [2023-02-24 07:57:44,540][784615] Num frames 2600... | |
| [2023-02-24 07:57:44,607][784615] Num frames 2700... | |
| [2023-02-24 07:57:44,671][784615] Num frames 2800... | |
| [2023-02-24 07:57:44,738][784615] Num frames 2900... | |
| [2023-02-24 07:57:44,810][784615] Num frames 3000... | |
| [2023-02-24 07:57:44,865][784615] Avg episode rewards: #0: 5.717, true rewards: #0: 4.289 | |
| [2023-02-24 07:57:44,866][784615] Avg episode reward: 5.717, avg true_objective: 4.289 | |
| [2023-02-24 07:57:44,925][784615] Num frames 3100... | |
| [2023-02-24 07:57:44,988][784615] Num frames 3200... | |
| [2023-02-24 07:57:45,057][784615] Num frames 3300... | |
| [2023-02-24 07:57:45,126][784615] Num frames 3400... | |
| [2023-02-24 07:57:45,200][784615] Num frames 3500... | |
| [2023-02-24 07:57:45,287][784615] Avg episode rewards: #0: 5.933, true rewards: #0: 4.432 | |
| [2023-02-24 07:57:45,287][784615] Avg episode reward: 5.933, avg true_objective: 4.432 | |
| [2023-02-24 07:57:45,332][784615] Num frames 3600... | |
| [2023-02-24 07:57:45,410][784615] Num frames 3700... | |
| [2023-02-24 07:57:45,468][784615] Num frames 3800... | |
| [2023-02-24 07:57:45,570][784615] Num frames 3900... | |
| [2023-02-24 07:57:45,634][784615] Num frames 4000... | |
| [2023-02-24 07:57:45,698][784615] Num frames 4100... | |
| [2023-02-24 07:57:45,762][784615] Num frames 4200... | |
| [2023-02-24 07:57:45,873][784615] Avg episode rewards: #0: 6.869, true rewards: #0: 4.758 | |
| [2023-02-24 07:57:45,874][784615] Avg episode reward: 6.869, avg true_objective: 4.758 | |
| [2023-02-24 07:57:45,886][784615] Num frames 4300... | |
| [2023-02-24 07:57:45,952][784615] Num frames 4400... | |
| [2023-02-24 07:57:46,014][784615] Num frames 4500... | |
| [2023-02-24 07:57:46,087][784615] Num frames 4600... | |
| [2023-02-24 07:57:46,149][784615] Num frames 4700... | |
| [2023-02-24 07:57:46,224][784615] Avg episode rewards: #0: 6.730, true rewards: #0: 4.730 | |
| [2023-02-24 07:57:46,225][784615] Avg episode reward: 6.730, avg true_objective: 4.730 | |
| [2023-02-24 07:57:48,353][784615] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4! | |
| [2023-02-24 07:58:39,896][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json | |
| [2023-02-24 07:58:39,896][784615] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2023-02-24 07:58:39,897][784615] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2023-02-24 07:58:39,897][784615] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2023-02-24 07:58:39,898][784615] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2023-02-24 07:58:39,898][784615] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2023-02-24 07:58:39,899][784615] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2023-02-24 07:58:39,899][784615] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2023-02-24 07:58:39,900][784615] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2023-02-24 07:58:39,900][784615] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2023-02-24 07:58:39,900][784615] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2023-02-24 07:58:39,901][784615] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2023-02-24 07:58:39,901][784615] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2023-02-24 07:58:39,902][784615] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2023-02-24 07:58:39,902][784615] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2023-02-24 07:58:39,911][784615] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-02-24 07:58:39,912][784615] RunningMeanStd input shape: (1,) | |
| [2023-02-24 07:58:39,919][784615] ConvEncoder: input_channels=3 | |
| [2023-02-24 07:58:39,943][784615] Conv encoder output size: 512 | |
| [2023-02-24 07:58:39,944][784615] Policy head output size: 512 | |
| [2023-02-24 07:58:39,980][784615] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... | |
| [2023-02-24 07:58:40,400][784615] Num frames 100... | |
| [2023-02-24 07:58:40,470][784615] Num frames 200... | |
| [2023-02-24 07:58:40,530][784615] Num frames 300... | |
| [2023-02-24 07:58:40,596][784615] Num frames 400... | |
| [2023-02-24 07:58:40,684][784615] Avg episode rewards: #0: 5.480, true rewards: #0: 4.480 | |
| [2023-02-24 07:58:40,685][784615] Avg episode reward: 5.480, avg true_objective: 4.480 | |
| [2023-02-24 07:58:40,719][784615] Num frames 500... | |
| [2023-02-24 07:58:40,787][784615] Num frames 600... | |
| [2023-02-24 07:58:40,850][784615] Num frames 700... | |
| [2023-02-24 07:58:40,918][784615] Num frames 800... | |
| [2023-02-24 07:58:41,005][784615] Num frames 900... | |
| [2023-02-24 07:58:41,078][784615] Avg episode rewards: #0: 6.640, true rewards: #0: 4.640 | |
| [2023-02-24 07:58:41,079][784615] Avg episode reward: 6.640, avg true_objective: 4.640 | |
| [2023-02-24 07:58:41,129][784615] Num frames 1000... | |
| [2023-02-24 07:58:41,202][784615] Num frames 1100... | |
| [2023-02-24 07:58:41,276][784615] Num frames 1200... | |
| [2023-02-24 07:58:41,360][784615] Num frames 1300... | |
| [2023-02-24 07:58:41,429][784615] Num frames 1400... | |
| [2023-02-24 07:58:41,493][784615] Num frames 1500... | |
| [2023-02-24 07:58:41,554][784615] Num frames 1600... | |
| [2023-02-24 07:58:41,667][784615] Avg episode rewards: #0: 8.653, true rewards: #0: 5.653 | |
| [2023-02-24 07:58:41,668][784615] Avg episode reward: 8.653, avg true_objective: 5.653 | |
| [2023-02-24 07:58:41,673][784615] Num frames 1700... | |
| [2023-02-24 07:58:41,734][784615] Num frames 1800... | |
| [2023-02-24 07:58:41,791][784615] Num frames 1900... | |
| [2023-02-24 07:58:41,848][784615] Num frames 2000... | |
| [2023-02-24 07:58:41,905][784615] Num frames 2100... | |
| [2023-02-24 07:58:41,961][784615] Num frames 2200... | |
| [2023-02-24 07:58:42,055][784615] Avg episode rewards: #0: 8.680, true rewards: #0: 5.680 | |
| [2023-02-24 07:58:42,056][784615] Avg episode reward: 8.680, avg true_objective: 5.680 | |
| [2023-02-24 07:58:42,077][784615] Num frames 2300... | |
| [2023-02-24 07:58:42,141][784615] Num frames 2400... | |
| [2023-02-24 07:58:42,203][784615] Num frames 2500... | |
| [2023-02-24 07:58:42,260][784615] Num frames 2600... | |
| [2023-02-24 07:58:42,346][784615] Avg episode rewards: #0: 7.712, true rewards: #0: 5.312 | |
| [2023-02-24 07:58:42,348][784615] Avg episode reward: 7.712, avg true_objective: 5.312 | |
| [2023-02-24 07:58:42,385][784615] Num frames 2700... | |
| [2023-02-24 07:58:42,449][784615] Num frames 2800... | |
| [2023-02-24 07:58:42,506][784615] Num frames 2900... | |
| [2023-02-24 07:58:42,563][784615] Num frames 3000... | |
| [2023-02-24 07:58:42,621][784615] Num frames 3100... | |
| [2023-02-24 07:58:42,688][784615] Num frames 3200... | |
| [2023-02-24 07:58:42,788][784615] Avg episode rewards: #0: 7.940, true rewards: #0: 5.440 | |
| [2023-02-24 07:58:42,788][784615] Avg episode reward: 7.940, avg true_objective: 5.440 | |
| [2023-02-24 07:58:42,817][784615] Num frames 3300... | |
| [2023-02-24 07:58:42,890][784615] Num frames 3400... | |
| [2023-02-24 07:58:42,964][784615] Num frames 3500... | |
| [2023-02-24 07:58:43,040][784615] Num frames 3600... | |
| [2023-02-24 07:58:43,128][784615] Avg episode rewards: #0: 7.354, true rewards: #0: 5.211 | |
| [2023-02-24 07:58:43,130][784615] Avg episode reward: 7.354, avg true_objective: 5.211 | |
| [2023-02-24 07:58:43,174][784615] Num frames 3700... | |
| [2023-02-24 07:58:43,248][784615] Num frames 3800... | |
| [2023-02-24 07:58:43,316][784615] Num frames 3900... | |
| [2023-02-24 07:58:43,383][784615] Num frames 4000... | |
| [2023-02-24 07:58:43,475][784615] Avg episode rewards: #0: 7.205, true rewards: #0: 5.080 | |
| [2023-02-24 07:58:43,476][784615] Avg episode reward: 7.205, avg true_objective: 5.080 | |
| [2023-02-24 07:58:43,498][784615] Num frames 4100... | |
| [2023-02-24 07:58:43,556][784615] Num frames 4200... | |
| [2023-02-24 07:58:43,614][784615] Num frames 4300... | |
| [2023-02-24 07:58:43,670][784615] Num frames 4400... | |
| [2023-02-24 07:58:43,727][784615] Num frames 4500... | |
| [2023-02-24 07:58:43,790][784615] Num frames 4600... | |
| [2023-02-24 07:58:43,891][784615] Avg episode rewards: #0: 7.413, true rewards: #0: 5.191 | |
| [2023-02-24 07:58:43,891][784615] Avg episode reward: 7.413, avg true_objective: 5.191 | |
| [2023-02-24 07:58:43,912][784615] Num frames 4700... | |
| [2023-02-24 07:58:43,982][784615] Num frames 4800... | |
| [2023-02-24 07:58:44,049][784615] Num frames 4900... | |
| [2023-02-24 07:58:44,117][784615] Num frames 5000... | |
| [2023-02-24 07:58:44,184][784615] Num frames 5100... | |
| [2023-02-24 07:58:44,250][784615] Avg episode rewards: #0: 7.220, true rewards: #0: 5.120 | |
| [2023-02-24 07:58:44,250][784615] Avg episode reward: 7.220, avg true_objective: 5.120 | |
| [2023-02-24 07:58:46,584][784615] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4! | |