[2023-02-24 07:55:59,139][784615] Saving configuration to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json... [2023-02-24 07:55:59,305][784615] Rollout worker 0 uses device cpu [2023-02-24 07:55:59,305][784615] Rollout worker 1 uses device cpu [2023-02-24 07:55:59,306][784615] Rollout worker 2 uses device cpu [2023-02-24 07:55:59,306][784615] Rollout worker 3 uses device cpu [2023-02-24 07:55:59,306][784615] Rollout worker 4 uses device cpu [2023-02-24 07:55:59,307][784615] Rollout worker 5 uses device cpu [2023-02-24 07:55:59,307][784615] Rollout worker 6 uses device cpu [2023-02-24 07:55:59,308][784615] Rollout worker 7 uses device cpu [2023-02-24 07:55:59,357][784615] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 07:55:59,358][784615] InferenceWorker_p0-w0: min num requests: 2 [2023-02-24 07:55:59,378][784615] Starting all processes... [2023-02-24 07:55:59,378][784615] Starting process learner_proc0 [2023-02-24 07:55:59,428][784615] Starting all processes... [2023-02-24 07:55:59,438][784615] Starting process inference_proc0-0 [2023-02-24 07:55:59,439][784615] Starting process rollout_proc0 [2023-02-24 07:55:59,439][784615] Starting process rollout_proc1 [2023-02-24 07:55:59,439][784615] Starting process rollout_proc2 [2023-02-24 07:55:59,439][784615] Starting process rollout_proc3 [2023-02-24 07:55:59,439][784615] Starting process rollout_proc4 [2023-02-24 07:55:59,439][784615] Starting process rollout_proc5 [2023-02-24 07:55:59,440][784615] Starting process rollout_proc6 [2023-02-24 07:55:59,441][784615] Starting process rollout_proc7 [2023-02-24 07:56:00,842][794035] Worker 3 uses CPU cores [3] [2023-02-24 07:56:00,862][794036] Worker 5 uses CPU cores [5] [2023-02-24 07:56:00,978][794037] Worker 4 uses CPU cores [4] [2023-02-24 07:56:00,998][794038] Worker 2 uses CPU cores [2] [2023-02-24 07:56:00,999][794019] Low niceness requires sudo! [2023-02-24 07:56:00,999][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 07:56:01,000][794019] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for learning process 0 [2023-02-24 07:56:01,022][794032] Low niceness requires sudo! [2023-02-24 07:56:01,022][794032] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 07:56:01,022][794032] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for inference process 0 [2023-02-24 07:56:01,025][794019] Num visible devices: 1 [2023-02-24 07:56:01,040][794032] Num visible devices: 1 [2023-02-24 07:56:01,050][794019] Starting seed is not provided [2023-02-24 07:56:01,050][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 07:56:01,050][794019] Initializing actor-critic model on device cuda:0 [2023-02-24 07:56:01,051][794019] RunningMeanStd input shape: (3, 72, 128) [2023-02-24 07:56:01,051][794019] RunningMeanStd input shape: (1,) [2023-02-24 07:56:01,058][794040] Worker 7 uses CPU cores [7] [2023-02-24 07:56:01,065][794019] ConvEncoder: input_channels=3 [2023-02-24 07:56:01,154][794034] Worker 1 uses CPU cores [1] [2023-02-24 07:56:01,186][794039] Worker 6 uses CPU cores [6] [2023-02-24 07:56:01,194][794033] Worker 0 uses CPU cores [0] [2023-02-24 07:56:01,198][794019] Conv encoder output size: 512 [2023-02-24 07:56:01,198][794019] Policy head output size: 512 [2023-02-24 07:56:01,216][794019] Created Actor Critic model with architecture: [2023-02-24 07:56:01,216][794019] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2023-02-24 07:56:04,122][794019] Using optimizer [2023-02-24 07:56:04,122][794019] No checkpoints found [2023-02-24 07:56:04,122][794019] Did not load from checkpoint, starting from scratch! [2023-02-24 07:56:04,122][794019] Initialized policy 0 weights for model version 0 [2023-02-24 07:56:04,124][794019] LearnerWorker_p0 finished initialization! [2023-02-24 07:56:04,124][794019] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 07:56:05,229][794032] RunningMeanStd input shape: (3, 72, 128) [2023-02-24 07:56:05,229][794032] RunningMeanStd input shape: (1,) [2023-02-24 07:56:05,237][794032] ConvEncoder: input_channels=3 [2023-02-24 07:56:05,307][794032] Conv encoder output size: 512 [2023-02-24 07:56:05,307][794032] Policy head output size: 512 [2023-02-24 07:56:06,350][784615] Inference worker 0-0 is ready! [2023-02-24 07:56:06,350][784615] All inference workers are ready! Signal rollout workers to start! [2023-02-24 07:56:06,367][794036] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:56:06,368][794039] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:56:06,368][794034] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:56:06,368][794035] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:56:06,369][794038] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:56:06,374][794040] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:56:06,390][794037] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:56:06,393][794033] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:56:06,669][794035] Decorrelating experience for 0 frames... [2023-02-24 07:56:06,672][794039] Decorrelating experience for 0 frames... [2023-02-24 07:56:06,678][794036] Decorrelating experience for 0 frames... [2023-02-24 07:56:06,681][794037] Decorrelating experience for 0 frames... [2023-02-24 07:56:06,683][794040] Decorrelating experience for 0 frames... [2023-02-24 07:56:06,929][794035] Decorrelating experience for 32 frames... [2023-02-24 07:56:06,946][794040] Decorrelating experience for 32 frames... [2023-02-24 07:56:06,958][794036] Decorrelating experience for 32 frames... [2023-02-24 07:56:06,977][794037] Decorrelating experience for 32 frames... [2023-02-24 07:56:06,986][794038] Decorrelating experience for 0 frames... [2023-02-24 07:56:06,997][794039] Decorrelating experience for 32 frames... [2023-02-24 07:56:07,033][794033] Decorrelating experience for 0 frames... [2023-02-24 07:56:07,227][794038] Decorrelating experience for 32 frames... [2023-02-24 07:56:07,268][794034] Decorrelating experience for 0 frames... [2023-02-24 07:56:07,269][794036] Decorrelating experience for 64 frames... [2023-02-24 07:56:07,278][794037] Decorrelating experience for 64 frames... [2023-02-24 07:56:07,293][794033] Decorrelating experience for 32 frames... [2023-02-24 07:56:07,502][794038] Decorrelating experience for 64 frames... [2023-02-24 07:56:07,545][794039] Decorrelating experience for 64 frames... [2023-02-24 07:56:07,584][794036] Decorrelating experience for 96 frames... [2023-02-24 07:56:07,591][794033] Decorrelating experience for 64 frames... [2023-02-24 07:56:07,594][794040] Decorrelating experience for 64 frames... [2023-02-24 07:56:07,600][794037] Decorrelating experience for 96 frames... [2023-02-24 07:56:07,628][794034] Decorrelating experience for 32 frames... [2023-02-24 07:56:07,822][794033] Decorrelating experience for 96 frames... [2023-02-24 07:56:07,846][794039] Decorrelating experience for 96 frames... [2023-02-24 07:56:07,890][794038] Decorrelating experience for 96 frames... [2023-02-24 07:56:07,895][794040] Decorrelating experience for 96 frames... [2023-02-24 07:56:08,065][794035] Decorrelating experience for 64 frames... [2023-02-24 07:56:08,153][784615] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-24 07:56:08,319][794034] Decorrelating experience for 64 frames... [2023-02-24 07:56:08,572][794035] Decorrelating experience for 96 frames... [2023-02-24 07:56:08,577][794034] Decorrelating experience for 96 frames... [2023-02-24 07:56:10,130][794019] Signal inference workers to stop experience collection... [2023-02-24 07:56:10,137][794032] InferenceWorker_p0-w0: stopping experience collection [2023-02-24 07:56:12,636][794019] Signal inference workers to resume experience collection... [2023-02-24 07:56:12,637][794032] InferenceWorker_p0-w0: resuming experience collection [2023-02-24 07:56:13,153][784615] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 4096. Throughput: 0: 494.8. Samples: 2474. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-02-24 07:56:13,154][784615] Avg episode reward: [(0, '2.494')] [2023-02-24 07:56:15,144][794032] Updated weights for policy 0, policy_version 10 (0.0249) [2023-02-24 07:56:17,664][794032] Updated weights for policy 0, policy_version 20 (0.0006) [2023-02-24 07:56:18,153][784615] Fps is (10 sec: 8601.6, 60 sec: 8601.6, 300 sec: 8601.6). Total num frames: 86016. Throughput: 0: 1166.6. Samples: 11666. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-24 07:56:18,154][784615] Avg episode reward: [(0, '4.437')] [2023-02-24 07:56:19,351][784615] Heartbeat connected on Batcher_0 [2023-02-24 07:56:19,354][784615] Heartbeat connected on LearnerWorker_p0 [2023-02-24 07:56:19,362][784615] Heartbeat connected on InferenceWorker_p0-w0 [2023-02-24 07:56:19,364][784615] Heartbeat connected on RolloutWorker_w0 [2023-02-24 07:56:19,365][784615] Heartbeat connected on RolloutWorker_w1 [2023-02-24 07:56:19,368][784615] Heartbeat connected on RolloutWorker_w3 [2023-02-24 07:56:19,373][784615] Heartbeat connected on RolloutWorker_w4 [2023-02-24 07:56:19,374][784615] Heartbeat connected on RolloutWorker_w5 [2023-02-24 07:56:19,375][784615] Heartbeat connected on RolloutWorker_w2 [2023-02-24 07:56:19,377][784615] Heartbeat connected on RolloutWorker_w7 [2023-02-24 07:56:19,378][784615] Heartbeat connected on RolloutWorker_w6 [2023-02-24 07:56:20,182][794032] Updated weights for policy 0, policy_version 30 (0.0006) [2023-02-24 07:56:22,465][794032] Updated weights for policy 0, policy_version 40 (0.0007) [2023-02-24 07:56:23,153][784615] Fps is (10 sec: 16793.7, 60 sec: 11468.8, 300 sec: 11468.8). Total num frames: 172032. Throughput: 0: 2462.0. Samples: 36930. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-02-24 07:56:23,154][784615] Avg episode reward: [(0, '4.501')] [2023-02-24 07:56:23,155][794019] Saving new best policy, reward=4.501! [2023-02-24 07:56:25,053][794032] Updated weights for policy 0, policy_version 50 (0.0007) [2023-02-24 07:56:27,490][794032] Updated weights for policy 0, policy_version 60 (0.0007) [2023-02-24 07:56:28,153][784615] Fps is (10 sec: 16793.6, 60 sec: 12697.6, 300 sec: 12697.6). Total num frames: 253952. Throughput: 0: 3086.0. Samples: 61720. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-02-24 07:56:28,154][784615] Avg episode reward: [(0, '4.402')] [2023-02-24 07:56:30,062][794032] Updated weights for policy 0, policy_version 70 (0.0007) [2023-02-24 07:56:32,640][794032] Updated weights for policy 0, policy_version 80 (0.0007) [2023-02-24 07:56:33,153][784615] Fps is (10 sec: 15974.4, 60 sec: 13271.1, 300 sec: 13271.1). Total num frames: 331776. Throughput: 0: 2951.7. Samples: 73792. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-02-24 07:56:33,154][784615] Avg episode reward: [(0, '4.289')] [2023-02-24 07:56:35,246][794032] Updated weights for policy 0, policy_version 90 (0.0007) [2023-02-24 07:56:37,855][794032] Updated weights for policy 0, policy_version 100 (0.0008) [2023-02-24 07:56:38,153][784615] Fps is (10 sec: 15974.5, 60 sec: 13789.9, 300 sec: 13789.9). Total num frames: 413696. Throughput: 0: 3249.9. Samples: 97496. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-02-24 07:56:38,154][784615] Avg episode reward: [(0, '4.457')] [2023-02-24 07:56:40,454][794032] Updated weights for policy 0, policy_version 110 (0.0007) [2023-02-24 07:56:43,061][794032] Updated weights for policy 0, policy_version 120 (0.0006) [2023-02-24 07:56:43,153][784615] Fps is (10 sec: 15973.9, 60 sec: 14043.3, 300 sec: 14043.3). Total num frames: 491520. Throughput: 0: 3461.9. Samples: 121166. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-02-24 07:56:43,154][784615] Avg episode reward: [(0, '4.401')] [2023-02-24 07:56:45,699][794032] Updated weights for policy 0, policy_version 130 (0.0007) [2023-02-24 07:56:48,153][784615] Fps is (10 sec: 15564.7, 60 sec: 14233.6, 300 sec: 14233.6). Total num frames: 569344. Throughput: 0: 3324.0. Samples: 132960. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-02-24 07:56:48,154][784615] Avg episode reward: [(0, '4.667')] [2023-02-24 07:56:48,157][794019] Saving new best policy, reward=4.667! [2023-02-24 07:56:48,334][794032] Updated weights for policy 0, policy_version 140 (0.0007) [2023-02-24 07:56:50,891][794032] Updated weights for policy 0, policy_version 150 (0.0006) [2023-02-24 07:56:53,153][784615] Fps is (10 sec: 15565.3, 60 sec: 14381.5, 300 sec: 14381.5). Total num frames: 647168. Throughput: 0: 3474.6. Samples: 156358. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-02-24 07:56:53,154][784615] Avg episode reward: [(0, '4.622')] [2023-02-24 07:56:53,523][794032] Updated weights for policy 0, policy_version 160 (0.0008) [2023-02-24 07:56:56,158][794032] Updated weights for policy 0, policy_version 170 (0.0007) [2023-02-24 07:56:58,153][784615] Fps is (10 sec: 15565.0, 60 sec: 14499.9, 300 sec: 14499.9). Total num frames: 724992. Throughput: 0: 3940.0. Samples: 179774. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-02-24 07:56:58,153][784615] Avg episode reward: [(0, '4.603')] [2023-02-24 07:56:58,738][794032] Updated weights for policy 0, policy_version 180 (0.0008) [2023-02-24 07:57:01,298][794032] Updated weights for policy 0, policy_version 190 (0.0007) [2023-02-24 07:57:03,153][784615] Fps is (10 sec: 15974.3, 60 sec: 14671.1, 300 sec: 14671.1). Total num frames: 806912. Throughput: 0: 3998.0. Samples: 191576. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-24 07:57:03,154][784615] Avg episode reward: [(0, '4.850')] [2023-02-24 07:57:03,155][794019] Saving new best policy, reward=4.850! [2023-02-24 07:57:03,931][794032] Updated weights for policy 0, policy_version 200 (0.0007) [2023-02-24 07:57:06,517][794032] Updated weights for policy 0, policy_version 210 (0.0007) [2023-02-24 07:57:08,153][784615] Fps is (10 sec: 15974.2, 60 sec: 14745.6, 300 sec: 14745.6). Total num frames: 884736. Throughput: 0: 3958.6. Samples: 215068. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-24 07:57:08,154][784615] Avg episode reward: [(0, '5.203')] [2023-02-24 07:57:08,156][794019] Saving new best policy, reward=5.203! [2023-02-24 07:57:09,207][794032] Updated weights for policy 0, policy_version 220 (0.0006) [2023-02-24 07:57:11,788][794032] Updated weights for policy 0, policy_version 230 (0.0006) [2023-02-24 07:57:13,153][784615] Fps is (10 sec: 15564.9, 60 sec: 15974.4, 300 sec: 14808.6). Total num frames: 962560. Throughput: 0: 3932.8. Samples: 238698. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-02-24 07:57:13,154][784615] Avg episode reward: [(0, '4.831')] [2023-02-24 07:57:14,400][794032] Updated weights for policy 0, policy_version 240 (0.0006) [2023-02-24 07:57:17,007][794032] Updated weights for policy 0, policy_version 250 (0.0008) [2023-02-24 07:57:18,153][784615] Fps is (10 sec: 15564.8, 60 sec: 15906.1, 300 sec: 14862.6). Total num frames: 1040384. Throughput: 0: 3928.0. Samples: 250550. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-02-24 07:57:18,154][784615] Avg episode reward: [(0, '6.010')] [2023-02-24 07:57:18,157][794019] Saving new best policy, reward=6.010! [2023-02-24 07:57:19,591][794032] Updated weights for policy 0, policy_version 260 (0.0006) [2023-02-24 07:57:21,628][784615] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 784615], exiting... [2023-02-24 07:57:21,629][794019] Stopping Batcher_0... [2023-02-24 07:57:21,629][794019] Loop batcher_evt_loop terminating... [2023-02-24 07:57:21,628][784615] Runner profile tree view: main_loop: 82.2510 [2023-02-24 07:57:21,630][784615] Collected {0: 1093632}, FPS: 13296.3 [2023-02-24 07:57:21,638][794039] Stopping RolloutWorker_w6... [2023-02-24 07:57:21,638][794035] Stopping RolloutWorker_w3... [2023-02-24 07:57:21,638][794035] Loop rollout_proc3_evt_loop terminating... [2023-02-24 07:57:21,638][794034] Stopping RolloutWorker_w1... [2023-02-24 07:57:21,639][794039] Loop rollout_proc6_evt_loop terminating... [2023-02-24 07:57:21,639][794034] Loop rollout_proc1_evt_loop terminating... [2023-02-24 07:57:21,645][794033] Stopping RolloutWorker_w0... [2023-02-24 07:57:21,646][794033] Loop rollout_proc0_evt_loop terminating... [2023-02-24 07:57:21,646][794037] Stopping RolloutWorker_w4... [2023-02-24 07:57:21,647][794037] Loop rollout_proc4_evt_loop terminating... [2023-02-24 07:57:21,647][794038] Stopping RolloutWorker_w2... [2023-02-24 07:57:21,648][794038] Loop rollout_proc2_evt_loop terminating... [2023-02-24 07:57:21,651][794036] Stopping RolloutWorker_w5... [2023-02-24 07:57:21,651][794036] Loop rollout_proc5_evt_loop terminating... [2023-02-24 07:57:21,652][794040] Stopping RolloutWorker_w7... [2023-02-24 07:57:21,653][794040] Loop rollout_proc7_evt_loop terminating... [2023-02-24 07:57:21,667][794019] Saving /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... [2023-02-24 07:57:21,688][794032] Weights refcount: 2 0 [2023-02-24 07:57:21,695][794032] Stopping InferenceWorker_p0-w0... [2023-02-24 07:57:21,696][794032] Loop inference_proc0-0_evt_loop terminating... [2023-02-24 07:57:21,825][794019] Stopping LearnerWorker_p0... [2023-02-24 07:57:21,826][794019] Loop learner_proc0_evt_loop terminating... [2023-02-24 07:57:37,644][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json [2023-02-24 07:57:37,644][784615] Overriding arg 'num_workers' with value 1 passed from command line [2023-02-24 07:57:37,645][784615] Adding new argument 'no_render'=True that is not in the saved config file! [2023-02-24 07:57:37,645][784615] Adding new argument 'save_video'=True that is not in the saved config file! [2023-02-24 07:57:37,645][784615] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-02-24 07:57:37,646][784615] Adding new argument 'video_name'=None that is not in the saved config file! [2023-02-24 07:57:37,646][784615] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2023-02-24 07:57:37,646][784615] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-02-24 07:57:37,647][784615] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2023-02-24 07:57:37,647][784615] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2023-02-24 07:57:37,647][784615] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-02-24 07:57:37,648][784615] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-02-24 07:57:37,648][784615] Adding new argument 'train_script'=None that is not in the saved config file! [2023-02-24 07:57:37,648][784615] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-02-24 07:57:37,649][784615] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-02-24 07:57:37,655][784615] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 07:57:37,656][784615] RunningMeanStd input shape: (3, 72, 128) [2023-02-24 07:57:37,657][784615] RunningMeanStd input shape: (1,) [2023-02-24 07:57:37,665][784615] ConvEncoder: input_channels=3 [2023-02-24 07:57:37,755][784615] Conv encoder output size: 512 [2023-02-24 07:57:37,755][784615] Policy head output size: 512 [2023-02-24 07:57:40,369][784615] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... [2023-02-24 07:57:42,531][784615] Num frames 100... [2023-02-24 07:57:42,595][784615] Num frames 200... [2023-02-24 07:57:42,663][784615] Num frames 300... [2023-02-24 07:57:42,732][784615] Num frames 400... [2023-02-24 07:57:42,832][784615] Avg episode rewards: #0: 6.800, true rewards: #0: 4.800 [2023-02-24 07:57:42,833][784615] Avg episode reward: 6.800, avg true_objective: 4.800 [2023-02-24 07:57:42,848][784615] Num frames 500... [2023-02-24 07:57:42,911][784615] Num frames 600... [2023-02-24 07:57:42,969][784615] Num frames 700... [2023-02-24 07:57:43,043][784615] Num frames 800... [2023-02-24 07:57:43,113][784615] Avg episode rewards: #0: 5.630, true rewards: #0: 4.130 [2023-02-24 07:57:43,113][784615] Avg episode reward: 5.630, avg true_objective: 4.130 [2023-02-24 07:57:43,175][784615] Num frames 900... [2023-02-24 07:57:43,244][784615] Num frames 1000... [2023-02-24 07:57:43,309][784615] Num frames 1100... [2023-02-24 07:57:43,371][784615] Num frames 1200... [2023-02-24 07:57:43,470][784615] Avg episode rewards: #0: 5.580, true rewards: #0: 4.247 [2023-02-24 07:57:43,471][784615] Avg episode reward: 5.580, avg true_objective: 4.247 [2023-02-24 07:57:43,487][784615] Num frames 1300... [2023-02-24 07:57:43,564][784615] Num frames 1400... [2023-02-24 07:57:43,628][784615] Num frames 1500... [2023-02-24 07:57:43,689][784615] Num frames 1600... [2023-02-24 07:57:43,781][784615] Avg episode rewards: #0: 5.145, true rewards: #0: 4.145 [2023-02-24 07:57:43,781][784615] Avg episode reward: 5.145, avg true_objective: 4.145 [2023-02-24 07:57:43,817][784615] Num frames 1700... [2023-02-24 07:57:43,887][784615] Num frames 1800... [2023-02-24 07:57:43,956][784615] Num frames 1900... [2023-02-24 07:57:44,024][784615] Num frames 2000... [2023-02-24 07:57:44,089][784615] Num frames 2100... [2023-02-24 07:57:44,147][784615] Avg episode rewards: #0: 5.612, true rewards: #0: 4.212 [2023-02-24 07:57:44,148][784615] Avg episode reward: 5.612, avg true_objective: 4.212 [2023-02-24 07:57:44,208][784615] Num frames 2200... [2023-02-24 07:57:44,288][784615] Num frames 2300... [2023-02-24 07:57:44,350][784615] Num frames 2400... [2023-02-24 07:57:44,463][784615] Avg episode rewards: #0: 5.317, true rewards: #0: 4.150 [2023-02-24 07:57:44,463][784615] Avg episode reward: 5.317, avg true_objective: 4.150 [2023-02-24 07:57:44,471][784615] Num frames 2500... [2023-02-24 07:57:44,540][784615] Num frames 2600... [2023-02-24 07:57:44,607][784615] Num frames 2700... [2023-02-24 07:57:44,671][784615] Num frames 2800... [2023-02-24 07:57:44,738][784615] Num frames 2900... [2023-02-24 07:57:44,810][784615] Num frames 3000... [2023-02-24 07:57:44,865][784615] Avg episode rewards: #0: 5.717, true rewards: #0: 4.289 [2023-02-24 07:57:44,866][784615] Avg episode reward: 5.717, avg true_objective: 4.289 [2023-02-24 07:57:44,925][784615] Num frames 3100... [2023-02-24 07:57:44,988][784615] Num frames 3200... [2023-02-24 07:57:45,057][784615] Num frames 3300... [2023-02-24 07:57:45,126][784615] Num frames 3400... [2023-02-24 07:57:45,200][784615] Num frames 3500... [2023-02-24 07:57:45,287][784615] Avg episode rewards: #0: 5.933, true rewards: #0: 4.432 [2023-02-24 07:57:45,287][784615] Avg episode reward: 5.933, avg true_objective: 4.432 [2023-02-24 07:57:45,332][784615] Num frames 3600... [2023-02-24 07:57:45,410][784615] Num frames 3700... [2023-02-24 07:57:45,468][784615] Num frames 3800... [2023-02-24 07:57:45,570][784615] Num frames 3900... [2023-02-24 07:57:45,634][784615] Num frames 4000... [2023-02-24 07:57:45,698][784615] Num frames 4100... [2023-02-24 07:57:45,762][784615] Num frames 4200... [2023-02-24 07:57:45,873][784615] Avg episode rewards: #0: 6.869, true rewards: #0: 4.758 [2023-02-24 07:57:45,874][784615] Avg episode reward: 6.869, avg true_objective: 4.758 [2023-02-24 07:57:45,886][784615] Num frames 4300... [2023-02-24 07:57:45,952][784615] Num frames 4400... [2023-02-24 07:57:46,014][784615] Num frames 4500... [2023-02-24 07:57:46,087][784615] Num frames 4600... [2023-02-24 07:57:46,149][784615] Num frames 4700... [2023-02-24 07:57:46,224][784615] Avg episode rewards: #0: 6.730, true rewards: #0: 4.730 [2023-02-24 07:57:46,225][784615] Avg episode reward: 6.730, avg true_objective: 4.730 [2023-02-24 07:57:48,353][784615] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4! [2023-02-24 07:58:39,896][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json [2023-02-24 07:58:39,896][784615] Overriding arg 'num_workers' with value 1 passed from command line [2023-02-24 07:58:39,897][784615] Adding new argument 'no_render'=True that is not in the saved config file! [2023-02-24 07:58:39,897][784615] Adding new argument 'save_video'=True that is not in the saved config file! [2023-02-24 07:58:39,898][784615] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-02-24 07:58:39,898][784615] Adding new argument 'video_name'=None that is not in the saved config file! [2023-02-24 07:58:39,899][784615] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2023-02-24 07:58:39,899][784615] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-02-24 07:58:39,900][784615] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2023-02-24 07:58:39,900][784615] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2023-02-24 07:58:39,900][784615] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-02-24 07:58:39,901][784615] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-02-24 07:58:39,901][784615] Adding new argument 'train_script'=None that is not in the saved config file! [2023-02-24 07:58:39,902][784615] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-02-24 07:58:39,902][784615] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-02-24 07:58:39,911][784615] RunningMeanStd input shape: (3, 72, 128) [2023-02-24 07:58:39,912][784615] RunningMeanStd input shape: (1,) [2023-02-24 07:58:39,919][784615] ConvEncoder: input_channels=3 [2023-02-24 07:58:39,943][784615] Conv encoder output size: 512 [2023-02-24 07:58:39,944][784615] Policy head output size: 512 [2023-02-24 07:58:39,980][784615] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... [2023-02-24 07:58:40,400][784615] Num frames 100... [2023-02-24 07:58:40,470][784615] Num frames 200... [2023-02-24 07:58:40,530][784615] Num frames 300... [2023-02-24 07:58:40,596][784615] Num frames 400... [2023-02-24 07:58:40,684][784615] Avg episode rewards: #0: 5.480, true rewards: #0: 4.480 [2023-02-24 07:58:40,685][784615] Avg episode reward: 5.480, avg true_objective: 4.480 [2023-02-24 07:58:40,719][784615] Num frames 500... [2023-02-24 07:58:40,787][784615] Num frames 600... [2023-02-24 07:58:40,850][784615] Num frames 700... [2023-02-24 07:58:40,918][784615] Num frames 800... [2023-02-24 07:58:41,005][784615] Num frames 900... [2023-02-24 07:58:41,078][784615] Avg episode rewards: #0: 6.640, true rewards: #0: 4.640 [2023-02-24 07:58:41,079][784615] Avg episode reward: 6.640, avg true_objective: 4.640 [2023-02-24 07:58:41,129][784615] Num frames 1000... [2023-02-24 07:58:41,202][784615] Num frames 1100... [2023-02-24 07:58:41,276][784615] Num frames 1200... [2023-02-24 07:58:41,360][784615] Num frames 1300... [2023-02-24 07:58:41,429][784615] Num frames 1400... [2023-02-24 07:58:41,493][784615] Num frames 1500... [2023-02-24 07:58:41,554][784615] Num frames 1600... [2023-02-24 07:58:41,667][784615] Avg episode rewards: #0: 8.653, true rewards: #0: 5.653 [2023-02-24 07:58:41,668][784615] Avg episode reward: 8.653, avg true_objective: 5.653 [2023-02-24 07:58:41,673][784615] Num frames 1700... [2023-02-24 07:58:41,734][784615] Num frames 1800... [2023-02-24 07:58:41,791][784615] Num frames 1900... [2023-02-24 07:58:41,848][784615] Num frames 2000... [2023-02-24 07:58:41,905][784615] Num frames 2100... [2023-02-24 07:58:41,961][784615] Num frames 2200... [2023-02-24 07:58:42,055][784615] Avg episode rewards: #0: 8.680, true rewards: #0: 5.680 [2023-02-24 07:58:42,056][784615] Avg episode reward: 8.680, avg true_objective: 5.680 [2023-02-24 07:58:42,077][784615] Num frames 2300... [2023-02-24 07:58:42,141][784615] Num frames 2400... [2023-02-24 07:58:42,203][784615] Num frames 2500... [2023-02-24 07:58:42,260][784615] Num frames 2600... [2023-02-24 07:58:42,346][784615] Avg episode rewards: #0: 7.712, true rewards: #0: 5.312 [2023-02-24 07:58:42,348][784615] Avg episode reward: 7.712, avg true_objective: 5.312 [2023-02-24 07:58:42,385][784615] Num frames 2700... [2023-02-24 07:58:42,449][784615] Num frames 2800... [2023-02-24 07:58:42,506][784615] Num frames 2900... [2023-02-24 07:58:42,563][784615] Num frames 3000... [2023-02-24 07:58:42,621][784615] Num frames 3100... [2023-02-24 07:58:42,688][784615] Num frames 3200... [2023-02-24 07:58:42,788][784615] Avg episode rewards: #0: 7.940, true rewards: #0: 5.440 [2023-02-24 07:58:42,788][784615] Avg episode reward: 7.940, avg true_objective: 5.440 [2023-02-24 07:58:42,817][784615] Num frames 3300... [2023-02-24 07:58:42,890][784615] Num frames 3400... [2023-02-24 07:58:42,964][784615] Num frames 3500... [2023-02-24 07:58:43,040][784615] Num frames 3600... [2023-02-24 07:58:43,128][784615] Avg episode rewards: #0: 7.354, true rewards: #0: 5.211 [2023-02-24 07:58:43,130][784615] Avg episode reward: 7.354, avg true_objective: 5.211 [2023-02-24 07:58:43,174][784615] Num frames 3700... [2023-02-24 07:58:43,248][784615] Num frames 3800... [2023-02-24 07:58:43,316][784615] Num frames 3900... [2023-02-24 07:58:43,383][784615] Num frames 4000... [2023-02-24 07:58:43,475][784615] Avg episode rewards: #0: 7.205, true rewards: #0: 5.080 [2023-02-24 07:58:43,476][784615] Avg episode reward: 7.205, avg true_objective: 5.080 [2023-02-24 07:58:43,498][784615] Num frames 4100... [2023-02-24 07:58:43,556][784615] Num frames 4200... [2023-02-24 07:58:43,614][784615] Num frames 4300... [2023-02-24 07:58:43,670][784615] Num frames 4400... [2023-02-24 07:58:43,727][784615] Num frames 4500... [2023-02-24 07:58:43,790][784615] Num frames 4600... [2023-02-24 07:58:43,891][784615] Avg episode rewards: #0: 7.413, true rewards: #0: 5.191 [2023-02-24 07:58:43,891][784615] Avg episode reward: 7.413, avg true_objective: 5.191 [2023-02-24 07:58:43,912][784615] Num frames 4700... [2023-02-24 07:58:43,982][784615] Num frames 4800... [2023-02-24 07:58:44,049][784615] Num frames 4900... [2023-02-24 07:58:44,117][784615] Num frames 5000... [2023-02-24 07:58:44,184][784615] Num frames 5100... [2023-02-24 07:58:44,250][784615] Avg episode rewards: #0: 7.220, true rewards: #0: 5.120 [2023-02-24 07:58:44,250][784615] Avg episode reward: 7.220, avg true_objective: 5.120 [2023-02-24 07:58:46,584][784615] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4! [2023-02-24 07:59:12,402][784615] The model has been pushed to https://huggingface.co/chqmatteo/rl_course_vizdoom_health_gathering_supreme [2023-02-24 08:02:31,136][784615] Environment doom_basic already registered, overwriting... [2023-02-24 08:02:31,137][784615] Environment doom_two_colors_easy already registered, overwriting... [2023-02-24 08:02:31,137][784615] Environment doom_two_colors_hard already registered, overwriting... [2023-02-24 08:02:31,138][784615] Environment doom_dm already registered, overwriting... [2023-02-24 08:02:31,138][784615] Environment doom_dwango5 already registered, overwriting... [2023-02-24 08:02:31,139][784615] Environment doom_my_way_home_flat_actions already registered, overwriting... [2023-02-24 08:02:31,139][784615] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2023-02-24 08:02:31,139][784615] Environment doom_my_way_home already registered, overwriting... [2023-02-24 08:02:31,140][784615] Environment doom_deadly_corridor already registered, overwriting... [2023-02-24 08:02:31,140][784615] Environment doom_defend_the_center already registered, overwriting... [2023-02-24 08:02:31,140][784615] Environment doom_defend_the_line already registered, overwriting... [2023-02-24 08:02:31,141][784615] Environment doom_health_gathering already registered, overwriting... [2023-02-24 08:02:31,141][784615] Environment doom_health_gathering_supreme already registered, overwriting... [2023-02-24 08:02:31,142][784615] Environment doom_battle already registered, overwriting... [2023-02-24 08:02:31,142][784615] Environment doom_battle2 already registered, overwriting... [2023-02-24 08:02:31,142][784615] Environment doom_duel_bots already registered, overwriting... [2023-02-24 08:02:31,142][784615] Environment doom_deathmatch_bots already registered, overwriting... [2023-02-24 08:02:31,143][784615] Environment doom_duel already registered, overwriting... [2023-02-24 08:02:31,143][784615] Environment doom_deathmatch_full already registered, overwriting... [2023-02-24 08:02:31,143][784615] Environment doom_benchmark already registered, overwriting... [2023-02-24 08:02:31,144][784615] register_encoder_factory: [2023-02-24 08:02:31,153][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json [2023-02-24 08:02:31,154][784615] Overriding arg 'train_for_env_steps' with value 40000000 passed from command line [2023-02-24 08:02:31,157][784615] Experiment dir /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment already exists! [2023-02-24 08:02:31,158][784615] Resuming existing experiment from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment... [2023-02-24 08:02:31,158][784615] Weights and Biases integration disabled [2023-02-24 08:02:31,159][784615] Environment var CUDA_VISIBLE_DEVICES is 1 [2023-02-24 08:03:09,472][784615] Environment doom_basic already registered, overwriting... [2023-02-24 08:03:09,474][784615] Environment doom_two_colors_easy already registered, overwriting... [2023-02-24 08:03:09,475][784615] Environment doom_two_colors_hard already registered, overwriting... [2023-02-24 08:03:09,476][784615] Environment doom_dm already registered, overwriting... [2023-02-24 08:03:09,477][784615] Environment doom_dwango5 already registered, overwriting... [2023-02-24 08:03:09,477][784615] Environment doom_my_way_home_flat_actions already registered, overwriting... [2023-02-24 08:03:09,478][784615] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2023-02-24 08:03:09,478][784615] Environment doom_my_way_home already registered, overwriting... [2023-02-24 08:03:09,479][784615] Environment doom_deadly_corridor already registered, overwriting... [2023-02-24 08:03:09,479][784615] Environment doom_defend_the_center already registered, overwriting... [2023-02-24 08:03:09,480][784615] Environment doom_defend_the_line already registered, overwriting... [2023-02-24 08:03:09,480][784615] Environment doom_health_gathering already registered, overwriting... [2023-02-24 08:03:09,481][784615] Environment doom_health_gathering_supreme already registered, overwriting... [2023-02-24 08:03:09,481][784615] Environment doom_battle already registered, overwriting... [2023-02-24 08:03:09,482][784615] Environment doom_battle2 already registered, overwriting... [2023-02-24 08:03:09,482][784615] Environment doom_duel_bots already registered, overwriting... [2023-02-24 08:03:09,483][784615] Environment doom_deathmatch_bots already registered, overwriting... [2023-02-24 08:03:09,483][784615] Environment doom_duel already registered, overwriting... [2023-02-24 08:03:09,483][784615] Environment doom_deathmatch_full already registered, overwriting... [2023-02-24 08:03:09,484][784615] Environment doom_benchmark already registered, overwriting... [2023-02-24 08:03:09,484][784615] register_encoder_factory: [2023-02-24 08:03:09,494][784615] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json [2023-02-24 08:03:09,495][784615] Overriding arg 'train_for_env_steps' with value 40000000 passed from command line [2023-02-24 08:03:09,498][784615] Experiment dir /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment already exists! [2023-02-24 08:03:09,499][784615] Resuming existing experiment from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment... [2023-02-24 08:03:09,499][784615] Weights and Biases integration disabled [2023-02-24 08:03:09,500][784615] Environment var CUDA_VISIBLE_DEVICES is 1 [2023-02-24 08:06:20,559][795538] Saving configuration to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json... [2023-02-24 08:06:20,725][795538] Rollout worker 0 uses device cpu [2023-02-24 08:06:20,726][795538] Rollout worker 1 uses device cpu [2023-02-24 08:06:20,726][795538] Rollout worker 2 uses device cpu [2023-02-24 08:06:20,727][795538] Rollout worker 3 uses device cpu [2023-02-24 08:06:20,727][795538] Rollout worker 4 uses device cpu [2023-02-24 08:06:20,727][795538] Rollout worker 5 uses device cpu [2023-02-24 08:06:20,728][795538] Rollout worker 6 uses device cpu [2023-02-24 08:06:20,728][795538] Rollout worker 7 uses device cpu [2023-02-24 08:06:20,767][795538] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 08:06:20,768][795538] InferenceWorker_p0-w0: min num requests: 2 [2023-02-24 08:06:20,819][795538] Starting all processes... [2023-02-24 08:06:20,820][795538] Starting process learner_proc0 [2023-02-24 08:06:20,869][795538] Starting all processes... [2023-02-24 08:06:20,872][795538] Starting process inference_proc0-0 [2023-02-24 08:06:20,873][795538] Starting process rollout_proc0 [2023-02-24 08:06:20,873][795538] Starting process rollout_proc1 [2023-02-24 08:06:20,873][795538] Starting process rollout_proc2 [2023-02-24 08:06:20,873][795538] Starting process rollout_proc3 [2023-02-24 08:06:20,874][795538] Starting process rollout_proc4 [2023-02-24 08:06:20,874][795538] Starting process rollout_proc5 [2023-02-24 08:06:20,874][795538] Starting process rollout_proc6 [2023-02-24 08:06:20,875][795538] Starting process rollout_proc7 [2023-02-24 08:06:22,179][795628] Worker 1 uses CPU cores [1] [2023-02-24 08:06:22,214][795634] Worker 6 uses CPU cores [6] [2023-02-24 08:06:22,319][795626] Low niceness requires sudo! [2023-02-24 08:06:22,319][795626] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 08:06:22,319][795626] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for inference process 0 [2023-02-24 08:06:22,336][795626] Num visible devices: 1 [2023-02-24 08:06:22,343][795613] Low niceness requires sudo! [2023-02-24 08:06:22,343][795613] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 08:06:22,344][795613] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [0]) for learning process 0 [2023-02-24 08:06:22,361][795613] Num visible devices: 1 [2023-02-24 08:06:22,366][795632] Worker 5 uses CPU cores [5] [2023-02-24 08:06:22,394][795613] Starting seed is not provided [2023-02-24 08:06:22,394][795613] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 08:06:22,394][795613] Initializing actor-critic model on device cuda:0 [2023-02-24 08:06:22,395][795613] RunningMeanStd input shape: (3, 72, 128) [2023-02-24 08:06:22,395][795613] RunningMeanStd input shape: (1,) [2023-02-24 08:06:22,409][795613] ConvEncoder: input_channels=3 [2023-02-24 08:06:22,497][795613] Conv encoder output size: 512 [2023-02-24 08:06:22,497][795613] Policy head output size: 512 [2023-02-24 08:06:22,506][795613] Created Actor Critic model with architecture: [2023-02-24 08:06:22,507][795613] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2023-02-24 08:06:22,514][795631] Worker 4 uses CPU cores [4] [2023-02-24 08:06:22,542][795633] Worker 7 uses CPU cores [7] [2023-02-24 08:06:22,548][795630] Worker 3 uses CPU cores [3] [2023-02-24 08:06:22,590][795629] Worker 2 uses CPU cores [2] [2023-02-24 08:06:22,604][795627] Worker 0 uses CPU cores [0] [2023-02-24 08:06:25,136][795613] Using optimizer [2023-02-24 08:06:25,137][795613] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000268_1097728.pth... [2023-02-24 08:06:25,196][795613] Loading model from checkpoint [2023-02-24 08:06:25,198][795613] Loaded experiment state at self.train_step=268, self.env_steps=1097728 [2023-02-24 08:06:25,198][795613] Initialized policy 0 weights for model version 268 [2023-02-24 08:06:25,199][795613] LearnerWorker_p0 finished initialization! [2023-02-24 08:06:25,199][795613] Using GPUs [0] for process 0 (actually maps to GPUs [1]) [2023-02-24 08:06:26,272][795626] RunningMeanStd input shape: (3, 72, 128) [2023-02-24 08:06:26,272][795626] RunningMeanStd input shape: (1,) [2023-02-24 08:06:26,280][795626] ConvEncoder: input_channels=3 [2023-02-24 08:06:26,341][795626] Conv encoder output size: 512 [2023-02-24 08:06:26,342][795626] Policy head output size: 512 [2023-02-24 08:06:27,341][795538] Inference worker 0-0 is ready! [2023-02-24 08:06:27,341][795538] All inference workers are ready! Signal rollout workers to start! [2023-02-24 08:06:27,358][795628] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:06:27,358][795631] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:06:27,358][795627] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:06:27,359][795629] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:06:27,364][795634] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:06:27,381][795632] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:06:27,380][795630] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:06:27,394][795633] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:06:27,569][795627] Decorrelating experience for 0 frames... [2023-02-24 08:06:27,619][795634] Decorrelating experience for 0 frames... [2023-02-24 08:06:27,650][795629] Decorrelating experience for 0 frames... [2023-02-24 08:06:27,665][795630] Decorrelating experience for 0 frames... [2023-02-24 08:06:27,805][795634] Decorrelating experience for 32 frames... [2023-02-24 08:06:27,881][795630] Decorrelating experience for 32 frames... [2023-02-24 08:06:27,882][795627] Decorrelating experience for 32 frames... [2023-02-24 08:06:27,882][795632] Decorrelating experience for 0 frames... [2023-02-24 08:06:28,090][795630] Decorrelating experience for 64 frames... [2023-02-24 08:06:28,092][795632] Decorrelating experience for 32 frames... [2023-02-24 08:06:28,106][795627] Decorrelating experience for 64 frames... [2023-02-24 08:06:28,308][795632] Decorrelating experience for 64 frames... [2023-02-24 08:06:28,376][795628] Decorrelating experience for 0 frames... [2023-02-24 08:06:28,383][795630] Decorrelating experience for 96 frames... [2023-02-24 08:06:28,383][795629] Decorrelating experience for 32 frames... [2023-02-24 08:06:28,395][795627] Decorrelating experience for 96 frames... [2023-02-24 08:06:28,625][795628] Decorrelating experience for 32 frames... [2023-02-24 08:06:28,660][795633] Decorrelating experience for 0 frames... [2023-02-24 08:06:28,720][795632] Decorrelating experience for 96 frames... [2023-02-24 08:06:28,729][795631] Decorrelating experience for 0 frames... [2023-02-24 08:06:28,907][795633] Decorrelating experience for 32 frames... [2023-02-24 08:06:28,955][795628] Decorrelating experience for 64 frames... [2023-02-24 08:06:28,975][795634] Decorrelating experience for 64 frames... [2023-02-24 08:06:29,141][795633] Decorrelating experience for 64 frames... [2023-02-24 08:06:29,169][795631] Decorrelating experience for 32 frames... [2023-02-24 08:06:29,237][795628] Decorrelating experience for 96 frames... [2023-02-24 08:06:29,253][795629] Decorrelating experience for 64 frames... [2023-02-24 08:06:29,425][795631] Decorrelating experience for 64 frames... [2023-02-24 08:06:29,531][795629] Decorrelating experience for 96 frames... [2023-02-24 08:06:29,537][795634] Decorrelating experience for 96 frames... [2023-02-24 08:06:29,541][795633] Decorrelating experience for 96 frames... [2023-02-24 08:06:29,684][795631] Decorrelating experience for 96 frames... [2023-02-24 08:06:29,698][795538] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 1097728. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-02-24 08:06:30,658][795613] Signal inference workers to stop experience collection... [2023-02-24 08:06:30,662][795626] InferenceWorker_p0-w0: stopping experience collection [2023-02-24 08:06:32,942][795613] Signal inference workers to resume experience collection... [2023-02-24 08:06:32,942][795626] InferenceWorker_p0-w0: resuming experience collection [2023-02-24 08:06:34,697][795538] Fps is (10 sec: 5734.4, 60 sec: 5734.4, 300 sec: 5734.4). Total num frames: 1126400. Throughput: 0: 628.8. Samples: 3144. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-02-24 08:06:34,699][795538] Avg episode reward: [(0, '5.223')] [2023-02-24 08:06:35,433][795626] Updated weights for policy 0, policy_version 278 (0.0234) [2023-02-24 08:06:38,065][795626] Updated weights for policy 0, policy_version 288 (0.0006) [2023-02-24 08:06:39,698][795538] Fps is (10 sec: 10649.4, 60 sec: 10649.4, 300 sec: 10649.4). Total num frames: 1204224. Throughput: 0: 2626.3. Samples: 26264. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-02-24 08:06:39,699][795538] Avg episode reward: [(0, '6.797')] [2023-02-24 08:06:39,702][795613] Saving new best policy, reward=6.797! [2023-02-24 08:06:40,702][795626] Updated weights for policy 0, policy_version 298 (0.0006) [2023-02-24 08:06:40,762][795538] Heartbeat connected on Batcher_0 [2023-02-24 08:06:40,764][795538] Heartbeat connected on LearnerWorker_p0 [2023-02-24 08:06:40,771][795538] Heartbeat connected on InferenceWorker_p0-w0 [2023-02-24 08:06:40,775][795538] Heartbeat connected on RolloutWorker_w0 [2023-02-24 08:06:40,777][795538] Heartbeat connected on RolloutWorker_w2 [2023-02-24 08:06:40,778][795538] Heartbeat connected on RolloutWorker_w1 [2023-02-24 08:06:40,779][795538] Heartbeat connected on RolloutWorker_w3 [2023-02-24 08:06:40,783][795538] Heartbeat connected on RolloutWorker_w4 [2023-02-24 08:06:40,784][795538] Heartbeat connected on RolloutWorker_w5 [2023-02-24 08:06:40,819][795538] Heartbeat connected on RolloutWorker_w7 [2023-02-24 08:06:40,825][795538] Heartbeat connected on RolloutWorker_w6 [2023-02-24 08:06:43,242][795626] Updated weights for policy 0, policy_version 308 (0.0007) [2023-02-24 08:06:44,698][795538] Fps is (10 sec: 15564.8, 60 sec: 12288.0, 300 sec: 12288.0). Total num frames: 1282048. Throughput: 0: 2536.4. Samples: 38046. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) [2023-02-24 08:06:44,698][795538] Avg episode reward: [(0, '7.215')] [2023-02-24 08:06:44,700][795613] Saving new best policy, reward=7.215! [2023-02-24 08:06:45,889][795626] Updated weights for policy 0, policy_version 318 (0.0007) [2023-02-24 08:06:48,495][795626] Updated weights for policy 0, policy_version 328 (0.0007) [2023-02-24 08:06:49,698][795538] Fps is (10 sec: 15565.1, 60 sec: 13107.2, 300 sec: 13107.2). Total num frames: 1359872. Throughput: 0: 3079.6. Samples: 61592. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-02-24 08:06:49,699][795538] Avg episode reward: [(0, '8.493')] [2023-02-24 08:06:49,702][795613] Saving new best policy, reward=8.493! [2023-02-24 08:06:51,107][795626] Updated weights for policy 0, policy_version 338 (0.0007) [2023-02-24 08:06:53,724][795626] Updated weights for policy 0, policy_version 348 (0.0007) [2023-02-24 08:06:54,698][795538] Fps is (10 sec: 15564.7, 60 sec: 13598.7, 300 sec: 13598.7). Total num frames: 1437696. Throughput: 0: 3404.3. Samples: 85108. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-02-24 08:06:54,699][795538] Avg episode reward: [(0, '10.669')] [2023-02-24 08:06:54,700][795613] Saving new best policy, reward=10.669! [2023-02-24 08:06:56,371][795626] Updated weights for policy 0, policy_version 358 (0.0007) [2023-02-24 08:06:58,972][795626] Updated weights for policy 0, policy_version 368 (0.0007) [2023-02-24 08:06:59,697][795538] Fps is (10 sec: 15564.8, 60 sec: 13926.4, 300 sec: 13926.4). Total num frames: 1515520. Throughput: 0: 3226.5. Samples: 96796. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-02-24 08:06:59,698][795538] Avg episode reward: [(0, '13.259')] [2023-02-24 08:06:59,701][795613] Saving new best policy, reward=13.259! [2023-02-24 08:07:01,613][795626] Updated weights for policy 0, policy_version 378 (0.0007) [2023-02-24 08:07:04,206][795626] Updated weights for policy 0, policy_version 388 (0.0007) [2023-02-24 08:07:04,698][795538] Fps is (10 sec: 15564.9, 60 sec: 14160.5, 300 sec: 14160.5). Total num frames: 1593344. Throughput: 0: 3437.5. Samples: 120312. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-02-24 08:07:04,698][795538] Avg episode reward: [(0, '13.989')] [2023-02-24 08:07:04,699][795613] Saving new best policy, reward=13.989! [2023-02-24 08:07:06,831][795626] Updated weights for policy 0, policy_version 398 (0.0007) [2023-02-24 08:07:08,206][795538] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 795538], exiting... [2023-02-24 08:07:08,207][795538] Runner profile tree view: main_loop: 47.3880 [2023-02-24 08:07:08,208][795613] Stopping Batcher_0... [2023-02-24 08:07:08,208][795613] Loop batcher_evt_loop terminating... [2023-02-24 08:07:08,208][795538] Collected {0: 1650688}, FPS: 11668.8 [2023-02-24 08:07:08,209][795613] Saving /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000403_1650688.pth... [2023-02-24 08:07:08,216][795632] Stopping RolloutWorker_w5... [2023-02-24 08:07:08,216][795632] Loop rollout_proc5_evt_loop terminating... [2023-02-24 08:07:08,218][795630] Stopping RolloutWorker_w3... [2023-02-24 08:07:08,218][795630] Loop rollout_proc3_evt_loop terminating... [2023-02-24 08:07:08,218][795628] Stopping RolloutWorker_w1... [2023-02-24 08:07:08,218][795633] Stopping RolloutWorker_w7... [2023-02-24 08:07:08,219][795628] Loop rollout_proc1_evt_loop terminating... [2023-02-24 08:07:08,219][795633] Loop rollout_proc7_evt_loop terminating... [2023-02-24 08:07:08,221][795631] Stopping RolloutWorker_w4... [2023-02-24 08:07:08,221][795631] Loop rollout_proc4_evt_loop terminating... [2023-02-24 08:07:08,228][795629] Stopping RolloutWorker_w2... [2023-02-24 08:07:08,228][795629] Loop rollout_proc2_evt_loop terminating... [2023-02-24 08:07:08,230][795627] Stopping RolloutWorker_w0... [2023-02-24 08:07:08,230][795634] Stopping RolloutWorker_w6... [2023-02-24 08:07:08,230][795634] Loop rollout_proc6_evt_loop terminating... [2023-02-24 08:07:08,230][795627] Loop rollout_proc0_evt_loop terminating... [2023-02-24 08:07:08,252][795626] Weights refcount: 2 0 [2023-02-24 08:07:08,256][795626] Stopping InferenceWorker_p0-w0... [2023-02-24 08:07:08,260][795626] Loop inference_proc0-0_evt_loop terminating... [2023-02-24 08:07:08,376][795613] Stopping LearnerWorker_p0... [2023-02-24 08:07:08,377][795613] Loop learner_proc0_evt_loop terminating... [2023-02-24 08:07:38,064][795538] Loading existing experiment configuration from /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/config.json [2023-02-24 08:07:38,065][795538] Overriding arg 'num_workers' with value 1 passed from command line [2023-02-24 08:07:38,065][795538] Adding new argument 'no_render'=True that is not in the saved config file! [2023-02-24 08:07:38,065][795538] Adding new argument 'save_video'=True that is not in the saved config file! [2023-02-24 08:07:38,066][795538] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-02-24 08:07:38,066][795538] Adding new argument 'video_name'=None that is not in the saved config file! [2023-02-24 08:07:38,066][795538] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2023-02-24 08:07:38,067][795538] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-02-24 08:07:38,067][795538] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2023-02-24 08:07:38,068][795538] Adding new argument 'hf_repository'='chqmatteo/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2023-02-24 08:07:38,068][795538] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-02-24 08:07:38,068][795538] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-02-24 08:07:38,068][795538] Adding new argument 'train_script'=None that is not in the saved config file! [2023-02-24 08:07:38,069][795538] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-02-24 08:07:38,069][795538] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-02-24 08:07:38,076][795538] Doom resolution: 160x120, resize resolution: (128, 72) [2023-02-24 08:07:38,077][795538] RunningMeanStd input shape: (3, 72, 128) [2023-02-24 08:07:38,078][795538] RunningMeanStd input shape: (1,) [2023-02-24 08:07:38,086][795538] ConvEncoder: input_channels=3 [2023-02-24 08:07:38,169][795538] Conv encoder output size: 512 [2023-02-24 08:07:38,170][795538] Policy head output size: 512 [2023-02-24 08:07:40,757][795538] Loading state from checkpoint /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000403_1650688.pth... [2023-02-24 08:07:42,930][795538] Num frames 100... [2023-02-24 08:07:43,007][795538] Num frames 200... [2023-02-24 08:07:43,075][795538] Num frames 300... [2023-02-24 08:07:43,138][795538] Num frames 400... [2023-02-24 08:07:43,207][795538] Num frames 500... [2023-02-24 08:07:43,275][795538] Num frames 600... [2023-02-24 08:07:43,349][795538] Num frames 700... [2023-02-24 08:07:43,409][795538] Avg episode rewards: #0: 16.090, true rewards: #0: 7.090 [2023-02-24 08:07:43,410][795538] Avg episode reward: 16.090, avg true_objective: 7.090 [2023-02-24 08:07:43,472][795538] Num frames 800... [2023-02-24 08:07:43,536][795538] Num frames 900... [2023-02-24 08:07:43,605][795538] Num frames 1000... [2023-02-24 08:07:43,689][795538] Num frames 1100... [2023-02-24 08:07:43,790][795538] Avg episode rewards: #0: 10.785, true rewards: #0: 5.785 [2023-02-24 08:07:43,791][795538] Avg episode reward: 10.785, avg true_objective: 5.785 [2023-02-24 08:07:43,819][795538] Num frames 1200... [2023-02-24 08:07:43,885][795538] Num frames 1300... [2023-02-24 08:07:43,956][795538] Num frames 1400... [2023-02-24 08:07:44,024][795538] Num frames 1500... [2023-02-24 08:07:44,102][795538] Num frames 1600... [2023-02-24 08:07:44,182][795538] Num frames 1700... [2023-02-24 08:07:44,280][795538] Avg episode rewards: #0: 10.217, true rewards: #0: 5.883 [2023-02-24 08:07:44,280][795538] Avg episode reward: 10.217, avg true_objective: 5.883 [2023-02-24 08:07:44,305][795538] Num frames 1800... [2023-02-24 08:07:44,373][795538] Num frames 1900... [2023-02-24 08:07:44,443][795538] Num frames 2000... [2023-02-24 08:07:44,519][795538] Num frames 2100... [2023-02-24 08:07:44,607][795538] Num frames 2200... [2023-02-24 08:07:44,678][795538] Num frames 2300... [2023-02-24 08:07:44,753][795538] Num frames 2400... [2023-02-24 08:07:44,838][795538] Num frames 2500... [2023-02-24 08:07:44,919][795538] Num frames 2600... [2023-02-24 08:07:45,012][795538] Avg episode rewards: #0: 12.653, true rewards: #0: 6.652 [2023-02-24 08:07:45,013][795538] Avg episode reward: 12.653, avg true_objective: 6.652 [2023-02-24 08:07:45,040][795538] Num frames 2700... [2023-02-24 08:07:45,115][795538] Num frames 2800... [2023-02-24 08:07:45,187][795538] Num frames 2900... [2023-02-24 08:07:45,261][795538] Num frames 3000... [2023-02-24 08:07:45,381][795538] Num frames 3100... [2023-02-24 08:07:45,448][795538] Num frames 3200... [2023-02-24 08:07:45,513][795538] Num frames 3300... [2023-02-24 08:07:45,581][795538] Num frames 3400... [2023-02-24 08:07:45,652][795538] Num frames 3500... [2023-02-24 08:07:45,717][795538] Num frames 3600... [2023-02-24 08:07:45,790][795538] Num frames 3700... [2023-02-24 08:07:45,860][795538] Num frames 3800... [2023-02-24 08:07:45,935][795538] Num frames 3900... [2023-02-24 08:07:46,006][795538] Num frames 4000... [2023-02-24 08:07:46,078][795538] Num frames 4100... [2023-02-24 08:07:46,213][795538] Avg episode rewards: #0: 17.194, true rewards: #0: 8.394 [2023-02-24 08:07:46,214][795538] Avg episode reward: 17.194, avg true_objective: 8.394 [2023-02-24 08:07:46,216][795538] Num frames 4200... [2023-02-24 08:07:46,291][795538] Num frames 4300... [2023-02-24 08:07:46,369][795538] Num frames 4400... [2023-02-24 08:07:46,449][795538] Num frames 4500... [2023-02-24 08:07:46,522][795538] Num frames 4600... [2023-02-24 08:07:46,591][795538] Num frames 4700... [2023-02-24 08:07:46,662][795538] Num frames 4800... [2023-02-24 08:07:46,732][795538] Num frames 4900... [2023-02-24 08:07:46,831][795538] Avg episode rewards: #0: 16.942, true rewards: #0: 8.275 [2023-02-24 08:07:46,831][795538] Avg episode reward: 16.942, avg true_objective: 8.275 [2023-02-24 08:07:46,855][795538] Num frames 5000... [2023-02-24 08:07:46,919][795538] Num frames 5100... [2023-02-24 08:07:46,986][795538] Num frames 5200... [2023-02-24 08:07:47,056][795538] Num frames 5300... [2023-02-24 08:07:47,141][795538] Num frames 5400... [2023-02-24 08:07:47,210][795538] Num frames 5500... [2023-02-24 08:07:47,278][795538] Num frames 5600... [2023-02-24 08:07:47,382][795538] Avg episode rewards: #0: 16.394, true rewards: #0: 8.109 [2023-02-24 08:07:47,383][795538] Avg episode reward: 16.394, avg true_objective: 8.109 [2023-02-24 08:07:47,406][795538] Num frames 5700... [2023-02-24 08:07:47,483][795538] Num frames 5800... [2023-02-24 08:07:47,567][795538] Num frames 5900... [2023-02-24 08:07:47,636][795538] Num frames 6000... [2023-02-24 08:07:47,706][795538] Num frames 6100... [2023-02-24 08:07:47,783][795538] Num frames 6200... [2023-02-24 08:07:47,861][795538] Num frames 6300... [2023-02-24 08:07:47,935][795538] Num frames 6400... [2023-02-24 08:07:48,009][795538] Num frames 6500... [2023-02-24 08:07:48,093][795538] Num frames 6600... [2023-02-24 08:07:48,166][795538] Num frames 6700... [2023-02-24 08:07:48,237][795538] Num frames 6800... [2023-02-24 08:07:48,313][795538] Num frames 6900... [2023-02-24 08:07:48,383][795538] Num frames 7000... [2023-02-24 08:07:48,450][795538] Num frames 7100... [2023-02-24 08:07:48,516][795538] Num frames 7200... [2023-02-24 08:07:48,594][795538] Num frames 7300... [2023-02-24 08:07:48,669][795538] Num frames 7400... [2023-02-24 08:07:48,742][795538] Num frames 7500... [2023-02-24 08:07:48,814][795538] Num frames 7600... [2023-02-24 08:07:48,895][795538] Num frames 7700... [2023-02-24 08:07:48,996][795538] Avg episode rewards: #0: 21.345, true rewards: #0: 9.720 [2023-02-24 08:07:48,997][795538] Avg episode reward: 21.345, avg true_objective: 9.720 [2023-02-24 08:07:49,018][795538] Num frames 7800... [2023-02-24 08:07:49,092][795538] Num frames 7900... [2023-02-24 08:07:49,171][795538] Num frames 8000... [2023-02-24 08:07:49,241][795538] Num frames 8100... [2023-02-24 08:07:49,311][795538] Num frames 8200... [2023-02-24 08:07:49,424][795538] Avg episode rewards: #0: 20.098, true rewards: #0: 9.209 [2023-02-24 08:07:49,425][795538] Avg episode reward: 20.098, avg true_objective: 9.209 [2023-02-24 08:07:49,434][795538] Num frames 8300... [2023-02-24 08:07:49,502][795538] Num frames 8400... [2023-02-24 08:07:49,568][795538] Num frames 8500... [2023-02-24 08:07:49,643][795538] Num frames 8600... [2023-02-24 08:07:49,707][795538] Num frames 8700... [2023-02-24 08:07:49,763][795538] Avg episode rewards: #0: 18.704, true rewards: #0: 8.704 [2023-02-24 08:07:49,764][795538] Avg episode reward: 18.704, avg true_objective: 8.704 [2023-02-24 08:07:53,791][795538] Replay video saved to /mnt/chqma/data-ssd-01/dataset/oss/RWKV-LM/deep-rl-class/notebooks/unit8/train_dir/default_experiment/replay.mp4!