[2024-08-26 14:51:14,698][98398] Saving configuration to /home/ai24/condaprojects/droid/d0/train_dir/default_experiment/config.json... [2024-08-26 14:51:14,698][98398] Rollout worker 0 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 1 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 2 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 3 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 4 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 5 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 6 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 7 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 8 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 9 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 10 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 11 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 12 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 13 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 14 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 15 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 16 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 17 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 18 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 19 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 20 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 21 uses device cpu [2024-08-26 14:51:14,699][98398] Rollout worker 22 uses device cpu [2024-08-26 14:51:14,700][98398] Rollout worker 23 uses device cpu [2024-08-26 14:51:14,813][98398] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-08-26 14:51:14,813][98398] InferenceWorker_p0-w0: min num requests: 8 [2024-08-26 14:51:14,846][98398] Starting all processes... [2024-08-26 14:51:14,846][98398] Starting process learner_proc0 [2024-08-26 14:51:15,765][98398] Starting all processes... [2024-08-26 14:51:15,767][98522] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-08-26 14:51:15,767][98522] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2024-08-26 14:51:15,775][98398] Starting process inference_proc0-0 [2024-08-26 14:51:15,776][98398] Starting process rollout_proc0 [2024-08-26 14:51:15,776][98398] Starting process rollout_proc1 [2024-08-26 14:51:15,776][98398] Starting process rollout_proc2 [2024-08-26 14:51:15,778][98398] Starting process rollout_proc3 [2024-08-26 14:51:15,780][98398] Starting process rollout_proc4 [2024-08-26 14:51:15,782][98398] Starting process rollout_proc5 [2024-08-26 14:51:15,784][98398] Starting process rollout_proc6 [2024-08-26 14:51:15,785][98398] Starting process rollout_proc7 [2024-08-26 14:51:15,788][98398] Starting process rollout_proc8 [2024-08-26 14:51:15,793][98398] Starting process rollout_proc9 [2024-08-26 14:51:15,795][98398] Starting process rollout_proc10 [2024-08-26 14:51:15,795][98398] Starting process rollout_proc11 [2024-08-26 14:51:15,797][98398] Starting process rollout_proc12 [2024-08-26 14:51:15,797][98398] Starting process rollout_proc13 [2024-08-26 14:51:15,797][98398] Starting process rollout_proc14 [2024-08-26 14:51:15,820][98522] Num visible devices: 1 [2024-08-26 14:51:15,985][98522] Starting seed is not provided [2024-08-26 14:51:15,985][98522] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-08-26 14:51:15,985][98522] Initializing actor-critic model on device cuda:0 [2024-08-26 14:51:15,986][98522] RunningMeanStd input shape: (3, 72, 128) [2024-08-26 14:51:15,992][98522] RunningMeanStd input shape: (1,) [2024-08-26 14:51:15,999][98522] ConvEncoder: input_channels=3 [2024-08-26 14:51:16,205][98522] Conv encoder output size: 512 [2024-08-26 14:51:16,205][98522] Policy head output size: 512 [2024-08-26 14:51:16,232][98522] Created Actor Critic model with architecture: [2024-08-26 14:51:16,233][98522] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2024-08-26 14:51:16,485][98522] Using optimizer [2024-08-26 14:51:17,021][98398] Starting process rollout_proc15 [2024-08-26 14:51:17,025][98579] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,039][98398] Starting process rollout_proc16 [2024-08-26 14:51:17,042][98578] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,086][98398] Starting process rollout_proc17 [2024-08-26 14:51:17,097][98581] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,107][98398] Starting process rollout_proc18 [2024-08-26 14:51:17,108][98398] Starting process rollout_proc19 [2024-08-26 14:51:17,115][98398] Starting process rollout_proc20 [2024-08-26 14:51:17,120][98589] Worker 12 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,127][98580] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,135][98398] Starting process rollout_proc21 [2024-08-26 14:51:17,152][98398] Starting process rollout_proc22 [2024-08-26 14:51:17,152][98577] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,154][98398] Starting process rollout_proc23 [2024-08-26 14:51:17,157][98585] Worker 8 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,176][98597] Worker 13 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,177][98584] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,180][98605] Worker 14 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,182][98576] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-08-26 14:51:17,182][98576] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2024-08-26 14:51:17,199][98576] Num visible devices: 1 [2024-08-26 14:51:17,236][98587] Worker 10 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,246][98583] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,277][98582] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,360][98586] Worker 9 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,401][98522] No checkpoints found [2024-08-26 14:51:17,401][98522] Did not load from checkpoint, starting from scratch! [2024-08-26 14:51:17,401][98522] Initialized policy 0 weights for model version 0 [2024-08-26 14:51:17,402][98588] Worker 11 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:17,405][98522] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2024-08-26 14:51:17,412][98522] LearnerWorker_p0 finished initialization! [2024-08-26 14:51:17,645][98576] RunningMeanStd input shape: (3, 72, 128) [2024-08-26 14:51:17,645][98576] RunningMeanStd input shape: (1,) [2024-08-26 14:51:17,650][98576] ConvEncoder: input_channels=3 [2024-08-26 14:51:17,696][98576] Conv encoder output size: 512 [2024-08-26 14:51:17,696][98576] Policy head output size: 512 [2024-08-26 14:51:18,110][99128] Worker 15 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,193][99354] Worker 23 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,195][99289] Worker 20 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,222][99263] Worker 19 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,225][99160] Worker 16 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,231][99265] Worker 18 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,236][99254] Worker 17 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,237][99353] Worker 22 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,250][98398] Inference worker 0-0 is ready! [2024-08-26 14:51:18,250][98398] All inference workers are ready! Signal rollout workers to start! [2024-08-26 14:51:18,250][98398] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-08-26 14:51:18,251][99322] Worker 21 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31] [2024-08-26 14:51:18,318][98605] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,326][98578] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,326][98584] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,317][98588] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,322][98587] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,328][99289] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,328][98586] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,329][98580] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,329][98597] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,329][99354] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,331][98583] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,331][98589] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,332][98577] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,335][99263] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,339][99128] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,340][98579] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,341][98581] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,342][98582] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,349][98585] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,360][99322] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,365][99265] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,371][99353] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,379][99254] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,382][99160] Doom resolution: 160x120, resize resolution: (128, 72) [2024-08-26 14:51:18,705][98577] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,705][99322] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,705][98580] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,705][98581] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,705][98586] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,707][98579] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,707][98605] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,707][98587] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,708][99265] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,843][98577] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,843][98581] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,846][98587] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,846][98584] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,847][99322] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,850][98580] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,851][98579] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,854][99263] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,888][98583] Decorrelating experience for 0 frames... [2024-08-26 14:51:18,981][98584] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,981][98586] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,986][98581] Decorrelating experience for 64 frames... [2024-08-26 14:51:18,986][99263] Decorrelating experience for 32 frames... [2024-08-26 14:51:18,988][99322] Decorrelating experience for 64 frames... [2024-08-26 14:51:18,988][98605] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,002][98578] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,006][99289] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,046][98587] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,122][98583] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,122][98588] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,128][99353] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,135][98577] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,136][99265] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,142][99160] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,149][98581] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,194][98578] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,258][98583] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,258][98582] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,259][99160] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,260][99128] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,277][98589] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,279][99263] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,288][98586] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,330][98578] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,345][98580] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,397][98579] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,397][98597] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,399][99354] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,403][99353] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,425][99263] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,428][99254] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,434][98586] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,480][98578] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,516][98581] Decorrelating experience for 128 frames... [2024-08-26 14:51:19,525][98597] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,545][98579] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,545][98587] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,563][99353] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,588][99128] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,597][99289] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,650][98585] Decorrelating experience for 0 frames... [2024-08-26 14:51:19,663][98577] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,668][98583] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,683][99354] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,698][99263] Decorrelating experience for 128 frames... [2024-08-26 14:51:19,708][98578] Decorrelating experience for 128 frames... [2024-08-26 14:51:19,727][99128] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,758][98580] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,781][99289] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,781][98584] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,820][98581] Decorrelating experience for 160 frames... [2024-08-26 14:51:19,833][98597] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,856][98577] Decorrelating experience for 128 frames... [2024-08-26 14:51:19,861][98586] Decorrelating experience for 128 frames... [2024-08-26 14:51:19,884][99128] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,913][99322] Decorrelating experience for 96 frames... [2024-08-26 14:51:19,914][99254] Decorrelating experience for 32 frames... [2024-08-26 14:51:19,933][98580] Decorrelating experience for 128 frames... [2024-08-26 14:51:19,972][99354] Decorrelating experience for 64 frames... [2024-08-26 14:51:19,982][98587] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,001][98583] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,004][98588] Decorrelating experience for 32 frames... [2024-08-26 14:51:20,039][99353] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,049][99289] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,055][98577] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,062][98579] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,061][99254] Decorrelating experience for 64 frames... [2024-08-26 14:51:20,098][98578] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,114][99128] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,120][99354] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,126][98581] Decorrelating experience for 192 frames... [2024-08-26 14:51:20,183][98605] Decorrelating experience for 64 frames... [2024-08-26 14:51:20,185][98585] Decorrelating experience for 32 frames... [2024-08-26 14:51:20,221][99353] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,247][99289] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,255][98597] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,259][98579] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,259][99160] Decorrelating experience for 64 frames... [2024-08-26 14:51:20,280][98587] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,304][98588] Decorrelating experience for 64 frames... [2024-08-26 14:51:20,335][98581] Decorrelating experience for 224 frames... [2024-08-26 14:51:20,335][99322] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,350][98583] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,366][98577] Decorrelating experience for 192 frames... [2024-08-26 14:51:20,405][98582] Decorrelating experience for 32 frames... [2024-08-26 14:51:20,413][99128] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,423][98580] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,432][98605] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,468][99160] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,485][99289] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,509][99353] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,562][98589] Decorrelating experience for 32 frames... [2024-08-26 14:51:20,563][98588] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,566][98578] Decorrelating experience for 192 frames... [2024-08-26 14:51:20,567][98581] Decorrelating experience for 256 frames... [2024-08-26 14:51:20,588][98577] Decorrelating experience for 224 frames... [2024-08-26 14:51:20,604][98605] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,635][98587] Decorrelating experience for 192 frames... [2024-08-26 14:51:20,667][98579] Decorrelating experience for 192 frames... [2024-08-26 14:51:20,696][98582] Decorrelating experience for 64 frames... [2024-08-26 14:51:20,706][98589] Decorrelating experience for 64 frames... [2024-08-26 14:51:20,707][99160] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,720][99254] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,722][98580] Decorrelating experience for 192 frames... [2024-08-26 14:51:20,738][99265] Decorrelating experience for 64 frames... [2024-08-26 14:51:20,802][98588] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,822][98581] Decorrelating experience for 288 frames... [2024-08-26 14:51:20,829][99353] Decorrelating experience for 192 frames... [2024-08-26 14:51:20,830][98577] Decorrelating experience for 256 frames... [2024-08-26 14:51:20,838][98585] Decorrelating experience for 64 frames... [2024-08-26 14:51:20,845][98578] Decorrelating experience for 224 frames... [2024-08-26 14:51:20,860][98589] Decorrelating experience for 96 frames... [2024-08-26 14:51:20,860][98587] Decorrelating experience for 224 frames... [2024-08-26 14:51:20,864][99128] Decorrelating experience for 192 frames... [2024-08-26 14:51:20,874][98597] Decorrelating experience for 128 frames... [2024-08-26 14:51:20,944][98580] Decorrelating experience for 224 frames... [2024-08-26 14:51:20,982][98588] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,988][98579] Decorrelating experience for 224 frames... [2024-08-26 14:51:20,989][98605] Decorrelating experience for 160 frames... [2024-08-26 14:51:20,995][99160] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,005][98585] Decorrelating experience for 96 frames... [2024-08-26 14:51:21,059][99353] Decorrelating experience for 224 frames... [2024-08-26 14:51:21,070][98597] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,085][99128] Decorrelating experience for 224 frames... [2024-08-26 14:51:21,100][99263] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,135][99322] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,139][98577] Decorrelating experience for 288 frames... [2024-08-26 14:51:21,142][98578] Decorrelating experience for 256 frames... [2024-08-26 14:51:21,142][98581] Decorrelating experience for 320 frames... [2024-08-26 14:51:21,142][98589] Decorrelating experience for 128 frames... [2024-08-26 14:51:21,231][98580] Decorrelating experience for 256 frames... [2024-08-26 14:51:21,239][98585] Decorrelating experience for 128 frames... [2024-08-26 14:51:21,252][99265] Decorrelating experience for 96 frames... [2024-08-26 14:51:21,283][98579] Decorrelating experience for 256 frames... [2024-08-26 14:51:21,300][98588] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,327][98587] Decorrelating experience for 256 frames... [2024-08-26 14:51:21,331][99128] Decorrelating experience for 256 frames... [2024-08-26 14:51:21,389][99160] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,396][99254] Decorrelating experience for 128 frames... [2024-08-26 14:51:21,397][99322] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,398][98597] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,438][98589] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,444][98585] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,462][98578] Decorrelating experience for 288 frames... [2024-08-26 14:51:21,467][98583] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,522][98579] Decorrelating experience for 288 frames... [2024-08-26 14:51:21,525][99265] Decorrelating experience for 128 frames... [2024-08-26 14:51:21,531][98581] Decorrelating experience for 352 frames... [2024-08-26 14:51:21,579][98588] Decorrelating experience for 224 frames... [2024-08-26 14:51:21,601][98587] Decorrelating experience for 288 frames... [2024-08-26 14:51:21,613][99289] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,616][99263] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,659][99353] Decorrelating experience for 256 frames... [2024-08-26 14:51:21,667][98586] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,696][98577] Decorrelating experience for 320 frames... [2024-08-26 14:51:21,715][98585] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,758][98589] Decorrelating experience for 192 frames... [2024-08-26 14:51:21,770][98582] Decorrelating experience for 96 frames... [2024-08-26 14:51:21,771][99322] Decorrelating experience for 224 frames... [2024-08-26 14:51:21,823][99254] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,838][98588] Decorrelating experience for 256 frames... [2024-08-26 14:51:21,850][99263] Decorrelating experience for 224 frames... [2024-08-26 14:51:21,858][98578] Decorrelating experience for 320 frames... [2024-08-26 14:51:21,934][99265] Decorrelating experience for 160 frames... [2024-08-26 14:51:21,950][98597] Decorrelating experience for 224 frames... [2024-08-26 14:51:21,961][99354] Decorrelating experience for 128 frames... [2024-08-26 14:51:21,963][98585] Decorrelating experience for 224 frames... [2024-08-26 14:51:21,981][98582] Decorrelating experience for 128 frames... [2024-08-26 14:51:21,991][98587] Decorrelating experience for 320 frames... [2024-08-26 14:51:22,003][98577] Decorrelating experience for 352 frames... [2024-08-26 14:51:22,005][98580] Decorrelating experience for 288 frames... [2024-08-26 14:51:22,084][99160] Decorrelating experience for 224 frames... [2024-08-26 14:51:22,091][99254] Decorrelating experience for 192 frames... [2024-08-26 14:51:22,094][98588] Decorrelating experience for 288 frames... [2024-08-26 14:51:22,109][99128] Decorrelating experience for 288 frames... [2024-08-26 14:51:22,152][98589] Decorrelating experience for 224 frames... [2024-08-26 14:51:22,175][98578] Decorrelating experience for 352 frames... [2024-08-26 14:51:22,192][98582] Decorrelating experience for 160 frames... [2024-08-26 14:51:22,211][98597] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,242][99322] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,256][99289] Decorrelating experience for 224 frames... [2024-08-26 14:51:22,258][99263] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,259][98583] Decorrelating experience for 224 frames... [2024-08-26 14:51:22,302][98580] Decorrelating experience for 320 frames... [2024-08-26 14:51:22,319][99265] Decorrelating experience for 192 frames... [2024-08-26 14:51:22,320][98586] Decorrelating experience for 192 frames... [2024-08-26 14:51:22,332][99254] Decorrelating experience for 224 frames... [2024-08-26 14:51:22,394][98588] Decorrelating experience for 320 frames... [2024-08-26 14:51:22,407][98579] Decorrelating experience for 320 frames... [2024-08-26 14:51:22,452][98587] Decorrelating experience for 352 frames... [2024-08-26 14:51:22,466][98582] Decorrelating experience for 192 frames... [2024-08-26 14:51:22,509][99322] Decorrelating experience for 288 frames... [2024-08-26 14:51:22,518][99289] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,542][99160] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,542][98586] Decorrelating experience for 224 frames... [2024-08-26 14:51:22,556][99265] Decorrelating experience for 224 frames... [2024-08-26 14:51:22,567][98583] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,608][99128] Decorrelating experience for 320 frames... [2024-08-26 14:51:22,683][98588] Decorrelating experience for 352 frames... [2024-08-26 14:51:22,695][99254] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,709][98580] Decorrelating experience for 352 frames... [2024-08-26 14:51:22,714][98579] Decorrelating experience for 352 frames... [2024-08-26 14:51:22,723][99354] Decorrelating experience for 160 frames... [2024-08-26 14:51:22,775][98589] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,802][98586] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,817][99265] Decorrelating experience for 256 frames... [2024-08-26 14:51:22,851][99289] Decorrelating experience for 288 frames... [2024-08-26 14:51:22,852][98582] Decorrelating experience for 224 frames... [2024-08-26 14:51:22,853][98583] Decorrelating experience for 288 frames... [2024-08-26 14:51:22,890][98605] Decorrelating experience for 192 frames... [2024-08-26 14:51:22,940][99263] Decorrelating experience for 288 frames... [2024-08-26 14:51:22,971][99160] Decorrelating experience for 288 frames... [2024-08-26 14:51:22,979][99128] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,014][99354] Decorrelating experience for 192 frames... [2024-08-26 14:51:23,024][98584] Decorrelating experience for 96 frames... [2024-08-26 14:51:23,025][98585] Decorrelating experience for 256 frames... [2024-08-26 14:51:23,070][99254] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,073][98586] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,084][98398] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-08-26 14:51:23,085][98398] Avg episode reward: [(0, '0.993')] [2024-08-26 14:51:23,146][99289] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,163][98582] Decorrelating experience for 256 frames... [2024-08-26 14:51:23,163][98589] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,191][98583] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,241][99263] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,246][98605] Decorrelating experience for 224 frames... [2024-08-26 14:51:23,264][99265] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,266][99354] Decorrelating experience for 224 frames... [2024-08-26 14:51:23,318][98585] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,336][98584] Decorrelating experience for 128 frames... [2024-08-26 14:51:23,359][98522] Signal inference workers to stop experience collection... [2024-08-26 14:51:23,365][99353] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,373][98576] InferenceWorker_p0-w0: stopping experience collection [2024-08-26 14:51:23,387][98586] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,428][99160] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,467][99254] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,469][99322] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,482][98589] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,492][98582] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,516][98583] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,524][98605] Decorrelating experience for 256 frames... [2024-08-26 14:51:23,546][98597] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,565][99263] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,585][99354] Decorrelating experience for 256 frames... [2024-08-26 14:51:23,623][99265] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,651][98584] Decorrelating experience for 160 frames... [2024-08-26 14:51:23,666][99353] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,677][99289] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,712][98586] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,729][98585] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,751][99160] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,787][99254] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,802][98582] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,844][99322] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,856][98597] Decorrelating experience for 320 frames... [2024-08-26 14:51:23,874][99354] Decorrelating experience for 288 frames... [2024-08-26 14:51:23,922][99265] Decorrelating experience for 352 frames... [2024-08-26 14:51:23,929][98584] Decorrelating experience for 192 frames... [2024-08-26 14:51:23,993][98589] Decorrelating experience for 352 frames... [2024-08-26 14:51:24,025][99353] Decorrelating experience for 352 frames... [2024-08-26 14:51:24,034][98585] Decorrelating experience for 352 frames... [2024-08-26 14:51:24,064][98522] Signal inference workers to resume experience collection... [2024-08-26 14:51:24,064][98576] InferenceWorker_p0-w0: resuming experience collection [2024-08-26 14:51:24,090][98582] Decorrelating experience for 352 frames... [2024-08-26 14:51:24,135][98597] Decorrelating experience for 352 frames... [2024-08-26 14:51:24,142][99354] Decorrelating experience for 320 frames... [2024-08-26 14:51:24,147][98584] Decorrelating experience for 224 frames... [2024-08-26 14:51:24,152][98605] Decorrelating experience for 288 frames... [2024-08-26 14:51:24,362][98584] Decorrelating experience for 256 frames... [2024-08-26 14:51:24,387][99354] Decorrelating experience for 352 frames... [2024-08-26 14:51:24,388][98605] Decorrelating experience for 320 frames... [2024-08-26 14:51:24,604][98584] Decorrelating experience for 288 frames... [2024-08-26 14:51:24,668][98605] Decorrelating experience for 352 frames... [2024-08-26 14:51:24,880][98584] Decorrelating experience for 320 frames... [2024-08-26 14:51:25,148][98584] Decorrelating experience for 352 frames... [2024-08-26 14:51:25,286][98576] Updated weights for policy 0, policy_version 12 (0.0011) [2024-08-26 14:51:26,204][98576] Updated weights for policy 0, policy_version 25 (0.0009) [2024-08-26 14:51:26,688][98576] Updated weights for policy 0, policy_version 35 (0.0011) [2024-08-26 14:51:27,177][98576] Updated weights for policy 0, policy_version 45 (0.0011) [2024-08-26 14:51:27,611][98576] Updated weights for policy 0, policy_version 55 (0.0009) [2024-08-26 14:51:28,079][98576] Updated weights for policy 0, policy_version 65 (0.0010) [2024-08-26 14:51:28,084][98398] Fps is (10 sec: 27074.1, 60 sec: 27074.1, 300 sec: 27074.1). Total num frames: 266240. Throughput: 0: 4385.1. Samples: 43122. Policy #0 lag: (min: 0.0, avg: 3.0, max: 9.0) [2024-08-26 14:51:28,084][98398] Avg episode reward: [(0, '4.311')] [2024-08-26 14:51:28,094][98522] Saving new best policy, reward=4.311! [2024-08-26 14:51:28,413][98522] Signal inference workers to stop experience collection... (50 times) [2024-08-26 14:51:28,423][98522] Signal inference workers to resume experience collection... (50 times) [2024-08-26 14:51:28,423][98576] InferenceWorker_p0-w0: stopping experience collection (50 times) [2024-08-26 14:51:28,430][98576] InferenceWorker_p0-w0: resuming experience collection (50 times) [2024-08-26 14:51:28,505][98576] Updated weights for policy 0, policy_version 75 (0.0013) [2024-08-26 14:51:29,048][98576] Updated weights for policy 0, policy_version 85 (0.0010) [2024-08-26 14:51:29,632][98576] Updated weights for policy 0, policy_version 96 (0.0009) [2024-08-26 14:51:30,146][98576] Updated weights for policy 0, policy_version 107 (0.0011) [2024-08-26 14:51:30,625][98576] Updated weights for policy 0, policy_version 117 (0.0009) [2024-08-26 14:51:31,066][98576] Updated weights for policy 0, policy_version 127 (0.0011) [2024-08-26 14:51:31,502][98576] Updated weights for policy 0, policy_version 137 (0.0008) [2024-08-26 14:51:31,748][98522] Signal inference workers to stop experience collection... (100 times) [2024-08-26 14:51:31,748][98522] Signal inference workers to resume experience collection... (100 times) [2024-08-26 14:51:31,763][98576] InferenceWorker_p0-w0: stopping experience collection (100 times) [2024-08-26 14:51:31,763][98576] InferenceWorker_p0-w0: resuming experience collection (100 times) [2024-08-26 14:51:31,952][98576] Updated weights for policy 0, policy_version 147 (0.0010) [2024-08-26 14:51:32,469][98576] Updated weights for policy 0, policy_version 157 (0.0010) [2024-08-26 14:51:33,006][98576] Updated weights for policy 0, policy_version 167 (0.0011) [2024-08-26 14:51:33,084][98398] Fps is (10 sec: 69223.1, 60 sec: 46665.3, 300 sec: 46665.3). Total num frames: 692224. Throughput: 0: 11620.7. Samples: 172380. Policy #0 lag: (min: 0.0, avg: 3.9, max: 10.0) [2024-08-26 14:51:33,084][98398] Avg episode reward: [(0, '4.302')] [2024-08-26 14:51:33,460][98576] Updated weights for policy 0, policy_version 177 (0.0016) [2024-08-26 14:51:33,940][98576] Updated weights for policy 0, policy_version 187 (0.0011) [2024-08-26 14:51:34,373][98576] Updated weights for policy 0, policy_version 197 (0.0010) [2024-08-26 14:51:34,809][98398] Heartbeat connected on Batcher_0 [2024-08-26 14:51:34,821][98398] Heartbeat connected on RolloutWorker_w4 [2024-08-26 14:51:34,822][98398] Heartbeat connected on RolloutWorker_w0 [2024-08-26 14:51:34,822][98398] Heartbeat connected on RolloutWorker_w2 [2024-08-26 14:51:34,822][98398] Heartbeat connected on RolloutWorker_w5 [2024-08-26 14:51:34,822][98398] Heartbeat connected on RolloutWorker_w1 [2024-08-26 14:51:34,822][98398] Heartbeat connected on RolloutWorker_w3 [2024-08-26 14:51:34,823][98398] Heartbeat connected on RolloutWorker_w6 [2024-08-26 14:51:34,824][98398] Heartbeat connected on RolloutWorker_w7 [2024-08-26 14:51:34,825][98398] Heartbeat connected on InferenceWorker_p0-w0 [2024-08-26 14:51:34,830][98398] Heartbeat connected on RolloutWorker_w9 [2024-08-26 14:51:34,830][98398] Heartbeat connected on RolloutWorker_w8 [2024-08-26 14:51:34,831][98398] Heartbeat connected on RolloutWorker_w10 [2024-08-26 14:51:34,831][98398] Heartbeat connected on RolloutWorker_w11 [2024-08-26 14:51:34,831][98398] Heartbeat connected on RolloutWorker_w12 [2024-08-26 14:51:34,832][98522] Signal inference workers to stop experience collection... (150 times) [2024-08-26 14:51:34,834][98398] Heartbeat connected on RolloutWorker_w13 [2024-08-26 14:51:34,839][98398] Heartbeat connected on RolloutWorker_w15 [2024-08-26 14:51:34,839][98398] Heartbeat connected on RolloutWorker_w17 [2024-08-26 14:51:34,839][98398] Heartbeat connected on RolloutWorker_w16 [2024-08-26 14:51:34,839][98398] Heartbeat connected on RolloutWorker_w14 [2024-08-26 14:51:34,839][98398] Heartbeat connected on RolloutWorker_w18 [2024-08-26 14:51:34,840][98398] Heartbeat connected on RolloutWorker_w19 [2024-08-26 14:51:34,841][98398] Heartbeat connected on RolloutWorker_w20 [2024-08-26 14:51:34,842][98398] Heartbeat connected on RolloutWorker_w21 [2024-08-26 14:51:34,844][98398] Heartbeat connected on RolloutWorker_w22 [2024-08-26 14:51:34,845][98398] Heartbeat connected on RolloutWorker_w23 [2024-08-26 14:51:34,845][98398] Heartbeat connected on LearnerWorker_p0 [2024-08-26 14:51:34,845][98522] Signal inference workers to resume experience collection... (150 times) [2024-08-26 14:51:34,847][98576] InferenceWorker_p0-w0: stopping experience collection (150 times) [2024-08-26 14:51:34,849][98576] Updated weights for policy 0, policy_version 207 (0.0010) [2024-08-26 14:51:34,856][98576] InferenceWorker_p0-w0: resuming experience collection (150 times) [2024-08-26 14:51:35,268][98576] Updated weights for policy 0, policy_version 217 (0.0008) [2024-08-26 14:51:35,753][98576] Updated weights for policy 0, policy_version 228 (0.0010) [2024-08-26 14:51:36,307][98576] Updated weights for policy 0, policy_version 238 (0.0011) [2024-08-26 14:51:36,826][98576] Updated weights for policy 0, policy_version 248 (0.0010) [2024-08-26 14:51:37,293][98576] Updated weights for policy 0, policy_version 258 (0.0008) [2024-08-26 14:51:37,744][98576] Updated weights for policy 0, policy_version 268 (0.0011) [2024-08-26 14:51:38,084][98398] Fps is (10 sec: 85604.5, 60 sec: 56584.9, 300 sec: 56584.9). Total num frames: 1122304. Throughput: 0: 11944.7. Samples: 236910. Policy #0 lag: (min: 1.0, avg: 4.7, max: 11.0) [2024-08-26 14:51:38,085][98398] Avg episode reward: [(0, '4.527')] [2024-08-26 14:51:38,097][98522] Saving new best policy, reward=4.527! [2024-08-26 14:51:38,220][98576] Updated weights for policy 0, policy_version 278 (0.0015) [2024-08-26 14:51:38,286][98522] Signal inference workers to stop experience collection... (200 times) [2024-08-26 14:51:38,292][98576] InferenceWorker_p0-w0: stopping experience collection (200 times) [2024-08-26 14:51:38,302][98522] Signal inference workers to resume experience collection... (200 times) [2024-08-26 14:51:38,302][98576] InferenceWorker_p0-w0: resuming experience collection (200 times) [2024-08-26 14:51:38,648][98576] Updated weights for policy 0, policy_version 288 (0.0009) [2024-08-26 14:51:39,110][98576] Updated weights for policy 0, policy_version 298 (0.0011) [2024-08-26 14:51:39,547][98576] Updated weights for policy 0, policy_version 308 (0.0009) [2024-08-26 14:51:40,000][98576] Updated weights for policy 0, policy_version 318 (0.0010) [2024-08-26 14:51:40,459][98576] Updated weights for policy 0, policy_version 328 (0.0011) [2024-08-26 14:51:41,041][98576] Updated weights for policy 0, policy_version 338 (0.0010) [2024-08-26 14:51:41,529][98576] Updated weights for policy 0, policy_version 349 (0.0009) [2024-08-26 14:51:41,600][98522] Signal inference workers to stop experience collection... (250 times) [2024-08-26 14:51:41,608][98576] InferenceWorker_p0-w0: stopping experience collection (250 times) [2024-08-26 14:51:41,612][98522] Signal inference workers to resume experience collection... (250 times) [2024-08-26 14:51:41,615][98576] InferenceWorker_p0-w0: resuming experience collection (250 times) [2024-08-26 14:51:41,990][98576] Updated weights for policy 0, policy_version 359 (0.0011) [2024-08-26 14:51:42,402][98576] Updated weights for policy 0, policy_version 369 (0.0011) [2024-08-26 14:51:42,888][98576] Updated weights for policy 0, policy_version 379 (0.0011) [2024-08-26 14:51:43,084][98398] Fps is (10 sec: 87243.5, 60 sec: 63005.3, 300 sec: 63005.3). Total num frames: 1564672. Throughput: 0: 14836.9. Samples: 368460. Policy #0 lag: (min: 0.0, avg: 5.8, max: 10.0) [2024-08-26 14:51:43,085][98398] Avg episode reward: [(0, '4.298')] [2024-08-26 14:51:43,340][98576] Updated weights for policy 0, policy_version 389 (0.0010) [2024-08-26 14:51:43,771][98576] Updated weights for policy 0, policy_version 399 (0.0014) [2024-08-26 14:51:44,310][98576] Updated weights for policy 0, policy_version 409 (0.0010) [2024-08-26 14:51:44,876][98576] Updated weights for policy 0, policy_version 419 (0.0009) [2024-08-26 14:51:45,298][98576] Updated weights for policy 0, policy_version 429 (0.0008) [2024-08-26 14:51:45,357][98522] Signal inference workers to stop experience collection... (300 times) [2024-08-26 14:51:45,364][98522] Signal inference workers to resume experience collection... (300 times) [2024-08-26 14:51:45,371][98576] InferenceWorker_p0-w0: stopping experience collection (300 times) [2024-08-26 14:51:45,371][98576] InferenceWorker_p0-w0: resuming experience collection (300 times) [2024-08-26 14:51:45,756][98576] Updated weights for policy 0, policy_version 439 (0.0010) [2024-08-26 14:51:46,247][98576] Updated weights for policy 0, policy_version 449 (0.0009) [2024-08-26 14:51:46,696][98576] Updated weights for policy 0, policy_version 459 (0.0013) [2024-08-26 14:51:47,167][98576] Updated weights for policy 0, policy_version 469 (0.0011) [2024-08-26 14:51:47,613][98576] Updated weights for policy 0, policy_version 479 (0.0013) [2024-08-26 14:51:48,084][98398] Fps is (10 sec: 87654.9, 60 sec: 66999.2, 300 sec: 66999.2). Total num frames: 1998848. Throughput: 0: 16698.4. Samples: 498180. Policy #0 lag: (min: 2.0, avg: 6.4, max: 12.0) [2024-08-26 14:51:48,085][98398] Avg episode reward: [(0, '4.265')] [2024-08-26 14:51:48,094][98576] Updated weights for policy 0, policy_version 489 (0.0010) [2024-08-26 14:51:48,541][98576] Updated weights for policy 0, policy_version 499 (0.0011) [2024-08-26 14:51:48,881][98522] Signal inference workers to stop experience collection... (350 times) [2024-08-26 14:51:48,888][98576] InferenceWorker_p0-w0: stopping experience collection (350 times) [2024-08-26 14:51:48,891][98522] Signal inference workers to resume experience collection... (350 times) [2024-08-26 14:51:48,894][98576] InferenceWorker_p0-w0: resuming experience collection (350 times) [2024-08-26 14:51:49,083][98576] Updated weights for policy 0, policy_version 509 (0.0012) [2024-08-26 14:51:49,551][98576] Updated weights for policy 0, policy_version 519 (0.0010) [2024-08-26 14:51:50,018][98576] Updated weights for policy 0, policy_version 529 (0.0010) [2024-08-26 14:51:50,479][98576] Updated weights for policy 0, policy_version 539 (0.0010) [2024-08-26 14:51:50,934][98576] Updated weights for policy 0, policy_version 549 (0.0012) [2024-08-26 14:51:51,368][98576] Updated weights for policy 0, policy_version 559 (0.0009) [2024-08-26 14:51:51,809][98576] Updated weights for policy 0, policy_version 569 (0.0010) [2024-08-26 14:51:52,260][98576] Updated weights for policy 0, policy_version 579 (0.0009) [2024-08-26 14:51:52,707][98576] Updated weights for policy 0, policy_version 589 (0.0012) [2024-08-26 14:51:53,084][98398] Fps is (10 sec: 87245.6, 60 sec: 69964.1, 300 sec: 69964.1). Total num frames: 2437120. Throughput: 0: 16196.8. Samples: 564198. Policy #0 lag: (min: 1.0, avg: 5.2, max: 11.0) [2024-08-26 14:51:53,085][98398] Avg episode reward: [(0, '4.609')] [2024-08-26 14:51:53,107][98522] Signal inference workers to stop experience collection... (400 times) [2024-08-26 14:51:53,115][98576] InferenceWorker_p0-w0: stopping experience collection (400 times) [2024-08-26 14:51:53,128][98522] Signal inference workers to resume experience collection... (400 times) [2024-08-26 14:51:53,128][98522] Saving new best policy, reward=4.609! [2024-08-26 14:51:53,128][98576] InferenceWorker_p0-w0: resuming experience collection (400 times) [2024-08-26 14:51:53,314][98576] Updated weights for policy 0, policy_version 599 (0.0012) [2024-08-26 14:51:53,757][98576] Updated weights for policy 0, policy_version 609 (0.0010) [2024-08-26 14:51:54,279][98576] Updated weights for policy 0, policy_version 619 (0.0010) [2024-08-26 14:51:54,731][98576] Updated weights for policy 0, policy_version 629 (0.0011) [2024-08-26 14:51:55,178][98576] Updated weights for policy 0, policy_version 639 (0.0009) [2024-08-26 14:51:55,633][98576] Updated weights for policy 0, policy_version 649 (0.0009) [2024-08-26 14:51:56,087][98576] Updated weights for policy 0, policy_version 659 (0.0011) [2024-08-26 14:51:56,290][98522] Signal inference workers to stop experience collection... (450 times) [2024-08-26 14:51:56,290][98522] Signal inference workers to resume experience collection... (450 times) [2024-08-26 14:51:56,300][98576] InferenceWorker_p0-w0: stopping experience collection (450 times) [2024-08-26 14:51:56,300][98576] InferenceWorker_p0-w0: resuming experience collection (450 times) [2024-08-26 14:51:56,541][98576] Updated weights for policy 0, policy_version 669 (0.0012) [2024-08-26 14:51:57,000][98576] Updated weights for policy 0, policy_version 679 (0.0010) [2024-08-26 14:51:57,537][98576] Updated weights for policy 0, policy_version 689 (0.0008) [2024-08-26 14:51:58,014][98576] Updated weights for policy 0, policy_version 699 (0.0016) [2024-08-26 14:51:58,085][98398] Fps is (10 sec: 86423.5, 60 sec: 71875.6, 300 sec: 71875.6). Total num frames: 2863104. Throughput: 0: 17430.4. Samples: 694326. Policy #0 lag: (min: 0.0, avg: 5.3, max: 11.0) [2024-08-26 14:51:58,085][98398] Avg episode reward: [(0, '4.465')] [2024-08-26 14:51:58,481][98576] Updated weights for policy 0, policy_version 709 (0.0010) [2024-08-26 14:51:58,990][98576] Updated weights for policy 0, policy_version 719 (0.0011) [2024-08-26 14:51:59,426][98576] Updated weights for policy 0, policy_version 729 (0.0009) [2024-08-26 14:51:59,475][98522] Signal inference workers to stop experience collection... (500 times) [2024-08-26 14:51:59,475][98522] Signal inference workers to resume experience collection... (500 times) [2024-08-26 14:51:59,486][98576] InferenceWorker_p0-w0: stopping experience collection (500 times) [2024-08-26 14:51:59,486][98576] InferenceWorker_p0-w0: resuming experience collection (500 times) [2024-08-26 14:51:59,891][98576] Updated weights for policy 0, policy_version 739 (0.0010) [2024-08-26 14:52:00,333][98576] Updated weights for policy 0, policy_version 749 (0.0010) [2024-08-26 14:52:00,749][98576] Updated weights for policy 0, policy_version 759 (0.0011) [2024-08-26 14:52:01,224][98576] Updated weights for policy 0, policy_version 769 (0.0010) [2024-08-26 14:52:01,745][98576] Updated weights for policy 0, policy_version 779 (0.0010) [2024-08-26 14:52:02,203][98576] Updated weights for policy 0, policy_version 789 (0.0011) [2024-08-26 14:52:02,650][98576] Updated weights for policy 0, policy_version 799 (0.0008) [2024-08-26 14:52:03,084][98398] Fps is (10 sec: 86835.0, 60 sec: 73727.1, 300 sec: 73727.1). Total num frames: 3305472. Throughput: 0: 18420.9. Samples: 825882. Policy #0 lag: (min: 0.0, avg: 4.8, max: 10.0) [2024-08-26 14:52:03,085][98398] Avg episode reward: [(0, '4.489')] [2024-08-26 14:52:03,159][98576] Updated weights for policy 0, policy_version 809 (0.0011) [2024-08-26 14:52:03,588][98576] Updated weights for policy 0, policy_version 819 (0.0008) [2024-08-26 14:52:04,051][98576] Updated weights for policy 0, policy_version 829 (0.0009) [2024-08-26 14:52:04,484][98522] Signal inference workers to stop experience collection... (550 times) [2024-08-26 14:52:04,494][98576] InferenceWorker_p0-w0: stopping experience collection (550 times) [2024-08-26 14:52:04,495][98522] Signal inference workers to resume experience collection... (550 times) [2024-08-26 14:52:04,496][98576] Updated weights for policy 0, policy_version 839 (0.0009) [2024-08-26 14:52:04,501][98576] InferenceWorker_p0-w0: resuming experience collection (550 times) [2024-08-26 14:52:04,955][98576] Updated weights for policy 0, policy_version 849 (0.0011) [2024-08-26 14:52:05,446][98576] Updated weights for policy 0, policy_version 859 (0.0008) [2024-08-26 14:52:05,957][98576] Updated weights for policy 0, policy_version 869 (0.0010) [2024-08-26 14:52:06,419][98576] Updated weights for policy 0, policy_version 879 (0.0010) [2024-08-26 14:52:06,904][98576] Updated weights for policy 0, policy_version 889 (0.0011) [2024-08-26 14:52:07,371][98576] Updated weights for policy 0, policy_version 899 (0.0009) [2024-08-26 14:52:07,808][98576] Updated weights for policy 0, policy_version 909 (0.0009) [2024-08-26 14:52:08,084][98398] Fps is (10 sec: 88067.5, 60 sec: 75124.6, 300 sec: 75124.6). Total num frames: 3743744. Throughput: 0: 19803.8. Samples: 891168. Policy #0 lag: (min: 0.0, avg: 4.9, max: 10.0) [2024-08-26 14:52:08,084][98398] Avg episode reward: [(0, '4.461')] [2024-08-26 14:52:08,290][98576] Updated weights for policy 0, policy_version 919 (0.0010) [2024-08-26 14:52:08,721][98576] Updated weights for policy 0, policy_version 929 (0.0008) [2024-08-26 14:52:09,177][98576] Updated weights for policy 0, policy_version 939 (0.0012) [2024-08-26 14:52:09,661][98576] Updated weights for policy 0, policy_version 949 (0.0009) [2024-08-26 14:52:10,127][98576] Updated weights for policy 0, policy_version 959 (0.0011) [2024-08-26 14:52:10,300][98522] Signal inference workers to stop experience collection... (600 times) [2024-08-26 14:52:10,300][98522] Signal inference workers to resume experience collection... (600 times) [2024-08-26 14:52:10,310][98576] InferenceWorker_p0-w0: stopping experience collection (600 times) [2024-08-26 14:52:10,311][98576] InferenceWorker_p0-w0: resuming experience collection (600 times) [2024-08-26 14:52:10,620][98576] Updated weights for policy 0, policy_version 969 (0.0008) [2024-08-26 14:52:11,039][98398] Component Batcher_0 stopped! [2024-08-26 14:52:11,039][98522] Stopping Batcher_0... [2024-08-26 14:52:11,040][98522] Loop batcher_evt_loop terminating... [2024-08-26 14:52:11,040][98522] Saving /home/ai24/condaprojects/droid/d0/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-08-26 14:52:11,054][98576] Weights refcount: 2 0 [2024-08-26 14:52:11,056][98576] Stopping InferenceWorker_p0-w0... [2024-08-26 14:52:11,056][98398] Component InferenceWorker_p0-w0 stopped! [2024-08-26 14:52:11,056][98576] Loop inference_proc0-0_evt_loop terminating... [2024-08-26 14:52:11,086][98522] Saving /home/ai24/condaprojects/droid/d0/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... [2024-08-26 14:52:11,124][99289] Stopping RolloutWorker_w20... [2024-08-26 14:52:11,125][98398] Component RolloutWorker_w20 stopped! [2024-08-26 14:52:11,125][99289] Loop rollout_proc20_evt_loop terminating... [2024-08-26 14:52:11,127][98579] Stopping RolloutWorker_w2... [2024-08-26 14:52:11,127][98398] Component RolloutWorker_w2 stopped! [2024-08-26 14:52:11,127][98579] Loop rollout_proc2_evt_loop terminating... [2024-08-26 14:52:11,128][98398] Component RolloutWorker_w17 stopped! [2024-08-26 14:52:11,128][99254] Stopping RolloutWorker_w17... [2024-08-26 14:52:11,128][99254] Loop rollout_proc17_evt_loop terminating... [2024-08-26 14:52:11,129][98578] Stopping RolloutWorker_w1... [2024-08-26 14:52:11,129][98398] Component RolloutWorker_w1 stopped! [2024-08-26 14:52:11,129][98586] Stopping RolloutWorker_w9... [2024-08-26 14:52:11,129][98398] Component RolloutWorker_w9 stopped! [2024-08-26 14:52:11,129][98578] Loop rollout_proc1_evt_loop terminating... [2024-08-26 14:52:11,129][98582] Stopping RolloutWorker_w4... [2024-08-26 14:52:11,129][98398] Component RolloutWorker_w4 stopped! [2024-08-26 14:52:11,129][98586] Loop rollout_proc9_evt_loop terminating... [2024-08-26 14:52:11,129][98584] Stopping RolloutWorker_w6... [2024-08-26 14:52:11,129][98582] Loop rollout_proc4_evt_loop terminating... [2024-08-26 14:52:11,129][98398] Component RolloutWorker_w6 stopped! [2024-08-26 14:52:11,129][98584] Loop rollout_proc6_evt_loop terminating... [2024-08-26 14:52:11,130][98398] Component RolloutWorker_w10 stopped! [2024-08-26 14:52:11,130][98587] Stopping RolloutWorker_w10... [2024-08-26 14:52:11,130][98587] Loop rollout_proc10_evt_loop terminating... [2024-08-26 14:52:11,141][99322] Stopping RolloutWorker_w21... [2024-08-26 14:52:11,141][98585] Stopping RolloutWorker_w8... [2024-08-26 14:52:11,141][99322] Loop rollout_proc21_evt_loop terminating... [2024-08-26 14:52:11,141][98398] Component RolloutWorker_w21 stopped! [2024-08-26 14:52:11,141][99128] Stopping RolloutWorker_w15... [2024-08-26 14:52:11,141][98585] Loop rollout_proc8_evt_loop terminating... [2024-08-26 14:52:11,141][98522] Stopping LearnerWorker_p0... [2024-08-26 14:52:11,141][98581] Stopping RolloutWorker_w5... [2024-08-26 14:52:11,142][98398] Component RolloutWorker_w8 stopped! [2024-08-26 14:52:11,141][98580] Stopping RolloutWorker_w3... [2024-08-26 14:52:11,142][99128] Loop rollout_proc15_evt_loop terminating... [2024-08-26 14:52:11,142][98398] Component RolloutWorker_w15 stopped! [2024-08-26 14:52:11,142][98522] Loop learner_proc0_evt_loop terminating... [2024-08-26 14:52:11,142][98589] Stopping RolloutWorker_w12... [2024-08-26 14:52:11,142][98581] Loop rollout_proc5_evt_loop terminating... [2024-08-26 14:52:11,142][98577] Stopping RolloutWorker_w0... [2024-08-26 14:52:11,142][98398] Component LearnerWorker_p0 stopped! [2024-08-26 14:52:11,142][99160] Stopping RolloutWorker_w16... [2024-08-26 14:52:11,142][98605] Stopping RolloutWorker_w14... [2024-08-26 14:52:11,142][98588] Stopping RolloutWorker_w11... [2024-08-26 14:52:11,142][98398] Component RolloutWorker_w5 stopped! [2024-08-26 14:52:11,142][98580] Loop rollout_proc3_evt_loop terminating... [2024-08-26 14:52:11,142][98589] Loop rollout_proc12_evt_loop terminating... [2024-08-26 14:52:11,142][98577] Loop rollout_proc0_evt_loop terminating... [2024-08-26 14:52:11,142][99160] Loop rollout_proc16_evt_loop terminating... [2024-08-26 14:52:11,142][98605] Loop rollout_proc14_evt_loop terminating... [2024-08-26 14:52:11,142][98398] Component RolloutWorker_w3 stopped! [2024-08-26 14:52:11,142][98588] Loop rollout_proc11_evt_loop terminating... [2024-08-26 14:52:11,142][98398] Component RolloutWorker_w12 stopped! [2024-08-26 14:52:11,142][98398] Component RolloutWorker_w0 stopped! [2024-08-26 14:52:11,142][98398] Component RolloutWorker_w16 stopped! [2024-08-26 14:52:11,142][99354] Stopping RolloutWorker_w23... [2024-08-26 14:52:11,142][98398] Component RolloutWorker_w14 stopped! [2024-08-26 14:52:11,143][99354] Loop rollout_proc23_evt_loop terminating... [2024-08-26 14:52:11,143][98398] Component RolloutWorker_w11 stopped! [2024-08-26 14:52:11,143][99263] Stopping RolloutWorker_w19... [2024-08-26 14:52:11,143][98398] Component RolloutWorker_w23 stopped! [2024-08-26 14:52:11,143][98398] Component RolloutWorker_w19 stopped! [2024-08-26 14:52:11,143][98597] Stopping RolloutWorker_w13... [2024-08-26 14:52:11,143][99263] Loop rollout_proc19_evt_loop terminating... [2024-08-26 14:52:11,143][98398] Component RolloutWorker_w13 stopped! [2024-08-26 14:52:11,143][98597] Loop rollout_proc13_evt_loop terminating... [2024-08-26 14:52:11,143][98398] Component RolloutWorker_w22 stopped! [2024-08-26 14:52:11,143][99353] Stopping RolloutWorker_w22... [2024-08-26 14:52:11,144][99353] Loop rollout_proc22_evt_loop terminating... [2024-08-26 14:52:11,144][98398] Component RolloutWorker_w7 stopped! [2024-08-26 14:52:11,144][98583] Stopping RolloutWorker_w7... [2024-08-26 14:52:11,144][98583] Loop rollout_proc7_evt_loop terminating... [2024-08-26 14:52:11,147][98398] Component RolloutWorker_w18 stopped! [2024-08-26 14:52:11,147][99265] Stopping RolloutWorker_w18... [2024-08-26 14:52:11,147][98398] Waiting for process learner_proc0 to stop... [2024-08-26 14:52:11,147][99265] Loop rollout_proc18_evt_loop terminating... [2024-08-26 14:52:11,958][98398] Waiting for process inference_proc0-0 to join... [2024-08-26 14:52:11,958][98398] Waiting for process rollout_proc0 to join... [2024-08-26 14:52:11,958][98398] Waiting for process rollout_proc1 to join... [2024-08-26 14:52:11,958][98398] Waiting for process rollout_proc2 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc3 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc4 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc5 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc6 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc7 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc8 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc9 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc10 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc11 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc12 to join... [2024-08-26 14:52:11,959][98398] Waiting for process rollout_proc13 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc14 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc15 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc16 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc17 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc18 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc19 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc20 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc21 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc22 to join... [2024-08-26 14:52:11,960][98398] Waiting for process rollout_proc23 to join... [2024-08-26 14:52:11,961][98398] Batcher 0 profile tree view: batching: 8.8489, releasing_batches: 5.4352 [2024-08-26 14:52:11,961][98398] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0001 wait_policy_total: 5.1096 update_model: 1.3600 weight_update: 0.0010 one_step: 0.0039 handle_policy_step: 44.6662 deserialize: 5.8751, stack: 0.2202, obs_to_device_normalize: 11.8517, forward: 17.3113, send_messages: 3.6109 prepare_outputs: 4.5991 to_cpu: 2.7106 [2024-08-26 14:52:11,961][98398] Learner 0 profile tree view: misc: 0.0035, prepare_batch: 7.8784 train: 18.0238 epoch_init: 0.0032, minibatch_init: 0.0038, losses_postprocess: 0.3948, kl_divergence: 0.4595, after_optimizer: 1.2354 calculate_losses: 8.3610 losses_init: 0.0016, forward_head: 0.9243, bptt_initial: 3.4646, tail: 0.6984, advantages_returns: 0.2220, losses: 1.4552 bptt: 1.4319 bptt_forward_core: 1.3695 update: 7.2580 clip: 0.7661 [2024-08-26 14:52:11,961][98398] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.0191, enqueue_policy_requests: 1.4591, env_step: 29.5664, overhead: 1.6877, complete_rollouts: 0.0252 save_policy_outputs: 1.6654 split_output_tensors: 0.5414 [2024-08-26 14:52:11,961][98398] RolloutWorker_w23 profile tree view: wait_for_trajectories: 0.0188, enqueue_policy_requests: 1.4002, env_step: 29.0792, overhead: 1.6564, complete_rollouts: 0.0229 save_policy_outputs: 1.6343 split_output_tensors: 0.5361 [2024-08-26 14:52:11,961][98398] Loop Runner_EvtLoop terminating... [2024-08-26 14:52:11,961][98398] Runner profile tree view: main_loop: 57.1156 [2024-08-26 14:52:11,961][98398] Collected {0: 4005888}, FPS: 70136.5