sf_log.txt

[2023-02-22 18:52:22,088][18494] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-02-22 18:52:22,094][18494] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-02-22 18:52:22,455][18508] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-02-22 18:52:22,468][18508] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-02-22 18:52:22,636][18513] Worker 4 uses CPU cores [0]
[2023-02-22 18:52:22,646][18516] Worker 6 uses CPU cores [0]
[2023-02-22 18:52:22,654][18510] Worker 2 uses CPU cores [0]
[2023-02-22 18:52:22,720][18512] Worker 3 uses CPU cores [1]
[2023-02-22 18:52:22,733][18509] Worker 0 uses CPU cores [0]
[2023-02-22 18:52:22,750][18515] Worker 7 uses CPU cores [1]
[2023-02-22 18:52:22,921][18511] Worker 1 uses CPU cores [1]
[2023-02-22 18:52:22,924][18514] Worker 5 uses CPU cores [1]
[2023-02-22 18:52:23,261][18508] Num visible devices: 1
[2023-02-22 18:52:23,262][18494] Num visible devices: 1
[2023-02-22 18:52:23,263][18494] Starting seed is not provided
[2023-02-22 18:52:23,263][18494] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-02-22 18:52:23,264][18494] Initializing actor-critic model on device cuda:0
[2023-02-22 18:52:23,265][18494] RunningMeanStd input shape: (3, 72, 128)
[2023-02-22 18:52:23,266][18494] RunningMeanStd input shape: (1,)
[2023-02-22 18:52:23,286][18494] ConvEncoder: input_channels=3
[2023-02-22 18:52:23,468][18494] Conv encoder output size: 512
[2023-02-22 18:52:23,469][18494] Policy head output size: 512
[2023-02-22 18:52:23,492][18494] Created Actor Critic model with architecture:
[2023-02-22 18:52:23,493][18494] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ELU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ELU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ELU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ELU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): GRU(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2023-02-22 18:52:25,754][18494] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-02-22 18:52:25,755][18494] No checkpoints found
[2023-02-22 18:52:25,756][18494] Did not load from checkpoint, starting from scratch!
[2023-02-22 18:52:25,756][18494] Initialized policy 0 weights for model version 0
[2023-02-22 18:52:25,760][18494] LearnerWorker_p0 finished initialization!
[2023-02-22 18:52:25,761][18494] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-02-22 18:52:25,974][18508] RunningMeanStd input shape: (3, 72, 128)
[2023-02-22 18:52:25,975][18508] RunningMeanStd input shape: (1,)
[2023-02-22 18:52:25,991][18508] ConvEncoder: input_channels=3
[2023-02-22 18:52:26,091][18508] Conv encoder output size: 512
[2023-02-22 18:52:26,091][18508] Policy head output size: 512
[2023-02-22 18:52:28,437][18509] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-02-22 18:52:28,444][18510] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-02-22 18:52:28,475][18511] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-02-22 18:52:28,474][18516] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-02-22 18:52:28,477][18515] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-02-22 18:52:28,486][18514] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-02-22 18:52:28,489][18512] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-02-22 18:52:28,491][18513] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-02-22 18:52:29,641][18511] Decorrelating experience for 0 frames...
[2023-02-22 18:52:29,642][18512] Decorrelating experience for 0 frames...
[2023-02-22 18:52:29,643][18514] Decorrelating experience for 0 frames...
[2023-02-22 18:52:29,881][18509] Decorrelating experience for 0 frames...
[2023-02-22 18:52:29,876][18513] Decorrelating experience for 0 frames...
[2023-02-22 18:52:29,885][18510] Decorrelating experience for 0 frames...
[2023-02-22 18:52:29,901][18516] Decorrelating experience for 0 frames...
[2023-02-22 18:52:30,514][18514] Decorrelating experience for 32 frames...
[2023-02-22 18:52:30,560][18515] Decorrelating experience for 0 frames...
[2023-02-22 18:52:30,890][18511] Decorrelating experience for 32 frames...
[2023-02-22 18:52:30,982][18509] Decorrelating experience for 32 frames...
[2023-02-22 18:52:30,985][18513] Decorrelating experience for 32 frames...
[2023-02-22 18:52:30,994][18510] Decorrelating experience for 32 frames...
[2023-02-22 18:52:31,638][18515] Decorrelating experience for 32 frames...
[2023-02-22 18:52:31,764][18516] Decorrelating experience for 32 frames...
[2023-02-22 18:52:32,185][18510] Decorrelating experience for 64 frames...
[2023-02-22 18:52:32,191][18513] Decorrelating experience for 64 frames...
[2023-02-22 18:52:32,387][18514] Decorrelating experience for 64 frames...
[2023-02-22 18:52:32,394][18512] Decorrelating experience for 32 frames...
[2023-02-22 18:52:32,402][18511] Decorrelating experience for 64 frames...
[2023-02-22 18:52:33,107][18515] Decorrelating experience for 64 frames...
[2023-02-22 18:52:33,500][18514] Decorrelating experience for 96 frames...
[2023-02-22 18:52:33,509][18510] Decorrelating experience for 96 frames...
[2023-02-22 18:52:33,513][18513] Decorrelating experience for 96 frames...
[2023-02-22 18:52:33,697][18512] Decorrelating experience for 64 frames...
[2023-02-22 18:52:34,207][18516] Decorrelating experience for 64 frames...
[2023-02-22 18:52:34,214][18509] Decorrelating experience for 64 frames...
[2023-02-22 18:52:34,623][18515] Decorrelating experience for 96 frames...
[2023-02-22 18:52:34,629][18511] Decorrelating experience for 96 frames...
[2023-02-22 18:52:35,339][18512] Decorrelating experience for 96 frames...
[2023-02-22 18:52:36,064][18516] Decorrelating experience for 96 frames...
[2023-02-22 18:52:36,085][18509] Decorrelating experience for 96 frames...
[2023-02-22 18:52:42,053][18494] Signal inference workers to stop experience collection...
[2023-02-22 18:52:42,116][18508] InferenceWorker_p0-w0: stopping experience collection
[2023-02-22 18:52:44,184][18494] Signal inference workers to resume experience collection...
[2023-02-22 18:52:44,187][18508] InferenceWorker_p0-w0: resuming experience collection
[2023-02-22 18:52:51,922][18508] Updated weights for policy 0, policy_version 10 (0.0338)
[2023-02-22 18:53:03,897][18508] Updated weights for policy 0, policy_version 20 (0.0021)
[2023-02-22 18:53:10,170][18494] Saving new best policy, reward=4.191!
[2023-02-22 18:53:12,948][18508] Updated weights for policy 0, policy_version 30 (0.0014)
[2023-02-22 18:53:15,156][18494] Saving new best policy, reward=4.205!
[2023-02-22 18:53:24,545][18508] Updated weights for policy 0, policy_version 40 (0.0025)
[2023-02-22 18:53:25,231][18494] Saving new best policy, reward=4.251!
[2023-02-22 18:53:30,169][18494] Saving new best policy, reward=4.532!
[2023-02-22 18:53:33,956][18508] Updated weights for policy 0, policy_version 50 (0.0026)
[2023-02-22 18:53:45,138][18508] Updated weights for policy 0, policy_version 60 (0.0016)
[2023-02-22 18:53:50,186][18494] Saving new best policy, reward=4.588!
[2023-02-22 18:53:54,603][18508] Updated weights for policy 0, policy_version 70 (0.0023)
[2023-02-22 18:54:05,216][18494] Saving new best policy, reward=4.629!
[2023-02-22 18:54:05,221][18508] Updated weights for policy 0, policy_version 80 (0.0030)
[2023-02-22 18:54:10,165][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000083_339968.pth...
[2023-02-22 18:54:19,873][18508] Updated weights for policy 0, policy_version 90 (0.0027)
[2023-02-22 18:54:25,154][18494] Saving new best policy, reward=4.694!
[2023-02-22 18:54:29,311][18508] Updated weights for policy 0, policy_version 100 (0.0017)
[2023-02-22 18:54:30,189][18494] Saving new best policy, reward=4.702!
[2023-02-22 18:54:40,931][18508] Updated weights for policy 0, policy_version 110 (0.0017)
[2023-02-22 18:54:45,156][18494] Saving new best policy, reward=4.806!
[2023-02-22 18:54:49,636][18508] Updated weights for policy 0, policy_version 120 (0.0030)
[2023-02-22 18:55:00,261][18494] Saving new best policy, reward=4.984!
[2023-02-22 18:55:01,559][18508] Updated weights for policy 0, policy_version 130 (0.0013)
[2023-02-22 18:55:10,154][18508] Updated weights for policy 0, policy_version 140 (0.0011)
[2023-02-22 18:55:20,171][18494] Saving new best policy, reward=5.100!
[2023-02-22 18:55:21,905][18508] Updated weights for policy 0, policy_version 150 (0.0019)
[2023-02-22 18:55:25,160][18494] Saving new best policy, reward=5.200!
[2023-02-22 18:55:30,400][18508] Updated weights for policy 0, policy_version 160 (0.0013)
[2023-02-22 18:55:42,165][18508] Updated weights for policy 0, policy_version 170 (0.0020)
[2023-02-22 18:55:50,168][18494] Saving new best policy, reward=5.430!
[2023-02-22 18:55:50,642][18508] Updated weights for policy 0, policy_version 180 (0.0018)
[2023-02-22 18:55:55,162][18494] Saving new best policy, reward=5.935!
[2023-02-22 18:56:02,727][18508] Updated weights for policy 0, policy_version 190 (0.0014)
[2023-02-22 18:56:05,160][18494] Saving new best policy, reward=5.956!
[2023-02-22 18:56:10,166][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000198_811008.pth...
[2023-02-22 18:56:10,313][18494] Saving new best policy, reward=6.201!
[2023-02-22 18:56:11,589][18508] Updated weights for policy 0, policy_version 200 (0.0015)
[2023-02-22 18:56:15,158][18494] Saving new best policy, reward=6.805!
[2023-02-22 18:56:20,171][18494] Saving new best policy, reward=6.821!
[2023-02-22 18:56:23,530][18508] Updated weights for policy 0, policy_version 210 (0.0012)
[2023-02-22 18:56:25,161][18494] Saving new best policy, reward=6.914!
[2023-02-22 18:56:30,224][18494] Saving new best policy, reward=6.963!
[2023-02-22 18:56:32,783][18508] Updated weights for policy 0, policy_version 220 (0.0028)
[2023-02-22 18:56:35,160][18494] Saving new best policy, reward=7.309!
[2023-02-22 18:56:43,943][18508] Updated weights for policy 0, policy_version 230 (0.0021)
[2023-02-22 18:56:53,708][18508] Updated weights for policy 0, policy_version 240 (0.0012)
[2023-02-22 18:57:04,323][18508] Updated weights for policy 0, policy_version 250 (0.0014)
[2023-02-22 18:57:14,724][18508] Updated weights for policy 0, policy_version 260 (0.0012)
[2023-02-22 18:57:20,167][18494] Saving new best policy, reward=7.435!
[2023-02-22 18:57:25,048][18508] Updated weights for policy 0, policy_version 270 (0.0017)
[2023-02-22 18:57:35,159][18494] Saving new best policy, reward=7.552!
[2023-02-22 18:57:35,781][18508] Updated weights for policy 0, policy_version 280 (0.0016)
[2023-02-22 18:57:40,169][18494] Saving new best policy, reward=7.882!
[2023-02-22 18:57:45,158][18494] Saving new best policy, reward=8.241!
[2023-02-22 18:57:45,586][18508] Updated weights for policy 0, policy_version 290 (0.0015)
[2023-02-22 18:57:55,163][18494] Saving new best policy, reward=8.245!
[2023-02-22 18:57:56,872][18508] Updated weights for policy 0, policy_version 300 (0.0025)
[2023-02-22 18:58:00,167][18494] Saving new best policy, reward=9.512!
[2023-02-22 18:58:05,157][18494] Saving new best policy, reward=10.187!
[2023-02-22 18:58:06,102][18508] Updated weights for policy 0, policy_version 310 (0.0013)
[2023-02-22 18:58:10,176][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000314_1286144.pth...
[2023-02-22 18:58:10,377][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000083_339968.pth
[2023-02-22 18:58:10,391][18494] Saving new best policy, reward=11.235!
[2023-02-22 18:58:15,154][18494] Saving new best policy, reward=12.271!
[2023-02-22 18:58:18,369][18508] Updated weights for policy 0, policy_version 320 (0.0032)
[2023-02-22 18:58:20,164][18494] Saving new best policy, reward=12.977!
[2023-02-22 18:58:25,159][18494] Saving new best policy, reward=13.710!
[2023-02-22 18:58:26,845][18508] Updated weights for policy 0, policy_version 330 (0.0013)
[2023-02-22 18:58:39,520][18508] Updated weights for policy 0, policy_version 340 (0.0019)
[2023-02-22 18:58:52,509][18508] Updated weights for policy 0, policy_version 350 (0.0018)
[2023-02-22 18:58:55,199][18494] Saving new best policy, reward=13.881!
[2023-02-22 18:59:00,167][18494] Saving new best policy, reward=15.240!
[2023-02-22 18:59:02,958][18508] Updated weights for policy 0, policy_version 360 (0.0014)
[2023-02-22 18:59:10,168][18494] Saving new best policy, reward=15.378!
[2023-02-22 18:59:13,687][18508] Updated weights for policy 0, policy_version 370 (0.0011)
[2023-02-22 18:59:15,167][18494] Saving new best policy, reward=16.206!
[2023-02-22 18:59:23,567][18508] Updated weights for policy 0, policy_version 380 (0.0019)
[2023-02-22 18:59:25,162][18494] Saving new best policy, reward=16.728!
[2023-02-22 18:59:30,168][18494] Saving new best policy, reward=17.859!
[2023-02-22 18:59:34,808][18508] Updated weights for policy 0, policy_version 390 (0.0012)
[2023-02-22 18:59:40,161][18494] Saving new best policy, reward=18.218!
[2023-02-22 18:59:44,145][18508] Updated weights for policy 0, policy_version 400 (0.0021)
[2023-02-22 18:59:45,155][18494] Saving new best policy, reward=18.396!
[2023-02-22 18:59:50,170][18494] Saving new best policy, reward=19.251!
[2023-02-22 18:59:55,604][18508] Updated weights for policy 0, policy_version 410 (0.0013)
[2023-02-22 19:00:04,352][18508] Updated weights for policy 0, policy_version 420 (0.0011)
[2023-02-22 19:00:10,162][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000425_1740800.pth...
[2023-02-22 19:00:10,307][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000198_811008.pth
[2023-02-22 19:00:16,510][18508] Updated weights for policy 0, policy_version 430 (0.0016)
[2023-02-22 19:00:20,169][18494] Saving new best policy, reward=19.433!
[2023-02-22 19:00:24,918][18508] Updated weights for policy 0, policy_version 440 (0.0023)
[2023-02-22 19:00:25,159][18494] Saving new best policy, reward=19.548!
[2023-02-22 19:00:36,903][18508] Updated weights for policy 0, policy_version 450 (0.0011)
[2023-02-22 19:00:45,179][18508] Updated weights for policy 0, policy_version 460 (0.0019)
[2023-02-22 19:00:57,269][18508] Updated weights for policy 0, policy_version 470 (0.0012)
[2023-02-22 19:01:05,633][18508] Updated weights for policy 0, policy_version 480 (0.0013)
[2023-02-22 19:01:17,487][18508] Updated weights for policy 0, policy_version 490 (0.0023)
[2023-02-22 19:01:25,180][18494] Saving new best policy, reward=20.348!
[2023-02-22 19:01:26,585][18508] Updated weights for policy 0, policy_version 500 (0.0016)
[2023-02-22 19:01:30,170][18494] Saving new best policy, reward=20.421!
[2023-02-22 19:01:37,786][18508] Updated weights for policy 0, policy_version 510 (0.0016)
[2023-02-22 19:01:45,167][18494] Saving new best policy, reward=20.458!
[2023-02-22 19:01:47,346][18508] Updated weights for policy 0, policy_version 520 (0.0030)
[2023-02-22 19:01:50,193][18494] Saving new best policy, reward=21.617!
[2023-02-22 19:01:55,159][18494] Saving new best policy, reward=22.472!
[2023-02-22 19:01:58,324][18508] Updated weights for policy 0, policy_version 530 (0.0018)
[2023-02-22 19:02:00,170][18494] Saving new best policy, reward=22.824!
[2023-02-22 19:02:08,289][18508] Updated weights for policy 0, policy_version 540 (0.0011)
[2023-02-22 19:02:10,171][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000541_2215936.pth...
[2023-02-22 19:02:10,309][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000314_1286144.pth
[2023-02-22 19:02:18,832][18508] Updated weights for policy 0, policy_version 550 (0.0028)
[2023-02-22 19:02:29,136][18508] Updated weights for policy 0, policy_version 560 (0.0015)
[2023-02-22 19:02:38,966][18508] Updated weights for policy 0, policy_version 570 (0.0027)
[2023-02-22 19:02:50,118][18508] Updated weights for policy 0, policy_version 580 (0.0019)
[2023-02-22 19:02:59,465][18508] Updated weights for policy 0, policy_version 590 (0.0014)
[2023-02-22 19:03:11,008][18508] Updated weights for policy 0, policy_version 600 (0.0013)
[2023-02-22 19:03:23,769][18508] Updated weights for policy 0, policy_version 610 (0.0029)
[2023-02-22 19:03:35,094][18508] Updated weights for policy 0, policy_version 620 (0.0034)
[2023-02-22 19:03:44,110][18508] Updated weights for policy 0, policy_version 630 (0.0026)
[2023-02-22 19:03:55,449][18508] Updated weights for policy 0, policy_version 640 (0.0021)
[2023-02-22 19:04:04,961][18508] Updated weights for policy 0, policy_version 650 (0.0022)
[2023-02-22 19:04:10,173][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000653_2674688.pth...
[2023-02-22 19:04:10,350][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000425_1740800.pth
[2023-02-22 19:04:15,773][18508] Updated weights for policy 0, policy_version 660 (0.0016)
[2023-02-22 19:04:25,669][18508] Updated weights for policy 0, policy_version 670 (0.0011)
[2023-02-22 19:04:36,111][18508] Updated weights for policy 0, policy_version 680 (0.0020)
[2023-02-22 19:04:47,035][18508] Updated weights for policy 0, policy_version 690 (0.0032)
[2023-02-22 19:04:55,222][18494] Saving new best policy, reward=23.394!
[2023-02-22 19:04:56,779][18508] Updated weights for policy 0, policy_version 700 (0.0015)
[2023-02-22 19:05:08,006][18508] Updated weights for policy 0, policy_version 710 (0.0013)
[2023-02-22 19:05:17,384][18508] Updated weights for policy 0, policy_version 720 (0.0020)
[2023-02-22 19:05:28,657][18508] Updated weights for policy 0, policy_version 730 (0.0022)
[2023-02-22 19:05:37,495][18508] Updated weights for policy 0, policy_version 740 (0.0036)
[2023-02-22 19:05:49,212][18508] Updated weights for policy 0, policy_version 750 (0.0015)
[2023-02-22 19:05:57,872][18508] Updated weights for policy 0, policy_version 760 (0.0021)
[2023-02-22 19:06:09,765][18508] Updated weights for policy 0, policy_version 770 (0.0044)
[2023-02-22 19:06:10,168][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000770_3153920.pth...
[2023-02-22 19:06:10,307][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000541_2215936.pth
[2023-02-22 19:06:18,309][18508] Updated weights for policy 0, policy_version 780 (0.0015)
[2023-02-22 19:06:30,077][18508] Updated weights for policy 0, policy_version 790 (0.0015)
[2023-02-22 19:06:38,169][18508] Updated weights for policy 0, policy_version 800 (0.0018)
[2023-02-22 19:06:45,158][18494] Saving new best policy, reward=24.124!
[2023-02-22 19:06:50,166][18494] Saving new best policy, reward=24.920!
[2023-02-22 19:06:50,533][18508] Updated weights for policy 0, policy_version 810 (0.0039)
[2023-02-22 19:06:55,155][18494] Saving new best policy, reward=26.242!
[2023-02-22 19:06:59,472][18508] Updated weights for policy 0, policy_version 820 (0.0015)
[2023-02-22 19:07:10,964][18508] Updated weights for policy 0, policy_version 830 (0.0031)
[2023-02-22 19:07:20,255][18508] Updated weights for policy 0, policy_version 840 (0.0026)
[2023-02-22 19:07:31,400][18508] Updated weights for policy 0, policy_version 850 (0.0018)
[2023-02-22 19:07:40,844][18508] Updated weights for policy 0, policy_version 860 (0.0014)
[2023-02-22 19:07:50,165][18494] Saving new best policy, reward=26.314!
[2023-02-22 19:07:55,359][18508] Updated weights for policy 0, policy_version 870 (0.0042)
[2023-02-22 19:08:05,366][18508] Updated weights for policy 0, policy_version 880 (0.0017)
[2023-02-22 19:08:10,176][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000883_3616768.pth...
[2023-02-22 19:08:10,356][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000653_2674688.pth
[2023-02-22 19:08:15,695][18508] Updated weights for policy 0, policy_version 890 (0.0013)
[2023-02-22 19:08:26,420][18508] Updated weights for policy 0, policy_version 900 (0.0031)
[2023-02-22 19:08:35,909][18508] Updated weights for policy 0, policy_version 910 (0.0037)
[2023-02-22 19:08:46,932][18508] Updated weights for policy 0, policy_version 920 (0.0032)
[2023-02-22 19:08:56,357][18508] Updated weights for policy 0, policy_version 930 (0.0014)
[2023-02-22 19:09:07,688][18508] Updated weights for policy 0, policy_version 940 (0.0013)
[2023-02-22 19:09:16,391][18508] Updated weights for policy 0, policy_version 950 (0.0011)
[2023-02-22 19:09:28,579][18508] Updated weights for policy 0, policy_version 960 (0.0011)
[2023-02-22 19:09:36,897][18508] Updated weights for policy 0, policy_version 970 (0.0014)
[2023-02-22 19:09:48,991][18508] Updated weights for policy 0, policy_version 980 (0.0033)
[2023-02-22 19:09:57,581][18508] Updated weights for policy 0, policy_version 990 (0.0011)
[2023-02-22 19:10:09,423][18508] Updated weights for policy 0, policy_version 1000 (0.0026)
[2023-02-22 19:10:10,165][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001001_4100096.pth...
[2023-02-22 19:10:10,282][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000770_3153920.pth
[2023-02-22 19:10:17,695][18508] Updated weights for policy 0, policy_version 1010 (0.0012)
[2023-02-22 19:10:29,555][18508] Updated weights for policy 0, policy_version 1020 (0.0020)
[2023-02-22 19:10:38,054][18508] Updated weights for policy 0, policy_version 1030 (0.0026)
[2023-02-22 19:10:49,652][18508] Updated weights for policy 0, policy_version 1040 (0.0028)
[2023-02-22 19:10:58,506][18508] Updated weights for policy 0, policy_version 1050 (0.0012)
[2023-02-22 19:11:10,211][18508] Updated weights for policy 0, policy_version 1060 (0.0012)
[2023-02-22 19:11:19,404][18508] Updated weights for policy 0, policy_version 1070 (0.0011)
[2023-02-22 19:11:30,700][18508] Updated weights for policy 0, policy_version 1080 (0.0023)
[2023-02-22 19:11:39,896][18508] Updated weights for policy 0, policy_version 1090 (0.0033)
[2023-02-22 19:11:50,890][18508] Updated weights for policy 0, policy_version 1100 (0.0011)
[2023-02-22 19:12:00,614][18508] Updated weights for policy 0, policy_version 1110 (0.0015)
[2023-02-22 19:12:10,171][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001118_4579328.pth...
[2023-02-22 19:12:10,311][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000883_3616768.pth
[2023-02-22 19:12:12,804][18508] Updated weights for policy 0, policy_version 1120 (0.0020)
[2023-02-22 19:12:26,550][18508] Updated weights for policy 0, policy_version 1130 (0.0018)
[2023-02-22 19:12:35,018][18508] Updated weights for policy 0, policy_version 1140 (0.0018)
[2023-02-22 19:12:46,742][18508] Updated weights for policy 0, policy_version 1150 (0.0021)
[2023-02-22 19:12:55,110][18508] Updated weights for policy 0, policy_version 1160 (0.0018)
[2023-02-22 19:13:07,263][18508] Updated weights for policy 0, policy_version 1170 (0.0016)
[2023-02-22 19:13:15,160][18494] Saving new best policy, reward=26.727!
[2023-02-22 19:13:15,694][18508] Updated weights for policy 0, policy_version 1180 (0.0014)
[2023-02-22 19:13:27,579][18508] Updated weights for policy 0, policy_version 1190 (0.0021)
[2023-02-22 19:13:36,030][18508] Updated weights for policy 0, policy_version 1200 (0.0028)
[2023-02-22 19:13:47,959][18508] Updated weights for policy 0, policy_version 1210 (0.0020)
[2023-02-22 19:13:56,937][18508] Updated weights for policy 0, policy_version 1220 (0.0017)
[2023-02-22 19:13:59,627][18494] Stopping Batcher_0...
[2023-02-22 19:13:59,629][18494] Loop batcher_evt_loop terminating...
[2023-02-22 19:13:59,635][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth...
[2023-02-22 19:13:59,669][18512] Stopping RolloutWorker_w3...
[2023-02-22 19:13:59,686][18511] Stopping RolloutWorker_w1...
[2023-02-22 19:13:59,691][18508] Weights refcount: 2 0
[2023-02-22 19:13:59,687][18511] Loop rollout_proc1_evt_loop terminating...
[2023-02-22 19:13:59,682][18512] Loop rollout_proc3_evt_loop terminating...
[2023-02-22 19:13:59,695][18508] Stopping InferenceWorker_p0-w0...
[2023-02-22 19:13:59,700][18508] Loop inference_proc0-0_evt_loop terminating...
[2023-02-22 19:13:59,714][18514] Stopping RolloutWorker_w5...
[2023-02-22 19:13:59,715][18514] Loop rollout_proc5_evt_loop terminating...
[2023-02-22 19:13:59,724][18515] Stopping RolloutWorker_w7...
[2023-02-22 19:13:59,728][18515] Loop rollout_proc7_evt_loop terminating...
[2023-02-22 19:13:59,832][18513] Stopping RolloutWorker_w4...
[2023-02-22 19:13:59,833][18513] Loop rollout_proc4_evt_loop terminating...
[2023-02-22 19:13:59,839][18509] Stopping RolloutWorker_w0...
[2023-02-22 19:13:59,844][18494] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001001_4100096.pth
[2023-02-22 19:13:59,862][18509] Loop rollout_proc0_evt_loop terminating...
[2023-02-22 19:13:59,864][18494] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001222_5005312.pth...
[2023-02-22 19:13:59,879][18510] Stopping RolloutWorker_w2...
[2023-02-22 19:13:59,889][18510] Loop rollout_proc2_evt_loop terminating...
[2023-02-22 19:13:59,900][18516] Stopping RolloutWorker_w6...
[2023-02-22 19:13:59,913][18516] Loop rollout_proc6_evt_loop terminating...
[2023-02-22 19:14:00,175][18494] Stopping LearnerWorker_p0...
[2023-02-22 19:14:00,176][18494] Loop learner_proc0_evt_loop terminating...