[2023-09-12 13:21:22,562][09743] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-09-12 13:21:22,562][09743] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-09-12 13:21:22,598][09743] Num visible devices: 1
[2023-09-12 13:21:22,637][09743] Starting seed is not provided
[2023-09-12 13:21:22,638][09743] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-09-12 13:21:22,638][09743] Initializing actor-critic model on device cuda:0
[2023-09-12 13:21:22,638][09743] RunningMeanStd input shape: (3, 72, 128)
[2023-09-12 13:21:22,639][09743] RunningMeanStd input shape: (1,)
[2023-09-12 13:21:22,659][09743] ConvEncoder: input_channels=3
[2023-09-12 13:21:22,911][09743] Conv encoder output size: 512
[2023-09-12 13:21:22,911][09743] Policy head output size: 512
[2023-09-12 13:21:22,935][09743] Created Actor Critic model with architecture:
[2023-09-12 13:21:22,935][09743] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ELU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ELU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ELU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ELU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): GRU(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=4, bias=True)
  )
)
[2023-09-12 13:21:24,096][09743] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-09-12 13:21:24,096][09743] No checkpoints found
[2023-09-12 13:21:24,097][09743] Did not load from checkpoint, starting from scratch!
[2023-09-12 13:21:24,097][09743] Initialized policy 0 weights for model version 0
[2023-09-12 13:21:24,098][09743] LearnerWorker_p0 finished initialization!
[2023-09-12 13:21:24,098][09743] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-09-12 13:21:24,463][09929] Worker 1 uses CPU cores [4, 5, 6, 7]
[2023-09-12 13:21:24,475][09931] Worker 2 uses CPU cores [8, 9, 10, 11]
[2023-09-12 13:21:24,499][09964] Worker 5 uses CPU cores [20, 21, 22, 23]
[2023-09-12 13:21:24,535][09967] Worker 4 uses CPU cores [16, 17, 18, 19]
[2023-09-12 13:21:24,545][09928] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-09-12 13:21:24,545][09928] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-09-12 13:21:24,566][09928] Num visible devices: 1
[2023-09-12 13:21:24,567][09932] Worker 3 uses CPU cores [12, 13, 14, 15]
[2023-09-12 13:21:24,645][09965] Worker 7 uses CPU cores [28, 29, 30, 31]
[2023-09-12 13:21:24,665][09968] Worker 6 uses CPU cores [24, 25, 26, 27]
[2023-09-12 13:21:24,689][09930] Worker 0 uses CPU cores [0, 1, 2, 3]
[2023-09-12 13:21:25,314][09928] RunningMeanStd input shape: (3, 72, 128)
[2023-09-12 13:21:25,315][09928] RunningMeanStd input shape: (1,)
[2023-09-12 13:21:25,326][09928] ConvEncoder: input_channels=3
[2023-09-12 13:21:25,447][09928] Conv encoder output size: 512
[2023-09-12 13:21:25,448][09928] Policy head output size: 512
[2023-09-12 13:21:25,839][09964] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-09-12 13:21:25,839][09968] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-09-12 13:21:25,839][09967] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-09-12 13:21:25,840][09965] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-09-12 13:21:25,840][09931] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-09-12 13:21:25,848][09932] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-09-12 13:21:25,851][09930] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-09-12 13:21:25,852][09929] Doom resolution: 160x120, resize resolution: (128, 72)
[2023-09-12 13:21:26,147][09967] Decorrelating experience for 0 frames...
[2023-09-12 13:21:26,147][09965] Decorrelating experience for 0 frames...
[2023-09-12 13:21:26,215][09964] Decorrelating experience for 0 frames...
[2023-09-12 13:21:26,239][09929] Decorrelating experience for 0 frames...
[2023-09-12 13:21:26,258][09968] Decorrelating experience for 0 frames...
[2023-09-12 13:21:26,273][09931] Decorrelating experience for 0 frames...
[2023-09-12 13:21:26,286][09930] Decorrelating experience for 0 frames...
[2023-09-12 13:21:26,418][09967] Decorrelating experience for 32 frames...
[2023-09-12 13:21:26,493][09964] Decorrelating experience for 32 frames...
[2023-09-12 13:21:26,522][09965] Decorrelating experience for 32 frames...
[2023-09-12 13:21:26,525][09929] Decorrelating experience for 32 frames...
[2023-09-12 13:21:26,551][09932] Decorrelating experience for 0 frames...
[2023-09-12 13:21:26,556][09931] Decorrelating experience for 32 frames...
[2023-09-12 13:21:26,568][09930] Decorrelating experience for 32 frames...
[2023-09-12 13:21:26,775][09967] Decorrelating experience for 64 frames...
[2023-09-12 13:21:26,821][09932] Decorrelating experience for 32 frames...
[2023-09-12 13:21:26,852][09964] Decorrelating experience for 64 frames...
[2023-09-12 13:21:26,919][09931] Decorrelating experience for 64 frames...
[2023-09-12 13:21:26,929][09930] Decorrelating experience for 64 frames...
[2023-09-12 13:21:27,103][09929] Decorrelating experience for 64 frames...
[2023-09-12 13:21:27,160][09968] Decorrelating experience for 32 frames...
[2023-09-12 13:21:27,164][09965] Decorrelating experience for 64 frames...
[2023-09-12 13:21:27,195][09932] Decorrelating experience for 64 frames...
[2023-09-12 13:21:27,201][09964] Decorrelating experience for 96 frames...
[2023-09-12 13:21:27,330][09931] Decorrelating experience for 96 frames...
[2023-09-12 13:21:27,451][09929] Decorrelating experience for 96 frames...
[2023-09-12 13:21:27,465][09967] Decorrelating experience for 96 frames...
[2023-09-12 13:21:27,498][09965] Decorrelating experience for 96 frames...
[2023-09-12 13:21:27,507][09930] Decorrelating experience for 96 frames...
[2023-09-12 13:21:27,588][09968] Decorrelating experience for 64 frames...
[2023-09-12 13:21:27,634][09932] Decorrelating experience for 96 frames...
[2023-09-12 13:21:27,903][09968] Decorrelating experience for 96 frames...
[2023-09-12 13:21:28,649][09743] Signal inference workers to stop experience collection...
[2023-09-12 13:21:28,653][09928] InferenceWorker_p0-w0: stopping experience collection
[2023-09-12 13:21:32,650][09743] Signal inference workers to resume experience collection...
[2023-09-12 13:21:32,651][09928] InferenceWorker_p0-w0: resuming experience collection
[2023-09-12 13:21:35,991][09928] Updated weights for policy 0, policy_version 10 (0.0392)
[2023-09-12 13:21:39,213][09928] Updated weights for policy 0, policy_version 20 (0.0009)
[2023-09-12 13:21:42,363][09928] Updated weights for policy 0, policy_version 30 (0.0009)
[2023-09-12 13:21:42,527][09743] Saving new best policy, reward=-1.655!
[2023-09-12 13:21:45,525][09928] Updated weights for policy 0, policy_version 40 (0.0009)
[2023-09-12 13:21:47,532][09743] Saving new best policy, reward=-0.936!
[2023-09-12 13:21:48,749][09928] Updated weights for policy 0, policy_version 50 (0.0009)
[2023-09-12 13:21:51,927][09928] Updated weights for policy 0, policy_version 60 (0.0008)
[2023-09-12 13:21:52,564][09743] Saving new best policy, reward=0.078!
[2023-09-12 13:21:55,197][09928] Updated weights for policy 0, policy_version 70 (0.0009)
[2023-09-12 13:21:57,529][09743] Saving new best policy, reward=0.521!
[2023-09-12 13:21:58,483][09928] Updated weights for policy 0, policy_version 80 (0.0010)
[2023-09-12 13:22:01,740][09928] Updated weights for policy 0, policy_version 90 (0.0008)
[2023-09-12 13:22:02,527][09743] Saving new best policy, reward=0.599!
[2023-09-12 13:22:05,213][09928] Updated weights for policy 0, policy_version 100 (0.0012)
[2023-09-12 13:22:07,589][09743] Saving new best policy, reward=0.680!
[2023-09-12 13:22:08,636][09928] Updated weights for policy 0, policy_version 110 (0.0017)
[2023-09-12 13:22:11,996][09928] Updated weights for policy 0, policy_version 120 (0.0009)
[2023-09-12 13:22:12,526][09743] Saving new best policy, reward=0.735!
[2023-09-12 13:22:15,370][09928] Updated weights for policy 0, policy_version 130 (0.0008)
[2023-09-12 13:22:17,527][09743] Saving new best policy, reward=0.755!
[2023-09-12 13:22:18,771][09928] Updated weights for policy 0, policy_version 140 (0.0009)
[2023-09-12 13:22:22,142][09928] Updated weights for policy 0, policy_version 150 (0.0009)
[2023-09-12 13:22:22,526][09743] Saving new best policy, reward=0.781!
[2023-09-12 13:22:25,580][09928] Updated weights for policy 0, policy_version 160 (0.0008)
[2023-09-12 13:22:27,573][09743] Saving new best policy, reward=0.791!
[2023-09-12 13:22:28,937][09928] Updated weights for policy 0, policy_version 170 (0.0009)
[2023-09-12 13:22:32,402][09928] Updated weights for policy 0, policy_version 180 (0.0008)
[2023-09-12 13:22:32,526][09743] Saving new best policy, reward=0.794!
[2023-09-12 13:22:35,725][09928] Updated weights for policy 0, policy_version 190 (0.0009)
[2023-09-12 13:22:37,529][09743] Saving new best policy, reward=0.806!
[2023-09-12 13:22:39,128][09928] Updated weights for policy 0, policy_version 200 (0.0010)
[2023-09-12 13:22:42,471][09928] Updated weights for policy 0, policy_version 210 (0.0009)
[2023-09-12 13:22:45,922][09928] Updated weights for policy 0, policy_version 220 (0.0009)
[2023-09-12 13:22:47,533][09743] Saving new best policy, reward=0.815!
[2023-09-12 13:22:49,374][09928] Updated weights for policy 0, policy_version 230 (0.0009)
[2023-09-12 13:22:52,742][09928] Updated weights for policy 0, policy_version 240 (0.0009)
[2023-09-12 13:22:54,862][09743] Stopping Batcher_0...
[2023-09-12 13:22:54,862][09743] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000246_1007616.pth...
[2023-09-12 13:22:54,863][09743] Loop batcher_evt_loop terminating...
[2023-09-12 13:22:54,876][09965] Stopping RolloutWorker_w7...
[2023-09-12 13:22:54,877][09965] Loop rollout_proc7_evt_loop terminating...
[2023-09-12 13:22:54,877][09932] Stopping RolloutWorker_w3...
[2023-09-12 13:22:54,877][09930] Stopping RolloutWorker_w0...
[2023-09-12 13:22:54,877][09968] Stopping RolloutWorker_w6...
[2023-09-12 13:22:54,877][09932] Loop rollout_proc3_evt_loop terminating...
[2023-09-12 13:22:54,877][09930] Loop rollout_proc0_evt_loop terminating...
[2023-09-12 13:22:54,878][09968] Loop rollout_proc6_evt_loop terminating...
[2023-09-12 13:22:54,880][09964] Stopping RolloutWorker_w5...
[2023-09-12 13:22:54,880][09931] Stopping RolloutWorker_w2...
[2023-09-12 13:22:54,880][09964] Loop rollout_proc5_evt_loop terminating...
[2023-09-12 13:22:54,880][09931] Loop rollout_proc2_evt_loop terminating...
[2023-09-12 13:22:54,881][09929] Stopping RolloutWorker_w1...
[2023-09-12 13:22:54,881][09929] Loop rollout_proc1_evt_loop terminating...
[2023-09-12 13:22:54,882][09967] Stopping RolloutWorker_w4...
[2023-09-12 13:22:54,882][09967] Loop rollout_proc4_evt_loop terminating...
[2023-09-12 13:22:54,885][09928] Weights refcount: 2 0
[2023-09-12 13:22:54,887][09928] Stopping InferenceWorker_p0-w0...
[2023-09-12 13:22:54,887][09928] Loop inference_proc0-0_evt_loop terminating...
[2023-09-12 13:22:54,931][09743] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000246_1007616.pth...
[2023-09-12 13:22:55,021][09743] Stopping LearnerWorker_p0...
[2023-09-12 13:22:55,022][09743] Loop learner_proc0_evt_loop terminating...