Hugging Face's logo

MattStammers
/
vizdoom_deadly_corridor

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files Metrics Community
MattStammers commited on
Commit
b7d69bb
•
1 Parent(s): c2d6f88

Upload folder using huggingface_hub

Browse files
Files changed (14) hide show
  1. .gitattributes +1 -0
  2. .summary/0/events.out.tfevents.1694524907.rhmmedcatt-ProLiant-ML350-Gen10 +3 -0
  3. .summary/1/events.out.tfevents.1694524907.rhmmedcatt-ProLiant-ML350-Gen10 +3 -0
  4. README.md +56 -0
  5. checkpoint_p0/best_000002847_11661312_reward_21.699.pth +3 -0
  6. checkpoint_p0/checkpoint_000002853_11685888.pth +3 -0
  7. checkpoint_p0/checkpoint_000002874_11771904.pth +3 -0
  8. checkpoint_p1/best_000002435_9973760_reward_18.719.pth +3 -0
  9. checkpoint_p1/checkpoint_000002419_9908224.pth +3 -0
  10. checkpoint_p1/checkpoint_000002442_10002432.pth +3 -0
  11. config.json +143 -0
  12. git.diff +0 -0
  13. replay.mp4 +3 -0
  14. sf_log.txt +920 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ replay.mp4 filter=lfs diff=lfs merge=lfs -text
.summary/0/events.out.tfevents.1694524907.rhmmedcatt-ProLiant-ML350-Gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7132bf77bba7fa440fe35773b4d6c3c54d99ca24de267c03e60b43ec3d08cc40
3
+ size 1348009
.summary/1/events.out.tfevents.1694524907.rhmmedcatt-ProLiant-ML350-Gen10 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e126149dc195b5edcab614968c99a4e19aa2e5302f1ffadbaf02921877c0280
3
+ size 1248965
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sample-factory
3
+ tags:
4
+ - deep-reinforcement-learning
5
+ - reinforcement-learning
6
+ - sample-factory
7
+ model-index:
8
+ - name: APPO
9
+ results:
10
+ - task:
11
+ type: reinforcement-learning
12
+ name: reinforcement-learning
13
+ dataset:
14
+ name: doom_deadly_corridor
15
+ type: doom_deadly_corridor
16
+ metrics:
17
+ - type: mean_reward
18
+ value: 20.86 +/- 5.79
19
+ name: mean_reward
20
+ verified: false
21
+ ---
22
+
23
+ A(n) **APPO** model trained on the **doom_deadly_corridor** environment.
24
+
25
+ This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
26
+ Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
27
+
28
+
29
+ ## Downloading the model
30
+
31
+ After installing Sample-Factory, download the model with:
32
+ ```
33
+ python -m sample_factory.huggingface.load_from_hub -r MattStammers/vizdoom_deadly_corridor
34
+ ```
35
+
36
+
37
+ ## Using the model
38
+
39
+ To run the model after download, use the `enjoy` script corresponding to this environment:
40
+ ```
41
+ python -m <path.to.enjoy.module> --algo=APPO --env=doom_deadly_corridor --train_dir=./train_dir --experiment=vizdoom_deadly_corridor
42
+ ```
43
+
44
+
45
+ You can also upload models to the Hugging Face Hub using the same script with the `--push_to_hub` flag.
46
+ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
47
+
48
+ ## Training with this model
49
+
50
+ To continue training with this model, use the `train` script corresponding to this environment:
51
+ ```
52
+ python -m <path.to.train.module> --algo=APPO --env=doom_deadly_corridor --train_dir=./train_dir --experiment=vizdoom_deadly_corridor --restart_behavior=resume --train_for_env_steps=10000000000
53
+ ```
54
+
55
+ Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
56
+
checkpoint_p0/best_000002847_11661312_reward_21.699.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3787fd51f97ddc7462a2a171992bcb63189ccf5361a49cd1f6ec9ace86e4fe44
3
+ size 34965478
checkpoint_p0/checkpoint_000002853_11685888.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:29091343064f94471180f5f627474243e88202720096a6f28ededb3219cadf51
3
+ size 34965892
checkpoint_p0/checkpoint_000002874_11771904.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cabec58d13a54122faaad9dad9863a0e57069c64a4b2908005200f8f0f5a62cd
3
+ size 34965892
checkpoint_p1/best_000002435_9973760_reward_18.719.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bf1da9264c6d1db57d3bd787f0b18f31e4ac76a5820e553f2435a09cc15ca8a6
3
+ size 34965478
checkpoint_p1/checkpoint_000002419_9908224.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0aa13810bf6376adb120c9a94e58006216af56539c9390d181af1abe053497a1
3
+ size 34965892
checkpoint_p1/checkpoint_000002442_10002432.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9aac2385b23d010e851811fc624fb49ae3401154c4f661e21162f431727fc064
3
+ size 34965892
config.json ADDED
@@ -0,0 +1,143 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "help": false,
3
+ "algo": "APPO",
4
+ "env": "doom_deadly_corridor",
5
+ "experiment": "default_experiment",
6
+ "train_dir": "/home/cogstack/Documents/optuna/environments/sample_factory/train_dir",
7
+ "restart_behavior": "restart",
8
+ "device": "gpu",
9
+ "seed": null,
10
+ "num_policies": 2,
11
+ "async_rl": true,
12
+ "serial_mode": false,
13
+ "batched_sampling": false,
14
+ "num_batches_to_accumulate": 2,
15
+ "worker_num_splits": 2,
16
+ "policy_workers_per_policy": 1,
17
+ "max_policy_lag": 1000,
18
+ "num_workers": 8,
19
+ "num_envs_per_worker": 4,
20
+ "batch_size": 1024,
21
+ "num_batches_per_epoch": 1,
22
+ "num_epochs": 1,
23
+ "rollout": 32,
24
+ "recurrence": 32,
25
+ "shuffle_minibatches": false,
26
+ "gamma": 0.99,
27
+ "reward_scale": 1.0,
28
+ "reward_clip": 1000.0,
29
+ "value_bootstrap": false,
30
+ "normalize_returns": true,
31
+ "exploration_loss_coeff": 0.001,
32
+ "value_loss_coeff": 0.5,
33
+ "kl_loss_coeff": 0.0,
34
+ "exploration_loss": "symmetric_kl",
35
+ "gae_lambda": 0.95,
36
+ "ppo_clip_ratio": 0.1,
37
+ "ppo_clip_value": 0.2,
38
+ "with_vtrace": false,
39
+ "vtrace_rho": 1.0,
40
+ "vtrace_c": 1.0,
41
+ "optimizer": "adam",
42
+ "adam_eps": 1e-06,
43
+ "adam_beta1": 0.9,
44
+ "adam_beta2": 0.999,
45
+ "max_grad_norm": 4.0,
46
+ "learning_rate": 0.0001,
47
+ "lr_schedule": "constant",
48
+ "lr_schedule_kl_threshold": 0.008,
49
+ "lr_adaptive_min": 1e-06,
50
+ "lr_adaptive_max": 0.01,
51
+ "obs_subtract_mean": 0.0,
52
+ "obs_scale": 255.0,
53
+ "normalize_input": true,
54
+ "normalize_input_keys": null,
55
+ "decorrelate_experience_max_seconds": 0,
56
+ "decorrelate_envs_on_one_worker": true,
57
+ "actor_worker_gpus": [],
58
+ "set_workers_cpu_affinity": true,
59
+ "force_envs_single_thread": false,
60
+ "default_niceness": 0,
61
+ "log_to_file": true,
62
+ "experiment_summaries_interval": 10,
63
+ "flush_summaries_interval": 30,
64
+ "stats_avg": 100,
65
+ "summaries_use_frameskip": true,
66
+ "heartbeat_interval": 20,
67
+ "heartbeat_reporting_interval": 600,
68
+ "train_for_env_steps": 10000000,
69
+ "train_for_seconds": 10000000000,
70
+ "save_every_sec": 120,
71
+ "keep_checkpoints": 2,
72
+ "load_checkpoint_kind": "latest",
73
+ "save_milestones_sec": -1,
74
+ "save_best_every_sec": 5,
75
+ "save_best_metric": "reward",
76
+ "save_best_after": 100000,
77
+ "benchmark": false,
78
+ "encoder_mlp_layers": [
79
+ 512,
80
+ 512
81
+ ],
82
+ "encoder_conv_architecture": "convnet_simple",
83
+ "encoder_conv_mlp_layers": [
84
+ 512
85
+ ],
86
+ "use_rnn": true,
87
+ "rnn_size": 512,
88
+ "rnn_type": "gru",
89
+ "rnn_num_layers": 1,
90
+ "decoder_mlp_layers": [],
91
+ "nonlinearity": "elu",
92
+ "policy_initialization": "orthogonal",
93
+ "policy_init_gain": 1.0,
94
+ "actor_critic_share_weights": true,
95
+ "adaptive_stddev": true,
96
+ "continuous_tanh_scale": 0.0,
97
+ "initial_stddev": 1.0,
98
+ "use_env_info_cache": false,
99
+ "env_gpu_actions": false,
100
+ "env_gpu_observations": true,
101
+ "env_frameskip": 4,
102
+ "env_framestack": 1,
103
+ "pixel_format": "CHW",
104
+ "use_record_episode_statistics": false,
105
+ "with_wandb": true,
106
+ "wandb_user": "matt-stammers",
107
+ "wandb_project": "sample_factory",
108
+ "wandb_group": null,
109
+ "wandb_job_type": "SF",
110
+ "wandb_tags": [],
111
+ "with_pbt": false,
112
+ "pbt_mix_policies_in_one_env": true,
113
+ "pbt_period_env_steps": 5000000,
114
+ "pbt_start_mutation": 20000000,
115
+ "pbt_replace_fraction": 0.3,
116
+ "pbt_mutation_rate": 0.15,
117
+ "pbt_replace_reward_gap": 0.1,
118
+ "pbt_replace_reward_gap_absolute": 1e-06,
119
+ "pbt_optimize_gamma": false,
120
+ "pbt_target_objective": "true_objective",
121
+ "pbt_perturb_min": 1.1,
122
+ "pbt_perturb_max": 1.5,
123
+ "num_agents": -1,
124
+ "num_humans": 0,
125
+ "num_bots": -1,
126
+ "start_bot_difficulty": null,
127
+ "timelimit": null,
128
+ "res_w": 128,
129
+ "res_h": 72,
130
+ "wide_aspect_ratio": false,
131
+ "eval_env_frameskip": 1,
132
+ "fps": 35,
133
+ "command_line": "--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000",
134
+ "cli_args": {
135
+ "env": "doom_health_gathering_supreme",
136
+ "num_workers": 8,
137
+ "num_envs_per_worker": 4,
138
+ "train_for_env_steps": 4000000
139
+ },
140
+ "git_hash": "b12d96985caa7a7552d0840afdd14065f56f9f9a",
141
+ "git_repo_name": "https://github.com/MattStammers/optuna.git",
142
+ "wandb_unique_id": "default_experiment_20230912_141858_570479"
143
+ }
git.diff ADDED
The diff for this file is too large to render. See raw diff
 
replay.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0890f486e8b823f80c9d2df634d31ff9d97b3f267437a02b78a9a9b519cd363
3
+ size 2375534
sf_log.txt ADDED
@@ -0,0 +1,920 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [2023-09-12 14:21:51,930][119377] Using GPUs [0] for process 0 (actually maps to GPUs [0])
2
+ [2023-09-12 14:21:51,931][119377] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
3
+ [2023-09-12 14:21:51,949][119377] Num visible devices: 1
4
+ [2023-09-12 14:21:51,977][119377] Starting seed is not provided
5
+ [2023-09-12 14:21:51,978][119377] Using GPUs [0] for process 0 (actually maps to GPUs [0])
6
+ [2023-09-12 14:21:51,978][119377] Initializing actor-critic model on device cuda:0
7
+ [2023-09-12 14:21:51,978][119377] RunningMeanStd input shape: (3, 72, 128)
8
+ [2023-09-12 14:21:51,979][119377] RunningMeanStd input shape: (1,)
9
+ [2023-09-12 14:21:51,992][119377] ConvEncoder: input_channels=3
10
+ [2023-09-12 14:21:52,103][119377] Conv encoder output size: 512
11
+ [2023-09-12 14:21:52,104][119377] Policy head output size: 512
12
+ [2023-09-12 14:21:52,118][119377] Created Actor Critic model with architecture:
13
+ [2023-09-12 14:21:52,118][119377] ActorCriticSharedWeights(
14
+ (obs_normalizer): ObservationNormalizer(
15
+ (running_mean_std): RunningMeanStdDictInPlace(
16
+ (running_mean_std): ModuleDict(
17
+ (obs): RunningMeanStdInPlace()
18
+ )
19
+ )
20
+ )
21
+ (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
22
+ (encoder): VizdoomEncoder(
23
+ (basic_encoder): ConvEncoder(
24
+ (enc): RecursiveScriptModule(
25
+ original_name=ConvEncoderImpl
26
+ (conv_head): RecursiveScriptModule(
27
+ original_name=Sequential
28
+ (0): RecursiveScriptModule(original_name=Conv2d)
29
+ (1): RecursiveScriptModule(original_name=ELU)
30
+ (2): RecursiveScriptModule(original_name=Conv2d)
31
+ (3): RecursiveScriptModule(original_name=ELU)
32
+ (4): RecursiveScriptModule(original_name=Conv2d)
33
+ (5): RecursiveScriptModule(original_name=ELU)
34
+ )
35
+ (mlp_layers): RecursiveScriptModule(
36
+ original_name=Sequential
37
+ (0): RecursiveScriptModule(original_name=Linear)
38
+ (1): RecursiveScriptModule(original_name=ELU)
39
+ )
40
+ )
41
+ )
42
+ )
43
+ (core): ModelCoreRNN(
44
+ (core): GRU(512, 512)
45
+ )
46
+ (decoder): MlpDecoder(
47
+ (mlp): Identity()
48
+ )
49
+ (critic_linear): Linear(in_features=512, out_features=1, bias=True)
50
+ (action_parameterization): ActionParameterizationDefault(
51
+ (distribution_linear): Linear(in_features=512, out_features=11, bias=True)
52
+ )
53
+ )
54
+ [2023-09-12 14:21:53,135][119377] Using optimizer <class 'torch.optim.adam.Adam'>
55
+ [2023-09-12 14:21:53,136][119377] No checkpoints found
56
+ [2023-09-12 14:21:53,136][119377] Did not load from checkpoint, starting from scratch!
57
+ [2023-09-12 14:21:53,136][119377] Initialized policy 0 weights for model version 0
58
+ [2023-09-12 14:21:53,137][119377] LearnerWorker_p0 finished initialization!
59
+ [2023-09-12 14:21:53,138][119377] Using GPUs [0] for process 0 (actually maps to GPUs [0])
60
+ [2023-09-12 14:21:53,600][119700] Using GPUs [1] for process 1 (actually maps to GPUs [1])
61
+ [2023-09-12 14:21:53,600][119700] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1
62
+ [2023-09-12 14:21:53,638][119700] Num visible devices: 1
63
+ [2023-09-12 14:21:53,679][119700] Starting seed is not provided
64
+ [2023-09-12 14:21:53,680][119700] Using GPUs [0] for process 1 (actually maps to GPUs [1])
65
+ [2023-09-12 14:21:53,680][119700] Initializing actor-critic model on device cuda:0
66
+ [2023-09-12 14:21:53,680][119700] RunningMeanStd input shape: (3, 72, 128)
67
+ [2023-09-12 14:21:53,681][119700] RunningMeanStd input shape: (1,)
68
+ [2023-09-12 14:21:53,703][119700] ConvEncoder: input_channels=3
69
+ [2023-09-12 14:21:53,931][119700] Conv encoder output size: 512
70
+ [2023-09-12 14:21:53,932][119700] Policy head output size: 512
71
+ [2023-09-12 14:21:53,949][119700] Created Actor Critic model with architecture:
72
+ [2023-09-12 14:21:53,949][119700] ActorCriticSharedWeights(
73
+ (obs_normalizer): ObservationNormalizer(
74
+ (running_mean_std): RunningMeanStdDictInPlace(
75
+ (running_mean_std): ModuleDict(
76
+ (obs): RunningMeanStdInPlace()
77
+ )
78
+ )
79
+ )
80
+ (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
81
+ (encoder): VizdoomEncoder(
82
+ (basic_encoder): ConvEncoder(
83
+ (enc): RecursiveScriptModule(
84
+ original_name=ConvEncoderImpl
85
+ (conv_head): RecursiveScriptModule(
86
+ original_name=Sequential
87
+ (0): RecursiveScriptModule(original_name=Conv2d)
88
+ (1): RecursiveScriptModule(original_name=ELU)
89
+ (2): RecursiveScriptModule(original_name=Conv2d)
90
+ (3): RecursiveScriptModule(original_name=ELU)
91
+ (4): RecursiveScriptModule(original_name=Conv2d)
92
+ (5): RecursiveScriptModule(original_name=ELU)
93
+ )
94
+ (mlp_layers): RecursiveScriptModule(
95
+ original_name=Sequential
96
+ (0): RecursiveScriptModule(original_name=Linear)
97
+ (1): RecursiveScriptModule(original_name=ELU)
98
+ )
99
+ )
100
+ )
101
+ )
102
+ (core): ModelCoreRNN(
103
+ (core): GRU(512, 512)
104
+ )
105
+ (decoder): MlpDecoder(
106
+ (mlp): Identity()
107
+ )
108
+ (critic_linear): Linear(in_features=512, out_features=1, bias=True)
109
+ (action_parameterization): ActionParameterizationDefault(
110
+ (distribution_linear): Linear(in_features=512, out_features=11, bias=True)
111
+ )
112
+ )
113
+ [2023-09-12 14:21:55,320][119700] Using optimizer <class 'torch.optim.adam.Adam'>
114
+ [2023-09-12 14:21:55,321][119700] No checkpoints found
115
+ [2023-09-12 14:21:55,321][119700] Did not load from checkpoint, starting from scratch!
116
+ [2023-09-12 14:21:55,321][119700] Initialized policy 1 weights for model version 0
117
+ [2023-09-12 14:21:55,323][119700] LearnerWorker_p1 finished initialization!
118
+ [2023-09-12 14:21:55,324][119700] Using GPUs [0] for process 1 (actually maps to GPUs [1])
119
+ [2023-09-12 14:21:55,850][119817] Worker 2 uses CPU cores [8, 9, 10, 11]
120
+ [2023-09-12 14:21:55,874][119814] Using GPUs [0] for process 0 (actually maps to GPUs [0])
121
+ [2023-09-12 14:21:55,875][119814] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
122
+ [2023-09-12 14:21:55,877][119819] Worker 3 uses CPU cores [12, 13, 14, 15]
123
+ [2023-09-12 14:21:55,892][119814] Num visible devices: 1
124
+ [2023-09-12 14:21:56,016][119815] Using GPUs [1] for process 1 (actually maps to GPUs [1])
125
+ [2023-09-12 14:21:56,017][119815] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1
126
+ [2023-09-12 14:21:56,020][119816] Worker 0 uses CPU cores [0, 1, 2, 3]
127
+ [2023-09-12 14:21:56,035][119815] Num visible devices: 1
128
+ [2023-09-12 14:21:56,059][119818] Worker 1 uses CPU cores [4, 5, 6, 7]
129
+ [2023-09-12 14:21:56,080][119918] Worker 7 uses CPU cores [28, 29, 30, 31]
130
+ [2023-09-12 14:21:56,108][119917] Worker 6 uses CPU cores [24, 25, 26, 27]
131
+ [2023-09-12 14:21:56,118][119882] Worker 4 uses CPU cores [16, 17, 18, 19]
132
+ [2023-09-12 14:21:56,121][119916] Worker 5 uses CPU cores [20, 21, 22, 23]
133
+ [2023-09-12 14:21:56,575][119814] RunningMeanStd input shape: (3, 72, 128)
134
+ [2023-09-12 14:21:56,576][119814] RunningMeanStd input shape: (1,)
135
+ [2023-09-12 14:21:56,588][119814] ConvEncoder: input_channels=3
136
+ [2023-09-12 14:21:56,649][119815] RunningMeanStd input shape: (3, 72, 128)
137
+ [2023-09-12 14:21:56,649][119815] RunningMeanStd input shape: (1,)
138
+ [2023-09-12 14:21:56,661][119815] ConvEncoder: input_channels=3
139
+ [2023-09-12 14:21:56,693][119814] Conv encoder output size: 512
140
+ [2023-09-12 14:21:56,693][119814] Policy head output size: 512
141
+ [2023-09-12 14:21:56,765][119815] Conv encoder output size: 512
142
+ [2023-09-12 14:21:56,765][119815] Policy head output size: 512
143
+ [2023-09-12 14:21:57,090][119818] Doom resolution: 160x120, resize resolution: (128, 72)
144
+ [2023-09-12 14:21:57,092][119816] Doom resolution: 160x120, resize resolution: (128, 72)
145
+ [2023-09-12 14:21:57,102][119817] Doom resolution: 160x120, resize resolution: (128, 72)
146
+ [2023-09-12 14:21:57,102][119916] Doom resolution: 160x120, resize resolution: (128, 72)
147
+ [2023-09-12 14:21:57,105][119882] Doom resolution: 160x120, resize resolution: (128, 72)
148
+ [2023-09-12 14:21:57,109][119819] Doom resolution: 160x120, resize resolution: (128, 72)
149
+ [2023-09-12 14:21:57,109][119918] Doom resolution: 160x120, resize resolution: (128, 72)
150
+ [2023-09-12 14:21:57,111][119917] Doom resolution: 160x120, resize resolution: (128, 72)
151
+ [2023-09-12 14:21:57,415][119818] Decorrelating experience for 0 frames...
152
+ [2023-09-12 14:21:57,482][119916] Decorrelating experience for 0 frames...
153
+ [2023-09-12 14:21:57,532][119816] Decorrelating experience for 0 frames...
154
+ [2023-09-12 14:21:57,550][119882] Decorrelating experience for 0 frames...
155
+ [2023-09-12 14:21:57,570][119817] Decorrelating experience for 0 frames...
156
+ [2023-09-12 14:21:57,737][119819] Decorrelating experience for 0 frames...
157
+ [2023-09-12 14:21:57,807][119918] Decorrelating experience for 0 frames...
158
+ [2023-09-12 14:21:57,820][119882] Decorrelating experience for 32 frames...
159
+ [2023-09-12 14:21:57,824][119816] Decorrelating experience for 32 frames...
160
+ [2023-09-12 14:21:57,825][119818] Decorrelating experience for 32 frames...
161
+ [2023-09-12 14:21:57,838][119916] Decorrelating experience for 32 frames...
162
+ [2023-09-12 14:21:58,014][119819] Decorrelating experience for 32 frames...
163
+ [2023-09-12 14:21:58,150][119918] Decorrelating experience for 32 frames...
164
+ [2023-09-12 14:21:58,161][119817] Decorrelating experience for 32 frames...
165
+ [2023-09-12 14:21:58,199][119818] Decorrelating experience for 64 frames...
166
+ [2023-09-12 14:21:58,199][119882] Decorrelating experience for 64 frames...
167
+ [2023-09-12 14:21:58,209][119917] Decorrelating experience for 0 frames...
168
+ [2023-09-12 14:21:58,239][119816] Decorrelating experience for 64 frames...
169
+ [2023-09-12 14:21:58,523][119818] Decorrelating experience for 96 frames...
170
+ [2023-09-12 14:21:58,524][119918] Decorrelating experience for 64 frames...
171
+ [2023-09-12 14:21:58,581][119917] Decorrelating experience for 32 frames...
172
+ [2023-09-12 14:21:58,616][119817] Decorrelating experience for 64 frames...
173
+ [2023-09-12 14:21:58,857][119882] Decorrelating experience for 96 frames...
174
+ [2023-09-12 14:21:58,866][119916] Decorrelating experience for 64 frames...
175
+ [2023-09-12 14:21:58,927][119918] Decorrelating experience for 96 frames...
176
+ [2023-09-12 14:21:59,043][119817] Decorrelating experience for 96 frames...
177
+ [2023-09-12 14:21:59,094][119917] Decorrelating experience for 64 frames...
178
+ [2023-09-12 14:21:59,244][119819] Decorrelating experience for 64 frames...
179
+ [2023-09-12 14:21:59,245][119916] Decorrelating experience for 96 frames...
180
+ [2023-09-12 14:21:59,423][119817] Multiple policies in trajectory buffer: [0 1] (-1 means inactive agent)
181
+ [2023-09-12 14:21:59,430][119882] Multiple policies in trajectory buffer: [0 1] (-1 means inactive agent)
182
+ [2023-09-12 14:21:59,452][119818] Multiple policies in trajectory buffer: [0 1] (-1 means inactive agent)
183
+ [2023-09-12 14:21:59,471][119917] Decorrelating experience for 96 frames...
184
+ [2023-09-12 14:21:59,527][119918] Multiple policies in trajectory buffer: [0 1] (-1 means inactive agent)
185
+ [2023-09-12 14:21:59,655][119816] Decorrelating experience for 96 frames...
186
+ [2023-09-12 14:21:59,726][119916] Multiple policies in trajectory buffer: [0 1] (-1 means inactive agent)
187
+ [2023-09-12 14:21:59,868][119917] Multiple policies in trajectory buffer: [0 1] (-1 means inactive agent)
188
+ [2023-09-12 14:21:59,993][119819] Decorrelating experience for 96 frames...
189
+ [2023-09-12 14:22:00,396][119819] Multiple policies in trajectory buffer: [0 1] (-1 means inactive agent)
190
+ [2023-09-12 14:22:00,536][119700] Signal inference workers to stop experience collection...
191
+ [2023-09-12 14:22:00,542][119815] InferenceWorker_p1-w0: stopping experience collection
192
+ [2023-09-12 14:22:00,544][119814] InferenceWorker_p0-w0: stopping experience collection
193
+ [2023-09-12 14:22:03,599][119700] Signal inference workers to resume experience collection...
194
+ [2023-09-12 14:22:03,600][119815] InferenceWorker_p1-w0: resuming experience collection
195
+ [2023-09-12 14:22:03,600][119814] InferenceWorker_p0-w0: resuming experience collection
196
+ [2023-09-12 14:22:05,328][119377] Signal inference workers to stop experience collection...
197
+ [2023-09-12 14:22:05,827][119377] Signal inference workers to resume experience collection...
198
+ [2023-09-12 14:22:07,160][119816] Multiple policies in trajectory buffer: [0 1] (-1 means inactive agent)
199
+ [2023-09-12 14:22:10,105][119814] Updated weights for policy 0, policy_version 10 (0.0010)
200
+ [2023-09-12 14:22:10,828][119815] Updated weights for policy 1, policy_version 10 (0.0373)
201
+ [2023-09-12 14:22:16,582][119814] Updated weights for policy 0, policy_version 20 (0.0010)
202
+ [2023-09-12 14:22:17,424][119815] Updated weights for policy 1, policy_version 20 (0.0009)
203
+ [2023-09-12 14:22:22,055][119700] Saving new best policy, reward=1.387!
204
+ [2023-09-12 14:22:22,055][119377] Saving new best policy, reward=1.098!
205
+ [2023-09-12 14:22:23,106][119815] Updated weights for policy 1, policy_version 30 (0.0009)
206
+ [2023-09-12 14:22:23,994][119814] Updated weights for policy 0, policy_version 30 (0.0010)
207
+ [2023-09-12 14:22:27,060][119377] Saving new best policy, reward=2.156!
208
+ [2023-09-12 14:22:27,060][119700] Saving new best policy, reward=1.787!
209
+ [2023-09-12 14:22:29,846][119814] Updated weights for policy 0, policy_version 40 (0.0009)
210
+ [2023-09-12 14:22:32,055][119700] Saving new best policy, reward=2.187!
211
+ [2023-09-12 14:22:32,097][119377] Saving new best policy, reward=2.296!
212
+ [2023-09-12 14:22:32,986][119815] Updated weights for policy 1, policy_version 40 (0.0009)
213
+ [2023-09-12 14:22:35,759][119814] Updated weights for policy 0, policy_version 50 (0.0011)
214
+ [2023-09-12 14:22:37,061][119700] Saving new best policy, reward=2.516!
215
+ [2023-09-12 14:22:37,137][119377] Saving new best policy, reward=3.257!
216
+ [2023-09-12 14:22:39,787][119815] Updated weights for policy 1, policy_version 50 (0.0010)
217
+ [2023-09-12 14:22:42,055][119700] Saving new best policy, reward=2.854!
218
+ [2023-09-12 14:22:43,162][119814] Updated weights for policy 0, policy_version 60 (0.0011)
219
+ [2023-09-12 14:22:45,300][119815] Updated weights for policy 1, policy_version 60 (0.0009)
220
+ [2023-09-12 14:22:47,059][119377] Saving new best policy, reward=3.264!
221
+ [2023-09-12 14:22:47,060][119700] Saving new best policy, reward=3.298!
222
+ [2023-09-12 14:22:50,465][119814] Updated weights for policy 0, policy_version 70 (0.0011)
223
+ [2023-09-12 14:22:51,179][119815] Updated weights for policy 1, policy_version 70 (0.0009)
224
+ [2023-09-12 14:22:52,055][119700] Saving new best policy, reward=3.469!
225
+ [2023-09-12 14:22:52,137][119377] Saving new best policy, reward=3.531!
226
+ [2023-09-12 14:22:57,060][119377] Saving new best policy, reward=3.595!
227
+ [2023-09-12 14:22:57,106][119815] Updated weights for policy 1, policy_version 80 (0.0009)
228
+ [2023-09-12 14:22:57,417][119814] Updated weights for policy 0, policy_version 80 (0.0009)
229
+ [2023-09-12 14:23:02,055][119377] Saving new best policy, reward=3.701!
230
+ [2023-09-12 14:23:02,055][119700] Saving new best policy, reward=3.647!
231
+ [2023-09-12 14:23:02,714][119814] Updated weights for policy 0, policy_version 90 (0.0009)
232
+ [2023-09-12 14:23:02,717][119815] Updated weights for policy 1, policy_version 90 (0.0009)
233
+ [2023-09-12 14:23:07,060][119700] Saving new best policy, reward=3.695!
234
+ [2023-09-12 14:23:07,800][119815] Updated weights for policy 1, policy_version 100 (0.0009)
235
+ [2023-09-12 14:23:08,185][119814] Updated weights for policy 0, policy_version 100 (0.0010)
236
+ [2023-09-12 14:23:12,055][119700] Saving new best policy, reward=4.035!
237
+ [2023-09-12 14:23:13,166][119815] Updated weights for policy 1, policy_version 110 (0.0009)
238
+ [2023-09-12 14:23:13,406][119814] Updated weights for policy 0, policy_version 110 (0.0009)
239
+ [2023-09-12 14:23:17,060][119377] Saving new best policy, reward=3.916!
240
+ [2023-09-12 14:23:18,444][119814] Updated weights for policy 0, policy_version 120 (0.0010)
241
+ [2023-09-12 14:23:19,127][119815] Updated weights for policy 1, policy_version 120 (0.0009)
242
+ [2023-09-12 14:23:23,633][119814] Updated weights for policy 0, policy_version 130 (0.0010)
243
+ [2023-09-12 14:23:24,098][119815] Updated weights for policy 1, policy_version 130 (0.0008)
244
+ [2023-09-12 14:23:28,748][119814] Updated weights for policy 0, policy_version 140 (0.0009)
245
+ [2023-09-12 14:23:29,960][119815] Updated weights for policy 1, policy_version 140 (0.0009)
246
+ [2023-09-12 14:23:32,055][119700] Saving new best policy, reward=4.304!
247
+ [2023-09-12 14:23:34,319][119814] Updated weights for policy 0, policy_version 150 (0.0010)
248
+ [2023-09-12 14:23:34,492][119815] Updated weights for policy 1, policy_version 150 (0.0009)
249
+ [2023-09-12 14:23:37,061][119377] Saving new best policy, reward=4.129!
250
+ [2023-09-12 14:23:38,562][119814] Updated weights for policy 0, policy_version 160 (0.0009)
251
+ [2023-09-12 14:23:40,869][119815] Updated weights for policy 1, policy_version 160 (0.0009)
252
+ [2023-09-12 14:23:44,152][119814] Updated weights for policy 0, policy_version 170 (0.0009)
253
+ [2023-09-12 14:23:47,059][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000167_684032.pth...
254
+ [2023-09-12 14:23:47,126][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000175_716800.pth...
255
+ [2023-09-12 14:23:49,588][119815] Updated weights for policy 1, policy_version 170 (0.0009)
256
+ [2023-09-12 14:23:50,305][119814] Updated weights for policy 0, policy_version 180 (0.0009)
257
+ [2023-09-12 14:23:56,143][119815] Updated weights for policy 1, policy_version 180 (0.0008)
258
+ [2023-09-12 14:23:56,537][119814] Updated weights for policy 0, policy_version 190 (0.0009)
259
+ [2023-09-12 14:24:02,766][119815] Updated weights for policy 1, policy_version 190 (0.0009)
260
+ [2023-09-12 14:24:02,832][119814] Updated weights for policy 0, policy_version 200 (0.0009)
261
+ [2023-09-12 14:24:08,540][119814] Updated weights for policy 0, policy_version 210 (0.0009)
262
+ [2023-09-12 14:24:10,870][119815] Updated weights for policy 1, policy_version 200 (0.0010)
263
+ [2023-09-12 14:24:12,055][119377] Saving new best policy, reward=4.356!
264
+ [2023-09-12 14:24:14,669][119814] Updated weights for policy 0, policy_version 220 (0.0009)
265
+ [2023-09-12 14:24:17,061][119377] Saving new best policy, reward=4.784!
266
+ [2023-09-12 14:24:18,863][119815] Updated weights for policy 1, policy_version 210 (0.0009)
267
+ [2023-09-12 14:24:20,355][119814] Updated weights for policy 0, policy_version 230 (0.0009)
268
+ [2023-09-12 14:24:26,133][119814] Updated weights for policy 0, policy_version 240 (0.0010)
269
+ [2023-09-12 14:24:26,370][119815] Updated weights for policy 1, policy_version 220 (0.0009)
270
+ [2023-09-12 14:24:31,751][119814] Updated weights for policy 0, policy_version 250 (0.0009)
271
+ [2023-09-12 14:24:34,143][119815] Updated weights for policy 1, policy_version 230 (0.0009)
272
+ [2023-09-12 14:24:37,513][119814] Updated weights for policy 0, policy_version 260 (0.0009)
273
+ [2023-09-12 14:24:41,487][119815] Updated weights for policy 1, policy_version 240 (0.0009)
274
+ [2023-09-12 14:24:43,725][119814] Updated weights for policy 0, policy_version 270 (0.0009)
275
+ [2023-09-12 14:24:47,059][119377] Saving new best policy, reward=5.386!
276
+ [2023-09-12 14:24:48,493][119815] Updated weights for policy 1, policy_version 250 (0.0010)
277
+ [2023-09-12 14:24:49,830][119814] Updated weights for policy 0, policy_version 280 (0.0009)
278
+ [2023-09-12 14:24:53,641][119815] Updated weights for policy 1, policy_version 260 (0.0009)
279
+ [2023-09-12 14:24:57,694][119814] Updated weights for policy 0, policy_version 290 (0.0009)
280
+ [2023-09-12 14:24:59,741][119815] Updated weights for policy 1, policy_version 270 (0.0008)
281
+ [2023-09-12 14:25:03,504][119814] Updated weights for policy 0, policy_version 300 (0.0009)
282
+ [2023-09-12 14:25:06,536][119815] Updated weights for policy 1, policy_version 280 (0.0008)
283
+ [2023-09-12 14:25:07,060][119377] Saving new best policy, reward=5.606!
284
+ [2023-09-12 14:25:10,332][119814] Updated weights for policy 0, policy_version 310 (0.0009)
285
+ [2023-09-12 14:25:13,751][119815] Updated weights for policy 1, policy_version 290 (0.0009)
286
+ [2023-09-12 14:25:15,935][119814] Updated weights for policy 0, policy_version 320 (0.0009)
287
+ [2023-09-12 14:25:20,424][119815] Updated weights for policy 1, policy_version 300 (0.0010)
288
+ [2023-09-12 14:25:22,493][119814] Updated weights for policy 0, policy_version 330 (0.0009)
289
+ [2023-09-12 14:25:26,902][119815] Updated weights for policy 1, policy_version 310 (0.0009)
290
+ [2023-09-12 14:25:28,668][119814] Updated weights for policy 0, policy_version 340 (0.0008)
291
+ [2023-09-12 14:25:34,036][119815] Updated weights for policy 1, policy_version 320 (0.0008)
292
+ [2023-09-12 14:25:34,804][119814] Updated weights for policy 0, policy_version 350 (0.0009)
293
+ [2023-09-12 14:25:37,060][119377] Saving new best policy, reward=5.668!
294
+ [2023-09-12 14:25:40,467][119814] Updated weights for policy 0, policy_version 360 (0.0009)
295
+ [2023-09-12 14:25:41,830][119815] Updated weights for policy 1, policy_version 330 (0.0010)
296
+ [2023-09-12 14:25:46,420][119814] Updated weights for policy 0, policy_version 370 (0.0009)
297
+ [2023-09-12 14:25:47,061][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000335_1372160.pth...
298
+ [2023-09-12 14:25:47,061][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000371_1519616.pth...
299
+ [2023-09-12 14:25:50,194][119815] Updated weights for policy 1, policy_version 340 (0.0009)
300
+ [2023-09-12 14:25:52,673][119814] Updated weights for policy 0, policy_version 380 (0.0009)
301
+ [2023-09-12 14:25:57,059][119377] Saving new best policy, reward=5.670!
302
+ [2023-09-12 14:25:57,060][119700] Saving new best policy, reward=4.341!
303
+ [2023-09-12 14:25:57,819][119815] Updated weights for policy 1, policy_version 350 (0.0009)
304
+ [2023-09-12 14:25:58,449][119814] Updated weights for policy 0, policy_version 390 (0.0008)
305
+ [2023-09-12 14:26:02,055][119377] Saving new best policy, reward=6.050!
306
+ [2023-09-12 14:26:02,055][119700] Saving new best policy, reward=4.525!
307
+ [2023-09-12 14:26:04,220][119814] Updated weights for policy 0, policy_version 400 (0.0009)
308
+ [2023-09-12 14:26:05,881][119815] Updated weights for policy 1, policy_version 360 (0.0009)
309
+ [2023-09-12 14:26:09,707][119814] Updated weights for policy 0, policy_version 410 (0.0009)
310
+ [2023-09-12 14:26:12,056][119700] Saving new best policy, reward=4.964!
311
+ [2023-09-12 14:26:13,650][119815] Updated weights for policy 1, policy_version 370 (0.0010)
312
+ [2023-09-12 14:26:16,415][119814] Updated weights for policy 0, policy_version 420 (0.0009)
313
+ [2023-09-12 14:26:20,294][119815] Updated weights for policy 1, policy_version 380 (0.0009)
314
+ [2023-09-12 14:26:21,922][119814] Updated weights for policy 0, policy_version 430 (0.0008)
315
+ [2023-09-12 14:26:26,146][119815] Updated weights for policy 1, policy_version 390 (0.0009)
316
+ [2023-09-12 14:26:26,206][119814] Updated weights for policy 0, policy_version 440 (0.0009)
317
+ [2023-09-12 14:26:30,855][119814] Updated weights for policy 0, policy_version 450 (0.0009)
318
+ [2023-09-12 14:26:31,850][119815] Updated weights for policy 1, policy_version 400 (0.0009)
319
+ [2023-09-12 14:26:32,055][119377] Saving new best policy, reward=6.263!
320
+ [2023-09-12 14:26:36,056][119814] Updated weights for policy 0, policy_version 460 (0.0009)
321
+ [2023-09-12 14:26:36,839][119815] Updated weights for policy 1, policy_version 410 (0.0008)
322
+ [2023-09-12 14:26:41,073][119814] Updated weights for policy 0, policy_version 470 (0.0009)
323
+ [2023-09-12 14:26:41,846][119815] Updated weights for policy 1, policy_version 420 (0.0008)
324
+ [2023-09-12 14:26:42,055][119377] Saving new best policy, reward=6.681!
325
+ [2023-09-12 14:26:45,889][119814] Updated weights for policy 0, policy_version 480 (0.0009)
326
+ [2023-09-12 14:26:48,987][119815] Updated weights for policy 1, policy_version 430 (0.0009)
327
+ [2023-09-12 14:26:49,807][119814] Updated weights for policy 0, policy_version 490 (0.0010)
328
+ [2023-09-12 14:26:52,056][119700] Saving new best policy, reward=5.258!
329
+ [2023-09-12 14:26:54,253][119814] Updated weights for policy 0, policy_version 500 (0.0009)
330
+ [2023-09-12 14:26:56,235][119815] Updated weights for policy 1, policy_version 440 (0.0010)
331
+ [2023-09-12 14:26:58,908][119814] Updated weights for policy 0, policy_version 510 (0.0010)
332
+ [2023-09-12 14:27:02,133][119815] Updated weights for policy 1, policy_version 450 (0.0009)
333
+ [2023-09-12 14:27:03,650][119814] Updated weights for policy 0, policy_version 520 (0.0009)
334
+ [2023-09-12 14:27:07,825][119815] Updated weights for policy 1, policy_version 460 (0.0009)
335
+ [2023-09-12 14:27:09,563][119814] Updated weights for policy 0, policy_version 530 (0.0009)
336
+ [2023-09-12 14:27:13,611][119815] Updated weights for policy 1, policy_version 470 (0.0009)
337
+ [2023-09-12 14:27:16,338][119814] Updated weights for policy 0, policy_version 540 (0.0009)
338
+ [2023-09-12 14:27:21,603][119815] Updated weights for policy 1, policy_version 480 (0.0009)
339
+ [2023-09-12 14:27:22,394][119814] Updated weights for policy 0, policy_version 550 (0.0009)
340
+ [2023-09-12 14:27:28,077][119814] Updated weights for policy 0, policy_version 560 (0.0009)
341
+ [2023-09-12 14:27:29,669][119815] Updated weights for policy 1, policy_version 490 (0.0012)
342
+ [2023-09-12 14:27:32,056][119377] Saving new best policy, reward=6.985!
343
+ [2023-09-12 14:27:34,006][119814] Updated weights for policy 0, policy_version 570 (0.0009)
344
+ [2023-09-12 14:27:37,617][119815] Updated weights for policy 1, policy_version 500 (0.0009)
345
+ [2023-09-12 14:27:39,964][119814] Updated weights for policy 0, policy_version 580 (0.0009)
346
+ [2023-09-12 14:27:45,767][119814] Updated weights for policy 0, policy_version 590 (0.0009)
347
+ [2023-09-12 14:27:45,963][119815] Updated weights for policy 1, policy_version 510 (0.0009)
348
+ [2023-09-12 14:27:47,061][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000591_2420736.pth...
349
+ [2023-09-12 14:27:47,092][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000512_2097152.pth...
350
+ [2023-09-12 14:27:47,112][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000175_716800.pth
351
+ [2023-09-12 14:27:47,146][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000167_684032.pth
352
+ [2023-09-12 14:27:51,900][119814] Updated weights for policy 0, policy_version 600 (0.0009)
353
+ [2023-09-12 14:27:53,646][119815] Updated weights for policy 1, policy_version 520 (0.0008)
354
+ [2023-09-12 14:27:57,060][119700] Saving new best policy, reward=6.073!
355
+ [2023-09-12 14:27:57,368][119814] Updated weights for policy 0, policy_version 610 (0.0009)
356
+ [2023-09-12 14:28:00,705][119815] Updated weights for policy 1, policy_version 530 (0.0009)
357
+ [2023-09-12 14:28:02,055][119377] Saving new best policy, reward=7.476!
358
+ [2023-09-12 14:28:03,945][119814] Updated weights for policy 0, policy_version 620 (0.0008)
359
+ [2023-09-12 14:28:07,060][119700] Saving new best policy, reward=6.364!
360
+ [2023-09-12 14:28:07,979][119815] Updated weights for policy 1, policy_version 540 (0.0010)
361
+ [2023-09-12 14:28:10,130][119814] Updated weights for policy 0, policy_version 630 (0.0009)
362
+ [2023-09-12 14:28:15,773][119814] Updated weights for policy 0, policy_version 640 (0.0009)
363
+ [2023-09-12 14:28:16,927][119815] Updated weights for policy 1, policy_version 550 (0.0008)
364
+ [2023-09-12 14:28:21,474][119814] Updated weights for policy 0, policy_version 650 (0.0009)
365
+ [2023-09-12 14:28:23,883][119815] Updated weights for policy 1, policy_version 560 (0.0008)
366
+ [2023-09-12 14:28:27,617][119814] Updated weights for policy 0, policy_version 660 (0.0008)
367
+ [2023-09-12 14:28:30,702][119815] Updated weights for policy 1, policy_version 570 (0.0008)
368
+ [2023-09-12 14:28:33,950][119814] Updated weights for policy 0, policy_version 670 (0.0009)
369
+ [2023-09-12 14:28:37,059][119377] Saving new best policy, reward=7.684!
370
+ [2023-09-12 14:28:38,203][119815] Updated weights for policy 1, policy_version 580 (0.0009)
371
+ [2023-09-12 14:28:39,947][119814] Updated weights for policy 0, policy_version 680 (0.0009)
372
+ [2023-09-12 14:28:44,053][119815] Updated weights for policy 1, policy_version 590 (0.0009)
373
+ [2023-09-12 14:28:46,642][119814] Updated weights for policy 0, policy_version 690 (0.0008)
374
+ [2023-09-12 14:28:51,683][119815] Updated weights for policy 1, policy_version 600 (0.0008)
375
+ [2023-09-12 14:28:52,478][119814] Updated weights for policy 0, policy_version 700 (0.0010)
376
+ [2023-09-12 14:28:57,061][119377] Saving new best policy, reward=7.751!
377
+ [2023-09-12 14:28:57,061][119700] Saving new best policy, reward=6.430!
378
+ [2023-09-12 14:28:58,506][119814] Updated weights for policy 0, policy_version 710 (0.0009)
379
+ [2023-09-12 14:28:59,016][119815] Updated weights for policy 1, policy_version 610 (0.0008)
380
+ [2023-09-12 14:29:04,675][119814] Updated weights for policy 0, policy_version 720 (0.0009)
381
+ [2023-09-12 14:29:05,634][119815] Updated weights for policy 1, policy_version 620 (0.0009)
382
+ [2023-09-12 14:29:07,060][119377] Saving new best policy, reward=8.052!
383
+ [2023-09-12 14:29:07,104][119700] Saving new best policy, reward=6.456!
384
+ [2023-09-12 14:29:10,927][119814] Updated weights for policy 0, policy_version 730 (0.0009)
385
+ [2023-09-12 14:29:12,055][119377] Saving new best policy, reward=9.382!
386
+ [2023-09-12 14:29:12,913][119815] Updated weights for policy 1, policy_version 630 (0.0008)
387
+ [2023-09-12 14:29:17,067][119814] Updated weights for policy 0, policy_version 740 (0.0010)
388
+ [2023-09-12 14:29:19,476][119815] Updated weights for policy 1, policy_version 640 (0.0009)
389
+ [2023-09-12 14:29:23,948][119814] Updated weights for policy 0, policy_version 750 (0.0008)
390
+ [2023-09-12 14:29:24,867][119815] Updated weights for policy 1, policy_version 650 (0.0008)
391
+ [2023-09-12 14:29:30,173][119814] Updated weights for policy 0, policy_version 760 (0.0009)
392
+ [2023-09-12 14:29:32,318][119815] Updated weights for policy 1, policy_version 660 (0.0008)
393
+ [2023-09-12 14:29:36,505][119814] Updated weights for policy 0, policy_version 770 (0.0010)
394
+ [2023-09-12 14:29:38,827][119815] Updated weights for policy 1, policy_version 670 (0.0008)
395
+ [2023-09-12 14:29:43,639][119814] Updated weights for policy 0, policy_version 780 (0.0009)
396
+ [2023-09-12 14:29:45,410][119815] Updated weights for policy 1, policy_version 680 (0.0009)
397
+ [2023-09-12 14:29:47,059][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000682_2793472.pth...
398
+ [2023-09-12 14:29:47,059][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000787_3223552.pth...
399
+ [2023-09-12 14:29:47,114][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000335_1372160.pth
400
+ [2023-09-12 14:29:47,125][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000371_1519616.pth
401
+ [2023-09-12 14:29:48,130][119814] Updated weights for policy 0, policy_version 790 (0.0009)
402
+ [2023-09-12 14:29:51,237][119815] Updated weights for policy 1, policy_version 690 (0.0008)
403
+ [2023-09-12 14:29:52,582][119814] Updated weights for policy 0, policy_version 800 (0.0009)
404
+ [2023-09-12 14:29:57,014][119815] Updated weights for policy 1, policy_version 700 (0.0008)
405
+ [2023-09-12 14:29:57,261][119814] Updated weights for policy 0, policy_version 810 (0.0009)
406
+ [2023-09-12 14:30:01,797][119815] Updated weights for policy 1, policy_version 710 (0.0008)
407
+ [2023-09-12 14:30:02,708][119814] Updated weights for policy 0, policy_version 820 (0.0009)
408
+ [2023-09-12 14:30:06,674][119814] Updated weights for policy 0, policy_version 830 (0.0009)
409
+ [2023-09-12 14:30:08,970][119815] Updated weights for policy 1, policy_version 720 (0.0009)
410
+ [2023-09-12 14:30:10,526][119814] Updated weights for policy 0, policy_version 840 (0.0009)
411
+ [2023-09-12 14:30:12,056][119377] Saving new best policy, reward=9.944!
412
+ [2023-09-12 14:30:16,503][119815] Updated weights for policy 1, policy_version 730 (0.0008)
413
+ [2023-09-12 14:30:16,909][119814] Updated weights for policy 0, policy_version 850 (0.0008)
414
+ [2023-09-12 14:30:17,059][119377] Saving new best policy, reward=11.313!
415
+ [2023-09-12 14:30:22,056][119700] Saving new best policy, reward=6.617!
416
+ [2023-09-12 14:30:22,056][119377] Saving new best policy, reward=11.463!
417
+ [2023-09-12 14:30:22,314][119814] Updated weights for policy 0, policy_version 860 (0.0009)
418
+ [2023-09-12 14:30:25,841][119815] Updated weights for policy 1, policy_version 740 (0.0009)
419
+ [2023-09-12 14:30:27,060][119377] Saving new best policy, reward=12.552!
420
+ [2023-09-12 14:30:27,516][119814] Updated weights for policy 0, policy_version 870 (0.0009)
421
+ [2023-09-12 14:30:32,056][119700] Saving new best policy, reward=6.785!
422
+ [2023-09-12 14:30:33,150][119814] Updated weights for policy 0, policy_version 880 (0.0009)
423
+ [2023-09-12 14:30:35,434][119815] Updated weights for policy 1, policy_version 750 (0.0009)
424
+ [2023-09-12 14:30:38,888][119814] Updated weights for policy 0, policy_version 890 (0.0010)
425
+ [2023-09-12 14:30:43,300][119815] Updated weights for policy 1, policy_version 760 (0.0009)
426
+ [2023-09-12 14:30:44,925][119814] Updated weights for policy 0, policy_version 900 (0.0009)
427
+ [2023-09-12 14:30:50,297][119815] Updated weights for policy 1, policy_version 770 (0.0008)
428
+ [2023-09-12 14:30:50,719][119814] Updated weights for policy 0, policy_version 910 (0.0009)
429
+ [2023-09-12 14:30:56,473][119814] Updated weights for policy 0, policy_version 920 (0.0008)
430
+ [2023-09-12 14:30:57,059][119377] Saving new best policy, reward=12.813!
431
+ [2023-09-12 14:30:58,519][119815] Updated weights for policy 1, policy_version 780 (0.0008)
432
+ [2023-09-12 14:31:02,679][119814] Updated weights for policy 0, policy_version 930 (0.0009)
433
+ [2023-09-12 14:31:04,733][119815] Updated weights for policy 1, policy_version 790 (0.0008)
434
+ [2023-09-12 14:31:08,689][119814] Updated weights for policy 0, policy_version 940 (0.0009)
435
+ [2023-09-12 14:31:11,583][119815] Updated weights for policy 1, policy_version 800 (0.0008)
436
+ [2023-09-12 14:31:15,001][119814] Updated weights for policy 0, policy_version 950 (0.0009)
437
+ [2023-09-12 14:31:17,062][119700] Saving new best policy, reward=7.440!
438
+ [2023-09-12 14:31:17,115][119377] Saving new best policy, reward=13.457!
439
+ [2023-09-12 14:31:20,148][119815] Updated weights for policy 1, policy_version 810 (0.0010)
440
+ [2023-09-12 14:31:20,662][119814] Updated weights for policy 0, policy_version 960 (0.0009)
441
+ [2023-09-12 14:31:26,703][119814] Updated weights for policy 0, policy_version 970 (0.0008)
442
+ [2023-09-12 14:31:27,581][119815] Updated weights for policy 1, policy_version 820 (0.0008)
443
+ [2023-09-12 14:31:32,055][119377] Saving new best policy, reward=13.472!
444
+ [2023-09-12 14:31:32,055][119700] Saving new best policy, reward=7.678!
445
+ [2023-09-12 14:31:32,235][119814] Updated weights for policy 0, policy_version 980 (0.0009)
446
+ [2023-09-12 14:31:34,968][119815] Updated weights for policy 1, policy_version 830 (0.0009)
447
+ [2023-09-12 14:31:38,793][119814] Updated weights for policy 0, policy_version 990 (0.0009)
448
+ [2023-09-12 14:31:41,363][119815] Updated weights for policy 1, policy_version 840 (0.0009)
449
+ [2023-09-12 14:31:42,056][119377] Saving new best policy, reward=14.281!
450
+ [2023-09-12 14:31:44,940][119814] Updated weights for policy 0, policy_version 1000 (0.0009)
451
+ [2023-09-12 14:31:47,060][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000846_3465216.pth...
452
+ [2023-09-12 14:31:47,060][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001003_4108288.pth...
453
+ [2023-09-12 14:31:47,113][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000512_2097152.pth
454
+ [2023-09-12 14:31:47,113][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000591_2420736.pth
455
+ [2023-09-12 14:31:50,411][119814] Updated weights for policy 0, policy_version 1010 (0.0009)
456
+ [2023-09-12 14:31:50,457][119815] Updated weights for policy 1, policy_version 850 (0.0009)
457
+ [2023-09-12 14:31:55,644][119814] Updated weights for policy 0, policy_version 1020 (0.0009)
458
+ [2023-09-12 14:31:57,069][119700] Saving new best policy, reward=7.932!
459
+ [2023-09-12 14:31:59,071][119815] Updated weights for policy 1, policy_version 860 (0.0010)
460
+ [2023-09-12 14:32:01,893][119814] Updated weights for policy 0, policy_version 1030 (0.0009)
461
+ [2023-09-12 14:32:05,115][119815] Updated weights for policy 1, policy_version 870 (0.0009)
462
+ [2023-09-12 14:32:07,776][119814] Updated weights for policy 0, policy_version 1040 (0.0009)
463
+ [2023-09-12 14:32:13,061][119814] Updated weights for policy 0, policy_version 1050 (0.0009)
464
+ [2023-09-12 14:32:14,199][119815] Updated weights for policy 1, policy_version 880 (0.0009)
465
+ [2023-09-12 14:32:17,060][119377] Saving new best policy, reward=14.491!
466
+ [2023-09-12 14:32:19,478][119814] Updated weights for policy 0, policy_version 1060 (0.0009)
467
+ [2023-09-12 14:32:21,705][119815] Updated weights for policy 1, policy_version 890 (0.0009)
468
+ [2023-09-12 14:32:25,390][119814] Updated weights for policy 0, policy_version 1070 (0.0009)
469
+ [2023-09-12 14:32:30,186][119815] Updated weights for policy 1, policy_version 900 (0.0009)
470
+ [2023-09-12 14:32:31,109][119814] Updated weights for policy 0, policy_version 1080 (0.0009)
471
+ [2023-09-12 14:32:37,177][119814] Updated weights for policy 0, policy_version 1090 (0.0009)
472
+ [2023-09-12 14:32:38,061][119815] Updated weights for policy 1, policy_version 910 (0.0009)
473
+ [2023-09-12 14:32:42,650][119814] Updated weights for policy 0, policy_version 1100 (0.0009)
474
+ [2023-09-12 14:32:45,278][119815] Updated weights for policy 1, policy_version 920 (0.0009)
475
+ [2023-09-12 14:32:48,593][119814] Updated weights for policy 0, policy_version 1110 (0.0009)
476
+ [2023-09-12 14:32:52,766][119815] Updated weights for policy 1, policy_version 930 (0.0010)
477
+ [2023-09-12 14:32:53,912][119814] Updated weights for policy 0, policy_version 1120 (0.0009)
478
+ [2023-09-12 14:32:58,013][119814] Updated weights for policy 0, policy_version 1130 (0.0008)
479
+ [2023-09-12 14:33:00,195][119815] Updated weights for policy 1, policy_version 940 (0.0009)
480
+ [2023-09-12 14:33:02,499][119814] Updated weights for policy 0, policy_version 1140 (0.0009)
481
+ [2023-09-12 14:33:06,059][119815] Updated weights for policy 1, policy_version 950 (0.0009)
482
+ [2023-09-12 14:33:07,055][119814] Updated weights for policy 0, policy_version 1150 (0.0009)
483
+ [2023-09-12 14:33:11,418][119814] Updated weights for policy 0, policy_version 1160 (0.0009)
484
+ [2023-09-12 14:33:12,447][119815] Updated weights for policy 1, policy_version 960 (0.0009)
485
+ [2023-09-12 14:33:15,734][119814] Updated weights for policy 0, policy_version 1170 (0.0009)
486
+ [2023-09-12 14:33:18,289][119815] Updated weights for policy 1, policy_version 970 (0.0009)
487
+ [2023-09-12 14:33:20,671][119814] Updated weights for policy 0, policy_version 1180 (0.0008)
488
+ [2023-09-12 14:33:23,606][119815] Updated weights for policy 1, policy_version 980 (0.0008)
489
+ [2023-09-12 14:33:27,533][119814] Updated weights for policy 0, policy_version 1190 (0.0009)
490
+ [2023-09-12 14:33:29,960][119815] Updated weights for policy 1, policy_version 990 (0.0009)
491
+ [2023-09-12 14:33:33,244][119814] Updated weights for policy 0, policy_version 1200 (0.0009)
492
+ [2023-09-12 14:33:38,159][119814] Updated weights for policy 0, policy_version 1210 (0.0009)
493
+ [2023-09-12 14:33:40,314][119815] Updated weights for policy 1, policy_version 1000 (0.0008)
494
+ [2023-09-12 14:33:42,055][119377] Saving new best policy, reward=14.873!
495
+ [2023-09-12 14:33:42,056][119700] Saving new best policy, reward=8.275!
496
+ [2023-09-12 14:33:44,159][119814] Updated weights for policy 0, policy_version 1220 (0.0009)
497
+ [2023-09-12 14:33:47,059][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001224_5013504.pth...
498
+ [2023-09-12 14:33:47,069][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001010_4136960.pth...
499
+ [2023-09-12 14:33:47,071][119815] Updated weights for policy 1, policy_version 1010 (0.0009)
500
+ [2023-09-12 14:33:47,114][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000000787_3223552.pth
501
+ [2023-09-12 14:33:47,125][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000682_2793472.pth
502
+ [2023-09-12 14:33:50,291][119814] Updated weights for policy 0, policy_version 1230 (0.0009)
503
+ [2023-09-12 14:33:54,357][119815] Updated weights for policy 1, policy_version 1020 (0.0009)
504
+ [2023-09-12 14:33:57,060][119700] Saving new best policy, reward=8.325!
505
+ [2023-09-12 14:33:57,164][119814] Updated weights for policy 0, policy_version 1240 (0.0009)
506
+ [2023-09-12 14:34:01,301][119815] Updated weights for policy 1, policy_version 1030 (0.0008)
507
+ [2023-09-12 14:34:03,825][119814] Updated weights for policy 0, policy_version 1250 (0.0009)
508
+ [2023-09-12 14:34:06,583][119815] Updated weights for policy 1, policy_version 1040 (0.0009)
509
+ [2023-09-12 14:34:10,034][119814] Updated weights for policy 0, policy_version 1260 (0.0009)
510
+ [2023-09-12 14:34:13,323][119815] Updated weights for policy 1, policy_version 1050 (0.0010)
511
+ [2023-09-12 14:34:16,527][119814] Updated weights for policy 0, policy_version 1270 (0.0009)
512
+ [2023-09-12 14:34:17,061][119377] Saving new best policy, reward=15.672!
513
+ [2023-09-12 14:34:17,073][119700] Saving new best policy, reward=9.196!
514
+ [2023-09-12 14:34:21,389][119815] Updated weights for policy 1, policy_version 1060 (0.0010)
515
+ [2023-09-12 14:34:22,342][119814] Updated weights for policy 0, policy_version 1280 (0.0009)
516
+ [2023-09-12 14:34:28,061][119815] Updated weights for policy 1, policy_version 1070 (0.0009)
517
+ [2023-09-12 14:34:28,815][119814] Updated weights for policy 0, policy_version 1290 (0.0009)
518
+ [2023-09-12 14:34:34,181][119814] Updated weights for policy 0, policy_version 1300 (0.0008)
519
+ [2023-09-12 14:34:37,254][119815] Updated weights for policy 1, policy_version 1080 (0.0010)
520
+ [2023-09-12 14:34:39,937][119814] Updated weights for policy 0, policy_version 1310 (0.0009)
521
+ [2023-09-12 14:34:44,827][119815] Updated weights for policy 1, policy_version 1090 (0.0008)
522
+ [2023-09-12 14:34:45,964][119814] Updated weights for policy 0, policy_version 1320 (0.0009)
523
+ [2023-09-12 14:34:51,765][119815] Updated weights for policy 1, policy_version 1100 (0.0009)
524
+ [2023-09-12 14:34:52,090][119814] Updated weights for policy 0, policy_version 1330 (0.0008)
525
+ [2023-09-12 14:34:58,201][119814] Updated weights for policy 0, policy_version 1340 (0.0008)
526
+ [2023-09-12 14:34:59,158][119815] Updated weights for policy 1, policy_version 1110 (0.0009)
527
+ [2023-09-12 14:35:04,942][119815] Updated weights for policy 1, policy_version 1120 (0.0009)
528
+ [2023-09-12 14:35:05,104][119814] Updated weights for policy 0, policy_version 1350 (0.0009)
529
+ [2023-09-12 14:35:10,589][119814] Updated weights for policy 0, policy_version 1360 (0.0009)
530
+ [2023-09-12 14:35:12,055][119377] Saving new best policy, reward=16.013!
531
+ [2023-09-12 14:35:13,644][119815] Updated weights for policy 1, policy_version 1130 (0.0009)
532
+ [2023-09-12 14:35:16,410][119814] Updated weights for policy 0, policy_version 1370 (0.0009)
533
+ [2023-09-12 14:35:21,186][119815] Updated weights for policy 1, policy_version 1140 (0.0008)
534
+ [2023-09-12 14:35:22,056][119700] Saving new best policy, reward=9.676!
535
+ [2023-09-12 14:35:22,402][119814] Updated weights for policy 0, policy_version 1380 (0.0009)
536
+ [2023-09-12 14:35:26,696][119815] Updated weights for policy 1, policy_version 1150 (0.0008)
537
+ [2023-09-12 14:35:29,053][119814] Updated weights for policy 0, policy_version 1390 (0.0009)
538
+ [2023-09-12 14:35:34,290][119814] Updated weights for policy 0, policy_version 1400 (0.0009)
539
+ [2023-09-12 14:35:36,076][119815] Updated weights for policy 1, policy_version 1160 (0.0009)
540
+ [2023-09-12 14:35:39,905][119814] Updated weights for policy 0, policy_version 1410 (0.0009)
541
+ [2023-09-12 14:35:44,021][119815] Updated weights for policy 1, policy_version 1170 (0.0008)
542
+ [2023-09-12 14:35:45,885][119814] Updated weights for policy 0, policy_version 1420 (0.0008)
543
+ [2023-09-12 14:35:47,060][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001174_4808704.pth...
544
+ [2023-09-12 14:35:47,060][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001422_5824512.pth...
545
+ [2023-09-12 14:35:47,123][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000000846_3465216.pth
546
+ [2023-09-12 14:35:47,127][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001003_4108288.pth
547
+ [2023-09-12 14:35:47,130][119700] Saving new best policy, reward=10.540!
548
+ [2023-09-12 14:35:51,413][119814] Updated weights for policy 0, policy_version 1430 (0.0009)
549
+ [2023-09-12 14:35:52,910][119815] Updated weights for policy 1, policy_version 1180 (0.0009)
550
+ [2023-09-12 14:35:57,327][119814] Updated weights for policy 0, policy_version 1440 (0.0009)
551
+ [2023-09-12 14:36:00,859][119815] Updated weights for policy 1, policy_version 1190 (0.0008)
552
+ [2023-09-12 14:36:02,601][119814] Updated weights for policy 0, policy_version 1450 (0.0009)
553
+ [2023-09-12 14:36:06,672][119814] Updated weights for policy 0, policy_version 1460 (0.0009)
554
+ [2023-09-12 14:36:07,744][119815] Updated weights for policy 1, policy_version 1200 (0.0009)
555
+ [2023-09-12 14:36:11,229][119814] Updated weights for policy 0, policy_version 1470 (0.0009)
556
+ [2023-09-12 14:36:14,014][119815] Updated weights for policy 1, policy_version 1210 (0.0009)
557
+ [2023-09-12 14:36:15,216][119814] Updated weights for policy 0, policy_version 1480 (0.0009)
558
+ [2023-09-12 14:36:19,276][119814] Updated weights for policy 0, policy_version 1490 (0.0009)
559
+ [2023-09-12 14:36:20,831][119815] Updated weights for policy 1, policy_version 1220 (0.0009)
560
+ [2023-09-12 14:36:22,055][119377] Saving new best policy, reward=16.058!
561
+ [2023-09-12 14:36:23,646][119814] Updated weights for policy 0, policy_version 1500 (0.0008)
562
+ [2023-09-12 14:36:26,774][119815] Updated weights for policy 1, policy_version 1230 (0.0009)
563
+ [2023-09-12 14:36:27,062][119700] Saving new best policy, reward=10.603!
564
+ [2023-09-12 14:36:28,013][119814] Updated weights for policy 0, policy_version 1510 (0.0009)
565
+ [2023-09-12 14:36:32,055][119377] Saving new best policy, reward=16.730!
566
+ [2023-09-12 14:36:32,402][119815] Updated weights for policy 1, policy_version 1240 (0.0008)
567
+ [2023-09-12 14:36:32,678][119814] Updated weights for policy 0, policy_version 1520 (0.0009)
568
+ [2023-09-12 14:36:38,664][119815] Updated weights for policy 1, policy_version 1250 (0.0010)
569
+ [2023-09-12 14:36:39,238][119814] Updated weights for policy 0, policy_version 1530 (0.0009)
570
+ [2023-09-12 14:36:45,601][119815] Updated weights for policy 1, policy_version 1260 (0.0009)
571
+ [2023-09-12 14:36:45,824][119814] Updated weights for policy 0, policy_version 1540 (0.0010)
572
+ [2023-09-12 14:36:51,674][119814] Updated weights for policy 0, policy_version 1550 (0.0009)
573
+ [2023-09-12 14:36:53,404][119815] Updated weights for policy 1, policy_version 1270 (0.0008)
574
+ [2023-09-12 14:36:58,280][119814] Updated weights for policy 0, policy_version 1560 (0.0009)
575
+ [2023-09-12 14:36:59,377][119815] Updated weights for policy 1, policy_version 1280 (0.0008)
576
+ [2023-09-12 14:37:03,623][119814] Updated weights for policy 0, policy_version 1570 (0.0009)
577
+ [2023-09-12 14:37:08,485][119815] Updated weights for policy 1, policy_version 1290 (0.0009)
578
+ [2023-09-12 14:37:09,492][119814] Updated weights for policy 0, policy_version 1580 (0.0009)
579
+ [2023-09-12 14:37:14,923][119815] Updated weights for policy 1, policy_version 1300 (0.0009)
580
+ [2023-09-12 14:37:16,450][119814] Updated weights for policy 0, policy_version 1590 (0.0008)
581
+ [2023-09-12 14:37:21,084][119815] Updated weights for policy 1, policy_version 1310 (0.0008)
582
+ [2023-09-12 14:37:22,583][119814] Updated weights for policy 0, policy_version 1600 (0.0009)
583
+ [2023-09-12 14:37:28,627][119814] Updated weights for policy 0, policy_version 1610 (0.0009)
584
+ [2023-09-12 14:37:28,813][119815] Updated weights for policy 1, policy_version 1320 (0.0008)
585
+ [2023-09-12 14:37:32,055][119700] Saving new best policy, reward=10.973!
586
+ [2023-09-12 14:37:34,700][119815] Updated weights for policy 1, policy_version 1330 (0.0009)
587
+ [2023-09-12 14:37:35,451][119814] Updated weights for policy 0, policy_version 1620 (0.0009)
588
+ [2023-09-12 14:37:41,265][119814] Updated weights for policy 0, policy_version 1630 (0.0008)
589
+ [2023-09-12 14:37:42,056][119700] Saving new best policy, reward=11.602!
590
+ [2023-09-12 14:37:43,522][119815] Updated weights for policy 1, policy_version 1340 (0.0008)
591
+ [2023-09-12 14:37:46,922][119814] Updated weights for policy 0, policy_version 1640 (0.0008)
592
+ [2023-09-12 14:37:47,065][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001345_5509120.pth...
593
+ [2023-09-12 14:37:47,066][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001640_6717440.pth...
594
+ [2023-09-12 14:37:47,119][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001010_4136960.pth
595
+ [2023-09-12 14:37:47,132][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001224_5013504.pth
596
+ [2023-09-12 14:37:47,141][119377] Saving new best policy, reward=16.777!
597
+ [2023-09-12 14:37:50,304][119815] Updated weights for policy 1, policy_version 1350 (0.0009)
598
+ [2023-09-12 14:37:52,055][119377] Saving new best policy, reward=17.284!
599
+ [2023-09-12 14:37:52,688][119814] Updated weights for policy 0, policy_version 1650 (0.0009)
600
+ [2023-09-12 14:37:57,060][119377] Saving new best policy, reward=17.463!
601
+ [2023-09-12 14:37:58,257][119815] Updated weights for policy 1, policy_version 1360 (0.0009)
602
+ [2023-09-12 14:37:58,706][119814] Updated weights for policy 0, policy_version 1660 (0.0009)
603
+ [2023-09-12 14:38:04,604][119814] Updated weights for policy 0, policy_version 1670 (0.0009)
604
+ [2023-09-12 14:38:06,082][119815] Updated weights for policy 1, policy_version 1370 (0.0009)
605
+ [2023-09-12 14:38:07,060][119700] Saving new best policy, reward=11.958!
606
+ [2023-09-12 14:38:10,522][119814] Updated weights for policy 0, policy_version 1680 (0.0009)
607
+ [2023-09-12 14:38:14,710][119815] Updated weights for policy 1, policy_version 1380 (0.0009)
608
+ [2023-09-12 14:38:16,054][119814] Updated weights for policy 0, policy_version 1690 (0.0009)
609
+ [2023-09-12 14:38:21,825][119814] Updated weights for policy 0, policy_version 1700 (0.0010)
610
+ [2023-09-12 14:38:22,056][119377] Saving new best policy, reward=17.938!
611
+ [2023-09-12 14:38:22,056][119700] Saving new best policy, reward=12.182!
612
+ [2023-09-12 14:38:23,044][119815] Updated weights for policy 1, policy_version 1390 (0.0009)
613
+ [2023-09-12 14:38:27,060][119700] Saving new best policy, reward=13.264!
614
+ [2023-09-12 14:38:27,725][119814] Updated weights for policy 0, policy_version 1710 (0.0009)
615
+ [2023-09-12 14:38:30,135][119815] Updated weights for policy 1, policy_version 1400 (0.0008)
616
+ [2023-09-12 14:38:36,153][119815] Updated weights for policy 1, policy_version 1410 (0.0009)
617
+ [2023-09-12 14:38:36,440][119814] Updated weights for policy 0, policy_version 1720 (0.0012)
618
+ [2023-09-12 14:38:42,056][119700] Saving new best policy, reward=13.494!
619
+ [2023-09-12 14:38:42,502][119815] Updated weights for policy 1, policy_version 1420 (0.0009)
620
+ [2023-09-12 14:38:42,915][119814] Updated weights for policy 0, policy_version 1730 (0.0009)
621
+ [2023-09-12 14:38:49,090][119815] Updated weights for policy 1, policy_version 1430 (0.0009)
622
+ [2023-09-12 14:38:49,490][119814] Updated weights for policy 0, policy_version 1740 (0.0009)
623
+ [2023-09-12 14:38:55,645][119814] Updated weights for policy 0, policy_version 1750 (0.0009)
624
+ [2023-09-12 14:38:56,790][119815] Updated weights for policy 1, policy_version 1440 (0.0009)
625
+ [2023-09-12 14:39:01,723][119814] Updated weights for policy 0, policy_version 1760 (0.0010)
626
+ [2023-09-12 14:39:06,757][119814] Updated weights for policy 0, policy_version 1770 (0.0009)
627
+ [2023-09-12 14:39:07,480][119815] Updated weights for policy 1, policy_version 1450 (0.0008)
628
+ [2023-09-12 14:39:12,223][119814] Updated weights for policy 0, policy_version 1780 (0.0010)
629
+ [2023-09-12 14:39:14,721][119815] Updated weights for policy 1, policy_version 1460 (0.0009)
630
+ [2023-09-12 14:39:17,061][119377] Saving new best policy, reward=18.905!
631
+ [2023-09-12 14:39:18,137][119814] Updated weights for policy 0, policy_version 1790 (0.0009)
632
+ [2023-09-12 14:39:19,137][119815] Updated weights for policy 1, policy_version 1470 (0.0010)
633
+ [2023-09-12 14:39:22,623][119814] Updated weights for policy 0, policy_version 1800 (0.0009)
634
+ [2023-09-12 14:39:25,388][119815] Updated weights for policy 1, policy_version 1480 (0.0009)
635
+ [2023-09-12 14:39:27,581][119814] Updated weights for policy 0, policy_version 1810 (0.0009)
636
+ [2023-09-12 14:39:29,803][119815] Updated weights for policy 1, policy_version 1490 (0.0008)
637
+ [2023-09-12 14:39:32,055][119700] Saving new best policy, reward=13.942!
638
+ [2023-09-12 14:39:32,967][119814] Updated weights for policy 0, policy_version 1820 (0.0009)
639
+ [2023-09-12 14:39:36,459][119815] Updated weights for policy 1, policy_version 1500 (0.0008)
640
+ [2023-09-12 14:39:37,110][119814] Updated weights for policy 0, policy_version 1830 (0.0008)
641
+ [2023-09-12 14:39:41,780][119814] Updated weights for policy 0, policy_version 1840 (0.0009)
642
+ [2023-09-12 14:39:41,963][119815] Updated weights for policy 1, policy_version 1510 (0.0009)
643
+ [2023-09-12 14:39:42,056][119700] Saving new best policy, reward=14.371!
644
+ [2023-09-12 14:39:46,060][119814] Updated weights for policy 0, policy_version 1850 (0.0009)
645
+ [2023-09-12 14:39:47,060][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001517_6213632.pth...
646
+ [2023-09-12 14:39:47,061][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001852_7585792.pth...
647
+ [2023-09-12 14:39:47,115][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001422_5824512.pth
648
+ [2023-09-12 14:39:47,126][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001174_4808704.pth
649
+ [2023-09-12 14:39:49,355][119815] Updated weights for policy 1, policy_version 1520 (0.0009)
650
+ [2023-09-12 14:39:51,632][119814] Updated weights for policy 0, policy_version 1860 (0.0009)
651
+ [2023-09-12 14:39:55,670][119815] Updated weights for policy 1, policy_version 1530 (0.0008)
652
+ [2023-09-12 14:39:58,045][119814] Updated weights for policy 0, policy_version 1870 (0.0008)
653
+ [2023-09-12 14:40:03,009][119815] Updated weights for policy 1, policy_version 1540 (0.0009)
654
+ [2023-09-12 14:40:04,454][119814] Updated weights for policy 0, policy_version 1880 (0.0009)
655
+ [2023-09-12 14:40:10,680][119814] Updated weights for policy 0, policy_version 1890 (0.0009)
656
+ [2023-09-12 14:40:11,048][119815] Updated weights for policy 1, policy_version 1550 (0.0010)
657
+ [2023-09-12 14:40:16,294][119814] Updated weights for policy 0, policy_version 1900 (0.0009)
658
+ [2023-09-12 14:40:20,227][119815] Updated weights for policy 1, policy_version 1560 (0.0008)
659
+ [2023-09-12 14:40:22,133][119814] Updated weights for policy 0, policy_version 1910 (0.0008)
660
+ [2023-09-12 14:40:26,516][119815] Updated weights for policy 1, policy_version 1570 (0.0008)
661
+ [2023-09-12 14:40:27,061][119377] Saving new best policy, reward=20.304!
662
+ [2023-09-12 14:40:28,850][119814] Updated weights for policy 0, policy_version 1920 (0.0008)
663
+ [2023-09-12 14:40:33,570][119815] Updated weights for policy 1, policy_version 1580 (0.0008)
664
+ [2023-09-12 14:40:34,589][119814] Updated weights for policy 0, policy_version 1930 (0.0008)
665
+ [2023-09-12 14:40:37,060][119700] Saving new best policy, reward=14.716!
666
+ [2023-09-12 14:40:40,805][119814] Updated weights for policy 0, policy_version 1940 (0.0009)
667
+ [2023-09-12 14:40:41,356][119815] Updated weights for policy 1, policy_version 1590 (0.0009)
668
+ [2023-09-12 14:40:42,055][119700] Saving new best policy, reward=15.108!
669
+ [2023-09-12 14:40:46,505][119814] Updated weights for policy 0, policy_version 1950 (0.0009)
670
+ [2023-09-12 14:40:50,028][119815] Updated weights for policy 1, policy_version 1600 (0.0008)
671
+ [2023-09-12 14:40:52,130][119814] Updated weights for policy 0, policy_version 1960 (0.0009)
672
+ [2023-09-12 14:40:56,859][119815] Updated weights for policy 1, policy_version 1610 (0.0009)
673
+ [2023-09-12 14:40:58,701][119814] Updated weights for policy 0, policy_version 1970 (0.0008)
674
+ [2023-09-12 14:41:03,955][119815] Updated weights for policy 1, policy_version 1620 (0.0010)
675
+ [2023-09-12 14:41:05,016][119814] Updated weights for policy 0, policy_version 1980 (0.0009)
676
+ [2023-09-12 14:41:07,062][119700] Saving new best policy, reward=15.880!
677
+ [2023-09-12 14:41:11,194][119814] Updated weights for policy 0, policy_version 1990 (0.0009)
678
+ [2023-09-12 14:41:11,970][119815] Updated weights for policy 1, policy_version 1630 (0.0009)
679
+ [2023-09-12 14:41:16,986][119814] Updated weights for policy 0, policy_version 2000 (0.0009)
680
+ [2023-09-12 14:41:19,388][119815] Updated weights for policy 1, policy_version 1640 (0.0009)
681
+ [2023-09-12 14:41:23,500][119814] Updated weights for policy 0, policy_version 2010 (0.0008)
682
+ [2023-09-12 14:41:25,964][119815] Updated weights for policy 1, policy_version 1650 (0.0009)
683
+ [2023-09-12 14:41:29,506][119814] Updated weights for policy 0, policy_version 2020 (0.0009)
684
+ [2023-09-12 14:41:33,252][119815] Updated weights for policy 1, policy_version 1660 (0.0010)
685
+ [2023-09-12 14:41:36,028][119814] Updated weights for policy 0, policy_version 2030 (0.0008)
686
+ [2023-09-12 14:41:40,598][119815] Updated weights for policy 1, policy_version 1670 (0.0010)
687
+ [2023-09-12 14:41:42,055][119377] Saving new best policy, reward=20.403!
688
+ [2023-09-12 14:41:42,056][119700] Saving new best policy, reward=16.295!
689
+ [2023-09-12 14:41:42,414][119814] Updated weights for policy 0, policy_version 2040 (0.0009)
690
+ [2023-09-12 14:41:46,051][119815] Updated weights for policy 1, policy_version 1680 (0.0009)
691
+ [2023-09-12 14:41:47,060][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001681_6885376.pth...
692
+ [2023-09-12 14:41:47,060][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002046_8380416.pth...
693
+ [2023-09-12 14:41:47,112][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001345_5509120.pth
694
+ [2023-09-12 14:41:47,114][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001640_6717440.pth
695
+ [2023-09-12 14:41:49,193][119814] Updated weights for policy 0, policy_version 2050 (0.0008)
696
+ [2023-09-12 14:41:53,731][119815] Updated weights for policy 1, policy_version 1690 (0.0009)
697
+ [2023-09-12 14:41:54,660][119814] Updated weights for policy 0, policy_version 2060 (0.0009)
698
+ [2023-09-12 14:41:59,010][119815] Updated weights for policy 1, policy_version 1700 (0.0009)
699
+ [2023-09-12 14:42:01,966][119814] Updated weights for policy 0, policy_version 2070 (0.0008)
700
+ [2023-09-12 14:42:05,063][119815] Updated weights for policy 1, policy_version 1710 (0.0009)
701
+ [2023-09-12 14:42:08,729][119814] Updated weights for policy 0, policy_version 2080 (0.0009)
702
+ [2023-09-12 14:42:11,167][119815] Updated weights for policy 1, policy_version 1720 (0.0009)
703
+ [2023-09-12 14:42:15,114][119814] Updated weights for policy 0, policy_version 2090 (0.0009)
704
+ [2023-09-12 14:42:18,267][119815] Updated weights for policy 1, policy_version 1730 (0.0010)
705
+ [2023-09-12 14:42:21,799][119814] Updated weights for policy 0, policy_version 2100 (0.0009)
706
+ [2023-09-12 14:42:25,420][119815] Updated weights for policy 1, policy_version 1740 (0.0009)
707
+ [2023-09-12 14:42:27,257][119814] Updated weights for policy 0, policy_version 2110 (0.0009)
708
+ [2023-09-12 14:42:31,606][119814] Updated weights for policy 0, policy_version 2120 (0.0009)
709
+ [2023-09-12 14:42:31,854][119815] Updated weights for policy 1, policy_version 1750 (0.0008)
710
+ [2023-09-12 14:42:36,295][119814] Updated weights for policy 0, policy_version 2130 (0.0009)
711
+ [2023-09-12 14:42:37,590][119815] Updated weights for policy 1, policy_version 1760 (0.0009)
712
+ [2023-09-12 14:42:41,039][119814] Updated weights for policy 0, policy_version 2140 (0.0008)
713
+ [2023-09-12 14:42:43,147][119815] Updated weights for policy 1, policy_version 1770 (0.0009)
714
+ [2023-09-12 14:42:45,702][119814] Updated weights for policy 0, policy_version 2150 (0.0009)
715
+ [2023-09-12 14:42:48,972][119815] Updated weights for policy 1, policy_version 1780 (0.0008)
716
+ [2023-09-12 14:42:50,121][119814] Updated weights for policy 0, policy_version 2160 (0.0008)
717
+ [2023-09-12 14:42:52,055][119700] Saving new best policy, reward=16.404!
718
+ [2023-09-12 14:42:54,784][119815] Updated weights for policy 1, policy_version 1790 (0.0009)
719
+ [2023-09-12 14:42:54,821][119814] Updated weights for policy 0, policy_version 2170 (0.0009)
720
+ [2023-09-12 14:42:59,378][119815] Updated weights for policy 1, policy_version 1800 (0.0009)
721
+ [2023-09-12 14:43:01,730][119814] Updated weights for policy 0, policy_version 2180 (0.0009)
722
+ [2023-09-12 14:43:07,512][119814] Updated weights for policy 0, policy_version 2190 (0.0008)
723
+ [2023-09-12 14:43:07,759][119815] Updated weights for policy 1, policy_version 1810 (0.0009)
724
+ [2023-09-12 14:43:12,055][119377] Saving new best policy, reward=20.623!
725
+ [2023-09-12 14:43:13,709][119814] Updated weights for policy 0, policy_version 2200 (0.0008)
726
+ [2023-09-12 14:43:14,531][119815] Updated weights for policy 1, policy_version 1820 (0.0009)
727
+ [2023-09-12 14:43:17,060][119700] Saving new best policy, reward=16.503!
728
+ [2023-09-12 14:43:20,410][119814] Updated weights for policy 0, policy_version 2210 (0.0009)
729
+ [2023-09-12 14:43:20,891][119815] Updated weights for policy 1, policy_version 1830 (0.0009)
730
+ [2023-09-12 14:43:26,573][119814] Updated weights for policy 0, policy_version 2220 (0.0009)
731
+ [2023-09-12 14:43:27,672][119815] Updated weights for policy 1, policy_version 1840 (0.0008)
732
+ [2023-09-12 14:43:32,982][119814] Updated weights for policy 0, policy_version 2230 (0.0009)
733
+ [2023-09-12 14:43:34,424][119815] Updated weights for policy 1, policy_version 1850 (0.0010)
734
+ [2023-09-12 14:43:39,626][119814] Updated weights for policy 0, policy_version 2240 (0.0009)
735
+ [2023-09-12 14:43:40,442][119815] Updated weights for policy 1, policy_version 1860 (0.0009)
736
+ [2023-09-12 14:43:42,143][119377] Saving new best policy, reward=20.783!
737
+ [2023-09-12 14:43:45,857][119814] Updated weights for policy 0, policy_version 2250 (0.0009)
738
+ [2023-09-12 14:43:47,061][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002252_9224192.pth...
739
+ [2023-09-12 14:43:47,061][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001869_7655424.pth...
740
+ [2023-09-12 14:43:47,134][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001517_6213632.pth
741
+ [2023-09-12 14:43:47,134][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000001852_7585792.pth
742
+ [2023-09-12 14:43:47,498][119815] Updated weights for policy 1, policy_version 1870 (0.0008)
743
+ [2023-09-12 14:43:51,252][119814] Updated weights for policy 0, policy_version 2260 (0.0010)
744
+ [2023-09-12 14:43:52,056][119700] Saving new best policy, reward=17.051!
745
+ [2023-09-12 14:43:55,594][119815] Updated weights for policy 1, policy_version 1880 (0.0009)
746
+ [2023-09-12 14:43:57,752][119814] Updated weights for policy 0, policy_version 2270 (0.0009)
747
+ [2023-09-12 14:44:02,117][119377] Saving new best policy, reward=21.145!
748
+ [2023-09-12 14:44:02,203][119815] Updated weights for policy 1, policy_version 1890 (0.0009)
749
+ [2023-09-12 14:44:03,897][119814] Updated weights for policy 0, policy_version 2280 (0.0009)
750
+ [2023-09-12 14:44:08,101][119815] Updated weights for policy 1, policy_version 1900 (0.0008)
751
+ [2023-09-12 14:44:10,738][119814] Updated weights for policy 0, policy_version 2290 (0.0009)
752
+ [2023-09-12 14:44:13,042][119815] Updated weights for policy 1, policy_version 1910 (0.0009)
753
+ [2023-09-12 14:44:18,009][119815] Updated weights for policy 1, policy_version 1920 (0.0008)
754
+ [2023-09-12 14:44:18,585][119814] Updated weights for policy 0, policy_version 2300 (0.0008)
755
+ [2023-09-12 14:44:24,502][119814] Updated weights for policy 0, policy_version 2310 (0.0009)
756
+ [2023-09-12 14:44:25,651][119815] Updated weights for policy 1, policy_version 1930 (0.0009)
757
+ [2023-09-12 14:44:30,988][119815] Updated weights for policy 1, policy_version 1940 (0.0009)
758
+ [2023-09-12 14:44:31,705][119814] Updated weights for policy 0, policy_version 2320 (0.0009)
759
+ [2023-09-12 14:44:37,218][119814] Updated weights for policy 0, policy_version 2330 (0.0008)
760
+ [2023-09-12 14:44:39,208][119815] Updated weights for policy 1, policy_version 1950 (0.0008)
761
+ [2023-09-12 14:44:43,493][119814] Updated weights for policy 0, policy_version 2340 (0.0010)
762
+ [2023-09-12 14:44:46,489][119815] Updated weights for policy 1, policy_version 1960 (0.0009)
763
+ [2023-09-12 14:44:47,062][119700] Saving new best policy, reward=17.231!
764
+ [2023-09-12 14:44:50,004][119814] Updated weights for policy 0, policy_version 2350 (0.0008)
765
+ [2023-09-12 14:44:53,168][119815] Updated weights for policy 1, policy_version 1970 (0.0009)
766
+ [2023-09-12 14:44:56,669][119814] Updated weights for policy 0, policy_version 2360 (0.0009)
767
+ [2023-09-12 14:44:57,063][119700] Saving new best policy, reward=17.244!
768
+ [2023-09-12 14:44:59,919][119815] Updated weights for policy 1, policy_version 1980 (0.0009)
769
+ [2023-09-12 14:45:04,782][119814] Updated weights for policy 0, policy_version 2370 (0.0009)
770
+ [2023-09-12 14:45:04,932][119815] Updated weights for policy 1, policy_version 1990 (0.0008)
771
+ [2023-09-12 14:45:11,460][119814] Updated weights for policy 0, policy_version 2380 (0.0009)
772
+ [2023-09-12 14:45:11,831][119815] Updated weights for policy 1, policy_version 2000 (0.0010)
773
+ [2023-09-12 14:45:16,785][119814] Updated weights for policy 0, policy_version 2390 (0.0009)
774
+ [2023-09-12 14:45:22,169][119814] Updated weights for policy 0, policy_version 2400 (0.0009)
775
+ [2023-09-12 14:45:22,389][119815] Updated weights for policy 1, policy_version 2010 (0.0008)
776
+ [2023-09-12 14:45:28,521][119814] Updated weights for policy 0, policy_version 2410 (0.0008)
777
+ [2023-09-12 14:45:29,132][119815] Updated weights for policy 1, policy_version 2020 (0.0010)
778
+ [2023-09-12 14:45:32,055][119700] Saving new best policy, reward=17.421!
779
+ [2023-09-12 14:45:34,463][119814] Updated weights for policy 0, policy_version 2420 (0.0009)
780
+ [2023-09-12 14:45:37,059][119377] Saving new best policy, reward=21.352!
781
+ [2023-09-12 14:45:37,365][119815] Updated weights for policy 1, policy_version 2030 (0.0010)
782
+ [2023-09-12 14:45:38,923][119814] Updated weights for policy 0, policy_version 2430 (0.0009)
783
+ [2023-09-12 14:45:42,759][119815] Updated weights for policy 1, policy_version 2040 (0.0008)
784
+ [2023-09-12 14:45:43,890][119814] Updated weights for policy 0, policy_version 2440 (0.0009)
785
+ [2023-09-12 14:45:47,060][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002447_10022912.pth...
786
+ [2023-09-12 14:45:47,060][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000002047_8384512.pth...
787
+ [2023-09-12 14:45:47,117][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002046_8380416.pth
788
+ [2023-09-12 14:45:47,120][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001681_6885376.pth
789
+ [2023-09-12 14:45:48,144][119815] Updated weights for policy 1, policy_version 2050 (0.0008)
790
+ [2023-09-12 14:45:48,749][119814] Updated weights for policy 0, policy_version 2450 (0.0009)
791
+ [2023-09-12 14:45:52,949][119815] Updated weights for policy 1, policy_version 2060 (0.0009)
792
+ [2023-09-12 14:45:53,993][119814] Updated weights for policy 0, policy_version 2460 (0.0009)
793
+ [2023-09-12 14:45:58,018][119815] Updated weights for policy 1, policy_version 2070 (0.0008)
794
+ [2023-09-12 14:45:58,945][119814] Updated weights for policy 0, policy_version 2470 (0.0008)
795
+ [2023-09-12 14:46:03,007][119814] Updated weights for policy 0, policy_version 2480 (0.0009)
796
+ [2023-09-12 14:46:04,678][119815] Updated weights for policy 1, policy_version 2080 (0.0009)
797
+ [2023-09-12 14:46:07,547][119814] Updated weights for policy 0, policy_version 2490 (0.0009)
798
+ [2023-09-12 14:46:11,008][119815] Updated weights for policy 1, policy_version 2090 (0.0008)
799
+ [2023-09-12 14:46:11,618][119814] Updated weights for policy 0, policy_version 2500 (0.0009)
800
+ [2023-09-12 14:46:16,057][119814] Updated weights for policy 0, policy_version 2510 (0.0008)
801
+ [2023-09-12 14:46:18,084][119815] Updated weights for policy 1, policy_version 2100 (0.0009)
802
+ [2023-09-12 14:46:22,881][119814] Updated weights for policy 0, policy_version 2520 (0.0009)
803
+ [2023-09-12 14:46:24,285][119815] Updated weights for policy 1, policy_version 2110 (0.0009)
804
+ [2023-09-12 14:46:29,361][119814] Updated weights for policy 0, policy_version 2530 (0.0009)
805
+ [2023-09-12 14:46:30,692][119815] Updated weights for policy 1, policy_version 2120 (0.0009)
806
+ [2023-09-12 14:46:32,056][119700] Saving new best policy, reward=18.123!
807
+ [2023-09-12 14:46:34,956][119814] Updated weights for policy 0, policy_version 2540 (0.0008)
808
+ [2023-09-12 14:46:40,027][119815] Updated weights for policy 1, policy_version 2130 (0.0009)
809
+ [2023-09-12 14:46:40,705][119814] Updated weights for policy 0, policy_version 2550 (0.0009)
810
+ [2023-09-12 14:46:45,980][119815] Updated weights for policy 1, policy_version 2140 (0.0009)
811
+ [2023-09-12 14:46:47,875][119814] Updated weights for policy 0, policy_version 2560 (0.0009)
812
+ [2023-09-12 14:46:52,721][119815] Updated weights for policy 1, policy_version 2150 (0.0010)
813
+ [2023-09-12 14:46:55,428][119814] Updated weights for policy 0, policy_version 2570 (0.0009)
814
+ [2023-09-12 14:46:58,157][119815] Updated weights for policy 1, policy_version 2160 (0.0010)
815
+ [2023-09-12 14:47:02,979][119814] Updated weights for policy 0, policy_version 2580 (0.0009)
816
+ [2023-09-12 14:47:03,580][119815] Updated weights for policy 1, policy_version 2170 (0.0010)
817
+ [2023-09-12 14:47:08,673][119814] Updated weights for policy 0, policy_version 2590 (0.0009)
818
+ [2023-09-12 14:47:12,195][119815] Updated weights for policy 1, policy_version 2180 (0.0008)
819
+ [2023-09-12 14:47:14,589][119814] Updated weights for policy 0, policy_version 2600 (0.0009)
820
+ [2023-09-12 14:47:19,719][119815] Updated weights for policy 1, policy_version 2190 (0.0009)
821
+ [2023-09-12 14:47:20,178][119814] Updated weights for policy 0, policy_version 2610 (0.0008)
822
+ [2023-09-12 14:47:24,876][119815] Updated weights for policy 1, policy_version 2200 (0.0009)
823
+ [2023-09-12 14:47:27,178][119814] Updated weights for policy 0, policy_version 2620 (0.0009)
824
+ [2023-09-12 14:47:31,477][119815] Updated weights for policy 1, policy_version 2210 (0.0008)
825
+ [2023-09-12 14:47:33,424][119814] Updated weights for policy 0, policy_version 2630 (0.0009)
826
+ [2023-09-12 14:47:37,506][119815] Updated weights for policy 1, policy_version 2220 (0.0009)
827
+ [2023-09-12 14:47:39,750][119814] Updated weights for policy 0, policy_version 2640 (0.0008)
828
+ [2023-09-12 14:47:45,522][119814] Updated weights for policy 0, policy_version 2650 (0.0009)
829
+ [2023-09-12 14:47:45,870][119815] Updated weights for policy 1, policy_version 2230 (0.0009)
830
+ [2023-09-12 14:47:47,060][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000002232_9142272.pth...
831
+ [2023-09-12 14:47:47,060][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002652_10862592.pth...
832
+ [2023-09-12 14:47:47,113][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002252_9224192.pth
833
+ [2023-09-12 14:47:47,113][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000001869_7655424.pth
834
+ [2023-09-12 14:47:51,280][119815] Updated weights for policy 1, policy_version 2240 (0.0009)
835
+ [2023-09-12 14:47:52,912][119814] Updated weights for policy 0, policy_version 2660 (0.0010)
836
+ [2023-09-12 14:47:57,731][119815] Updated weights for policy 1, policy_version 2250 (0.0008)
837
+ [2023-09-12 14:48:00,494][119814] Updated weights for policy 0, policy_version 2670 (0.0009)
838
+ [2023-09-12 14:48:02,690][119815] Updated weights for policy 1, policy_version 2260 (0.0009)
839
+ [2023-09-12 14:48:07,515][119814] Updated weights for policy 0, policy_version 2680 (0.0009)
840
+ [2023-09-12 14:48:09,836][119815] Updated weights for policy 1, policy_version 2270 (0.0009)
841
+ [2023-09-12 14:48:13,404][119814] Updated weights for policy 0, policy_version 2690 (0.0008)
842
+ [2023-09-12 14:48:17,529][119815] Updated weights for policy 1, policy_version 2280 (0.0008)
843
+ [2023-09-12 14:48:19,451][119814] Updated weights for policy 0, policy_version 2700 (0.0008)
844
+ [2023-09-12 14:48:25,077][119815] Updated weights for policy 1, policy_version 2290 (0.0008)
845
+ [2023-09-12 14:48:25,308][119814] Updated weights for policy 0, policy_version 2710 (0.0009)
846
+ [2023-09-12 14:48:31,624][119814] Updated weights for policy 0, policy_version 2720 (0.0009)
847
+ [2023-09-12 14:48:32,248][119815] Updated weights for policy 1, policy_version 2300 (0.0009)
848
+ [2023-09-12 14:48:37,596][119814] Updated weights for policy 0, policy_version 2730 (0.0008)
849
+ [2023-09-12 14:48:39,672][119815] Updated weights for policy 1, policy_version 2310 (0.0009)
850
+ [2023-09-12 14:48:43,628][119814] Updated weights for policy 0, policy_version 2740 (0.0009)
851
+ [2023-09-12 14:48:47,060][119700] Saving new best policy, reward=18.298!
852
+ [2023-09-12 14:48:47,202][119815] Updated weights for policy 1, policy_version 2320 (0.0009)
853
+ [2023-09-12 14:48:49,449][119814] Updated weights for policy 0, policy_version 2750 (0.0009)
854
+ [2023-09-12 14:48:53,597][119815] Updated weights for policy 1, policy_version 2330 (0.0010)
855
+ [2023-09-12 14:48:55,723][119814] Updated weights for policy 0, policy_version 2760 (0.0010)
856
+ [2023-09-12 14:48:57,437][119815] Updated weights for policy 1, policy_version 2340 (0.0009)
857
+ [2023-09-12 14:49:01,292][119814] Updated weights for policy 0, policy_version 2770 (0.0009)
858
+ [2023-09-12 14:49:02,470][119815] Updated weights for policy 1, policy_version 2350 (0.0010)
859
+ [2023-09-12 14:49:07,029][119814] Updated weights for policy 0, policy_version 2780 (0.0010)
860
+ [2023-09-12 14:49:07,411][119815] Updated weights for policy 1, policy_version 2360 (0.0010)
861
+ [2023-09-12 14:49:12,339][119814] Updated weights for policy 0, policy_version 2790 (0.0009)
862
+ [2023-09-12 14:49:13,240][119815] Updated weights for policy 1, policy_version 2370 (0.0009)
863
+ [2023-09-12 14:49:17,186][119814] Updated weights for policy 0, policy_version 2800 (0.0010)
864
+ [2023-09-12 14:49:18,622][119815] Updated weights for policy 1, policy_version 2380 (0.0009)
865
+ [2023-09-12 14:49:22,027][119814] Updated weights for policy 0, policy_version 2810 (0.0009)
866
+ [2023-09-12 14:49:22,056][119700] Saving new best policy, reward=18.560!
867
+ [2023-09-12 14:49:24,886][119815] Updated weights for policy 1, policy_version 2390 (0.0008)
868
+ [2023-09-12 14:49:26,983][119814] Updated weights for policy 0, policy_version 2820 (0.0009)
869
+ [2023-09-12 14:49:32,710][119814] Updated weights for policy 0, policy_version 2830 (0.0009)
870
+ [2023-09-12 14:49:32,770][119815] Updated weights for policy 1, policy_version 2400 (0.0008)
871
+ [2023-09-12 14:49:38,154][119814] Updated weights for policy 0, policy_version 2840 (0.0009)
872
+ [2023-09-12 14:49:41,905][119815] Updated weights for policy 1, policy_version 2410 (0.0009)
873
+ [2023-09-12 14:49:42,117][119377] Saving new best policy, reward=21.699!
874
+ [2023-09-12 14:49:44,187][119814] Updated weights for policy 0, policy_version 2850 (0.0008)
875
+ [2023-09-12 14:49:47,060][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002853_11685888.pth...
876
+ [2023-09-12 14:49:47,102][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000002419_9908224.pth...
877
+ [2023-09-12 14:49:47,126][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002447_10022912.pth
878
+ [2023-09-12 14:49:47,156][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000002047_8384512.pth
879
+ [2023-09-12 14:49:47,762][119815] Updated weights for policy 1, policy_version 2420 (0.0008)
880
+ [2023-09-12 14:49:51,624][119814] Updated weights for policy 0, policy_version 2860 (0.0009)
881
+ [2023-09-12 14:49:53,095][119815] Updated weights for policy 1, policy_version 2430 (0.0008)
882
+ [2023-09-12 14:49:57,059][119700] Saving new best policy, reward=18.719!
883
+ [2023-09-12 14:49:58,416][119814] Updated weights for policy 0, policy_version 2870 (0.0008)
884
+ [2023-09-12 14:49:59,474][119815] Updated weights for policy 1, policy_version 2440 (0.0008)
885
+ [2023-09-12 14:50:00,953][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002874_11771904.pth...
886
+ [2023-09-12 14:50:00,953][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000002442_10002432.pth...
887
+ [2023-09-12 14:50:00,953][119700] Stopping Batcher_1...
888
+ [2023-09-12 14:50:00,965][119700] Loop batcher_evt_loop terminating...
889
+ [2023-09-12 14:50:00,969][119816] Stopping RolloutWorker_w0...
890
+ [2023-09-12 14:50:00,969][119816] Loop rollout_proc0_evt_loop terminating...
891
+ [2023-09-12 14:50:00,969][119817] Stopping RolloutWorker_w2...
892
+ [2023-09-12 14:50:00,970][119817] Loop rollout_proc2_evt_loop terminating...
893
+ [2023-09-12 14:50:00,970][119818] Stopping RolloutWorker_w1...
894
+ [2023-09-12 14:50:00,971][119819] Stopping RolloutWorker_w3...
895
+ [2023-09-12 14:50:00,971][119818] Loop rollout_proc1_evt_loop terminating...
896
+ [2023-09-12 14:50:00,971][119819] Loop rollout_proc3_evt_loop terminating...
897
+ [2023-09-12 14:50:00,971][119882] Stopping RolloutWorker_w4...
898
+ [2023-09-12 14:50:00,971][119882] Loop rollout_proc4_evt_loop terminating...
899
+ [2023-09-12 14:50:00,971][119916] Stopping RolloutWorker_w5...
900
+ [2023-09-12 14:50:00,972][119916] Loop rollout_proc5_evt_loop terminating...
901
+ [2023-09-12 14:50:00,973][119814] Weights refcount: 2 0
902
+ [2023-09-12 14:50:00,973][119917] Stopping RolloutWorker_w6...
903
+ [2023-09-12 14:50:00,974][119917] Loop rollout_proc6_evt_loop terminating...
904
+ [2023-09-12 14:50:00,975][119814] Stopping InferenceWorker_p0-w0...
905
+ [2023-09-12 14:50:00,975][119814] Loop inference_proc0-0_evt_loop terminating...
906
+ [2023-09-12 14:50:00,976][119815] Weights refcount: 2 0
907
+ [2023-09-12 14:50:00,977][119815] Stopping InferenceWorker_p1-w0...
908
+ [2023-09-12 14:50:00,978][119815] Loop inference_proc1-0_evt_loop terminating...
909
+ [2023-09-12 14:50:00,968][119377] Stopping Batcher_0...
910
+ [2023-09-12 14:50:00,979][119918] Stopping RolloutWorker_w7...
911
+ [2023-09-12 14:50:00,979][119918] Loop rollout_proc7_evt_loop terminating...
912
+ [2023-09-12 14:50:00,989][119377] Loop batcher_evt_loop terminating...
913
+ [2023-09-12 14:50:01,018][119700] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000002232_9142272.pth
914
+ [2023-09-12 14:50:01,023][119377] Removing /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002652_10862592.pth
915
+ [2023-09-12 14:50:01,028][119700] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p1/checkpoint_000002442_10002432.pth...
916
+ [2023-09-12 14:50:01,034][119377] Saving /home/cogstack/Documents/optuna/environments/sample_factory/train_dir/default_experiment/checkpoint_p0/checkpoint_000002874_11771904.pth...
917
+ [2023-09-12 14:50:01,119][119700] Stopping LearnerWorker_p1...
918
+ [2023-09-12 14:50:01,120][119700] Loop learner_proc1_evt_loop terminating...
919
+ [2023-09-12 14:50:01,121][119377] Stopping LearnerWorker_p0...
920
+ [2023-09-12 14:50:01,122][119377] Loop learner_proc0_evt_loop terminating...