Hugging Face's logo

markeidsaune
/
rl_course_vizdoom_health_gathering_supreme

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files Metrics Community
markeidsaune commited on
Commit
10eab5d
·
1 Parent(s): 02b6692

Upload folder using huggingface_hub

Browse files
Files changed (8) hide show
  1. .summary/0/events.out.tfevents.1684975218.barry +3 -0
  2. README.md +1 -1
  3. checkpoint_p0/best_000002236_9158656_reward_32.078.pth +3 -0
  4. checkpoint_p0/checkpoint_000002209_9048064.pth +3 -0
  5. checkpoint_p0/checkpoint_000002443_10006528.pth +3 -0
  6. config.json +1 -1
  7. replay.mp4 +2 -2
  8. sf_log.txt +977 -0
.summary/0/events.out.tfevents.1684975218.barry ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5cdc49930528db6cdc50e71d84efaec3d6200a38d581253d01080b6af6485e3
3
+ size 220052
README.md CHANGED
@@ -15,7 +15,7 @@ model-index:
15
  type: doom_health_gathering_supreme
16
  metrics:
17
  - type: mean_reward
18
- value: 9.31 +/- 4.84
19
  name: mean_reward
20
  verified: false
21
  ---
 
15
  type: doom_health_gathering_supreme
16
  metrics:
17
  - type: mean_reward
18
+ value: 12.85 +/- 6.79
19
  name: mean_reward
20
  verified: false
21
  ---
checkpoint_p0/best_000002236_9158656_reward_32.078.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2e62807a89bc5f4f65d5ab6e174a4dbfd768e962c6e867a38798a07c2081fcd
3
+ size 34928806
checkpoint_p0/checkpoint_000002209_9048064.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75ae5263cb9e1495fd07805afd2ce39ebb7dc595e36c9f10499a6d51d30874ca
3
+ size 34929220
checkpoint_p0/checkpoint_000002443_10006528.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4047f0f7065c2ff501bd24de2ca2cf24a8200cffbcf58a37ee8f9ed469386850
3
+ size 34929220
config.json CHANGED
@@ -65,7 +65,7 @@
65
  "summaries_use_frameskip": true,
66
  "heartbeat_interval": 20,
67
  "heartbeat_reporting_interval": 600,
68
- "train_for_env_steps": 4000000,
69
  "train_for_seconds": 10000000000,
70
  "save_every_sec": 120,
71
  "keep_checkpoints": 2,
 
65
  "summaries_use_frameskip": true,
66
  "heartbeat_interval": 20,
67
  "heartbeat_reporting_interval": 600,
68
+ "train_for_env_steps": 10000000,
69
  "train_for_seconds": 10000000000,
70
  "save_every_sec": 120,
71
  "keep_checkpoints": 2,
replay.mp4 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d8a5102fe8b96e9a3dc6b25b55f2fbd74dabba05e05fe71f0edfbe6d06b57370
3
- size 17482115
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:694b7594a71c86a676f78fb9c31063cd258dbac73e6eccc1431e44f197e39558
3
+ size 24847612
sf_log.txt CHANGED
@@ -704,3 +704,980 @@ main_loop: 191.8832
704
  [2023-05-24 20:37:12,284][2722668] Avg episode rewards: #0: 20.614, true rewards: #0: 9.314
705
  [2023-05-24 20:37:12,285][2722668] Avg episode reward: 20.614, avg true_objective: 9.314
706
  [2023-05-24 20:37:34,803][2722668] Replay video saved to /home/mark/rl_course/unit8/train_dir/default_experiment/replay.mp4!
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
704
  [2023-05-24 20:37:12,284][2722668] Avg episode rewards: #0: 20.614, true rewards: #0: 9.314
705
  [2023-05-24 20:37:12,285][2722668] Avg episode reward: 20.614, avg true_objective: 9.314
706
  [2023-05-24 20:37:34,803][2722668] Replay video saved to /home/mark/rl_course/unit8/train_dir/default_experiment/replay.mp4!
707
+ [2023-05-24 20:39:03,801][2722668] The model has been pushed to https://huggingface.co/markeidsaune/rl_course_vizdoom_health_gathering_supreme
708
+ [2023-05-24 20:40:18,815][2722668] Environment doom_basic already registered, overwriting...
709
+ [2023-05-24 20:40:18,816][2722668] Environment doom_two_colors_easy already registered, overwriting...
710
+ [2023-05-24 20:40:18,817][2722668] Environment doom_two_colors_hard already registered, overwriting...
711
+ [2023-05-24 20:40:18,817][2722668] Environment doom_dm already registered, overwriting...
712
+ [2023-05-24 20:40:18,818][2722668] Environment doom_dwango5 already registered, overwriting...
713
+ [2023-05-24 20:40:18,819][2722668] Environment doom_my_way_home_flat_actions already registered, overwriting...
714
+ [2023-05-24 20:40:18,820][2722668] Environment doom_defend_the_center_flat_actions already registered, overwriting...
715
+ [2023-05-24 20:40:18,821][2722668] Environment doom_my_way_home already registered, overwriting...
716
+ [2023-05-24 20:40:18,821][2722668] Environment doom_deadly_corridor already registered, overwriting...
717
+ [2023-05-24 20:40:18,822][2722668] Environment doom_defend_the_center already registered, overwriting...
718
+ [2023-05-24 20:40:18,823][2722668] Environment doom_defend_the_line already registered, overwriting...
719
+ [2023-05-24 20:40:18,823][2722668] Environment doom_health_gathering already registered, overwriting...
720
+ [2023-05-24 20:40:18,824][2722668] Environment doom_health_gathering_supreme already registered, overwriting...
721
+ [2023-05-24 20:40:18,825][2722668] Environment doom_battle already registered, overwriting...
722
+ [2023-05-24 20:40:18,825][2722668] Environment doom_battle2 already registered, overwriting...
723
+ [2023-05-24 20:40:18,826][2722668] Environment doom_duel_bots already registered, overwriting...
724
+ [2023-05-24 20:40:18,826][2722668] Environment doom_deathmatch_bots already registered, overwriting...
725
+ [2023-05-24 20:40:18,827][2722668] Environment doom_duel already registered, overwriting...
726
+ [2023-05-24 20:40:18,828][2722668] Environment doom_deathmatch_full already registered, overwriting...
727
+ [2023-05-24 20:40:18,828][2722668] Environment doom_benchmark already registered, overwriting...
728
+ [2023-05-24 20:40:18,829][2722668] register_encoder_factory: <function make_vizdoom_encoder at 0x7f9e60e3bc70>
729
+ [2023-05-24 20:40:18,843][2722668] Loading existing experiment configuration from /home/mark/rl_course/unit8/train_dir/default_experiment/config.json
730
+ [2023-05-24 20:40:18,844][2722668] Overriding arg 'train_for_env_steps' with value 10000000 passed from command line
731
+ [2023-05-24 20:40:18,849][2722668] Experiment dir /home/mark/rl_course/unit8/train_dir/default_experiment already exists!
732
+ [2023-05-24 20:40:18,850][2722668] Resuming existing experiment from /home/mark/rl_course/unit8/train_dir/default_experiment...
733
+ [2023-05-24 20:40:18,850][2722668] Weights and Biases integration disabled
734
+ [2023-05-24 20:40:18,854][2722668] Environment var CUDA_VISIBLE_DEVICES is 0,1
735
+
736
+ [2023-05-24 20:40:20,634][2722668] Starting experiment with the following configuration:
737
+ help=False
738
+ algo=APPO
739
+ env=doom_health_gathering_supreme
740
+ experiment=default_experiment
741
+ train_dir=/home/mark/rl_course/unit8/train_dir
742
+ restart_behavior=resume
743
+ device=gpu
744
+ seed=None
745
+ num_policies=1
746
+ async_rl=True
747
+ serial_mode=False
748
+ batched_sampling=False
749
+ num_batches_to_accumulate=2
750
+ worker_num_splits=2
751
+ policy_workers_per_policy=1
752
+ max_policy_lag=1000
753
+ num_workers=8
754
+ num_envs_per_worker=4
755
+ batch_size=1024
756
+ num_batches_per_epoch=1
757
+ num_epochs=1
758
+ rollout=32
759
+ recurrence=32
760
+ shuffle_minibatches=False
761
+ gamma=0.99
762
+ reward_scale=1.0
763
+ reward_clip=1000.0
764
+ value_bootstrap=False
765
+ normalize_returns=True
766
+ exploration_loss_coeff=0.001
767
+ value_loss_coeff=0.5
768
+ kl_loss_coeff=0.0
769
+ exploration_loss=symmetric_kl
770
+ gae_lambda=0.95
771
+ ppo_clip_ratio=0.1
772
+ ppo_clip_value=0.2
773
+ with_vtrace=False
774
+ vtrace_rho=1.0
775
+ vtrace_c=1.0
776
+ optimizer=adam
777
+ adam_eps=1e-06
778
+ adam_beta1=0.9
779
+ adam_beta2=0.999
780
+ max_grad_norm=4.0
781
+ learning_rate=0.0001
782
+ lr_schedule=constant
783
+ lr_schedule_kl_threshold=0.008
784
+ lr_adaptive_min=1e-06
785
+ lr_adaptive_max=0.01
786
+ obs_subtract_mean=0.0
787
+ obs_scale=255.0
788
+ normalize_input=True
789
+ normalize_input_keys=None
790
+ decorrelate_experience_max_seconds=0
791
+ decorrelate_envs_on_one_worker=True
792
+ actor_worker_gpus=[]
793
+ set_workers_cpu_affinity=True
794
+ force_envs_single_thread=False
795
+ default_niceness=0
796
+ log_to_file=True
797
+ experiment_summaries_interval=10
798
+ flush_summaries_interval=30
799
+ stats_avg=100
800
+ summaries_use_frameskip=True
801
+ heartbeat_interval=20
802
+ heartbeat_reporting_interval=600
803
+ train_for_env_steps=10000000
804
+ train_for_seconds=10000000000
805
+ save_every_sec=120
806
+ keep_checkpoints=2
807
+ load_checkpoint_kind=latest
808
+ save_milestones_sec=-1
809
+ save_best_every_sec=5
810
+ save_best_metric=reward
811
+ save_best_after=100000
812
+ benchmark=False
813
+ encoder_mlp_layers=[512, 512]
814
+ encoder_conv_architecture=convnet_simple
815
+ encoder_conv_mlp_layers=[512]
816
+ use_rnn=True
817
+ rnn_size=512
818
+ rnn_type=gru
819
+ rnn_num_layers=1
820
+ decoder_mlp_layers=[]
821
+ nonlinearity=elu
822
+ policy_initialization=orthogonal
823
+ policy_init_gain=1.0
824
+ actor_critic_share_weights=True
825
+ adaptive_stddev=True
826
+ continuous_tanh_scale=0.0
827
+ initial_stddev=1.0
828
+ use_env_info_cache=False
829
+ env_gpu_actions=False
830
+ env_gpu_observations=True
831
+ env_frameskip=4
832
+ env_framestack=1
833
+ pixel_format=CHW
834
+ use_record_episode_statistics=False
835
+ with_wandb=False
836
+ wandb_user=None
837
+ wandb_project=sample_factory
838
+ wandb_group=None
839
+ wandb_job_type=SF
840
+ wandb_tags=[]
841
+ with_pbt=False
842
+ pbt_mix_policies_in_one_env=True
843
+ pbt_period_env_steps=5000000
844
+ pbt_start_mutation=20000000
845
+ pbt_replace_fraction=0.3
846
+ pbt_mutation_rate=0.15
847
+ pbt_replace_reward_gap=0.1
848
+ pbt_replace_reward_gap_absolute=1e-06
849
+ pbt_optimize_gamma=False
850
+ pbt_target_objective=true_objective
851
+ pbt_perturb_min=1.1
852
+ pbt_perturb_max=1.5
853
+ num_agents=-1
854
+ num_humans=0
855
+ num_bots=-1
856
+ start_bot_difficulty=None
857
+ timelimit=None
858
+ res_w=128
859
+ res_h=72
860
+ wide_aspect_ratio=False
861
+ eval_env_frameskip=1
862
+ fps=35
863
+ command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000
864
+ cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000}
865
+ git_hash=unknown
866
+ git_repo_name=not a git repository
867
+ [2023-05-24 20:40:20,636][2722668] Saving configuration to /home/mark/rl_course/unit8/train_dir/default_experiment/config.json...
868
+ [2023-05-24 20:40:20,638][2722668] Rollout worker 0 uses device cpu
869
+ [2023-05-24 20:40:20,638][2722668] Rollout worker 1 uses device cpu
870
+ [2023-05-24 20:40:20,640][2722668] Rollout worker 2 uses device cpu
871
+ [2023-05-24 20:40:20,641][2722668] Rollout worker 3 uses device cpu
872
+ [2023-05-24 20:40:20,642][2722668] Rollout worker 4 uses device cpu
873
+ [2023-05-24 20:40:20,643][2722668] Rollout worker 5 uses device cpu
874
+ [2023-05-24 20:40:20,644][2722668] Rollout worker 6 uses device cpu
875
+ [2023-05-24 20:40:20,645][2722668] Rollout worker 7 uses device cpu
876
+ [2023-05-24 20:40:20,676][2722668] Using GPUs [0] for process 0 (actually maps to GPUs [0])
877
+ [2023-05-24 20:40:20,677][2722668] InferenceWorker_p0-w0: min num requests: 2
878
+ [2023-05-24 20:40:20,706][2722668] Starting all processes...
879
+ [2023-05-24 20:40:20,707][2722668] Starting process learner_proc0
880
+ [2023-05-24 20:40:20,756][2722668] Starting all processes...
881
+ [2023-05-24 20:40:20,762][2722668] Starting process inference_proc0-0
882
+ [2023-05-24 20:40:20,762][2722668] Starting process rollout_proc0
883
+ [2023-05-24 20:40:20,762][2722668] Starting process rollout_proc1
884
+ [2023-05-24 20:40:20,763][2722668] Starting process rollout_proc2
885
+ [2023-05-24 20:40:20,763][2722668] Starting process rollout_proc3
886
+ [2023-05-24 20:40:20,763][2722668] Starting process rollout_proc4
887
+ [2023-05-24 20:40:20,764][2722668] Starting process rollout_proc5
888
+ [2023-05-24 20:40:20,764][2722668] Starting process rollout_proc6
889
+ [2023-05-24 20:40:20,765][2722668] Starting process rollout_proc7
890
+ [2023-05-24 20:40:22,441][2740685] Worker 1 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
891
+ [2023-05-24 20:40:22,497][2740681] Using GPUs [0] for process 0 (actually maps to GPUs [0])
892
+ [2023-05-24 20:40:22,497][2740681] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
893
+ [2023-05-24 20:40:22,515][2740681] Num visible devices: 1
894
+ [2023-05-24 20:40:22,523][2740682] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
895
+ [2023-05-24 20:40:22,526][2740692] Worker 6 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
896
+ [2023-05-24 20:40:22,527][2740668] Using GPUs [0] for process 0 (actually maps to GPUs [0])
897
+ [2023-05-24 20:40:22,527][2740668] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
898
+ [2023-05-24 20:40:22,529][2740690] Worker 5 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
899
+ [2023-05-24 20:40:22,529][2740687] Worker 4 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
900
+ [2023-05-24 20:40:22,544][2740668] Num visible devices: 1
901
+ [2023-05-24 20:40:22,545][2740686] Worker 2 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
902
+ [2023-05-24 20:40:22,586][2740691] Worker 7 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
903
+ [2023-05-24 20:40:22,589][2740668] Starting seed is not provided
904
+ [2023-05-24 20:40:22,589][2740668] Using GPUs [0] for process 0 (actually maps to GPUs [0])
905
+ [2023-05-24 20:40:22,589][2740668] Initializing actor-critic model on device cuda:0
906
+ [2023-05-24 20:40:22,589][2740668] RunningMeanStd input shape: (3, 72, 128)
907
+ [2023-05-24 20:40:22,590][2740668] RunningMeanStd input shape: (1,)
908
+ [2023-05-24 20:40:22,600][2740668] ConvEncoder: input_channels=3
909
+ [2023-05-24 20:40:22,627][2740684] Worker 3 uses CPU cores [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
910
+ [2023-05-24 20:40:22,722][2740668] Conv encoder output size: 512
911
+ [2023-05-24 20:40:22,722][2740668] Policy head output size: 512
912
+ [2023-05-24 20:40:22,747][2740668] Created Actor Critic model with architecture:
913
+ [2023-05-24 20:40:22,747][2740668] ActorCriticSharedWeights(
914
+ (obs_normalizer): ObservationNormalizer(
915
+ (running_mean_std): RunningMeanStdDictInPlace(
916
+ (running_mean_std): ModuleDict(
917
+ (obs): RunningMeanStdInPlace()
918
+ )
919
+ )
920
+ )
921
+ (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
922
+ (encoder): VizdoomEncoder(
923
+ (basic_encoder): ConvEncoder(
924
+ (enc): RecursiveScriptModule(
925
+ original_name=ConvEncoderImpl
926
+ (conv_head): RecursiveScriptModule(
927
+ original_name=Sequential
928
+ (0): RecursiveScriptModule(original_name=Conv2d)
929
+ (1): RecursiveScriptModule(original_name=ELU)
930
+ (2): RecursiveScriptModule(original_name=Conv2d)
931
+ (3): RecursiveScriptModule(original_name=ELU)
932
+ (4): RecursiveScriptModule(original_name=Conv2d)
933
+ (5): RecursiveScriptModule(original_name=ELU)
934
+ )
935
+ (mlp_layers): RecursiveScriptModule(
936
+ original_name=Sequential
937
+ (0): RecursiveScriptModule(original_name=Linear)
938
+ (1): RecursiveScriptModule(original_name=ELU)
939
+ )
940
+ )
941
+ )
942
+ )
943
+ (core): ModelCoreRNN(
944
+ (core): GRU(512, 512)
945
+ )
946
+ (decoder): MlpDecoder(
947
+ (mlp): Identity()
948
+ )
949
+ (critic_linear): Linear(in_features=512, out_features=1, bias=True)
950
+ (action_parameterization): ActionParameterizationDefault(
951
+ (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
952
+ )
953
+ )
954
+ [2023-05-24 20:40:25,214][2740668] Using optimizer <class 'torch.optim.adam.Adam'>
955
+ [2023-05-24 20:40:25,215][2740668] Loading state from checkpoint /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
956
+ [2023-05-24 20:40:25,237][2740668] Loading model from checkpoint
957
+ [2023-05-24 20:40:25,241][2740668] Loaded experiment state at self.train_step=978, self.env_steps=4005888
958
+ [2023-05-24 20:40:25,242][2740668] Initialized policy 0 weights for model version 978
959
+ [2023-05-24 20:40:25,244][2740668] LearnerWorker_p0 finished initialization!
960
+ [2023-05-24 20:40:25,244][2740668] Using GPUs [0] for process 0 (actually maps to GPUs [0])
961
+ [2023-05-24 20:40:25,355][2740681] RunningMeanStd input shape: (3, 72, 128)
962
+ [2023-05-24 20:40:25,356][2740681] RunningMeanStd input shape: (1,)
963
+ [2023-05-24 20:40:25,371][2740681] ConvEncoder: input_channels=3
964
+ [2023-05-24 20:40:25,509][2740681] Conv encoder output size: 512
965
+ [2023-05-24 20:40:25,509][2740681] Policy head output size: 512
966
+ [2023-05-24 20:40:27,915][2722668] Inference worker 0-0 is ready!
967
+ [2023-05-24 20:40:27,917][2722668] All inference workers are ready! Signal rollout workers to start!
968
+ [2023-05-24 20:40:27,960][2740685] Doom resolution: 160x120, resize resolution: (128, 72)
969
+ [2023-05-24 20:40:27,965][2740690] Doom resolution: 160x120, resize resolution: (128, 72)
970
+ [2023-05-24 20:40:27,967][2740682] Doom resolution: 160x120, resize resolution: (128, 72)
971
+ [2023-05-24 20:40:27,968][2740686] Doom resolution: 160x120, resize resolution: (128, 72)
972
+ [2023-05-24 20:40:27,968][2740687] Doom resolution: 160x120, resize resolution: (128, 72)
973
+ [2023-05-24 20:40:27,970][2740692] Doom resolution: 160x120, resize resolution: (128, 72)
974
+ [2023-05-24 20:40:28,012][2740691] Doom resolution: 160x120, resize resolution: (128, 72)
975
+ [2023-05-24 20:40:28,016][2740684] Doom resolution: 160x120, resize resolution: (128, 72)
976
+ [2023-05-24 20:40:28,553][2740685] Decorrelating experience for 0 frames...
977
+ [2023-05-24 20:40:28,556][2740686] Decorrelating experience for 0 frames...
978
+ [2023-05-24 20:40:28,557][2740682] Decorrelating experience for 0 frames...
979
+ [2023-05-24 20:40:28,559][2740692] Decorrelating experience for 0 frames...
980
+ [2023-05-24 20:40:28,560][2740687] Decorrelating experience for 0 frames...
981
+ [2023-05-24 20:40:28,563][2740690] Decorrelating experience for 0 frames...
982
+ [2023-05-24 20:40:28,854][2722668] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 4005888. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
983
+ [2023-05-24 20:40:28,871][2740686] Decorrelating experience for 32 frames...
984
+ [2023-05-24 20:40:28,876][2740692] Decorrelating experience for 32 frames...
985
+ [2023-05-24 20:40:28,882][2740690] Decorrelating experience for 32 frames...
986
+ [2023-05-24 20:40:28,884][2740684] Decorrelating experience for 0 frames...
987
+ [2023-05-24 20:40:28,910][2740691] Decorrelating experience for 0 frames...
988
+ [2023-05-24 20:40:29,200][2740684] Decorrelating experience for 32 frames...
989
+ [2023-05-24 20:40:29,227][2740682] Decorrelating experience for 32 frames...
990
+ [2023-05-24 20:40:29,228][2740686] Decorrelating experience for 64 frames...
991
+ [2023-05-24 20:40:29,229][2740691] Decorrelating experience for 32 frames...
992
+ [2023-05-24 20:40:29,521][2740685] Decorrelating experience for 32 frames...
993
+ [2023-05-24 20:40:29,543][2740692] Decorrelating experience for 64 frames...
994
+ [2023-05-24 20:40:29,556][2740684] Decorrelating experience for 64 frames...
995
+ [2023-05-24 20:40:29,585][2740687] Decorrelating experience for 32 frames...
996
+ [2023-05-24 20:40:29,600][2740691] Decorrelating experience for 64 frames...
997
+ [2023-05-24 20:40:29,841][2740682] Decorrelating experience for 64 frames...
998
+ [2023-05-24 20:40:29,897][2740686] Decorrelating experience for 96 frames...
999
+ [2023-05-24 20:40:29,918][2740684] Decorrelating experience for 96 frames...
1000
+ [2023-05-24 20:40:29,952][2740687] Decorrelating experience for 64 frames...
1001
+ [2023-05-24 20:40:29,966][2740691] Decorrelating experience for 96 frames...
1002
+ [2023-05-24 20:40:30,191][2740685] Decorrelating experience for 64 frames...
1003
+ [2023-05-24 20:40:30,233][2740682] Decorrelating experience for 96 frames...
1004
+ [2023-05-24 20:40:30,248][2740690] Decorrelating experience for 64 frames...
1005
+ [2023-05-24 20:40:30,312][2740687] Decorrelating experience for 96 frames...
1006
+ [2023-05-24 20:40:30,541][2740692] Decorrelating experience for 96 frames...
1007
+ [2023-05-24 20:40:30,615][2740690] Decorrelating experience for 96 frames...
1008
+ [2023-05-24 20:40:30,870][2740685] Decorrelating experience for 96 frames...
1009
+ [2023-05-24 20:40:31,319][2740668] Signal inference workers to stop experience collection...
1010
+ [2023-05-24 20:40:31,322][2740681] InferenceWorker_p0-w0: stopping experience collection
1011
+ [2023-05-24 20:40:32,820][2740668] Signal inference workers to resume experience collection...
1012
+ [2023-05-24 20:40:32,821][2740681] InferenceWorker_p0-w0: resuming experience collection
1013
+ [2023-05-24 20:40:33,854][2722668] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 4009984. Throughput: 0: 97.6. Samples: 488. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
1014
+ [2023-05-24 20:40:33,856][2722668] Avg episode reward: [(0, '4.377')]
1015
+ [2023-05-24 20:40:35,239][2740681] Updated weights for policy 0, policy_version 988 (0.0452)
1016
+ [2023-05-24 20:40:37,059][2740681] Updated weights for policy 0, policy_version 998 (0.0008)
1017
+ [2023-05-24 20:40:38,854][2722668] Fps is (10 sec: 11878.3, 60 sec: 11878.3, 300 sec: 11878.3). Total num frames: 4124672. Throughput: 0: 2054.2. Samples: 20542. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0)
1018
+ [2023-05-24 20:40:38,855][2722668] Avg episode reward: [(0, '24.933')]
1019
+ [2023-05-24 20:40:38,872][2740681] Updated weights for policy 0, policy_version 1008 (0.0009)
1020
+ [2023-05-24 20:40:40,669][2722668] Heartbeat connected on Batcher_0
1021
+ [2023-05-24 20:40:40,678][2740681] Updated weights for policy 0, policy_version 1018 (0.0009)
1022
+ [2023-05-24 20:40:40,678][2722668] Heartbeat connected on LearnerWorker_p0
1023
+ [2023-05-24 20:40:40,680][2722668] Heartbeat connected on InferenceWorker_p0-w0
1024
+ [2023-05-24 20:40:40,681][2722668] Heartbeat connected on RolloutWorker_w0
1025
+ [2023-05-24 20:40:40,687][2722668] Heartbeat connected on RolloutWorker_w1
1026
+ [2023-05-24 20:40:40,690][2722668] Heartbeat connected on RolloutWorker_w2
1027
+ [2023-05-24 20:40:40,693][2722668] Heartbeat connected on RolloutWorker_w3
1028
+ [2023-05-24 20:40:40,698][2722668] Heartbeat connected on RolloutWorker_w4
1029
+ [2023-05-24 20:40:40,699][2722668] Heartbeat connected on RolloutWorker_w5
1030
+ [2023-05-24 20:40:40,701][2722668] Heartbeat connected on RolloutWorker_w6
1031
+ [2023-05-24 20:40:40,706][2722668] Heartbeat connected on RolloutWorker_w7
1032
+ [2023-05-24 20:40:42,493][2740681] Updated weights for policy 0, policy_version 1028 (0.0009)
1033
+ [2023-05-24 20:40:43,854][2722668] Fps is (10 sec: 22937.9, 60 sec: 15564.8, 300 sec: 15564.8). Total num frames: 4239360. Throughput: 0: 3632.0. Samples: 54480. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1034
+ [2023-05-24 20:40:43,855][2722668] Avg episode reward: [(0, '23.081')]
1035
+ [2023-05-24 20:40:44,347][2740681] Updated weights for policy 0, policy_version 1038 (0.0009)
1036
+ [2023-05-24 20:40:46,219][2740681] Updated weights for policy 0, policy_version 1048 (0.0008)
1037
+ [2023-05-24 20:40:48,027][2740681] Updated weights for policy 0, policy_version 1058 (0.0009)
1038
+ [2023-05-24 20:40:48,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 17203.2, 300 sec: 17203.2). Total num frames: 4349952. Throughput: 0: 3548.5. Samples: 70970. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
1039
+ [2023-05-24 20:40:48,855][2722668] Avg episode reward: [(0, '22.824')]
1040
+ [2023-05-24 20:40:49,860][2740681] Updated weights for policy 0, policy_version 1068 (0.0009)
1041
+ [2023-05-24 20:40:51,644][2740681] Updated weights for policy 0, policy_version 1078 (0.0009)
1042
+ [2023-05-24 20:40:53,471][2740681] Updated weights for policy 0, policy_version 1088 (0.0008)
1043
+ [2023-05-24 20:40:53,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 18350.0, 300 sec: 18350.0). Total num frames: 4464640. Throughput: 0: 4191.5. Samples: 104788. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1044
+ [2023-05-24 20:40:53,855][2722668] Avg episode reward: [(0, '23.583')]
1045
+ [2023-05-24 20:40:55,305][2740681] Updated weights for policy 0, policy_version 1098 (0.0008)
1046
+ [2023-05-24 20:40:57,111][2740681] Updated weights for policy 0, policy_version 1108 (0.0008)
1047
+ [2023-05-24 20:40:58,854][2722668] Fps is (10 sec: 22528.2, 60 sec: 18978.2, 300 sec: 18978.2). Total num frames: 4575232. Throughput: 0: 4619.2. Samples: 138576. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1048
+ [2023-05-24 20:40:58,855][2722668] Avg episode reward: [(0, '23.463')]
1049
+ [2023-05-24 20:40:58,940][2740681] Updated weights for policy 0, policy_version 1118 (0.0009)
1050
+ [2023-05-24 20:41:00,782][2740681] Updated weights for policy 0, policy_version 1128 (0.0009)
1051
+ [2023-05-24 20:41:02,592][2740681] Updated weights for policy 0, policy_version 1138 (0.0008)
1052
+ [2023-05-24 20:41:03,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 19426.7, 300 sec: 19426.7). Total num frames: 4685824. Throughput: 0: 4439.7. Samples: 155390. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
1053
+ [2023-05-24 20:41:03,855][2722668] Avg episode reward: [(0, '23.344')]
1054
+ [2023-05-24 20:41:04,437][2740681] Updated weights for policy 0, policy_version 1148 (0.0008)
1055
+ [2023-05-24 20:41:06,265][2740681] Updated weights for policy 0, policy_version 1158 (0.0009)
1056
+ [2023-05-24 20:41:08,070][2740681] Updated weights for policy 0, policy_version 1168 (0.0009)
1057
+ [2023-05-24 20:41:08,854][2722668] Fps is (10 sec: 22527.7, 60 sec: 19865.6, 300 sec: 19865.6). Total num frames: 4800512. Throughput: 0: 4725.7. Samples: 189030. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
1058
+ [2023-05-24 20:41:08,855][2722668] Avg episode reward: [(0, '23.984')]
1059
+ [2023-05-24 20:41:09,863][2740681] Updated weights for policy 0, policy_version 1178 (0.0009)
1060
+ [2023-05-24 20:41:11,658][2740681] Updated weights for policy 0, policy_version 1188 (0.0008)
1061
+ [2023-05-24 20:41:13,513][2740681] Updated weights for policy 0, policy_version 1198 (0.0008)
1062
+ [2023-05-24 20:41:13,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 20115.9, 300 sec: 20115.9). Total num frames: 4911104. Throughput: 0: 4952.5. Samples: 222862. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1063
+ [2023-05-24 20:41:13,855][2722668] Avg episode reward: [(0, '28.266')]
1064
+ [2023-05-24 20:41:13,866][2740668] Saving new best policy, reward=28.266!
1065
+ [2023-05-24 20:41:15,358][2740681] Updated weights for policy 0, policy_version 1208 (0.0009)
1066
+ [2023-05-24 20:41:17,206][2740681] Updated weights for policy 0, policy_version 1218 (0.0009)
1067
+ [2023-05-24 20:41:18,854][2722668] Fps is (10 sec: 22528.3, 60 sec: 20398.1, 300 sec: 20398.1). Total num frames: 5025792. Throughput: 0: 5310.3. Samples: 239450. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1068
+ [2023-05-24 20:41:18,855][2722668] Avg episode reward: [(0, '24.734')]
1069
+ [2023-05-24 20:41:19,040][2740681] Updated weights for policy 0, policy_version 1228 (0.0009)
1070
+ [2023-05-24 20:41:20,866][2740681] Updated weights for policy 0, policy_version 1238 (0.0009)
1071
+ [2023-05-24 20:41:22,720][2740681] Updated weights for policy 0, policy_version 1248 (0.0009)
1072
+ [2023-05-24 20:41:23,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 20554.4, 300 sec: 20554.4). Total num frames: 5136384. Throughput: 0: 5610.5. Samples: 273016. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
1073
+ [2023-05-24 20:41:23,855][2722668] Avg episode reward: [(0, '28.659')]
1074
+ [2023-05-24 20:41:23,857][2740668] Saving new best policy, reward=28.659!
1075
+ [2023-05-24 20:41:24,558][2740681] Updated weights for policy 0, policy_version 1258 (0.0008)
1076
+ [2023-05-24 20:41:26,381][2740681] Updated weights for policy 0, policy_version 1268 (0.0009)
1077
+ [2023-05-24 20:41:28,188][2740681] Updated weights for policy 0, policy_version 1278 (0.0009)
1078
+ [2023-05-24 20:41:28,854][2722668] Fps is (10 sec: 22118.1, 60 sec: 20684.8, 300 sec: 20684.8). Total num frames: 5246976. Throughput: 0: 5600.2. Samples: 306490. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1079
+ [2023-05-24 20:41:28,855][2722668] Avg episode reward: [(0, '24.789')]
1080
+ [2023-05-24 20:41:30,008][2740681] Updated weights for policy 0, policy_version 1288 (0.0009)
1081
+ [2023-05-24 20:41:31,855][2740681] Updated weights for policy 0, policy_version 1298 (0.0008)
1082
+ [2023-05-24 20:41:33,670][2740681] Updated weights for policy 0, policy_version 1308 (0.0008)
1083
+ [2023-05-24 20:41:33,859][2722668] Fps is (10 sec: 22517.7, 60 sec: 22526.3, 300 sec: 20856.6). Total num frames: 5361664. Throughput: 0: 5606.0. Samples: 323266. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1084
+ [2023-05-24 20:41:33,861][2722668] Avg episode reward: [(0, '27.411')]
1085
+ [2023-05-24 20:41:35,483][2740681] Updated weights for policy 0, policy_version 1318 (0.0009)
1086
+ [2023-05-24 20:41:37,314][2740681] Updated weights for policy 0, policy_version 1328 (0.0009)
1087
+ [2023-05-24 20:41:38,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22459.7, 300 sec: 20948.1). Total num frames: 5472256. Throughput: 0: 5602.2. Samples: 356886. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
1088
+ [2023-05-24 20:41:38,855][2722668] Avg episode reward: [(0, '27.920')]
1089
+ [2023-05-24 20:41:39,173][2740681] Updated weights for policy 0, policy_version 1338 (0.0009)
1090
+ [2023-05-24 20:41:40,976][2740681] Updated weights for policy 0, policy_version 1348 (0.0008)
1091
+ [2023-05-24 20:41:42,799][2740681] Updated weights for policy 0, policy_version 1358 (0.0008)
1092
+ [2023-05-24 20:41:43,854][2722668] Fps is (10 sec: 22128.9, 60 sec: 22391.5, 300 sec: 21026.2). Total num frames: 5582848. Throughput: 0: 5599.6. Samples: 390556. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1093
+ [2023-05-24 20:41:43,855][2722668] Avg episode reward: [(0, '26.898')]
1094
+ [2023-05-24 20:41:44,619][2740681] Updated weights for policy 0, policy_version 1368 (0.0009)
1095
+ [2023-05-24 20:41:46,452][2740681] Updated weights for policy 0, policy_version 1378 (0.0009)
1096
+ [2023-05-24 20:41:48,289][2740681] Updated weights for policy 0, policy_version 1388 (0.0008)
1097
+ [2023-05-24 20:41:48,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22459.8, 300 sec: 21145.6). Total num frames: 5697536. Throughput: 0: 5600.0. Samples: 407388. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1098
+ [2023-05-24 20:41:48,855][2722668] Avg episode reward: [(0, '27.218')]
1099
+ [2023-05-24 20:41:50,124][2740681] Updated weights for policy 0, policy_version 1398 (0.0008)
1100
+ [2023-05-24 20:41:51,976][2740681] Updated weights for policy 0, policy_version 1408 (0.0009)
1101
+ [2023-05-24 20:41:53,854][2722668] Fps is (10 sec: 22118.1, 60 sec: 22323.2, 300 sec: 21154.6). Total num frames: 5804032. Throughput: 0: 5593.0. Samples: 440716. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1102
+ [2023-05-24 20:41:53,855][2722668] Avg episode reward: [(0, '27.321')]
1103
+ [2023-05-24 20:41:53,860][2740681] Updated weights for policy 0, policy_version 1418 (0.0008)
1104
+ [2023-05-24 20:41:55,705][2740681] Updated weights for policy 0, policy_version 1428 (0.0008)
1105
+ [2023-05-24 20:41:57,556][2740681] Updated weights for policy 0, policy_version 1438 (0.0009)
1106
+ [2023-05-24 20:41:58,854][2722668] Fps is (10 sec: 21708.7, 60 sec: 22323.2, 300 sec: 21208.2). Total num frames: 5914624. Throughput: 0: 5573.1. Samples: 473652. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
1107
+ [2023-05-24 20:41:58,855][2722668] Avg episode reward: [(0, '21.784')]
1108
+ [2023-05-24 20:41:59,430][2740681] Updated weights for policy 0, policy_version 1448 (0.0009)
1109
+ [2023-05-24 20:42:01,259][2740681] Updated weights for policy 0, policy_version 1458 (0.0008)
1110
+ [2023-05-24 20:42:03,088][2740681] Updated weights for policy 0, policy_version 1468 (0.0008)
1111
+ [2023-05-24 20:42:03,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22391.5, 300 sec: 21299.2). Total num frames: 6029312. Throughput: 0: 5574.8. Samples: 490318. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1112
+ [2023-05-24 20:42:03,855][2722668] Avg episode reward: [(0, '27.011')]
1113
+ [2023-05-24 20:42:04,905][2740681] Updated weights for policy 0, policy_version 1478 (0.0009)
1114
+ [2023-05-24 20:42:06,738][2740681] Updated weights for policy 0, policy_version 1488 (0.0009)
1115
+ [2023-05-24 20:42:08,562][2740681] Updated weights for policy 0, policy_version 1498 (0.0009)
1116
+ [2023-05-24 20:42:08,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22323.2, 300 sec: 21340.2). Total num frames: 6139904. Throughput: 0: 5577.9. Samples: 524020. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1117
+ [2023-05-24 20:42:08,855][2722668] Avg episode reward: [(0, '26.691')]
1118
+ [2023-05-24 20:42:10,377][2740681] Updated weights for policy 0, policy_version 1508 (0.0009)
1119
+ [2023-05-24 20:42:12,210][2740681] Updated weights for policy 0, policy_version 1518 (0.0008)
1120
+ [2023-05-24 20:42:13,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22323.2, 300 sec: 21377.2). Total num frames: 6250496. Throughput: 0: 5582.2. Samples: 557688. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1121
+ [2023-05-24 20:42:13,856][2722668] Avg episode reward: [(0, '28.372')]
1122
+ [2023-05-24 20:42:14,046][2740681] Updated weights for policy 0, policy_version 1528 (0.0010)
1123
+ [2023-05-24 20:42:15,895][2740681] Updated weights for policy 0, policy_version 1538 (0.0009)
1124
+ [2023-05-24 20:42:17,697][2740681] Updated weights for policy 0, policy_version 1548 (0.0008)
1125
+ [2023-05-24 20:42:18,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22323.1, 300 sec: 21448.1). Total num frames: 6365184. Throughput: 0: 5580.9. Samples: 574382. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1126
+ [2023-05-24 20:42:18,855][2722668] Avg episode reward: [(0, '28.683')]
1127
+ [2023-05-24 20:42:18,861][2740668] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001554_6365184.pth...
1128
+ [2023-05-24 20:42:18,904][2740668] Removing /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000569_2330624.pth
1129
+ [2023-05-24 20:42:18,910][2740668] Saving new best policy, reward=28.683!
1130
+ [2023-05-24 20:42:19,545][2740681] Updated weights for policy 0, policy_version 1558 (0.0009)
1131
+ [2023-05-24 20:42:21,365][2740681] Updated weights for policy 0, policy_version 1568 (0.0008)
1132
+ [2023-05-24 20:42:23,174][2740681] Updated weights for policy 0, policy_version 1578 (0.0009)
1133
+ [2023-05-24 20:42:23,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22323.2, 300 sec: 21477.3). Total num frames: 6475776. Throughput: 0: 5583.6. Samples: 608146. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1134
+ [2023-05-24 20:42:23,855][2722668] Avg episode reward: [(0, '28.246')]
1135
+ [2023-05-24 20:42:24,992][2740681] Updated weights for policy 0, policy_version 1588 (0.0009)
1136
+ [2023-05-24 20:42:26,845][2740681] Updated weights for policy 0, policy_version 1598 (0.0009)
1137
+ [2023-05-24 20:42:28,717][2740681] Updated weights for policy 0, policy_version 1608 (0.0009)
1138
+ [2023-05-24 20:42:28,854][2722668] Fps is (10 sec: 22118.3, 60 sec: 22323.2, 300 sec: 21504.0). Total num frames: 6586368. Throughput: 0: 5578.0. Samples: 641568. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1139
+ [2023-05-24 20:42:28,855][2722668] Avg episode reward: [(0, '30.804')]
1140
+ [2023-05-24 20:42:28,860][2740668] Saving new best policy, reward=30.804!
1141
+ [2023-05-24 20:42:30,573][2740681] Updated weights for policy 0, policy_version 1618 (0.0008)
1142
+ [2023-05-24 20:42:32,405][2740681] Updated weights for policy 0, policy_version 1628 (0.0008)
1143
+ [2023-05-24 20:42:33,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22256.7, 300 sec: 21528.6). Total num frames: 6696960. Throughput: 0: 5570.7. Samples: 658070. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1144
+ [2023-05-24 20:42:33,855][2722668] Avg episode reward: [(0, '24.207')]
1145
+ [2023-05-24 20:42:34,261][2740681] Updated weights for policy 0, policy_version 1638 (0.0008)
1146
+ [2023-05-24 20:42:36,166][2740681] Updated weights for policy 0, policy_version 1648 (0.0009)
1147
+ [2023-05-24 20:42:38,015][2740681] Updated weights for policy 0, policy_version 1658 (0.0009)
1148
+ [2023-05-24 20:42:38,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22254.9, 300 sec: 21551.3). Total num frames: 6807552. Throughput: 0: 5559.1. Samples: 690874. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
1149
+ [2023-05-24 20:42:38,855][2722668] Avg episode reward: [(0, '28.196')]
1150
+ [2023-05-24 20:42:39,844][2740681] Updated weights for policy 0, policy_version 1668 (0.0008)
1151
+ [2023-05-24 20:42:41,662][2740681] Updated weights for policy 0, policy_version 1678 (0.0008)
1152
+ [2023-05-24 20:42:43,455][2740681] Updated weights for policy 0, policy_version 1688 (0.0008)
1153
+ [2023-05-24 20:42:43,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22323.2, 300 sec: 21602.6). Total num frames: 6922240. Throughput: 0: 5578.1. Samples: 724666. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1154
+ [2023-05-24 20:42:43,855][2722668] Avg episode reward: [(0, '31.920')]
1155
+ [2023-05-24 20:42:43,856][2740668] Saving new best policy, reward=31.920!
1156
+ [2023-05-24 20:42:45,312][2740681] Updated weights for policy 0, policy_version 1698 (0.0009)
1157
+ [2023-05-24 20:42:47,157][2740681] Updated weights for policy 0, policy_version 1708 (0.0008)
1158
+ [2023-05-24 20:42:48,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22254.9, 300 sec: 21621.0). Total num frames: 7032832. Throughput: 0: 5577.6. Samples: 741310. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1159
+ [2023-05-24 20:42:48,855][2722668] Avg episode reward: [(0, '27.695')]
1160
+ [2023-05-24 20:42:48,995][2740681] Updated weights for policy 0, policy_version 1718 (0.0008)
1161
+ [2023-05-24 20:42:50,839][2740681] Updated weights for policy 0, policy_version 1728 (0.0008)
1162
+ [2023-05-24 20:42:52,669][2740681] Updated weights for policy 0, policy_version 1738 (0.0008)
1163
+ [2023-05-24 20:42:53,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22323.2, 300 sec: 21638.2). Total num frames: 7143424. Throughput: 0: 5573.2. Samples: 774812. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1164
+ [2023-05-24 20:42:53,855][2722668] Avg episode reward: [(0, '27.728')]
1165
+ [2023-05-24 20:42:54,522][2740681] Updated weights for policy 0, policy_version 1748 (0.0009)
1166
+ [2023-05-24 20:42:56,333][2740681] Updated weights for policy 0, policy_version 1758 (0.0008)
1167
+ [2023-05-24 20:42:58,208][2740681] Updated weights for policy 0, policy_version 1768 (0.0009)
1168
+ [2023-05-24 20:42:58,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22323.2, 300 sec: 21654.2). Total num frames: 7254016. Throughput: 0: 5569.2. Samples: 808304. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1169
+ [2023-05-24 20:42:58,855][2722668] Avg episode reward: [(0, '29.743')]
1170
+ [2023-05-24 20:43:00,033][2740681] Updated weights for policy 0, policy_version 1778 (0.0009)
1171
+ [2023-05-24 20:43:01,872][2740681] Updated weights for policy 0, policy_version 1788 (0.0008)
1172
+ [2023-05-24 20:43:03,729][2740681] Updated weights for policy 0, policy_version 1798 (0.0008)
1173
+ [2023-05-24 20:43:03,854][2722668] Fps is (10 sec: 22118.3, 60 sec: 22254.9, 300 sec: 21669.2). Total num frames: 7364608. Throughput: 0: 5567.1. Samples: 824900. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1174
+ [2023-05-24 20:43:03,855][2722668] Avg episode reward: [(0, '28.198')]
1175
+ [2023-05-24 20:43:05,562][2740681] Updated weights for policy 0, policy_version 1808 (0.0008)
1176
+ [2023-05-24 20:43:07,388][2740681] Updated weights for policy 0, policy_version 1818 (0.0009)
1177
+ [2023-05-24 20:43:08,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22323.2, 300 sec: 21708.8). Total num frames: 7479296. Throughput: 0: 5561.0. Samples: 858390. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1178
+ [2023-05-24 20:43:08,855][2722668] Avg episode reward: [(0, '29.986')]
1179
+ [2023-05-24 20:43:09,179][2740681] Updated weights for policy 0, policy_version 1828 (0.0009)
1180
+ [2023-05-24 20:43:10,969][2740681] Updated weights for policy 0, policy_version 1838 (0.0008)
1181
+ [2023-05-24 20:43:12,795][2740681] Updated weights for policy 0, policy_version 1848 (0.0008)
1182
+ [2023-05-24 20:43:13,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22323.2, 300 sec: 21721.2). Total num frames: 7589888. Throughput: 0: 5572.1. Samples: 892314. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1183
+ [2023-05-24 20:43:13,855][2722668] Avg episode reward: [(0, '26.336')]
1184
+ [2023-05-24 20:43:14,619][2740681] Updated weights for policy 0, policy_version 1858 (0.0008)
1185
+ [2023-05-24 20:43:16,431][2740681] Updated weights for policy 0, policy_version 1868 (0.0008)
1186
+ [2023-05-24 20:43:18,272][2740681] Updated weights for policy 0, policy_version 1878 (0.0009)
1187
+ [2023-05-24 20:43:18,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22323.2, 300 sec: 21757.0). Total num frames: 7704576. Throughput: 0: 5580.9. Samples: 909212. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1188
+ [2023-05-24 20:43:18,855][2722668] Avg episode reward: [(0, '27.271')]
1189
+ [2023-05-24 20:43:20,078][2740681] Updated weights for policy 0, policy_version 1888 (0.0008)
1190
+ [2023-05-24 20:43:21,909][2740681] Updated weights for policy 0, policy_version 1898 (0.0008)
1191
+ [2023-05-24 20:43:23,753][2740681] Updated weights for policy 0, policy_version 1908 (0.0009)
1192
+ [2023-05-24 20:43:23,854][2722668] Fps is (10 sec: 22528.3, 60 sec: 22323.3, 300 sec: 21767.3). Total num frames: 7815168. Throughput: 0: 5600.1. Samples: 942878. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1193
+ [2023-05-24 20:43:23,855][2722668] Avg episode reward: [(0, '29.434')]
1194
+ [2023-05-24 20:43:25,586][2740681] Updated weights for policy 0, policy_version 1918 (0.0009)
1195
+ [2023-05-24 20:43:27,395][2740681] Updated weights for policy 0, policy_version 1928 (0.0009)
1196
+ [2023-05-24 20:43:28,854][2722668] Fps is (10 sec: 22528.2, 60 sec: 22391.5, 300 sec: 21799.8). Total num frames: 7929856. Throughput: 0: 5597.6. Samples: 976560. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1197
+ [2023-05-24 20:43:28,855][2722668] Avg episode reward: [(0, '29.442')]
1198
+ [2023-05-24 20:43:29,195][2740681] Updated weights for policy 0, policy_version 1938 (0.0008)
1199
+ [2023-05-24 20:43:31,023][2740681] Updated weights for policy 0, policy_version 1948 (0.0009)
1200
+ [2023-05-24 20:43:32,827][2740681] Updated weights for policy 0, policy_version 1958 (0.0009)
1201
+ [2023-05-24 20:43:33,854][2722668] Fps is (10 sec: 22527.6, 60 sec: 22391.5, 300 sec: 21808.4). Total num frames: 8040448. Throughput: 0: 5604.0. Samples: 993490. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1202
+ [2023-05-24 20:43:33,855][2722668] Avg episode reward: [(0, '27.952')]
1203
+ [2023-05-24 20:43:34,670][2740681] Updated weights for policy 0, policy_version 1968 (0.0008)
1204
+ [2023-05-24 20:43:36,467][2740681] Updated weights for policy 0, policy_version 1978 (0.0008)
1205
+ [2023-05-24 20:43:38,244][2740681] Updated weights for policy 0, policy_version 1988 (0.0008)
1206
+ [2023-05-24 20:43:38,854][2722668] Fps is (10 sec: 22527.8, 60 sec: 22459.7, 300 sec: 21838.1). Total num frames: 8155136. Throughput: 0: 5611.8. Samples: 1027342. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1207
+ [2023-05-24 20:43:38,855][2722668] Avg episode reward: [(0, '28.687')]
1208
+ [2023-05-24 20:43:40,098][2740681] Updated weights for policy 0, policy_version 1998 (0.0010)
1209
+ [2023-05-24 20:43:41,961][2740681] Updated weights for policy 0, policy_version 2008 (0.0009)
1210
+ [2023-05-24 20:43:43,806][2740681] Updated weights for policy 0, policy_version 2018 (0.0009)
1211
+ [2023-05-24 20:43:43,854][2722668] Fps is (10 sec: 22528.3, 60 sec: 22391.5, 300 sec: 21845.3). Total num frames: 8265728. Throughput: 0: 5605.6. Samples: 1060556. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
1212
+ [2023-05-24 20:43:43,855][2722668] Avg episode reward: [(0, '26.381')]
1213
+ [2023-05-24 20:43:45,628][2740681] Updated weights for policy 0, policy_version 2028 (0.0008)
1214
+ [2023-05-24 20:43:47,479][2740681] Updated weights for policy 0, policy_version 2038 (0.0010)
1215
+ [2023-05-24 20:43:48,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22391.5, 300 sec: 21852.2). Total num frames: 8376320. Throughput: 0: 5609.4. Samples: 1077324. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
1216
+ [2023-05-24 20:43:48,855][2722668] Avg episode reward: [(0, '28.460')]
1217
+ [2023-05-24 20:43:49,334][2740681] Updated weights for policy 0, policy_version 2048 (0.0008)
1218
+ [2023-05-24 20:43:51,158][2740681] Updated weights for policy 0, policy_version 2058 (0.0009)
1219
+ [2023-05-24 20:43:52,989][2740681] Updated weights for policy 0, policy_version 2068 (0.0008)
1220
+ [2023-05-24 20:43:53,854][2722668] Fps is (10 sec: 22118.2, 60 sec: 22391.5, 300 sec: 21858.7). Total num frames: 8486912. Throughput: 0: 5608.4. Samples: 1110768. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
1221
+ [2023-05-24 20:43:53,855][2722668] Avg episode reward: [(0, '26.161')]
1222
+ [2023-05-24 20:43:54,818][2740681] Updated weights for policy 0, policy_version 2078 (0.0008)
1223
+ [2023-05-24 20:43:56,656][2740681] Updated weights for policy 0, policy_version 2088 (0.0009)
1224
+ [2023-05-24 20:43:58,457][2740681] Updated weights for policy 0, policy_version 2098 (0.0009)
1225
+ [2023-05-24 20:43:58,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22459.7, 300 sec: 21884.3). Total num frames: 8601600. Throughput: 0: 5601.3. Samples: 1144374. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
1226
+ [2023-05-24 20:43:58,855][2722668] Avg episode reward: [(0, '26.754')]
1227
+ [2023-05-24 20:44:00,317][2740681] Updated weights for policy 0, policy_version 2108 (0.0008)
1228
+ [2023-05-24 20:44:02,200][2740681] Updated weights for policy 0, policy_version 2118 (0.0008)
1229
+ [2023-05-24 20:44:03,854][2722668] Fps is (10 sec: 22118.2, 60 sec: 22391.5, 300 sec: 21870.7). Total num frames: 8708096. Throughput: 0: 5596.8. Samples: 1161066. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1230
+ [2023-05-24 20:44:03,855][2722668] Avg episode reward: [(0, '28.003')]
1231
+ [2023-05-24 20:44:04,042][2740681] Updated weights for policy 0, policy_version 2128 (0.0009)
1232
+ [2023-05-24 20:44:05,869][2740681] Updated weights for policy 0, policy_version 2138 (0.0008)
1233
+ [2023-05-24 20:44:07,696][2740681] Updated weights for policy 0, policy_version 2148 (0.0008)
1234
+ [2023-05-24 20:44:08,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22391.5, 300 sec: 21895.0). Total num frames: 8822784. Throughput: 0: 5596.4. Samples: 1194718. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
1235
+ [2023-05-24 20:44:08,855][2722668] Avg episode reward: [(0, '28.981')]
1236
+ [2023-05-24 20:44:09,544][2740681] Updated weights for policy 0, policy_version 2158 (0.0008)
1237
+ [2023-05-24 20:44:11,378][2740681] Updated weights for policy 0, policy_version 2168 (0.0008)
1238
+ [2023-05-24 20:44:13,204][2740681] Updated weights for policy 0, policy_version 2178 (0.0009)
1239
+ [2023-05-24 20:44:13,854][2722668] Fps is (10 sec: 22528.2, 60 sec: 22391.5, 300 sec: 21899.9). Total num frames: 8933376. Throughput: 0: 5586.7. Samples: 1227960. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1240
+ [2023-05-24 20:44:13,855][2722668] Avg episode reward: [(0, '31.399')]
1241
+ [2023-05-24 20:44:15,008][2740681] Updated weights for policy 0, policy_version 2188 (0.0008)
1242
+ [2023-05-24 20:44:16,831][2740681] Updated weights for policy 0, policy_version 2198 (0.0008)
1243
+ [2023-05-24 20:44:18,630][2740681] Updated weights for policy 0, policy_version 2208 (0.0008)
1244
+ [2023-05-24 20:44:18,854][2722668] Fps is (10 sec: 22528.0, 60 sec: 22391.5, 300 sec: 21922.5). Total num frames: 9048064. Throughput: 0: 5588.3. Samples: 1244962. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
1245
+ [2023-05-24 20:44:18,855][2722668] Avg episode reward: [(0, '29.507')]
1246
+ [2023-05-24 20:44:18,860][2740668] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002209_9048064.pth...
1247
+ [2023-05-24 20:44:18,903][2740668] Removing /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth
1248
+ [2023-05-24 20:44:20,450][2740681] Updated weights for policy 0, policy_version 2218 (0.0008)
1249
+ [2023-05-24 20:44:22,295][2740681] Updated weights for policy 0, policy_version 2228 (0.0009)
1250
+ [2023-05-24 20:44:23,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22391.4, 300 sec: 21926.7). Total num frames: 9158656. Throughput: 0: 5588.7. Samples: 1278834. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0)
1251
+ [2023-05-24 20:44:23,855][2722668] Avg episode reward: [(0, '32.078')]
1252
+ [2023-05-24 20:44:23,857][2740668] Saving new best policy, reward=32.078!
1253
+ [2023-05-24 20:44:24,099][2740681] Updated weights for policy 0, policy_version 2238 (0.0009)
1254
+ [2023-05-24 20:44:25,919][2740681] Updated weights for policy 0, policy_version 2248 (0.0009)
1255
+ [2023-05-24 20:44:27,741][2740681] Updated weights for policy 0, policy_version 2258 (0.0008)
1256
+ [2023-05-24 20:44:28,854][2722668] Fps is (10 sec: 22527.9, 60 sec: 22391.5, 300 sec: 21947.7). Total num frames: 9273344. Throughput: 0: 5597.3. Samples: 1312434. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
1257
+ [2023-05-24 20:44:28,855][2722668] Avg episode reward: [(0, '27.981')]
1258
+ [2023-05-24 20:44:29,555][2740681] Updated weights for policy 0, policy_version 2268 (0.0009)
1259
+ [2023-05-24 20:44:31,380][2740681] Updated weights for policy 0, policy_version 2278 (0.0009)
1260
+ [2023-05-24 20:44:33,224][2740681] Updated weights for policy 0, policy_version 2288 (0.0010)
1261
+ [2023-05-24 20:44:33,854][2722668] Fps is (10 sec: 22528.1, 60 sec: 22391.5, 300 sec: 21951.2). Total num frames: 9383936. Throughput: 0: 5598.3. Samples: 1329246. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
1262
+ [2023-05-24 20:44:33,855][2722668] Avg episode reward: [(0, '29.901')]
1263
+ [2023-05-24 20:44:35,056][2740681] Updated weights for policy 0, policy_version 2298 (0.0010)
1264
+ [2023-05-24 20:44:36,882][2740681] Updated weights for policy 0, policy_version 2308 (0.0009)
1265
+ [2023-05-24 20:44:38,786][2740681] Updated weights for policy 0, policy_version 2318 (0.0009)
1266
+ [2023-05-24 20:44:38,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22323.2, 300 sec: 21954.6). Total num frames: 9494528. Throughput: 0: 5599.2. Samples: 1362734. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
1267
+ [2023-05-24 20:44:38,855][2722668] Avg episode reward: [(0, '30.109')]
1268
+ [2023-05-24 20:44:40,619][2740681] Updated weights for policy 0, policy_version 2328 (0.0009)
1269
+ [2023-05-24 20:44:42,459][2740681] Updated weights for policy 0, policy_version 2338 (0.0009)
1270
+ [2023-05-24 20:44:43,854][2722668] Fps is (10 sec: 22118.4, 60 sec: 22323.1, 300 sec: 21957.8). Total num frames: 9605120. Throughput: 0: 5586.2. Samples: 1395754. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
1271
+ [2023-05-24 20:44:43,855][2722668] Avg episode reward: [(0, '30.028')]
1272
+ [2023-05-24 20:44:44,395][2740681] Updated weights for policy 0, policy_version 2348 (0.0008)
1273
+ [2023-05-24 20:44:46,339][2740681] Updated weights for policy 0, policy_version 2358 (0.0009)
1274
+ [2023-05-24 20:44:48,184][2740681] Updated weights for policy 0, policy_version 2368 (0.0009)
1275
+ [2023-05-24 20:44:48,854][2722668] Fps is (10 sec: 21708.7, 60 sec: 22254.9, 300 sec: 21945.1). Total num frames: 9711616. Throughput: 0: 5571.5. Samples: 1411784. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0)
1276
+ [2023-05-24 20:44:48,856][2722668] Avg episode reward: [(0, '29.331')]
1277
+ [2023-05-24 20:44:50,046][2740681] Updated weights for policy 0, policy_version 2378 (0.0009)
1278
+ [2023-05-24 20:44:51,883][2740681] Updated weights for policy 0, policy_version 2388 (0.0009)
1279
+ [2023-05-24 20:44:53,680][2740681] Updated weights for policy 0, policy_version 2398 (0.0009)
1280
+ [2023-05-24 20:44:53,854][2722668] Fps is (10 sec: 21708.9, 60 sec: 22254.9, 300 sec: 21948.4). Total num frames: 9822208. Throughput: 0: 5557.7. Samples: 1444814. Policy #0 lag: (min: 0.0, avg: 0.8, max: 1.0)
1281
+ [2023-05-24 20:44:53,855][2722668] Avg episode reward: [(0, '27.804')]
1282
+ [2023-05-24 20:44:55,523][2740681] Updated weights for policy 0, policy_version 2408 (0.0008)
1283
+ [2023-05-24 20:44:57,342][2740681] Updated weights for policy 0, policy_version 2418 (0.0008)
1284
+ [2023-05-24 20:44:58,854][2722668] Fps is (10 sec: 22118.5, 60 sec: 22186.7, 300 sec: 21951.5). Total num frames: 9932800. Throughput: 0: 5563.8. Samples: 1478332. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
1285
+ [2023-05-24 20:44:58,855][2722668] Avg episode reward: [(0, '29.302')]
1286
+ [2023-05-24 20:44:59,233][2740681] Updated weights for policy 0, policy_version 2428 (0.0010)
1287
+ [2023-05-24 20:45:01,067][2740681] Updated weights for policy 0, policy_version 2438 (0.0008)
1288
+ [2023-05-24 20:45:01,991][2740668] Stopping Batcher_0...
1289
+ [2023-05-24 20:45:01,991][2740668] Loop batcher_evt_loop terminating...
1290
+ [2023-05-24 20:45:01,991][2740668] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002443_10006528.pth...
1291
+ [2023-05-24 20:45:01,998][2722668] Component Batcher_0 stopped!
1292
+ [2023-05-24 20:45:02,003][2740684] Stopping RolloutWorker_w3...
1293
+ [2023-05-24 20:45:02,004][2740684] Loop rollout_proc3_evt_loop terminating...
1294
+ [2023-05-24 20:45:02,003][2722668] Component RolloutWorker_w3 stopped!
1295
+ [2023-05-24 20:45:02,004][2740691] Stopping RolloutWorker_w7...
1296
+ [2023-05-24 20:45:02,004][2740686] Stopping RolloutWorker_w2...
1297
+ [2023-05-24 20:45:02,005][2740691] Loop rollout_proc7_evt_loop terminating...
1298
+ [2023-05-24 20:45:02,004][2740685] Stopping RolloutWorker_w1...
1299
+ [2023-05-24 20:45:02,004][2740692] Stopping RolloutWorker_w6...
1300
+ [2023-05-24 20:45:02,005][2740682] Stopping RolloutWorker_w0...
1301
+ [2023-05-24 20:45:02,005][2740687] Stopping RolloutWorker_w4...
1302
+ [2023-05-24 20:45:02,005][2740686] Loop rollout_proc2_evt_loop terminating...
1303
+ [2023-05-24 20:45:02,005][2740685] Loop rollout_proc1_evt_loop terminating...
1304
+ [2023-05-24 20:45:02,005][2740692] Loop rollout_proc6_evt_loop terminating...
1305
+ [2023-05-24 20:45:02,005][2740690] Stopping RolloutWorker_w5...
1306
+ [2023-05-24 20:45:02,005][2740687] Loop rollout_proc4_evt_loop terminating...
1307
+ [2023-05-24 20:45:02,005][2740682] Loop rollout_proc0_evt_loop terminating...
1308
+ [2023-05-24 20:45:02,005][2740690] Loop rollout_proc5_evt_loop terminating...
1309
+ [2023-05-24 20:45:02,005][2722668] Component RolloutWorker_w7 stopped!
1310
+ [2023-05-24 20:45:02,006][2740681] Weights refcount: 2 0
1311
+ [2023-05-24 20:45:02,006][2722668] Component RolloutWorker_w2 stopped!
1312
+ [2023-05-24 20:45:02,007][2740681] Stopping InferenceWorker_p0-w0...
1313
+ [2023-05-24 20:45:02,008][2740681] Loop inference_proc0-0_evt_loop terminating...
1314
+ [2023-05-24 20:45:02,007][2722668] Component RolloutWorker_w6 stopped!
1315
+ [2023-05-24 20:45:02,008][2722668] Component RolloutWorker_w1 stopped!
1316
+ [2023-05-24 20:45:02,009][2722668] Component RolloutWorker_w0 stopped!
1317
+ [2023-05-24 20:45:02,010][2722668] Component RolloutWorker_w4 stopped!
1318
+ [2023-05-24 20:45:02,011][2722668] Component RolloutWorker_w5 stopped!
1319
+ [2023-05-24 20:45:02,012][2722668] Component InferenceWorker_p0-w0 stopped!
1320
+ [2023-05-24 20:45:02,036][2740668] Removing /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001554_6365184.pth
1321
+ [2023-05-24 20:45:02,042][2740668] Saving /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002443_10006528.pth...
1322
+ [2023-05-24 20:45:02,095][2740668] Stopping LearnerWorker_p0...
1323
+ [2023-05-24 20:45:02,096][2740668] Loop learner_proc0_evt_loop terminating...
1324
+ [2023-05-24 20:45:02,095][2722668] Component LearnerWorker_p0 stopped!
1325
+ [2023-05-24 20:45:02,097][2722668] Waiting for process learner_proc0 to stop...
1326
+ [2023-05-24 20:45:02,894][2722668] Waiting for process inference_proc0-0 to join...
1327
+ [2023-05-24 20:45:02,896][2722668] Waiting for process rollout_proc0 to join...
1328
+ [2023-05-24 20:45:02,898][2722668] Waiting for process rollout_proc1 to join...
1329
+ [2023-05-24 20:45:02,899][2722668] Waiting for process rollout_proc2 to join...
1330
+ [2023-05-24 20:45:02,901][2722668] Waiting for process rollout_proc3 to join...
1331
+ [2023-05-24 20:45:02,902][2722668] Waiting for process rollout_proc4 to join...
1332
+ [2023-05-24 20:45:02,903][2722668] Waiting for process rollout_proc5 to join...
1333
+ [2023-05-24 20:45:02,904][2722668] Waiting for process rollout_proc6 to join...
1334
+ [2023-05-24 20:45:02,904][2722668] Waiting for process rollout_proc7 to join...
1335
+ [2023-05-24 20:45:02,905][2722668] Batcher 0 profile tree view:
1336
+ batching: 13.3147, releasing_batches: 0.0342
1337
+ [2023-05-24 20:45:02,906][2722668] InferenceWorker_p0-w0 profile tree view:
1338
+ wait_policy: 0.0001
1339
+ wait_policy_total: 6.5332
1340
+ update_model: 4.1118
1341
+ weight_update: 0.0009
1342
+ one_step: 0.0019
1343
+ handle_policy_step: 246.0819
1344
+ deserialize: 9.9033, stack: 1.4958, obs_to_device_normalize: 61.1206, forward: 106.2121, send_messages: 16.6256
1345
+ prepare_outputs: 39.1913
1346
+ to_cpu: 25.6796
1347
+ [2023-05-24 20:45:02,907][2722668] Learner 0 profile tree view:
1348
+ misc: 0.0066, prepare_batch: 11.8106
1349
+ train: 39.8102
1350
+ epoch_init: 0.0084, minibatch_init: 0.0091, losses_postprocess: 0.3513, kl_divergence: 0.3257, after_optimizer: 0.5826
1351
+ calculate_losses: 11.6125
1352
+ losses_init: 0.0055, forward_head: 1.1278, bptt_initial: 6.9832, tail: 0.6039, advantages_returns: 0.1700, losses: 1.2730
1353
+ bptt: 1.2362
1354
+ bptt_forward_core: 1.1854
1355
+ update: 26.4486
1356
+ clip: 1.7273
1357
+ [2023-05-24 20:45:02,908][2722668] RolloutWorker_w0 profile tree view:
1358
+ wait_for_trajectories: 0.2403, enqueue_policy_requests: 11.0413, env_step: 176.9094, overhead: 13.4279, complete_rollouts: 0.3495
1359
+ save_policy_outputs: 13.2120
1360
+ split_output_tensors: 6.4741
1361
+ [2023-05-24 20:45:02,909][2722668] RolloutWorker_w7 profile tree view:
1362
+ wait_for_trajectories: 0.2375, enqueue_policy_requests: 11.0937, env_step: 177.0302, overhead: 13.4463, complete_rollouts: 0.3416
1363
+ save_policy_outputs: 13.1739
1364
+ split_output_tensors: 6.4360
1365
+ [2023-05-24 20:45:02,910][2722668] Loop Runner_EvtLoop terminating...
1366
+ [2023-05-24 20:45:02,911][2722668] Runner profile tree view:
1367
+ main_loop: 282.2049
1368
+ [2023-05-24 20:45:02,912][2722668] Collected {0: 10006528}, FPS: 21263.4
1369
+ [2023-05-24 20:45:02,990][2722668] Loading existing experiment configuration from /home/mark/rl_course/unit8/train_dir/default_experiment/config.json
1370
+ [2023-05-24 20:45:02,991][2722668] Overriding arg 'num_workers' with value 1 passed from command line
1371
+ [2023-05-24 20:45:02,992][2722668] Adding new argument 'no_render'=True that is not in the saved config file!
1372
+ [2023-05-24 20:45:02,993][2722668] Adding new argument 'save_video'=True that is not in the saved config file!
1373
+ [2023-05-24 20:45:02,994][2722668] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
1374
+ [2023-05-24 20:45:02,995][2722668] Adding new argument 'video_name'=None that is not in the saved config file!
1375
+ [2023-05-24 20:45:02,996][2722668] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
1376
+ [2023-05-24 20:45:02,997][2722668] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
1377
+ [2023-05-24 20:45:02,997][2722668] Adding new argument 'push_to_hub'=False that is not in the saved config file!
1378
+ [2023-05-24 20:45:02,998][2722668] Adding new argument 'hf_repository'=None that is not in the saved config file!
1379
+ [2023-05-24 20:45:02,999][2722668] Adding new argument 'policy_index'=0 that is not in the saved config file!
1380
+ [2023-05-24 20:45:03,000][2722668] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
1381
+ [2023-05-24 20:45:03,002][2722668] Adding new argument 'train_script'=None that is not in the saved config file!
1382
+ [2023-05-24 20:45:03,003][2722668] Adding new argument 'enjoy_script'=None that is not in the saved config file!
1383
+ [2023-05-24 20:45:03,004][2722668] Using frameskip 1 and render_action_repeat=4 for evaluation
1384
+ [2023-05-24 20:45:03,009][2722668] RunningMeanStd input shape: (3, 72, 128)
1385
+ [2023-05-24 20:45:03,011][2722668] RunningMeanStd input shape: (1,)
1386
+ [2023-05-24 20:45:03,027][2722668] ConvEncoder: input_channels=3
1387
+ [2023-05-24 20:45:03,078][2722668] Conv encoder output size: 512
1388
+ [2023-05-24 20:45:03,079][2722668] Policy head output size: 512
1389
+ [2023-05-24 20:45:03,110][2722668] Loading state from checkpoint /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002443_10006528.pth...
1390
+ [2023-05-24 20:45:03,949][2722668] Num frames 100...
1391
+ [2023-05-24 20:45:04,111][2722668] Num frames 200...
1392
+ [2023-05-24 20:45:04,278][2722668] Num frames 300...
1393
+ [2023-05-24 20:45:04,443][2722668] Num frames 400...
1394
+ [2023-05-24 20:45:04,602][2722668] Num frames 500...
1395
+ [2023-05-24 20:45:04,761][2722668] Num frames 600...
1396
+ [2023-05-24 20:45:04,831][2722668] Avg episode rewards: #0: 11.080, true rewards: #0: 6.080
1397
+ [2023-05-24 20:45:04,833][2722668] Avg episode reward: 11.080, avg true_objective: 6.080
1398
+ [2023-05-24 20:45:04,986][2722668] Num frames 700...
1399
+ [2023-05-24 20:45:05,145][2722668] Num frames 800...
1400
+ [2023-05-24 20:45:05,309][2722668] Num frames 900...
1401
+ [2023-05-24 20:45:05,472][2722668] Num frames 1000...
1402
+ [2023-05-24 20:45:05,642][2722668] Num frames 1100...
1403
+ [2023-05-24 20:45:05,781][2722668] Avg episode rewards: #0: 9.260, true rewards: #0: 5.760
1404
+ [2023-05-24 20:45:05,783][2722668] Avg episode reward: 9.260, avg true_objective: 5.760
1405
+ [2023-05-24 20:45:05,862][2722668] Num frames 1200...
1406
+ [2023-05-24 20:45:06,020][2722668] Num frames 1300...
1407
+ [2023-05-24 20:45:06,184][2722668] Num frames 1400...
1408
+ [2023-05-24 20:45:06,346][2722668] Num frames 1500...
1409
+ [2023-05-24 20:45:06,502][2722668] Num frames 1600...
1410
+ [2023-05-24 20:45:06,661][2722668] Num frames 1700...
1411
+ [2023-05-24 20:45:06,820][2722668] Num frames 1800...
1412
+ [2023-05-24 20:45:06,982][2722668] Num frames 1900...
1413
+ [2023-05-24 20:45:07,143][2722668] Num frames 2000...
1414
+ [2023-05-24 20:45:07,302][2722668] Num frames 2100...
1415
+ [2023-05-24 20:45:07,464][2722668] Num frames 2200...
1416
+ [2023-05-24 20:45:07,626][2722668] Num frames 2300...
1417
+ [2023-05-24 20:45:07,780][2722668] Num frames 2400...
1418
+ [2023-05-24 20:45:07,948][2722668] Num frames 2500...
1419
+ [2023-05-24 20:45:08,091][2722668] Num frames 2600...
1420
+ [2023-05-24 20:45:08,249][2722668] Num frames 2700...
1421
+ [2023-05-24 20:45:08,407][2722668] Num frames 2800...
1422
+ [2023-05-24 20:45:08,566][2722668] Num frames 2900...
1423
+ [2023-05-24 20:45:08,723][2722668] Num frames 3000...
1424
+ [2023-05-24 20:45:08,876][2722668] Num frames 3100...
1425
+ [2023-05-24 20:45:09,008][2722668] Num frames 3200...
1426
+ [2023-05-24 20:45:09,146][2722668] Avg episode rewards: #0: 25.173, true rewards: #0: 10.840
1427
+ [2023-05-24 20:45:09,148][2722668] Avg episode reward: 25.173, avg true_objective: 10.840
1428
+ [2023-05-24 20:45:09,227][2722668] Num frames 3300...
1429
+ [2023-05-24 20:45:09,385][2722668] Num frames 3400...
1430
+ [2023-05-24 20:45:09,556][2722668] Num frames 3500...
1431
+ [2023-05-24 20:45:09,716][2722668] Num frames 3600...
1432
+ [2023-05-24 20:45:09,874][2722668] Num frames 3700...
1433
+ [2023-05-24 20:45:10,029][2722668] Num frames 3800...
1434
+ [2023-05-24 20:45:10,193][2722668] Num frames 3900...
1435
+ [2023-05-24 20:45:10,343][2722668] Num frames 4000...
1436
+ [2023-05-24 20:45:10,425][2722668] Avg episode rewards: #0: 23.547, true rewards: #0: 10.048
1437
+ [2023-05-24 20:45:10,426][2722668] Avg episode reward: 23.547, avg true_objective: 10.048
1438
+ [2023-05-24 20:45:10,540][2722668] Num frames 4100...
1439
+ [2023-05-24 20:45:10,691][2722668] Num frames 4200...
1440
+ [2023-05-24 20:45:10,847][2722668] Num frames 4300...
1441
+ [2023-05-24 20:45:10,985][2722668] Num frames 4400...
1442
+ [2023-05-24 20:45:11,128][2722668] Num frames 4500...
1443
+ [2023-05-24 20:45:11,287][2722668] Num frames 4600...
1444
+ [2023-05-24 20:45:11,425][2722668] Num frames 4700...
1445
+ [2023-05-24 20:45:11,584][2722668] Num frames 4800...
1446
+ [2023-05-24 20:45:11,731][2722668] Num frames 4900...
1447
+ [2023-05-24 20:45:11,885][2722668] Num frames 5000...
1448
+ [2023-05-24 20:45:12,052][2722668] Num frames 5100...
1449
+ [2023-05-24 20:45:12,222][2722668] Num frames 5200...
1450
+ [2023-05-24 20:45:12,382][2722668] Num frames 5300...
1451
+ [2023-05-24 20:45:12,540][2722668] Num frames 5400...
1452
+ [2023-05-24 20:45:12,701][2722668] Num frames 5500...
1453
+ [2023-05-24 20:45:12,866][2722668] Num frames 5600...
1454
+ [2023-05-24 20:45:13,045][2722668] Num frames 5700...
1455
+ [2023-05-24 20:45:13,208][2722668] Num frames 5800...
1456
+ [2023-05-24 20:45:13,370][2722668] Num frames 5900...
1457
+ [2023-05-24 20:45:13,528][2722668] Num frames 6000...
1458
+ [2023-05-24 20:45:13,697][2722668] Num frames 6100...
1459
+ [2023-05-24 20:45:13,787][2722668] Avg episode rewards: #0: 30.038, true rewards: #0: 12.238
1460
+ [2023-05-24 20:45:13,788][2722668] Avg episode reward: 30.038, avg true_objective: 12.238
1461
+ [2023-05-24 20:45:13,924][2722668] Num frames 6200...
1462
+ [2023-05-24 20:45:14,079][2722668] Num frames 6300...
1463
+ [2023-05-24 20:45:14,241][2722668] Num frames 6400...
1464
+ [2023-05-24 20:45:14,398][2722668] Num frames 6500...
1465
+ [2023-05-24 20:45:14,558][2722668] Num frames 6600...
1466
+ [2023-05-24 20:45:14,737][2722668] Num frames 6700...
1467
+ [2023-05-24 20:45:14,882][2722668] Num frames 6800...
1468
+ [2023-05-24 20:45:15,043][2722668] Num frames 6900...
1469
+ [2023-05-24 20:45:15,204][2722668] Num frames 7000...
1470
+ [2023-05-24 20:45:15,364][2722668] Num frames 7100...
1471
+ [2023-05-24 20:45:15,522][2722668] Num frames 7200...
1472
+ [2023-05-24 20:45:15,688][2722668] Num frames 7300...
1473
+ [2023-05-24 20:45:15,846][2722668] Num frames 7400...
1474
+ [2023-05-24 20:45:15,909][2722668] Avg episode rewards: #0: 30.005, true rewards: #0: 12.338
1475
+ [2023-05-24 20:45:15,911][2722668] Avg episode reward: 30.005, avg true_objective: 12.338
1476
+ [2023-05-24 20:45:16,075][2722668] Num frames 7500...
1477
+ [2023-05-24 20:45:16,224][2722668] Num frames 7600...
1478
+ [2023-05-24 20:45:16,381][2722668] Num frames 7700...
1479
+ [2023-05-24 20:45:16,534][2722668] Num frames 7800...
1480
+ [2023-05-24 20:45:16,695][2722668] Num frames 7900...
1481
+ [2023-05-24 20:45:16,853][2722668] Num frames 8000...
1482
+ [2023-05-24 20:45:17,011][2722668] Num frames 8100...
1483
+ [2023-05-24 20:45:17,174][2722668] Num frames 8200...
1484
+ [2023-05-24 20:45:17,343][2722668] Num frames 8300...
1485
+ [2023-05-24 20:45:17,509][2722668] Num frames 8400...
1486
+ [2023-05-24 20:45:17,683][2722668] Num frames 8500...
1487
+ [2023-05-24 20:45:17,851][2722668] Num frames 8600...
1488
+ [2023-05-24 20:45:18,023][2722668] Num frames 8700...
1489
+ [2023-05-24 20:45:18,188][2722668] Num frames 8800...
1490
+ [2023-05-24 20:45:18,359][2722668] Num frames 8900...
1491
+ [2023-05-24 20:45:18,550][2722668] Avg episode rewards: #0: 31.680, true rewards: #0: 12.823
1492
+ [2023-05-24 20:45:18,552][2722668] Avg episode reward: 31.680, avg true_objective: 12.823
1493
+ [2023-05-24 20:45:18,597][2722668] Num frames 9000...
1494
+ [2023-05-24 20:45:18,755][2722668] Num frames 9100...
1495
+ [2023-05-24 20:45:18,914][2722668] Num frames 9200...
1496
+ [2023-05-24 20:45:19,078][2722668] Num frames 9300...
1497
+ [2023-05-24 20:45:19,182][2722668] Avg episode rewards: #0: 28.535, true rewards: #0: 11.660
1498
+ [2023-05-24 20:45:19,184][2722668] Avg episode reward: 28.535, avg true_objective: 11.660
1499
+ [2023-05-24 20:45:19,306][2722668] Num frames 9400...
1500
+ [2023-05-24 20:45:19,469][2722668] Num frames 9500...
1501
+ [2023-05-24 20:45:19,667][2722668] Avg episode rewards: #0: 25.649, true rewards: #0: 10.649
1502
+ [2023-05-24 20:45:19,669][2722668] Avg episode reward: 25.649, avg true_objective: 10.649
1503
+ [2023-05-24 20:45:19,701][2722668] Num frames 9600...
1504
+ [2023-05-24 20:45:19,863][2722668] Num frames 9700...
1505
+ [2023-05-24 20:45:20,026][2722668] Num frames 9800...
1506
+ [2023-05-24 20:45:20,185][2722668] Num frames 9900...
1507
+ [2023-05-24 20:45:20,338][2722668] Num frames 10000...
1508
+ [2023-05-24 20:45:20,502][2722668] Num frames 10100...
1509
+ [2023-05-24 20:45:20,661][2722668] Num frames 10200...
1510
+ [2023-05-24 20:45:20,831][2722668] Num frames 10300...
1511
+ [2023-05-24 20:45:21,032][2722668] Avg episode rewards: #0: 24.484, true rewards: #0: 10.384
1512
+ [2023-05-24 20:45:21,034][2722668] Avg episode reward: 24.484, avg true_objective: 10.384
1513
+ [2023-05-24 20:45:46,594][2722668] Replay video saved to /home/mark/rl_course/unit8/train_dir/default_experiment/replay.mp4!
1514
+ [2023-05-24 20:45:49,050][2722668] Loading existing experiment configuration from /home/mark/rl_course/unit8/train_dir/default_experiment/config.json
1515
+ [2023-05-24 20:45:49,051][2722668] Overriding arg 'num_workers' with value 1 passed from command line
1516
+ [2023-05-24 20:45:49,052][2722668] Adding new argument 'no_render'=True that is not in the saved config file!
1517
+ [2023-05-24 20:45:49,053][2722668] Adding new argument 'save_video'=True that is not in the saved config file!
1518
+ [2023-05-24 20:45:49,055][2722668] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
1519
+ [2023-05-24 20:45:49,055][2722668] Adding new argument 'video_name'=None that is not in the saved config file!
1520
+ [2023-05-24 20:45:49,056][2722668] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
1521
+ [2023-05-24 20:45:49,057][2722668] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
1522
+ [2023-05-24 20:45:49,057][2722668] Adding new argument 'push_to_hub'=True that is not in the saved config file!
1523
+ [2023-05-24 20:45:49,058][2722668] Adding new argument 'hf_repository'='markeidsaune/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
1524
+ [2023-05-24 20:45:49,059][2722668] Adding new argument 'policy_index'=0 that is not in the saved config file!
1525
+ [2023-05-24 20:45:49,059][2722668] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
1526
+ [2023-05-24 20:45:49,060][2722668] Adding new argument 'train_script'=None that is not in the saved config file!
1527
+ [2023-05-24 20:45:49,060][2722668] Adding new argument 'enjoy_script'=None that is not in the saved config file!
1528
+ [2023-05-24 20:45:49,061][2722668] Using frameskip 1 and render_action_repeat=4 for evaluation
1529
+ [2023-05-24 20:45:49,074][2722668] RunningMeanStd input shape: (3, 72, 128)
1530
+ [2023-05-24 20:45:49,075][2722668] RunningMeanStd input shape: (1,)
1531
+ [2023-05-24 20:45:49,091][2722668] ConvEncoder: input_channels=3
1532
+ [2023-05-24 20:45:49,146][2722668] Conv encoder output size: 512
1533
+ [2023-05-24 20:45:49,147][2722668] Policy head output size: 512
1534
+ [2023-05-24 20:45:49,189][2722668] Loading state from checkpoint /home/mark/rl_course/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000002443_10006528.pth...
1535
+ [2023-05-24 20:45:50,034][2722668] Num frames 100...
1536
+ [2023-05-24 20:45:50,198][2722668] Num frames 200...
1537
+ [2023-05-24 20:45:50,358][2722668] Num frames 300...
1538
+ [2023-05-24 20:45:50,518][2722668] Num frames 400...
1539
+ [2023-05-24 20:45:50,677][2722668] Num frames 500...
1540
+ [2023-05-24 20:45:50,848][2722668] Num frames 600...
1541
+ [2023-05-24 20:45:51,020][2722668] Num frames 700...
1542
+ [2023-05-24 20:45:51,193][2722668] Num frames 800...
1543
+ [2023-05-24 20:45:51,355][2722668] Num frames 900...
1544
+ [2023-05-24 20:45:51,436][2722668] Avg episode rewards: #0: 18.150, true rewards: #0: 9.150
1545
+ [2023-05-24 20:45:51,437][2722668] Avg episode reward: 18.150, avg true_objective: 9.150
1546
+ [2023-05-24 20:45:51,578][2722668] Num frames 1000...
1547
+ [2023-05-24 20:45:51,736][2722668] Num frames 1100...
1548
+ [2023-05-24 20:45:51,896][2722668] Num frames 1200...
1549
+ [2023-05-24 20:45:52,055][2722668] Num frames 1300...
1550
+ [2023-05-24 20:45:52,213][2722668] Num frames 1400...
1551
+ [2023-05-24 20:45:52,362][2722668] Avg episode rewards: #0: 12.795, true rewards: #0: 7.295
1552
+ [2023-05-24 20:45:52,364][2722668] Avg episode reward: 12.795, avg true_objective: 7.295
1553
+ [2023-05-24 20:45:52,433][2722668] Num frames 1500...
1554
+ [2023-05-24 20:45:52,589][2722668] Num frames 1600...
1555
+ [2023-05-24 20:45:52,746][2722668] Num frames 1700...
1556
+ [2023-05-24 20:45:52,901][2722668] Num frames 1800...
1557
+ [2023-05-24 20:45:53,064][2722668] Num frames 1900...
1558
+ [2023-05-24 20:45:53,223][2722668] Num frames 2000...
1559
+ [2023-05-24 20:45:53,382][2722668] Num frames 2100...
1560
+ [2023-05-24 20:45:53,542][2722668] Num frames 2200...
1561
+ [2023-05-24 20:45:53,704][2722668] Num frames 2300...
1562
+ [2023-05-24 20:45:53,868][2722668] Num frames 2400...
1563
+ [2023-05-24 20:45:54,030][2722668] Num frames 2500...
1564
+ [2023-05-24 20:45:54,193][2722668] Num frames 2600...
1565
+ [2023-05-24 20:45:54,356][2722668] Num frames 2700...
1566
+ [2023-05-24 20:45:54,523][2722668] Num frames 2800...
1567
+ [2023-05-24 20:45:54,687][2722668] Num frames 2900...
1568
+ [2023-05-24 20:45:54,851][2722668] Num frames 3000...
1569
+ [2023-05-24 20:45:55,017][2722668] Num frames 3100...
1570
+ [2023-05-24 20:45:55,187][2722668] Num frames 3200...
1571
+ [2023-05-24 20:45:55,357][2722668] Num frames 3300...
1572
+ [2023-05-24 20:45:55,527][2722668] Num frames 3400...
1573
+ [2023-05-24 20:45:55,705][2722668] Num frames 3500...
1574
+ [2023-05-24 20:45:55,857][2722668] Avg episode rewards: #0: 27.196, true rewards: #0: 11.863
1575
+ [2023-05-24 20:45:55,859][2722668] Avg episode reward: 27.196, avg true_objective: 11.863
1576
+ [2023-05-24 20:45:55,930][2722668] Num frames 3600...
1577
+ [2023-05-24 20:45:56,085][2722668] Num frames 3700...
1578
+ [2023-05-24 20:45:56,248][2722668] Num frames 3800...
1579
+ [2023-05-24 20:45:56,408][2722668] Num frames 3900...
1580
+ [2023-05-24 20:45:56,569][2722668] Num frames 4000...
1581
+ [2023-05-24 20:45:56,731][2722668] Num frames 4100...
1582
+ [2023-05-24 20:45:56,918][2722668] Avg episode rewards: #0: 23.455, true rewards: #0: 10.455
1583
+ [2023-05-24 20:45:56,920][2722668] Avg episode reward: 23.455, avg true_objective: 10.455
1584
+ [2023-05-24 20:45:56,953][2722668] Num frames 4200...
1585
+ [2023-05-24 20:45:57,115][2722668] Num frames 4300...
1586
+ [2023-05-24 20:45:57,275][2722668] Num frames 4400...
1587
+ [2023-05-24 20:45:57,434][2722668] Num frames 4500...
1588
+ [2023-05-24 20:45:57,587][2722668] Num frames 4600...
1589
+ [2023-05-24 20:45:57,742][2722668] Num frames 4700...
1590
+ [2023-05-24 20:45:57,901][2722668] Num frames 4800...
1591
+ [2023-05-24 20:45:58,082][2722668] Avg episode rewards: #0: 21.556, true rewards: #0: 9.756
1592
+ [2023-05-24 20:45:58,084][2722668] Avg episode reward: 21.556, avg true_objective: 9.756
1593
+ [2023-05-24 20:45:58,124][2722668] Num frames 4900...
1594
+ [2023-05-24 20:45:58,280][2722668] Num frames 5000...
1595
+ [2023-05-24 20:45:58,441][2722668] Num frames 5100...
1596
+ [2023-05-24 20:45:58,593][2722668] Num frames 5200...
1597
+ [2023-05-24 20:45:58,763][2722668] Num frames 5300...
1598
+ [2023-05-24 20:45:58,924][2722668] Num frames 5400...
1599
+ [2023-05-24 20:45:59,080][2722668] Num frames 5500...
1600
+ [2023-05-24 20:45:59,247][2722668] Num frames 5600...
1601
+ [2023-05-24 20:45:59,412][2722668] Num frames 5700...
1602
+ [2023-05-24 20:45:59,570][2722668] Num frames 5800...
1603
+ [2023-05-24 20:45:59,734][2722668] Num frames 5900...
1604
+ [2023-05-24 20:45:59,794][2722668] Avg episode rewards: #0: 21.503, true rewards: #0: 9.837
1605
+ [2023-05-24 20:45:59,796][2722668] Avg episode reward: 21.503, avg true_objective: 9.837
1606
+ [2023-05-24 20:45:59,956][2722668] Num frames 6000...
1607
+ [2023-05-24 20:46:00,109][2722668] Num frames 6100...
1608
+ [2023-05-24 20:46:00,269][2722668] Num frames 6200...
1609
+ [2023-05-24 20:46:00,431][2722668] Num frames 6300...
1610
+ [2023-05-24 20:46:00,600][2722668] Num frames 6400...
1611
+ [2023-05-24 20:46:00,775][2722668] Num frames 6500...
1612
+ [2023-05-24 20:46:00,943][2722668] Num frames 6600...
1613
+ [2023-05-24 20:46:01,108][2722668] Num frames 6700...
1614
+ [2023-05-24 20:46:01,269][2722668] Num frames 6800...
1615
+ [2023-05-24 20:46:01,435][2722668] Num frames 6900...
1616
+ [2023-05-24 20:46:01,604][2722668] Num frames 7000...
1617
+ [2023-05-24 20:46:01,760][2722668] Num frames 7100...
1618
+ [2023-05-24 20:46:01,918][2722668] Num frames 7200...
1619
+ [2023-05-24 20:46:02,075][2722668] Num frames 7300...
1620
+ [2023-05-24 20:46:02,238][2722668] Num frames 7400...
1621
+ [2023-05-24 20:46:02,404][2722668] Num frames 7500...
1622
+ [2023-05-24 20:46:02,562][2722668] Num frames 7600...
1623
+ [2023-05-24 20:46:02,701][2722668] Num frames 7700...
1624
+ [2023-05-24 20:46:02,838][2722668] Num frames 7800...
1625
+ [2023-05-24 20:46:02,978][2722668] Num frames 7900...
1626
+ [2023-05-24 20:46:03,113][2722668] Num frames 8000...
1627
+ [2023-05-24 20:46:03,171][2722668] Avg episode rewards: #0: 27.145, true rewards: #0: 11.431
1628
+ [2023-05-24 20:46:03,172][2722668] Avg episode reward: 27.145, avg true_objective: 11.431
1629
+ [2023-05-24 20:46:03,308][2722668] Num frames 8100...
1630
+ [2023-05-24 20:46:03,454][2722668] Num frames 8200...
1631
+ [2023-05-24 20:46:03,607][2722668] Num frames 8300...
1632
+ [2023-05-24 20:46:03,767][2722668] Num frames 8400...
1633
+ [2023-05-24 20:46:03,932][2722668] Num frames 8500...
1634
+ [2023-05-24 20:46:04,090][2722668] Num frames 8600...
1635
+ [2023-05-24 20:46:04,229][2722668] Num frames 8700...
1636
+ [2023-05-24 20:46:04,385][2722668] Num frames 8800...
1637
+ [2023-05-24 20:46:04,523][2722668] Num frames 8900...
1638
+ [2023-05-24 20:46:04,693][2722668] Num frames 9000...
1639
+ [2023-05-24 20:46:04,862][2722668] Num frames 9100...
1640
+ [2023-05-24 20:46:05,034][2722668] Num frames 9200...
1641
+ [2023-05-24 20:46:05,195][2722668] Num frames 9300...
1642
+ [2023-05-24 20:46:05,359][2722668] Num frames 9400...
1643
+ [2023-05-24 20:46:05,520][2722668] Num frames 9500...
1644
+ [2023-05-24 20:46:05,679][2722668] Num frames 9600...
1645
+ [2023-05-24 20:46:05,828][2722668] Num frames 9700...
1646
+ [2023-05-24 20:46:05,988][2722668] Num frames 9800...
1647
+ [2023-05-24 20:46:06,151][2722668] Num frames 9900...
1648
+ [2023-05-24 20:46:06,303][2722668] Num frames 10000...
1649
+ [2023-05-24 20:46:06,463][2722668] Num frames 10100...
1650
+ [2023-05-24 20:46:06,525][2722668] Avg episode rewards: #0: 30.752, true rewards: #0: 12.628
1651
+ [2023-05-24 20:46:06,526][2722668] Avg episode reward: 30.752, avg true_objective: 12.628
1652
+ [2023-05-24 20:46:06,676][2722668] Num frames 10200...
1653
+ [2023-05-24 20:46:06,836][2722668] Num frames 10300...
1654
+ [2023-05-24 20:46:07,005][2722668] Num frames 10400...
1655
+ [2023-05-24 20:46:07,182][2722668] Num frames 10500...
1656
+ [2023-05-24 20:46:07,348][2722668] Num frames 10600...
1657
+ [2023-05-24 20:46:07,512][2722668] Num frames 10700...
1658
+ [2023-05-24 20:46:07,676][2722668] Num frames 10800...
1659
+ [2023-05-24 20:46:07,833][2722668] Num frames 10900...
1660
+ [2023-05-24 20:46:08,004][2722668] Num frames 11000...
1661
+ [2023-05-24 20:46:08,175][2722668] Num frames 11100...
1662
+ [2023-05-24 20:46:08,340][2722668] Num frames 11200...
1663
+ [2023-05-24 20:46:08,506][2722668] Num frames 11300...
1664
+ [2023-05-24 20:46:08,673][2722668] Num frames 11400...
1665
+ [2023-05-24 20:46:08,838][2722668] Num frames 11500...
1666
+ [2023-05-24 20:46:09,009][2722668] Num frames 11600...
1667
+ [2023-05-24 20:46:09,172][2722668] Num frames 11700...
1668
+ [2023-05-24 20:46:09,341][2722668] Num frames 11800...
1669
+ [2023-05-24 20:46:09,499][2722668] Num frames 11900...
1670
+ [2023-05-24 20:46:09,657][2722668] Num frames 12000...
1671
+ [2023-05-24 20:46:09,822][2722668] Num frames 12100...
1672
+ [2023-05-24 20:46:09,991][2722668] Num frames 12200...
1673
+ [2023-05-24 20:46:10,053][2722668] Avg episode rewards: #0: 33.780, true rewards: #0: 13.558
1674
+ [2023-05-24 20:46:10,054][2722668] Avg episode reward: 33.780, avg true_objective: 13.558
1675
+ [2023-05-24 20:46:10,212][2722668] Num frames 12300...
1676
+ [2023-05-24 20:46:10,366][2722668] Num frames 12400...
1677
+ [2023-05-24 20:46:10,526][2722668] Num frames 12500...
1678
+ [2023-05-24 20:46:10,698][2722668] Num frames 12600...
1679
+ [2023-05-24 20:46:10,868][2722668] Num frames 12700...
1680
+ [2023-05-24 20:46:11,033][2722668] Num frames 12800...
1681
+ [2023-05-24 20:46:11,165][2722668] Avg episode rewards: #0: 31.947, true rewards: #0: 12.847
1682
+ [2023-05-24 20:46:11,167][2722668] Avg episode reward: 31.947, avg true_objective: 12.847
1683
+ [2023-05-24 20:46:42,742][2722668] Replay video saved to /home/mark/rl_course/unit8/train_dir/default_experiment/replay.mp4!