[2023-07-24 00:29:43,173][00294] Saving configuration to /content/train_dir/default_experiment/config.json... [2023-07-24 00:29:43,176][00294] Rollout worker 0 uses device cpu [2023-07-24 00:29:43,181][00294] Rollout worker 1 uses device cpu [2023-07-24 00:29:43,182][00294] Rollout worker 2 uses device cpu [2023-07-24 00:29:43,190][00294] Rollout worker 3 uses device cpu [2023-07-24 00:29:43,191][00294] Rollout worker 4 uses device cpu [2023-07-24 00:29:43,198][00294] Rollout worker 5 uses device cpu [2023-07-24 00:29:43,201][00294] Rollout worker 6 uses device cpu [2023-07-24 00:29:43,202][00294] Rollout worker 7 uses device cpu [2023-07-24 00:29:43,365][00294] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:29:43,366][00294] InferenceWorker_p0-w0: min num requests: 2 [2023-07-24 00:29:43,399][00294] Starting all processes... [2023-07-24 00:29:43,400][00294] Starting process learner_proc0 [2023-07-24 00:29:43,456][00294] Starting all processes... [2023-07-24 00:29:43,469][00294] Starting process inference_proc0-0 [2023-07-24 00:29:43,469][00294] Starting process rollout_proc0 [2023-07-24 00:29:43,472][00294] Starting process rollout_proc1 [2023-07-24 00:29:43,472][00294] Starting process rollout_proc2 [2023-07-24 00:29:43,472][00294] Starting process rollout_proc3 [2023-07-24 00:29:43,473][00294] Starting process rollout_proc4 [2023-07-24 00:29:43,473][00294] Starting process rollout_proc5 [2023-07-24 00:29:43,473][00294] Starting process rollout_proc6 [2023-07-24 00:29:43,473][00294] Starting process rollout_proc7 [2023-07-24 00:29:45,751][00294] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 294], exiting... [2023-07-24 00:29:45,753][00294] Runner profile tree view: main_loop: 2.3549 [2023-07-24 00:29:45,755][00294] Collected {}, FPS: 0.0 [2023-07-24 00:30:00,773][08962] Worker 6 uses CPU cores [0] [2023-07-24 00:30:00,806][08943] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:30:00,819][08943] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-07-24 00:30:00,843][08961] Worker 4 uses CPU cores [0] [2023-07-24 00:30:00,879][08959] Worker 2 uses CPU cores [0] [2023-07-24 00:30:00,897][08943] Num visible devices: 1 [2023-07-24 00:30:00,928][08943] Starting seed is not provided [2023-07-24 00:30:00,929][08943] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:30:00,930][08943] Initializing actor-critic model on device cuda:0 [2023-07-24 00:30:00,931][08943] RunningMeanStd input shape: (23,) [2023-07-24 00:30:00,933][08943] Stopping Batcher_0... [2023-07-24 00:30:00,934][08943] Loop batcher_evt_loop terminating... [2023-07-24 00:30:00,934][08943] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:30:00,936][08943] RunningMeanStd input shape: (1,) [2023-07-24 00:30:00,980][08962] Stopping RolloutWorker_w6... [2023-07-24 00:30:01,013][08943] ConvEncoder: input_channels=3 [2023-07-24 00:30:00,992][08962] Loop rollout_proc6_evt_loop terminating... [2023-07-24 00:30:01,031][08961] Stopping RolloutWorker_w4... [2023-07-24 00:30:01,046][08961] Loop rollout_proc4_evt_loop terminating... [2023-07-24 00:30:01,059][08959] Stopping RolloutWorker_w2... [2023-07-24 00:30:01,107][08963] Worker 5 uses CPU cores [1] [2023-07-24 00:30:01,110][08960] Worker 3 uses CPU cores [1] [2023-07-24 00:30:01,113][08959] Loop rollout_proc2_evt_loop terminating... [2023-07-24 00:30:01,235][08964] Worker 7 uses CPU cores [1] [2023-07-24 00:30:01,252][08958] Worker 0 uses CPU cores [0] [2023-07-24 00:30:01,250][08963] Stopping RolloutWorker_w5... [2023-07-24 00:30:01,273][08960] Stopping RolloutWorker_w3... [2023-07-24 00:30:01,280][08960] Loop rollout_proc3_evt_loop terminating... [2023-07-24 00:30:01,274][08963] Loop rollout_proc5_evt_loop terminating... [2023-07-24 00:30:01,320][08957] Worker 1 uses CPU cores [1] [2023-07-24 00:30:01,326][08964] Stopping RolloutWorker_w7... [2023-07-24 00:30:01,332][08958] Stopping RolloutWorker_w0... [2023-07-24 00:30:01,334][08958] Loop rollout_proc0_evt_loop terminating... [2023-07-24 00:30:01,331][08964] Loop rollout_proc7_evt_loop terminating... [2023-07-24 00:30:01,367][08957] Stopping RolloutWorker_w1... [2023-07-24 00:30:01,369][08957] Loop rollout_proc1_evt_loop terminating... [2023-07-24 00:30:01,596][08943] Conv encoder output size: 512 [2023-07-24 00:30:01,598][08943] Policy head output size: 640 [2023-07-24 00:30:01,704][08943] Created Actor Critic model with architecture: [2023-07-24 00:30:01,704][08943] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (measurements): RunningMeanStdInPlace() (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) (measurements_head): Sequential( (0): Linear(in_features=23, out_features=128, bias=True) (1): ELU(alpha=1.0) (2): Linear(in_features=128, out_features=128, bias=True) (3): ELU(alpha=1.0) ) ) (core): ModelCoreRNN( (core): GRU(640, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=39, bias=True) ) ) [2023-07-24 00:30:10,905][08943] Using optimizer [2023-07-24 00:30:10,906][08943] No checkpoints found [2023-07-24 00:30:10,907][08943] Did not load from checkpoint, starting from scratch! [2023-07-24 00:30:10,907][08943] Initialized policy 0 weights for model version 0 [2023-07-24 00:30:10,910][08943] LearnerWorker_p0 finished initialization! [2023-07-24 00:30:10,911][08943] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth... [2023-07-24 00:30:10,932][08943] Stopping LearnerWorker_p0... [2023-07-24 00:30:10,933][08943] Loop learner_proc0_evt_loop terminating... [2023-07-24 00:30:43,721][00294] Environment doom_basic already registered, overwriting... [2023-07-24 00:30:43,724][00294] Environment doom_two_colors_easy already registered, overwriting... [2023-07-24 00:30:43,726][00294] Environment doom_two_colors_hard already registered, overwriting... [2023-07-24 00:30:43,727][00294] Environment doom_dm already registered, overwriting... [2023-07-24 00:30:43,728][00294] Environment doom_dwango5 already registered, overwriting... [2023-07-24 00:30:43,732][00294] Environment doom_my_way_home_flat_actions already registered, overwriting... [2023-07-24 00:30:43,733][00294] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2023-07-24 00:30:43,736][00294] Environment doom_my_way_home already registered, overwriting... [2023-07-24 00:30:43,737][00294] Environment doom_deadly_corridor already registered, overwriting... [2023-07-24 00:30:43,738][00294] Environment doom_defend_the_center already registered, overwriting... [2023-07-24 00:30:43,739][00294] Environment doom_defend_the_line already registered, overwriting... [2023-07-24 00:30:43,741][00294] Environment doom_health_gathering already registered, overwriting... [2023-07-24 00:30:43,742][00294] Environment doom_health_gathering_supreme already registered, overwriting... [2023-07-24 00:30:43,743][00294] Environment doom_battle already registered, overwriting... [2023-07-24 00:30:43,746][00294] Environment doom_battle2 already registered, overwriting... [2023-07-24 00:30:43,749][00294] Environment doom_duel_bots already registered, overwriting... [2023-07-24 00:30:43,750][00294] Environment doom_deathmatch_bots already registered, overwriting... [2023-07-24 00:30:43,751][00294] Environment doom_duel already registered, overwriting... [2023-07-24 00:30:43,753][00294] Environment doom_deathmatch_full already registered, overwriting... [2023-07-24 00:30:43,754][00294] Environment doom_benchmark already registered, overwriting... [2023-07-24 00:30:43,755][00294] register_encoder_factory: [2023-07-24 00:30:43,791][00294] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-07-24 00:30:43,796][00294] Experiment dir /content/train_dir/default_experiment already exists! [2023-07-24 00:30:43,797][00294] Resuming existing experiment from /content/train_dir/default_experiment... [2023-07-24 00:30:43,799][00294] Weights and Biases integration disabled [2023-07-24 00:30:43,805][00294] Environment var CUDA_VISIBLE_DEVICES is 0 [2023-07-24 00:30:46,569][00294] Starting experiment with the following configuration: help=False algo=APPO env=doom_deathmatch_bots experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=gpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=4 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=4000000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=3 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_deathmatch_bots --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 cli_args={'env': 'doom_deathmatch_bots', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} git_hash=unknown git_repo_name=not a git repository [2023-07-24 00:30:46,571][00294] Saving configuration to /content/train_dir/default_experiment/config.json... [2023-07-24 00:30:46,575][00294] Rollout worker 0 uses device cpu [2023-07-24 00:30:46,577][00294] Rollout worker 1 uses device cpu [2023-07-24 00:30:46,581][00294] Rollout worker 2 uses device cpu [2023-07-24 00:30:46,582][00294] Rollout worker 3 uses device cpu [2023-07-24 00:30:46,584][00294] Rollout worker 4 uses device cpu [2023-07-24 00:30:46,585][00294] Rollout worker 5 uses device cpu [2023-07-24 00:30:46,586][00294] Rollout worker 6 uses device cpu [2023-07-24 00:30:46,587][00294] Rollout worker 7 uses device cpu [2023-07-24 00:30:46,722][00294] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:30:46,725][00294] InferenceWorker_p0-w0: min num requests: 2 [2023-07-24 00:30:46,767][00294] Starting all processes... [2023-07-24 00:30:46,769][00294] Starting process learner_proc0 [2023-07-24 00:30:46,847][00294] Starting all processes... [2023-07-24 00:30:46,861][00294] Starting process inference_proc0-0 [2023-07-24 00:30:46,863][00294] Starting process rollout_proc0 [2023-07-24 00:30:46,863][00294] Starting process rollout_proc1 [2023-07-24 00:30:46,863][00294] Starting process rollout_proc2 [2023-07-24 00:30:46,863][00294] Starting process rollout_proc3 [2023-07-24 00:30:46,863][00294] Starting process rollout_proc4 [2023-07-24 00:30:46,863][00294] Starting process rollout_proc5 [2023-07-24 00:30:46,863][00294] Starting process rollout_proc6 [2023-07-24 00:30:46,863][00294] Starting process rollout_proc7 [2023-07-24 00:31:04,113][09272] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:31:04,115][09272] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-07-24 00:31:04,341][09272] Num visible devices: 1 [2023-07-24 00:31:04,467][09259] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:31:04,475][09259] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-07-24 00:31:04,614][09259] Num visible devices: 1 [2023-07-24 00:31:04,681][09259] Starting seed is not provided [2023-07-24 00:31:04,684][09259] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:31:04,684][09259] Initializing actor-critic model on device cuda:0 [2023-07-24 00:31:04,688][09259] RunningMeanStd input shape: (23,) [2023-07-24 00:31:04,688][09259] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:31:04,692][09259] RunningMeanStd input shape: (1,) [2023-07-24 00:31:04,862][09281] Worker 6 uses CPU cores [0] [2023-07-24 00:31:04,863][09273] Worker 0 uses CPU cores [0] [2023-07-24 00:31:04,883][09278] Worker 5 uses CPU cores [1] [2023-07-24 00:31:04,912][09259] ConvEncoder: input_channels=3 [2023-07-24 00:31:04,988][09275] Worker 2 uses CPU cores [0] [2023-07-24 00:31:05,012][09274] Worker 1 uses CPU cores [1] [2023-07-24 00:31:05,029][09277] Worker 4 uses CPU cores [0] [2023-07-24 00:31:05,178][09282] Worker 7 uses CPU cores [1] [2023-07-24 00:31:05,211][09276] Worker 3 uses CPU cores [1] [2023-07-24 00:31:05,438][09259] Conv encoder output size: 512 [2023-07-24 00:31:05,440][09259] Policy head output size: 640 [2023-07-24 00:31:05,473][09259] Created Actor Critic model with architecture: [2023-07-24 00:31:05,474][09259] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (measurements): RunningMeanStdInPlace() (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) (measurements_head): Sequential( (0): Linear(in_features=23, out_features=128, bias=True) (1): ELU(alpha=1.0) (2): Linear(in_features=128, out_features=128, bias=True) (3): ELU(alpha=1.0) ) ) (core): ModelCoreRNN( (core): GRU(640, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=39, bias=True) ) ) [2023-07-24 00:31:06,712][00294] Heartbeat connected on Batcher_0 [2023-07-24 00:31:06,723][00294] Heartbeat connected on InferenceWorker_p0-w0 [2023-07-24 00:31:06,738][00294] Heartbeat connected on RolloutWorker_w0 [2023-07-24 00:31:06,742][00294] Heartbeat connected on RolloutWorker_w1 [2023-07-24 00:31:06,746][00294] Heartbeat connected on RolloutWorker_w2 [2023-07-24 00:31:06,751][00294] Heartbeat connected on RolloutWorker_w3 [2023-07-24 00:31:06,756][00294] Heartbeat connected on RolloutWorker_w4 [2023-07-24 00:31:06,762][00294] Heartbeat connected on RolloutWorker_w5 [2023-07-24 00:31:06,764][00294] Heartbeat connected on RolloutWorker_w6 [2023-07-24 00:31:06,770][00294] Heartbeat connected on RolloutWorker_w7 [2023-07-24 00:31:08,107][09259] Using optimizer [2023-07-24 00:31:08,108][09259] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth... [2023-07-24 00:31:08,124][09259] Loading model from checkpoint [2023-07-24 00:31:08,125][09259] Loaded experiment state at self.train_step=0, self.env_steps=0 [2023-07-24 00:31:08,126][09259] Initialized policy 0 weights for model version 0 [2023-07-24 00:31:08,131][09259] LearnerWorker_p0 finished initialization! [2023-07-24 00:31:08,131][09259] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:31:08,131][00294] Heartbeat connected on LearnerWorker_p0 [2023-07-24 00:31:08,264][09272] RunningMeanStd input shape: (23,) [2023-07-24 00:31:08,265][09272] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:31:08,265][09272] RunningMeanStd input shape: (1,) [2023-07-24 00:31:08,278][09272] ConvEncoder: input_channels=3 [2023-07-24 00:31:08,385][09272] Conv encoder output size: 512 [2023-07-24 00:31:08,386][09272] Policy head output size: 640 [2023-07-24 00:31:08,500][00294] Inference worker 0-0 is ready! [2023-07-24 00:31:08,503][00294] All inference workers are ready! Signal rollout workers to start! [2023-07-24 00:31:08,806][00294] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:31:08,840][09275] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:31:08,847][09282] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:31:08,843][09276] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:31:08,852][09275] Port 40500 is available [2023-07-24 00:31:08,852][09275] Using port 40500 [2023-07-24 00:31:08,863][09282] Port 41000 is available [2023-07-24 00:31:08,865][09274] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:31:08,868][09282] Using port 41000 [2023-07-24 00:31:08,867][09276] Port 40600 is available [2023-07-24 00:31:08,874][09276] Using port 40600 [2023-07-24 00:31:08,876][09277] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:31:08,878][09273] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:31:08,890][09281] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:31:08,889][09277] Port 40700 is available [2023-07-24 00:31:08,893][09277] Using port 40700 [2023-07-24 00:31:08,897][09273] Port 40300 is available [2023-07-24 00:31:08,897][09273] Using port 40300 [2023-07-24 00:31:08,904][09281] Port 40900 is available [2023-07-24 00:31:08,889][09274] Port 40400 is available [2023-07-24 00:31:08,904][09281] Using port 40900 [2023-07-24 00:31:08,893][09278] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:31:08,906][09274] Using port 40400 [2023-07-24 00:31:08,921][09278] Port 40800 is available [2023-07-24 00:31:08,922][09278] Using port 40800 [2023-07-24 00:31:09,075][09275] Port 40501 is available [2023-07-24 00:31:09,077][09275] Using port 40501 [2023-07-24 00:31:09,082][09275] Using port 40500 on host... [2023-07-24 00:31:09,118][09281] Port 40901 is available [2023-07-24 00:31:09,118][09281] Using port 40901 [2023-07-24 00:31:09,116][09277] Port 40701 is available [2023-07-24 00:31:09,119][09277] Using port 40701 [2023-07-24 00:31:09,122][09273] Port 40301 is available [2023-07-24 00:31:09,122][09273] Using port 40301 [2023-07-24 00:31:09,130][09277] Using port 40700 on host... [2023-07-24 00:31:09,127][09281] Using port 40900 on host... [2023-07-24 00:31:09,125][09273] Using port 40300 on host... [2023-07-24 00:31:09,134][09282] Port 41001 is available [2023-07-24 00:31:09,135][09282] Using port 41001 [2023-07-24 00:31:09,137][09276] Port 40601 is available [2023-07-24 00:31:09,143][09276] Using port 40601 [2023-07-24 00:31:09,146][09282] Using port 41000 on host... [2023-07-24 00:31:09,149][09276] Using port 40600 on host... [2023-07-24 00:31:09,192][09274] Port 40401 is available [2023-07-24 00:31:09,201][09274] Using port 40401 [2023-07-24 00:31:09,196][09278] Port 40801 is available [2023-07-24 00:31:09,203][09278] Using port 40801 [2023-07-24 00:31:09,212][09274] Using port 40400 on host... [2023-07-24 00:31:09,216][09278] Using port 40800 on host... [2023-07-24 00:31:10,855][09278] Initialized w:5 v:0 player:0 [2023-07-24 00:31:10,858][09282] Initialized w:7 v:0 player:0 [2023-07-24 00:31:10,862][09276] Initialized w:3 v:0 player:0 [2023-07-24 00:31:10,865][09274] Initialized w:1 v:0 player:0 [2023-07-24 00:31:10,867][09273] Initialized w:0 v:0 player:0 [2023-07-24 00:31:10,875][09277] Initialized w:4 v:0 player:0 [2023-07-24 00:31:10,877][09281] Initialized w:6 v:0 player:0 [2023-07-24 00:31:10,882][09273] Decorrelating experience for 0 frames... [2023-07-24 00:31:10,884][09273] Using port 40301 on host... [2023-07-24 00:31:10,884][09277] Decorrelating experience for 0 frames... [2023-07-24 00:31:10,881][09281] Decorrelating experience for 0 frames... [2023-07-24 00:31:10,881][09276] Decorrelating experience for 0 frames... [2023-07-24 00:31:10,882][09282] Decorrelating experience for 0 frames... [2023-07-24 00:31:10,880][09278] Decorrelating experience for 0 frames... [2023-07-24 00:31:10,888][09277] Using port 40701 on host... [2023-07-24 00:31:10,889][09281] Using port 40901 on host... [2023-07-24 00:31:10,884][09274] Decorrelating experience for 0 frames... [2023-07-24 00:31:10,891][09275] Initialized w:2 v:0 player:0 [2023-07-24 00:31:10,893][09276] Using port 40601 on host... [2023-07-24 00:31:10,888][09278] Using port 40801 on host... [2023-07-24 00:31:10,894][09282] Using port 41001 on host... [2023-07-24 00:31:10,891][09274] Using port 40401 on host... [2023-07-24 00:31:10,893][09275] Decorrelating experience for 0 frames... [2023-07-24 00:31:10,902][09275] Using port 40501 on host... [2023-07-24 00:31:12,537][09281] Initialized w:6 v:1 player:0 [2023-07-24 00:31:12,539][09277] Initialized w:4 v:1 player:0 [2023-07-24 00:31:12,542][09281] Decorrelating experience for 32 frames... [2023-07-24 00:31:12,548][09277] Decorrelating experience for 32 frames... [2023-07-24 00:31:12,551][09273] Initialized w:0 v:1 player:0 [2023-07-24 00:31:12,554][09275] Initialized w:2 v:1 player:0 [2023-07-24 00:31:12,563][09275] Decorrelating experience for 32 frames... [2023-07-24 00:31:12,553][09273] Decorrelating experience for 32 frames... [2023-07-24 00:31:12,573][09276] Initialized w:3 v:1 player:0 [2023-07-24 00:31:12,579][09278] Initialized w:5 v:1 player:0 [2023-07-24 00:31:12,576][09276] Decorrelating experience for 32 frames... [2023-07-24 00:31:12,585][09274] Initialized w:1 v:1 player:0 [2023-07-24 00:31:12,584][09278] Decorrelating experience for 32 frames... [2023-07-24 00:31:12,589][09274] Decorrelating experience for 32 frames... [2023-07-24 00:31:12,597][09282] Initialized w:7 v:1 player:0 [2023-07-24 00:31:12,603][09282] Decorrelating experience for 32 frames... [2023-07-24 00:31:13,167][09277] Port 40702 is available [2023-07-24 00:31:13,167][09277] Using port 40702 [2023-07-24 00:31:13,193][09275] Port 40502 is available [2023-07-24 00:31:13,194][09275] Using port 40502 [2023-07-24 00:31:13,194][09273] Port 40302 is available [2023-07-24 00:31:13,198][09273] Using port 40302 [2023-07-24 00:31:13,193][09281] Port 40902 is available [2023-07-24 00:31:13,199][09281] Using port 40902 [2023-07-24 00:31:13,230][09282] Port 41002 is available [2023-07-24 00:31:13,238][09282] Using port 41002 [2023-07-24 00:31:13,233][09278] Port 40802 is available [2023-07-24 00:31:13,241][09278] Using port 40802 [2023-07-24 00:31:13,241][09276] Port 40602 is available [2023-07-24 00:31:13,245][09276] Using port 40602 [2023-07-24 00:31:13,254][09274] Port 40402 is available [2023-07-24 00:31:13,258][09274] Using port 40402 [2023-07-24 00:31:13,390][09277] Port 40703 is available [2023-07-24 00:31:13,395][09277] Using port 40703 [2023-07-24 00:31:13,403][09277] Using port 40702 on host... [2023-07-24 00:31:13,415][09275] Port 40503 is available [2023-07-24 00:31:13,416][09275] Using port 40503 [2023-07-24 00:31:13,422][09281] Port 40903 is available [2023-07-24 00:31:13,422][09281] Using port 40903 [2023-07-24 00:31:13,428][09281] Using port 40902 on host... [2023-07-24 00:31:13,430][09275] Using port 40502 on host... [2023-07-24 00:31:13,432][09273] Port 40303 is available [2023-07-24 00:31:13,433][09273] Using port 40303 [2023-07-24 00:31:13,444][09273] Using port 40302 on host... [2023-07-24 00:31:13,492][09278] Port 40803 is available [2023-07-24 00:31:13,497][09278] Using port 40803 [2023-07-24 00:31:13,500][09274] Port 40403 is available [2023-07-24 00:31:13,504][09274] Using port 40403 [2023-07-24 00:31:13,507][09282] Port 41003 is available [2023-07-24 00:31:13,506][09278] Using port 40802 on host... [2023-07-24 00:31:13,508][09282] Using port 41003 [2023-07-24 00:31:13,512][09274] Using port 40402 on host... [2023-07-24 00:31:13,514][09282] Using port 41002 on host... [2023-07-24 00:31:13,521][09276] Port 40603 is available [2023-07-24 00:31:13,528][09276] Using port 40603 [2023-07-24 00:31:13,537][09276] Using port 40602 on host... [2023-07-24 00:31:13,806][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:31:15,122][09277] Initialized w:4 v:2 player:0 [2023-07-24 00:31:15,124][09277] Decorrelating experience for 64 frames... [2023-07-24 00:31:15,158][09281] Initialized w:6 v:2 player:0 [2023-07-24 00:31:15,160][09281] Decorrelating experience for 64 frames... [2023-07-24 00:31:15,166][09275] Initialized w:2 v:2 player:0 [2023-07-24 00:31:15,174][09275] Decorrelating experience for 64 frames... [2023-07-24 00:31:15,185][09273] Initialized w:0 v:2 player:0 [2023-07-24 00:31:15,191][09273] Decorrelating experience for 64 frames... [2023-07-24 00:31:15,215][09274] Initialized w:1 v:2 player:0 [2023-07-24 00:31:15,219][09278] Initialized w:5 v:2 player:0 [2023-07-24 00:31:15,218][09274] Decorrelating experience for 64 frames... [2023-07-24 00:31:15,221][09278] Decorrelating experience for 64 frames... [2023-07-24 00:31:15,232][09282] Initialized w:7 v:2 player:0 [2023-07-24 00:31:15,234][09282] Decorrelating experience for 64 frames... [2023-07-24 00:31:15,255][09276] Initialized w:3 v:2 player:0 [2023-07-24 00:31:15,258][09276] Decorrelating experience for 64 frames... [2023-07-24 00:31:15,809][09277] Using port 40703 on host... [2023-07-24 00:31:15,936][09281] Using port 40903 on host... [2023-07-24 00:31:15,948][09274] Using port 40403 on host... [2023-07-24 00:31:15,981][09278] Using port 40803 on host... [2023-07-24 00:31:15,984][09282] Using port 41003 on host... [2023-07-24 00:31:16,023][09276] Using port 40603 on host... [2023-07-24 00:31:16,048][09275] Using port 40503 on host... [2023-07-24 00:31:16,116][09273] Using port 40303 on host... [2023-07-24 00:31:18,081][09277] Initialized w:4 v:3 player:0 [2023-07-24 00:31:18,082][09277] Decorrelating experience for 96 frames... [2023-07-24 00:31:18,130][09281] Initialized w:6 v:3 player:0 [2023-07-24 00:31:18,134][09281] Decorrelating experience for 96 frames... [2023-07-24 00:31:18,192][09275] Initialized w:2 v:3 player:0 [2023-07-24 00:31:18,206][09275] Decorrelating experience for 96 frames... [2023-07-24 00:31:18,297][09273] Initialized w:0 v:3 player:0 [2023-07-24 00:31:18,304][09273] Decorrelating experience for 96 frames... [2023-07-24 00:31:18,731][09282] Initialized w:7 v:3 player:0 [2023-07-24 00:31:18,735][09282] Decorrelating experience for 96 frames... [2023-07-24 00:31:18,740][09278] Initialized w:5 v:3 player:0 [2023-07-24 00:31:18,743][09274] Initialized w:1 v:3 player:0 [2023-07-24 00:31:18,745][09274] Decorrelating experience for 96 frames... [2023-07-24 00:31:18,747][09278] Decorrelating experience for 96 frames... [2023-07-24 00:31:18,806][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:31:18,849][09276] Initialized w:3 v:3 player:0 [2023-07-24 00:31:18,866][09276] Decorrelating experience for 96 frames... [2023-07-24 00:31:23,806][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 52.3. Samples: 784. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:31:28,368][09259] Signal inference workers to stop experience collection... [2023-07-24 00:31:28,404][09272] InferenceWorker_p0-w0: stopping experience collection [2023-07-24 00:31:28,806][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 79.5. Samples: 1590. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:31:32,808][09259] Signal inference workers to resume experience collection... [2023-07-24 00:31:32,809][09272] InferenceWorker_p0-w0: resuming experience collection [2023-07-24 00:31:33,806][00294] Fps is (10 sec: 409.6, 60 sec: 163.8, 300 sec: 163.8). Total num frames: 4096. Throughput: 0: 91.8. Samples: 2296. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-07-24 00:31:38,810][00294] Fps is (10 sec: 1228.3, 60 sec: 409.5, 300 sec: 409.5). Total num frames: 12288. Throughput: 0: 132.6. Samples: 3980. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-07-24 00:31:42,118][00294] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 294], exiting... [2023-07-24 00:31:42,126][09259] Stopping Batcher_0... [2023-07-24 00:31:42,120][00294] Runner profile tree view: main_loop: 55.3536 [2023-07-24 00:31:42,127][09259] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000004_16384.pth... [2023-07-24 00:31:42,128][00294] Collected {0: 16384}, FPS: 296.0 [2023-07-24 00:31:42,128][09259] Loop batcher_evt_loop terminating... [2023-07-24 00:31:42,161][09276] EvtLoop [rollout_proc3_evt_loop, process=rollout_proc3] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance3'), args=(1, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:31:42,216][09276] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc3_evt_loop [2023-07-24 00:31:42,195][09278] EvtLoop [rollout_proc5_evt_loop, process=rollout_proc5] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance5'), args=(0, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:31:42,232][09278] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc5_evt_loop [2023-07-24 00:31:42,196][09274] EvtLoop [rollout_proc1_evt_loop, process=rollout_proc1] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance1'), args=(0, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:31:42,259][09274] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc1_evt_loop [2023-07-24 00:31:42,211][09282] EvtLoop [rollout_proc7_evt_loop, process=rollout_proc7] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance7'), args=(1, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:31:42,264][09282] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc7_evt_loop [2023-07-24 00:31:42,328][09272] Weights refcount: 2 0 [2023-07-24 00:31:42,344][09272] Stopping InferenceWorker_p0-w0... [2023-07-24 00:31:42,344][09272] Loop inference_proc0-0_evt_loop terminating... [2023-07-24 00:31:42,417][09259] Stopping LearnerWorker_p0... [2023-07-24 00:31:42,417][09259] Loop learner_proc0_evt_loop terminating... [2023-07-24 00:31:42,369][09277] EvtLoop [rollout_proc4_evt_loop, process=rollout_proc4] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance4'), args=(1, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:31:42,425][09277] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc4_evt_loop [2023-07-24 00:31:42,376][09275] EvtLoop [rollout_proc2_evt_loop, process=rollout_proc2] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance2'), args=(0, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:31:42,460][09275] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc2_evt_loop [2023-07-24 00:31:42,362][09273] EvtLoop [rollout_proc0_evt_loop, process=rollout_proc0] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance0'), args=(1, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:31:42,475][09273] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc0_evt_loop [2023-07-24 00:31:42,352][09281] EvtLoop [rollout_proc6_evt_loop, process=rollout_proc6] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance6'), args=(1, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:31:42,477][09281] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc6_evt_loop [2023-07-24 00:32:22,596][00294] Environment doom_basic already registered, overwriting... [2023-07-24 00:32:22,598][00294] Environment doom_two_colors_easy already registered, overwriting... [2023-07-24 00:32:22,600][00294] Environment doom_two_colors_hard already registered, overwriting... [2023-07-24 00:32:22,601][00294] Environment doom_dm already registered, overwriting... [2023-07-24 00:32:22,603][00294] Environment doom_dwango5 already registered, overwriting... [2023-07-24 00:32:22,604][00294] Environment doom_my_way_home_flat_actions already registered, overwriting... [2023-07-24 00:32:22,605][00294] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2023-07-24 00:32:22,607][00294] Environment doom_my_way_home already registered, overwriting... [2023-07-24 00:32:22,608][00294] Environment doom_deadly_corridor already registered, overwriting... [2023-07-24 00:32:22,609][00294] Environment doom_defend_the_center already registered, overwriting... [2023-07-24 00:32:22,610][00294] Environment doom_defend_the_line already registered, overwriting... [2023-07-24 00:32:22,612][00294] Environment doom_health_gathering already registered, overwriting... [2023-07-24 00:32:22,613][00294] Environment doom_health_gathering_supreme already registered, overwriting... [2023-07-24 00:32:22,614][00294] Environment doom_battle already registered, overwriting... [2023-07-24 00:32:22,616][00294] Environment doom_battle2 already registered, overwriting... [2023-07-24 00:32:22,617][00294] Environment doom_duel_bots already registered, overwriting... [2023-07-24 00:32:22,618][00294] Environment doom_deathmatch_bots already registered, overwriting... [2023-07-24 00:32:22,619][00294] Environment doom_duel already registered, overwriting... [2023-07-24 00:32:22,621][00294] Environment doom_deathmatch_full already registered, overwriting... [2023-07-24 00:32:22,622][00294] Environment doom_benchmark already registered, overwriting... [2023-07-24 00:32:22,623][00294] register_encoder_factory: [2023-07-24 00:32:22,649][00294] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-07-24 00:32:22,650][00294] Overriding arg 'train_for_env_steps' with value 6000000 passed from command line [2023-07-24 00:32:22,662][00294] Experiment dir /content/train_dir/default_experiment already exists! [2023-07-24 00:32:22,663][00294] Resuming existing experiment from /content/train_dir/default_experiment... [2023-07-24 00:32:22,664][00294] Weights and Biases integration disabled [2023-07-24 00:32:22,668][00294] Environment var CUDA_VISIBLE_DEVICES is 0 [2023-07-24 00:32:25,371][00294] Starting experiment with the following configuration: help=False algo=APPO env=doom_deathmatch_bots experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=gpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=4 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=6000000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=3 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_deathmatch_bots --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 cli_args={'env': 'doom_deathmatch_bots', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} git_hash=unknown git_repo_name=not a git repository [2023-07-24 00:32:25,374][00294] Saving configuration to /content/train_dir/default_experiment/config.json... [2023-07-24 00:32:25,382][00294] Rollout worker 0 uses device cpu [2023-07-24 00:32:25,383][00294] Rollout worker 1 uses device cpu [2023-07-24 00:32:25,386][00294] Rollout worker 2 uses device cpu [2023-07-24 00:32:25,387][00294] Rollout worker 3 uses device cpu [2023-07-24 00:32:25,389][00294] Rollout worker 4 uses device cpu [2023-07-24 00:32:25,391][00294] Rollout worker 5 uses device cpu [2023-07-24 00:32:25,392][00294] Rollout worker 6 uses device cpu [2023-07-24 00:32:25,394][00294] Rollout worker 7 uses device cpu [2023-07-24 00:32:25,534][00294] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:32:25,536][00294] InferenceWorker_p0-w0: min num requests: 2 [2023-07-24 00:32:25,579][00294] Starting all processes... [2023-07-24 00:32:25,581][00294] Starting process learner_proc0 [2023-07-24 00:32:25,652][00294] Starting all processes... [2023-07-24 00:32:25,666][00294] Starting process inference_proc0-0 [2023-07-24 00:32:25,666][00294] Starting process rollout_proc0 [2023-07-24 00:32:25,668][00294] Starting process rollout_proc1 [2023-07-24 00:32:25,668][00294] Starting process rollout_proc2 [2023-07-24 00:32:25,668][00294] Starting process rollout_proc3 [2023-07-24 00:32:25,668][00294] Starting process rollout_proc4 [2023-07-24 00:32:25,668][00294] Starting process rollout_proc5 [2023-07-24 00:32:25,668][00294] Starting process rollout_proc6 [2023-07-24 00:32:25,668][00294] Starting process rollout_proc7 [2023-07-24 00:32:42,538][13861] Worker 1 uses CPU cores [1] [2023-07-24 00:32:42,701][13866] Worker 6 uses CPU cores [0] [2023-07-24 00:32:42,936][13867] Worker 7 uses CPU cores [1] [2023-07-24 00:32:42,944][13846] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:32:42,945][13846] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-07-24 00:32:42,993][13846] Num visible devices: 1 [2023-07-24 00:32:43,016][13846] Starting seed is not provided [2023-07-24 00:32:43,017][13846] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:32:43,018][13846] Initializing actor-critic model on device cuda:0 [2023-07-24 00:32:43,019][13846] RunningMeanStd input shape: (23,) [2023-07-24 00:32:43,021][13846] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:32:43,022][13846] RunningMeanStd input shape: (1,) [2023-07-24 00:32:43,186][13846] ConvEncoder: input_channels=3 [2023-07-24 00:32:43,220][13863] Worker 3 uses CPU cores [1] [2023-07-24 00:32:43,243][13859] Worker 0 uses CPU cores [0] [2023-07-24 00:32:43,399][13865] Worker 5 uses CPU cores [1] [2023-07-24 00:32:43,468][13864] Worker 4 uses CPU cores [0] [2023-07-24 00:32:43,491][13860] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:32:43,491][13860] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-07-24 00:32:43,520][13862] Worker 2 uses CPU cores [0] [2023-07-24 00:32:43,528][13860] Num visible devices: 1 [2023-07-24 00:32:43,657][13846] Conv encoder output size: 512 [2023-07-24 00:32:43,659][13846] Policy head output size: 640 [2023-07-24 00:32:43,691][13846] Created Actor Critic model with architecture: [2023-07-24 00:32:43,691][13846] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (measurements): RunningMeanStdInPlace() (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) (measurements_head): Sequential( (0): Linear(in_features=23, out_features=128, bias=True) (1): ELU(alpha=1.0) (2): Linear(in_features=128, out_features=128, bias=True) (3): ELU(alpha=1.0) ) ) (core): ModelCoreRNN( (core): GRU(640, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=39, bias=True) ) ) [2023-07-24 00:32:45,524][00294] Heartbeat connected on Batcher_0 [2023-07-24 00:32:45,535][00294] Heartbeat connected on InferenceWorker_p0-w0 [2023-07-24 00:32:45,548][00294] Heartbeat connected on RolloutWorker_w0 [2023-07-24 00:32:45,549][00294] Heartbeat connected on RolloutWorker_w1 [2023-07-24 00:32:45,553][00294] Heartbeat connected on RolloutWorker_w2 [2023-07-24 00:32:45,558][00294] Heartbeat connected on RolloutWorker_w3 [2023-07-24 00:32:45,568][00294] Heartbeat connected on RolloutWorker_w5 [2023-07-24 00:32:45,572][00294] Heartbeat connected on RolloutWorker_w4 [2023-07-24 00:32:45,576][00294] Heartbeat connected on RolloutWorker_w6 [2023-07-24 00:32:45,583][00294] Heartbeat connected on RolloutWorker_w7 [2023-07-24 00:32:46,374][13846] Using optimizer [2023-07-24 00:32:46,375][13846] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000004_16384.pth... [2023-07-24 00:32:46,411][13846] Loading model from checkpoint [2023-07-24 00:32:46,417][13846] Loaded experiment state at self.train_step=4, self.env_steps=16384 [2023-07-24 00:32:46,417][13846] Initialized policy 0 weights for model version 4 [2023-07-24 00:32:46,420][13846] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:32:46,427][13846] LearnerWorker_p0 finished initialization! [2023-07-24 00:32:46,428][00294] Heartbeat connected on LearnerWorker_p0 [2023-07-24 00:32:46,525][13860] RunningMeanStd input shape: (23,) [2023-07-24 00:32:46,526][13860] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:32:46,527][13860] RunningMeanStd input shape: (1,) [2023-07-24 00:32:46,541][13860] ConvEncoder: input_channels=3 [2023-07-24 00:32:46,651][13860] Conv encoder output size: 512 [2023-07-24 00:32:46,653][13860] Policy head output size: 640 [2023-07-24 00:32:46,725][00294] Inference worker 0-0 is ready! [2023-07-24 00:32:46,726][00294] All inference workers are ready! Signal rollout workers to start! [2023-07-24 00:32:47,001][13862] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:32:47,003][13865] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:32:47,005][13867] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:32:47,006][13863] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:32:47,004][13861] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:32:47,004][13859] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:32:47,013][13864] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:32:47,014][13866] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:32:47,016][13865] Port 40800 is available [2023-07-24 00:32:47,017][13867] Port 41000 is available [2023-07-24 00:32:47,019][13861] Port 40400 is available [2023-07-24 00:32:47,016][13865] Using port 40800 [2023-07-24 00:32:47,019][13861] Using port 40400 [2023-07-24 00:32:47,020][13863] Port 40600 is available [2023-07-24 00:32:47,018][13867] Using port 41000 [2023-07-24 00:32:47,021][13863] Using port 40600 [2023-07-24 00:32:47,027][13862] Port 40500 is available [2023-07-24 00:32:47,027][13862] Using port 40500 [2023-07-24 00:32:47,030][13859] Port 40300 is available [2023-07-24 00:32:47,045][13859] Using port 40300 [2023-07-24 00:32:47,038][13864] Port 40700 is available [2023-07-24 00:32:47,051][13864] Using port 40700 [2023-07-24 00:32:47,034][13866] Port 40900 is available [2023-07-24 00:32:47,055][13866] Using port 40900 [2023-07-24 00:32:47,254][13861] Port 40401 is available [2023-07-24 00:32:47,256][13867] Port 41001 is available [2023-07-24 00:32:47,259][13865] Port 40801 is available [2023-07-24 00:32:47,257][13867] Using port 41001 [2023-07-24 00:32:47,255][13861] Using port 40401 [2023-07-24 00:32:47,260][13865] Using port 40801 [2023-07-24 00:32:47,263][13863] Port 40601 is available [2023-07-24 00:32:47,264][13863] Using port 40601 [2023-07-24 00:32:47,269][13867] Using port 41000 on host... [2023-07-24 00:32:47,266][13861] Using port 40400 on host... [2023-07-24 00:32:47,268][13865] Using port 40800 on host... [2023-07-24 00:32:47,276][13863] Using port 40600 on host... [2023-07-24 00:32:47,301][13862] Port 40501 is available [2023-07-24 00:32:47,313][13862] Using port 40501 [2023-07-24 00:32:47,322][13864] Port 40701 is available [2023-07-24 00:32:47,319][13859] Port 40301 is available [2023-07-24 00:32:47,323][13864] Using port 40701 [2023-07-24 00:32:47,323][13859] Using port 40301 [2023-07-24 00:32:47,316][13866] Port 40901 is available [2023-07-24 00:32:47,329][13866] Using port 40901 [2023-07-24 00:32:47,327][13862] Using port 40500 on host... [2023-07-24 00:32:47,333][13864] Using port 40700 on host... [2023-07-24 00:32:47,336][13859] Using port 40300 on host... [2023-07-24 00:32:47,335][13866] Using port 40900 on host... [2023-07-24 00:32:47,668][00294] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 16384. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:32:48,882][13867] Initialized w:7 v:0 player:0 [2023-07-24 00:32:48,883][13863] Initialized w:3 v:0 player:0 [2023-07-24 00:32:48,885][13867] Decorrelating experience for 0 frames... [2023-07-24 00:32:48,886][13865] Initialized w:5 v:0 player:0 [2023-07-24 00:32:48,891][13863] Decorrelating experience for 0 frames... [2023-07-24 00:32:48,892][13861] Initialized w:1 v:0 player:0 [2023-07-24 00:32:48,895][13865] Decorrelating experience for 0 frames... [2023-07-24 00:32:48,899][13867] Using port 41001 on host... [2023-07-24 00:32:48,900][13863] Using port 40601 on host... [2023-07-24 00:32:48,897][13861] Decorrelating experience for 0 frames... [2023-07-24 00:32:48,901][13865] Using port 40801 on host... [2023-07-24 00:32:48,904][13861] Using port 40401 on host... [2023-07-24 00:32:48,996][13859] Initialized w:0 v:0 player:0 [2023-07-24 00:32:49,005][13864] Initialized w:4 v:0 player:0 [2023-07-24 00:32:49,006][13862] Initialized w:2 v:0 player:0 [2023-07-24 00:32:49,011][13866] Initialized w:6 v:0 player:0 [2023-07-24 00:32:49,004][13859] Decorrelating experience for 0 frames... [2023-07-24 00:32:49,017][13862] Decorrelating experience for 0 frames... [2023-07-24 00:32:49,018][13864] Decorrelating experience for 0 frames... [2023-07-24 00:32:49,014][13866] Decorrelating experience for 0 frames... [2023-07-24 00:32:49,019][13859] Using port 40301 on host... [2023-07-24 00:32:49,021][13862] Using port 40501 on host... [2023-07-24 00:32:49,025][13864] Using port 40701 on host... [2023-07-24 00:32:49,023][13866] Using port 40901 on host... [2023-07-24 00:32:50,490][13867] Initialized w:7 v:1 player:0 [2023-07-24 00:32:50,492][13863] Initialized w:3 v:1 player:0 [2023-07-24 00:32:50,495][13861] Initialized w:1 v:1 player:0 [2023-07-24 00:32:50,497][13867] Decorrelating experience for 32 frames... [2023-07-24 00:32:50,499][13863] Decorrelating experience for 32 frames... [2023-07-24 00:32:50,501][13861] Decorrelating experience for 32 frames... [2023-07-24 00:32:50,502][13865] Initialized w:5 v:1 player:0 [2023-07-24 00:32:50,509][13865] Decorrelating experience for 32 frames... [2023-07-24 00:32:50,695][13866] Initialized w:6 v:1 player:0 [2023-07-24 00:32:50,704][13862] Initialized w:2 v:1 player:0 [2023-07-24 00:32:50,702][13866] Decorrelating experience for 32 frames... [2023-07-24 00:32:50,708][13864] Initialized w:4 v:1 player:0 [2023-07-24 00:32:50,712][13859] Initialized w:0 v:1 player:0 [2023-07-24 00:32:50,718][13862] Decorrelating experience for 32 frames... [2023-07-24 00:32:50,716][13859] Decorrelating experience for 32 frames... [2023-07-24 00:32:50,714][13864] Decorrelating experience for 32 frames... [2023-07-24 00:32:51,246][13863] Port 40602 is available [2023-07-24 00:32:51,239][13867] Port 41002 is available [2023-07-24 00:32:51,247][13867] Using port 41002 [2023-07-24 00:32:51,247][13863] Using port 40602 [2023-07-24 00:32:51,260][13861] Port 40402 is available [2023-07-24 00:32:51,261][13861] Using port 40402 [2023-07-24 00:32:51,267][13865] Port 40802 is available [2023-07-24 00:32:51,267][13865] Using port 40802 [2023-07-24 00:32:51,474][13859] Port 40302 is available [2023-07-24 00:32:51,474][13859] Using port 40302 [2023-07-24 00:32:51,485][13867] Port 41003 is available [2023-07-24 00:32:51,484][13864] Port 40702 is available [2023-07-24 00:32:51,486][13867] Using port 41003 [2023-07-24 00:32:51,489][13864] Using port 40702 [2023-07-24 00:32:51,492][13862] Port 40502 is available [2023-07-24 00:32:51,493][13862] Using port 40502 [2023-07-24 00:32:51,496][13866] Port 40902 is available [2023-07-24 00:32:51,496][13866] Using port 40902 [2023-07-24 00:32:51,494][13863] Port 40603 is available [2023-07-24 00:32:51,499][13863] Using port 40603 [2023-07-24 00:32:51,501][13867] Using port 41002 on host... [2023-07-24 00:32:51,498][13861] Port 40403 is available [2023-07-24 00:32:51,504][13861] Using port 40403 [2023-07-24 00:32:51,506][13865] Port 40803 is available [2023-07-24 00:32:51,509][13863] Using port 40602 on host... [2023-07-24 00:32:51,511][13865] Using port 40803 [2023-07-24 00:32:51,517][13861] Using port 40402 on host... [2023-07-24 00:32:51,521][13865] Using port 40802 on host... [2023-07-24 00:32:51,704][13859] Port 40303 is available [2023-07-24 00:32:51,706][13859] Using port 40303 [2023-07-24 00:32:51,711][13859] Using port 40302 on host... [2023-07-24 00:32:51,713][13864] Port 40703 is available [2023-07-24 00:32:51,717][13864] Using port 40703 [2023-07-24 00:32:51,722][13866] Port 40903 is available [2023-07-24 00:32:51,723][13866] Using port 40903 [2023-07-24 00:32:51,720][13862] Port 40503 is available [2023-07-24 00:32:51,728][13864] Using port 40702 on host... [2023-07-24 00:32:51,727][13862] Using port 40503 [2023-07-24 00:32:51,732][13866] Using port 40902 on host... [2023-07-24 00:32:51,731][13862] Using port 40502 on host... [2023-07-24 00:32:52,668][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 16384. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:32:53,135][13863] Initialized w:3 v:2 player:0 [2023-07-24 00:32:53,139][13867] Initialized w:7 v:2 player:0 [2023-07-24 00:32:53,137][13863] Decorrelating experience for 64 frames... [2023-07-24 00:32:53,143][13867] Decorrelating experience for 64 frames... [2023-07-24 00:32:53,150][13865] Initialized w:5 v:2 player:0 [2023-07-24 00:32:53,156][13865] Decorrelating experience for 64 frames... [2023-07-24 00:32:53,165][13861] Initialized w:1 v:2 player:0 [2023-07-24 00:32:53,170][13861] Decorrelating experience for 64 frames... [2023-07-24 00:32:53,422][13866] Initialized w:6 v:2 player:0 [2023-07-24 00:32:53,428][13859] Initialized w:0 v:2 player:0 [2023-07-24 00:32:53,426][13864] Initialized w:4 v:2 player:0 [2023-07-24 00:32:53,431][13862] Initialized w:2 v:2 player:0 [2023-07-24 00:32:53,436][13859] Decorrelating experience for 64 frames... [2023-07-24 00:32:53,438][13862] Decorrelating experience for 64 frames... [2023-07-24 00:32:53,427][13866] Decorrelating experience for 64 frames... [2023-07-24 00:32:53,445][13864] Decorrelating experience for 64 frames... [2023-07-24 00:32:53,824][13863] Using port 40603 on host... [2023-07-24 00:32:53,826][13867] Using port 41003 on host... [2023-07-24 00:32:53,844][13865] Using port 40803 on host... [2023-07-24 00:32:53,854][13861] Using port 40403 on host... [2023-07-24 00:32:54,111][13866] Using port 40903 on host... [2023-07-24 00:32:54,130][13859] Using port 40303 on host... [2023-07-24 00:32:54,127][13864] Using port 40703 on host... [2023-07-24 00:32:54,139][13862] Using port 40503 on host... [2023-07-24 00:32:55,886][13863] Initialized w:3 v:3 player:0 [2023-07-24 00:32:55,904][13863] Decorrelating experience for 96 frames... [2023-07-24 00:32:55,920][13865] Initialized w:5 v:3 player:0 [2023-07-24 00:32:55,922][13865] Decorrelating experience for 96 frames... [2023-07-24 00:32:55,926][13867] Initialized w:7 v:3 player:0 [2023-07-24 00:32:55,943][13867] Decorrelating experience for 96 frames... [2023-07-24 00:32:55,985][13861] Initialized w:1 v:3 player:0 [2023-07-24 00:32:55,988][13861] Decorrelating experience for 96 frames... [2023-07-24 00:32:56,181][13859] Initialized w:0 v:3 player:0 [2023-07-24 00:32:56,183][13859] Decorrelating experience for 96 frames... [2023-07-24 00:32:56,194][13864] Initialized w:4 v:3 player:0 [2023-07-24 00:32:56,204][13866] Initialized w:6 v:3 player:0 [2023-07-24 00:32:56,219][13864] Decorrelating experience for 96 frames... [2023-07-24 00:32:56,230][13866] Decorrelating experience for 96 frames... [2023-07-24 00:32:56,247][13862] Initialized w:2 v:3 player:0 [2023-07-24 00:32:56,260][13862] Decorrelating experience for 96 frames... [2023-07-24 00:32:57,668][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 16384. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:33:02,672][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 16384. Throughput: 0: 70.8. Samples: 1062. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:33:07,668][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 16384. Throughput: 0: 80.2. Samples: 1604. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:33:07,926][13846] Signal inference workers to stop experience collection... [2023-07-24 00:33:07,946][13860] InferenceWorker_p0-w0: stopping experience collection [2023-07-24 00:33:09,225][13846] Signal inference workers to resume experience collection... [2023-07-24 00:33:09,226][13860] InferenceWorker_p0-w0: resuming experience collection [2023-07-24 00:33:11,481][00294] Keyboard interrupt detected in the event loop EvtLoop [Runner_EvtLoop, process=main process 294], exiting... [2023-07-24 00:33:11,499][13846] Stopping Batcher_0... [2023-07-24 00:33:11,500][13846] Loop batcher_evt_loop terminating... [2023-07-24 00:33:11,496][00294] Runner profile tree view: main_loop: 45.9175 [2023-07-24 00:33:11,501][00294] Collected {0: 20480}, FPS: 89.2 [2023-07-24 00:33:11,573][13865] EvtLoop [rollout_proc5_evt_loop, process=rollout_proc5] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance5'), args=(0, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:33:11,594][13859] EvtLoop [rollout_proc0_evt_loop, process=rollout_proc0] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance0'), args=(1, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:33:11,615][00294] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-07-24 00:33:11,618][00294] Overriding arg 'num_workers' with value 1 passed from command line [2023-07-24 00:33:11,627][00294] Adding new argument 'no_render'=True that is not in the saved config file! [2023-07-24 00:33:11,630][00294] Adding new argument 'save_video'=True that is not in the saved config file! [2023-07-24 00:33:11,635][00294] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-07-24 00:33:11,638][13865] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc5_evt_loop [2023-07-24 00:33:11,637][00294] Adding new argument 'video_name'=None that is not in the saved config file! [2023-07-24 00:33:11,641][00294] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2023-07-24 00:33:11,570][13863] EvtLoop [rollout_proc3_evt_loop, process=rollout_proc3] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance3'), args=(1, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:33:11,645][13863] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc3_evt_loop [2023-07-24 00:33:11,580][13861] EvtLoop [rollout_proc1_evt_loop, process=rollout_proc1] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance1'), args=(0, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:33:11,650][13861] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc1_evt_loop [2023-07-24 00:33:11,645][00294] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-07-24 00:33:11,653][13867] EvtLoop [rollout_proc7_evt_loop, process=rollout_proc7] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance7'), args=(0, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:33:11,659][13867] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc7_evt_loop [2023-07-24 00:33:11,663][13864] EvtLoop [rollout_proc4_evt_loop, process=rollout_proc4] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance4'), args=(0, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:33:11,654][00294] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2023-07-24 00:33:11,715][13864] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc4_evt_loop [2023-07-24 00:33:11,715][00294] Adding new argument 'hf_repository'='Corianas/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2023-07-24 00:33:11,568][13862] EvtLoop [rollout_proc2_evt_loop, process=rollout_proc2] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance2'), args=(0, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:33:11,718][13862] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc2_evt_loop [2023-07-24 00:33:11,717][00294] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-07-24 00:33:11,726][00294] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-07-24 00:33:11,727][00294] Adding new argument 'train_script'=None that is not in the saved config file! [2023-07-24 00:33:11,728][00294] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-07-24 00:33:11,729][00294] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-07-24 00:33:11,674][13866] EvtLoop [rollout_proc6_evt_loop, process=rollout_proc6] unhandled exception in slot='advance_rollouts' connected to emitter=Emitter(object_id='InferenceWorker_p0-w0', signal_name='advance6'), args=(1, 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/signal_slot/signal_slot.py", line 355, in _process_signal slot_callable(*args) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/rollout_worker.py", line 241, in advance_rollouts complete_rollouts, episodic_stats = runner.advance_rollouts(policy_id, self.timing) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/sampling/non_batched_sampling.py", line 634, in advance_rollouts new_obs, rewards, terminated, truncated, infos = e.step(actions) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/algo/utils/make_env.py", line 115, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/reward_shaping.py", line 219, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/additional_input.py", line 96, in step obs, rew, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 508, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 117, in step observation, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sample_factory/envs/env_wrappers.py", line 86, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/gymnasium/core.py", line 447, in step return self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/wrappers/multiplayer_stats.py", line 54, in step obs, reward, terminated, truncated, info = self.env.step(action) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/multiplayer/doom_multiagent.py", line 204, in step return super().step(actions) File "/usr/local/lib/python3.10/dist-packages/sf_examples/vizdoom/doom/doom_gym.py", line 452, in step reward = self.game.make_action(actions_flattened, self.skip_frames) vizdoom.vizdoom.SignalException: Signal SIGINT received. ViZDoom instance has been closed. [2023-07-24 00:33:11,732][13866] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc6_evt_loop [2023-07-24 00:33:11,702][13860] Weights refcount: 2 0 [2023-07-24 00:33:11,708][13859] Unhandled exception Signal SIGINT received. ViZDoom instance has been closed. in evt loop rollout_proc0_evt_loop [2023-07-24 00:33:11,766][13860] Stopping InferenceWorker_p0-w0... [2023-07-24 00:33:11,767][13860] Loop inference_proc0-0_evt_loop terminating... [2023-07-24 00:33:11,902][00294] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:33:11,914][00294] Port 40300 is available [2023-07-24 00:33:11,916][00294] Using port 40300 [2023-07-24 00:33:11,935][00294] RunningMeanStd input shape: (23,) [2023-07-24 00:33:11,940][00294] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:33:11,948][00294] RunningMeanStd input shape: (1,) [2023-07-24 00:33:12,012][00294] ConvEncoder: input_channels=3 [2023-07-24 00:33:12,320][00294] Conv encoder output size: 512 [2023-07-24 00:33:12,327][00294] Policy head output size: 640 [2023-07-24 00:33:13,520][13846] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000006_24576.pth... [2023-07-24 00:33:13,686][13846] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth [2023-07-24 00:33:13,698][13846] Stopping LearnerWorker_p0... [2023-07-24 00:33:13,699][13846] Loop learner_proc0_evt_loop terminating... [2023-07-24 00:33:17,759][00294] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000006_24576.pth... [2023-07-24 00:33:17,844][00294] Using port 40300 on host... [2023-07-24 00:33:18,947][00294] Initialized w:0 v:0 player:0 [2023-07-24 00:33:20,708][00294] Num frames 100... [2023-07-24 00:33:21,047][00294] Num frames 200... [2023-07-24 00:33:21,395][00294] Num frames 300... [2023-07-24 00:33:21,731][00294] Num frames 400... [2023-07-24 00:33:22,076][00294] Num frames 500... [2023-07-24 00:33:22,424][00294] Num frames 600... [2023-07-24 00:33:22,777][00294] Num frames 700... [2023-07-24 00:33:23,020][00294] Num frames 800... [2023-07-24 00:33:23,255][00294] Num frames 900... [2023-07-24 00:33:23,486][00294] Num frames 1000... [2023-07-24 00:33:23,708][00294] Num frames 1100... [2023-07-24 00:33:23,936][00294] Num frames 1200... [2023-07-24 00:33:24,155][00294] Num frames 1300... [2023-07-24 00:33:24,392][00294] Num frames 1400... [2023-07-24 00:33:24,641][00294] Num frames 1500... [2023-07-24 00:33:24,865][00294] Num frames 1600... [2023-07-24 00:33:25,083][00294] Num frames 1700... [2023-07-24 00:33:25,328][00294] Num frames 1800... [2023-07-24 00:33:25,565][00294] Num frames 1900... [2023-07-24 00:33:25,796][00294] Num frames 2000... [2023-07-24 00:33:26,019][00294] Num frames 2100... [2023-07-24 00:33:26,242][00294] Num frames 2200... [2023-07-24 00:33:26,474][00294] Num frames 2300... [2023-07-24 00:33:26,706][00294] Num frames 2400... [2023-07-24 00:33:26,996][00294] Num frames 2500... [2023-07-24 00:33:27,228][00294] Num frames 2600... [2023-07-24 00:33:27,452][00294] Num frames 2700... [2023-07-24 00:33:27,814][00294] Num frames 2800... [2023-07-24 00:33:28,186][00294] Num frames 2900... [2023-07-24 00:33:28,411][00294] Num frames 3000... [2023-07-24 00:33:28,625][00294] Num frames 3100... [2023-07-24 00:33:28,844][00294] Num frames 3200... [2023-07-24 00:33:29,067][00294] Num frames 3300... [2023-07-24 00:33:29,284][00294] Num frames 3400... [2023-07-24 00:33:29,519][00294] Num frames 3500... [2023-07-24 00:33:29,747][00294] Num frames 3600... [2023-07-24 00:33:29,975][00294] Num frames 3700... [2023-07-24 00:33:30,196][00294] Num frames 3800... [2023-07-24 00:33:30,418][00294] Num frames 3900... [2023-07-24 00:33:30,638][00294] Num frames 4000... [2023-07-24 00:33:30,864][00294] Num frames 4100... [2023-07-24 00:33:31,083][00294] Num frames 4200... [2023-07-24 00:33:31,314][00294] Num frames 4300... [2023-07-24 00:33:31,545][00294] Num frames 4400... [2023-07-24 00:33:31,759][00294] Num frames 4500... [2023-07-24 00:33:31,981][00294] Num frames 4600... [2023-07-24 00:33:32,199][00294] Num frames 4700... [2023-07-24 00:33:32,429][00294] Num frames 4800... [2023-07-24 00:33:32,659][00294] Num frames 4900... [2023-07-24 00:33:32,937][00294] Num frames 5000... [2023-07-24 00:33:33,275][00294] Num frames 5100... [2023-07-24 00:33:33,620][00294] Num frames 5200... [2023-07-24 00:33:33,962][00294] Num frames 5300... [2023-07-24 00:33:34,299][00294] Num frames 5400... [2023-07-24 00:33:34,643][00294] Num frames 5500... [2023-07-24 00:33:34,965][00294] Num frames 5600... [2023-07-24 00:33:35,306][00294] Num frames 5700... [2023-07-24 00:33:35,647][00294] Num frames 5800... [2023-07-24 00:33:35,992][00294] Num frames 5900... [2023-07-24 00:33:39,936][00294] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-07-24 00:33:39,937][00294] Overriding arg 'num_workers' with value 1 passed from command line [2023-07-24 00:33:39,940][00294] Adding new argument 'no_render'=True that is not in the saved config file! [2023-07-24 00:33:39,942][00294] Adding new argument 'save_video'=True that is not in the saved config file! [2023-07-24 00:33:39,946][00294] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-07-24 00:33:39,948][00294] Adding new argument 'video_name'=None that is not in the saved config file! [2023-07-24 00:33:39,950][00294] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2023-07-24 00:33:39,951][00294] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-07-24 00:33:39,952][00294] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2023-07-24 00:33:39,954][00294] Adding new argument 'hf_repository'='Corianas/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2023-07-24 00:33:39,955][00294] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-07-24 00:33:39,960][00294] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-07-24 00:33:39,961][00294] Adding new argument 'train_script'=None that is not in the saved config file! [2023-07-24 00:33:39,962][00294] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-07-24 00:33:39,963][00294] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-07-24 00:33:40,006][00294] Port 40300 is available [2023-07-24 00:33:40,008][00294] Using port 40300 [2023-07-24 00:33:40,012][00294] RunningMeanStd input shape: (23,) [2023-07-24 00:33:40,013][00294] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:33:40,016][00294] RunningMeanStd input shape: (1,) [2023-07-24 00:33:40,032][00294] ConvEncoder: input_channels=3 [2023-07-24 00:33:40,069][00294] Conv encoder output size: 512 [2023-07-24 00:33:40,072][00294] Policy head output size: 640 [2023-07-24 00:33:40,095][00294] No checkpoints found [2023-07-24 00:33:59,547][00294] Environment doom_basic already registered, overwriting... [2023-07-24 00:33:59,550][00294] Environment doom_two_colors_easy already registered, overwriting... [2023-07-24 00:33:59,552][00294] Environment doom_two_colors_hard already registered, overwriting... [2023-07-24 00:33:59,555][00294] Environment doom_dm already registered, overwriting... [2023-07-24 00:33:59,558][00294] Environment doom_dwango5 already registered, overwriting... [2023-07-24 00:33:59,559][00294] Environment doom_my_way_home_flat_actions already registered, overwriting... [2023-07-24 00:33:59,561][00294] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2023-07-24 00:33:59,562][00294] Environment doom_my_way_home already registered, overwriting... [2023-07-24 00:33:59,563][00294] Environment doom_deadly_corridor already registered, overwriting... [2023-07-24 00:33:59,564][00294] Environment doom_defend_the_center already registered, overwriting... [2023-07-24 00:33:59,565][00294] Environment doom_defend_the_line already registered, overwriting... [2023-07-24 00:33:59,566][00294] Environment doom_health_gathering already registered, overwriting... [2023-07-24 00:33:59,567][00294] Environment doom_health_gathering_supreme already registered, overwriting... [2023-07-24 00:33:59,568][00294] Environment doom_battle already registered, overwriting... [2023-07-24 00:33:59,570][00294] Environment doom_battle2 already registered, overwriting... [2023-07-24 00:33:59,572][00294] Environment doom_duel_bots already registered, overwriting... [2023-07-24 00:33:59,573][00294] Environment doom_deathmatch_bots already registered, overwriting... [2023-07-24 00:33:59,574][00294] Environment doom_duel already registered, overwriting... [2023-07-24 00:33:59,575][00294] Environment doom_deathmatch_full already registered, overwriting... [2023-07-24 00:33:59,576][00294] Environment doom_benchmark already registered, overwriting... [2023-07-24 00:33:59,578][00294] register_encoder_factory: [2023-07-24 00:33:59,605][00294] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-07-24 00:33:59,607][00294] Overriding arg 'num_envs_per_worker' with value 8 passed from command line [2023-07-24 00:33:59,620][00294] Experiment dir /content/train_dir/default_experiment already exists! [2023-07-24 00:33:59,621][00294] Resuming existing experiment from /content/train_dir/default_experiment... [2023-07-24 00:33:59,623][00294] Weights and Biases integration disabled [2023-07-24 00:33:59,628][00294] Environment var CUDA_VISIBLE_DEVICES is 0 [2023-07-24 00:34:02,714][00294] Starting experiment with the following configuration: help=False algo=APPO env=doom_deathmatch_bots experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=gpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=8 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=6000000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=3 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_deathmatch_bots --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 cli_args={'env': 'doom_deathmatch_bots', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} git_hash=unknown git_repo_name=not a git repository [2023-07-24 00:34:02,719][00294] Saving configuration to /content/train_dir/default_experiment/config.json... [2023-07-24 00:34:02,722][00294] Rollout worker 0 uses device cpu [2023-07-24 00:34:02,724][00294] Rollout worker 1 uses device cpu [2023-07-24 00:34:02,726][00294] Rollout worker 2 uses device cpu [2023-07-24 00:34:02,727][00294] Rollout worker 3 uses device cpu [2023-07-24 00:34:02,728][00294] Rollout worker 4 uses device cpu [2023-07-24 00:34:02,729][00294] Rollout worker 5 uses device cpu [2023-07-24 00:34:02,730][00294] Rollout worker 6 uses device cpu [2023-07-24 00:34:02,732][00294] Rollout worker 7 uses device cpu [2023-07-24 00:34:02,948][00294] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:34:02,951][00294] InferenceWorker_p0-w0: min num requests: 2 [2023-07-24 00:34:02,992][00294] Starting all processes... [2023-07-24 00:34:02,994][00294] Starting process learner_proc0 [2023-07-24 00:34:03,070][00294] Starting all processes... [2023-07-24 00:34:03,090][00294] Starting process inference_proc0-0 [2023-07-24 00:34:03,091][00294] Starting process rollout_proc0 [2023-07-24 00:34:03,094][00294] Starting process rollout_proc1 [2023-07-24 00:34:03,094][00294] Starting process rollout_proc2 [2023-07-24 00:34:03,094][00294] Starting process rollout_proc3 [2023-07-24 00:34:03,094][00294] Starting process rollout_proc4 [2023-07-24 00:34:03,094][00294] Starting process rollout_proc5 [2023-07-24 00:34:03,094][00294] Starting process rollout_proc6 [2023-07-24 00:34:03,100][00294] Starting process rollout_proc7 [2023-07-24 00:34:20,373][14525] Worker 0 uses CPU cores [0] [2023-07-24 00:34:20,704][14532] Worker 7 uses CPU cores [1] [2023-07-24 00:34:20,887][14526] Worker 2 uses CPU cores [0] [2023-07-24 00:34:20,905][14528] Worker 3 uses CPU cores [1] [2023-07-24 00:34:20,991][14524] Worker 1 uses CPU cores [1] [2023-07-24 00:34:21,013][14531] Worker 5 uses CPU cores [1] [2023-07-24 00:34:21,036][14511] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:34:21,037][14511] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2023-07-24 00:34:21,038][14530] Worker 6 uses CPU cores [0] [2023-07-24 00:34:21,070][14527] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:34:21,071][14527] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2023-07-24 00:34:21,076][14511] Num visible devices: 1 [2023-07-24 00:34:21,089][14529] Worker 4 uses CPU cores [0] [2023-07-24 00:34:21,097][14511] Starting seed is not provided [2023-07-24 00:34:21,098][14511] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:34:21,098][14511] Initializing actor-critic model on device cuda:0 [2023-07-24 00:34:21,098][14511] RunningMeanStd input shape: (23,) [2023-07-24 00:34:21,099][14511] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:34:21,100][14511] RunningMeanStd input shape: (1,) [2023-07-24 00:34:21,110][14527] Num visible devices: 1 [2023-07-24 00:34:21,121][14511] ConvEncoder: input_channels=3 [2023-07-24 00:34:21,305][14511] Conv encoder output size: 512 [2023-07-24 00:34:21,307][14511] Policy head output size: 640 [2023-07-24 00:34:21,340][14511] Created Actor Critic model with architecture: [2023-07-24 00:34:21,341][14511] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (measurements): RunningMeanStdInPlace() (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) (measurements_head): Sequential( (0): Linear(in_features=23, out_features=128, bias=True) (1): ELU(alpha=1.0) (2): Linear(in_features=128, out_features=128, bias=True) (3): ELU(alpha=1.0) ) ) (core): ModelCoreRNN( (core): GRU(640, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=39, bias=True) ) ) [2023-07-24 00:34:21,467][14511] Using optimizer [2023-07-24 00:34:21,468][14511] No checkpoints found [2023-07-24 00:34:21,468][14511] Did not load from checkpoint, starting from scratch! [2023-07-24 00:34:21,469][14511] Initialized policy 0 weights for model version 0 [2023-07-24 00:34:21,471][14511] LearnerWorker_p0 finished initialization! [2023-07-24 00:34:21,472][14511] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2023-07-24 00:34:21,565][14527] RunningMeanStd input shape: (23,) [2023-07-24 00:34:21,566][14527] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 00:34:21,567][14527] RunningMeanStd input shape: (1,) [2023-07-24 00:34:21,580][14527] ConvEncoder: input_channels=3 [2023-07-24 00:34:21,685][14527] Conv encoder output size: 512 [2023-07-24 00:34:21,686][14527] Policy head output size: 640 [2023-07-24 00:34:21,753][00294] Inference worker 0-0 is ready! [2023-07-24 00:34:21,755][00294] All inference workers are ready! Signal rollout workers to start! [2023-07-24 00:34:22,009][14529] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:34:22,011][14525] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:34:22,014][14530] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:34:22,015][14526] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:34:22,021][14529] Port 40700 is available [2023-07-24 00:34:22,025][14525] Port 40300 is available [2023-07-24 00:34:22,027][14530] Port 40900 is available [2023-07-24 00:34:22,022][14529] Using port 40700 [2023-07-24 00:34:22,030][14526] Port 40500 is available [2023-07-24 00:34:22,032][14524] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:34:22,030][14526] Using port 40500 [2023-07-24 00:34:22,028][14530] Using port 40900 [2023-07-24 00:34:22,026][14525] Using port 40300 [2023-07-24 00:34:22,037][14531] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:34:22,038][14532] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:34:22,041][14528] Doom resolution: 160x120, resize resolution: (128, 72) [2023-07-24 00:34:22,052][14531] Port 40800 is available [2023-07-24 00:34:22,054][14524] Port 40400 is available [2023-07-24 00:34:22,052][14531] Using port 40800 [2023-07-24 00:34:22,055][14532] Port 41000 is available [2023-07-24 00:34:22,054][14524] Using port 40400 [2023-07-24 00:34:22,056][14532] Using port 41000 [2023-07-24 00:34:22,057][14528] Port 40600 is available [2023-07-24 00:34:22,058][14528] Using port 40600 [2023-07-24 00:34:22,284][14529] Port 40701 is available [2023-07-24 00:34:22,288][14529] Using port 40701 [2023-07-24 00:34:22,292][14530] Port 40901 is available [2023-07-24 00:34:22,293][14530] Using port 40901 [2023-07-24 00:34:22,288][14526] Port 40501 is available [2023-07-24 00:34:22,295][14525] Port 40301 is available [2023-07-24 00:34:22,296][14532] Port 41001 is available [2023-07-24 00:34:22,296][14526] Using port 40501 [2023-07-24 00:34:22,300][14531] Port 40801 is available [2023-07-24 00:34:22,297][14532] Using port 41001 [2023-07-24 00:34:22,296][14525] Using port 40301 [2023-07-24 00:34:22,302][14524] Port 40401 is available [2023-07-24 00:34:22,300][14531] Using port 40801 [2023-07-24 00:34:22,305][14528] Port 40601 is available [2023-07-24 00:34:22,302][14524] Using port 40401 [2023-07-24 00:34:22,305][14528] Using port 40601 [2023-07-24 00:34:22,535][14529] Port 40702 is available [2023-07-24 00:34:22,537][14529] Using port 40702 [2023-07-24 00:34:22,538][14526] Port 40502 is available [2023-07-24 00:34:22,542][14526] Using port 40502 [2023-07-24 00:34:22,542][14530] Port 40902 is available [2023-07-24 00:34:22,544][14530] Using port 40902 [2023-07-24 00:34:22,552][14525] Port 40302 is available [2023-07-24 00:34:22,554][14525] Using port 40302 [2023-07-24 00:34:22,556][14524] Port 40402 is available [2023-07-24 00:34:22,554][14532] Port 41002 is available [2023-07-24 00:34:22,558][14532] Using port 41002 [2023-07-24 00:34:22,556][14524] Using port 40402 [2023-07-24 00:34:22,558][14531] Port 40802 is available [2023-07-24 00:34:22,566][14531] Using port 40802 [2023-07-24 00:34:22,561][14528] Port 40602 is available [2023-07-24 00:34:22,569][14528] Using port 40602 [2023-07-24 00:34:22,797][14532] Port 41003 is available [2023-07-24 00:34:22,799][14532] Using port 41003 [2023-07-24 00:34:22,800][14524] Port 40403 is available [2023-07-24 00:34:22,798][14526] Port 40503 is available [2023-07-24 00:34:22,801][14526] Using port 40503 [2023-07-24 00:34:22,803][14531] Port 40803 is available [2023-07-24 00:34:22,806][14529] Port 40703 is available [2023-07-24 00:34:22,801][14524] Using port 40403 [2023-07-24 00:34:22,806][14530] Port 40903 is available [2023-07-24 00:34:22,810][14532] Using port 41000 on host... [2023-07-24 00:34:22,804][14531] Using port 40803 [2023-07-24 00:34:22,806][14529] Using port 40703 [2023-07-24 00:34:22,807][14530] Using port 40903 [2023-07-24 00:34:22,805][14528] Port 40603 is available [2023-07-24 00:34:22,812][14528] Using port 40603 [2023-07-24 00:34:22,817][14529] Using port 40700 on host... [2023-07-24 00:34:22,809][14526] Using port 40500 on host... [2023-07-24 00:34:22,818][14525] Port 40303 is available [2023-07-24 00:34:22,818][14525] Using port 40303 [2023-07-24 00:34:22,817][14524] Using port 40400 on host... [2023-07-24 00:34:22,815][14530] Using port 40900 on host... [2023-07-24 00:34:22,820][14531] Using port 40800 on host... [2023-07-24 00:34:22,821][14528] Using port 40600 on host... [2023-07-24 00:34:22,824][14525] Using port 40300 on host... [2023-07-24 00:34:22,940][00294] Heartbeat connected on Batcher_0 [2023-07-24 00:34:22,946][00294] Heartbeat connected on LearnerWorker_p0 [2023-07-24 00:34:22,982][00294] Heartbeat connected on InferenceWorker_p0-w0 [2023-07-24 00:34:24,464][14530] Initialized w:6 v:0 player:0 [2023-07-24 00:34:24,471][14530] Decorrelating experience for 0 frames... [2023-07-24 00:34:24,475][14524] Initialized w:1 v:0 player:0 [2023-07-24 00:34:24,474][14529] Initialized w:4 v:0 player:0 [2023-07-24 00:34:24,480][14525] Initialized w:0 v:0 player:0 [2023-07-24 00:34:24,481][14532] Initialized w:7 v:0 player:0 [2023-07-24 00:34:24,484][14532] Decorrelating experience for 0 frames... [2023-07-24 00:34:24,482][14524] Decorrelating experience for 0 frames... [2023-07-24 00:34:24,487][14531] Initialized w:5 v:0 player:0 [2023-07-24 00:34:24,485][14526] Initialized w:2 v:0 player:0 [2023-07-24 00:34:24,488][14528] Initialized w:3 v:0 player:0 [2023-07-24 00:34:24,475][14530] Using port 40901 on host... [2023-07-24 00:34:24,491][14524] Using port 40401 on host... [2023-07-24 00:34:24,478][14529] Decorrelating experience for 0 frames... [2023-07-24 00:34:24,487][14525] Decorrelating experience for 0 frames... [2023-07-24 00:34:24,495][14532] Using port 41001 on host... [2023-07-24 00:34:24,495][14531] Decorrelating experience for 0 frames... [2023-07-24 00:34:24,496][14528] Decorrelating experience for 0 frames... [2023-07-24 00:34:24,496][14526] Decorrelating experience for 0 frames... [2023-07-24 00:34:24,493][14529] Using port 40701 on host... [2023-07-24 00:34:24,499][14531] Using port 40801 on host... [2023-07-24 00:34:24,499][14528] Using port 40601 on host... [2023-07-24 00:34:24,500][14525] Using port 40301 on host... [2023-07-24 00:34:24,498][14526] Using port 40501 on host... [2023-07-24 00:34:24,628][00294] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:34:26,127][14526] Initialized w:2 v:1 player:0 [2023-07-24 00:34:26,130][14525] Initialized w:0 v:1 player:0 [2023-07-24 00:34:26,134][14525] Decorrelating experience for 32 frames... [2023-07-24 00:34:26,132][14526] Decorrelating experience for 32 frames... [2023-07-24 00:34:26,136][14530] Initialized w:6 v:1 player:0 [2023-07-24 00:34:26,146][14529] Initialized w:4 v:1 player:0 [2023-07-24 00:34:26,151][14532] Initialized w:7 v:1 player:0 [2023-07-24 00:34:26,144][14530] Decorrelating experience for 32 frames... [2023-07-24 00:34:26,152][14532] Decorrelating experience for 32 frames... [2023-07-24 00:34:26,150][14529] Decorrelating experience for 32 frames... [2023-07-24 00:34:26,161][14524] Initialized w:1 v:1 player:0 [2023-07-24 00:34:26,163][14531] Initialized w:5 v:1 player:0 [2023-07-24 00:34:26,167][14524] Decorrelating experience for 32 frames... [2023-07-24 00:34:26,166][14531] Decorrelating experience for 32 frames... [2023-07-24 00:34:26,170][14528] Initialized w:3 v:1 player:0 [2023-07-24 00:34:26,174][14528] Decorrelating experience for 32 frames... [2023-07-24 00:34:26,467][14526] Using port 40502 on host... [2023-07-24 00:34:26,482][14525] Using port 40302 on host... [2023-07-24 00:34:26,488][14530] Using port 40902 on host... [2023-07-24 00:34:26,490][14529] Using port 40702 on host... [2023-07-24 00:34:26,495][14532] Using port 41002 on host... [2023-07-24 00:34:26,521][14531] Using port 40802 on host... [2023-07-24 00:34:26,519][14524] Using port 40402 on host... [2023-07-24 00:34:26,537][14528] Using port 40602 on host... [2023-07-24 00:34:28,161][14530] Initialized w:6 v:2 player:0 [2023-07-24 00:34:28,161][14532] Initialized w:7 v:2 player:0 [2023-07-24 00:34:28,166][14530] Decorrelating experience for 64 frames... [2023-07-24 00:34:28,168][14525] Initialized w:0 v:2 player:0 [2023-07-24 00:34:28,169][14531] Initialized w:5 v:2 player:0 [2023-07-24 00:34:28,164][14532] Decorrelating experience for 64 frames... [2023-07-24 00:34:28,173][14526] Initialized w:2 v:2 player:0 [2023-07-24 00:34:28,176][14526] Decorrelating experience for 64 frames... [2023-07-24 00:34:28,177][14525] Decorrelating experience for 64 frames... [2023-07-24 00:34:28,171][14531] Decorrelating experience for 64 frames... [2023-07-24 00:34:28,179][14529] Initialized w:4 v:2 player:0 [2023-07-24 00:34:28,183][14529] Decorrelating experience for 64 frames... [2023-07-24 00:34:28,189][14524] Initialized w:1 v:2 player:0 [2023-07-24 00:34:28,195][14524] Decorrelating experience for 64 frames... [2023-07-24 00:34:28,206][14528] Initialized w:3 v:2 player:0 [2023-07-24 00:34:28,210][14528] Decorrelating experience for 64 frames... [2023-07-24 00:34:28,829][14526] Using port 40503 on host... [2023-07-24 00:34:28,826][14532] Using port 41003 on host... [2023-07-24 00:34:28,852][14525] Using port 40303 on host... [2023-07-24 00:34:28,855][14530] Using port 40903 on host... [2023-07-24 00:34:28,863][14531] Using port 40803 on host... [2023-07-24 00:34:28,867][14529] Using port 40703 on host... [2023-07-24 00:34:28,870][14528] Using port 40603 on host... [2023-07-24 00:34:28,874][14524] Using port 40403 on host... [2023-07-24 00:34:29,628][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:34:30,521][14532] Initialized w:7 v:3 player:0 [2023-07-24 00:34:30,528][14532] Decorrelating experience for 96 frames... [2023-07-24 00:34:30,533][14531] Initialized w:5 v:3 player:0 [2023-07-24 00:34:30,536][14524] Initialized w:1 v:3 player:0 [2023-07-24 00:34:30,542][14528] Initialized w:3 v:3 player:0 [2023-07-24 00:34:30,535][14531] Decorrelating experience for 96 frames... [2023-07-24 00:34:30,539][14524] Decorrelating experience for 96 frames... [2023-07-24 00:34:30,548][14525] Initialized w:0 v:3 player:0 [2023-07-24 00:34:30,547][14528] Decorrelating experience for 96 frames... [2023-07-24 00:34:30,554][14526] Initialized w:2 v:3 player:0 [2023-07-24 00:34:30,556][14526] Decorrelating experience for 96 frames... [2023-07-24 00:34:30,551][14525] Decorrelating experience for 96 frames... [2023-07-24 00:34:30,572][14530] Initialized w:6 v:3 player:0 [2023-07-24 00:34:30,574][14530] Decorrelating experience for 96 frames... [2023-07-24 00:34:30,575][14529] Initialized w:4 v:3 player:0 [2023-07-24 00:34:30,577][14529] Decorrelating experience for 96 frames... [2023-07-24 00:34:31,825][14526] Port 40504 is available [2023-07-24 00:34:31,825][14526] Using port 40504 [2023-07-24 00:34:31,919][14525] Port 40304 is available [2023-07-24 00:34:31,919][14525] Using port 40304 [2023-07-24 00:34:31,954][14530] Port 40904 is available [2023-07-24 00:34:31,955][14530] Using port 40904 [2023-07-24 00:34:31,945][14529] Port 40704 is available [2023-07-24 00:34:31,967][14529] Using port 40704 [2023-07-24 00:34:32,107][14526] Port 40505 is available [2023-07-24 00:34:32,109][14526] Using port 40505 [2023-07-24 00:34:32,166][14525] Port 40305 is available [2023-07-24 00:34:32,182][14525] Using port 40305 [2023-07-24 00:34:32,186][14530] Port 40905 is available [2023-07-24 00:34:32,191][14530] Using port 40905 [2023-07-24 00:34:32,199][14528] Port 40604 is available [2023-07-24 00:34:32,199][14528] Using port 40604 [2023-07-24 00:34:32,200][14532] Port 41004 is available [2023-07-24 00:34:32,201][14532] Using port 41004 [2023-07-24 00:34:32,207][14529] Port 40705 is available [2023-07-24 00:34:32,215][14524] Port 40404 is available [2023-07-24 00:34:32,215][14524] Using port 40404 [2023-07-24 00:34:32,207][14529] Using port 40705 [2023-07-24 00:34:32,238][14531] Port 40804 is available [2023-07-24 00:34:32,238][14531] Using port 40804 [2023-07-24 00:34:32,380][14526] Port 40506 is available [2023-07-24 00:34:32,383][14526] Using port 40506 [2023-07-24 00:34:32,435][14525] Port 40306 is available [2023-07-24 00:34:32,442][14525] Using port 40306 [2023-07-24 00:34:32,443][14530] Port 40906 is available [2023-07-24 00:34:32,449][14530] Using port 40906 [2023-07-24 00:34:32,459][14529] Port 40706 is available [2023-07-24 00:34:32,462][14529] Using port 40706 [2023-07-24 00:34:32,595][14526] Port 40507 is available [2023-07-24 00:34:32,597][14526] Using port 40507 [2023-07-24 00:34:32,608][14526] Using port 40504 on host... [2023-07-24 00:34:32,650][14528] Port 40605 is available [2023-07-24 00:34:32,650][14528] Using port 40605 [2023-07-24 00:34:32,631][14532] Port 41005 is available [2023-07-24 00:34:32,644][14525] Port 40307 is available [2023-07-24 00:34:32,655][14525] Using port 40307 [2023-07-24 00:34:32,651][14532] Using port 41005 [2023-07-24 00:34:32,657][14530] Port 40907 is available [2023-07-24 00:34:32,662][14530] Using port 40907 [2023-07-24 00:34:32,666][14525] Using port 40304 on host... [2023-07-24 00:34:32,673][14530] Using port 40904 on host... [2023-07-24 00:34:32,670][14529] Port 40707 is available [2023-07-24 00:34:32,675][14529] Using port 40707 [2023-07-24 00:34:32,683][14529] Using port 40704 on host... [2023-07-24 00:34:32,701][14524] Port 40405 is available [2023-07-24 00:34:32,701][14531] Port 40805 is available [2023-07-24 00:34:32,702][14531] Using port 40805 [2023-07-24 00:34:32,701][14524] Using port 40405 [2023-07-24 00:34:33,091][14528] Port 40606 is available [2023-07-24 00:34:33,091][14528] Using port 40606 [2023-07-24 00:34:33,102][14532] Port 41006 is available [2023-07-24 00:34:33,102][14532] Using port 41006 [2023-07-24 00:34:33,145][14524] Port 40406 is available [2023-07-24 00:34:33,158][14524] Using port 40406 [2023-07-24 00:34:33,168][14531] Port 40806 is available [2023-07-24 00:34:33,169][14531] Using port 40806 [2023-07-24 00:34:33,424][14528] Port 40607 is available [2023-07-24 00:34:33,431][14528] Using port 40607 [2023-07-24 00:34:33,429][14532] Port 41007 is available [2023-07-24 00:34:33,434][14532] Using port 41007 [2023-07-24 00:34:33,442][14528] Using port 40604 on host... [2023-07-24 00:34:33,440][14532] Using port 41004 on host... [2023-07-24 00:34:33,458][14524] Port 40407 is available [2023-07-24 00:34:33,458][14524] Using port 40407 [2023-07-24 00:34:33,457][14531] Port 40807 is available [2023-07-24 00:34:33,475][14531] Using port 40807 [2023-07-24 00:34:33,492][14524] Using port 40404 on host... [2023-07-24 00:34:33,494][14531] Using port 40804 on host... [2023-07-24 00:34:34,628][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:34:35,126][14526] Initialized w:2 v:4 player:0 [2023-07-24 00:34:35,128][14526] Decorrelating experience for 128 frames... [2023-07-24 00:34:35,175][14529] Initialized w:4 v:4 player:0 [2023-07-24 00:34:35,179][14529] Decorrelating experience for 128 frames... [2023-07-24 00:34:35,188][14525] Initialized w:0 v:4 player:0 [2023-07-24 00:34:35,192][14525] Decorrelating experience for 128 frames... [2023-07-24 00:34:35,199][14530] Initialized w:6 v:4 player:0 [2023-07-24 00:34:35,216][14530] Decorrelating experience for 128 frames... [2023-07-24 00:34:35,620][14528] Initialized w:3 v:4 player:0 [2023-07-24 00:34:35,626][14528] Decorrelating experience for 128 frames... [2023-07-24 00:34:35,628][14531] Initialized w:5 v:4 player:0 [2023-07-24 00:34:35,630][14532] Initialized w:7 v:4 player:0 [2023-07-24 00:34:35,634][14524] Initialized w:1 v:4 player:0 [2023-07-24 00:34:35,633][14531] Decorrelating experience for 128 frames... [2023-07-24 00:34:35,640][14532] Decorrelating experience for 128 frames... [2023-07-24 00:34:35,641][14524] Decorrelating experience for 128 frames... [2023-07-24 00:34:37,394][14531] Using port 40805 on host... [2023-07-24 00:34:37,451][14528] Using port 40605 on host... [2023-07-24 00:34:37,493][14524] Using port 40405 on host... [2023-07-24 00:34:37,501][14532] Using port 41005 on host... [2023-07-24 00:34:37,689][14526] Using port 40505 on host... [2023-07-24 00:34:37,774][14525] Using port 40305 on host... [2023-07-24 00:34:37,889][14529] Using port 40705 on host... [2023-07-24 00:34:37,972][14530] Using port 40905 on host... [2023-07-24 00:34:39,628][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:34:40,321][14531] Initialized w:5 v:5 player:0 [2023-07-24 00:34:40,327][14528] Initialized w:3 v:5 player:0 [2023-07-24 00:34:40,329][14531] Decorrelating experience for 160 frames... [2023-07-24 00:34:40,331][14528] Decorrelating experience for 160 frames... [2023-07-24 00:34:40,339][14524] Initialized w:1 v:5 player:0 [2023-07-24 00:34:40,349][14524] Decorrelating experience for 160 frames... [2023-07-24 00:34:40,359][14532] Initialized w:7 v:5 player:0 [2023-07-24 00:34:40,363][14532] Decorrelating experience for 160 frames... [2023-07-24 00:34:40,686][14526] Initialized w:2 v:5 player:0 [2023-07-24 00:34:40,690][14526] Decorrelating experience for 160 frames... [2023-07-24 00:34:40,722][14525] Initialized w:0 v:5 player:0 [2023-07-24 00:34:40,728][14525] Decorrelating experience for 160 frames... [2023-07-24 00:34:40,768][14529] Initialized w:4 v:5 player:0 [2023-07-24 00:34:40,781][14529] Decorrelating experience for 160 frames... [2023-07-24 00:34:40,913][14530] Initialized w:6 v:5 player:0 [2023-07-24 00:34:40,915][14530] Decorrelating experience for 160 frames... [2023-07-24 00:34:42,751][14528] Using port 40606 on host... [2023-07-24 00:34:42,810][14524] Using port 40406 on host... [2023-07-24 00:34:42,845][14532] Using port 41006 on host... [2023-07-24 00:34:42,886][14531] Using port 40806 on host... [2023-07-24 00:34:43,203][14525] Using port 40306 on host... [2023-07-24 00:34:43,212][14526] Using port 40506 on host... [2023-07-24 00:34:43,252][14529] Using port 40706 on host... [2023-07-24 00:34:43,406][14530] Using port 40906 on host... [2023-07-24 00:34:44,628][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:34:45,152][14528] Initialized w:3 v:6 player:0 [2023-07-24 00:34:45,162][14528] Decorrelating experience for 192 frames... [2023-07-24 00:34:45,186][14524] Initialized w:1 v:6 player:0 [2023-07-24 00:34:45,194][14532] Initialized w:7 v:6 player:0 [2023-07-24 00:34:45,191][14524] Decorrelating experience for 192 frames... [2023-07-24 00:34:45,196][14532] Decorrelating experience for 192 frames... [2023-07-24 00:34:45,309][14531] Initialized w:5 v:6 player:0 [2023-07-24 00:34:45,312][14531] Decorrelating experience for 192 frames... [2023-07-24 00:34:45,615][14525] Initialized w:0 v:6 player:0 [2023-07-24 00:34:45,623][14525] Decorrelating experience for 192 frames... [2023-07-24 00:34:45,625][14526] Initialized w:2 v:6 player:0 [2023-07-24 00:34:45,632][14526] Decorrelating experience for 192 frames... [2023-07-24 00:34:45,650][14529] Initialized w:4 v:6 player:0 [2023-07-24 00:34:45,655][14529] Decorrelating experience for 192 frames... [2023-07-24 00:34:45,844][14530] Initialized w:6 v:6 player:0 [2023-07-24 00:34:45,848][14530] Decorrelating experience for 192 frames... [2023-07-24 00:34:47,151][14528] Using port 40607 on host... [2023-07-24 00:34:47,211][14524] Using port 40407 on host... [2023-07-24 00:34:47,233][14532] Using port 41007 on host... [2023-07-24 00:34:47,261][14531] Using port 40807 on host... [2023-07-24 00:34:47,495][14525] Using port 40307 on host... [2023-07-24 00:34:47,530][14526] Using port 40507 on host... [2023-07-24 00:34:47,580][14529] Using port 40707 on host... [2023-07-24 00:34:47,700][14530] Using port 40907 on host... [2023-07-24 00:34:48,898][14528] Initialized w:3 v:7 player:0 [2023-07-24 00:34:48,903][14528] Decorrelating experience for 224 frames... [2023-07-24 00:34:48,942][14524] Initialized w:1 v:7 player:0 [2023-07-24 00:34:48,945][14524] Decorrelating experience for 224 frames... [2023-07-24 00:34:48,958][14532] Initialized w:7 v:7 player:0 [2023-07-24 00:34:48,967][14532] Decorrelating experience for 224 frames... [2023-07-24 00:34:48,985][14531] Initialized w:5 v:7 player:0 [2023-07-24 00:34:48,988][14531] Decorrelating experience for 224 frames... [2023-07-24 00:34:49,210][14525] Initialized w:0 v:7 player:0 [2023-07-24 00:34:49,218][14525] Decorrelating experience for 224 frames... [2023-07-24 00:34:49,245][14526] Initialized w:2 v:7 player:0 [2023-07-24 00:34:49,253][14526] Decorrelating experience for 224 frames... [2023-07-24 00:34:49,294][14529] Initialized w:4 v:7 player:0 [2023-07-24 00:34:49,296][14529] Decorrelating experience for 224 frames... [2023-07-24 00:34:49,440][14530] Initialized w:6 v:7 player:0 [2023-07-24 00:34:49,444][14530] Decorrelating experience for 224 frames... [2023-07-24 00:34:49,628][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:34:51,072][00294] Heartbeat connected on RolloutWorker_w3 [2023-07-24 00:34:51,131][00294] Heartbeat connected on RolloutWorker_w1 [2023-07-24 00:34:51,181][00294] Heartbeat connected on RolloutWorker_w7 [2023-07-24 00:34:51,207][00294] Heartbeat connected on RolloutWorker_w5 [2023-07-24 00:34:51,618][00294] Heartbeat connected on RolloutWorker_w2 [2023-07-24 00:34:51,644][00294] Heartbeat connected on RolloutWorker_w0 [2023-07-24 00:34:51,667][00294] Heartbeat connected on RolloutWorker_w4 [2023-07-24 00:34:51,719][00294] Heartbeat connected on RolloutWorker_w6 [2023-07-24 00:34:54,628][00294] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 3.3. Samples: 100. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2023-07-24 00:34:57,322][14511] Signal inference workers to stop experience collection... [2023-07-24 00:34:57,378][14527] InferenceWorker_p0-w0: stopping experience collection [2023-07-24 00:34:59,076][14511] Signal inference workers to resume experience collection... [2023-07-24 00:34:59,077][14527] InferenceWorker_p0-w0: resuming experience collection [2023-07-24 00:34:59,628][00294] Fps is (10 sec: 409.6, 60 sec: 117.0, 300 sec: 117.0). Total num frames: 4096. Throughput: 0: 67.8. Samples: 2372. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2023-07-24 00:35:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 204.8, 300 sec: 204.8). Total num frames: 8192. Throughput: 0: 84.0. Samples: 3360. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) [2023-07-24 00:35:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 364.1, 300 sec: 364.1). Total num frames: 16384. Throughput: 0: 93.1. Samples: 4188. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:35:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 491.5, 300 sec: 491.5). Total num frames: 24576. Throughput: 0: 147.1. Samples: 6620. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:35:19,628][00294] Fps is (10 sec: 2048.0, 60 sec: 670.3, 300 sec: 670.3). Total num frames: 36864. Throughput: 0: 218.7. Samples: 9840. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:35:21,677][14527] Updated weights for policy 0, policy_version 10 (0.1216) [2023-07-24 00:35:24,628][00294] Fps is (10 sec: 2047.9, 60 sec: 750.9, 300 sec: 750.9). Total num frames: 45056. Throughput: 0: 244.5. Samples: 11004. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:35:29,630][00294] Fps is (10 sec: 1228.6, 60 sec: 819.2, 300 sec: 756.2). Total num frames: 49152. Throughput: 0: 292.1. Samples: 13144. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:35:34,632][00294] Fps is (10 sec: 1637.8, 60 sec: 1023.9, 300 sec: 877.7). Total num frames: 61440. Throughput: 0: 340.6. Samples: 15328. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0) [2023-07-24 00:35:39,628][00294] Fps is (10 sec: 2048.3, 60 sec: 1160.5, 300 sec: 928.4). Total num frames: 69632. Throughput: 0: 366.2. Samples: 16580. Policy #0 lag: (min: 0.0, avg: 0.2, max: 2.0) [2023-07-24 00:35:44,628][00294] Fps is (10 sec: 1639.1, 60 sec: 1297.1, 300 sec: 972.8). Total num frames: 77824. Throughput: 0: 386.3. Samples: 19756. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:35:46,269][14527] Updated weights for policy 0, policy_version 20 (0.0022) [2023-07-24 00:35:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1012.0). Total num frames: 86016. Throughput: 0: 423.1. Samples: 22400. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:35:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1570.1, 300 sec: 1046.8). Total num frames: 94208. Throughput: 0: 429.6. Samples: 23520. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:35:59,629][00294] Fps is (10 sec: 1228.6, 60 sec: 1570.1, 300 sec: 1034.8). Total num frames: 98304. Throughput: 0: 415.9. Samples: 25336. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-07-24 00:35:59,637][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000024_98304.pth... [2023-07-24 00:36:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1570.1, 300 sec: 1024.0). Total num frames: 102400. Throughput: 0: 380.8. Samples: 26976. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-07-24 00:36:09,635][00294] Fps is (10 sec: 1228.2, 60 sec: 1570.0, 300 sec: 1053.2). Total num frames: 110592. Throughput: 0: 374.9. Samples: 27876. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:36:14,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1570.1, 300 sec: 1079.8). Total num frames: 118784. Throughput: 0: 372.4. Samples: 29900. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-07-24 00:36:17,458][14527] Updated weights for policy 0, policy_version 30 (0.0042) [2023-07-24 00:36:19,628][00294] Fps is (10 sec: 1639.5, 60 sec: 1501.9, 300 sec: 1104.1). Total num frames: 126976. Throughput: 0: 377.0. Samples: 32292. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-07-24 00:36:24,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1433.6, 300 sec: 1092.3). Total num frames: 131072. Throughput: 0: 372.8. Samples: 33356. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-07-24 00:36:29,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1501.9, 300 sec: 1114.1). Total num frames: 139264. Throughput: 0: 347.5. Samples: 35396. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:36:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.7, 300 sec: 1134.3). Total num frames: 147456. Throughput: 0: 345.5. Samples: 37948. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) [2023-07-24 00:36:39,628][00294] Fps is (10 sec: 2048.2, 60 sec: 1501.9, 300 sec: 1183.3). Total num frames: 159744. Throughput: 0: 356.5. Samples: 39564. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:36:41,515][14527] Updated weights for policy 0, policy_version 40 (0.0043) [2023-07-24 00:36:44,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1501.9, 300 sec: 1199.5). Total num frames: 167936. Throughput: 0: 379.4. Samples: 42408. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:36:49,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1501.9, 300 sec: 1214.7). Total num frames: 176128. Throughput: 0: 390.6. Samples: 44552. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:36:54,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1433.6, 300 sec: 1201.5). Total num frames: 180224. Throughput: 0: 394.0. Samples: 45604. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:36:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1501.9, 300 sec: 1215.6). Total num frames: 188416. Throughput: 0: 395.4. Samples: 47692. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) [2023-07-24 00:37:04,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1570.1, 300 sec: 1228.8). Total num frames: 196608. Throughput: 0: 410.1. Samples: 50748. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:37:06,561][14527] Updated weights for policy 0, policy_version 50 (0.0021) [2023-07-24 00:37:09,630][00294] Fps is (10 sec: 1638.2, 60 sec: 1570.3, 300 sec: 1241.2). Total num frames: 204800. Throughput: 0: 418.0. Samples: 52168. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:37:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1570.2, 300 sec: 1252.9). Total num frames: 212992. Throughput: 0: 415.8. Samples: 54108. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:37:19,628][00294] Fps is (10 sec: 1638.7, 60 sec: 1570.1, 300 sec: 1263.9). Total num frames: 221184. Throughput: 0: 399.8. Samples: 55940. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:37:24,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1570.1, 300 sec: 1251.5). Total num frames: 225280. Throughput: 0: 384.9. Samples: 56884. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:37:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1638.4, 300 sec: 1284.2). Total num frames: 237568. Throughput: 0: 375.6. Samples: 59312. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:37:33,924][14527] Updated weights for policy 0, policy_version 60 (0.0053) [2023-07-24 00:37:34,628][00294] Fps is (10 sec: 2048.2, 60 sec: 1638.4, 300 sec: 1293.5). Total num frames: 245760. Throughput: 0: 389.1. Samples: 62060. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:37:39,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1501.8, 300 sec: 1281.3). Total num frames: 249856. Throughput: 0: 388.3. Samples: 63076. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:37:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1433.6, 300 sec: 1269.8). Total num frames: 253952. Throughput: 0: 380.4. Samples: 64812. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:37:49,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1433.6, 300 sec: 1278.7). Total num frames: 262144. Throughput: 0: 352.0. Samples: 66588. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:37:54,632][00294] Fps is (10 sec: 1637.8, 60 sec: 1501.8, 300 sec: 1287.3). Total num frames: 270336. Throughput: 0: 340.0. Samples: 67468. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:37:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1276.4). Total num frames: 274432. Throughput: 0: 343.4. Samples: 69560. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:37:59,640][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000067_274432.pth... [2023-07-24 00:38:04,629][00294] Fps is (10 sec: 819.4, 60 sec: 1365.3, 300 sec: 1266.0). Total num frames: 278528. Throughput: 0: 338.5. Samples: 71172. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:38:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1256.1). Total num frames: 282624. Throughput: 0: 332.4. Samples: 71840. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:38:10,455][14527] Updated weights for policy 0, policy_version 70 (0.0063) [2023-07-24 00:38:14,628][00294] Fps is (10 sec: 819.3, 60 sec: 1228.8, 300 sec: 1246.6). Total num frames: 286720. Throughput: 0: 308.6. Samples: 73200. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:38:19,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1228.8, 300 sec: 1254.9). Total num frames: 294912. Throughput: 0: 282.1. Samples: 74756. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:38:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1245.9). Total num frames: 299008. Throughput: 0: 279.5. Samples: 75652. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:38:29,628][00294] Fps is (10 sec: 1638.6, 60 sec: 1228.8, 300 sec: 1270.6). Total num frames: 311296. Throughput: 0: 292.5. Samples: 77976. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:38:34,633][00294] Fps is (10 sec: 2047.0, 60 sec: 1228.7, 300 sec: 1277.9). Total num frames: 319488. Throughput: 0: 315.3. Samples: 80780. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:38:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1269.0). Total num frames: 323584. Throughput: 0: 317.4. Samples: 81752. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 00:38:41,018][14527] Updated weights for policy 0, policy_version 80 (0.0036) [2023-07-24 00:38:44,628][00294] Fps is (10 sec: 1229.4, 60 sec: 1297.1, 300 sec: 1276.1). Total num frames: 331776. Throughput: 0: 310.9. Samples: 83552. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:38:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1267.4). Total num frames: 335872. Throughput: 0: 314.9. Samples: 85344. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:38:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.9, 300 sec: 1274.3). Total num frames: 344064. Throughput: 0: 320.2. Samples: 86248. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:38:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1280.9). Total num frames: 352256. Throughput: 0: 350.3. Samples: 88964. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:39:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1287.3). Total num frames: 360448. Throughput: 0: 371.2. Samples: 91460. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:39:09,377][14527] Updated weights for policy 0, policy_version 90 (0.0053) [2023-07-24 00:39:09,635][00294] Fps is (10 sec: 1638.2, 60 sec: 1433.6, 300 sec: 1293.5). Total num frames: 368640. Throughput: 0: 371.4. Samples: 92364. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:39:14,633][00294] Fps is (10 sec: 1228.2, 60 sec: 1433.5, 300 sec: 1285.3). Total num frames: 372736. Throughput: 0: 359.4. Samples: 94152. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:39:19,628][00294] Fps is (10 sec: 819.3, 60 sec: 1365.4, 300 sec: 1277.4). Total num frames: 376832. Throughput: 0: 337.1. Samples: 95948. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:39:24,628][00294] Fps is (10 sec: 1639.1, 60 sec: 1501.9, 300 sec: 1319.1). Total num frames: 389120. Throughput: 0: 342.4. Samples: 97160. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:39:29,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1433.6, 300 sec: 1346.8). Total num frames: 397312. Throughput: 0: 365.2. Samples: 99984. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:39:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1360.7). Total num frames: 401408. Throughput: 0: 372.0. Samples: 102084. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:39:37,085][14527] Updated weights for policy 0, policy_version 100 (0.0020) [2023-07-24 00:39:39,632][00294] Fps is (10 sec: 1228.4, 60 sec: 1433.5, 300 sec: 1388.5). Total num frames: 409600. Throughput: 0: 372.0. Samples: 102988. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:39:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1402.4). Total num frames: 413696. Throughput: 0: 351.3. Samples: 104772. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:39:49,628][00294] Fps is (10 sec: 1229.2, 60 sec: 1433.6, 300 sec: 1430.1). Total num frames: 421888. Throughput: 0: 340.2. Samples: 106768. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:39:54,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1501.9, 300 sec: 1457.9). Total num frames: 434176. Throughput: 0: 351.7. Samples: 108188. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:39:59,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1433.6, 300 sec: 1457.9). Total num frames: 438272. Throughput: 0: 358.2. Samples: 110272. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:39:59,654][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000107_438272.pth... [2023-07-24 00:39:59,973][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000024_98304.pth [2023-07-24 00:40:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1444.0). Total num frames: 442368. Throughput: 0: 347.9. Samples: 111604. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:40:09,634][00294] Fps is (10 sec: 818.9, 60 sec: 1297.0, 300 sec: 1430.1). Total num frames: 446464. Throughput: 0: 336.5. Samples: 112304. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:40:11,556][14527] Updated weights for policy 0, policy_version 110 (0.0093) [2023-07-24 00:40:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.2, 300 sec: 1402.4). Total num frames: 450560. Throughput: 0: 304.6. Samples: 113692. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:40:19,628][00294] Fps is (10 sec: 819.6, 60 sec: 1297.1, 300 sec: 1388.5). Total num frames: 454656. Throughput: 0: 289.3. Samples: 115104. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:40:23,589][14529] DAMAGECOUNT value on done: 60.0 [2023-07-24 00:40:23,592][14529] Sum rewards: -2.449, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-0.550', 'AMMO2': '0.015', 'ARMOR': '0.020', 'weapon4': '0.024', 'HITCOUNT': '0.050', 'AMMO4': '0.076', 'WEAPON4': '0.100', 'AMMO3': '0.128', 'DAMAGECOUNT': '0.180', 'weapon3': '0.672', 'WEAPON3': '0.700', 'weapon2': '0.886', 'FRAGCOUNT': '2.000'} [2023-07-24 00:40:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1402.4). Total num frames: 462848. Throughput: 0: 288.6. Samples: 115976. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:40:24,631][00294] Avg episode reward: [(0, '-2.703')] [2023-07-24 00:40:24,634][14511] Saving new best policy, reward=-2.703! [2023-07-24 00:40:25,875][14530] DAMAGECOUNT value on done: 90.0 [2023-07-24 00:40:26,521][14526] DAMAGECOUNT value on done: 40.0 [2023-07-24 00:40:26,537][14526] Sum rewards: -7.172, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.493', 'AMMO5': '0.005', 'AMMO2': '0.008', 'weapon5': '0.008', 'HITCOUNT': '0.030', 'AMMO4': '0.039', 'ARMOR': '0.069', 'weapon4': '0.098', 'WEAPON5': '0.100', 'WEAPON4': '0.100', 'DAMAGECOUNT': '0.120', 'AMMO3': '0.144', 'weapon3': '0.534', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon2': '1.066'} [2023-07-24 00:40:27,464][14525] DAMAGECOUNT value on done: 19.0 [2023-07-24 00:40:27,471][14525] Sum rewards: -8.156, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.557', 'FRAGCOUNT': '-1.500', 'AMMO5': '0.004', 'AMMO2': '0.004', 'WEAPON1': '0.010', 'AMMO4': '0.021', 'HITCOUNT': '0.030', 'weapon4': '0.036', 'WEAPON4': '0.050', 'DAMAGECOUNT': '0.057', 'weapon5': '0.066', 'WEAPON5': '0.100', 'AMMO3': '0.140', 'ARMOR': '0.408', 'weapon3': '0.652', 'WEAPON3': '0.750', 'weapon2': '0.822'} [2023-07-24 00:40:28,443][14529] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:40:28,452][14529] Sum rewards: -7.661, reward structure: {'DEATHCOUNT': '-8.250', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.800', 'AMMO5': '0.005', 'AMMO2': '0.009', 'HITCOUNT': '0.020', 'weapon5': '0.038', 'AMMO4': '0.044', 'DAMAGECOUNT': '0.075', 'WEAPON5': '0.100', 'AMMO3': '0.134', 'WEAPON3': '0.700', 'weapon2': '0.842', 'weapon3': '0.922'} [2023-07-24 00:40:29,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1297.1, 300 sec: 1402.4). Total num frames: 475136. Throughput: 0: 309.6. Samples: 118704. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:40:29,634][00294] Avg episode reward: [(0, '-6.486')] [2023-07-24 00:40:30,073][14530] DAMAGECOUNT value on done: 164.0 [2023-07-24 00:40:30,074][14530] Sum rewards: -7.344, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.067', 'AMMO5': '0.003', 'ARMOR': '0.020', 'HITCOUNT': '0.030', 'AMMO2': '0.031', 'weapon5': '0.068', 'weapon4': '0.086', 'WEAPON5': '0.100', 'AMMO3': '0.123', 'AMMO4': '0.157', 'WEAPON4': '0.200', 'weapon3': '0.292', 'DAMAGECOUNT': '0.492', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon2': '1.222'} [2023-07-24 00:40:30,977][14526] DAMAGECOUNT value on done: 85.0 [2023-07-24 00:40:30,987][14526] Sum rewards: -6.102, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.400', 'AMMO5': '0.005', 'WEAPON1': '0.020', 'ARMOR': '0.025', 'AMMO2': '0.036', 'HITCOUNT': '0.060', 'WEAPON5': '0.100', 'weapon4': '0.108', 'WEAPON4': '0.150', 'AMMO3': '0.154', 'AMMO4': '0.181', 'DAMAGECOUNT': '0.255', 'weapon3': '0.560', 'WEAPON3': '0.750', 'weapon2': '0.894', 'FRAGCOUNT': '1.000'} [2023-07-24 00:40:31,822][14525] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:40:33,434][14529] DAMAGECOUNT value on done: 30.0 [2023-07-24 00:40:33,435][14529] Sum rewards: -5.939, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.768', 'AMMO2': '0.004', 'AMMO5': '0.005', 'AMMO4': '0.022', 'weapon4': '0.026', 'HITCOUNT': '0.040', 'WEAPON4': '0.050', 'DAMAGECOUNT': '0.090', 'WEAPON5': '0.100', 'AMMO3': '0.112', 'ARMOR': '0.120', 'WEAPON3': '0.600', 'weapon3': '0.662', 'weapon2': '0.998', 'FRAGCOUNT': '1.000'} [2023-07-24 00:40:34,635][00294] Fps is (10 sec: 1637.3, 60 sec: 1296.9, 300 sec: 1388.4). Total num frames: 479232. Throughput: 0: 318.2. Samples: 121088. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:40:34,638][00294] Avg episode reward: [(0, '-6.345')] [2023-07-24 00:40:35,856][14532] DAMAGECOUNT value on done: 85.0 [2023-07-24 00:40:35,857][14532] Sum rewards: -4.800, reward structure: {'DEATHCOUNT': '-9.000', 'AMMO5': '0.003', 'AMMO2': '0.010', 'ARMOR': '0.024', 'AMMO4': '0.048', 'WEAPON4': '0.050', 'weapon4': '0.078', 'HITCOUNT': '0.080', 'AMMO3': '0.088', 'DAMAGECOUNT': '0.255', 'WEAPON3': '0.450', 'weapon3': '0.456', 'HEALTH': '0.484', 'FRAGCOUNT': '1.000', 'weapon2': '1.174'} [2023-07-24 00:40:36,262][14530] DAMAGECOUNT value on done: 10.0 [2023-07-24 00:40:37,316][14526] DAMAGECOUNT value on done: 110.0 [2023-07-24 00:40:38,466][14525] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:40:38,628][14531] DAMAGECOUNT value on done: 155.0 [2023-07-24 00:40:39,120][14524] DAMAGECOUNT value on done: 75.0 [2023-07-24 00:40:39,126][14524] Sum rewards: -6.892, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.216', 'AMMO5': '0.005', 'weapon5': '0.034', 'AMMO2': '0.036', 'weapon4': '0.078', 'HITCOUNT': '0.080', 'WEAPON5': '0.100', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.103', 'AMMO4': '0.179', 'DAMAGECOUNT': '0.225', 'WEAPON4': '0.300', 'weapon3': '0.488', 'WEAPON3': '0.550', 'ARMOR': '0.560', 'FRAGCOUNT': '1.000', 'weapon2': '1.036'} [2023-07-24 00:40:39,376][14528] DAMAGECOUNT value on done: 5.0 [2023-07-24 00:40:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.9, 300 sec: 1374.6). Total num frames: 483328. Throughput: 0: 305.8. Samples: 121948. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:40:39,632][00294] Avg episode reward: [(0, '-6.426')] [2023-07-24 00:40:39,899][14529] DAMAGECOUNT value on done: 10.0 [2023-07-24 00:40:42,577][14527] Updated weights for policy 0, policy_version 120 (0.0061) [2023-07-24 00:40:43,016][14530] DAMAGECOUNT value on done: 85.0 [2023-07-24 00:40:43,025][14530] Sum rewards: -6.196, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.950', 'AMMO2': '0.020', 'ARMOR': '0.025', 'AMMO3': '0.060', 'HITCOUNT': '0.070', 'AMMO4': '0.100', 'weapon4': '0.196', 'WEAPON4': '0.250', 'WEAPON3': '0.250', 'DAMAGECOUNT': '0.255', 'weapon3': '0.396', 'FRAGCOUNT': '1.000', 'weapon2': '1.132'} [2023-07-24 00:40:43,140][14532] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:40:43,850][14526] DAMAGECOUNT value on done: 130.0 [2023-07-24 00:40:43,861][14526] Sum rewards: -8.299, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-1.500', 'HEALTH': '-1.151', 'AMMO2': '0.000', 'AMMO4': '0.001', 'AMMO5': '0.011', 'weapon5': '0.030', 'ARMOR': '0.060', 'HITCOUNT': '0.090', 'WEAPON5': '0.150', 'AMMO3': '0.165', 'DAMAGECOUNT': '0.390', 'weapon3': '0.696', 'WEAPON3': '0.750', 'weapon2': '1.008'} [2023-07-24 00:40:44,633][00294] Fps is (10 sec: 1229.0, 60 sec: 1297.0, 300 sec: 1374.6). Total num frames: 491520. Throughput: 0: 298.6. Samples: 123708. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:40:44,636][00294] Avg episode reward: [(0, '-6.496')] [2023-07-24 00:40:45,371][14525] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:40:45,816][14531] DAMAGECOUNT value on done: 45.0 [2023-07-24 00:40:46,354][14529] DAMAGECOUNT value on done: 85.0 [2023-07-24 00:40:46,440][14524] DAMAGECOUNT value on done: 11.0 [2023-07-24 00:40:47,041][14528] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:40:49,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1297.0, 300 sec: 1374.6). Total num frames: 499712. Throughput: 0: 307.3. Samples: 125432. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:40:49,632][00294] Avg episode reward: [(0, '-6.589')] [2023-07-24 00:40:50,039][14530] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:40:50,054][14532] DAMAGECOUNT value on done: 60.0 [2023-07-24 00:40:50,771][14526] DAMAGECOUNT value on done: 55.0 [2023-07-24 00:40:51,475][14531] DAMAGECOUNT value on done: 29.0 [2023-07-24 00:40:51,781][14525] DAMAGECOUNT value on done: 10.0 [2023-07-24 00:40:51,830][14524] DAMAGECOUNT value on done: 50.0 [2023-07-24 00:40:52,142][14528] DAMAGECOUNT value on done: 15.0 [2023-07-24 00:40:52,530][14529] DAMAGECOUNT value on done: 100.0 [2023-07-24 00:40:52,530][14529] Sum rewards: -6.234, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.182', 'AMMO5': '0.005', 'weapon5': '0.010', 'AMMO2': '0.027', 'ARMOR': '0.048', 'HITCOUNT': '0.050', 'AMMO3': '0.086', 'WEAPON5': '0.100', 'AMMO4': '0.134', 'weapon4': '0.196', 'weapon3': '0.292', 'DAMAGECOUNT': '0.300', 'WEAPON4': '0.300', 'WEAPON3': '0.450', 'weapon2': '0.950', 'FRAGCOUNT': '1.000'} [2023-07-24 00:40:54,351][14532] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:40:54,629][00294] Fps is (10 sec: 1639.0, 60 sec: 1228.8, 300 sec: 1388.5). Total num frames: 507904. Throughput: 0: 318.7. Samples: 126644. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:40:54,631][00294] Avg episode reward: [(0, '-6.792')] [2023-07-24 00:40:54,798][14530] DAMAGECOUNT value on done: 95.0 [2023-07-24 00:40:55,621][14526] DAMAGECOUNT value on done: 5.0 [2023-07-24 00:40:55,894][14531] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:40:56,184][14524] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:40:56,515][14528] DAMAGECOUNT value on done: 67.0 [2023-07-24 00:40:56,630][14525] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:40:57,113][14529] DAMAGECOUNT value on done: 125.0 [2023-07-24 00:40:57,114][14529] Sum rewards: -7.052, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.554', 'AMMO5': '0.005', 'AMMO2': '0.015', 'ARMOR': '0.040', 'HITCOUNT': '0.060', 'AMMO4': '0.077', 'WEAPON5': '0.100', 'AMMO3': '0.112', 'weapon4': '0.124', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.375', 'weapon3': '0.580', 'WEAPON3': '0.600', 'weapon2': '0.964', 'FRAGCOUNT': '1.000'} [2023-07-24 00:40:58,836][14532] DAMAGECOUNT value on done: 17.0 [2023-07-24 00:40:59,434][14530] DAMAGECOUNT value on done: 10.0 [2023-07-24 00:40:59,437][14530] Sum rewards: -5.822, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.646', 'HITCOUNT': '0.010', 'ARMOR': '0.016', 'DAMAGECOUNT': '0.030', 'AMMO2': '0.034', 'AMMO3': '0.095', 'weapon4': '0.120', 'AMMO4': '0.169', 'WEAPON4': '0.250', 'WEAPON3': '0.550', 'weapon3': '0.694', 'weapon2': '0.856', 'FRAGCOUNT': '1.000'} [2023-07-24 00:40:59,628][00294] Fps is (10 sec: 1638.6, 60 sec: 1297.1, 300 sec: 1402.4). Total num frames: 516096. Throughput: 0: 347.2. Samples: 129316. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:40:59,636][00294] Avg episode reward: [(0, '-6.936')] [2023-07-24 00:41:00,448][14526] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:41:01,150][14531] DAMAGECOUNT value on done: 13.0 [2023-07-24 00:41:01,471][14524] DAMAGECOUNT value on done: 41.0 [2023-07-24 00:41:01,699][14525] DAMAGECOUNT value on done: 80.0 [2023-07-24 00:41:02,017][14528] DAMAGECOUNT value on done: 80.0 [2023-07-24 00:41:02,957][14529] DAMAGECOUNT value on done: 125.0 [2023-07-24 00:41:02,958][14529] Sum rewards: -10.820, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-2.715', 'ARMOR': '0.008', 'AMMO2': '0.018', 'weapon4': '0.038', 'AMMO4': '0.091', 'HITCOUNT': '0.120', 'WEAPON4': '0.200', 'AMMO3': '0.201', 'DAMAGECOUNT': '0.375', 'weapon3': '0.538', 'WEAPON3': '0.950', 'FRAGCOUNT': '1.000', 'weapon2': '1.106'} [2023-07-24 00:41:04,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1297.1, 300 sec: 1388.5). Total num frames: 520192. Throughput: 0: 360.8. Samples: 131340. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:41:04,630][00294] Avg episode reward: [(0, '-7.036')] [2023-07-24 00:41:04,819][14532] DAMAGECOUNT value on done: 10.0 [2023-07-24 00:41:07,080][14530] DAMAGECOUNT value on done: 115.0 [2023-07-24 00:41:07,598][14531] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:41:07,804][14524] DAMAGECOUNT value on done: 5.0 [2023-07-24 00:41:07,808][14524] Sum rewards: -7.914, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.587', 'FRAGCOUNT': '-1.500', 'AMMO5': '0.005', 'HITCOUNT': '0.010', 'weapon5': '0.012', 'DAMAGECOUNT': '0.015', 'AMMO2': '0.029', 'AMMO3': '0.076', 'WEAPON5': '0.100', 'ARMOR': '0.104', 'AMMO4': '0.146', 'weapon4': '0.150', 'weapon3': '0.252', 'WEAPON4': '0.300', 'WEAPON3': '0.400', 'weapon2': '1.074'} [2023-07-24 00:41:08,315][14528] DAMAGECOUNT value on done: 10.0 [2023-07-24 00:41:08,433][14526] DAMAGECOUNT value on done: 30.0 [2023-07-24 00:41:08,434][14526] Sum rewards: -5.061, reward structure: {'DEATHCOUNT': '-9.750', 'AMMO5': '0.003', 'WEAPON1': '0.010', 'weapon5': '0.012', 'HITCOUNT': '0.030', 'AMMO2': '0.036', 'WEAPON5': '0.050', 'ARMOR': '0.076', 'DAMAGECOUNT': '0.090', 'AMMO3': '0.098', 'weapon4': '0.142', 'AMMO4': '0.179', 'WEAPON4': '0.350', 'HEALTH': '0.452', 'WEAPON3': '0.500', 'weapon3': '0.576', 'FRAGCOUNT': '1.000', 'weapon2': '1.086'} [2023-07-24 00:41:09,478][14525] DAMAGECOUNT value on done: 5.0 [2023-07-24 00:41:09,629][00294] Fps is (10 sec: 819.2, 60 sec: 1297.2, 300 sec: 1374.6). Total num frames: 524288. Throughput: 0: 360.5. Samples: 132200. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:41:09,632][00294] Avg episode reward: [(0, '-6.992')] [2023-07-24 00:41:12,378][14527] Updated weights for policy 0, policy_version 130 (0.0029) [2023-07-24 00:41:12,785][14532] DAMAGECOUNT value on done: 14.0 [2023-07-24 00:41:14,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1365.3, 300 sec: 1374.6). Total num frames: 532480. Throughput: 0: 338.7. Samples: 133944. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:41:14,635][00294] Avg episode reward: [(0, '-7.048')] [2023-07-24 00:41:15,143][14531] DAMAGECOUNT value on done: 95.0 [2023-07-24 00:41:15,146][14531] Sum rewards: -10.294, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.420', 'weapon5': '0.006', 'AMMO5': '0.010', 'AMMO2': '0.018', 'weapon4': '0.046', 'ARMOR': '0.069', 'HITCOUNT': '0.080', 'AMMO4': '0.088', 'AMMO3': '0.110', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.285', 'weapon3': '0.476', 'WEAPON3': '0.600', 'weapon2': '0.938', 'FRAGCOUNT': '1.000'} [2023-07-24 00:41:15,486][14524] DAMAGECOUNT value on done: 15.0 [2023-07-24 00:41:16,072][14528] DAMAGECOUNT value on done: 15.0 [2023-07-24 00:41:18,989][14532] DAMAGECOUNT value on done: 30.0 [2023-07-24 00:41:19,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1433.6, 300 sec: 1388.5). Total num frames: 540672. Throughput: 0: 330.0. Samples: 135936. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:41:19,633][00294] Avg episode reward: [(0, '-7.098')] [2023-07-24 00:41:20,418][14531] DAMAGECOUNT value on done: 50.0 [2023-07-24 00:41:20,544][14524] DAMAGECOUNT value on done: 20.0 [2023-07-24 00:41:20,918][14528] DAMAGECOUNT value on done: 65.0 [2023-07-24 00:41:20,923][14528] Sum rewards: -9.621, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.069', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.005', 'ARMOR': '0.008', 'weapon5': '0.008', 'AMMO2': '0.012', 'AMMO4': '0.062', 'HITCOUNT': '0.070', 'AMMO3': '0.094', 'WEAPON5': '0.100', 'weapon4': '0.126', 'DAMAGECOUNT': '0.195', 'WEAPON4': '0.200', 'WEAPON3': '0.300', 'weapon3': '0.346', 'weapon2': '1.172'} [2023-07-24 00:41:24,628][00294] Fps is (10 sec: 1638.7, 60 sec: 1433.6, 300 sec: 1388.5). Total num frames: 548864. Throughput: 0: 341.5. Samples: 137316. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:41:24,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:41:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1388.5). Total num frames: 557056. Throughput: 0: 362.7. Samples: 140028. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:41:29,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:41:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.5, 300 sec: 1360.7). Total num frames: 561152. Throughput: 0: 364.9. Samples: 141852. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:41:34,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:41:39,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1433.6, 300 sec: 1360.7). Total num frames: 569344. Throughput: 0: 357.3. Samples: 142724. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:41:39,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:41:41,515][14527] Updated weights for policy 0, policy_version 140 (0.0042) [2023-07-24 00:41:44,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1365.4, 300 sec: 1346.8). Total num frames: 573440. Throughput: 0: 338.2. Samples: 144536. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:41:44,637][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:41:49,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1365.4, 300 sec: 1360.7). Total num frames: 581632. Throughput: 0: 347.0. Samples: 146956. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:41:49,634][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:41:54,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 589824. Throughput: 0: 358.9. Samples: 148352. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:41:54,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:41:59,633][00294] Fps is (10 sec: 1637.7, 60 sec: 1365.2, 300 sec: 1360.7). Total num frames: 598016. Throughput: 0: 372.1. Samples: 150688. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:41:59,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:41:59,642][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000146_598016.pth... [2023-07-24 00:41:59,843][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000067_274432.pth [2023-07-24 00:42:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1360.7). Total num frames: 606208. Throughput: 0: 366.9. Samples: 152448. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:42:04,639][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:09,628][00294] Fps is (10 sec: 1229.4, 60 sec: 1433.6, 300 sec: 1346.8). Total num frames: 610304. Throughput: 0: 356.2. Samples: 153344. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:42:09,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:09,840][14527] Updated weights for policy 0, policy_version 150 (0.0045) [2023-07-24 00:42:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1346.8). Total num frames: 618496. Throughput: 0: 335.6. Samples: 155128. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:42:14,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1360.7). Total num frames: 626688. Throughput: 0: 355.7. Samples: 157860. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:42:19,636][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1346.8). Total num frames: 634880. Throughput: 0: 367.1. Samples: 159244. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:42:24,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1346.8). Total num frames: 643072. Throughput: 0: 370.1. Samples: 161192. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:42:29,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1346.8). Total num frames: 647168. Throughput: 0: 356.2. Samples: 162984. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:42:34,632][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:39,463][14527] Updated weights for policy 0, policy_version 160 (0.0042) [2023-07-24 00:42:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1360.7). Total num frames: 655360. Throughput: 0: 345.2. Samples: 163888. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:42:39,637][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1501.9, 300 sec: 1360.7). Total num frames: 663552. Throughput: 0: 342.0. Samples: 166076. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:42:44,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1501.9, 300 sec: 1360.7). Total num frames: 671744. Throughput: 0: 363.9. Samples: 168824. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:42:49,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1360.7). Total num frames: 675840. Throughput: 0: 369.5. Samples: 169972. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:42:54,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:42:59,629][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.7, 300 sec: 1374.6). Total num frames: 684032. Throughput: 0: 370.2. Samples: 171788. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:42:59,637][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1374.6). Total num frames: 688128. Throughput: 0: 350.1. Samples: 173616. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:43:04,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:08,614][14527] Updated weights for policy 0, policy_version 170 (0.0052) [2023-07-24 00:43:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1388.5). Total num frames: 696320. Throughput: 0: 339.7. Samples: 174532. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:43:09,630][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1388.5). Total num frames: 704512. Throughput: 0: 353.4. Samples: 177096. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:43:14,630][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1402.4). Total num frames: 712704. Throughput: 0: 372.1. Samples: 179728. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:43:19,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:24,631][00294] Fps is (10 sec: 1638.0, 60 sec: 1433.5, 300 sec: 1388.5). Total num frames: 720896. Throughput: 0: 371.4. Samples: 180600. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:43:24,636][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1374.6). Total num frames: 724992. Throughput: 0: 362.8. Samples: 182404. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:43:29,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:34,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1433.6, 300 sec: 1388.5). Total num frames: 733184. Throughput: 0: 340.9. Samples: 184164. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:43:34,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:36,980][14527] Updated weights for policy 0, policy_version 180 (0.0075) [2023-07-24 00:43:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1388.5). Total num frames: 741376. Throughput: 0: 339.4. Samples: 185244. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:43:39,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1402.4). Total num frames: 749568. Throughput: 0: 360.0. Samples: 187988. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:43:44,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1402.4). Total num frames: 757760. Throughput: 0: 368.9. Samples: 190216. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:43:49,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1388.5). Total num frames: 761856. Throughput: 0: 368.5. Samples: 191116. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:43:54,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:59,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1374.6). Total num frames: 765952. Throughput: 0: 346.9. Samples: 192708. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:43:59,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:43:59,648][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000187_765952.pth... [2023-07-24 00:43:59,979][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000107_438272.pth [2023-07-24 00:44:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 770048. Throughput: 0: 318.8. Samples: 194076. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:44:04,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1360.7). Total num frames: 774144. Throughput: 0: 314.9. Samples: 194768. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:44:09,637][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:10,473][14527] Updated weights for policy 0, policy_version 190 (0.0048) [2023-07-24 00:44:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1374.6). Total num frames: 782336. Throughput: 0: 312.3. Samples: 196456. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:44:14,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:19,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1297.1, 300 sec: 1360.7). Total num frames: 790528. Throughput: 0: 315.7. Samples: 198372. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:44:19,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:24,632][00294] Fps is (10 sec: 1228.4, 60 sec: 1228.8, 300 sec: 1346.8). Total num frames: 794624. Throughput: 0: 311.5. Samples: 199264. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:44:24,634][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:29,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1346.8). Total num frames: 798720. Throughput: 0: 289.6. Samples: 201020. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:44:29,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:34,628][00294] Fps is (10 sec: 1229.2, 60 sec: 1228.8, 300 sec: 1346.8). Total num frames: 806912. Throughput: 0: 278.5. Samples: 202748. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:44:34,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1360.7). Total num frames: 815104. Throughput: 0: 281.6. Samples: 203788. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:44:39,636][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:41,890][14527] Updated weights for policy 0, policy_version 200 (0.0040) [2023-07-24 00:44:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1360.7). Total num frames: 823296. Throughput: 0: 307.1. Samples: 206528. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:44:44,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:49,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1228.8, 300 sec: 1346.8). Total num frames: 831488. Throughput: 0: 325.2. Samples: 208708. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:44:49,632][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1346.8). Total num frames: 835584. Throughput: 0: 329.2. Samples: 209584. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:44:54,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:44:59,635][00294] Fps is (10 sec: 818.7, 60 sec: 1228.7, 300 sec: 1346.8). Total num frames: 839680. Throughput: 0: 330.6. Samples: 211336. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:44:59,637][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1360.7). Total num frames: 847872. Throughput: 0: 327.0. Samples: 213088. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:45:04,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:09,628][00294] Fps is (10 sec: 1639.5, 60 sec: 1365.3, 300 sec: 1374.6). Total num frames: 856064. Throughput: 0: 335.9. Samples: 214380. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:45:09,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:11,364][14527] Updated weights for policy 0, policy_version 210 (0.0025) [2023-07-24 00:45:14,634][00294] Fps is (10 sec: 1637.5, 60 sec: 1365.2, 300 sec: 1388.4). Total num frames: 864256. Throughput: 0: 355.5. Samples: 217020. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:45:14,637][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:19,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1365.3, 300 sec: 1388.5). Total num frames: 872448. Throughput: 0: 359.5. Samples: 218924. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:45:19,634][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:24,628][00294] Fps is (10 sec: 1229.5, 60 sec: 1365.4, 300 sec: 1360.7). Total num frames: 876544. Throughput: 0: 355.8. Samples: 219800. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:45:24,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:29,630][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 880640. Throughput: 0: 332.6. Samples: 221496. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:45:29,636][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1374.6). Total num frames: 888832. Throughput: 0: 329.2. Samples: 223520. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:45:34,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:39,624][14527] Updated weights for policy 0, policy_version 220 (0.0046) [2023-07-24 00:45:39,628][00294] Fps is (10 sec: 2048.3, 60 sec: 1433.6, 300 sec: 1388.5). Total num frames: 901120. Throughput: 0: 339.1. Samples: 224844. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:45:39,630][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1374.6). Total num frames: 905216. Throughput: 0: 356.4. Samples: 227372. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:45:44,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:49,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1360.7). Total num frames: 909312. Throughput: 0: 355.5. Samples: 229084. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:45:49,634][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 917504. Throughput: 0: 345.4. Samples: 229924. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:45:54,636][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:59,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1365.5, 300 sec: 1360.7). Total num frames: 921600. Throughput: 0: 324.6. Samples: 231624. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:45:59,632][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:45:59,646][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000225_921600.pth... [2023-07-24 00:46:00,027][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000146_598016.pth [2023-07-24 00:46:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1360.7). Total num frames: 925696. Throughput: 0: 314.8. Samples: 233088. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:46:04,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:09,631][00294] Fps is (10 sec: 1228.6, 60 sec: 1297.0, 300 sec: 1360.7). Total num frames: 933888. Throughput: 0: 314.2. Samples: 233940. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:46:09,639][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:14,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1228.9, 300 sec: 1346.8). Total num frames: 937984. Throughput: 0: 313.3. Samples: 235592. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:46:14,633][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:15,746][14527] Updated weights for policy 0, policy_version 230 (0.0045) [2023-07-24 00:46:19,628][00294] Fps is (10 sec: 819.4, 60 sec: 1160.6, 300 sec: 1332.9). Total num frames: 942080. Throughput: 0: 298.1. Samples: 236936. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:46:19,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:24,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1228.8, 300 sec: 1332.9). Total num frames: 950272. Throughput: 0: 287.3. Samples: 237772. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:46:24,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1332.9). Total num frames: 954368. Throughput: 0: 269.6. Samples: 239504. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:46:29,637][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1332.9). Total num frames: 962560. Throughput: 0: 278.6. Samples: 241620. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:46:34,630][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1160.5, 300 sec: 1346.8). Total num frames: 970752. Throughput: 0: 289.6. Samples: 242956. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:46:39,634][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1332.9). Total num frames: 974848. Throughput: 0: 304.3. Samples: 245316. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:46:44,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:47,739][14527] Updated weights for policy 0, policy_version 240 (0.0029) [2023-07-24 00:46:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1332.9). Total num frames: 983040. Throughput: 0: 309.7. Samples: 247024. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:46:49,631][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:54,630][00294] Fps is (10 sec: 1638.0, 60 sec: 1228.8, 300 sec: 1332.9). Total num frames: 991232. Throughput: 0: 310.1. Samples: 247896. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:46:54,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:46:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 995328. Throughput: 0: 310.1. Samples: 249548. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:46:59,636][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:47:04,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1297.1, 300 sec: 1332.9). Total num frames: 1003520. Throughput: 0: 336.4. Samples: 252072. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:47:04,635][00294] Avg episode reward: [(0, '-7.136')] [2023-07-24 00:47:05,058][14529] DAMAGECOUNT value on done: 65.0 [2023-07-24 00:47:07,953][14532] DAMAGECOUNT value on done: 135.0 [2023-07-24 00:47:07,956][14532] Sum rewards: -2.893, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.944', 'weapon5': '0.002', 'AMMO5': '0.005', 'WEAPON1': '0.010', 'AMMO2': '0.017', 'HITCOUNT': '0.040', 'weapon4': '0.080', 'AMMO4': '0.084', 'AMMO3': '0.097', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'ARMOR': '0.108', 'DAMAGECOUNT': '0.150', 'WEAPON3': '0.500', 'weapon3': '0.698', 'FRAGCOUNT': '1.000', 'weapon2': '1.060'} [2023-07-24 00:47:08,310][14530] DAMAGECOUNT value on done: 124.0 [2023-07-24 00:47:08,311][14530] Sum rewards: -13.049, reward structure: {'DEATHCOUNT': '-15.000', 'HEALTH': '-2.760', 'AMMO2': '0.008', 'weapon5': '0.014', 'AMMO5': '0.015', 'WEAPON1': '0.020', 'ARMOR': '0.032', 'HITCOUNT': '0.040', 'AMMO4': '0.042', 'DAMAGECOUNT': '0.102', 'WEAPON5': '0.200', 'AMMO3': '0.212', 'weapon3': '0.700', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.100', 'weapon2': '1.226'} [2023-07-24 00:47:09,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1297.1, 300 sec: 1332.9). Total num frames: 1011712. Throughput: 0: 346.2. Samples: 253352. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:47:09,632][00294] Avg episode reward: [(0, '-7.141')] [2023-07-24 00:47:10,805][14524] DAMAGECOUNT value on done: 210.0 [2023-07-24 00:47:10,808][14529] DAMAGECOUNT value on done: 124.0 [2023-07-24 00:47:10,809][14529] Sum rewards: -3.006, reward structure: {'DEATHCOUNT': '-4.500', 'HEALTH': '-1.195', 'FRAGCOUNT': '-0.500', 'AMMO4': '-0.004', 'AMMO2': '-0.001', 'AMMO5': '0.010', 'weapon5': '0.030', 'AMMO3': '0.079', 'HITCOUNT': '0.110', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.297', 'WEAPON3': '0.400', 'weapon2': '0.962', 'weapon3': '1.106'} [2023-07-24 00:47:10,805][14524] Sum rewards: -4.591, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-0.568', 'FRAGCOUNT': '-0.500', 'weapon5': '0.012', 'AMMO5': '0.013', 'AMMO2': '0.024', 'ARMOR': '0.032', 'weapon4': '0.060', 'AMMO3': '0.081', 'HITCOUNT': '0.090', 'WEAPON4': '0.100', 'AMMO4': '0.120', 'WEAPON5': '0.150', 'WEAPON3': '0.400', 'DAMAGECOUNT': '0.405', 'weapon3': '0.594', 'weapon2': '1.146'} [2023-07-24 00:47:11,050][14526] DAMAGECOUNT value on done: 68.0 [2023-07-24 00:47:11,060][14526] Sum rewards: -5.064, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.617', 'AMMO2': '0.017', 'WEAPON1': '0.020', 'HITCOUNT': '0.040', 'AMMO3': '0.070', 'AMMO4': '0.082', 'DAMAGECOUNT': '0.084', 'ARMOR': '0.108', 'weapon4': '0.174', 'WEAPON4': '0.200', 'WEAPON3': '0.350', 'weapon3': '0.654', 'FRAGCOUNT': '1.000', 'weapon2': '1.004'} [2023-07-24 00:47:12,047][14531] DAMAGECOUNT value on done: 230.0 [2023-07-24 00:47:12,048][14531] Sum rewards: -5.363, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.995', 'AMMO5': '0.005', 'AMMO2': '0.018', 'WEAPON4': '0.050', 'HITCOUNT': '0.070', 'ARMOR': '0.072', 'AMMO4': '0.089', 'WEAPON5': '0.100', 'AMMO3': '0.173', 'DAMAGECOUNT': '0.225', 'weapon2': '0.936', 'weapon3': '0.944', 'WEAPON3': '0.950', 'FRAGCOUNT': '2.000'} [2023-07-24 00:47:12,628][14528] DAMAGECOUNT value on done: 45.0 [2023-07-24 00:47:13,632][14525] DAMAGECOUNT value on done: 19.0 [2023-07-24 00:47:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.0). Total num frames: 1015808. Throughput: 0: 350.5. Samples: 255276. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:47:14,635][00294] Avg episode reward: [(0, '-6.957')] [2023-07-24 00:47:15,026][14532] DAMAGECOUNT value on done: 65.0 [2023-07-24 00:47:15,745][14530] DAMAGECOUNT value on done: 259.0 [2023-07-24 00:47:15,770][14530] Sum rewards: -4.872, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.809', 'AMMO5': '0.003', 'AMMO2': '0.004', 'WEAPON1': '0.010', 'weapon5': '0.016', 'AMMO4': '0.021', 'WEAPON5': '0.050', 'HITCOUNT': '0.060', 'AMMO3': '0.132', 'DAMAGECOUNT': '0.285', 'ARMOR': '0.496', 'WEAPON3': '0.700', 'weapon3': '0.720', 'FRAGCOUNT': '1.000', 'weapon2': '1.440'} [2023-07-24 00:47:17,767][14524] DAMAGECOUNT value on done: 26.0 [2023-07-24 00:47:17,788][14527] Updated weights for policy 0, policy_version 250 (0.0041) [2023-07-24 00:47:18,361][14531] DAMAGECOUNT value on done: 60.0 [2023-07-24 00:47:18,863][14529] DAMAGECOUNT value on done: 74.0 [2023-07-24 00:47:19,180][14526] DAMAGECOUNT value on done: 121.0 [2023-07-24 00:47:19,198][14528] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:47:19,182][14526] Sum rewards: -6.914, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.049', 'AMMO5': '0.005', 'AMMO2': '0.008', 'weapon4': '0.010', 'AMMO4': '0.039', 'HITCOUNT': '0.040', 'weapon5': '0.052', 'WEAPON5': '0.100', 'WEAPON4': '0.100', 'AMMO3': '0.101', 'DAMAGECOUNT': '0.108', 'WEAPON3': '0.550', 'weapon3': '0.716', 'FRAGCOUNT': '1.000', 'weapon2': '1.056'} [2023-07-24 00:47:19,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1024000. Throughput: 0: 339.8. Samples: 256912. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:47:19,638][00294] Avg episode reward: [(0, '-6.938')] [2023-07-24 00:47:21,458][14532] DAMAGECOUNT value on done: 60.0 [2023-07-24 00:47:21,807][14525] DAMAGECOUNT value on done: 35.0 [2023-07-24 00:47:23,952][14530] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:47:24,630][00294] Fps is (10 sec: 1228.7, 60 sec: 1297.0, 300 sec: 1305.2). Total num frames: 1028096. Throughput: 0: 329.0. Samples: 257760. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:47:24,634][00294] Avg episode reward: [(0, '-6.983')] [2023-07-24 00:47:25,343][14524] DAMAGECOUNT value on done: 59.0 [2023-07-24 00:47:25,910][14531] DAMAGECOUNT value on done: 29.0 [2023-07-24 00:47:26,290][14529] DAMAGECOUNT value on done: 40.0 [2023-07-24 00:47:26,485][14528] DAMAGECOUNT value on done: 80.0 [2023-07-24 00:47:26,607][14526] DAMAGECOUNT value on done: 295.0 [2023-07-24 00:47:26,610][14526] Sum rewards: -5.059, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.326', 'AMMO5': '0.005', 'AMMO2': '0.023', 'ARMOR': '0.032', 'weapon5': '0.036', 'weapon4': '0.060', 'AMMO3': '0.076', 'WEAPON5': '0.100', 'HITCOUNT': '0.110', 'AMMO4': '0.116', 'WEAPON4': '0.150', 'WEAPON3': '0.450', 'DAMAGECOUNT': '0.555', 'weapon3': '0.796', 'FRAGCOUNT': '1.000', 'weapon2': '1.008'} [2023-07-24 00:47:27,988][14532] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:47:28,184][14525] DAMAGECOUNT value on done: 65.0 [2023-07-24 00:47:28,187][14525] Sum rewards: -2.219, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.325', 'AMMO2': '0.013', 'HITCOUNT': '0.040', 'AMMO4': '0.065', 'DAMAGECOUNT': '0.120', 'AMMO3': '0.121', 'ARMOR': '0.132', 'WEAPON4': '0.150', 'weapon4': '0.204', 'WEAPON3': '0.600', 'weapon3': '0.774', 'weapon2': '0.886', 'FRAGCOUNT': '1.000'} [2023-07-24 00:47:29,378][14530] DAMAGECOUNT value on done: 280.0 [2023-07-24 00:47:29,380][14530] Sum rewards: -4.246, reward structure: {'DEATHCOUNT': '-9.000', 'AMMO5': '0.003', 'AMMO2': '0.024', 'weapon5': '0.030', 'ARMOR': '0.032', 'WEAPON5': '0.050', 'HEALTH': '0.104', 'AMMO3': '0.108', 'weapon4': '0.118', 'AMMO4': '0.120', 'HITCOUNT': '0.120', 'WEAPON4': '0.150', 'WEAPON3': '0.550', 'weapon3': '0.568', 'DAMAGECOUNT': '0.585', 'FRAGCOUNT': '1.000', 'weapon2': '1.192'} [2023-07-24 00:47:29,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1365.3, 300 sec: 1319.0). Total num frames: 1036288. Throughput: 0: 318.2. Samples: 259636. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:47:29,633][00294] Avg episode reward: [(0, '-6.875')] [2023-07-24 00:47:30,580][14524] DAMAGECOUNT value on done: 115.0 [2023-07-24 00:47:30,593][14529] DAMAGECOUNT value on done: 97.0 [2023-07-24 00:47:30,870][14526] DAMAGECOUNT value on done: 190.0 [2023-07-24 00:47:30,873][14526] Sum rewards: -4.829, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.532', 'AMMO2': '0.010', 'ARMOR': '0.044', 'AMMO4': '0.048', 'HITCOUNT': '0.050', 'AMMO3': '0.113', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.180', 'weapon4': '0.202', 'WEAPON3': '0.650', 'weapon2': '0.800', 'weapon3': '0.956', 'FRAGCOUNT': '1.000'} [2023-07-24 00:47:31,147][14531] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:47:31,513][14528] DAMAGECOUNT value on done: 81.0 [2023-07-24 00:47:31,520][14528] Sum rewards: -9.731, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.682', 'FRAGCOUNT': '-0.500', 'ARMOR': '0.004', 'AMMO5': '0.010', 'WEAPON1': '0.010', 'AMMO2': '0.015', 'HITCOUNT': '0.020', 'weapon5': '0.022', 'DAMAGECOUNT': '0.042', 'AMMO4': '0.074', 'AMMO3': '0.151', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'weapon4': '0.312', 'weapon3': '0.720', 'WEAPON3': '0.750', 'weapon2': '1.172'} [2023-07-24 00:47:32,597][14525] DAMAGECOUNT value on done: 7.0 [2023-07-24 00:47:32,648][14532] DAMAGECOUNT value on done: 84.0 [2023-07-24 00:47:33,875][14530] DAMAGECOUNT value on done: 80.0 [2023-07-24 00:47:33,877][14530] Sum rewards: -8.503, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.765', 'AMMO2': '0.003', 'weapon4': '0.010', 'AMMO4': '0.013', 'WEAPON4': '0.050', 'HITCOUNT': '0.070', 'DAMAGECOUNT': '0.165', 'AMMO3': '0.175', 'ARMOR': '0.400', 'WEAPON3': '0.800', 'weapon3': '0.856', 'FRAGCOUNT': '1.000', 'weapon2': '1.220'} [2023-07-24 00:47:34,628][00294] Fps is (10 sec: 1638.6, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1044480. Throughput: 0: 339.6. Samples: 262308. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:47:34,631][00294] Avg episode reward: [(0, '-6.874')] [2023-07-24 00:47:34,765][14524] DAMAGECOUNT value on done: 41.0 [2023-07-24 00:47:35,260][14531] DAMAGECOUNT value on done: 128.0 [2023-07-24 00:47:35,264][14531] Sum rewards: -4.075, reward structure: {'DEATHCOUNT': '-6.750', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.003', 'AMMO2': '0.004', 'weapon5': '0.016', 'AMMO4': '0.020', 'HEALTH': '0.028', 'ARMOR': '0.037', 'WEAPON4': '0.050', 'WEAPON5': '0.050', 'HITCOUNT': '0.060', 'AMMO3': '0.087', 'weapon4': '0.220', 'DAMAGECOUNT': '0.345', 'WEAPON3': '0.500', 'weapon2': '0.878', 'weapon3': '0.878'} [2023-07-24 00:47:35,446][14528] DAMAGECOUNT value on done: 172.0 [2023-07-24 00:47:35,448][14528] Sum rewards: -9.568, reward structure: {'DEATHCOUNT': '-13.500', 'HEALTH': '-0.617', 'AMMO5': '0.017', 'weapon4': '0.018', 'AMMO2': '0.023', 'weapon5': '0.030', 'HITCOUNT': '0.050', 'AMMO4': '0.113', 'AMMO3': '0.150', 'WEAPON4': '0.200', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.276', 'weapon3': '0.630', 'WEAPON3': '0.700', 'FRAGCOUNT': '1.000', 'weapon2': '1.092'} [2023-07-24 00:47:35,743][14529] DAMAGECOUNT value on done: 220.0 [2023-07-24 00:47:35,744][14529] Sum rewards: -4.749, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.224', 'AMMO2': '0.008', 'weapon4': '0.026', 'ARMOR': '0.035', 'AMMO4': '0.042', 'HITCOUNT': '0.100', 'WEAPON4': '0.100', 'AMMO3': '0.150', 'DAMAGECOUNT': '0.360', 'WEAPON3': '0.750', 'weapon3': '0.834', 'weapon2': '1.070', 'FRAGCOUNT': '2.000'} [2023-07-24 00:47:36,310][14526] DAMAGECOUNT value on done: 80.0 [2023-07-24 00:47:37,096][14532] DAMAGECOUNT value on done: 95.0 [2023-07-24 00:47:37,100][14532] Sum rewards: -2.050, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-1.142', 'AMMO2': '0.029', 'AMMO3': '0.041', 'HITCOUNT': '0.080', 'ARMOR': '0.089', 'AMMO4': '0.144', 'weapon4': '0.224', 'WEAPON3': '0.250', 'DAMAGECOUNT': '0.255', 'WEAPON4': '0.350', 'weapon3': '0.600', 'FRAGCOUNT': '1.000', 'weapon2': '1.280'} [2023-07-24 00:47:38,511][14525] DAMAGECOUNT value on done: 30.0 [2023-07-24 00:47:39,628][00294] Fps is (10 sec: 1638.6, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1052672. Throughput: 0: 347.1. Samples: 263516. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:47:39,633][00294] Avg episode reward: [(0, '-6.841')] [2023-07-24 00:47:40,280][14530] DAMAGECOUNT value on done: 155.0 [2023-07-24 00:47:40,876][14524] DAMAGECOUNT value on done: 115.0 [2023-07-24 00:47:40,878][14524] Sum rewards: -6.329, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.080', 'AMMO4': '-0.026', 'AMMO2': '-0.005', 'weapon5': '0.002', 'AMMO5': '0.005', 'ARMOR': '0.045', 'WEAPON5': '0.050', 'HITCOUNT': '0.090', 'AMMO3': '0.194', 'DAMAGECOUNT': '0.330', 'weapon3': '0.914', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.050', 'weapon2': '1.102'} [2023-07-24 00:47:41,969][14531] DAMAGECOUNT value on done: 40.0 [2023-07-24 00:47:42,272][14528] DAMAGECOUNT value on done: 125.0 [2023-07-24 00:47:42,273][14528] Sum rewards: -7.357, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.130', 'AMMO4': '-0.013', 'AMMO2': '-0.003', 'ARMOR': '0.008', 'HITCOUNT': '0.060', 'AMMO3': '0.142', 'DAMAGECOUNT': '0.345', 'weapon3': '0.648', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon2': '1.336'} [2023-07-24 00:47:42,391][14529] DAMAGECOUNT value on done: 175.0 [2023-07-24 00:47:42,398][14529] Sum rewards: -5.438, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.524', 'AMMO2': '0.009', 'ARMOR': '0.028', 'AMMO4': '0.044', 'HITCOUNT': '0.070', 'AMMO3': '0.143', 'DAMAGECOUNT': '0.150', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon2': '1.018', 'weapon3': '1.074'} [2023-07-24 00:47:43,042][14526] DAMAGECOUNT value on done: 224.0 [2023-07-24 00:47:43,042][14526] Sum rewards: -8.047, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.595', 'AMMO4': '-0.008', 'AMMO2': '-0.002', 'ARMOR': '0.024', 'HITCOUNT': '0.090', 'AMMO3': '0.182', 'DAMAGECOUNT': '0.402', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.000', 'weapon2': '1.012', 'weapon3': '1.098'} [2023-07-24 00:47:44,315][14532] DAMAGECOUNT value on done: 14.0 [2023-07-24 00:47:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 1056768. Throughput: 0: 348.4. Samples: 265228. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:47:44,631][00294] Avg episode reward: [(0, '-6.891')] [2023-07-24 00:47:45,870][14525] DAMAGECOUNT value on done: 245.0 [2023-07-24 00:47:45,871][14525] Sum rewards: -5.238, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.972', 'AMMO5': '0.003', 'AMMO2': '0.016', 'weapon5': '0.018', 'ARMOR': '0.024', 'WEAPON5': '0.050', 'AMMO4': '0.079', 'weapon4': '0.098', 'WEAPON4': '0.100', 'AMMO3': '0.119', 'HITCOUNT': '0.170', 'WEAPON3': '0.650', 'weapon3': '0.680', 'DAMAGECOUNT': '0.735', 'weapon2': '0.992', 'FRAGCOUNT': '2.000'} [2023-07-24 00:47:47,694][14524] DAMAGECOUNT value on done: 30.0 [2023-07-24 00:47:48,389][14530] DAMAGECOUNT value on done: 10.0 [2023-07-24 00:47:48,460][14531] DAMAGECOUNT value on done: 240.0 [2023-07-24 00:47:48,462][14531] Sum rewards: -8.435, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-3.490', 'AMMO2': '0.013', 'AMMO5': '0.018', 'WEAPON1': '0.020', 'ARMOR': '0.028', 'AMMO4': '0.064', 'weapon5': '0.076', 'weapon4': '0.090', 'WEAPON4': '0.100', 'HITCOUNT': '0.110', 'AMMO3': '0.135', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.435', 'weapon3': '0.624', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon2': '1.042'} [2023-07-24 00:47:48,679][14528] DAMAGECOUNT value on done: 62.0 [2023-07-24 00:47:49,435][14527] Updated weights for policy 0, policy_version 260 (0.0038) [2023-07-24 00:47:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1064960. Throughput: 0: 329.6. Samples: 266904. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:47:49,632][00294] Avg episode reward: [(0, '-6.931')] [2023-07-24 00:47:50,759][14529] DAMAGECOUNT value on done: 163.0 [2023-07-24 00:47:50,765][14532] DAMAGECOUNT value on done: 215.0 [2023-07-24 00:47:50,767][14532] Sum rewards: -4.709, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.794', 'AMMO2': '0.001', 'AMMO4': '0.005', 'AMMO5': '0.012', 'WEAPON1': '0.020', 'weapon5': '0.040', 'WEAPON4': '0.050', 'weapon4': '0.074', 'AMMO3': '0.100', 'HITCOUNT': '0.130', 'WEAPON5': '0.200', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.555', 'weapon2': '0.884', 'weapon3': '0.964', 'FRAGCOUNT': '1.000'} [2023-07-24 00:47:51,782][14526] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:47:51,783][14526] Sum rewards: -7.904, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.630', 'AMMO2': '0.020', 'HITCOUNT': '0.030', 'weapon4': '0.032', 'DAMAGECOUNT': '0.075', 'ARMOR': '0.100', 'AMMO4': '0.100', 'AMMO3': '0.132', 'WEAPON4': '0.250', 'WEAPON3': '0.750', 'weapon3': '0.904', 'FRAGCOUNT': '1.000', 'weapon2': '1.082'} [2023-07-24 00:47:54,179][14524] DAMAGECOUNT value on done: 20.0 [2023-07-24 00:47:54,237][14525] DAMAGECOUNT value on done: 157.0 [2023-07-24 00:47:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 1069056. Throughput: 0: 319.8. Samples: 267744. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:47:54,634][00294] Avg episode reward: [(0, '-6.890')] [2023-07-24 00:47:54,770][14531] DAMAGECOUNT value on done: 60.0 [2023-07-24 00:47:54,896][14528] DAMAGECOUNT value on done: 105.0 [2023-07-24 00:47:54,905][14528] Sum rewards: -7.300, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.220', 'AMMO2': '0.001', 'AMMO5': '0.003', 'AMMO4': '0.003', 'WEAPON1': '0.010', 'HITCOUNT': '0.040', 'ARMOR': '0.040', 'WEAPON4': '0.050', 'WEAPON5': '0.050', 'weapon4': '0.074', 'AMMO3': '0.119', 'DAMAGECOUNT': '0.120', 'WEAPON3': '0.650', 'weapon3': '0.794', 'weapon2': '0.966', 'FRAGCOUNT': '1.000'} [2023-07-24 00:47:55,582][14530] DAMAGECOUNT value on done: 205.0 [2023-07-24 00:47:57,649][14526] DAMAGECOUNT value on done: 135.0 [2023-07-24 00:47:59,318][14525] DAMAGECOUNT value on done: 37.0 [2023-07-24 00:47:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1077248. Throughput: 0: 326.7. Samples: 269976. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:47:59,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:47:59,645][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000263_1077248.pth... [2023-07-24 00:47:59,823][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000187_765952.pth [2023-07-24 00:48:04,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1365.3, 300 sec: 1319.0). Total num frames: 1085440. Throughput: 0: 339.9. Samples: 272208. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:48:04,633][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:09,633][00294] Fps is (10 sec: 818.8, 60 sec: 1228.7, 300 sec: 1291.3). Total num frames: 1085440. Throughput: 0: 336.7. Samples: 272912. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:48:09,638][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:14,633][00294] Fps is (10 sec: 819.0, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 1093632. Throughput: 0: 324.9. Samples: 274256. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:48:14,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:19,628][00294] Fps is (10 sec: 1229.4, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1097728. Throughput: 0: 296.2. Samples: 275636. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:48:19,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:24,628][00294] Fps is (10 sec: 819.6, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1101824. Throughput: 0: 284.5. Samples: 276320. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:48:24,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:27,285][14527] Updated weights for policy 0, policy_version 270 (0.0066) [2023-07-24 00:48:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1110016. Throughput: 0: 280.2. Samples: 277836. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:48:29,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1118208. Throughput: 0: 298.8. Samples: 280352. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:48:34,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:39,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1126400. Throughput: 0: 310.7. Samples: 281724. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:48:39,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:44,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1228.7, 300 sec: 1263.5). Total num frames: 1130496. Throughput: 0: 310.1. Samples: 283932. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:48:44,634][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:49,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1138688. Throughput: 0: 300.5. Samples: 285732. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:48:49,634][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:54,629][00294] Fps is (10 sec: 1229.1, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1142784. Throughput: 0: 304.8. Samples: 286628. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:48:54,633][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:48:54,785][14527] Updated weights for policy 0, policy_version 280 (0.0032) [2023-07-24 00:48:59,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1150976. Throughput: 0: 316.1. Samples: 288480. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:48:59,637][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:04,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 1159168. Throughput: 0: 345.7. Samples: 291192. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:49:04,637][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:09,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 1167360. Throughput: 0: 361.2. Samples: 292576. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:49:09,633][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:14,633][00294] Fps is (10 sec: 1637.6, 60 sec: 1365.3, 300 sec: 1305.1). Total num frames: 1175552. Throughput: 0: 368.0. Samples: 294396. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:49:14,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 1179648. Throughput: 0: 351.1. Samples: 296152. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:49:19,633][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:24,628][00294] Fps is (10 sec: 819.6, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 1183744. Throughput: 0: 339.8. Samples: 297016. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:49:24,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:25,434][14527] Updated weights for policy 0, policy_version 290 (0.0019) [2023-07-24 00:49:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 1191936. Throughput: 0: 339.3. Samples: 299200. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:49:29,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:34,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1204224. Throughput: 0: 360.2. Samples: 301940. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 00:49:34,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 1208320. Throughput: 0: 363.0. Samples: 302964. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:49:39,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.7, 300 sec: 1305.2). Total num frames: 1216512. Throughput: 0: 360.1. Samples: 304684. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:49:44,633][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 1220608. Throughput: 0: 338.0. Samples: 306400. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:49:49,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:53,706][14527] Updated weights for policy 0, policy_version 300 (0.0039) [2023-07-24 00:49:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1228800. Throughput: 0: 326.5. Samples: 307268. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:49:54,638][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1236992. Throughput: 0: 342.3. Samples: 309800. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:49:59,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:49:59,651][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000302_1236992.pth... [2023-07-24 00:49:59,860][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000225_921600.pth [2023-07-24 00:50:04,632][00294] Fps is (10 sec: 1637.7, 60 sec: 1433.5, 300 sec: 1319.0). Total num frames: 1245184. Throughput: 0: 356.4. Samples: 312192. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:50:04,637][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:09,633][00294] Fps is (10 sec: 1228.2, 60 sec: 1365.2, 300 sec: 1305.2). Total num frames: 1249280. Throughput: 0: 356.9. Samples: 313076. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:50:09,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:14,629][00294] Fps is (10 sec: 819.5, 60 sec: 1297.2, 300 sec: 1291.3). Total num frames: 1253376. Throughput: 0: 337.6. Samples: 314392. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:50:14,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:19,628][00294] Fps is (10 sec: 819.6, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 1257472. Throughput: 0: 306.1. Samples: 315716. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:50:19,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:24,629][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 1261568. Throughput: 0: 297.7. Samples: 316360. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:50:24,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:29,629][00294] Fps is (10 sec: 819.1, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1265664. Throughput: 0: 290.0. Samples: 317736. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:50:29,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:30,147][14527] Updated weights for policy 0, policy_version 310 (0.0055) [2023-07-24 00:50:34,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1160.5, 300 sec: 1263.5). Total num frames: 1273856. Throughput: 0: 300.4. Samples: 319916. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:50:34,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:39,629][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1282048. Throughput: 0: 310.6. Samples: 321244. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:50:39,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:44,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1290240. Throughput: 0: 296.1. Samples: 323124. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:50:44,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:49,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 1294336. Throughput: 0: 281.4. Samples: 324852. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:50:49,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:54,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1302528. Throughput: 0: 280.4. Samples: 325692. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:50:54,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 1306624. Throughput: 0: 295.5. Samples: 327688. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:50:59,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:50:59,952][14527] Updated weights for policy 0, policy_version 320 (0.0037) [2023-07-24 00:51:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 1318912. Throughput: 0: 325.9. Samples: 330380. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:51:04,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 1323008. Throughput: 0: 338.3. Samples: 331584. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:51:09,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:14,632][00294] Fps is (10 sec: 818.9, 60 sec: 1228.7, 300 sec: 1305.1). Total num frames: 1327104. Throughput: 0: 345.8. Samples: 333296. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:51:14,643][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 1335296. Throughput: 0: 335.4. Samples: 335008. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:51:19,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:24,628][00294] Fps is (10 sec: 1639.0, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1343488. Throughput: 0: 325.1. Samples: 335872. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:51:24,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:29,207][14527] Updated weights for policy 0, policy_version 330 (0.0032) [2023-07-24 00:51:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1351680. Throughput: 0: 334.9. Samples: 338192. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:51:29,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1359872. Throughput: 0: 356.1. Samples: 340876. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:51:34,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1363968. Throughput: 0: 356.5. Samples: 341736. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:51:39,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:44,631][00294] Fps is (10 sec: 819.0, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 1368064. Throughput: 0: 349.7. Samples: 343424. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:51:44,633][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 1376256. Throughput: 0: 327.6. Samples: 345124. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:51:49,633][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:54,629][00294] Fps is (10 sec: 1638.8, 60 sec: 1365.3, 300 sec: 1319.0). Total num frames: 1384448. Throughput: 0: 321.7. Samples: 346060. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:51:54,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:58,293][14527] Updated weights for policy 0, policy_version 340 (0.0031) [2023-07-24 00:51:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1392640. Throughput: 0: 342.1. Samples: 348688. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:51:59,632][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:51:59,646][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000340_1392640.pth... [2023-07-24 00:51:59,840][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000263_1077248.pth [2023-07-24 00:52:04,629][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1400832. Throughput: 0: 354.9. Samples: 350980. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:52:04,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:09,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1365.3, 300 sec: 1319.0). Total num frames: 1404928. Throughput: 0: 354.8. Samples: 351840. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:52:09,637][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 1409024. Throughput: 0: 340.3. Samples: 353504. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:52:14,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:19,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1417216. Throughput: 0: 313.2. Samples: 354968. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:52:19,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 1421312. Throughput: 0: 309.7. Samples: 355672. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:52:24,636][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:29,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1425408. Throughput: 0: 311.4. Samples: 357436. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:52:29,630][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:34,630][00294] Fps is (10 sec: 819.1, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 1429504. Throughput: 0: 311.5. Samples: 359140. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:52:34,634][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:34,900][14527] Updated weights for policy 0, policy_version 350 (0.0027) [2023-07-24 00:52:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1437696. Throughput: 0: 308.0. Samples: 359920. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:52:39,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:44,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1228.9, 300 sec: 1277.4). Total num frames: 1441792. Throughput: 0: 288.6. Samples: 361676. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:52:44,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:49,629][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1449984. Throughput: 0: 276.6. Samples: 363428. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:52:49,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1458176. Throughput: 0: 282.2. Samples: 364540. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:52:54,630][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:52:59,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1466368. Throughput: 0: 306.5. Samples: 367296. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:52:59,637][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:03,105][14527] Updated weights for policy 0, policy_version 360 (0.0025) [2023-07-24 00:53:04,628][00294] Fps is (10 sec: 1638.3, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 1474560. Throughput: 0: 324.4. Samples: 369568. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:53:04,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 1478656. Throughput: 0: 328.8. Samples: 370468. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:53:09,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 1482752. Throughput: 0: 329.0. Samples: 372240. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:53:14,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 1490944. Throughput: 0: 332.3. Samples: 374092. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 00:53:19,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:24,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1503232. Throughput: 0: 346.1. Samples: 375496. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:53:24,630][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:29,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 1511424. Throughput: 0: 367.8. Samples: 378228. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:53:29,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:32,317][14527] Updated weights for policy 0, policy_version 370 (0.0023) [2023-07-24 00:53:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1515520. Throughput: 0: 371.8. Samples: 380160. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:53:34,635][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:39,635][00294] Fps is (10 sec: 818.7, 60 sec: 1365.2, 300 sec: 1319.0). Total num frames: 1519616. Throughput: 0: 367.4. Samples: 381076. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) [2023-07-24 00:53:39,641][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1527808. Throughput: 0: 346.7. Samples: 382896. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:53:44,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:49,628][00294] Fps is (10 sec: 1639.5, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 1536000. Throughput: 0: 344.2. Samples: 385056. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:53:49,631][00294] Avg episode reward: [(0, '-6.976')] [2023-07-24 00:53:53,073][14529] DAMAGECOUNT value on done: 130.0 [2023-07-24 00:53:53,853][14532] DAMAGECOUNT value on done: 139.0 [2023-07-24 00:53:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 1544192. Throughput: 0: 355.3. Samples: 386456. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:53:54,632][00294] Avg episode reward: [(0, '-6.971')] [2023-07-24 00:53:56,301][14524] DAMAGECOUNT value on done: 295.0 [2023-07-24 00:53:56,305][14524] Sum rewards: -8.108, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.165', 'AMMO5': '0.005', 'weapon5': '0.006', 'AMMO2': '0.019', 'ARMOR': '0.052', 'HITCOUNT': '0.090', 'AMMO4': '0.092', 'WEAPON5': '0.100', 'weapon4': '0.126', 'AMMO3': '0.166', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.255', 'WEAPON3': '0.850', 'weapon3': '0.896', 'weapon2': '1.200', 'FRAGCOUNT': '2.000'} [2023-07-24 00:53:56,481][14528] DAMAGECOUNT value on done: 52.0 [2023-07-24 00:53:57,521][14530] DAMAGECOUNT value on done: 154.0 [2023-07-24 00:53:57,523][14530] Sum rewards: -2.213, reward structure: {'DEATHCOUNT': '-6.750', 'AMMO5': '0.003', 'AMMO2': '0.008', 'weapon5': '0.008', 'WEAPON1': '0.010', 'HITCOUNT': '0.030', 'ARMOR': '0.036', 'AMMO4': '0.039', 'WEAPON5': '0.050', 'DAMAGECOUNT': '0.090', 'AMMO3': '0.113', 'HEALTH': '0.205', 'WEAPON3': '0.600', 'FRAGCOUNT': '1.000', 'weapon2': '1.084', 'weapon3': '1.262'} [2023-07-24 00:53:58,195][14531] DAMAGECOUNT value on done: 245.0 [2023-07-24 00:53:58,675][14529] DAMAGECOUNT value on done: 202.0 [2023-07-24 00:53:59,232][14532] DAMAGECOUNT value on done: 220.0 [2023-07-24 00:53:59,251][14532] Sum rewards: -4.051, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.070', 'AMMO5': '0.003', 'AMMO2': '0.007', 'ARMOR': '0.008', 'AMMO4': '0.036', 'WEAPON5': '0.050', 'WEAPON4': '0.100', 'HITCOUNT': '0.110', 'AMMO3': '0.116', 'weapon4': '0.162', 'DAMAGECOUNT': '0.465', 'WEAPON3': '0.650', 'weapon3': '0.780', 'FRAGCOUNT': '1.000', 'weapon2': '1.032'} [2023-07-24 00:53:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 1552384. Throughput: 0: 373.1. Samples: 389028. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:53:59,635][00294] Avg episode reward: [(0, '-6.795')] [2023-07-24 00:53:59,651][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000379_1552384.pth... [2023-07-24 00:53:59,850][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000302_1236992.pth [2023-07-24 00:54:00,938][14527] Updated weights for policy 0, policy_version 380 (0.0039) [2023-07-24 00:54:02,114][14526] DAMAGECOUNT value on done: 163.0 [2023-07-24 00:54:03,461][14524] DAMAGECOUNT value on done: 51.0 [2023-07-24 00:54:03,946][14525] DAMAGECOUNT value on done: 40.0 [2023-07-24 00:54:03,946][14528] DAMAGECOUNT value on done: 203.0 [2023-07-24 00:54:03,947][14528] Sum rewards: -4.892, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.434', 'weapon5': '0.004', 'AMMO5': '0.013', 'AMMO2': '0.022', 'weapon7': '0.080', 'AMMO4': '0.110', 'AMMO3': '0.118', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON5': '0.150', 'weapon4': '0.158', 'HITCOUNT': '0.200', 'WEAPON7': '0.200', 'WEAPON4': '0.250', 'ARMOR': '0.498', 'WEAPON3': '0.500', 'DAMAGECOUNT': '0.534', 'weapon3': '0.740', 'FRAGCOUNT': '1.000', 'weapon2': '1.226'} [2023-07-24 00:54:04,113][14530] DAMAGECOUNT value on done: 359.0 [2023-07-24 00:54:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 1556480. Throughput: 0: 369.5. Samples: 390720. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:54:04,635][00294] Avg episode reward: [(0, '-6.795')] [2023-07-24 00:54:05,275][14529] DAMAGECOUNT value on done: 96.0 [2023-07-24 00:54:05,281][14529] Sum rewards: -7.018, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-1.500', 'AMMO5': '0.007', 'WEAPON1': '0.010', 'AMMO2': '0.011', 'HITCOUNT': '0.020', 'weapon5': '0.040', 'WEAPON4': '0.050', 'AMMO4': '0.054', 'weapon4': '0.056', 'DAMAGECOUNT': '0.066', 'AMMO3': '0.090', 'WEAPON5': '0.150', 'HEALTH': '0.210', 'WEAPON3': '0.250', 'ARMOR': '0.448', 'weapon3': '0.716', 'weapon2': '1.304'} [2023-07-24 00:54:06,232][14531] DAMAGECOUNT value on done: 115.0 [2023-07-24 00:54:06,944][14532] DAMAGECOUNT value on done: 105.0 [2023-07-24 00:54:06,950][14532] Sum rewards: -9.170, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-3.163', 'AMMO2': '0.002', 'AMMO4': '0.011', 'HITCOUNT': '0.030', 'ARMOR': '0.040', 'weapon4': '0.040', 'WEAPON4': '0.050', 'DAMAGECOUNT': '0.135', 'AMMO3': '0.234', 'weapon2': '0.960', 'weapon3': '1.190', 'WEAPON3': '1.300', 'FRAGCOUNT': '2.000'} [2023-07-24 00:54:08,497][14526] DAMAGECOUNT value on done: 296.0 [2023-07-24 00:54:08,498][14526] Sum rewards: -0.475, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.061', 'AMMO2': '0.003', 'WEAPON1': '0.010', 'AMMO4': '0.017', 'AMMO3': '0.103', 'HITCOUNT': '0.110', 'WEAPON3': '0.500', 'DAMAGECOUNT': '0.525', 'weapon3': '1.152', 'weapon2': '1.166', 'FRAGCOUNT': '2.000'} [2023-07-24 00:54:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 1564672. Throughput: 0: 357.1. Samples: 391564. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:54:09,632][00294] Avg episode reward: [(0, '-6.755')] [2023-07-24 00:54:10,686][14525] DAMAGECOUNT value on done: 85.0 [2023-07-24 00:54:10,966][14530] DAMAGECOUNT value on done: 80.0 [2023-07-24 00:54:10,967][14530] Sum rewards: -8.512, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-1.500', 'HEALTH': '-1.344', 'weapon5': '0.006', 'AMMO5': '0.007', 'WEAPON1': '0.010', 'AMMO2': '0.014', 'HITCOUNT': '0.050', 'AMMO4': '0.068', 'AMMO3': '0.077', 'ARMOR': '0.096', 'WEAPON5': '0.150', 'DAMAGECOUNT': '0.165', 'WEAPON4': '0.200', 'weapon4': '0.258', 'WEAPON3': '0.350', 'weapon3': '0.722', 'weapon2': '1.158'} [2023-07-24 00:54:11,125][14524] DAMAGECOUNT value on done: 114.0 [2023-07-24 00:54:11,125][14524] Sum rewards: -3.726, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-2.141', 'AMMO5': '0.005', 'weapon5': '0.014', 'AMMO2': '0.018', 'ARMOR': '0.044', 'HITCOUNT': '0.050', 'AMMO4': '0.089', 'WEAPON5': '0.100', 'AMMO3': '0.144', 'DAMAGECOUNT': '0.165', 'weapon4': '0.232', 'WEAPON4': '0.250', 'WEAPON3': '0.700', 'weapon3': '0.796', 'weapon2': '0.808', 'FRAGCOUNT': '1.000'} [2023-07-24 00:54:11,656][14528] DAMAGECOUNT value on done: 221.0 [2023-07-24 00:54:11,656][14528] Sum rewards: -2.257, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.814', 'AMMO2': '0.012', 'AMMO4': '0.057', 'ARMOR': '0.080', 'HITCOUNT': '0.100', 'AMMO3': '0.105', 'weapon4': '0.112', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.423', 'WEAPON3': '0.550', 'weapon3': '0.836', 'FRAGCOUNT': '1.000', 'weapon2': '1.132'} [2023-07-24 00:54:12,029][14529] DAMAGECOUNT value on done: 180.0 [2023-07-24 00:54:13,889][14531] DAMAGECOUNT value on done: 169.0 [2023-07-24 00:54:13,894][14531] Sum rewards: -2.526, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.354', 'WEAPON1': '0.010', 'AMMO2': '0.013', 'AMMO4': '0.065', 'AMMO3': '0.096', 'WEAPON4': '0.100', 'HITCOUNT': '0.120', 'weapon4': '0.158', 'DAMAGECOUNT': '0.420', 'WEAPON3': '0.450', 'ARMOR': '0.531', 'weapon3': '0.692', 'weapon2': '1.422', 'FRAGCOUNT': '2.000'} [2023-07-24 00:54:14,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1433.5, 300 sec: 1319.0). Total num frames: 1568768. Throughput: 0: 334.7. Samples: 393292. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:54:14,634][00294] Avg episode reward: [(0, '-6.649')] [2023-07-24 00:54:14,762][14532] DAMAGECOUNT value on done: 0.0 [2023-07-24 00:54:14,866][14526] DAMAGECOUNT value on done: 430.0 [2023-07-24 00:54:16,080][14525] DAMAGECOUNT value on done: 90.0 [2023-07-24 00:54:16,083][14525] Sum rewards: -5.984, reward structure: {'DEATHCOUNT': '-6.750', 'FRAGCOUNT': '-1.500', 'HEALTH': '-1.380', 'AMMO5': '0.009', 'HITCOUNT': '0.020', 'ARMOR': '0.040', 'WEAPON1': '0.040', 'AMMO3': '0.054', 'AMMO2': '0.057', 'DAMAGECOUNT': '0.075', 'weapon5': '0.092', 'WEAPON5': '0.200', 'WEAPON3': '0.250', 'AMMO4': '0.286', 'weapon4': '0.352', 'WEAPON4': '0.400', 'weapon3': '0.440', 'weapon2': '1.330'} [2023-07-24 00:54:16,278][14530] DAMAGECOUNT value on done: 295.0 [2023-07-24 00:54:17,155][14529] DAMAGECOUNT value on done: 272.0 [2023-07-24 00:54:17,169][14524] DAMAGECOUNT value on done: 165.0 [2023-07-24 00:54:17,161][14529] Sum rewards: -7.292, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.715', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.000', 'AMMO4': '0.001', 'AMMO5': '0.005', 'weapon5': '0.006', 'WEAPON1': '0.010', 'ARMOR': '0.036', 'WEAPON4': '0.050', 'WEAPON5': '0.050', 'weapon4': '0.086', 'AMMO3': '0.119', 'HITCOUNT': '0.130', 'DAMAGECOUNT': '0.525', 'WEAPON3': '0.600', 'weapon3': '0.804', 'weapon2': '1.500'} [2023-07-24 00:54:17,350][14528] DAMAGECOUNT value on done: 111.0 [2023-07-24 00:54:18,449][14531] DAMAGECOUNT value on done: 5.0 [2023-07-24 00:54:18,870][14532] DAMAGECOUNT value on done: 164.0 [2023-07-24 00:54:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 1576960. Throughput: 0: 346.1. Samples: 395736. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 00:54:19,631][00294] Avg episode reward: [(0, '-6.553')] [2023-07-24 00:54:19,793][14526] DAMAGECOUNT value on done: 320.0 [2023-07-24 00:54:19,793][14526] Sum rewards: -9.639, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-1.606', 'AMMO2': '0.014', 'WEAPON1': '0.040', 'ARMOR': '0.060', 'AMMO4': '0.069', 'weapon4': '0.118', 'HITCOUNT': '0.120', 'AMMO3': '0.136', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.390', 'weapon3': '0.648', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon2': '1.322'} [2023-07-24 00:54:21,133][14524] DAMAGECOUNT value on done: 110.0 [2023-07-24 00:54:21,137][14524] Sum rewards: -2.771, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.924', 'AMMO2': '0.006', 'AMMO5': '0.030', 'AMMO4': '0.030', 'WEAPON1': '0.040', 'weapon5': '0.058', 'AMMO3': '0.069', 'HITCOUNT': '0.070', 'weapon4': '0.098', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.207', 'WEAPON5': '0.300', 'WEAPON3': '0.400', 'ARMOR': '0.404', 'weapon3': '0.516', 'weapon2': '1.524', 'FRAGCOUNT': '2.000'} [2023-07-24 00:54:21,300][14525] DAMAGECOUNT value on done: 32.0 [2023-07-24 00:54:21,303][14525] Sum rewards: -9.665, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.582', 'FRAGCOUNT': '-1.500', 'ARMOR': '0.004', 'AMMO5': '0.005', 'AMMO2': '0.011', 'WEAPON1': '0.020', 'weapon4': '0.028', 'HITCOUNT': '0.030', 'AMMO4': '0.054', 'weapon5': '0.074', 'DAMAGECOUNT': '0.075', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.110', 'WEAPON3': '0.500', 'weapon3': '0.502', 'weapon2': '1.554'} [2023-07-24 00:54:21,427][14530] DAMAGECOUNT value on done: 80.0 [2023-07-24 00:54:21,449][14528] DAMAGECOUNT value on done: 182.0 [2023-07-24 00:54:22,078][14529] DAMAGECOUNT value on done: 347.0 [2023-07-24 00:54:22,081][14529] Sum rewards: -10.348, reward structure: {'DEATHCOUNT': '-13.500', 'HEALTH': '-2.026', 'weapon5': '0.002', 'AMMO2': '0.010', 'AMMO5': '0.018', 'AMMO4': '0.049', 'HITCOUNT': '0.080', 'WEAPON4': '0.100', 'weapon4': '0.122', 'AMMO3': '0.161', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.381', 'WEAPON3': '0.900', 'weapon3': '0.936', 'FRAGCOUNT': '1.000', 'weapon2': '1.220'} [2023-07-24 00:54:22,865][14531] DAMAGECOUNT value on done: 223.0 [2023-07-24 00:54:22,865][14531] Sum rewards: -5.134, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.204', 'AMMO5': '0.010', 'AMMO2': '0.025', 'HITCOUNT': '0.050', 'weapon5': '0.064', 'WEAPON5': '0.100', 'AMMO3': '0.106', 'AMMO4': '0.124', 'weapon4': '0.178', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.285', 'ARMOR': '0.480', 'WEAPON3': '0.550', 'weapon3': '0.654', 'FRAGCOUNT': '1.000', 'weapon2': '1.194'} [2023-07-24 00:54:23,545][14532] DAMAGECOUNT value on done: 99.0 [2023-07-24 00:54:24,629][00294] Fps is (10 sec: 1638.7, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1585152. Throughput: 0: 355.4. Samples: 397068. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) [2023-07-24 00:54:24,632][00294] Avg episode reward: [(0, '-6.623')] [2023-07-24 00:54:26,122][14526] DAMAGECOUNT value on done: 122.0 [2023-07-24 00:54:26,123][14526] Sum rewards: -9.324, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.007', 'ARMOR': '0.004', 'AMMO2': '0.007', 'WEAPON1': '0.020', 'AMMO4': '0.035', 'WEAPON4': '0.050', 'HITCOUNT': '0.050', 'weapon4': '0.080', 'DAMAGECOUNT': '0.126', 'AMMO3': '0.198', 'WEAPON3': '0.950', 'FRAGCOUNT': '1.000', 'weapon3': '1.032', 'weapon2': '1.130'} [2023-07-24 00:54:28,485][14525] DAMAGECOUNT value on done: 115.0 [2023-07-24 00:54:28,487][14524] DAMAGECOUNT value on done: 190.0 [2023-07-24 00:54:28,487][14524] Sum rewards: -7.534, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.830', 'AMMO4': '-0.031', 'AMMO2': '-0.006', 'ARMOR': '0.004', 'AMMO5': '0.005', 'WEAPON1': '0.020', 'HITCOUNT': '0.060', 'WEAPON5': '0.100', 'AMMO3': '0.207', 'DAMAGECOUNT': '0.225', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.050', 'weapon2': '1.146', 'weapon3': '1.266'} [2023-07-24 00:54:28,486][14525] Sum rewards: -6.047, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-2.297', 'AMMO5': '0.005', 'weapon5': '0.006', 'AMMO2': '0.015', 'AMMO4': '0.077', 'HITCOUNT': '0.080', 'ARMOR': '0.080', 'AMMO3': '0.094', 'WEAPON5': '0.100', 'weapon4': '0.178', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.255', 'WEAPON3': '0.550', 'weapon3': '0.662', 'FRAGCOUNT': '1.000', 'weapon2': '1.198'} [2023-07-24 00:54:28,784][14528] DAMAGECOUNT value on done: 205.0 [2023-07-24 00:54:28,947][14530] DAMAGECOUNT value on done: 365.0 [2023-07-24 00:54:28,948][14530] Sum rewards: -1.184, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.248', 'AMMO5': '0.005', 'AMMO2': '0.009', 'WEAPON1': '0.020', 'AMMO4': '0.046', 'weapon4': '0.082', 'ARMOR': '0.088', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.102', 'HITCOUNT': '0.170', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.630', 'weapon2': '0.900', 'FRAGCOUNT': '1.000', 'weapon3': '1.262'} [2023-07-24 00:54:29,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1297.0, 300 sec: 1305.2). Total num frames: 1589248. Throughput: 0: 351.7. Samples: 398724. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:54:29,636][00294] Avg episode reward: [(0, '-6.709')] [2023-07-24 00:54:30,248][14529] DAMAGECOUNT value on done: 180.0 [2023-07-24 00:54:31,382][14531] DAMAGECOUNT value on done: 100.0 [2023-07-24 00:54:32,369][14532] DAMAGECOUNT value on done: 129.0 [2023-07-24 00:54:32,371][14532] Sum rewards: -10.510, reward structure: {'DEATHCOUNT': '-13.500', 'HEALTH': '-2.062', 'AMMO2': '0.009', 'AMMO5': '0.013', 'AMMO4': '0.042', 'ARMOR': '0.048', 'WEAPON1': '0.050', 'weapon4': '0.068', 'HITCOUNT': '0.100', 'WEAPON4': '0.100', 'AMMO3': '0.127', 'weapon5': '0.162', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.345', 'weapon3': '0.628', 'WEAPON3': '0.700', 'FRAGCOUNT': '1.000', 'weapon2': '1.360'} [2023-07-24 00:54:34,054][14527] Updated weights for policy 0, policy_version 390 (0.0056) [2023-07-24 00:54:34,629][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.0). Total num frames: 1597440. Throughput: 0: 333.5. Samples: 400064. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:54:34,633][00294] Avg episode reward: [(0, '-6.784')] [2023-07-24 00:54:35,526][14526] DAMAGECOUNT value on done: 269.0 [2023-07-24 00:54:35,529][14526] Sum rewards: -6.704, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.520', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.003', 'AMMO5': '0.004', 'weapon4': '0.004', 'AMMO4': '0.013', 'WEAPON1': '0.020', 'weapon5': '0.026', 'WEAPON4': '0.050', 'HITCOUNT': '0.060', 'WEAPON5': '0.100', 'DAMAGECOUNT': '0.135', 'AMMO3': '0.149', 'WEAPON3': '0.750', 'weapon2': '0.948', 'weapon3': '1.304'} [2023-07-24 00:54:37,459][14524] DAMAGECOUNT value on done: 119.0 [2023-07-24 00:54:37,460][14524] Sum rewards: -11.593, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.616', 'FRAGCOUNT': '-1.500', 'AMMO5': '0.004', 'WEAPON1': '0.020', 'AMMO2': '0.021', 'weapon5': '0.058', 'AMMO3': '0.073', 'HITCOUNT': '0.080', 'ARMOR': '0.084', 'weapon4': '0.092', 'WEAPON5': '0.100', 'AMMO4': '0.104', 'DAMAGECOUNT': '0.267', 'WEAPON4': '0.300', 'WEAPON3': '0.400', 'weapon3': '0.504', 'weapon2': '1.666'} [2023-07-24 00:54:37,742][14528] DAMAGECOUNT value on done: 102.0 [2023-07-24 00:54:37,794][14525] DAMAGECOUNT value on done: 268.0 [2023-07-24 00:54:38,192][14530] DAMAGECOUNT value on done: 245.0 [2023-07-24 00:54:38,193][14530] Sum rewards: -5.101, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-0.028', 'AMMO5': '0.012', 'AMMO2': '0.020', 'weapon5': '0.038', 'AMMO4': '0.098', 'WEAPON4': '0.100', 'AMMO3': '0.168', 'HITCOUNT': '0.180', 'WEAPON5': '0.250', 'ARMOR': '0.400', 'DAMAGECOUNT': '0.705', 'WEAPON3': '0.800', 'weapon3': '0.866', 'weapon2': '1.290', 'FRAGCOUNT': '2.000'} [2023-07-24 00:54:39,541][14529] DAMAGECOUNT value on done: 278.0 [2023-07-24 00:54:39,543][14529] Sum rewards: -5.308, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.956', 'AMMO2': '0.012', 'ARMOR': '0.025', 'weapon4': '0.034', 'AMMO4': '0.062', 'WEAPON4': '0.100', 'HITCOUNT': '0.100', 'AMMO3': '0.114', 'DAMAGECOUNT': '0.345', 'WEAPON3': '0.600', 'FRAGCOUNT': '1.000', 'weapon2': '1.100', 'weapon3': '1.156'} [2023-07-24 00:54:39,628][00294] Fps is (10 sec: 819.3, 60 sec: 1297.2, 300 sec: 1291.3). Total num frames: 1597440. Throughput: 0: 317.2. Samples: 400732. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:54:39,634][00294] Avg episode reward: [(0, '-6.774')] [2023-07-24 00:54:40,409][14531] DAMAGECOUNT value on done: 245.0 [2023-07-24 00:54:41,498][14532] DAMAGECOUNT value on done: 295.0 [2023-07-24 00:54:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 1605632. Throughput: 0: 289.2. Samples: 402040. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:54:44,636][00294] Avg episode reward: [(0, '-6.739')] [2023-07-24 00:54:44,781][14526] DAMAGECOUNT value on done: 25.0 [2023-07-24 00:54:46,419][14524] DAMAGECOUNT value on done: 150.0 [2023-07-24 00:54:46,419][14524] Sum rewards: -9.825, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.540', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.007', 'AMMO2': '0.020', 'weapon5': '0.032', 'AMMO4': '0.098', 'AMMO3': '0.124', 'HITCOUNT': '0.130', 'weapon4': '0.132', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.390', 'ARMOR': '0.488', 'WEAPON3': '0.600', 'weapon3': '0.784', 'weapon2': '1.310'} [2023-07-24 00:54:46,752][14528] DAMAGECOUNT value on done: 165.0 [2023-07-24 00:54:47,000][14525] DAMAGECOUNT value on done: 182.0 [2023-07-24 00:54:47,373][14530] DAMAGECOUNT value on done: 275.0 [2023-07-24 00:54:48,529][14531] DAMAGECOUNT value on done: 132.0 [2023-07-24 00:54:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1609728. Throughput: 0: 285.3. Samples: 403560. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:54:49,631][00294] Avg episode reward: [(0, '-6.794')] [2023-07-24 00:54:51,870][14526] DAMAGECOUNT value on done: 225.0 [2023-07-24 00:54:53,047][14525] DAMAGECOUNT value on done: 37.0 [2023-07-24 00:54:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 1617920. Throughput: 0: 292.8. Samples: 404740. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:54:54,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:54:59,629][00294] Fps is (10 sec: 2047.9, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 1630208. Throughput: 0: 315.7. Samples: 407500. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:54:59,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:04,629][00294] Fps is (10 sec: 1638.2, 60 sec: 1297.0, 300 sec: 1305.2). Total num frames: 1634304. Throughput: 0: 308.8. Samples: 409632. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:55:04,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:05,520][14527] Updated weights for policy 0, policy_version 400 (0.0051) [2023-07-24 00:55:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 1638400. Throughput: 0: 297.9. Samples: 410472. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:55:09,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:14,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 1646592. Throughput: 0: 300.6. Samples: 412252. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:55:14,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1332.9). Total num frames: 1654784. Throughput: 0: 315.1. Samples: 414244. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 00:55:19,635][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1346.8). Total num frames: 1662976. Throughput: 0: 331.8. Samples: 415664. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:55:24,635][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1346.8). Total num frames: 1671168. Throughput: 0: 364.1. Samples: 418424. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:55:29,635][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1332.9). Total num frames: 1675264. Throughput: 0: 368.8. Samples: 420156. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 00:55:34,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:36,770][14527] Updated weights for policy 0, policy_version 410 (0.0022) [2023-07-24 00:55:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 1683456. Throughput: 0: 361.5. Samples: 421008. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:55:39,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1687552. Throughput: 0: 340.0. Samples: 422800. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:55:44,635][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 1695744. Throughput: 0: 342.1. Samples: 425024. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:55:49,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:54,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1501.9, 300 sec: 1360.7). Total num frames: 1708032. Throughput: 0: 353.2. Samples: 426364. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:55:54,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1712128. Throughput: 0: 367.6. Samples: 428792. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:55:59,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:55:59,654][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000418_1712128.pth... [2023-07-24 00:55:59,951][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000340_1392640.pth [2023-07-24 00:56:04,630][00294] Fps is (10 sec: 819.1, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1716224. Throughput: 0: 361.7. Samples: 430520. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:56:04,632][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:05,728][14527] Updated weights for policy 0, policy_version 420 (0.0030) [2023-07-24 00:56:09,637][00294] Fps is (10 sec: 1227.8, 60 sec: 1433.4, 300 sec: 1346.8). Total num frames: 1724416. Throughput: 0: 349.4. Samples: 431388. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:56:09,640][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:14,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1728512. Throughput: 0: 326.8. Samples: 433128. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:56:14,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:19,628][00294] Fps is (10 sec: 1229.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1736704. Throughput: 0: 343.6. Samples: 435620. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 00:56:19,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1744896. Throughput: 0: 354.6. Samples: 436964. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:56:24,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1753088. Throughput: 0: 361.5. Samples: 439068. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:56:29,632][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1757184. Throughput: 0: 344.7. Samples: 440536. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:56:34,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:34,994][14527] Updated weights for policy 0, policy_version 430 (0.0031) [2023-07-24 00:56:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1332.9). Total num frames: 1761280. Throughput: 0: 329.9. Samples: 441208. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:56:39,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1769472. Throughput: 0: 305.7. Samples: 442548. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:56:44,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:49,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 1769472. Throughput: 0: 299.1. Samples: 443980. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:56:49,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:54,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1305.2). Total num frames: 1777664. Throughput: 0: 298.5. Samples: 444816. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:56:54,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:56:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 1785856. Throughput: 0: 319.1. Samples: 447488. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:56:59,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 1794048. Throughput: 0: 310.7. Samples: 449600. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 00:57:04,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:08,681][14527] Updated weights for policy 0, policy_version 440 (0.0036) [2023-07-24 00:57:09,631][00294] Fps is (10 sec: 1637.9, 60 sec: 1297.2, 300 sec: 1332.9). Total num frames: 1802240. Throughput: 0: 303.6. Samples: 450628. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:57:09,640][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:14,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 1810432. Throughput: 0: 301.7. Samples: 452644. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:57:14,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:19,628][00294] Fps is (10 sec: 1638.9, 60 sec: 1365.3, 300 sec: 1346.8). Total num frames: 1818624. Throughput: 0: 332.6. Samples: 455504. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) [2023-07-24 00:57:19,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 1826816. Throughput: 0: 351.1. Samples: 457008. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:57:24,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:29,636][00294] Fps is (10 sec: 1637.2, 60 sec: 1365.2, 300 sec: 1374.6). Total num frames: 1835008. Throughput: 0: 368.8. Samples: 459148. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 00:57:29,641][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:34,629][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 1839104. Throughput: 0: 375.3. Samples: 460868. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:57:34,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:35,751][14527] Updated weights for policy 0, policy_version 450 (0.0065) [2023-07-24 00:57:39,628][00294] Fps is (10 sec: 819.8, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 1843200. Throughput: 0: 375.4. Samples: 461708. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:57:39,639][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 1851392. Throughput: 0: 355.8. Samples: 463500. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:57:44,638][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1501.9, 300 sec: 1360.7). Total num frames: 1859584. Throughput: 0: 368.8. Samples: 466196. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:57:49,637][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1501.9, 300 sec: 1360.7). Total num frames: 1867776. Throughput: 0: 375.7. Samples: 467532. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:57:54,632][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1501.9, 300 sec: 1360.7). Total num frames: 1875968. Throughput: 0: 370.1. Samples: 469300. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:57:59,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:57:59,652][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000458_1875968.pth... [2023-07-24 00:57:59,923][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000379_1552384.pth [2023-07-24 00:58:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1346.8). Total num frames: 1875968. Throughput: 0: 343.8. Samples: 470976. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:58:04,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:06,283][14527] Updated weights for policy 0, policy_version 460 (0.0041) [2023-07-24 00:58:09,632][00294] Fps is (10 sec: 819.0, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 1884160. Throughput: 0: 328.7. Samples: 471800. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:58:09,639][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1360.7). Total num frames: 1892352. Throughput: 0: 328.6. Samples: 473932. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:58:14,630][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:19,628][00294] Fps is (10 sec: 1638.8, 60 sec: 1365.3, 300 sec: 1346.8). Total num frames: 1900544. Throughput: 0: 349.9. Samples: 476612. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:58:19,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:24,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1365.3, 300 sec: 1346.8). Total num frames: 1908736. Throughput: 0: 354.0. Samples: 477636. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:58:24,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.2, 300 sec: 1346.8). Total num frames: 1912832. Throughput: 0: 353.3. Samples: 479400. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:58:29,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:34,629][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1346.8). Total num frames: 1916928. Throughput: 0: 330.8. Samples: 481080. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:58:34,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:37,609][14527] Updated weights for policy 0, policy_version 470 (0.0029) [2023-07-24 00:58:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1346.8). Total num frames: 1925120. Throughput: 0: 319.6. Samples: 481916. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:58:39,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:44,629][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1346.8). Total num frames: 1933312. Throughput: 0: 320.3. Samples: 483712. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:58:44,638][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:49,630][00294] Fps is (10 sec: 819.1, 60 sec: 1228.8, 300 sec: 1319.0). Total num frames: 1933312. Throughput: 0: 319.2. Samples: 485340. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:58:49,635][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:54,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1319.0). Total num frames: 1941504. Throughput: 0: 315.1. Samples: 485980. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:58:54,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:58:59,630][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1319.0). Total num frames: 1945600. Throughput: 0: 297.6. Samples: 487324. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:58:59,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 1949696. Throughput: 0: 273.9. Samples: 488936. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 00:59:04,637][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:09,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1228.9, 300 sec: 1319.1). Total num frames: 1957888. Throughput: 0: 270.1. Samples: 489792. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:59:09,630][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:13,282][14527] Updated weights for policy 0, policy_version 480 (0.0034) [2023-07-24 00:59:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 1966080. Throughput: 0: 281.2. Samples: 492052. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:59:14,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 1974272. Throughput: 0: 302.4. Samples: 494688. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 00:59:19,637][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1332.9). Total num frames: 1982464. Throughput: 0: 304.5. Samples: 495620. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:59:24,639][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 1986560. Throughput: 0: 302.5. Samples: 497324. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 00:59:29,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:34,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1332.9). Total num frames: 1990656. Throughput: 0: 304.4. Samples: 499036. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 00:59:34,637][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1332.9). Total num frames: 1998848. Throughput: 0: 309.0. Samples: 499884. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:59:39,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:43,128][14527] Updated weights for policy 0, policy_version 490 (0.0041) [2023-07-24 00:59:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1346.8). Total num frames: 2007040. Throughput: 0: 338.2. Samples: 502544. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 00:59:44,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1346.8). Total num frames: 2015232. Throughput: 0: 354.6. Samples: 504892. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:59:49,633][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 2019328. Throughput: 0: 354.0. Samples: 505724. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:59:54,632][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1332.9). Total num frames: 2027520. Throughput: 0: 342.7. Samples: 507472. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 00:59:59,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 00:59:59,654][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000495_2027520.pth... [2023-07-24 00:59:59,943][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000418_1712128.pth [2023-07-24 01:00:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 2031616. Throughput: 0: 320.1. Samples: 509092. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:00:04,635][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 01:00:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 2039808. Throughput: 0: 326.8. Samples: 510324. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:00:09,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 01:00:13,851][14527] Updated weights for policy 0, policy_version 500 (0.0051) [2023-07-24 01:00:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 2048000. Throughput: 0: 347.7. Samples: 512972. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:00:14,634][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 01:00:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 2052096. Throughput: 0: 352.3. Samples: 514888. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:00:19,631][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 01:00:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 2060288. Throughput: 0: 352.2. Samples: 515732. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:00:24,636][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 01:00:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.0). Total num frames: 2064384. Throughput: 0: 331.6. Samples: 517468. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:00:29,637][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 01:00:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 2072576. Throughput: 0: 323.4. Samples: 519444. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:00:34,630][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 01:00:35,791][14532] DAMAGECOUNT value on done: 164.0 [2023-07-24 01:00:37,448][14524] DAMAGECOUNT value on done: 375.0 [2023-07-24 01:00:37,888][14528] DAMAGECOUNT value on done: 204.0 [2023-07-24 01:00:37,897][14528] Sum rewards: -4.124, reward structure: {'DEATHCOUNT': '-7.500', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.003', 'ARMOR': '0.004', 'WEAPON1': '0.010', 'weapon5': '0.014', 'AMMO2': '0.015', 'WEAPON5': '0.050', 'AMMO3': '0.065', 'AMMO4': '0.077', 'WEAPON4': '0.100', 'HEALTH': '0.110', 'HITCOUNT': '0.120', 'weapon4': '0.132', 'WEAPON3': '0.350', 'DAMAGECOUNT': '0.456', 'weapon3': '0.868', 'weapon2': '1.502'} [2023-07-24 01:00:39,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 2080768. Throughput: 0: 333.5. Samples: 520732. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:00:39,631][00294] Avg episode reward: [(0, '-6.848')] [2023-07-24 01:00:39,850][14529] DAMAGECOUNT value on done: 380.0 [2023-07-24 01:00:39,852][14529] Sum rewards: -6.064, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.253', 'AMMO5': '0.015', 'AMMO2': '0.019', 'ARMOR': '0.044', 'weapon5': '0.044', 'weapon4': '0.068', 'AMMO4': '0.093', 'WEAPON4': '0.100', 'HITCOUNT': '0.190', 'AMMO3': '0.200', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.750', 'weapon3': '0.850', 'WEAPON3': '1.050', 'weapon2': '1.316', 'FRAGCOUNT': '2.500'} [2023-07-24 01:00:41,053][14532] DAMAGECOUNT value on done: 418.0 [2023-07-24 01:00:41,055][14532] Sum rewards: -7.386, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.130', 'AMMO5': '0.010', 'ARMOR': '0.013', 'AMMO2': '0.014', 'AMMO4': '0.068', 'WEAPON5': '0.100', 'AMMO3': '0.167', 'HITCOUNT': '0.170', 'WEAPON4': '0.200', 'weapon4': '0.236', 'DAMAGECOUNT': '0.594', 'WEAPON3': '0.950', 'weapon2': '0.996', 'weapon3': '1.226', 'FRAGCOUNT': '2.000'} [2023-07-24 01:00:41,523][14531] DAMAGECOUNT value on done: 305.0 [2023-07-24 01:00:43,398][14524] DAMAGECOUNT value on done: 201.0 [2023-07-24 01:00:43,915][14527] Updated weights for policy 0, policy_version 510 (0.0035) [2023-07-24 01:00:44,185][14528] DAMAGECOUNT value on done: 233.0 [2023-07-24 01:00:44,632][00294] Fps is (10 sec: 1637.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 2088960. Throughput: 0: 349.2. Samples: 523188. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:00:44,641][00294] Avg episode reward: [(0, '-6.871')] [2023-07-24 01:00:45,849][14529] DAMAGECOUNT value on done: 231.0 [2023-07-24 01:00:48,121][14530] DAMAGECOUNT value on done: 209.0 [2023-07-24 01:00:49,513][14532] DAMAGECOUNT value on done: 204.0 [2023-07-24 01:00:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 2093056. Throughput: 0: 343.9. Samples: 524568. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:00:49,632][00294] Avg episode reward: [(0, '-6.826')] [2023-07-24 01:00:50,194][14531] DAMAGECOUNT value on done: 263.0 [2023-07-24 01:00:50,195][14531] Sum rewards: -3.112, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.835', 'WEAPON1': '0.010', 'AMMO2': '0.010', 'weapon4': '0.028', 'ARMOR': '0.040', 'WEAPON4': '0.050', 'AMMO4': '0.051', 'AMMO3': '0.158', 'HITCOUNT': '0.160', 'DAMAGECOUNT': '0.444', 'WEAPON3': '0.850', 'weapon2': '1.118', 'weapon3': '1.554', 'FRAGCOUNT': '3.000'} [2023-07-24 01:00:52,503][14524] DAMAGECOUNT value on done: 114.0 [2023-07-24 01:00:53,485][14528] DAMAGECOUNT value on done: 381.0 [2023-07-24 01:00:53,491][14528] Sum rewards: -6.044, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.557', 'AMMO5': '0.003', 'AMMO2': '0.009', 'AMMO4': '0.045', 'weapon4': '0.096', 'HITCOUNT': '0.100', 'WEAPON4': '0.100', 'AMMO3': '0.128', 'DAMAGECOUNT': '0.480', 'WEAPON3': '0.600', 'FRAGCOUNT': '1.000', 'weapon2': '1.064', 'weapon3': '1.388'} [2023-07-24 01:00:54,632][00294] Fps is (10 sec: 819.2, 60 sec: 1297.0, 300 sec: 1305.2). Total num frames: 2097152. Throughput: 0: 331.3. Samples: 525232. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:00:54,635][00294] Avg episode reward: [(0, '-6.801')] [2023-07-24 01:00:55,378][14529] DAMAGECOUNT value on done: 176.0 [2023-07-24 01:00:55,378][14529] Sum rewards: 1.482, reward structure: {'DEATHCOUNT': '-3.750', 'HEALTH': '-0.300', 'AMMO5': '0.005', 'AMMO2': '0.016', 'AMMO3': '0.048', 'AMMO4': '0.079', 'HITCOUNT': '0.080', 'weapon4': '0.096', 'WEAPON4': '0.100', 'WEAPON3': '0.200', 'DAMAGECOUNT': '0.240', 'ARMOR': '0.400', 'weapon3': '1.018', 'weapon2': '1.250', 'FRAGCOUNT': '2.000'} [2023-07-24 01:00:57,857][14530] DAMAGECOUNT value on done: 459.0 [2023-07-24 01:00:57,857][14530] Sum rewards: -8.241, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-1.066', 'AMMO5': '0.003', 'weapon5': '0.016', 'WEAPON1': '0.020', 'AMMO2': '0.026', 'WEAPON5': '0.050', 'ARMOR': '0.098', 'HITCOUNT': '0.100', 'AMMO4': '0.130', 'weapon4': '0.144', 'AMMO3': '0.168', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.300', 'WEAPON3': '0.850', 'weapon3': '0.866', 'FRAGCOUNT': '1.000', 'weapon2': '1.554'} [2023-07-24 01:00:58,308][14526] DAMAGECOUNT value on done: 230.0 [2023-07-24 01:00:58,657][14525] DAMAGECOUNT value on done: 206.0 [2023-07-24 01:00:58,666][14525] Sum rewards: -4.029, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.075', 'AMMO2': '0.003', 'AMMO5': '0.007', 'AMMO4': '0.015', 'WEAPON1': '0.040', 'WEAPON4': '0.050', 'weapon4': '0.080', 'WEAPON5': '0.100', 'HITCOUNT': '0.130', 'AMMO3': '0.160', 'weapon2': '0.496', 'DAMAGECOUNT': '0.498', 'WEAPON3': '0.950', 'FRAGCOUNT': '1.000', 'weapon3': '1.766'} [2023-07-24 01:00:58,771][14532] DAMAGECOUNT value on done: 126.0 [2023-07-24 01:00:59,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 2101248. Throughput: 0: 301.3. Samples: 526532. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:00:59,632][00294] Avg episode reward: [(0, '-6.724')] [2023-07-24 01:00:59,695][14531] DAMAGECOUNT value on done: 299.0 [2023-07-24 01:00:59,696][14531] Sum rewards: -7.097, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.434', 'AMMO5': '0.007', 'AMMO2': '0.008', 'weapon5': '0.010', 'AMMO4': '0.038', 'weapon4': '0.074', 'WEAPON5': '0.100', 'WEAPON4': '0.100', 'ARMOR': '0.108', 'HITCOUNT': '0.130', 'AMMO3': '0.188', 'DAMAGECOUNT': '0.390', 'FRAGCOUNT': '0.500', 'WEAPON3': '1.000', 'weapon2': '1.022', 'weapon3': '1.162'} [2023-07-24 01:01:01,689][14524] DAMAGECOUNT value on done: 195.0 [2023-07-24 01:01:02,643][14528] DAMAGECOUNT value on done: 186.0 [2023-07-24 01:01:04,628][00294] Fps is (10 sec: 819.5, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2105344. Throughput: 0: 287.6. Samples: 527832. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:01:04,631][00294] Avg episode reward: [(0, '-6.711')] [2023-07-24 01:01:05,041][14529] DAMAGECOUNT value on done: 417.0 [2023-07-24 01:01:05,042][14529] Sum rewards: -1.028, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.324', 'AMMO2': '0.014', 'AMMO4': '0.071', 'ARMOR': '0.096', 'AMMO3': '0.113', 'weapon4': '0.136', 'WEAPON4': '0.150', 'HITCOUNT': '0.180', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.711', 'weapon2': '0.790', 'FRAGCOUNT': '1.000', 'weapon3': '1.384'} [2023-07-24 01:01:06,965][14532] DAMAGECOUNT value on done: 179.0 [2023-07-24 01:01:07,754][14530] DAMAGECOUNT value on done: 135.0 [2023-07-24 01:01:07,758][14530] Sum rewards: -6.936, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.070', 'AMMO2': '0.004', 'AMMO5': '0.007', 'AMMO4': '0.022', 'HITCOUNT': '0.060', 'ARMOR': '0.064', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'weapon4': '0.116', 'DAMAGECOUNT': '0.165', 'AMMO3': '0.184', 'WEAPON3': '0.900', 'FRAGCOUNT': '1.000', 'weapon2': '1.076', 'weapon3': '1.084'} [2023-07-24 01:01:07,993][14526] DAMAGECOUNT value on done: 461.0 [2023-07-24 01:01:07,994][14526] Sum rewards: -4.374, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.226', 'AMMO5': '0.003', 'AMMO2': '0.009', 'weapon5': '0.020', 'ARMOR': '0.024', 'AMMO4': '0.044', 'WEAPON5': '0.050', 'weapon4': '0.066', 'WEAPON4': '0.100', 'HITCOUNT': '0.120', 'AMMO3': '0.122', 'DAMAGECOUNT': '0.495', 'WEAPON3': '0.650', 'weapon3': '0.882', 'FRAGCOUNT': '1.000', 'weapon2': '1.518'} [2023-07-24 01:01:08,029][14531] DAMAGECOUNT value on done: 179.0 [2023-07-24 01:01:08,038][14531] Sum rewards: -2.764, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.970', 'AMMO2': '0.003', 'AMMO5': '0.007', 'weapon5': '0.008', 'AMMO4': '0.013', 'ARMOR': '0.088', 'AMMO3': '0.089', 'WEAPON4': '0.100', 'HITCOUNT': '0.120', 'WEAPON5': '0.150', 'weapon4': '0.232', 'DAMAGECOUNT': '0.522', 'WEAPON3': '0.550', 'weapon2': '1.006', 'weapon3': '1.068', 'FRAGCOUNT': '2.000'} [2023-07-24 01:01:08,518][14525] DAMAGECOUNT value on done: 220.0 [2023-07-24 01:01:08,519][14525] Sum rewards: 0.498, reward structure: {'DEATHCOUNT': '-4.500', 'AMMO5': '0.008', 'AMMO2': '0.013', 'WEAPON1': '0.020', 'weapon5': '0.022', 'AMMO3': '0.055', 'AMMO4': '0.067', 'HITCOUNT': '0.100', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'ARMOR': '0.104', 'weapon4': '0.184', 'WEAPON3': '0.300', 'DAMAGECOUNT': '0.405', 'HEALTH': '0.416', 'FRAGCOUNT': '1.000', 'weapon2': '1.018', 'weapon3': '1.086'} [2023-07-24 01:01:09,303][14524] DAMAGECOUNT value on done: 311.0 [2023-07-24 01:01:09,304][14524] Sum rewards: -4.686, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.748', 'AMMO4': '-0.013', 'AMMO2': '-0.003', 'ARMOR': '0.024', 'HITCOUNT': '0.060', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'weapon7': '0.112', 'AMMO3': '0.121', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.603', 'weapon3': '0.874', 'FRAGCOUNT': '1.000', 'weapon2': '1.384'} [2023-07-24 01:01:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 2109440. Throughput: 0: 285.3. Samples: 528572. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:01:09,634][00294] Avg episode reward: [(0, '-6.439')] [2023-07-24 01:01:09,742][14528] DAMAGECOUNT value on done: 307.0 [2023-07-24 01:01:09,747][14528] Sum rewards: -7.299, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.610', 'AMMO2': '0.028', 'ARMOR': '0.052', 'HITCOUNT': '0.100', 'AMMO4': '0.137', 'AMMO3': '0.153', 'weapon4': '0.164', 'WEAPON4': '0.300', 'DAMAGECOUNT': '0.375', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon3': '1.136', 'weapon2': '1.316'} [2023-07-24 01:01:12,084][14532] DAMAGECOUNT value on done: 142.0 [2023-07-24 01:01:12,489][14529] DAMAGECOUNT value on done: 332.0 [2023-07-24 01:01:12,783][14531] DAMAGECOUNT value on done: 303.0 [2023-07-24 01:01:12,788][14531] Sum rewards: -4.572, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.010', 'AMMO5': '0.005', 'AMMO2': '0.007', 'ARMOR': '0.008', 'WEAPON1': '0.020', 'AMMO4': '0.035', 'weapon5': '0.046', 'WEAPON4': '0.050', 'weapon4': '0.060', 'HITCOUNT': '0.070', 'WEAPON5': '0.100', 'AMMO3': '0.115', 'DAMAGECOUNT': '0.240', 'WEAPON3': '0.650', 'weapon2': '0.862', 'FRAGCOUNT': '1.000', 'weapon3': '1.420'} [2023-07-24 01:01:13,787][14530] DAMAGECOUNT value on done: 315.0 [2023-07-24 01:01:13,796][14530] Sum rewards: -9.229, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.759', 'AMMO5': '0.005', 'weapon5': '0.006', 'AMMO2': '0.007', 'ARMOR': '0.008', 'HITCOUNT': '0.020', 'AMMO4': '0.034', 'DAMAGECOUNT': '0.060', 'weapon4': '0.086', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.160', 'WEAPON3': '0.850', 'weapon3': '0.986', 'FRAGCOUNT': '1.000', 'weapon2': '1.358'} [2023-07-24 01:01:13,869][14526] DAMAGECOUNT value on done: 540.0 [2023-07-24 01:01:13,877][14526] Sum rewards: -3.194, reward structure: {'DEATHCOUNT': '-9.000', 'AMMO2': '0.004', 'AMMO5': '0.008', 'AMMO4': '0.018', 'weapon5': '0.038', 'HEALTH': '0.070', 'HITCOUNT': '0.080', 'ARMOR': '0.086', 'WEAPON5': '0.100', 'AMMO3': '0.133', 'DAMAGECOUNT': '0.330', 'WEAPON3': '0.700', 'weapon2': '0.926', 'weapon3': '1.314', 'FRAGCOUNT': '2.000'} [2023-07-24 01:01:13,872][14524] DAMAGECOUNT value on done: 440.0 [2023-07-24 01:01:13,879][14524] Sum rewards: -5.478, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.548', 'AMMO2': '0.003', 'weapon5': '0.010', 'AMMO4': '0.014', 'AMMO5': '0.015', 'WEAPON4': '0.050', 'WEAPON5': '0.100', 'ARMOR': '0.144', 'AMMO3': '0.148', 'HITCOUNT': '0.190', 'weapon4': '0.194', 'DAMAGECOUNT': '0.750', 'WEAPON3': '0.850', 'weapon2': '1.000', 'weapon3': '1.102', 'FRAGCOUNT': '2.000'} [2023-07-24 01:01:14,131][14528] DAMAGECOUNT value on done: 235.0 [2023-07-24 01:01:14,215][14525] DAMAGECOUNT value on done: 124.0 [2023-07-24 01:01:14,216][14525] Sum rewards: -3.563, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.754', 'AMMO2': '0.007', 'weapon5': '0.008', 'AMMO5': '0.010', 'AMMO4': '0.033', 'HITCOUNT': '0.050', 'WEAPON4': '0.050', 'weapon4': '0.060', 'WEAPON5': '0.100', 'DAMAGECOUNT': '0.102', 'AMMO3': '0.155', 'ARMOR': '0.548', 'WEAPON3': '0.850', 'FRAGCOUNT': '1.000', 'weapon3': '1.036', 'weapon2': '1.432'} [2023-07-24 01:01:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 2117632. Throughput: 0: 293.2. Samples: 530660. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:01:14,632][00294] Avg episode reward: [(0, '-6.319')] [2023-07-24 01:01:16,403][14532] DAMAGECOUNT value on done: 169.0 [2023-07-24 01:01:17,177][14531] DAMAGECOUNT value on done: 155.0 [2023-07-24 01:01:17,797][14529] DAMAGECOUNT value on done: 417.0 [2023-07-24 01:01:17,798][14529] Sum rewards: -5.107, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.859', 'AMMO2': '0.015', 'HITCOUNT': '0.060', 'AMMO4': '0.074', 'ARMOR': '0.076', 'weapon4': '0.132', 'AMMO3': '0.133', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.210', 'WEAPON3': '0.700', 'FRAGCOUNT': '1.000', 'weapon2': '1.100', 'weapon3': '1.102'} [2023-07-24 01:01:18,079][14524] DAMAGECOUNT value on done: 264.0 [2023-07-24 01:01:18,353][14528] DAMAGECOUNT value on done: 127.0 [2023-07-24 01:01:19,427][14530] DAMAGECOUNT value on done: 259.0 [2023-07-24 01:01:19,428][14530] Sum rewards: -6.552, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.232', 'AMMO5': '0.005', 'AMMO2': '0.010', 'WEAPON1': '0.010', 'weapon5': '0.038', 'AMMO4': '0.047', 'WEAPON5': '0.050', 'ARMOR': '0.072', 'WEAPON4': '0.100', 'AMMO3': '0.135', 'HITCOUNT': '0.170', 'weapon4': '0.226', 'DAMAGECOUNT': '0.537', 'WEAPON3': '0.750', 'weapon3': '0.908', 'FRAGCOUNT': '1.000', 'weapon2': '1.122'} [2023-07-24 01:01:19,517][14526] DAMAGECOUNT value on done: 440.0 [2023-07-24 01:01:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2125824. Throughput: 0: 305.9. Samples: 533208. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:01:19,635][00294] Avg episode reward: [(0, '-6.436')] [2023-07-24 01:01:19,851][14525] DAMAGECOUNT value on done: 323.0 [2023-07-24 01:01:19,852][14525] Sum rewards: -4.900, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.212', 'AMMO2': '0.001', 'AMMO5': '0.003', 'AMMO4': '0.005', 'ARMOR': '0.016', 'WEAPON5': '0.050', 'weapon5': '0.058', 'HITCOUNT': '0.100', 'AMMO3': '0.170', 'WEAPON3': '0.800', 'weapon3': '0.854', 'DAMAGECOUNT': '0.873', 'weapon2': '1.632', 'FRAGCOUNT': '3.000'} [2023-07-24 01:01:20,260][14527] Updated weights for policy 0, policy_version 520 (0.0050) [2023-07-24 01:01:22,346][14532] DAMAGECOUNT value on done: 530.0 [2023-07-24 01:01:22,348][14532] Sum rewards: -5.009, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-2.237', 'AMMO4': '-0.018', 'AMMO2': '-0.004', 'AMMO5': '0.007', 'weapon5': '0.048', 'ARMOR': '0.096', 'HITCOUNT': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.140', 'DAMAGECOUNT': '0.705', 'WEAPON3': '0.800', 'weapon3': '0.978', 'FRAGCOUNT': '1.000', 'weapon2': '1.524'} [2023-07-24 01:01:23,636][14531] DAMAGECOUNT value on done: 250.0 [2023-07-24 01:01:24,284][14529] DAMAGECOUNT value on done: 203.0 [2023-07-24 01:01:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2134016. Throughput: 0: 296.4. Samples: 534068. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:01:24,631][00294] Avg episode reward: [(0, '-6.425')] [2023-07-24 01:01:25,143][14524] DAMAGECOUNT value on done: 165.0 [2023-07-24 01:01:25,426][14528] DAMAGECOUNT value on done: 325.0 [2023-07-24 01:01:25,437][14528] Sum rewards: -6.091, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.308', 'AMMO5': '0.007', 'AMMO2': '0.016', 'ARMOR': '0.040', 'weapon5': '0.042', 'AMMO4': '0.080', 'weapon4': '0.086', 'HITCOUNT': '0.100', 'AMMO3': '0.117', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.480', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.700', 'weapon3': '1.134', 'weapon2': '1.364'} [2023-07-24 01:01:26,254][14526] DAMAGECOUNT value on done: 251.0 [2023-07-24 01:01:26,261][14526] Sum rewards: -5.859, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.495', 'AMMO2': '0.012', 'AMMO5': '0.015', 'weapon5': '0.022', 'ARMOR': '0.040', 'AMMO4': '0.058', 'weapon4': '0.070', 'WEAPON4': '0.100', 'HITCOUNT': '0.130', 'WEAPON5': '0.150', 'AMMO3': '0.158', 'DAMAGECOUNT': '0.387', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon2': '1.104', 'weapon3': '1.140'} [2023-07-24 01:01:26,287][14530] DAMAGECOUNT value on done: 575.0 [2023-07-24 01:01:26,293][14530] Sum rewards: -3.724, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.189', 'weapon7': '0.008', 'AMMO2': '0.013', 'AMMO4': '0.066', 'weapon4': '0.114', 'ARMOR': '0.140', 'WEAPON4': '0.150', 'AMMO3': '0.152', 'HITCOUNT': '0.200', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'DAMAGECOUNT': '0.630', 'WEAPON3': '0.850', 'weapon2': '0.992', 'weapon3': '1.300', 'FRAGCOUNT': '2.000'} [2023-07-24 01:01:26,592][14525] DAMAGECOUNT value on done: 215.0 [2023-07-24 01:01:26,605][14525] Sum rewards: -3.859, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.257', 'weapon5': '0.002', 'AMMO5': '0.012', 'AMMO2': '0.014', 'ARMOR': '0.044', 'HITCOUNT': '0.050', 'AMMO4': '0.071', 'AMMO3': '0.116', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.300', 'weapon4': '0.322', 'WEAPON3': '0.650', 'weapon3': '0.698', 'FRAGCOUNT': '1.000', 'weapon2': '1.268'} [2023-07-24 01:01:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2138112. Throughput: 0: 279.0. Samples: 535744. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:01:29,632][00294] Avg episode reward: [(0, '-6.299')] [2023-07-24 01:01:30,868][14529] DAMAGECOUNT value on done: 318.0 [2023-07-24 01:01:31,436][14531] DAMAGECOUNT value on done: 142.0 [2023-07-24 01:01:32,880][14526] DAMAGECOUNT value on done: 381.0 [2023-07-24 01:01:32,953][14530] DAMAGECOUNT value on done: 280.0 [2023-07-24 01:01:32,959][14530] Sum rewards: -2.511, reward structure: {'DEATHCOUNT': '-7.500', 'WEAPON1': '0.010', 'AMMO5': '0.012', 'AMMO2': '0.016', 'HITCOUNT': '0.030', 'AMMO3': '0.068', 'AMMO4': '0.079', 'DAMAGECOUNT': '0.105', 'weapon5': '0.120', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'weapon4': '0.198', 'HEALTH': '0.207', 'WEAPON3': '0.350', 'ARMOR': '0.512', 'weapon3': '0.860', 'FRAGCOUNT': '1.000', 'weapon2': '1.122'} [2023-07-24 01:01:33,272][14525] DAMAGECOUNT value on done: 323.0 [2023-07-24 01:01:34,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 2142208. Throughput: 0: 287.5. Samples: 537504. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:01:34,633][00294] Avg episode reward: [(0, '-6.309')] [2023-07-24 01:01:39,313][14530] DAMAGECOUNT value on done: 332.0 [2023-07-24 01:01:39,319][14526] DAMAGECOUNT value on done: 205.0 [2023-07-24 01:01:39,320][14526] Sum rewards: -6.156, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.980', 'AMMO2': '0.001', 'AMMO5': '0.005', 'AMMO4': '0.005', 'WEAPON5': '0.100', 'HITCOUNT': '0.130', 'AMMO3': '0.176', 'DAMAGECOUNT': '0.540', 'WEAPON3': '0.800', 'weapon2': '0.932', 'FRAGCOUNT': '1.000', 'weapon3': '1.634'} [2023-07-24 01:01:39,585][14525] DAMAGECOUNT value on done: 241.0 [2023-07-24 01:01:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 2150400. Throughput: 0: 292.0. Samples: 538372. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:01:39,631][00294] Avg episode reward: [(0, '-6.328')] [2023-07-24 01:01:44,456][14526] DAMAGECOUNT value on done: 400.0 [2023-07-24 01:01:44,461][14526] Sum rewards: -7.769, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-3.192', 'AMMO4': '-0.027', 'AMMO2': '-0.005', 'AMMO5': '0.015', 'weapon5': '0.030', 'ARMOR': '0.080', 'HITCOUNT': '0.140', 'AMMO3': '0.199', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.525', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.150', 'weapon2': '1.300', 'weapon3': '1.316'} [2023-07-24 01:01:44,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1228.9, 300 sec: 1332.9). Total num frames: 2162688. Throughput: 0: 321.9. Samples: 541016. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:01:44,634][00294] Avg episode reward: [(0, '-6.363')] [2023-07-24 01:01:44,791][14525] DAMAGECOUNT value on done: 102.0 [2023-07-24 01:01:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 2166784. Throughput: 0: 343.8. Samples: 543304. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:01:49,638][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:01:50,471][14527] Updated weights for policy 0, policy_version 530 (0.0033) [2023-07-24 01:01:54,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1297.1, 300 sec: 1319.0). Total num frames: 2174976. Throughput: 0: 346.8. Samples: 544180. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:01:54,635][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:01:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 2179072. Throughput: 0: 338.7. Samples: 545900. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:01:59,634][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:01:59,653][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000532_2179072.pth... [2023-07-24 01:01:59,885][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000458_1875968.pth [2023-07-24 01:02:04,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2187264. Throughput: 0: 319.6. Samples: 547588. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:02:04,635][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 2195456. Throughput: 0: 328.1. Samples: 548832. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:02:09,638][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 2203648. Throughput: 0: 350.4. Samples: 551512. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:02:14,634][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:19,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2207744. Throughput: 0: 354.5. Samples: 553456. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:02:19,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:19,840][14527] Updated weights for policy 0, policy_version 540 (0.0026) [2023-07-24 01:02:24,629][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 2211840. Throughput: 0: 354.1. Samples: 554308. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:02:24,634][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:29,629][00294] Fps is (10 sec: 1228.9, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2220032. Throughput: 0: 333.9. Samples: 556040. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:02:29,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:34,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 2228224. Throughput: 0: 328.0. Samples: 558064. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:02:34,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:39,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 2236416. Throughput: 0: 338.1. Samples: 559396. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:02:39,632][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:44,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2244608. Throughput: 0: 355.7. Samples: 561908. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:02:44,634][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:49,632][00294] Fps is (10 sec: 1228.5, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2248704. Throughput: 0: 356.9. Samples: 563648. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:02:49,635][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:51,467][14527] Updated weights for policy 0, policy_version 550 (0.0039) [2023-07-24 01:02:54,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1365.4, 300 sec: 1291.3). Total num frames: 2256896. Throughput: 0: 348.1. Samples: 564496. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:02:54,635][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:02:59,628][00294] Fps is (10 sec: 819.5, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2256896. Throughput: 0: 320.5. Samples: 565936. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:02:59,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2265088. Throughput: 0: 307.3. Samples: 567284. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:03:04,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2269184. Throughput: 0: 306.9. Samples: 568120. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:03:09,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1263.5). Total num frames: 2273280. Throughput: 0: 305.9. Samples: 569804. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:03:14,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 2281472. Throughput: 0: 297.7. Samples: 571460. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:03:19,642][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 2285568. Throughput: 0: 287.5. Samples: 572332. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:03:24,638][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:28,984][14527] Updated weights for policy 0, policy_version 560 (0.0048) [2023-07-24 01:03:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2293760. Throughput: 0: 269.1. Samples: 574016. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:03:29,636][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2301952. Throughput: 0: 277.4. Samples: 576128. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:03:34,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2310144. Throughput: 0: 287.6. Samples: 577440. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:03:39,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.6, 300 sec: 1291.3). Total num frames: 2314240. Throughput: 0: 308.2. Samples: 579804. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:03:44,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:49,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.6, 300 sec: 1277.4). Total num frames: 2318336. Throughput: 0: 316.3. Samples: 581516. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:03:49,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 2326528. Throughput: 0: 316.0. Samples: 582340. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:03:54,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2330624. Throughput: 0: 315.6. Samples: 584004. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:03:59,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:03:59,689][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000570_2334720.pth... [2023-07-24 01:03:59,686][14527] Updated weights for policy 0, policy_version 570 (0.0036) [2023-07-24 01:03:59,881][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000495_2027520.pth [2023-07-24 01:04:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 2342912. Throughput: 0: 334.3. Samples: 586504. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:04:04,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:09,630][00294] Fps is (10 sec: 1638.2, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 2347008. Throughput: 0: 344.5. Samples: 587836. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:04:09,640][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2355200. Throughput: 0: 350.3. Samples: 589780. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:04:14,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:19,630][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.0, 300 sec: 1277.4). Total num frames: 2359296. Throughput: 0: 341.3. Samples: 591488. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:04:19,636][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2367488. Throughput: 0: 331.3. Samples: 592348. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:04:24,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:28,824][14527] Updated weights for policy 0, policy_version 580 (0.0023) [2023-07-24 01:04:29,628][00294] Fps is (10 sec: 1638.6, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2375680. Throughput: 0: 322.8. Samples: 594332. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:04:29,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:31,714][14526] Large shaping reward -2.549 for [('FRAGCOUNT', -1.5, -1.0), ('DEATHCOUNT', -0.75, 1.0), ('HEALTH', -0.3, -100.0), ('AMMO5', -0.0005, -1.0), ('weapon5', 0.002)] [2023-07-24 01:04:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2383872. Throughput: 0: 344.4. Samples: 597016. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:04:34,636][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:39,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2392064. Throughput: 0: 353.5. Samples: 598248. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:04:39,632][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:44,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2396160. Throughput: 0: 354.7. Samples: 599968. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:04:44,632][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:49,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2400256. Throughput: 0: 337.9. Samples: 601708. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:04:49,632][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:54,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2408448. Throughput: 0: 327.4. Samples: 602568. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:04:54,634][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:04:58,035][14527] Updated weights for policy 0, policy_version 590 (0.0047) [2023-07-24 01:04:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 2416640. Throughput: 0: 336.3. Samples: 604912. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:04:59,632][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:00,364][14524] Large shaping reward -2.561 for [('FRAGCOUNT', -1.5, -1.0), ('DEATHCOUNT', -0.75, 1.0), ('HEALTH', 0.17500000000000002, 35.0), ('AMMO2', -0.0049, -49.0), ('WEAPON3', -0.05, -1.0), ('AMMO3', -0.005, -10.0), ('WEAPON4', -0.05, -1.0), ('AMMO4', -0.0245, -49.0), ('WEAPON5', -0.05, -1.0), ('AMMO5', -0.002, -4.0), ('AMMO6', -0.1, -100.0), ('WEAPON7', -0.1, -1.0), ('AMMO7', -0.1, -100.0)] [2023-07-24 01:05:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2424832. Throughput: 0: 357.6. Samples: 607580. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:05:04,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1291.3). Total num frames: 2428928. Throughput: 0: 355.2. Samples: 608332. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:05:09,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2433024. Throughput: 0: 340.5. Samples: 609656. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:05:14,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1291.3). Total num frames: 2441216. Throughput: 0: 311.5. Samples: 611032. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:05:19,634][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:24,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2441216. Throughput: 0: 298.8. Samples: 611692. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:05:24,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:29,631][00294] Fps is (10 sec: 818.9, 60 sec: 1228.7, 300 sec: 1277.4). Total num frames: 2449408. Throughput: 0: 292.2. Samples: 613116. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:05:29,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:33,826][14527] Updated weights for policy 0, policy_version 600 (0.0022) [2023-07-24 01:05:34,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2457600. Throughput: 0: 298.8. Samples: 615156. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:05:34,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:39,628][00294] Fps is (10 sec: 1638.9, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2465792. Throughput: 0: 309.7. Samples: 616504. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:05:39,630][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2473984. Throughput: 0: 310.7. Samples: 618892. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:05:44,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2478080. Throughput: 0: 289.6. Samples: 620612. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:05:49,636][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:54,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2482176. Throughput: 0: 291.9. Samples: 621468. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:05:54,634][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 2490368. Throughput: 0: 300.8. Samples: 623192. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:05:59,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:05:59,648][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000608_2490368.pth... [2023-07-24 01:05:59,851][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000532_2179072.pth [2023-07-24 01:06:04,057][14527] Updated weights for policy 0, policy_version 610 (0.0046) [2023-07-24 01:06:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 2498560. Throughput: 0: 325.0. Samples: 625656. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:06:04,636][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1319.0). Total num frames: 2506752. Throughput: 0: 339.5. Samples: 626968. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:06:09,635][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 2510848. Throughput: 0: 352.6. Samples: 628984. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:06:14,639][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 2519040. Throughput: 0: 346.0. Samples: 630724. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:06:19,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:24,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2523136. Throughput: 0: 334.9. Samples: 631576. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:06:24,632][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1319.1). Total num frames: 2531328. Throughput: 0: 323.6. Samples: 633456. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:06:29,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:33,383][14527] Updated weights for policy 0, policy_version 620 (0.0035) [2023-07-24 01:06:34,628][00294] Fps is (10 sec: 1638.6, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 2539520. Throughput: 0: 344.6. Samples: 636120. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:06:34,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:39,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2547712. Throughput: 0: 354.0. Samples: 637400. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:06:39,633][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 2551808. Throughput: 0: 354.2. Samples: 639132. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:06:44,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:49,628][00294] Fps is (10 sec: 819.3, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2555904. Throughput: 0: 338.0. Samples: 640864. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:06:49,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2564096. Throughput: 0: 328.3. Samples: 641740. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:06:54,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:06:54,776][14532] Large shaping reward -2.569 for [('FRAGCOUNT', -1.5, -1.0), ('DEATHCOUNT', -0.75, 1.0), ('AMMO2', 0.0004, 2.0), ('WEAPON3', -0.05, -1.0), ('AMMO4', 0.002, 2.0), ('WEAPON5', -0.05, -1.0), ('AMMO5', -0.0015, -3.0), ('AMMO6', -0.06, -60.0), ('WEAPON7', -0.1, -1.0), ('AMMO7', -0.06, -60.0)] [2023-07-24 01:06:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2572288. Throughput: 0: 333.0. Samples: 643968. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:06:59,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:07:02,606][14527] Updated weights for policy 0, policy_version 630 (0.0034) [2023-07-24 01:07:04,633][00294] Fps is (10 sec: 1637.7, 60 sec: 1365.2, 300 sec: 1305.1). Total num frames: 2580480. Throughput: 0: 354.3. Samples: 646668. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:07:04,635][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:07:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2588672. Throughput: 0: 357.5. Samples: 647664. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:07:09,639][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:07:14,628][00294] Fps is (10 sec: 1229.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2592768. Throughput: 0: 353.4. Samples: 649360. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:07:14,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:07:19,630][00294] Fps is (10 sec: 819.1, 60 sec: 1297.0, 300 sec: 1305.2). Total num frames: 2596864. Throughput: 0: 325.9. Samples: 650788. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:07:19,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:07:24,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2600960. Throughput: 0: 312.3. Samples: 651452. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:07:24,637][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:07:29,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2609152. Throughput: 0: 305.7. Samples: 652888. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:07:29,631][00294] Avg episode reward: [(0, '-6.384')] [2023-07-24 01:07:30,088][14532] DAMAGECOUNT value on done: 199.0 [2023-07-24 01:07:30,916][14524] DAMAGECOUNT value on done: 484.0 [2023-07-24 01:07:30,926][14524] Sum rewards: -3.466, reward structure: {'DEATHCOUNT': '-7.500', 'FRAGCOUNT': '-0.500', 'HEALTH': '-0.302', 'WEAPON1': '0.010', 'AMMO5': '0.015', 'AMMO2': '0.016', 'weapon7': '0.032', 'HITCOUNT': '0.040', 'ARMOR': '0.052', 'weapon5': '0.052', 'AMMO4': '0.082', 'AMMO3': '0.101', 'DAMAGECOUNT': '0.120', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON7': '0.200', 'WEAPON5': '0.200', 'WEAPON4': '0.250', 'WEAPON3': '0.550', 'weapon4': '0.574', 'weapon2': '0.972', 'weapon3': '1.250'} [2023-07-24 01:07:31,585][14528] DAMAGECOUNT value on done: 274.0 [2023-07-24 01:07:31,590][14528] Sum rewards: -6.505, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.644', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.008', 'WEAPON1': '0.010', 'HITCOUNT': '0.010', 'AMMO5': '0.013', 'ARMOR': '0.024', 'weapon4': '0.032', 'AMMO4': '0.038', 'weapon7': '0.068', 'WEAPON4': '0.100', 'AMMO6': '0.120', 'AMMO7': '0.120', 'weapon5': '0.128', 'AMMO3': '0.144', 'WEAPON7': '0.200', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.210', 'WEAPON3': '0.850', 'weapon2': '0.918', 'weapon3': '1.446'} [2023-07-24 01:07:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2613248. Throughput: 0: 304.4. Samples: 654560. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:07:34,633][00294] Avg episode reward: [(0, '-6.274')] [2023-07-24 01:07:37,040][14532] DAMAGECOUNT value on done: 618.0 [2023-07-24 01:07:37,043][14532] Sum rewards: -4.142, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.920', 'AMMO5': '0.012', 'WEAPON1': '0.020', 'AMMO2': '0.039', 'HITCOUNT': '0.070', 'ARMOR': '0.084', 'weapon5': '0.086', 'AMMO3': '0.128', 'AMMO4': '0.196', 'weapon4': '0.228', 'WEAPON5': '0.250', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.600', 'WEAPON3': '0.800', 'weapon3': '1.246', 'weapon2': '1.518', 'FRAGCOUNT': '2.000'} [2023-07-24 01:07:37,626][14524] DAMAGECOUNT value on done: 451.0 [2023-07-24 01:07:37,627][14524] Sum rewards: -8.054, reward structure: {'DEATHCOUNT': '-11.250', 'FRAGCOUNT': '-2.000', 'HEALTH': '-0.510', 'weapon7': '0.006', 'AMMO5': '0.012', 'AMMO2': '0.015', 'weapon5': '0.018', 'AMMO4': '0.074', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.148', 'HITCOUNT': '0.160', 'weapon4': '0.194', 'WEAPON4': '0.200', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.750', 'WEAPON3': '0.850', 'weapon2': '1.306', 'weapon3': '1.422'} [2023-07-24 01:07:37,946][14531] DAMAGECOUNT value on done: 538.0 [2023-07-24 01:07:37,956][14531] Sum rewards: 1.957, reward structure: {'DEATHCOUNT': '-4.500', 'HEALTH': '-0.101', 'AMMO2': '0.014', 'AMMO4': '0.070', 'WEAPON4': '0.100', 'AMMO3': '0.102', 'ARMOR': '0.108', 'HITCOUNT': '0.120', 'weapon4': '0.200', 'WEAPON3': '0.450', 'DAMAGECOUNT': '0.699', 'weapon3': '1.294', 'weapon2': '1.400', 'FRAGCOUNT': '2.000'} [2023-07-24 01:07:38,377][14528] DAMAGECOUNT value on done: 398.0 [2023-07-24 01:07:38,382][14528] Sum rewards: -6.265, reward structure: {'DEATHCOUNT': '-9.750', 'FRAGCOUNT': '-2.000', 'HEALTH': '-0.477', 'AMMO2': '0.005', 'WEAPON1': '0.010', 'AMMO5': '0.022', 'AMMO4': '0.024', 'WEAPON4': '0.050', 'weapon7': '0.078', 'AMMO3': '0.108', 'AMMO6': '0.120', 'AMMO7': '0.120', 'weapon4': '0.156', 'HITCOUNT': '0.170', 'weapon5': '0.176', 'WEAPON7': '0.200', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.495', 'ARMOR': '0.548', 'WEAPON3': '0.700', 'weapon2': '1.236', 'weapon3': '1.444'} [2023-07-24 01:07:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.6, 300 sec: 1263.5). Total num frames: 2617344. Throughput: 0: 305.6. Samples: 655492. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:07:39,631][00294] Avg episode reward: [(0, '-6.152')] [2023-07-24 01:07:40,703][14527] Updated weights for policy 0, policy_version 640 (0.0070) [2023-07-24 01:07:43,340][14532] DAMAGECOUNT value on done: 464.0 [2023-07-24 01:07:43,340][14532] Sum rewards: 2.021, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.545', 'AMMO2': '0.002', 'AMMO5': '0.007', 'AMMO4': '0.008', 'weapon5': '0.062', 'AMMO3': '0.080', 'HITCOUNT': '0.150', 'WEAPON5': '0.150', 'WEAPON3': '0.500', 'DAMAGECOUNT': '0.780', 'weapon3': '1.480', 'weapon2': '1.596', 'FRAGCOUNT': '3.000'} [2023-07-24 01:07:44,082][14524] DAMAGECOUNT value on done: 339.0 [2023-07-24 01:07:44,083][14524] Sum rewards: -4.951, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.660', 'AMMO5': '0.007', 'AMMO2': '0.024', 'AMMO4': '0.120', 'WEAPON5': '0.150', 'AMMO3': '0.154', 'HITCOUNT': '0.190', 'weapon4': '0.234', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.675', 'WEAPON3': '0.900', 'weapon3': '1.396', 'weapon2': '1.608', 'FRAGCOUNT': '3.000'} [2023-07-24 01:07:44,525][14531] DAMAGECOUNT value on done: 528.0 [2023-07-24 01:07:44,530][14531] Sum rewards: -2.714, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.562', 'weapon7': '0.006', 'AMMO2': '0.007', 'AMMO5': '0.010', 'WEAPON1': '0.010', 'AMMO4': '0.033', 'weapon5': '0.078', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.133', 'HITCOUNT': '0.190', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.795', 'WEAPON3': '0.800', 'weapon3': '1.506', 'weapon2': '1.530', 'FRAGCOUNT': '2.000'} [2023-07-24 01:07:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2625536. Throughput: 0: 293.5. Samples: 657176. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:07:44,634][00294] Avg episode reward: [(0, '-6.017')] [2023-07-24 01:07:44,820][14528] DAMAGECOUNT value on done: 441.0 [2023-07-24 01:07:45,665][14529] DAMAGECOUNT value on done: 613.0 [2023-07-24 01:07:45,666][14529] Sum rewards: -2.984, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.530', 'AMMO5': '0.009', 'AMMO2': '0.012', 'AMMO4': '0.061', 'weapon7': '0.064', 'weapon5': '0.068', 'HITCOUNT': '0.150', 'WEAPON5': '0.150', 'AMMO3': '0.162', 'weapon4': '0.172', 'WEAPON4': '0.200', 'AMMO6': '0.360', 'AMMO7': '0.360', 'WEAPON7': '0.400', 'ARMOR': '0.482', 'DAMAGECOUNT': '0.699', 'weapon2': '0.850', 'WEAPON3': '0.900', 'weapon3': '1.446', 'FRAGCOUNT': '2.000'} [2023-07-24 01:07:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 2629632. Throughput: 0: 270.3. Samples: 658832. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:07:49,633][00294] Avg episode reward: [(0, '-5.902')] [2023-07-24 01:07:51,044][14532] DAMAGECOUNT value on done: 276.0 [2023-07-24 01:07:51,047][14532] Sum rewards: -6.758, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-0.976', 'AMMO4': '-0.018', 'AMMO2': '-0.004', 'WEAPON1': '0.010', 'AMMO5': '0.015', 'weapon5': '0.044', 'AMMO3': '0.109', 'HITCOUNT': '0.120', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.450', 'ARMOR': '0.468', 'WEAPON3': '0.600', 'weapon3': '1.342', 'weapon2': '1.632', 'FRAGCOUNT': '2.000'} [2023-07-24 01:07:51,581][14524] DAMAGECOUNT value on done: 274.0 [2023-07-24 01:07:52,021][14531] DAMAGECOUNT value on done: 450.0 [2023-07-24 01:07:52,027][14531] Sum rewards: -8.588, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.297', 'FRAGCOUNT': '-0.500', 'weapon7': '0.008', 'AMMO5': '0.010', 'ARMOR': '0.012', 'weapon5': '0.018', 'AMMO2': '0.022', 'HITCOUNT': '0.080', 'AMMO4': '0.111', 'weapon4': '0.152', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'AMMO3': '0.226', 'DAMAGECOUNT': '0.453', 'weapon2': '1.154', 'WEAPON3': '1.300', 'weapon3': '1.662'} [2023-07-24 01:07:52,488][14529] DAMAGECOUNT value on done: 296.0 [2023-07-24 01:07:52,495][14529] Sum rewards: -7.419, reward structure: {'DEATHCOUNT': '-9.750', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.624', 'AMMO5': '0.010', 'AMMO2': '0.013', 'weapon5': '0.028', 'AMMO4': '0.063', 'ARMOR': '0.068', 'HITCOUNT': '0.070', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.119', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.195', 'WEAPON5': '0.200', 'weapon4': '0.266', 'WEAPON3': '0.650', 'weapon2': '1.078', 'weapon3': '1.246'} [2023-07-24 01:07:52,633][14528] DAMAGECOUNT value on done: 271.0 [2023-07-24 01:07:54,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 2633728. Throughput: 0: 266.8. Samples: 659672. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:07:54,631][00294] Avg episode reward: [(0, '-5.866')] [2023-07-24 01:07:55,271][14530] DAMAGECOUNT value on done: 294.0 [2023-07-24 01:07:55,272][14530] Sum rewards: -8.988, reward structure: {'DEATHCOUNT': '-14.250', 'HEALTH': '-1.152', 'weapon5': '0.012', 'AMMO5': '0.015', 'AMMO2': '0.039', 'HITCOUNT': '0.080', 'ARMOR': '0.104', 'AMMO4': '0.197', 'WEAPON5': '0.200', 'AMMO3': '0.211', 'DAMAGECOUNT': '0.255', 'WEAPON4': '0.400', 'weapon4': '0.418', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.050', 'weapon3': '1.136', 'weapon2': '1.296'} [2023-07-24 01:07:56,839][14532] DAMAGECOUNT value on done: 344.0 [2023-07-24 01:07:56,851][14532] Sum rewards: -2.340, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.548', 'AMMO2': '0.023', 'weapon7': '0.054', 'AMMO3': '0.080', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'ARMOR': '0.103', 'AMMO4': '0.113', 'HITCOUNT': '0.200', 'WEAPON4': '0.350', 'DAMAGECOUNT': '0.495', 'weapon4': '0.512', 'WEAPON3': '0.550', 'weapon2': '0.896', 'FRAGCOUNT': '1.000', 'weapon3': '1.282'} [2023-07-24 01:07:57,237][14524] DAMAGECOUNT value on done: 664.0 [2023-07-24 01:07:57,243][14524] Sum rewards: -3.690, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.496', 'AMMO4': '-0.001', 'AMMO2': '-0.000', 'AMMO5': '0.019', 'weapon5': '0.028', 'AMMO3': '0.161', 'HITCOUNT': '0.230', 'WEAPON5': '0.300', 'WEAPON3': '1.000', 'DAMAGECOUNT': '1.059', 'weapon2': '1.156', 'weapon3': '1.854', 'FRAGCOUNT': '3.500'} [2023-07-24 01:07:57,602][14531] DAMAGECOUNT value on done: 419.0 [2023-07-24 01:07:57,605][14531] Sum rewards: -4.259, reward structure: {'DEATHCOUNT': '-11.250', 'AMMO2': '0.011', 'AMMO5': '0.017', 'AMMO4': '0.055', 'weapon5': '0.058', 'AMMO3': '0.154', 'HITCOUNT': '0.240', 'WEAPON5': '0.300', 'HEALTH': '0.363', 'DAMAGECOUNT': '0.720', 'WEAPON3': '0.850', 'FRAGCOUNT': '1.000', 'weapon2': '1.290', 'weapon3': '1.932'} [2023-07-24 01:07:57,811][14528] DAMAGECOUNT value on done: 342.0 [2023-07-24 01:07:57,815][14528] Sum rewards: -4.306, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.510', 'AMMO2': '0.012', 'AMMO5': '0.013', 'WEAPON1': '0.030', 'HITCOUNT': '0.040', 'AMMO4': '0.058', 'weapon7': '0.086', 'DAMAGECOUNT': '0.105', 'AMMO3': '0.108', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON4': '0.200', 'WEAPON7': '0.200', 'WEAPON5': '0.250', 'weapon4': '0.382', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon2': '1.012', 'weapon3': '1.068'} [2023-07-24 01:07:58,172][14529] DAMAGECOUNT value on done: 231.0 [2023-07-24 01:07:58,178][14529] Sum rewards: -3.448, reward structure: {'DEATHCOUNT': '-5.250', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.720', 'AMMO5': '0.005', 'AMMO2': '0.014', 'weapon5': '0.024', 'AMMO3': '0.054', 'HITCOUNT': '0.060', 'AMMO4': '0.070', 'WEAPON5': '0.100', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.165', 'WEAPON3': '0.350', 'weapon4': '0.462', 'ARMOR': '0.492', 'weapon3': '0.874', 'weapon2': '1.202'} [2023-07-24 01:07:59,237][14524] Large shaping reward -2.534 for [('FRAGCOUNT', -1.5, -1.0), ('DEATHCOUNT', -0.75, 1.0), ('HEALTH', -0.28500000000000003, -95.0), ('AMMO5', -0.0005, -1.0), ('weapon5', 0.002)] [2023-07-24 01:07:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 2641920. Throughput: 0: 278.0. Samples: 661868. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:07:59,638][00294] Avg episode reward: [(0, '-5.610')] [2023-07-24 01:07:59,651][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000645_2641920.pth... [2023-07-24 01:07:59,855][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000570_2334720.pth [2023-07-24 01:08:00,307][14530] DAMAGECOUNT value on done: 819.0 [2023-07-24 01:08:00,309][14530] Sum rewards: -4.568, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.740', 'AMMO2': '0.010', 'AMMO5': '0.020', 'WEAPON1': '0.020', 'AMMO4': '0.047', 'ARMOR': '0.048', 'weapon5': '0.072', 'AMMO3': '0.175', 'HITCOUNT': '0.260', 'WEAPON5': '0.300', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.000', 'weapon2': '1.066', 'DAMAGECOUNT': '1.080', 'weapon3': '1.824'} [2023-07-24 01:08:00,638][14525] DAMAGECOUNT value on done: 611.0 [2023-07-24 01:08:00,642][14525] Sum rewards: -1.570, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.610', 'AMMO5': '0.007', 'AMMO2': '0.019', 'WEAPON1': '0.020', 'ARMOR': '0.032', 'weapon4': '0.052', 'AMMO4': '0.094', 'WEAPON4': '0.100', 'AMMO3': '0.163', 'weapon5': '0.164', 'WEAPON5': '0.200', 'HITCOUNT': '0.310', 'WEAPON3': '0.900', 'DAMAGECOUNT': '1.215', 'weapon3': '1.286', 'weapon2': '1.478', 'FRAGCOUNT': '3.000'} [2023-07-24 01:08:01,431][14532] DAMAGECOUNT value on done: 378.0 [2023-07-24 01:08:01,436][14532] Sum rewards: -13.646, reward structure: {'DEATHCOUNT': '-14.250', 'HEALTH': '-3.317', 'FRAGCOUNT': '-2.000', 'AMMO2': '0.006', 'AMMO5': '0.015', 'AMMO4': '0.032', 'weapon5': '0.044', 'ARMOR': '0.056', 'HITCOUNT': '0.170', 'AMMO3': '0.196', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.708', 'WEAPON3': '1.150', 'weapon2': '1.296', 'weapon3': '1.948'} [2023-07-24 01:08:01,521][14526] DAMAGECOUNT value on done: 300.0 [2023-07-24 01:08:01,796][14524] DAMAGECOUNT value on done: 720.0 [2023-07-24 01:08:01,796][14524] Sum rewards: -0.755, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.401', 'AMMO2': '0.015', 'AMMO5': '0.020', 'ARMOR': '0.036', 'weapon5': '0.042', 'weapon7': '0.042', 'weapon4': '0.062', 'AMMO4': '0.074', 'WEAPON4': '0.100', 'AMMO3': '0.150', 'AMMO6': '0.160', 'AMMO7': '0.160', 'HITCOUNT': '0.190', 'WEAPON5': '0.200', 'WEAPON7': '0.200', 'DAMAGECOUNT': '0.840', 'WEAPON3': '0.850', 'weapon2': '1.080', 'weapon3': '1.676', 'FRAGCOUNT': '3.000'} [2023-07-24 01:08:02,103][14531] DAMAGECOUNT value on done: 407.0 [2023-07-24 01:08:02,105][14531] Sum rewards: -6.422, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.479', 'AMMO5': '0.009', 'AMMO2': '0.019', 'WEAPON1': '0.020', 'weapon5': '0.034', 'ARMOR': '0.060', 'HITCOUNT': '0.080', 'AMMO4': '0.094', 'AMMO3': '0.120', 'WEAPON5': '0.200', 'WEAPON4': '0.200', 'weapon4': '0.216', 'DAMAGECOUNT': '0.312', 'WEAPON3': '0.650', 'weapon3': '1.092', 'weapon2': '1.450'} [2023-07-24 01:08:02,415][14528] DAMAGECOUNT value on done: 438.0 [2023-07-24 01:08:02,422][14528] Sum rewards: -3.531, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.998', 'AMMO2': '0.007', 'AMMO5': '0.010', 'WEAPON1': '0.010', 'weapon7': '0.014', 'AMMO4': '0.033', 'ARMOR': '0.036', 'WEAPON5': '0.050', 'WEAPON4': '0.050', 'weapon5': '0.052', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'weapon4': '0.126', 'AMMO3': '0.142', 'HITCOUNT': '0.220', 'DAMAGECOUNT': '0.609', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon3': '1.352', 'weapon2': '1.656'} [2023-07-24 01:08:03,305][14529] DAMAGECOUNT value on done: 471.0 [2023-07-24 01:08:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1160.6, 300 sec: 1291.3). Total num frames: 2650112. Throughput: 0: 302.9. Samples: 664420. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:08:04,630][00294] Avg episode reward: [(0, '-5.476')] [2023-07-24 01:08:05,708][14530] DAMAGECOUNT value on done: 314.0 [2023-07-24 01:08:05,715][14530] Sum rewards: -2.895, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.665', 'AMMO5': '0.005', 'AMMO2': '0.011', 'weapon5': '0.014', 'WEAPON4': '0.050', 'AMMO4': '0.055', 'weapon4': '0.068', 'WEAPON5': '0.100', 'HITCOUNT': '0.130', 'AMMO3': '0.142', 'ARMOR': '0.466', 'DAMAGECOUNT': '0.537', 'WEAPON3': '0.850', 'weapon3': '1.532', 'weapon2': '1.560', 'FRAGCOUNT': '2.000'} [2023-07-24 01:08:06,411][14525] DAMAGECOUNT value on done: 355.0 [2023-07-24 01:08:06,412][14525] Sum rewards: -1.890, reward structure: {'DEATHCOUNT': '-7.500', 'AMMO5': '0.013', 'AMMO2': '0.021', 'weapon5': '0.034', 'HITCOUNT': '0.060', 'ARMOR': '0.064', 'weapon7': '0.064', 'HEALTH': '0.070', 'WEAPON5': '0.100', 'AMMO4': '0.105', 'AMMO6': '0.120', 'AMMO7': '0.120', 'AMMO3': '0.144', 'WEAPON4': '0.200', 'WEAPON7': '0.200', 'weapon4': '0.398', 'DAMAGECOUNT': '0.405', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.650', 'weapon3': '1.050', 'weapon2': '1.292'} [2023-07-24 01:08:06,729][14532] DAMAGECOUNT value on done: 361.0 [2023-07-24 01:08:07,150][14524] DAMAGECOUNT value on done: 348.0 [2023-07-24 01:08:07,151][14524] Sum rewards: -6.352, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.414', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.014', 'AMMO5': '0.017', 'ARMOR': '0.040', 'WEAPON1': '0.040', 'AMMO4': '0.070', 'HITCOUNT': '0.070', 'weapon5': '0.082', 'WEAPON4': '0.100', 'AMMO3': '0.116', 'DAMAGECOUNT': '0.252', 'WEAPON5': '0.300', 'weapon4': '0.302', 'WEAPON3': '0.650', 'weapon2': '0.814', 'weapon3': '1.694'} [2023-07-24 01:08:07,613][14531] DAMAGECOUNT value on done: 415.0 [2023-07-24 01:08:07,619][14531] Sum rewards: -6.132, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.875', 'AMMO2': '0.011', 'AMMO5': '0.011', 'AMMO4': '0.054', 'HITCOUNT': '0.100', 'ARMOR': '0.120', 'AMMO3': '0.131', 'weapon5': '0.160', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'weapon4': '0.326', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.650', 'weapon3': '0.740', 'DAMAGECOUNT': '0.780', 'weapon2': '1.510'} [2023-07-24 01:08:07,694][14526] DAMAGECOUNT value on done: 619.0 [2023-07-24 01:08:08,229][14528] DAMAGECOUNT value on done: 418.0 [2023-07-24 01:08:08,233][14528] Sum rewards: -3.979, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.760', 'AMMO2': '0.009', 'AMMO5': '0.020', 'weapon4': '0.032', 'AMMO4': '0.046', 'WEAPON4': '0.100', 'AMMO3': '0.148', 'weapon5': '0.186', 'HITCOUNT': '0.210', 'WEAPON5': '0.400', 'FRAGCOUNT': '0.500', 'weapon2': '0.788', 'DAMAGECOUNT': '0.873', 'WEAPON3': '0.900', 'weapon3': '1.818'} [2023-07-24 01:08:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1160.5, 300 sec: 1305.2). Total num frames: 2658304. Throughput: 0: 309.3. Samples: 665372. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:08:09,637][00294] Avg episode reward: [(0, '-5.312')] [2023-07-24 01:08:10,899][14529] DAMAGECOUNT value on done: 432.0 [2023-07-24 01:08:12,987][14527] Updated weights for policy 0, policy_version 650 (0.0056) [2023-07-24 01:08:13,478][14532] DAMAGECOUNT value on done: 703.0 [2023-07-24 01:08:13,483][14532] Sum rewards: -6.198, reward structure: {'DEATHCOUNT': '-9.750', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.200', 'AMMO2': '0.007', 'WEAPON1': '0.010', 'AMMO5': '0.015', 'AMMO4': '0.036', 'weapon7': '0.044', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'weapon5': '0.128', 'AMMO3': '0.157', 'HITCOUNT': '0.160', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.519', 'WEAPON3': '0.700', 'weapon2': '1.312', 'weapon3': '1.614'} [2023-07-24 01:08:13,809][14524] DAMAGECOUNT value on done: 210.0 [2023-07-24 01:08:13,810][14524] Sum rewards: -0.033, reward structure: {'DEATHCOUNT': '-4.500', 'HEALTH': '-0.375', 'AMMO2': '0.002', 'AMMO4': '0.009', 'ARMOR': '0.024', 'WEAPON1': '0.040', 'AMMO3': '0.046', 'HITCOUNT': '0.050', 'weapon7': '0.080', 'AMMO6': '0.120', 'AMMO7': '0.120', 'DAMAGECOUNT': '0.135', 'WEAPON4': '0.150', 'WEAPON7': '0.200', 'WEAPON3': '0.300', 'weapon4': '0.346', 'weapon3': '0.958', 'FRAGCOUNT': '1.000', 'weapon2': '1.262'} [2023-07-24 01:08:13,934][14530] DAMAGECOUNT value on done: 445.0 [2023-07-24 01:08:13,936][14530] Sum rewards: -2.925, reward structure: {'DEATHCOUNT': '-7.500', 'FRAGCOUNT': '-0.500', 'HEALTH': '-0.290', 'AMMO2': '0.003', 'AMMO5': '0.006', 'AMMO4': '0.014', 'WEAPON1': '0.020', 'WEAPON4': '0.050', 'weapon7': '0.066', 'weapon5': '0.068', 'HITCOUNT': '0.070', 'weapon4': '0.104', 'AMMO3': '0.106', 'WEAPON5': '0.150', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON7': '0.200', 'DAMAGECOUNT': '0.390', 'ARMOR': '0.400', 'WEAPON3': '0.650', 'weapon2': '1.138', 'weapon3': '1.610'} [2023-07-24 01:08:14,454][14531] DAMAGECOUNT value on done: 480.0 [2023-07-24 01:08:14,459][14531] Sum rewards: -3.589, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.990', 'WEAPON1': '0.010', 'AMMO5': '0.012', 'AMMO2': '0.023', 'weapon5': '0.050', 'AMMO4': '0.113', 'weapon4': '0.116', 'AMMO3': '0.141', 'HITCOUNT': '0.250', 'WEAPON4': '0.250', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.690', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon2': '1.196', 'weapon3': '1.500'} [2023-07-24 01:08:14,569][14525] DAMAGECOUNT value on done: 276.0 [2023-07-24 01:08:14,570][14525] Sum rewards: -2.356, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.620', 'AMMO2': '0.005', 'weapon5': '0.014', 'AMMO5': '0.018', 'AMMO4': '0.027', 'ARMOR': '0.052', 'HITCOUNT': '0.080', 'AMMO3': '0.146', 'WEAPON4': '0.150', 'weapon4': '0.158', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.456', 'WEAPON3': '0.900', 'weapon2': '1.118', 'weapon3': '1.390', 'FRAGCOUNT': '2.000'} [2023-07-24 01:08:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 2666496. Throughput: 0: 314.8. Samples: 667056. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:08:14,633][00294] Avg episode reward: [(0, '-5.216')] [2023-07-24 01:08:15,269][14528] DAMAGECOUNT value on done: 557.0 [2023-07-24 01:08:15,274][14528] Sum rewards: -4.390, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.530', 'AMMO2': '0.010', 'AMMO5': '0.015', 'ARMOR': '0.048', 'AMMO4': '0.048', 'AMMO3': '0.120', 'weapon5': '0.164', 'HITCOUNT': '0.200', 'WEAPON4': '0.200', 'weapon4': '0.216', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.696', 'WEAPON3': '0.700', 'FRAGCOUNT': '1.000', 'weapon2': '1.050', 'weapon3': '1.372'} [2023-07-24 01:08:15,703][14526] DAMAGECOUNT value on done: 820.0 [2023-07-24 01:08:15,718][14526] Sum rewards: -7.397, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-1.288', 'AMMO5': '0.005', 'weapon5': '0.006', 'AMMO2': '0.008', 'weapon4': '0.030', 'ARMOR': '0.036', 'AMMO4': '0.039', 'WEAPON4': '0.050', 'WEAPON5': '0.100', 'AMMO3': '0.203', 'HITCOUNT': '0.260', 'weapon2': '0.782', 'DAMAGECOUNT': '0.840', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.100', 'weapon3': '2.182'} [2023-07-24 01:08:17,915][14529] DAMAGECOUNT value on done: 549.0 [2023-07-24 01:08:17,915][14529] Sum rewards: -6.200, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-0.980', 'AMMO5': '0.015', 'AMMO2': '0.029', 'weapon5': '0.076', 'HITCOUNT': '0.110', 'AMMO4': '0.145', 'AMMO3': '0.184', 'WEAPON5': '0.200', 'WEAPON4': '0.250', 'weapon4': '0.264', 'DAMAGECOUNT': '0.396', 'ARMOR': '0.517', 'weapon2': '0.972', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.000', 'weapon3': '1.622'} [2023-07-24 01:08:19,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.6, 300 sec: 1291.3). Total num frames: 2666496. Throughput: 0: 314.6. Samples: 668716. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:08:19,636][00294] Avg episode reward: [(0, '-5.120')] [2023-07-24 01:08:21,300][14530] DAMAGECOUNT value on done: 403.0 [2023-07-24 01:08:21,302][14530] Sum rewards: -9.312, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.102', 'FRAGCOUNT': '-1.500', 'AMMO5': '0.005', 'AMMO2': '0.026', 'AMMO3': '0.096', 'weapon5': '0.098', 'HITCOUNT': '0.130', 'AMMO4': '0.131', 'WEAPON5': '0.150', 'weapon4': '0.244', 'WEAPON4': '0.400', 'DAMAGECOUNT': '0.432', 'WEAPON3': '0.500', 'ARMOR': '0.522', 'weapon3': '0.562', 'weapon2': '2.244'} [2023-07-24 01:08:21,849][14525] DAMAGECOUNT value on done: 493.0 [2023-07-24 01:08:21,855][14525] Sum rewards: -7.169, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.910', 'AMMO2': '0.001', 'AMMO4': '0.007', 'AMMO5': '0.010', 'weapon5': '0.012', 'WEAPON1': '0.020', 'ARMOR': '0.084', 'HITCOUNT': '0.120', 'WEAPON5': '0.200', 'AMMO3': '0.231', 'FRAGCOUNT': '0.500', 'DAMAGECOUNT': '0.510', 'weapon2': '0.940', 'WEAPON3': '1.300', 'weapon3': '2.056'} [2023-07-24 01:08:22,116][14531] DAMAGECOUNT value on done: 424.0 [2023-07-24 01:08:22,123][14531] Sum rewards: -2.892, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.208', 'AMMO4': '-0.020', 'AMMO2': '-0.004', 'WEAPON1': '0.010', 'AMMO5': '0.017', 'AMMO3': '0.100', 'weapon7': '0.104', 'weapon5': '0.166', 'HITCOUNT': '0.190', 'WEAPON5': '0.250', 'AMMO6': '0.320', 'AMMO7': '0.320', 'WEAPON7': '0.400', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.846', 'weapon2': '1.346', 'weapon3': '1.420'} [2023-07-24 01:08:22,707][14526] DAMAGECOUNT value on done: 465.0 [2023-07-24 01:08:24,517][14529] DAMAGECOUNT value on done: 498.0 [2023-07-24 01:08:24,518][14529] Sum rewards: -4.625, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.020', 'AMMO2': '0.011', 'AMMO5': '0.012', 'weapon4': '0.026', 'weapon5': '0.048', 'WEAPON4': '0.050', 'AMMO4': '0.054', 'AMMO3': '0.157', 'WEAPON5': '0.200', 'HITCOUNT': '0.220', 'DAMAGECOUNT': '0.885', 'WEAPON3': '0.900', 'weapon2': '0.960', 'weapon3': '1.872', 'FRAGCOUNT': '3.000'} [2023-07-24 01:08:24,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2674688. Throughput: 0: 312.1. Samples: 669536. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:08:24,632][00294] Avg episode reward: [(0, '-5.247')] [2023-07-24 01:08:26,423][14530] DAMAGECOUNT value on done: 741.0 [2023-07-24 01:08:26,425][14530] Sum rewards: -5.289, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.533', 'FRAGCOUNT': '-1.000', 'AMMO2': '0.017', 'AMMO5': '0.021', 'WEAPON1': '0.030', 'weapon7': '0.032', 'AMMO4': '0.084', 'AMMO3': '0.108', 'weapon4': '0.114', 'WEAPON4': '0.150', 'HITCOUNT': '0.160', 'weapon5': '0.160', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON5': '0.450', 'DAMAGECOUNT': '0.498', 'WEAPON3': '0.650', 'weapon2': '1.078', 'weapon3': '1.342'} [2023-07-24 01:08:26,927][14525] DAMAGECOUNT value on done: 270.0 [2023-07-24 01:08:27,519][14526] DAMAGECOUNT value on done: 383.0 [2023-07-24 01:08:27,521][14526] Sum rewards: -3.387, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.038', 'AMMO5': '0.010', 'AMMO2': '0.017', 'ARMOR': '0.048', 'weapon5': '0.058', 'AMMO4': '0.084', 'AMMO3': '0.140', 'HITCOUNT': '0.140', 'WEAPON4': '0.150', 'WEAPON5': '0.200', 'weapon4': '0.206', 'DAMAGECOUNT': '0.396', 'WEAPON3': '0.850', 'weapon2': '1.290', 'FRAGCOUNT': '1.500', 'weapon3': '1.562'} [2023-07-24 01:08:29,269][14529] DAMAGECOUNT value on done: 368.0 [2023-07-24 01:08:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2682880. Throughput: 0: 329.8. Samples: 672016. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:08:29,630][00294] Avg episode reward: [(0, '-5.160')] [2023-07-24 01:08:31,450][14530] DAMAGECOUNT value on done: 330.0 [2023-07-24 01:08:31,878][14525] DAMAGECOUNT value on done: 362.0 [2023-07-24 01:08:31,883][14525] Sum rewards: -10.425, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-3.000', 'HEALTH': '-2.507', 'AMMO4': '-0.019', 'AMMO2': '-0.004', 'AMMO5': '0.033', 'ARMOR': '0.040', 'HITCOUNT': '0.040', 'AMMO3': '0.086', 'weapon5': '0.110', 'DAMAGECOUNT': '0.117', 'WEAPON5': '0.350', 'WEAPON3': '0.500', 'weapon3': '0.946', 'weapon2': '1.882'} [2023-07-24 01:08:32,356][14526] DAMAGECOUNT value on done: 491.0 [2023-07-24 01:08:32,358][14526] Sum rewards: -8.510, reward structure: {'DEATHCOUNT': '-9.750', 'FRAGCOUNT': '-2.000', 'HEALTH': '-1.823', 'AMMO2': '0.006', 'WEAPON1': '0.020', 'AMMO5': '0.022', 'AMMO4': '0.030', 'ARMOR': '0.052', 'HITCOUNT': '0.070', 'weapon5': '0.076', 'AMMO3': '0.157', 'DAMAGECOUNT': '0.330', 'WEAPON5': '0.450', 'WEAPON3': '0.950', 'weapon2': '1.108', 'weapon3': '1.792'} [2023-07-24 01:08:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2691072. Throughput: 0: 348.1. Samples: 674496. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:08:34,634][00294] Avg episode reward: [(0, '-5.275')] [2023-07-24 01:08:38,167][14530] DAMAGECOUNT value on done: 360.0 [2023-07-24 01:08:38,173][14530] Sum rewards: -11.727, reward structure: {'DEATHCOUNT': '-11.250', 'FRAGCOUNT': '-3.000', 'HEALTH': '-2.064', 'AMMO5': '0.011', 'AMMO2': '0.011', 'WEAPON1': '0.020', 'HITCOUNT': '0.030', 'ARMOR': '0.048', 'AMMO4': '0.056', 'weapon5': '0.060', 'DAMAGECOUNT': '0.084', 'weapon4': '0.132', 'AMMO3': '0.164', 'WEAPON4': '0.200', 'WEAPON5': '0.250', 'WEAPON3': '0.850', 'weapon3': '1.322', 'weapon2': '1.348'} [2023-07-24 01:08:39,170][14525] DAMAGECOUNT value on done: 575.0 [2023-07-24 01:08:39,171][14525] Sum rewards: -2.599, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.489', 'AMMO2': '0.001', 'AMMO4': '0.006', 'AMMO5': '0.015', 'WEAPON1': '0.030', 'ARMOR': '0.040', 'AMMO3': '0.139', 'HITCOUNT': '0.180', 'weapon5': '0.182', 'WEAPON5': '0.300', 'WEAPON3': '0.800', 'DAMAGECOUNT': '1.002', 'weapon2': '1.086', 'weapon3': '1.858', 'FRAGCOUNT': '2.000'} [2023-07-24 01:08:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2699264. Throughput: 0: 348.7. Samples: 675364. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:08:39,634][00294] Avg episode reward: [(0, '-5.287')] [2023-07-24 01:08:39,846][14526] DAMAGECOUNT value on done: 515.0 [2023-07-24 01:08:39,848][14526] Sum rewards: -2.010, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.297', 'AMMO5': '0.005', 'AMMO2': '0.028', 'ARMOR': '0.034', 'weapon5': '0.066', 'WEAPON5': '0.100', 'AMMO4': '0.139', 'AMMO3': '0.155', 'HITCOUNT': '0.240', 'WEAPON4': '0.300', 'weapon4': '0.452', 'WEAPON3': '0.650', 'weapon2': '0.814', 'DAMAGECOUNT': '0.930', 'weapon3': '1.374', 'FRAGCOUNT': '1.500'} [2023-07-24 01:08:42,762][14527] Updated weights for policy 0, policy_version 660 (0.0030) [2023-07-24 01:08:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 2703360. Throughput: 0: 338.2. Samples: 677088. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:08:44,636][00294] Avg episode reward: [(0, '-5.279')] [2023-07-24 01:08:46,408][14525] DAMAGECOUNT value on done: 349.0 [2023-07-24 01:08:46,408][14525] Sum rewards: -4.848, reward structure: {'DEATHCOUNT': '-8.250', 'FRAGCOUNT': '-1.000', 'HEALTH': '-0.332', 'AMMO4': '-0.002', 'AMMO2': '-0.000', 'AMMO5': '0.010', 'WEAPON1': '0.030', 'ARMOR': '0.040', 'HITCOUNT': '0.060', 'AMMO3': '0.125', 'weapon5': '0.142', 'WEAPON5': '0.150', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.735', 'weapon2': '1.082', 'weapon3': '1.662'} [2023-07-24 01:08:46,977][14526] DAMAGECOUNT value on done: 480.0 [2023-07-24 01:08:46,981][14526] Sum rewards: -1.786, reward structure: {'DEATHCOUNT': '-5.250', 'FRAGCOUNT': '-0.500', 'HEALTH': '-0.181', 'AMMO2': '0.008', 'AMMO5': '0.010', 'WEAPON1': '0.020', 'AMMO4': '0.040', 'weapon5': '0.056', 'AMMO3': '0.073', 'HITCOUNT': '0.080', 'weapon7': '0.084', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'WEAPON4': '0.100', 'ARMOR': '0.108', 'WEAPON5': '0.150', 'DAMAGECOUNT': '0.240', 'weapon4': '0.250', 'WEAPON3': '0.400', 'weapon3': '1.046', 'weapon2': '1.180'} [2023-07-24 01:08:49,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2707456. Throughput: 0: 320.3. Samples: 678832. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:08:49,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:08:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 2719744. Throughput: 0: 323.0. Samples: 679908. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:08:54,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:08:59,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 2727936. Throughput: 0: 346.6. Samples: 682652. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:08:59,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:04,630][00294] Fps is (10 sec: 1638.2, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 2736128. Throughput: 0: 358.8. Samples: 684864. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:09:04,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2740224. Throughput: 0: 359.7. Samples: 685724. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:09:09,635][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:11,223][14527] Updated weights for policy 0, policy_version 670 (0.0035) [2023-07-24 01:09:14,628][00294] Fps is (10 sec: 819.3, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 2744320. Throughput: 0: 343.5. Samples: 687472. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:09:14,636][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 2752512. Throughput: 0: 328.4. Samples: 689276. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:09:19,639][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 2760704. Throughput: 0: 339.0. Samples: 690620. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:09:24,630][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:29,631][00294] Fps is (10 sec: 1638.0, 60 sec: 1433.5, 300 sec: 1305.2). Total num frames: 2768896. Throughput: 0: 351.8. Samples: 692920. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:09:29,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:34,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 2768896. Throughput: 0: 343.9. Samples: 694308. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:09:34,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:39,631][00294] Fps is (10 sec: 819.2, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 2777088. Throughput: 0: 334.9. Samples: 694980. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:09:39,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:44,630][00294] Fps is (10 sec: 819.1, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2777088. Throughput: 0: 304.1. Samples: 696336. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:09:44,638][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:47,136][14527] Updated weights for policy 0, policy_version 680 (0.0088) [2023-07-24 01:09:49,628][00294] Fps is (10 sec: 819.4, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 2785280. Throughput: 0: 283.7. Samples: 697632. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:09:49,630][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:51,282][14526] Large shaping reward -2.634 for [('FRAGCOUNT', -1.5, -1.0), ('DEATHCOUNT', -0.75, 1.0), ('HEALTH', 0.025, 5.0), ('AMMO2', 0.0004, 2.0), ('WEAPON3', -0.05, -1.0), ('AMMO3', -0.009000000000000001, -18.0), ('AMMO4', 0.002, 2.0), ('WEAPON5', -0.05, -1.0), ('AMMO5', -0.0025, -5.0), ('AMMO6', -0.1, -100.0), ('WEAPON7', -0.1, -1.0), ('AMMO7', -0.1, -100.0)] [2023-07-24 01:09:54,628][00294] Fps is (10 sec: 1638.6, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2793472. Throughput: 0: 283.7. Samples: 698492. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:09:54,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2801664. Throughput: 0: 300.7. Samples: 701004. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:09:59,636][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:09:59,655][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000684_2801664.pth... [2023-07-24 01:09:59,861][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000608_2490368.pth [2023-07-24 01:10:04,632][00294] Fps is (10 sec: 1637.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2809856. Throughput: 0: 315.1. Samples: 703456. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:10:04,642][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2813952. Throughput: 0: 305.2. Samples: 704352. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:10:09,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:14,628][00294] Fps is (10 sec: 819.5, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2818048. Throughput: 0: 292.1. Samples: 706064. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:10:14,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:17,867][14527] Updated weights for policy 0, policy_version 690 (0.0048) [2023-07-24 01:10:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 2826240. Throughput: 0: 299.0. Samples: 707764. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:10:19,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 2834432. Throughput: 0: 307.8. Samples: 708828. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:10:24,630][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 2842624. Throughput: 0: 336.5. Samples: 711476. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:10:29,636][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2850816. Throughput: 0: 355.7. Samples: 713640. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:10:34,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 2854912. Throughput: 0: 355.9. Samples: 714508. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:10:39,635][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.4, 300 sec: 1291.3). Total num frames: 2859008. Throughput: 0: 338.2. Samples: 716224. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:10:44,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:48,007][14527] Updated weights for policy 0, policy_version 700 (0.0030) [2023-07-24 01:10:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2867200. Throughput: 0: 321.8. Samples: 717936. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:10:49,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2875392. Throughput: 0: 332.2. Samples: 719300. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:10:54,637][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:10:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2883584. Throughput: 0: 353.8. Samples: 721984. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:10:59,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 2891776. Throughput: 0: 357.6. Samples: 723856. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:11:04,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2895872. Throughput: 0: 353.2. Samples: 724724. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:11:09,636][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 2899968. Throughput: 0: 332.4. Samples: 726436. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:11:14,635][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:17,627][14527] Updated weights for policy 0, policy_version 710 (0.0057) [2023-07-24 01:11:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2908160. Throughput: 0: 330.8. Samples: 728524. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:11:19,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2916352. Throughput: 0: 340.6. Samples: 729836. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:11:24,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2924544. Throughput: 0: 356.6. Samples: 732272. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:11:29,636][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:34,631][00294] Fps is (10 sec: 1638.0, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2932736. Throughput: 0: 357.2. Samples: 734012. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:11:34,637][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:39,632][00294] Fps is (10 sec: 818.9, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 2932736. Throughput: 0: 343.4. Samples: 734756. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:11:39,635][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:44,628][00294] Fps is (10 sec: 819.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 2940928. Throughput: 0: 314.2. Samples: 736124. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:11:44,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:49,633][00294] Fps is (10 sec: 819.1, 60 sec: 1228.7, 300 sec: 1277.4). Total num frames: 2940928. Throughput: 0: 303.9. Samples: 737532. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:11:49,636][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:50,189][14531] Large shaping reward -2.536 for [('FRAGCOUNT', -1.5, -1.0), ('DEATHCOUNT', -0.75, 1.0), ('HEALTH', -0.28500000000000003, -95.0), ('AMMO5', -0.0005, -1.0)] [2023-07-24 01:11:51,452][14527] Updated weights for policy 0, policy_version 720 (0.0051) [2023-07-24 01:11:54,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2949120. Throughput: 0: 301.8. Samples: 738304. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:11:54,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:59,628][00294] Fps is (10 sec: 1639.2, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2957312. Throughput: 0: 305.5. Samples: 740184. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) [2023-07-24 01:11:59,635][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:11:59,649][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000722_2957312.pth... [2023-07-24 01:11:59,880][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000645_2641920.pth [2023-07-24 01:12:04,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2965504. Throughput: 0: 303.6. Samples: 742188. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:12:04,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1263.5). Total num frames: 2965504. Throughput: 0: 293.2. Samples: 743032. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:12:09,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:14,629][00294] Fps is (10 sec: 819.1, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 2973696. Throughput: 0: 277.2. Samples: 744748. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:12:14,635][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2981888. Throughput: 0: 282.1. Samples: 746708. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:12:19,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:22,599][14527] Updated weights for policy 0, policy_version 730 (0.0029) [2023-07-24 01:12:24,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 2990080. Throughput: 0: 295.7. Samples: 748060. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:12:24,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 2998272. Throughput: 0: 322.5. Samples: 750636. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) [2023-07-24 01:12:29,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.9, 300 sec: 1319.1). Total num frames: 3006464. Throughput: 0: 330.2. Samples: 752388. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2023-07-24 01:12:34,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:39,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.9, 300 sec: 1291.3). Total num frames: 3006464. Throughput: 0: 331.9. Samples: 753240. Policy #0 lag: (min: 0.0, avg: 1.4, max: 4.0) [2023-07-24 01:12:39,638][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:44,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3014656. Throughput: 0: 327.6. Samples: 754928. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:12:44,632][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:49,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1365.4, 300 sec: 1319.0). Total num frames: 3022848. Throughput: 0: 334.3. Samples: 757232. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:12:49,639][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:52,311][14527] Updated weights for policy 0, policy_version 740 (0.0022) [2023-07-24 01:12:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3031040. Throughput: 0: 345.5. Samples: 758580. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:12:54,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:12:59,628][00294] Fps is (10 sec: 1638.6, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3039232. Throughput: 0: 356.6. Samples: 760796. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:12:59,636][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3047424. Throughput: 0: 351.7. Samples: 762536. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:13:04,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 3047424. Throughput: 0: 340.7. Samples: 763392. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:13:09,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.4, 300 sec: 1319.1). Total num frames: 3055616. Throughput: 0: 321.0. Samples: 765080. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:13:14,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3063808. Throughput: 0: 340.6. Samples: 767716. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) [2023-07-24 01:13:19,635][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:21,356][14527] Updated weights for policy 0, policy_version 750 (0.0055) [2023-07-24 01:13:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3072000. Throughput: 0: 350.5. Samples: 769012. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) [2023-07-24 01:13:24,635][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3080192. Throughput: 0: 355.4. Samples: 770920. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:13:29,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3084288. Throughput: 0: 342.9. Samples: 772664. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) [2023-07-24 01:13:34,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3088384. Throughput: 0: 332.2. Samples: 773528. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) [2023-07-24 01:13:39,637][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3096576. Throughput: 0: 328.5. Samples: 775580. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) [2023-07-24 01:13:44,639][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:49,629][00294] Fps is (10 sec: 1638.2, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3104768. Throughput: 0: 344.3. Samples: 778028. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) [2023-07-24 01:13:49,640][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:52,897][14527] Updated weights for policy 0, policy_version 760 (0.0064) [2023-07-24 01:13:54,632][00294] Fps is (10 sec: 1637.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3112960. Throughput: 0: 342.8. Samples: 778820. Policy #0 lag: (min: 0.0, avg: 1.5, max: 3.0) [2023-07-24 01:13:54,641][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:59,628][00294] Fps is (10 sec: 819.3, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3112960. Throughput: 0: 336.2. Samples: 780208. Policy #0 lag: (min: 0.0, avg: 1.5, max: 3.0) [2023-07-24 01:13:59,631][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:13:59,651][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000760_3112960.pth... [2023-07-24 01:13:59,920][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000684_2801664.pth [2023-07-24 01:14:04,629][00294] Fps is (10 sec: 819.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3121152. Throughput: 0: 306.8. Samples: 781524. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:14:04,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:14:09,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3121152. Throughput: 0: 292.9. Samples: 782192. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:14:09,633][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:14:14,633][00294] Fps is (10 sec: 818.9, 60 sec: 1228.7, 300 sec: 1277.4). Total num frames: 3129344. Throughput: 0: 281.3. Samples: 783580. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:14:14,640][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:14:19,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3137536. Throughput: 0: 291.4. Samples: 785776. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:14:19,634][00294] Avg episode reward: [(0, '-5.307')] [2023-07-24 01:14:20,759][14524] DAMAGECOUNT value on done: 684.0 [2023-07-24 01:14:20,766][14524] Sum rewards: -5.625, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-0.540', 'AMMO2': '0.001', 'AMMO4': '0.005', 'WEAPON1': '0.010', 'AMMO5': '0.022', 'ARMOR': '0.032', 'HITCOUNT': '0.130', 'AMMO3': '0.136', 'weapon5': '0.150', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.600', 'WEAPON3': '0.750', 'weapon2': '1.238', 'weapon3': '1.590', 'FRAGCOUNT': '2.000'} [2023-07-24 01:14:20,969][14528] DAMAGECOUNT value on done: 527.0 [2023-07-24 01:14:20,974][14528] Sum rewards: -3.250, reward structure: {'DEATHCOUNT': '-7.500', 'FRAGCOUNT': '-0.500', 'HEALTH': '-0.104', 'AMMO5': '0.005', 'WEAPON1': '0.010', 'AMMO2': '0.011', 'AMMO4': '0.053', 'HITCOUNT': '0.090', 'weapon5': '0.094', 'WEAPON5': '0.100', 'AMMO3': '0.106', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.759', 'weapon2': '1.512', 'weapon3': '1.514'} [2023-07-24 01:14:21,189][14532] DAMAGECOUNT value on done: 974.0 [2023-07-24 01:14:21,196][14532] Sum rewards: 0.855, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-0.338', 'AMMO2': '0.007', 'AMMO5': '0.015', 'WEAPON1': '0.030', 'AMMO4': '0.035', 'AMMO3': '0.082', 'weapon5': '0.090', 'weapon7': '0.092', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'HITCOUNT': '0.160', 'WEAPON5': '0.300', 'WEAPON3': '0.450', 'weapon2': '1.146', 'weapon3': '1.310', 'DAMAGECOUNT': '1.425', 'FRAGCOUNT': '2.500'} [2023-07-24 01:14:24,628][00294] Fps is (10 sec: 1639.0, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3145728. Throughput: 0: 301.2. Samples: 787084. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:14:24,632][00294] Avg episode reward: [(0, '-5.198')] [2023-07-24 01:14:25,741][14524] DAMAGECOUNT value on done: 584.0 [2023-07-24 01:14:25,987][14528] DAMAGECOUNT value on done: 503.0 [2023-07-24 01:14:25,987][14528] Sum rewards: -5.665, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.200', 'AMMO2': '0.003', 'WEAPON1': '0.010', 'ARMOR': '0.012', 'AMMO5': '0.012', 'AMMO4': '0.015', 'weapon7': '0.066', 'HITCOUNT': '0.080', 'WEAPON4': '0.100', 'weapon5': '0.100', 'AMMO3': '0.159', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON7': '0.200', 'WEAPON5': '0.250', 'weapon4': '0.272', 'DAMAGECOUNT': '0.315', 'WEAPON3': '0.850', 'weapon2': '0.894', 'FRAGCOUNT': '1.000', 'weapon3': '1.376'} [2023-07-24 01:14:26,542][14532] DAMAGECOUNT value on done: 826.0 [2023-07-24 01:14:26,543][14532] Sum rewards: -4.038, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.346', 'AMMO2': '0.009', 'ARMOR': '0.012', 'AMMO5': '0.022', 'WEAPON1': '0.030', 'AMMO4': '0.047', 'weapon5': '0.096', 'WEAPON4': '0.100', 'HITCOUNT': '0.150', 'AMMO3': '0.159', 'weapon4': '0.170', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.624', 'WEAPON3': '0.950', 'weapon2': '0.960', 'FRAGCOUNT': '1.000', 'weapon3': '1.628'} [2023-07-24 01:14:26,781][14531] DAMAGECOUNT value on done: 708.0 [2023-07-24 01:14:26,785][14531] Sum rewards: -6.940, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.958', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.001', 'AMMO4': '0.003', 'AMMO5': '0.021', 'WEAPON1': '0.030', 'ARMOR': '0.040', 'WEAPON4': '0.050', 'HITCOUNT': '0.130', 'AMMO3': '0.138', 'weapon5': '0.140', 'weapon4': '0.148', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.510', 'WEAPON3': '0.850', 'weapon2': '1.296', 'weapon3': '1.560'} [2023-07-24 01:14:27,739][14527] Updated weights for policy 0, policy_version 770 (0.0037) [2023-07-24 01:14:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3153920. Throughput: 0: 307.8. Samples: 789432. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:14:29,630][00294] Avg episode reward: [(0, '-5.193')] [2023-07-24 01:14:31,862][14524] DAMAGECOUNT value on done: 686.0 [2023-07-24 01:14:31,866][14524] Sum rewards: -5.218, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.570', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.009', 'AMMO5': '0.016', 'WEAPON1': '0.030', 'weapon4': '0.040', 'AMMO4': '0.045', 'WEAPON4': '0.100', 'AMMO3': '0.133', 'weapon5': '0.206', 'HITCOUNT': '0.220', 'WEAPON5': '0.400', 'WEAPON3': '0.750', 'weapon2': '0.980', 'DAMAGECOUNT': '1.041', 'weapon3': '1.632'} [2023-07-24 01:14:32,072][14528] DAMAGECOUNT value on done: 441.0 [2023-07-24 01:14:32,421][14532] DAMAGECOUNT value on done: 634.0 [2023-07-24 01:14:32,424][14532] Sum rewards: -5.464, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-2.026', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.005', 'WEAPON1': '0.010', 'AMMO5': '0.010', 'AMMO4': '0.023', 'ARMOR': '0.032', 'weapon5': '0.138', 'AMMO3': '0.146', 'WEAPON5': '0.150', 'HITCOUNT': '0.170', 'DAMAGECOUNT': '0.510', 'WEAPON3': '0.800', 'weapon2': '0.822', 'weapon3': '1.746'} [2023-07-24 01:14:32,629][14531] DAMAGECOUNT value on done: 741.0 [2023-07-24 01:14:32,629][14531] Sum rewards: -4.508, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.210', 'FRAGCOUNT': '0.000', 'AMMO5': '0.019', 'AMMO2': '0.035', 'weapon7': '0.036', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.111', 'HITCOUNT': '0.140', 'AMMO4': '0.174', 'weapon5': '0.180', 'WEAPON4': '0.250', 'weapon4': '0.324', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.639', 'WEAPON3': '0.650', 'weapon3': '1.064', 'weapon2': '1.430'} [2023-07-24 01:14:34,635][00294] Fps is (10 sec: 1228.0, 60 sec: 1228.7, 300 sec: 1291.3). Total num frames: 3158016. Throughput: 0: 291.7. Samples: 791156. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:14:34,638][00294] Avg episode reward: [(0, '-5.115')] [2023-07-24 01:14:38,434][14524] DAMAGECOUNT value on done: 319.0 [2023-07-24 01:14:38,437][14524] Sum rewards: -2.086, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.912', 'AMMO5': '0.007', 'AMMO2': '0.036', 'ARMOR': '0.036', 'WEAPON1': '0.040', 'HITCOUNT': '0.050', 'weapon5': '0.080', 'AMMO3': '0.098', 'DAMAGECOUNT': '0.135', 'AMMO4': '0.179', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'weapon4': '0.220', 'WEAPON3': '0.650', 'weapon2': '0.920', 'weapon3': '1.474', 'FRAGCOUNT': '2.000'} [2023-07-24 01:14:39,175][14528] DAMAGECOUNT value on done: 361.0 [2023-07-24 01:14:39,187][14528] Sum rewards: -7.556, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.916', 'AMMO5': '0.005', 'weapon5': '0.030', 'AMMO2': '0.031', 'ARMOR': '0.044', 'WEAPON5': '0.050', 'HITCOUNT': '0.080', 'AMMO4': '0.153', 'weapon4': '0.190', 'AMMO3': '0.195', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.270', 'weapon3': '0.948', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.500', 'weapon2': '1.614'} [2023-07-24 01:14:39,361][14532] DAMAGECOUNT value on done: 441.0 [2023-07-24 01:14:39,363][14532] Sum rewards: -4.529, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.285', 'AMMO5': '0.005', 'AMMO2': '0.008', 'AMMO4': '0.038', 'ARMOR': '0.040', 'WEAPON5': '0.050', 'WEAPON4': '0.050', 'HITCOUNT': '0.130', 'AMMO3': '0.144', 'weapon4': '0.188', 'DAMAGECOUNT': '0.495', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon2': '1.138', 'weapon3': '1.470'} [2023-07-24 01:14:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3162112. Throughput: 0: 293.3. Samples: 792016. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:14:39,635][00294] Avg episode reward: [(0, '-5.089')] [2023-07-24 01:14:39,896][14531] DAMAGECOUNT value on done: 857.0 [2023-07-24 01:14:39,898][14531] Sum rewards: -3.202, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.190', 'weapon5': '0.002', 'AMMO2': '0.003', 'AMMO5': '0.007', 'AMMO4': '0.015', 'ARMOR': '0.020', 'weapon7': '0.050', 'WEAPON4': '0.100', 'weapon4': '0.142', 'WEAPON5': '0.150', 'AMMO6': '0.160', 'AMMO7': '0.160', 'AMMO3': '0.199', 'WEAPON7': '0.200', 'HITCOUNT': '0.300', 'WEAPON3': '0.950', 'weapon2': '1.008', 'DAMAGECOUNT': '1.221', 'weapon3': '1.800', 'FRAGCOUNT': '2.000'} [2023-07-24 01:14:40,250][14529] DAMAGECOUNT value on done: 788.0 [2023-07-24 01:14:40,251][14529] Sum rewards: -3.646, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.755', 'AMMO2': '0.004', 'AMMO5': '0.020', 'AMMO4': '0.021', 'ARMOR': '0.037', 'WEAPON4': '0.050', 'weapon5': '0.052', 'HITCOUNT': '0.140', 'weapon4': '0.144', 'AMMO3': '0.157', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.525', 'WEAPON3': '0.750', 'weapon2': '1.252', 'weapon3': '1.456', 'FRAGCOUNT': '2.000'} [2023-07-24 01:14:44,628][00294] Fps is (10 sec: 1229.6, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3170304. Throughput: 0: 300.3. Samples: 793720. Policy #0 lag: (min: 0.0, avg: 1.5, max: 3.0) [2023-07-24 01:14:44,631][00294] Avg episode reward: [(0, '-5.001')] [2023-07-24 01:14:45,205][14524] DAMAGECOUNT value on done: 804.0 [2023-07-24 01:14:45,207][14524] Sum rewards: -4.639, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.108', 'AMMO5': '0.003', 'AMMO2': '0.014', 'weapon5': '0.016', 'WEAPON5': '0.050', 'AMMO4': '0.068', 'AMMO3': '0.120', 'HITCOUNT': '0.130', 'WEAPON4': '0.200', 'weapon4': '0.372', 'DAMAGECOUNT': '0.420', 'ARMOR': '0.501', 'WEAPON3': '0.700', 'FRAGCOUNT': '1.000', 'weapon3': '1.396', 'weapon2': '1.480'} [2023-07-24 01:14:45,612][14528] DAMAGECOUNT value on done: 422.0 [2023-07-24 01:14:45,612][14528] Sum rewards: -5.535, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.038', 'AMMO2': '0.024', 'weapon4': '0.076', 'HITCOUNT': '0.080', 'ARMOR': '0.084', 'AMMO3': '0.090', 'AMMO4': '0.120', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.240', 'WEAPON3': '0.550', 'weapon3': '0.944', 'FRAGCOUNT': '1.000', 'weapon2': '1.894'} [2023-07-24 01:14:45,704][14532] DAMAGECOUNT value on done: 669.0 [2023-07-24 01:14:45,704][14532] Sum rewards: -0.987, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.676', 'AMMO5': '0.010', 'WEAPON1': '0.010', 'AMMO2': '0.011', 'weapon5': '0.030', 'AMMO4': '0.053', 'WEAPON4': '0.100', 'WEAPON5': '0.150', 'AMMO3': '0.156', 'weapon4': '0.212', 'HITCOUNT': '0.240', 'ARMOR': '0.428', 'weapon2': '0.638', 'WEAPON3': '0.850', 'DAMAGECOUNT': '0.975', 'weapon3': '1.826', 'FRAGCOUNT': '3.000'} [2023-07-24 01:14:46,068][14531] DAMAGECOUNT value on done: 534.0 [2023-07-24 01:14:46,074][14531] Sum rewards: -5.741, reward structure: {'DEATHCOUNT': '-6.750', 'FRAGCOUNT': '-2.000', 'HEALTH': '-1.294', 'AMMO4': '-0.007', 'AMMO2': '-0.001', 'AMMO5': '0.017', 'ARMOR': '0.048', 'HITCOUNT': '0.100', 'weapon5': '0.116', 'AMMO3': '0.117', 'DAMAGECOUNT': '0.345', 'WEAPON5': '0.350', 'WEAPON3': '0.700', 'weapon2': '0.984', 'weapon3': '1.534'} [2023-07-24 01:14:46,091][14529] DAMAGECOUNT value on done: 366.0 [2023-07-24 01:14:46,093][14529] Sum rewards: -9.089, reward structure: {'DEATHCOUNT': '-9.750', 'FRAGCOUNT': '-3.000', 'HEALTH': '-0.908', 'AMMO5': '0.005', 'weapon5': '0.014', 'AMMO2': '0.021', 'ARMOR': '0.064', 'HITCOUNT': '0.080', 'AMMO3': '0.089', 'WEAPON5': '0.100', 'AMMO4': '0.102', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.210', 'weapon4': '0.246', 'WEAPON3': '0.550', 'weapon3': '1.298', 'weapon2': '1.640'} [2023-07-24 01:14:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3178496. Throughput: 0: 326.0. Samples: 796196. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:14:49,638][00294] Avg episode reward: [(0, '-4.932')] [2023-07-24 01:14:50,042][14524] DAMAGECOUNT value on done: 973.0 [2023-07-24 01:14:50,046][14524] Sum rewards: -3.206, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.848', 'AMMO2': '0.012', 'AMMO5': '0.019', 'ARMOR': '0.032', 'AMMO4': '0.059', 'weapon5': '0.088', 'AMMO3': '0.103', 'HITCOUNT': '0.150', 'WEAPON4': '0.150', 'weapon4': '0.286', 'WEAPON5': '0.300', 'WEAPON3': '0.600', 'weapon2': '0.748', 'DAMAGECOUNT': '0.759', 'FRAGCOUNT': '1.000', 'weapon3': '1.586'} [2023-07-24 01:14:50,287][14530] DAMAGECOUNT value on done: 563.0 [2023-07-24 01:14:50,292][14530] Sum rewards: -6.287, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.312', 'AMMO4': '-0.023', 'AMMO2': '-0.004', 'AMMO5': '0.017', 'WEAPON1': '0.020', 'weapon5': '0.030', 'ARMOR': '0.105', 'AMMO3': '0.160', 'HITCOUNT': '0.230', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.807', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.000', 'weapon2': '1.030', 'weapon3': '1.802'} [2023-07-24 01:14:50,404][14528] DAMAGECOUNT value on done: 518.0 [2023-07-24 01:14:50,417][14528] Sum rewards: -2.596, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.084', 'AMMO2': '0.001', 'AMMO4': '0.003', 'WEAPON1': '0.010', 'AMMO5': '0.011', 'ARMOR': '0.032', 'HITCOUNT': '0.090', 'AMMO3': '0.129', 'weapon5': '0.200', 'DAMAGECOUNT': '0.240', 'WEAPON5': '0.250', 'WEAPON3': '0.700', 'weapon2': '0.746', 'FRAGCOUNT': '1.000', 'weapon3': '1.826'} [2023-07-24 01:14:50,548][14532] DAMAGECOUNT value on done: 503.0 [2023-07-24 01:14:50,552][14532] Sum rewards: -7.398, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.580', 'FRAGCOUNT': '-1.500', 'AMMO2': '0.011', 'weapon7': '0.018', 'WEAPON1': '0.020', 'AMMO5': '0.022', 'AMMO4': '0.057', 'AMMO3': '0.116', 'HITCOUNT': '0.140', 'weapon5': '0.142', 'WEAPON4': '0.150', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'weapon4': '0.242', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.375', 'ARMOR': '0.478', 'WEAPON3': '0.700', 'weapon2': '0.938', 'weapon3': '1.172'} [2023-07-24 01:14:50,858][14529] DAMAGECOUNT value on done: 346.0 [2023-07-24 01:14:50,878][14529] Sum rewards: -8.965, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.047', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.004', 'ARMOR': '0.004', 'AMMO2': '0.007', 'AMMO4': '0.033', 'WEAPON4': '0.050', 'weapon5': '0.064', 'WEAPON5': '0.100', 'weapon4': '0.104', 'HITCOUNT': '0.130', 'AMMO3': '0.169', 'DAMAGECOUNT': '0.345', 'WEAPON3': '0.850', 'weapon3': '1.448', 'weapon2': '1.524'} [2023-07-24 01:14:50,891][14531] DAMAGECOUNT value on done: 489.0 [2023-07-24 01:14:50,892][14531] Sum rewards: -7.118, reward structure: {'DEATHCOUNT': '-8.250', 'FRAGCOUNT': '-3.000', 'HEALTH': '-0.105', 'WEAPON1': '0.010', 'AMMO2': '0.012', 'AMMO5': '0.015', 'ARMOR': '0.036', 'WEAPON4': '0.050', 'AMMO4': '0.057', 'weapon5': '0.074', 'AMMO3': '0.089', 'HITCOUNT': '0.090', 'weapon4': '0.188', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.246', 'WEAPON3': '0.500', 'weapon3': '1.234', 'weapon2': '1.436'} [2023-07-24 01:14:54,522][14525] DAMAGECOUNT value on done: 774.0 [2023-07-24 01:14:54,528][14525] Sum rewards: -7.246, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.340', 'FRAGCOUNT': '-1.500', 'AMMO5': '0.010', 'AMMO2': '0.016', 'AMMO4': '0.078', 'WEAPON4': '0.100', 'weapon5': '0.104', 'HITCOUNT': '0.110', 'AMMO3': '0.161', 'weapon4': '0.162', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.489', 'ARMOR': '0.493', 'WEAPON3': '0.900', 'weapon3': '1.356', 'weapon2': '1.414'} [2023-07-24 01:14:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 3186688. Throughput: 0: 339.7. Samples: 797480. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:14:54,631][00294] Avg episode reward: [(0, '-4.993')] [2023-07-24 01:14:55,554][14524] DAMAGECOUNT value on done: 553.0 [2023-07-24 01:14:55,557][14524] Sum rewards: -0.965, reward structure: {'DEATHCOUNT': '-6.750', 'AMMO2': '0.005', 'AMMO5': '0.005', 'WEAPON1': '0.010', 'AMMO4': '0.025', 'HEALTH': '0.030', 'HITCOUNT': '0.060', 'AMMO3': '0.104', 'WEAPON5': '0.150', 'weapon5': '0.178', 'ARMOR': '0.476', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.615', 'FRAGCOUNT': '1.000', 'weapon2': '1.118', 'weapon3': '1.408'} [2023-07-24 01:14:55,706][14530] DAMAGECOUNT value on done: 1052.0 [2023-07-24 01:14:55,716][14530] Sum rewards: -0.931, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.179', 'AMMO2': '0.005', 'AMMO5': '0.007', 'ARMOR': '0.008', 'weapon5': '0.020', 'AMMO4': '0.026', 'HITCOUNT': '0.080', 'weapon7': '0.080', 'WEAPON4': '0.100', 'weapon4': '0.114', 'AMMO3': '0.116', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON5': '0.150', 'WEAPON7': '0.200', 'DAMAGECOUNT': '0.699', 'WEAPON3': '0.700', 'weapon3': '1.160', 'weapon2': '1.292', 'FRAGCOUNT': '2.000'} [2023-07-24 01:14:56,103][14526] DAMAGECOUNT value on done: 380.0 [2023-07-24 01:14:56,104][14526] Sum rewards: -2.488, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.958', 'weapon5': '0.002', 'AMMO5': '0.005', 'AMMO2': '0.018', 'WEAPON1': '0.020', 'ARMOR': '0.028', 'weapon7': '0.054', 'AMMO4': '0.087', 'HITCOUNT': '0.090', 'WEAPON5': '0.100', 'AMMO3': '0.126', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON7': '0.200', 'DAMAGECOUNT': '0.240', 'WEAPON4': '0.250', 'weapon4': '0.402', 'weapon2': '0.630', 'WEAPON3': '0.750', 'weapon3': '1.398', 'FRAGCOUNT': '2.000'} [2023-07-24 01:14:56,105][14528] DAMAGECOUNT value on done: 646.0 [2023-07-24 01:14:56,106][14528] Sum rewards: -6.998, reward structure: {'DEATHCOUNT': '-13.500', 'HEALTH': '-1.131', 'AMMO2': '0.019', 'AMMO5': '0.020', 'WEAPON1': '0.020', 'weapon5': '0.028', 'ARMOR': '0.032', 'weapon4': '0.038', 'WEAPON4': '0.050', 'AMMO4': '0.094', 'HITCOUNT': '0.150', 'AMMO3': '0.164', 'WEAPON5': '0.400', 'DAMAGECOUNT': '0.684', 'WEAPON3': '0.950', 'weapon2': '1.356', 'weapon3': '1.628', 'FRAGCOUNT': '2.000'} [2023-07-24 01:14:56,308][14532] DAMAGECOUNT value on done: 476.0 [2023-07-24 01:14:56,316][14532] Sum rewards: -3.830, reward structure: {'DEATHCOUNT': '-6.750', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.198', 'AMMO5': '0.009', 'WEAPON1': '0.010', 'AMMO2': '0.027', 'weapon4': '0.076', 'AMMO3': '0.081', 'HITCOUNT': '0.100', 'WEAPON4': '0.100', 'AMMO4': '0.134', 'weapon5': '0.140', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.345', 'WEAPON3': '0.450', 'weapon2': '1.238', 'weapon3': '1.708'} [2023-07-24 01:14:56,595][14529] DAMAGECOUNT value on done: 703.0 [2023-07-24 01:14:56,596][14529] Sum rewards: -4.923, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.170', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.012', 'WEAPON1': '0.020', 'AMMO2': '0.031', 'ARMOR': '0.052', 'AMMO3': '0.104', 'HITCOUNT': '0.110', 'weapon5': '0.128', 'AMMO4': '0.154', 'weapon4': '0.292', 'WEAPON4': '0.300', 'WEAPON5': '0.300', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.696', 'weapon2': '0.792', 'weapon3': '1.356'} [2023-07-24 01:14:56,875][14531] DAMAGECOUNT value on done: 538.0 [2023-07-24 01:14:56,876][14531] Sum rewards: -4.650, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '0.006', 'weapon7': '0.010', 'AMMO2': '0.023', 'ARMOR': '0.076', 'AMMO3': '0.078', 'AMMO4': '0.112', 'HITCOUNT': '0.120', 'WEAPON4': '0.150', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'DAMAGECOUNT': '0.369', 'weapon4': '0.370', 'WEAPON3': '0.450', 'weapon3': '0.528', 'FRAGCOUNT': '1.000', 'weapon2': '1.958'} [2023-07-24 01:14:58,718][14527] Updated weights for policy 0, policy_version 780 (0.0045) [2023-07-24 01:14:59,634][00294] Fps is (10 sec: 1637.5, 60 sec: 1365.2, 300 sec: 1305.2). Total num frames: 3194880. Throughput: 0: 352.2. Samples: 799428. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:14:59,641][00294] Avg episode reward: [(0, '-4.828')] [2023-07-24 01:15:01,441][14525] DAMAGECOUNT value on done: 395.0 [2023-07-24 01:15:02,284][14530] DAMAGECOUNT value on done: 443.0 [2023-07-24 01:15:02,285][14530] Sum rewards: -3.068, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.414', 'AMMO2': '0.013', 'AMMO5': '0.017', 'ARMOR': '0.050', 'HITCOUNT': '0.060', 'AMMO4': '0.064', 'AMMO3': '0.117', 'weapon5': '0.222', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.387', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon2': '1.206', 'weapon3': '1.460'} [2023-07-24 01:15:02,629][14526] DAMAGECOUNT value on done: 712.0 [2023-07-24 01:15:02,644][14526] Sum rewards: -4.222, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.800', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.006', 'AMMO5': '0.009', 'WEAPON1': '0.020', 'AMMO4': '0.028', 'weapon5': '0.042', 'ARMOR': '0.056', 'AMMO3': '0.116', 'HITCOUNT': '0.120', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.279', 'WEAPON3': '0.700', 'weapon2': '1.490', 'weapon3': '1.512'} [2023-07-24 01:15:03,048][14529] DAMAGECOUNT value on done: 698.0 [2023-07-24 01:15:03,048][14529] Sum rewards: -2.891, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.924', 'ARMOR': '0.004', 'WEAPON1': '0.020', 'AMMO5': '0.020', 'AMMO2': '0.021', 'weapon5': '0.052', 'WEAPON4': '0.100', 'AMMO4': '0.103', 'AMMO3': '0.121', 'weapon4': '0.184', 'HITCOUNT': '0.230', 'WEAPON5': '0.250', 'WEAPON3': '0.750', 'DAMAGECOUNT': '0.798', 'weapon2': '0.932', 'FRAGCOUNT': '1.000', 'weapon3': '1.698'} [2023-07-24 01:15:04,385][14524] DAMAGECOUNT value on done: 240.0 [2023-07-24 01:15:04,388][14524] Sum rewards: -6.941, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.882', 'FRAGCOUNT': '-1.500', 'weapon5': '0.006', 'AMMO5': '0.015', 'AMMO2': '0.017', 'WEAPON1': '0.020', 'HITCOUNT': '0.030', 'ARMOR': '0.044', 'AMMO4': '0.085', 'DAMAGECOUNT': '0.090', 'AMMO3': '0.148', 'WEAPON5': '0.200', 'WEAPON4': '0.200', 'weapon4': '0.276', 'WEAPON3': '0.900', 'weapon2': '0.904', 'weapon3': '1.756'} [2023-07-24 01:15:04,637][00294] Fps is (10 sec: 1227.7, 60 sec: 1296.9, 300 sec: 1305.1). Total num frames: 3198976. Throughput: 0: 339.9. Samples: 801076. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:15:04,641][00294] Avg episode reward: [(0, '-4.847')] [2023-07-24 01:15:05,067][14528] DAMAGECOUNT value on done: 843.0 [2023-07-24 01:15:05,071][14528] Sum rewards: -5.208, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-1.500', 'AMMO5': '0.003', 'AMMO2': '0.006', 'AMMO4': '0.030', 'WEAPON4': '0.050', 'ARMOR': '0.056', 'weapon4': '0.086', 'WEAPON5': '0.100', 'weapon5': '0.116', 'AMMO3': '0.121', 'HITCOUNT': '0.220', 'HEALTH': '0.300', 'WEAPON3': '0.700', 'weapon2': '0.780', 'DAMAGECOUNT': '0.858', 'weapon3': '1.866'} [2023-07-24 01:15:05,213][14532] DAMAGECOUNT value on done: 922.0 [2023-07-24 01:15:05,214][14532] Sum rewards: -3.608, reward structure: {'DEATHCOUNT': '-9.750', 'AMMO2': '0.013', 'AMMO5': '0.015', 'ARMOR': '0.036', 'weapon5': '0.048', 'WEAPON4': '0.050', 'weapon4': '0.052', 'AMMO4': '0.065', 'weapon7': '0.068', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'AMMO3': '0.109', 'HITCOUNT': '0.150', 'WEAPON5': '0.200', 'HEALTH': '0.268', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.657', 'weapon3': '1.428', 'weapon2': '1.582'} [2023-07-24 01:15:05,693][14531] DAMAGECOUNT value on done: 644.0 [2023-07-24 01:15:05,719][14531] Sum rewards: -2.134, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.405', 'AMMO5': '0.005', 'AMMO2': '0.014', 'WEAPON1': '0.020', 'weapon4': '0.062', 'AMMO4': '0.067', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.105', 'HITCOUNT': '0.180', 'DAMAGECOUNT': '0.492', 'WEAPON3': '0.550', 'FRAGCOUNT': '1.000', 'weapon2': '1.218', 'weapon3': '1.858'} [2023-07-24 01:15:07,706][14525] DAMAGECOUNT value on done: 487.0 [2023-07-24 01:15:07,711][14525] Sum rewards: -4.789, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.974', 'AMMO5': '0.003', 'AMMO2': '0.023', 'ARMOR': '0.032', 'weapon5': '0.044', 'WEAPON5': '0.050', 'AMMO4': '0.114', 'AMMO3': '0.168', 'HITCOUNT': '0.200', 'weapon4': '0.204', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.633', 'WEAPON3': '0.950', 'weapon3': '1.306', 'weapon2': '1.458', 'FRAGCOUNT': '3.000'} [2023-07-24 01:15:08,597][14530] DAMAGECOUNT value on done: 584.0 [2023-07-24 01:15:08,602][14530] Sum rewards: -3.751, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.804', 'AMMO5': '0.005', 'AMMO2': '0.020', 'AMMO3': '0.082', 'AMMO4': '0.099', 'WEAPON5': '0.100', 'HITCOUNT': '0.160', 'WEAPON4': '0.200', 'weapon4': '0.248', 'DAMAGECOUNT': '0.417', 'ARMOR': '0.440', 'WEAPON3': '0.500', 'FRAGCOUNT': '1.000', 'weapon2': '1.274', 'weapon3': '1.508'} [2023-07-24 01:15:08,966][14526] DAMAGECOUNT value on done: 1043.0 [2023-07-24 01:15:08,978][14526] Sum rewards: -4.752, reward structure: {'DEATHCOUNT': '-8.250', 'FRAGCOUNT': '-1.000', 'HEALTH': '-0.180', 'AMMO5': '0.015', 'AMMO2': '0.021', 'ARMOR': '0.028', 'AMMO3': '0.040', 'HITCOUNT': '0.070', 'weapon5': '0.076', 'weapon7': '0.088', 'WEAPON4': '0.100', 'AMMO4': '0.106', 'WEAPON5': '0.200', 'WEAPON3': '0.200', 'WEAPON7': '0.200', 'AMMO6': '0.200', 'AMMO7': '0.200', 'weapon3': '0.250', 'weapon4': '0.366', 'DAMAGECOUNT': '0.669', 'weapon2': '1.848'} [2023-07-24 01:15:09,429][14529] DAMAGECOUNT value on done: 879.0 [2023-07-24 01:15:09,433][14529] Sum rewards: -2.803, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.195', 'AMMO5': '0.007', 'AMMO2': '0.009', 'WEAPON1': '0.030', 'AMMO4': '0.045', 'weapon4': '0.070', 'ARMOR': '0.076', 'weapon5': '0.086', 'WEAPON4': '0.100', 'AMMO3': '0.148', 'WEAPON5': '0.150', 'HITCOUNT': '0.240', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.850', 'DAMAGECOUNT': '0.990', 'weapon2': '1.130', 'weapon3': '1.460'} [2023-07-24 01:15:09,628][00294] Fps is (10 sec: 819.6, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3203072. Throughput: 0: 329.5. Samples: 801912. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:15:09,631][00294] Avg episode reward: [(0, '-4.909')] [2023-07-24 01:15:12,902][14531] DAMAGECOUNT value on done: 519.0 [2023-07-24 01:15:12,904][14531] Sum rewards: -3.389, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.831', 'AMMO5': '0.003', 'AMMO2': '0.004', 'weapon5': '0.018', 'AMMO4': '0.019', 'WEAPON5': '0.050', 'HITCOUNT': '0.080', 'AMMO3': '0.118', 'DAMAGECOUNT': '0.285', 'ARMOR': '0.440', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon3': '1.296', 'weapon2': '1.730'} [2023-07-24 01:15:13,689][14525] DAMAGECOUNT value on done: 552.0 [2023-07-24 01:15:14,325][14530] DAMAGECOUNT value on done: 522.0 [2023-07-24 01:15:14,329][14530] Sum rewards: -0.873, reward structure: {'DEATHCOUNT': '-7.500', 'AMMO5': '0.010', 'AMMO2': '0.014', 'weapon5': '0.028', 'AMMO4': '0.067', 'HEALTH': '0.084', 'weapon4': '0.096', 'WEAPON4': '0.100', 'HITCOUNT': '0.120', 'AMMO3': '0.133', 'WEAPON5': '0.150', 'DAMAGECOUNT': '0.357', 'ARMOR': '0.484', 'WEAPON3': '0.650', 'weapon2': '0.740', 'weapon3': '1.594', 'FRAGCOUNT': '2.000'} [2023-07-24 01:15:14,539][14526] DAMAGECOUNT value on done: 655.0 [2023-07-24 01:15:14,542][14526] Sum rewards: -6.736, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.900', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.007', 'weapon5': '0.010', 'AMMO2': '0.024', 'ARMOR': '0.104', 'AMMO4': '0.119', 'WEAPON5': '0.150', 'AMMO3': '0.159', 'HITCOUNT': '0.190', 'weapon4': '0.344', 'WEAPON4': '0.350', 'DAMAGECOUNT': '0.570', 'WEAPON3': '0.900', 'weapon2': '1.202', 'weapon3': '1.284'} [2023-07-24 01:15:14,628][00294] Fps is (10 sec: 1229.9, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 3211264. Throughput: 0: 320.4. Samples: 803848. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:15:14,630][00294] Avg episode reward: [(0, '-4.800')] [2023-07-24 01:15:14,816][14529] DAMAGECOUNT value on done: 853.0 [2023-07-24 01:15:14,818][14529] Sum rewards: -6.277, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.018', 'ARMOR': '0.008', 'AMMO2': '0.009', 'AMMO4': '0.043', 'WEAPON4': '0.050', 'weapon4': '0.098', 'AMMO3': '0.160', 'HITCOUNT': '0.290', 'WEAPON3': '1.000', 'DAMAGECOUNT': '1.065', 'weapon2': '1.406', 'weapon3': '1.612', 'FRAGCOUNT': '2.000'} [2023-07-24 01:15:18,126][14525] DAMAGECOUNT value on done: 500.0 [2023-07-24 01:15:19,025][14530] DAMAGECOUNT value on done: 1137.0 [2023-07-24 01:15:19,028][14530] Sum rewards: -0.447, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.664', 'AMMO2': '0.004', 'AMMO5': '0.010', 'ARMOR': '0.012', 'AMMO4': '0.018', 'WEAPON1': '0.030', 'weapon5': '0.042', 'WEAPON4': '0.050', 'AMMO3': '0.131', 'WEAPON5': '0.200', 'weapon4': '0.254', 'HITCOUNT': '0.350', 'WEAPON3': '0.650', 'DAMAGECOUNT': '1.188', 'weapon2': '1.194', 'weapon3': '1.584', 'FRAGCOUNT': '5.000'} [2023-07-24 01:15:19,132][14526] DAMAGECOUNT value on done: 517.0 [2023-07-24 01:15:19,136][14526] Sum rewards: -1.138, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.340', 'AMMO5': '0.005', 'AMMO2': '0.018', 'ARMOR': '0.026', 'WEAPON1': '0.030', 'AMMO4': '0.091', 'AMMO3': '0.100', 'weapon5': '0.124', 'HITCOUNT': '0.140', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'weapon4': '0.164', 'DAMAGECOUNT': '0.402', 'WEAPON3': '0.600', 'weapon2': '1.104', 'weapon3': '1.598', 'FRAGCOUNT': '2.000'} [2023-07-24 01:15:19,436][14529] DAMAGECOUNT value on done: 1188.0 [2023-07-24 01:15:19,441][14529] Sum rewards: -3.513, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.030', 'AMMO5': '0.005', 'weapon5': '0.008', 'AMMO2': '0.022', 'ARMOR': '0.024', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON5': '0.100', 'WEAPON7': '0.100', 'AMMO4': '0.109', 'HITCOUNT': '0.110', 'AMMO3': '0.111', 'weapon7': '0.140', 'weapon4': '0.210', 'WEAPON4': '0.250', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.960', 'weapon2': '1.154', 'weapon3': '1.414', 'FRAGCOUNT': '2.000'} [2023-07-24 01:15:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3219456. Throughput: 0: 339.6. Samples: 806436. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:15:19,631][00294] Avg episode reward: [(0, '-4.695')] [2023-07-24 01:15:24,553][14525] DAMAGECOUNT value on done: 519.0 [2023-07-24 01:15:24,562][14525] Sum rewards: -2.903, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.047', 'AMMO5': '0.015', 'AMMO2': '0.025', 'ARMOR': '0.068', 'weapon5': '0.122', 'AMMO4': '0.126', 'HITCOUNT': '0.150', 'weapon4': '0.164', 'AMMO3': '0.166', 'WEAPON4': '0.200', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.471', 'WEAPON3': '0.850', 'weapon2': '1.112', 'weapon3': '1.374', 'FRAGCOUNT': '2.000'} [2023-07-24 01:15:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3227648. Throughput: 0: 346.7. Samples: 807616. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:15:24,637][00294] Avg episode reward: [(0, '-4.693')] [2023-07-24 01:15:26,044][14530] DAMAGECOUNT value on done: 654.0 [2023-07-24 01:15:26,045][14530] Sum rewards: -0.190, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.480', 'AMMO2': '0.003', 'AMMO5': '0.004', 'AMMO4': '0.016', 'WEAPON1': '0.020', 'ARMOR': '0.032', 'weapon5': '0.060', 'WEAPON5': '0.100', 'WEAPON4': '0.100', 'weapon4': '0.118', 'AMMO3': '0.139', 'HITCOUNT': '0.230', 'WEAPON3': '0.750', 'DAMAGECOUNT': '0.972', 'weapon3': '1.196', 'weapon2': '1.300', 'FRAGCOUNT': '3.000'} [2023-07-24 01:15:26,205][14526] DAMAGECOUNT value on done: 732.0 [2023-07-24 01:15:26,208][14526] Sum rewards: -1.469, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.936', 'AMMO5': '0.005', 'AMMO2': '0.013', 'WEAPON1': '0.020', 'AMMO4': '0.065', 'ARMOR': '0.082', 'weapon5': '0.086', 'AMMO3': '0.088', 'HITCOUNT': '0.110', 'weapon4': '0.130', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.723', 'weapon3': '0.922', 'weapon2': '1.822'} [2023-07-24 01:15:29,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 3231744. Throughput: 0: 346.4. Samples: 809308. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:15:29,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:15:30,859][14525] DAMAGECOUNT value on done: 585.0 [2023-07-24 01:15:31,050][14527] Updated weights for policy 0, policy_version 790 (0.0043) [2023-07-24 01:15:32,148][14530] DAMAGECOUNT value on done: 505.0 [2023-07-24 01:15:32,150][14530] Sum rewards: -5.084, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.770', 'AMMO5': '0.003', 'weapon5': '0.014', 'AMMO2': '0.022', 'WEAPON5': '0.050', 'weapon7': '0.066', 'AMMO4': '0.109', 'HITCOUNT': '0.150', 'AMMO3': '0.181', 'WEAPON4': '0.200', 'weapon4': '0.256', 'AMMO6': '0.260', 'AMMO7': '0.260', 'WEAPON7': '0.300', 'DAMAGECOUNT': '0.435', 'ARMOR': '0.490', 'WEAPON3': '0.800', 'weapon3': '0.994', 'FRAGCOUNT': '1.000', 'weapon2': '1.596'} [2023-07-24 01:15:32,287][14526] DAMAGECOUNT value on done: 1055.0 [2023-07-24 01:15:32,289][14526] Sum rewards: -2.824, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.198', 'AMMO2': '0.008', 'AMMO5': '0.012', 'weapon5': '0.020', 'AMMO4': '0.038', 'WEAPON4': '0.100', 'WEAPON5': '0.200', 'AMMO3': '0.244', 'HITCOUNT': '0.370', 'ARMOR': '0.481', 'WEAPON3': '1.200', 'weapon2': '1.438', 'DAMAGECOUNT': '1.620', 'weapon3': '1.642', 'FRAGCOUNT': '4.000'} [2023-07-24 01:15:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.5, 300 sec: 1305.2). Total num frames: 3239936. Throughput: 0: 328.5. Samples: 810980. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:15:34,633][00294] Avg episode reward: [(0, '-4.571')] [2023-07-24 01:15:38,231][14525] DAMAGECOUNT value on done: 528.0 [2023-07-24 01:15:38,231][14525] Sum rewards: -4.872, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.982', 'AMMO4': '-0.016', 'AMMO2': '-0.003', 'AMMO5': '0.005', 'ARMOR': '0.080', 'WEAPON5': '0.100', 'HITCOUNT': '0.130', 'AMMO3': '0.151', 'DAMAGECOUNT': '0.537', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon3': '1.378', 'weapon2': '1.748'} [2023-07-24 01:15:39,392][14526] DAMAGECOUNT value on done: 625.0 [2023-07-24 01:15:39,395][14526] Sum rewards: -5.928, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.450', 'weapon5': '0.006', 'AMMO5': '0.020', 'AMMO2': '0.033', 'ARMOR': '0.096', 'HITCOUNT': '0.110', 'weapon4': '0.132', 'AMMO3': '0.151', 'AMMO4': '0.163', 'WEAPON4': '0.300', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.435', 'WEAPON3': '0.900', 'FRAGCOUNT': '1.000', 'weapon2': '1.224', 'weapon3': '1.402'} [2023-07-24 01:15:39,630][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3244032. Throughput: 0: 320.0. Samples: 811880. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:15:39,642][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:15:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3252224. Throughput: 0: 329.2. Samples: 814240. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:15:44,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:15:49,628][00294] Fps is (10 sec: 1638.7, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3260416. Throughput: 0: 351.8. Samples: 816904. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:15:49,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:15:54,635][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3268608. Throughput: 0: 352.4. Samples: 817772. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:15:54,638][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:15:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.2, 300 sec: 1291.3). Total num frames: 3272704. Throughput: 0: 347.9. Samples: 819504. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:15:59,641][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:15:59,656][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000799_3272704.pth... [2023-07-24 01:15:59,908][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000722_2957312.pth [2023-07-24 01:16:00,779][14527] Updated weights for policy 0, policy_version 800 (0.0035) [2023-07-24 01:16:04,630][00294] Fps is (10 sec: 819.1, 60 sec: 1297.2, 300 sec: 1291.3). Total num frames: 3276800. Throughput: 0: 323.7. Samples: 821004. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:16:04,635][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:09,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3284992. Throughput: 0: 312.5. Samples: 821680. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:16:09,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:14,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 3289088. Throughput: 0: 311.2. Samples: 823312. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:16:14,635][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:19,630][00294] Fps is (10 sec: 819.4, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3293184. Throughput: 0: 312.5. Samples: 825044. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:16:19,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3301376. Throughput: 0: 307.3. Samples: 825708. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:16:24,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:29,632][00294] Fps is (10 sec: 1228.4, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 3305472. Throughput: 0: 291.8. Samples: 827372. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:16:29,635][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:34,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 3309568. Throughput: 0: 270.6. Samples: 829080. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:16:34,632][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:37,917][14527] Updated weights for policy 0, policy_version 810 (0.0069) [2023-07-24 01:16:39,628][00294] Fps is (10 sec: 1229.2, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3317760. Throughput: 0: 271.1. Samples: 829972. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:16:39,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3325952. Throughput: 0: 288.6. Samples: 832492. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0) [2023-07-24 01:16:44,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3334144. Throughput: 0: 309.6. Samples: 834936. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:16:49,632][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3342336. Throughput: 0: 313.8. Samples: 835800. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:16:54,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:16:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3346432. Throughput: 0: 315.5. Samples: 837508. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:16:59,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3350528. Throughput: 0: 316.2. Samples: 839272. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:17:04,632][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:07,277][14527] Updated weights for policy 0, policy_version 820 (0.0060) [2023-07-24 01:17:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 3358720. Throughput: 0: 324.9. Samples: 840328. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:17:09,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3366912. Throughput: 0: 346.7. Samples: 842972. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:17:14,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3375104. Throughput: 0: 361.2. Samples: 845336. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:17:19,630][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3383296. Throughput: 0: 363.6. Samples: 846332. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:17:24,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1291.3). Total num frames: 3387392. Throughput: 0: 352.4. Samples: 848348. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:17:29,632][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:34,324][14527] Updated weights for policy 0, policy_version 830 (0.0037) [2023-07-24 01:17:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1501.9, 300 sec: 1332.9). Total num frames: 3399680. Throughput: 0: 351.6. Samples: 850760. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:17:34,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:39,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1501.9, 300 sec: 1332.9). Total num frames: 3407872. Throughput: 0: 364.4. Samples: 852200. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:17:39,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:44,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1501.8, 300 sec: 1332.9). Total num frames: 3416064. Throughput: 0: 380.6. Samples: 854636. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:17:44,636][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 3420160. Throughput: 0: 379.1. Samples: 856332. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:17:49,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:54,628][00294] Fps is (10 sec: 819.3, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3424256. Throughput: 0: 373.8. Samples: 857148. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:17:54,640][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 3432448. Throughput: 0: 352.0. Samples: 858812. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:17:59,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:17:59,657][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000838_3432448.pth... [2023-07-24 01:17:59,898][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000760_3112960.pth [2023-07-24 01:18:03,602][14527] Updated weights for policy 0, policy_version 840 (0.0025) [2023-07-24 01:18:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1501.9, 300 sec: 1332.9). Total num frames: 3440640. Throughput: 0: 350.1. Samples: 861092. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:18:04,641][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1501.9, 300 sec: 1332.9). Total num frames: 3448832. Throughput: 0: 356.9. Samples: 862392. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:18:09,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 3452928. Throughput: 0: 359.4. Samples: 864520. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:18:14,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:19,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3457024. Throughput: 0: 338.8. Samples: 866008. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:18:19,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3465216. Throughput: 0: 321.2. Samples: 866652. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:18:24,639][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3469312. Throughput: 0: 296.1. Samples: 867960. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:18:29,635][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:34,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3473408. Throughput: 0: 289.2. Samples: 869344. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:18:34,635][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:39,629][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 3477504. Throughput: 0: 290.3. Samples: 870212. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:18:39,637][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:40,783][14527] Updated weights for policy 0, policy_version 850 (0.0079) [2023-07-24 01:18:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.6, 300 sec: 1291.3). Total num frames: 3485696. Throughput: 0: 303.1. Samples: 872452. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:18:44,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:49,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3493888. Throughput: 0: 292.9. Samples: 874272. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:18:49,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3497984. Throughput: 0: 282.8. Samples: 875120. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:18:54,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:18:59,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 3502080. Throughput: 0: 273.4. Samples: 876824. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:18:59,635][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1319.1). Total num frames: 3510272. Throughput: 0: 287.6. Samples: 878952. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:19:04,636][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:09,630][00294] Fps is (10 sec: 1638.1, 60 sec: 1160.5, 300 sec: 1319.1). Total num frames: 3518464. Throughput: 0: 302.5. Samples: 880264. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:19:09,639][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:10,413][14527] Updated weights for policy 0, policy_version 860 (0.0031) [2023-07-24 01:19:14,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1228.8, 300 sec: 1319.0). Total num frames: 3526656. Throughput: 0: 326.2. Samples: 882640. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:19:14,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:19,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3530752. Throughput: 0: 333.5. Samples: 884352. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:19:19,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:24,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3538944. Throughput: 0: 332.7. Samples: 885184. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:19:24,632][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3543040. Throughput: 0: 320.1. Samples: 886856. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:19:29,639][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 3551232. Throughput: 0: 334.1. Samples: 889308. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:19:34,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:39,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1365.3, 300 sec: 1319.0). Total num frames: 3559424. Throughput: 0: 344.6. Samples: 890628. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:19:39,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:40,331][14527] Updated weights for policy 0, policy_version 870 (0.0063) [2023-07-24 01:19:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3567616. Throughput: 0: 352.7. Samples: 892696. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:19:44,632][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:49,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3571712. Throughput: 0: 343.0. Samples: 894388. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:19:49,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3579904. Throughput: 0: 331.7. Samples: 895192. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:19:54,637][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3584000. Throughput: 0: 320.5. Samples: 897064. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:19:59,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:19:59,650][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000875_3584000.pth... [2023-07-24 01:19:59,856][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000799_3272704.pth [2023-07-24 01:20:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3592192. Throughput: 0: 340.3. Samples: 899664. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:20:04,630][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1319.1). Total num frames: 3600384. Throughput: 0: 350.6. Samples: 900960. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:20:09,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:11,163][14527] Updated weights for policy 0, policy_version 880 (0.0058) [2023-07-24 01:20:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3604480. Throughput: 0: 350.8. Samples: 902640. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:20:14,637][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3612672. Throughput: 0: 333.8. Samples: 904328. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:20:19,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:24,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3616768. Throughput: 0: 323.2. Samples: 905172. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:20:24,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 3629056. Throughput: 0: 325.4. Samples: 907340. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:20:29,635][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:34,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3633152. Throughput: 0: 331.8. Samples: 909320. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:20:34,638][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3637248. Throughput: 0: 329.6. Samples: 910024. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:20:39,633][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3641344. Throughput: 0: 317.5. Samples: 911352. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:20:44,636][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:47,297][14527] Updated weights for policy 0, policy_version 890 (0.0029) [2023-07-24 01:20:49,630][00294] Fps is (10 sec: 819.0, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3645440. Throughput: 0: 288.6. Samples: 912652. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:20:49,635][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:54,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 3649536. Throughput: 0: 275.1. Samples: 913340. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:20:54,634][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:20:59,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3657728. Throughput: 0: 273.2. Samples: 914932. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:20:59,636][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:21:04,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3665920. Throughput: 0: 292.4. Samples: 917488. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:21:04,638][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:21:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3674112. Throughput: 0: 303.4. Samples: 918824. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:21:09,631][00294] Avg episode reward: [(0, '-4.619')] [2023-07-24 01:21:10,410][14524] DAMAGECOUNT value on done: 889.0 [2023-07-24 01:21:10,411][14524] Sum rewards: -2.980, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.108', 'AMMO2': '0.001', 'AMMO4': '0.005', 'AMMO5': '0.013', 'weapon5': '0.018', 'WEAPON5': '0.050', 'HITCOUNT': '0.150', 'AMMO3': '0.154', 'DAMAGECOUNT': '0.615', 'WEAPON3': '0.950', 'weapon2': '1.124', 'FRAGCOUNT': '2.000', 'weapon3': '2.048'} [2023-07-24 01:21:11,546][14528] DAMAGECOUNT value on done: 762.0 [2023-07-24 01:21:11,548][14528] Sum rewards: -7.612, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.270', 'FRAGCOUNT': '-0.500', 'weapon7': '0.008', 'AMMO5': '0.010', 'AMMO2': '0.023', 'weapon5': '0.098', 'AMMO4': '0.116', 'AMMO3': '0.122', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON5': '0.200', 'HITCOUNT': '0.240', 'WEAPON4': '0.300', 'weapon4': '0.382', 'ARMOR': '0.400', 'DAMAGECOUNT': '0.705', 'WEAPON3': '0.750', 'weapon2': '0.862', 'weapon3': '1.592'} [2023-07-24 01:21:12,976][14532] DAMAGECOUNT value on done: 1144.0 [2023-07-24 01:21:12,979][14532] Sum rewards: -3.015, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.038', 'AMMO4': '-0.001', 'AMMO2': '-0.000', 'AMMO5': '0.017', 'weapon5': '0.092', 'AMMO3': '0.152', 'HITCOUNT': '0.170', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.510', 'WEAPON3': '0.850', 'weapon2': '1.396', 'weapon3': '1.586', 'FRAGCOUNT': '2.000'} [2023-07-24 01:21:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3678208. Throughput: 0: 298.0. Samples: 920748. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:21:14,633][00294] Avg episode reward: [(0, '-4.630')] [2023-07-24 01:21:17,184][14524] DAMAGECOUNT value on done: 953.0 [2023-07-24 01:21:17,185][14524] Sum rewards: 0.908, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.334', 'AMMO2': '0.011', 'AMMO5': '0.012', 'weapon5': '0.020', 'ARMOR': '0.040', 'AMMO4': '0.055', 'AMMO3': '0.064', 'weapon7': '0.094', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'WEAPON7': '0.200', 'HITCOUNT': '0.250', 'WEAPON3': '0.350', 'weapon4': '0.592', 'weapon3': '0.942', 'FRAGCOUNT': '1.000', 'DAMAGECOUNT': '1.107', 'weapon2': '1.214'} [2023-07-24 01:21:18,215][14528] DAMAGECOUNT value on done: 879.0 [2023-07-24 01:21:18,215][14528] Sum rewards: -3.884, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.748', 'AMMO2': '0.005', 'AMMO5': '0.009', 'AMMO4': '0.025', 'ARMOR': '0.056', 'weapon5': '0.082', 'AMMO3': '0.100', 'weapon4': '0.132', 'WEAPON4': '0.150', 'WEAPON5': '0.200', 'HITCOUNT': '0.280', 'WEAPON3': '0.650', 'DAMAGECOUNT': '1.128', 'weapon3': '1.356', 'weapon2': '1.690', 'FRAGCOUNT': '2.500'} [2023-07-24 01:21:19,370][14527] Updated weights for policy 0, policy_version 900 (0.0027) [2023-07-24 01:21:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3686400. Throughput: 0: 291.9. Samples: 922456. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:21:19,634][00294] Avg episode reward: [(0, '-4.612')] [2023-07-24 01:21:19,960][14532] DAMAGECOUNT value on done: 1051.0 [2023-07-24 01:21:19,962][14532] Sum rewards: -8.156, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-1.290', 'AMMO2': '0.008', 'WEAPON1': '0.010', 'AMMO5': '0.013', 'ARMOR': '0.032', 'AMMO4': '0.040', 'weapon5': '0.100', 'HITCOUNT': '0.110', 'AMMO3': '0.182', 'WEAPON5': '0.200', 'FRAGCOUNT': '0.500', 'DAMAGECOUNT': '0.675', 'WEAPON3': '0.950', 'weapon2': '1.318', 'weapon3': '1.746'} [2023-07-24 01:21:21,601][14531] DAMAGECOUNT value on done: 899.0 [2023-07-24 01:21:21,605][14531] Sum rewards: -3.415, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.984', 'AMMO4': '-0.003', 'AMMO2': '-0.001', 'WEAPON1': '0.010', 'ARMOR': '0.012', 'AMMO5': '0.014', 'AMMO3': '0.117', 'weapon5': '0.138', 'HITCOUNT': '0.140', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.573', 'WEAPON3': '0.750', 'weapon2': '0.980', 'FRAGCOUNT': '1.000', 'weapon3': '1.888'} [2023-07-24 01:21:24,400][14524] DAMAGECOUNT value on done: 944.0 [2023-07-24 01:21:24,401][14524] Sum rewards: -2.129, reward structure: {'DEATHCOUNT': '-9.000', 'AMMO5': '0.003', 'AMMO2': '0.011', 'WEAPON5': '0.050', 'AMMO4': '0.054', 'weapon5': '0.074', 'AMMO3': '0.097', 'WEAPON4': '0.100', 'HEALTH': '0.117', 'weapon4': '0.176', 'HITCOUNT': '0.200', 'ARMOR': '0.555', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.774', 'FRAGCOUNT': '1.000', 'weapon3': '1.394', 'weapon2': '1.666'} [2023-07-24 01:21:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 3690496. Throughput: 0: 294.8. Samples: 923292. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:21:24,630][00294] Avg episode reward: [(0, '-4.611')] [2023-07-24 01:21:25,465][14528] DAMAGECOUNT value on done: 614.0 [2023-07-24 01:21:25,469][14528] Sum rewards: -5.061, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.321', 'AMMO4': '-0.005', 'AMMO2': '-0.001', 'AMMO5': '0.011', 'WEAPON1': '0.020', 'WEAPON4': '0.050', 'ARMOR': '0.064', 'HITCOUNT': '0.070', 'AMMO3': '0.132', 'weapon4': '0.170', 'weapon5': '0.194', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.519', 'WEAPON3': '0.700', 'weapon3': '0.768', 'FRAGCOUNT': '1.000', 'weapon2': '1.968'} [2023-07-24 01:21:26,869][14532] DAMAGECOUNT value on done: 695.0 [2023-07-24 01:21:28,161][14531] DAMAGECOUNT value on done: 861.0 [2023-07-24 01:21:28,165][14531] Sum rewards: -7.517, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.960', 'AMMO5': '0.010', 'AMMO2': '0.013', 'weapon5': '0.022', 'ARMOR': '0.024', 'AMMO4': '0.067', 'HITCOUNT': '0.110', 'WEAPON4': '0.150', 'weapon4': '0.168', 'WEAPON5': '0.200', 'AMMO3': '0.207', 'DAMAGECOUNT': '0.360', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.000', 'weapon2': '1.466', 'weapon3': '1.646'} [2023-07-24 01:21:29,556][14524] DAMAGECOUNT value on done: 829.0 [2023-07-24 01:21:29,558][14524] Sum rewards: -3.657, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.001', 'AMMO5': '0.010', 'AMMO2': '0.014', 'weapon5': '0.062', 'AMMO4': '0.071', 'WEAPON5': '0.100', 'AMMO3': '0.109', 'weapon4': '0.110', 'WEAPON4': '0.150', 'HITCOUNT': '0.290', 'ARMOR': '0.420', 'WEAPON3': '0.600', 'weapon3': '1.334', 'FRAGCOUNT': '1.500', 'DAMAGECOUNT': '1.530', 'weapon2': '1.544'} [2023-07-24 01:21:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1319.1). Total num frames: 3698688. Throughput: 0: 308.5. Samples: 925236. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:21:29,631][00294] Avg episode reward: [(0, '-4.598')] [2023-07-24 01:21:30,346][14528] DAMAGECOUNT value on done: 543.0 [2023-07-24 01:21:30,353][14528] Sum rewards: -2.659, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.190', 'AMMO2': '0.006', 'AMMO5': '0.017', 'AMMO4': '0.030', 'ARMOR': '0.040', 'weapon7': '0.084', 'AMMO3': '0.091', 'weapon5': '0.096', 'HITCOUNT': '0.110', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON4': '0.150', 'WEAPON7': '0.200', 'WEAPON5': '0.250', 'WEAPON3': '0.400', 'weapon4': '0.482', 'DAMAGECOUNT': '0.546', 'FRAGCOUNT': '1.000', 'weapon2': '1.104', 'weapon3': '1.184'} [2023-07-24 01:21:31,658][14532] DAMAGECOUNT value on done: 505.0 [2023-07-24 01:21:31,666][14532] Sum rewards: -1.896, reward structure: {'DEATHCOUNT': '-5.250', 'FRAGCOUNT': '-0.500', 'HEALTH': '-0.192', 'AMMO5': '0.003', 'AMMO2': '0.030', 'weapon5': '0.040', 'WEAPON5': '0.050', 'AMMO3': '0.060', 'HITCOUNT': '0.070', 'WEAPON4': '0.100', 'AMMO4': '0.150', 'DAMAGECOUNT': '0.192', 'WEAPON3': '0.300', 'weapon4': '0.338', 'ARMOR': '0.591', 'weapon3': '0.906', 'weapon2': '1.216'} [2023-07-24 01:21:31,897][14529] DAMAGECOUNT value on done: 914.0 [2023-07-24 01:21:31,910][14529] Sum rewards: 0.169, reward structure: {'DEATHCOUNT': '-4.500', 'HEALTH': '-1.335', 'AMMO4': '-0.027', 'AMMO2': '-0.005', 'AMMO5': '0.005', 'weapon5': '0.008', 'WEAPON1': '0.020', 'AMMO3': '0.061', 'weapon7': '0.070', 'HITCOUNT': '0.100', 'WEAPON5': '0.100', 'WEAPON4': '0.100', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON7': '0.200', 'weapon4': '0.294', 'DAMAGECOUNT': '0.378', 'WEAPON3': '0.450', 'weapon2': '0.872', 'weapon3': '1.138', 'FRAGCOUNT': '2.000'} [2023-07-24 01:21:32,830][14531] DAMAGECOUNT value on done: 1084.0 [2023-07-24 01:21:32,834][14531] Sum rewards: -4.233, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.120', 'AMMO5': '0.004', 'AMMO2': '0.014', 'weapon5': '0.070', 'AMMO4': '0.072', 'WEAPON5': '0.100', 'WEAPON4': '0.100', 'AMMO3': '0.154', 'weapon4': '0.164', 'HITCOUNT': '0.210', 'DAMAGECOUNT': '0.681', 'WEAPON3': '0.700', 'FRAGCOUNT': '1.000', 'weapon2': '1.230', 'weapon3': '1.388'} [2023-07-24 01:21:34,353][14524] DAMAGECOUNT value on done: 1019.0 [2023-07-24 01:21:34,357][14524] Sum rewards: -6.608, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-0.606', 'AMMO5': '0.014', 'AMMO2': '0.017', 'WEAPON1': '0.020', 'AMMO4': '0.086', 'weapon4': '0.120', 'weapon5': '0.132', 'WEAPON4': '0.150', 'AMMO3': '0.193', 'HITCOUNT': '0.210', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.645', 'weapon2': '0.950', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.100', 'weapon3': '1.860'} [2023-07-24 01:21:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 3706880. Throughput: 0: 336.6. Samples: 927800. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:21:34,634][00294] Avg episode reward: [(0, '-4.592')] [2023-07-24 01:21:35,039][14528] DAMAGECOUNT value on done: 997.0 [2023-07-24 01:21:35,041][14528] Sum rewards: 0.062, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.630', 'AMMO2': '0.006', 'AMMO5': '0.025', 'weapon4': '0.028', 'weapon5': '0.028', 'AMMO4': '0.029', 'WEAPON1': '0.030', 'ARMOR': '0.040', 'WEAPON4': '0.100', 'AMMO3': '0.205', 'HITCOUNT': '0.400', 'WEAPON5': '0.400', 'WEAPON3': '1.000', 'weapon2': '1.574', 'weapon3': '1.602', 'DAMAGECOUNT': '1.725', 'FRAGCOUNT': '5.000'} [2023-07-24 01:21:36,481][14532] DAMAGECOUNT value on done: 899.0 [2023-07-24 01:21:36,482][14532] Sum rewards: -3.569, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-2.817', 'AMMO4': '-0.052', 'AMMO2': '-0.010', 'ARMOR': '0.004', 'AMMO5': '0.010', 'weapon7': '0.068', 'weapon5': '0.082', 'WEAPON5': '0.100', 'WEAPON4': '0.100', 'weapon4': '0.120', 'AMMO3': '0.159', 'HITCOUNT': '0.160', 'AMMO6': '0.320', 'AMMO7': '0.320', 'WEAPON7': '0.400', 'DAMAGECOUNT': '0.690', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.050', 'weapon2': '1.432', 'weapon3': '1.546'} [2023-07-24 01:21:37,749][14529] DAMAGECOUNT value on done: 784.0 [2023-07-24 01:21:37,753][14529] Sum rewards: -0.168, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.016', 'AMMO2': '0.007', 'AMMO5': '0.009', 'AMMO4': '0.034', 'ARMOR': '0.040', 'weapon5': '0.058', 'HITCOUNT': '0.130', 'AMMO3': '0.134', 'WEAPON4': '0.150', 'weapon7': '0.160', 'WEAPON5': '0.200', 'AMMO6': '0.220', 'AMMO7': '0.220', 'WEAPON7': '0.300', 'weapon4': '0.556', 'WEAPON3': '0.650', 'weapon2': '0.822', 'weapon3': '1.154', 'DAMAGECOUNT': '1.254', 'FRAGCOUNT': '3.000'} [2023-07-24 01:21:38,219][14531] DAMAGECOUNT value on done: 827.0 [2023-07-24 01:21:38,226][14531] Sum rewards: -2.285, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.360', 'AMMO5': '0.012', 'weapon5': '0.018', 'AMMO2': '0.023', 'ARMOR': '0.028', 'weapon4': '0.032', 'weapon7': '0.050', 'AMMO3': '0.099', 'WEAPON4': '0.100', 'AMMO4': '0.114', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON7': '0.200', 'HITCOUNT': '0.230', 'WEAPON5': '0.250', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.879', 'weapon2': '1.538', 'weapon3': '1.612'} [2023-07-24 01:21:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 3715072. Throughput: 0: 347.6. Samples: 928980. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:21:39,630][00294] Avg episode reward: [(0, '-4.351')] [2023-07-24 01:21:40,611][14524] DAMAGECOUNT value on done: 1291.0 [2023-07-24 01:21:40,612][14524] Sum rewards: -6.641, reward structure: {'DEATHCOUNT': '-13.500', 'HEALTH': '-1.820', 'AMMO5': '0.007', 'WEAPON1': '0.020', 'AMMO2': '0.029', 'weapon5': '0.088', 'AMMO3': '0.129', 'AMMO4': '0.143', 'WEAPON5': '0.200', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'HITCOUNT': '0.230', 'WEAPON4': '0.300', 'weapon4': '0.442', 'ARMOR': '0.511', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.954', 'weapon3': '1.350', 'weapon2': '1.476', 'FRAGCOUNT': '1.500'} [2023-07-24 01:21:41,595][14528] DAMAGECOUNT value on done: 690.0 [2023-07-24 01:21:44,272][14532] DAMAGECOUNT value on done: 668.0 [2023-07-24 01:21:44,272][14532] Sum rewards: -5.687, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.100', 'AMMO5': '0.004', 'AMMO2': '0.006', 'AMMO4': '0.030', 'WEAPON4': '0.050', 'weapon5': '0.056', 'WEAPON5': '0.100', 'weapon4': '0.106', 'HITCOUNT': '0.120', 'AMMO3': '0.139', 'DAMAGECOUNT': '0.495', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon3': '1.400', 'weapon2': '1.856'} [2023-07-24 01:21:44,342][14529] DAMAGECOUNT value on done: 586.0 [2023-07-24 01:21:44,356][14529] Sum rewards: -2.933, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.929', 'AMMO5': '0.019', 'AMMO2': '0.028', 'weapon4': '0.076', 'WEAPON4': '0.100', 'AMMO3': '0.101', 'AMMO4': '0.138', 'weapon5': '0.172', 'HITCOUNT': '0.200', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON5': '0.300', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.720', 'weapon2': '0.862', 'weapon3': '1.780'} [2023-07-24 01:21:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3719168. Throughput: 0: 349.8. Samples: 930672. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:21:44,631][00294] Avg episode reward: [(0, '-4.375')] [2023-07-24 01:21:46,117][14531] DAMAGECOUNT value on done: 578.0 [2023-07-24 01:21:46,850][14530] DAMAGECOUNT value on done: 883.0 [2023-07-24 01:21:47,786][14524] DAMAGECOUNT value on done: 703.0 [2023-07-24 01:21:48,864][14528] DAMAGECOUNT value on done: 806.0 [2023-07-24 01:21:48,865][14528] Sum rewards: -2.421, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.170', 'AMMO5': '0.003', 'ARMOR': '0.005', 'AMMO2': '0.011', 'AMMO4': '0.052', 'WEAPON4': '0.100', 'AMMO3': '0.110', 'HITCOUNT': '0.110', 'weapon4': '0.222', 'DAMAGECOUNT': '0.480', 'WEAPON3': '0.600', 'weapon3': '1.428', 'weapon2': '1.628', 'FRAGCOUNT': '2.000'} [2023-07-24 01:21:49,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 3723264. Throughput: 0: 331.4. Samples: 932400. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:21:49,638][00294] Avg episode reward: [(0, '-4.313')] [2023-07-24 01:21:50,502][14527] Updated weights for policy 0, policy_version 910 (0.0041) [2023-07-24 01:21:50,895][14532] DAMAGECOUNT value on done: 746.0 [2023-07-24 01:21:50,907][14532] Sum rewards: -2.891, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.260', 'AMMO5': '0.007', 'ARMOR': '0.008', 'AMMO2': '0.009', 'AMMO4': '0.045', 'weapon5': '0.072', 'weapon7': '0.130', 'AMMO3': '0.150', 'WEAPON5': '0.150', 'HITCOUNT': '0.200', 'WEAPON4': '0.200', 'AMMO6': '0.220', 'AMMO7': '0.220', 'WEAPON7': '0.300', 'weapon4': '0.300', 'DAMAGECOUNT': '0.810', 'WEAPON3': '0.850', 'weapon3': '1.240', 'weapon2': '1.458', 'FRAGCOUNT': '2.000'} [2023-07-24 01:21:51,832][14529] DAMAGECOUNT value on done: 902.0 [2023-07-24 01:21:51,837][14529] Sum rewards: -4.339, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.131', 'AMMO5': '0.012', 'AMMO2': '0.026', 'ARMOR': '0.104', 'weapon5': '0.104', 'HITCOUNT': '0.120', 'weapon4': '0.124', 'AMMO4': '0.129', 'AMMO3': '0.184', 'WEAPON4': '0.250', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.597', 'weapon2': '0.996', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.000', 'weapon3': '1.846'} [2023-07-24 01:21:53,249][14531] DAMAGECOUNT value on done: 877.0 [2023-07-24 01:21:53,250][14531] Sum rewards: -3.523, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.864', 'FRAGCOUNT': '-0.500', 'WEAPON1': '0.010', 'AMMO5': '0.015', 'AMMO2': '0.016', 'ARMOR': '0.049', 'AMMO4': '0.081', 'weapon7': '0.096', 'weapon5': '0.108', 'AMMO3': '0.143', 'HITCOUNT': '0.150', 'WEAPON4': '0.250', 'WEAPON5': '0.300', 'AMMO6': '0.320', 'AMMO7': '0.320', 'weapon4': '0.354', 'WEAPON7': '0.400', 'WEAPON3': '0.900', 'DAMAGECOUNT': '1.017', 'weapon3': '1.272', 'weapon2': '1.290'} [2023-07-24 01:21:54,238][14530] DAMAGECOUNT value on done: 1274.0 [2023-07-24 01:21:54,238][14530] Sum rewards: -1.183, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.066', 'AMMO2': '0.016', 'weapon7': '0.050', 'AMMO4': '0.078', 'AMMO3': '0.123', 'weapon4': '0.140', 'WEAPON4': '0.150', 'HITCOUNT': '0.220', 'AMMO6': '0.300', 'WEAPON7': '0.300', 'AMMO7': '0.300', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.666', 'weapon2': '1.068', 'weapon3': '1.872', 'FRAGCOUNT': '2.000'} [2023-07-24 01:21:54,385][14525] DAMAGECOUNT value on done: 951.0 [2023-07-24 01:21:54,387][14525] Sum rewards: -6.959, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-2.000', 'HEALTH': '-1.086', 'AMMO2': '0.005', 'WEAPON1': '0.020', 'AMMO5': '0.022', 'AMMO4': '0.023', 'WEAPON4': '0.050', 'ARMOR': '0.056', 'AMMO3': '0.112', 'HITCOUNT': '0.130', 'weapon4': '0.132', 'weapon5': '0.214', 'WEAPON5': '0.500', 'DAMAGECOUNT': '0.531', 'WEAPON3': '0.700', 'weapon2': '1.224', 'weapon3': '1.408'} [2023-07-24 01:21:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3731456. Throughput: 0: 320.6. Samples: 933252. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:21:54,631][00294] Avg episode reward: [(0, '-4.149')] [2023-07-24 01:21:54,668][14524] DAMAGECOUNT value on done: 489.0 [2023-07-24 01:21:54,684][14524] Sum rewards: 1.870, reward structure: {'DEATHCOUNT': '-4.500', 'HEALTH': '-0.442', 'AMMO2': '0.001', 'AMMO4': '0.007', 'AMMO5': '0.009', 'weapon4': '0.016', 'WEAPON1': '0.030', 'weapon5': '0.038', 'WEAPON4': '0.050', 'AMMO3': '0.066', 'WEAPON5': '0.150', 'HITCOUNT': '0.200', 'WEAPON3': '0.350', 'ARMOR': '0.504', 'DAMAGECOUNT': '0.747', 'weapon2': '1.098', 'weapon3': '1.544', 'FRAGCOUNT': '2.000'} [2023-07-24 01:21:54,927][14526] DAMAGECOUNT value on done: 650.0 [2023-07-24 01:21:54,930][14526] Sum rewards: -3.150, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.318', 'AMMO2': '0.012', 'AMMO5': '0.012', 'ARMOR': '0.060', 'AMMO4': '0.060', 'AMMO3': '0.115', 'weapon5': '0.144', 'WEAPON4': '0.150', 'HITCOUNT': '0.230', 'WEAPON5': '0.250', 'weapon4': '0.252', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.810', 'weapon3': '1.172', 'weapon2': '1.750', 'FRAGCOUNT': '3.000'} [2023-07-24 01:21:55,651][14528] DAMAGECOUNT value on done: 1080.0 [2023-07-24 01:21:55,653][14528] Sum rewards: -4.055, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.270', 'AMMO4': '-0.025', 'AMMO2': '-0.005', 'AMMO5': '0.015', 'WEAPON1': '0.020', 'ARMOR': '0.048', 'WEAPON4': '0.050', 'weapon7': '0.070', 'weapon5': '0.100', 'HITCOUNT': '0.120', 'AMMO6': '0.120', 'AMMO7': '0.120', 'AMMO3': '0.128', 'weapon4': '0.134', 'WEAPON7': '0.200', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.711', 'WEAPON3': '0.800', 'weapon3': '1.202', 'FRAGCOUNT': '1.500', 'weapon2': '1.556'} [2023-07-24 01:21:57,059][14529] DAMAGECOUNT value on done: 818.0 [2023-07-24 01:21:57,067][14529] Sum rewards: -2.652, reward structure: {'DEATHCOUNT': '-9.000', 'AMMO5': '0.008', 'WEAPON1': '0.010', 'AMMO2': '0.020', 'WEAPON4': '0.050', 'ARMOR': '0.056', 'HITCOUNT': '0.070', 'weapon5': '0.096', 'AMMO4': '0.100', 'AMMO3': '0.127', 'WEAPON5': '0.150', 'weapon4': '0.304', 'DAMAGECOUNT': '0.360', 'WEAPON3': '0.650', 'HEALTH': '0.652', 'FRAGCOUNT': '1.000', 'weapon2': '1.286', 'weapon3': '1.408'} [2023-07-24 01:21:57,081][14532] DAMAGECOUNT value on done: 1172.0 [2023-07-24 01:21:57,083][14532] Sum rewards: 1.690, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.714', 'AMMO2': '0.010', 'AMMO4': '0.050', 'AMMO3': '0.072', 'ARMOR': '0.080', 'weapon7': '0.104', 'AMMO6': '0.120', 'AMMO7': '0.120', 'HITCOUNT': '0.130', 'WEAPON7': '0.200', 'WEAPON4': '0.300', 'weapon4': '0.400', 'WEAPON3': '0.450', 'DAMAGECOUNT': '0.750', 'weapon3': '0.850', 'weapon2': '1.018', 'FRAGCOUNT': '3.000'} [2023-07-24 01:21:58,580][14530] DAMAGECOUNT value on done: 718.0 [2023-07-24 01:21:58,589][14530] Sum rewards: -5.318, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.654', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.012', 'AMMO2': '0.015', 'ARMOR': '0.068', 'AMMO4': '0.073', 'AMMO3': '0.149', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'HITCOUNT': '0.220', 'weapon5': '0.240', 'weapon4': '0.256', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.825', 'weapon3': '1.088', 'weapon2': '1.540'} [2023-07-24 01:21:58,654][14531] DAMAGECOUNT value on done: 779.0 [2023-07-24 01:21:58,660][14531] Sum rewards: -4.347, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.313', 'FRAGCOUNT': '-0.500', 'AMMO4': '-0.027', 'AMMO2': '-0.005', 'AMMO5': '0.007', 'weapon5': '0.060', 'AMMO3': '0.087', 'HITCOUNT': '0.150', 'WEAPON5': '0.150', 'DAMAGECOUNT': '0.405', 'ARMOR': '0.452', 'WEAPON3': '0.500', 'weapon3': '1.304', 'weapon2': '1.882'} [2023-07-24 01:21:58,748][14525] DAMAGECOUNT value on done: 535.0 [2023-07-24 01:21:58,751][14525] Sum rewards: -9.529, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-3.010', 'AMMO2': '0.003', 'AMMO5': '0.015', 'AMMO4': '0.017', 'WEAPON1': '0.020', 'WEAPON4': '0.050', 'ARMOR': '0.060', 'HITCOUNT': '0.080', 'weapon4': '0.156', 'AMMO3': '0.181', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.420', 'WEAPON3': '0.950', 'FRAGCOUNT': '1.000', 'weapon3': '1.262', 'weapon2': '1.816'} [2023-07-24 01:21:59,414][14526] DAMAGECOUNT value on done: 907.0 [2023-07-24 01:21:59,418][14526] Sum rewards: -8.701, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-3.000', 'HEALTH': '-2.008', 'AMMO5': '0.007', 'weapon5': '0.022', 'AMMO2': '0.031', 'ARMOR': '0.040', 'AMMO3': '0.119', 'HITCOUNT': '0.150', 'WEAPON5': '0.150', 'AMMO4': '0.153', 'weapon4': '0.236', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.585', 'WEAPON3': '0.750', 'weapon2': '1.276', 'weapon3': '1.538'} [2023-07-24 01:21:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 3739648. Throughput: 0: 327.4. Samples: 935480. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:21:59,630][00294] Avg episode reward: [(0, '-4.105')] [2023-07-24 01:21:59,654][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000913_3739648.pth... [2023-07-24 01:21:59,851][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000838_3432448.pth [2023-07-24 01:22:01,613][14529] DAMAGECOUNT value on done: 1074.0 [2023-07-24 01:22:01,616][14529] Sum rewards: -1.651, reward structure: {'DEATHCOUNT': '-8.250', 'ARMOR': '0.008', 'AMMO5': '0.012', 'AMMO2': '0.017', 'WEAPON1': '0.020', 'AMMO4': '0.085', 'AMMO3': '0.098', 'WEAPON4': '0.100', 'weapon5': '0.104', 'HITCOUNT': '0.200', 'weapon4': '0.206', 'WEAPON5': '0.300', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.585', 'HEALTH': '0.629', 'FRAGCOUNT': '1.000', 'weapon3': '1.142', 'weapon2': '1.542'} [2023-07-24 01:22:03,366][14531] DAMAGECOUNT value on done: 694.0 [2023-07-24 01:22:03,368][14531] Sum rewards: -5.134, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.113', 'AMMO5': '0.007', 'AMMO2': '0.013', 'weapon5': '0.056', 'weapon7': '0.062', 'AMMO4': '0.066', 'ARMOR': '0.072', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.149', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'HITCOUNT': '0.160', 'weapon4': '0.398', 'DAMAGECOUNT': '0.525', 'WEAPON3': '0.800', 'weapon3': '0.950', 'weapon2': '1.620', 'FRAGCOUNT': '2.000'} [2023-07-24 01:22:03,723][14530] DAMAGECOUNT value on done: 774.0 [2023-07-24 01:22:03,723][14530] Sum rewards: -10.580, reward structure: {'DEATHCOUNT': '-14.250', 'HEALTH': '-3.160', 'AMMO5': '0.007', 'ARMOR': '0.008', 'WEAPON1': '0.010', 'AMMO2': '0.025', 'weapon5': '0.058', 'weapon4': '0.106', 'AMMO4': '0.124', 'HITCOUNT': '0.140', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'AMMO3': '0.241', 'DAMAGECOUNT': '0.570', 'FRAGCOUNT': '1.000', 'weapon2': '1.126', 'WEAPON3': '1.300', 'weapon3': '1.764'} [2023-07-24 01:22:03,991][14525] DAMAGECOUNT value on done: 712.0 [2023-07-24 01:22:03,992][14525] Sum rewards: -1.557, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.850', 'AMMO5': '0.005', 'AMMO2': '0.014', 'weapon5': '0.028', 'ARMOR': '0.068', 'AMMO4': '0.072', 'WEAPON5': '0.100', 'AMMO3': '0.125', 'WEAPON4': '0.150', 'HITCOUNT': '0.220', 'weapon4': '0.344', 'DAMAGECOUNT': '0.675', 'WEAPON3': '0.700', 'weapon2': '1.206', 'weapon3': '1.586', 'FRAGCOUNT': '3.000'} [2023-07-24 01:22:04,631][00294] Fps is (10 sec: 1637.9, 60 sec: 1365.3, 300 sec: 1319.0). Total num frames: 3747840. Throughput: 0: 346.5. Samples: 938048. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:22:04,634][00294] Avg episode reward: [(0, '-4.101')] [2023-07-24 01:22:05,135][14526] DAMAGECOUNT value on done: 1480.0 [2023-07-24 01:22:05,139][14526] Sum rewards: 0.611, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-0.584', 'AMMO5': '0.009', 'WEAPON1': '0.010', 'AMMO2': '0.013', 'WEAPON4': '0.050', 'AMMO4': '0.064', 'weapon4': '0.114', 'AMMO3': '0.140', 'weapon5': '0.160', 'WEAPON5': '0.200', 'HITCOUNT': '0.270', 'WEAPON3': '0.700', 'weapon3': '1.180', 'DAMAGECOUNT': '1.311', 'weapon2': '1.724', 'FRAGCOUNT': '2.000'} [2023-07-24 01:22:09,566][14529] DAMAGECOUNT value on done: 1292.0 [2023-07-24 01:22:09,567][14529] Sum rewards: -2.844, reward structure: {'DEATHCOUNT': '-6.750', 'FRAGCOUNT': '-1.000', 'HEALTH': '-0.810', 'AMMO5': '0.015', 'WEAPON1': '0.020', 'AMMO2': '0.024', 'AMMO3': '0.100', 'AMMO4': '0.120', 'weapon5': '0.144', 'HITCOUNT': '0.200', 'WEAPON5': '0.200', 'WEAPON4': '0.300', 'weapon4': '0.306', 'WEAPON3': '0.550', 'weapon3': '0.994', 'DAMAGECOUNT': '1.317', 'weapon2': '1.426'} [2023-07-24 01:22:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3751936. Throughput: 0: 347.6. Samples: 938932. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:22:09,635][00294] Avg episode reward: [(0, '-4.074')] [2023-07-24 01:22:12,361][14530] DAMAGECOUNT value on done: 702.0 [2023-07-24 01:22:12,362][14530] Sum rewards: -3.836, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.560', 'AMMO5': '0.005', 'AMMO2': '0.019', 'ARMOR': '0.044', 'AMMO4': '0.097', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'WEAPON5': '0.100', 'HITCOUNT': '0.150', 'AMMO3': '0.155', 'weapon4': '0.164', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.540', 'WEAPON3': '0.850', 'FRAGCOUNT': '1.000', 'weapon2': '1.042', 'weapon3': '1.308'} [2023-07-24 01:22:12,708][14525] DAMAGECOUNT value on done: 737.0 [2023-07-24 01:22:12,709][14525] Sum rewards: -5.434, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.185', 'AMMO4': '-0.003', 'AMMO2': '-0.001', 'AMMO5': '0.012', 'WEAPON1': '0.020', 'ARMOR': '0.040', 'WEAPON4': '0.050', 'weapon5': '0.088', 'weapon4': '0.118', 'HITCOUNT': '0.120', 'AMMO3': '0.140', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.555', 'WEAPON3': '0.900', 'weapon2': '1.388', 'weapon3': '1.624', 'FRAGCOUNT': '2.000'} [2023-07-24 01:22:13,743][14526] DAMAGECOUNT value on done: 1210.0 [2023-07-24 01:22:13,744][14526] Sum rewards: -7.641, reward structure: {'DEATHCOUNT': '-15.000', 'HEALTH': '-1.952', 'AMMO2': '0.003', 'AMMO5': '0.012', 'AMMO4': '0.016', 'WEAPON4': '0.050', 'ARMOR': '0.057', 'weapon4': '0.072', 'weapon5': '0.176', 'WEAPON5': '0.200', 'AMMO3': '0.233', 'HITCOUNT': '0.240', 'WEAPON3': '1.200', 'weapon2': '1.348', 'weapon3': '1.538', 'DAMAGECOUNT': '1.665', 'FRAGCOUNT': '2.500'} [2023-07-24 01:22:14,628][00294] Fps is (10 sec: 1229.2, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3760128. Throughput: 0: 341.9. Samples: 940620. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:22:14,636][00294] Avg episode reward: [(0, '-4.083')] [2023-07-24 01:22:17,879][14529] DAMAGECOUNT value on done: 1417.0 [2023-07-24 01:22:17,883][14529] Sum rewards: -0.728, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.774', 'AMMO2': '0.018', 'WEAPON1': '0.020', 'weapon7': '0.052', 'ARMOR': '0.080', 'AMMO4': '0.091', 'AMMO3': '0.116', 'AMMO6': '0.160', 'AMMO7': '0.160', 'HITCOUNT': '0.170', 'WEAPON4': '0.200', 'WEAPON7': '0.200', 'weapon4': '0.570', 'DAMAGECOUNT': '0.687', 'WEAPON3': '0.700', 'weapon3': '1.014', 'weapon2': '1.308', 'FRAGCOUNT': '2.000'} [2023-07-24 01:22:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 3764224. Throughput: 0: 322.5. Samples: 942312. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:22:19,637][00294] Avg episode reward: [(0, '-4.044')] [2023-07-24 01:22:20,431][14527] Updated weights for policy 0, policy_version 920 (0.0053) [2023-07-24 01:22:20,723][14530] DAMAGECOUNT value on done: 1400.0 [2023-07-24 01:22:20,725][14530] Sum rewards: 0.359, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.154', 'weapon7': '0.006', 'AMMO5': '0.013', 'AMMO2': '0.017', 'WEAPON1': '0.040', 'AMMO4': '0.083', 'AMMO3': '0.107', 'weapon5': '0.176', 'HITCOUNT': '0.190', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON5': '0.300', 'WEAPON4': '0.300', 'weapon4': '0.352', 'WEAPON3': '0.700', 'weapon2': '0.750', 'DAMAGECOUNT': '0.789', 'weapon3': '1.590', 'FRAGCOUNT': '3.000'} [2023-07-24 01:22:21,102][14525] DAMAGECOUNT value on done: 920.0 [2023-07-24 01:22:21,108][14525] Sum rewards: 3.770, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.722', 'AMMO2': '0.003', 'AMMO5': '0.012', 'AMMO4': '0.015', 'ARMOR': '0.032', 'AMMO3': '0.130', 'HITCOUNT': '0.170', 'weapon5': '0.172', 'WEAPON5': '0.200', 'WEAPON3': '0.650', 'weapon2': '1.086', 'DAMAGECOUNT': '1.260', 'weapon3': '2.012', 'FRAGCOUNT': '4.000'} [2023-07-24 01:22:22,152][14526] DAMAGECOUNT value on done: 517.0 [2023-07-24 01:22:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3772416. Throughput: 0: 315.1. Samples: 943160. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:22:24,636][00294] Avg episode reward: [(0, '-3.866')] [2023-07-24 01:22:26,120][14530] DAMAGECOUNT value on done: 884.0 [2023-07-24 01:22:26,125][14530] Sum rewards: 1.160, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.640', 'WEAPON1': '0.010', 'AMMO5': '0.015', 'AMMO2': '0.018', 'weapon5': '0.046', 'AMMO3': '0.067', 'ARMOR': '0.072', 'AMMO4': '0.091', 'weapon4': '0.168', 'HITCOUNT': '0.170', 'WEAPON4': '0.250', 'WEAPON5': '0.250', 'WEAPON3': '0.450', 'DAMAGECOUNT': '0.690', 'weapon2': '1.294', 'weapon3': '1.458', 'FRAGCOUNT': '2.000'} [2023-07-24 01:22:26,460][14525] DAMAGECOUNT value on done: 688.0 [2023-07-24 01:22:26,462][14525] Sum rewards: -9.234, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-1.842', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.005', 'AMMO2': '0.024', 'AMMO4': '0.117', 'weapon5': '0.122', 'WEAPON5': '0.150', 'HITCOUNT': '0.170', 'AMMO3': '0.189', 'ARMOR': '0.400', 'DAMAGECOUNT': '0.507', 'weapon3': '1.022', 'WEAPON3': '1.050', 'weapon2': '2.102'} [2023-07-24 01:22:27,212][14526] DAMAGECOUNT value on done: 1183.0 [2023-07-24 01:22:27,214][14526] Sum rewards: -2.275, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.452', 'AMMO5': '0.007', 'AMMO2': '0.028', 'WEAPON4': '0.050', 'WEAPON5': '0.100', 'AMMO3': '0.131', 'AMMO4': '0.141', 'weapon5': '0.144', 'weapon4': '0.234', 'HITCOUNT': '0.270', 'ARMOR': '0.400', 'WEAPON3': '0.750', 'weapon2': '1.322', 'DAMAGECOUNT': '1.353', 'weapon3': '1.496', 'FRAGCOUNT': '1.500'} [2023-07-24 01:22:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 3780608. Throughput: 0: 334.7. Samples: 945732. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:22:29,640][00294] Avg episode reward: [(0, '-3.817')] [2023-07-24 01:22:30,959][14530] DAMAGECOUNT value on done: 715.0 [2023-07-24 01:22:30,960][14530] Sum rewards: -1.151, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-2.071', 'AMMO2': '0.002', 'AMMO4': '0.008', 'AMMO5': '0.010', 'WEAPON5': '0.050', 'weapon5': '0.058', 'AMMO3': '0.116', 'HITCOUNT': '0.130', 'WEAPON4': '0.200', 'weapon4': '0.264', 'ARMOR': '0.484', 'DAMAGECOUNT': '0.630', 'WEAPON3': '0.700', 'weapon3': '1.108', 'weapon2': '1.660', 'FRAGCOUNT': '3.000'} [2023-07-24 01:22:31,220][14525] DAMAGECOUNT value on done: 889.0 [2023-07-24 01:22:31,221][14525] Sum rewards: -3.947, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-0.894', 'AMMO5': '0.010', 'weapon5': '0.014', 'AMMO2': '0.015', 'AMMO4': '0.072', 'AMMO3': '0.116', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'HITCOUNT': '0.280', 'weapon4': '0.316', 'ARMOR': '0.456', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.912', 'weapon2': '1.316', 'weapon3': '1.590', 'FRAGCOUNT': '2.000'} [2023-07-24 01:22:31,838][14526] DAMAGECOUNT value on done: 1327.0 [2023-07-24 01:22:31,841][14526] Sum rewards: -3.577, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.946', 'AMMO5': '0.004', 'AMMO2': '0.005', 'AMMO4': '0.025', 'ARMOR': '0.052', 'weapon5': '0.052', 'weapon7': '0.076', 'HITCOUNT': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.119', 'WEAPON4': '0.150', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON7': '0.200', 'weapon4': '0.388', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.816', 'weapon3': '0.858', 'FRAGCOUNT': '1.000', 'weapon2': '1.654'} [2023-07-24 01:22:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 3784704. Throughput: 0: 348.6. Samples: 948088. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:22:34,631][00294] Avg episode reward: [(0, '-3.733')] [2023-07-24 01:22:37,827][14525] DAMAGECOUNT value on done: 1039.0 [2023-07-24 01:22:37,828][14525] Sum rewards: 2.002, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.360', 'AMMO5': '0.014', 'ARMOR': '0.015', 'AMMO2': '0.018', 'weapon7': '0.076', 'AMMO4': '0.088', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'weapon5': '0.102', 'weapon4': '0.102', 'AMMO3': '0.103', 'WEAPON4': '0.150', 'WEAPON5': '0.200', 'HITCOUNT': '0.310', 'WEAPON3': '0.650', 'weapon2': '0.820', 'DAMAGECOUNT': '1.533', 'weapon3': '2.130', 'FRAGCOUNT': '4.000'} [2023-07-24 01:22:39,133][14526] DAMAGECOUNT value on done: 965.0 [2023-07-24 01:22:39,134][14526] Sum rewards: -6.983, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.070', 'FRAGCOUNT': '-2.000', 'AMMO2': '0.009', 'ARMOR': '0.010', 'AMMO5': '0.015', 'weapon7': '0.022', 'AMMO4': '0.043', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.128', 'weapon4': '0.164', 'WEAPON4': '0.200', 'weapon5': '0.234', 'HITCOUNT': '0.240', 'WEAPON5': '0.350', 'WEAPON3': '0.750', 'DAMAGECOUNT': '1.020', 'weapon3': '1.254', 'weapon2': '1.348'} [2023-07-24 01:22:39,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 3792896. Throughput: 0: 348.7. Samples: 948944. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:22:39,643][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:22:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 3796992. Throughput: 0: 336.2. Samples: 950608. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:22:44,635][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:22:49,630][00294] Fps is (10 sec: 819.3, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 3801088. Throughput: 0: 308.7. Samples: 951940. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:22:49,636][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:22:54,509][14527] Updated weights for policy 0, policy_version 930 (0.0051) [2023-07-24 01:22:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 3809280. Throughput: 0: 303.7. Samples: 952600. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:22:54,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:22:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 3813376. Throughput: 0: 301.7. Samples: 954196. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:22:59,632][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:04,629][00294] Fps is (10 sec: 819.2, 60 sec: 1160.6, 300 sec: 1249.6). Total num frames: 3817472. Throughput: 0: 303.8. Samples: 955984. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:23:04,634][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 3825664. Throughput: 0: 306.0. Samples: 956932. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:23:09,633][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:14,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1160.5, 300 sec: 1263.5). Total num frames: 3829760. Throughput: 0: 287.8. Samples: 958684. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:23:14,639][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 3837952. Throughput: 0: 274.1. Samples: 960424. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:23:19,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1263.5). Total num frames: 3842048. Throughput: 0: 274.4. Samples: 961292. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:23:24,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:27,103][14527] Updated weights for policy 0, policy_version 940 (0.0042) [2023-07-24 01:23:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3854336. Throughput: 0: 295.0. Samples: 963884. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:23:29,632][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3858432. Throughput: 0: 318.8. Samples: 966284. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:23:34,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:39,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 3866624. Throughput: 0: 323.6. Samples: 967164. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:23:39,633][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3870720. Throughput: 0: 325.9. Samples: 968860. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:23:44,633][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:49,630][00294] Fps is (10 sec: 1228.9, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 3878912. Throughput: 0: 324.4. Samples: 970580. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:23:49,632][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 3887104. Throughput: 0: 328.0. Samples: 971692. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:23:54,638][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:57,387][14527] Updated weights for policy 0, policy_version 950 (0.0056) [2023-07-24 01:23:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3895296. Throughput: 0: 347.5. Samples: 974320. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:23:59,633][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:23:59,655][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000951_3895296.pth... [2023-07-24 01:23:59,867][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000875_3584000.pth [2023-07-24 01:24:04,629][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 3899392. Throughput: 0: 354.5. Samples: 976376. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:24:04,634][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 3907584. Throughput: 0: 354.3. Samples: 977236. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:24:09,632][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:14,629][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 3911680. Throughput: 0: 335.2. Samples: 978968. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:24:14,637][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:19,632][00294] Fps is (10 sec: 1228.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 3919872. Throughput: 0: 322.8. Samples: 980812. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:24:19,634][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:24,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 3928064. Throughput: 0: 332.6. Samples: 982132. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:24:24,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:26,279][14527] Updated weights for policy 0, policy_version 960 (0.0040) [2023-07-24 01:24:29,628][00294] Fps is (10 sec: 1639.0, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3936256. Throughput: 0: 353.6. Samples: 984772. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:24:29,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:34,633][00294] Fps is (10 sec: 1228.2, 60 sec: 1365.2, 300 sec: 1291.3). Total num frames: 3940352. Throughput: 0: 354.0. Samples: 986512. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:24:34,637][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 3944448. Throughput: 0: 348.3. Samples: 987364. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:24:39,636][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:44,628][00294] Fps is (10 sec: 1229.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 3952640. Throughput: 0: 328.2. Samples: 989088. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:24:44,634][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 3960832. Throughput: 0: 330.8. Samples: 991264. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:24:49,638][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3969024. Throughput: 0: 341.2. Samples: 992588. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:24:54,638][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:24:56,375][14527] Updated weights for policy 0, policy_version 970 (0.0024) [2023-07-24 01:24:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 3977216. Throughput: 0: 356.5. Samples: 995008. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:24:59,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:04,633][00294] Fps is (10 sec: 1228.2, 60 sec: 1365.2, 300 sec: 1291.3). Total num frames: 3981312. Throughput: 0: 347.3. Samples: 996440. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:25:04,636][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:09,632][00294] Fps is (10 sec: 818.9, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 3985408. Throughput: 0: 332.2. Samples: 997080. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:25:09,642][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:14,628][00294] Fps is (10 sec: 819.6, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 3989504. Throughput: 0: 304.0. Samples: 998452. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:25:14,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:19,630][00294] Fps is (10 sec: 819.4, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 3993600. Throughput: 0: 294.7. Samples: 999772. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:25:19,633][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:24,631][00294] Fps is (10 sec: 1228.4, 60 sec: 1228.7, 300 sec: 1263.5). Total num frames: 4001792. Throughput: 0: 291.4. Samples: 1000476. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:25:24,639][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:29,630][00294] Fps is (10 sec: 1638.3, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 4009984. Throughput: 0: 305.1. Samples: 1002816. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:25:29,638][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:32,686][14527] Updated weights for policy 0, policy_version 980 (0.0040) [2023-07-24 01:25:34,628][00294] Fps is (10 sec: 1638.9, 60 sec: 1297.2, 300 sec: 1291.3). Total num frames: 4018176. Throughput: 0: 307.7. Samples: 1005112. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:25:34,635][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:39,632][00294] Fps is (10 sec: 819.1, 60 sec: 1228.7, 300 sec: 1277.4). Total num frames: 4018176. Throughput: 0: 296.8. Samples: 1005944. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:25:39,635][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4026368. Throughput: 0: 281.9. Samples: 1007692. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:25:44,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:49,628][00294] Fps is (10 sec: 1639.0, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4034560. Throughput: 0: 289.1. Samples: 1009448. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:25:49,632][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4042752. Throughput: 0: 302.1. Samples: 1010672. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:25:54,634][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4050944. Throughput: 0: 329.3. Samples: 1013272. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:25:59,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:25:59,648][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000989_4050944.pth... [2023-07-24 01:25:59,866][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000913_3739648.pth [2023-07-24 01:26:03,175][14527] Updated weights for policy 0, policy_version 990 (0.0031) [2023-07-24 01:26:04,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1228.9, 300 sec: 1291.3). Total num frames: 4055040. Throughput: 0: 343.5. Samples: 1015228. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:26:04,632][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.9, 300 sec: 1291.3). Total num frames: 4059136. Throughput: 0: 346.2. Samples: 1016056. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:26:09,633][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:14,629][00294] Fps is (10 sec: 1228.9, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4067328. Throughput: 0: 332.5. Samples: 1017780. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:26:14,640][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 4075520. Throughput: 0: 324.8. Samples: 1019728. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:26:19,634][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:24,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 4083712. Throughput: 0: 336.2. Samples: 1021072. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:26:24,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 4091904. Throughput: 0: 354.8. Samples: 1023656. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:26:29,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:32,459][14527] Updated weights for policy 0, policy_version 1000 (0.0027) [2023-07-24 01:26:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4096000. Throughput: 0: 353.2. Samples: 1025344. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:26:34,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:39,636][00294] Fps is (10 sec: 1227.9, 60 sec: 1433.5, 300 sec: 1305.1). Total num frames: 4104192. Throughput: 0: 344.5. Samples: 1026176. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:26:39,639][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:44,630][00294] Fps is (10 sec: 819.0, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 4104192. Throughput: 0: 324.3. Samples: 1027868. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:26:44,633][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:47,419][14525] Large shaping reward -2.519 for [('FRAGCOUNT', -1.5, -1.0), ('DEATHCOUNT', -0.75, 1.0), ('HEALTH', -0.27, -90.0), ('AMMO5', -0.0005, -1.0), ('weapon5', 0.002)] [2023-07-24 01:26:49,628][00294] Fps is (10 sec: 1229.7, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4116480. Throughput: 0: 332.8. Samples: 1030204. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:26:49,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:54,628][00294] Fps is (10 sec: 2048.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4124672. Throughput: 0: 343.2. Samples: 1031500. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:26:54,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:26:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4128768. Throughput: 0: 353.1. Samples: 1033668. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:26:59,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:02,532][14527] Updated weights for policy 0, policy_version 1010 (0.0022) [2023-07-24 01:27:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 4136960. Throughput: 0: 347.5. Samples: 1035364. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:27:04,636][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:09,634][00294] Fps is (10 sec: 1637.5, 60 sec: 1433.5, 300 sec: 1305.1). Total num frames: 4145152. Throughput: 0: 337.3. Samples: 1036252. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:27:09,638][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 4149248. Throughput: 0: 318.0. Samples: 1037964. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:27:14,633][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:19,634][00294] Fps is (10 sec: 819.2, 60 sec: 1296.9, 300 sec: 1291.3). Total num frames: 4153344. Throughput: 0: 330.5. Samples: 1040220. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:27:19,636][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4161536. Throughput: 0: 331.0. Samples: 1041068. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:27:24,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:29,629][00294] Fps is (10 sec: 1229.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4165632. Throughput: 0: 324.1. Samples: 1042452. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:27:29,631][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:34,633][00294] Fps is (10 sec: 818.8, 60 sec: 1228.7, 300 sec: 1277.4). Total num frames: 4169728. Throughput: 0: 301.5. Samples: 1043772. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:27:34,635][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:39,632][00294] Fps is (10 sec: 819.0, 60 sec: 1160.6, 300 sec: 1277.4). Total num frames: 4173824. Throughput: 0: 287.2. Samples: 1044424. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:27:39,635][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:40,785][14527] Updated weights for policy 0, policy_version 1020 (0.0070) [2023-07-24 01:27:44,629][00294] Fps is (10 sec: 819.5, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 4177920. Throughput: 0: 272.9. Samples: 1045948. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:27:44,636][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:49,628][00294] Fps is (10 sec: 1229.2, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 4186112. Throughput: 0: 284.2. Samples: 1048152. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:27:49,640][00294] Avg episode reward: [(0, '-3.620')] [2023-07-24 01:27:54,011][14524] DAMAGECOUNT value on done: 1001.0 [2023-07-24 01:27:54,577][14528] DAMAGECOUNT value on done: 939.0 [2023-07-24 01:27:54,578][14528] Sum rewards: -0.402, reward structure: {'DEATHCOUNT': '-9.000', 'AMMO5': '0.003', 'HEALTH': '0.014', 'AMMO2': '0.016', 'ARMOR': '0.020', 'weapon5': '0.032', 'WEAPON5': '0.050', 'AMMO4': '0.077', 'AMMO3': '0.110', 'WEAPON4': '0.150', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'HITCOUNT': '0.210', 'weapon4': '0.316', 'DAMAGECOUNT': '0.531', 'WEAPON3': '0.650', 'weapon2': '1.060', 'weapon3': '1.760', 'FRAGCOUNT': '3.000'} [2023-07-24 01:27:54,628][00294] Fps is (10 sec: 2048.1, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4198400. Throughput: 0: 293.9. Samples: 1049476. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:27:54,636][00294] Avg episode reward: [(0, '-3.676')] [2023-07-24 01:27:57,766][14532] DAMAGECOUNT value on done: 1254.0 [2023-07-24 01:27:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4202496. Throughput: 0: 306.8. Samples: 1051768. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:27:59,631][00294] Avg episode reward: [(0, '-3.623')] [2023-07-24 01:27:59,652][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001026_4202496.pth... [2023-07-24 01:27:59,912][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000951_3895296.pth [2023-07-24 01:28:01,351][14524] DAMAGECOUNT value on done: 1443.0 [2023-07-24 01:28:01,354][14524] Sum rewards: -3.388, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.859', 'AMMO5': '0.010', 'WEAPON1': '0.010', 'AMMO2': '0.010', 'ARMOR': '0.040', 'AMMO4': '0.052', 'weapon5': '0.052', 'weapon4': '0.162', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'AMMO3': '0.203', 'HITCOUNT': '0.360', 'WEAPON3': '1.050', 'weapon2': '1.150', 'DAMAGECOUNT': '1.470', 'weapon3': '1.752', 'FRAGCOUNT': '3.000'} [2023-07-24 01:28:02,006][14528] DAMAGECOUNT value on done: 1114.0 [2023-07-24 01:28:02,010][14528] Sum rewards: -1.273, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.780', 'AMMO5': '0.005', 'AMMO2': '0.007', 'weapon5': '0.016', 'WEAPON1': '0.030', 'AMMO4': '0.033', 'WEAPON5': '0.100', 'AMMO3': '0.109', 'WEAPON4': '0.150', 'HITCOUNT': '0.170', 'weapon4': '0.214', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.705', 'weapon3': '1.230', 'weapon2': '1.538', 'FRAGCOUNT': '3.000'} [2023-07-24 01:28:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 4206592. Throughput: 0: 292.9. Samples: 1053400. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:28:04,632][00294] Avg episode reward: [(0, '-3.606')] [2023-07-24 01:28:05,902][14532] DAMAGECOUNT value on done: 1489.0 [2023-07-24 01:28:05,921][14532] Sum rewards: -5.719, reward structure: {'DEATHCOUNT': '-10.500', 'FRAGCOUNT': '-1.000', 'HEALTH': '-0.985', 'AMMO5': '0.015', 'AMMO2': '0.016', 'WEAPON1': '0.020', 'AMMO4': '0.082', 'AMMO3': '0.117', 'WEAPON4': '0.150', 'weapon4': '0.226', 'HITCOUNT': '0.230', 'WEAPON5': '0.250', 'weapon5': '0.250', 'ARMOR': '0.400', 'WEAPON3': '0.750', 'weapon2': '1.206', 'DAMAGECOUNT': '1.314', 'weapon3': '1.740'} [2023-07-24 01:28:08,020][14531] DAMAGECOUNT value on done: 1480.0 [2023-07-24 01:28:08,022][14531] Sum rewards: 2.297, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.250', 'AMMO5': '0.005', 'AMMO2': '0.007', 'WEAPON1': '0.030', 'AMMO4': '0.033', 'WEAPON4': '0.100', 'AMMO3': '0.139', 'weapon5': '0.142', 'WEAPON5': '0.150', 'weapon4': '0.202', 'HITCOUNT': '0.310', 'WEAPON3': '0.700', 'weapon3': '1.156', 'DAMAGECOUNT': '1.743', 'weapon2': '1.830', 'FRAGCOUNT': '6.000'} [2023-07-24 01:28:08,127][14524] DAMAGECOUNT value on done: 984.0 [2023-07-24 01:28:08,749][14528] DAMAGECOUNT value on done: 941.0 [2023-07-24 01:28:08,770][14528] Sum rewards: -4.408, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.889', 'AMMO5': '0.011', 'AMMO2': '0.023', 'ARMOR': '0.032', 'AMMO4': '0.116', 'weapon5': '0.134', 'HITCOUNT': '0.190', 'AMMO3': '0.212', 'weapon4': '0.288', 'WEAPON5': '0.300', 'WEAPON4': '0.350', 'weapon2': '0.934', 'DAMAGECOUNT': '0.981', 'WEAPON3': '1.150', 'weapon3': '1.760', 'FRAGCOUNT': '4.000'} [2023-07-24 01:28:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.6, 300 sec: 1305.2). Total num frames: 4214784. Throughput: 0: 292.1. Samples: 1054212. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:28:09,632][00294] Avg episode reward: [(0, '-3.549')] [2023-07-24 01:28:12,664][14527] Updated weights for policy 0, policy_version 1030 (0.0022) [2023-07-24 01:28:12,936][14532] DAMAGECOUNT value on done: 765.0 [2023-07-24 01:28:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 4218880. Throughput: 0: 299.3. Samples: 1055920. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:28:14,631][00294] Avg episode reward: [(0, '-3.592')] [2023-07-24 01:28:14,771][14531] DAMAGECOUNT value on done: 955.0 [2023-07-24 01:28:14,776][14531] Sum rewards: -6.506, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.452', 'FRAGCOUNT': '-0.500', 'WEAPON1': '0.010', 'AMMO5': '0.016', 'ARMOR': '0.040', 'AMMO2': '0.041', 'HITCOUNT': '0.100', 'AMMO3': '0.128', 'weapon5': '0.156', 'AMMO4': '0.206', 'DAMAGECOUNT': '0.282', 'WEAPON5': '0.300', 'weapon4': '0.322', 'WEAPON4': '0.400', 'WEAPON3': '0.750', 'weapon3': '1.214', 'weapon2': '1.230'} [2023-07-24 01:28:14,814][14524] DAMAGECOUNT value on done: 1010.0 [2023-07-24 01:28:14,835][14524] Sum rewards: -7.042, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.798', 'FRAGCOUNT': '-0.500', 'AMMO4': '-0.030', 'AMMO2': '-0.006', 'AMMO5': '0.003', 'WEAPON5': '0.050', 'weapon5': '0.072', 'weapon7': '0.084', 'AMMO3': '0.094', 'AMMO6': '0.120', 'AMMO7': '0.120', 'HITCOUNT': '0.160', 'WEAPON7': '0.200', 'DAMAGECOUNT': '0.543', 'WEAPON3': '0.600', 'weapon3': '1.088', 'weapon2': '1.908'} [2023-07-24 01:28:15,159][14528] DAMAGECOUNT value on done: 583.0 [2023-07-24 01:28:15,163][14528] Sum rewards: -4.156, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.570', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.010', 'AMMO2': '0.014', 'ARMOR': '0.016', 'HITCOUNT': '0.030', 'AMMO4': '0.067', 'weapon5': '0.084', 'AMMO3': '0.099', 'DAMAGECOUNT': '0.120', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'weapon4': '0.342', 'WEAPON3': '0.600', 'weapon3': '1.286', 'weapon2': '1.396'} [2023-07-24 01:28:17,655][14532] DAMAGECOUNT value on done: 880.0 [2023-07-24 01:28:17,657][14532] Sum rewards: -4.058, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.925', 'AMMO4': '-0.009', 'AMMO2': '-0.002', 'weapon5': '0.002', 'AMMO5': '0.013', 'weapon7': '0.014', 'WEAPON4': '0.100', 'AMMO3': '0.138', 'WEAPON5': '0.150', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'weapon4': '0.204', 'HITCOUNT': '0.350', 'WEAPON3': '0.900', 'FRAGCOUNT': '1.000', 'DAMAGECOUNT': '1.125', 'weapon3': '1.478', 'weapon2': '1.554'} [2023-07-24 01:28:19,212][14531] DAMAGECOUNT value on done: 1154.0 [2023-07-24 01:28:19,276][14524] DAMAGECOUNT value on done: 1178.0 [2023-07-24 01:28:19,278][14524] Sum rewards: -4.428, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.736', 'ARMOR': '0.004', 'AMMO5': '0.014', 'AMMO2': '0.031', 'weapon5': '0.062', 'HITCOUNT': '0.110', 'AMMO3': '0.141', 'AMMO4': '0.153', 'WEAPON5': '0.200', 'weapon4': '0.246', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.477', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon3': '1.160', 'weapon2': '1.460'} [2023-07-24 01:28:19,561][14528] DAMAGECOUNT value on done: 1097.0 [2023-07-24 01:28:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.2, 300 sec: 1319.1). Total num frames: 4231168. Throughput: 0: 325.2. Samples: 1058404. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:28:19,636][00294] Avg episode reward: [(0, '-3.678')] [2023-07-24 01:28:22,345][14532] DAMAGECOUNT value on done: 1162.0 [2023-07-24 01:28:22,348][14532] Sum rewards: -6.997, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.012', 'AMMO5': '0.005', 'weapon5': '0.014', 'AMMO2': '0.027', 'ARMOR': '0.084', 'WEAPON5': '0.100', 'AMMO4': '0.136', 'weapon4': '0.164', 'AMMO3': '0.199', 'HITCOUNT': '0.210', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.789', 'WEAPON3': '0.950', 'FRAGCOUNT': '1.000', 'weapon3': '1.176', 'weapon2': '1.910'} [2023-07-24 01:28:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4235264. Throughput: 0: 339.6. Samples: 1059704. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:28:24,631][00294] Avg episode reward: [(0, '-3.662')] [2023-07-24 01:28:24,785][14531] DAMAGECOUNT value on done: 1057.0 [2023-07-24 01:28:24,859][14524] DAMAGECOUNT value on done: 1561.0 [2023-07-24 01:28:24,859][14524] Sum rewards: -0.514, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.189', 'AMMO5': '0.006', 'AMMO2': '0.025', 'weapon5': '0.060', 'WEAPON5': '0.100', 'AMMO4': '0.126', 'AMMO3': '0.130', 'HITCOUNT': '0.190', 'weapon4': '0.194', 'WEAPON4': '0.200', 'ARMOR': '0.475', 'WEAPON3': '0.750', 'DAMAGECOUNT': '0.810', 'weapon2': '1.022', 'weapon3': '1.836', 'FRAGCOUNT': '2.000'} [2023-07-24 01:28:25,417][14528] DAMAGECOUNT value on done: 841.0 [2023-07-24 01:28:25,419][14528] Sum rewards: -5.615, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-0.526', 'AMMO5': '0.007', 'weapon5': '0.022', 'AMMO2': '0.050', 'ARMOR': '0.052', 'AMMO3': '0.140', 'HITCOUNT': '0.140', 'WEAPON5': '0.150', 'AMMO4': '0.249', 'WEAPON4': '0.450', 'DAMAGECOUNT': '0.453', 'FRAGCOUNT': '0.500', 'weapon4': '0.500', 'WEAPON3': '0.750', 'weapon2': '1.336', 'weapon3': '1.362'} [2023-07-24 01:28:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 4243456. Throughput: 0: 349.6. Samples: 1061680. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:28:29,636][00294] Avg episode reward: [(0, '-3.684')] [2023-07-24 01:28:30,103][14532] DAMAGECOUNT value on done: 1074.0 [2023-07-24 01:28:30,104][14532] Sum rewards: -1.419, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.786', 'ARMOR': '0.004', 'AMMO5': '0.005', 'AMMO2': '0.010', 'weapon5': '0.012', 'AMMO4': '0.049', 'WEAPON5': '0.100', 'AMMO3': '0.121', 'WEAPON4': '0.150', 'weapon4': '0.362', 'HITCOUNT': '0.380', 'WEAPON3': '0.750', 'DAMAGECOUNT': '1.218', 'weapon2': '1.232', 'weapon3': '1.474', 'FRAGCOUNT': '2.000'} [2023-07-24 01:28:33,332][14531] DAMAGECOUNT value on done: 708.0 [2023-07-24 01:28:33,373][14524] DAMAGECOUNT value on done: 814.0 [2023-07-24 01:28:34,004][14528] DAMAGECOUNT value on done: 1227.0 [2023-07-24 01:28:34,005][14528] Sum rewards: -1.953, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.002', 'AMMO5': '0.010', 'AMMO2': '0.035', 'weapon7': '0.044', 'ARMOR': '0.092', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'weapon5': '0.130', 'WEAPON5': '0.150', 'AMMO4': '0.176', 'AMMO3': '0.183', 'HITCOUNT': '0.200', 'WEAPON4': '0.300', 'weapon4': '0.526', 'weapon2': '0.674', 'WEAPON3': '0.950', 'DAMAGECOUNT': '1.263', 'weapon3': '1.766', 'FRAGCOUNT': '2.000'} [2023-07-24 01:28:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.2, 300 sec: 1291.3). Total num frames: 4247552. Throughput: 0: 337.7. Samples: 1063348. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:28:34,633][00294] Avg episode reward: [(0, '-3.605')] [2023-07-24 01:28:35,760][14529] DAMAGECOUNT value on done: 1294.0 [2023-07-24 01:28:35,777][14529] Sum rewards: 1.168, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.262', 'ARMOR': '0.008', 'AMMO5': '0.010', 'WEAPON1': '0.010', 'AMMO2': '0.017', 'AMMO4': '0.085', 'weapon4': '0.090', 'weapon5': '0.128', 'WEAPON4': '0.150', 'AMMO3': '0.153', 'WEAPON5': '0.200', 'HITCOUNT': '0.300', 'WEAPON3': '0.850', 'weapon2': '1.136', 'DAMAGECOUNT': '1.140', 'weapon3': '1.902', 'FRAGCOUNT': '5.000'} [2023-07-24 01:28:39,450][14532] DAMAGECOUNT value on done: 986.0 [2023-07-24 01:28:39,451][14532] Sum rewards: -0.717, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.262', 'WEAPON1': '0.010', 'AMMO5': '0.017', 'AMMO2': '0.021', 'AMMO3': '0.057', 'AMMO4': '0.104', 'HITCOUNT': '0.110', 'WEAPON4': '0.150', 'weapon4': '0.228', 'WEAPON5': '0.250', 'weapon5': '0.252', 'WEAPON3': '0.300', 'ARMOR': '0.440', 'DAMAGECOUNT': '0.720', 'weapon3': '1.084', 'weapon2': '1.302', 'FRAGCOUNT': '2.000'} [2023-07-24 01:28:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4251648. Throughput: 0: 326.4. Samples: 1064164. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:28:39,637][00294] Avg episode reward: [(0, '-3.589')] [2023-07-24 01:28:41,795][14529] DAMAGECOUNT value on done: 794.0 [2023-07-24 01:28:42,061][14531] DAMAGECOUNT value on done: 1837.0 [2023-07-24 01:28:42,065][14531] Sum rewards: -4.629, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.050', 'AMMO5': '0.007', 'WEAPON1': '0.020', 'AMMO2': '0.023', 'weapon7': '0.030', 'weapon5': '0.040', 'weapon4': '0.062', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO4': '0.112', 'WEAPON5': '0.150', 'AMMO3': '0.161', 'WEAPON4': '0.200', 'HITCOUNT': '0.240', 'WEAPON3': '0.900', 'weapon2': '1.372', 'DAMAGECOUNT': '1.380', 'weapon3': '1.674', 'FRAGCOUNT': '2.000'} [2023-07-24 01:28:42,135][14524] DAMAGECOUNT value on done: 856.0 [2023-07-24 01:28:42,139][14524] Sum rewards: -3.742, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.800', 'AMMO5': '0.017', 'ARMOR': '0.020', 'AMMO2': '0.040', 'weapon5': '0.096', 'AMMO3': '0.133', 'AMMO4': '0.199', 'HITCOUNT': '0.240', 'WEAPON4': '0.250', 'weapon4': '0.250', 'WEAPON5': '0.350', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.650', 'DAMAGECOUNT': '1.101', 'weapon2': '1.460', 'weapon3': '1.502'} [2023-07-24 01:28:42,333][14528] DAMAGECOUNT value on done: 1412.0 [2023-07-24 01:28:42,339][14528] Sum rewards: 0.049, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.931', 'AMMO4': '-0.005', 'AMMO2': '-0.001', 'AMMO5': '0.005', 'ARMOR': '0.036', 'weapon5': '0.074', 'AMMO3': '0.101', 'WEAPON5': '0.150', 'HITCOUNT': '0.260', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.996', 'weapon3': '1.266', 'weapon2': '1.548', 'FRAGCOUNT': '2.000'} [2023-07-24 01:28:42,402][14527] Updated weights for policy 0, policy_version 1040 (0.0048) [2023-07-24 01:28:42,996][14529] Large shaping reward -2.549 for [('FRAGCOUNT', -1.5, -1.0), ('DEATHCOUNT', -0.75, 1.0), ('HEALTH', -0.3, -100.0), ('AMMO5', -0.0005, -1.0), ('weapon5', 0.002)] [2023-07-24 01:28:44,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4259840. Throughput: 0: 317.1. Samples: 1066036. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:28:44,635][00294] Avg episode reward: [(0, '-3.616')] [2023-07-24 01:28:45,144][14532] DAMAGECOUNT value on done: 1362.0 [2023-07-24 01:28:45,148][14532] Sum rewards: -4.577, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.834', 'AMMO5': '0.005', 'weapon5': '0.016', 'AMMO2': '0.016', 'ARMOR': '0.040', 'AMMO4': '0.082', 'WEAPON5': '0.100', 'AMMO3': '0.122', 'HITCOUNT': '0.140', 'WEAPON4': '0.150', 'weapon4': '0.166', 'DAMAGECOUNT': '0.570', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon2': '1.352', 'weapon3': '1.498'} [2023-07-24 01:28:46,670][14529] DAMAGECOUNT value on done: 720.0 [2023-07-24 01:28:46,670][14529] Sum rewards: -5.819, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.448', 'ARMOR': '0.008', 'AMMO5': '0.009', 'WEAPON1': '0.010', 'weapon7': '0.022', 'AMMO2': '0.037', 'HITCOUNT': '0.050', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.103', 'WEAPON5': '0.150', 'AMMO4': '0.187', 'weapon5': '0.264', 'WEAPON4': '0.300', 'WEAPON3': '0.400', 'DAMAGECOUNT': '0.402', 'weapon4': '0.466', 'weapon3': '1.044', 'weapon2': '1.376'} [2023-07-24 01:28:46,815][14531] DAMAGECOUNT value on done: 814.0 [2023-07-24 01:28:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4268032. Throughput: 0: 339.9. Samples: 1068696. Policy #0 lag: (min: 0.0, avg: 1.1, max: 4.0) [2023-07-24 01:28:49,631][00294] Avg episode reward: [(0, '-3.586')] [2023-07-24 01:28:49,701][14530] DAMAGECOUNT value on done: 1339.0 [2023-07-24 01:28:49,702][14530] Sum rewards: -2.894, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.144', 'AMMO2': '0.008', 'AMMO5': '0.009', 'WEAPON1': '0.010', 'weapon4': '0.012', 'ARMOR': '0.036', 'AMMO4': '0.040', 'WEAPON4': '0.100', 'weapon5': '0.130', 'AMMO3': '0.138', 'WEAPON5': '0.200', 'HITCOUNT': '0.250', 'WEAPON3': '0.800', 'weapon2': '1.300', 'DAMAGECOUNT': '1.368', 'weapon3': '1.598', 'FRAGCOUNT': '2.000'} [2023-07-24 01:28:51,192][14529] DAMAGECOUNT value on done: 1160.0 [2023-07-24 01:28:51,192][14529] Sum rewards: -5.201, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.737', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.005', 'weapon5': '0.014', 'AMMO2': '0.021', 'WEAPON5': '0.050', 'AMMO4': '0.102', 'ARMOR': '0.104', 'AMMO3': '0.140', 'HITCOUNT': '0.200', 'weapon4': '0.230', 'WEAPON4': '0.250', 'WEAPON3': '0.750', 'DAMAGECOUNT': '0.774', 'weapon2': '1.088', 'weapon3': '1.558'} [2023-07-24 01:28:52,769][14531] DAMAGECOUNT value on done: 984.0 [2023-07-24 01:28:52,774][14531] Sum rewards: -3.453, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.750', 'weapon5': '0.006', 'AMMO5': '0.007', 'AMMO2': '0.022', 'ARMOR': '0.024', 'AMMO3': '0.094', 'AMMO4': '0.111', 'WEAPON5': '0.150', 'weapon4': '0.150', 'HITCOUNT': '0.190', 'WEAPON4': '0.300', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.810', 'FRAGCOUNT': '1.000', 'weapon3': '1.344', 'weapon2': '1.788'} [2023-07-24 01:28:54,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4276224. Throughput: 0: 348.7. Samples: 1069904. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:28:54,634][00294] Avg episode reward: [(0, '-3.563')] [2023-07-24 01:28:54,757][14525] DAMAGECOUNT value on done: 991.0 [2023-07-24 01:28:54,758][14525] Sum rewards: -3.546, reward structure: {'DEATHCOUNT': '-6.000', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.606', 'AMMO5': '0.017', 'AMMO2': '0.017', 'ARMOR': '0.020', 'HITCOUNT': '0.040', 'AMMO3': '0.075', 'weapon7': '0.082', 'AMMO4': '0.087', 'DAMAGECOUNT': '0.120', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'weapon5': '0.200', 'weapon4': '0.298', 'WEAPON3': '0.450', 'weapon2': '1.030', 'weapon3': '1.224'} [2023-07-24 01:28:55,694][14530] DAMAGECOUNT value on done: 1656.0 [2023-07-24 01:28:55,707][14530] Sum rewards: -2.253, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.929', 'AMMO5': '0.006', 'AMMO2': '0.008', 'WEAPON1': '0.010', 'weapon6': '0.038', 'ARMOR': '0.040', 'AMMO4': '0.042', 'WEAPON4': '0.050', 'weapon4': '0.054', 'AMMO3': '0.116', 'WEAPON5': '0.150', 'AMMO6': '0.197', 'AMMO7': '0.197', 'WEAPON6': '0.200', 'weapon5': '0.200', 'HITCOUNT': '0.250', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.750', 'DAMAGECOUNT': '1.146', 'weapon2': '1.248', 'weapon3': '1.724'} [2023-07-24 01:28:56,862][14526] DAMAGECOUNT value on done: 949.0 [2023-07-24 01:28:56,869][14526] Sum rewards: -5.557, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.362', 'AMMO5': '0.009', 'AMMO2': '0.016', 'weapon5': '0.062', 'AMMO4': '0.081', 'AMMO3': '0.153', 'WEAPON4': '0.200', 'WEAPON5': '0.200', 'HITCOUNT': '0.240', 'weapon4': '0.436', 'DAMAGECOUNT': '0.897', 'WEAPON3': '0.900', 'weapon2': '1.260', 'weapon3': '1.350', 'FRAGCOUNT': '2.000'} [2023-07-24 01:28:57,967][14529] DAMAGECOUNT value on done: 946.0 [2023-07-24 01:28:57,967][14529] Sum rewards: -6.360, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.886', 'AMMO5': '0.006', 'AMMO2': '0.014', 'WEAPON1': '0.020', 'ARMOR': '0.040', 'AMMO4': '0.070', 'weapon5': '0.074', 'WEAPON4': '0.100', 'HITCOUNT': '0.120', 'AMMO3': '0.133', 'WEAPON5': '0.150', 'weapon4': '0.164', 'DAMAGECOUNT': '0.384', 'WEAPON3': '0.650', 'weapon2': '1.544', 'weapon3': '1.556'} [2023-07-24 01:28:59,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 4280320. Throughput: 0: 347.9. Samples: 1071576. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:28:59,637][00294] Avg episode reward: [(0, '-3.605')] [2023-07-24 01:29:02,851][14525] DAMAGECOUNT value on done: 746.0 [2023-07-24 01:29:02,857][14525] Sum rewards: -7.312, reward structure: {'DEATHCOUNT': '-9.000', 'FRAGCOUNT': '-2.000', 'HEALTH': '-1.821', 'AMMO2': '0.011', 'ARMOR': '0.032', 'AMMO5': '0.033', 'AMMO4': '0.054', 'HITCOUNT': '0.090', 'weapon7': '0.094', 'WEAPON4': '0.100', 'AMMO3': '0.132', 'weapon5': '0.146', 'weapon4': '0.158', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON7': '0.200', 'WEAPON5': '0.450', 'DAMAGECOUNT': '0.633', 'weapon2': '0.770', 'WEAPON3': '0.800', 'weapon3': '1.486'} [2023-07-24 01:29:03,894][14530] DAMAGECOUNT value on done: 819.0 [2023-07-24 01:29:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4288512. Throughput: 0: 330.6. Samples: 1073280. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:29:04,633][00294] Avg episode reward: [(0, '-3.610')] [2023-07-24 01:29:05,170][14526] DAMAGECOUNT value on done: 947.0 [2023-07-24 01:29:06,300][14529] DAMAGECOUNT value on done: 1189.0 [2023-07-24 01:29:06,303][14529] Sum rewards: -7.256, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-3.298', 'AMMO5': '0.010', 'AMMO2': '0.020', 'WEAPON1': '0.030', 'ARMOR': '0.036', 'weapon5': '0.098', 'AMMO4': '0.102', 'AMMO3': '0.127', 'weapon4': '0.128', 'HITCOUNT': '0.130', 'WEAPON4': '0.200', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.345', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.850', 'weapon3': '1.414', 'weapon2': '1.552'} [2023-07-24 01:29:09,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4292608. Throughput: 0: 320.1. Samples: 1074108. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:29:09,637][00294] Avg episode reward: [(0, '-3.645')] [2023-07-24 01:29:09,797][14525] DAMAGECOUNT value on done: 837.0 [2023-07-24 01:29:09,799][14525] Sum rewards: -0.077, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.391', 'AMMO5': '0.009', 'AMMO2': '0.016', 'weapon5': '0.052', 'AMMO3': '0.073', 'AMMO4': '0.079', 'WEAPON5': '0.100', 'HITCOUNT': '0.150', 'DAMAGECOUNT': '0.375', 'WEAPON3': '0.450', 'FRAGCOUNT': '1.000', 'weapon3': '1.328', 'weapon2': '1.932'} [2023-07-24 01:29:10,477][14530] DAMAGECOUNT value on done: 984.0 [2023-07-24 01:29:10,479][14530] Sum rewards: -2.170, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.359', 'ARMOR': '0.008', 'WEAPON1': '0.010', 'AMMO2': '0.011', 'AMMO5': '0.011', 'AMMO4': '0.055', 'HITCOUNT': '0.100', 'weapon5': '0.112', 'AMMO3': '0.121', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'weapon4': '0.546', 'DAMAGECOUNT': '0.630', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon3': '1.076', 'weapon2': '1.408'} [2023-07-24 01:29:11,170][14526] DAMAGECOUNT value on done: 1600.0 [2023-07-24 01:29:11,173][14526] Sum rewards: 1.402, reward structure: {'DEATHCOUNT': '-2.250', 'AMMO5': '0.005', 'AMMO2': '0.008', 'weapon5': '0.012', 'ARMOR': '0.032', 'AMMO3': '0.033', 'AMMO4': '0.038', 'HITCOUNT': '0.100', 'WEAPON5': '0.100', 'HEALTH': '0.136', 'WEAPON3': '0.200', 'DAMAGECOUNT': '0.360', 'weapon2': '0.742', 'weapon3': '0.886', 'FRAGCOUNT': '1.000'} [2023-07-24 01:29:11,940][14529] DAMAGECOUNT value on done: 1677.0 [2023-07-24 01:29:11,941][14529] Sum rewards: -2.926, reward structure: {'DEATHCOUNT': '-12.000', 'weapon5': '0.004', 'AMMO5': '0.010', 'AMMO2': '0.013', 'WEAPON4': '0.050', 'AMMO4': '0.062', 'weapon4': '0.096', 'WEAPON5': '0.100', 'AMMO3': '0.118', 'HITCOUNT': '0.320', 'WEAPON3': '0.550', 'DAMAGECOUNT': '1.155', 'HEALTH': '1.228', 'weapon2': '1.522', 'weapon3': '1.846', 'FRAGCOUNT': '2.000'} [2023-07-24 01:29:12,227][14527] Updated weights for policy 0, policy_version 1050 (0.0020) [2023-07-24 01:29:14,296][14525] DAMAGECOUNT value on done: 831.0 [2023-07-24 01:29:14,297][14525] Sum rewards: -3.589, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.472', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.010', 'AMMO5': '0.013', 'weapon4': '0.014', 'WEAPON1': '0.020', 'AMMO4': '0.051', 'HITCOUNT': '0.080', 'WEAPON4': '0.100', 'AMMO3': '0.129', 'weapon5': '0.196', 'DAMAGECOUNT': '0.282', 'WEAPON5': '0.300', 'ARMOR': '0.424', 'WEAPON3': '0.600', 'weapon2': '1.184', 'weapon3': '1.730'} [2023-07-24 01:29:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4300800. Throughput: 0: 326.2. Samples: 1076360. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:29:14,640][00294] Avg episode reward: [(0, '-3.497')] [2023-07-24 01:29:15,023][14530] DAMAGECOUNT value on done: 887.0 [2023-07-24 01:29:15,024][14530] Sum rewards: -8.348, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.712', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.010', 'AMMO2': '0.018', 'ARMOR': '0.056', 'AMMO4': '0.090', 'WEAPON5': '0.100', 'AMMO3': '0.118', 'weapon5': '0.128', 'HITCOUNT': '0.200', 'WEAPON4': '0.250', 'weapon4': '0.472', 'DAMAGECOUNT': '0.555', 'WEAPON3': '0.650', 'weapon3': '1.056', 'weapon2': '1.410'} [2023-07-24 01:29:15,725][14526] DAMAGECOUNT value on done: 1426.0 [2023-07-24 01:29:15,729][14526] Sum rewards: -9.688, reward structure: {'DEATHCOUNT': '-11.250', 'FRAGCOUNT': '-2.000', 'HEALTH': '-1.972', 'AMMO5': '0.005', 'WEAPON1': '0.010', 'ARMOR': '0.016', 'AMMO2': '0.033', 'AMMO3': '0.119', 'HITCOUNT': '0.140', 'WEAPON5': '0.150', 'AMMO4': '0.167', 'weapon5': '0.200', 'WEAPON4': '0.400', 'weapon4': '0.418', 'DAMAGECOUNT': '0.648', 'WEAPON3': '0.750', 'weapon2': '1.094', 'weapon3': '1.384'} [2023-07-24 01:29:16,238][14529] DAMAGECOUNT value on done: 1607.0 [2023-07-24 01:29:16,243][14529] Sum rewards: -2.559, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.950', 'AMMO5': '0.017', 'WEAPON1': '0.030', 'AMMO2': '0.031', 'AMMO3': '0.102', 'weapon5': '0.154', 'AMMO4': '0.156', 'HITCOUNT': '0.190', 'WEAPON4': '0.200', 'weapon4': '0.266', 'WEAPON5': '0.400', 'DAMAGECOUNT': '0.570', 'WEAPON3': '0.650', 'weapon2': '1.100', 'weapon3': '1.274', 'FRAGCOUNT': '1.500'} [2023-07-24 01:29:18,613][14525] DAMAGECOUNT value on done: 995.0 [2023-07-24 01:29:18,618][14525] Sum rewards: -6.084, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.700', 'AMMO2': '0.006', 'AMMO5': '0.019', 'AMMO4': '0.028', 'ARMOR': '0.040', 'HITCOUNT': '0.090', 'weapon5': '0.100', 'WEAPON4': '0.150', 'AMMO3': '0.156', 'DAMAGECOUNT': '0.225', 'weapon4': '0.258', 'WEAPON5': '0.300', 'WEAPON3': '0.700', 'FRAGCOUNT': '1.000', 'weapon3': '1.368', 'weapon2': '1.676'} [2023-07-24 01:29:19,595][14530] DAMAGECOUNT value on done: 1576.0 [2023-07-24 01:29:19,596][14530] Sum rewards: -0.258, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.826', 'AMMO2': '0.002', 'AMMO4': '0.008', 'AMMO5': '0.012', 'ARMOR': '0.040', 'AMMO3': '0.058', 'WEAPON4': '0.100', 'weapon5': '0.164', 'HITCOUNT': '0.170', 'WEAPON5': '0.250', 'weapon4': '0.278', 'WEAPON3': '0.400', 'DAMAGECOUNT': '0.528', 'FRAGCOUNT': '1.000', 'weapon3': '1.134', 'weapon2': '1.674'} [2023-07-24 01:29:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4308992. Throughput: 0: 349.2. Samples: 1079064. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:29:19,636][00294] Avg episode reward: [(0, '-3.657')] [2023-07-24 01:29:20,671][14526] DAMAGECOUNT value on done: 634.0 [2023-07-24 01:29:20,671][14526] Sum rewards: -9.791, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.700', 'FRAGCOUNT': '-1.500', 'WEAPON1': '0.010', 'AMMO2': '0.011', 'AMMO5': '0.013', 'ARMOR': '0.032', 'AMMO4': '0.055', 'HITCOUNT': '0.090', 'AMMO3': '0.144', 'WEAPON4': '0.200', 'weapon5': '0.240', 'weapon4': '0.258', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.351', 'WEAPON3': '0.750', 'weapon2': '0.878', 'weapon3': '1.526'} [2023-07-24 01:29:24,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4317184. Throughput: 0: 350.7. Samples: 1079948. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:29:24,634][00294] Avg episode reward: [(0, '-3.721')] [2023-07-24 01:29:26,141][14525] DAMAGECOUNT value on done: 837.0 [2023-07-24 01:29:26,143][14525] Sum rewards: -3.781, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.112', 'AMMO5': '0.005', 'WEAPON1': '0.010', 'WEAPON5': '0.050', 'AMMO2': '0.056', 'HITCOUNT': '0.090', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.158', 'AMMO4': '0.279', 'weapon4': '0.442', 'DAMAGECOUNT': '0.447', 'ARMOR': '0.450', 'WEAPON4': '0.550', 'WEAPON3': '0.800', 'weapon3': '1.408', 'weapon2': '1.536', 'FRAGCOUNT': '2.000'} [2023-07-24 01:29:27,048][14530] DAMAGECOUNT value on done: 997.0 [2023-07-24 01:29:27,906][14526] DAMAGECOUNT value on done: 1298.0 [2023-07-24 01:29:27,909][14526] Sum rewards: -1.685, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.928', 'AMMO2': '0.009', 'AMMO5': '0.015', 'weapon5': '0.026', 'AMMO4': '0.047', 'HITCOUNT': '0.100', 'ARMOR': '0.108', 'AMMO3': '0.147', 'WEAPON4': '0.150', 'WEAPON5': '0.200', 'weapon4': '0.336', 'DAMAGECOUNT': '0.345', 'WEAPON3': '0.700', 'weapon3': '1.230', 'weapon2': '1.330', 'FRAGCOUNT': '2.000'} [2023-07-24 01:29:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4325376. Throughput: 0: 348.4. Samples: 1081712. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:29:29,633][00294] Avg episode reward: [(0, '-3.703')] [2023-07-24 01:29:34,437][14525] DAMAGECOUNT value on done: 1154.0 [2023-07-24 01:29:34,438][14525] Sum rewards: -4.874, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.158', 'AMMO5': '0.005', 'AMMO2': '0.014', 'weapon5': '0.034', 'WEAPON5': '0.050', 'ARMOR': '0.060', 'AMMO4': '0.070', 'AMMO3': '0.160', 'weapon4': '0.194', 'HITCOUNT': '0.210', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.795', 'WEAPON3': '0.900', 'FRAGCOUNT': '1.000', 'weapon3': '1.500', 'weapon2': '1.542'} [2023-07-24 01:29:34,628][00294] Fps is (10 sec: 819.3, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4325376. Throughput: 0: 326.4. Samples: 1083384. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:29:34,633][00294] Avg episode reward: [(0, '-3.703')] [2023-07-24 01:29:35,917][14530] DAMAGECOUNT value on done: 950.0 [2023-07-24 01:29:35,918][14530] Sum rewards: -4.809, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.620', 'AMMO2': '0.007', 'weapon5': '0.016', 'weapon7': '0.016', 'AMMO5': '0.030', 'AMMO4': '0.034', 'ARMOR': '0.036', 'WEAPON4': '0.050', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.147', 'HITCOUNT': '0.240', 'weapon4': '0.280', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.705', 'WEAPON3': '0.850', 'weapon3': '1.236', 'weapon2': '1.814', 'FRAGCOUNT': '2.000'} [2023-07-24 01:29:37,162][14526] DAMAGECOUNT value on done: 1404.0 [2023-07-24 01:29:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4333568. Throughput: 0: 314.3. Samples: 1084048. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:29:39,632][00294] Avg episode reward: [(0, '-3.725')] [2023-07-24 01:29:41,956][14525] DAMAGECOUNT value on done: 1219.0 [2023-07-24 01:29:41,958][14525] Sum rewards: -2.727, reward structure: {'DEATHCOUNT': '-6.000', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.135', 'AMMO5': '0.007', 'WEAPON1': '0.010', 'AMMO2': '0.013', 'AMMO4': '0.067', 'AMMO3': '0.106', 'weapon5': '0.134', 'HITCOUNT': '0.140', 'WEAPON4': '0.150', 'WEAPON5': '0.150', 'weapon4': '0.162', 'WEAPON3': '0.500', 'DAMAGECOUNT': '0.540', 'weapon3': '0.994', 'weapon2': '1.934'} [2023-07-24 01:29:43,670][14526] DAMAGECOUNT value on done: 1285.0 [2023-07-24 01:29:43,673][14526] Sum rewards: -3.375, reward structure: {'DEATHCOUNT': '-11.250', 'AMMO5': '0.010', 'WEAPON1': '0.020', 'AMMO2': '0.024', 'ARMOR': '0.036', 'weapon5': '0.054', 'HEALTH': '0.058', 'weapon4': '0.068', 'WEAPON4': '0.100', 'AMMO4': '0.119', 'AMMO3': '0.138', 'WEAPON5': '0.150', 'HITCOUNT': '0.200', 'WEAPON3': '0.800', 'DAMAGECOUNT': '0.960', 'weapon2': '1.184', 'weapon3': '1.954', 'FRAGCOUNT': '2.000'} [2023-07-24 01:29:44,636][00294] Fps is (10 sec: 1227.9, 60 sec: 1296.9, 300 sec: 1277.4). Total num frames: 4337664. Throughput: 0: 313.2. Samples: 1085672. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:29:44,638][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:29:45,375][14527] Updated weights for policy 0, policy_version 1060 (0.0029) [2023-07-24 01:29:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 4345856. Throughput: 0: 313.9. Samples: 1087404. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:29:49,632][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:29:54,628][00294] Fps is (10 sec: 1229.7, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 4349952. Throughput: 0: 310.9. Samples: 1088100. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:29:54,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:29:59,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 4354048. Throughput: 0: 296.4. Samples: 1089696. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:29:59,636][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:29:59,655][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001063_4354048.pth... [2023-07-24 01:29:59,928][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000989_4050944.pth [2023-07-24 01:30:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1263.5). Total num frames: 4358144. Throughput: 0: 273.6. Samples: 1091376. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:30:04,638][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 4366336. Throughput: 0: 272.5. Samples: 1092212. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:09,635][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:14,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4374528. Throughput: 0: 287.0. Samples: 1094628. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:30:14,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:17,257][14527] Updated weights for policy 0, policy_version 1070 (0.0027) [2023-07-24 01:30:19,636][00294] Fps is (10 sec: 1637.2, 60 sec: 1228.6, 300 sec: 1291.3). Total num frames: 4382720. Throughput: 0: 307.1. Samples: 1097208. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:19,644][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:24,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1160.6, 300 sec: 1277.4). Total num frames: 4386816. Throughput: 0: 311.2. Samples: 1098052. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:24,633][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:29,632][00294] Fps is (10 sec: 1229.3, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 4395008. Throughput: 0: 313.3. Samples: 1099768. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:29,636][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4399104. Throughput: 0: 312.3. Samples: 1101456. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:34,634][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:39,628][00294] Fps is (10 sec: 1229.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4407296. Throughput: 0: 318.8. Samples: 1102444. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:39,633][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:44,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1365.5, 300 sec: 1305.2). Total num frames: 4419584. Throughput: 0: 342.5. Samples: 1105108. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:44,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:48,426][14527] Updated weights for policy 0, policy_version 1080 (0.0048) [2023-07-24 01:30:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4423680. Throughput: 0: 354.0. Samples: 1107304. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:49,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:54,633][00294] Fps is (10 sec: 818.8, 60 sec: 1297.0, 300 sec: 1277.4). Total num frames: 4427776. Throughput: 0: 354.5. Samples: 1108164. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:30:54,638][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:30:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4435968. Throughput: 0: 339.3. Samples: 1109896. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:30:59,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:04,628][00294] Fps is (10 sec: 1229.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4440064. Throughput: 0: 320.1. Samples: 1111608. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:31:04,634][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4448256. Throughput: 0: 329.4. Samples: 1112876. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:31:09,640][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4456448. Throughput: 0: 349.4. Samples: 1115492. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:31:14,634][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:17,226][14527] Updated weights for policy 0, policy_version 1090 (0.0071) [2023-07-24 01:31:19,634][00294] Fps is (10 sec: 1637.5, 60 sec: 1365.4, 300 sec: 1291.3). Total num frames: 4464640. Throughput: 0: 355.1. Samples: 1117436. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:31:19,637][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1291.3). Total num frames: 4472832. Throughput: 0: 352.3. Samples: 1118296. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:31:24,633][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:29,628][00294] Fps is (10 sec: 819.6, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 4472832. Throughput: 0: 331.2. Samples: 1120012. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:31:29,633][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:34,628][00294] Fps is (10 sec: 819.2, 60 sec: 1365.3, 300 sec: 1277.4). Total num frames: 4481024. Throughput: 0: 325.8. Samples: 1121964. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) [2023-07-24 01:31:34,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4489216. Throughput: 0: 335.7. Samples: 1123268. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:31:39,633][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:44,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4497408. Throughput: 0: 353.3. Samples: 1125796. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:31:44,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:48,151][14527] Updated weights for policy 0, policy_version 1100 (0.0020) [2023-07-24 01:31:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4505600. Throughput: 0: 352.8. Samples: 1127484. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:31:49,634][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:54,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.2, 300 sec: 1277.4). Total num frames: 4505600. Throughput: 0: 340.1. Samples: 1128180. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:31:54,635][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:59,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 4513792. Throughput: 0: 311.3. Samples: 1129500. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:31:59,634][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:31:59,653][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001102_4513792.pth... [2023-07-24 01:31:59,950][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001026_4202496.pth [2023-07-24 01:32:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1263.5). Total num frames: 4517888. Throughput: 0: 297.4. Samples: 1130816. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:32:04,637][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:09,633][00294] Fps is (10 sec: 818.8, 60 sec: 1228.7, 300 sec: 1263.5). Total num frames: 4521984. Throughput: 0: 295.8. Samples: 1131608. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:32:09,638][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 4530176. Throughput: 0: 297.0. Samples: 1133376. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:32:14,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:19,628][00294] Fps is (10 sec: 1639.2, 60 sec: 1228.9, 300 sec: 1277.4). Total num frames: 4538368. Throughput: 0: 299.4. Samples: 1135436. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:32:19,635][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:24,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 4542464. Throughput: 0: 289.8. Samples: 1136308. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:32:24,633][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:26,107][14527] Updated weights for policy 0, policy_version 1110 (0.0067) [2023-07-24 01:32:29,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1277.4). Total num frames: 4546560. Throughput: 0: 270.9. Samples: 1137988. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:32:29,640][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4554752. Throughput: 0: 273.8. Samples: 1139804. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:32:34,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:39,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4562944. Throughput: 0: 287.6. Samples: 1141120. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:32:39,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4571136. Throughput: 0: 316.2. Samples: 1143728. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:32:44,636][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1277.4). Total num frames: 4575232. Throughput: 0: 325.2. Samples: 1145452. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:32:49,630][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4583424. Throughput: 0: 326.2. Samples: 1146284. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:32:54,637][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:32:58,537][14527] Updated weights for policy 0, policy_version 1120 (0.0021) [2023-07-24 01:32:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4587520. Throughput: 0: 325.5. Samples: 1148024. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) [2023-07-24 01:32:59,634][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4595712. Throughput: 0: 327.7. Samples: 1150184. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:04,634][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 4603904. Throughput: 0: 336.9. Samples: 1151468. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:09,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:14,632][00294] Fps is (10 sec: 1637.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4612096. Throughput: 0: 350.9. Samples: 1153780. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:14,636][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4616192. Throughput: 0: 348.2. Samples: 1155472. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:33:19,635][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:24,629][00294] Fps is (10 sec: 1229.1, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4624384. Throughput: 0: 337.8. Samples: 1156320. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:24,632][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:28,623][14527] Updated weights for policy 0, policy_version 1130 (0.0036) [2023-07-24 01:33:29,634][00294] Fps is (10 sec: 1228.1, 60 sec: 1365.2, 300 sec: 1291.3). Total num frames: 4628480. Throughput: 0: 318.1. Samples: 1158044. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:29,637][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:34,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4636672. Throughput: 0: 335.6. Samples: 1160556. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:34,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:39,628][00294] Fps is (10 sec: 1639.3, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4644864. Throughput: 0: 345.9. Samples: 1161848. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:39,635][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:44,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 4648960. Throughput: 0: 351.1. Samples: 1163824. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:44,637][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:49,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4657152. Throughput: 0: 340.4. Samples: 1165504. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:33:49,640][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:54,629][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 4661248. Throughput: 0: 330.6. Samples: 1166344. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:33:54,633][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:58,011][14527] Updated weights for policy 0, policy_version 1140 (0.0061) [2023-07-24 01:33:59,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4669440. Throughput: 0: 322.5. Samples: 1168292. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:33:59,642][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:33:59,662][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001140_4669440.pth... [2023-07-24 01:33:59,855][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001063_4354048.pth [2023-07-24 01:34:04,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4677632. Throughput: 0: 342.2. Samples: 1170872. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:34:04,639][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4685824. Throughput: 0: 349.9. Samples: 1172064. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:34:09,638][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4689920. Throughput: 0: 341.1. Samples: 1173392. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:34:14,631][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:19,631][00294] Fps is (10 sec: 819.0, 60 sec: 1297.0, 300 sec: 1277.4). Total num frames: 4694016. Throughput: 0: 314.3. Samples: 1174700. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:34:19,635][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:24,630][00294] Fps is (10 sec: 819.1, 60 sec: 1228.8, 300 sec: 1263.5). Total num frames: 4698112. Throughput: 0: 300.9. Samples: 1175388. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:34:24,634][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:29,628][00294] Fps is (10 sec: 819.4, 60 sec: 1228.9, 300 sec: 1277.4). Total num frames: 4702208. Throughput: 0: 286.3. Samples: 1176708. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:34:29,635][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:34,628][00294] Fps is (10 sec: 819.3, 60 sec: 1160.5, 300 sec: 1263.5). Total num frames: 4706304. Throughput: 0: 283.2. Samples: 1178248. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:34:34,637][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:35,192][14527] Updated weights for policy 0, policy_version 1150 (0.0056) [2023-07-24 01:34:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4718592. Throughput: 0: 292.7. Samples: 1179516. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:34:39,638][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:44,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4726784. Throughput: 0: 310.4. Samples: 1182260. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:34:44,633][00294] Avg episode reward: [(0, '-3.723')] [2023-07-24 01:34:48,707][14524] DAMAGECOUNT value on done: 1380.0 [2023-07-24 01:34:48,715][14524] Sum rewards: -2.493, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.380', 'AMMO2': '0.001', 'AMMO4': '0.003', 'AMMO5': '0.007', 'weapon5': '0.084', 'AMMO3': '0.105', 'WEAPON5': '0.150', 'HITCOUNT': '0.280', 'WEAPON3': '0.700', 'DAMAGECOUNT': '1.137', 'weapon3': '1.438', 'weapon2': '1.982', 'FRAGCOUNT': '2.000'} [2023-07-24 01:34:48,728][14528] DAMAGECOUNT value on done: 1099.0 [2023-07-24 01:34:48,732][14528] Sum rewards: -3.940, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.120', 'AMMO2': '0.007', 'AMMO5': '0.019', 'ARMOR': '0.020', 'AMMO4': '0.035', 'WEAPON4': '0.100', 'HITCOUNT': '0.100', 'weapon5': '0.104', 'AMMO3': '0.113', 'weapon4': '0.236', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.480', 'WEAPON3': '0.650', 'weapon2': '1.262', 'weapon3': '1.504', 'FRAGCOUNT': '2.000'} [2023-07-24 01:34:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4730880. Throughput: 0: 292.1. Samples: 1184016. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:34:49,630][00294] Avg episode reward: [(0, '-3.735')] [2023-07-24 01:34:53,684][14532] DAMAGECOUNT value on done: 1543.0 [2023-07-24 01:34:53,689][14532] Sum rewards: -1.709, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.614', 'AMMO5': '0.007', 'AMMO2': '0.014', 'AMMO4': '0.067', 'AMMO3': '0.149', 'WEAPON4': '0.150', 'WEAPON5': '0.150', 'weapon5': '0.184', 'HITCOUNT': '0.210', 'weapon4': '0.380', 'ARMOR': '0.464', 'WEAPON3': '0.850', 'DAMAGECOUNT': '0.867', 'weapon2': '1.170', 'weapon3': '1.742', 'FRAGCOUNT': '3.000'} [2023-07-24 01:34:54,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 4734976. Throughput: 0: 285.3. Samples: 1184904. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:34:54,633][00294] Avg episode reward: [(0, '-3.708')] [2023-07-24 01:34:55,643][14524] DAMAGECOUNT value on done: 1923.0 [2023-07-24 01:34:55,644][14524] Sum rewards: -3.429, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.030', 'AMMO5': '0.007', 'WEAPON1': '0.010', 'ARMOR': '0.016', 'AMMO2': '0.020', 'weapon5': '0.060', 'AMMO4': '0.101', 'AMMO3': '0.148', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'weapon4': '0.274', 'HITCOUNT': '0.300', 'WEAPON3': '0.900', 'weapon2': '1.170', 'DAMAGECOUNT': '1.440', 'FRAGCOUNT': '1.500', 'weapon3': '1.804'} [2023-07-24 01:34:55,704][14528] DAMAGECOUNT value on done: 1334.0 [2023-07-24 01:34:55,704][14528] Sum rewards: -2.980, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.234', 'weapon4': '0.002', 'AMMO5': '0.005', 'AMMO2': '0.012', 'weapon5': '0.048', 'AMMO4': '0.057', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.152', 'HITCOUNT': '0.170', 'ARMOR': '0.452', 'DAMAGECOUNT': '0.660', 'WEAPON3': '0.750', 'weapon2': '1.514', 'weapon3': '1.732', 'FRAGCOUNT': '3.000'} [2023-07-24 01:34:59,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 4739072. Throughput: 0: 294.2. Samples: 1186632. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:34:59,644][00294] Avg episode reward: [(0, '-3.725')] [2023-07-24 01:35:01,329][14532] DAMAGECOUNT value on done: 1606.0 [2023-07-24 01:35:02,473][14524] DAMAGECOUNT value on done: 1238.0 [2023-07-24 01:35:02,482][14524] Sum rewards: -6.235, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-0.506', 'AMMO2': '0.012', 'AMMO5': '0.020', 'WEAPON1': '0.020', 'AMMO4': '0.061', 'weapon5': '0.126', 'WEAPON4': '0.150', 'AMMO3': '0.175', 'HITCOUNT': '0.210', 'WEAPON5': '0.300', 'weapon4': '0.354', 'FRAGCOUNT': '0.500', 'DAMAGECOUNT': '0.762', 'WEAPON3': '0.900', 'weapon2': '0.966', 'weapon3': '1.714'} [2023-07-24 01:35:02,632][14528] DAMAGECOUNT value on done: 1131.0 [2023-07-24 01:35:02,636][14528] Sum rewards: -2.100, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.938', 'WEAPON1': '0.010', 'AMMO5': '0.014', 'AMMO2': '0.028', 'weapon5': '0.112', 'AMMO3': '0.120', 'HITCOUNT': '0.130', 'AMMO4': '0.142', 'WEAPON5': '0.250', 'WEAPON4': '0.350', 'ARMOR': '0.484', 'DAMAGECOUNT': '0.570', 'weapon4': '0.640', 'WEAPON3': '0.700', 'weapon2': '0.836', 'weapon3': '1.452', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:04,444][14531] DAMAGECOUNT value on done: 1765.0 [2023-07-24 01:35:04,450][14531] Sum rewards: -5.627, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-0.596', 'WEAPON1': '0.010', 'AMMO2': '0.019', 'ARMOR': '0.032', 'AMMO4': '0.092', 'WEAPON4': '0.150', 'AMMO3': '0.161', 'HITCOUNT': '0.210', 'weapon4': '0.336', 'WEAPON3': '0.750', 'DAMAGECOUNT': '0.855', 'weapon3': '1.374', 'weapon2': '1.730', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 4747264. Throughput: 0: 315.1. Samples: 1188880. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:35:04,630][00294] Avg episode reward: [(0, '-3.786')] [2023-07-24 01:35:04,659][14527] Updated weights for policy 0, policy_version 1160 (0.0037) [2023-07-24 01:35:06,576][14532] DAMAGECOUNT value on done: 890.0 [2023-07-24 01:35:06,577][14532] Sum rewards: -3.929, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.218', 'AMMO2': '0.012', 'AMMO5': '0.015', 'WEAPON1': '0.030', 'weapon5': '0.040', 'ARMOR': '0.048', 'AMMO4': '0.060', 'weapon4': '0.070', 'HITCOUNT': '0.140', 'WEAPON4': '0.150', 'AMMO3': '0.160', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.375', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon2': '1.126', 'weapon3': '1.962'} [2023-07-24 01:35:07,979][14524] DAMAGECOUNT value on done: 1040.0 [2023-07-24 01:35:07,987][14524] Sum rewards: -4.689, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.122', 'AMMO2': '0.009', 'WEAPON1': '0.010', 'AMMO5': '0.010', 'ARMOR': '0.020', 'HITCOUNT': '0.020', 'AMMO4': '0.044', 'WEAPON4': '0.050', 'DAMAGECOUNT': '0.090', 'WEAPON5': '0.100', 'weapon4': '0.116', 'AMMO3': '0.124', 'weapon5': '0.130', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon2': '1.262', 'weapon3': '1.548'} [2023-07-24 01:35:07,993][14528] DAMAGECOUNT value on done: 1391.0 [2023-07-24 01:35:07,995][14528] Sum rewards: -3.135, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.410', 'AMMO2': '0.001', 'AMMO4': '0.003', 'weapon7': '0.044', 'WEAPON4': '0.050', 'AMMO3': '0.097', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'HITCOUNT': '0.130', 'weapon4': '0.214', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.924', 'weapon3': '1.328', 'weapon2': '1.584', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:09,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4759552. Throughput: 0: 328.6. Samples: 1190176. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:35:09,636][00294] Avg episode reward: [(0, '-3.840')] [2023-07-24 01:35:09,867][14531] DAMAGECOUNT value on done: 1302.0 [2023-07-24 01:35:09,870][14531] Sum rewards: -2.812, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.112', 'AMMO5': '0.007', 'AMMO2': '0.016', 'ARMOR': '0.036', 'weapon4': '0.066', 'weapon7': '0.066', 'AMMO4': '0.078', 'WEAPON5': '0.100', 'weapon5': '0.106', 'AMMO6': '0.120', 'AMMO7': '0.120', 'AMMO3': '0.150', 'HITCOUNT': '0.180', 'WEAPON7': '0.200', 'WEAPON4': '0.250', 'WEAPON3': '0.700', 'DAMAGECOUNT': '1.041', 'weapon2': '1.282', 'weapon3': '1.782', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:12,124][14532] DAMAGECOUNT value on done: 1085.0 [2023-07-24 01:35:12,124][14532] Sum rewards: -0.438, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.314', 'AMMO5': '0.003', 'AMMO2': '0.006', 'AMMO4': '0.030', 'WEAPON5': '0.050', 'weapon5': '0.074', 'AMMO3': '0.086', 'WEAPON4': '0.100', 'HITCOUNT': '0.110', 'ARMOR': '0.494', 'WEAPON3': '0.500', 'weapon4': '0.532', 'DAMAGECOUNT': '0.615', 'weapon2': '0.984', 'FRAGCOUNT': '1.000', 'weapon3': '1.292'} [2023-07-24 01:35:14,441][14524] DAMAGECOUNT value on done: 1248.0 [2023-07-24 01:35:14,545][14528] DAMAGECOUNT value on done: 1327.0 [2023-07-24 01:35:14,553][14528] Sum rewards: -1.803, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-0.840', 'AMMO2': '0.002', 'AMMO5': '0.006', 'AMMO4': '0.011', 'ARMOR': '0.032', 'WEAPON4': '0.050', 'AMMO3': '0.101', 'weapon5': '0.118', 'HITCOUNT': '0.150', 'WEAPON5': '0.150', 'weapon4': '0.204', 'WEAPON3': '0.450', 'DAMAGECOUNT': '0.690', 'FRAGCOUNT': '1.000', 'weapon3': '1.314', 'weapon2': '1.508'} [2023-07-24 01:35:14,630][00294] Fps is (10 sec: 2048.0, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 4767744. Throughput: 0: 351.2. Samples: 1192512. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:35:14,636][00294] Avg episode reward: [(0, '-3.734')] [2023-07-24 01:35:16,940][14531] DAMAGECOUNT value on done: 1278.0 [2023-07-24 01:35:16,946][14531] Sum rewards: -8.887, reward structure: {'DEATHCOUNT': '-11.250', 'FRAGCOUNT': '-1.500', 'HEALTH': '-1.500', 'AMMO5': '0.018', 'AMMO2': '0.038', 'ARMOR': '0.072', 'weapon5': '0.104', 'HITCOUNT': '0.130', 'AMMO3': '0.136', 'AMMO4': '0.191', 'WEAPON5': '0.200', 'WEAPON4': '0.300', 'DAMAGECOUNT': '0.372', 'WEAPON3': '0.800', 'weapon4': '0.870', 'weapon3': '0.964', 'weapon2': '1.168'} [2023-07-24 01:35:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 4771840. Throughput: 0: 356.5. Samples: 1194292. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:35:19,631][00294] Avg episode reward: [(0, '-3.706')] [2023-07-24 01:35:19,889][14532] DAMAGECOUNT value on done: 1237.0 [2023-07-24 01:35:22,830][14528] DAMAGECOUNT value on done: 971.0 [2023-07-24 01:35:22,830][14528] Sum rewards: -2.760, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.366', 'AMMO5': '0.003', 'AMMO2': '0.011', 'weapon5': '0.020', 'WEAPON5': '0.050', 'AMMO4': '0.056', 'AMMO3': '0.078', 'HITCOUNT': '0.080', 'DAMAGECOUNT': '0.390', 'WEAPON3': '0.400', 'ARMOR': '0.428', 'FRAGCOUNT': '1.000', 'weapon3': '1.116', 'weapon2': '2.224'} [2023-07-24 01:35:22,901][14524] DAMAGECOUNT value on done: 2056.0 [2023-07-24 01:35:22,902][14524] Sum rewards: 3.318, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.014', 'AMMO2': '0.010', 'AMMO5': '0.015', 'AMMO4': '0.048', 'AMMO3': '0.082', 'weapon7': '0.086', 'WEAPON4': '0.100', 'AMMO6': '0.120', 'AMMO7': '0.120', 'weapon5': '0.146', 'WEAPON5': '0.200', 'WEAPON7': '0.200', 'weapon4': '0.202', 'HITCOUNT': '0.280', 'WEAPON3': '0.450', 'weapon3': '1.014', 'DAMAGECOUNT': '1.485', 'weapon2': '1.774', 'FRAGCOUNT': '3.000'} [2023-07-24 01:35:24,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4775936. Throughput: 0: 347.8. Samples: 1195168. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:35:24,637][00294] Avg episode reward: [(0, '-3.617')] [2023-07-24 01:35:25,245][14531] DAMAGECOUNT value on done: 1410.0 [2023-07-24 01:35:25,248][14531] Sum rewards: -4.252, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.190', 'AMMO2': '0.012', 'AMMO5': '0.017', 'WEAPON1': '0.020', 'AMMO4': '0.061', 'weapon5': '0.090', 'AMMO3': '0.160', 'HITCOUNT': '0.170', 'ARMOR': '0.400', 'WEAPON5': '0.400', 'WEAPON3': '0.900', 'FRAGCOUNT': '1.000', 'DAMAGECOUNT': '1.059', 'weapon2': '1.558', 'weapon3': '1.590'} [2023-07-24 01:35:27,782][14532] DAMAGECOUNT value on done: 1435.0 [2023-07-24 01:35:27,783][14532] Sum rewards: 0.519, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.570', 'AMMO5': '0.005', 'AMMO2': '0.017', 'weapon5': '0.034', 'ARMOR': '0.052', 'weapon7': '0.074', 'AMMO4': '0.085', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.121', 'HITCOUNT': '0.170', 'WEAPON4': '0.250', 'weapon4': '0.600', 'WEAPON3': '0.650', 'weapon2': '0.990', 'DAMAGECOUNT': '1.083', 'weapon3': '1.308', 'FRAGCOUNT': '3.000'} [2023-07-24 01:35:29,464][14524] DAMAGECOUNT value on done: 1053.0 [2023-07-24 01:35:29,469][14524] Sum rewards: 0.211, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.002', 'AMMO5': '0.007', 'weapon7': '0.012', 'AMMO2': '0.028', 'weapon5': '0.082', 'ARMOR': '0.088', 'AMMO3': '0.101', 'AMMO4': '0.141', 'WEAPON5': '0.150', 'HITCOUNT': '0.190', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON4': '0.250', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.717', 'weapon4': '0.748', 'weapon2': '1.060', 'weapon3': '1.138', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:29,498][14528] DAMAGECOUNT value on done: 1342.0 [2023-07-24 01:35:29,500][14528] Sum rewards: -5.983, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.576', 'FRAGCOUNT': '-1.500', 'AMMO2': '0.011', 'AMMO5': '0.025', 'AMMO4': '0.056', 'HITCOUNT': '0.080', 'AMMO3': '0.135', 'WEAPON4': '0.200', 'weapon4': '0.206', 'weapon5': '0.276', 'DAMAGECOUNT': '0.345', 'WEAPON5': '0.400', 'ARMOR': '0.527', 'WEAPON3': '0.800', 'weapon2': '0.824', 'weapon3': '1.458'} [2023-07-24 01:35:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4784128. Throughput: 0: 326.0. Samples: 1196928. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:35:29,630][00294] Avg episode reward: [(0, '-3.531')] [2023-07-24 01:35:29,891][14529] DAMAGECOUNT value on done: 1429.0 [2023-07-24 01:35:31,000][14531] DAMAGECOUNT value on done: 883.0 [2023-07-24 01:35:31,008][14531] Sum rewards: -2.857, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.740', 'AMMO5': '0.005', 'AMMO2': '0.007', 'WEAPON1': '0.020', 'weapon5': '0.022', 'AMMO4': '0.033', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'weapon4': '0.136', 'AMMO3': '0.157', 'HITCOUNT': '0.170', 'DAMAGECOUNT': '0.525', 'WEAPON3': '0.750', 'weapon2': '0.782', 'FRAGCOUNT': '1.000', 'weapon3': '2.326'} [2023-07-24 01:35:32,425][14527] Updated weights for policy 0, policy_version 1170 (0.0039) [2023-07-24 01:35:32,775][14532] DAMAGECOUNT value on done: 1231.0 [2023-07-24 01:35:32,779][14532] Sum rewards: -6.213, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-1.862', 'AMMO5': '0.017', 'AMMO2': '0.031', 'ARMOR': '0.033', 'weapon7': '0.036', 'HITCOUNT': '0.100', 'AMMO4': '0.152', 'AMMO3': '0.156', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON4': '0.200', 'WEAPON7': '0.200', 'weapon5': '0.220', 'WEAPON5': '0.300', 'weapon4': '0.304', 'DAMAGECOUNT': '0.735', 'WEAPON3': '0.850', 'weapon2': '1.076', 'weapon3': '1.668', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:33,970][14524] DAMAGECOUNT value on done: 1132.0 [2023-07-24 01:35:33,972][14524] Sum rewards: -2.703, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-2.040', 'AMMO2': '0.007', 'AMMO4': '0.036', 'weapon7': '0.068', 'AMMO3': '0.107', 'AMMO6': '0.120', 'AMMO7': '0.120', 'HITCOUNT': '0.180', 'WEAPON4': '0.200', 'WEAPON7': '0.200', 'weapon4': '0.346', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.828', 'weapon3': '0.882', 'weapon2': '1.942', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:34,088][14528] DAMAGECOUNT value on done: 1847.0 [2023-07-24 01:35:34,091][14528] Sum rewards: -5.541, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.824', 'WEAPON1': '0.010', 'ARMOR': '0.020', 'AMMO5': '0.023', 'AMMO2': '0.023', 'weapon4': '0.110', 'AMMO4': '0.116', 'WEAPON4': '0.150', 'weapon5': '0.158', 'AMMO3': '0.178', 'HITCOUNT': '0.320', 'WEAPON5': '0.450', 'weapon2': '0.830', 'WEAPON3': '1.050', 'DAMAGECOUNT': '1.305', 'FRAGCOUNT': '1.500', 'weapon3': '2.040'} [2023-07-24 01:35:34,535][14529] DAMAGECOUNT value on done: 1074.0 [2023-07-24 01:35:34,538][14529] Sum rewards: -6.162, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-2.435', 'AMMO2': '0.005', 'AMMO5': '0.017', 'WEAPON1': '0.020', 'AMMO4': '0.026', 'weapon5': '0.120', 'AMMO3': '0.143', 'HITCOUNT': '0.230', 'WEAPON4': '0.250', 'WEAPON5': '0.300', 'ARMOR': '0.493', 'weapon4': '0.634', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.840', 'weapon3': '1.076', 'weapon2': '1.418', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 4792320. Throughput: 0: 344.4. Samples: 1199516. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:35:34,631][00294] Avg episode reward: [(0, '-3.783')] [2023-07-24 01:35:36,191][14531] DAMAGECOUNT value on done: 1897.0 [2023-07-24 01:35:36,198][14531] Sum rewards: -5.968, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.944', 'weapon5': '0.002', 'AMMO2': '0.008', 'WEAPON1': '0.010', 'AMMO5': '0.013', 'AMMO4': '0.040', 'HITCOUNT': '0.080', 'ARMOR': '0.080', 'AMMO3': '0.129', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'weapon4': '0.174', 'DAMAGECOUNT': '0.180', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon3': '1.346', 'weapon2': '1.614'} [2023-07-24 01:35:37,618][14532] DAMAGECOUNT value on done: 1508.0 [2023-07-24 01:35:37,619][14532] Sum rewards: -4.548, reward structure: {'DEATHCOUNT': '-6.000', 'FRAGCOUNT': '-2.000', 'HEALTH': '-0.937', 'AMMO4': '-0.013', 'AMMO2': '-0.002', 'AMMO5': '0.028', 'WEAPON1': '0.030', 'WEAPON4': '0.050', 'AMMO3': '0.074', 'weapon4': '0.098', 'HITCOUNT': '0.130', 'weapon5': '0.136', 'WEAPON5': '0.300', 'DAMAGECOUNT': '0.438', 'WEAPON3': '0.500', 'weapon2': '1.144', 'weapon3': '1.476'} [2023-07-24 01:35:39,219][14529] DAMAGECOUNT value on done: 885.0 [2023-07-24 01:35:39,227][14529] Sum rewards: -4.993, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.236', 'AMMO2': '0.000', 'AMMO4': '0.002', 'AMMO5': '0.005', 'WEAPON1': '0.020', 'ARMOR': '0.057', 'HITCOUNT': '0.080', 'weapon5': '0.082', 'AMMO3': '0.125', 'WEAPON5': '0.150', 'DAMAGECOUNT': '0.495', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon2': '1.356', 'weapon3': '1.820'} [2023-07-24 01:35:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4800512. Throughput: 0: 353.4. Samples: 1200808. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:35:39,634][00294] Avg episode reward: [(0, '-3.833')] [2023-07-24 01:35:42,902][14531] DAMAGECOUNT value on done: 949.0 [2023-07-24 01:35:42,916][14531] Sum rewards: -5.046, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.622', 'FRAGCOUNT': '-0.500', 'ARMOR': '0.012', 'AMMO5': '0.014', 'AMMO2': '0.023', 'weapon7': '0.074', 'weapon5': '0.102', 'HITCOUNT': '0.110', 'AMMO4': '0.114', 'AMMO6': '0.120', 'AMMO7': '0.120', 'AMMO3': '0.128', 'WEAPON5': '0.200', 'WEAPON7': '0.200', 'WEAPON4': '0.300', 'DAMAGECOUNT': '0.405', 'WEAPON3': '0.600', 'weapon4': '0.826', 'weapon2': '0.902', 'weapon3': '1.076'} [2023-07-24 01:35:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4808704. Throughput: 0: 360.7. Samples: 1202864. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:35:44,638][00294] Avg episode reward: [(0, '-3.848')] [2023-07-24 01:35:45,128][14530] DAMAGECOUNT value on done: 1420.0 [2023-07-24 01:35:45,891][14529] DAMAGECOUNT value on done: 1519.0 [2023-07-24 01:35:45,892][14529] Sum rewards: 0.274, reward structure: {'DEATHCOUNT': '-8.250', 'AMMO5': '0.012', 'AMMO2': '0.020', 'ARMOR': '0.050', 'AMMO4': '0.102', 'AMMO3': '0.106', 'HEALTH': '0.148', 'weapon5': '0.172', 'WEAPON4': '0.250', 'WEAPON5': '0.250', 'HITCOUNT': '0.270', 'weapon4': '0.508', 'WEAPON3': '0.550', 'weapon2': '0.892', 'DAMAGECOUNT': '1.077', 'weapon3': '1.616', 'FRAGCOUNT': '2.500'} [2023-07-24 01:35:49,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 4816896. Throughput: 0: 349.5. Samples: 1204608. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:35:49,636][00294] Avg episode reward: [(0, '-3.837')] [2023-07-24 01:35:49,685][14531] DAMAGECOUNT value on done: 1384.0 [2023-07-24 01:35:49,689][14531] Sum rewards: -5.211, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.504', 'AMMO5': '0.004', 'WEAPON1': '0.020', 'AMMO2': '0.023', 'ARMOR': '0.037', 'WEAPON5': '0.100', 'AMMO4': '0.114', 'weapon5': '0.124', 'weapon4': '0.168', 'WEAPON4': '0.200', 'AMMO3': '0.209', 'HITCOUNT': '0.310', 'WEAPON3': '1.050', 'DAMAGECOUNT': '1.200', 'weapon2': '1.276', 'weapon3': '1.708', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:52,167][14530] DAMAGECOUNT value on done: 1786.0 [2023-07-24 01:35:52,171][14530] Sum rewards: -4.144, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.656', 'AMMO2': '0.004', 'AMMO5': '0.011', 'WEAPON1': '0.020', 'AMMO4': '0.020', 'ARMOR': '0.022', 'WEAPON4': '0.050', 'AMMO3': '0.114', 'weapon4': '0.114', 'HITCOUNT': '0.120', 'WEAPON5': '0.250', 'weapon5': '0.250', 'DAMAGECOUNT': '0.390', 'WEAPON3': '0.600', 'FRAGCOUNT': '1.000', 'weapon3': '1.316', 'weapon2': '1.480'} [2023-07-24 01:35:53,118][14529] DAMAGECOUNT value on done: 1075.0 [2023-07-24 01:35:53,123][14529] Sum rewards: -3.848, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.814', 'AMMO5': '0.005', 'ARMOR': '0.013', 'AMMO2': '0.015', 'weapon5': '0.034', 'AMMO4': '0.072', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.122', 'HITCOUNT': '0.130', 'weapon4': '0.148', 'DAMAGECOUNT': '0.387', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon3': '1.476', 'weapon2': '1.614'} [2023-07-24 01:35:53,145][14525] DAMAGECOUNT value on done: 1141.0 [2023-07-24 01:35:53,147][14525] Sum rewards: -4.547, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.776', 'FRAGCOUNT': '-0.500', 'AMMO4': '-0.024', 'AMMO2': '-0.005', 'AMMO5': '0.012', 'weapon5': '0.048', 'weapon7': '0.052', 'AMMO3': '0.102', 'HITCOUNT': '0.110', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON5': '0.150', 'WEAPON7': '0.200', 'DAMAGECOUNT': '0.450', 'WEAPON3': '0.650', 'weapon3': '1.474', 'weapon2': '1.770'} [2023-07-24 01:35:53,740][14526] DAMAGECOUNT value on done: 1186.0 [2023-07-24 01:35:53,747][14526] Sum rewards: -6.707, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.740', 'AMMO2': '0.006', 'AMMO5': '0.015', 'WEAPON1': '0.020', 'AMMO4': '0.029', 'weapon5': '0.030', 'ARMOR': '0.060', 'weapon4': '0.072', 'AMMO3': '0.139', 'WEAPON4': '0.150', 'HITCOUNT': '0.190', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.711', 'WEAPON3': '0.850', 'FRAGCOUNT': '1.000', 'weapon2': '1.322', 'weapon3': '1.740'} [2023-07-24 01:35:54,631][00294] Fps is (10 sec: 819.1, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4816896. Throughput: 0: 340.4. Samples: 1205496. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:35:54,640][00294] Avg episode reward: [(0, '-3.943')] [2023-07-24 01:35:58,246][14530] DAMAGECOUNT value on done: 903.0 [2023-07-24 01:35:58,247][14530] Sum rewards: -6.193, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.492', 'WEAPON1': '0.010', 'AMMO5': '0.012', 'AMMO2': '0.028', 'weapon5': '0.042', 'ARMOR': '0.055', 'HITCOUNT': '0.070', 'AMMO4': '0.139', 'AMMO3': '0.186', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.252', 'WEAPON4': '0.400', 'weapon4': '0.626', 'WEAPON3': '1.050', 'weapon2': '1.122', 'weapon3': '1.306', 'FRAGCOUNT': '2.000'} [2023-07-24 01:35:58,788][14525] DAMAGECOUNT value on done: 1220.0 [2023-07-24 01:35:58,791][14525] Sum rewards: 2.077, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-0.990', 'AMMO5': '0.017', 'AMMO2': '0.020', 'WEAPON1': '0.020', 'AMMO4': '0.098', 'AMMO3': '0.108', 'HITCOUNT': '0.140', 'WEAPON4': '0.200', 'weapon5': '0.240', 'WEAPON5': '0.300', 'WEAPON3': '0.500', 'weapon3': '1.252', 'DAMAGECOUNT': '1.422', 'weapon2': '1.500', 'FRAGCOUNT': '4.000'} [2023-07-24 01:35:58,829][14529] DAMAGECOUNT value on done: 1589.0 [2023-07-24 01:35:58,832][14529] Sum rewards: 1.575, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.105', 'AMMO2': '0.002', 'AMMO4': '0.009', 'AMMO5': '0.022', 'WEAPON4': '0.050', 'weapon4': '0.122', 'AMMO3': '0.153', 'weapon5': '0.158', 'HITCOUNT': '0.200', 'WEAPON5': '0.350', 'weapon2': '0.762', 'WEAPON3': '0.800', 'DAMAGECOUNT': '1.200', 'weapon3': '2.102', 'FRAGCOUNT': '5.000'} [2023-07-24 01:35:59,304][14526] DAMAGECOUNT value on done: 1204.0 [2023-07-24 01:35:59,313][14526] Sum rewards: 1.593, reward structure: {'DEATHCOUNT': '-3.000', 'HEALTH': '-0.650', 'AMMO5': '0.005', 'weapon5': '0.008', 'AMMO2': '0.020', 'ARMOR': '0.040', 'WEAPON4': '0.050', 'AMMO3': '0.058', 'weapon7': '0.084', 'WEAPON5': '0.100', 'AMMO4': '0.100', 'AMMO6': '0.120', 'AMMO7': '0.120', 'HITCOUNT': '0.130', 'weapon4': '0.184', 'WEAPON7': '0.200', 'WEAPON3': '0.450', 'DAMAGECOUNT': '0.771', 'weapon2': '0.802', 'FRAGCOUNT': '1.000', 'weapon3': '1.000'} [2023-07-24 01:35:59,628][00294] Fps is (10 sec: 819.2, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 4825088. Throughput: 0: 333.4. Samples: 1207516. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) [2023-07-24 01:35:59,634][00294] Avg episode reward: [(0, '-3.799')] [2023-07-24 01:35:59,647][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001178_4825088.pth... [2023-07-24 01:35:59,853][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001102_4513792.pth [2023-07-24 01:36:01,065][14527] Updated weights for policy 0, policy_version 1180 (0.0033) [2023-07-24 01:36:02,658][14530] DAMAGECOUNT value on done: 1194.0 [2023-07-24 01:36:02,660][14530] Sum rewards: -5.537, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.104', 'AMMO5': '0.010', 'WEAPON1': '0.010', 'ARMOR': '0.036', 'AMMO2': '0.047', 'weapon5': '0.096', 'AMMO3': '0.140', 'HITCOUNT': '0.160', 'WEAPON5': '0.200', 'AMMO4': '0.234', 'WEAPON4': '0.500', 'DAMAGECOUNT': '0.630', 'weapon4': '0.708', 'WEAPON3': '0.900', 'weapon2': '0.928', 'FRAGCOUNT': '1.000', 'weapon3': '1.468'} [2023-07-24 01:36:03,195][14525] DAMAGECOUNT value on done: 987.0 [2023-07-24 01:36:03,197][14525] Sum rewards: -6.558, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-2.666', 'AMMO5': '0.015', 'AMMO2': '0.019', 'ARMOR': '0.040', 'AMMO4': '0.092', 'WEAPON4': '0.150', 'HITCOUNT': '0.160', 'AMMO3': '0.187', 'weapon5': '0.242', 'WEAPON5': '0.350', 'weapon4': '0.412', 'DAMAGECOUNT': '0.450', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.000', 'weapon2': '1.084', 'weapon3': '1.408'} [2023-07-24 01:36:03,222][14529] DAMAGECOUNT value on done: 2044.0 [2023-07-24 01:36:03,224][14529] Sum rewards: -3.904, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.961', 'AMMO5': '0.007', 'AMMO2': '0.018', 'WEAPON1': '0.030', 'WEAPON5': '0.050', 'ARMOR': '0.076', 'AMMO4': '0.088', 'AMMO3': '0.124', 'weapon4': '0.214', 'HITCOUNT': '0.260', 'WEAPON4': '0.300', 'WEAPON3': '0.850', 'DAMAGECOUNT': '1.101', 'weapon2': '1.442', 'weapon3': '1.746', 'FRAGCOUNT': '4.000'} [2023-07-24 01:36:03,743][14526] DAMAGECOUNT value on done: 1940.0 [2023-07-24 01:36:03,747][14526] Sum rewards: -4.226, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-1.084', 'weapon5': '0.008', 'AMMO5': '0.015', 'AMMO2': '0.021', 'ARMOR': '0.024', 'AMMO4': '0.107', 'WEAPON5': '0.150', 'AMMO3': '0.207', 'WEAPON4': '0.250', 'weapon4': '0.332', 'HITCOUNT': '0.340', 'DAMAGECOUNT': '1.020', 'WEAPON3': '1.100', 'weapon2': '1.162', 'weapon3': '1.872', 'FRAGCOUNT': '3.000'} [2023-07-24 01:36:04,628][00294] Fps is (10 sec: 2048.2, 60 sec: 1501.9, 300 sec: 1319.1). Total num frames: 4837376. Throughput: 0: 354.1. Samples: 1210228. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:36:04,637][00294] Avg episode reward: [(0, '-3.739')] [2023-07-24 01:36:07,088][14530] DAMAGECOUNT value on done: 1177.0 [2023-07-24 01:36:07,094][14530] Sum rewards: -4.244, reward structure: {'DEATHCOUNT': '-10.500', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.007', 'AMMO2': '0.017', 'weapon5': '0.062', 'weapon7': '0.068', 'AMMO4': '0.086', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'AMMO3': '0.137', 'WEAPON5': '0.150', 'HITCOUNT': '0.210', 'HEALTH': '0.364', 'ARMOR': '0.400', 'WEAPON3': '0.750', 'DAMAGECOUNT': '0.870', 'weapon2': '1.654', 'weapon3': '1.680'} [2023-07-24 01:36:08,035][14529] DAMAGECOUNT value on done: 2181.0 [2023-07-24 01:36:08,035][14529] Sum rewards: 1.523, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.308', 'AMMO5': '0.004', 'AMMO2': '0.005', 'ARMOR': '0.016', 'AMMO4': '0.024', 'weapon5': '0.084', 'WEAPON5': '0.100', 'AMMO3': '0.125', 'HITCOUNT': '0.500', 'WEAPON3': '0.800', 'weapon2': '0.832', 'DAMAGECOUNT': '1.722', 'weapon3': '2.370', 'FRAGCOUNT': '3.500'} [2023-07-24 01:36:08,109][14525] DAMAGECOUNT value on done: 1045.0 [2023-07-24 01:36:08,718][14526] DAMAGECOUNT value on done: 1498.0 [2023-07-24 01:36:08,724][14526] Sum rewards: -5.099, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.336', 'AMMO2': '0.019', 'AMMO5': '0.020', 'WEAPON1': '0.030', 'AMMO4': '0.093', 'HITCOUNT': '0.110', 'AMMO3': '0.113', 'weapon4': '0.122', 'weapon5': '0.150', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.216', 'WEAPON5': '0.400', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon2': '1.058', 'weapon3': '1.806'} [2023-07-24 01:36:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4841472. Throughput: 0: 362.3. Samples: 1211472. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:36:09,631][00294] Avg episode reward: [(0, '-3.656')] [2023-07-24 01:36:12,639][14530] DAMAGECOUNT value on done: 2026.0 [2023-07-24 01:36:12,646][14530] Sum rewards: -2.207, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.462', 'AMMO2': '0.006', 'AMMO5': '0.007', 'WEAPON1': '0.010', 'AMMO4': '0.028', 'ARMOR': '0.036', 'weapon5': '0.044', 'weapon4': '0.058', 'WEAPON4': '0.100', 'WEAPON5': '0.150', 'AMMO3': '0.184', 'HITCOUNT': '0.350', 'weapon2': '0.946', 'WEAPON3': '1.050', 'DAMAGECOUNT': '1.350', 'weapon3': '2.186', 'FRAGCOUNT': '4.000'} [2023-07-24 01:36:13,555][14525] DAMAGECOUNT value on done: 1170.0 [2023-07-24 01:36:13,559][14525] Sum rewards: -5.503, reward structure: {'DEATHCOUNT': '-6.750', 'FRAGCOUNT': '-3.500', 'HEALTH': '-0.998', 'weapon7': '0.006', 'AMMO5': '0.014', 'WEAPON1': '0.020', 'weapon5': '0.026', 'AMMO2': '0.031', 'ARMOR': '0.072', 'AMMO3': '0.089', 'HITCOUNT': '0.130', 'AMMO4': '0.155', 'WEAPON5': '0.200', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON4': '0.400', 'weapon4': '0.440', 'DAMAGECOUNT': '0.525', 'WEAPON3': '0.550', 'weapon2': '1.190', 'weapon3': '1.296'} [2023-07-24 01:36:14,593][14526] DAMAGECOUNT value on done: 714.0 [2023-07-24 01:36:14,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 4849664. Throughput: 0: 361.9. Samples: 1213216. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:36:14,635][00294] Avg episode reward: [(0, '-3.616')] [2023-07-24 01:36:18,363][14530] DAMAGECOUNT value on done: 1106.0 [2023-07-24 01:36:18,367][14530] Sum rewards: -6.683, reward structure: {'DEATHCOUNT': '-7.500', 'FRAGCOUNT': '-3.000', 'HEALTH': '-0.634', 'AMMO5': '0.008', 'AMMO2': '0.013', 'ARMOR': '0.024', 'AMMO4': '0.066', 'weapon5': '0.066', 'AMMO3': '0.095', 'HITCOUNT': '0.120', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.327', 'WEAPON3': '0.500', 'weapon4': '0.668', 'weapon3': '0.810', 'weapon2': '1.404'} [2023-07-24 01:36:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 4853760. Throughput: 0: 342.9. Samples: 1214948. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:36:19,631][00294] Avg episode reward: [(0, '-3.657')] [2023-07-24 01:36:19,847][14525] DAMAGECOUNT value on done: 912.0 [2023-07-24 01:36:20,846][14526] DAMAGECOUNT value on done: 1508.0 [2023-07-24 01:36:20,847][14526] Sum rewards: -3.439, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-2.530', 'ARMOR': '0.016', 'AMMO2': '0.018', 'AMMO5': '0.025', 'AMMO4': '0.090', 'AMMO3': '0.095', 'weapon5': '0.110', 'HITCOUNT': '0.140', 'WEAPON4': '0.300', 'WEAPON5': '0.400', 'WEAPON3': '0.450', 'weapon4': '0.554', 'DAMAGECOUNT': '0.630', 'weapon2': '0.638', 'FRAGCOUNT': '1.000', 'weapon3': '1.374'} [2023-07-24 01:36:24,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 4861952. Throughput: 0: 333.9. Samples: 1215832. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:36:24,635][00294] Avg episode reward: [(0, '-3.736')] [2023-07-24 01:36:24,953][14530] DAMAGECOUNT value on done: 1173.0 [2023-07-24 01:36:24,956][14530] Sum rewards: -2.362, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.260', 'AMMO5': '0.013', 'AMMO2': '0.013', 'ARMOR': '0.040', 'WEAPON1': '0.040', 'WEAPON4': '0.050', 'AMMO4': '0.066', 'AMMO3': '0.089', 'weapon5': '0.104', 'weapon4': '0.188', 'WEAPON5': '0.200', 'HITCOUNT': '0.220', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.669', 'weapon2': '1.026', 'weapon3': '1.780', 'FRAGCOUNT': '2.000'} [2023-07-24 01:36:25,752][14525] DAMAGECOUNT value on done: 1684.0 [2023-07-24 01:36:25,755][14525] Sum rewards: -4.450, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.425', 'AMMO2': '0.002', 'AMMO4': '0.008', 'AMMO5': '0.010', 'ARMOR': '0.028', 'weapon5': '0.118', 'WEAPON5': '0.150', 'AMMO3': '0.157', 'HITCOUNT': '0.430', 'WEAPON3': '0.900', 'weapon2': '1.226', 'DAMAGECOUNT': '1.590', 'weapon3': '1.856', 'FRAGCOUNT': '2.500'} [2023-07-24 01:36:26,070][14526] DAMAGECOUNT value on done: 1662.0 [2023-07-24 01:36:26,072][14526] Sum rewards: -8.104, reward structure: {'DEATHCOUNT': '-9.750', 'FRAGCOUNT': '-4.000', 'HEALTH': '-0.653', 'AMMO5': '0.024', 'ARMOR': '0.032', 'AMMO2': '0.032', 'weapon7': '0.080', 'HITCOUNT': '0.090', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'AMMO3': '0.125', 'AMMO4': '0.161', 'WEAPON4': '0.200', 'weapon4': '0.406', 'weapon5': '0.436', 'WEAPON5': '0.450', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.774', 'weapon2': '0.886', 'weapon3': '1.602'} [2023-07-24 01:36:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 4870144. Throughput: 0: 340.6. Samples: 1218192. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:36:29,634][00294] Avg episode reward: [(0, '-3.827')] [2023-07-24 01:36:30,720][14525] DAMAGECOUNT value on done: 1458.0 [2023-07-24 01:36:30,720][14525] Sum rewards: -1.746, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.490', 'AMMO5': '0.005', 'AMMO2': '0.015', 'weapon5': '0.030', 'AMMO4': '0.075', 'WEAPON5': '0.100', 'ARMOR': '0.120', 'AMMO3': '0.143', 'HITCOUNT': '0.200', 'WEAPON4': '0.200', 'weapon4': '0.372', 'DAMAGECOUNT': '0.717', 'WEAPON3': '0.850', 'weapon2': '0.856', 'weapon3': '2.060', 'FRAGCOUNT': '3.000'} [2023-07-24 01:36:31,088][14526] DAMAGECOUNT value on done: 1434.0 [2023-07-24 01:36:31,088][14526] Sum rewards: -1.379, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.294', 'AMMO5': '0.017', 'AMMO2': '0.033', 'AMMO3': '0.079', 'HITCOUNT': '0.090', 'weapon5': '0.102', 'AMMO4': '0.165', 'WEAPON5': '0.250', 'WEAPON4': '0.300', 'DAMAGECOUNT': '0.447', 'ARMOR': '0.448', 'WEAPON3': '0.450', 'weapon3': '0.676', 'weapon4': '0.702', 'FRAGCOUNT': '1.000', 'weapon2': '1.656'} [2023-07-24 01:36:31,859][14527] Updated weights for policy 0, policy_version 1190 (0.0033) [2023-07-24 01:36:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 4878336. Throughput: 0: 349.1. Samples: 1220316. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:36:34,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:36:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4878336. Throughput: 0: 344.6. Samples: 1221004. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:36:39,635][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:36:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 4886528. Throughput: 0: 330.0. Samples: 1222364. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:36:44,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:36:49,631][00294] Fps is (10 sec: 819.0, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 4886528. Throughput: 0: 299.5. Samples: 1223708. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:36:49,639][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:36:54,633][00294] Fps is (10 sec: 818.8, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 4894720. Throughput: 0: 286.9. Samples: 1224384. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:36:54,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:36:59,628][00294] Fps is (10 sec: 1638.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 4902912. Throughput: 0: 283.2. Samples: 1225960. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:36:59,637][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:04,628][00294] Fps is (10 sec: 1639.1, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 4911104. Throughput: 0: 302.8. Samples: 1228576. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:37:04,639][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:06,365][14527] Updated weights for policy 0, policy_version 1200 (0.0064) [2023-07-24 01:37:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 4919296. Throughput: 0: 313.4. Samples: 1229936. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:37:09,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:14,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4923392. Throughput: 0: 303.4. Samples: 1231848. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:37:14,635][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:19,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 4927488. Throughput: 0: 295.0. Samples: 1233592. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:37:19,632][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:24,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 4935680. Throughput: 0: 299.1. Samples: 1234464. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:37:24,633][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 4943872. Throughput: 0: 314.8. Samples: 1236532. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:37:29,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 4952064. Throughput: 0: 347.1. Samples: 1239328. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:37:34,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:35,350][14527] Updated weights for policy 0, policy_version 1210 (0.0024) [2023-07-24 01:37:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 4960256. Throughput: 0: 363.2. Samples: 1240728. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:37:39,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 4968448. Throughput: 0: 373.4. Samples: 1242764. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:37:44,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.7, 300 sec: 1319.1). Total num frames: 4972544. Throughput: 0: 359.8. Samples: 1244768. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:37:49,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:54,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 4980736. Throughput: 0: 348.0. Samples: 1245596. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:37:54,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 4988928. Throughput: 0: 360.9. Samples: 1248088. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:37:59,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:37:59,648][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001218_4988928.pth... [2023-07-24 01:37:59,862][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001140_4669440.pth [2023-07-24 01:38:02,498][14527] Updated weights for policy 0, policy_version 1220 (0.0018) [2023-07-24 01:38:04,628][00294] Fps is (10 sec: 1638.8, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 4997120. Throughput: 0: 376.4. Samples: 1250528. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:38:04,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:09,635][00294] Fps is (10 sec: 1637.3, 60 sec: 1433.4, 300 sec: 1332.9). Total num frames: 5005312. Throughput: 0: 376.1. Samples: 1251392. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:38:09,637][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:14,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 5009408. Throughput: 0: 368.7. Samples: 1253124. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:38:14,633][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:19,628][00294] Fps is (10 sec: 819.7, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 5013504. Throughput: 0: 344.4. Samples: 1254828. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:38:19,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:24,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1433.6, 300 sec: 1333.0). Total num frames: 5021696. Throughput: 0: 335.3. Samples: 1255816. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:38:24,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 5029888. Throughput: 0: 348.4. Samples: 1258440. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:38:29,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:33,785][14527] Updated weights for policy 0, policy_version 1230 (0.0052) [2023-07-24 01:38:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 5038080. Throughput: 0: 352.1. Samples: 1260612. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:38:34,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 5042176. Throughput: 0: 353.2. Samples: 1261488. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:38:39,638][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5046272. Throughput: 0: 335.3. Samples: 1263176. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:38:44,635][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 5054464. Throughput: 0: 319.1. Samples: 1264888. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:38:49,635][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.4, 300 sec: 1332.9). Total num frames: 5062656. Throughput: 0: 328.8. Samples: 1266188. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:38:54,635][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:38:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 5070848. Throughput: 0: 339.9. Samples: 1268420. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:38:59,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5074944. Throughput: 0: 332.4. Samples: 1269788. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:39:04,639][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:08,570][14527] Updated weights for policy 0, policy_version 1240 (0.0028) [2023-07-24 01:39:09,630][00294] Fps is (10 sec: 819.1, 60 sec: 1228.9, 300 sec: 1319.0). Total num frames: 5079040. Throughput: 0: 324.7. Samples: 1270428. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:39:09,633][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.9, 300 sec: 1319.1). Total num frames: 5083136. Throughput: 0: 295.1. Samples: 1271720. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:39:14,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:19,628][00294] Fps is (10 sec: 819.3, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 5087232. Throughput: 0: 275.1. Samples: 1272992. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:39:19,637][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:24,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1319.1). Total num frames: 5091328. Throughput: 0: 274.4. Samples: 1273836. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:39:24,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1332.9). Total num frames: 5099520. Throughput: 0: 290.7. Samples: 1276256. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:39:29,641][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:34,633][00294] Fps is (10 sec: 1637.6, 60 sec: 1160.4, 300 sec: 1319.0). Total num frames: 5107712. Throughput: 0: 309.7. Samples: 1278828. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:39:34,638][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:39,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 5115904. Throughput: 0: 299.5. Samples: 1279664. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:39:39,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:42,300][14527] Updated weights for policy 0, policy_version 1250 (0.0051) [2023-07-24 01:39:44,635][00294] Fps is (10 sec: 1638.1, 60 sec: 1296.9, 300 sec: 1332.9). Total num frames: 5124096. Throughput: 0: 287.5. Samples: 1281360. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:39:44,638][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:49,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1228.7, 300 sec: 1332.9). Total num frames: 5128192. Throughput: 0: 295.8. Samples: 1283100. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:39:49,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:54,629][00294] Fps is (10 sec: 1229.4, 60 sec: 1228.8, 300 sec: 1346.8). Total num frames: 5136384. Throughput: 0: 302.0. Samples: 1284016. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:39:54,632][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:59,628][00294] Fps is (10 sec: 1638.8, 60 sec: 1228.8, 300 sec: 1346.8). Total num frames: 5144576. Throughput: 0: 332.7. Samples: 1286692. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:39:59,638][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:39:59,655][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001256_5144576.pth... [2023-07-24 01:39:59,836][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001178_4825088.pth [2023-07-24 01:40:04,629][00294] Fps is (10 sec: 1228.9, 60 sec: 1228.8, 300 sec: 1319.0). Total num frames: 5148672. Throughput: 0: 352.3. Samples: 1288844. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:40:04,637][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5156864. Throughput: 0: 352.3. Samples: 1289688. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:40:09,633][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:12,796][14527] Updated weights for policy 0, policy_version 1260 (0.0036) [2023-07-24 01:40:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5160960. Throughput: 0: 336.3. Samples: 1291388. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:40:14,639][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1332.9). Total num frames: 5169152. Throughput: 0: 318.3. Samples: 1293148. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:40:19,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 5177344. Throughput: 0: 327.4. Samples: 1294396. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:40:24,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 5185536. Throughput: 0: 348.4. Samples: 1297036. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:40:29,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1319.1). Total num frames: 5189632. Throughput: 0: 351.7. Samples: 1298924. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:40:34,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:39,633][00294] Fps is (10 sec: 1228.2, 60 sec: 1365.2, 300 sec: 1319.0). Total num frames: 5197824. Throughput: 0: 349.8. Samples: 1299760. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:40:39,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:44,067][14527] Updated weights for policy 0, policy_version 1270 (0.0074) [2023-07-24 01:40:44,631][00294] Fps is (10 sec: 1228.4, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5201920. Throughput: 0: 328.6. Samples: 1301480. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:40:44,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:49,628][00294] Fps is (10 sec: 819.6, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5206016. Throughput: 0: 326.0. Samples: 1303512. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:40:49,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:54,628][00294] Fps is (10 sec: 1638.9, 60 sec: 1365.4, 300 sec: 1332.9). Total num frames: 5218304. Throughput: 0: 336.1. Samples: 1304812. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:40:54,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:40:59,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5222400. Throughput: 0: 354.7. Samples: 1307348. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:40:59,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5230592. Throughput: 0: 353.9. Samples: 1309072. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:41:04,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:09,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5238784. Throughput: 0: 344.7. Samples: 1309908. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) [2023-07-24 01:41:09,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:13,357][14527] Updated weights for policy 0, policy_version 1280 (0.0037) [2023-07-24 01:41:14,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5242880. Throughput: 0: 324.7. Samples: 1311648. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:41:14,634][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5251072. Throughput: 0: 335.5. Samples: 1314020. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:41:19,635][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5259264. Throughput: 0: 343.2. Samples: 1315204. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:41:24,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5263360. Throughput: 0: 339.1. Samples: 1316740. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:41:29,635][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:34,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5267456. Throughput: 0: 323.5. Samples: 1318068. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:41:34,631][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:39,633][00294] Fps is (10 sec: 819.2, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 5271552. Throughput: 0: 309.9. Samples: 1318756. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:41:39,636][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:44,629][00294] Fps is (10 sec: 819.2, 60 sec: 1228.9, 300 sec: 1319.1). Total num frames: 5275648. Throughput: 0: 283.3. Samples: 1320096. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:41:44,638][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:49,362][14527] Updated weights for policy 0, policy_version 1290 (0.0050) [2023-07-24 01:41:49,469][14524] DAMAGECOUNT value on done: 1685.0 [2023-07-24 01:41:49,476][14524] Sum rewards: -4.456, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.248', 'AMMO2': '0.012', 'WEAPON1': '0.020', 'AMMO5': '0.025', 'AMMO4': '0.062', 'AMMO3': '0.128', 'weapon7': '0.134', 'weapon5': '0.148', 'WEAPON7': '0.200', 'AMMO6': '0.200', 'AMMO7': '0.200', 'HITCOUNT': '0.240', 'WEAPON4': '0.250', 'WEAPON5': '0.550', 'weapon4': '0.644', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.915', 'weapon2': '0.940', 'weapon3': '1.424', 'FRAGCOUNT': '2.000'} [2023-07-24 01:41:49,586][14528] DAMAGECOUNT value on done: 1239.0 [2023-07-24 01:41:49,594][14528] Sum rewards: -2.918, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.724', 'AMMO2': '0.006', 'AMMO4': '0.032', 'WEAPON4': '0.100', 'AMMO3': '0.124', 'HITCOUNT': '0.130', 'weapon4': '0.410', 'DAMAGECOUNT': '0.420', 'ARMOR': '0.552', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon2': '1.096', 'weapon3': '1.686'} [2023-07-24 01:41:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5283840. Throughput: 0: 279.8. Samples: 1321664. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:41:49,638][00294] Avg episode reward: [(0, '-3.822')] [2023-07-24 01:41:53,638][14532] DAMAGECOUNT value on done: 1723.0 [2023-07-24 01:41:53,658][14532] Sum rewards: -1.795, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-0.430', 'AMMO5': '0.003', 'ARMOR': '0.008', 'weapon5': '0.012', 'weapon4': '0.018', 'WEAPON1': '0.020', 'AMMO2': '0.020', 'WEAPON5': '0.050', 'WEAPON4': '0.100', 'AMMO4': '0.100', 'AMMO3': '0.108', 'HITCOUNT': '0.130', 'DAMAGECOUNT': '0.540', 'WEAPON3': '0.600', 'FRAGCOUNT': '1.000', 'weapon3': '1.260', 'weapon2': '1.416'} [2023-07-24 01:41:54,348][14524] DAMAGECOUNT value on done: 2103.0 [2023-07-24 01:41:54,352][14524] Sum rewards: -2.557, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.180', 'WEAPON1': '0.010', 'AMMO5': '0.012', 'AMMO2': '0.018', 'ARMOR': '0.036', 'AMMO4': '0.089', 'HITCOUNT': '0.170', 'AMMO3': '0.184', 'WEAPON4': '0.200', 'WEAPON5': '0.250', 'weapon5': '0.284', 'weapon4': '0.286', 'DAMAGECOUNT': '0.540', 'WEAPON3': '1.000', 'weapon2': '1.020', 'weapon3': '1.774', 'FRAGCOUNT': '4.000'} [2023-07-24 01:41:54,409][14528] DAMAGECOUNT value on done: 1763.0 [2023-07-24 01:41:54,413][14528] Sum rewards: 0.220, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.292', 'AMMO5': '0.007', 'AMMO2': '0.026', 'weapon4': '0.078', 'AMMO3': '0.106', 'AMMO4': '0.127', 'WEAPON5': '0.150', 'weapon5': '0.182', 'WEAPON4': '0.200', 'HITCOUNT': '0.410', 'WEAPON3': '0.650', 'DAMAGECOUNT': '1.287', 'weapon2': '1.356', 'weapon3': '1.932', 'FRAGCOUNT': '3.000'} [2023-07-24 01:41:54,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 5292032. Throughput: 0: 288.6. Samples: 1322896. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:41:54,633][00294] Avg episode reward: [(0, '-3.750')] [2023-07-24 01:41:58,878][14532] DAMAGECOUNT value on done: 1646.0 [2023-07-24 01:41:59,629][00294] Fps is (10 sec: 1638.3, 60 sec: 1297.1, 300 sec: 1319.0). Total num frames: 5300224. Throughput: 0: 307.8. Samples: 1325500. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:41:59,631][00294] Avg episode reward: [(0, '-3.750')] [2023-07-24 01:41:59,647][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001294_5300224.pth... [2023-07-24 01:41:59,881][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001218_4988928.pth [2023-07-24 01:41:59,969][14528] DAMAGECOUNT value on done: 1331.0 [2023-07-24 01:41:59,970][14528] Sum rewards: 0.743, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-1.327', 'AMMO2': '0.015', 'AMMO3': '0.074', 'weapon7': '0.074', 'AMMO4': '0.076', 'ARMOR': '0.096', 'AMMO6': '0.120', 'AMMO7': '0.120', 'HITCOUNT': '0.150', 'WEAPON4': '0.200', 'WEAPON7': '0.200', 'weapon4': '0.238', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.600', 'weapon3': '1.164', 'weapon2': '1.642', 'FRAGCOUNT': '2.000'} [2023-07-24 01:42:00,004][14524] DAMAGECOUNT value on done: 1398.0 [2023-07-24 01:42:00,008][14524] Sum rewards: -0.818, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.790', 'weapon7': '0.006', 'AMMO5': '0.009', 'AMMO2': '0.021', 'weapon5': '0.064', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO4': '0.104', 'AMMO3': '0.133', 'HITCOUNT': '0.160', 'WEAPON5': '0.200', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.480', 'WEAPON3': '0.600', 'weapon3': '0.962', 'weapon2': '1.034', 'weapon4': '1.148', 'FRAGCOUNT': '3.000'} [2023-07-24 01:42:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5304320. Throughput: 0: 297.6. Samples: 1327412. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:42:04,633][00294] Avg episode reward: [(0, '-3.670')] [2023-07-24 01:42:04,915][14531] DAMAGECOUNT value on done: 1995.0 [2023-07-24 01:42:04,932][14531] Sum rewards: -4.063, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.430', 'AMMO5': '0.013', 'AMMO2': '0.041', 'weapon5': '0.062', 'HITCOUNT': '0.170', 'AMMO3': '0.187', 'AMMO4': '0.207', 'WEAPON5': '0.250', 'weapon4': '0.296', 'WEAPON4': '0.300', 'ARMOR': '0.445', 'DAMAGECOUNT': '0.690', 'weapon2': '1.040', 'WEAPON3': '1.100', 'weapon3': '1.816', 'FRAGCOUNT': '3.000'} [2023-07-24 01:42:05,929][14532] DAMAGECOUNT value on done: 945.0 [2023-07-24 01:42:07,103][14528] DAMAGECOUNT value on done: 1476.0 [2023-07-24 01:42:07,104][14528] Sum rewards: -4.946, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.199', 'AMMO5': '0.020', 'AMMO2': '0.041', 'HITCOUNT': '0.060', 'weapon7': '0.068', 'AMMO3': '0.095', 'weapon5': '0.140', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON7': '0.200', 'AMMO4': '0.205', 'DAMAGECOUNT': '0.255', 'WEAPON5': '0.400', 'ARMOR': '0.444', 'WEAPON4': '0.550', 'WEAPON3': '0.550', 'weapon4': '0.866', 'weapon3': '0.868', 'weapon2': '0.920', 'FRAGCOUNT': '1.000'} [2023-07-24 01:42:07,237][14524] DAMAGECOUNT value on done: 1179.0 [2023-07-24 01:42:08,282][14529] DAMAGECOUNT value on done: 2089.0 [2023-07-24 01:42:08,291][14529] Sum rewards: 4.484, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.104', 'AMMO5': '0.005', 'AMMO2': '0.022', 'ARMOR': '0.048', 'WEAPON1': '0.050', 'WEAPON5': '0.100', 'AMMO3': '0.106', 'AMMO4': '0.111', 'WEAPON4': '0.200', 'weapon4': '0.262', 'HITCOUNT': '0.460', 'WEAPON3': '0.650', 'weapon2': '1.176', 'weapon3': '1.918', 'DAMAGECOUNT': '1.980', 'FRAGCOUNT': '5.000'} [2023-07-24 01:42:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1305.2). Total num frames: 5308416. Throughput: 0: 290.0. Samples: 1328252. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:42:09,631][00294] Avg episode reward: [(0, '-3.570')] [2023-07-24 01:42:12,490][14531] DAMAGECOUNT value on done: 1356.0 [2023-07-24 01:42:13,504][14532] DAMAGECOUNT value on done: 1241.0 [2023-07-24 01:42:13,506][14532] Sum rewards: -3.018, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.035', 'AMMO2': '0.003', 'AMMO5': '0.007', 'AMMO4': '0.015', 'WEAPON1': '0.020', 'ARMOR': '0.024', 'WEAPON4': '0.050', 'weapon7': '0.074', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.116', 'WEAPON5': '0.150', 'HITCOUNT': '0.200', 'weapon4': '0.364', 'DAMAGECOUNT': '0.468', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'weapon2': '1.010', 'weapon3': '1.716'} [2023-07-24 01:42:14,079][14528] DAMAGECOUNT value on done: 1544.0 [2023-07-24 01:42:14,096][14528] Sum rewards: -4.483, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.095', 'AMMO5': '0.003', 'AMMO2': '0.016', 'weapon5': '0.030', 'weapon7': '0.040', 'WEAPON5': '0.050', 'AMMO4': '0.081', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'HITCOUNT': '0.120', 'AMMO3': '0.181', 'WEAPON4': '0.250', 'weapon4': '0.568', 'DAMAGECOUNT': '0.651', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon3': '1.134', 'weapon2': '1.138'} [2023-07-24 01:42:14,295][14524] DAMAGECOUNT value on done: 1478.0 [2023-07-24 01:42:14,301][14524] Sum rewards: -7.380, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-2.678', 'AMMO5': '0.003', 'ARMOR': '0.008', 'weapon5': '0.024', 'AMMO2': '0.028', 'WEAPON5': '0.050', 'weapon7': '0.062', 'AMMO4': '0.139', 'weapon4': '0.196', 'WEAPON4': '0.200', 'AMMO3': '0.202', 'HITCOUNT': '0.220', 'AMMO6': '0.360', 'AMMO7': '0.360', 'WEAPON7': '0.400', 'DAMAGECOUNT': '0.690', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.150', 'weapon2': '1.350', 'weapon3': '1.606'} [2023-07-24 01:42:14,629][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1319.0). Total num frames: 5316608. Throughput: 0: 293.4. Samples: 1329944. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:42:14,634][00294] Avg episode reward: [(0, '-3.534')] [2023-07-24 01:42:16,480][14529] DAMAGECOUNT value on done: 1254.0 [2023-07-24 01:42:16,481][14529] Sum rewards: -7.052, reward structure: {'DEATHCOUNT': '-9.750', 'FRAGCOUNT': '-1.500', 'HEALTH': '-1.410', 'AMMO5': '0.013', 'AMMO2': '0.028', 'WEAPON1': '0.030', 'weapon5': '0.050', 'ARMOR': '0.060', 'AMMO3': '0.125', 'AMMO4': '0.140', 'HITCOUNT': '0.170', 'WEAPON5': '0.250', 'WEAPON4': '0.350', 'DAMAGECOUNT': '0.540', 'weapon4': '0.566', 'WEAPON3': '0.750', 'weapon2': '0.892', 'weapon3': '1.644'} [2023-07-24 01:42:18,438][14531] DAMAGECOUNT value on done: 1453.0 [2023-07-24 01:42:18,441][14531] Sum rewards: -1.254, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.033', 'AMMO5': '0.013', 'AMMO2': '0.019', 'WEAPON1': '0.020', 'ARMOR': '0.032', 'AMMO4': '0.096', 'AMMO3': '0.098', 'weapon5': '0.108', 'HITCOUNT': '0.150', 'WEAPON5': '0.200', 'WEAPON4': '0.250', 'DAMAGECOUNT': '0.525', 'WEAPON3': '0.600', 'weapon2': '0.802', 'weapon4': '0.866', 'weapon3': '1.500', 'FRAGCOUNT': '2.000'} [2023-07-24 01:42:18,886][14527] Updated weights for policy 0, policy_version 1300 (0.0026) [2023-07-24 01:42:19,226][14532] DAMAGECOUNT value on done: 1327.0 [2023-07-24 01:42:19,232][14532] Sum rewards: -5.753, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.800', 'AMMO5': '0.005', 'AMMO2': '0.023', 'WEAPON1': '0.040', 'HITCOUNT': '0.090', 'WEAPON5': '0.100', 'weapon5': '0.104', 'AMMO4': '0.112', 'AMMO3': '0.152', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.270', 'weapon4': '0.404', 'WEAPON3': '0.950', 'weapon2': '1.144', 'weapon3': '1.704', 'FRAGCOUNT': '2.000'} [2023-07-24 01:42:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 5324800. Throughput: 0: 306.0. Samples: 1331840. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:42:19,630][00294] Avg episode reward: [(0, '-3.667')] [2023-07-24 01:42:19,964][14528] DAMAGECOUNT value on done: 1441.0 [2023-07-24 01:42:19,973][14528] Sum rewards: -4.853, reward structure: {'DEATHCOUNT': '-14.250', 'HEALTH': '-1.830', 'weapon4': '0.012', 'AMMO5': '0.012', 'AMMO2': '0.014', 'WEAPON1': '0.020', 'WEAPON4': '0.050', 'AMMO4': '0.069', 'weapon5': '0.116', 'AMMO3': '0.188', 'WEAPON5': '0.250', 'HITCOUNT': '0.350', 'weapon2': '0.546', 'WEAPON3': '1.300', 'DAMAGECOUNT': '1.410', 'weapon3': '2.890', 'FRAGCOUNT': '4.000'} [2023-07-24 01:42:20,080][14524] DAMAGECOUNT value on done: 2364.0 [2023-07-24 01:42:20,086][14524] Sum rewards: -2.872, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.872', 'AMMO5': '0.006', 'WEAPON1': '0.010', 'AMMO2': '0.010', 'ARMOR': '0.028', 'AMMO4': '0.051', 'AMMO3': '0.123', 'WEAPON5': '0.150', 'WEAPON4': '0.200', 'HITCOUNT': '0.240', 'weapon5': '0.304', 'weapon4': '0.310', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.924', 'weapon3': '1.364', 'weapon2': '1.380', 'FRAGCOUNT': '1.500'} [2023-07-24 01:42:21,531][14529] DAMAGECOUNT value on done: 1216.0 [2023-07-24 01:42:21,543][14529] Sum rewards: -0.861, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.360', 'AMMO5': '0.003', 'weapon5': '0.006', 'WEAPON1': '0.010', 'AMMO2': '0.014', 'WEAPON5': '0.050', 'AMMO4': '0.071', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'weapon7': '0.102', 'AMMO3': '0.108', 'HITCOUNT': '0.240', 'ARMOR': '0.496', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.993', 'weapon2': '1.636', 'weapon3': '1.820', 'FRAGCOUNT': '2.000'} [2023-07-24 01:42:23,260][14531] DAMAGECOUNT value on done: 1490.0 [2023-07-24 01:42:23,264][14531] Sum rewards: -9.440, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.794', 'FRAGCOUNT': '-1.500', 'AMMO5': '0.016', 'ARMOR': '0.016', 'AMMO2': '0.020', 'WEAPON1': '0.030', 'HITCOUNT': '0.090', 'AMMO4': '0.099', 'AMMO3': '0.117', 'WEAPON4': '0.200', 'weapon4': '0.218', 'DAMAGECOUNT': '0.240', 'WEAPON5': '0.350', 'weapon5': '0.480', 'WEAPON3': '0.650', 'weapon3': '1.194', 'weapon2': '1.384'} [2023-07-24 01:42:23,951][14532] DAMAGECOUNT value on done: 1473.0 [2023-07-24 01:42:24,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1228.8, 300 sec: 1319.1). Total num frames: 5332992. Throughput: 0: 319.7. Samples: 1333144. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:42:24,631][00294] Avg episode reward: [(0, '-3.637')] [2023-07-24 01:42:24,642][14528] DAMAGECOUNT value on done: 1405.0 [2023-07-24 01:42:24,644][14528] Sum rewards: -2.519, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.910', 'AMMO5': '0.010', 'AMMO2': '0.011', 'ARMOR': '0.036', 'AMMO4': '0.054', 'HITCOUNT': '0.060', 'weapon5': '0.096', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.123', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.189', 'weapon4': '0.236', 'WEAPON3': '0.450', 'weapon3': '0.922', 'FRAGCOUNT': '1.000', 'weapon2': '2.104'} [2023-07-24 01:42:24,705][14524] DAMAGECOUNT value on done: 1355.0 [2023-07-24 01:42:24,718][14524] Sum rewards: -7.848, reward structure: {'DEATHCOUNT': '-13.500', 'HEALTH': '-1.730', 'AMMO5': '0.010', 'WEAPON1': '0.020', 'AMMO2': '0.023', 'ARMOR': '0.044', 'weapon5': '0.054', 'AMMO4': '0.115', 'AMMO3': '0.180', 'WEAPON5': '0.200', 'HITCOUNT': '0.210', 'WEAPON4': '0.250', 'weapon4': '0.372', 'DAMAGECOUNT': '0.906', 'FRAGCOUNT': '1.000', 'WEAPON3': '1.050', 'weapon2': '1.172', 'weapon3': '1.776'} [2023-07-24 01:42:26,211][14529] DAMAGECOUNT value on done: 1800.0 [2023-07-24 01:42:27,453][14530] DAMAGECOUNT value on done: 1650.0 [2023-07-24 01:42:27,459][14530] Sum rewards: -8.773, reward structure: {'DEATHCOUNT': '-15.750', 'HEALTH': '-2.790', 'weapon7': '0.010', 'AMMO5': '0.011', 'WEAPON1': '0.020', 'AMMO2': '0.042', 'weapon5': '0.052', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.177', 'HITCOUNT': '0.190', 'AMMO4': '0.208', 'WEAPON5': '0.250', 'WEAPON4': '0.300', 'weapon4': '0.404', 'DAMAGECOUNT': '0.690', 'weapon2': '1.060', 'WEAPON3': '1.100', 'weapon3': '1.952', 'FRAGCOUNT': '3.000'} [2023-07-24 01:42:28,330][14531] DAMAGECOUNT value on done: 1049.0 [2023-07-24 01:42:28,331][14531] Sum rewards: -4.678, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.280', 'AMMO5': '0.012', 'ARMOR': '0.012', 'AMMO2': '0.025', 'weapon5': '0.056', 'AMMO4': '0.126', 'HITCOUNT': '0.150', 'WEAPON5': '0.150', 'AMMO3': '0.156', 'WEAPON4': '0.250', 'weapon4': '0.422', 'DAMAGECOUNT': '0.498', 'WEAPON3': '0.850', 'weapon2': '1.274', 'weapon3': '1.870', 'FRAGCOUNT': '2.000'} [2023-07-24 01:42:29,204][14532] DAMAGECOUNT value on done: 1486.0 [2023-07-24 01:42:29,204][14532] Sum rewards: -2.588, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.558', 'ARMOR': '0.008', 'AMMO2': '0.015', 'AMMO5': '0.020', 'WEAPON1': '0.020', 'AMMO4': '0.073', 'weapon5': '0.088', 'WEAPON4': '0.100', 'weapon4': '0.116', 'AMMO3': '0.171', 'HITCOUNT': '0.200', 'WEAPON5': '0.400', 'DAMAGECOUNT': '0.765', 'WEAPON3': '0.950', 'weapon2': '1.070', 'weapon3': '1.974', 'FRAGCOUNT': '2.000'} [2023-07-24 01:42:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5341184. Throughput: 0: 346.1. Samples: 1335672. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:42:29,631][00294] Avg episode reward: [(0, '-3.618')] [2023-07-24 01:42:30,309][14524] DAMAGECOUNT value on done: 1296.0 [2023-07-24 01:42:30,353][14528] DAMAGECOUNT value on done: 2157.0 [2023-07-24 01:42:30,356][14528] Sum rewards: -9.073, reward structure: {'DEATHCOUNT': '-10.500', 'FRAGCOUNT': '-3.500', 'HEALTH': '-1.328', 'AMMO2': '0.022', 'AMMO5': '0.029', 'WEAPON1': '0.050', 'AMMO4': '0.111', 'AMMO3': '0.143', 'weapon5': '0.184', 'WEAPON4': '0.200', 'HITCOUNT': '0.230', 'weapon4': '0.406', 'WEAPON5': '0.550', 'weapon2': '0.730', 'WEAPON3': '0.750', 'DAMAGECOUNT': '0.930', 'weapon3': '1.920'} [2023-07-24 01:42:34,090][14529] DAMAGECOUNT value on done: 1156.0 [2023-07-24 01:42:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5345280. Throughput: 0: 349.4. Samples: 1337388. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:42:34,631][00294] Avg episode reward: [(0, '-3.598')] [2023-07-24 01:42:35,087][14531] DAMAGECOUNT value on done: 1992.0 [2023-07-24 01:42:35,428][14530] DAMAGECOUNT value on done: 2115.0 [2023-07-24 01:42:35,432][14530] Sum rewards: -4.625, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.351', 'AMMO5': '0.003', 'AMMO2': '0.016', 'WEAPON1': '0.020', 'AMMO4': '0.080', 'WEAPON5': '0.100', 'AMMO3': '0.113', 'HITCOUNT': '0.130', 'weapon5': '0.152', 'WEAPON4': '0.200', 'weapon4': '0.248', 'WEAPON3': '0.550', 'weapon3': '0.768', 'DAMAGECOUNT': '0.963', 'FRAGCOUNT': '1.000', 'weapon2': '2.384'} [2023-07-24 01:42:36,205][14532] DAMAGECOUNT value on done: 1528.0 [2023-07-24 01:42:37,594][14526] DAMAGECOUNT value on done: 1560.0 [2023-07-24 01:42:37,605][14526] Sum rewards: 0.280, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.740', 'AMMO2': '0.018', 'AMMO5': '0.022', 'ARMOR': '0.044', 'weapon7': '0.044', 'weapon5': '0.082', 'AMMO4': '0.089', 'AMMO3': '0.133', 'HITCOUNT': '0.170', 'WEAPON4': '0.250', 'WEAPON5': '0.350', 'AMMO6': '0.360', 'AMMO7': '0.360', 'WEAPON7': '0.400', 'weapon4': '0.638', 'weapon2': '0.762', 'WEAPON3': '0.800', 'DAMAGECOUNT': '1.122', 'weapon3': '1.626', 'FRAGCOUNT': '3.000'} [2023-07-24 01:42:38,113][14525] DAMAGECOUNT value on done: 1250.0 [2023-07-24 01:42:38,115][14525] Sum rewards: -3.283, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.978', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.007', 'AMMO2': '0.016', 'weapon7': '0.016', 'WEAPON1': '0.020', 'weapon5': '0.058', 'AMMO4': '0.077', 'HITCOUNT': '0.090', 'AMMO3': '0.148', 'WEAPON5': '0.150', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON4': '0.300', 'DAMAGECOUNT': '0.327', 'ARMOR': '0.452', 'weapon2': '0.574', 'WEAPON3': '0.800', 'weapon3': '1.118', 'weapon4': '1.192'} [2023-07-24 01:42:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5349376. Throughput: 0: 340.0. Samples: 1338196. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:42:39,636][00294] Avg episode reward: [(0, '-3.587')] [2023-07-24 01:42:40,195][14529] DAMAGECOUNT value on done: 1725.0 [2023-07-24 01:42:40,197][14529] Sum rewards: -1.794, reward structure: {'DEATHCOUNT': '-6.000', 'FRAGCOUNT': '-0.500', 'HEALTH': '-0.378', 'AMMO2': '0.010', 'ARMOR': '0.012', 'AMMO5': '0.020', 'AMMO4': '0.048', 'AMMO3': '0.098', 'HITCOUNT': '0.100', 'weapon5': '0.104', 'WEAPON4': '0.150', 'WEAPON5': '0.300', 'weapon4': '0.312', 'DAMAGECOUNT': '0.408', 'WEAPON3': '0.550', 'weapon2': '1.344', 'weapon3': '1.628'} [2023-07-24 01:42:41,589][14530] DAMAGECOUNT value on done: 993.0 [2023-07-24 01:42:43,766][14531] DAMAGECOUNT value on done: 1515.0 [2023-07-24 01:42:43,767][14531] Sum rewards: -1.598, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-2.624', 'AMMO4': '-0.023', 'AMMO2': '-0.005', 'AMMO5': '0.018', 'WEAPON1': '0.020', 'ARMOR': '0.040', 'WEAPON4': '0.050', 'weapon7': '0.114', 'AMMO3': '0.116', 'AMMO6': '0.120', 'AMMO7': '0.120', 'HITCOUNT': '0.150', 'WEAPON7': '0.200', 'weapon4': '0.230', 'weapon5': '0.286', 'WEAPON5': '0.450', 'WEAPON3': '0.650', 'weapon3': '0.990', 'DAMAGECOUNT': '1.134', 'weapon2': '1.616', 'FRAGCOUNT': '3.000'} [2023-07-24 01:42:44,215][14526] DAMAGECOUNT value on done: 1391.0 [2023-07-24 01:42:44,221][14526] Sum rewards: -5.688, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.018', 'AMMO5': '0.023', 'AMMO2': '0.023', 'weapon5': '0.070', 'AMMO4': '0.116', 'HITCOUNT': '0.150', 'AMMO3': '0.166', 'weapon4': '0.258', 'WEAPON4': '0.300', 'WEAPON5': '0.400', 'ARMOR': '0.481', 'DAMAGECOUNT': '0.561', 'WEAPON3': '1.000', 'FRAGCOUNT': '1.000', 'weapon3': '1.276', 'weapon2': '1.756'} [2023-07-24 01:42:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5357568. Throughput: 0: 319.9. Samples: 1339896. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 01:42:44,637][00294] Avg episode reward: [(0, '-3.583')] [2023-07-24 01:42:45,244][14525] DAMAGECOUNT value on done: 1474.0 [2023-07-24 01:42:45,244][14525] Sum rewards: -3.085, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.270', 'AMMO2': '0.013', 'AMMO5': '0.020', 'AMMO4': '0.066', 'weapon7': '0.088', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'weapon5': '0.110', 'AMMO3': '0.119', 'HITCOUNT': '0.120', 'WEAPON4': '0.150', 'weapon4': '0.256', 'WEAPON5': '0.400', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.762', 'weapon3': '1.238', 'weapon2': '1.592', 'FRAGCOUNT': '2.000'} [2023-07-24 01:42:46,750][14529] DAMAGECOUNT value on done: 2266.0 [2023-07-24 01:42:46,754][14529] Sum rewards: -2.934, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.152', 'AMMO5': '0.008', 'WEAPON1': '0.020', 'AMMO2': '0.040', 'AMMO3': '0.106', 'WEAPON5': '0.150', 'HITCOUNT': '0.160', 'AMMO4': '0.200', 'weapon5': '0.246', 'WEAPON4': '0.500', 'ARMOR': '0.509', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.666', 'weapon2': '0.824', 'weapon4': '1.020', 'weapon3': '1.118', 'FRAGCOUNT': '2.000'} [2023-07-24 01:42:47,642][14530] DAMAGECOUNT value on done: 1330.0 [2023-07-24 01:42:47,644][14530] Sum rewards: -3.956, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.406', 'WEAPON1': '0.010', 'AMMO2': '0.015', 'AMMO5': '0.016', 'ARMOR': '0.020', 'AMMO4': '0.076', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'HITCOUNT': '0.120', 'AMMO3': '0.127', 'WEAPON4': '0.150', 'weapon7': '0.154', 'WEAPON5': '0.300', 'weapon4': '0.380', 'weapon5': '0.384', 'DAMAGECOUNT': '0.408', 'WEAPON3': '0.750', 'weapon2': '0.906', 'FRAGCOUNT': '1.000', 'weapon3': '1.334'} [2023-07-24 01:42:48,903][14527] Updated weights for policy 0, policy_version 1310 (0.0054) [2023-07-24 01:42:49,216][14531] DAMAGECOUNT value on done: 1614.0 [2023-07-24 01:42:49,219][14531] Sum rewards: -2.075, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.126', 'AMMO5': '0.007', 'WEAPON1': '0.020', 'AMMO2': '0.035', 'weapon5': '0.062', 'AMMO3': '0.095', 'WEAPON5': '0.150', 'WEAPON4': '0.150', 'AMMO4': '0.176', 'HITCOUNT': '0.210', 'ARMOR': '0.400', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.690', 'weapon4': '0.698', 'weapon2': '0.912', 'FRAGCOUNT': '1.000', 'weapon3': '1.796'} [2023-07-24 01:42:49,322][14526] DAMAGECOUNT value on done: 2124.0 [2023-07-24 01:42:49,324][14526] Sum rewards: -2.080, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.341', 'AMMO2': '0.019', 'AMMO5': '0.032', 'WEAPON1': '0.080', 'weapon4': '0.084', 'AMMO4': '0.094', 'AMMO3': '0.100', 'WEAPON4': '0.100', 'HITCOUNT': '0.160', 'weapon5': '0.270', 'DAMAGECOUNT': '0.552', 'WEAPON3': '0.650', 'WEAPON5': '0.650', 'FRAGCOUNT': '1.000', 'weapon2': '1.110', 'weapon3': '1.860'} [2023-07-24 01:42:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5365760. Throughput: 0: 326.6. Samples: 1342108. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:42:49,631][00294] Avg episode reward: [(0, '-3.530')] [2023-07-24 01:42:49,941][14525] DAMAGECOUNT value on done: 1025.0 [2023-07-24 01:42:51,729][14529] DAMAGECOUNT value on done: 2256.0 [2023-07-24 01:42:52,701][14530] DAMAGECOUNT value on done: 1327.0 [2023-07-24 01:42:52,703][14530] Sum rewards: -3.645, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.840', 'AMMO2': '0.001', 'AMMO4': '0.002', 'AMMO5': '0.010', 'WEAPON4': '0.050', 'weapon5': '0.078', 'weapon7': '0.098', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.116', 'HITCOUNT': '0.140', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.450', 'WEAPON3': '0.550', 'FRAGCOUNT': '1.000', 'weapon2': '1.408', 'weapon3': '1.792'} [2023-07-24 01:42:54,374][14526] DAMAGECOUNT value on done: 1562.0 [2023-07-24 01:42:54,376][14526] Sum rewards: -1.017, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.858', 'ARMOR': '0.004', 'AMMO2': '0.017', 'AMMO5': '0.019', 'WEAPON1': '0.020', 'weapon7': '0.028', 'HITCOUNT': '0.070', 'AMMO4': '0.084', 'AMMO3': '0.125', 'weapon5': '0.172', 'DAMAGECOUNT': '0.192', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON4': '0.250', 'WEAPON5': '0.350', 'weapon2': '0.378', 'FRAGCOUNT': '0.500', 'weapon4': '0.626', 'WEAPON3': '0.700', 'weapon3': '1.706'} [2023-07-24 01:42:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5373952. Throughput: 0: 336.8. Samples: 1343408. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:42:54,631][00294] Avg episode reward: [(0, '-3.578')] [2023-07-24 01:42:54,929][14525] DAMAGECOUNT value on done: 1220.0 [2023-07-24 01:42:54,934][14525] Sum rewards: -2.988, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.552', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.010', 'AMMO2': '0.025', 'AMMO3': '0.085', 'AMMO4': '0.125', 'weapon4': '0.150', 'HITCOUNT': '0.180', 'WEAPON5': '0.200', 'WEAPON4': '0.200', 'weapon5': '0.306', 'ARMOR': '0.472', 'WEAPON3': '0.500', 'DAMAGECOUNT': '0.525', 'weapon3': '1.284', 'weapon2': '1.502'} [2023-07-24 01:42:58,124][14530] DAMAGECOUNT value on done: 2206.0 [2023-07-24 01:42:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5378048. Throughput: 0: 350.0. Samples: 1345692. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:42:59,633][00294] Avg episode reward: [(0, '-3.561')] [2023-07-24 01:43:00,310][14526] DAMAGECOUNT value on done: 1158.0 [2023-07-24 01:43:00,314][14526] Sum rewards: -2.994, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-2.000', 'AMMO5': '0.010', 'AMMO2': '0.051', 'ARMOR': '0.112', 'AMMO3': '0.122', 'WEAPON5': '0.200', 'AMMO4': '0.253', 'weapon5': '0.260', 'HITCOUNT': '0.360', 'WEAPON4': '0.500', 'weapon4': '0.514', 'weapon2': '0.728', 'WEAPON3': '0.750', 'FRAGCOUNT': '1.000', 'DAMAGECOUNT': '1.332', 'weapon3': '1.814'} [2023-07-24 01:43:01,455][14525] DAMAGECOUNT value on done: 1480.0 [2023-07-24 01:43:01,468][14525] Sum rewards: -4.487, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.512', 'AMMO5': '0.022', 'AMMO2': '0.024', 'WEAPON1': '0.030', 'ARMOR': '0.072', 'AMMO4': '0.122', 'AMMO3': '0.150', 'weapon5': '0.152', 'WEAPON4': '0.200', 'HITCOUNT': '0.230', 'WEAPON5': '0.300', 'weapon4': '0.312', 'WEAPON3': '0.900', 'DAMAGECOUNT': '0.930', 'weapon2': '1.120', 'weapon3': '1.710', 'FRAGCOUNT': '2.000'} [2023-07-24 01:43:04,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1277.4). Total num frames: 5382144. Throughput: 0: 345.8. Samples: 1347400. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:43:04,631][00294] Avg episode reward: [(0, '-3.642')] [2023-07-24 01:43:05,647][14530] DAMAGECOUNT value on done: 1220.0 [2023-07-24 01:43:05,650][14530] Sum rewards: -3.814, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.766', 'AMMO5': '0.003', 'AMMO2': '0.030', 'WEAPON5': '0.050', 'ARMOR': '0.072', 'AMMO3': '0.076', 'HITCOUNT': '0.090', 'AMMO4': '0.148', 'WEAPON4': '0.300', 'DAMAGECOUNT': '0.342', 'WEAPON3': '0.350', 'weapon4': '0.800', 'weapon3': '0.952', 'FRAGCOUNT': '1.000', 'weapon2': '1.740'} [2023-07-24 01:43:08,318][14526] DAMAGECOUNT value on done: 1633.0 [2023-07-24 01:43:08,324][14526] Sum rewards: -3.340, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.470', 'AMMO2': '0.008', 'AMMO5': '0.018', 'AMMO4': '0.041', 'HITCOUNT': '0.130', 'WEAPON4': '0.150', 'AMMO3': '0.153', 'weapon5': '0.286', 'DAMAGECOUNT': '0.375', 'WEAPON5': '0.400', 'ARMOR': '0.408', 'weapon4': '0.616', 'WEAPON3': '0.850', 'weapon2': '0.998', 'FRAGCOUNT': '1.000', 'weapon3': '1.446'} [2023-07-24 01:43:09,471][14525] DAMAGECOUNT value on done: 1183.0 [2023-07-24 01:43:09,474][14525] Sum rewards: -3.285, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.103', 'AMMO5': '0.015', 'AMMO2': '0.017', 'weapon4': '0.018', 'WEAPON4': '0.050', 'weapon5': '0.078', 'AMMO4': '0.084', 'AMMO3': '0.149', 'WEAPON5': '0.150', 'HITCOUNT': '0.230', 'DAMAGECOUNT': '0.813', 'WEAPON3': '0.850', 'weapon2': '1.058', 'weapon3': '2.306', 'FRAGCOUNT': '4.000'} [2023-07-24 01:43:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 5390336. Throughput: 0: 335.4. Samples: 1348236. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:43:09,637][00294] Avg episode reward: [(0, '-3.676')] [2023-07-24 01:43:13,724][14530] DAMAGECOUNT value on done: 1428.0 [2023-07-24 01:43:13,733][14530] Sum rewards: -1.315, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.695', 'WEAPON1': '0.010', 'AMMO5': '0.013', 'AMMO2': '0.031', 'ARMOR': '0.086', 'weapon5': '0.106', 'AMMO3': '0.150', 'AMMO4': '0.154', 'HITCOUNT': '0.190', 'WEAPON5': '0.300', 'WEAPON4': '0.350', 'weapon4': '0.516', 'DAMAGECOUNT': '0.765', 'WEAPON3': '0.800', 'weapon2': '0.924', 'weapon3': '1.734', 'FRAGCOUNT': '3.000'} [2023-07-24 01:43:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5398528. Throughput: 0: 317.2. Samples: 1349948. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:43:14,634][00294] Avg episode reward: [(0, '-3.664')] [2023-07-24 01:43:15,305][14526] DAMAGECOUNT value on done: 1849.0 [2023-07-24 01:43:15,311][14526] Sum rewards: -3.116, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.673', 'ARMOR': '0.008', 'WEAPON1': '0.010', 'weapon7': '0.016', 'weapon5': '0.022', 'AMMO5': '0.022', 'AMMO2': '0.044', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'HITCOUNT': '0.130', 'AMMO3': '0.150', 'AMMO4': '0.221', 'WEAPON5': '0.250', 'WEAPON4': '0.300', 'weapon4': '0.496', 'DAMAGECOUNT': '0.561', 'WEAPON3': '0.750', 'weapon2': '1.272', 'weapon3': '1.504', 'FRAGCOUNT': '2.000'} [2023-07-24 01:43:16,021][14525] DAMAGECOUNT value on done: 2129.0 [2023-07-24 01:43:16,023][14525] Sum rewards: -1.997, reward structure: {'DEATHCOUNT': '-12.000', 'HEALTH': '-1.613', 'AMMO2': '0.006', 'WEAPON1': '0.010', 'AMMO5': '0.016', 'AMMO4': '0.029', 'WEAPON4': '0.050', 'weapon7': '0.106', 'weapon4': '0.138', 'AMMO3': '0.186', 'AMMO6': '0.220', 'AMMO7': '0.220', 'weapon5': '0.238', 'WEAPON7': '0.300', 'HITCOUNT': '0.340', 'WEAPON5': '0.350', 'WEAPON3': '1.000', 'weapon2': '1.110', 'DAMAGECOUNT': '1.335', 'weapon3': '1.962', 'FRAGCOUNT': '4.000'} [2023-07-24 01:43:19,495][14527] Updated weights for policy 0, policy_version 1320 (0.0034) [2023-07-24 01:43:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5406720. Throughput: 0: 335.9. Samples: 1352504. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:43:19,633][00294] Avg episode reward: [(0, '-3.617')] [2023-07-24 01:43:20,576][14526] DAMAGECOUNT value on done: 1569.0 [2023-07-24 01:43:20,579][14526] Sum rewards: -5.071, reward structure: {'DEATHCOUNT': '-8.250', 'FRAGCOUNT': '-1.500', 'HEALTH': '-1.330', 'AMMO2': '0.016', 'AMMO5': '0.018', 'WEAPON1': '0.020', 'AMMO4': '0.077', 'weapon7': '0.106', 'AMMO3': '0.129', 'HITCOUNT': '0.130', 'WEAPON4': '0.150', 'weapon5': '0.160', 'AMMO6': '0.200', 'AMMO7': '0.200', 'WEAPON7': '0.200', 'WEAPON5': '0.300', 'weapon4': '0.380', 'DAMAGECOUNT': '0.405', 'WEAPON3': '0.850', 'weapon2': '1.030', 'weapon3': '1.638'} [2023-07-24 01:43:21,096][14525] DAMAGECOUNT value on done: 1556.0 [2023-07-24 01:43:21,107][14525] Sum rewards: -4.344, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.742', 'WEAPON1': '0.010', 'AMMO2': '0.016', 'AMMO5': '0.024', 'AMMO4': '0.081', 'HITCOUNT': '0.110', 'AMMO3': '0.145', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.294', 'weapon5': '0.398', 'WEAPON5': '0.400', 'ARMOR': '0.400', 'weapon4': '0.618', 'WEAPON3': '0.750', 'weapon2': '0.886', 'FRAGCOUNT': '1.000', 'weapon3': '1.566'} [2023-07-24 01:43:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5414912. Throughput: 0: 346.9. Samples: 1353808. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:43:24,637][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:43:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5419008. Throughput: 0: 354.3. Samples: 1355840. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:43:29,630][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:43:34,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5427200. Throughput: 0: 343.6. Samples: 1357568. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:43:34,632][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:43:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5431296. Throughput: 0: 334.3. Samples: 1358452. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:43:39,636][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:43:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5435392. Throughput: 0: 327.6. Samples: 1360432. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:43:44,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:43:49,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5443584. Throughput: 0: 338.8. Samples: 1362644. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:43:49,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:43:50,793][14527] Updated weights for policy 0, policy_version 1330 (0.0042) [2023-07-24 01:43:54,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5451776. Throughput: 0: 338.1. Samples: 1363452. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:43:54,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:43:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5455872. Throughput: 0: 330.8. Samples: 1364832. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0) [2023-07-24 01:43:59,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:43:59,653][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001332_5455872.pth... [2023-07-24 01:43:59,924][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001256_5144576.pth [2023-07-24 01:44:04,631][00294] Fps is (10 sec: 819.0, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 5459968. Throughput: 0: 303.1. Samples: 1366144. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:44:04,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 5464064. Throughput: 0: 289.9. Samples: 1366852. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:44:09,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:14,628][00294] Fps is (10 sec: 819.4, 60 sec: 1160.5, 300 sec: 1291.3). Total num frames: 5468160. Throughput: 0: 279.6. Samples: 1368420. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:44:14,637][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1160.5, 300 sec: 1305.2). Total num frames: 5476352. Throughput: 0: 292.8. Samples: 1370744. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:44:19,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1160.5, 300 sec: 1305.2). Total num frames: 5484544. Throughput: 0: 302.8. Samples: 1372076. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) [2023-07-24 01:44:24,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:25,261][14527] Updated weights for policy 0, policy_version 1340 (0.0041) [2023-07-24 01:44:29,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5492736. Throughput: 0: 308.5. Samples: 1374316. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:44:29,630][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:34,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5500928. Throughput: 0: 298.3. Samples: 1376068. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:44:34,632][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 5505024. Throughput: 0: 300.0. Samples: 1376952. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:44:39,634][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:44,628][00294] Fps is (10 sec: 819.2, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 5509120. Throughput: 0: 307.8. Samples: 1378684. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) [2023-07-24 01:44:44,641][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5521408. Throughput: 0: 337.7. Samples: 1381340. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:44:49,635][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:54,510][14527] Updated weights for policy 0, policy_version 1350 (0.0044) [2023-07-24 01:44:54,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5529600. Throughput: 0: 350.8. Samples: 1382636. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:44:54,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:44:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5533696. Throughput: 0: 359.0. Samples: 1384576. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:44:59,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:04,636][00294] Fps is (10 sec: 1227.9, 60 sec: 1365.2, 300 sec: 1305.1). Total num frames: 5541888. Throughput: 0: 345.3. Samples: 1386284. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:45:04,641][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5541888. Throughput: 0: 334.9. Samples: 1387148. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:45:09,635][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:14,628][00294] Fps is (10 sec: 1229.7, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 5554176. Throughput: 0: 329.3. Samples: 1389136. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:45:14,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:19,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 5562368. Throughput: 0: 348.9. Samples: 1391768. Policy #0 lag: (min: 0.0, avg: 1.0, max: 4.0) [2023-07-24 01:45:19,634][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:24,631][00294] Fps is (10 sec: 1228.5, 60 sec: 1365.3, 300 sec: 1291.3). Total num frames: 5566464. Throughput: 0: 355.4. Samples: 1392948. Policy #0 lag: (min: 0.0, avg: 1.0, max: 4.0) [2023-07-24 01:45:24,638][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:25,534][14527] Updated weights for policy 0, policy_version 1360 (0.0050) [2023-07-24 01:45:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5574656. Throughput: 0: 354.5. Samples: 1394636. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:45:29,632][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:34,628][00294] Fps is (10 sec: 1638.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5582848. Throughput: 0: 333.9. Samples: 1396364. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:45:34,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5582848. Throughput: 0: 323.8. Samples: 1397208. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:45:39,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:44,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 5595136. Throughput: 0: 335.0. Samples: 1399652. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:45:44,642][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:49,632][00294] Fps is (10 sec: 2047.2, 60 sec: 1365.2, 300 sec: 1305.1). Total num frames: 5603328. Throughput: 0: 353.6. Samples: 1402196. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:45:49,635][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5607424. Throughput: 0: 353.0. Samples: 1403032. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:45:54,634][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:55,072][14527] Updated weights for policy 0, policy_version 1370 (0.0023) [2023-07-24 01:45:59,628][00294] Fps is (10 sec: 1229.3, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5615616. Throughput: 0: 347.4. Samples: 1404768. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:45:59,636][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:45:59,650][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001371_5615616.pth... [2023-07-24 01:45:59,920][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001294_5300224.pth [2023-07-24 01:46:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.2, 300 sec: 1291.3). Total num frames: 5619712. Throughput: 0: 326.8. Samples: 1406472. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:46:04,637][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 5627904. Throughput: 0: 322.3. Samples: 1407452. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:46:09,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:14,628][00294] Fps is (10 sec: 1638.5, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5636096. Throughput: 0: 343.2. Samples: 1410080. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:46:14,635][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:19,629][00294] Fps is (10 sec: 1228.7, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5640192. Throughput: 0: 341.1. Samples: 1411712. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:46:19,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:24,632][00294] Fps is (10 sec: 818.9, 60 sec: 1297.0, 300 sec: 1291.3). Total num frames: 5644288. Throughput: 0: 337.2. Samples: 1412384. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:46:24,634][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:29,587][14527] Updated weights for policy 0, policy_version 1380 (0.0063) [2023-07-24 01:46:29,628][00294] Fps is (10 sec: 1228.9, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5652480. Throughput: 0: 312.5. Samples: 1413716. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:46:29,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:34,634][00294] Fps is (10 sec: 819.0, 60 sec: 1160.4, 300 sec: 1291.3). Total num frames: 5652480. Throughput: 0: 285.6. Samples: 1415048. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:46:34,637][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:39,631][00294] Fps is (10 sec: 819.0, 60 sec: 1297.0, 300 sec: 1305.2). Total num frames: 5660672. Throughput: 0: 281.0. Samples: 1415680. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:46:39,638][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:44,628][00294] Fps is (10 sec: 1639.3, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5668864. Throughput: 0: 285.3. Samples: 1417608. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:46:44,638][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:49,628][00294] Fps is (10 sec: 1638.8, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 5677056. Throughput: 0: 306.4. Samples: 1420260. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:46:49,638][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1291.3). Total num frames: 5681152. Throughput: 0: 311.3. Samples: 1421460. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:46:54,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:46:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5689344. Throughput: 0: 291.6. Samples: 1423200. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:46:59,634][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:00,948][14527] Updated weights for policy 0, policy_version 1390 (0.0026) [2023-07-24 01:47:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5693440. Throughput: 0: 292.9. Samples: 1424892. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) [2023-07-24 01:47:04,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5701632. Throughput: 0: 297.7. Samples: 1425780. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:47:09,634][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:14,628][00294] Fps is (10 sec: 1638.3, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5709824. Throughput: 0: 319.4. Samples: 1428088. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:47:14,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5718016. Throughput: 0: 348.8. Samples: 1430740. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:47:19,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:24,631][00294] Fps is (10 sec: 1638.0, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 5726208. Throughput: 0: 354.9. Samples: 1431652. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:47:24,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5730304. Throughput: 0: 349.9. Samples: 1433352. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:47:29,634][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:31,783][14527] Updated weights for policy 0, policy_version 1400 (0.0022) [2023-07-24 01:47:34,628][00294] Fps is (10 sec: 819.4, 60 sec: 1365.5, 300 sec: 1305.2). Total num frames: 5734400. Throughput: 0: 328.7. Samples: 1435052. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:47:34,635][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.4, 300 sec: 1305.2). Total num frames: 5742592. Throughput: 0: 321.2. Samples: 1435916. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:47:39,631][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5750784. Throughput: 0: 341.8. Samples: 1438580. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:47:44,630][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:49,630][00294] Fps is (10 sec: 1638.2, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5758976. Throughput: 0: 355.7. Samples: 1440900. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:47:49,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5763072. Throughput: 0: 355.0. Samples: 1441756. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:47:54,632][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:59,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5771264. Throughput: 0: 342.0. Samples: 1443480. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:47:59,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:47:59,649][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001409_5771264.pth... [2023-07-24 01:47:59,908][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001332_5455872.pth [2023-07-24 01:48:03,265][14527] Updated weights for policy 0, policy_version 1410 (0.0031) [2023-07-24 01:48:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5775360. Throughput: 0: 320.6. Samples: 1445168. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:48:04,634][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:48:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5783552. Throughput: 0: 326.9. Samples: 1446360. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:48:09,630][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:48:14,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1433.6, 300 sec: 1319.1). Total num frames: 5795840. Throughput: 0: 348.8. Samples: 1449048. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:48:14,639][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:48:19,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1291.3). Total num frames: 5795840. Throughput: 0: 355.8. Samples: 1451064. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:48:19,633][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:48:24,628][00294] Fps is (10 sec: 819.2, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5804032. Throughput: 0: 354.7. Samples: 1451876. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:48:24,632][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:48:29,631][00294] Fps is (10 sec: 1638.0, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5812224. Throughput: 0: 334.8. Samples: 1453648. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:48:29,637][00294] Avg episode reward: [(0, '-3.622')] [2023-07-24 01:48:32,948][14527] Updated weights for policy 0, policy_version 1420 (0.0020) [2023-07-24 01:48:34,081][14528] DAMAGECOUNT value on done: 1359.0 [2023-07-24 01:48:34,085][14528] Sum rewards: -7.738, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-0.944', 'WEAPON1': '0.010', 'AMMO5': '0.015', 'AMMO2': '0.037', 'ARMOR': '0.052', 'weapon4': '0.080', 'HITCOUNT': '0.100', 'weapon5': '0.110', 'AMMO3': '0.178', 'AMMO4': '0.186', 'WEAPON4': '0.200', 'WEAPON5': '0.250', 'DAMAGECOUNT': '0.360', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.900', 'weapon3': '1.258', 'weapon2': '1.720'} [2023-07-24 01:48:34,314][14524] DAMAGECOUNT value on done: 1758.0 [2023-07-24 01:48:34,317][14524] Sum rewards: -4.530, reward structure: {'DEATHCOUNT': '-8.250', 'FRAGCOUNT': '-1.500', 'HEALTH': '-0.470', 'AMMO5': '0.020', 'AMMO2': '0.028', 'WEAPON1': '0.030', 'weapon5': '0.032', 'HITCOUNT': '0.070', 'AMMO3': '0.121', 'AMMO4': '0.138', 'DAMAGECOUNT': '0.219', 'WEAPON5': '0.300', 'WEAPON4': '0.350', 'ARMOR': '0.474', 'WEAPON3': '0.550', 'weapon2': '0.838', 'weapon3': '1.236', 'weapon4': '1.284'} [2023-07-24 01:48:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5816320. Throughput: 0: 325.6. Samples: 1455552. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:48:34,633][00294] Avg episode reward: [(0, '-3.628')] [2023-07-24 01:48:38,395][14532] DAMAGECOUNT value on done: 1923.0 [2023-07-24 01:48:38,398][14532] Sum rewards: 0.748, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.866', 'AMMO2': '0.013', 'AMMO5': '0.014', 'WEAPON1': '0.040', 'weapon7': '0.052', 'AMMO4': '0.066', 'AMMO3': '0.090', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'WEAPON4': '0.150', 'HITCOUNT': '0.160', 'weapon5': '0.244', 'WEAPON5': '0.250', 'WEAPON3': '0.500', 'ARMOR': '0.554', 'DAMAGECOUNT': '0.600', 'weapon4': '0.630', 'weapon2': '0.768', 'FRAGCOUNT': '1.000', 'weapon3': '1.432'} [2023-07-24 01:48:38,614][14528] DAMAGECOUNT value on done: 1793.0 [2023-07-24 01:48:38,937][14524] DAMAGECOUNT value on done: 2168.0 [2023-07-24 01:48:39,628][00294] Fps is (10 sec: 1638.8, 60 sec: 1433.6, 300 sec: 1332.9). Total num frames: 5828608. Throughput: 0: 336.0. Samples: 1456876. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:48:39,631][00294] Avg episode reward: [(0, '-3.478')] [2023-07-24 01:48:43,497][14532] DAMAGECOUNT value on done: 1726.0 [2023-07-24 01:48:43,508][14532] Sum rewards: -4.614, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.255', 'AMMO2': '0.014', 'AMMO5': '0.020', 'weapon4': '0.020', 'WEAPON4': '0.050', 'HITCOUNT': '0.060', 'AMMO4': '0.070', 'AMMO3': '0.136', 'DAMAGECOUNT': '0.240', 'weapon5': '0.298', 'WEAPON5': '0.350', 'WEAPON3': '0.800', 'FRAGCOUNT': '1.000', 'weapon2': '1.478', 'weapon3': '1.604'} [2023-07-24 01:48:43,880][14528] DAMAGECOUNT value on done: 1461.0 [2023-07-24 01:48:43,880][14528] Sum rewards: -1.003, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-0.578', 'AMMO5': '0.007', 'AMMO2': '0.010', 'weapon7': '0.018', 'weapon5': '0.038', 'AMMO4': '0.049', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'WEAPON4': '0.100', 'HITCOUNT': '0.110', 'AMMO3': '0.123', 'WEAPON5': '0.150', 'weapon4': '0.264', 'DAMAGECOUNT': '0.390', 'WEAPON3': '0.700', 'FRAGCOUNT': '1.000', 'weapon2': '1.104', 'weapon3': '1.962'} [2023-07-24 01:48:44,495][14524] DAMAGECOUNT value on done: 1483.0 [2023-07-24 01:48:44,496][14524] Sum rewards: -0.969, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.818', 'AMMO5': '0.003', 'AMMO2': '0.014', 'WEAPON5': '0.050', 'AMMO4': '0.068', 'HITCOUNT': '0.080', 'ARMOR': '0.083', 'AMMO3': '0.096', 'WEAPON4': '0.150', 'weapon5': '0.226', 'DAMAGECOUNT': '0.255', 'weapon4': '0.402', 'WEAPON3': '0.500', 'weapon3': '0.906', 'FRAGCOUNT': '1.000', 'weapon2': '2.016'} [2023-07-24 01:48:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5832704. Throughput: 0: 354.9. Samples: 1459452. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:48:44,637][00294] Avg episode reward: [(0, '-3.443')] [2023-07-24 01:48:49,634][00294] Fps is (10 sec: 818.7, 60 sec: 1297.0, 300 sec: 1305.1). Total num frames: 5836800. Throughput: 0: 346.2. Samples: 1460748. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:48:49,637][00294] Avg episode reward: [(0, '-3.455')] [2023-07-24 01:48:53,351][14532] DAMAGECOUNT value on done: 1005.0 [2023-07-24 01:48:53,621][14528] DAMAGECOUNT value on done: 1814.0 [2023-07-24 01:48:53,622][14528] Sum rewards: -1.784, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.630', 'weapon7': '0.006', 'AMMO5': '0.010', 'AMMO2': '0.034', 'weapon5': '0.102', 'AMMO3': '0.151', 'AMMO4': '0.169', 'HITCOUNT': '0.180', 'WEAPON5': '0.200', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON4': '0.400', 'weapon4': '0.410', 'ARMOR': '0.496', 'WEAPON3': '0.900', 'DAMAGECOUNT': '1.014', 'weapon2': '1.206', 'weapon3': '1.718', 'FRAGCOUNT': '2.000'} [2023-07-24 01:48:54,078][14524] DAMAGECOUNT value on done: 1383.0 [2023-07-24 01:48:54,083][14524] Sum rewards: -8.285, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.530', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.014', 'WEAPON1': '0.020', 'AMMO5': '0.025', 'AMMO4': '0.069', 'AMMO3': '0.131', 'weapon4': '0.140', 'WEAPON4': '0.150', 'HITCOUNT': '0.160', 'weapon5': '0.252', 'WEAPON5': '0.350', 'DAMAGECOUNT': '0.612', 'WEAPON3': '0.900', 'weapon2': '1.432', 'weapon3': '1.740'} [2023-07-24 01:48:54,267][14531] DAMAGECOUNT value on done: 2311.0 [2023-07-24 01:48:54,267][14531] Sum rewards: -1.366, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.775', 'WEAPON1': '0.010', 'AMMO2': '0.022', 'AMMO5': '0.025', 'AMMO3': '0.103', 'AMMO4': '0.107', 'HITCOUNT': '0.140', 'weapon5': '0.208', 'WEAPON4': '0.250', 'weapon4': '0.358', 'WEAPON5': '0.400', 'ARMOR': '0.500', 'WEAPON3': '0.600', 'DAMAGECOUNT': '0.948', 'weapon2': '1.456', 'weapon3': '1.532', 'FRAGCOUNT': '2.000'} [2023-07-24 01:48:54,631][00294] Fps is (10 sec: 819.0, 60 sec: 1297.0, 300 sec: 1305.2). Total num frames: 5840896. Throughput: 0: 334.3. Samples: 1461404. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:48:54,633][00294] Avg episode reward: [(0, '-3.462')] [2023-07-24 01:48:58,889][14529] DAMAGECOUNT value on done: 2369.0 [2023-07-24 01:48:58,890][14529] Sum rewards: -1.897, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.900', 'AMMO5': '0.018', 'WEAPON1': '0.020', 'AMMO2': '0.022', 'weapon5': '0.026', 'AMMO4': '0.108', 'AMMO3': '0.176', 'HITCOUNT': '0.220', 'WEAPON4': '0.250', 'WEAPON5': '0.350', 'weapon4': '0.470', 'ARMOR': '0.485', 'DAMAGECOUNT': '0.840', 'weapon2': '0.882', 'WEAPON3': '1.050', 'weapon3': '1.836', 'FRAGCOUNT': '3.000'} [2023-07-24 01:48:59,628][00294] Fps is (10 sec: 819.7, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5844992. Throughput: 0: 303.7. Samples: 1462716. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:48:59,631][00294] Avg episode reward: [(0, '-3.414')] [2023-07-24 01:49:02,737][14532] DAMAGECOUNT value on done: 1370.0 [2023-07-24 01:49:02,744][14532] Sum rewards: -1.895, reward structure: {'DEATHCOUNT': '-6.750', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.010', 'AMMO2': '0.016', 'weapon5': '0.034', 'ARMOR': '0.044', 'AMMO4': '0.077', 'HEALTH': '0.088', 'HITCOUNT': '0.090', 'AMMO6': '0.100', 'WEAPON7': '0.100', 'AMMO7': '0.100', 'AMMO3': '0.101', 'WEAPON4': '0.150', 'WEAPON5': '0.200', 'weapon4': '0.234', 'DAMAGECOUNT': '0.387', 'WEAPON3': '0.500', 'weapon3': '1.458', 'weapon2': '1.666'} [2023-07-24 01:49:02,854][14528] DAMAGECOUNT value on done: 1579.0 [2023-07-24 01:49:02,855][14528] Sum rewards: -3.154, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.602', 'AMMO2': '0.020', 'WEAPON1': '0.020', 'AMMO5': '0.027', 'HITCOUNT': '0.030', 'weapon7': '0.068', 'AMMO4': '0.098', 'DAMAGECOUNT': '0.105', 'AMMO3': '0.108', 'WEAPON4': '0.200', 'weapon5': '0.268', 'AMMO6': '0.300', 'WEAPON7': '0.300', 'AMMO7': '0.300', 'WEAPON5': '0.400', 'ARMOR': '0.400', 'weapon4': '0.412', 'WEAPON3': '0.600', 'FRAGCOUNT': '1.000', 'weapon3': '1.182', 'weapon2': '1.360'} [2023-07-24 01:49:03,478][14524] DAMAGECOUNT value on done: 1598.0 [2023-07-24 01:49:03,739][14531] DAMAGECOUNT value on done: 1651.0 [2023-07-24 01:49:03,753][14531] Sum rewards: -1.921, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.664', 'AMMO5': '0.005', 'weapon5': '0.024', 'ARMOR': '0.028', 'AMMO2': '0.029', 'WEAPON1': '0.030', 'AMMO3': '0.090', 'WEAPON5': '0.100', 'AMMO4': '0.146', 'HITCOUNT': '0.220', 'WEAPON4': '0.250', 'weapon4': '0.550', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.885', 'weapon2': '1.344', 'weapon3': '1.592', 'FRAGCOUNT': '2.000'} [2023-07-24 01:49:04,628][00294] Fps is (10 sec: 1229.1, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5853184. Throughput: 0: 287.6. Samples: 1464004. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:49:04,632][00294] Avg episode reward: [(0, '-3.534')] [2023-07-24 01:49:08,660][14529] DAMAGECOUNT value on done: 1604.0 [2023-07-24 01:49:08,661][14529] Sum rewards: -3.379, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.452', 'WEAPON1': '0.010', 'AMMO2': '0.012', 'AMMO5': '0.015', 'ARMOR': '0.028', 'AMMO4': '0.059', 'weapon4': '0.064', 'AMMO3': '0.103', 'WEAPON4': '0.150', 'weapon5': '0.164', 'HITCOUNT': '0.260', 'WEAPON5': '0.300', 'FRAGCOUNT': '0.500', 'WEAPON3': '0.650', 'DAMAGECOUNT': '1.050', 'weapon2': '1.492', 'weapon3': '1.966'} [2023-07-24 01:49:09,628][00294] Fps is (10 sec: 819.2, 60 sec: 1160.5, 300 sec: 1305.2). Total num frames: 5853184. Throughput: 0: 285.7. Samples: 1464732. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:49:09,631][00294] Avg episode reward: [(0, '-3.513')] [2023-07-24 01:49:09,991][14532] DAMAGECOUNT value on done: 1742.0 [2023-07-24 01:49:09,998][14532] Sum rewards: -0.802, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-1.632', 'AMMO2': '0.002', 'AMMO4': '0.010', 'AMMO5': '0.013', 'WEAPON4': '0.100', 'AMMO3': '0.164', 'weapon5': '0.212', 'WEAPON5': '0.250', 'HITCOUNT': '0.270', 'weapon4': '0.396', 'weapon2': '0.752', 'WEAPON3': '1.000', 'DAMAGECOUNT': '1.245', 'weapon3': '2.166', 'FRAGCOUNT': '4.000'} [2023-07-24 01:49:10,013][14528] DAMAGECOUNT value on done: 1521.0 [2023-07-24 01:49:10,015][14528] Sum rewards: -0.099, reward structure: {'DEATHCOUNT': '-5.250', 'HEALTH': '-0.526', 'AMMO5': '0.010', 'AMMO2': '0.022', 'HITCOUNT': '0.050', 'weapon5': '0.050', 'AMMO3': '0.086', 'WEAPON4': '0.100', 'WEAPON5': '0.100', 'AMMO4': '0.107', 'weapon4': '0.214', 'DAMAGECOUNT': '0.240', 'ARMOR': '0.400', 'WEAPON3': '0.500', 'FRAGCOUNT': '1.000', 'weapon2': '1.160', 'weapon3': '1.638'} [2023-07-24 01:49:10,085][14527] Updated weights for policy 0, policy_version 1430 (0.0062) [2023-07-24 01:49:10,299][14524] DAMAGECOUNT value on done: 2439.0 [2023-07-24 01:49:10,422][14531] DAMAGECOUNT value on done: 1603.0 [2023-07-24 01:49:10,432][14531] Sum rewards: -7.559, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-2.035', 'FRAGCOUNT': '-0.500', 'weapon7': '0.014', 'WEAPON1': '0.020', 'AMMO5': '0.020', 'AMMO2': '0.032', 'ARMOR': '0.060', 'weapon5': '0.118', 'AMMO3': '0.132', 'HITCOUNT': '0.150', 'AMMO4': '0.160', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON4': '0.300', 'WEAPON5': '0.400', 'DAMAGECOUNT': '0.450', 'WEAPON3': '0.600', 'weapon4': '0.636', 'weapon3': '1.210', 'weapon2': '1.324'} [2023-07-24 01:49:13,836][14529] DAMAGECOUNT value on done: 1485.0 [2023-07-24 01:49:13,842][14529] Sum rewards: -9.238, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-2.210', 'FRAGCOUNT': '-0.500', 'AMMO2': '0.012', 'AMMO5': '0.012', 'WEAPON1': '0.020', 'ARMOR': '0.035', 'WEAPON4': '0.050', 'AMMO4': '0.058', 'HITCOUNT': '0.130', 'weapon4': '0.170', 'AMMO3': '0.204', 'WEAPON5': '0.250', 'weapon5': '0.296', 'DAMAGECOUNT': '0.807', 'weapon2': '1.124', 'WEAPON3': '1.200', 'weapon3': '1.854'} [2023-07-24 01:49:14,628][00294] Fps is (10 sec: 819.2, 60 sec: 1092.3, 300 sec: 1305.2). Total num frames: 5861376. Throughput: 0: 293.3. Samples: 1466848. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) [2023-07-24 01:49:14,630][00294] Avg episode reward: [(0, '-3.562')] [2023-07-24 01:49:14,696][14528] DAMAGECOUNT value on done: 1910.0 [2023-07-24 01:49:14,704][14528] Sum rewards: 0.718, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.595', 'WEAPON1': '0.010', 'AMMO5': '0.012', 'AMMO2': '0.024', 'weapon5': '0.118', 'AMMO4': '0.119', 'AMMO3': '0.138', 'WEAPON4': '0.150', 'WEAPON5': '0.250', 'HITCOUNT': '0.360', 'ARMOR': '0.408', 'weapon4': '0.540', 'WEAPON3': '0.800', 'weapon2': '0.952', 'DAMAGECOUNT': '1.515', 'weapon3': '1.666', 'FRAGCOUNT': '4.000'} [2023-07-24 01:49:14,734][14532] DAMAGECOUNT value on done: 1504.0 [2023-07-24 01:49:14,736][14532] Sum rewards: -3.945, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.242', 'AMMO2': '0.021', 'AMMO5': '0.023', 'WEAPON1': '0.030', 'HITCOUNT': '0.030', 'ARMOR': '0.036', 'DAMAGECOUNT': '0.093', 'AMMO4': '0.103', 'AMMO3': '0.120', 'WEAPON4': '0.200', 'WEAPON5': '0.400', 'weapon5': '0.434', 'weapon4': '0.464', 'weapon2': '0.582', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon3': '1.862'} [2023-07-24 01:49:15,063][14524] DAMAGECOUNT value on done: 1755.0 [2023-07-24 01:49:15,071][14524] Sum rewards: -1.830, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-2.460', 'AMMO2': '0.011', 'AMMO5': '0.012', 'WEAPON1': '0.040', 'AMMO4': '0.057', 'weapon5': '0.074', 'ARMOR': '0.121', 'AMMO3': '0.155', 'WEAPON4': '0.200', 'WEAPON5': '0.250', 'HITCOUNT': '0.330', 'weapon4': '0.388', 'WEAPON3': '1.000', 'weapon2': '1.130', 'DAMAGECOUNT': '1.200', 'weapon3': '1.912', 'FRAGCOUNT': '2.000'} [2023-07-24 01:49:15,374][14531] DAMAGECOUNT value on done: 1684.0 [2023-07-24 01:49:15,381][14531] Sum rewards: -5.272, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-3.943', 'AMMO2': '0.026', 'AMMO4': '0.127', 'HITCOUNT': '0.140', 'AMMO3': '0.141', 'WEAPON4': '0.400', 'DAMAGECOUNT': '0.582', 'weapon4': '0.830', 'WEAPON3': '0.900', 'ARMOR': '1.013', 'weapon3': '1.368', 'weapon2': '1.394', 'FRAGCOUNT': '3.000'} [2023-07-24 01:49:18,381][14529] DAMAGECOUNT value on done: 1970.0 [2023-07-24 01:49:18,384][14529] Sum rewards: -1.481, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.055', 'WEAPON1': '0.020', 'AMMO5': '0.032', 'AMMO2': '0.038', 'ARMOR': '0.044', 'AMMO3': '0.106', 'HITCOUNT': '0.140', 'AMMO4': '0.188', 'WEAPON4': '0.300', 'weapon5': '0.356', 'WEAPON5': '0.450', 'FRAGCOUNT': '0.500', 'DAMAGECOUNT': '0.510', 'weapon4': '0.586', 'WEAPON3': '0.650', 'weapon2': '1.026', 'weapon3': '1.378'} [2023-07-24 01:49:19,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5873664. Throughput: 0: 309.3. Samples: 1469472. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:49:19,634][00294] Avg episode reward: [(0, '-3.514')] [2023-07-24 01:49:20,028][14528] DAMAGECOUNT value on done: 2686.0 [2023-07-24 01:49:20,029][14528] Sum rewards: 5.576, reward structure: {'DEATHCOUNT': '-6.000', 'AMMO2': '0.009', 'AMMO5': '0.017', 'WEAPON1': '0.040', 'AMMO4': '0.044', 'WEAPON4': '0.050', 'AMMO3': '0.093', 'HEALTH': '0.150', 'weapon4': '0.174', 'WEAPON5': '0.250', 'HITCOUNT': '0.260', 'WEAPON3': '0.550', 'weapon5': '0.576', 'weapon2': '1.102', 'DAMAGECOUNT': '1.587', 'weapon3': '1.674', 'FRAGCOUNT': '5.000'} [2023-07-24 01:49:20,120][14532] DAMAGECOUNT value on done: 1741.0 [2023-07-24 01:49:20,123][14532] Sum rewards: -1.840, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.596', 'AMMO2': '0.001', 'AMMO4': '0.006', 'weapon7': '0.010', 'AMMO5': '0.018', 'WEAPON1': '0.020', 'ARMOR': '0.040', 'WEAPON4': '0.100', 'HITCOUNT': '0.110', 'AMMO3': '0.149', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'weapon4': '0.210', 'WEAPON5': '0.400', 'weapon5': '0.476', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.765', 'FRAGCOUNT': '1.000', 'weapon3': '1.238', 'weapon2': '1.462'} [2023-07-24 01:49:20,451][14524] DAMAGECOUNT value on done: 1560.0 [2023-07-24 01:49:20,453][14524] Sum rewards: -4.794, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.720', 'FRAGCOUNT': '-0.500', 'ARMOR': '0.016', 'AMMO5': '0.022', 'AMMO2': '0.030', 'WEAPON1': '0.040', 'AMMO3': '0.145', 'AMMO4': '0.149', 'HITCOUNT': '0.160', 'WEAPON4': '0.250', 'weapon5': '0.260', 'weapon4': '0.318', 'WEAPON5': '0.450', 'weapon2': '0.630', 'DAMAGECOUNT': '0.792', 'WEAPON3': '0.800', 'weapon3': '2.114'} [2023-07-24 01:49:20,760][14531] DAMAGECOUNT value on done: 1385.0 [2023-07-24 01:49:20,764][14531] Sum rewards: -1.591, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-0.272', 'AMMO2': '0.012', 'AMMO5': '0.012', 'weapon5': '0.030', 'ARMOR': '0.040', 'AMMO4': '0.059', 'weapon7': '0.088', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'WEAPON4': '0.100', 'AMMO3': '0.113', 'HITCOUNT': '0.140', 'WEAPON5': '0.250', 'weapon4': '0.332', 'WEAPON3': '0.650', 'weapon2': '0.926', 'DAMAGECOUNT': '1.008', 'weapon3': '1.620', 'FRAGCOUNT': '2.000'} [2023-07-24 01:49:24,006][14530] DAMAGECOUNT value on done: 1750.0 [2023-07-24 01:49:24,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5877760. Throughput: 0: 299.8. Samples: 1470368. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0) [2023-07-24 01:49:24,633][00294] Avg episode reward: [(0, '-3.296')] [2023-07-24 01:49:26,206][14532] DAMAGECOUNT value on done: 1563.0 [2023-07-24 01:49:26,218][14529] DAMAGECOUNT value on done: 1217.0 [2023-07-24 01:49:26,220][14529] Sum rewards: -2.950, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.620', 'AMMO5': '0.005', 'weapon7': '0.018', 'WEAPON1': '0.020', 'ARMOR': '0.032', 'AMMO2': '0.040', 'HITCOUNT': '0.090', 'WEAPON5': '0.100', 'AMMO3': '0.158', 'AMMO6': '0.160', 'AMMO7': '0.160', 'DAMAGECOUNT': '0.183', 'AMMO4': '0.200', 'WEAPON7': '0.200', 'WEAPON4': '0.450', 'WEAPON3': '0.700', 'weapon4': '0.958', 'FRAGCOUNT': '1.000', 'weapon2': '1.064', 'weapon3': '1.382'} [2023-07-24 01:49:27,190][14531] DAMAGECOUNT value on done: 2294.0 [2023-07-24 01:49:27,206][14531] Sum rewards: 3.609, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-0.607', 'ARMOR': '0.004', 'AMMO5': '0.005', 'AMMO2': '0.023', 'AMMO3': '0.069', 'WEAPON4': '0.100', 'weapon5': '0.112', 'AMMO4': '0.113', 'AMMO6': '0.120', 'AMMO7': '0.120', 'WEAPON5': '0.150', 'weapon7': '0.152', 'HITCOUNT': '0.160', 'WEAPON7': '0.200', 'weapon4': '0.304', 'WEAPON3': '0.500', 'DAMAGECOUNT': '0.906', 'weapon3': '0.998', 'weapon2': '2.180', 'FRAGCOUNT': '4.000'} [2023-07-24 01:49:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.9, 300 sec: 1305.2). Total num frames: 5885952. Throughput: 0: 280.9. Samples: 1472092. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0) [2023-07-24 01:49:29,634][00294] Avg episode reward: [(0, '-3.174')] [2023-07-24 01:49:31,339][14530] DAMAGECOUNT value on done: 2260.0 [2023-07-24 01:49:32,998][14529] DAMAGECOUNT value on done: 1994.0 [2023-07-24 01:49:33,019][14529] Sum rewards: -2.434, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-0.836', 'AMMO2': '0.023', 'WEAPON1': '0.030', 'AMMO4': '0.113', 'AMMO3': '0.167', 'HITCOUNT': '0.240', 'WEAPON4': '0.250', 'ARMOR': '0.448', 'weapon4': '0.448', 'weapon2': '0.786', 'DAMAGECOUNT': '0.807', 'WEAPON3': '0.850', 'weapon3': '1.990', 'FRAGCOUNT': '2.000'} [2023-07-24 01:49:33,031][14526] DAMAGECOUNT value on done: 1615.0 [2023-07-24 01:49:34,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5890048. Throughput: 0: 291.0. Samples: 1473840. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:49:34,635][00294] Avg episode reward: [(0, '-3.116')] [2023-07-24 01:49:34,940][14531] DAMAGECOUNT value on done: 1865.0 [2023-07-24 01:49:34,942][14531] Sum rewards: -1.706, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.958', 'WEAPON1': '0.020', 'AMMO2': '0.024', 'AMMO5': '0.030', 'AMMO4': '0.120', 'AMMO3': '0.148', 'weapon4': '0.180', 'WEAPON4': '0.200', 'weapon5': '0.200', 'HITCOUNT': '0.310', 'ARMOR': '0.444', 'WEAPON5': '0.500', 'WEAPON3': '0.950', 'weapon2': '1.030', 'DAMAGECOUNT': '1.050', 'FRAGCOUNT': '2.000', 'weapon3': '2.046'} [2023-07-24 01:49:35,363][14525] DAMAGECOUNT value on done: 1760.0 [2023-07-24 01:49:35,369][14525] Sum rewards: -1.670, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-1.268', 'AMMO2': '0.002', 'AMMO4': '0.008', 'AMMO5': '0.010', 'WEAPON1': '0.010', 'ARMOR': '0.040', 'weapon5': '0.110', 'AMMO3': '0.177', 'WEAPON5': '0.200', 'HITCOUNT': '0.280', 'weapon2': '1.006', 'WEAPON3': '1.050', 'DAMAGECOUNT': '1.530', 'weapon3': '2.426', 'FRAGCOUNT': '4.000'} [2023-07-24 01:49:38,406][14530] DAMAGECOUNT value on done: 1083.0 [2023-07-24 01:49:38,410][14530] Sum rewards: -5.126, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.762', 'AMMO5': '0.020', 'AMMO2': '0.020', 'WEAPON1': '0.030', 'ARMOR': '0.040', 'HITCOUNT': '0.070', 'AMMO4': '0.098', 'AMMO3': '0.121', 'weapon5': '0.122', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.270', 'WEAPON5': '0.350', 'weapon4': '0.534', 'WEAPON3': '0.600', 'FRAGCOUNT': '1.000', 'weapon2': '1.324', 'weapon3': '1.388'} [2023-07-24 01:49:39,607][14529] DAMAGECOUNT value on done: 2481.0 [2023-07-24 01:49:39,615][14529] Sum rewards: 0.952, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-0.100', 'AMMO5': '0.010', 'WEAPON1': '0.020', 'AMMO2': '0.027', 'weapon7': '0.034', 'weapon5': '0.036', 'AMMO3': '0.097', 'AMMO4': '0.135', 'WEAPON5': '0.150', 'HITCOUNT': '0.160', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON4': '0.300', 'ARMOR': '0.460', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.645', 'weapon4': '0.814', 'weapon2': '1.050', 'weapon3': '1.214', 'FRAGCOUNT': '3.000'} [2023-07-24 01:49:39,628][00294] Fps is (10 sec: 819.2, 60 sec: 1092.3, 300 sec: 1305.2). Total num frames: 5894144. Throughput: 0: 294.6. Samples: 1474660. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:49:39,637][00294] Avg episode reward: [(0, '-3.161')] [2023-07-24 01:49:39,762][14527] Updated weights for policy 0, policy_version 1440 (0.0026) [2023-07-24 01:49:39,814][14526] DAMAGECOUNT value on done: 1505.0 [2023-07-24 01:49:40,354][14531] DAMAGECOUNT value on done: 1874.0 [2023-07-24 01:49:40,356][14531] Sum rewards: 1.466, reward structure: {'DEATHCOUNT': '-6.000', 'HEALTH': '-1.150', 'AMMO5': '0.018', 'AMMO2': '0.028', 'WEAPON1': '0.030', 'weapon5': '0.030', 'weapon7': '0.030', 'AMMO3': '0.116', 'AMMO4': '0.139', 'HITCOUNT': '0.160', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'WEAPON4': '0.250', 'WEAPON5': '0.300', 'ARMOR': '0.400', 'weapon4': '0.458', 'WEAPON3': '0.550', 'DAMAGECOUNT': '0.780', 'weapon2': '1.286', 'weapon3': '1.442', 'FRAGCOUNT': '2.000'} [2023-07-24 01:49:40,905][14525] DAMAGECOUNT value on done: 1659.0 [2023-07-24 01:49:40,905][14525] Sum rewards: -4.830, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.900', 'AMMO2': '0.019', 'WEAPON1': '0.030', 'AMMO5': '0.037', 'ARMOR': '0.040', 'AMMO4': '0.095', 'HITCOUNT': '0.140', 'WEAPON4': '0.150', 'AMMO3': '0.167', 'weapon5': '0.342', 'weapon4': '0.368', 'FRAGCOUNT': '0.500', 'DAMAGECOUNT': '0.555', 'WEAPON5': '0.600', 'WEAPON3': '0.850', 'weapon2': '1.110', 'weapon3': '1.566'} [2023-07-24 01:49:43,326][14530] DAMAGECOUNT value on done: 1495.0 [2023-07-24 01:49:43,332][14530] Sum rewards: 1.456, reward structure: {'DEATHCOUNT': '-4.500', 'HEALTH': '-1.226', 'AMMO4': '-0.019', 'AMMO2': '-0.004', 'AMMO5': '0.010', 'WEAPON1': '0.020', 'AMMO3': '0.064', 'HITCOUNT': '0.150', 'WEAPON5': '0.200', 'weapon5': '0.262', 'WEAPON3': '0.350', 'ARMOR': '0.400', 'DAMAGECOUNT': '0.495', 'weapon3': '1.228', 'FRAGCOUNT': '2.000', 'weapon2': '2.026'} [2023-07-24 01:49:44,252][14529] DAMAGECOUNT value on done: 2542.0 [2023-07-24 01:49:44,258][14529] Sum rewards: -8.507, reward structure: {'DEATHCOUNT': '-12.750', 'HEALTH': '-2.176', 'FRAGCOUNT': '-0.500', 'AMMO5': '0.023', 'ARMOR': '0.040', 'AMMO2': '0.055', 'HITCOUNT': '0.120', 'AMMO3': '0.152', 'weapon5': '0.214', 'AMMO4': '0.273', 'WEAPON5': '0.400', 'WEAPON4': '0.500', 'weapon4': '0.542', 'DAMAGECOUNT': '0.858', 'WEAPON3': '0.950', 'weapon2': '1.030', 'weapon3': '1.762'} [2023-07-24 01:49:44,604][14526] DAMAGECOUNT value on done: 2264.0 [2023-07-24 01:49:44,615][14526] Sum rewards: -3.241, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.580', 'weapon5': '0.002', 'AMMO5': '0.005', 'WEAPON1': '0.010', 'AMMO2': '0.024', 'HITCOUNT': '0.100', 'WEAPON5': '0.100', 'AMMO4': '0.120', 'AMMO3': '0.125', 'WEAPON4': '0.150', 'DAMAGECOUNT': '0.420', 'ARMOR': '0.424', 'weapon4': '0.434', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon3': '1.318', 'weapon2': '1.706'} [2023-07-24 01:49:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1228.8, 300 sec: 1305.2). Total num frames: 5906432. Throughput: 0: 322.3. Samples: 1477220. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:49:44,636][00294] Avg episode reward: [(0, '-3.149')] [2023-07-24 01:49:45,970][14525] DAMAGECOUNT value on done: 1424.0 [2023-07-24 01:49:45,971][14525] Sum rewards: -3.024, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.265', 'AMMO5': '0.024', 'AMMO2': '0.045', 'ARMOR': '0.068', 'AMMO3': '0.154', 'HITCOUNT': '0.190', 'AMMO4': '0.227', 'WEAPON4': '0.450', 'WEAPON5': '0.500', 'weapon5': '0.508', 'weapon4': '0.744', 'WEAPON3': '0.850', 'weapon2': '0.986', 'weapon3': '1.048', 'DAMAGECOUNT': '1.197', 'FRAGCOUNT': '2.000'} [2023-07-24 01:49:49,039][14530] DAMAGECOUNT value on done: 1352.0 [2023-07-24 01:49:49,628][00294] Fps is (10 sec: 2048.0, 60 sec: 1297.2, 300 sec: 1305.2). Total num frames: 5914624. Throughput: 0: 347.0. Samples: 1479620. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:49:49,634][00294] Avg episode reward: [(0, '-3.226')] [2023-07-24 01:49:51,532][14526] DAMAGECOUNT value on done: 1662.0 [2023-07-24 01:49:51,533][14526] Sum rewards: -1.114, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-1.590', 'AMMO2': '0.004', 'ARMOR': '0.005', 'AMMO4': '0.020', 'AMMO5': '0.020', 'WEAPON1': '0.020', 'HITCOUNT': '0.080', 'WEAPON4': '0.100', 'AMMO6': '0.120', 'AMMO7': '0.120', 'AMMO3': '0.127', 'weapon7': '0.136', 'weapon5': '0.158', 'WEAPON7': '0.200', 'WEAPON5': '0.200', 'DAMAGECOUNT': '0.300', 'weapon4': '0.334', 'WEAPON3': '0.600', 'weapon3': '0.938', 'weapon2': '1.744', 'FRAGCOUNT': '2.000'} [2023-07-24 01:49:53,102][14525] DAMAGECOUNT value on done: 1306.0 [2023-07-24 01:49:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1305.2). Total num frames: 5918720. Throughput: 0: 349.8. Samples: 1480472. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:49:54,637][00294] Avg episode reward: [(0, '-3.121')] [2023-07-24 01:49:56,561][14530] DAMAGECOUNT value on done: 2435.0 [2023-07-24 01:49:56,563][14530] Sum rewards: -2.600, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-1.900', 'AMMO5': '0.022', 'AMMO2': '0.028', 'AMMO4': '0.138', 'AMMO3': '0.153', 'WEAPON4': '0.200', 'HITCOUNT': '0.230', 'WEAPON5': '0.250', 'weapon4': '0.352', 'weapon5': '0.482', 'ARMOR': '0.494', 'DAMAGECOUNT': '0.687', 'WEAPON3': '0.900', 'weapon3': '1.288', 'weapon2': '1.576', 'FRAGCOUNT': '3.000'} [2023-07-24 01:49:57,977][14526] DAMAGECOUNT value on done: 1393.0 [2023-07-24 01:49:57,989][14526] Sum rewards: -2.953, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-0.710', 'AMMO2': '0.019', 'AMMO5': '0.030', 'WEAPON1': '0.040', 'weapon5': '0.078', 'AMMO4': '0.095', 'AMMO3': '0.150', 'WEAPON4': '0.200', 'HITCOUNT': '0.230', 'weapon4': '0.264', 'WEAPON5': '0.450', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.705', 'weapon2': '1.238', 'FRAGCOUNT': '2.000', 'weapon3': '2.058'} [2023-07-24 01:49:59,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5926912. Throughput: 0: 340.9. Samples: 1482188. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) [2023-07-24 01:49:59,634][00294] Avg episode reward: [(0, '-3.094')] [2023-07-24 01:49:59,646][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001447_5926912.pth... [2023-07-24 01:49:59,930][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001371_5615616.pth [2023-07-24 01:50:00,539][14525] DAMAGECOUNT value on done: 1515.0 [2023-07-24 01:50:00,550][14525] Sum rewards: -8.231, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-2.350', 'FRAGCOUNT': '-1.500', 'WEAPON1': '0.020', 'AMMO5': '0.023', 'AMMO2': '0.030', 'HITCOUNT': '0.030', 'DAMAGECOUNT': '0.105', 'AMMO4': '0.149', 'weapon5': '0.172', 'AMMO3': '0.181', 'WEAPON4': '0.200', 'weapon4': '0.258', 'WEAPON5': '0.350', 'WEAPON3': '0.900', 'weapon2': '1.192', 'weapon3': '1.760'} [2023-07-24 01:50:03,451][14530] DAMAGECOUNT value on done: 1285.0 [2023-07-24 01:50:04,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1297.1, 300 sec: 1319.1). Total num frames: 5931008. Throughput: 0: 320.3. Samples: 1483884. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:50:04,637][00294] Avg episode reward: [(0, '-3.121')] [2023-07-24 01:50:05,329][14526] DAMAGECOUNT value on done: 1965.0 [2023-07-24 01:50:05,331][14526] Sum rewards: 0.724, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-0.680', 'WEAPON1': '0.010', 'AMMO5': '0.012', 'AMMO2': '0.023', 'ARMOR': '0.050', 'weapon7': '0.082', 'AMMO4': '0.115', 'AMMO3': '0.127', 'weapon5': '0.168', 'HITCOUNT': '0.180', 'WEAPON5': '0.200', 'AMMO6': '0.260', 'AMMO7': '0.260', 'WEAPON4': '0.300', 'WEAPON7': '0.300', 'weapon4': '0.586', 'WEAPON3': '0.700', 'DAMAGECOUNT': '0.996', 'weapon2': '1.238', 'weapon3': '1.296', 'FRAGCOUNT': '2.000'} [2023-07-24 01:50:06,417][14525] DAMAGECOUNT value on done: 1675.0 [2023-07-24 01:50:06,419][14525] Sum rewards: -4.935, reward structure: {'DEATHCOUNT': '-14.250', 'HEALTH': '-1.733', 'AMMO2': '0.010', 'WEAPON1': '0.010', 'weapon4': '0.014', 'AMMO5': '0.023', 'ARMOR': '0.024', 'weapon7': '0.040', 'AMMO4': '0.047', 'weapon5': '0.058', 'WEAPON4': '0.100', 'AMMO6': '0.160', 'AMMO7': '0.160', 'WEAPON7': '0.200', 'AMMO3': '0.213', 'HITCOUNT': '0.360', 'WEAPON5': '0.400', 'WEAPON3': '1.200', 'DAMAGECOUNT': '1.476', 'weapon2': '1.544', 'weapon3': '2.010', 'FRAGCOUNT': '3.000'} [2023-07-24 01:50:08,913][14527] Updated weights for policy 0, policy_version 1450 (0.0040) [2023-07-24 01:50:08,985][14530] DAMAGECOUNT value on done: 1548.0 [2023-07-24 01:50:08,987][14530] Sum rewards: -1.936, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-1.925', 'AMMO2': '0.007', 'WEAPON1': '0.020', 'weapon7': '0.028', 'AMMO5': '0.030', 'AMMO4': '0.035', 'weapon4': '0.072', 'WEAPON4': '0.100', 'HITCOUNT': '0.100', 'AMMO3': '0.114', 'weapon5': '0.298', 'AMMO6': '0.300', 'WEAPON7': '0.300', 'AMMO7': '0.300', 'DAMAGECOUNT': '0.360', 'WEAPON5': '0.450', 'ARMOR': '0.473', 'WEAPON3': '0.650', 'FRAGCOUNT': '1.000', 'weapon2': '1.140', 'weapon3': '1.712'} [2023-07-24 01:50:09,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 5939200. Throughput: 0: 325.0. Samples: 1484992. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) [2023-07-24 01:50:09,636][00294] Avg episode reward: [(0, '-3.051')] [2023-07-24 01:50:10,644][14526] DAMAGECOUNT value on done: 2038.0 [2023-07-24 01:50:11,940][14525] DAMAGECOUNT value on done: 2397.0 [2023-07-24 01:50:11,947][14525] Sum rewards: -0.588, reward structure: {'DEATHCOUNT': '-8.250', 'HEALTH': '-1.546', 'AMMO5': '0.005', 'AMMO2': '0.005', 'weapon4': '0.026', 'AMMO4': '0.027', 'WEAPON4': '0.050', 'weapon7': '0.090', 'AMMO6': '0.100', 'AMMO7': '0.100', 'WEAPON7': '0.100', 'WEAPON5': '0.100', 'AMMO3': '0.110', 'HITCOUNT': '0.220', 'ARMOR': '0.428', 'WEAPON3': '0.650', 'DAMAGECOUNT': '0.804', 'weapon2': '1.584', 'weapon3': '1.808', 'FRAGCOUNT': '3.000'} [2023-07-24 01:50:14,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 5947392. Throughput: 0: 346.1. Samples: 1487668. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:50:14,631][00294] Avg episode reward: [(0, '-3.001')] [2023-07-24 01:50:15,833][14526] DAMAGECOUNT value on done: 1764.0 [2023-07-24 01:50:15,838][14526] Sum rewards: -0.773, reward structure: {'DEATHCOUNT': '-6.000', 'FRAGCOUNT': '-0.500', 'WEAPON1': '0.010', 'AMMO2': '0.017', 'AMMO5': '0.020', 'ARMOR': '0.050', 'AMMO4': '0.085', 'AMMO3': '0.090', 'WEAPON4': '0.150', 'HITCOUNT': '0.200', 'HEALTH': '0.225', 'weapon5': '0.228', 'WEAPON5': '0.250', 'WEAPON3': '0.450', 'weapon4': '0.528', 'DAMAGECOUNT': '0.585', 'weapon2': '0.746', 'weapon3': '2.092'} [2023-07-24 01:50:17,268][14525] DAMAGECOUNT value on done: 1726.0 [2023-07-24 01:50:17,272][14525] Sum rewards: -2.781, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-1.910', 'AMMO2': '0.015', 'AMMO5': '0.023', 'weapon4': '0.036', 'WEAPON4': '0.050', 'AMMO4': '0.075', 'AMMO3': '0.142', 'HITCOUNT': '0.160', 'weapon5': '0.214', 'ARMOR': '0.400', 'WEAPON5': '0.450', 'DAMAGECOUNT': '0.510', 'WEAPON3': '0.800', 'weapon2': '1.024', 'FRAGCOUNT': '2.000', 'weapon3': '2.230'} [2023-07-24 01:50:19,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5955584. Throughput: 0: 353.4. Samples: 1489744. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 01:50:19,632][00294] Avg episode reward: [(0, '-2.955')] [2023-07-24 01:50:24,630][00294] Fps is (10 sec: 1228.6, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5959680. Throughput: 0: 353.7. Samples: 1490576. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 01:50:24,633][00294] Avg episode reward: [(0, '-2.955')] [2023-07-24 01:50:29,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5967872. Throughput: 0: 336.2. Samples: 1492348. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) [2023-07-24 01:50:29,636][00294] Avg episode reward: [(0, '-2.955')] [2023-07-24 01:50:34,628][00294] Fps is (10 sec: 1229.0, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5971968. Throughput: 0: 324.5. Samples: 1494224. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:50:34,634][00294] Avg episode reward: [(0, '-2.955')] [2023-07-24 01:50:38,415][14527] Updated weights for policy 0, policy_version 1460 (0.0031) [2023-07-24 01:50:39,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1433.6, 300 sec: 1305.2). Total num frames: 5980160. Throughput: 0: 335.1. Samples: 1495552. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) [2023-07-24 01:50:39,639][00294] Avg episode reward: [(0, '-2.955')] [2023-07-24 01:50:44,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 5988352. Throughput: 0: 356.5. Samples: 1498232. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) [2023-07-24 01:50:44,633][00294] Avg episode reward: [(0, '-2.955')] [2023-07-24 01:50:49,628][00294] Fps is (10 sec: 1638.4, 60 sec: 1365.3, 300 sec: 1319.1). Total num frames: 5996544. Throughput: 0: 357.4. Samples: 1499968. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:50:49,635][00294] Avg episode reward: [(0, '-2.955')] [2023-07-24 01:50:54,628][00294] Fps is (10 sec: 1228.8, 60 sec: 1365.3, 300 sec: 1305.2). Total num frames: 6000640. Throughput: 0: 351.8. Samples: 1500824. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) [2023-07-24 01:50:54,635][00294] Avg episode reward: [(0, '-2.955')] [2023-07-24 01:50:56,113][14511] Stopping Batcher_0... [2023-07-24 01:50:56,114][14511] Loop batcher_evt_loop terminating... [2023-07-24 01:50:56,117][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001466_6004736.pth... [2023-07-24 01:50:56,132][00294] Component Batcher_0 stopped! [2023-07-24 01:50:56,328][14527] Weights refcount: 2 0 [2023-07-24 01:50:56,345][14527] Stopping InferenceWorker_p0-w0... [2023-07-24 01:50:56,351][14527] Loop inference_proc0-0_evt_loop terminating... [2023-07-24 01:50:56,352][00294] Component InferenceWorker_p0-w0 stopped! [2023-07-24 01:50:56,390][14511] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001409_5771264.pth [2023-07-24 01:50:56,425][14511] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001466_6004736.pth... [2023-07-24 01:50:56,562][14526] Stopping RolloutWorker_w2... [2023-07-24 01:50:56,562][00294] Component RolloutWorker_w2 stopped! [2023-07-24 01:50:56,566][14526] Loop rollout_proc2_evt_loop terminating... [2023-07-24 01:50:56,765][14530] Stopping RolloutWorker_w6... [2023-07-24 01:50:56,764][00294] Component RolloutWorker_w6 stopped! [2023-07-24 01:50:56,769][14530] Loop rollout_proc6_evt_loop terminating... [2023-07-24 01:50:56,820][00294] Component RolloutWorker_w0 stopped! [2023-07-24 01:50:56,826][14525] Stopping RolloutWorker_w0... [2023-07-24 01:50:56,843][14525] Loop rollout_proc0_evt_loop terminating... [2023-07-24 01:50:56,862][00294] Component LearnerWorker_p0 stopped! [2023-07-24 01:50:56,867][14511] Stopping LearnerWorker_p0... [2023-07-24 01:50:56,867][14511] Loop learner_proc0_evt_loop terminating... [2023-07-24 01:50:56,911][00294] Component RolloutWorker_w1 stopped! [2023-07-24 01:50:56,916][14524] Stopping RolloutWorker_w1... [2023-07-24 01:50:56,917][14524] Loop rollout_proc1_evt_loop terminating... [2023-07-24 01:50:56,923][14529] Stopping RolloutWorker_w4... [2023-07-24 01:50:56,923][00294] Component RolloutWorker_w4 stopped! [2023-07-24 01:50:56,923][14529] Loop rollout_proc4_evt_loop terminating... [2023-07-24 01:50:56,953][00294] Component RolloutWorker_w7 stopped! [2023-07-24 01:50:56,956][14532] Stopping RolloutWorker_w7... [2023-07-24 01:50:56,956][14532] Loop rollout_proc7_evt_loop terminating... [2023-07-24 01:50:56,975][00294] Component RolloutWorker_w3 stopped! [2023-07-24 01:50:56,981][14528] Stopping RolloutWorker_w3... [2023-07-24 01:50:56,986][14528] Loop rollout_proc3_evt_loop terminating... [2023-07-24 01:50:57,036][00294] Component RolloutWorker_w5 stopped! [2023-07-24 01:50:57,045][14531] Stopping RolloutWorker_w5... [2023-07-24 01:50:57,046][14531] Loop rollout_proc5_evt_loop terminating... [2023-07-24 01:50:57,039][00294] Waiting for process learner_proc0 to stop... [2023-07-24 01:50:58,670][00294] Waiting for process inference_proc0-0 to join... [2023-07-24 01:50:59,095][00294] Waiting for process rollout_proc0 to join... [2023-07-24 01:51:01,987][00294] Waiting for process rollout_proc1 to join... [2023-07-24 01:51:01,990][00294] Waiting for process rollout_proc2 to join... [2023-07-24 01:51:01,993][00294] Waiting for process rollout_proc3 to join... [2023-07-24 01:51:01,994][00294] Waiting for process rollout_proc4 to join... [2023-07-24 01:51:01,996][00294] Waiting for process rollout_proc5 to join... [2023-07-24 01:51:01,998][00294] Waiting for process rollout_proc6 to join... [2023-07-24 01:51:02,000][00294] Waiting for process rollout_proc7 to join... [2023-07-24 01:51:02,002][00294] Batcher 0 profile tree view: batching: 52.0976, releasing_batches: 0.0474 [2023-07-24 01:51:02,003][00294] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0040 wait_policy_total: 2054.0026 update_model: 22.5141 weight_update: 0.0030 one_step: 0.0199 handle_policy_step: 2385.3268 deserialize: 40.6149, stack: 7.6315, obs_to_device_normalize: 320.0554, forward: 1682.0563, send_messages: 72.7838 prepare_outputs: 193.5749 to_cpu: 93.5898 [2023-07-24 01:51:02,005][00294] Learner 0 profile tree view: misc: 0.0087, prepare_batch: 28.7119 train: 152.2264 epoch_init: 0.0181, minibatch_init: 0.0554, losses_postprocess: 1.2165, kl_divergence: 4.8065, after_optimizer: 16.4942 calculate_losses: 51.8312 losses_init: 0.0288, forward_head: 4.6883, bptt_initial: 23.3772, tail: 6.4011, advantages_returns: 0.6147, losses: 11.4953 bptt: 4.4673 bptt_forward_core: 4.2833 update: 75.7822 clip: 49.1336 [2023-07-24 01:51:02,006][00294] RolloutWorker_w0 profile tree view: wait_for_trajectories: 2.1110, enqueue_policy_requests: 323.1946, env_step: 3839.2003, overhead: 102.2053, complete_rollouts: 13.8596 save_policy_outputs: 166.2986 split_output_tensors: 76.3908 [2023-07-24 01:51:02,008][00294] RolloutWorker_w7 profile tree view: wait_for_trajectories: 1.8224, enqueue_policy_requests: 322.6948, env_step: 3842.5977, overhead: 99.5418, complete_rollouts: 13.2269 save_policy_outputs: 167.5178 split_output_tensors: 77.0765 [2023-07-24 01:51:02,009][00294] Loop Runner_EvtLoop terminating... [2023-07-24 01:51:02,011][00294] Runner profile tree view: main_loop: 4619.0192 [2023-07-24 01:51:02,012][00294] Collected {0: 6004736}, FPS: 1300.0 [2023-07-24 01:51:02,053][00294] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2023-07-24 01:51:02,055][00294] Overriding arg 'num_workers' with value 1 passed from command line [2023-07-24 01:51:02,057][00294] Adding new argument 'no_render'=True that is not in the saved config file! [2023-07-24 01:51:02,059][00294] Adding new argument 'save_video'=True that is not in the saved config file! [2023-07-24 01:51:02,061][00294] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2023-07-24 01:51:02,063][00294] Adding new argument 'video_name'=None that is not in the saved config file! [2023-07-24 01:51:02,064][00294] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2023-07-24 01:51:02,065][00294] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2023-07-24 01:51:02,066][00294] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2023-07-24 01:51:02,068][00294] Adding new argument 'hf_repository'='Corianas/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2023-07-24 01:51:02,069][00294] Adding new argument 'policy_index'=0 that is not in the saved config file! [2023-07-24 01:51:02,070][00294] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2023-07-24 01:51:02,071][00294] Adding new argument 'train_script'=None that is not in the saved config file! [2023-07-24 01:51:02,072][00294] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2023-07-24 01:51:02,073][00294] Using frameskip 1 and render_action_repeat=4 for evaluation [2023-07-24 01:51:02,130][00294] Port 40300 is available [2023-07-24 01:51:02,133][00294] Using port 40300 [2023-07-24 01:51:02,136][00294] RunningMeanStd input shape: (23,) [2023-07-24 01:51:02,138][00294] RunningMeanStd input shape: (3, 72, 128) [2023-07-24 01:51:02,141][00294] RunningMeanStd input shape: (1,) [2023-07-24 01:51:02,161][00294] ConvEncoder: input_channels=3 [2023-07-24 01:51:02,231][00294] Conv encoder output size: 512 [2023-07-24 01:51:02,235][00294] Policy head output size: 640 [2023-07-24 01:51:02,272][00294] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001466_6004736.pth... [2023-07-24 01:51:02,309][00294] Using port 40300 on host... [2023-07-24 01:51:02,682][00294] Initialized w:0 v:0 player:0 [2023-07-24 01:51:02,953][00294] Num frames 100... [2023-07-24 01:51:03,214][00294] Num frames 200... [2023-07-24 01:51:03,482][00294] Num frames 300... [2023-07-24 01:51:03,750][00294] Num frames 400... [2023-07-24 01:51:04,008][00294] Num frames 500... [2023-07-24 01:51:04,263][00294] Num frames 600... [2023-07-24 01:51:04,529][00294] Num frames 700... [2023-07-24 01:51:04,781][00294] Num frames 800... [2023-07-24 01:51:05,044][00294] Num frames 900... [2023-07-24 01:51:05,299][00294] Num frames 1000... [2023-07-24 01:51:05,571][00294] Num frames 1100... [2023-07-24 01:51:05,827][00294] Num frames 1200... [2023-07-24 01:51:06,090][00294] Num frames 1300... [2023-07-24 01:51:06,355][00294] Num frames 1400... [2023-07-24 01:51:06,625][00294] Num frames 1500... [2023-07-24 01:51:06,876][00294] Num frames 1600... [2023-07-24 01:51:07,137][00294] Num frames 1700... [2023-07-24 01:51:07,388][00294] Num frames 1800... [2023-07-24 01:51:07,654][00294] Num frames 1900... [2023-07-24 01:51:07,906][00294] Num frames 2000... [2023-07-24 01:51:08,166][00294] Num frames 2100... [2023-07-24 01:51:08,417][00294] Num frames 2200... [2023-07-24 01:51:08,684][00294] Num frames 2300... [2023-07-24 01:51:08,937][00294] Num frames 2400... [2023-07-24 01:51:09,197][00294] Num frames 2500... [2023-07-24 01:51:09,449][00294] Num frames 2600... [2023-07-24 01:51:09,715][00294] Num frames 2700... [2023-07-24 01:51:09,969][00294] Num frames 2800... [2023-07-24 01:51:10,227][00294] Num frames 2900... [2023-07-24 01:51:10,479][00294] Num frames 3000... [2023-07-24 01:51:10,851][00294] Num frames 3100... [2023-07-24 01:51:11,235][00294] Num frames 3200... [2023-07-24 01:51:11,608][00294] Num frames 3300... [2023-07-24 01:51:11,987][00294] Num frames 3400... [2023-07-24 01:51:12,394][00294] Num frames 3500... [2023-07-24 01:51:12,834][00294] Num frames 3600... [2023-07-24 01:51:13,287][00294] Num frames 3700... [2023-07-24 01:51:13,759][00294] Num frames 3800... [2023-07-24 01:51:14,215][00294] Num frames 3900... [2023-07-24 01:51:14,652][00294] Num frames 4000... [2023-07-24 01:51:15,096][00294] Num frames 4100... [2023-07-24 01:51:15,570][00294] Num frames 4200... [2023-07-24 01:51:16,059][00294] Num frames 4300... [2023-07-24 01:51:16,565][00294] Num frames 4400... [2023-07-24 01:51:17,058][00294] Num frames 4500... [2023-07-24 01:51:17,520][00294] Num frames 4600... [2023-07-24 01:51:17,985][00294] Num frames 4700... [2023-07-24 01:51:18,389][00294] Num frames 4800... [2023-07-24 01:51:18,795][00294] Num frames 4900... [2023-07-24 01:51:19,192][00294] Num frames 5000... [2023-07-24 01:51:19,575][00294] Num frames 5100... [2023-07-24 01:51:19,972][00294] Num frames 5200... [2023-07-24 01:51:20,329][00294] Num frames 5300... [2023-07-24 01:51:20,590][00294] Num frames 5400... [2023-07-24 01:51:20,848][00294] Num frames 5500... [2023-07-24 01:51:21,108][00294] Num frames 5600... [2023-07-24 01:51:21,388][00294] Num frames 5700... [2023-07-24 01:51:21,641][00294] Num frames 5800... [2023-07-24 01:51:21,906][00294] Num frames 5900... [2023-07-24 01:51:22,156][00294] Num frames 6000... [2023-07-24 01:51:22,421][00294] Num frames 6100... [2023-07-24 01:51:22,674][00294] Num frames 6200... [2023-07-24 01:51:22,934][00294] Num frames 6300... [2023-07-24 01:51:23,189][00294] Num frames 6400... [2023-07-24 01:51:23,453][00294] Num frames 6500... [2023-07-24 01:51:23,702][00294] Num frames 6600... [2023-07-24 01:51:23,968][00294] Num frames 6700... [2023-07-24 01:51:24,222][00294] Num frames 6800... [2023-07-24 01:51:24,494][00294] Num frames 6900... [2023-07-24 01:51:24,748][00294] Num frames 7000... [2023-07-24 01:51:25,007][00294] Num frames 7100... [2023-07-24 01:51:25,264][00294] Num frames 7200... [2023-07-24 01:51:25,534][00294] Num frames 7300... [2023-07-24 01:51:25,785][00294] Num frames 7400... [2023-07-24 01:51:26,052][00294] Num frames 7500... [2023-07-24 01:51:26,305][00294] Num frames 7600... [2023-07-24 01:51:26,578][00294] Num frames 7700... [2023-07-24 01:51:26,833][00294] Num frames 7800... [2023-07-24 01:51:27,106][00294] Num frames 7900... [2023-07-24 01:51:27,379][00294] Num frames 8000... [2023-07-24 01:51:27,650][00294] Num frames 8100... [2023-07-24 01:51:27,938][00294] Num frames 8200... [2023-07-24 01:51:28,325][00294] Num frames 8300... [2023-07-24 01:51:28,720][00294] DAMAGECOUNT value on done: 227.0 [2023-07-24 01:51:28,729][00294] Sum rewards: 6.805, reward structure: {'DEATHCOUNT': '-10.500', 'HEALTH': '-4.365', 'ARMOR': '0.016', 'WEAPON1': '0.020', 'AMMO5': '0.020', 'AMMO2': '0.043', 'HITCOUNT': '0.160', 'AMMO6': '0.160', 'AMMO7': '0.160', 'AMMO3': '0.182', 'WEAPON7': '0.200', 'AMMO4': '0.216', 'weapon5': '0.272', 'WEAPON4': '0.300', 'weapon7': '0.322', 'WEAPON5': '0.400', 'DAMAGECOUNT': '0.681', 'WEAPON3': '1.200', 'weapon4': '1.752', 'FRAGCOUNT': '2.000', 'weapon2': '4.648', 'weapon3': '8.918'} [2023-07-24 01:51:28,805][00294] Avg episode rewards: #0: 6.805, true rewards: #0: 2.000 [2023-07-24 01:51:28,811][00294] Avg episode reward: 6.805, avg true_objective: 2.000 [2023-07-24 01:51:28,825][00294] Num frames 8400... [2023-07-24 01:51:29,222][00294] Num frames 8500... [2023-07-24 01:51:29,608][00294] Num frames 8600... [2023-07-24 01:51:30,000][00294] Num frames 8700... [2023-07-24 01:51:30,380][00294] Num frames 8800... [2023-07-24 01:51:30,778][00294] Num frames 8900... [2023-07-24 01:51:31,166][00294] Num frames 9000... [2023-07-24 01:51:31,569][00294] Num frames 9100... [2023-07-24 01:51:31,969][00294] Num frames 9200... [2023-07-24 01:51:32,368][00294] Num frames 9300... [2023-07-24 01:51:32,764][00294] Num frames 9400... [2023-07-24 01:51:33,074][00294] Num frames 9500... [2023-07-24 01:51:33,346][00294] Num frames 9600... [2023-07-24 01:51:33,598][00294] Num frames 9700... [2023-07-24 01:51:33,862][00294] Num frames 9800... [2023-07-24 01:51:34,118][00294] Num frames 9900... [2023-07-24 01:51:34,375][00294] Num frames 10000... [2023-07-24 01:51:34,631][00294] Num frames 10100... [2023-07-24 01:51:34,911][00294] Num frames 10200... [2023-07-24 01:51:35,181][00294] Num frames 10300... [2023-07-24 01:51:35,440][00294] Num frames 10400... [2023-07-24 01:51:35,695][00294] Num frames 10500... [2023-07-24 01:51:35,943][00294] Num frames 10600... [2023-07-24 01:51:36,199][00294] Num frames 10700... [2023-07-24 01:51:36,463][00294] Num frames 10800... [2023-07-24 01:51:36,718][00294] Num frames 10900... [2023-07-24 01:51:36,989][00294] Num frames 11000... [2023-07-24 01:51:37,245][00294] Num frames 11100... [2023-07-24 01:51:37,501][00294] Num frames 11200... [2023-07-24 01:51:37,753][00294] Num frames 11300... [2023-07-24 01:51:38,016][00294] Num frames 11400... [2023-07-24 01:51:38,274][00294] Num frames 11500... [2023-07-24 01:51:38,531][00294] Num frames 11600... [2023-07-24 01:51:38,789][00294] Num frames 11700... [2023-07-24 01:51:39,054][00294] Num frames 11800... [2023-07-24 01:51:39,311][00294] Num frames 11900... [2023-07-24 01:51:39,574][00294] Num frames 12000... [2023-07-24 01:51:39,834][00294] Num frames 12100... [2023-07-24 01:51:40,096][00294] Num frames 12200... [2023-07-24 01:51:40,349][00294] Num frames 12300... [2023-07-24 01:51:40,604][00294] Num frames 12400... [2023-07-24 01:51:40,860][00294] Num frames 12500... [2023-07-24 01:51:41,111][00294] Num frames 12600... [2023-07-24 01:51:41,377][00294] Num frames 12700... [2023-07-24 01:51:41,632][00294] Num frames 12800... [2023-07-24 01:51:41,897][00294] Num frames 12900... [2023-07-24 01:51:42,155][00294] Num frames 13000... [2023-07-24 01:51:42,413][00294] Num frames 13100... [2023-07-24 01:51:42,668][00294] Num frames 13200... [2023-07-24 01:51:42,946][00294] Num frames 13300... [2023-07-24 01:51:43,318][00294] Num frames 13400... [2023-07-24 01:51:43,684][00294] Num frames 13500... [2023-07-24 01:51:44,062][00294] Num frames 13600... [2023-07-24 01:51:44,456][00294] Num frames 13700... [2023-07-24 01:51:44,831][00294] Num frames 13800... [2023-07-24 01:51:45,219][00294] Num frames 13900... [2023-07-24 01:51:45,596][00294] Num frames 14000... [2023-07-24 01:51:45,975][00294] Num frames 14100... [2023-07-24 01:51:46,379][00294] Num frames 14200... [2023-07-24 01:51:46,760][00294] Num frames 14300... [2023-07-24 01:51:47,160][00294] Num frames 14400... [2023-07-24 01:51:47,557][00294] Num frames 14500... [2023-07-24 01:51:47,951][00294] Num frames 14600... [2023-07-24 01:51:48,241][00294] Num frames 14700... [2023-07-24 01:51:48,501][00294] Num frames 14800... [2023-07-24 01:51:48,758][00294] Num frames 14900... [2023-07-24 01:51:49,047][00294] Num frames 15000... [2023-07-24 01:51:49,303][00294] Num frames 15100... [2023-07-24 01:51:49,555][00294] Num frames 15200... [2023-07-24 01:51:49,807][00294] Num frames 15300... [2023-07-24 01:51:50,063][00294] Num frames 15400... [2023-07-24 01:51:50,335][00294] Num frames 15500... [2023-07-24 01:51:50,591][00294] Num frames 15600... [2023-07-24 01:51:50,839][00294] Num frames 15700... [2023-07-24 01:51:51,103][00294] Num frames 15800... [2023-07-24 01:51:51,372][00294] Num frames 15900... [2023-07-24 01:51:51,635][00294] Num frames 16000... [2023-07-24 01:51:51,895][00294] Num frames 16100... [2023-07-24 01:51:52,159][00294] Num frames 16200... [2023-07-24 01:51:52,431][00294] Num frames 16300... [2023-07-24 01:51:52,689][00294] Num frames 16400... [2023-07-24 01:51:52,943][00294] Num frames 16500... [2023-07-24 01:51:53,217][00294] Num frames 16600... [2023-07-24 01:51:53,494][00294] Num frames 16700... [2023-07-24 01:51:53,754][00294] DAMAGECOUNT value on done: 342.0 [2023-07-24 01:51:53,820][00294] Avg episode rewards: #0: 4.570, true rewards: #0: 1.000 [2023-07-24 01:51:53,822][00294] Avg episode reward: 4.570, avg true_objective: 1.000 [2023-07-24 01:51:53,838][00294] Num frames 16800... [2023-07-24 01:51:54,095][00294] Num frames 16900... [2023-07-24 01:51:54,362][00294] Num frames 17000... [2023-07-24 01:51:54,612][00294] Num frames 17100... [2023-07-24 01:51:54,869][00294] Num frames 17200... [2023-07-24 01:51:55,135][00294] Num frames 17300... [2023-07-24 01:51:55,393][00294] Num frames 17400... [2023-07-24 01:51:55,652][00294] Num frames 17500... [2023-07-24 01:51:55,897][00294] Num frames 17600... [2023-07-24 01:51:56,159][00294] Num frames 17700... [2023-07-24 01:51:56,425][00294] Num frames 17800... [2023-07-24 01:51:56,683][00294] Num frames 17900... [2023-07-24 01:51:56,935][00294] Num frames 18000... [2023-07-24 01:51:57,206][00294] Num frames 18100... [2023-07-24 01:51:57,480][00294] Num frames 18200... [2023-07-24 01:51:57,729][00294] Num frames 18300... [2023-07-24 01:51:58,000][00294] Num frames 18400... [2023-07-24 01:51:58,380][00294] Num frames 18500... [2023-07-24 01:51:58,764][00294] Num frames 18600... [2023-07-24 01:51:59,133][00294] Num frames 18700... [2023-07-24 01:51:59,514][00294] Num frames 18800... [2023-07-24 01:51:59,897][00294] Num frames 18900... [2023-07-24 01:52:00,288][00294] Num frames 19000... [2023-07-24 01:52:00,672][00294] Num frames 19100... [2023-07-24 01:52:01,068][00294] Num frames 19200... [2023-07-24 01:52:01,463][00294] Num frames 19300... [2023-07-24 01:52:01,856][00294] Num frames 19400... [2023-07-24 01:52:02,256][00294] Num frames 19500... [2023-07-24 01:52:02,659][00294] Num frames 19600... [2023-07-24 01:52:03,052][00294] Num frames 19700... [2023-07-24 01:52:03,345][00294] Num frames 19800... [2023-07-24 01:52:03,610][00294] Num frames 19900... [2023-07-24 01:52:03,862][00294] Num frames 20000... [2023-07-24 01:52:04,118][00294] Num frames 20100... [2023-07-24 01:52:04,369][00294] Num frames 20200... [2023-07-24 01:52:04,612][00294] Num frames 20300... [2023-07-24 01:52:04,875][00294] Num frames 20400... [2023-07-24 01:52:05,127][00294] Num frames 20500... [2023-07-24 01:52:05,385][00294] Num frames 20600... [2023-07-24 01:52:05,632][00294] Num frames 20700... [2023-07-24 01:52:05,887][00294] Num frames 20800... [2023-07-24 01:52:06,146][00294] Num frames 20900... [2023-07-24 01:52:06,400][00294] Num frames 21000... [2023-07-24 01:52:06,659][00294] Num frames 21100... [2023-07-24 01:52:06,926][00294] Num frames 21200... [2023-07-24 01:52:07,183][00294] Num frames 21300... [2023-07-24 01:52:07,448][00294] Num frames 21400... [2023-07-24 01:52:07,716][00294] Num frames 21500... [2023-07-24 01:52:07,984][00294] Num frames 21600... [2023-07-24 01:52:08,246][00294] Num frames 21700... [2023-07-24 01:52:08,506][00294] Num frames 21800... [2023-07-24 01:52:08,771][00294] Num frames 21900... [2023-07-24 01:52:09,030][00294] Num frames 22000... [2023-07-24 01:52:09,292][00294] Num frames 22100... [2023-07-24 01:52:09,553][00294] Num frames 22200... [2023-07-24 01:52:09,814][00294] Num frames 22300... [2023-07-24 01:52:10,087][00294] Num frames 22400... [2023-07-24 01:52:10,352][00294] Num frames 22500... [2023-07-24 01:52:10,609][00294] Num frames 22600... [2023-07-24 01:52:10,869][00294] Num frames 22700... [2023-07-24 01:52:11,128][00294] Num frames 22800... [2023-07-24 01:52:11,399][00294] Num frames 22900... [2023-07-24 01:52:11,655][00294] Num frames 23000... [2023-07-24 01:52:11,916][00294] Num frames 23100... [2023-07-24 01:52:12,181][00294] Num frames 23200... [2023-07-24 01:52:12,437][00294] Num frames 23300... [2023-07-24 01:52:12,704][00294] Num frames 23400... [2023-07-24 01:52:12,983][00294] Num frames 23500... [2023-07-24 01:52:13,306][00294] Num frames 23600... [2023-07-24 01:52:13,681][00294] Num frames 23700... [2023-07-24 01:52:14,061][00294] Num frames 23800... [2023-07-24 01:52:14,439][00294] Num frames 23900... [2023-07-24 01:52:14,837][00294] Num frames 24000... [2023-07-24 01:52:15,211][00294] Num frames 24100... [2023-07-24 01:52:15,602][00294] Num frames 24200... [2023-07-24 01:52:15,997][00294] Num frames 24300... [2023-07-24 01:52:16,392][00294] Num frames 24400... [2023-07-24 01:52:16,799][00294] Num frames 24500... [2023-07-24 01:52:17,206][00294] Num frames 24600... [2023-07-24 01:52:17,609][00294] Num frames 24700... [2023-07-24 01:52:18,000][00294] Num frames 24800... [2023-07-24 01:52:18,421][00294] Num frames 24900... [2023-07-24 01:52:18,682][00294] Num frames 25000... [2023-07-24 01:52:18,942][00294] Num frames 25100... [2023-07-24 01:52:19,242][00294] DAMAGECOUNT value on done: 672.0 [2023-07-24 01:52:19,245][00294] Sum rewards: 7.329, reward structure: {'DEATHCOUNT': '-9.750', 'HEALTH': '-3.935', 'AMMO5': '0.010', 'AMMO2': '0.012', 'ARMOR': '0.028', 'AMMO4': '0.059', 'WEAPON1': '0.060', 'AMMO3': '0.163', 'WEAPON5': '0.200', 'WEAPON4': '0.200', 'AMMO6': '0.200', 'WEAPON7': '0.200', 'AMMO7': '0.200', 'weapon7': '0.200', 'HITCOUNT': '0.240', 'weapon5': '0.304', 'DAMAGECOUNT': '0.990', 'WEAPON3': '1.100', 'FRAGCOUNT': '2.000', 'weapon4': '2.762', 'weapon2': '4.272', 'weapon3': '7.814'} [2023-07-24 01:52:19,312][00294] Avg episode rewards: #0: 5.490, true rewards: #0: 1.333 [2023-07-24 01:52:19,315][00294] Avg episode reward: 5.490, avg true_objective: 1.333 [2023-07-24 01:52:19,330][00294] Num frames 25200... [2023-07-24 01:52:19,591][00294] Num frames 25300... [2023-07-24 01:52:19,844][00294] Num frames 25400... [2023-07-24 01:52:20,096][00294] Num frames 25500... [2023-07-24 01:52:20,377][00294] Num frames 25600... [2023-07-24 01:52:20,640][00294] Num frames 25700... [2023-07-24 01:52:20,896][00294] Num frames 25800... [2023-07-24 01:52:21,151][00294] Num frames 25900... [2023-07-24 01:52:21,420][00294] Num frames 26000... [2023-07-24 01:52:21,672][00294] Num frames 26100... [2023-07-24 01:52:21,925][00294] Num frames 26200... [2023-07-24 01:52:22,180][00294] Num frames 26300... [2023-07-24 01:52:22,454][00294] Num frames 26400... [2023-07-24 01:52:22,710][00294] Num frames 26500... [2023-07-24 01:52:22,954][00294] Num frames 26600... [2023-07-24 01:52:23,211][00294] Num frames 26700... [2023-07-24 01:52:23,480][00294] Num frames 26800... [2023-07-24 01:52:23,743][00294] Num frames 26900... [2023-07-24 01:52:23,999][00294] Num frames 27000... [2023-07-24 01:52:24,264][00294] Num frames 27100... [2023-07-24 01:52:24,525][00294] Num frames 27200... [2023-07-24 01:52:24,780][00294] Num frames 27300... [2023-07-24 01:52:25,036][00294] Num frames 27400... [2023-07-24 01:52:25,303][00294] Num frames 27500... [2023-07-24 01:52:25,566][00294] Num frames 27600... [2023-07-24 01:52:25,818][00294] Num frames 27700... [2023-07-24 01:52:26,068][00294] Num frames 27800... [2023-07-24 01:52:26,324][00294] Num frames 27900... [2023-07-24 01:52:26,585][00294] Num frames 28000... [2023-07-24 01:52:26,838][00294] Num frames 28100... [2023-07-24 01:52:27,084][00294] Num frames 28200... [2023-07-24 01:52:27,348][00294] Num frames 28300... [2023-07-24 01:52:27,600][00294] Num frames 28400... [2023-07-24 01:52:27,855][00294] Num frames 28500... [2023-07-24 01:52:28,104][00294] Num frames 28600... [2023-07-24 01:52:28,372][00294] Num frames 28700... [2023-07-24 01:52:28,747][00294] Num frames 28800... [2023-07-24 01:52:29,111][00294] Num frames 28900... [2023-07-24 01:52:29,486][00294] Num frames 29000... [2023-07-24 01:52:29,858][00294] Num frames 29100... [2023-07-24 01:52:30,235][00294] Num frames 29200... [2023-07-24 01:52:30,618][00294] Num frames 29300... [2023-07-24 01:52:30,989][00294] Num frames 29400... [2023-07-24 01:52:31,379][00294] Num frames 29500... [2023-07-24 01:52:31,781][00294] Num frames 29600... [2023-07-24 01:52:32,170][00294] Num frames 29700... [2023-07-24 01:52:32,570][00294] Num frames 29800... [2023-07-24 01:52:32,964][00294] Num frames 29900... [2023-07-24 01:52:33,351][00294] Num frames 30000... [2023-07-24 01:52:33,629][00294] Num frames 30100... [2023-07-24 01:52:33,890][00294] Num frames 30200... [2023-07-24 01:52:34,149][00294] Num frames 30300... [2023-07-24 01:52:34,403][00294] Num frames 30400... [2023-07-24 01:52:34,651][00294] Num frames 30500... [2023-07-24 01:52:34,914][00294] Num frames 30600... [2023-07-24 01:52:35,161][00294] Num frames 30700... [2023-07-24 01:52:35,423][00294] Num frames 30800... [2023-07-24 01:52:35,676][00294] Num frames 30900... [2023-07-24 01:52:35,941][00294] Num frames 31000... [2023-07-24 01:52:36,191][00294] Num frames 31100... [2023-07-24 01:52:36,450][00294] Num frames 31200... [2023-07-24 01:52:36,696][00294] Num frames 31300... [2023-07-24 01:52:36,971][00294] Num frames 31400... [2023-07-24 01:52:37,232][00294] Num frames 31500... [2023-07-24 01:52:37,496][00294] Num frames 31600... [2023-07-24 01:52:37,750][00294] Num frames 31700... [2023-07-24 01:52:38,022][00294] Num frames 31800... [2023-07-24 01:52:38,277][00294] Num frames 31900... [2023-07-24 01:52:38,529][00294] Num frames 32000... [2023-07-24 01:52:38,787][00294] Num frames 32100... [2023-07-24 01:52:39,057][00294] Num frames 32200... [2023-07-24 01:52:39,318][00294] Num frames 32300... [2023-07-24 01:52:39,577][00294] Num frames 32400... [2023-07-24 01:52:39,836][00294] Num frames 32500... [2023-07-24 01:52:40,096][00294] Num frames 32600... [2023-07-24 01:52:40,371][00294] Num frames 32700... [2023-07-24 01:52:40,623][00294] Num frames 32800... [2023-07-24 01:52:40,887][00294] Num frames 32900... [2023-07-24 01:52:41,149][00294] Num frames 33000... [2023-07-24 01:52:41,433][00294] Num frames 33100... [2023-07-24 01:52:41,682][00294] Num frames 33200... [2023-07-24 01:52:41,947][00294] Num frames 33300... [2023-07-24 01:52:42,205][00294] Num frames 33400... [2023-07-24 01:52:42,465][00294] Num frames 33500... [2023-07-24 01:52:42,709][00294] DAMAGECOUNT value on done: 697.0 [2023-07-24 01:52:42,777][00294] Avg episode rewards: #0: 5.786, true rewards: #0: 1.000 [2023-07-24 01:52:42,780][00294] Avg episode reward: 5.786, avg true_objective: 1.000 [2023-07-24 01:52:42,799][00294] Num frames 33600... [2023-07-24 01:52:43,067][00294] Num frames 33700... [2023-07-24 01:52:43,323][00294] Num frames 33800... [2023-07-24 01:52:43,665][00294] Num frames 33900... [2023-07-24 01:52:44,069][00294] Num frames 34000... [2023-07-24 01:52:44,442][00294] Num frames 34100... [2023-07-24 01:52:44,831][00294] Num frames 34200... [2023-07-24 01:52:45,245][00294] Num frames 34300... [2023-07-24 01:52:45,615][00294] Num frames 34400... [2023-07-24 01:52:45,998][00294] Num frames 34500... [2023-07-24 01:52:46,405][00294] Num frames 34600... [2023-07-24 01:52:46,792][00294] Num frames 34700... [2023-07-24 01:52:47,201][00294] Num frames 34800... [2023-07-24 01:52:47,610][00294] Num frames 34900... [2023-07-24 01:52:47,993][00294] Num frames 35000... [2023-07-24 01:52:48,388][00294] Num frames 35100... [2023-07-24 01:52:48,688][00294] Num frames 35200... [2023-07-24 01:52:48,946][00294] Num frames 35300... [2023-07-24 01:52:49,219][00294] Num frames 35400... [2023-07-24 01:52:49,505][00294] Num frames 35500... [2023-07-24 01:52:49,770][00294] Num frames 35600... [2023-07-24 01:52:50,036][00294] Num frames 35700... [2023-07-24 01:52:50,316][00294] Num frames 35800... [2023-07-24 01:52:50,577][00294] Num frames 35900... [2023-07-24 01:52:50,836][00294] Num frames 36000... [2023-07-24 01:52:51,093][00294] Num frames 36100... [2023-07-24 01:52:51,367][00294] Num frames 36200... [2023-07-24 01:52:51,617][00294] Num frames 36300... [2023-07-24 01:52:51,875][00294] Num frames 36400... [2023-07-24 01:52:52,126][00294] Num frames 36500... [2023-07-24 01:52:52,394][00294] Num frames 36600... [2023-07-24 01:52:52,650][00294] Num frames 36700... [2023-07-24 01:52:52,901][00294] Num frames 36800... [2023-07-24 01:52:53,154][00294] Num frames 36900... [2023-07-24 01:52:53,423][00294] Num frames 37000... [2023-07-24 01:52:53,682][00294] Num frames 37100... [2023-07-24 01:52:53,935][00294] Num frames 37200... [2023-07-24 01:52:54,184][00294] Num frames 37300... [2023-07-24 01:52:54,449][00294] Num frames 37400... [2023-07-24 01:52:54,703][00294] Num frames 37500... [2023-07-24 01:52:54,956][00294] Num frames 37600... [2023-07-24 01:52:55,208][00294] Num frames 37700... [2023-07-24 01:52:55,485][00294] Num frames 37800... [2023-07-24 01:52:55,739][00294] Num frames 37900... [2023-07-24 01:52:55,994][00294] Num frames 38000... [2023-07-24 01:52:56,251][00294] Num frames 38100... [2023-07-24 01:52:56,523][00294] Num frames 38200... [2023-07-24 01:52:56,773][00294] Num frames 38300... [2023-07-24 01:52:57,023][00294] Num frames 38400... [2023-07-24 01:52:57,281][00294] Num frames 38500... [2023-07-24 01:52:57,554][00294] Num frames 38600... [2023-07-24 01:52:57,810][00294] Num frames 38700... [2023-07-24 01:52:58,072][00294] Num frames 38800... [2023-07-24 01:52:58,331][00294] Num frames 38900... [2023-07-24 01:52:58,655][00294] Num frames 39000... [2023-07-24 01:52:59,032][00294] Num frames 39100... [2023-07-24 01:52:59,446][00294] Num frames 39200... [2023-07-24 01:52:59,837][00294] Num frames 39300... [2023-07-24 01:53:00,218][00294] Num frames 39400... [2023-07-24 01:53:00,610][00294] Num frames 39500... [2023-07-24 01:53:00,994][00294] Num frames 39600... [2023-07-24 01:53:01,388][00294] Num frames 39700... [2023-07-24 01:53:01,791][00294] Num frames 39800... [2023-07-24 01:53:02,171][00294] Num frames 39900... [2023-07-24 01:53:02,556][00294] Num frames 40000... [2023-07-24 01:53:02,958][00294] Num frames 40100... [2023-07-24 01:53:03,362][00294] Num frames 40200... [2023-07-24 01:53:03,713][00294] Num frames 40300... [2023-07-24 01:53:03,969][00294] Num frames 40400... [2023-07-24 01:53:04,233][00294] Num frames 40500... [2023-07-24 01:53:04,481][00294] Num frames 40600... [2023-07-24 01:53:04,756][00294] Num frames 40700... [2023-07-24 01:53:05,006][00294] Num frames 40800... [2023-07-24 01:53:05,268][00294] Num frames 40900... [2023-07-24 01:53:05,529][00294] Num frames 41000... [2023-07-24 01:53:05,808][00294] Num frames 41100... [2023-07-24 01:53:06,063][00294] Num frames 41200... [2023-07-24 01:53:06,323][00294] Num frames 41300... [2023-07-24 01:53:06,584][00294] Num frames 41400... [2023-07-24 01:53:06,850][00294] Num frames 41500... [2023-07-24 01:53:07,107][00294] Num frames 41600... [2023-07-24 01:53:07,364][00294] Num frames 41700... [2023-07-24 01:53:07,616][00294] Num frames 41800... [2023-07-24 01:53:07,881][00294] Num frames 41900... [2023-07-24 01:53:08,119][00294] DAMAGECOUNT value on done: 807.0 [2023-07-24 01:53:08,122][00294] Sum rewards: 9.104, reward structure: {'DEATHCOUNT': '-7.500', 'HEALTH': '-3.016', 'AMMO2': '0.015', 'WEAPON1': '0.020', 'AMMO5': '0.029', 'AMMO4': '0.073', 'HITCOUNT': '0.090', 'AMMO3': '0.119', 'WEAPON4': '0.200', 'DAMAGECOUNT': '0.330', 'WEAPON5': '0.400', 'ARMOR': '0.508', 'weapon4': '0.670', 'WEAPON3': '0.900', 'FRAGCOUNT': '1.000', 'weapon5': '2.654', 'weapon2': '4.394', 'weapon3': '8.218'} [2023-07-24 01:53:08,189][00294] Avg episode rewards: #0: 6.450, true rewards: #0: 1.000 [2023-07-24 01:53:08,192][00294] Avg episode reward: 6.450, avg true_objective: 1.000 [2023-07-24 01:53:08,218][00294] Num frames 42000... [2023-07-24 01:53:08,476][00294] Num frames 42100... [2023-07-24 01:53:08,746][00294] Num frames 42200... [2023-07-24 01:53:09,007][00294] Num frames 42300... [2023-07-24 01:53:09,273][00294] Num frames 42400... [2023-07-24 01:53:09,525][00294] Num frames 42500... [2023-07-24 01:53:09,792][00294] Num frames 42600... [2023-07-24 01:53:10,050][00294] Num frames 42700... [2023-07-24 01:53:10,308][00294] Num frames 42800... [2023-07-24 01:53:10,568][00294] Num frames 42900... [2023-07-24 01:53:10,838][00294] Num frames 43000... [2023-07-24 01:53:11,094][00294] Num frames 43100... [2023-07-24 01:53:11,376][00294] Num frames 43200... [2023-07-24 01:53:11,635][00294] Num frames 43300... [2023-07-24 01:53:11,901][00294] Num frames 43400... [2023-07-24 01:53:12,149][00294] Num frames 43500... [2023-07-24 01:53:12,420][00294] Num frames 43600... [2023-07-24 01:53:12,685][00294] Num frames 43700... [2023-07-24 01:53:12,949][00294] Num frames 43800... [2023-07-24 01:53:13,202][00294] Num frames 43900... [2023-07-24 01:53:13,463][00294] Num frames 44000... [2023-07-24 01:53:13,770][00294] Num frames 44100... [2023-07-24 01:53:14,150][00294] Num frames 44200... [2023-07-24 01:53:14,523][00294] Num frames 44300... [2023-07-24 01:53:14,898][00294] Num frames 44400... [2023-07-24 01:53:15,287][00294] Num frames 44500... [2023-07-24 01:53:15,660][00294] Num frames 44600... [2023-07-24 01:53:16,036][00294] Num frames 44700... [2023-07-24 01:53:16,420][00294] Num frames 44800... [2023-07-24 01:53:16,809][00294] Num frames 44900... [2023-07-24 01:53:17,202][00294] Num frames 45000... [2023-07-24 01:53:17,595][00294] Num frames 45100... [2023-07-24 01:53:17,992][00294] Num frames 45200... [2023-07-24 01:53:18,394][00294] Num frames 45300... [2023-07-24 01:53:18,778][00294] Num frames 45400... [2023-07-24 01:53:19,034][00294] Num frames 45500... [2023-07-24 01:53:19,300][00294] Num frames 45600... [2023-07-24 01:53:19,557][00294] Num frames 45700... [2023-07-24 01:53:19,839][00294] Num frames 45800... [2023-07-24 01:53:20,087][00294] Num frames 45900... [2023-07-24 01:53:20,357][00294] Num frames 46000... [2023-07-24 01:53:20,612][00294] Num frames 46100... [2023-07-24 01:53:20,860][00294] Num frames 46200... [2023-07-24 01:53:21,129][00294] Num frames 46300... [2023-07-24 01:53:21,393][00294] Num frames 46400... [2023-07-24 01:53:21,651][00294] Num frames 46500... [2023-07-24 01:53:21,913][00294] Num frames 46600... [2023-07-24 01:53:22,179][00294] Num frames 46700... [2023-07-24 01:53:22,444][00294] Num frames 46800... [2023-07-24 01:53:22,701][00294] Num frames 46900... [2023-07-24 01:53:22,961][00294] Num frames 47000... [2023-07-24 01:53:23,235][00294] Num frames 47100... [2023-07-24 01:53:23,496][00294] Num frames 47200... [2023-07-24 01:53:23,762][00294] Num frames 47300... [2023-07-24 01:53:24,016][00294] Num frames 47400... [2023-07-24 01:53:24,282][00294] Num frames 47500... [2023-07-24 01:53:24,551][00294] Num frames 47600... [2023-07-24 01:53:24,813][00294] Num frames 47700... [2023-07-24 01:53:25,072][00294] Num frames 47800... [2023-07-24 01:53:25,333][00294] Num frames 47900... [2023-07-24 01:53:25,603][00294] Num frames 48000... [2023-07-24 01:53:25,862][00294] Num frames 48100... [2023-07-24 01:53:26,120][00294] Num frames 48200... [2023-07-24 01:53:26,394][00294] Num frames 48300... [2023-07-24 01:53:26,659][00294] Num frames 48400... [2023-07-24 01:53:26,915][00294] Num frames 48500... [2023-07-24 01:53:27,176][00294] Num frames 48600... [2023-07-24 01:53:27,453][00294] Num frames 48700... [2023-07-24 01:53:27,737][00294] Num frames 48800... [2023-07-24 01:53:28,000][00294] Num frames 48900... [2023-07-24 01:53:28,254][00294] Num frames 49000... [2023-07-24 01:53:28,532][00294] Num frames 49100... [2023-07-24 01:53:28,820][00294] Num frames 49200... [2023-07-24 01:53:29,205][00294] Num frames 49300... [2023-07-24 01:53:29,588][00294] Num frames 49400... [2023-07-24 01:53:29,963][00294] Num frames 49500... [2023-07-24 01:53:30,340][00294] Num frames 49600... [2023-07-24 01:53:30,721][00294] Num frames 49700... [2023-07-24 01:53:31,112][00294] Num frames 49800... [2023-07-24 01:53:31,508][00294] Num frames 49900... [2023-07-24 01:53:31,932][00294] Num frames 50000... [2023-07-24 01:53:32,349][00294] Num frames 50100... [2023-07-24 01:53:32,780][00294] Num frames 50200... [2023-07-24 01:53:33,201][00294] Num frames 50300... [2023-07-24 01:53:33,582][00294] DAMAGECOUNT value on done: 1143.0 [2023-07-24 01:53:33,591][00294] Sum rewards: 10.028, reward structure: {'DEATHCOUNT': '-9.000', 'HEALTH': '-3.725', 'AMMO2': '0.010', 'AMMO5': '0.019', 'AMMO4': '0.048', 'AMMO6': '0.120', 'AMMO7': '0.120', 'AMMO3': '0.196', 'WEAPON4': '0.200', 'WEAPON7': '0.200', 'HITCOUNT': '0.250', 'weapon7': '0.380', 'WEAPON5': '0.400', 'weapon5': '0.986', 'DAMAGECOUNT': '1.008', 'WEAPON3': '1.200', 'FRAGCOUNT': '3.000', 'weapon2': '4.024', 'weapon4': '4.792', 'weapon3': '5.800'} [2023-07-24 01:53:33,667][00294] Avg episode rewards: #0: 7.046, true rewards: #0: 1.333 [2023-07-24 01:53:33,670][00294] Avg episode reward: 7.046, avg true_objective: 1.333 [2023-07-24 01:53:33,699][00294] Num frames 50400... [2023-07-24 01:53:34,022][00294] Num frames 50500... [2023-07-24 01:53:34,284][00294] Num frames 50600... [2023-07-24 01:53:34,548][00294] Num frames 50700... [2023-07-24 01:53:34,813][00294] Num frames 50800... [2023-07-24 01:53:35,076][00294] Num frames 50900... [2023-07-24 01:53:35,331][00294] Num frames 51000... [2023-07-24 01:53:35,596][00294] Num frames 51100... [2023-07-24 01:53:35,854][00294] Num frames 51200... [2023-07-24 01:53:36,114][00294] Num frames 51300... [2023-07-24 01:53:36,373][00294] Num frames 51400... [2023-07-24 01:53:36,634][00294] Num frames 51500... [2023-07-24 01:53:36,901][00294] Num frames 51600... [2023-07-24 01:53:37,160][00294] Num frames 51700... [2023-07-24 01:53:37,415][00294] Num frames 51800... [2023-07-24 01:53:37,668][00294] Num frames 51900... [2023-07-24 01:53:37,934][00294] Num frames 52000... [2023-07-24 01:53:38,188][00294] Num frames 52100... [2023-07-24 01:53:38,454][00294] Num frames 52200... [2023-07-24 01:53:38,710][00294] Num frames 52300... [2023-07-24 01:53:38,981][00294] Num frames 52400... [2023-07-24 01:53:39,236][00294] Num frames 52500... [2023-07-24 01:53:39,506][00294] Num frames 52600... [2023-07-24 01:53:39,759][00294] Num frames 52700... [2023-07-24 01:53:40,033][00294] Num frames 52800... [2023-07-24 01:53:40,286][00294] Num frames 52900... [2023-07-24 01:53:40,553][00294] Num frames 53000... [2023-07-24 01:53:40,803][00294] Num frames 53100... [2023-07-24 01:53:41,080][00294] Num frames 53200... [2023-07-24 01:53:41,353][00294] Num frames 53300... [2023-07-24 01:53:41,616][00294] Num frames 53400... [2023-07-24 01:53:41,890][00294] Num frames 53500... [2023-07-24 01:53:42,151][00294] Num frames 53600... [2023-07-24 01:53:42,417][00294] Num frames 53700... [2023-07-24 01:53:42,678][00294] Num frames 53800... [2023-07-24 01:53:42,952][00294] Num frames 53900... [2023-07-24 01:53:43,207][00294] Num frames 54000... [2023-07-24 01:53:43,582][00294] Num frames 54100... [2023-07-24 01:53:43,983][00294] Num frames 54200... [2023-07-24 01:53:44,383][00294] Num frames 54300... [2023-07-24 01:53:44,763][00294] Num frames 54400... [2023-07-24 01:53:45,192][00294] Num frames 54500... [2023-07-24 01:53:45,588][00294] Num frames 54600... [2023-07-24 01:53:46,001][00294] Num frames 54700... [2023-07-24 01:53:46,450][00294] Num frames 54800... [2023-07-24 01:53:46,913][00294] Num frames 54900... [2023-07-24 01:53:47,367][00294] Num frames 55000... [2023-07-24 01:53:47,833][00294] Num frames 55100... [2023-07-24 01:53:48,292][00294] Num frames 55200... [2023-07-24 01:53:48,733][00294] Num frames 55300... [2023-07-24 01:53:49,189][00294] Num frames 55400... [2023-07-24 01:53:49,639][00294] Num frames 55500... [2023-07-24 01:53:50,070][00294] Num frames 55600... [2023-07-24 01:53:50,498][00294] Num frames 55700... [2023-07-24 01:53:50,940][00294] Num frames 55800... [2023-07-24 01:53:51,367][00294] Num frames 55900... [2023-07-24 01:53:51,812][00294] Num frames 56000... [2023-07-24 01:53:52,233][00294] Num frames 56100... [2023-07-24 01:53:52,635][00294] Num frames 56200... [2023-07-24 01:53:53,031][00294] Num frames 56300... [2023-07-24 01:53:53,294][00294] Num frames 56400... [2023-07-24 01:53:53,564][00294] Num frames 56500... [2023-07-24 01:53:53,835][00294] Num frames 56600... [2023-07-24 01:53:54,105][00294] Num frames 56700... [2023-07-24 01:53:54,364][00294] Num frames 56800... [2023-07-24 01:53:54,633][00294] Num frames 56900... [2023-07-24 01:53:54,905][00294] Num frames 57000... [2023-07-24 01:53:55,174][00294] Num frames 57100... [2023-07-24 01:53:55,447][00294] Num frames 57200... [2023-07-24 01:53:55,736][00294] Num frames 57300... [2023-07-24 01:53:56,000][00294] Num frames 57400... [2023-07-24 01:53:56,268][00294] Num frames 57500... [2023-07-24 01:53:56,521][00294] Num frames 57600... [2023-07-24 01:53:56,785][00294] Num frames 57700... [2023-07-24 01:53:57,047][00294] Num frames 57800... [2023-07-24 01:53:57,298][00294] Num frames 57900... [2023-07-24 01:53:57,554][00294] Num frames 58000... [2023-07-24 01:53:57,828][00294] Num frames 58100... [2023-07-24 01:53:58,082][00294] Num frames 58200... [2023-07-24 01:53:58,349][00294] Num frames 58300... [2023-07-24 01:53:58,609][00294] Num frames 58400... [2023-07-24 01:53:58,869][00294] Num frames 58500... [2023-07-24 01:53:59,122][00294] Num frames 58600... [2023-07-24 01:53:59,384][00294] Num frames 58700... [2023-07-24 01:53:59,618][00294] DAMAGECOUNT value on done: 1238.0 [2023-07-24 01:53:59,621][00294] Sum rewards: 4.684, reward structure: {'DEATHCOUNT': '-11.250', 'HEALTH': '-4.420', 'AMMO5': '0.014', 'AMMO2': '0.025', 'WEAPON1': '0.080', 'HITCOUNT': '0.090', 'AMMO4': '0.124', 'AMMO3': '0.169', 'DAMAGECOUNT': '0.285', 'WEAPON4': '0.300', 'WEAPON5': '0.300', 'WEAPON3': '0.900', 'weapon4': '0.952', 'FRAGCOUNT': '2.000', 'weapon5': '2.012', 'weapon2': '6.474', 'weapon3': '6.628'} [2023-07-24 01:53:59,688][00294] Avg episode rewards: #0: 6.709, true rewards: #0: 1.429 [2023-07-24 01:53:59,690][00294] Avg episode reward: 6.709, avg true_objective: 1.429 [2023-07-24 01:53:59,716][00294] Num frames 58800... [2023-07-24 01:53:59,983][00294] Num frames 58900... [2023-07-24 01:54:00,246][00294] Num frames 59000... [2023-07-24 01:54:00,511][00294] Num frames 59100... [2023-07-24 01:54:00,778][00294] Num frames 59200... [2023-07-24 01:54:01,038][00294] Num frames 59300... [2023-07-24 01:54:01,302][00294] Num frames 59400... [2023-07-24 01:54:01,553][00294] Num frames 59500... [2023-07-24 01:54:01,806][00294] Num frames 59600... [2023-07-24 01:54:02,066][00294] Num frames 59700... [2023-07-24 01:54:02,333][00294] Num frames 59800... [2023-07-24 01:54:02,584][00294] Num frames 59900... [2023-07-24 01:54:02,838][00294] Num frames 60000... [2023-07-24 01:54:03,126][00294] Num frames 60100... [2023-07-24 01:54:03,505][00294] Num frames 60200... [2023-07-24 01:54:03,893][00294] Num frames 60300... [2023-07-24 01:54:04,282][00294] Num frames 60400... [2023-07-24 01:54:04,686][00294] Num frames 60500... [2023-07-24 01:54:05,078][00294] Num frames 60600... [2023-07-24 01:54:05,465][00294] Num frames 60700... [2023-07-24 01:54:05,846][00294] Num frames 60800... [2023-07-24 01:54:06,243][00294] Num frames 60900... [2023-07-24 01:54:06,639][00294] Num frames 61000... [2023-07-24 01:54:07,054][00294] Num frames 61100... [2023-07-24 01:54:07,455][00294] Num frames 61200... [2023-07-24 01:54:07,864][00294] Num frames 61300... [2023-07-24 01:54:08,212][00294] Num frames 61400... [2023-07-24 01:54:08,476][00294] Num frames 61500... [2023-07-24 01:54:08,736][00294] Num frames 61600... [2023-07-24 01:54:08,997][00294] Num frames 61700... [2023-07-24 01:54:09,260][00294] Num frames 61800... [2023-07-24 01:54:09,513][00294] Num frames 61900... [2023-07-24 01:54:09,773][00294] Num frames 62000... [2023-07-24 01:54:10,037][00294] Num frames 62100... [2023-07-24 01:54:10,305][00294] Num frames 62200... [2023-07-24 01:54:10,572][00294] Num frames 62300... [2023-07-24 01:54:10,825][00294] Num frames 62400... [2023-07-24 01:54:11,093][00294] Num frames 62500... [2023-07-24 01:54:11,358][00294] Num frames 62600... [2023-07-24 01:54:11,625][00294] Num frames 62700... [2023-07-24 01:54:11,876][00294] Num frames 62800... [2023-07-24 01:54:12,145][00294] Num frames 62900... [2023-07-24 01:54:12,409][00294] Num frames 63000... [2023-07-24 01:54:12,667][00294] Num frames 63100... [2023-07-24 01:54:12,918][00294] Num frames 63200... [2023-07-24 01:54:13,174][00294] Num frames 63300... [2023-07-24 01:54:13,444][00294] Num frames 63400... [2023-07-24 01:54:13,704][00294] Num frames 63500... [2023-07-24 01:54:13,957][00294] Num frames 63600... [2023-07-24 01:54:14,253][00294] Num frames 63700... [2023-07-24 01:54:14,515][00294] Num frames 63800... [2023-07-24 01:54:14,777][00294] Num frames 63900... [2023-07-24 01:54:15,036][00294] Num frames 64000... [2023-07-24 01:54:15,297][00294] Num frames 64100... [2023-07-24 01:54:15,560][00294] Num frames 64200... [2023-07-24 01:54:15,806][00294] Num frames 64300... [2023-07-24 01:54:16,056][00294] Num frames 64400... [2023-07-24 01:54:16,315][00294] Num frames 64500... [2023-07-24 01:54:16,574][00294] Num frames 64600... [2023-07-24 01:54:16,835][00294] Num frames 64700... [2023-07-24 01:54:17,095][00294] Num frames 64800... [2023-07-24 01:54:17,358][00294] Num frames 64900... [2023-07-24 01:54:17,626][00294] Num frames 65000... [2023-07-24 01:54:17,882][00294] Num frames 65100... [2023-07-24 01:54:18,162][00294] Num frames 65200... [2023-07-24 01:54:18,546][00294] Num frames 65300... [2023-07-24 01:54:18,929][00294] Num frames 65400... [2023-07-24 01:54:19,320][00294] Num frames 65500... [2023-07-24 01:54:19,738][00294] Num frames 65600... [2023-07-24 01:54:20,132][00294] Num frames 65700... [2023-07-24 01:54:20,540][00294] Num frames 65800... [2023-07-24 01:54:20,930][00294] Num frames 65900... [2023-07-24 01:54:21,329][00294] Num frames 66000... [2023-07-24 01:54:21,738][00294] Num frames 66100... [2023-07-24 01:54:22,131][00294] Num frames 66200... [2023-07-24 01:54:22,532][00294] Num frames 66300... [2023-07-24 01:54:22,919][00294] Num frames 66400... [2023-07-24 01:54:23,304][00294] Num frames 66500... [2023-07-24 01:54:23,551][00294] Num frames 66600... [2023-07-24 01:54:23,801][00294] Num frames 66700... [2023-07-24 01:54:24,052][00294] Num frames 66800... [2023-07-24 01:54:24,303][00294] Num frames 66900... [2023-07-24 01:54:24,564][00294] Num frames 67000... [2023-07-24 01:54:24,832][00294] Num frames 67100... [2023-07-24 01:54:25,072][00294] DAMAGECOUNT value on done: 1298.0 [2023-07-24 01:54:25,140][00294] Avg episode rewards: #0: 7.438, true rewards: #0: 1.250 [2023-07-24 01:54:25,142][00294] Avg episode reward: 7.438, avg true_objective: 1.250 [2023-07-24 01:54:25,174][00294] Num frames 67200... [2023-07-24 01:54:25,440][00294] Num frames 67300... [2023-07-24 01:54:25,704][00294] Num frames 67400... [2023-07-24 01:54:25,964][00294] Num frames 67500... [2023-07-24 01:54:26,228][00294] Num frames 67600... [2023-07-24 01:54:26,485][00294] Num frames 67700... [2023-07-24 01:54:26,742][00294] Num frames 67800... [2023-07-24 01:54:27,009][00294] Num frames 67900... [2023-07-24 01:54:27,270][00294] Num frames 68000... [2023-07-24 01:54:27,524][00294] Num frames 68100... [2023-07-24 01:54:27,786][00294] Num frames 68200... [2023-07-24 01:54:28,042][00294] Num frames 68300... [2023-07-24 01:54:28,303][00294] Num frames 68400... [2023-07-24 01:54:28,567][00294] Num frames 68500... [2023-07-24 01:54:28,829][00294] Num frames 68600... [2023-07-24 01:54:29,082][00294] Num frames 68700... [2023-07-24 01:54:29,352][00294] Num frames 68800... [2023-07-24 01:54:29,615][00294] Num frames 68900... [2023-07-24 01:54:29,873][00294] Num frames 69000... [2023-07-24 01:54:30,134][00294] Num frames 69100... [2023-07-24 01:54:30,394][00294] Num frames 69200... [2023-07-24 01:54:30,657][00294] Num frames 69300... [2023-07-24 01:54:30,924][00294] Num frames 69400... [2023-07-24 01:54:31,183][00294] Num frames 69500... [2023-07-24 01:54:31,437][00294] Num frames 69600... [2023-07-24 01:54:31,705][00294] Num frames 69700... [2023-07-24 01:54:31,971][00294] Num frames 69800... [2023-07-24 01:54:32,246][00294] Num frames 69900... [2023-07-24 01:54:32,513][00294] Num frames 70000... [2023-07-24 01:54:32,770][00294] Num frames 70100... [2023-07-24 01:54:33,035][00294] Num frames 70200... [2023-07-24 01:54:33,313][00294] Num frames 70300... [2023-07-24 01:54:33,708][00294] Num frames 70400... [2023-07-24 01:54:34,123][00294] Num frames 70500... [2023-07-24 01:54:34,527][00294] Num frames 70600... [2023-07-24 01:54:34,931][00294] Num frames 70700... [2023-07-24 01:54:35,353][00294] Num frames 70800... [2023-07-24 01:54:35,728][00294] Num frames 70900... [2023-07-24 01:54:36,118][00294] Num frames 71000... [2023-07-24 01:54:36,524][00294] Num frames 71100... [2023-07-24 01:54:36,936][00294] Num frames 71200... [2023-07-24 01:54:37,340][00294] Num frames 71300... [2023-07-24 01:54:37,735][00294] Num frames 71400... [2023-07-24 01:54:38,138][00294] Num frames 71500... [2023-07-24 01:54:38,509][00294] Num frames 71600... [2023-07-24 01:54:38,767][00294] Num frames 71700... [2023-07-24 01:54:39,038][00294] Num frames 71800... [2023-07-24 01:54:39,316][00294] Num frames 71900... [2023-07-24 01:54:39,581][00294] Num frames 72000... [2023-07-24 01:54:39,846][00294] Num frames 72100... [2023-07-24 01:54:40,113][00294] Num frames 72200... [2023-07-24 01:54:40,393][00294] Num frames 72300... [2023-07-24 01:54:40,654][00294] Num frames 72400... [2023-07-24 01:54:40,920][00294] Num frames 72500... [2023-07-24 01:54:41,209][00294] Num frames 72600... [2023-07-24 01:54:41,476][00294] Num frames 72700... [2023-07-24 01:54:41,738][00294] Num frames 72800... [2023-07-24 01:54:41,994][00294] Num frames 72900... [2023-07-24 01:54:42,265][00294] Num frames 73000... [2023-07-24 01:54:42,532][00294] Num frames 73100... [2023-07-24 01:54:42,797][00294] Num frames 73200... [2023-07-24 01:54:43,054][00294] Num frames 73300... [2023-07-24 01:54:43,319][00294] Num frames 73400... [2023-07-24 01:54:43,593][00294] Num frames 73500... [2023-07-24 01:54:43,852][00294] Num frames 73600... [2023-07-24 01:54:44,108][00294] Num frames 73700... [2023-07-24 01:54:44,374][00294] Num frames 73800... [2023-07-24 01:54:44,646][00294] Num frames 73900... [2023-07-24 01:54:44,909][00294] Num frames 74000... [2023-07-24 01:54:45,181][00294] Num frames 74100... [2023-07-24 01:54:45,469][00294] Num frames 74200... [2023-07-24 01:54:45,735][00294] Num frames 74300... [2023-07-24 01:54:45,996][00294] Num frames 74400... [2023-07-24 01:54:46,257][00294] Num frames 74500... [2023-07-24 01:54:46,520][00294] Num frames 74600... [2023-07-24 01:54:46,780][00294] Num frames 74700... [2023-07-24 01:54:47,040][00294] Num frames 74800... [2023-07-24 01:54:47,299][00294] Num frames 74900... [2023-07-24 01:54:47,571][00294] Num frames 75000... [2023-07-24 01:54:47,826][00294] Num frames 75100... [2023-07-24 01:54:48,082][00294] Num frames 75200... [2023-07-24 01:54:48,340][00294] Num frames 75300... [2023-07-24 01:54:48,695][00294] Num frames 75400... [2023-07-24 01:54:49,081][00294] Num frames 75500... [2023-07-24 01:54:49,434][00294] DAMAGECOUNT value on done: 1433.0 [2023-07-24 01:54:49,511][00294] Avg episode rewards: #0: 7.398, true rewards: #0: 1.111 [2023-07-24 01:54:49,514][00294] Avg episode reward: 7.398, avg true_objective: 1.111 [2023-07-24 01:54:49,580][00294] Num frames 75600... [2023-07-24 01:54:49,984][00294] Num frames 75700... [2023-07-24 01:54:50,365][00294] Num frames 75800... [2023-07-24 01:54:50,781][00294] Num frames 75900... [2023-07-24 01:54:51,191][00294] Num frames 76000... [2023-07-24 01:54:51,578][00294] Num frames 76100... [2023-07-24 01:54:51,989][00294] Num frames 76200... [2023-07-24 01:54:52,392][00294] Num frames 76300... [2023-07-24 01:54:52,793][00294] Num frames 76400... [2023-07-24 01:54:53,182][00294] Num frames 76500... [2023-07-24 01:54:53,581][00294] Num frames 76600... [2023-07-24 01:54:53,855][00294] Num frames 76700... [2023-07-24 01:54:54,112][00294] Num frames 76800... [2023-07-24 01:54:54,393][00294] Num frames 76900... [2023-07-24 01:54:54,651][00294] Num frames 77000... [2023-07-24 01:54:54,918][00294] Num frames 77100... [2023-07-24 01:54:55,172][00294] Num frames 77200... [2023-07-24 01:54:55,444][00294] Num frames 77300... [2023-07-24 01:54:55,706][00294] Num frames 77400... [2023-07-24 01:54:55,981][00294] Num frames 77500... [2023-07-24 01:54:56,239][00294] Num frames 77600... [2023-07-24 01:54:56,514][00294] Num frames 77700... [2023-07-24 01:54:56,777][00294] Num frames 77800... [2023-07-24 01:54:57,049][00294] Num frames 77900... [2023-07-24 01:54:57,319][00294] Num frames 78000... [2023-07-24 01:54:57,582][00294] Num frames 78100... [2023-07-24 01:54:57,842][00294] Num frames 78200... [2023-07-24 01:54:58,113][00294] Num frames 78300... [2023-07-24 01:54:58,379][00294] Num frames 78400... [2023-07-24 01:54:58,641][00294] Num frames 78500... [2023-07-24 01:54:58,926][00294] Num frames 78600... [2023-07-24 01:54:59,188][00294] Num frames 78700... [2023-07-24 01:54:59,462][00294] Num frames 78800... [2023-07-24 01:54:59,726][00294] Num frames 78900... [2023-07-24 01:54:59,996][00294] Num frames 79000... [2023-07-24 01:55:00,267][00294] Num frames 79100... [2023-07-24 01:55:00,536][00294] Num frames 79200... [2023-07-24 01:55:00,807][00294] Num frames 79300... [2023-07-24 01:55:01,076][00294] Num frames 79400... [2023-07-24 01:55:01,342][00294] Num frames 79500... [2023-07-24 01:55:01,597][00294] Num frames 79600... [2023-07-24 01:55:01,853][00294] Num frames 79700... [2023-07-24 01:55:02,120][00294] Num frames 79800... [2023-07-24 01:55:02,385][00294] Num frames 79900... [2023-07-24 01:55:02,654][00294] Num frames 80000... [2023-07-24 01:55:02,926][00294] Num frames 80100... [2023-07-24 01:55:03,196][00294] Num frames 80200... [2023-07-24 01:55:03,482][00294] Num frames 80300... [2023-07-24 01:55:03,809][00294] Num frames 80400... [2023-07-24 01:55:04,208][00294] Num frames 80500... [2023-07-24 01:55:04,590][00294] Num frames 80600... [2023-07-24 01:55:04,966][00294] Num frames 80700... [2023-07-24 01:55:05,353][00294] Num frames 80800... [2023-07-24 01:55:05,720][00294] Num frames 80900... [2023-07-24 01:55:06,101][00294] Num frames 81000... [2023-07-24 01:55:06,508][00294] Num frames 81100... [2023-07-24 01:55:06,898][00294] Num frames 81200... [2023-07-24 01:55:07,323][00294] Num frames 81300... [2023-07-24 01:55:07,728][00294] Num frames 81400... [2023-07-24 01:55:08,124][00294] Num frames 81500... [2023-07-24 01:55:08,536][00294] Num frames 81600... [2023-07-24 01:55:08,877][00294] Num frames 81700... [2023-07-24 01:55:09,138][00294] Num frames 81800... [2023-07-24 01:55:09,408][00294] Num frames 81900... [2023-07-24 01:55:09,668][00294] Num frames 82000... [2023-07-24 01:55:09,917][00294] Num frames 82100... [2023-07-24 01:55:10,187][00294] Num frames 82200... [2023-07-24 01:55:10,472][00294] Num frames 82300... [2023-07-24 01:55:10,743][00294] Num frames 82400... [2023-07-24 01:55:10,995][00294] Num frames 82500... [2023-07-24 01:55:11,258][00294] Num frames 82600... [2023-07-24 01:55:11,533][00294] Num frames 82700... [2023-07-24 01:55:11,781][00294] Num frames 82800... [2023-07-24 01:55:12,059][00294] Num frames 82900... [2023-07-24 01:55:12,322][00294] Num frames 83000... [2023-07-24 01:55:12,602][00294] Num frames 83100... [2023-07-24 01:55:12,866][00294] Num frames 83200... [2023-07-24 01:55:13,138][00294] Num frames 83300... [2023-07-24 01:55:13,399][00294] Num frames 83400... [2023-07-24 01:55:13,665][00294] Num frames 83500... [2023-07-24 01:55:13,911][00294] Num frames 83600... [2023-07-24 01:55:14,173][00294] Num frames 83700... [2023-07-24 01:55:14,435][00294] Num frames 83800... [2023-07-24 01:55:14,708][00294] Num frames 83900... [2023-07-24 01:55:14,950][00294] DAMAGECOUNT value on done: 1780.0 [2023-07-24 01:55:14,957][00294] Sum rewards: 12.318, reward structure: {'DEATHCOUNT': '-6.750', 'HEALTH': '-2.660', 'AMMO2': '0.009', 'AMMO5': '0.030', 'AMMO4': '0.046', 'AMMO3': '0.100', 'WEAPON4': '0.100', 'HITCOUNT': '0.260', 'WEAPON5': '0.300', 'weapon5': '0.682', 'WEAPON3': '0.800', 'DAMAGECOUNT': '1.041', 'weapon4': '2.668', 'FRAGCOUNT': '3.000', 'weapon2': '4.470', 'weapon3': '8.222'} [2023-07-24 01:55:15,022][00294] Avg episode rewards: #0: 7.890, true rewards: #0: 1.300 [2023-07-24 01:55:15,025][00294] Avg episode reward: 7.890, avg true_objective: 1.300 [2023-07-24 02:05:45,948][00294] Replay video saved to /content/train_dir/default_experiment/replay.mp4!