2023-06-18 16:39:11 - SimpleLog - INFO: - General Configs: 2023-06-18 16:39:11 - SimpleLog - INFO: - ================================================================================ 2023-06-18 16:39:11 - SimpleLog - INFO: - Name Value Type 2023-06-18 16:39:11 - SimpleLog - INFO: - env_name gym 2023-06-18 16:39:11 - SimpleLog - INFO: - algo_name SAC 2023-06-18 16:39:11 - SimpleLog - INFO: - mode train 2023-06-18 16:39:11 - SimpleLog - INFO: - device cpu 2023-06-18 16:39:11 - SimpleLog - INFO: - seed 1 2023-06-18 16:39:11 - SimpleLog - INFO: - max_episode 200 2023-06-18 16:39:11 - SimpleLog - INFO: - max_step 200 2023-06-18 16:39:11 - SimpleLog - INFO: - collect_traj 0 2023-06-18 16:39:11 - SimpleLog - INFO: - mp_backend single 2023-06-18 16:39:11 - SimpleLog - INFO: - n_workers 2 2023-06-18 16:39:11 - SimpleLog - INFO: - n_learners 1 2023-06-18 16:39:11 - SimpleLog - INFO: - share_buffer 1 2023-06-18 16:39:11 - SimpleLog - INFO: - online_eval 1 2023-06-18 16:39:11 - SimpleLog - INFO: - online_eval_episode 10 2023-06-18 16:39:11 - SimpleLog - INFO: - model_save_fre 100 2023-06-18 16:39:11 - SimpleLog - INFO: - load_checkpoint 0 2023-06-18 16:39:11 - SimpleLog - INFO: - load_path Train_CartPole-v1_SAC_20230618-162702 2023-06-18 16:39:11 - SimpleLog - INFO: - load_model_step best 2023-06-18 16:39:11 - SimpleLog - INFO: - interact_summary_fre 1 2023-06-18 16:39:11 - SimpleLog - INFO: - model_summary_fre 1 2023-06-18 16:39:11 - SimpleLog - INFO: - ================================================================================ 2023-06-18 16:39:11 - SimpleLog - INFO: - Algo Configs: 2023-06-18 16:39:11 - SimpleLog - INFO: - ================================================================================ 2023-06-18 16:39:11 - SimpleLog - INFO: - Name Value Type 2023-06-18 16:39:11 - SimpleLog - INFO: - critic1_lr 0.001 2023-06-18 16:39:11 - SimpleLog - INFO: - critic2_lr 0.001 2023-06-18 16:39:11 - SimpleLog - INFO: - actor_lr 0.0003 2023-06-18 16:39:11 - SimpleLog - INFO: - gamma 0.95 2023-06-18 16:39:11 - SimpleLog - INFO: - tau 0.005 2023-06-18 16:39:11 - SimpleLog - INFO: - alpha 0.1 2023-06-18 16:39:11 - SimpleLog - INFO: - automatic_entropy_tuning 0 2023-06-18 16:39:11 - SimpleLog - INFO: - batch_size 64 2023-06-18 16:39:11 - SimpleLog - INFO: - hidden_dim 64 2023-06-18 16:39:11 - SimpleLog - INFO: - n_epochs 1 2023-06-18 16:39:11 - SimpleLog - INFO: - start_steps 10000 2023-06-18 16:39:11 - SimpleLog - INFO: - target_update_fre 1 2023-06-18 16:39:11 - SimpleLog - INFO: - buffer_size 1000000 2023-06-18 16:39:11 - SimpleLog - INFO: - min_policy 0 2023-06-18 16:39:11 - SimpleLog - INFO: - alpha_lr 0.0001 2023-06-18 16:39:11 - SimpleLog - INFO: - n_steps_per_learn 1 2023-06-18 16:39:11 - SimpleLog - INFO: - action_type continuous 2023-06-18 16:39:11 - SimpleLog - INFO: - independ_actor 1 2023-06-18 16:39:11 - SimpleLog - INFO: - share_optimizer 0 2023-06-18 16:39:11 - SimpleLog - INFO: - actor_layers [{'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}] 2023-06-18 16:39:11 - SimpleLog - INFO: - critic_layers [{'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}] 2023-06-18 16:39:11 - SimpleLog - INFO: - value_layers [{'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [256], 'activation': 'relu'}] 2023-06-18 16:39:11 - SimpleLog - INFO: - buffer_type REPLAY_QUE 2023-06-18 16:39:11 - SimpleLog - INFO: - epsilon_decay 500 2023-06-18 16:39:11 - SimpleLog - INFO: - epsilon_end 0.01 2023-06-18 16:39:11 - SimpleLog - INFO: - epsilon_start 0.95 2023-06-18 16:39:11 - SimpleLog - INFO: - lr 0.0001 2023-06-18 16:39:11 - SimpleLog - INFO: - target_update 1 2023-06-18 16:39:11 - SimpleLog - INFO: - ================================================================================ 2023-06-18 16:39:11 - SimpleLog - INFO: - Env Configs: 2023-06-18 16:39:11 - SimpleLog - INFO: - ================================================================================ 2023-06-18 16:39:11 - SimpleLog - INFO: - Name Value Type 2023-06-18 16:39:11 - SimpleLog - INFO: - id Pendulum-v1 2023-06-18 16:39:11 - SimpleLog - INFO: - render_mode None 2023-06-18 16:39:11 - SimpleLog - INFO: - wrapper None 2023-06-18 16:39:11 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] 2023-06-18 16:39:11 - SimpleLog - INFO: - ================================================================================ 2023-06-18 16:39:11 - SimpleLog - INFO: - Start training! 2023-06-18 16:39:13 - SimpleLog - INFO: - update_step: 100, online_eval_reward: -1380.018 2023-06-18 16:39:13 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1380.018, save the best model! 2023-06-18 16:39:13 - SimpleLog - INFO: - Interactor 0 finished episode 1 with reward -1247.240 in 200 steps 2023-06-18 16:39:13 - SimpleLog - INFO: - Interactor 1 finished episode 2 with reward -1239.670 in 200 steps 2023-06-18 16:39:14 - SimpleLog - INFO: - update_step: 200, online_eval_reward: -1594.462 2023-06-18 16:39:15 - SimpleLog - INFO: - update_step: 300, online_eval_reward: -1575.933 2023-06-18 16:39:15 - SimpleLog - INFO: - Interactor 0 finished episode 3 with reward -1465.696 in 200 steps 2023-06-18 16:39:15 - SimpleLog - INFO: - Interactor 1 finished episode 4 with reward -1538.397 in 200 steps 2023-06-18 16:39:16 - SimpleLog - INFO: - update_step: 400, online_eval_reward: -1550.200 2023-06-18 16:39:17 - SimpleLog - INFO: - update_step: 500, online_eval_reward: -1517.290 2023-06-18 16:39:18 - SimpleLog - INFO: - Interactor 0 finished episode 5 with reward -1516.613 in 200 steps 2023-06-18 16:39:18 - SimpleLog - INFO: - Interactor 1 finished episode 6 with reward -1697.405 in 200 steps 2023-06-18 16:39:18 - SimpleLog - INFO: - update_step: 600, online_eval_reward: -1474.664 2023-06-18 16:39:19 - SimpleLog - INFO: - update_step: 700, online_eval_reward: -1480.853 2023-06-18 16:39:20 - SimpleLog - INFO: - Interactor 0 finished episode 7 with reward -1479.039 in 200 steps 2023-06-18 16:39:20 - SimpleLog - INFO: - Interactor 1 finished episode 8 with reward -1621.124 in 200 steps 2023-06-18 16:39:21 - SimpleLog - INFO: - update_step: 800, online_eval_reward: -1444.541 2023-06-18 16:39:22 - SimpleLog - INFO: - update_step: 900, online_eval_reward: -1432.778 2023-06-18 16:39:22 - SimpleLog - INFO: - Interactor 0 finished episode 9 with reward -1375.822 in 200 steps 2023-06-18 16:39:22 - SimpleLog - INFO: - Interactor 1 finished episode 10 with reward -1548.226 in 200 steps 2023-06-18 16:39:23 - SimpleLog - INFO: - update_step: 1000, online_eval_reward: -1379.687 2023-06-18 16:39:23 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1379.687, save the best model! 2023-06-18 16:39:24 - SimpleLog - INFO: - update_step: 1100, online_eval_reward: -1365.993 2023-06-18 16:39:24 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1365.993, save the best model! 2023-06-18 16:39:25 - SimpleLog - INFO: - Interactor 0 finished episode 11 with reward -1417.818 in 200 steps 2023-06-18 16:39:25 - SimpleLog - INFO: - Interactor 1 finished episode 12 with reward -1484.637 in 200 steps 2023-06-18 16:39:26 - SimpleLog - INFO: - update_step: 1200, online_eval_reward: -1289.023 2023-06-18 16:39:26 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1289.023, save the best model! 2023-06-18 16:39:27 - SimpleLog - INFO: - update_step: 1300, online_eval_reward: -1274.835 2023-06-18 16:39:27 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1274.835, save the best model! 2023-06-18 16:39:27 - SimpleLog - INFO: - Interactor 0 finished episode 13 with reward -1299.286 in 200 steps 2023-06-18 16:39:27 - SimpleLog - INFO: - Interactor 1 finished episode 14 with reward -1410.494 in 200 steps 2023-06-18 16:39:28 - SimpleLog - INFO: - update_step: 1400, online_eval_reward: -1234.458 2023-06-18 16:39:28 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1234.458, save the best model! 2023-06-18 16:39:29 - SimpleLog - INFO: - update_step: 1500, online_eval_reward: -1167.915 2023-06-18 16:39:29 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1167.915, save the best model! 2023-06-18 16:39:30 - SimpleLog - INFO: - Interactor 0 finished episode 15 with reward -1114.071 in 200 steps 2023-06-18 16:39:30 - SimpleLog - INFO: - Interactor 1 finished episode 16 with reward -1412.817 in 200 steps 2023-06-18 16:39:30 - SimpleLog - INFO: - update_step: 1600, online_eval_reward: -1164.202 2023-06-18 16:39:30 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1164.202, save the best model! 2023-06-18 16:39:32 - SimpleLog - INFO: - update_step: 1700, online_eval_reward: -1135.373 2023-06-18 16:39:32 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1135.373, save the best model! 2023-06-18 16:39:32 - SimpleLog - INFO: - Interactor 0 finished episode 17 with reward -1036.607 in 200 steps 2023-06-18 16:39:32 - SimpleLog - INFO: - Interactor 1 finished episode 18 with reward -1192.079 in 200 steps 2023-06-18 16:39:33 - SimpleLog - INFO: - update_step: 1800, online_eval_reward: -1136.757 2023-06-18 16:39:34 - SimpleLog - INFO: - update_step: 1900, online_eval_reward: -1096.014 2023-06-18 16:39:34 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1096.014, save the best model! 2023-06-18 16:39:35 - SimpleLog - INFO: - Interactor 0 finished episode 19 with reward -1018.449 in 200 steps 2023-06-18 16:39:35 - SimpleLog - INFO: - Interactor 1 finished episode 20 with reward -1162.681 in 200 steps 2023-06-18 16:39:35 - SimpleLog - INFO: - update_step: 2000, online_eval_reward: -1065.251 2023-06-18 16:39:35 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -1065.251, save the best model! 2023-06-18 16:39:37 - SimpleLog - INFO: - update_step: 2100, online_eval_reward: -1124.240 2023-06-18 16:39:37 - SimpleLog - INFO: - Interactor 0 finished episode 21 with reward -971.930 in 200 steps 2023-06-18 16:39:37 - SimpleLog - INFO: - Interactor 1 finished episode 22 with reward -1129.776 in 200 steps 2023-06-18 16:39:38 - SimpleLog - INFO: - update_step: 2200, online_eval_reward: -880.723 2023-06-18 16:39:38 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -880.723, save the best model! 2023-06-18 16:39:39 - SimpleLog - INFO: - update_step: 2300, online_eval_reward: -993.409 2023-06-18 16:39:40 - SimpleLog - INFO: - Interactor 0 finished episode 23 with reward -940.286 in 200 steps 2023-06-18 16:39:40 - SimpleLog - INFO: - Interactor 1 finished episode 24 with reward -1006.670 in 200 steps 2023-06-18 16:39:40 - SimpleLog - INFO: - update_step: 2400, online_eval_reward: -874.374 2023-06-18 16:39:40 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -874.374, save the best model! 2023-06-18 16:39:42 - SimpleLog - INFO: - update_step: 2500, online_eval_reward: -1023.463 2023-06-18 16:39:42 - SimpleLog - INFO: - Interactor 0 finished episode 25 with reward -854.571 in 200 steps 2023-06-18 16:39:42 - SimpleLog - INFO: - Interactor 1 finished episode 26 with reward -995.997 in 200 steps 2023-06-18 16:39:43 - SimpleLog - INFO: - update_step: 2600, online_eval_reward: -970.227 2023-06-18 16:39:44 - SimpleLog - INFO: - update_step: 2700, online_eval_reward: -970.407 2023-06-18 16:39:45 - SimpleLog - INFO: - Interactor 0 finished episode 27 with reward -891.371 in 200 steps 2023-06-18 16:39:45 - SimpleLog - INFO: - Interactor 1 finished episode 28 with reward -884.375 in 200 steps 2023-06-18 16:39:45 - SimpleLog - INFO: - update_step: 2800, online_eval_reward: -988.688 2023-06-18 16:39:47 - SimpleLog - INFO: - update_step: 2900, online_eval_reward: -844.578 2023-06-18 16:39:47 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -844.578, save the best model! 2023-06-18 16:39:47 - SimpleLog - INFO: - Interactor 0 finished episode 29 with reward -874.244 in 200 steps 2023-06-18 16:39:47 - SimpleLog - INFO: - Interactor 1 finished episode 30 with reward -950.054 in 200 steps 2023-06-18 16:39:48 - SimpleLog - INFO: - update_step: 3000, online_eval_reward: -931.215 2023-06-18 16:39:49 - SimpleLog - INFO: - update_step: 3100, online_eval_reward: -969.424 2023-06-18 16:39:50 - SimpleLog - INFO: - Interactor 0 finished episode 31 with reward -761.690 in 200 steps 2023-06-18 16:39:50 - SimpleLog - INFO: - Interactor 1 finished episode 32 with reward -810.092 in 200 steps 2023-06-18 16:39:50 - SimpleLog - INFO: - update_step: 3200, online_eval_reward: -858.450 2023-06-18 16:39:52 - SimpleLog - INFO: - update_step: 3300, online_eval_reward: -974.745 2023-06-18 16:39:52 - SimpleLog - INFO: - Interactor 0 finished episode 33 with reward -720.616 in 200 steps 2023-06-18 16:39:52 - SimpleLog - INFO: - Interactor 1 finished episode 34 with reward -763.590 in 200 steps 2023-06-18 16:39:53 - SimpleLog - INFO: - update_step: 3400, online_eval_reward: -865.729 2023-06-18 16:39:54 - SimpleLog - INFO: - update_step: 3500, online_eval_reward: -703.698 2023-06-18 16:39:54 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -703.698, save the best model! 2023-06-18 16:39:55 - SimpleLog - INFO: - Interactor 0 finished episode 35 with reward -697.547 in 200 steps 2023-06-18 16:39:55 - SimpleLog - INFO: - Interactor 1 finished episode 36 with reward -750.590 in 200 steps 2023-06-18 16:39:55 - SimpleLog - INFO: - update_step: 3600, online_eval_reward: -863.473 2023-06-18 16:39:57 - SimpleLog - INFO: - update_step: 3700, online_eval_reward: -859.878 2023-06-18 16:39:57 - SimpleLog - INFO: - Interactor 0 finished episode 37 with reward -731.446 in 200 steps 2023-06-18 16:39:57 - SimpleLog - INFO: - Interactor 1 finished episode 38 with reward -874.953 in 200 steps 2023-06-18 16:39:58 - SimpleLog - INFO: - update_step: 3800, online_eval_reward: -855.164 2023-06-18 16:39:59 - SimpleLog - INFO: - update_step: 3900, online_eval_reward: -630.874 2023-06-18 16:39:59 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -630.874, save the best model! 2023-06-18 16:40:00 - SimpleLog - INFO: - Interactor 0 finished episode 39 with reward -545.310 in 200 steps 2023-06-18 16:40:00 - SimpleLog - INFO: - Interactor 1 finished episode 40 with reward -751.373 in 200 steps 2023-06-18 16:40:00 - SimpleLog - INFO: - update_step: 4000, online_eval_reward: -756.478 2023-06-18 16:40:02 - SimpleLog - INFO: - update_step: 4100, online_eval_reward: -863.402 2023-06-18 16:40:02 - SimpleLog - INFO: - Interactor 0 finished episode 41 with reward -646.219 in 200 steps 2023-06-18 16:40:02 - SimpleLog - INFO: - Interactor 1 finished episode 42 with reward -753.741 in 200 steps 2023-06-18 16:40:03 - SimpleLog - INFO: - update_step: 4200, online_eval_reward: -626.108 2023-06-18 16:40:03 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -626.108, save the best model! 2023-06-18 16:40:04 - SimpleLog - INFO: - update_step: 4300, online_eval_reward: -748.564 2023-06-18 16:40:05 - SimpleLog - INFO: - Interactor 0 finished episode 43 with reward -632.094 in 200 steps 2023-06-18 16:40:05 - SimpleLog - INFO: - Interactor 1 finished episode 44 with reward -751.567 in 200 steps 2023-06-18 16:40:05 - SimpleLog - INFO: - update_step: 4400, online_eval_reward: -649.743 2023-06-18 16:40:06 - SimpleLog - INFO: - update_step: 4500, online_eval_reward: -766.485 2023-06-18 16:40:07 - SimpleLog - INFO: - Interactor 0 finished episode 45 with reward -627.878 in 200 steps 2023-06-18 16:40:07 - SimpleLog - INFO: - Interactor 1 finished episode 46 with reward -628.071 in 200 steps 2023-06-18 16:40:08 - SimpleLog - INFO: - update_step: 4600, online_eval_reward: -781.563 2023-06-18 16:40:09 - SimpleLog - INFO: - update_step: 4700, online_eval_reward: -628.591 2023-06-18 16:40:10 - SimpleLog - INFO: - Interactor 0 finished episode 47 with reward -509.268 in 200 steps 2023-06-18 16:40:10 - SimpleLog - INFO: - Interactor 1 finished episode 48 with reward -377.213 in 200 steps 2023-06-18 16:40:10 - SimpleLog - INFO: - update_step: 4800, online_eval_reward: -504.638 2023-06-18 16:40:10 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -504.638, save the best model! 2023-06-18 16:40:11 - SimpleLog - INFO: - update_step: 4900, online_eval_reward: -530.351 2023-06-18 16:40:12 - SimpleLog - INFO: - Interactor 0 finished episode 49 with reward -383.350 in 200 steps 2023-06-18 16:40:12 - SimpleLog - INFO: - Interactor 1 finished episode 50 with reward -501.978 in 200 steps 2023-06-18 16:40:13 - SimpleLog - INFO: - update_step: 5000, online_eval_reward: -504.956 2023-06-18 16:40:14 - SimpleLog - INFO: - update_step: 5100, online_eval_reward: -632.072 2023-06-18 16:40:14 - SimpleLog - INFO: - Interactor 0 finished episode 51 with reward -255.278 in 200 steps 2023-06-18 16:40:14 - SimpleLog - INFO: - Interactor 1 finished episode 52 with reward -500.879 in 200 steps 2023-06-18 16:40:15 - SimpleLog - INFO: - update_step: 5200, online_eval_reward: -641.763 2023-06-18 16:40:16 - SimpleLog - INFO: - update_step: 5300, online_eval_reward: -508.721 2023-06-18 16:40:17 - SimpleLog - INFO: - Interactor 0 finished episode 53 with reward -257.886 in 200 steps 2023-06-18 16:40:17 - SimpleLog - INFO: - Interactor 1 finished episode 54 with reward -500.677 in 200 steps 2023-06-18 16:40:18 - SimpleLog - INFO: - update_step: 5400, online_eval_reward: -624.826 2023-06-18 16:40:19 - SimpleLog - INFO: - update_step: 5500, online_eval_reward: -504.471 2023-06-18 16:40:19 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -504.471, save the best model! 2023-06-18 16:40:19 - SimpleLog - INFO: - Interactor 0 finished episode 55 with reward -259.141 in 200 steps 2023-06-18 16:40:19 - SimpleLog - INFO: - Interactor 1 finished episode 56 with reward -501.679 in 200 steps 2023-06-18 16:40:20 - SimpleLog - INFO: - update_step: 5600, online_eval_reward: -381.350 2023-06-18 16:40:20 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -381.350, save the best model! 2023-06-18 16:40:21 - SimpleLog - INFO: - update_step: 5700, online_eval_reward: -627.468 2023-06-18 16:40:22 - SimpleLog - INFO: - Interactor 0 finished episode 57 with reward -384.113 in 200 steps 2023-06-18 16:40:22 - SimpleLog - INFO: - Interactor 1 finished episode 58 with reward -401.693 in 200 steps 2023-06-18 16:40:23 - SimpleLog - INFO: - update_step: 5800, online_eval_reward: -627.865 2023-06-18 16:40:24 - SimpleLog - INFO: - update_step: 5900, online_eval_reward: -524.992 2023-06-18 16:40:24 - SimpleLog - INFO: - Interactor 0 finished episode 59 with reward -386.282 in 200 steps 2023-06-18 16:40:24 - SimpleLog - INFO: - Interactor 1 finished episode 60 with reward -377.906 in 200 steps 2023-06-18 16:40:25 - SimpleLog - INFO: - update_step: 6000, online_eval_reward: -504.850 2023-06-18 16:40:26 - SimpleLog - INFO: - update_step: 6100, online_eval_reward: -622.662 2023-06-18 16:40:27 - SimpleLog - INFO: - Interactor 0 finished episode 61 with reward -264.980 in 200 steps 2023-06-18 16:40:27 - SimpleLog - INFO: - Interactor 1 finished episode 62 with reward -376.024 in 200 steps 2023-06-18 16:40:27 - SimpleLog - INFO: - update_step: 6200, online_eval_reward: -629.334 2023-06-18 16:40:29 - SimpleLog - INFO: - update_step: 6300, online_eval_reward: -505.557 2023-06-18 16:40:29 - SimpleLog - INFO: - Interactor 0 finished episode 63 with reward -258.428 in 200 steps 2023-06-18 16:40:29 - SimpleLog - INFO: - Interactor 1 finished episode 64 with reward -377.041 in 200 steps 2023-06-18 16:40:30 - SimpleLog - INFO: - update_step: 6400, online_eval_reward: -506.662 2023-06-18 16:40:31 - SimpleLog - INFO: - update_step: 6500, online_eval_reward: -504.294 2023-06-18 16:40:32 - SimpleLog - INFO: - Interactor 0 finished episode 65 with reward -144.788 in 200 steps 2023-06-18 16:40:32 - SimpleLog - INFO: - Interactor 1 finished episode 66 with reward -250.861 in 200 steps 2023-06-18 16:40:32 - SimpleLog - INFO: - update_step: 6600, online_eval_reward: -506.432 2023-06-18 16:40:34 - SimpleLog - INFO: - update_step: 6700, online_eval_reward: -503.358 2023-06-18 16:40:34 - SimpleLog - INFO: - Interactor 0 finished episode 67 with reward -384.900 in 200 steps 2023-06-18 16:40:34 - SimpleLog - INFO: - Interactor 1 finished episode 68 with reward -503.686 in 200 steps 2023-06-18 16:40:35 - SimpleLog - INFO: - update_step: 6800, online_eval_reward: -610.874 2023-06-18 16:40:36 - SimpleLog - INFO: - update_step: 6900, online_eval_reward: -507.600 2023-06-18 16:40:37 - SimpleLog - INFO: - Interactor 0 finished episode 69 with reward -6.160 in 200 steps 2023-06-18 16:40:37 - SimpleLog - INFO: - Interactor 1 finished episode 70 with reward -249.738 in 200 steps 2023-06-18 16:40:37 - SimpleLog - INFO: - update_step: 7000, online_eval_reward: -516.111 2023-06-18 16:40:39 - SimpleLog - INFO: - update_step: 7100, online_eval_reward: -504.814 2023-06-18 16:40:39 - SimpleLog - INFO: - Interactor 0 finished episode 71 with reward -130.383 in 200 steps 2023-06-18 16:40:39 - SimpleLog - INFO: - Interactor 1 finished episode 72 with reward -248.035 in 200 steps 2023-06-18 16:40:40 - SimpleLog - INFO: - update_step: 7200, online_eval_reward: -504.119 2023-06-18 16:40:41 - SimpleLog - INFO: - update_step: 7300, online_eval_reward: -252.920 2023-06-18 16:40:41 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -252.920, save the best model! 2023-06-18 16:40:42 - SimpleLog - INFO: - Interactor 0 finished episode 73 with reward -129.843 in 200 steps 2023-06-18 16:40:42 - SimpleLog - INFO: - Interactor 1 finished episode 74 with reward -123.091 in 200 steps 2023-06-18 16:40:42 - SimpleLog - INFO: - update_step: 7400, online_eval_reward: -504.046 2023-06-18 16:40:44 - SimpleLog - INFO: - update_step: 7500, online_eval_reward: -506.457 2023-06-18 16:40:44 - SimpleLog - INFO: - Interactor 0 finished episode 75 with reward -9.244 in 200 steps 2023-06-18 16:40:44 - SimpleLog - INFO: - Interactor 1 finished episode 76 with reward -374.890 in 200 steps 2023-06-18 16:40:45 - SimpleLog - INFO: - update_step: 7600, online_eval_reward: -253.852 2023-06-18 16:40:46 - SimpleLog - INFO: - update_step: 7700, online_eval_reward: -497.678 2023-06-18 16:40:47 - SimpleLog - INFO: - Interactor 0 finished episode 77 with reward -257.436 in 200 steps 2023-06-18 16:40:47 - SimpleLog - INFO: - Interactor 1 finished episode 78 with reward -248.164 in 200 steps 2023-06-18 16:40:47 - SimpleLog - INFO: - update_step: 7800, online_eval_reward: -518.276 2023-06-18 16:40:49 - SimpleLog - INFO: - update_step: 7900, online_eval_reward: -496.236 2023-06-18 16:40:49 - SimpleLog - INFO: - Interactor 0 finished episode 79 with reward -129.262 in 200 steps 2023-06-18 16:40:49 - SimpleLog - INFO: - Interactor 1 finished episode 80 with reward -378.180 in 200 steps 2023-06-18 16:40:50 - SimpleLog - INFO: - update_step: 8000, online_eval_reward: -505.930 2023-06-18 16:40:51 - SimpleLog - INFO: - update_step: 8100, online_eval_reward: -505.368 2023-06-18 16:40:52 - SimpleLog - INFO: - Interactor 0 finished episode 81 with reward -382.849 in 200 steps 2023-06-18 16:40:52 - SimpleLog - INFO: - Interactor 1 finished episode 82 with reward -501.592 in 200 steps 2023-06-18 16:40:52 - SimpleLog - INFO: - update_step: 8200, online_eval_reward: -512.991 2023-06-18 16:40:54 - SimpleLog - INFO: - update_step: 8300, online_eval_reward: -504.157 2023-06-18 16:40:54 - SimpleLog - INFO: - Interactor 0 finished episode 83 with reward -289.496 in 200 steps 2023-06-18 16:40:54 - SimpleLog - INFO: - Interactor 1 finished episode 84 with reward -379.101 in 200 steps 2023-06-18 16:40:55 - SimpleLog - INFO: - update_step: 8400, online_eval_reward: -471.989 2023-06-18 16:40:56 - SimpleLog - INFO: - update_step: 8500, online_eval_reward: -378.024 2023-06-18 16:40:57 - SimpleLog - INFO: - Interactor 0 finished episode 85 with reward -383.413 in 200 steps 2023-06-18 16:40:57 - SimpleLog - INFO: - Interactor 1 finished episode 86 with reward -377.348 in 200 steps 2023-06-18 16:40:57 - SimpleLog - INFO: - update_step: 8600, online_eval_reward: -504.045 2023-06-18 16:40:59 - SimpleLog - INFO: - update_step: 8700, online_eval_reward: -622.723 2023-06-18 16:40:59 - SimpleLog - INFO: - Interactor 0 finished episode 87 with reward -258.051 in 200 steps 2023-06-18 16:40:59 - SimpleLog - INFO: - Interactor 1 finished episode 88 with reward -378.418 in 200 steps 2023-06-18 16:41:00 - SimpleLog - INFO: - update_step: 8800, online_eval_reward: -504.854 2023-06-18 16:41:01 - SimpleLog - INFO: - update_step: 8900, online_eval_reward: -501.113 2023-06-18 16:41:02 - SimpleLog - INFO: - Interactor 0 finished episode 89 with reward -265.978 in 200 steps 2023-06-18 16:41:02 - SimpleLog - INFO: - Interactor 1 finished episode 90 with reward -377.551 in 200 steps 2023-06-18 16:41:02 - SimpleLog - INFO: - update_step: 9000, online_eval_reward: -627.357 2023-06-18 16:41:04 - SimpleLog - INFO: - update_step: 9100, online_eval_reward: -378.242 2023-06-18 16:41:04 - SimpleLog - INFO: - Interactor 0 finished episode 91 with reward -376.627 in 200 steps 2023-06-18 16:41:04 - SimpleLog - INFO: - Interactor 1 finished episode 92 with reward -379.475 in 200 steps 2023-06-18 16:41:05 - SimpleLog - INFO: - update_step: 9200, online_eval_reward: -523.777 2023-06-18 16:41:06 - SimpleLog - INFO: - update_step: 9300, online_eval_reward: -392.154 2023-06-18 16:41:07 - SimpleLog - INFO: - Interactor 0 finished episode 93 with reward -257.879 in 200 steps 2023-06-18 16:41:07 - SimpleLog - INFO: - Interactor 1 finished episode 94 with reward -364.066 in 200 steps 2023-06-18 16:41:07 - SimpleLog - INFO: - update_step: 9400, online_eval_reward: -505.073 2023-06-18 16:41:09 - SimpleLog - INFO: - update_step: 9500, online_eval_reward: -392.630 2023-06-18 16:41:09 - SimpleLog - INFO: - Interactor 0 finished episode 95 with reward -256.403 in 200 steps 2023-06-18 16:41:09 - SimpleLog - INFO: - Interactor 1 finished episode 96 with reward -378.919 in 200 steps 2023-06-18 16:41:10 - SimpleLog - INFO: - update_step: 9600, online_eval_reward: -611.083 2023-06-18 16:41:11 - SimpleLog - INFO: - update_step: 9700, online_eval_reward: -575.006 2023-06-18 16:41:12 - SimpleLog - INFO: - Interactor 0 finished episode 97 with reward -257.988 in 200 steps 2023-06-18 16:41:12 - SimpleLog - INFO: - Interactor 1 finished episode 98 with reward -375.953 in 200 steps 2023-06-18 16:41:12 - SimpleLog - INFO: - update_step: 9800, online_eval_reward: -504.909 2023-06-18 16:41:14 - SimpleLog - INFO: - update_step: 9900, online_eval_reward: -521.546 2023-06-18 16:41:14 - SimpleLog - INFO: - Interactor 0 finished episode 99 with reward -257.814 in 200 steps 2023-06-18 16:41:14 - SimpleLog - INFO: - Interactor 1 finished episode 100 with reward -376.679 in 200 steps 2023-06-18 16:41:15 - SimpleLog - INFO: - update_step: 10000, online_eval_reward: -498.104 2023-06-18 16:41:16 - SimpleLog - INFO: - update_step: 10100, online_eval_reward: -505.148 2023-06-18 16:41:17 - SimpleLog - INFO: - Interactor 0 finished episode 101 with reward -383.380 in 200 steps 2023-06-18 16:41:17 - SimpleLog - INFO: - Interactor 1 finished episode 102 with reward -376.594 in 200 steps 2023-06-18 16:41:17 - SimpleLog - INFO: - update_step: 10200, online_eval_reward: -502.743 2023-06-18 16:41:18 - SimpleLog - INFO: - update_step: 10300, online_eval_reward: -495.894 2023-06-18 16:41:19 - SimpleLog - INFO: - Interactor 0 finished episode 103 with reward -258.215 in 200 steps 2023-06-18 16:41:19 - SimpleLog - INFO: - Interactor 1 finished episode 104 with reward -252.459 in 200 steps 2023-06-18 16:41:20 - SimpleLog - INFO: - update_step: 10400, online_eval_reward: -428.521 2023-06-18 16:41:21 - SimpleLog - INFO: - update_step: 10500, online_eval_reward: -509.165 2023-06-18 16:41:21 - SimpleLog - INFO: - Interactor 0 finished episode 105 with reward -237.254 in 200 steps 2023-06-18 16:41:21 - SimpleLog - INFO: - Interactor 1 finished episode 106 with reward -249.686 in 200 steps 2023-06-18 16:41:22 - SimpleLog - INFO: - update_step: 10600, online_eval_reward: -378.710 2023-06-18 16:41:23 - SimpleLog - INFO: - update_step: 10700, online_eval_reward: -467.916 2023-06-18 16:41:24 - SimpleLog - INFO: - Interactor 0 finished episode 107 with reward -380.161 in 200 steps 2023-06-18 16:41:24 - SimpleLog - INFO: - Interactor 1 finished episode 108 with reward -377.340 in 200 steps 2023-06-18 16:41:25 - SimpleLog - INFO: - update_step: 10800, online_eval_reward: -504.152 2023-06-18 16:41:26 - SimpleLog - INFO: - update_step: 10900, online_eval_reward: -616.149 2023-06-18 16:41:26 - SimpleLog - INFO: - Interactor 0 finished episode 109 with reward -256.526 in 200 steps 2023-06-18 16:41:26 - SimpleLog - INFO: - Interactor 1 finished episode 110 with reward -250.052 in 200 steps 2023-06-18 16:41:27 - SimpleLog - INFO: - update_step: 11000, online_eval_reward: -624.551 2023-06-18 16:41:28 - SimpleLog - INFO: - update_step: 11100, online_eval_reward: -626.679 2023-06-18 16:41:29 - SimpleLog - INFO: - Interactor 0 finished episode 111 with reward -255.673 in 200 steps 2023-06-18 16:41:29 - SimpleLog - INFO: - Interactor 1 finished episode 112 with reward -249.592 in 200 steps 2023-06-18 16:41:30 - SimpleLog - INFO: - update_step: 11200, online_eval_reward: -504.051 2023-06-18 16:41:31 - SimpleLog - INFO: - update_step: 11300, online_eval_reward: -627.940 2023-06-18 16:41:31 - SimpleLog - INFO: - Interactor 0 finished episode 113 with reward -257.611 in 200 steps 2023-06-18 16:41:31 - SimpleLog - INFO: - Interactor 1 finished episode 114 with reward -375.881 in 200 steps 2023-06-18 16:41:32 - SimpleLog - INFO: - update_step: 11400, online_eval_reward: -603.049 2023-06-18 16:41:33 - SimpleLog - INFO: - update_step: 11500, online_eval_reward: -471.987 2023-06-18 16:41:34 - SimpleLog - INFO: - Interactor 0 finished episode 115 with reward -382.501 in 200 steps 2023-06-18 16:41:34 - SimpleLog - INFO: - Interactor 1 finished episode 116 with reward -252.038 in 200 steps 2023-06-18 16:41:35 - SimpleLog - INFO: - update_step: 11600, online_eval_reward: -503.790 2023-06-18 16:41:36 - SimpleLog - INFO: - update_step: 11700, online_eval_reward: -503.284 2023-06-18 16:41:36 - SimpleLog - INFO: - Interactor 0 finished episode 117 with reward -264.405 in 200 steps 2023-06-18 16:41:36 - SimpleLog - INFO: - Interactor 1 finished episode 118 with reward -502.513 in 200 steps 2023-06-18 16:41:37 - SimpleLog - INFO: - update_step: 11800, online_eval_reward: -627.830 2023-06-18 16:41:38 - SimpleLog - INFO: - update_step: 11900, online_eval_reward: -622.682 2023-06-18 16:41:39 - SimpleLog - INFO: - Interactor 0 finished episode 119 with reward -344.907 in 200 steps 2023-06-18 16:41:39 - SimpleLog - INFO: - Interactor 1 finished episode 120 with reward -405.190 in 200 steps 2023-06-18 16:41:40 - SimpleLog - INFO: - update_step: 12000, online_eval_reward: -503.674 2023-06-18 16:41:41 - SimpleLog - INFO: - update_step: 12100, online_eval_reward: -628.032 2023-06-18 16:41:42 - SimpleLog - INFO: - Interactor 0 finished episode 121 with reward -384.215 in 200 steps 2023-06-18 16:41:42 - SimpleLog - INFO: - Interactor 1 finished episode 122 with reward -432.613 in 200 steps 2023-06-18 16:41:42 - SimpleLog - INFO: - update_step: 12200, online_eval_reward: -503.886 2023-06-18 16:41:44 - SimpleLog - INFO: - update_step: 12300, online_eval_reward: -503.432 2023-06-18 16:41:44 - SimpleLog - INFO: - Interactor 0 finished episode 123 with reward -383.556 in 200 steps 2023-06-18 16:41:44 - SimpleLog - INFO: - Interactor 1 finished episode 124 with reward -415.292 in 200 steps 2023-06-18 16:41:45 - SimpleLog - INFO: - update_step: 12400, online_eval_reward: -549.750 2023-06-18 16:41:46 - SimpleLog - INFO: - update_step: 12500, online_eval_reward: -500.803 2023-06-18 16:41:47 - SimpleLog - INFO: - Interactor 0 finished episode 125 with reward -384.129 in 200 steps 2023-06-18 16:41:47 - SimpleLog - INFO: - Interactor 1 finished episode 126 with reward -378.644 in 200 steps 2023-06-18 16:41:47 - SimpleLog - INFO: - update_step: 12600, online_eval_reward: -614.173 2023-06-18 16:41:49 - SimpleLog - INFO: - update_step: 12700, online_eval_reward: -504.940 2023-06-18 16:41:49 - SimpleLog - INFO: - Interactor 0 finished episode 127 with reward -384.349 in 200 steps 2023-06-18 16:41:49 - SimpleLog - INFO: - Interactor 1 finished episode 128 with reward -378.360 in 200 steps 2023-06-18 16:41:50 - SimpleLog - INFO: - update_step: 12800, online_eval_reward: -627.797 2023-06-18 16:41:51 - SimpleLog - INFO: - update_step: 12900, online_eval_reward: -597.391 2023-06-18 16:41:52 - SimpleLog - INFO: - Interactor 0 finished episode 129 with reward -383.978 in 200 steps 2023-06-18 16:41:52 - SimpleLog - INFO: - Interactor 1 finished episode 130 with reward -380.662 in 200 steps 2023-06-18 16:41:52 - SimpleLog - INFO: - update_step: 13000, online_eval_reward: -504.481 2023-06-18 16:41:54 - SimpleLog - INFO: - update_step: 13100, online_eval_reward: -502.841 2023-06-18 16:41:54 - SimpleLog - INFO: - Interactor 0 finished episode 131 with reward -383.082 in 200 steps 2023-06-18 16:41:54 - SimpleLog - INFO: - Interactor 1 finished episode 132 with reward -377.854 in 200 steps 2023-06-18 16:41:55 - SimpleLog - INFO: - update_step: 13200, online_eval_reward: -606.172 2023-06-18 16:41:56 - SimpleLog - INFO: - update_step: 13300, online_eval_reward: -627.079 2023-06-18 16:41:57 - SimpleLog - INFO: - Interactor 0 finished episode 133 with reward -261.498 in 200 steps 2023-06-18 16:41:57 - SimpleLog - INFO: - Interactor 1 finished episode 134 with reward -378.427 in 200 steps 2023-06-18 16:41:57 - SimpleLog - INFO: - update_step: 13400, online_eval_reward: -621.482 2023-06-18 16:41:59 - SimpleLog - INFO: - update_step: 13500, online_eval_reward: -627.802 2023-06-18 16:41:59 - SimpleLog - INFO: - Interactor 0 finished episode 135 with reward -259.911 in 200 steps 2023-06-18 16:41:59 - SimpleLog - INFO: - Interactor 1 finished episode 136 with reward -254.313 in 200 steps 2023-06-18 16:42:00 - SimpleLog - INFO: - update_step: 13600, online_eval_reward: -554.930 2023-06-18 16:42:01 - SimpleLog - INFO: - update_step: 13700, online_eval_reward: -568.048 2023-06-18 16:42:02 - SimpleLog - INFO: - Interactor 0 finished episode 137 with reward -292.799 in 200 steps 2023-06-18 16:42:02 - SimpleLog - INFO: - Interactor 1 finished episode 138 with reward -378.388 in 200 steps 2023-06-18 16:42:02 - SimpleLog - INFO: - update_step: 13800, online_eval_reward: -536.149 2023-06-18 16:42:03 - SimpleLog - INFO: - update_step: 13900, online_eval_reward: -506.025 2023-06-18 16:42:04 - SimpleLog - INFO: - Interactor 0 finished episode 139 with reward -259.192 in 200 steps 2023-06-18 16:42:04 - SimpleLog - INFO: - Interactor 1 finished episode 140 with reward -378.459 in 200 steps 2023-06-18 16:42:05 - SimpleLog - INFO: - update_step: 14000, online_eval_reward: -521.703 2023-06-18 16:42:06 - SimpleLog - INFO: - update_step: 14100, online_eval_reward: -510.447 2023-06-18 16:42:07 - SimpleLog - INFO: - Interactor 0 finished episode 141 with reward -261.268 in 200 steps 2023-06-18 16:42:07 - SimpleLog - INFO: - Interactor 1 finished episode 142 with reward -378.171 in 200 steps 2023-06-18 16:42:07 - SimpleLog - INFO: - update_step: 14200, online_eval_reward: -522.750 2023-06-18 16:42:09 - SimpleLog - INFO: - update_step: 14300, online_eval_reward: -499.859 2023-06-18 16:42:09 - SimpleLog - INFO: - Interactor 0 finished episode 143 with reward -258.966 in 200 steps 2023-06-18 16:42:09 - SimpleLog - INFO: - Interactor 1 finished episode 144 with reward -377.578 in 200 steps 2023-06-18 16:42:10 - SimpleLog - INFO: - update_step: 14400, online_eval_reward: -503.152 2023-06-18 16:42:11 - SimpleLog - INFO: - update_step: 14500, online_eval_reward: -504.464 2023-06-18 16:42:12 - SimpleLog - INFO: - Interactor 0 finished episode 145 with reward -259.010 in 200 steps 2023-06-18 16:42:12 - SimpleLog - INFO: - Interactor 1 finished episode 146 with reward -378.754 in 200 steps 2023-06-18 16:42:12 - SimpleLog - INFO: - update_step: 14600, online_eval_reward: -407.771 2023-06-18 16:42:13 - SimpleLog - INFO: - update_step: 14700, online_eval_reward: -504.594 2023-06-18 16:42:14 - SimpleLog - INFO: - Interactor 0 finished episode 147 with reward -265.800 in 200 steps 2023-06-18 16:42:14 - SimpleLog - INFO: - Interactor 1 finished episode 148 with reward -379.020 in 200 steps 2023-06-18 16:42:15 - SimpleLog - INFO: - update_step: 14800, online_eval_reward: -521.266 2023-06-18 16:42:16 - SimpleLog - INFO: - update_step: 14900, online_eval_reward: -504.800 2023-06-18 16:42:17 - SimpleLog - INFO: - Interactor 0 finished episode 149 with reward -259.717 in 200 steps 2023-06-18 16:42:17 - SimpleLog - INFO: - Interactor 1 finished episode 150 with reward -371.442 in 200 steps 2023-06-18 16:42:17 - SimpleLog - INFO: - update_step: 15000, online_eval_reward: -557.607 2023-06-18 16:42:18 - SimpleLog - INFO: - update_step: 15100, online_eval_reward: -503.430 2023-06-18 16:42:19 - SimpleLog - INFO: - Interactor 0 finished episode 151 with reward -259.884 in 200 steps 2023-06-18 16:42:19 - SimpleLog - INFO: - Interactor 1 finished episode 152 with reward -378.490 in 200 steps 2023-06-18 16:42:20 - SimpleLog - INFO: - update_step: 15200, online_eval_reward: -503.907 2023-06-18 16:42:21 - SimpleLog - INFO: - update_step: 15300, online_eval_reward: -504.686 2023-06-18 16:42:22 - SimpleLog - INFO: - Interactor 0 finished episode 153 with reward -137.131 in 200 steps 2023-06-18 16:42:22 - SimpleLog - INFO: - Interactor 1 finished episode 154 with reward -256.148 in 200 steps 2023-06-18 16:42:22 - SimpleLog - INFO: - update_step: 15400, online_eval_reward: -503.950 2023-06-18 16:42:23 - SimpleLog - INFO: - update_step: 15500, online_eval_reward: -377.807 2023-06-18 16:42:24 - SimpleLog - INFO: - Interactor 0 finished episode 155 with reward -182.937 in 200 steps 2023-06-18 16:42:24 - SimpleLog - INFO: - Interactor 1 finished episode 156 with reward -228.954 in 200 steps 2023-06-18 16:42:25 - SimpleLog - INFO: - update_step: 15600, online_eval_reward: -633.895 2023-06-18 16:42:26 - SimpleLog - INFO: - update_step: 15700, online_eval_reward: -504.156 2023-06-18 16:42:27 - SimpleLog - INFO: - Interactor 0 finished episode 157 with reward -135.201 in 200 steps 2023-06-18 16:42:27 - SimpleLog - INFO: - Interactor 1 finished episode 158 with reward -127.702 in 200 steps 2023-06-18 16:42:27 - SimpleLog - INFO: - update_step: 15800, online_eval_reward: -504.442 2023-06-18 16:42:28 - SimpleLog - INFO: - update_step: 15900, online_eval_reward: -378.320 2023-06-18 16:42:29 - SimpleLog - INFO: - Interactor 0 finished episode 159 with reward -140.012 in 200 steps 2023-06-18 16:42:29 - SimpleLog - INFO: - Interactor 1 finished episode 160 with reward -254.871 in 200 steps 2023-06-18 16:42:30 - SimpleLog - INFO: - update_step: 16000, online_eval_reward: -758.459 2023-06-18 16:42:31 - SimpleLog - INFO: - update_step: 16100, online_eval_reward: -528.790 2023-06-18 16:42:32 - SimpleLog - INFO: - Interactor 0 finished episode 161 with reward -260.244 in 200 steps 2023-06-18 16:42:32 - SimpleLog - INFO: - Interactor 1 finished episode 162 with reward -378.559 in 200 steps 2023-06-18 16:42:32 - SimpleLog - INFO: - update_step: 16200, online_eval_reward: -512.799 2023-06-18 16:42:33 - SimpleLog - INFO: - update_step: 16300, online_eval_reward: -500.630 2023-06-18 16:42:34 - SimpleLog - INFO: - Interactor 0 finished episode 163 with reward -135.817 in 200 steps 2023-06-18 16:42:34 - SimpleLog - INFO: - Interactor 1 finished episode 164 with reward -260.366 in 200 steps 2023-06-18 16:42:35 - SimpleLog - INFO: - update_step: 16400, online_eval_reward: -514.297 2023-06-18 16:42:36 - SimpleLog - INFO: - update_step: 16500, online_eval_reward: -504.048 2023-06-18 16:42:37 - SimpleLog - INFO: - Interactor 0 finished episode 165 with reward -13.423 in 200 steps 2023-06-18 16:42:37 - SimpleLog - INFO: - Interactor 1 finished episode 166 with reward -132.449 in 200 steps 2023-06-18 16:42:37 - SimpleLog - INFO: - update_step: 16600, online_eval_reward: -504.361 2023-06-18 16:42:39 - SimpleLog - INFO: - update_step: 16700, online_eval_reward: -503.206 2023-06-18 16:42:39 - SimpleLog - INFO: - Interactor 0 finished episode 167 with reward -139.457 in 200 steps 2023-06-18 16:42:39 - SimpleLog - INFO: - Interactor 1 finished episode 168 with reward -374.078 in 200 steps 2023-06-18 16:42:40 - SimpleLog - INFO: - update_step: 16800, online_eval_reward: -453.025 2023-06-18 16:42:41 - SimpleLog - INFO: - update_step: 16900, online_eval_reward: -377.225 2023-06-18 16:42:42 - SimpleLog - INFO: - Interactor 0 finished episode 169 with reward -277.210 in 200 steps 2023-06-18 16:42:42 - SimpleLog - INFO: - Interactor 1 finished episode 170 with reward -252.749 in 200 steps 2023-06-18 16:42:42 - SimpleLog - INFO: - update_step: 17000, online_eval_reward: -494.961 2023-06-18 16:42:44 - SimpleLog - INFO: - update_step: 17100, online_eval_reward: -628.388 2023-06-18 16:42:44 - SimpleLog - INFO: - Interactor 0 finished episode 171 with reward -186.962 in 200 steps 2023-06-18 16:42:44 - SimpleLog - INFO: - Interactor 1 finished episode 172 with reward -279.426 in 200 steps 2023-06-18 16:42:45 - SimpleLog - INFO: - update_step: 17200, online_eval_reward: -403.423 2023-06-18 16:42:46 - SimpleLog - INFO: - update_step: 17300, online_eval_reward: -377.556 2023-06-18 16:42:47 - SimpleLog - INFO: - Interactor 0 finished episode 173 with reward -129.997 in 200 steps 2023-06-18 16:42:47 - SimpleLog - INFO: - Interactor 1 finished episode 174 with reward -249.593 in 200 steps 2023-06-18 16:42:47 - SimpleLog - INFO: - update_step: 17400, online_eval_reward: -125.928 2023-06-18 16:42:47 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -125.928, save the best model! 2023-06-18 16:42:49 - SimpleLog - INFO: - update_step: 17500, online_eval_reward: -252.264 2023-06-18 16:42:49 - SimpleLog - INFO: - Interactor 0 finished episode 175 with reward -333.461 in 200 steps 2023-06-18 16:42:49 - SimpleLog - INFO: - Interactor 1 finished episode 176 with reward -380.684 in 200 steps 2023-06-18 16:42:50 - SimpleLog - INFO: - update_step: 17600, online_eval_reward: -377.967 2023-06-18 16:42:51 - SimpleLog - INFO: - update_step: 17700, online_eval_reward: -502.527 2023-06-18 16:42:52 - SimpleLog - INFO: - Interactor 0 finished episode 177 with reward -382.695 in 200 steps 2023-06-18 16:42:52 - SimpleLog - INFO: - Interactor 1 finished episode 178 with reward -377.542 in 200 steps 2023-06-18 16:42:52 - SimpleLog - INFO: - update_step: 17800, online_eval_reward: -503.022 2023-06-18 16:42:54 - SimpleLog - INFO: - update_step: 17900, online_eval_reward: -509.103 2023-06-18 16:42:54 - SimpleLog - INFO: - Interactor 0 finished episode 179 with reward -382.172 in 200 steps 2023-06-18 16:42:54 - SimpleLog - INFO: - Interactor 1 finished episode 180 with reward -500.410 in 200 steps 2023-06-18 16:42:55 - SimpleLog - INFO: - update_step: 18000, online_eval_reward: -518.620 2023-06-18 16:42:56 - SimpleLog - INFO: - update_step: 18100, online_eval_reward: -502.824 2023-06-18 16:42:57 - SimpleLog - INFO: - Interactor 0 finished episode 181 with reward -381.368 in 200 steps 2023-06-18 16:42:57 - SimpleLog - INFO: - Interactor 1 finished episode 182 with reward -500.005 in 200 steps 2023-06-18 16:42:58 - SimpleLog - INFO: - update_step: 18200, online_eval_reward: -748.451 2023-06-18 16:42:59 - SimpleLog - INFO: - update_step: 18300, online_eval_reward: -519.708 2023-06-18 16:42:59 - SimpleLog - INFO: - Interactor 0 finished episode 183 with reward -382.336 in 200 steps 2023-06-18 16:42:59 - SimpleLog - INFO: - Interactor 1 finished episode 184 with reward -377.370 in 200 steps 2023-06-18 16:43:00 - SimpleLog - INFO: - update_step: 18400, online_eval_reward: -503.253 2023-06-18 16:43:01 - SimpleLog - INFO: - update_step: 18500, online_eval_reward: -504.352 2023-06-18 16:43:02 - SimpleLog - INFO: - Interactor 0 finished episode 185 with reward -381.589 in 200 steps 2023-06-18 16:43:02 - SimpleLog - INFO: - Interactor 1 finished episode 186 with reward -498.094 in 200 steps 2023-06-18 16:43:03 - SimpleLog - INFO: - update_step: 18600, online_eval_reward: -621.553 2023-06-18 16:43:04 - SimpleLog - INFO: - update_step: 18700, online_eval_reward: -503.396 2023-06-18 16:43:04 - SimpleLog - INFO: - Interactor 0 finished episode 187 with reward -381.901 in 200 steps 2023-06-18 16:43:04 - SimpleLog - INFO: - Interactor 1 finished episode 188 with reward -500.203 in 200 steps 2023-06-18 16:43:05 - SimpleLog - INFO: - update_step: 18800, online_eval_reward: -378.566 2023-06-18 16:43:06 - SimpleLog - INFO: - update_step: 18900, online_eval_reward: -503.793 2023-06-18 16:43:07 - SimpleLog - INFO: - Interactor 0 finished episode 189 with reward -381.694 in 200 steps 2023-06-18 16:43:07 - SimpleLog - INFO: - Interactor 1 finished episode 190 with reward -500.428 in 200 steps 2023-06-18 16:43:08 - SimpleLog - INFO: - update_step: 19000, online_eval_reward: -502.545 2023-06-18 16:43:09 - SimpleLog - INFO: - update_step: 19100, online_eval_reward: -501.932 2023-06-18 16:43:10 - SimpleLog - INFO: - Interactor 0 finished episode 191 with reward -268.149 in 200 steps 2023-06-18 16:43:10 - SimpleLog - INFO: - Interactor 1 finished episode 192 with reward -376.541 in 200 steps 2023-06-18 16:43:10 - SimpleLog - INFO: - update_step: 19200, online_eval_reward: -558.227 2023-06-18 16:43:12 - SimpleLog - INFO: - update_step: 19300, online_eval_reward: -517.582 2023-06-18 16:43:12 - SimpleLog - INFO: - Interactor 0 finished episode 193 with reward -259.277 in 200 steps 2023-06-18 16:43:12 - SimpleLog - INFO: - Interactor 1 finished episode 194 with reward -377.712 in 200 steps 2023-06-18 16:43:13 - SimpleLog - INFO: - update_step: 19400, online_eval_reward: -500.636 2023-06-18 16:43:14 - SimpleLog - INFO: - update_step: 19500, online_eval_reward: -377.581 2023-06-18 16:43:15 - SimpleLog - INFO: - Interactor 0 finished episode 195 with reward -258.805 in 200 steps 2023-06-18 16:43:15 - SimpleLog - INFO: - Interactor 1 finished episode 196 with reward -374.762 in 200 steps 2023-06-18 16:43:15 - SimpleLog - INFO: - update_step: 19600, online_eval_reward: -125.934 2023-06-18 16:43:17 - SimpleLog - INFO: - update_step: 19700, online_eval_reward: -502.898 2023-06-18 16:43:17 - SimpleLog - INFO: - Interactor 0 finished episode 197 with reward -134.881 in 200 steps 2023-06-18 16:43:17 - SimpleLog - INFO: - Interactor 1 finished episode 198 with reward -274.250 in 200 steps 2023-06-18 16:43:18 - SimpleLog - INFO: - update_step: 19800, online_eval_reward: -377.947 2023-06-18 16:43:19 - SimpleLog - INFO: - update_step: 19900, online_eval_reward: -380.967 2023-06-18 16:43:20 - SimpleLog - INFO: - Interactor 0 finished episode 199 with reward -131.490 in 200 steps 2023-06-18 16:43:20 - SimpleLog - INFO: - Interactor 1 finished episode 200 with reward -254.782 in 200 steps 2023-06-18 16:43:20 - SimpleLog - INFO: - Finish training! Time cost: 248.411 s