2023-05-30 00:09:23 - SimpleLog - INFO: - General Configs: 2023-05-30 00:09:23 - SimpleLog - INFO: - ================================================================================ 2023-05-30 00:09:23 - SimpleLog - INFO: - Name Value Type 2023-05-30 00:09:23 - SimpleLog - INFO: - env_name gym 2023-05-30 00:09:23 - SimpleLog - INFO: - algo_name TD3 2023-05-30 00:09:23 - SimpleLog - INFO: - mode train 2023-05-30 00:09:23 - SimpleLog - INFO: - device cuda 2023-05-30 00:09:23 - SimpleLog - INFO: - seed 2023 2023-05-30 00:09:23 - SimpleLog - INFO: - max_episode 10000 2023-05-30 00:09:23 - SimpleLog - INFO: - max_step 1000 2023-05-30 00:09:23 - SimpleLog - INFO: - collect_traj False 2023-05-30 00:09:23 - SimpleLog - INFO: - mp_backend single 2023-05-30 00:09:23 - SimpleLog - INFO: - n_workers 1 2023-05-30 00:09:23 - SimpleLog - INFO: - n_learners 1 2023-05-30 00:09:23 - SimpleLog - INFO: - share_buffer True 2023-05-30 00:09:23 - SimpleLog - INFO: - online_eval True 2023-05-30 00:09:23 - SimpleLog - INFO: - online_eval_episode 10 2023-05-30 00:09:23 - SimpleLog - INFO: - model_save_fre 70000 2023-05-30 00:09:23 - SimpleLog - INFO: - load_checkpoint False 2023-05-30 00:09:23 - SimpleLog - INFO: - load_path Train_single_BipedalWalkerHardcore-v3_TD3_20230528-151042 2023-05-30 00:09:23 - SimpleLog - INFO: - load_model_step best 2023-05-30 00:09:23 - SimpleLog - INFO: - eval_per_episode 50 2023-05-30 00:09:23 - SimpleLog - INFO: - ================================================================================ 2023-05-30 00:09:23 - SimpleLog - INFO: - Algo Configs: 2023-05-30 00:09:23 - SimpleLog - INFO: - ================================================================================ 2023-05-30 00:09:23 - SimpleLog - INFO: - Name Value Type 2023-05-30 00:09:23 - SimpleLog - INFO: - action_type DPG 2023-05-30 00:09:23 - SimpleLog - INFO: - buffer_type REPLAY_QUE 2023-05-30 00:09:23 - SimpleLog - INFO: - explore_steps 2048 2023-05-30 00:09:23 - SimpleLog - INFO: - policy_freq 2 2023-05-30 00:09:23 - SimpleLog - INFO: - actor_lr 0.0001 2023-05-30 00:09:23 - SimpleLog - INFO: - critic_lr 0.00012 2023-05-30 00:09:23 - SimpleLog - INFO: - gamma 0.99 2023-05-30 00:09:23 - SimpleLog - INFO: - tau 0.005 2023-05-30 00:09:23 - SimpleLog - INFO: - policy_noise 0.2 2023-05-30 00:09:23 - SimpleLog - INFO: - expl_noise 0.25 2023-05-30 00:09:23 - SimpleLog - INFO: - noise_clip 0.5 2023-05-30 00:09:23 - SimpleLog - INFO: - batch_size 256 2023-05-30 00:09:23 - SimpleLog - INFO: - buffer_size 60000 2023-05-30 00:09:23 - SimpleLog - INFO: - actor_layers [{'layer_type': 'linear', 'layer_size': [200], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [200], 'activation': 'relu'}] 2023-05-30 00:09:23 - SimpleLog - INFO: - critic_layers [{'layer_type': 'linear', 'layer_size': [200], 'activation': 'relu'}, {'layer_type': 'linear', 'layer_size': [200], 'activation': 'relu'}] 2023-05-30 00:09:23 - SimpleLog - INFO: - ================================================================================ 2023-05-30 00:09:23 - SimpleLog - INFO: - Env Configs: 2023-05-30 00:09:23 - SimpleLog - INFO: - ================================================================================ 2023-05-30 00:09:23 - SimpleLog - INFO: - Name Value Type 2023-05-30 00:09:23 - SimpleLog - INFO: - id BipedalWalkerHardcore-v3 2023-05-30 00:09:23 - SimpleLog - INFO: - render_mode None 2023-05-30 00:09:23 - SimpleLog - INFO: - wrapper None 2023-05-30 00:09:23 - SimpleLog - INFO: - ignore_params ['wrapper', 'ignore_params'] 2023-05-30 00:09:23 - SimpleLog - INFO: - ================================================================================ 2023-05-30 00:09:23 - SimpleLog - INFO: - obs_space: Box([-3.1415927 -5. -5. -5. -3.1415927 -5. -3.1415927 -5. -0. -3.1415927 -5. -3.1415927 -5. -0. -1. -1. -1. -1. -1. -1. -1. -1. -1. -1. ], [3.1415927 5. 5. 5. 3.1415927 5. 3.1415927 5. 5. 3.1415927 5. 3.1415927 5. 5. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. ], (24,), float32), n_actions: Box(-1.0, 1.0, (4,), float32) 2023-05-30 00:09:23 - SimpleLog - INFO: - Start training! 2023-05-30 00:09:23 - SimpleLog - INFO: - episode: 0, ep_reward: -118.109, ep_step: 136 2023-05-30 00:09:24 - SimpleLog - INFO: - episode: 1, ep_reward: -52.068, ep_step: 1000 2023-05-30 00:09:24 - SimpleLog - INFO: - episode: 2, ep_reward: -47.616, ep_step: 1000 2023-05-30 00:09:25 - SimpleLog - INFO: - episode: 3, ep_reward: -112.923, ep_step: 491 2023-05-30 00:09:25 - SimpleLog - INFO: - episode: 4, ep_reward: -120.951, ep_step: 42 2023-05-30 00:09:25 - SimpleLog - INFO: - episode: 5, ep_reward: -110.589, ep_step: 99 2023-05-30 00:09:26 - SimpleLog - INFO: - episode: 6, ep_reward: -115.993, ep_step: 81 2023-05-30 00:09:26 - SimpleLog - INFO: - episode: 7, ep_reward: -113.001, ep_step: 73 2023-05-30 00:09:26 - SimpleLog - INFO: - episode: 8, ep_reward: -105.139, ep_step: 56 2023-05-30 00:09:26 - SimpleLog - INFO: - episode: 9, ep_reward: -97.453, ep_step: 80 2023-05-30 00:09:28 - SimpleLog - INFO: - episode: 10, ep_reward: -46.015, ep_step: 1000 2023-05-30 00:09:29 - SimpleLog - INFO: - episode: 11, ep_reward: -129.777, ep_step: 401 2023-05-30 00:09:29 - SimpleLog - INFO: - episode: 12, ep_reward: -106.822, ep_step: 118 2023-05-30 00:09:31 - SimpleLog - INFO: - episode: 13, ep_reward: -47.969, ep_step: 1000 2023-05-30 00:09:31 - SimpleLog - INFO: - episode: 14, ep_reward: -135.421, ep_step: 96 2023-05-30 00:09:31 - SimpleLog - INFO: - episode: 15, ep_reward: -129.174, ep_step: 119 2023-05-30 00:09:31 - SimpleLog - INFO: - episode: 16, ep_reward: -118.952, ep_step: 48 2023-05-30 00:09:31 - SimpleLog - INFO: - episode: 17, ep_reward: -136.749, ep_step: 144 2023-05-30 00:09:32 - SimpleLog - INFO: - episode: 18, ep_reward: -128.281, ep_step: 79 2023-05-30 00:09:32 - SimpleLog - INFO: - episode: 19, ep_reward: -130.081, ep_step: 112 2023-05-30 00:09:32 - SimpleLog - INFO: - episode: 20, ep_reward: -115.879, ep_step: 114 2023-05-30 00:09:32 - SimpleLog - INFO: - episode: 21, ep_reward: -113.565, ep_step: 106 2023-05-30 00:09:33 - SimpleLog - INFO: - episode: 22, ep_reward: -141.891, ep_step: 145 2023-05-30 00:09:33 - SimpleLog - INFO: - episode: 23, ep_reward: -134.299, ep_step: 85 2023-05-30 00:09:33 - SimpleLog - INFO: - episode: 24, ep_reward: -140.157, ep_step: 119 2023-05-30 00:09:35 - SimpleLog - INFO: - episode: 25, ep_reward: -72.603, ep_step: 1000 2023-05-30 00:09:37 - SimpleLog - INFO: - episode: 26, ep_reward: -65.186, ep_step: 1000 2023-05-30 00:09:37 - SimpleLog - INFO: - episode: 27, ep_reward: -121.202, ep_step: 101 2023-05-30 00:09:38 - SimpleLog - INFO: - episode: 28, ep_reward: -126.576, ep_step: 91 2023-05-30 00:09:40 - SimpleLog - INFO: - episode: 29, ep_reward: -54.432, ep_step: 1000 2023-05-30 00:09:40 - SimpleLog - INFO: - episode: 30, ep_reward: -116.445, ep_step: 238 2023-05-30 00:09:40 - SimpleLog - INFO: - episode: 31, ep_reward: -122.008, ep_step: 96 2023-05-30 00:09:40 - SimpleLog - INFO: - episode: 32, ep_reward: -122.802, ep_step: 87 2023-05-30 00:09:41 - SimpleLog - INFO: - episode: 33, ep_reward: -124.089, ep_step: 97 2023-05-30 00:09:41 - SimpleLog - INFO: - episode: 34, ep_reward: -124.596, ep_step: 89 2023-05-30 00:09:41 - SimpleLog - INFO: - episode: 35, ep_reward: -124.609, ep_step: 75 2023-05-30 00:09:41 - SimpleLog - INFO: - episode: 36, ep_reward: -128.658, ep_step: 151 2023-05-30 00:09:42 - SimpleLog - INFO: - episode: 37, ep_reward: -125.054, ep_step: 105 2023-05-30 00:09:42 - SimpleLog - INFO: - episode: 38, ep_reward: -128.101, ep_step: 111 2023-05-30 00:09:42 - SimpleLog - INFO: - episode: 39, ep_reward: -142.047, ep_step: 171 2023-05-30 00:09:43 - SimpleLog - INFO: - episode: 40, ep_reward: -142.014, ep_step: 193 2023-05-30 00:09:43 - SimpleLog - INFO: - episode: 41, ep_reward: -134.687, ep_step: 154 2023-05-30 00:09:43 - SimpleLog - INFO: - episode: 42, ep_reward: -127.684, ep_step: 80 2023-05-30 00:09:43 - SimpleLog - INFO: - episode: 43, ep_reward: -127.255, ep_step: 78 2023-05-30 00:09:43 - SimpleLog - INFO: - episode: 44, ep_reward: -118.566, ep_step: 103 2023-05-30 00:09:44 - SimpleLog - INFO: - episode: 45, ep_reward: -125.442, ep_step: 81 2023-05-30 00:09:46 - SimpleLog - INFO: - episode: 46, ep_reward: -42.512, ep_step: 1000 2023-05-30 00:09:46 - SimpleLog - INFO: - episode: 47, ep_reward: -101.374, ep_step: 130 2023-05-30 00:09:46 - SimpleLog - INFO: - episode: 48, ep_reward: -99.874, ep_step: 118 2023-05-30 00:09:47 - SimpleLog - INFO: - episode: 49, ep_reward: -109.870, ep_step: 147 2023-05-30 00:09:47 - SimpleLog - INFO: - episode: 50, ep_reward: -102.754, ep_step: 148 2023-05-30 00:09:47 - SimpleLog - INFO: - episode: 51, ep_reward: -103.472, ep_step: 108 2023-05-30 00:09:47 - SimpleLog - INFO: - episode: 52, ep_reward: -101.477, ep_step: 78 2023-05-30 00:09:47 - SimpleLog - INFO: - episode: 53, ep_reward: -103.939, ep_step: 94 2023-05-30 00:09:48 - SimpleLog - INFO: - episode: 54, ep_reward: -106.504, ep_step: 94 2023-05-30 00:09:48 - SimpleLog - INFO: - episode: 55, ep_reward: -110.334, ep_step: 101 2023-05-30 00:09:48 - SimpleLog - INFO: - episode: 56, ep_reward: -106.655, ep_step: 169 2023-05-30 00:09:48 - SimpleLog - INFO: - episode: 57, ep_reward: -103.846, ep_step: 94 2023-05-30 00:09:49 - SimpleLog - INFO: - episode: 58, ep_reward: -108.000, ep_step: 97 2023-05-30 00:09:49 - SimpleLog - INFO: - episode: 59, ep_reward: -110.319, ep_step: 99 2023-05-30 00:09:49 - SimpleLog - INFO: - episode: 60, ep_reward: -107.814, ep_step: 160 2023-05-30 00:09:49 - SimpleLog - INFO: - episode: 61, ep_reward: -94.304, ep_step: 141 2023-05-30 00:09:50 - SimpleLog - INFO: - episode: 62, ep_reward: -105.712, ep_step: 127 2023-05-30 00:09:50 - SimpleLog - INFO: - episode: 63, ep_reward: -103.006, ep_step: 117 2023-05-30 00:09:50 - SimpleLog - INFO: - episode: 64, ep_reward: -97.814, ep_step: 115 2023-05-30 00:09:51 - SimpleLog - INFO: - episode: 65, ep_reward: -123.613, ep_step: 193 2023-05-30 00:09:51 - SimpleLog - INFO: - episode: 66, ep_reward: -107.269, ep_step: 109 2023-05-30 00:09:51 - SimpleLog - INFO: - episode: 67, ep_reward: -100.103, ep_step: 87 2023-05-30 00:09:51 - SimpleLog - INFO: - episode: 68, ep_reward: -107.168, ep_step: 97 2023-05-30 00:09:51 - SimpleLog - INFO: - episode: 69, ep_reward: -121.250, ep_step: 89 2023-05-30 00:09:52 - SimpleLog - INFO: - episode: 70, ep_reward: -116.318, ep_step: 136 2023-05-30 00:09:52 - SimpleLog - INFO: - episode: 71, ep_reward: -128.897, ep_step: 97 2023-05-30 00:09:52 - SimpleLog - INFO: - episode: 72, ep_reward: -118.330, ep_step: 85 2023-05-30 00:09:52 - SimpleLog - INFO: - episode: 73, ep_reward: -105.642, ep_step: 98 2023-05-30 00:09:52 - SimpleLog - INFO: - episode: 74, ep_reward: -123.085, ep_step: 79 2023-05-30 00:09:53 - SimpleLog - INFO: - episode: 75, ep_reward: -118.701, ep_step: 85 2023-05-30 00:09:53 - SimpleLog - INFO: - episode: 76, ep_reward: -115.639, ep_step: 78 2023-05-30 00:09:53 - SimpleLog - INFO: - episode: 77, ep_reward: -119.810, ep_step: 85 2023-05-30 00:09:53 - SimpleLog - INFO: - episode: 78, ep_reward: -103.641, ep_step: 126 2023-05-30 00:09:53 - SimpleLog - INFO: - episode: 79, ep_reward: -135.836, ep_step: 105 2023-05-30 00:09:54 - SimpleLog - INFO: - episode: 80, ep_reward: -108.003, ep_step: 94 2023-05-30 00:09:54 - SimpleLog - INFO: - episode: 81, ep_reward: -120.050, ep_step: 102 2023-05-30 00:09:54 - SimpleLog - INFO: - episode: 82, ep_reward: -97.621, ep_step: 68 2023-05-30 00:09:54 - SimpleLog - INFO: - episode: 83, ep_reward: -115.458, ep_step: 165 2023-05-30 00:09:54 - SimpleLog - INFO: - episode: 84, ep_reward: -114.714, ep_step: 80 2023-05-30 00:09:55 - SimpleLog - INFO: - episode: 85, ep_reward: -116.994, ep_step: 103 2023-05-30 00:09:55 - SimpleLog - INFO: - episode: 86, ep_reward: -137.198, ep_step: 259 2023-05-30 00:09:55 - SimpleLog - INFO: - episode: 87, ep_reward: -111.293, ep_step: 122 2023-05-30 00:09:55 - SimpleLog - INFO: - episode: 88, ep_reward: -118.572, ep_step: 60 2023-05-30 00:09:56 - SimpleLog - INFO: - episode: 89, ep_reward: -115.596, ep_step: 48 2023-05-30 00:09:56 - SimpleLog - INFO: - episode: 90, ep_reward: -115.849, ep_step: 57 2023-05-30 00:09:56 - SimpleLog - INFO: - episode: 91, ep_reward: -119.205, ep_step: 124 2023-05-30 00:09:56 - SimpleLog - INFO: - episode: 92, ep_reward: -110.410, ep_step: 46 2023-05-30 00:09:56 - SimpleLog - INFO: - episode: 93, ep_reward: -123.405, ep_step: 76 2023-05-30 00:09:56 - SimpleLog - INFO: - episode: 94, ep_reward: -111.966, ep_step: 49 2023-05-30 00:09:57 - SimpleLog - INFO: - episode: 95, ep_reward: -117.326, ep_step: 150 2023-05-30 00:09:57 - SimpleLog - INFO: - episode: 96, ep_reward: -117.241, ep_step: 55 2023-05-30 00:09:57 - SimpleLog - INFO: - episode: 97, ep_reward: -117.404, ep_step: 126 2023-05-30 00:09:57 - SimpleLog - INFO: - episode: 98, ep_reward: -114.397, ep_step: 59 2023-05-30 00:09:58 - SimpleLog - INFO: - episode: 99, ep_reward: -120.218, ep_step: 269 2023-05-30 00:10:00 - SimpleLog - INFO: - episode: 100, ep_reward: -54.364, ep_step: 1000 2023-05-30 00:10:02 - SimpleLog - INFO: - episode: 101, ep_reward: -83.903, ep_step: 1000 2023-05-30 00:10:04 - SimpleLog - INFO: - episode: 102, ep_reward: -63.988, ep_step: 1000 2023-05-30 00:10:05 - SimpleLog - INFO: - episode: 103, ep_reward: -156.025, ep_step: 506 2023-05-30 00:10:07 - SimpleLog - INFO: - episode: 104, ep_reward: -61.555, ep_step: 1000 2023-05-30 00:10:07 - SimpleLog - INFO: - episode: 105, ep_reward: -110.052, ep_step: 105 2023-05-30 00:10:09 - SimpleLog - INFO: - episode: 106, ep_reward: -53.630, ep_step: 1000 2023-05-30 00:10:11 - SimpleLog - INFO: - episode: 107, ep_reward: -56.104, ep_step: 1000 2023-05-30 00:10:13 - SimpleLog - INFO: - episode: 108, ep_reward: -64.533, ep_step: 1000 2023-05-30 00:10:15 - SimpleLog - INFO: - episode: 109, ep_reward: -76.086, ep_step: 1000 2023-05-30 00:10:18 - SimpleLog - INFO: - episode: 110, ep_reward: -183.528, ep_step: 979 2023-05-30 00:10:19 - SimpleLog - INFO: - episode: 111, ep_reward: -154.750, ep_step: 516 2023-05-30 00:10:20 - SimpleLog - INFO: - episode: 112, ep_reward: -159.547, ep_step: 579 2023-05-30 00:10:20 - SimpleLog - INFO: - episode: 113, ep_reward: -121.647, ep_step: 72 2023-05-30 00:10:21 - SimpleLog - INFO: - episode: 114, ep_reward: -167.084, ep_step: 664 2023-05-30 00:10:22 - SimpleLog - INFO: - episode: 115, ep_reward: -120.125, ep_step: 68 2023-05-30 00:10:24 - SimpleLog - INFO: - episode: 116, ep_reward: -180.500, ep_step: 909 2023-05-30 00:10:25 - SimpleLog - INFO: - episode: 117, ep_reward: -161.718, ep_step: 609 2023-05-30 00:10:27 - SimpleLog - INFO: - episode: 118, ep_reward: -75.790, ep_step: 1000 2023-05-30 00:10:29 - SimpleLog - INFO: - episode: 119, ep_reward: -173.191, ep_step: 817 2023-05-30 00:10:31 - SimpleLog - INFO: - episode: 120, ep_reward: -180.724, ep_step: 895 2023-05-30 00:10:33 - SimpleLog - INFO: - episode: 121, ep_reward: -175.454, ep_step: 888 2023-05-30 00:10:35 - SimpleLog - INFO: - episode: 122, ep_reward: -160.865, ep_step: 665 2023-05-30 00:10:37 - SimpleLog - INFO: - episode: 123, ep_reward: -71.415, ep_step: 1000 2023-05-30 00:10:39 - SimpleLog - INFO: - episode: 124, ep_reward: -74.497, ep_step: 1000 2023-05-30 00:10:41 - SimpleLog - INFO: - episode: 125, ep_reward: -77.529, ep_step: 1000 2023-05-30 00:10:42 - SimpleLog - INFO: - episode: 126, ep_reward: -109.738, ep_step: 61 2023-05-30 00:10:44 - SimpleLog - INFO: - episode: 127, ep_reward: -74.404, ep_step: 1000 2023-05-30 00:10:46 - SimpleLog - INFO: - episode: 128, ep_reward: -57.532, ep_step: 1000 2023-05-30 00:10:46 - SimpleLog - INFO: - episode: 129, ep_reward: -112.859, ep_step: 70 2023-05-30 00:10:47 - SimpleLog - INFO: - episode: 130, ep_reward: -147.227, ep_step: 173 2023-05-30 00:10:47 - SimpleLog - INFO: - episode: 131, ep_reward: -116.996, ep_step: 75 2023-05-30 00:10:47 - SimpleLog - INFO: - episode: 132, ep_reward: -120.778, ep_step: 79 2023-05-30 00:10:47 - SimpleLog - INFO: - episode: 133, ep_reward: -109.593, ep_step: 64 2023-05-30 00:10:47 - SimpleLog - INFO: - episode: 134, ep_reward: -116.021, ep_step: 189 2023-05-30 00:10:50 - SimpleLog - INFO: - episode: 135, ep_reward: -64.159, ep_step: 1000 2023-05-30 00:10:52 - SimpleLog - INFO: - episode: 136, ep_reward: -70.756, ep_step: 1000 2023-05-30 00:10:54 - SimpleLog - INFO: - episode: 137, ep_reward: -65.094, ep_step: 1000 2023-05-30 00:10:56 - SimpleLog - INFO: - episode: 138, ep_reward: -55.175, ep_step: 1000 2023-05-30 00:10:59 - SimpleLog - INFO: - episode: 139, ep_reward: -47.194, ep_step: 1000 2023-05-30 00:11:01 - SimpleLog - INFO: - episode: 140, ep_reward: -41.067, ep_step: 1000 2023-05-30 00:11:03 - SimpleLog - INFO: - episode: 141, ep_reward: -40.782, ep_step: 1000 2023-05-30 00:11:05 - SimpleLog - INFO: - episode: 142, ep_reward: -40.009, ep_step: 1000 2023-05-30 00:11:08 - SimpleLog - INFO: - episode: 143, ep_reward: -47.356, ep_step: 1000 2023-05-30 00:11:10 - SimpleLog - INFO: - episode: 144, ep_reward: -48.580, ep_step: 1000 2023-05-30 00:11:12 - SimpleLog - INFO: - episode: 145, ep_reward: -54.439, ep_step: 1000 2023-05-30 00:11:14 - SimpleLog - INFO: - episode: 146, ep_reward: -60.690, ep_step: 1000 2023-05-30 00:11:16 - SimpleLog - INFO: - episode: 147, ep_reward: -55.913, ep_step: 1000 2023-05-30 00:11:19 - SimpleLog - INFO: - episode: 148, ep_reward: -53.773, ep_step: 1000 2023-05-30 00:11:21 - SimpleLog - INFO: - episode: 149, ep_reward: -65.735, ep_step: 1000 2023-05-30 00:11:23 - SimpleLog - INFO: - episode: 150, ep_reward: -67.227, ep_step: 1000 2023-05-30 00:11:26 - SimpleLog - INFO: - episode: 151, ep_reward: -66.363, ep_step: 1000 2023-05-30 00:11:28 - SimpleLog - INFO: - episode: 152, ep_reward: -74.205, ep_step: 1000 2023-05-30 00:11:30 - SimpleLog - INFO: - episode: 153, ep_reward: -50.752, ep_step: 1000 2023-05-30 00:11:32 - SimpleLog - INFO: - episode: 154, ep_reward: -47.004, ep_step: 1000 2023-05-30 00:11:35 - SimpleLog - INFO: - episode: 155, ep_reward: -59.197, ep_step: 1000 2023-05-30 00:11:37 - SimpleLog - INFO: - episode: 156, ep_reward: -53.276, ep_step: 1000 2023-05-30 00:11:39 - SimpleLog - INFO: - episode: 157, ep_reward: -51.727, ep_step: 1000 2023-05-30 00:11:41 - SimpleLog - INFO: - episode: 158, ep_reward: -45.629, ep_step: 1000 2023-05-30 00:11:44 - SimpleLog - INFO: - episode: 159, ep_reward: -43.302, ep_step: 1000 2023-05-30 00:11:46 - SimpleLog - INFO: - episode: 160, ep_reward: -45.260, ep_step: 1000 2023-05-30 00:11:48 - SimpleLog - INFO: - episode: 161, ep_reward: -42.460, ep_step: 1000 2023-05-30 00:11:50 - SimpleLog - INFO: - episode: 162, ep_reward: -41.699, ep_step: 1000 2023-05-30 00:11:54 - SimpleLog - INFO: - update_step: 70000, online_eval_reward: -40.015 2023-05-30 00:11:54 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -40.015, save the best model! 2023-05-30 00:11:55 - SimpleLog - INFO: - episode: 163, ep_reward: -44.805, ep_step: 1000 2023-05-30 00:11:57 - SimpleLog - INFO: - episode: 164, ep_reward: -41.027, ep_step: 1000 2023-05-30 00:11:59 - SimpleLog - INFO: - episode: 165, ep_reward: -43.469, ep_step: 1000 2023-05-30 00:12:01 - SimpleLog - INFO: - episode: 166, ep_reward: -52.759, ep_step: 1000 2023-05-30 00:12:04 - SimpleLog - INFO: - episode: 167, ep_reward: -55.477, ep_step: 1000 2023-05-30 00:12:06 - SimpleLog - INFO: - episode: 168, ep_reward: -63.006, ep_step: 1000 2023-05-30 00:12:08 - SimpleLog - INFO: - episode: 169, ep_reward: -62.527, ep_step: 1000 2023-05-30 00:12:10 - SimpleLog - INFO: - episode: 170, ep_reward: -55.866, ep_step: 1000 2023-05-30 00:12:13 - SimpleLog - INFO: - episode: 171, ep_reward: -60.461, ep_step: 1000 2023-05-30 00:12:15 - SimpleLog - INFO: - episode: 172, ep_reward: -66.784, ep_step: 1000 2023-05-30 00:12:17 - SimpleLog - INFO: - episode: 173, ep_reward: -62.443, ep_step: 1000 2023-05-30 00:12:19 - SimpleLog - INFO: - episode: 174, ep_reward: -58.784, ep_step: 1000 2023-05-30 00:12:22 - SimpleLog - INFO: - episode: 175, ep_reward: -155.470, ep_step: 920 2023-05-30 00:12:24 - SimpleLog - INFO: - episode: 176, ep_reward: -63.969, ep_step: 1000 2023-05-30 00:12:26 - SimpleLog - INFO: - episode: 177, ep_reward: -66.538, ep_step: 1000 2023-05-30 00:12:28 - SimpleLog - INFO: - episode: 178, ep_reward: -57.102, ep_step: 1000 2023-05-30 00:12:31 - SimpleLog - INFO: - episode: 179, ep_reward: -52.257, ep_step: 1000 2023-05-30 00:12:33 - SimpleLog - INFO: - episode: 180, ep_reward: -58.195, ep_step: 1000 2023-05-30 00:12:35 - SimpleLog - INFO: - episode: 181, ep_reward: -58.537, ep_step: 1000 2023-05-30 00:12:37 - SimpleLog - INFO: - episode: 182, ep_reward: -56.102, ep_step: 1000 2023-05-30 00:12:40 - SimpleLog - INFO: - episode: 183, ep_reward: -62.051, ep_step: 1000 2023-05-30 00:12:42 - SimpleLog - INFO: - episode: 184, ep_reward: -61.817, ep_step: 1000 2023-05-30 00:12:44 - SimpleLog - INFO: - episode: 185, ep_reward: -64.057, ep_step: 1000 2023-05-30 00:12:46 - SimpleLog - INFO: - episode: 186, ep_reward: -59.727, ep_step: 1000 2023-05-30 00:12:49 - SimpleLog - INFO: - episode: 187, ep_reward: -57.955, ep_step: 1000 2023-05-30 00:12:51 - SimpleLog - INFO: - episode: 188, ep_reward: -56.170, ep_step: 1000 2023-05-30 00:12:53 - SimpleLog - INFO: - episode: 189, ep_reward: -53.152, ep_step: 1000 2023-05-30 00:12:56 - SimpleLog - INFO: - episode: 190, ep_reward: -52.327, ep_step: 1000 2023-05-30 00:12:58 - SimpleLog - INFO: - episode: 191, ep_reward: -52.558, ep_step: 1000 2023-05-30 00:13:00 - SimpleLog - INFO: - episode: 192, ep_reward: -61.252, ep_step: 1000 2023-05-30 00:13:03 - SimpleLog - INFO: - episode: 193, ep_reward: -58.474, ep_step: 1000 2023-05-30 00:13:05 - SimpleLog - INFO: - episode: 194, ep_reward: -58.873, ep_step: 1000 2023-05-30 00:13:08 - SimpleLog - INFO: - episode: 195, ep_reward: -48.840, ep_step: 1000 2023-05-30 00:13:10 - SimpleLog - INFO: - episode: 196, ep_reward: -44.370, ep_step: 1000 2023-05-30 00:13:12 - SimpleLog - INFO: - episode: 197, ep_reward: -44.943, ep_step: 1000 2023-05-30 00:13:15 - SimpleLog - INFO: - episode: 198, ep_reward: -44.273, ep_step: 1000 2023-05-30 00:13:17 - SimpleLog - INFO: - episode: 199, ep_reward: -46.435, ep_step: 1000 2023-05-30 00:13:19 - SimpleLog - INFO: - episode: 200, ep_reward: -47.364, ep_step: 1000 2023-05-30 00:13:22 - SimpleLog - INFO: - episode: 201, ep_reward: -47.856, ep_step: 1000 2023-05-30 00:13:24 - SimpleLog - INFO: - episode: 202, ep_reward: -47.451, ep_step: 1000 2023-05-30 00:13:26 - SimpleLog - INFO: - episode: 203, ep_reward: -52.730, ep_step: 1000 2023-05-30 00:13:29 - SimpleLog - INFO: - episode: 204, ep_reward: -47.120, ep_step: 1000 2023-05-30 00:13:31 - SimpleLog - INFO: - episode: 205, ep_reward: -46.637, ep_step: 1000 2023-05-30 00:13:33 - SimpleLog - INFO: - episode: 206, ep_reward: -45.290, ep_step: 1000 2023-05-30 00:13:36 - SimpleLog - INFO: - episode: 207, ep_reward: -47.236, ep_step: 1000 2023-05-30 00:13:38 - SimpleLog - INFO: - episode: 208, ep_reward: -46.276, ep_step: 1000 2023-05-30 00:13:40 - SimpleLog - INFO: - episode: 209, ep_reward: -43.488, ep_step: 1000 2023-05-30 00:13:42 - SimpleLog - INFO: - episode: 210, ep_reward: -36.709, ep_step: 1000 2023-05-30 00:13:45 - SimpleLog - INFO: - episode: 211, ep_reward: -34.178, ep_step: 1000 2023-05-30 00:13:47 - SimpleLog - INFO: - episode: 212, ep_reward: -35.912, ep_step: 1000 2023-05-30 00:13:49 - SimpleLog - INFO: - episode: 213, ep_reward: -32.318, ep_step: 1000 2023-05-30 00:13:52 - SimpleLog - INFO: - episode: 214, ep_reward: -36.048, ep_step: 1000 2023-05-30 00:13:54 - SimpleLog - INFO: - episode: 215, ep_reward: -35.314, ep_step: 1000 2023-05-30 00:13:56 - SimpleLog - INFO: - episode: 216, ep_reward: -32.354, ep_step: 1000 2023-05-30 00:13:58 - SimpleLog - INFO: - episode: 217, ep_reward: -35.408, ep_step: 1000 2023-05-30 00:14:01 - SimpleLog - INFO: - episode: 218, ep_reward: -35.801, ep_step: 1000 2023-05-30 00:14:03 - SimpleLog - INFO: - episode: 219, ep_reward: -32.564, ep_step: 1000 2023-05-30 00:14:05 - SimpleLog - INFO: - episode: 220, ep_reward: -33.301, ep_step: 1000 2023-05-30 00:14:08 - SimpleLog - INFO: - episode: 221, ep_reward: -33.370, ep_step: 1000 2023-05-30 00:14:10 - SimpleLog - INFO: - episode: 222, ep_reward: -32.305, ep_step: 1000 2023-05-30 00:14:12 - SimpleLog - INFO: - episode: 223, ep_reward: -32.902, ep_step: 1000 2023-05-30 00:14:15 - SimpleLog - INFO: - episode: 224, ep_reward: -30.996, ep_step: 1000 2023-05-30 00:14:17 - SimpleLog - INFO: - episode: 225, ep_reward: -30.603, ep_step: 1000 2023-05-30 00:14:19 - SimpleLog - INFO: - episode: 226, ep_reward: -32.677, ep_step: 1000 2023-05-30 00:14:22 - SimpleLog - INFO: - episode: 227, ep_reward: -31.969, ep_step: 1000 2023-05-30 00:14:24 - SimpleLog - INFO: - episode: 228, ep_reward: -31.623, ep_step: 1000 2023-05-30 00:14:26 - SimpleLog - INFO: - episode: 229, ep_reward: -32.908, ep_step: 1000 2023-05-30 00:14:29 - SimpleLog - INFO: - episode: 230, ep_reward: -36.537, ep_step: 1000 2023-05-30 00:14:31 - SimpleLog - INFO: - episode: 231, ep_reward: -33.943, ep_step: 1000 2023-05-30 00:14:31 - SimpleLog - INFO: - episode: 232, ep_reward: -116.992, ep_step: 66 2023-05-30 00:14:31 - SimpleLog - INFO: - episode: 233, ep_reward: -114.717, ep_step: 68 2023-05-30 00:14:32 - SimpleLog - INFO: - episode: 234, ep_reward: -114.847, ep_step: 65 2023-05-30 00:14:32 - SimpleLog - INFO: - episode: 235, ep_reward: -112.555, ep_step: 76 2023-05-30 00:14:34 - SimpleLog - INFO: - episode: 236, ep_reward: -34.579, ep_step: 1000 2023-05-30 00:14:37 - SimpleLog - INFO: - update_step: 140000, online_eval_reward: -33.451 2023-05-30 00:14:37 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -33.451, save the best model! 2023-05-30 00:14:38 - SimpleLog - INFO: - episode: 237, ep_reward: -35.157, ep_step: 1000 2023-05-30 00:14:39 - SimpleLog - INFO: - episode: 238, ep_reward: -113.293, ep_step: 74 2023-05-30 00:14:39 - SimpleLog - INFO: - episode: 239, ep_reward: -110.388, ep_step: 68 2023-05-30 00:14:39 - SimpleLog - INFO: - episode: 240, ep_reward: -109.484, ep_step: 78 2023-05-30 00:14:41 - SimpleLog - INFO: - episode: 241, ep_reward: -31.981, ep_step: 1000 2023-05-30 00:14:44 - SimpleLog - INFO: - episode: 242, ep_reward: -30.228, ep_step: 1000 2023-05-30 00:14:46 - SimpleLog - INFO: - episode: 243, ep_reward: -33.612, ep_step: 1000 2023-05-30 00:14:48 - SimpleLog - INFO: - episode: 244, ep_reward: -31.668, ep_step: 1000 2023-05-30 00:14:51 - SimpleLog - INFO: - episode: 245, ep_reward: -31.584, ep_step: 1000 2023-05-30 00:14:53 - SimpleLog - INFO: - episode: 246, ep_reward: -32.860, ep_step: 1000 2023-05-30 00:14:55 - SimpleLog - INFO: - episode: 247, ep_reward: -36.441, ep_step: 1000 2023-05-30 00:14:58 - SimpleLog - INFO: - episode: 248, ep_reward: -36.685, ep_step: 1000 2023-05-30 00:15:00 - SimpleLog - INFO: - episode: 249, ep_reward: -45.423, ep_step: 1000 2023-05-30 00:15:03 - SimpleLog - INFO: - episode: 250, ep_reward: -34.406, ep_step: 1000 2023-05-30 00:15:05 - SimpleLog - INFO: - episode: 251, ep_reward: -33.309, ep_step: 1000 2023-05-30 00:15:07 - SimpleLog - INFO: - episode: 252, ep_reward: -30.270, ep_step: 1000 2023-05-30 00:15:09 - SimpleLog - INFO: - episode: 253, ep_reward: -31.295, ep_step: 1000 2023-05-30 00:15:12 - SimpleLog - INFO: - episode: 254, ep_reward: -34.181, ep_step: 1000 2023-05-30 00:15:14 - SimpleLog - INFO: - episode: 255, ep_reward: -30.277, ep_step: 1000 2023-05-30 00:15:16 - SimpleLog - INFO: - episode: 256, ep_reward: -32.459, ep_step: 1000 2023-05-30 00:15:19 - SimpleLog - INFO: - episode: 257, ep_reward: -34.225, ep_step: 1000 2023-05-30 00:15:21 - SimpleLog - INFO: - episode: 258, ep_reward: -30.676, ep_step: 1000 2023-05-30 00:15:23 - SimpleLog - INFO: - episode: 259, ep_reward: -29.956, ep_step: 1000 2023-05-30 00:15:25 - SimpleLog - INFO: - episode: 260, ep_reward: -32.135, ep_step: 1000 2023-05-30 00:15:28 - SimpleLog - INFO: - episode: 261, ep_reward: -29.431, ep_step: 1000 2023-05-30 00:15:30 - SimpleLog - INFO: - episode: 262, ep_reward: -30.580, ep_step: 1000 2023-05-30 00:15:32 - SimpleLog - INFO: - episode: 263, ep_reward: -32.688, ep_step: 1000 2023-05-30 00:15:35 - SimpleLog - INFO: - episode: 264, ep_reward: -29.884, ep_step: 1000 2023-05-30 00:15:37 - SimpleLog - INFO: - episode: 265, ep_reward: -27.994, ep_step: 1000 2023-05-30 00:15:39 - SimpleLog - INFO: - episode: 266, ep_reward: -27.957, ep_step: 1000 2023-05-30 00:15:42 - SimpleLog - INFO: - episode: 267, ep_reward: -25.772, ep_step: 1000 2023-05-30 00:15:44 - SimpleLog - INFO: - episode: 268, ep_reward: -25.188, ep_step: 1000 2023-05-30 00:15:46 - SimpleLog - INFO: - episode: 269, ep_reward: -30.401, ep_step: 1000 2023-05-30 00:15:48 - SimpleLog - INFO: - episode: 270, ep_reward: -24.796, ep_step: 1000 2023-05-30 00:15:51 - SimpleLog - INFO: - episode: 271, ep_reward: -30.243, ep_step: 1000 2023-05-30 00:15:53 - SimpleLog - INFO: - episode: 272, ep_reward: -24.682, ep_step: 1000 2023-05-30 00:15:55 - SimpleLog - INFO: - episode: 273, ep_reward: -21.693, ep_step: 1000 2023-05-30 00:15:58 - SimpleLog - INFO: - episode: 274, ep_reward: -25.083, ep_step: 1000 2023-05-30 00:16:00 - SimpleLog - INFO: - episode: 275, ep_reward: -90.359, ep_step: 1000 2023-05-30 00:16:01 - SimpleLog - INFO: - episode: 276, ep_reward: -152.787, ep_step: 409 2023-05-30 00:16:03 - SimpleLog - INFO: - episode: 277, ep_reward: -95.065, ep_step: 1000 2023-05-30 00:16:05 - SimpleLog - INFO: - episode: 278, ep_reward: -91.216, ep_step: 1000 2023-05-30 00:16:06 - SimpleLog - INFO: - episode: 279, ep_reward: -101.595, ep_step: 141 2023-05-30 00:16:08 - SimpleLog - INFO: - episode: 280, ep_reward: -81.754, ep_step: 1000 2023-05-30 00:16:10 - SimpleLog - INFO: - episode: 281, ep_reward: -68.954, ep_step: 1000 2023-05-30 00:16:13 - SimpleLog - INFO: - episode: 282, ep_reward: -85.420, ep_step: 1000 2023-05-30 00:16:15 - SimpleLog - INFO: - episode: 283, ep_reward: -84.397, ep_step: 1000 2023-05-30 00:16:17 - SimpleLog - INFO: - episode: 284, ep_reward: -73.835, ep_step: 1000 2023-05-30 00:16:19 - SimpleLog - INFO: - episode: 285, ep_reward: -80.727, ep_step: 1000 2023-05-30 00:16:22 - SimpleLog - INFO: - episode: 286, ep_reward: -71.310, ep_step: 1000 2023-05-30 00:16:22 - SimpleLog - INFO: - episode: 287, ep_reward: -96.672, ep_step: 106 2023-05-30 00:16:24 - SimpleLog - INFO: - episode: 288, ep_reward: -76.402, ep_step: 1000 2023-05-30 00:16:26 - SimpleLog - INFO: - episode: 289, ep_reward: -58.954, ep_step: 1000 2023-05-30 00:16:29 - SimpleLog - INFO: - episode: 290, ep_reward: -63.853, ep_step: 1000 2023-05-30 00:16:30 - SimpleLog - INFO: - episode: 291, ep_reward: -110.630, ep_step: 346 2023-05-30 00:16:32 - SimpleLog - INFO: - episode: 292, ep_reward: -50.306, ep_step: 1000 2023-05-30 00:16:34 - SimpleLog - INFO: - episode: 293, ep_reward: -42.970, ep_step: 1000 2023-05-30 00:16:36 - SimpleLog - INFO: - episode: 294, ep_reward: -39.492, ep_step: 1000 2023-05-30 00:16:39 - SimpleLog - INFO: - episode: 295, ep_reward: -65.665, ep_step: 1000 2023-05-30 00:16:41 - SimpleLog - INFO: - episode: 296, ep_reward: -29.827, ep_step: 1000 2023-05-30 00:16:43 - SimpleLog - INFO: - episode: 297, ep_reward: -39.396, ep_step: 1000 2023-05-30 00:16:46 - SimpleLog - INFO: - episode: 298, ep_reward: -34.979, ep_step: 1000 2023-05-30 00:16:48 - SimpleLog - INFO: - episode: 299, ep_reward: -68.775, ep_step: 1000 2023-05-30 00:16:50 - SimpleLog - INFO: - episode: 300, ep_reward: -34.188, ep_step: 1000 2023-05-30 00:16:52 - SimpleLog - INFO: - episode: 301, ep_reward: -44.681, ep_step: 1000 2023-05-30 00:16:55 - SimpleLog - INFO: - episode: 302, ep_reward: -34.878, ep_step: 1000 2023-05-30 00:16:57 - SimpleLog - INFO: - episode: 303, ep_reward: -30.539, ep_step: 1000 2023-05-30 00:16:59 - SimpleLog - INFO: - episode: 304, ep_reward: -34.755, ep_step: 1000 2023-05-30 00:17:02 - SimpleLog - INFO: - episode: 305, ep_reward: -37.418, ep_step: 1000 2023-05-30 00:17:04 - SimpleLog - INFO: - episode: 306, ep_reward: -33.432, ep_step: 1000 2023-05-30 00:17:06 - SimpleLog - INFO: - episode: 307, ep_reward: -28.793, ep_step: 1000 2023-05-30 00:17:09 - SimpleLog - INFO: - episode: 308, ep_reward: -30.387, ep_step: 1000 2023-05-30 00:17:11 - SimpleLog - INFO: - episode: 309, ep_reward: -30.979, ep_step: 1000 2023-05-30 00:17:13 - SimpleLog - INFO: - episode: 310, ep_reward: -33.146, ep_step: 1000 2023-05-30 00:17:15 - SimpleLog - INFO: - episode: 311, ep_reward: -34.349, ep_step: 1000 2023-05-30 00:17:18 - SimpleLog - INFO: - episode: 312, ep_reward: -32.290, ep_step: 1000 2023-05-30 00:17:20 - SimpleLog - INFO: - update_step: 210000, online_eval_reward: -20.387 2023-05-30 00:17:20 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -20.387, save the best model! 2023-05-30 00:17:22 - SimpleLog - INFO: - episode: 313, ep_reward: -32.793, ep_step: 1000 2023-05-30 00:17:24 - SimpleLog - INFO: - episode: 314, ep_reward: -34.499, ep_step: 1000 2023-05-30 00:17:27 - SimpleLog - INFO: - episode: 315, ep_reward: -33.782, ep_step: 1000 2023-05-30 00:17:29 - SimpleLog - INFO: - episode: 316, ep_reward: -35.937, ep_step: 1000 2023-05-30 00:17:31 - SimpleLog - INFO: - episode: 317, ep_reward: -37.823, ep_step: 1000 2023-05-30 00:17:34 - SimpleLog - INFO: - episode: 318, ep_reward: -40.585, ep_step: 1000 2023-05-30 00:17:36 - SimpleLog - INFO: - episode: 319, ep_reward: -42.324, ep_step: 1000 2023-05-30 00:17:38 - SimpleLog - INFO: - episode: 320, ep_reward: -37.915, ep_step: 1000 2023-05-30 00:17:38 - SimpleLog - INFO: - episode: 321, ep_reward: -100.609, ep_step: 75 2023-05-30 00:17:41 - SimpleLog - INFO: - episode: 322, ep_reward: -40.304, ep_step: 1000 2023-05-30 00:17:43 - SimpleLog - INFO: - episode: 323, ep_reward: -40.564, ep_step: 1000 2023-05-30 00:17:45 - SimpleLog - INFO: - episode: 324, ep_reward: -38.856, ep_step: 1000 2023-05-30 00:17:48 - SimpleLog - INFO: - episode: 325, ep_reward: -38.709, ep_step: 1000 2023-05-30 00:17:50 - SimpleLog - INFO: - episode: 326, ep_reward: -38.646, ep_step: 1000 2023-05-30 00:17:52 - SimpleLog - INFO: - episode: 327, ep_reward: -40.528, ep_step: 1000 2023-05-30 00:17:54 - SimpleLog - INFO: - episode: 328, ep_reward: -37.690, ep_step: 1000 2023-05-30 00:17:55 - SimpleLog - INFO: - episode: 329, ep_reward: -100.195, ep_step: 74 2023-05-30 00:17:57 - SimpleLog - INFO: - episode: 330, ep_reward: -33.627, ep_step: 1000 2023-05-30 00:17:59 - SimpleLog - INFO: - episode: 331, ep_reward: -37.718, ep_step: 1000 2023-05-30 00:17:59 - SimpleLog - INFO: - episode: 332, ep_reward: -102.050, ep_step: 75 2023-05-30 00:17:59 - SimpleLog - INFO: - episode: 333, ep_reward: -102.847, ep_step: 68 2023-05-30 00:18:00 - SimpleLog - INFO: - episode: 334, ep_reward: -102.827, ep_step: 61 2023-05-30 00:18:00 - SimpleLog - INFO: - episode: 335, ep_reward: -114.231, ep_step: 104 2023-05-30 00:18:00 - SimpleLog - INFO: - episode: 336, ep_reward: -103.060, ep_step: 63 2023-05-30 00:18:00 - SimpleLog - INFO: - episode: 337, ep_reward: -102.896, ep_step: 57 2023-05-30 00:18:00 - SimpleLog - INFO: - episode: 338, ep_reward: -104.254, ep_step: 63 2023-05-30 00:18:00 - SimpleLog - INFO: - episode: 339, ep_reward: -102.261, ep_step: 62 2023-05-30 00:18:03 - SimpleLog - INFO: - episode: 340, ep_reward: -40.965, ep_step: 1000 2023-05-30 00:18:05 - SimpleLog - INFO: - episode: 341, ep_reward: -40.533, ep_step: 1000 2023-05-30 00:18:07 - SimpleLog - INFO: - episode: 342, ep_reward: -35.693, ep_step: 1000 2023-05-30 00:18:10 - SimpleLog - INFO: - episode: 343, ep_reward: -43.018, ep_step: 1000 2023-05-30 00:18:12 - SimpleLog - INFO: - episode: 344, ep_reward: -32.275, ep_step: 1000 2023-05-30 00:18:14 - SimpleLog - INFO: - episode: 345, ep_reward: -28.397, ep_step: 1000 2023-05-30 00:18:16 - SimpleLog - INFO: - episode: 346, ep_reward: -32.663, ep_step: 1000 2023-05-30 00:18:19 - SimpleLog - INFO: - episode: 347, ep_reward: -31.387, ep_step: 1000 2023-05-30 00:18:21 - SimpleLog - INFO: - episode: 348, ep_reward: -28.800, ep_step: 1000 2023-05-30 00:18:23 - SimpleLog - INFO: - episode: 349, ep_reward: -30.323, ep_step: 1000 2023-05-30 00:18:25 - SimpleLog - INFO: - episode: 350, ep_reward: -28.135, ep_step: 1000 2023-05-30 00:18:26 - SimpleLog - INFO: - episode: 351, ep_reward: -141.778, ep_step: 191 2023-05-30 00:18:26 - SimpleLog - INFO: - episode: 352, ep_reward: -133.059, ep_step: 109 2023-05-30 00:18:26 - SimpleLog - INFO: - episode: 353, ep_reward: -126.403, ep_step: 76 2023-05-30 00:18:26 - SimpleLog - INFO: - episode: 354, ep_reward: -134.086, ep_step: 116 2023-05-30 00:18:29 - SimpleLog - INFO: - episode: 355, ep_reward: -43.453, ep_step: 1000 2023-05-30 00:18:31 - SimpleLog - INFO: - episode: 356, ep_reward: -45.745, ep_step: 1000 2023-05-30 00:18:33 - SimpleLog - INFO: - episode: 357, ep_reward: -69.387, ep_step: 1000 2023-05-30 00:18:36 - SimpleLog - INFO: - episode: 358, ep_reward: -67.011, ep_step: 1000 2023-05-30 00:18:38 - SimpleLog - INFO: - episode: 359, ep_reward: -66.741, ep_step: 1000 2023-05-30 00:18:40 - SimpleLog - INFO: - episode: 360, ep_reward: -63.182, ep_step: 1000 2023-05-30 00:18:42 - SimpleLog - INFO: - episode: 361, ep_reward: -53.255, ep_step: 1000 2023-05-30 00:18:45 - SimpleLog - INFO: - episode: 362, ep_reward: -53.998, ep_step: 1000 2023-05-30 00:18:47 - SimpleLog - INFO: - episode: 363, ep_reward: -49.972, ep_step: 1000 2023-05-30 00:18:49 - SimpleLog - INFO: - episode: 364, ep_reward: -53.499, ep_step: 1000 2023-05-30 00:18:52 - SimpleLog - INFO: - episode: 365, ep_reward: -43.733, ep_step: 1000 2023-05-30 00:18:54 - SimpleLog - INFO: - episode: 366, ep_reward: -60.157, ep_step: 1000 2023-05-30 00:18:56 - SimpleLog - INFO: - episode: 367, ep_reward: -43.657, ep_step: 1000 2023-05-30 00:18:58 - SimpleLog - INFO: - episode: 368, ep_reward: -52.347, ep_step: 1000 2023-05-30 00:19:01 - SimpleLog - INFO: - episode: 369, ep_reward: -48.452, ep_step: 1000 2023-05-30 00:19:03 - SimpleLog - INFO: - episode: 370, ep_reward: -43.827, ep_step: 1000 2023-05-30 00:19:05 - SimpleLog - INFO: - episode: 371, ep_reward: -46.290, ep_step: 1000 2023-05-30 00:19:07 - SimpleLog - INFO: - episode: 372, ep_reward: -44.977, ep_step: 1000 2023-05-30 00:19:08 - SimpleLog - INFO: - episode: 373, ep_reward: -115.258, ep_step: 291 2023-05-30 00:19:09 - SimpleLog - INFO: - episode: 374, ep_reward: -133.635, ep_step: 327 2023-05-30 00:19:11 - SimpleLog - INFO: - episode: 375, ep_reward: -69.164, ep_step: 1000 2023-05-30 00:19:12 - SimpleLog - INFO: - episode: 376, ep_reward: -121.647, ep_step: 192 2023-05-30 00:19:14 - SimpleLog - INFO: - episode: 377, ep_reward: -74.400, ep_step: 1000 2023-05-30 00:19:16 - SimpleLog - INFO: - episode: 378, ep_reward: -60.565, ep_step: 1000 2023-05-30 00:19:17 - SimpleLog - INFO: - episode: 379, ep_reward: -106.783, ep_step: 239 2023-05-30 00:19:19 - SimpleLog - INFO: - episode: 380, ep_reward: -62.589, ep_step: 1000 2023-05-30 00:19:21 - SimpleLog - INFO: - episode: 381, ep_reward: -60.968, ep_step: 1000 2023-05-30 00:19:24 - SimpleLog - INFO: - episode: 382, ep_reward: -58.465, ep_step: 1000 2023-05-30 00:19:26 - SimpleLog - INFO: - episode: 383, ep_reward: -64.831, ep_step: 1000 2023-05-30 00:19:28 - SimpleLog - INFO: - episode: 384, ep_reward: -63.519, ep_step: 1000 2023-05-30 00:19:30 - SimpleLog - INFO: - episode: 385, ep_reward: -58.811, ep_step: 1000 2023-05-30 00:19:33 - SimpleLog - INFO: - episode: 386, ep_reward: -59.063, ep_step: 1000 2023-05-30 00:19:35 - SimpleLog - INFO: - episode: 387, ep_reward: -41.526, ep_step: 1000 2023-05-30 00:19:37 - SimpleLog - INFO: - episode: 388, ep_reward: -55.822, ep_step: 1000 2023-05-30 00:19:38 - SimpleLog - INFO: - episode: 389, ep_reward: -108.467, ep_step: 117 2023-05-30 00:19:40 - SimpleLog - INFO: - episode: 390, ep_reward: -55.634, ep_step: 1000 2023-05-30 00:19:42 - SimpleLog - INFO: - episode: 391, ep_reward: -48.361, ep_step: 1000 2023-05-30 00:19:44 - SimpleLog - INFO: - episode: 392, ep_reward: -133.994, ep_step: 727 2023-05-30 00:19:46 - SimpleLog - INFO: - episode: 393, ep_reward: -40.824, ep_step: 1000 2023-05-30 00:19:49 - SimpleLog - INFO: - episode: 394, ep_reward: -41.955, ep_step: 1000 2023-05-30 00:19:51 - SimpleLog - INFO: - episode: 395, ep_reward: -42.978, ep_step: 1000 2023-05-30 00:19:53 - SimpleLog - INFO: - episode: 396, ep_reward: -44.014, ep_step: 1000 2023-05-30 00:19:55 - SimpleLog - INFO: - episode: 397, ep_reward: -42.373, ep_step: 1000 2023-05-30 00:19:58 - SimpleLog - INFO: - episode: 398, ep_reward: -46.095, ep_step: 1000 2023-05-30 00:20:00 - SimpleLog - INFO: - episode: 399, ep_reward: -45.208, ep_step: 1000 2023-05-30 00:20:02 - SimpleLog - INFO: - update_step: 280000, online_eval_reward: -102.544 2023-05-30 00:20:04 - SimpleLog - INFO: - episode: 400, ep_reward: -42.977, ep_step: 1000 2023-05-30 00:20:07 - SimpleLog - INFO: - episode: 401, ep_reward: -95.159, ep_step: 1000 2023-05-30 00:20:09 - SimpleLog - INFO: - episode: 402, ep_reward: -33.435, ep_step: 1000 2023-05-30 00:20:11 - SimpleLog - INFO: - episode: 403, ep_reward: -31.891, ep_step: 1000 2023-05-30 00:20:14 - SimpleLog - INFO: - episode: 404, ep_reward: -29.151, ep_step: 1000 2023-05-30 00:20:16 - SimpleLog - INFO: - episode: 405, ep_reward: -28.749, ep_step: 1000 2023-05-30 00:20:18 - SimpleLog - INFO: - episode: 406, ep_reward: -29.222, ep_step: 1000 2023-05-30 00:20:20 - SimpleLog - INFO: - episode: 407, ep_reward: -31.196, ep_step: 1000 2023-05-30 00:20:23 - SimpleLog - INFO: - episode: 408, ep_reward: -31.967, ep_step: 1000 2023-05-30 00:20:25 - SimpleLog - INFO: - episode: 409, ep_reward: -33.384, ep_step: 1000 2023-05-30 00:20:27 - SimpleLog - INFO: - episode: 410, ep_reward: -34.648, ep_step: 1000 2023-05-30 00:20:30 - SimpleLog - INFO: - episode: 411, ep_reward: -37.848, ep_step: 1000 2023-05-30 00:20:32 - SimpleLog - INFO: - episode: 412, ep_reward: -61.208, ep_step: 1000 2023-05-30 00:20:34 - SimpleLog - INFO: - episode: 413, ep_reward: -57.388, ep_step: 1000 2023-05-30 00:20:36 - SimpleLog - INFO: - episode: 414, ep_reward: -34.543, ep_step: 1000 2023-05-30 00:20:39 - SimpleLog - INFO: - episode: 415, ep_reward: -34.802, ep_step: 1000 2023-05-30 00:20:41 - SimpleLog - INFO: - episode: 416, ep_reward: -34.934, ep_step: 1000 2023-05-30 00:20:41 - SimpleLog - INFO: - episode: 417, ep_reward: -104.043, ep_step: 58 2023-05-30 00:20:43 - SimpleLog - INFO: - episode: 418, ep_reward: -31.920, ep_step: 1000 2023-05-30 00:20:44 - SimpleLog - INFO: - episode: 419, ep_reward: -100.605, ep_step: 81 2023-05-30 00:20:44 - SimpleLog - INFO: - episode: 420, ep_reward: -105.574, ep_step: 89 2023-05-30 00:20:44 - SimpleLog - INFO: - episode: 421, ep_reward: -107.380, ep_step: 78 2023-05-30 00:20:46 - SimpleLog - INFO: - episode: 422, ep_reward: -29.672, ep_step: 1000 2023-05-30 00:20:49 - SimpleLog - INFO: - episode: 423, ep_reward: -35.937, ep_step: 1000 2023-05-30 00:20:49 - SimpleLog - INFO: - episode: 424, ep_reward: -99.817, ep_step: 59 2023-05-30 00:20:49 - SimpleLog - INFO: - episode: 425, ep_reward: -100.371, ep_step: 64 2023-05-30 00:20:51 - SimpleLog - INFO: - episode: 426, ep_reward: -42.066, ep_step: 1000 2023-05-30 00:20:51 - SimpleLog - INFO: - episode: 427, ep_reward: -121.268, ep_step: 71 2023-05-30 00:20:51 - SimpleLog - INFO: - episode: 428, ep_reward: -120.878, ep_step: 73 2023-05-30 00:20:52 - SimpleLog - INFO: - episode: 429, ep_reward: -109.721, ep_step: 135 2023-05-30 00:20:52 - SimpleLog - INFO: - episode: 430, ep_reward: -118.125, ep_step: 58 2023-05-30 00:20:52 - SimpleLog - INFO: - episode: 431, ep_reward: -116.006, ep_step: 153 2023-05-30 00:20:52 - SimpleLog - INFO: - episode: 432, ep_reward: -104.656, ep_step: 62 2023-05-30 00:20:53 - SimpleLog - INFO: - episode: 433, ep_reward: -116.796, ep_step: 51 2023-05-30 00:20:53 - SimpleLog - INFO: - episode: 434, ep_reward: -118.939, ep_step: 62 2023-05-30 00:20:55 - SimpleLog - INFO: - episode: 435, ep_reward: -54.372, ep_step: 1000 2023-05-30 00:20:55 - SimpleLog - INFO: - episode: 436, ep_reward: -101.526, ep_step: 115 2023-05-30 00:20:55 - SimpleLog - INFO: - episode: 437, ep_reward: -123.780, ep_step: 78 2023-05-30 00:20:58 - SimpleLog - INFO: - episode: 438, ep_reward: -61.819, ep_step: 1000 2023-05-30 00:20:58 - SimpleLog - INFO: - episode: 439, ep_reward: -125.481, ep_step: 75 2023-05-30 00:20:58 - SimpleLog - INFO: - episode: 440, ep_reward: -121.256, ep_step: 64 2023-05-30 00:20:58 - SimpleLog - INFO: - episode: 441, ep_reward: -119.802, ep_step: 63 2023-05-30 00:20:58 - SimpleLog - INFO: - episode: 442, ep_reward: -117.526, ep_step: 51 2023-05-30 00:20:59 - SimpleLog - INFO: - episode: 443, ep_reward: -126.934, ep_step: 86 2023-05-30 00:20:59 - SimpleLog - INFO: - episode: 444, ep_reward: -116.020, ep_step: 52 2023-05-30 00:20:59 - SimpleLog - INFO: - episode: 445, ep_reward: -117.387, ep_step: 56 2023-05-30 00:20:59 - SimpleLog - INFO: - episode: 446, ep_reward: -117.699, ep_step: 53 2023-05-30 00:20:59 - SimpleLog - INFO: - episode: 447, ep_reward: -110.929, ep_step: 157 2023-05-30 00:21:02 - SimpleLog - INFO: - episode: 448, ep_reward: -57.712, ep_step: 1000 2023-05-30 00:21:02 - SimpleLog - INFO: - episode: 449, ep_reward: -119.307, ep_step: 62 2023-05-30 00:21:02 - SimpleLog - INFO: - episode: 450, ep_reward: -132.669, ep_step: 155 2023-05-30 00:21:02 - SimpleLog - INFO: - episode: 451, ep_reward: -129.935, ep_step: 128 2023-05-30 00:21:05 - SimpleLog - INFO: - episode: 452, ep_reward: -52.967, ep_step: 1000 2023-05-30 00:21:07 - SimpleLog - INFO: - episode: 453, ep_reward: -38.234, ep_step: 1000 2023-05-30 00:21:09 - SimpleLog - INFO: - episode: 454, ep_reward: -34.212, ep_step: 1000 2023-05-30 00:21:12 - SimpleLog - INFO: - episode: 455, ep_reward: -38.836, ep_step: 1000 2023-05-30 00:21:14 - SimpleLog - INFO: - episode: 456, ep_reward: -38.825, ep_step: 1000 2023-05-30 00:21:16 - SimpleLog - INFO: - episode: 457, ep_reward: -38.093, ep_step: 1000 2023-05-30 00:21:18 - SimpleLog - INFO: - episode: 458, ep_reward: -39.229, ep_step: 1000 2023-05-30 00:21:21 - SimpleLog - INFO: - episode: 459, ep_reward: -38.576, ep_step: 1000 2023-05-30 00:21:23 - SimpleLog - INFO: - episode: 460, ep_reward: -41.699, ep_step: 1000 2023-05-30 00:21:25 - SimpleLog - INFO: - episode: 461, ep_reward: -39.089, ep_step: 1000 2023-05-30 00:21:28 - SimpleLog - INFO: - episode: 462, ep_reward: -37.823, ep_step: 1000 2023-05-30 00:21:30 - SimpleLog - INFO: - episode: 463, ep_reward: -46.631, ep_step: 1000 2023-05-30 00:21:32 - SimpleLog - INFO: - episode: 464, ep_reward: -54.700, ep_step: 1000 2023-05-30 00:21:34 - SimpleLog - INFO: - episode: 465, ep_reward: -51.005, ep_step: 1000 2023-05-30 00:21:37 - SimpleLog - INFO: - episode: 466, ep_reward: -52.798, ep_step: 1000 2023-05-30 00:21:39 - SimpleLog - INFO: - episode: 467, ep_reward: -55.178, ep_step: 1000 2023-05-30 00:21:41 - SimpleLog - INFO: - episode: 468, ep_reward: -60.558, ep_step: 1000 2023-05-30 00:21:43 - SimpleLog - INFO: - episode: 469, ep_reward: -56.942, ep_step: 1000 2023-05-30 00:21:46 - SimpleLog - INFO: - episode: 470, ep_reward: -53.593, ep_step: 1000 2023-05-30 00:21:48 - SimpleLog - INFO: - episode: 471, ep_reward: -42.452, ep_step: 1000 2023-05-30 00:21:50 - SimpleLog - INFO: - episode: 472, ep_reward: -51.297, ep_step: 1000 2023-05-30 00:21:52 - SimpleLog - INFO: - episode: 473, ep_reward: -47.171, ep_step: 1000 2023-05-30 00:21:55 - SimpleLog - INFO: - episode: 474, ep_reward: -47.509, ep_step: 1000 2023-05-30 00:21:57 - SimpleLog - INFO: - episode: 475, ep_reward: -49.103, ep_step: 1000 2023-05-30 00:21:59 - SimpleLog - INFO: - episode: 476, ep_reward: -48.353, ep_step: 1000 2023-05-30 00:22:02 - SimpleLog - INFO: - episode: 477, ep_reward: -49.342, ep_step: 1000 2023-05-30 00:22:04 - SimpleLog - INFO: - episode: 478, ep_reward: -50.554, ep_step: 1000 2023-05-30 00:22:06 - SimpleLog - INFO: - episode: 479, ep_reward: -68.018, ep_step: 1000 2023-05-30 00:22:09 - SimpleLog - INFO: - episode: 480, ep_reward: -72.258, ep_step: 1000 2023-05-30 00:22:11 - SimpleLog - INFO: - episode: 481, ep_reward: -70.648, ep_step: 1000 2023-05-30 00:22:13 - SimpleLog - INFO: - episode: 482, ep_reward: -72.857, ep_step: 1000 2023-05-30 00:22:16 - SimpleLog - INFO: - episode: 483, ep_reward: -76.494, ep_step: 1000 2023-05-30 00:22:18 - SimpleLog - INFO: - episode: 484, ep_reward: -74.990, ep_step: 1000 2023-05-30 00:22:20 - SimpleLog - INFO: - episode: 485, ep_reward: -66.514, ep_step: 1000 2023-05-30 00:22:23 - SimpleLog - INFO: - episode: 486, ep_reward: -62.529, ep_step: 1000 2023-05-30 00:22:25 - SimpleLog - INFO: - episode: 487, ep_reward: -48.896, ep_step: 1000 2023-05-30 00:22:27 - SimpleLog - INFO: - episode: 488, ep_reward: -43.008, ep_step: 1000 2023-05-30 00:22:29 - SimpleLog - INFO: - episode: 489, ep_reward: -38.415, ep_step: 1000 2023-05-30 00:22:32 - SimpleLog - INFO: - episode: 490, ep_reward: -37.853, ep_step: 1000 2023-05-30 00:22:34 - SimpleLog - INFO: - episode: 491, ep_reward: -40.704, ep_step: 1000 2023-05-30 00:22:36 - SimpleLog - INFO: - episode: 492, ep_reward: -31.165, ep_step: 1000 2023-05-30 00:22:39 - SimpleLog - INFO: - episode: 493, ep_reward: -35.692, ep_step: 1000 2023-05-30 00:22:41 - SimpleLog - INFO: - episode: 494, ep_reward: -27.913, ep_step: 1000 2023-05-30 00:22:41 - SimpleLog - INFO: - episode: 495, ep_reward: -96.796, ep_step: 100 2023-05-30 00:22:45 - SimpleLog - INFO: - update_step: 350000, online_eval_reward: -27.177 2023-05-30 00:22:45 - SimpleLog - INFO: - episode: 496, ep_reward: -30.728, ep_step: 1000 2023-05-30 00:22:46 - SimpleLog - INFO: - episode: 497, ep_reward: -90.966, ep_step: 111 2023-05-30 00:22:46 - SimpleLog - INFO: - episode: 498, ep_reward: -98.532, ep_step: 94 2023-05-30 00:22:48 - SimpleLog - INFO: - episode: 499, ep_reward: -35.932, ep_step: 1000 2023-05-30 00:22:50 - SimpleLog - INFO: - episode: 500, ep_reward: -39.748, ep_step: 1000 2023-05-30 00:22:53 - SimpleLog - INFO: - episode: 501, ep_reward: -29.185, ep_step: 1000 2023-05-30 00:22:55 - SimpleLog - INFO: - episode: 502, ep_reward: -28.721, ep_step: 1000 2023-05-30 00:22:57 - SimpleLog - INFO: - episode: 503, ep_reward: -41.033, ep_step: 1000 2023-05-30 00:22:59 - SimpleLog - INFO: - episode: 504, ep_reward: -38.682, ep_step: 1000 2023-05-30 00:23:02 - SimpleLog - INFO: - episode: 505, ep_reward: -31.534, ep_step: 1000 2023-05-30 00:23:04 - SimpleLog - INFO: - episode: 506, ep_reward: -45.957, ep_step: 1000 2023-05-30 00:23:06 - SimpleLog - INFO: - episode: 507, ep_reward: -45.665, ep_step: 1000 2023-05-30 00:23:09 - SimpleLog - INFO: - episode: 508, ep_reward: -45.511, ep_step: 1000 2023-05-30 00:23:11 - SimpleLog - INFO: - episode: 509, ep_reward: -44.997, ep_step: 1000 2023-05-30 00:23:13 - SimpleLog - INFO: - episode: 510, ep_reward: -42.454, ep_step: 1000 2023-05-30 00:23:15 - SimpleLog - INFO: - episode: 511, ep_reward: -36.286, ep_step: 1000 2023-05-30 00:23:18 - SimpleLog - INFO: - episode: 512, ep_reward: -31.438, ep_step: 1000 2023-05-30 00:23:20 - SimpleLog - INFO: - episode: 513, ep_reward: -28.530, ep_step: 1000 2023-05-30 00:23:22 - SimpleLog - INFO: - episode: 514, ep_reward: -39.770, ep_step: 1000 2023-05-30 00:23:24 - SimpleLog - INFO: - episode: 515, ep_reward: -29.535, ep_step: 1000 2023-05-30 00:23:27 - SimpleLog - INFO: - episode: 516, ep_reward: -33.318, ep_step: 1000 2023-05-30 00:23:29 - SimpleLog - INFO: - episode: 517, ep_reward: -37.696, ep_step: 1000 2023-05-30 00:23:31 - SimpleLog - INFO: - episode: 518, ep_reward: -41.769, ep_step: 1000 2023-05-30 00:23:34 - SimpleLog - INFO: - episode: 519, ep_reward: -42.962, ep_step: 1000 2023-05-30 00:23:36 - SimpleLog - INFO: - episode: 520, ep_reward: -42.189, ep_step: 1000 2023-05-30 00:23:36 - SimpleLog - INFO: - episode: 521, ep_reward: -147.243, ep_step: 193 2023-05-30 00:23:39 - SimpleLog - INFO: - episode: 522, ep_reward: -38.936, ep_step: 1000 2023-05-30 00:23:41 - SimpleLog - INFO: - episode: 523, ep_reward: -40.269, ep_step: 1000 2023-05-30 00:23:43 - SimpleLog - INFO: - episode: 524, ep_reward: -34.213, ep_step: 1000 2023-05-30 00:23:43 - SimpleLog - INFO: - episode: 525, ep_reward: -118.246, ep_step: 68 2023-05-30 00:23:45 - SimpleLog - INFO: - episode: 526, ep_reward: -35.751, ep_step: 1000 2023-05-30 00:23:48 - SimpleLog - INFO: - episode: 527, ep_reward: -38.132, ep_step: 1000 2023-05-30 00:23:50 - SimpleLog - INFO: - episode: 528, ep_reward: -41.429, ep_step: 1000 2023-05-30 00:23:52 - SimpleLog - INFO: - episode: 529, ep_reward: -39.317, ep_step: 1000 2023-05-30 00:23:55 - SimpleLog - INFO: - episode: 530, ep_reward: -38.372, ep_step: 1000 2023-05-30 00:23:57 - SimpleLog - INFO: - episode: 531, ep_reward: -41.588, ep_step: 1000 2023-05-30 00:23:59 - SimpleLog - INFO: - episode: 532, ep_reward: -41.937, ep_step: 1000 2023-05-30 00:24:01 - SimpleLog - INFO: - episode: 533, ep_reward: -46.751, ep_step: 1000 2023-05-30 00:24:04 - SimpleLog - INFO: - episode: 534, ep_reward: -34.126, ep_step: 1000 2023-05-30 00:24:04 - SimpleLog - INFO: - episode: 535, ep_reward: -96.705, ep_step: 77 2023-05-30 00:24:06 - SimpleLog - INFO: - episode: 536, ep_reward: -29.952, ep_step: 1000 2023-05-30 00:24:06 - SimpleLog - INFO: - episode: 537, ep_reward: -100.454, ep_step: 75 2023-05-30 00:24:07 - SimpleLog - INFO: - episode: 538, ep_reward: -98.125, ep_step: 90 2023-05-30 00:24:09 - SimpleLog - INFO: - episode: 539, ep_reward: -33.803, ep_step: 1000 2023-05-30 00:24:09 - SimpleLog - INFO: - episode: 540, ep_reward: -95.791, ep_step: 77 2023-05-30 00:24:11 - SimpleLog - INFO: - episode: 541, ep_reward: -27.327, ep_step: 1000 2023-05-30 00:24:14 - SimpleLog - INFO: - episode: 542, ep_reward: -34.958, ep_step: 1000 2023-05-30 00:24:16 - SimpleLog - INFO: - episode: 543, ep_reward: -36.069, ep_step: 1000 2023-05-30 00:24:19 - SimpleLog - INFO: - episode: 544, ep_reward: -32.683, ep_step: 1000 2023-05-30 00:24:21 - SimpleLog - INFO: - episode: 545, ep_reward: -33.722, ep_step: 1000 2023-05-30 00:24:23 - SimpleLog - INFO: - episode: 546, ep_reward: -33.134, ep_step: 1000 2023-05-30 00:24:26 - SimpleLog - INFO: - episode: 547, ep_reward: -33.120, ep_step: 1000 2023-05-30 00:24:28 - SimpleLog - INFO: - episode: 548, ep_reward: -27.584, ep_step: 1000 2023-05-30 00:24:30 - SimpleLog - INFO: - episode: 549, ep_reward: -32.223, ep_step: 1000 2023-05-30 00:24:33 - SimpleLog - INFO: - episode: 550, ep_reward: -30.153, ep_step: 1000 2023-05-30 00:24:35 - SimpleLog - INFO: - episode: 551, ep_reward: -29.409, ep_step: 1000 2023-05-30 00:24:37 - SimpleLog - INFO: - episode: 552, ep_reward: -27.841, ep_step: 1000 2023-05-30 00:24:40 - SimpleLog - INFO: - episode: 553, ep_reward: -29.602, ep_step: 1000 2023-05-30 00:24:42 - SimpleLog - INFO: - episode: 554, ep_reward: -33.136, ep_step: 1000 2023-05-30 00:24:44 - SimpleLog - INFO: - episode: 555, ep_reward: -31.964, ep_step: 1000 2023-05-30 00:24:47 - SimpleLog - INFO: - episode: 556, ep_reward: -30.280, ep_step: 1000 2023-05-30 00:24:49 - SimpleLog - INFO: - episode: 557, ep_reward: -32.463, ep_step: 1000 2023-05-30 00:24:51 - SimpleLog - INFO: - episode: 558, ep_reward: -30.861, ep_step: 1000 2023-05-30 00:24:54 - SimpleLog - INFO: - episode: 559, ep_reward: -31.246, ep_step: 1000 2023-05-30 00:24:56 - SimpleLog - INFO: - episode: 560, ep_reward: -32.823, ep_step: 1000 2023-05-30 00:24:58 - SimpleLog - INFO: - episode: 561, ep_reward: -31.243, ep_step: 1000 2023-05-30 00:25:01 - SimpleLog - INFO: - episode: 562, ep_reward: -30.165, ep_step: 1000 2023-05-30 00:25:03 - SimpleLog - INFO: - episode: 563, ep_reward: -29.693, ep_step: 1000 2023-05-30 00:25:05 - SimpleLog - INFO: - episode: 564, ep_reward: -31.091, ep_step: 1000 2023-05-30 00:25:08 - SimpleLog - INFO: - episode: 565, ep_reward: -28.433, ep_step: 1000 2023-05-30 00:25:10 - SimpleLog - INFO: - episode: 566, ep_reward: -29.701, ep_step: 1000 2023-05-30 00:25:12 - SimpleLog - INFO: - episode: 567, ep_reward: -26.792, ep_step: 1000 2023-05-30 00:25:15 - SimpleLog - INFO: - episode: 568, ep_reward: -25.638, ep_step: 1000 2023-05-30 00:25:17 - SimpleLog - INFO: - episode: 569, ep_reward: -27.137, ep_step: 1000 2023-05-30 00:25:19 - SimpleLog - INFO: - episode: 570, ep_reward: -24.801, ep_step: 1000 2023-05-30 00:25:22 - SimpleLog - INFO: - episode: 571, ep_reward: -30.310, ep_step: 1000 2023-05-30 00:25:24 - SimpleLog - INFO: - episode: 572, ep_reward: -35.086, ep_step: 1000 2023-05-30 00:25:29 - SimpleLog - INFO: - update_step: 420000, online_eval_reward: -21.593 2023-05-30 00:25:29 - SimpleLog - INFO: - episode: 573, ep_reward: -34.457, ep_step: 1000 2023-05-30 00:25:31 - SimpleLog - INFO: - episode: 574, ep_reward: -29.525, ep_step: 1000 2023-05-30 00:25:33 - SimpleLog - INFO: - episode: 575, ep_reward: -26.447, ep_step: 1000 2023-05-30 00:25:35 - SimpleLog - INFO: - episode: 576, ep_reward: -23.896, ep_step: 1000 2023-05-30 00:25:38 - SimpleLog - INFO: - episode: 577, ep_reward: -34.023, ep_step: 1000 2023-05-30 00:25:40 - SimpleLog - INFO: - episode: 578, ep_reward: -26.390, ep_step: 1000 2023-05-30 00:25:40 - SimpleLog - INFO: - episode: 579, ep_reward: -130.755, ep_step: 173 2023-05-30 00:25:43 - SimpleLog - INFO: - episode: 580, ep_reward: -35.398, ep_step: 1000 2023-05-30 00:25:45 - SimpleLog - INFO: - episode: 581, ep_reward: -26.830, ep_step: 1000 2023-05-30 00:25:47 - SimpleLog - INFO: - episode: 582, ep_reward: -34.580, ep_step: 1000 2023-05-30 00:25:50 - SimpleLog - INFO: - episode: 583, ep_reward: -38.710, ep_step: 1000 2023-05-30 00:25:52 - SimpleLog - INFO: - episode: 584, ep_reward: -23.131, ep_step: 1000 2023-05-30 00:25:54 - SimpleLog - INFO: - episode: 585, ep_reward: -35.131, ep_step: 1000 2023-05-30 00:25:56 - SimpleLog - INFO: - episode: 586, ep_reward: -34.034, ep_step: 1000 2023-05-30 00:25:59 - SimpleLog - INFO: - episode: 587, ep_reward: -53.190, ep_step: 1000 2023-05-30 00:26:01 - SimpleLog - INFO: - episode: 588, ep_reward: -39.379, ep_step: 1000 2023-05-30 00:26:06 - SimpleLog - INFO: - episode: 589, ep_reward: -34.350, ep_step: 1000 2023-05-30 00:26:08 - SimpleLog - INFO: - episode: 590, ep_reward: -31.579, ep_step: 1000 2023-05-30 00:26:10 - SimpleLog - INFO: - episode: 591, ep_reward: -73.161, ep_step: 1000 2023-05-30 00:26:13 - SimpleLog - INFO: - episode: 592, ep_reward: -68.964, ep_step: 1000 2023-05-30 00:26:15 - SimpleLog - INFO: - episode: 593, ep_reward: -41.520, ep_step: 1000 2023-05-30 00:26:17 - SimpleLog - INFO: - episode: 594, ep_reward: -36.335, ep_step: 1000 2023-05-30 00:26:20 - SimpleLog - INFO: - episode: 595, ep_reward: -28.615, ep_step: 1000 2023-05-30 00:26:22 - SimpleLog - INFO: - episode: 596, ep_reward: -41.941, ep_step: 1000 2023-05-30 00:26:24 - SimpleLog - INFO: - episode: 597, ep_reward: -17.133, ep_step: 1000 2023-05-30 00:26:26 - SimpleLog - INFO: - episode: 598, ep_reward: -28.352, ep_step: 1000 2023-05-30 00:26:29 - SimpleLog - INFO: - episode: 599, ep_reward: -26.289, ep_step: 1000 2023-05-30 00:26:31 - SimpleLog - INFO: - episode: 600, ep_reward: -27.025, ep_step: 1000 2023-05-30 00:26:33 - SimpleLog - INFO: - episode: 601, ep_reward: -12.538, ep_step: 1000 2023-05-30 00:26:36 - SimpleLog - INFO: - episode: 602, ep_reward: -28.345, ep_step: 1000 2023-05-30 00:26:38 - SimpleLog - INFO: - episode: 603, ep_reward: -24.632, ep_step: 1000 2023-05-30 00:26:40 - SimpleLog - INFO: - episode: 604, ep_reward: -29.229, ep_step: 1000 2023-05-30 00:26:43 - SimpleLog - INFO: - episode: 605, ep_reward: -26.798, ep_step: 1000 2023-05-30 00:26:45 - SimpleLog - INFO: - episode: 606, ep_reward: -27.735, ep_step: 1000 2023-05-30 00:26:47 - SimpleLog - INFO: - episode: 607, ep_reward: -27.223, ep_step: 1000 2023-05-30 00:26:47 - SimpleLog - INFO: - episode: 608, ep_reward: -107.559, ep_step: 46 2023-05-30 00:26:50 - SimpleLog - INFO: - episode: 609, ep_reward: -27.603, ep_step: 1000 2023-05-30 00:26:52 - SimpleLog - INFO: - episode: 610, ep_reward: -26.152, ep_step: 1000 2023-05-30 00:26:54 - SimpleLog - INFO: - episode: 611, ep_reward: -23.348, ep_step: 1000 2023-05-30 00:26:56 - SimpleLog - INFO: - episode: 612, ep_reward: -25.269, ep_step: 1000 2023-05-30 00:26:59 - SimpleLog - INFO: - episode: 613, ep_reward: -25.686, ep_step: 1000 2023-05-30 00:27:01 - SimpleLog - INFO: - episode: 614, ep_reward: -27.834, ep_step: 1000 2023-05-30 00:27:03 - SimpleLog - INFO: - episode: 615, ep_reward: -24.519, ep_step: 1000 2023-05-30 00:27:06 - SimpleLog - INFO: - episode: 616, ep_reward: -25.361, ep_step: 1000 2023-05-30 00:27:08 - SimpleLog - INFO: - episode: 617, ep_reward: -25.407, ep_step: 1000 2023-05-30 00:27:11 - SimpleLog - INFO: - episode: 618, ep_reward: -27.178, ep_step: 1000 2023-05-30 00:27:13 - SimpleLog - INFO: - episode: 619, ep_reward: -24.991, ep_step: 1000 2023-05-30 00:27:15 - SimpleLog - INFO: - episode: 620, ep_reward: -25.899, ep_step: 1000 2023-05-30 00:27:18 - SimpleLog - INFO: - episode: 621, ep_reward: -21.733, ep_step: 1000 2023-05-30 00:27:20 - SimpleLog - INFO: - episode: 622, ep_reward: -21.251, ep_step: 1000 2023-05-30 00:27:22 - SimpleLog - INFO: - episode: 623, ep_reward: -21.753, ep_step: 1000 2023-05-30 00:27:25 - SimpleLog - INFO: - episode: 624, ep_reward: -23.147, ep_step: 1000 2023-05-30 00:27:26 - SimpleLog - INFO: - episode: 625, ep_reward: -102.569, ep_step: 329 2023-05-30 00:27:28 - SimpleLog - INFO: - episode: 626, ep_reward: -20.633, ep_step: 1000 2023-05-30 00:27:30 - SimpleLog - INFO: - episode: 627, ep_reward: -12.929, ep_step: 1000 2023-05-30 00:27:32 - SimpleLog - INFO: - episode: 628, ep_reward: -22.982, ep_step: 1000 2023-05-30 00:27:35 - SimpleLog - INFO: - episode: 629, ep_reward: -12.551, ep_step: 1000 2023-05-30 00:27:37 - SimpleLog - INFO: - episode: 630, ep_reward: -22.625, ep_step: 1000 2023-05-30 00:27:39 - SimpleLog - INFO: - episode: 631, ep_reward: -26.188, ep_step: 1000 2023-05-30 00:27:42 - SimpleLog - INFO: - episode: 632, ep_reward: -21.803, ep_step: 1000 2023-05-30 00:27:44 - SimpleLog - INFO: - episode: 633, ep_reward: -21.801, ep_step: 1000 2023-05-30 00:27:46 - SimpleLog - INFO: - episode: 634, ep_reward: -26.925, ep_step: 1000 2023-05-30 00:27:48 - SimpleLog - INFO: - episode: 635, ep_reward: -24.485, ep_step: 1000 2023-05-30 00:27:51 - SimpleLog - INFO: - episode: 636, ep_reward: -24.170, ep_step: 1000 2023-05-30 00:27:53 - SimpleLog - INFO: - episode: 637, ep_reward: -22.092, ep_step: 1000 2023-05-30 00:27:55 - SimpleLog - INFO: - episode: 638, ep_reward: -24.802, ep_step: 1000 2023-05-30 00:27:57 - SimpleLog - INFO: - episode: 639, ep_reward: -23.219, ep_step: 1000 2023-05-30 00:28:00 - SimpleLog - INFO: - episode: 640, ep_reward: -19.640, ep_step: 1000 2023-05-30 00:28:02 - SimpleLog - INFO: - episode: 641, ep_reward: -21.346, ep_step: 1000 2023-05-30 00:28:04 - SimpleLog - INFO: - episode: 642, ep_reward: -18.837, ep_step: 1000 2023-05-30 00:28:07 - SimpleLog - INFO: - episode: 643, ep_reward: -19.843, ep_step: 1000 2023-05-30 00:28:09 - SimpleLog - INFO: - episode: 644, ep_reward: -21.073, ep_step: 1000 2023-05-30 00:28:11 - SimpleLog - INFO: - episode: 645, ep_reward: -13.798, ep_step: 1000 2023-05-30 00:28:14 - SimpleLog - INFO: - update_step: 490000, online_eval_reward: -5.597 2023-05-30 00:28:14 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: -5.597, save the best model! 2023-05-30 00:28:15 - SimpleLog - INFO: - episode: 646, ep_reward: -22.229, ep_step: 1000 2023-05-30 00:28:18 - SimpleLog - INFO: - episode: 647, ep_reward: -24.759, ep_step: 1000 2023-05-30 00:28:20 - SimpleLog - INFO: - episode: 648, ep_reward: -23.260, ep_step: 1000 2023-05-30 00:28:22 - SimpleLog - INFO: - episode: 649, ep_reward: -22.544, ep_step: 1000 2023-05-30 00:28:25 - SimpleLog - INFO: - episode: 650, ep_reward: -24.611, ep_step: 1000 2023-05-30 00:28:27 - SimpleLog - INFO: - episode: 651, ep_reward: -32.788, ep_step: 1000 2023-05-30 00:28:27 - SimpleLog - INFO: - episode: 652, ep_reward: -90.398, ep_step: 166 2023-05-30 00:28:28 - SimpleLog - INFO: - episode: 653, ep_reward: -88.731, ep_step: 258 2023-05-30 00:28:28 - SimpleLog - INFO: - episode: 654, ep_reward: -107.323, ep_step: 59 2023-05-30 00:28:30 - SimpleLog - INFO: - episode: 655, ep_reward: -22.738, ep_step: 1000 2023-05-30 00:28:30 - SimpleLog - INFO: - episode: 656, ep_reward: -106.151, ep_step: 53 2023-05-30 00:28:31 - SimpleLog - INFO: - episode: 657, ep_reward: -107.718, ep_step: 50 2023-05-30 00:28:31 - SimpleLog - INFO: - episode: 658, ep_reward: -106.311, ep_step: 52 2023-05-30 00:28:31 - SimpleLog - INFO: - episode: 659, ep_reward: -106.247, ep_step: 52 2023-05-30 00:28:31 - SimpleLog - INFO: - episode: 660, ep_reward: -102.310, ep_step: 45 2023-05-30 00:28:31 - SimpleLog - INFO: - episode: 661, ep_reward: -100.481, ep_step: 70 2023-05-30 00:28:31 - SimpleLog - INFO: - episode: 662, ep_reward: -98.326, ep_step: 74 2023-05-30 00:28:31 - SimpleLog - INFO: - episode: 663, ep_reward: -99.464, ep_step: 68 2023-05-30 00:28:32 - SimpleLog - INFO: - episode: 664, ep_reward: -102.651, ep_step: 79 2023-05-30 00:28:32 - SimpleLog - INFO: - episode: 665, ep_reward: -101.877, ep_step: 51 2023-05-30 00:28:32 - SimpleLog - INFO: - episode: 666, ep_reward: -99.904, ep_step: 65 2023-05-30 00:28:32 - SimpleLog - INFO: - episode: 667, ep_reward: -100.876, ep_step: 57 2023-05-30 00:28:32 - SimpleLog - INFO: - episode: 668, ep_reward: -101.189, ep_step: 68 2023-05-30 00:28:32 - SimpleLog - INFO: - episode: 669, ep_reward: -100.583, ep_step: 69 2023-05-30 00:28:32 - SimpleLog - INFO: - episode: 670, ep_reward: -100.270, ep_step: 67 2023-05-30 00:28:33 - SimpleLog - INFO: - episode: 671, ep_reward: -101.173, ep_step: 66 2023-05-30 00:28:33 - SimpleLog - INFO: - episode: 672, ep_reward: -97.478, ep_step: 74 2023-05-30 00:28:33 - SimpleLog - INFO: - episode: 673, ep_reward: -100.578, ep_step: 64 2023-05-30 00:28:33 - SimpleLog - INFO: - episode: 674, ep_reward: -100.747, ep_step: 63 2023-05-30 00:28:33 - SimpleLog - INFO: - episode: 675, ep_reward: -101.932, ep_step: 40 2023-05-30 00:28:33 - SimpleLog - INFO: - episode: 676, ep_reward: -100.796, ep_step: 50 2023-05-30 00:28:33 - SimpleLog - INFO: - episode: 677, ep_reward: -98.129, ep_step: 68 2023-05-30 00:28:34 - SimpleLog - INFO: - episode: 678, ep_reward: -98.767, ep_step: 85 2023-05-30 00:28:34 - SimpleLog - INFO: - episode: 679, ep_reward: -101.053, ep_step: 66 2023-05-30 00:28:34 - SimpleLog - INFO: - episode: 680, ep_reward: -99.286, ep_step: 56 2023-05-30 00:28:34 - SimpleLog - INFO: - episode: 681, ep_reward: -100.750, ep_step: 57 2023-05-30 00:28:34 - SimpleLog - INFO: - episode: 682, ep_reward: -101.811, ep_step: 45 2023-05-30 00:28:34 - SimpleLog - INFO: - episode: 683, ep_reward: -102.020, ep_step: 50 2023-05-30 00:28:34 - SimpleLog - INFO: - episode: 684, ep_reward: -103.258, ep_step: 47 2023-05-30 00:28:34 - SimpleLog - INFO: - episode: 685, ep_reward: -101.301, ep_step: 47 2023-05-30 00:28:35 - SimpleLog - INFO: - episode: 686, ep_reward: -101.354, ep_step: 57 2023-05-30 00:28:35 - SimpleLog - INFO: - episode: 687, ep_reward: -101.246, ep_step: 60 2023-05-30 00:28:35 - SimpleLog - INFO: - episode: 688, ep_reward: -100.849, ep_step: 59 2023-05-30 00:28:35 - SimpleLog - INFO: - episode: 689, ep_reward: -103.237, ep_step: 51 2023-05-30 00:28:35 - SimpleLog - INFO: - episode: 690, ep_reward: -101.232, ep_step: 56 2023-05-30 00:28:35 - SimpleLog - INFO: - episode: 691, ep_reward: -102.046, ep_step: 54 2023-05-30 00:28:35 - SimpleLog - INFO: - episode: 692, ep_reward: -101.940, ep_step: 54 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 693, ep_reward: -101.736, ep_step: 53 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 694, ep_reward: -103.000, ep_step: 49 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 695, ep_reward: -102.688, ep_step: 43 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 696, ep_reward: -101.794, ep_step: 50 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 697, ep_reward: -100.907, ep_step: 53 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 698, ep_reward: -101.237, ep_step: 56 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 699, ep_reward: -101.976, ep_step: 51 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 700, ep_reward: -102.691, ep_step: 47 2023-05-30 00:28:36 - SimpleLog - INFO: - episode: 701, ep_reward: -102.017, ep_step: 46 2023-05-30 00:28:37 - SimpleLog - INFO: - episode: 702, ep_reward: -101.484, ep_step: 58 2023-05-30 00:28:37 - SimpleLog - INFO: - episode: 703, ep_reward: -101.622, ep_step: 48 2023-05-30 00:28:37 - SimpleLog - INFO: - episode: 704, ep_reward: -102.817, ep_step: 49 2023-05-30 00:28:37 - SimpleLog - INFO: - episode: 705, ep_reward: -102.287, ep_step: 50 2023-05-30 00:28:37 - SimpleLog - INFO: - episode: 706, ep_reward: -103.099, ep_step: 43 2023-05-30 00:28:37 - SimpleLog - INFO: - episode: 707, ep_reward: -102.357, ep_step: 51 2023-05-30 00:28:37 - SimpleLog - INFO: - episode: 708, ep_reward: -102.327, ep_step: 45 2023-05-30 00:28:37 - SimpleLog - INFO: - episode: 709, ep_reward: -101.291, ep_step: 58 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 710, ep_reward: -100.951, ep_step: 56 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 711, ep_reward: -100.696, ep_step: 56 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 712, ep_reward: -101.409, ep_step: 54 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 713, ep_reward: -100.886, ep_step: 56 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 714, ep_reward: -100.925, ep_step: 53 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 715, ep_reward: -101.822, ep_step: 54 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 716, ep_reward: -103.199, ep_step: 44 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 717, ep_reward: -100.717, ep_step: 57 2023-05-30 00:28:38 - SimpleLog - INFO: - episode: 718, ep_reward: -101.311, ep_step: 49 2023-05-30 00:28:39 - SimpleLog - INFO: - episode: 719, ep_reward: -101.935, ep_step: 54 2023-05-30 00:28:39 - SimpleLog - INFO: - episode: 720, ep_reward: -101.723, ep_step: 57 2023-05-30 00:28:39 - SimpleLog - INFO: - episode: 721, ep_reward: -101.715, ep_step: 54 2023-05-30 00:28:39 - SimpleLog - INFO: - episode: 722, ep_reward: -102.182, ep_step: 49 2023-05-30 00:28:39 - SimpleLog - INFO: - episode: 723, ep_reward: -102.563, ep_step: 44 2023-05-30 00:28:39 - SimpleLog - INFO: - episode: 724, ep_reward: -102.536, ep_step: 57 2023-05-30 00:28:39 - SimpleLog - INFO: - episode: 725, ep_reward: -102.315, ep_step: 46 2023-05-30 00:28:39 - SimpleLog - INFO: - episode: 726, ep_reward: -102.639, ep_step: 45 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 727, ep_reward: -103.047, ep_step: 52 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 728, ep_reward: -102.284, ep_step: 53 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 729, ep_reward: -102.668, ep_step: 48 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 730, ep_reward: -102.004, ep_step: 50 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 731, ep_reward: -103.037, ep_step: 45 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 732, ep_reward: -102.245, ep_step: 51 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 733, ep_reward: -102.758, ep_step: 49 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 734, ep_reward: -102.456, ep_step: 53 2023-05-30 00:28:40 - SimpleLog - INFO: - episode: 735, ep_reward: -102.280, ep_step: 46 2023-05-30 00:28:41 - SimpleLog - INFO: - episode: 736, ep_reward: -102.031, ep_step: 50 2023-05-30 00:28:41 - SimpleLog - INFO: - episode: 737, ep_reward: -101.145, ep_step: 57 2023-05-30 00:28:41 - SimpleLog - INFO: - episode: 738, ep_reward: -102.221, ep_step: 46 2023-05-30 00:28:41 - SimpleLog - INFO: - episode: 739, ep_reward: -101.251, ep_step: 63 2023-05-30 00:28:41 - SimpleLog - INFO: - episode: 740, ep_reward: -101.705, ep_step: 55 2023-05-30 00:28:41 - SimpleLog - INFO: - episode: 741, ep_reward: -102.032, ep_step: 47 2023-05-30 00:28:41 - SimpleLog - INFO: - episode: 742, ep_reward: -100.639, ep_step: 57 2023-05-30 00:28:41 - SimpleLog - INFO: - episode: 743, ep_reward: -101.607, ep_step: 48 2023-05-30 00:28:42 - SimpleLog - INFO: - episode: 744, ep_reward: -102.457, ep_step: 44 2023-05-30 00:28:42 - SimpleLog - INFO: - episode: 745, ep_reward: -99.480, ep_step: 71 2023-05-30 00:28:42 - SimpleLog - INFO: - episode: 746, ep_reward: -101.998, ep_step: 55 2023-05-30 00:28:42 - SimpleLog - INFO: - episode: 747, ep_reward: -99.953, ep_step: 59 2023-05-30 00:28:42 - SimpleLog - INFO: - episode: 748, ep_reward: -99.346, ep_step: 82 2023-05-30 00:28:42 - SimpleLog - INFO: - episode: 749, ep_reward: -98.628, ep_step: 71 2023-05-30 00:28:42 - SimpleLog - INFO: - episode: 750, ep_reward: -102.721, ep_step: 63 2023-05-30 00:28:43 - SimpleLog - INFO: - episode: 751, ep_reward: -101.589, ep_step: 54 2023-05-30 00:28:43 - SimpleLog - INFO: - episode: 752, ep_reward: -99.970, ep_step: 67 2023-05-30 00:28:43 - SimpleLog - INFO: - episode: 753, ep_reward: -99.595, ep_step: 60 2023-05-30 00:28:43 - SimpleLog - INFO: - episode: 754, ep_reward: -103.683, ep_step: 62 2023-05-30 00:28:43 - SimpleLog - INFO: - episode: 755, ep_reward: -102.227, ep_step: 68 2023-05-30 00:28:43 - SimpleLog - INFO: - episode: 756, ep_reward: -101.360, ep_step: 69 2023-05-30 00:28:44 - SimpleLog - INFO: - episode: 757, ep_reward: -105.574, ep_step: 61 2023-05-30 00:28:44 - SimpleLog - INFO: - episode: 758, ep_reward: -100.592, ep_step: 66 2023-05-30 00:28:44 - SimpleLog - INFO: - episode: 759, ep_reward: -99.397, ep_step: 73 2023-05-30 00:28:44 - SimpleLog - INFO: - episode: 760, ep_reward: -101.894, ep_step: 60 2023-05-30 00:28:44 - SimpleLog - INFO: - episode: 761, ep_reward: -102.278, ep_step: 60 2023-05-30 00:28:44 - SimpleLog - INFO: - episode: 762, ep_reward: -102.967, ep_step: 52 2023-05-30 00:28:44 - SimpleLog - INFO: - episode: 763, ep_reward: -100.576, ep_step: 61 2023-05-30 00:28:45 - SimpleLog - INFO: - episode: 764, ep_reward: -100.126, ep_step: 63 2023-05-30 00:28:45 - SimpleLog - INFO: - episode: 765, ep_reward: -103.011, ep_step: 55 2023-05-30 00:28:45 - SimpleLog - INFO: - episode: 766, ep_reward: -102.666, ep_step: 62 2023-05-30 00:28:45 - SimpleLog - INFO: - episode: 767, ep_reward: -101.848, ep_step: 54 2023-05-30 00:28:45 - SimpleLog - INFO: - episode: 768, ep_reward: -102.000, ep_step: 55 2023-05-30 00:28:45 - SimpleLog - INFO: - episode: 769, ep_reward: -103.652, ep_step: 63 2023-05-30 00:28:45 - SimpleLog - INFO: - episode: 770, ep_reward: -103.320, ep_step: 55 2023-05-30 00:28:46 - SimpleLog - INFO: - episode: 771, ep_reward: -96.412, ep_step: 77 2023-05-30 00:28:46 - SimpleLog - INFO: - episode: 772, ep_reward: -102.886, ep_step: 59 2023-05-30 00:28:46 - SimpleLog - INFO: - episode: 773, ep_reward: -98.226, ep_step: 72 2023-05-30 00:28:46 - SimpleLog - INFO: - episode: 774, ep_reward: -101.779, ep_step: 61 2023-05-30 00:28:46 - SimpleLog - INFO: - episode: 775, ep_reward: -99.752, ep_step: 67 2023-05-30 00:28:46 - SimpleLog - INFO: - episode: 776, ep_reward: -101.291, ep_step: 60 2023-05-30 00:28:46 - SimpleLog - INFO: - episode: 777, ep_reward: -96.643, ep_step: 76 2023-05-30 00:28:47 - SimpleLog - INFO: - episode: 778, ep_reward: -97.839, ep_step: 69 2023-05-30 00:28:47 - SimpleLog - INFO: - episode: 779, ep_reward: -98.106, ep_step: 72 2023-05-30 00:28:47 - SimpleLog - INFO: - episode: 780, ep_reward: -95.081, ep_step: 98 2023-05-30 00:28:47 - SimpleLog - INFO: - episode: 781, ep_reward: -98.014, ep_step: 72 2023-05-30 00:28:47 - SimpleLog - INFO: - episode: 782, ep_reward: -101.352, ep_step: 58 2023-05-30 00:28:47 - SimpleLog - INFO: - episode: 783, ep_reward: -98.156, ep_step: 75 2023-05-30 00:28:48 - SimpleLog - INFO: - episode: 784, ep_reward: -98.510, ep_step: 69 2023-05-30 00:28:48 - SimpleLog - INFO: - episode: 785, ep_reward: -101.104, ep_step: 61 2023-05-30 00:28:48 - SimpleLog - INFO: - episode: 786, ep_reward: -100.121, ep_step: 66 2023-05-30 00:28:48 - SimpleLog - INFO: - episode: 787, ep_reward: -99.096, ep_step: 69 2023-05-30 00:28:48 - SimpleLog - INFO: - episode: 788, ep_reward: -99.086, ep_step: 65 2023-05-30 00:28:48 - SimpleLog - INFO: - episode: 789, ep_reward: -100.589, ep_step: 63 2023-05-30 00:28:49 - SimpleLog - INFO: - episode: 790, ep_reward: -100.395, ep_step: 65 2023-05-30 00:28:49 - SimpleLog - INFO: - episode: 791, ep_reward: -101.469, ep_step: 64 2023-05-30 00:28:49 - SimpleLog - INFO: - episode: 792, ep_reward: -101.193, ep_step: 73 2023-05-30 00:28:49 - SimpleLog - INFO: - episode: 793, ep_reward: -99.618, ep_step: 65 2023-05-30 00:28:49 - SimpleLog - INFO: - episode: 794, ep_reward: -99.933, ep_step: 68 2023-05-30 00:28:49 - SimpleLog - INFO: - episode: 795, ep_reward: -97.562, ep_step: 73 2023-05-30 00:28:49 - SimpleLog - INFO: - episode: 796, ep_reward: -99.910, ep_step: 57 2023-05-30 00:28:50 - SimpleLog - INFO: - episode: 797, ep_reward: -97.059, ep_step: 75 2023-05-30 00:28:50 - SimpleLog - INFO: - episode: 798, ep_reward: -99.796, ep_step: 60 2023-05-30 00:28:50 - SimpleLog - INFO: - episode: 799, ep_reward: -99.966, ep_step: 71 2023-05-30 00:28:50 - SimpleLog - INFO: - episode: 800, ep_reward: -99.854, ep_step: 60 2023-05-30 00:28:50 - SimpleLog - INFO: - episode: 801, ep_reward: -99.711, ep_step: 63 2023-05-30 00:28:50 - SimpleLog - INFO: - episode: 802, ep_reward: -101.833, ep_step: 58 2023-05-30 00:28:50 - SimpleLog - INFO: - episode: 803, ep_reward: -100.878, ep_step: 65 2023-05-30 00:28:51 - SimpleLog - INFO: - episode: 804, ep_reward: -95.724, ep_step: 79 2023-05-30 00:28:51 - SimpleLog - INFO: - episode: 805, ep_reward: -97.530, ep_step: 80 2023-05-30 00:28:51 - SimpleLog - INFO: - episode: 806, ep_reward: -99.038, ep_step: 66 2023-05-30 00:28:51 - SimpleLog - INFO: - episode: 807, ep_reward: -97.154, ep_step: 86 2023-05-30 00:28:51 - SimpleLog - INFO: - episode: 808, ep_reward: -97.814, ep_step: 68 2023-05-30 00:28:52 - SimpleLog - INFO: - episode: 809, ep_reward: -97.060, ep_step: 67 2023-05-30 00:28:52 - SimpleLog - INFO: - episode: 810, ep_reward: -98.344, ep_step: 65 2023-05-30 00:28:52 - SimpleLog - INFO: - episode: 811, ep_reward: -98.433, ep_step: 64 2023-05-30 00:28:52 - SimpleLog - INFO: - episode: 812, ep_reward: -97.832, ep_step: 70 2023-05-30 00:28:52 - SimpleLog - INFO: - episode: 813, ep_reward: -99.509, ep_step: 77 2023-05-30 00:28:52 - SimpleLog - INFO: - episode: 814, ep_reward: -101.139, ep_step: 82 2023-05-30 00:28:53 - SimpleLog - INFO: - episode: 815, ep_reward: -99.506, ep_step: 76 2023-05-30 00:28:53 - SimpleLog - INFO: - episode: 816, ep_reward: -99.844, ep_step: 63 2023-05-30 00:28:53 - SimpleLog - INFO: - episode: 817, ep_reward: -101.619, ep_step: 82 2023-05-30 00:28:53 - SimpleLog - INFO: - episode: 818, ep_reward: -97.712, ep_step: 77 2023-05-30 00:28:53 - SimpleLog - INFO: - episode: 819, ep_reward: -99.781, ep_step: 96 2023-05-30 00:28:53 - SimpleLog - INFO: - episode: 820, ep_reward: -100.543, ep_step: 78 2023-05-30 00:28:54 - SimpleLog - INFO: - episode: 821, ep_reward: -98.359, ep_step: 77 2023-05-30 00:28:54 - SimpleLog - INFO: - episode: 822, ep_reward: -97.525, ep_step: 79 2023-05-30 00:28:54 - SimpleLog - INFO: - episode: 823, ep_reward: -97.541, ep_step: 79 2023-05-30 00:28:54 - SimpleLog - INFO: - episode: 824, ep_reward: -98.779, ep_step: 86 2023-05-30 00:28:54 - SimpleLog - INFO: - episode: 825, ep_reward: -99.669, ep_step: 67 2023-05-30 00:28:55 - SimpleLog - INFO: - episode: 826, ep_reward: -97.655, ep_step: 85 2023-05-30 00:28:55 - SimpleLog - INFO: - episode: 827, ep_reward: -99.931, ep_step: 61 2023-05-30 00:28:55 - SimpleLog - INFO: - episode: 828, ep_reward: -98.279, ep_step: 71 2023-05-30 00:28:55 - SimpleLog - INFO: - episode: 829, ep_reward: -96.916, ep_step: 78 2023-05-30 00:28:55 - SimpleLog - INFO: - episode: 830, ep_reward: -97.035, ep_step: 89 2023-05-30 00:28:55 - SimpleLog - INFO: - episode: 831, ep_reward: -97.772, ep_step: 71 2023-05-30 00:28:56 - SimpleLog - INFO: - episode: 832, ep_reward: -97.043, ep_step: 83 2023-05-30 00:28:56 - SimpleLog - INFO: - episode: 833, ep_reward: -99.161, ep_step: 61 2023-05-30 00:28:56 - SimpleLog - INFO: - episode: 834, ep_reward: -97.182, ep_step: 106 2023-05-30 00:28:56 - SimpleLog - INFO: - episode: 835, ep_reward: -98.507, ep_step: 99 2023-05-30 00:28:56 - SimpleLog - INFO: - episode: 836, ep_reward: -98.645, ep_step: 92 2023-05-30 00:28:57 - SimpleLog - INFO: - episode: 837, ep_reward: -98.930, ep_step: 96 2023-05-30 00:28:57 - SimpleLog - INFO: - episode: 838, ep_reward: -98.506, ep_step: 103 2023-05-30 00:28:57 - SimpleLog - INFO: - episode: 839, ep_reward: -95.791, ep_step: 89 2023-05-30 00:28:58 - SimpleLog - INFO: - episode: 840, ep_reward: -99.498, ep_step: 339 2023-05-30 00:28:59 - SimpleLog - INFO: - episode: 841, ep_reward: -117.219, ep_step: 434 2023-05-30 00:28:59 - SimpleLog - INFO: - episode: 842, ep_reward: -104.806, ep_step: 252 2023-05-30 00:29:01 - SimpleLog - INFO: - episode: 843, ep_reward: -118.706, ep_step: 536 2023-05-30 00:29:02 - SimpleLog - INFO: - episode: 844, ep_reward: -106.812, ep_step: 417 2023-05-30 00:29:02 - SimpleLog - INFO: - episode: 845, ep_reward: -106.729, ep_step: 309 2023-05-30 00:29:03 - SimpleLog - INFO: - episode: 846, ep_reward: -103.170, ep_step: 222 2023-05-30 00:29:03 - SimpleLog - INFO: - episode: 847, ep_reward: -99.830, ep_step: 212 2023-05-30 00:29:06 - SimpleLog - INFO: - episode: 848, ep_reward: -58.089, ep_step: 1000 2023-05-30 00:29:08 - SimpleLog - INFO: - episode: 849, ep_reward: -69.934, ep_step: 1000 2023-05-30 00:29:10 - SimpleLog - INFO: - episode: 850, ep_reward: -76.827, ep_step: 1000 2023-05-30 00:29:12 - SimpleLog - INFO: - episode: 851, ep_reward: -63.702, ep_step: 1000 2023-05-30 00:29:15 - SimpleLog - INFO: - episode: 852, ep_reward: -56.620, ep_step: 1000 2023-05-30 00:29:17 - SimpleLog - INFO: - episode: 853, ep_reward: -46.418, ep_step: 1000 2023-05-30 00:29:19 - SimpleLog - INFO: - episode: 854, ep_reward: -51.936, ep_step: 1000 2023-05-30 00:29:22 - SimpleLog - INFO: - episode: 855, ep_reward: -52.928, ep_step: 1000 2023-05-30 00:29:24 - SimpleLog - INFO: - episode: 856, ep_reward: -53.750, ep_step: 1000 2023-05-30 00:29:26 - SimpleLog - INFO: - episode: 857, ep_reward: -56.519, ep_step: 1000 2023-05-30 00:29:29 - SimpleLog - INFO: - episode: 858, ep_reward: -57.407, ep_step: 1000 2023-05-30 00:29:31 - SimpleLog - INFO: - episode: 859, ep_reward: -69.866, ep_step: 1000 2023-05-30 00:29:33 - SimpleLog - INFO: - episode: 860, ep_reward: -63.354, ep_step: 1000 2023-05-30 00:29:35 - SimpleLog - INFO: - episode: 861, ep_reward: -51.221, ep_step: 1000 2023-05-30 00:29:38 - SimpleLog - INFO: - episode: 862, ep_reward: -58.558, ep_step: 1000 2023-05-30 00:29:40 - SimpleLog - INFO: - episode: 863, ep_reward: -79.319, ep_step: 1000 2023-05-30 00:29:42 - SimpleLog - INFO: - episode: 864, ep_reward: -89.410, ep_step: 1000 2023-05-30 00:29:45 - SimpleLog - INFO: - episode: 865, ep_reward: -81.581, ep_step: 1000 2023-05-30 00:29:47 - SimpleLog - INFO: - episode: 866, ep_reward: -80.213, ep_step: 1000 2023-05-30 00:29:49 - SimpleLog - INFO: - episode: 867, ep_reward: -86.013, ep_step: 1000 2023-05-30 00:29:51 - SimpleLog - INFO: - episode: 868, ep_reward: -67.907, ep_step: 1000 2023-05-30 00:29:54 - SimpleLog - INFO: - episode: 869, ep_reward: -44.111, ep_step: 1000 2023-05-30 00:29:56 - SimpleLog - INFO: - episode: 870, ep_reward: -35.961, ep_step: 1000 2023-05-30 00:29:58 - SimpleLog - INFO: - episode: 871, ep_reward: -52.056, ep_step: 1000 2023-05-30 00:30:01 - SimpleLog - INFO: - episode: 872, ep_reward: -41.699, ep_step: 1000 2023-05-30 00:30:03 - SimpleLog - INFO: - episode: 873, ep_reward: -46.684, ep_step: 1000 2023-05-30 00:30:05 - SimpleLog - INFO: - episode: 874, ep_reward: -54.400, ep_step: 1000 2023-05-30 00:30:07 - SimpleLog - INFO: - episode: 875, ep_reward: -47.312, ep_step: 1000 2023-05-30 00:30:10 - SimpleLog - INFO: - episode: 876, ep_reward: -43.435, ep_step: 1000 2023-05-30 00:30:12 - SimpleLog - INFO: - episode: 877, ep_reward: -53.039, ep_step: 1000 2023-05-30 00:30:14 - SimpleLog - INFO: - episode: 878, ep_reward: -52.348, ep_step: 1000 2023-05-30 00:30:17 - SimpleLog - INFO: - episode: 879, ep_reward: -48.313, ep_step: 1000 2023-05-30 00:30:19 - SimpleLog - INFO: - episode: 880, ep_reward: -48.636, ep_step: 1000 2023-05-30 00:30:21 - SimpleLog - INFO: - episode: 881, ep_reward: -45.125, ep_step: 1000 2023-05-30 00:30:23 - SimpleLog - INFO: - episode: 882, ep_reward: -48.202, ep_step: 1000 2023-05-30 00:30:26 - SimpleLog - INFO: - episode: 883, ep_reward: -46.072, ep_step: 1000 2023-05-30 00:30:28 - SimpleLog - INFO: - episode: 884, ep_reward: -48.278, ep_step: 1000 2023-05-30 00:30:30 - SimpleLog - INFO: - episode: 885, ep_reward: -49.735, ep_step: 1000 2023-05-30 00:30:32 - SimpleLog - INFO: - episode: 886, ep_reward: -48.050, ep_step: 1000 2023-05-30 00:30:35 - SimpleLog - INFO: - episode: 887, ep_reward: -48.380, ep_step: 1000 2023-05-30 00:30:37 - SimpleLog - INFO: - episode: 888, ep_reward: -58.274, ep_step: 1000 2023-05-30 00:30:39 - SimpleLog - INFO: - episode: 889, ep_reward: -64.562, ep_step: 1000 2023-05-30 00:30:41 - SimpleLog - INFO: - episode: 890, ep_reward: -62.743, ep_step: 1000 2023-05-30 00:30:44 - SimpleLog - INFO: - episode: 891, ep_reward: -54.366, ep_step: 1000 2023-05-30 00:30:46 - SimpleLog - INFO: - episode: 892, ep_reward: -58.269, ep_step: 1000 2023-05-30 00:30:46 - SimpleLog - INFO: - episode: 893, ep_reward: -118.793, ep_step: 83 2023-05-30 00:30:48 - SimpleLog - INFO: - episode: 894, ep_reward: -47.079, ep_step: 1000 2023-05-30 00:30:51 - SimpleLog - INFO: - episode: 895, ep_reward: -41.215, ep_step: 1000 2023-05-30 00:30:53 - SimpleLog - INFO: - episode: 896, ep_reward: -36.822, ep_step: 1000 2023-05-30 00:30:56 - SimpleLog - INFO: - update_step: 560000, online_eval_reward: -22.881 2023-05-30 00:30:57 - SimpleLog - INFO: - episode: 897, ep_reward: -39.592, ep_step: 1000 2023-05-30 00:31:00 - SimpleLog - INFO: - episode: 898, ep_reward: -41.215, ep_step: 1000 2023-05-30 00:31:02 - SimpleLog - INFO: - episode: 899, ep_reward: -34.819, ep_step: 1000 2023-05-30 00:31:04 - SimpleLog - INFO: - episode: 900, ep_reward: -32.531, ep_step: 1000 2023-05-30 00:31:06 - SimpleLog - INFO: - episode: 901, ep_reward: -34.440, ep_step: 1000 2023-05-30 00:31:07 - SimpleLog - INFO: - episode: 902, ep_reward: -104.162, ep_step: 83 2023-05-30 00:31:09 - SimpleLog - INFO: - episode: 903, ep_reward: -36.181, ep_step: 1000 2023-05-30 00:31:11 - SimpleLog - INFO: - episode: 904, ep_reward: -41.947, ep_step: 1000 2023-05-30 00:31:11 - SimpleLog - INFO: - episode: 905, ep_reward: -100.404, ep_step: 87 2023-05-30 00:31:14 - SimpleLog - INFO: - episode: 906, ep_reward: -32.509, ep_step: 1000 2023-05-30 00:31:16 - SimpleLog - INFO: - episode: 907, ep_reward: -35.277, ep_step: 1000 2023-05-30 00:31:18 - SimpleLog - INFO: - episode: 908, ep_reward: -34.027, ep_step: 1000 2023-05-30 00:31:21 - SimpleLog - INFO: - episode: 909, ep_reward: -32.956, ep_step: 1000 2023-05-30 00:31:23 - SimpleLog - INFO: - episode: 910, ep_reward: -33.729, ep_step: 1000 2023-05-30 00:31:25 - SimpleLog - INFO: - episode: 911, ep_reward: -32.125, ep_step: 1000 2023-05-30 00:31:27 - SimpleLog - INFO: - episode: 912, ep_reward: -34.708, ep_step: 1000 2023-05-30 00:31:30 - SimpleLog - INFO: - episode: 913, ep_reward: -34.407, ep_step: 1000 2023-05-30 00:31:32 - SimpleLog - INFO: - episode: 914, ep_reward: -32.976, ep_step: 1000 2023-05-30 00:31:32 - SimpleLog - INFO: - episode: 915, ep_reward: -97.433, ep_step: 97 2023-05-30 00:31:34 - SimpleLog - INFO: - episode: 916, ep_reward: -33.402, ep_step: 1000 2023-05-30 00:31:35 - SimpleLog - INFO: - episode: 917, ep_reward: -105.913, ep_step: 57 2023-05-30 00:31:37 - SimpleLog - INFO: - episode: 918, ep_reward: -35.892, ep_step: 1000 2023-05-30 00:31:39 - SimpleLog - INFO: - episode: 919, ep_reward: -31.008, ep_step: 1000 2023-05-30 00:31:39 - SimpleLog - INFO: - episode: 920, ep_reward: -101.766, ep_step: 70 2023-05-30 00:31:40 - SimpleLog - INFO: - episode: 921, ep_reward: -106.279, ep_step: 67 2023-05-30 00:31:40 - SimpleLog - INFO: - episode: 922, ep_reward: -103.836, ep_step: 71 2023-05-30 00:31:40 - SimpleLog - INFO: - episode: 923, ep_reward: -97.430, ep_step: 99 2023-05-30 00:31:40 - SimpleLog - INFO: - episode: 924, ep_reward: -106.806, ep_step: 66 2023-05-30 00:31:40 - SimpleLog - INFO: - episode: 925, ep_reward: -107.793, ep_step: 61 2023-05-30 00:31:40 - SimpleLog - INFO: - episode: 926, ep_reward: -117.388, ep_step: 75 2023-05-30 00:31:43 - SimpleLog - INFO: - episode: 927, ep_reward: -42.089, ep_step: 1000 2023-05-30 00:31:45 - SimpleLog - INFO: - episode: 928, ep_reward: -25.720, ep_step: 1000 2023-05-30 00:31:47 - SimpleLog - INFO: - episode: 929, ep_reward: -29.705, ep_step: 1000 2023-05-30 00:31:49 - SimpleLog - INFO: - episode: 930, ep_reward: -28.296, ep_step: 1000 2023-05-30 00:31:52 - SimpleLog - INFO: - episode: 931, ep_reward: -35.692, ep_step: 1000 2023-05-30 00:31:54 - SimpleLog - INFO: - episode: 932, ep_reward: -35.253, ep_step: 1000 2023-05-30 00:31:56 - SimpleLog - INFO: - episode: 933, ep_reward: -35.263, ep_step: 1000 2023-05-30 00:31:58 - SimpleLog - INFO: - episode: 934, ep_reward: -33.912, ep_step: 1000 2023-05-30 00:32:01 - SimpleLog - INFO: - episode: 935, ep_reward: -33.655, ep_step: 1000 2023-05-30 00:32:01 - SimpleLog - INFO: - episode: 936, ep_reward: -96.839, ep_step: 112 2023-05-30 00:32:03 - SimpleLog - INFO: - episode: 937, ep_reward: -34.753, ep_step: 1000 2023-05-30 00:32:05 - SimpleLog - INFO: - episode: 938, ep_reward: -30.967, ep_step: 1000 2023-05-30 00:32:08 - SimpleLog - INFO: - episode: 939, ep_reward: -31.713, ep_step: 1000 2023-05-30 00:32:10 - SimpleLog - INFO: - episode: 940, ep_reward: -29.268, ep_step: 1000 2023-05-30 00:32:12 - SimpleLog - INFO: - episode: 941, ep_reward: -27.829, ep_step: 1000 2023-05-30 00:32:15 - SimpleLog - INFO: - episode: 942, ep_reward: -31.268, ep_step: 1000 2023-05-30 00:32:17 - SimpleLog - INFO: - episode: 943, ep_reward: -25.515, ep_step: 1000 2023-05-30 00:32:19 - SimpleLog - INFO: - episode: 944, ep_reward: -26.048, ep_step: 1000 2023-05-30 00:32:21 - SimpleLog - INFO: - episode: 945, ep_reward: -25.702, ep_step: 1000 2023-05-30 00:32:24 - SimpleLog - INFO: - episode: 946, ep_reward: -29.153, ep_step: 1000 2023-05-30 00:32:26 - SimpleLog - INFO: - episode: 947, ep_reward: -32.686, ep_step: 1000 2023-05-30 00:32:28 - SimpleLog - INFO: - episode: 948, ep_reward: -32.137, ep_step: 1000 2023-05-30 00:32:30 - SimpleLog - INFO: - episode: 949, ep_reward: -28.735, ep_step: 1000 2023-05-30 00:32:33 - SimpleLog - INFO: - episode: 950, ep_reward: -27.244, ep_step: 1000 2023-05-30 00:32:35 - SimpleLog - INFO: - episode: 951, ep_reward: -27.555, ep_step: 1000 2023-05-30 00:32:37 - SimpleLog - INFO: - episode: 952, ep_reward: -56.641, ep_step: 1000 2023-05-30 00:32:38 - SimpleLog - INFO: - episode: 953, ep_reward: -143.342, ep_step: 170 2023-05-30 00:32:40 - SimpleLog - INFO: - episode: 954, ep_reward: -31.562, ep_step: 1000 2023-05-30 00:32:42 - SimpleLog - INFO: - episode: 955, ep_reward: -27.154, ep_step: 1000 2023-05-30 00:32:42 - SimpleLog - INFO: - episode: 956, ep_reward: -116.896, ep_step: 123 2023-05-30 00:32:43 - SimpleLog - INFO: - episode: 957, ep_reward: -110.364, ep_step: 132 2023-05-30 00:32:43 - SimpleLog - INFO: - episode: 958, ep_reward: -123.974, ep_step: 111 2023-05-30 00:32:43 - SimpleLog - INFO: - episode: 959, ep_reward: -120.231, ep_step: 93 2023-05-30 00:32:45 - SimpleLog - INFO: - episode: 960, ep_reward: -37.529, ep_step: 1000 2023-05-30 00:32:46 - SimpleLog - INFO: - episode: 961, ep_reward: -106.518, ep_step: 277 2023-05-30 00:32:48 - SimpleLog - INFO: - episode: 962, ep_reward: -47.194, ep_step: 1000 2023-05-30 00:32:51 - SimpleLog - INFO: - episode: 963, ep_reward: -28.393, ep_step: 1000 2023-05-30 00:32:51 - SimpleLog - INFO: - episode: 964, ep_reward: -97.304, ep_step: 329 2023-05-30 00:32:52 - SimpleLog - INFO: - episode: 965, ep_reward: -102.516, ep_step: 368 2023-05-30 00:32:54 - SimpleLog - INFO: - episode: 966, ep_reward: -29.820, ep_step: 1000 2023-05-30 00:32:57 - SimpleLog - INFO: - episode: 967, ep_reward: -30.611, ep_step: 1000 2023-05-30 00:32:57 - SimpleLog - INFO: - episode: 968, ep_reward: -85.582, ep_step: 186 2023-05-30 00:32:57 - SimpleLog - INFO: - episode: 969, ep_reward: -93.328, ep_step: 96 2023-05-30 00:32:58 - SimpleLog - INFO: - episode: 970, ep_reward: -92.236, ep_step: 106 2023-05-30 00:33:00 - SimpleLog - INFO: - episode: 971, ep_reward: -59.620, ep_step: 1000 2023-05-30 00:33:00 - SimpleLog - INFO: - episode: 972, ep_reward: -95.704, ep_step: 233 2023-05-30 00:33:01 - SimpleLog - INFO: - episode: 973, ep_reward: -89.547, ep_step: 229 2023-05-30 00:33:01 - SimpleLog - INFO: - episode: 974, ep_reward: -88.324, ep_step: 243 2023-05-30 00:33:04 - SimpleLog - INFO: - episode: 975, ep_reward: -58.366, ep_step: 1000 2023-05-30 00:33:06 - SimpleLog - INFO: - episode: 976, ep_reward: -43.228, ep_step: 1000 2023-05-30 00:33:08 - SimpleLog - INFO: - episode: 977, ep_reward: -47.929, ep_step: 1000 2023-05-30 00:33:08 - SimpleLog - INFO: - episode: 978, ep_reward: -96.565, ep_step: 86 2023-05-30 00:33:11 - SimpleLog - INFO: - episode: 979, ep_reward: -55.101, ep_step: 1000 2023-05-30 00:33:13 - SimpleLog - INFO: - episode: 980, ep_reward: -33.273, ep_step: 1000 2023-05-30 00:33:15 - SimpleLog - INFO: - episode: 981, ep_reward: -31.154, ep_step: 1000 2023-05-30 00:33:18 - SimpleLog - INFO: - episode: 982, ep_reward: -30.373, ep_step: 1000 2023-05-30 00:33:20 - SimpleLog - INFO: - episode: 983, ep_reward: -31.727, ep_step: 1000 2023-05-30 00:33:20 - SimpleLog - INFO: - episode: 984, ep_reward: -95.352, ep_step: 104 2023-05-30 00:33:22 - SimpleLog - INFO: - episode: 985, ep_reward: -30.546, ep_step: 1000 2023-05-30 00:33:25 - SimpleLog - INFO: - episode: 986, ep_reward: -28.602, ep_step: 1000 2023-05-30 00:33:27 - SimpleLog - INFO: - episode: 987, ep_reward: -29.741, ep_step: 1000 2023-05-30 00:33:29 - SimpleLog - INFO: - episode: 988, ep_reward: -26.819, ep_step: 1000 2023-05-30 00:33:31 - SimpleLog - INFO: - episode: 989, ep_reward: -58.039, ep_step: 1000 2023-05-30 00:33:34 - SimpleLog - INFO: - episode: 990, ep_reward: -30.172, ep_step: 1000 2023-05-30 00:33:37 - SimpleLog - INFO: - update_step: 630000, online_eval_reward: -17.610 2023-05-30 00:33:38 - SimpleLog - INFO: - episode: 991, ep_reward: -40.109, ep_step: 1000 2023-05-30 00:33:40 - SimpleLog - INFO: - episode: 992, ep_reward: -34.320, ep_step: 1000 2023-05-30 00:33:42 - SimpleLog - INFO: - episode: 993, ep_reward: -35.822, ep_step: 1000 2023-05-30 00:33:45 - SimpleLog - INFO: - episode: 994, ep_reward: -23.897, ep_step: 1000 2023-05-30 00:33:47 - SimpleLog - INFO: - episode: 995, ep_reward: -34.233, ep_step: 1000 2023-05-30 00:33:49 - SimpleLog - INFO: - episode: 996, ep_reward: -28.496, ep_step: 1000 2023-05-30 00:33:52 - SimpleLog - INFO: - episode: 997, ep_reward: -26.977, ep_step: 1000 2023-05-30 00:33:54 - SimpleLog - INFO: - episode: 998, ep_reward: -25.980, ep_step: 1000 2023-05-30 00:33:56 - SimpleLog - INFO: - episode: 999, ep_reward: -29.853, ep_step: 1000 2023-05-30 00:33:58 - SimpleLog - INFO: - episode: 1000, ep_reward: -30.893, ep_step: 1000 2023-05-30 00:34:01 - SimpleLog - INFO: - episode: 1001, ep_reward: -25.388, ep_step: 1000 2023-05-30 00:34:03 - SimpleLog - INFO: - episode: 1002, ep_reward: -25.801, ep_step: 1000 2023-05-30 00:34:04 - SimpleLog - INFO: - episode: 1003, ep_reward: -122.498, ep_step: 467 2023-05-30 00:34:06 - SimpleLog - INFO: - episode: 1004, ep_reward: -25.023, ep_step: 1000 2023-05-30 00:34:09 - SimpleLog - INFO: - episode: 1005, ep_reward: -24.495, ep_step: 1000 2023-05-30 00:34:11 - SimpleLog - INFO: - episode: 1006, ep_reward: -31.384, ep_step: 1000 2023-05-30 00:34:13 - SimpleLog - INFO: - episode: 1007, ep_reward: -22.644, ep_step: 1000 2023-05-30 00:34:14 - SimpleLog - INFO: - episode: 1008, ep_reward: -115.821, ep_step: 348 2023-05-30 00:34:15 - SimpleLog - INFO: - episode: 1009, ep_reward: -112.988, ep_step: 477 2023-05-30 00:34:16 - SimpleLog - INFO: - episode: 1010, ep_reward: -96.728, ep_step: 306 2023-05-30 00:34:16 - SimpleLog - INFO: - episode: 1011, ep_reward: -95.127, ep_step: 273 2023-05-30 00:34:17 - SimpleLog - INFO: - episode: 1012, ep_reward: -93.506, ep_step: 249 2023-05-30 00:34:17 - SimpleLog - INFO: - episode: 1013, ep_reward: -89.659, ep_step: 223 2023-05-30 00:34:18 - SimpleLog - INFO: - episode: 1014, ep_reward: -97.447, ep_step: 283 2023-05-30 00:34:19 - SimpleLog - INFO: - episode: 1015, ep_reward: -94.664, ep_step: 274 2023-05-30 00:34:21 - SimpleLog - INFO: - episode: 1016, ep_reward: -34.767, ep_step: 1000 2023-05-30 00:34:22 - SimpleLog - INFO: - episode: 1017, ep_reward: -116.721, ep_step: 583 2023-05-30 00:34:24 - SimpleLog - INFO: - episode: 1018, ep_reward: -28.332, ep_step: 1000 2023-05-30 00:34:27 - SimpleLog - INFO: - episode: 1019, ep_reward: -41.216, ep_step: 1000 2023-05-30 00:34:29 - SimpleLog - INFO: - episode: 1020, ep_reward: -36.861, ep_step: 1000 2023-05-30 00:34:29 - SimpleLog - INFO: - episode: 1021, ep_reward: -95.414, ep_step: 136 2023-05-30 00:34:32 - SimpleLog - INFO: - episode: 1022, ep_reward: -33.550, ep_step: 1000 2023-05-30 00:34:34 - SimpleLog - INFO: - episode: 1023, ep_reward: -32.305, ep_step: 1000 2023-05-30 00:34:36 - SimpleLog - INFO: - episode: 1024, ep_reward: -31.753, ep_step: 1000 2023-05-30 00:34:38 - SimpleLog - INFO: - episode: 1025, ep_reward: -34.469, ep_step: 1000 2023-05-30 00:34:41 - SimpleLog - INFO: - episode: 1026, ep_reward: -34.499, ep_step: 1000 2023-05-30 00:34:43 - SimpleLog - INFO: - episode: 1027, ep_reward: -34.812, ep_step: 1000 2023-05-30 00:34:45 - SimpleLog - INFO: - episode: 1028, ep_reward: -36.931, ep_step: 1000 2023-05-30 00:34:47 - SimpleLog - INFO: - episode: 1029, ep_reward: -89.853, ep_step: 1000 2023-05-30 00:34:50 - SimpleLog - INFO: - episode: 1030, ep_reward: -35.886, ep_step: 1000 2023-05-30 00:34:52 - SimpleLog - INFO: - episode: 1031, ep_reward: -37.350, ep_step: 1000 2023-05-30 00:34:54 - SimpleLog - INFO: - episode: 1032, ep_reward: -34.791, ep_step: 1000 2023-05-30 00:34:56 - SimpleLog - INFO: - episode: 1033, ep_reward: -37.192, ep_step: 1000 2023-05-30 00:34:59 - SimpleLog - INFO: - episode: 1034, ep_reward: -39.672, ep_step: 1000 2023-05-30 00:35:01 - SimpleLog - INFO: - episode: 1035, ep_reward: -37.012, ep_step: 1000 2023-05-30 00:35:03 - SimpleLog - INFO: - episode: 1036, ep_reward: -36.380, ep_step: 1000 2023-05-30 00:35:05 - SimpleLog - INFO: - episode: 1037, ep_reward: -39.068, ep_step: 1000 2023-05-30 00:35:08 - SimpleLog - INFO: - episode: 1038, ep_reward: -36.162, ep_step: 1000 2023-05-30 00:35:10 - SimpleLog - INFO: - episode: 1039, ep_reward: -39.389, ep_step: 1000 2023-05-30 00:35:12 - SimpleLog - INFO: - episode: 1040, ep_reward: -35.296, ep_step: 1000 2023-05-30 00:35:15 - SimpleLog - INFO: - episode: 1041, ep_reward: -35.965, ep_step: 1000 2023-05-30 00:35:17 - SimpleLog - INFO: - episode: 1042, ep_reward: -38.558, ep_step: 1000 2023-05-30 00:35:19 - SimpleLog - INFO: - episode: 1043, ep_reward: -39.190, ep_step: 1000 2023-05-30 00:35:21 - SimpleLog - INFO: - episode: 1044, ep_reward: -33.790, ep_step: 1000 2023-05-30 00:35:24 - SimpleLog - INFO: - episode: 1045, ep_reward: -38.457, ep_step: 1000 2023-05-30 00:35:26 - SimpleLog - INFO: - episode: 1046, ep_reward: -35.963, ep_step: 1000 2023-05-30 00:35:28 - SimpleLog - INFO: - episode: 1047, ep_reward: -38.298, ep_step: 1000 2023-05-30 00:35:29 - SimpleLog - INFO: - episode: 1048, ep_reward: -107.510, ep_step: 398 2023-05-30 00:35:32 - SimpleLog - INFO: - episode: 1049, ep_reward: -33.649, ep_step: 1000 2023-05-30 00:35:34 - SimpleLog - INFO: - episode: 1050, ep_reward: -36.596, ep_step: 1000 2023-05-30 00:35:36 - SimpleLog - INFO: - episode: 1051, ep_reward: -34.476, ep_step: 1000 2023-05-30 00:35:38 - SimpleLog - INFO: - episode: 1052, ep_reward: -36.784, ep_step: 1000 2023-05-30 00:35:41 - SimpleLog - INFO: - episode: 1053, ep_reward: -37.216, ep_step: 1000 2023-05-30 00:35:43 - SimpleLog - INFO: - episode: 1054, ep_reward: -36.878, ep_step: 1000 2023-05-30 00:35:43 - SimpleLog - INFO: - episode: 1055, ep_reward: -99.191, ep_step: 220 2023-05-30 00:35:46 - SimpleLog - INFO: - episode: 1056, ep_reward: -41.551, ep_step: 1000 2023-05-30 00:35:46 - SimpleLog - INFO: - episode: 1057, ep_reward: -102.694, ep_step: 323 2023-05-30 00:35:47 - SimpleLog - INFO: - episode: 1058, ep_reward: -105.725, ep_step: 224 2023-05-30 00:35:48 - SimpleLog - INFO: - episode: 1059, ep_reward: -103.481, ep_step: 265 2023-05-30 00:35:49 - SimpleLog - INFO: - episode: 1060, ep_reward: -107.325, ep_step: 674 2023-05-30 00:35:51 - SimpleLog - INFO: - episode: 1061, ep_reward: -25.560, ep_step: 1000 2023-05-30 00:35:52 - SimpleLog - INFO: - episode: 1062, ep_reward: -100.212, ep_step: 251 2023-05-30 00:35:54 - SimpleLog - INFO: - episode: 1063, ep_reward: -92.475, ep_step: 869 2023-05-30 00:35:56 - SimpleLog - INFO: - episode: 1064, ep_reward: -22.342, ep_step: 1000 2023-05-30 00:35:58 - SimpleLog - INFO: - episode: 1065, ep_reward: 10.634, ep_step: 1000 2023-05-30 00:36:01 - SimpleLog - INFO: - episode: 1066, ep_reward: -1.530, ep_step: 1000 2023-05-30 00:36:03 - SimpleLog - INFO: - episode: 1067, ep_reward: -22.063, ep_step: 1000 2023-05-30 00:36:05 - SimpleLog - INFO: - episode: 1068, ep_reward: 19.165, ep_step: 1000 2023-05-30 00:36:08 - SimpleLog - INFO: - episode: 1069, ep_reward: -20.751, ep_step: 1000 2023-05-30 00:36:10 - SimpleLog - INFO: - episode: 1070, ep_reward: -10.657, ep_step: 1000 2023-05-30 00:36:11 - SimpleLog - INFO: - episode: 1071, ep_reward: -99.135, ep_step: 295 2023-05-30 00:36:13 - SimpleLog - INFO: - episode: 1072, ep_reward: -68.851, ep_step: 1000 2023-05-30 00:36:15 - SimpleLog - INFO: - episode: 1073, ep_reward: -66.176, ep_step: 1000 2023-05-30 00:36:19 - SimpleLog - INFO: - update_step: 700000, online_eval_reward: -55.129 2023-05-30 00:36:19 - SimpleLog - INFO: - episode: 1074, ep_reward: -41.820, ep_step: 1000 2023-05-30 00:36:22 - SimpleLog - INFO: - episode: 1075, ep_reward: -57.210, ep_step: 1000 2023-05-30 00:36:24 - SimpleLog - INFO: - episode: 1076, ep_reward: -41.484, ep_step: 1000 2023-05-30 00:36:26 - SimpleLog - INFO: - episode: 1077, ep_reward: -41.837, ep_step: 1000 2023-05-30 00:36:29 - SimpleLog - INFO: - episode: 1078, ep_reward: -38.587, ep_step: 1000 2023-05-30 00:36:31 - SimpleLog - INFO: - episode: 1079, ep_reward: -27.111, ep_step: 1000 2023-05-30 00:36:33 - SimpleLog - INFO: - episode: 1080, ep_reward: -42.744, ep_step: 1000 2023-05-30 00:36:35 - SimpleLog - INFO: - episode: 1081, ep_reward: -48.780, ep_step: 1000 2023-05-30 00:36:38 - SimpleLog - INFO: - episode: 1082, ep_reward: -58.123, ep_step: 1000 2023-05-30 00:36:40 - SimpleLog - INFO: - episode: 1083, ep_reward: -41.969, ep_step: 1000 2023-05-30 00:36:42 - SimpleLog - INFO: - episode: 1084, ep_reward: -36.989, ep_step: 1000 2023-05-30 00:36:42 - SimpleLog - INFO: - episode: 1085, ep_reward: -100.060, ep_step: 90 2023-05-30 00:36:45 - SimpleLog - INFO: - episode: 1086, ep_reward: -33.872, ep_step: 1000 2023-05-30 00:36:47 - SimpleLog - INFO: - episode: 1087, ep_reward: -34.643, ep_step: 1000 2023-05-30 00:36:49 - SimpleLog - INFO: - episode: 1088, ep_reward: -31.144, ep_step: 1000 2023-05-30 00:36:52 - SimpleLog - INFO: - episode: 1089, ep_reward: -21.532, ep_step: 1000 2023-05-30 00:36:54 - SimpleLog - INFO: - episode: 1090, ep_reward: 9.741, ep_step: 1000 2023-05-30 00:36:56 - SimpleLog - INFO: - episode: 1091, ep_reward: 22.637, ep_step: 1000 2023-05-30 00:36:58 - SimpleLog - INFO: - episode: 1092, ep_reward: 0.283, ep_step: 1000 2023-05-30 00:36:59 - SimpleLog - INFO: - episode: 1093, ep_reward: -89.119, ep_step: 249 2023-05-30 00:36:59 - SimpleLog - INFO: - episode: 1094, ep_reward: -98.364, ep_step: 212 2023-05-30 00:37:00 - SimpleLog - INFO: - episode: 1095, ep_reward: -93.669, ep_step: 260 2023-05-30 00:37:01 - SimpleLog - INFO: - episode: 1096, ep_reward: -92.714, ep_step: 235 2023-05-30 00:37:03 - SimpleLog - INFO: - episode: 1097, ep_reward: 27.536, ep_step: 1000 2023-05-30 00:37:03 - SimpleLog - INFO: - episode: 1098, ep_reward: -93.834, ep_step: 212 2023-05-30 00:37:04 - SimpleLog - INFO: - episode: 1099, ep_reward: -100.111, ep_step: 302 2023-05-30 00:37:06 - SimpleLog - INFO: - episode: 1100, ep_reward: 19.391, ep_step: 1000 2023-05-30 00:37:09 - SimpleLog - INFO: - episode: 1101, ep_reward: 12.116, ep_step: 1000 2023-05-30 00:37:09 - SimpleLog - INFO: - episode: 1102, ep_reward: -91.153, ep_step: 239 2023-05-30 00:37:11 - SimpleLog - INFO: - episode: 1103, ep_reward: -2.471, ep_step: 1000 2023-05-30 00:37:14 - SimpleLog - INFO: - episode: 1104, ep_reward: -33.635, ep_step: 1000 2023-05-30 00:37:16 - SimpleLog - INFO: - episode: 1105, ep_reward: -15.387, ep_step: 1000 2023-05-30 00:37:17 - SimpleLog - INFO: - episode: 1106, ep_reward: -92.145, ep_step: 260 2023-05-30 00:37:19 - SimpleLog - INFO: - episode: 1107, ep_reward: 25.316, ep_step: 1000 2023-05-30 00:37:21 - SimpleLog - INFO: - episode: 1108, ep_reward: -17.722, ep_step: 1000 2023-05-30 00:37:23 - SimpleLog - INFO: - episode: 1109, ep_reward: -5.865, ep_step: 1000 2023-05-30 00:37:26 - SimpleLog - INFO: - episode: 1110, ep_reward: -3.604, ep_step: 1000 2023-05-30 00:37:26 - SimpleLog - INFO: - episode: 1111, ep_reward: -96.828, ep_step: 163 2023-05-30 00:37:28 - SimpleLog - INFO: - episode: 1112, ep_reward: 26.325, ep_step: 1000 2023-05-30 00:37:29 - SimpleLog - INFO: - episode: 1113, ep_reward: -92.215, ep_step: 258 2023-05-30 00:37:31 - SimpleLog - INFO: - episode: 1114, ep_reward: -77.194, ep_step: 753 2023-05-30 00:37:33 - SimpleLog - INFO: - episode: 1115, ep_reward: 25.425, ep_step: 1000 2023-05-30 00:37:33 - SimpleLog - INFO: - episode: 1116, ep_reward: -90.441, ep_step: 216 2023-05-30 00:37:35 - SimpleLog - INFO: - episode: 1117, ep_reward: -106.374, ep_step: 629 2023-05-30 00:37:37 - SimpleLog - INFO: - episode: 1118, ep_reward: 22.542, ep_step: 1000 2023-05-30 00:37:38 - SimpleLog - INFO: - episode: 1119, ep_reward: -69.997, ep_step: 582 2023-05-30 00:37:41 - SimpleLog - INFO: - episode: 1120, ep_reward: -76.065, ep_step: 966 2023-05-30 00:37:43 - SimpleLog - INFO: - episode: 1121, ep_reward: 22.836, ep_step: 1000 2023-05-30 00:37:43 - SimpleLog - INFO: - episode: 1122, ep_reward: -94.330, ep_step: 273 2023-05-30 00:37:46 - SimpleLog - INFO: - episode: 1123, ep_reward: 16.669, ep_step: 1000 2023-05-30 00:37:48 - SimpleLog - INFO: - episode: 1124, ep_reward: 13.107, ep_step: 1000 2023-05-30 00:37:49 - SimpleLog - INFO: - episode: 1125, ep_reward: -79.593, ep_step: 446 2023-05-30 00:37:51 - SimpleLog - INFO: - episode: 1126, ep_reward: 20.108, ep_step: 1000 2023-05-30 00:37:53 - SimpleLog - INFO: - episode: 1127, ep_reward: -63.686, ep_step: 591 2023-05-30 00:37:55 - SimpleLog - INFO: - episode: 1128, ep_reward: 28.986, ep_step: 1000 2023-05-30 00:37:56 - SimpleLog - INFO: - episode: 1129, ep_reward: -86.915, ep_step: 515 2023-05-30 00:37:58 - SimpleLog - INFO: - episode: 1130, ep_reward: 16.230, ep_step: 1000 2023-05-30 00:37:59 - SimpleLog - INFO: - episode: 1131, ep_reward: -73.802, ep_step: 468 2023-05-30 00:38:00 - SimpleLog - INFO: - episode: 1132, ep_reward: -61.025, ep_step: 506 2023-05-30 00:38:03 - SimpleLog - INFO: - episode: 1133, ep_reward: 20.628, ep_step: 1000 2023-05-30 00:38:04 - SimpleLog - INFO: - episode: 1134, ep_reward: -102.200, ep_step: 358 2023-05-30 00:38:06 - SimpleLog - INFO: - episode: 1135, ep_reward: 18.346, ep_step: 1000 2023-05-30 00:38:07 - SimpleLog - INFO: - episode: 1136, ep_reward: -87.635, ep_step: 554 2023-05-30 00:38:09 - SimpleLog - INFO: - episode: 1137, ep_reward: 21.916, ep_step: 1000 2023-05-30 00:38:12 - SimpleLog - INFO: - episode: 1138, ep_reward: 12.796, ep_step: 1000 2023-05-30 00:38:14 - SimpleLog - INFO: - episode: 1139, ep_reward: -67.465, ep_step: 826 2023-05-30 00:38:14 - SimpleLog - INFO: - episode: 1140, ep_reward: -89.583, ep_step: 326 2023-05-30 00:38:15 - SimpleLog - INFO: - episode: 1141, ep_reward: -78.705, ep_step: 399 2023-05-30 00:38:17 - SimpleLog - INFO: - episode: 1142, ep_reward: -77.946, ep_step: 972 2023-05-30 00:38:20 - SimpleLog - INFO: - episode: 1143, ep_reward: -77.190, ep_step: 984 2023-05-30 00:38:22 - SimpleLog - INFO: - episode: 1144, ep_reward: 17.371, ep_step: 1000 2023-05-30 00:38:24 - SimpleLog - INFO: - episode: 1145, ep_reward: 17.266, ep_step: 1000 2023-05-30 00:38:26 - SimpleLog - INFO: - episode: 1146, ep_reward: 20.800, ep_step: 1000 2023-05-30 00:38:28 - SimpleLog - INFO: - episode: 1147, ep_reward: -66.603, ep_step: 556 2023-05-30 00:38:30 - SimpleLog - INFO: - episode: 1148, ep_reward: 16.892, ep_step: 1000 2023-05-30 00:38:32 - SimpleLog - INFO: - episode: 1149, ep_reward: 17.771, ep_step: 1000 2023-05-30 00:38:33 - SimpleLog - INFO: - episode: 1150, ep_reward: -103.778, ep_step: 197 2023-05-30 00:38:35 - SimpleLog - INFO: - episode: 1151, ep_reward: 17.575, ep_step: 1000 2023-05-30 00:38:37 - SimpleLog - INFO: - episode: 1152, ep_reward: 18.793, ep_step: 1000 2023-05-30 00:38:40 - SimpleLog - INFO: - episode: 1153, ep_reward: 18.089, ep_step: 1000 2023-05-30 00:38:42 - SimpleLog - INFO: - episode: 1154, ep_reward: 18.867, ep_step: 1000 2023-05-30 00:38:44 - SimpleLog - INFO: - episode: 1155, ep_reward: 17.126, ep_step: 1000 2023-05-30 00:38:46 - SimpleLog - INFO: - episode: 1156, ep_reward: 18.914, ep_step: 1000 2023-05-30 00:38:49 - SimpleLog - INFO: - episode: 1157, ep_reward: 17.750, ep_step: 1000 2023-05-30 00:38:50 - SimpleLog - INFO: - episode: 1158, ep_reward: -81.758, ep_step: 435 2023-05-30 00:38:52 - SimpleLog - INFO: - episode: 1159, ep_reward: 18.787, ep_step: 1000 2023-05-30 00:38:54 - SimpleLog - INFO: - episode: 1160, ep_reward: 16.803, ep_step: 1000 2023-05-30 00:38:56 - SimpleLog - INFO: - episode: 1161, ep_reward: -80.612, ep_step: 685 2023-05-30 00:39:00 - SimpleLog - INFO: - update_step: 770000, online_eval_reward: 12.206 2023-05-30 00:39:00 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 12.206, save the best model! 2023-05-30 00:39:00 - SimpleLog - INFO: - episode: 1162, ep_reward: 17.843, ep_step: 1000 2023-05-30 00:39:02 - SimpleLog - INFO: - episode: 1163, ep_reward: 16.274, ep_step: 1000 2023-05-30 00:39:04 - SimpleLog - INFO: - episode: 1164, ep_reward: -44.882, ep_step: 1000 2023-05-30 00:39:07 - SimpleLog - INFO: - episode: 1165, ep_reward: 17.509, ep_step: 1000 2023-05-30 00:39:09 - SimpleLog - INFO: - episode: 1166, ep_reward: 17.042, ep_step: 1000 2023-05-30 00:39:11 - SimpleLog - INFO: - episode: 1167, ep_reward: 3.633, ep_step: 1000 2023-05-30 00:39:13 - SimpleLog - INFO: - episode: 1168, ep_reward: 18.284, ep_step: 1000 2023-05-30 00:39:16 - SimpleLog - INFO: - episode: 1169, ep_reward: 13.920, ep_step: 1000 2023-05-30 00:39:18 - SimpleLog - INFO: - episode: 1170, ep_reward: 16.906, ep_step: 1000 2023-05-30 00:39:20 - SimpleLog - INFO: - episode: 1171, ep_reward: 15.164, ep_step: 1000 2023-05-30 00:39:23 - SimpleLog - INFO: - episode: 1172, ep_reward: 17.187, ep_step: 1000 2023-05-30 00:39:25 - SimpleLog - INFO: - episode: 1173, ep_reward: 1.546, ep_step: 1000 2023-05-30 00:39:27 - SimpleLog - INFO: - episode: 1174, ep_reward: -46.005, ep_step: 1000 2023-05-30 00:39:29 - SimpleLog - INFO: - episode: 1175, ep_reward: -46.932, ep_step: 1000 2023-05-30 00:39:32 - SimpleLog - INFO: - episode: 1176, ep_reward: -45.607, ep_step: 1000 2023-05-30 00:39:34 - SimpleLog - INFO: - episode: 1177, ep_reward: 1.275, ep_step: 1000 2023-05-30 00:39:36 - SimpleLog - INFO: - episode: 1178, ep_reward: 17.802, ep_step: 1000 2023-05-30 00:39:38 - SimpleLog - INFO: - episode: 1179, ep_reward: 16.302, ep_step: 1000 2023-05-30 00:39:40 - SimpleLog - INFO: - episode: 1180, ep_reward: -111.731, ep_step: 550 2023-05-30 00:39:42 - SimpleLog - INFO: - episode: 1181, ep_reward: 15.468, ep_step: 1000 2023-05-30 00:39:44 - SimpleLog - INFO: - episode: 1182, ep_reward: 16.079, ep_step: 1000 2023-05-30 00:39:46 - SimpleLog - INFO: - episode: 1183, ep_reward: -77.901, ep_step: 905 2023-05-30 00:39:47 - SimpleLog - INFO: - episode: 1184, ep_reward: -103.332, ep_step: 87 2023-05-30 00:39:47 - SimpleLog - INFO: - episode: 1185, ep_reward: -103.373, ep_step: 79 2023-05-30 00:39:47 - SimpleLog - INFO: - episode: 1186, ep_reward: -116.119, ep_step: 219 2023-05-30 00:39:47 - SimpleLog - INFO: - episode: 1187, ep_reward: -102.475, ep_step: 88 2023-05-30 00:39:48 - SimpleLog - INFO: - episode: 1188, ep_reward: -103.301, ep_step: 76 2023-05-30 00:39:48 - SimpleLog - INFO: - episode: 1189, ep_reward: -112.134, ep_step: 233 2023-05-30 00:39:49 - SimpleLog - INFO: - episode: 1190, ep_reward: -99.552, ep_step: 211 2023-05-30 00:39:49 - SimpleLog - INFO: - episode: 1191, ep_reward: -107.148, ep_step: 94 2023-05-30 00:39:49 - SimpleLog - INFO: - episode: 1192, ep_reward: -118.690, ep_step: 226 2023-05-30 00:39:50 - SimpleLog - INFO: - episode: 1193, ep_reward: -106.982, ep_step: 91 2023-05-30 00:39:50 - SimpleLog - INFO: - episode: 1194, ep_reward: -97.327, ep_step: 182 2023-05-30 00:39:50 - SimpleLog - INFO: - episode: 1195, ep_reward: -106.977, ep_step: 145 2023-05-30 00:39:51 - SimpleLog - INFO: - episode: 1196, ep_reward: -109.060, ep_step: 157 2023-05-30 00:39:51 - SimpleLog - INFO: - episode: 1197, ep_reward: -109.909, ep_step: 150 2023-05-30 00:39:51 - SimpleLog - INFO: - episode: 1198, ep_reward: -102.704, ep_step: 74 2023-05-30 00:39:51 - SimpleLog - INFO: - episode: 1199, ep_reward: -102.455, ep_step: 86 2023-05-30 00:39:52 - SimpleLog - INFO: - episode: 1200, ep_reward: -115.166, ep_step: 155 2023-05-30 00:39:52 - SimpleLog - INFO: - episode: 1201, ep_reward: -105.455, ep_step: 88 2023-05-30 00:39:52 - SimpleLog - INFO: - episode: 1202, ep_reward: -103.393, ep_step: 84 2023-05-30 00:39:52 - SimpleLog - INFO: - episode: 1203, ep_reward: -103.163, ep_step: 95 2023-05-30 00:39:53 - SimpleLog - INFO: - episode: 1204, ep_reward: -103.490, ep_step: 68 2023-05-30 00:39:53 - SimpleLog - INFO: - episode: 1205, ep_reward: -106.658, ep_step: 96 2023-05-30 00:39:53 - SimpleLog - INFO: - episode: 1206, ep_reward: -102.126, ep_step: 78 2023-05-30 00:39:53 - SimpleLog - INFO: - episode: 1207, ep_reward: -103.133, ep_step: 76 2023-05-30 00:39:53 - SimpleLog - INFO: - episode: 1208, ep_reward: -103.453, ep_step: 107 2023-05-30 00:39:54 - SimpleLog - INFO: - episode: 1209, ep_reward: -102.323, ep_step: 135 2023-05-30 00:39:54 - SimpleLog - INFO: - episode: 1210, ep_reward: -101.953, ep_step: 77 2023-05-30 00:39:54 - SimpleLog - INFO: - episode: 1211, ep_reward: -121.401, ep_step: 216 2023-05-30 00:39:55 - SimpleLog - INFO: - episode: 1212, ep_reward: -113.918, ep_step: 141 2023-05-30 00:39:55 - SimpleLog - INFO: - episode: 1213, ep_reward: -102.486, ep_step: 75 2023-05-30 00:39:55 - SimpleLog - INFO: - episode: 1214, ep_reward: -111.128, ep_step: 136 2023-05-30 00:39:55 - SimpleLog - INFO: - episode: 1215, ep_reward: -102.356, ep_step: 73 2023-05-30 00:39:55 - SimpleLog - INFO: - episode: 1216, ep_reward: -102.059, ep_step: 65 2023-05-30 00:39:56 - SimpleLog - INFO: - episode: 1217, ep_reward: -104.973, ep_step: 60 2023-05-30 00:39:56 - SimpleLog - INFO: - episode: 1218, ep_reward: -103.264, ep_step: 69 2023-05-30 00:39:56 - SimpleLog - INFO: - episode: 1219, ep_reward: -101.611, ep_step: 76 2023-05-30 00:39:56 - SimpleLog - INFO: - episode: 1220, ep_reward: -102.715, ep_step: 70 2023-05-30 00:39:56 - SimpleLog - INFO: - episode: 1221, ep_reward: -103.853, ep_step: 80 2023-05-30 00:39:56 - SimpleLog - INFO: - episode: 1222, ep_reward: -104.376, ep_step: 74 2023-05-30 00:39:57 - SimpleLog - INFO: - episode: 1223, ep_reward: -104.105, ep_step: 92 2023-05-30 00:39:57 - SimpleLog - INFO: - episode: 1224, ep_reward: -102.077, ep_step: 69 2023-05-30 00:39:57 - SimpleLog - INFO: - episode: 1225, ep_reward: -102.072, ep_step: 74 2023-05-30 00:39:57 - SimpleLog - INFO: - episode: 1226, ep_reward: -103.783, ep_step: 86 2023-05-30 00:39:57 - SimpleLog - INFO: - episode: 1227, ep_reward: -104.842, ep_step: 70 2023-05-30 00:39:58 - SimpleLog - INFO: - episode: 1228, ep_reward: -102.856, ep_step: 66 2023-05-30 00:39:58 - SimpleLog - INFO: - episode: 1229, ep_reward: -102.396, ep_step: 67 2023-05-30 00:39:58 - SimpleLog - INFO: - episode: 1230, ep_reward: -103.414, ep_step: 82 2023-05-30 00:39:58 - SimpleLog - INFO: - episode: 1231, ep_reward: -101.280, ep_step: 66 2023-05-30 00:39:58 - SimpleLog - INFO: - episode: 1232, ep_reward: -102.482, ep_step: 83 2023-05-30 00:39:59 - SimpleLog - INFO: - episode: 1233, ep_reward: -122.582, ep_step: 268 2023-05-30 00:39:59 - SimpleLog - INFO: - episode: 1234, ep_reward: -101.659, ep_step: 68 2023-05-30 00:39:59 - SimpleLog - INFO: - episode: 1235, ep_reward: -102.257, ep_step: 64 2023-05-30 00:39:59 - SimpleLog - INFO: - episode: 1236, ep_reward: -105.224, ep_step: 78 2023-05-30 00:39:59 - SimpleLog - INFO: - episode: 1237, ep_reward: -102.242, ep_step: 64 2023-05-30 00:40:00 - SimpleLog - INFO: - episode: 1238, ep_reward: -107.032, ep_step: 91 2023-05-30 00:40:00 - SimpleLog - INFO: - episode: 1239, ep_reward: -106.093, ep_step: 78 2023-05-30 00:40:00 - SimpleLog - INFO: - episode: 1240, ep_reward: -105.416, ep_step: 96 2023-05-30 00:40:00 - SimpleLog - INFO: - episode: 1241, ep_reward: -102.604, ep_step: 60 2023-05-30 00:40:00 - SimpleLog - INFO: - episode: 1242, ep_reward: -104.883, ep_step: 67 2023-05-30 00:40:01 - SimpleLog - INFO: - episode: 1243, ep_reward: -103.136, ep_step: 77 2023-05-30 00:40:01 - SimpleLog - INFO: - episode: 1244, ep_reward: -108.909, ep_step: 85 2023-05-30 00:40:01 - SimpleLog - INFO: - episode: 1245, ep_reward: -118.693, ep_step: 64 2023-05-30 00:40:01 - SimpleLog - INFO: - episode: 1246, ep_reward: -107.031, ep_step: 69 2023-05-30 00:40:01 - SimpleLog - INFO: - episode: 1247, ep_reward: -107.656, ep_step: 69 2023-05-30 00:40:01 - SimpleLog - INFO: - episode: 1248, ep_reward: -103.065, ep_step: 55 2023-05-30 00:40:01 - SimpleLog - INFO: - episode: 1249, ep_reward: -104.266, ep_step: 67 2023-05-30 00:40:02 - SimpleLog - INFO: - episode: 1250, ep_reward: -107.745, ep_step: 92 2023-05-30 00:40:02 - SimpleLog - INFO: - episode: 1251, ep_reward: -103.217, ep_step: 72 2023-05-30 00:40:02 - SimpleLog - INFO: - episode: 1252, ep_reward: -105.009, ep_step: 78 2023-05-30 00:40:02 - SimpleLog - INFO: - episode: 1253, ep_reward: -103.500, ep_step: 59 2023-05-30 00:40:02 - SimpleLog - INFO: - episode: 1254, ep_reward: -107.651, ep_step: 72 2023-05-30 00:40:03 - SimpleLog - INFO: - episode: 1255, ep_reward: -105.453, ep_step: 76 2023-05-30 00:40:03 - SimpleLog - INFO: - episode: 1256, ep_reward: -105.082, ep_step: 67 2023-05-30 00:40:03 - SimpleLog - INFO: - episode: 1257, ep_reward: -102.114, ep_step: 65 2023-05-30 00:40:03 - SimpleLog - INFO: - episode: 1258, ep_reward: -102.868, ep_step: 86 2023-05-30 00:40:03 - SimpleLog - INFO: - episode: 1259, ep_reward: -102.549, ep_step: 82 2023-05-30 00:40:03 - SimpleLog - INFO: - episode: 1260, ep_reward: -104.379, ep_step: 69 2023-05-30 00:40:04 - SimpleLog - INFO: - episode: 1261, ep_reward: -108.550, ep_step: 105 2023-05-30 00:40:04 - SimpleLog - INFO: - episode: 1262, ep_reward: -128.362, ep_step: 328 2023-05-30 00:40:05 - SimpleLog - INFO: - episode: 1263, ep_reward: -123.182, ep_step: 248 2023-05-30 00:40:07 - SimpleLog - INFO: - episode: 1264, ep_reward: -71.761, ep_step: 1000 2023-05-30 00:40:10 - SimpleLog - INFO: - episode: 1265, ep_reward: -68.557, ep_step: 1000 2023-05-30 00:40:10 - SimpleLog - INFO: - episode: 1266, ep_reward: -105.366, ep_step: 102 2023-05-30 00:40:10 - SimpleLog - INFO: - episode: 1267, ep_reward: -109.191, ep_step: 164 2023-05-30 00:40:10 - SimpleLog - INFO: - episode: 1268, ep_reward: -106.826, ep_step: 111 2023-05-30 00:40:11 - SimpleLog - INFO: - episode: 1269, ep_reward: -106.130, ep_step: 106 2023-05-30 00:40:11 - SimpleLog - INFO: - episode: 1270, ep_reward: -102.886, ep_step: 109 2023-05-30 00:40:11 - SimpleLog - INFO: - episode: 1271, ep_reward: -101.914, ep_step: 90 2023-05-30 00:40:11 - SimpleLog - INFO: - episode: 1272, ep_reward: -104.502, ep_step: 96 2023-05-30 00:40:14 - SimpleLog - INFO: - episode: 1273, ep_reward: -88.199, ep_step: 1000 2023-05-30 00:40:14 - SimpleLog - INFO: - episode: 1274, ep_reward: -152.904, ep_step: 361 2023-05-30 00:40:15 - SimpleLog - INFO: - episode: 1275, ep_reward: -156.781, ep_step: 454 2023-05-30 00:40:18 - SimpleLog - INFO: - episode: 1276, ep_reward: 0.435, ep_step: 1000 2023-05-30 00:40:19 - SimpleLog - INFO: - episode: 1277, ep_reward: -140.742, ep_step: 559 2023-05-30 00:40:21 - SimpleLog - INFO: - episode: 1278, ep_reward: -67.616, ep_step: 1000 2023-05-30 00:40:24 - SimpleLog - INFO: - episode: 1279, ep_reward: -65.185, ep_step: 1000 2023-05-30 00:40:26 - SimpleLog - INFO: - episode: 1280, ep_reward: -60.615, ep_step: 1000 2023-05-30 00:40:28 - SimpleLog - INFO: - episode: 1281, ep_reward: -56.487, ep_step: 1000 2023-05-30 00:40:30 - SimpleLog - INFO: - episode: 1282, ep_reward: -57.272, ep_step: 1000 2023-05-30 00:40:33 - SimpleLog - INFO: - episode: 1283, ep_reward: -55.495, ep_step: 1000 2023-05-30 00:40:35 - SimpleLog - INFO: - episode: 1284, ep_reward: -50.715, ep_step: 1000 2023-05-30 00:40:35 - SimpleLog - INFO: - episode: 1285, ep_reward: -121.847, ep_step: 48 2023-05-30 00:40:35 - SimpleLog - INFO: - episode: 1286, ep_reward: -114.014, ep_step: 50 2023-05-30 00:40:38 - SimpleLog - INFO: - episode: 1287, ep_reward: -52.222, ep_step: 1000 2023-05-30 00:40:40 - SimpleLog - INFO: - episode: 1288, ep_reward: -47.340, ep_step: 1000 2023-05-30 00:40:42 - SimpleLog - INFO: - episode: 1289, ep_reward: -47.571, ep_step: 1000 2023-05-30 00:40:44 - SimpleLog - INFO: - episode: 1290, ep_reward: -47.433, ep_step: 1000 2023-05-30 00:40:47 - SimpleLog - INFO: - episode: 1291, ep_reward: -46.689, ep_step: 1000 2023-05-30 00:40:49 - SimpleLog - INFO: - episode: 1292, ep_reward: -45.055, ep_step: 1000 2023-05-30 00:40:51 - SimpleLog - INFO: - episode: 1293, ep_reward: -44.915, ep_step: 1000 2023-05-30 00:40:53 - SimpleLog - INFO: - episode: 1294, ep_reward: -43.745, ep_step: 1000 2023-05-30 00:40:56 - SimpleLog - INFO: - episode: 1295, ep_reward: -47.893, ep_step: 1000 2023-05-30 00:40:58 - SimpleLog - INFO: - episode: 1296, ep_reward: -50.427, ep_step: 1000 2023-05-30 00:41:00 - SimpleLog - INFO: - episode: 1297, ep_reward: -51.273, ep_step: 1000 2023-05-30 00:41:02 - SimpleLog - INFO: - episode: 1298, ep_reward: -51.685, ep_step: 1000 2023-05-30 00:41:05 - SimpleLog - INFO: - episode: 1299, ep_reward: -51.469, ep_step: 1000 2023-05-30 00:41:07 - SimpleLog - INFO: - episode: 1300, ep_reward: -49.912, ep_step: 1000 2023-05-30 00:41:09 - SimpleLog - INFO: - episode: 1301, ep_reward: -13.092, ep_step: 1000 2023-05-30 00:41:12 - SimpleLog - INFO: - episode: 1302, ep_reward: -21.864, ep_step: 1000 2023-05-30 00:41:14 - SimpleLog - INFO: - episode: 1303, ep_reward: -47.464, ep_step: 1000 2023-05-30 00:41:16 - SimpleLog - INFO: - episode: 1304, ep_reward: -26.635, ep_step: 1000 2023-05-30 00:41:18 - SimpleLog - INFO: - episode: 1305, ep_reward: -42.646, ep_step: 1000 2023-05-30 00:41:21 - SimpleLog - INFO: - episode: 1306, ep_reward: -16.107, ep_step: 1000 2023-05-30 00:41:21 - SimpleLog - INFO: - episode: 1307, ep_reward: -130.422, ep_step: 196 2023-05-30 00:41:23 - SimpleLog - INFO: - episode: 1308, ep_reward: -46.884, ep_step: 1000 2023-05-30 00:41:26 - SimpleLog - INFO: - episode: 1309, ep_reward: -28.104, ep_step: 1000 2023-05-30 00:41:28 - SimpleLog - INFO: - episode: 1310, ep_reward: -44.283, ep_step: 1000 2023-05-30 00:41:30 - SimpleLog - INFO: - episode: 1311, ep_reward: -3.246, ep_step: 1000 2023-05-30 00:41:32 - SimpleLog - INFO: - episode: 1312, ep_reward: -46.874, ep_step: 1000 2023-05-30 00:41:35 - SimpleLog - INFO: - episode: 1313, ep_reward: -42.505, ep_step: 1000 2023-05-30 00:41:37 - SimpleLog - INFO: - episode: 1314, ep_reward: -22.974, ep_step: 1000 2023-05-30 00:41:41 - SimpleLog - INFO: - update_step: 840000, online_eval_reward: -13.696 2023-05-30 00:41:41 - SimpleLog - INFO: - episode: 1315, ep_reward: -44.403, ep_step: 1000 2023-05-30 00:41:43 - SimpleLog - INFO: - episode: 1316, ep_reward: -20.810, ep_step: 1000 2023-05-30 00:41:46 - SimpleLog - INFO: - episode: 1317, ep_reward: -41.984, ep_step: 1000 2023-05-30 00:41:48 - SimpleLog - INFO: - episode: 1318, ep_reward: -42.175, ep_step: 1000 2023-05-30 00:41:50 - SimpleLog - INFO: - episode: 1319, ep_reward: -41.827, ep_step: 1000 2023-05-30 00:41:52 - SimpleLog - INFO: - episode: 1320, ep_reward: -45.354, ep_step: 1000 2023-05-30 00:41:55 - SimpleLog - INFO: - episode: 1321, ep_reward: -42.648, ep_step: 1000 2023-05-30 00:41:57 - SimpleLog - INFO: - episode: 1322, ep_reward: -41.762, ep_step: 1000 2023-05-30 00:41:59 - SimpleLog - INFO: - episode: 1323, ep_reward: -45.375, ep_step: 1000 2023-05-30 00:42:01 - SimpleLog - INFO: - episode: 1324, ep_reward: -43.999, ep_step: 1000 2023-05-30 00:42:04 - SimpleLog - INFO: - episode: 1325, ep_reward: -38.560, ep_step: 1000 2023-05-30 00:42:06 - SimpleLog - INFO: - episode: 1326, ep_reward: -2.208, ep_step: 1000 2023-05-30 00:42:08 - SimpleLog - INFO: - episode: 1327, ep_reward: -42.878, ep_step: 1000 2023-05-30 00:42:11 - SimpleLog - INFO: - episode: 1328, ep_reward: -43.014, ep_step: 1000 2023-05-30 00:42:12 - SimpleLog - INFO: - episode: 1329, ep_reward: -101.655, ep_step: 484 2023-05-30 00:42:14 - SimpleLog - INFO: - episode: 1330, ep_reward: -46.344, ep_step: 1000 2023-05-30 00:42:16 - SimpleLog - INFO: - episode: 1331, ep_reward: -46.461, ep_step: 1000 2023-05-30 00:42:19 - SimpleLog - INFO: - episode: 1332, ep_reward: -47.024, ep_step: 1000 2023-05-30 00:42:21 - SimpleLog - INFO: - episode: 1333, ep_reward: -51.257, ep_step: 1000 2023-05-30 00:42:23 - SimpleLog - INFO: - episode: 1334, ep_reward: -50.204, ep_step: 1000 2023-05-30 00:42:26 - SimpleLog - INFO: - episode: 1335, ep_reward: -4.482, ep_step: 1000 2023-05-30 00:42:28 - SimpleLog - INFO: - episode: 1336, ep_reward: -49.547, ep_step: 1000 2023-05-30 00:42:30 - SimpleLog - INFO: - episode: 1337, ep_reward: -47.930, ep_step: 1000 2023-05-30 00:42:31 - SimpleLog - INFO: - episode: 1338, ep_reward: -113.529, ep_step: 235 2023-05-30 00:42:33 - SimpleLog - INFO: - episode: 1339, ep_reward: -40.664, ep_step: 1000 2023-05-30 00:42:35 - SimpleLog - INFO: - episode: 1340, ep_reward: -41.889, ep_step: 1000 2023-05-30 00:42:38 - SimpleLog - INFO: - episode: 1341, ep_reward: -47.768, ep_step: 1000 2023-05-30 00:42:39 - SimpleLog - INFO: - episode: 1342, ep_reward: -115.412, ep_step: 850 2023-05-30 00:42:42 - SimpleLog - INFO: - episode: 1343, ep_reward: -41.674, ep_step: 1000 2023-05-30 00:42:43 - SimpleLog - INFO: - episode: 1344, ep_reward: -101.392, ep_step: 644 2023-05-30 00:42:46 - SimpleLog - INFO: - episode: 1345, ep_reward: -33.455, ep_step: 1000 2023-05-30 00:42:48 - SimpleLog - INFO: - episode: 1346, ep_reward: -31.322, ep_step: 1000 2023-05-30 00:42:50 - SimpleLog - INFO: - episode: 1347, ep_reward: -31.101, ep_step: 1000 2023-05-30 00:42:52 - SimpleLog - INFO: - episode: 1348, ep_reward: -30.175, ep_step: 1000 2023-05-30 00:42:55 - SimpleLog - INFO: - episode: 1349, ep_reward: -29.185, ep_step: 1000 2023-05-30 00:42:57 - SimpleLog - INFO: - episode: 1350, ep_reward: -30.519, ep_step: 1000 2023-05-30 00:42:59 - SimpleLog - INFO: - episode: 1351, ep_reward: -28.327, ep_step: 1000 2023-05-30 00:43:00 - SimpleLog - INFO: - episode: 1352, ep_reward: -136.808, ep_step: 140 2023-05-30 00:43:02 - SimpleLog - INFO: - episode: 1353, ep_reward: -28.659, ep_step: 1000 2023-05-30 00:43:02 - SimpleLog - INFO: - episode: 1354, ep_reward: -140.543, ep_step: 124 2023-05-30 00:43:04 - SimpleLog - INFO: - episode: 1355, ep_reward: -31.223, ep_step: 1000 2023-05-30 00:43:07 - SimpleLog - INFO: - episode: 1356, ep_reward: -30.084, ep_step: 1000 2023-05-30 00:43:09 - SimpleLog - INFO: - episode: 1357, ep_reward: -29.579, ep_step: 1000 2023-05-30 00:43:11 - SimpleLog - INFO: - episode: 1358, ep_reward: -31.126, ep_step: 1000 2023-05-30 00:43:13 - SimpleLog - INFO: - episode: 1359, ep_reward: -29.092, ep_step: 1000 2023-05-30 00:43:16 - SimpleLog - INFO: - episode: 1360, ep_reward: -28.496, ep_step: 1000 2023-05-30 00:43:18 - SimpleLog - INFO: - episode: 1361, ep_reward: -27.957, ep_step: 1000 2023-05-30 00:43:20 - SimpleLog - INFO: - episode: 1362, ep_reward: -28.268, ep_step: 1000 2023-05-30 00:43:23 - SimpleLog - INFO: - episode: 1363, ep_reward: -28.367, ep_step: 1000 2023-05-30 00:43:25 - SimpleLog - INFO: - episode: 1364, ep_reward: -25.940, ep_step: 1000 2023-05-30 00:43:27 - SimpleLog - INFO: - episode: 1365, ep_reward: -27.365, ep_step: 1000 2023-05-30 00:43:29 - SimpleLog - INFO: - episode: 1366, ep_reward: -26.661, ep_step: 1000 2023-05-30 00:43:32 - SimpleLog - INFO: - episode: 1367, ep_reward: -28.506, ep_step: 1000 2023-05-30 00:43:34 - SimpleLog - INFO: - episode: 1368, ep_reward: -27.136, ep_step: 1000 2023-05-30 00:43:36 - SimpleLog - INFO: - episode: 1369, ep_reward: -29.147, ep_step: 1000 2023-05-30 00:43:39 - SimpleLog - INFO: - episode: 1370, ep_reward: -26.968, ep_step: 1000 2023-05-30 00:43:41 - SimpleLog - INFO: - episode: 1371, ep_reward: 5.236, ep_step: 1000 2023-05-30 00:43:43 - SimpleLog - INFO: - episode: 1372, ep_reward: -26.371, ep_step: 1000 2023-05-30 00:43:45 - SimpleLog - INFO: - episode: 1373, ep_reward: -28.073, ep_step: 1000 2023-05-30 00:43:48 - SimpleLog - INFO: - episode: 1374, ep_reward: -27.332, ep_step: 1000 2023-05-30 00:43:50 - SimpleLog - INFO: - episode: 1375, ep_reward: -27.185, ep_step: 1000 2023-05-30 00:43:52 - SimpleLog - INFO: - episode: 1376, ep_reward: -12.950, ep_step: 1000 2023-05-30 00:43:55 - SimpleLog - INFO: - episode: 1377, ep_reward: -8.148, ep_step: 1000 2023-05-30 00:43:57 - SimpleLog - INFO: - episode: 1378, ep_reward: -26.097, ep_step: 1000 2023-05-30 00:43:59 - SimpleLog - INFO: - episode: 1379, ep_reward: -26.467, ep_step: 1000 2023-05-30 00:44:02 - SimpleLog - INFO: - episode: 1380, ep_reward: -24.976, ep_step: 1000 2023-05-30 00:44:04 - SimpleLog - INFO: - episode: 1381, ep_reward: -25.882, ep_step: 1000 2023-05-30 00:44:06 - SimpleLog - INFO: - episode: 1382, ep_reward: -24.421, ep_step: 1000 2023-05-30 00:44:09 - SimpleLog - INFO: - episode: 1383, ep_reward: -25.913, ep_step: 1000 2023-05-30 00:44:11 - SimpleLog - INFO: - episode: 1384, ep_reward: -25.080, ep_step: 1000 2023-05-30 00:44:13 - SimpleLog - INFO: - episode: 1385, ep_reward: -26.474, ep_step: 1000 2023-05-30 00:44:16 - SimpleLog - INFO: - episode: 1386, ep_reward: -26.759, ep_step: 1000 2023-05-30 00:44:18 - SimpleLog - INFO: - episode: 1387, ep_reward: -26.496, ep_step: 1000 2023-05-30 00:44:20 - SimpleLog - INFO: - episode: 1388, ep_reward: -26.192, ep_step: 1000 2023-05-30 00:44:21 - SimpleLog - INFO: - update_step: 910000, online_eval_reward: -107.262 2023-05-30 00:44:23 - SimpleLog - INFO: - episode: 1389, ep_reward: -27.073, ep_step: 1000 2023-05-30 00:44:23 - SimpleLog - INFO: - episode: 1390, ep_reward: -101.416, ep_step: 74 2023-05-30 00:44:25 - SimpleLog - INFO: - episode: 1391, ep_reward: -28.568, ep_step: 1000 2023-05-30 00:44:27 - SimpleLog - INFO: - episode: 1392, ep_reward: -26.661, ep_step: 1000 2023-05-30 00:44:28 - SimpleLog - INFO: - episode: 1393, ep_reward: -118.550, ep_step: 91 2023-05-30 00:44:28 - SimpleLog - INFO: - episode: 1394, ep_reward: -103.001, ep_step: 72 2023-05-30 00:44:28 - SimpleLog - INFO: - episode: 1395, ep_reward: -112.572, ep_step: 100 2023-05-30 00:44:28 - SimpleLog - INFO: - episode: 1396, ep_reward: -101.256, ep_step: 76 2023-05-30 00:44:28 - SimpleLog - INFO: - episode: 1397, ep_reward: -101.369, ep_step: 72 2023-05-30 00:44:28 - SimpleLog - INFO: - episode: 1398, ep_reward: -104.425, ep_step: 51 2023-05-30 00:44:29 - SimpleLog - INFO: - episode: 1399, ep_reward: -101.454, ep_step: 94 2023-05-30 00:44:29 - SimpleLog - INFO: - episode: 1400, ep_reward: -105.497, ep_step: 58 2023-05-30 00:44:29 - SimpleLog - INFO: - episode: 1401, ep_reward: -102.961, ep_step: 58 2023-05-30 00:44:29 - SimpleLog - INFO: - episode: 1402, ep_reward: -103.518, ep_step: 91 2023-05-30 00:44:29 - SimpleLog - INFO: - episode: 1403, ep_reward: -103.259, ep_step: 67 2023-05-30 00:44:29 - SimpleLog - INFO: - episode: 1404, ep_reward: -105.593, ep_step: 46 2023-05-30 00:44:30 - SimpleLog - INFO: - episode: 1405, ep_reward: -103.627, ep_step: 49 2023-05-30 00:44:30 - SimpleLog - INFO: - episode: 1406, ep_reward: -104.489, ep_step: 55 2023-05-30 00:44:30 - SimpleLog - INFO: - episode: 1407, ep_reward: -105.604, ep_step: 66 2023-05-30 00:44:30 - SimpleLog - INFO: - episode: 1408, ep_reward: -106.525, ep_step: 45 2023-05-30 00:44:30 - SimpleLog - INFO: - episode: 1409, ep_reward: -104.284, ep_step: 58 2023-05-30 00:44:30 - SimpleLog - INFO: - episode: 1410, ep_reward: -103.969, ep_step: 60 2023-05-30 00:44:30 - SimpleLog - INFO: - episode: 1411, ep_reward: -103.542, ep_step: 88 2023-05-30 00:44:31 - SimpleLog - INFO: - episode: 1412, ep_reward: -104.553, ep_step: 64 2023-05-30 00:44:33 - SimpleLog - INFO: - episode: 1413, ep_reward: -74.798, ep_step: 1000 2023-05-30 00:44:35 - SimpleLog - INFO: - episode: 1414, ep_reward: -74.121, ep_step: 1000 2023-05-30 00:44:35 - SimpleLog - INFO: - episode: 1415, ep_reward: -106.444, ep_step: 55 2023-05-30 00:44:35 - SimpleLog - INFO: - episode: 1416, ep_reward: -105.273, ep_step: 65 2023-05-30 00:44:35 - SimpleLog - INFO: - episode: 1417, ep_reward: -106.623, ep_step: 53 2023-05-30 00:44:36 - SimpleLog - INFO: - episode: 1418, ep_reward: -104.938, ep_step: 78 2023-05-30 00:44:36 - SimpleLog - INFO: - episode: 1419, ep_reward: -103.639, ep_step: 55 2023-05-30 00:44:36 - SimpleLog - INFO: - episode: 1420, ep_reward: -101.391, ep_step: 77 2023-05-30 00:44:36 - SimpleLog - INFO: - episode: 1421, ep_reward: -104.067, ep_step: 51 2023-05-30 00:44:36 - SimpleLog - INFO: - episode: 1422, ep_reward: -104.469, ep_step: 61 2023-05-30 00:44:37 - SimpleLog - INFO: - episode: 1423, ep_reward: -129.327, ep_step: 361 2023-05-30 00:44:37 - SimpleLog - INFO: - episode: 1424, ep_reward: -103.569, ep_step: 65 2023-05-30 00:44:37 - SimpleLog - INFO: - episode: 1425, ep_reward: -104.287, ep_step: 64 2023-05-30 00:44:37 - SimpleLog - INFO: - episode: 1426, ep_reward: -104.113, ep_step: 60 2023-05-30 00:44:38 - SimpleLog - INFO: - episode: 1427, ep_reward: -104.574, ep_step: 62 2023-05-30 00:44:38 - SimpleLog - INFO: - episode: 1428, ep_reward: -104.251, ep_step: 66 2023-05-30 00:44:38 - SimpleLog - INFO: - episode: 1429, ep_reward: -103.970, ep_step: 58 2023-05-30 00:44:38 - SimpleLog - INFO: - episode: 1430, ep_reward: -106.341, ep_step: 66 2023-05-30 00:44:38 - SimpleLog - INFO: - episode: 1431, ep_reward: -103.394, ep_step: 59 2023-05-30 00:44:38 - SimpleLog - INFO: - episode: 1432, ep_reward: -104.123, ep_step: 47 2023-05-30 00:44:38 - SimpleLog - INFO: - episode: 1433, ep_reward: -102.899, ep_step: 55 2023-05-30 00:44:39 - SimpleLog - INFO: - episode: 1434, ep_reward: -103.251, ep_step: 55 2023-05-30 00:44:39 - SimpleLog - INFO: - episode: 1435, ep_reward: -104.325, ep_step: 64 2023-05-30 00:44:39 - SimpleLog - INFO: - episode: 1436, ep_reward: -104.135, ep_step: 56 2023-05-30 00:44:39 - SimpleLog - INFO: - episode: 1437, ep_reward: -102.942, ep_step: 54 2023-05-30 00:44:39 - SimpleLog - INFO: - episode: 1438, ep_reward: -104.157, ep_step: 61 2023-05-30 00:44:39 - SimpleLog - INFO: - episode: 1439, ep_reward: -103.495, ep_step: 56 2023-05-30 00:44:39 - SimpleLog - INFO: - episode: 1440, ep_reward: -104.573, ep_step: 62 2023-05-30 00:44:40 - SimpleLog - INFO: - episode: 1441, ep_reward: -103.314, ep_step: 57 2023-05-30 00:44:40 - SimpleLog - INFO: - episode: 1442, ep_reward: -104.325, ep_step: 59 2023-05-30 00:44:40 - SimpleLog - INFO: - episode: 1443, ep_reward: -103.868, ep_step: 61 2023-05-30 00:44:40 - SimpleLog - INFO: - episode: 1444, ep_reward: -104.329, ep_step: 56 2023-05-30 00:44:40 - SimpleLog - INFO: - episode: 1445, ep_reward: -103.368, ep_step: 57 2023-05-30 00:44:40 - SimpleLog - INFO: - episode: 1446, ep_reward: -103.168, ep_step: 57 2023-05-30 00:44:40 - SimpleLog - INFO: - episode: 1447, ep_reward: -103.901, ep_step: 55 2023-05-30 00:44:40 - SimpleLog - INFO: - episode: 1448, ep_reward: -103.468, ep_step: 52 2023-05-30 00:44:41 - SimpleLog - INFO: - episode: 1449, ep_reward: -103.946, ep_step: 56 2023-05-30 00:44:41 - SimpleLog - INFO: - episode: 1450, ep_reward: -103.173, ep_step: 50 2023-05-30 00:44:41 - SimpleLog - INFO: - episode: 1451, ep_reward: -103.452, ep_step: 54 2023-05-30 00:44:41 - SimpleLog - INFO: - episode: 1452, ep_reward: -103.117, ep_step: 50 2023-05-30 00:44:41 - SimpleLog - INFO: - episode: 1453, ep_reward: -103.613, ep_step: 53 2023-05-30 00:44:41 - SimpleLog - INFO: - episode: 1454, ep_reward: -104.014, ep_step: 57 2023-05-30 00:44:41 - SimpleLog - INFO: - episode: 1455, ep_reward: -104.135, ep_step: 55 2023-05-30 00:44:41 - SimpleLog - INFO: - episode: 1456, ep_reward: -104.119, ep_step: 55 2023-05-30 00:44:42 - SimpleLog - INFO: - episode: 1457, ep_reward: -103.202, ep_step: 50 2023-05-30 00:44:42 - SimpleLog - INFO: - episode: 1458, ep_reward: -102.555, ep_step: 61 2023-05-30 00:44:42 - SimpleLog - INFO: - episode: 1459, ep_reward: -103.245, ep_step: 55 2023-05-30 00:44:42 - SimpleLog - INFO: - episode: 1460, ep_reward: -103.742, ep_step: 57 2023-05-30 00:44:42 - SimpleLog - INFO: - episode: 1461, ep_reward: -102.949, ep_step: 47 2023-05-30 00:44:42 - SimpleLog - INFO: - episode: 1462, ep_reward: -102.936, ep_step: 54 2023-05-30 00:44:42 - SimpleLog - INFO: - episode: 1463, ep_reward: -104.228, ep_step: 63 2023-05-30 00:44:43 - SimpleLog - INFO: - episode: 1464, ep_reward: -103.827, ep_step: 58 2023-05-30 00:44:43 - SimpleLog - INFO: - episode: 1465, ep_reward: -103.160, ep_step: 54 2023-05-30 00:44:43 - SimpleLog - INFO: - episode: 1466, ep_reward: -103.276, ep_step: 48 2023-05-30 00:44:43 - SimpleLog - INFO: - episode: 1467, ep_reward: -103.191, ep_step: 69 2023-05-30 00:44:43 - SimpleLog - INFO: - episode: 1468, ep_reward: -103.698, ep_step: 54 2023-05-30 00:44:43 - SimpleLog - INFO: - episode: 1469, ep_reward: -104.914, ep_step: 89 2023-05-30 00:44:43 - SimpleLog - INFO: - episode: 1470, ep_reward: -103.341, ep_step: 46 2023-05-30 00:44:43 - SimpleLog - INFO: - episode: 1471, ep_reward: -103.598, ep_step: 63 2023-05-30 00:44:44 - SimpleLog - INFO: - episode: 1472, ep_reward: -103.971, ep_step: 63 2023-05-30 00:44:44 - SimpleLog - INFO: - episode: 1473, ep_reward: -103.175, ep_step: 50 2023-05-30 00:44:44 - SimpleLog - INFO: - episode: 1474, ep_reward: -103.376, ep_step: 49 2023-05-30 00:44:44 - SimpleLog - INFO: - episode: 1475, ep_reward: -103.317, ep_step: 49 2023-05-30 00:44:44 - SimpleLog - INFO: - episode: 1476, ep_reward: -103.344, ep_step: 45 2023-05-30 00:44:44 - SimpleLog - INFO: - episode: 1477, ep_reward: -102.847, ep_step: 47 2023-05-30 00:44:44 - SimpleLog - INFO: - episode: 1478, ep_reward: -102.871, ep_step: 47 2023-05-30 00:44:44 - SimpleLog - INFO: - episode: 1479, ep_reward: -103.281, ep_step: 57 2023-05-30 00:44:45 - SimpleLog - INFO: - episode: 1480, ep_reward: -102.704, ep_step: 52 2023-05-30 00:44:45 - SimpleLog - INFO: - episode: 1481, ep_reward: -103.745, ep_step: 73 2023-05-30 00:44:45 - SimpleLog - INFO: - episode: 1482, ep_reward: -102.980, ep_step: 45 2023-05-30 00:44:45 - SimpleLog - INFO: - episode: 1483, ep_reward: -103.200, ep_step: 54 2023-05-30 00:44:45 - SimpleLog - INFO: - episode: 1484, ep_reward: -102.827, ep_step: 52 2023-05-30 00:44:45 - SimpleLog - INFO: - episode: 1485, ep_reward: -104.153, ep_step: 73 2023-05-30 00:44:45 - SimpleLog - INFO: - episode: 1486, ep_reward: -103.897, ep_step: 76 2023-05-30 00:44:46 - SimpleLog - INFO: - episode: 1487, ep_reward: -104.320, ep_step: 80 2023-05-30 00:44:46 - SimpleLog - INFO: - episode: 1488, ep_reward: -103.806, ep_step: 67 2023-05-30 00:44:46 - SimpleLog - INFO: - episode: 1489, ep_reward: -100.108, ep_step: 85 2023-05-30 00:44:46 - SimpleLog - INFO: - episode: 1490, ep_reward: -103.446, ep_step: 82 2023-05-30 00:44:46 - SimpleLog - INFO: - episode: 1491, ep_reward: -105.353, ep_step: 107 2023-05-30 00:44:47 - SimpleLog - INFO: - episode: 1492, ep_reward: -101.780, ep_step: 89 2023-05-30 00:44:47 - SimpleLog - INFO: - episode: 1493, ep_reward: -116.600, ep_step: 218 2023-05-30 00:44:47 - SimpleLog - INFO: - episode: 1494, ep_reward: -100.880, ep_step: 84 2023-05-30 00:44:48 - SimpleLog - INFO: - episode: 1495, ep_reward: -103.258, ep_step: 69 2023-05-30 00:44:48 - SimpleLog - INFO: - episode: 1496, ep_reward: -111.892, ep_step: 179 2023-05-30 00:44:48 - SimpleLog - INFO: - episode: 1497, ep_reward: -103.246, ep_step: 55 2023-05-30 00:44:48 - SimpleLog - INFO: - episode: 1498, ep_reward: -103.151, ep_step: 46 2023-05-30 00:44:48 - SimpleLog - INFO: - episode: 1499, ep_reward: -103.312, ep_step: 62 2023-05-30 00:44:48 - SimpleLog - INFO: - episode: 1500, ep_reward: -103.172, ep_step: 61 2023-05-30 00:44:49 - SimpleLog - INFO: - episode: 1501, ep_reward: -103.685, ep_step: 63 2023-05-30 00:44:49 - SimpleLog - INFO: - episode: 1502, ep_reward: -103.550, ep_step: 68 2023-05-30 00:44:49 - SimpleLog - INFO: - episode: 1503, ep_reward: -103.031, ep_step: 59 2023-05-30 00:44:49 - SimpleLog - INFO: - episode: 1504, ep_reward: -103.015, ep_step: 54 2023-05-30 00:44:49 - SimpleLog - INFO: - episode: 1505, ep_reward: -103.143, ep_step: 62 2023-05-30 00:44:49 - SimpleLog - INFO: - episode: 1506, ep_reward: -104.931, ep_step: 72 2023-05-30 00:44:50 - SimpleLog - INFO: - episode: 1507, ep_reward: -104.688, ep_step: 84 2023-05-30 00:44:50 - SimpleLog - INFO: - episode: 1508, ep_reward: -103.364, ep_step: 67 2023-05-30 00:44:50 - SimpleLog - INFO: - episode: 1509, ep_reward: -107.565, ep_step: 123 2023-05-30 00:44:50 - SimpleLog - INFO: - episode: 1510, ep_reward: -104.043, ep_step: 66 2023-05-30 00:44:50 - SimpleLog - INFO: - episode: 1511, ep_reward: -105.163, ep_step: 70 2023-05-30 00:44:50 - SimpleLog - INFO: - episode: 1512, ep_reward: -103.306, ep_step: 62 2023-05-30 00:44:51 - SimpleLog - INFO: - episode: 1513, ep_reward: -126.965, ep_step: 279 2023-05-30 00:44:51 - SimpleLog - INFO: - episode: 1514, ep_reward: -104.839, ep_step: 80 2023-05-30 00:44:51 - SimpleLog - INFO: - episode: 1515, ep_reward: -103.546, ep_step: 69 2023-05-30 00:44:52 - SimpleLog - INFO: - episode: 1516, ep_reward: -105.911, ep_step: 92 2023-05-30 00:44:52 - SimpleLog - INFO: - episode: 1517, ep_reward: -103.956, ep_step: 66 2023-05-30 00:44:52 - SimpleLog - INFO: - episode: 1518, ep_reward: -102.618, ep_step: 72 2023-05-30 00:44:52 - SimpleLog - INFO: - episode: 1519, ep_reward: -105.702, ep_step: 87 2023-05-30 00:44:52 - SimpleLog - INFO: - episode: 1520, ep_reward: -105.566, ep_step: 79 2023-05-30 00:44:53 - SimpleLog - INFO: - episode: 1521, ep_reward: -104.397, ep_step: 77 2023-05-30 00:44:53 - SimpleLog - INFO: - episode: 1522, ep_reward: -109.054, ep_step: 130 2023-05-30 00:44:53 - SimpleLog - INFO: - episode: 1523, ep_reward: -108.853, ep_step: 131 2023-05-30 00:44:53 - SimpleLog - INFO: - episode: 1524, ep_reward: -104.138, ep_step: 82 2023-05-30 00:44:53 - SimpleLog - INFO: - episode: 1525, ep_reward: -106.160, ep_step: 78 2023-05-30 00:44:54 - SimpleLog - INFO: - episode: 1526, ep_reward: -103.703, ep_step: 70 2023-05-30 00:44:54 - SimpleLog - INFO: - episode: 1527, ep_reward: -105.627, ep_step: 89 2023-05-30 00:44:54 - SimpleLog - INFO: - episode: 1528, ep_reward: -103.886, ep_step: 65 2023-05-30 00:44:54 - SimpleLog - INFO: - episode: 1529, ep_reward: -105.844, ep_step: 83 2023-05-30 00:44:54 - SimpleLog - INFO: - episode: 1530, ep_reward: -105.227, ep_step: 96 2023-05-30 00:44:57 - SimpleLog - INFO: - episode: 1531, ep_reward: -96.064, ep_step: 1000 2023-05-30 00:44:59 - SimpleLog - INFO: - episode: 1532, ep_reward: -95.412, ep_step: 1000 2023-05-30 00:45:01 - SimpleLog - INFO: - episode: 1533, ep_reward: -90.860, ep_step: 1000 2023-05-30 00:45:01 - SimpleLog - INFO: - episode: 1534, ep_reward: -102.699, ep_step: 92 2023-05-30 00:45:02 - SimpleLog - INFO: - episode: 1535, ep_reward: -104.841, ep_step: 84 2023-05-30 00:45:02 - SimpleLog - INFO: - episode: 1536, ep_reward: -105.648, ep_step: 106 2023-05-30 00:45:02 - SimpleLog - INFO: - episode: 1537, ep_reward: -104.333, ep_step: 81 2023-05-30 00:45:02 - SimpleLog - INFO: - episode: 1538, ep_reward: -103.258, ep_step: 89 2023-05-30 00:45:03 - SimpleLog - INFO: - episode: 1539, ep_reward: -119.349, ep_step: 282 2023-05-30 00:45:03 - SimpleLog - INFO: - episode: 1540, ep_reward: -103.023, ep_step: 95 2023-05-30 00:45:03 - SimpleLog - INFO: - episode: 1541, ep_reward: -104.529, ep_step: 95 2023-05-30 00:45:04 - SimpleLog - INFO: - episode: 1542, ep_reward: -102.189, ep_step: 98 2023-05-30 00:45:04 - SimpleLog - INFO: - episode: 1543, ep_reward: -103.875, ep_step: 92 2023-05-30 00:45:04 - SimpleLog - INFO: - episode: 1544, ep_reward: -102.472, ep_step: 96 2023-05-30 00:45:04 - SimpleLog - INFO: - episode: 1545, ep_reward: -103.053, ep_step: 118 2023-05-30 00:45:07 - SimpleLog - INFO: - episode: 1546, ep_reward: -72.710, ep_step: 1000 2023-05-30 00:45:07 - SimpleLog - INFO: - episode: 1547, ep_reward: -103.112, ep_step: 96 2023-05-30 00:45:07 - SimpleLog - INFO: - episode: 1548, ep_reward: -104.150, ep_step: 90 2023-05-30 00:45:09 - SimpleLog - INFO: - episode: 1549, ep_reward: -72.485, ep_step: 1000 2023-05-30 00:45:10 - SimpleLog - INFO: - episode: 1550, ep_reward: -105.789, ep_step: 282 2023-05-30 00:45:10 - SimpleLog - INFO: - episode: 1551, ep_reward: -103.876, ep_step: 103 2023-05-30 00:45:10 - SimpleLog - INFO: - episode: 1552, ep_reward: -106.379, ep_step: 79 2023-05-30 00:45:11 - SimpleLog - INFO: - episode: 1553, ep_reward: -106.813, ep_step: 79 2023-05-30 00:45:11 - SimpleLog - INFO: - episode: 1554, ep_reward: -102.246, ep_step: 98 2023-05-30 00:45:11 - SimpleLog - INFO: - episode: 1555, ep_reward: -102.642, ep_step: 94 2023-05-30 00:45:11 - SimpleLog - INFO: - episode: 1556, ep_reward: -106.460, ep_step: 120 2023-05-30 00:45:12 - SimpleLog - INFO: - episode: 1557, ep_reward: -102.100, ep_step: 105 2023-05-30 00:45:12 - SimpleLog - INFO: - episode: 1558, ep_reward: -100.221, ep_step: 147 2023-05-30 00:45:12 - SimpleLog - INFO: - episode: 1559, ep_reward: -103.810, ep_step: 115 2023-05-30 00:45:12 - SimpleLog - INFO: - episode: 1560, ep_reward: -102.904, ep_step: 134 2023-05-30 00:45:13 - SimpleLog - INFO: - episode: 1561, ep_reward: -100.072, ep_step: 103 2023-05-30 00:45:14 - SimpleLog - INFO: - episode: 1562, ep_reward: -113.913, ep_step: 354 2023-05-30 00:45:16 - SimpleLog - INFO: - episode: 1563, ep_reward: -87.991, ep_step: 1000 2023-05-30 00:45:16 - SimpleLog - INFO: - episode: 1564, ep_reward: -99.850, ep_step: 100 2023-05-30 00:45:16 - SimpleLog - INFO: - episode: 1565, ep_reward: -104.566, ep_step: 89 2023-05-30 00:45:17 - SimpleLog - INFO: - episode: 1566, ep_reward: -99.568, ep_step: 112 2023-05-30 00:45:17 - SimpleLog - INFO: - episode: 1567, ep_reward: -99.686, ep_step: 138 2023-05-30 00:45:17 - SimpleLog - INFO: - episode: 1568, ep_reward: -103.583, ep_step: 93 2023-05-30 00:45:17 - SimpleLog - INFO: - episode: 1569, ep_reward: -102.701, ep_step: 105 2023-05-30 00:45:18 - SimpleLog - INFO: - episode: 1570, ep_reward: -105.606, ep_step: 118 2023-05-30 00:45:18 - SimpleLog - INFO: - episode: 1571, ep_reward: -100.449, ep_step: 91 2023-05-30 00:45:18 - SimpleLog - INFO: - episode: 1572, ep_reward: -100.197, ep_step: 94 2023-05-30 00:45:18 - SimpleLog - INFO: - episode: 1573, ep_reward: -104.991, ep_step: 121 2023-05-30 00:45:21 - SimpleLog - INFO: - episode: 1574, ep_reward: -96.317, ep_step: 1000 2023-05-30 00:45:21 - SimpleLog - INFO: - episode: 1575, ep_reward: -102.208, ep_step: 78 2023-05-30 00:45:21 - SimpleLog - INFO: - episode: 1576, ep_reward: -97.954, ep_step: 102 2023-05-30 00:45:21 - SimpleLog - INFO: - episode: 1577, ep_reward: -95.386, ep_step: 102 2023-05-30 00:45:21 - SimpleLog - INFO: - episode: 1578, ep_reward: -95.079, ep_step: 84 2023-05-30 00:45:22 - SimpleLog - INFO: - episode: 1579, ep_reward: -98.759, ep_step: 92 2023-05-30 00:45:22 - SimpleLog - INFO: - episode: 1580, ep_reward: -96.037, ep_step: 73 2023-05-30 00:45:22 - SimpleLog - INFO: - episode: 1581, ep_reward: -96.057, ep_step: 71 2023-05-30 00:45:22 - SimpleLog - INFO: - episode: 1582, ep_reward: -101.777, ep_step: 53 2023-05-30 00:45:22 - SimpleLog - INFO: - episode: 1583, ep_reward: -100.462, ep_step: 76 2023-05-30 00:45:22 - SimpleLog - INFO: - episode: 1584, ep_reward: -96.745, ep_step: 65 2023-05-30 00:45:23 - SimpleLog - INFO: - episode: 1585, ep_reward: -105.699, ep_step: 80 2023-05-30 00:45:23 - SimpleLog - INFO: - episode: 1586, ep_reward: -101.700, ep_step: 70 2023-05-30 00:45:23 - SimpleLog - INFO: - episode: 1587, ep_reward: -98.473, ep_step: 98 2023-05-30 00:45:23 - SimpleLog - INFO: - episode: 1588, ep_reward: -104.735, ep_step: 105 2023-05-30 00:45:23 - SimpleLog - INFO: - episode: 1589, ep_reward: -93.875, ep_step: 112 2023-05-30 00:45:24 - SimpleLog - INFO: - episode: 1590, ep_reward: -106.476, ep_step: 91 2023-05-30 00:45:24 - SimpleLog - INFO: - episode: 1591, ep_reward: -104.719, ep_step: 81 2023-05-30 00:45:24 - SimpleLog - INFO: - episode: 1592, ep_reward: -99.014, ep_step: 105 2023-05-30 00:45:25 - SimpleLog - INFO: - episode: 1593, ep_reward: -115.653, ep_step: 386 2023-05-30 00:45:27 - SimpleLog - INFO: - episode: 1594, ep_reward: -58.182, ep_step: 1000 2023-05-30 00:45:30 - SimpleLog - INFO: - episode: 1595, ep_reward: -62.990, ep_step: 1000 2023-05-30 00:45:32 - SimpleLog - INFO: - episode: 1596, ep_reward: -51.634, ep_step: 1000 2023-05-30 00:45:34 - SimpleLog - INFO: - episode: 1597, ep_reward: -40.985, ep_step: 1000 2023-05-30 00:45:36 - SimpleLog - INFO: - episode: 1598, ep_reward: -41.954, ep_step: 1000 2023-05-30 00:45:39 - SimpleLog - INFO: - episode: 1599, ep_reward: -45.179, ep_step: 1000 2023-05-30 00:45:41 - SimpleLog - INFO: - episode: 1600, ep_reward: -43.753, ep_step: 1000 2023-05-30 00:45:43 - SimpleLog - INFO: - episode: 1601, ep_reward: -55.754, ep_step: 1000 2023-05-30 00:45:46 - SimpleLog - INFO: - episode: 1602, ep_reward: -59.373, ep_step: 1000 2023-05-30 00:45:48 - SimpleLog - INFO: - episode: 1603, ep_reward: -50.216, ep_step: 1000 2023-05-30 00:45:50 - SimpleLog - INFO: - episode: 1604, ep_reward: -50.014, ep_step: 1000 2023-05-30 00:45:52 - SimpleLog - INFO: - episode: 1605, ep_reward: -51.760, ep_step: 1000 2023-05-30 00:45:55 - SimpleLog - INFO: - episode: 1606, ep_reward: -49.364, ep_step: 1000 2023-05-30 00:45:57 - SimpleLog - INFO: - episode: 1607, ep_reward: -52.586, ep_step: 1000 2023-05-30 00:45:59 - SimpleLog - INFO: - episode: 1608, ep_reward: -55.846, ep_step: 1000 2023-05-30 00:46:02 - SimpleLog - INFO: - episode: 1609, ep_reward: -66.561, ep_step: 1000 2023-05-30 00:46:04 - SimpleLog - INFO: - episode: 1610, ep_reward: -68.759, ep_step: 1000 2023-05-30 00:46:06 - SimpleLog - INFO: - episode: 1611, ep_reward: -68.663, ep_step: 1000 2023-05-30 00:46:08 - SimpleLog - INFO: - episode: 1612, ep_reward: -70.109, ep_step: 1000 2023-05-30 00:46:09 - SimpleLog - INFO: - episode: 1613, ep_reward: -108.308, ep_step: 130 2023-05-30 00:46:09 - SimpleLog - INFO: - episode: 1614, ep_reward: -109.913, ep_step: 135 2023-05-30 00:46:11 - SimpleLog - INFO: - episode: 1615, ep_reward: -64.976, ep_step: 1000 2023-05-30 00:46:12 - SimpleLog - INFO: - episode: 1616, ep_reward: -107.806, ep_step: 151 2023-05-30 00:46:14 - SimpleLog - INFO: - episode: 1617, ep_reward: -133.821, ep_step: 831 2023-05-30 00:46:14 - SimpleLog - INFO: - episode: 1618, ep_reward: -143.170, ep_step: 383 2023-05-30 00:46:15 - SimpleLog - INFO: - episode: 1619, ep_reward: -127.502, ep_step: 364 2023-05-30 00:46:18 - SimpleLog - INFO: - episode: 1620, ep_reward: -41.164, ep_step: 1000 2023-05-30 00:46:20 - SimpleLog - INFO: - episode: 1621, ep_reward: -42.514, ep_step: 1000 2023-05-30 00:46:21 - SimpleLog - INFO: - episode: 1622, ep_reward: -124.728, ep_step: 660 2023-05-30 00:46:24 - SimpleLog - INFO: - episode: 1623, ep_reward: -25.731, ep_step: 1000 2023-05-30 00:46:26 - SimpleLog - INFO: - episode: 1624, ep_reward: -41.495, ep_step: 1000 2023-05-30 00:46:28 - SimpleLog - INFO: - episode: 1625, ep_reward: -41.904, ep_step: 1000 2023-05-30 00:46:31 - SimpleLog - INFO: - episode: 1626, ep_reward: -46.270, ep_step: 1000 2023-05-30 00:46:33 - SimpleLog - INFO: - episode: 1627, ep_reward: -44.703, ep_step: 1000 2023-05-30 00:46:35 - SimpleLog - INFO: - episode: 1628, ep_reward: -43.985, ep_step: 1000 2023-05-30 00:46:37 - SimpleLog - INFO: - episode: 1629, ep_reward: -45.094, ep_step: 1000 2023-05-30 00:46:40 - SimpleLog - INFO: - episode: 1630, ep_reward: -46.982, ep_step: 1000 2023-05-30 00:46:41 - SimpleLog - INFO: - episode: 1631, ep_reward: -123.614, ep_step: 563 2023-05-30 00:46:43 - SimpleLog - INFO: - episode: 1632, ep_reward: -47.943, ep_step: 1000 2023-05-30 00:46:46 - SimpleLog - INFO: - episode: 1633, ep_reward: -45.339, ep_step: 1000 2023-05-30 00:46:48 - SimpleLog - INFO: - episode: 1634, ep_reward: -44.472, ep_step: 1000 2023-05-30 00:46:50 - SimpleLog - INFO: - episode: 1635, ep_reward: -46.776, ep_step: 1000 2023-05-30 00:46:52 - SimpleLog - INFO: - episode: 1636, ep_reward: -45.581, ep_step: 1000 2023-05-30 00:46:55 - SimpleLog - INFO: - episode: 1637, ep_reward: -45.053, ep_step: 1000 2023-05-30 00:46:57 - SimpleLog - INFO: - episode: 1638, ep_reward: -50.623, ep_step: 1000 2023-05-30 00:46:59 - SimpleLog - INFO: - episode: 1639, ep_reward: -50.769, ep_step: 1000 2023-05-30 00:47:01 - SimpleLog - INFO: - episode: 1640, ep_reward: -50.087, ep_step: 1000 2023-05-30 00:47:04 - SimpleLog - INFO: - update_step: 980000, online_eval_reward: -51.255 2023-05-30 00:47:06 - SimpleLog - INFO: - episode: 1641, ep_reward: -49.769, ep_step: 1000 2023-05-30 00:47:08 - SimpleLog - INFO: - episode: 1642, ep_reward: -48.151, ep_step: 1000 2023-05-30 00:47:10 - SimpleLog - INFO: - episode: 1643, ep_reward: -46.346, ep_step: 1000 2023-05-30 00:47:13 - SimpleLog - INFO: - episode: 1644, ep_reward: -43.982, ep_step: 1000 2023-05-30 00:47:15 - SimpleLog - INFO: - episode: 1645, ep_reward: -44.460, ep_step: 1000 2023-05-30 00:47:17 - SimpleLog - INFO: - episode: 1646, ep_reward: -42.738, ep_step: 1000 2023-05-30 00:47:20 - SimpleLog - INFO: - episode: 1647, ep_reward: -42.356, ep_step: 1000 2023-05-30 00:47:22 - SimpleLog - INFO: - episode: 1648, ep_reward: -42.926, ep_step: 1000 2023-05-30 00:47:24 - SimpleLog - INFO: - episode: 1649, ep_reward: -42.639, ep_step: 1000 2023-05-30 00:47:26 - SimpleLog - INFO: - episode: 1650, ep_reward: -40.965, ep_step: 1000 2023-05-30 00:47:29 - SimpleLog - INFO: - episode: 1651, ep_reward: -39.991, ep_step: 1000 2023-05-30 00:47:31 - SimpleLog - INFO: - episode: 1652, ep_reward: -41.049, ep_step: 1000 2023-05-30 00:47:33 - SimpleLog - INFO: - episode: 1653, ep_reward: -40.350, ep_step: 1000 2023-05-30 00:47:35 - SimpleLog - INFO: - episode: 1654, ep_reward: -140.903, ep_step: 921 2023-05-30 00:47:38 - SimpleLog - INFO: - episode: 1655, ep_reward: -45.930, ep_step: 1000 2023-05-30 00:47:40 - SimpleLog - INFO: - episode: 1656, ep_reward: -141.888, ep_step: 883 2023-05-30 00:47:40 - SimpleLog - INFO: - episode: 1657, ep_reward: -113.367, ep_step: 341 2023-05-30 00:47:41 - SimpleLog - INFO: - episode: 1658, ep_reward: -107.251, ep_step: 296 2023-05-30 00:47:42 - SimpleLog - INFO: - episode: 1659, ep_reward: -95.490, ep_step: 243 2023-05-30 00:47:42 - SimpleLog - INFO: - episode: 1660, ep_reward: -105.120, ep_step: 255 2023-05-30 00:47:43 - SimpleLog - INFO: - episode: 1661, ep_reward: -113.412, ep_step: 431 2023-05-30 00:47:45 - SimpleLog - INFO: - episode: 1662, ep_reward: -5.901, ep_step: 1000 2023-05-30 00:47:47 - SimpleLog - INFO: - episode: 1663, ep_reward: -128.338, ep_step: 595 2023-05-30 00:47:49 - SimpleLog - INFO: - episode: 1664, ep_reward: -142.603, ep_step: 903 2023-05-30 00:47:51 - SimpleLog - INFO: - episode: 1665, ep_reward: -50.442, ep_step: 1000 2023-05-30 00:47:53 - SimpleLog - INFO: - episode: 1666, ep_reward: -45.265, ep_step: 1000 2023-05-30 00:47:56 - SimpleLog - INFO: - episode: 1667, ep_reward: -41.126, ep_step: 1000 2023-05-30 00:47:58 - SimpleLog - INFO: - episode: 1668, ep_reward: -129.967, ep_step: 880 2023-05-30 00:47:59 - SimpleLog - INFO: - episode: 1669, ep_reward: -103.342, ep_step: 456 2023-05-30 00:48:01 - SimpleLog - INFO: - episode: 1670, ep_reward: -136.584, ep_step: 799 2023-05-30 00:48:03 - SimpleLog - INFO: - episode: 1671, ep_reward: -111.412, ep_step: 918 2023-05-30 00:48:05 - SimpleLog - INFO: - episode: 1672, ep_reward: -39.062, ep_step: 1000 2023-05-30 00:48:07 - SimpleLog - INFO: - episode: 1673, ep_reward: -41.014, ep_step: 1000 2023-05-30 00:48:09 - SimpleLog - INFO: - episode: 1674, ep_reward: -42.316, ep_step: 1000 2023-05-30 00:48:10 - SimpleLog - INFO: - episode: 1675, ep_reward: -101.132, ep_step: 108 2023-05-30 00:48:12 - SimpleLog - INFO: - episode: 1676, ep_reward: -41.507, ep_step: 1000 2023-05-30 00:48:14 - SimpleLog - INFO: - episode: 1677, ep_reward: -43.423, ep_step: 1000 2023-05-30 00:48:17 - SimpleLog - INFO: - episode: 1678, ep_reward: -43.719, ep_step: 1000 2023-05-30 00:48:19 - SimpleLog - INFO: - episode: 1679, ep_reward: -42.720, ep_step: 1000 2023-05-30 00:48:21 - SimpleLog - INFO: - episode: 1680, ep_reward: -43.123, ep_step: 1000 2023-05-30 00:48:24 - SimpleLog - INFO: - episode: 1681, ep_reward: -42.542, ep_step: 1000 2023-05-30 00:48:26 - SimpleLog - INFO: - episode: 1682, ep_reward: -43.316, ep_step: 1000 2023-05-30 00:48:28 - SimpleLog - INFO: - episode: 1683, ep_reward: -41.320, ep_step: 1000 2023-05-30 00:48:30 - SimpleLog - INFO: - episode: 1684, ep_reward: -41.621, ep_step: 1000 2023-05-30 00:48:33 - SimpleLog - INFO: - episode: 1685, ep_reward: -39.654, ep_step: 1000 2023-05-30 00:48:35 - SimpleLog - INFO: - episode: 1686, ep_reward: -40.276, ep_step: 1000 2023-05-30 00:48:37 - SimpleLog - INFO: - episode: 1687, ep_reward: 2.293, ep_step: 1000 2023-05-30 00:48:40 - SimpleLog - INFO: - episode: 1688, ep_reward: -39.676, ep_step: 1000 2023-05-30 00:48:42 - SimpleLog - INFO: - episode: 1689, ep_reward: -40.504, ep_step: 1000 2023-05-30 00:48:44 - SimpleLog - INFO: - episode: 1690, ep_reward: -39.589, ep_step: 1000 2023-05-30 00:48:46 - SimpleLog - INFO: - episode: 1691, ep_reward: -38.121, ep_step: 1000 2023-05-30 00:48:49 - SimpleLog - INFO: - episode: 1692, ep_reward: -37.686, ep_step: 1000 2023-05-30 00:48:51 - SimpleLog - INFO: - episode: 1693, ep_reward: -39.229, ep_step: 1000 2023-05-30 00:48:51 - SimpleLog - INFO: - episode: 1694, ep_reward: -88.756, ep_step: 180 2023-05-30 00:48:54 - SimpleLog - INFO: - episode: 1695, ep_reward: -39.387, ep_step: 1000 2023-05-30 00:48:56 - SimpleLog - INFO: - episode: 1696, ep_reward: -37.889, ep_step: 1000 2023-05-30 00:48:58 - SimpleLog - INFO: - episode: 1697, ep_reward: -36.391, ep_step: 1000 2023-05-30 00:49:00 - SimpleLog - INFO: - episode: 1698, ep_reward: -40.527, ep_step: 1000 2023-05-30 00:49:03 - SimpleLog - INFO: - episode: 1699, ep_reward: -37.632, ep_step: 1000 2023-05-30 00:49:05 - SimpleLog - INFO: - episode: 1700, ep_reward: -38.647, ep_step: 1000 2023-05-30 00:49:07 - SimpleLog - INFO: - episode: 1701, ep_reward: -40.396, ep_step: 1000 2023-05-30 00:49:08 - SimpleLog - INFO: - episode: 1702, ep_reward: -101.671, ep_step: 209 2023-05-30 00:49:10 - SimpleLog - INFO: - episode: 1703, ep_reward: -40.350, ep_step: 1000 2023-05-30 00:49:12 - SimpleLog - INFO: - episode: 1704, ep_reward: -41.186, ep_step: 1000 2023-05-30 00:49:14 - SimpleLog - INFO: - episode: 1705, ep_reward: -42.079, ep_step: 1000 2023-05-30 00:49:17 - SimpleLog - INFO: - episode: 1706, ep_reward: -42.560, ep_step: 1000 2023-05-30 00:49:19 - SimpleLog - INFO: - episode: 1707, ep_reward: -40.140, ep_step: 1000 2023-05-30 00:49:21 - SimpleLog - INFO: - episode: 1708, ep_reward: -39.733, ep_step: 1000 2023-05-30 00:49:24 - SimpleLog - INFO: - episode: 1709, ep_reward: -38.450, ep_step: 1000 2023-05-30 00:49:26 - SimpleLog - INFO: - episode: 1710, ep_reward: -38.065, ep_step: 1000 2023-05-30 00:49:28 - SimpleLog - INFO: - episode: 1711, ep_reward: -38.042, ep_step: 1000 2023-05-30 00:49:30 - SimpleLog - INFO: - episode: 1712, ep_reward: -37.532, ep_step: 1000 2023-05-30 00:49:33 - SimpleLog - INFO: - episode: 1713, ep_reward: -35.578, ep_step: 1000 2023-05-30 00:49:35 - SimpleLog - INFO: - episode: 1714, ep_reward: -34.860, ep_step: 1000 2023-05-30 00:49:37 - SimpleLog - INFO: - episode: 1715, ep_reward: -103.241, ep_step: 654 2023-05-30 00:49:39 - SimpleLog - INFO: - episode: 1716, ep_reward: -36.308, ep_step: 1000 2023-05-30 00:49:41 - SimpleLog - INFO: - episode: 1717, ep_reward: -0.171, ep_step: 1000 2023-05-30 00:49:43 - SimpleLog - INFO: - episode: 1718, ep_reward: 1.766, ep_step: 1000 2023-05-30 00:49:46 - SimpleLog - INFO: - update_step: 1050000, online_eval_reward: -20.915 2023-05-30 00:49:48 - SimpleLog - INFO: - episode: 1719, ep_reward: -18.746, ep_step: 1000 2023-05-30 00:49:50 - SimpleLog - INFO: - episode: 1720, ep_reward: -17.087, ep_step: 1000 2023-05-30 00:49:52 - SimpleLog - INFO: - episode: 1721, ep_reward: 2.641, ep_step: 1000 2023-05-30 00:49:54 - SimpleLog - INFO: - episode: 1722, ep_reward: 20.508, ep_step: 1000 2023-05-30 00:49:57 - SimpleLog - INFO: - episode: 1723, ep_reward: -13.901, ep_step: 1000 2023-05-30 00:49:59 - SimpleLog - INFO: - episode: 1724, ep_reward: -0.407, ep_step: 1000 2023-05-30 00:50:01 - SimpleLog - INFO: - episode: 1725, ep_reward: 18.456, ep_step: 1000 2023-05-30 00:50:03 - SimpleLog - INFO: - episode: 1726, ep_reward: 17.026, ep_step: 1000 2023-05-30 00:50:06 - SimpleLog - INFO: - episode: 1727, ep_reward: 16.662, ep_step: 1000 2023-05-30 00:50:08 - SimpleLog - INFO: - episode: 1728, ep_reward: 6.197, ep_step: 1000 2023-05-30 00:50:10 - SimpleLog - INFO: - episode: 1729, ep_reward: 5.816, ep_step: 1000 2023-05-30 00:50:13 - SimpleLog - INFO: - episode: 1730, ep_reward: -33.211, ep_step: 1000 2023-05-30 00:50:15 - SimpleLog - INFO: - episode: 1731, ep_reward: -14.607, ep_step: 1000 2023-05-30 00:50:17 - SimpleLog - INFO: - episode: 1732, ep_reward: -32.758, ep_step: 1000 2023-05-30 00:50:19 - SimpleLog - INFO: - episode: 1733, ep_reward: -32.692, ep_step: 1000 2023-05-30 00:50:22 - SimpleLog - INFO: - episode: 1734, ep_reward: -30.709, ep_step: 1000 2023-05-30 00:50:24 - SimpleLog - INFO: - episode: 1735, ep_reward: 6.901, ep_step: 1000 2023-05-30 00:50:26 - SimpleLog - INFO: - episode: 1736, ep_reward: -32.015, ep_step: 1000 2023-05-30 00:50:28 - SimpleLog - INFO: - episode: 1737, ep_reward: 7.598, ep_step: 1000 2023-05-30 00:50:31 - SimpleLog - INFO: - episode: 1738, ep_reward: -31.129, ep_step: 1000 2023-05-30 00:50:33 - SimpleLog - INFO: - episode: 1739, ep_reward: 4.924, ep_step: 1000 2023-05-30 00:50:35 - SimpleLog - INFO: - episode: 1740, ep_reward: 6.483, ep_step: 1000 2023-05-30 00:50:38 - SimpleLog - INFO: - episode: 1741, ep_reward: 23.526, ep_step: 1000 2023-05-30 00:50:40 - SimpleLog - INFO: - episode: 1742, ep_reward: -80.795, ep_step: 986 2023-05-30 00:50:42 - SimpleLog - INFO: - episode: 1743, ep_reward: 19.675, ep_step: 1000 2023-05-30 00:50:44 - SimpleLog - INFO: - episode: 1744, ep_reward: 12.624, ep_step: 1000 2023-05-30 00:50:47 - SimpleLog - INFO: - episode: 1745, ep_reward: 21.714, ep_step: 1000 2023-05-30 00:50:49 - SimpleLog - INFO: - episode: 1746, ep_reward: 21.030, ep_step: 1000 2023-05-30 00:50:51 - SimpleLog - INFO: - episode: 1747, ep_reward: 24.213, ep_step: 1000 2023-05-30 00:50:53 - SimpleLog - INFO: - episode: 1748, ep_reward: 22.249, ep_step: 1000 2023-05-30 00:50:56 - SimpleLog - INFO: - episode: 1749, ep_reward: 25.506, ep_step: 1000 2023-05-30 00:50:58 - SimpleLog - INFO: - episode: 1750, ep_reward: 86.898, ep_step: 1000 2023-05-30 00:51:00 - SimpleLog - INFO: - episode: 1751, ep_reward: 24.055, ep_step: 1000 2023-05-30 00:51:03 - SimpleLog - INFO: - episode: 1752, ep_reward: 18.997, ep_step: 1000 2023-05-30 00:51:05 - SimpleLog - INFO: - episode: 1753, ep_reward: -3.521, ep_step: 1000 2023-05-30 00:51:05 - SimpleLog - INFO: - episode: 1754, ep_reward: -118.269, ep_step: 161 2023-05-30 00:51:07 - SimpleLog - INFO: - episode: 1755, ep_reward: 22.425, ep_step: 1000 2023-05-30 00:51:10 - SimpleLog - INFO: - episode: 1756, ep_reward: 24.952, ep_step: 1000 2023-05-30 00:51:12 - SimpleLog - INFO: - episode: 1757, ep_reward: 46.726, ep_step: 1000 2023-05-30 00:51:13 - SimpleLog - INFO: - episode: 1758, ep_reward: -148.711, ep_step: 553 2023-05-30 00:51:13 - SimpleLog - INFO: - episode: 1759, ep_reward: -110.113, ep_step: 76 2023-05-30 00:51:13 - SimpleLog - INFO: - episode: 1760, ep_reward: -108.465, ep_step: 57 2023-05-30 00:51:14 - SimpleLog - INFO: - episode: 1761, ep_reward: -107.259, ep_step: 45 2023-05-30 00:51:14 - SimpleLog - INFO: - episode: 1762, ep_reward: -108.984, ep_step: 50 2023-05-30 00:51:14 - SimpleLog - INFO: - episode: 1763, ep_reward: -101.127, ep_step: 74 2023-05-30 00:51:14 - SimpleLog - INFO: - episode: 1764, ep_reward: -102.195, ep_step: 95 2023-05-30 00:51:14 - SimpleLog - INFO: - episode: 1765, ep_reward: -99.455, ep_step: 72 2023-05-30 00:51:14 - SimpleLog - INFO: - episode: 1766, ep_reward: -105.003, ep_step: 82 2023-05-30 00:51:15 - SimpleLog - INFO: - episode: 1767, ep_reward: -110.372, ep_step: 87 2023-05-30 00:51:15 - SimpleLog - INFO: - episode: 1768, ep_reward: -107.385, ep_step: 282 2023-05-30 00:51:15 - SimpleLog - INFO: - episode: 1769, ep_reward: -103.552, ep_step: 71 2023-05-30 00:51:16 - SimpleLog - INFO: - episode: 1770, ep_reward: -103.592, ep_step: 65 2023-05-30 00:51:16 - SimpleLog - INFO: - episode: 1771, ep_reward: -102.855, ep_step: 67 2023-05-30 00:51:16 - SimpleLog - INFO: - episode: 1772, ep_reward: -111.342, ep_step: 297 2023-05-30 00:51:17 - SimpleLog - INFO: - episode: 1773, ep_reward: -103.270, ep_step: 72 2023-05-30 00:51:17 - SimpleLog - INFO: - episode: 1774, ep_reward: -101.929, ep_step: 64 2023-05-30 00:51:17 - SimpleLog - INFO: - episode: 1775, ep_reward: -102.583, ep_step: 61 2023-05-30 00:51:17 - SimpleLog - INFO: - episode: 1776, ep_reward: -104.872, ep_step: 90 2023-05-30 00:51:17 - SimpleLog - INFO: - episode: 1777, ep_reward: -103.110, ep_step: 123 2023-05-30 00:51:18 - SimpleLog - INFO: - episode: 1778, ep_reward: -106.000, ep_step: 162 2023-05-30 00:51:20 - SimpleLog - INFO: - episode: 1779, ep_reward: -79.784, ep_step: 795 2023-05-30 00:51:22 - SimpleLog - INFO: - episode: 1780, ep_reward: -102.831, ep_step: 985 2023-05-30 00:51:22 - SimpleLog - INFO: - episode: 1781, ep_reward: -120.670, ep_step: 276 2023-05-30 00:51:24 - SimpleLog - INFO: - episode: 1782, ep_reward: -134.454, ep_step: 597 2023-05-30 00:51:24 - SimpleLog - INFO: - episode: 1783, ep_reward: -141.696, ep_step: 197 2023-05-30 00:51:24 - SimpleLog - INFO: - episode: 1784, ep_reward: -108.975, ep_step: 75 2023-05-30 00:51:25 - SimpleLog - INFO: - episode: 1785, ep_reward: -133.912, ep_step: 191 2023-05-30 00:51:25 - SimpleLog - INFO: - episode: 1786, ep_reward: -108.367, ep_step: 100 2023-05-30 00:51:25 - SimpleLog - INFO: - episode: 1787, ep_reward: -125.397, ep_step: 146 2023-05-30 00:51:27 - SimpleLog - INFO: - episode: 1788, ep_reward: -123.862, ep_step: 723 2023-05-30 00:51:29 - SimpleLog - INFO: - episode: 1789, ep_reward: 13.864, ep_step: 1000 2023-05-30 00:51:29 - SimpleLog - INFO: - episode: 1790, ep_reward: -110.230, ep_step: 57 2023-05-30 00:51:30 - SimpleLog - INFO: - episode: 1791, ep_reward: -90.139, ep_step: 433 2023-05-30 00:51:33 - SimpleLog - INFO: - episode: 1792, ep_reward: 16.752, ep_step: 1000 2023-05-30 00:51:35 - SimpleLog - INFO: - episode: 1793, ep_reward: 16.646, ep_step: 1000 2023-05-30 00:51:37 - SimpleLog - INFO: - episode: 1794, ep_reward: 17.158, ep_step: 1000 2023-05-30 00:51:39 - SimpleLog - INFO: - episode: 1795, ep_reward: -72.640, ep_step: 815 2023-05-30 00:51:41 - SimpleLog - INFO: - episode: 1796, ep_reward: 22.058, ep_step: 1000 2023-05-30 00:51:43 - SimpleLog - INFO: - episode: 1797, ep_reward: -68.764, ep_step: 753 2023-05-30 00:51:44 - SimpleLog - INFO: - episode: 1798, ep_reward: -85.062, ep_step: 452 2023-05-30 00:51:45 - SimpleLog - INFO: - episode: 1799, ep_reward: -54.798, ep_step: 567 2023-05-30 00:51:46 - SimpleLog - INFO: - episode: 1800, ep_reward: -96.013, ep_step: 146 2023-05-30 00:51:46 - SimpleLog - INFO: - episode: 1801, ep_reward: -88.775, ep_step: 127 2023-05-30 00:51:48 - SimpleLog - INFO: - episode: 1802, ep_reward: -62.593, ep_step: 691 2023-05-30 00:51:49 - SimpleLog - INFO: - episode: 1803, ep_reward: -58.262, ep_step: 530 2023-05-30 00:51:51 - SimpleLog - INFO: - episode: 1804, ep_reward: 21.661, ep_step: 1000 2023-05-30 00:51:53 - SimpleLog - INFO: - episode: 1805, ep_reward: 13.946, ep_step: 1000 2023-05-30 00:51:55 - SimpleLog - INFO: - episode: 1806, ep_reward: -59.842, ep_step: 819 2023-05-30 00:51:58 - SimpleLog - INFO: - episode: 1807, ep_reward: 19.439, ep_step: 1000 2023-05-30 00:52:00 - SimpleLog - INFO: - episode: 1808, ep_reward: 15.026, ep_step: 1000 2023-05-30 00:52:02 - SimpleLog - INFO: - episode: 1809, ep_reward: 83.096, ep_step: 1000 2023-05-30 00:52:04 - SimpleLog - INFO: - episode: 1810, ep_reward: 85.084, ep_step: 1000 2023-05-30 00:52:07 - SimpleLog - INFO: - episode: 1811, ep_reward: -0.152, ep_step: 1000 2023-05-30 00:52:09 - SimpleLog - INFO: - episode: 1812, ep_reward: 2.094, ep_step: 1000 2023-05-30 00:52:11 - SimpleLog - INFO: - episode: 1813, ep_reward: -13.083, ep_step: 935 2023-05-30 00:52:13 - SimpleLog - INFO: - episode: 1814, ep_reward: 47.753, ep_step: 1000 2023-05-30 00:52:15 - SimpleLog - INFO: - episode: 1815, ep_reward: -76.208, ep_step: 525 2023-05-30 00:52:17 - SimpleLog - INFO: - episode: 1816, ep_reward: -38.480, ep_step: 1000 2023-05-30 00:52:18 - SimpleLog - INFO: - episode: 1817, ep_reward: -64.407, ep_step: 664 2023-05-30 00:52:21 - SimpleLog - INFO: - episode: 1818, ep_reward: 1.247, ep_step: 1000 2023-05-30 00:52:23 - SimpleLog - INFO: - episode: 1819, ep_reward: 3.424, ep_step: 1000 2023-05-30 00:52:27 - SimpleLog - INFO: - update_step: 1120000, online_eval_reward: -5.617 2023-05-30 00:52:27 - SimpleLog - INFO: - episode: 1820, ep_reward: 1.835, ep_step: 1000 2023-05-30 00:52:30 - SimpleLog - INFO: - episode: 1821, ep_reward: 4.731, ep_step: 1000 2023-05-30 00:52:32 - SimpleLog - INFO: - episode: 1822, ep_reward: -31.016, ep_step: 1000 2023-05-30 00:52:34 - SimpleLog - INFO: - episode: 1823, ep_reward: 21.311, ep_step: 1000 2023-05-30 00:52:36 - SimpleLog - INFO: - episode: 1824, ep_reward: 21.196, ep_step: 1000 2023-05-30 00:52:39 - SimpleLog - INFO: - episode: 1825, ep_reward: -35.750, ep_step: 1000 2023-05-30 00:52:41 - SimpleLog - INFO: - episode: 1826, ep_reward: -38.640, ep_step: 1000 2023-05-30 00:52:43 - SimpleLog - INFO: - episode: 1827, ep_reward: -30.826, ep_step: 1000 2023-05-30 00:52:44 - SimpleLog - INFO: - episode: 1828, ep_reward: -110.385, ep_step: 361 2023-05-30 00:52:44 - SimpleLog - INFO: - episode: 1829, ep_reward: -136.317, ep_step: 182 2023-05-30 00:52:47 - SimpleLog - INFO: - episode: 1830, ep_reward: -47.388, ep_step: 1000 2023-05-30 00:52:47 - SimpleLog - INFO: - episode: 1831, ep_reward: -109.523, ep_step: 385 2023-05-30 00:52:50 - SimpleLog - INFO: - episode: 1832, ep_reward: -14.240, ep_step: 1000 2023-05-30 00:52:52 - SimpleLog - INFO: - episode: 1833, ep_reward: -42.265, ep_step: 1000 2023-05-30 00:52:54 - SimpleLog - INFO: - episode: 1834, ep_reward: -10.414, ep_step: 1000 2023-05-30 00:52:57 - SimpleLog - INFO: - episode: 1835, ep_reward: -78.934, ep_step: 1000 2023-05-30 00:52:59 - SimpleLog - INFO: - episode: 1836, ep_reward: -37.601, ep_step: 1000 2023-05-30 00:53:01 - SimpleLog - INFO: - episode: 1837, ep_reward: -38.620, ep_step: 1000 2023-05-30 00:53:03 - SimpleLog - INFO: - episode: 1838, ep_reward: -45.522, ep_step: 1000 2023-05-30 00:53:06 - SimpleLog - INFO: - episode: 1839, ep_reward: -39.759, ep_step: 1000 2023-05-30 00:53:08 - SimpleLog - INFO: - episode: 1840, ep_reward: -16.944, ep_step: 1000 2023-05-30 00:53:10 - SimpleLog - INFO: - episode: 1841, ep_reward: -66.252, ep_step: 1000 2023-05-30 00:53:12 - SimpleLog - INFO: - episode: 1842, ep_reward: -37.742, ep_step: 1000 2023-05-30 00:53:15 - SimpleLog - INFO: - episode: 1843, ep_reward: -38.153, ep_step: 1000 2023-05-30 00:53:17 - SimpleLog - INFO: - episode: 1844, ep_reward: -36.083, ep_step: 1000 2023-05-30 00:53:19 - SimpleLog - INFO: - episode: 1845, ep_reward: -37.958, ep_step: 1000 2023-05-30 00:53:22 - SimpleLog - INFO: - episode: 1846, ep_reward: -37.003, ep_step: 1000 2023-05-30 00:53:24 - SimpleLog - INFO: - episode: 1847, ep_reward: -36.330, ep_step: 1000 2023-05-30 00:53:24 - SimpleLog - INFO: - episode: 1848, ep_reward: -84.333, ep_step: 223 2023-05-30 00:53:27 - SimpleLog - INFO: - episode: 1849, ep_reward: -39.545, ep_step: 1000 2023-05-30 00:53:29 - SimpleLog - INFO: - episode: 1850, ep_reward: -13.909, ep_step: 1000 2023-05-30 00:53:31 - SimpleLog - INFO: - episode: 1851, ep_reward: -0.930, ep_step: 1000 2023-05-30 00:53:33 - SimpleLog - INFO: - episode: 1852, ep_reward: -43.181, ep_step: 1000 2023-05-30 00:53:36 - SimpleLog - INFO: - episode: 1853, ep_reward: -40.796, ep_step: 1000 2023-05-30 00:53:38 - SimpleLog - INFO: - episode: 1854, ep_reward: -17.300, ep_step: 1000 2023-05-30 00:53:40 - SimpleLog - INFO: - episode: 1855, ep_reward: -17.035, ep_step: 1000 2023-05-30 00:53:43 - SimpleLog - INFO: - episode: 1856, ep_reward: 2.510, ep_step: 1000 2023-05-30 00:53:43 - SimpleLog - INFO: - episode: 1857, ep_reward: -114.153, ep_step: 89 2023-05-30 00:53:43 - SimpleLog - INFO: - episode: 1858, ep_reward: -115.061, ep_step: 80 2023-05-30 00:53:43 - SimpleLog - INFO: - episode: 1859, ep_reward: -129.171, ep_step: 120 2023-05-30 00:53:45 - SimpleLog - INFO: - episode: 1860, ep_reward: -0.504, ep_step: 1000 2023-05-30 00:53:46 - SimpleLog - INFO: - episode: 1861, ep_reward: -118.656, ep_step: 91 2023-05-30 00:53:48 - SimpleLog - INFO: - episode: 1862, ep_reward: -22.131, ep_step: 1000 2023-05-30 00:53:50 - SimpleLog - INFO: - episode: 1863, ep_reward: -15.707, ep_step: 1000 2023-05-30 00:53:53 - SimpleLog - INFO: - episode: 1864, ep_reward: -36.263, ep_step: 1000 2023-05-30 00:53:55 - SimpleLog - INFO: - episode: 1865, ep_reward: -42.537, ep_step: 1000 2023-05-30 00:53:57 - SimpleLog - INFO: - episode: 1866, ep_reward: -20.337, ep_step: 1000 2023-05-30 00:53:59 - SimpleLog - INFO: - episode: 1867, ep_reward: 15.669, ep_step: 1000 2023-05-30 00:54:01 - SimpleLog - INFO: - episode: 1868, ep_reward: -104.765, ep_step: 528 2023-05-30 00:54:03 - SimpleLog - INFO: - episode: 1869, ep_reward: 8.372, ep_step: 1000 2023-05-30 00:54:05 - SimpleLog - INFO: - episode: 1870, ep_reward: 17.656, ep_step: 1000 2023-05-30 00:54:07 - SimpleLog - INFO: - episode: 1871, ep_reward: 0.291, ep_step: 1000 2023-05-30 00:54:10 - SimpleLog - INFO: - episode: 1872, ep_reward: -19.992, ep_step: 1000 2023-05-30 00:54:12 - SimpleLog - INFO: - episode: 1873, ep_reward: -17.513, ep_step: 1000 2023-05-30 00:54:14 - SimpleLog - INFO: - episode: 1874, ep_reward: 7.766, ep_step: 1000 2023-05-30 00:54:16 - SimpleLog - INFO: - episode: 1875, ep_reward: -15.337, ep_step: 1000 2023-05-30 00:54:18 - SimpleLog - INFO: - episode: 1876, ep_reward: -70.304, ep_step: 718 2023-05-30 00:54:20 - SimpleLog - INFO: - episode: 1877, ep_reward: 9.952, ep_step: 1000 2023-05-30 00:54:23 - SimpleLog - INFO: - episode: 1878, ep_reward: 17.195, ep_step: 1000 2023-05-30 00:54:25 - SimpleLog - INFO: - episode: 1879, ep_reward: 12.516, ep_step: 1000 2023-05-30 00:54:27 - SimpleLog - INFO: - episode: 1880, ep_reward: 18.853, ep_step: 1000 2023-05-30 00:54:30 - SimpleLog - INFO: - episode: 1881, ep_reward: 16.352, ep_step: 1000 2023-05-30 00:54:32 - SimpleLog - INFO: - episode: 1882, ep_reward: 19.246, ep_step: 1000 2023-05-30 00:54:34 - SimpleLog - INFO: - episode: 1883, ep_reward: 87.078, ep_step: 1000 2023-05-30 00:54:36 - SimpleLog - INFO: - episode: 1884, ep_reward: 22.536, ep_step: 1000 2023-05-30 00:54:39 - SimpleLog - INFO: - episode: 1885, ep_reward: 18.562, ep_step: 1000 2023-05-30 00:54:40 - SimpleLog - INFO: - episode: 1886, ep_reward: -61.715, ep_step: 733 2023-05-30 00:54:43 - SimpleLog - INFO: - episode: 1887, ep_reward: 20.156, ep_step: 1000 2023-05-30 00:54:44 - SimpleLog - INFO: - episode: 1888, ep_reward: -74.137, ep_step: 765 2023-05-30 00:54:46 - SimpleLog - INFO: - episode: 1889, ep_reward: -83.797, ep_step: 609 2023-05-30 00:54:48 - SimpleLog - INFO: - episode: 1890, ep_reward: 19.311, ep_step: 1000 2023-05-30 00:54:50 - SimpleLog - INFO: - episode: 1891, ep_reward: 60.781, ep_step: 1000 2023-05-30 00:54:51 - SimpleLog - INFO: - episode: 1892, ep_reward: -55.249, ep_step: 586 2023-05-30 00:54:54 - SimpleLog - INFO: - episode: 1893, ep_reward: 12.015, ep_step: 1000 2023-05-30 00:54:55 - SimpleLog - INFO: - episode: 1894, ep_reward: -60.035, ep_step: 615 2023-05-30 00:54:57 - SimpleLog - INFO: - episode: 1895, ep_reward: 61.846, ep_step: 1000 2023-05-30 00:54:59 - SimpleLog - INFO: - episode: 1896, ep_reward: -55.641, ep_step: 560 2023-05-30 00:55:01 - SimpleLog - INFO: - episode: 1897, ep_reward: 15.385, ep_step: 1000 2023-05-30 00:55:03 - SimpleLog - INFO: - episode: 1898, ep_reward: 19.050, ep_step: 1000 2023-05-30 00:55:05 - SimpleLog - INFO: - episode: 1899, ep_reward: 20.549, ep_step: 1000 2023-05-30 00:55:07 - SimpleLog - INFO: - update_step: 1190000, online_eval_reward: 3.856 2023-05-30 00:55:09 - SimpleLog - INFO: - episode: 1900, ep_reward: 19.377, ep_step: 1000 2023-05-30 00:55:11 - SimpleLog - INFO: - episode: 1901, ep_reward: 86.256, ep_step: 1000 2023-05-30 00:55:12 - SimpleLog - INFO: - episode: 1902, ep_reward: -88.031, ep_step: 236 2023-05-30 00:55:13 - SimpleLog - INFO: - episode: 1903, ep_reward: -48.656, ep_step: 496 2023-05-30 00:55:14 - SimpleLog - INFO: - episode: 1904, ep_reward: -84.231, ep_step: 303 2023-05-30 00:55:15 - SimpleLog - INFO: - episode: 1905, ep_reward: -65.979, ep_step: 420 2023-05-30 00:55:17 - SimpleLog - INFO: - episode: 1906, ep_reward: 19.736, ep_step: 1000 2023-05-30 00:55:19 - SimpleLog - INFO: - episode: 1907, ep_reward: 6.873, ep_step: 1000 2023-05-30 00:55:22 - SimpleLog - INFO: - episode: 1908, ep_reward: 6.484, ep_step: 1000 2023-05-30 00:55:24 - SimpleLog - INFO: - episode: 1909, ep_reward: 15.796, ep_step: 1000 2023-05-30 00:55:26 - SimpleLog - INFO: - episode: 1910, ep_reward: 9.944, ep_step: 1000 2023-05-30 00:55:28 - SimpleLog - INFO: - episode: 1911, ep_reward: 85.373, ep_step: 1000 2023-05-30 00:55:31 - SimpleLog - INFO: - episode: 1912, ep_reward: 2.888, ep_step: 1000 2023-05-30 00:55:33 - SimpleLog - INFO: - episode: 1913, ep_reward: -79.517, ep_step: 858 2023-05-30 00:55:34 - SimpleLog - INFO: - episode: 1914, ep_reward: -66.186, ep_step: 636 2023-05-30 00:55:36 - SimpleLog - INFO: - episode: 1915, ep_reward: 16.441, ep_step: 1000 2023-05-30 00:55:39 - SimpleLog - INFO: - episode: 1916, ep_reward: 15.017, ep_step: 1000 2023-05-30 00:55:41 - SimpleLog - INFO: - episode: 1917, ep_reward: -41.183, ep_step: 1000 2023-05-30 00:55:43 - SimpleLog - INFO: - episode: 1918, ep_reward: -98.677, ep_step: 897 2023-05-30 00:55:45 - SimpleLog - INFO: - episode: 1919, ep_reward: 10.981, ep_step: 1000 2023-05-30 00:55:48 - SimpleLog - INFO: - episode: 1920, ep_reward: -39.289, ep_step: 1000 2023-05-30 00:55:50 - SimpleLog - INFO: - episode: 1921, ep_reward: 4.302, ep_step: 1000 2023-05-30 00:55:52 - SimpleLog - INFO: - episode: 1922, ep_reward: -42.024, ep_step: 1000 2023-05-30 00:55:54 - SimpleLog - INFO: - episode: 1923, ep_reward: -83.908, ep_step: 953 2023-05-30 00:55:56 - SimpleLog - INFO: - episode: 1924, ep_reward: -132.940, ep_step: 682 2023-05-30 00:55:57 - SimpleLog - INFO: - episode: 1925, ep_reward: -67.255, ep_step: 544 2023-05-30 00:55:59 - SimpleLog - INFO: - episode: 1926, ep_reward: 86.715, ep_step: 1000 2023-05-30 00:56:02 - SimpleLog - INFO: - episode: 1927, ep_reward: 41.903, ep_step: 1000 2023-05-30 00:56:03 - SimpleLog - INFO: - episode: 1928, ep_reward: -59.782, ep_step: 619 2023-05-30 00:56:05 - SimpleLog - INFO: - episode: 1929, ep_reward: 11.411, ep_step: 1000 2023-05-30 00:56:08 - SimpleLog - INFO: - episode: 1930, ep_reward: 3.106, ep_step: 1000 2023-05-30 00:56:09 - SimpleLog - INFO: - episode: 1931, ep_reward: -85.811, ep_step: 670 2023-05-30 00:56:12 - SimpleLog - INFO: - episode: 1932, ep_reward: 80.790, ep_step: 1000 2023-05-30 00:56:12 - SimpleLog - INFO: - episode: 1933, ep_reward: -104.534, ep_step: 267 2023-05-30 00:56:13 - SimpleLog - INFO: - episode: 1934, ep_reward: -101.669, ep_step: 334 2023-05-30 00:56:15 - SimpleLog - INFO: - episode: 1935, ep_reward: 11.680, ep_step: 1000 2023-05-30 00:56:16 - SimpleLog - INFO: - episode: 1936, ep_reward: -113.615, ep_step: 314 2023-05-30 00:56:18 - SimpleLog - INFO: - episode: 1937, ep_reward: 16.412, ep_step: 1000 2023-05-30 00:56:21 - SimpleLog - INFO: - episode: 1938, ep_reward: 87.341, ep_step: 1000 2023-05-30 00:56:23 - SimpleLog - INFO: - episode: 1939, ep_reward: 12.671, ep_step: 1000 2023-05-30 00:56:25 - SimpleLog - INFO: - episode: 1940, ep_reward: 15.723, ep_step: 1000 2023-05-30 00:56:27 - SimpleLog - INFO: - episode: 1941, ep_reward: 61.449, ep_step: 1000 2023-05-30 00:56:30 - SimpleLog - INFO: - episode: 1942, ep_reward: 11.719, ep_step: 1000 2023-05-30 00:56:30 - SimpleLog - INFO: - episode: 1943, ep_reward: -102.113, ep_step: 201 2023-05-30 00:56:32 - SimpleLog - INFO: - episode: 1944, ep_reward: 19.041, ep_step: 1000 2023-05-30 00:56:35 - SimpleLog - INFO: - episode: 1945, ep_reward: 14.936, ep_step: 1000 2023-05-30 00:56:37 - SimpleLog - INFO: - episode: 1946, ep_reward: 16.353, ep_step: 1000 2023-05-30 00:56:39 - SimpleLog - INFO: - episode: 1947, ep_reward: -81.313, ep_step: 671 2023-05-30 00:56:40 - SimpleLog - INFO: - episode: 1948, ep_reward: -69.104, ep_step: 804 2023-05-30 00:56:43 - SimpleLog - INFO: - episode: 1949, ep_reward: 22.382, ep_step: 1000 2023-05-30 00:56:45 - SimpleLog - INFO: - episode: 1950, ep_reward: 85.810, ep_step: 1000 2023-05-30 00:56:47 - SimpleLog - INFO: - episode: 1951, ep_reward: -0.568, ep_step: 1000 2023-05-30 00:56:48 - SimpleLog - INFO: - episode: 1952, ep_reward: -63.201, ep_step: 443 2023-05-30 00:56:50 - SimpleLog - INFO: - episode: 1953, ep_reward: 91.209, ep_step: 1000 2023-05-30 00:56:53 - SimpleLog - INFO: - episode: 1954, ep_reward: 0.093, ep_step: 1000 2023-05-30 00:56:55 - SimpleLog - INFO: - episode: 1955, ep_reward: 0.435, ep_step: 1000 2023-05-30 00:56:55 - SimpleLog - INFO: - episode: 1956, ep_reward: -89.355, ep_step: 201 2023-05-30 00:56:58 - SimpleLog - INFO: - episode: 1957, ep_reward: 20.677, ep_step: 1000 2023-05-30 00:56:59 - SimpleLog - INFO: - episode: 1958, ep_reward: -72.779, ep_step: 482 2023-05-30 00:57:01 - SimpleLog - INFO: - episode: 1959, ep_reward: -8.401, ep_step: 1000 2023-05-30 00:57:03 - SimpleLog - INFO: - episode: 1960, ep_reward: 0.819, ep_step: 1000 2023-05-30 00:57:06 - SimpleLog - INFO: - episode: 1961, ep_reward: 0.066, ep_step: 1000 2023-05-30 00:57:08 - SimpleLog - INFO: - episode: 1962, ep_reward: -37.847, ep_step: 1000 2023-05-30 00:57:10 - SimpleLog - INFO: - episode: 1963, ep_reward: 1.259, ep_step: 1000 2023-05-30 00:57:12 - SimpleLog - INFO: - episode: 1964, ep_reward: 1.448, ep_step: 1000 2023-05-30 00:57:15 - SimpleLog - INFO: - episode: 1965, ep_reward: 23.510, ep_step: 1000 2023-05-30 00:57:16 - SimpleLog - INFO: - episode: 1966, ep_reward: -36.404, ep_step: 556 2023-05-30 00:57:18 - SimpleLog - INFO: - episode: 1967, ep_reward: -29.845, ep_step: 1000 2023-05-30 00:57:20 - SimpleLog - INFO: - episode: 1968, ep_reward: 24.148, ep_step: 1000 2023-05-30 00:57:23 - SimpleLog - INFO: - episode: 1969, ep_reward: -16.588, ep_step: 1000 2023-05-30 00:57:25 - SimpleLog - INFO: - episode: 1970, ep_reward: -31.578, ep_step: 1000 2023-05-30 00:57:27 - SimpleLog - INFO: - episode: 1971, ep_reward: 3.407, ep_step: 1000 2023-05-30 00:57:30 - SimpleLog - INFO: - episode: 1972, ep_reward: 23.073, ep_step: 1000 2023-05-30 00:57:32 - SimpleLog - INFO: - episode: 1973, ep_reward: -35.071, ep_step: 1000 2023-05-30 00:57:34 - SimpleLog - INFO: - episode: 1974, ep_reward: -34.048, ep_step: 1000 2023-05-30 00:57:35 - SimpleLog - INFO: - episode: 1975, ep_reward: -52.630, ep_step: 550 2023-05-30 00:57:38 - SimpleLog - INFO: - episode: 1976, ep_reward: 9.975, ep_step: 1000 2023-05-30 00:57:40 - SimpleLog - INFO: - episode: 1977, ep_reward: -34.480, ep_step: 1000 2023-05-30 00:57:41 - SimpleLog - INFO: - episode: 1978, ep_reward: -45.812, ep_step: 619 2023-05-30 00:57:44 - SimpleLog - INFO: - episode: 1979, ep_reward: 1.641, ep_step: 1000 2023-05-30 00:57:45 - SimpleLog - INFO: - episode: 1980, ep_reward: -46.796, ep_step: 470 2023-05-30 00:57:47 - SimpleLog - INFO: - update_step: 1260000, online_eval_reward: -80.328 2023-05-30 00:57:47 - SimpleLog - INFO: - episode: 1981, ep_reward: 0.002, ep_step: 1000 2023-05-30 00:57:48 - SimpleLog - INFO: - episode: 1982, ep_reward: -41.747, ep_step: 522 2023-05-30 00:57:50 - SimpleLog - INFO: - episode: 1983, ep_reward: -82.555, ep_step: 761 2023-05-30 00:57:51 - SimpleLog - INFO: - episode: 1984, ep_reward: -78.744, ep_step: 342 2023-05-30 00:57:53 - SimpleLog - INFO: - episode: 1985, ep_reward: 91.028, ep_step: 1000 2023-05-30 00:57:56 - SimpleLog - INFO: - episode: 1986, ep_reward: 19.875, ep_step: 1000 2023-05-30 00:57:58 - SimpleLog - INFO: - episode: 1987, ep_reward: 17.982, ep_step: 1000 2023-05-30 00:57:59 - SimpleLog - INFO: - episode: 1988, ep_reward: -65.550, ep_step: 457 2023-05-30 00:58:01 - SimpleLog - INFO: - episode: 1989, ep_reward: -71.481, ep_step: 808 2023-05-30 00:58:02 - SimpleLog - INFO: - episode: 1990, ep_reward: -76.023, ep_step: 673 2023-05-30 00:58:04 - SimpleLog - INFO: - episode: 1991, ep_reward: -83.644, ep_step: 955 2023-05-30 00:58:07 - SimpleLog - INFO: - episode: 1992, ep_reward: 12.186, ep_step: 1000 2023-05-30 00:58:08 - SimpleLog - INFO: - episode: 1993, ep_reward: -59.228, ep_step: 571 2023-05-30 00:58:10 - SimpleLog - INFO: - episode: 1994, ep_reward: 15.935, ep_step: 1000 2023-05-30 00:58:13 - SimpleLog - INFO: - episode: 1995, ep_reward: 10.852, ep_step: 1000 2023-05-30 00:58:14 - SimpleLog - INFO: - episode: 1996, ep_reward: -77.689, ep_step: 789 2023-05-30 00:58:17 - SimpleLog - INFO: - episode: 1997, ep_reward: 11.235, ep_step: 1000 2023-05-30 00:58:18 - SimpleLog - INFO: - episode: 1998, ep_reward: -92.895, ep_step: 781 2023-05-30 00:58:21 - SimpleLog - INFO: - episode: 1999, ep_reward: 11.823, ep_step: 1000 2023-05-30 00:58:23 - SimpleLog - INFO: - episode: 2000, ep_reward: 62.845, ep_step: 1000 2023-05-30 00:58:25 - SimpleLog - INFO: - episode: 2001, ep_reward: -92.798, ep_step: 905 2023-05-30 00:58:26 - SimpleLog - INFO: - episode: 2002, ep_reward: -56.589, ep_step: 437 2023-05-30 00:58:26 - SimpleLog - INFO: - episode: 2003, ep_reward: -97.845, ep_step: 189 2023-05-30 00:58:28 - SimpleLog - INFO: - episode: 2004, ep_reward: -75.792, ep_step: 863 2023-05-30 00:58:30 - SimpleLog - INFO: - episode: 2005, ep_reward: -70.994, ep_step: 629 2023-05-30 00:58:32 - SimpleLog - INFO: - episode: 2006, ep_reward: 18.669, ep_step: 1000 2023-05-30 00:58:34 - SimpleLog - INFO: - episode: 2007, ep_reward: 7.049, ep_step: 1000 2023-05-30 00:58:37 - SimpleLog - INFO: - episode: 2008, ep_reward: 86.659, ep_step: 1000 2023-05-30 00:58:39 - SimpleLog - INFO: - episode: 2009, ep_reward: 15.415, ep_step: 1000 2023-05-30 00:58:41 - SimpleLog - INFO: - episode: 2010, ep_reward: -3.063, ep_step: 1000 2023-05-30 00:58:43 - SimpleLog - INFO: - episode: 2011, ep_reward: 12.868, ep_step: 1000 2023-05-30 00:58:45 - SimpleLog - INFO: - episode: 2012, ep_reward: -47.167, ep_step: 488 2023-05-30 00:58:46 - SimpleLog - INFO: - episode: 2013, ep_reward: -50.642, ep_step: 517 2023-05-30 00:58:48 - SimpleLog - INFO: - episode: 2014, ep_reward: 87.683, ep_step: 1000 2023-05-30 00:58:50 - SimpleLog - INFO: - episode: 2015, ep_reward: 16.383, ep_step: 1000 2023-05-30 00:58:51 - SimpleLog - INFO: - episode: 2016, ep_reward: -91.998, ep_step: 241 2023-05-30 00:58:51 - SimpleLog - INFO: - episode: 2017, ep_reward: -100.039, ep_step: 230 2023-05-30 00:58:54 - SimpleLog - INFO: - episode: 2018, ep_reward: 88.818, ep_step: 1000 2023-05-30 00:58:55 - SimpleLog - INFO: - episode: 2019, ep_reward: -58.316, ep_step: 618 2023-05-30 00:58:57 - SimpleLog - INFO: - episode: 2020, ep_reward: 82.859, ep_step: 1000 2023-05-30 00:59:00 - SimpleLog - INFO: - episode: 2021, ep_reward: -2.470, ep_step: 1000 2023-05-30 00:59:02 - SimpleLog - INFO: - episode: 2022, ep_reward: 20.320, ep_step: 1000 2023-05-30 00:59:04 - SimpleLog - INFO: - episode: 2023, ep_reward: 11.563, ep_step: 1000 2023-05-30 00:59:06 - SimpleLog - INFO: - episode: 2024, ep_reward: 88.826, ep_step: 1000 2023-05-30 00:59:08 - SimpleLog - INFO: - episode: 2025, ep_reward: -75.012, ep_step: 619 2023-05-30 00:59:09 - SimpleLog - INFO: - episode: 2026, ep_reward: -68.619, ep_step: 531 2023-05-30 00:59:11 - SimpleLog - INFO: - episode: 2027, ep_reward: 21.198, ep_step: 1000 2023-05-30 00:59:13 - SimpleLog - INFO: - episode: 2028, ep_reward: -119.634, ep_step: 570 2023-05-30 00:59:13 - SimpleLog - INFO: - episode: 2029, ep_reward: -119.517, ep_step: 106 2023-05-30 00:59:15 - SimpleLog - INFO: - episode: 2030, ep_reward: 18.019, ep_step: 1000 2023-05-30 00:59:16 - SimpleLog - INFO: - episode: 2031, ep_reward: -50.239, ep_step: 527 2023-05-30 00:59:17 - SimpleLog - INFO: - episode: 2032, ep_reward: -131.750, ep_step: 145 2023-05-30 00:59:17 - SimpleLog - INFO: - episode: 2033, ep_reward: -114.611, ep_step: 61 2023-05-30 00:59:18 - SimpleLog - INFO: - episode: 2034, ep_reward: -51.330, ep_step: 543 2023-05-30 00:59:18 - SimpleLog - INFO: - episode: 2035, ep_reward: -131.726, ep_step: 143 2023-05-30 00:59:20 - SimpleLog - INFO: - episode: 2036, ep_reward: -48.241, ep_step: 468 2023-05-30 00:59:20 - SimpleLog - INFO: - episode: 2037, ep_reward: -109.439, ep_step: 67 2023-05-30 00:59:20 - SimpleLog - INFO: - episode: 2038, ep_reward: -129.220, ep_step: 236 2023-05-30 00:59:20 - SimpleLog - INFO: - episode: 2039, ep_reward: -131.823, ep_step: 101 2023-05-30 00:59:22 - SimpleLog - INFO: - episode: 2040, ep_reward: -64.756, ep_step: 542 2023-05-30 00:59:22 - SimpleLog - INFO: - episode: 2041, ep_reward: -110.528, ep_step: 52 2023-05-30 00:59:23 - SimpleLog - INFO: - episode: 2042, ep_reward: -55.592, ep_step: 593 2023-05-30 00:59:25 - SimpleLog - INFO: - episode: 2043, ep_reward: 13.874, ep_step: 1000 2023-05-30 00:59:28 - SimpleLog - INFO: - episode: 2044, ep_reward: 4.488, ep_step: 1000 2023-05-30 00:59:30 - SimpleLog - INFO: - episode: 2045, ep_reward: 66.261, ep_step: 1000 2023-05-30 00:59:32 - SimpleLog - INFO: - episode: 2046, ep_reward: 25.450, ep_step: 1000 2023-05-30 00:59:35 - SimpleLog - INFO: - episode: 2047, ep_reward: 1.534, ep_step: 1000 2023-05-30 00:59:36 - SimpleLog - INFO: - episode: 2048, ep_reward: -53.356, ep_step: 580 2023-05-30 00:59:38 - SimpleLog - INFO: - episode: 2049, ep_reward: 35.037, ep_step: 1000 2023-05-30 00:59:40 - SimpleLog - INFO: - episode: 2050, ep_reward: 20.957, ep_step: 1000 2023-05-30 00:59:43 - SimpleLog - INFO: - episode: 2051, ep_reward: 70.096, ep_step: 1000 2023-05-30 00:59:45 - SimpleLog - INFO: - episode: 2052, ep_reward: 18.875, ep_step: 1000 2023-05-30 00:59:47 - SimpleLog - INFO: - episode: 2053, ep_reward: 22.487, ep_step: 1000 2023-05-30 00:59:49 - SimpleLog - INFO: - episode: 2054, ep_reward: 19.110, ep_step: 1000 2023-05-30 00:59:52 - SimpleLog - INFO: - episode: 2055, ep_reward: 18.615, ep_step: 1000 2023-05-30 00:59:53 - SimpleLog - INFO: - episode: 2056, ep_reward: -38.542, ep_step: 628 2023-05-30 00:59:55 - SimpleLog - INFO: - episode: 2057, ep_reward: -33.149, ep_step: 1000 2023-05-30 00:59:58 - SimpleLog - INFO: - episode: 2058, ep_reward: 88.290, ep_step: 1000 2023-05-30 01:00:00 - SimpleLog - INFO: - episode: 2059, ep_reward: -43.985, ep_step: 1000 2023-05-30 01:00:02 - SimpleLog - INFO: - episode: 2060, ep_reward: -33.485, ep_step: 1000 2023-05-30 01:00:04 - SimpleLog - INFO: - episode: 2061, ep_reward: -34.677, ep_step: 1000 2023-05-30 01:00:07 - SimpleLog - INFO: - episode: 2062, ep_reward: 16.397, ep_step: 1000 2023-05-30 01:00:09 - SimpleLog - INFO: - episode: 2063, ep_reward: 16.642, ep_step: 1000 2023-05-30 01:00:11 - SimpleLog - INFO: - episode: 2064, ep_reward: 15.273, ep_step: 1000 2023-05-30 01:00:14 - SimpleLog - INFO: - episode: 2065, ep_reward: -19.120, ep_step: 1000 2023-05-30 01:00:15 - SimpleLog - INFO: - episode: 2066, ep_reward: -50.115, ep_step: 495 2023-05-30 01:00:17 - SimpleLog - INFO: - episode: 2067, ep_reward: 86.793, ep_step: 1000 2023-05-30 01:00:18 - SimpleLog - INFO: - episode: 2068, ep_reward: -45.970, ep_step: 625 2023-05-30 01:00:21 - SimpleLog - INFO: - episode: 2069, ep_reward: 18.144, ep_step: 1000 2023-05-30 01:00:23 - SimpleLog - INFO: - episode: 2070, ep_reward: 19.178, ep_step: 1000 2023-05-30 01:00:25 - SimpleLog - INFO: - episode: 2071, ep_reward: 18.787, ep_step: 1000 2023-05-30 01:00:28 - SimpleLog - INFO: - update_step: 1330000, online_eval_reward: 23.925 2023-05-30 01:00:28 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 23.925, save the best model! 2023-05-30 01:00:29 - SimpleLog - INFO: - episode: 2072, ep_reward: 19.323, ep_step: 1000 2023-05-30 01:00:32 - SimpleLog - INFO: - episode: 2073, ep_reward: 19.451, ep_step: 1000 2023-05-30 01:00:33 - SimpleLog - INFO: - episode: 2074, ep_reward: -53.447, ep_step: 559 2023-05-30 01:00:35 - SimpleLog - INFO: - episode: 2075, ep_reward: 25.554, ep_step: 1000 2023-05-30 01:00:38 - SimpleLog - INFO: - episode: 2076, ep_reward: 18.497, ep_step: 1000 2023-05-30 01:00:38 - SimpleLog - INFO: - episode: 2077, ep_reward: -73.063, ep_step: 330 2023-05-30 01:00:41 - SimpleLog - INFO: - episode: 2078, ep_reward: 17.139, ep_step: 1000 2023-05-30 01:00:43 - SimpleLog - INFO: - episode: 2079, ep_reward: -20.475, ep_step: 1000 2023-05-30 01:00:45 - SimpleLog - INFO: - episode: 2080, ep_reward: 59.111, ep_step: 1000 2023-05-30 01:00:47 - SimpleLog - INFO: - episode: 2081, ep_reward: -18.015, ep_step: 1000 2023-05-30 01:00:50 - SimpleLog - INFO: - episode: 2082, ep_reward: 21.201, ep_step: 1000 2023-05-30 01:00:51 - SimpleLog - INFO: - episode: 2083, ep_reward: -59.054, ep_step: 664 2023-05-30 01:00:53 - SimpleLog - INFO: - episode: 2084, ep_reward: -6.414, ep_step: 939 2023-05-30 01:00:55 - SimpleLog - INFO: - episode: 2085, ep_reward: -68.423, ep_step: 510 2023-05-30 01:00:57 - SimpleLog - INFO: - episode: 2086, ep_reward: 17.707, ep_step: 1000 2023-05-30 01:00:59 - SimpleLog - INFO: - episode: 2087, ep_reward: 22.779, ep_step: 1000 2023-05-30 01:01:01 - SimpleLog - INFO: - episode: 2088, ep_reward: 82.400, ep_step: 1000 2023-05-30 01:01:04 - SimpleLog - INFO: - episode: 2089, ep_reward: 14.546, ep_step: 1000 2023-05-30 01:01:05 - SimpleLog - INFO: - episode: 2090, ep_reward: -49.469, ep_step: 495 2023-05-30 01:01:07 - SimpleLog - INFO: - episode: 2091, ep_reward: 29.604, ep_step: 1000 2023-05-30 01:01:08 - SimpleLog - INFO: - episode: 2092, ep_reward: -64.475, ep_step: 412 2023-05-30 01:01:10 - SimpleLog - INFO: - episode: 2093, ep_reward: 14.136, ep_step: 1000 2023-05-30 01:01:11 - SimpleLog - INFO: - episode: 2094, ep_reward: -50.225, ep_step: 499 2023-05-30 01:01:14 - SimpleLog - INFO: - episode: 2095, ep_reward: 14.790, ep_step: 1000 2023-05-30 01:01:16 - SimpleLog - INFO: - episode: 2096, ep_reward: 78.839, ep_step: 1000 2023-05-30 01:01:17 - SimpleLog - INFO: - episode: 2097, ep_reward: -51.805, ep_step: 540 2023-05-30 01:01:20 - SimpleLog - INFO: - episode: 2098, ep_reward: 8.263, ep_step: 1000 2023-05-30 01:01:22 - SimpleLog - INFO: - episode: 2099, ep_reward: 42.653, ep_step: 1000 2023-05-30 01:01:23 - SimpleLog - INFO: - episode: 2100, ep_reward: -49.311, ep_step: 488 2023-05-30 01:01:23 - SimpleLog - INFO: - episode: 2101, ep_reward: -92.250, ep_step: 172 2023-05-30 01:01:24 - SimpleLog - INFO: - episode: 2102, ep_reward: -95.002, ep_step: 173 2023-05-30 01:01:26 - SimpleLog - INFO: - episode: 2103, ep_reward: 24.655, ep_step: 1000 2023-05-30 01:01:27 - SimpleLog - INFO: - episode: 2104, ep_reward: -63.310, ep_step: 439 2023-05-30 01:01:28 - SimpleLog - INFO: - episode: 2105, ep_reward: -60.483, ep_step: 383 2023-05-30 01:01:29 - SimpleLog - INFO: - episode: 2106, ep_reward: -101.559, ep_step: 339 2023-05-30 01:01:29 - SimpleLog - INFO: - episode: 2107, ep_reward: -96.102, ep_step: 184 2023-05-30 01:01:30 - SimpleLog - INFO: - episode: 2108, ep_reward: -63.126, ep_step: 409 2023-05-30 01:01:31 - SimpleLog - INFO: - episode: 2109, ep_reward: -59.634, ep_step: 405 2023-05-30 01:01:31 - SimpleLog - INFO: - episode: 2110, ep_reward: -92.782, ep_step: 172 2023-05-30 01:01:34 - SimpleLog - INFO: - episode: 2111, ep_reward: -30.760, ep_step: 1000 2023-05-30 01:01:34 - SimpleLog - INFO: - episode: 2112, ep_reward: -62.831, ep_step: 373 2023-05-30 01:01:36 - SimpleLog - INFO: - episode: 2113, ep_reward: -50.837, ep_step: 494 2023-05-30 01:01:38 - SimpleLog - INFO: - episode: 2114, ep_reward: 22.515, ep_step: 1000 2023-05-30 01:01:39 - SimpleLog - INFO: - episode: 2115, ep_reward: -44.459, ep_step: 396 2023-05-30 01:01:41 - SimpleLog - INFO: - episode: 2116, ep_reward: 14.576, ep_step: 1000 2023-05-30 01:01:43 - SimpleLog - INFO: - episode: 2117, ep_reward: 25.111, ep_step: 1000 2023-05-30 01:01:44 - SimpleLog - INFO: - episode: 2118, ep_reward: -97.213, ep_step: 191 2023-05-30 01:01:46 - SimpleLog - INFO: - episode: 2119, ep_reward: -42.759, ep_step: 1000 2023-05-30 01:01:46 - SimpleLog - INFO: - episode: 2120, ep_reward: -96.231, ep_step: 136 2023-05-30 01:01:47 - SimpleLog - INFO: - episode: 2121, ep_reward: -49.491, ep_step: 509 2023-05-30 01:01:48 - SimpleLog - INFO: - episode: 2122, ep_reward: -69.579, ep_step: 308 2023-05-30 01:01:49 - SimpleLog - INFO: - episode: 2123, ep_reward: -63.782, ep_step: 412 2023-05-30 01:01:51 - SimpleLog - INFO: - episode: 2124, ep_reward: 15.991, ep_step: 1000 2023-05-30 01:01:54 - SimpleLog - INFO: - episode: 2125, ep_reward: -45.328, ep_step: 1000 2023-05-30 01:01:56 - SimpleLog - INFO: - episode: 2126, ep_reward: 29.197, ep_step: 1000 2023-05-30 01:01:58 - SimpleLog - INFO: - episode: 2127, ep_reward: -1.025, ep_step: 1000 2023-05-30 01:01:59 - SimpleLog - INFO: - episode: 2128, ep_reward: -52.096, ep_step: 533 2023-05-30 01:02:00 - SimpleLog - INFO: - episode: 2129, ep_reward: -76.238, ep_step: 237 2023-05-30 01:02:01 - SimpleLog - INFO: - episode: 2130, ep_reward: -50.973, ep_step: 536 2023-05-30 01:02:03 - SimpleLog - INFO: - episode: 2131, ep_reward: 9.906, ep_step: 1000 2023-05-30 01:02:04 - SimpleLog - INFO: - episode: 2132, ep_reward: -63.921, ep_step: 282 2023-05-30 01:02:06 - SimpleLog - INFO: - episode: 2133, ep_reward: 15.054, ep_step: 1000 2023-05-30 01:02:09 - SimpleLog - INFO: - episode: 2134, ep_reward: 79.138, ep_step: 1000 2023-05-30 01:02:11 - SimpleLog - INFO: - episode: 2135, ep_reward: -29.662, ep_step: 1000 2023-05-30 01:02:13 - SimpleLog - INFO: - episode: 2136, ep_reward: 15.440, ep_step: 1000 2023-05-30 01:02:15 - SimpleLog - INFO: - episode: 2137, ep_reward: 20.383, ep_step: 1000 2023-05-30 01:02:18 - SimpleLog - INFO: - episode: 2138, ep_reward: 18.943, ep_step: 1000 2023-05-30 01:02:20 - SimpleLog - INFO: - episode: 2139, ep_reward: -45.742, ep_step: 1000 2023-05-30 01:02:22 - SimpleLog - INFO: - episode: 2140, ep_reward: 25.660, ep_step: 1000 2023-05-30 01:02:24 - SimpleLog - INFO: - episode: 2141, ep_reward: -27.625, ep_step: 1000 2023-05-30 01:02:27 - SimpleLog - INFO: - episode: 2142, ep_reward: -48.357, ep_step: 1000 2023-05-30 01:02:29 - SimpleLog - INFO: - episode: 2143, ep_reward: 23.326, ep_step: 1000 2023-05-30 01:02:31 - SimpleLog - INFO: - episode: 2144, ep_reward: 86.481, ep_step: 1000 2023-05-30 01:02:32 - SimpleLog - INFO: - episode: 2145, ep_reward: -93.440, ep_step: 215 2023-05-30 01:02:34 - SimpleLog - INFO: - episode: 2146, ep_reward: -30.937, ep_step: 1000 2023-05-30 01:02:36 - SimpleLog - INFO: - episode: 2147, ep_reward: 21.449, ep_step: 1000 2023-05-30 01:02:39 - SimpleLog - INFO: - episode: 2148, ep_reward: 91.643, ep_step: 1000 2023-05-30 01:02:41 - SimpleLog - INFO: - episode: 2149, ep_reward: -53.881, ep_step: 1000 2023-05-30 01:02:43 - SimpleLog - INFO: - episode: 2150, ep_reward: 17.128, ep_step: 1000 2023-05-30 01:02:45 - SimpleLog - INFO: - episode: 2151, ep_reward: -51.854, ep_step: 1000 2023-05-30 01:02:48 - SimpleLog - INFO: - episode: 2152, ep_reward: -3.733, ep_step: 1000 2023-05-30 01:02:50 - SimpleLog - INFO: - episode: 2153, ep_reward: -52.574, ep_step: 1000 2023-05-30 01:02:51 - SimpleLog - INFO: - episode: 2154, ep_reward: -59.528, ep_step: 432 2023-05-30 01:02:52 - SimpleLog - INFO: - episode: 2155, ep_reward: -69.222, ep_step: 465 2023-05-30 01:02:54 - SimpleLog - INFO: - episode: 2156, ep_reward: -53.288, ep_step: 1000 2023-05-30 01:02:56 - SimpleLog - INFO: - episode: 2157, ep_reward: 18.041, ep_step: 1000 2023-05-30 01:02:59 - SimpleLog - INFO: - episode: 2158, ep_reward: 15.754, ep_step: 1000 2023-05-30 01:03:01 - SimpleLog - INFO: - episode: 2159, ep_reward: -22.236, ep_step: 1000 2023-05-30 01:03:03 - SimpleLog - INFO: - episode: 2160, ep_reward: -51.342, ep_step: 1000 2023-05-30 01:03:04 - SimpleLog - INFO: - episode: 2161, ep_reward: -57.907, ep_step: 405 2023-05-30 01:03:05 - SimpleLog - INFO: - episode: 2162, ep_reward: -66.991, ep_step: 521 2023-05-30 01:03:10 - SimpleLog - INFO: - update_step: 1400000, online_eval_reward: 47.143 2023-05-30 01:03:10 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 47.143, save the best model! 2023-05-30 01:03:10 - SimpleLog - INFO: - episode: 2163, ep_reward: -51.705, ep_step: 1000 2023-05-30 01:03:12 - SimpleLog - INFO: - episode: 2164, ep_reward: -49.118, ep_step: 1000 2023-05-30 01:03:14 - SimpleLog - INFO: - episode: 2165, ep_reward: 0.330, ep_step: 1000 2023-05-30 01:03:16 - SimpleLog - INFO: - episode: 2166, ep_reward: 4.740, ep_step: 1000 2023-05-30 01:03:19 - SimpleLog - INFO: - episode: 2167, ep_reward: 25.146, ep_step: 1000 2023-05-30 01:03:21 - SimpleLog - INFO: - episode: 2168, ep_reward: -32.313, ep_step: 1000 2023-05-30 01:03:23 - SimpleLog - INFO: - episode: 2169, ep_reward: 14.883, ep_step: 1000 2023-05-30 01:03:26 - SimpleLog - INFO: - episode: 2170, ep_reward: 15.813, ep_step: 1000 2023-05-30 01:03:26 - SimpleLog - INFO: - episode: 2171, ep_reward: -117.137, ep_step: 59 2023-05-30 01:03:28 - SimpleLog - INFO: - episode: 2172, ep_reward: 15.860, ep_step: 1000 2023-05-30 01:03:30 - SimpleLog - INFO: - episode: 2173, ep_reward: -39.489, ep_step: 1000 2023-05-30 01:03:31 - SimpleLog - INFO: - episode: 2174, ep_reward: -90.703, ep_step: 187 2023-05-30 01:03:33 - SimpleLog - INFO: - episode: 2175, ep_reward: 3.970, ep_step: 1000 2023-05-30 01:03:35 - SimpleLog - INFO: - episode: 2176, ep_reward: 1.780, ep_step: 1000 2023-05-30 01:03:37 - SimpleLog - INFO: - episode: 2177, ep_reward: 30.575, ep_step: 1000 2023-05-30 01:03:40 - SimpleLog - INFO: - episode: 2178, ep_reward: -18.433, ep_step: 1000 2023-05-30 01:03:42 - SimpleLog - INFO: - episode: 2179, ep_reward: -34.022, ep_step: 1000 2023-05-30 01:03:44 - SimpleLog - INFO: - episode: 2180, ep_reward: -24.312, ep_step: 1000 2023-05-30 01:03:46 - SimpleLog - INFO: - episode: 2181, ep_reward: -48.589, ep_step: 540 2023-05-30 01:03:46 - SimpleLog - INFO: - episode: 2182, ep_reward: -81.131, ep_step: 391 2023-05-30 01:03:47 - SimpleLog - INFO: - episode: 2183, ep_reward: -39.925, ep_step: 377 2023-05-30 01:03:50 - SimpleLog - INFO: - episode: 2184, ep_reward: -19.527, ep_step: 1000 2023-05-30 01:03:50 - SimpleLog - INFO: - episode: 2185, ep_reward: -80.713, ep_step: 309 2023-05-30 01:03:53 - SimpleLog - INFO: - episode: 2186, ep_reward: 23.472, ep_step: 1000 2023-05-30 01:03:55 - SimpleLog - INFO: - episode: 2187, ep_reward: 2.171, ep_step: 1000 2023-05-30 01:03:57 - SimpleLog - INFO: - episode: 2188, ep_reward: 2.501, ep_step: 1000 2023-05-30 01:03:59 - SimpleLog - INFO: - episode: 2189, ep_reward: -1.716, ep_step: 1000 2023-05-30 01:04:00 - SimpleLog - INFO: - episode: 2190, ep_reward: -81.306, ep_step: 221 2023-05-30 01:04:02 - SimpleLog - INFO: - episode: 2191, ep_reward: 19.240, ep_step: 1000 2023-05-30 01:04:03 - SimpleLog - INFO: - episode: 2192, ep_reward: -51.923, ep_step: 514 2023-05-30 01:04:06 - SimpleLog - INFO: - episode: 2193, ep_reward: 22.697, ep_step: 1000 2023-05-30 01:04:07 - SimpleLog - INFO: - episode: 2194, ep_reward: -52.716, ep_step: 564 2023-05-30 01:04:09 - SimpleLog - INFO: - episode: 2195, ep_reward: 33.241, ep_step: 1000 2023-05-30 01:04:10 - SimpleLog - INFO: - episode: 2196, ep_reward: -25.214, ep_step: 533 2023-05-30 01:04:13 - SimpleLog - INFO: - episode: 2197, ep_reward: -85.748, ep_step: 968 2023-05-30 01:04:14 - SimpleLog - INFO: - episode: 2198, ep_reward: -47.074, ep_step: 487 2023-05-30 01:04:16 - SimpleLog - INFO: - episode: 2199, ep_reward: 9.771, ep_step: 1000 2023-05-30 01:04:18 - SimpleLog - INFO: - episode: 2200, ep_reward: 5.731, ep_step: 1000 2023-05-30 01:04:20 - SimpleLog - INFO: - episode: 2201, ep_reward: -58.519, ep_step: 620 2023-05-30 01:04:21 - SimpleLog - INFO: - episode: 2202, ep_reward: -52.588, ep_step: 526 2023-05-30 01:04:23 - SimpleLog - INFO: - episode: 2203, ep_reward: 26.055, ep_step: 1000 2023-05-30 01:04:25 - SimpleLog - INFO: - episode: 2204, ep_reward: 26.674, ep_step: 1000 2023-05-30 01:04:28 - SimpleLog - INFO: - episode: 2205, ep_reward: -28.340, ep_step: 1000 2023-05-30 01:04:30 - SimpleLog - INFO: - episode: 2206, ep_reward: 88.481, ep_step: 1000 2023-05-30 01:04:32 - SimpleLog - INFO: - episode: 2207, ep_reward: 21.878, ep_step: 1000 2023-05-30 01:04:34 - SimpleLog - INFO: - episode: 2208, ep_reward: 27.035, ep_step: 1000 2023-05-30 01:04:37 - SimpleLog - INFO: - episode: 2209, ep_reward: 25.904, ep_step: 1000 2023-05-30 01:04:39 - SimpleLog - INFO: - episode: 2210, ep_reward: 35.299, ep_step: 1000 2023-05-30 01:04:41 - SimpleLog - INFO: - episode: 2211, ep_reward: 13.964, ep_step: 1000 2023-05-30 01:04:44 - SimpleLog - INFO: - episode: 2212, ep_reward: 21.072, ep_step: 1000 2023-05-30 01:04:46 - SimpleLog - INFO: - episode: 2213, ep_reward: 24.548, ep_step: 1000 2023-05-30 01:04:48 - SimpleLog - INFO: - episode: 2214, ep_reward: 23.687, ep_step: 1000 2023-05-30 01:04:50 - SimpleLog - INFO: - episode: 2215, ep_reward: 3.747, ep_step: 1000 2023-05-30 01:04:52 - SimpleLog - INFO: - episode: 2216, ep_reward: -105.423, ep_step: 913 2023-05-30 01:04:55 - SimpleLog - INFO: - episode: 2217, ep_reward: 13.744, ep_step: 1000 2023-05-30 01:04:57 - SimpleLog - INFO: - episode: 2218, ep_reward: 24.767, ep_step: 1000 2023-05-30 01:04:59 - SimpleLog - INFO: - episode: 2219, ep_reward: 27.037, ep_step: 1000 2023-05-30 01:05:02 - SimpleLog - INFO: - episode: 2220, ep_reward: 25.452, ep_step: 1000 2023-05-30 01:05:03 - SimpleLog - INFO: - episode: 2221, ep_reward: -66.158, ep_step: 586 2023-05-30 01:05:05 - SimpleLog - INFO: - episode: 2222, ep_reward: 23.028, ep_step: 1000 2023-05-30 01:05:07 - SimpleLog - INFO: - episode: 2223, ep_reward: 18.504, ep_step: 1000 2023-05-30 01:05:09 - SimpleLog - INFO: - episode: 2224, ep_reward: -92.132, ep_step: 534 2023-05-30 01:05:11 - SimpleLog - INFO: - episode: 2225, ep_reward: 24.024, ep_step: 1000 2023-05-30 01:05:13 - SimpleLog - INFO: - episode: 2226, ep_reward: 24.320, ep_step: 1000 2023-05-30 01:05:16 - SimpleLog - INFO: - episode: 2227, ep_reward: 23.147, ep_step: 1000 2023-05-30 01:05:18 - SimpleLog - INFO: - episode: 2228, ep_reward: 87.426, ep_step: 1000 2023-05-30 01:05:19 - SimpleLog - INFO: - episode: 2229, ep_reward: -82.857, ep_step: 597 2023-05-30 01:05:21 - SimpleLog - INFO: - episode: 2230, ep_reward: -50.559, ep_step: 1000 2023-05-30 01:05:24 - SimpleLog - INFO: - episode: 2231, ep_reward: -55.061, ep_step: 1000 2023-05-30 01:05:26 - SimpleLog - INFO: - episode: 2232, ep_reward: -1.515, ep_step: 867 2023-05-30 01:05:28 - SimpleLog - INFO: - episode: 2233, ep_reward: -50.801, ep_step: 1000 2023-05-30 01:05:30 - SimpleLog - INFO: - episode: 2234, ep_reward: 24.465, ep_step: 1000 2023-05-30 01:05:31 - SimpleLog - INFO: - episode: 2235, ep_reward: -48.940, ep_step: 452 2023-05-30 01:05:34 - SimpleLog - INFO: - episode: 2236, ep_reward: 20.534, ep_step: 1000 2023-05-30 01:05:36 - SimpleLog - INFO: - episode: 2237, ep_reward: 17.596, ep_step: 1000 2023-05-30 01:05:38 - SimpleLog - INFO: - episode: 2238, ep_reward: 87.122, ep_step: 1000 2023-05-30 01:05:40 - SimpleLog - INFO: - episode: 2239, ep_reward: 19.906, ep_step: 1000 2023-05-30 01:05:43 - SimpleLog - INFO: - episode: 2240, ep_reward: -53.922, ep_step: 1000 2023-05-30 01:05:45 - SimpleLog - INFO: - episode: 2241, ep_reward: 85.081, ep_step: 1000 2023-05-30 01:05:47 - SimpleLog - INFO: - episode: 2242, ep_reward: 18.442, ep_step: 1000 2023-05-30 01:05:48 - SimpleLog - INFO: - episode: 2243, ep_reward: -44.060, ep_step: 412 2023-05-30 01:05:51 - SimpleLog - INFO: - update_step: 1470000, online_eval_reward: 12.758 2023-05-30 01:05:51 - SimpleLog - INFO: - episode: 2244, ep_reward: -68.029, ep_step: 519 2023-05-30 01:05:52 - SimpleLog - INFO: - episode: 2245, ep_reward: -104.867, ep_step: 220 2023-05-30 01:05:54 - SimpleLog - INFO: - episode: 2246, ep_reward: 16.871, ep_step: 1000 2023-05-30 01:05:56 - SimpleLog - INFO: - episode: 2247, ep_reward: 16.244, ep_step: 1000 2023-05-30 01:05:58 - SimpleLog - INFO: - episode: 2248, ep_reward: -60.379, ep_step: 662 2023-05-30 01:06:00 - SimpleLog - INFO: - episode: 2249, ep_reward: -35.985, ep_step: 1000 2023-05-30 01:06:02 - SimpleLog - INFO: - episode: 2250, ep_reward: 18.879, ep_step: 1000 2023-05-30 01:06:05 - SimpleLog - INFO: - episode: 2251, ep_reward: 13.138, ep_step: 1000 2023-05-30 01:06:07 - SimpleLog - INFO: - episode: 2252, ep_reward: 9.754, ep_step: 1000 2023-05-30 01:06:08 - SimpleLog - INFO: - episode: 2253, ep_reward: -31.434, ep_step: 588 2023-05-30 01:06:11 - SimpleLog - INFO: - episode: 2254, ep_reward: -38.002, ep_step: 1000 2023-05-30 01:06:13 - SimpleLog - INFO: - episode: 2255, ep_reward: 20.382, ep_step: 1000 2023-05-30 01:06:15 - SimpleLog - INFO: - episode: 2256, ep_reward: 5.212, ep_step: 1000 2023-05-30 01:06:17 - SimpleLog - INFO: - episode: 2257, ep_reward: 23.498, ep_step: 1000 2023-05-30 01:06:20 - SimpleLog - INFO: - episode: 2258, ep_reward: 1.630, ep_step: 1000 2023-05-30 01:06:22 - SimpleLog - INFO: - episode: 2259, ep_reward: 6.930, ep_step: 1000 2023-05-30 01:06:24 - SimpleLog - INFO: - episode: 2260, ep_reward: 21.728, ep_step: 1000 2023-05-30 01:06:26 - SimpleLog - INFO: - episode: 2261, ep_reward: 21.462, ep_step: 1000 2023-05-30 01:06:27 - SimpleLog - INFO: - episode: 2262, ep_reward: -46.133, ep_step: 450 2023-05-30 01:06:29 - SimpleLog - INFO: - episode: 2263, ep_reward: -48.853, ep_step: 479 2023-05-30 01:06:31 - SimpleLog - INFO: - episode: 2264, ep_reward: 19.524, ep_step: 1000 2023-05-30 01:06:33 - SimpleLog - INFO: - episode: 2265, ep_reward: -1.883, ep_step: 1000 2023-05-30 01:06:35 - SimpleLog - INFO: - episode: 2266, ep_reward: 20.129, ep_step: 1000 2023-05-30 01:06:38 - SimpleLog - INFO: - episode: 2267, ep_reward: 80.520, ep_step: 1000 2023-05-30 01:06:40 - SimpleLog - INFO: - episode: 2268, ep_reward: 16.926, ep_step: 1000 2023-05-30 01:06:42 - SimpleLog - INFO: - episode: 2269, ep_reward: 6.111, ep_step: 1000 2023-05-30 01:06:44 - SimpleLog - INFO: - episode: 2270, ep_reward: 16.621, ep_step: 1000 2023-05-30 01:06:46 - SimpleLog - INFO: - episode: 2271, ep_reward: -49.849, ep_step: 524 2023-05-30 01:06:47 - SimpleLog - INFO: - episode: 2272, ep_reward: -53.488, ep_step: 587 2023-05-30 01:06:49 - SimpleLog - INFO: - episode: 2273, ep_reward: 5.169, ep_step: 1000 2023-05-30 01:06:51 - SimpleLog - INFO: - episode: 2274, ep_reward: -86.259, ep_step: 627 2023-05-30 01:06:52 - SimpleLog - INFO: - episode: 2275, ep_reward: -83.593, ep_step: 602 2023-05-30 01:06:54 - SimpleLog - INFO: - episode: 2276, ep_reward: 18.335, ep_step: 1000 2023-05-30 01:06:56 - SimpleLog - INFO: - episode: 2277, ep_reward: 20.314, ep_step: 1000 2023-05-30 01:06:59 - SimpleLog - INFO: - episode: 2278, ep_reward: 21.545, ep_step: 1000 2023-05-30 01:07:01 - SimpleLog - INFO: - episode: 2279, ep_reward: 18.415, ep_step: 1000 2023-05-30 01:07:03 - SimpleLog - INFO: - episode: 2280, ep_reward: 10.419, ep_step: 1000 2023-05-30 01:07:04 - SimpleLog - INFO: - episode: 2281, ep_reward: -72.542, ep_step: 504 2023-05-30 01:07:07 - SimpleLog - INFO: - episode: 2282, ep_reward: 26.976, ep_step: 1000 2023-05-30 01:07:08 - SimpleLog - INFO: - episode: 2283, ep_reward: -53.185, ep_step: 470 2023-05-30 01:07:10 - SimpleLog - INFO: - episode: 2284, ep_reward: -2.889, ep_step: 1000 2023-05-30 01:07:12 - SimpleLog - INFO: - episode: 2285, ep_reward: 39.777, ep_step: 1000 2023-05-30 01:07:14 - SimpleLog - INFO: - episode: 2286, ep_reward: 83.391, ep_step: 1000 2023-05-30 01:07:17 - SimpleLog - INFO: - episode: 2287, ep_reward: 22.938, ep_step: 1000 2023-05-30 01:07:19 - SimpleLog - INFO: - episode: 2288, ep_reward: 83.514, ep_step: 1000 2023-05-30 01:07:21 - SimpleLog - INFO: - episode: 2289, ep_reward: 22.052, ep_step: 1000 2023-05-30 01:07:24 - SimpleLog - INFO: - episode: 2290, ep_reward: 0.212, ep_step: 1000 2023-05-30 01:07:26 - SimpleLog - INFO: - episode: 2291, ep_reward: 87.110, ep_step: 1000 2023-05-30 01:07:28 - SimpleLog - INFO: - episode: 2292, ep_reward: 25.345, ep_step: 1000 2023-05-30 01:07:30 - SimpleLog - INFO: - episode: 2293, ep_reward: 76.111, ep_step: 1000 2023-05-30 01:07:33 - SimpleLog - INFO: - episode: 2294, ep_reward: -2.313, ep_step: 1000 2023-05-30 01:07:35 - SimpleLog - INFO: - episode: 2295, ep_reward: -0.337, ep_step: 1000 2023-05-30 01:07:37 - SimpleLog - INFO: - episode: 2296, ep_reward: 77.686, ep_step: 1000 2023-05-30 01:07:40 - SimpleLog - INFO: - episode: 2297, ep_reward: 3.215, ep_step: 1000 2023-05-30 01:07:41 - SimpleLog - INFO: - episode: 2298, ep_reward: -52.001, ep_step: 555 2023-05-30 01:07:43 - SimpleLog - INFO: - episode: 2299, ep_reward: 6.439, ep_step: 1000 2023-05-30 01:07:45 - SimpleLog - INFO: - episode: 2300, ep_reward: 3.955, ep_step: 1000 2023-05-30 01:07:48 - SimpleLog - INFO: - episode: 2301, ep_reward: -19.451, ep_step: 1000 2023-05-30 01:07:50 - SimpleLog - INFO: - episode: 2302, ep_reward: -37.250, ep_step: 1000 2023-05-30 01:07:52 - SimpleLog - INFO: - episode: 2303, ep_reward: 0.869, ep_step: 1000 2023-05-30 01:07:54 - SimpleLog - INFO: - episode: 2304, ep_reward: 23.502, ep_step: 1000 2023-05-30 01:07:57 - SimpleLog - INFO: - episode: 2305, ep_reward: 23.608, ep_step: 1000 2023-05-30 01:07:58 - SimpleLog - INFO: - episode: 2306, ep_reward: -56.058, ep_step: 455 2023-05-30 01:07:59 - SimpleLog - INFO: - episode: 2307, ep_reward: -44.705, ep_step: 466 2023-05-30 01:08:01 - SimpleLog - INFO: - episode: 2308, ep_reward: 7.823, ep_step: 1000 2023-05-30 01:08:03 - SimpleLog - INFO: - episode: 2309, ep_reward: 24.831, ep_step: 1000 2023-05-30 01:08:06 - SimpleLog - INFO: - episode: 2310, ep_reward: -36.363, ep_step: 1000 2023-05-30 01:08:08 - SimpleLog - INFO: - episode: 2311, ep_reward: 77.746, ep_step: 1000 2023-05-30 01:08:10 - SimpleLog - INFO: - episode: 2312, ep_reward: 25.715, ep_step: 1000 2023-05-30 01:08:12 - SimpleLog - INFO: - episode: 2313, ep_reward: 9.409, ep_step: 1000 2023-05-30 01:08:15 - SimpleLog - INFO: - episode: 2314, ep_reward: 79.255, ep_step: 1000 2023-05-30 01:08:16 - SimpleLog - INFO: - episode: 2315, ep_reward: -51.863, ep_step: 446 2023-05-30 01:08:17 - SimpleLog - INFO: - episode: 2316, ep_reward: 5.706, ep_step: 578 2023-05-30 01:08:18 - SimpleLog - INFO: - episode: 2317, ep_reward: -44.588, ep_step: 528 2023-05-30 01:08:19 - SimpleLog - INFO: - episode: 2318, ep_reward: -86.033, ep_step: 150 2023-05-30 01:08:19 - SimpleLog - INFO: - episode: 2319, ep_reward: -65.022, ep_step: 385 2023-05-30 01:08:22 - SimpleLog - INFO: - episode: 2320, ep_reward: 23.976, ep_step: 1000 2023-05-30 01:08:23 - SimpleLog - INFO: - episode: 2321, ep_reward: 7.252, ep_step: 749 2023-05-30 01:08:26 - SimpleLog - INFO: - episode: 2322, ep_reward: 8.737, ep_step: 1000 2023-05-30 01:08:28 - SimpleLog - INFO: - episode: 2323, ep_reward: -10.926, ep_step: 1000 2023-05-30 01:08:31 - SimpleLog - INFO: - update_step: 1540000, online_eval_reward: -53.228 2023-05-30 01:08:31 - SimpleLog - INFO: - episode: 2324, ep_reward: 27.375, ep_step: 1000 2023-05-30 01:08:33 - SimpleLog - INFO: - episode: 2325, ep_reward: 9.793, ep_step: 1000 2023-05-30 01:08:36 - SimpleLog - INFO: - episode: 2326, ep_reward: 23.597, ep_step: 1000 2023-05-30 01:08:37 - SimpleLog - INFO: - episode: 2327, ep_reward: -46.016, ep_step: 509 2023-05-30 01:08:38 - SimpleLog - INFO: - episode: 2328, ep_reward: -45.632, ep_step: 387 2023-05-30 01:08:40 - SimpleLog - INFO: - episode: 2329, ep_reward: 21.516, ep_step: 1000 2023-05-30 01:08:42 - SimpleLog - INFO: - episode: 2330, ep_reward: 24.987, ep_step: 1000 2023-05-30 01:08:43 - SimpleLog - INFO: - episode: 2331, ep_reward: -53.204, ep_step: 340 2023-05-30 01:08:45 - SimpleLog - INFO: - episode: 2332, ep_reward: 23.683, ep_step: 1000 2023-05-30 01:08:47 - SimpleLog - INFO: - episode: 2333, ep_reward: 6.027, ep_step: 704 2023-05-30 01:08:49 - SimpleLog - INFO: - episode: 2334, ep_reward: 10.047, ep_step: 1000 2023-05-30 01:08:50 - SimpleLog - INFO: - episode: 2335, ep_reward: -51.622, ep_step: 465 2023-05-30 01:08:50 - SimpleLog - INFO: - episode: 2336, ep_reward: -94.774, ep_step: 151 2023-05-30 01:08:51 - SimpleLog - INFO: - episode: 2337, ep_reward: -47.167, ep_step: 430 2023-05-30 01:08:52 - SimpleLog - INFO: - episode: 2338, ep_reward: -57.350, ep_step: 469 2023-05-30 01:08:55 - SimpleLog - INFO: - episode: 2339, ep_reward: -5.300, ep_step: 1000 2023-05-30 01:08:57 - SimpleLog - INFO: - episode: 2340, ep_reward: 22.536, ep_step: 1000 2023-05-30 01:08:59 - SimpleLog - INFO: - episode: 2341, ep_reward: -48.259, ep_step: 1000 2023-05-30 01:09:02 - SimpleLog - INFO: - episode: 2342, ep_reward: 93.313, ep_step: 1000 2023-05-30 01:09:04 - SimpleLog - INFO: - episode: 2343, ep_reward: 29.795, ep_step: 1000 2023-05-30 01:09:05 - SimpleLog - INFO: - episode: 2344, ep_reward: -52.425, ep_step: 482 2023-05-30 01:09:07 - SimpleLog - INFO: - episode: 2345, ep_reward: 84.127, ep_step: 1000 2023-05-30 01:09:08 - SimpleLog - INFO: - episode: 2346, ep_reward: -41.039, ep_step: 504 2023-05-30 01:09:09 - SimpleLog - INFO: - episode: 2347, ep_reward: -44.110, ep_step: 394 2023-05-30 01:09:10 - SimpleLog - INFO: - episode: 2348, ep_reward: -43.245, ep_step: 578 2023-05-30 01:09:13 - SimpleLog - INFO: - episode: 2349, ep_reward: -31.515, ep_step: 964 2023-05-30 01:09:15 - SimpleLog - INFO: - episode: 2350, ep_reward: 81.976, ep_step: 1000 2023-05-30 01:09:17 - SimpleLog - INFO: - episode: 2351, ep_reward: 11.289, ep_step: 1000 2023-05-30 01:09:19 - SimpleLog - INFO: - episode: 2352, ep_reward: -68.497, ep_step: 592 2023-05-30 01:09:21 - SimpleLog - INFO: - episode: 2353, ep_reward: 97.362, ep_step: 1000 2023-05-30 01:09:23 - SimpleLog - INFO: - episode: 2354, ep_reward: 30.855, ep_step: 1000 2023-05-30 01:09:25 - SimpleLog - INFO: - episode: 2355, ep_reward: 11.444, ep_step: 1000 2023-05-30 01:09:27 - SimpleLog - INFO: - episode: 2356, ep_reward: -37.949, ep_step: 638 2023-05-30 01:09:29 - SimpleLog - INFO: - episode: 2357, ep_reward: 44.848, ep_step: 1000 2023-05-30 01:09:31 - SimpleLog - INFO: - episode: 2358, ep_reward: -27.548, ep_step: 1000 2023-05-30 01:09:33 - SimpleLog - INFO: - episode: 2359, ep_reward: 6.486, ep_step: 723 2023-05-30 01:09:35 - SimpleLog - INFO: - episode: 2360, ep_reward: 81.309, ep_step: 1000 2023-05-30 01:09:38 - SimpleLog - INFO: - episode: 2361, ep_reward: 22.127, ep_step: 1000 2023-05-30 01:09:40 - SimpleLog - INFO: - episode: 2362, ep_reward: 84.228, ep_step: 1000 2023-05-30 01:09:41 - SimpleLog - INFO: - episode: 2363, ep_reward: -49.579, ep_step: 482 2023-05-30 01:09:43 - SimpleLog - INFO: - episode: 2364, ep_reward: 30.786, ep_step: 1000 2023-05-30 01:09:46 - SimpleLog - INFO: - episode: 2365, ep_reward: 30.720, ep_step: 1000 2023-05-30 01:09:48 - SimpleLog - INFO: - episode: 2366, ep_reward: 30.443, ep_step: 1000 2023-05-30 01:09:50 - SimpleLog - INFO: - episode: 2367, ep_reward: 23.315, ep_step: 1000 2023-05-30 01:09:52 - SimpleLog - INFO: - episode: 2368, ep_reward: 82.272, ep_step: 1000 2023-05-30 01:09:55 - SimpleLog - INFO: - episode: 2369, ep_reward: -10.964, ep_step: 1000 2023-05-30 01:09:57 - SimpleLog - INFO: - episode: 2370, ep_reward: 84.563, ep_step: 1000 2023-05-30 01:09:59 - SimpleLog - INFO: - episode: 2371, ep_reward: 84.357, ep_step: 1000 2023-05-30 01:10:02 - SimpleLog - INFO: - episode: 2372, ep_reward: 85.894, ep_step: 1000 2023-05-30 01:10:04 - SimpleLog - INFO: - episode: 2373, ep_reward: 85.361, ep_step: 1000 2023-05-30 01:10:06 - SimpleLog - INFO: - episode: 2374, ep_reward: -53.450, ep_step: 1000 2023-05-30 01:10:07 - SimpleLog - INFO: - episode: 2375, ep_reward: -49.173, ep_step: 479 2023-05-30 01:10:10 - SimpleLog - INFO: - episode: 2376, ep_reward: 85.722, ep_step: 1000 2023-05-30 01:10:12 - SimpleLog - INFO: - episode: 2377, ep_reward: 28.099, ep_step: 1000 2023-05-30 01:10:14 - SimpleLog - INFO: - episode: 2378, ep_reward: 25.544, ep_step: 1000 2023-05-30 01:10:16 - SimpleLog - INFO: - episode: 2379, ep_reward: 82.897, ep_step: 1000 2023-05-30 01:10:18 - SimpleLog - INFO: - episode: 2380, ep_reward: 4.186, ep_step: 880 2023-05-30 01:10:21 - SimpleLog - INFO: - episode: 2381, ep_reward: -36.892, ep_step: 1000 2023-05-30 01:10:22 - SimpleLog - INFO: - episode: 2382, ep_reward: 7.270, ep_step: 717 2023-05-30 01:10:24 - SimpleLog - INFO: - episode: 2383, ep_reward: 83.569, ep_step: 1000 2023-05-30 01:10:27 - SimpleLog - INFO: - episode: 2384, ep_reward: 84.631, ep_step: 1000 2023-05-30 01:10:29 - SimpleLog - INFO: - episode: 2385, ep_reward: 96.065, ep_step: 1000 2023-05-30 01:10:31 - SimpleLog - INFO: - episode: 2386, ep_reward: 83.461, ep_step: 1000 2023-05-30 01:10:32 - SimpleLog - INFO: - episode: 2387, ep_reward: -32.339, ep_step: 519 2023-05-30 01:10:35 - SimpleLog - INFO: - episode: 2388, ep_reward: 83.936, ep_step: 1000 2023-05-30 01:10:36 - SimpleLog - INFO: - episode: 2389, ep_reward: -51.294, ep_step: 481 2023-05-30 01:10:38 - SimpleLog - INFO: - episode: 2390, ep_reward: 85.376, ep_step: 1000 2023-05-30 01:10:40 - SimpleLog - INFO: - episode: 2391, ep_reward: 84.026, ep_step: 1000 2023-05-30 01:10:43 - SimpleLog - INFO: - episode: 2392, ep_reward: 8.207, ep_step: 1000 2023-05-30 01:10:45 - SimpleLog - INFO: - episode: 2393, ep_reward: 32.244, ep_step: 1000 2023-05-30 01:10:47 - SimpleLog - INFO: - episode: 2394, ep_reward: 9.540, ep_step: 1000 2023-05-30 01:10:49 - SimpleLog - INFO: - episode: 2395, ep_reward: 84.767, ep_step: 1000 2023-05-30 01:10:52 - SimpleLog - INFO: - episode: 2396, ep_reward: 13.234, ep_step: 1000 2023-05-30 01:10:54 - SimpleLog - INFO: - episode: 2397, ep_reward: 82.693, ep_step: 1000 2023-05-30 01:10:56 - SimpleLog - INFO: - episode: 2398, ep_reward: -43.910, ep_step: 1000 2023-05-30 01:10:59 - SimpleLog - INFO: - episode: 2399, ep_reward: 90.523, ep_step: 1000 2023-05-30 01:11:01 - SimpleLog - INFO: - episode: 2400, ep_reward: 90.740, ep_step: 1000 2023-05-30 01:11:03 - SimpleLog - INFO: - episode: 2401, ep_reward: 85.054, ep_step: 1000 2023-05-30 01:11:06 - SimpleLog - INFO: - episode: 2402, ep_reward: 89.279, ep_step: 1000 2023-05-30 01:11:07 - SimpleLog - INFO: - episode: 2403, ep_reward: -7.584, ep_step: 836 2023-05-30 01:11:08 - SimpleLog - INFO: - episode: 2404, ep_reward: -66.093, ep_step: 480 2023-05-30 01:11:11 - SimpleLog - INFO: - update_step: 1610000, online_eval_reward: 6.125 2023-05-30 01:11:12 - SimpleLog - INFO: - episode: 2405, ep_reward: 13.706, ep_step: 1000 2023-05-30 01:11:14 - SimpleLog - INFO: - episode: 2406, ep_reward: 83.769, ep_step: 1000 2023-05-30 01:11:15 - SimpleLog - INFO: - episode: 2407, ep_reward: -62.049, ep_step: 543 2023-05-30 01:11:18 - SimpleLog - INFO: - episode: 2408, ep_reward: 23.669, ep_step: 1000 2023-05-30 01:11:19 - SimpleLog - INFO: - episode: 2409, ep_reward: -36.848, ep_step: 582 2023-05-30 01:11:21 - SimpleLog - INFO: - episode: 2410, ep_reward: 27.371, ep_step: 1000 2023-05-30 01:11:24 - SimpleLog - INFO: - episode: 2411, ep_reward: -2.076, ep_step: 1000 2023-05-30 01:11:26 - SimpleLog - INFO: - episode: 2412, ep_reward: 82.830, ep_step: 1000 2023-05-30 01:11:28 - SimpleLog - INFO: - episode: 2413, ep_reward: 22.364, ep_step: 1000 2023-05-30 01:11:30 - SimpleLog - INFO: - episode: 2414, ep_reward: 12.976, ep_step: 1000 2023-05-30 01:11:33 - SimpleLog - INFO: - episode: 2415, ep_reward: 14.358, ep_step: 1000 2023-05-30 01:11:35 - SimpleLog - INFO: - episode: 2416, ep_reward: 83.465, ep_step: 1000 2023-05-30 01:11:37 - SimpleLog - INFO: - episode: 2417, ep_reward: 2.670, ep_step: 733 2023-05-30 01:11:39 - SimpleLog - INFO: - episode: 2418, ep_reward: 93.326, ep_step: 1000 2023-05-30 01:11:41 - SimpleLog - INFO: - episode: 2419, ep_reward: 81.725, ep_step: 1000 2023-05-30 01:11:41 - SimpleLog - INFO: - episode: 2420, ep_reward: -92.741, ep_step: 156 2023-05-30 01:11:44 - SimpleLog - INFO: - episode: 2421, ep_reward: 25.229, ep_step: 1000 2023-05-30 01:11:46 - SimpleLog - INFO: - episode: 2422, ep_reward: 23.012, ep_step: 1000 2023-05-30 01:11:48 - SimpleLog - INFO: - episode: 2423, ep_reward: 83.222, ep_step: 1000 2023-05-30 01:11:50 - SimpleLog - INFO: - episode: 2424, ep_reward: 92.091, ep_step: 1000 2023-05-30 01:11:53 - SimpleLog - INFO: - episode: 2425, ep_reward: 26.543, ep_step: 1000 2023-05-30 01:11:55 - SimpleLog - INFO: - episode: 2426, ep_reward: 82.017, ep_step: 1000 2023-05-30 01:11:57 - SimpleLog - INFO: - episode: 2427, ep_reward: 84.028, ep_step: 1000 2023-05-30 01:11:59 - SimpleLog - INFO: - episode: 2428, ep_reward: 28.909, ep_step: 1000 2023-05-30 01:12:02 - SimpleLog - INFO: - episode: 2429, ep_reward: 94.257, ep_step: 1000 2023-05-30 01:12:04 - SimpleLog - INFO: - episode: 2430, ep_reward: 8.061, ep_step: 874 2023-05-30 01:12:06 - SimpleLog - INFO: - episode: 2431, ep_reward: 6.935, ep_step: 862 2023-05-30 01:12:08 - SimpleLog - INFO: - episode: 2432, ep_reward: 86.129, ep_step: 1000 2023-05-30 01:12:10 - SimpleLog - INFO: - episode: 2433, ep_reward: 0.009, ep_step: 782 2023-05-30 01:12:12 - SimpleLog - INFO: - episode: 2434, ep_reward: 94.831, ep_step: 1000 2023-05-30 01:12:14 - SimpleLog - INFO: - episode: 2435, ep_reward: 18.580, ep_step: 1000 2023-05-30 01:12:16 - SimpleLog - INFO: - episode: 2436, ep_reward: -31.257, ep_step: 1000 2023-05-30 01:12:18 - SimpleLog - INFO: - episode: 2437, ep_reward: -11.467, ep_step: 625 2023-05-30 01:12:20 - SimpleLog - INFO: - episode: 2438, ep_reward: 96.347, ep_step: 1000 2023-05-30 01:12:22 - SimpleLog - INFO: - episode: 2439, ep_reward: 18.005, ep_step: 1000 2023-05-30 01:12:25 - SimpleLog - INFO: - episode: 2440, ep_reward: 17.268, ep_step: 1000 2023-05-30 01:12:27 - SimpleLog - INFO: - episode: 2441, ep_reward: 95.708, ep_step: 1000 2023-05-30 01:12:29 - SimpleLog - INFO: - episode: 2442, ep_reward: 17.593, ep_step: 810 2023-05-30 01:12:31 - SimpleLog - INFO: - episode: 2443, ep_reward: 16.216, ep_step: 1000 2023-05-30 01:12:33 - SimpleLog - INFO: - episode: 2444, ep_reward: 14.896, ep_step: 1000 2023-05-30 01:12:35 - SimpleLog - INFO: - episode: 2445, ep_reward: 109.835, ep_step: 1000 2023-05-30 01:12:38 - SimpleLog - INFO: - episode: 2446, ep_reward: 23.539, ep_step: 1000 2023-05-30 01:12:40 - SimpleLog - INFO: - episode: 2447, ep_reward: 92.400, ep_step: 1000 2023-05-30 01:12:42 - SimpleLog - INFO: - episode: 2448, ep_reward: 20.764, ep_step: 1000 2023-05-30 01:12:44 - SimpleLog - INFO: - episode: 2449, ep_reward: 15.128, ep_step: 1000 2023-05-30 01:12:47 - SimpleLog - INFO: - episode: 2450, ep_reward: 85.312, ep_step: 1000 2023-05-30 01:12:49 - SimpleLog - INFO: - episode: 2451, ep_reward: 8.796, ep_step: 924 2023-05-30 01:12:51 - SimpleLog - INFO: - episode: 2452, ep_reward: 86.894, ep_step: 1000 2023-05-30 01:12:53 - SimpleLog - INFO: - episode: 2453, ep_reward: -10.273, ep_step: 1000 2023-05-30 01:12:56 - SimpleLog - INFO: - episode: 2454, ep_reward: 27.344, ep_step: 1000 2023-05-30 01:12:58 - SimpleLog - INFO: - episode: 2455, ep_reward: 95.837, ep_step: 1000 2023-05-30 01:13:00 - SimpleLog - INFO: - episode: 2456, ep_reward: 28.858, ep_step: 1000 2023-05-30 01:13:00 - SimpleLog - INFO: - episode: 2457, ep_reward: -106.938, ep_step: 104 2023-05-30 01:13:01 - SimpleLog - INFO: - episode: 2458, ep_reward: -101.463, ep_step: 94 2023-05-30 01:13:01 - SimpleLog - INFO: - episode: 2459, ep_reward: -103.508, ep_step: 292 2023-05-30 01:13:02 - SimpleLog - INFO: - episode: 2460, ep_reward: -123.669, ep_step: 450 2023-05-30 01:13:04 - SimpleLog - INFO: - episode: 2461, ep_reward: -143.468, ep_step: 620 2023-05-30 01:13:04 - SimpleLog - INFO: - episode: 2462, ep_reward: -109.530, ep_step: 96 2023-05-30 01:13:04 - SimpleLog - INFO: - episode: 2463, ep_reward: -108.319, ep_step: 196 2023-05-30 01:13:05 - SimpleLog - INFO: - episode: 2464, ep_reward: -105.009, ep_step: 189 2023-05-30 01:13:05 - SimpleLog - INFO: - episode: 2465, ep_reward: -131.221, ep_step: 246 2023-05-30 01:13:06 - SimpleLog - INFO: - episode: 2466, ep_reward: -138.757, ep_step: 272 2023-05-30 01:13:06 - SimpleLog - INFO: - episode: 2467, ep_reward: -113.795, ep_step: 116 2023-05-30 01:13:06 - SimpleLog - INFO: - episode: 2468, ep_reward: -114.509, ep_step: 81 2023-05-30 01:13:07 - SimpleLog - INFO: - episode: 2469, ep_reward: -123.840, ep_step: 120 2023-05-30 01:13:09 - SimpleLog - INFO: - episode: 2470, ep_reward: 13.778, ep_step: 1000 2023-05-30 01:13:11 - SimpleLog - INFO: - episode: 2471, ep_reward: 10.962, ep_step: 1000 2023-05-30 01:13:12 - SimpleLog - INFO: - episode: 2472, ep_reward: -88.024, ep_step: 413 2023-05-30 01:13:14 - SimpleLog - INFO: - episode: 2473, ep_reward: 8.726, ep_step: 1000 2023-05-30 01:13:15 - SimpleLog - INFO: - episode: 2474, ep_reward: -78.701, ep_step: 231 2023-05-30 01:13:15 - SimpleLog - INFO: - episode: 2475, ep_reward: -106.380, ep_step: 291 2023-05-30 01:13:17 - SimpleLog - INFO: - episode: 2476, ep_reward: -58.136, ep_step: 824 2023-05-30 01:13:18 - SimpleLog - INFO: - episode: 2477, ep_reward: -95.051, ep_step: 195 2023-05-30 01:13:20 - SimpleLog - INFO: - episode: 2478, ep_reward: 5.729, ep_step: 1000 2023-05-30 01:13:22 - SimpleLog - INFO: - episode: 2479, ep_reward: 68.883, ep_step: 1000 2023-05-30 01:13:23 - SimpleLog - INFO: - episode: 2480, ep_reward: -160.975, ep_step: 481 2023-05-30 01:13:25 - SimpleLog - INFO: - episode: 2481, ep_reward: -157.710, ep_step: 977 2023-05-30 01:13:27 - SimpleLog - INFO: - episode: 2482, ep_reward: 0.334, ep_step: 674 2023-05-30 01:13:28 - SimpleLog - INFO: - episode: 2483, ep_reward: -35.121, ep_step: 508 2023-05-30 01:13:30 - SimpleLog - INFO: - episode: 2484, ep_reward: -28.772, ep_step: 767 2023-05-30 01:13:32 - SimpleLog - INFO: - episode: 2485, ep_reward: 5.377, ep_step: 901 2023-05-30 01:13:34 - SimpleLog - INFO: - episode: 2486, ep_reward: 87.681, ep_step: 1000 2023-05-30 01:13:36 - SimpleLog - INFO: - episode: 2487, ep_reward: 81.218, ep_step: 1000 2023-05-30 01:13:39 - SimpleLog - INFO: - episode: 2488, ep_reward: 78.059, ep_step: 1000 2023-05-30 01:13:41 - SimpleLog - INFO: - episode: 2489, ep_reward: 93.213, ep_step: 1000 2023-05-30 01:13:43 - SimpleLog - INFO: - episode: 2490, ep_reward: 81.804, ep_step: 1000 2023-05-30 01:13:45 - SimpleLog - INFO: - episode: 2491, ep_reward: 117.892, ep_step: 1000 2023-05-30 01:13:47 - SimpleLog - INFO: - episode: 2492, ep_reward: 17.812, ep_step: 841 2023-05-30 01:13:51 - SimpleLog - INFO: - update_step: 1680000, online_eval_reward: 62.434 2023-05-30 01:13:51 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 62.434, save the best model! 2023-05-30 01:13:51 - SimpleLog - INFO: - episode: 2493, ep_reward: 84.092, ep_step: 1000 2023-05-30 01:13:54 - SimpleLog - INFO: - episode: 2494, ep_reward: 92.357, ep_step: 1000 2023-05-30 01:13:56 - SimpleLog - INFO: - episode: 2495, ep_reward: 21.249, ep_step: 1000 2023-05-30 01:13:58 - SimpleLog - INFO: - episode: 2496, ep_reward: 84.398, ep_step: 1000 2023-05-30 01:13:59 - SimpleLog - INFO: - episode: 2497, ep_reward: -65.919, ep_step: 416 2023-05-30 01:13:59 - SimpleLog - INFO: - episode: 2498, ep_reward: -96.352, ep_step: 169 2023-05-30 01:14:01 - SimpleLog - INFO: - episode: 2499, ep_reward: -28.719, ep_step: 896 2023-05-30 01:14:04 - SimpleLog - INFO: - episode: 2500, ep_reward: 83.559, ep_step: 1000 2023-05-30 01:14:06 - SimpleLog - INFO: - episode: 2501, ep_reward: 83.920, ep_step: 1000 2023-05-30 01:14:08 - SimpleLog - INFO: - episode: 2502, ep_reward: 122.920, ep_step: 1000 2023-05-30 01:14:10 - SimpleLog - INFO: - episode: 2503, ep_reward: 85.600, ep_step: 1000 2023-05-30 01:14:13 - SimpleLog - INFO: - episode: 2504, ep_reward: 136.182, ep_step: 1000 2023-05-30 01:14:15 - SimpleLog - INFO: - episode: 2505, ep_reward: 90.881, ep_step: 1000 2023-05-30 01:14:17 - SimpleLog - INFO: - episode: 2506, ep_reward: 95.437, ep_step: 1000 2023-05-30 01:14:19 - SimpleLog - INFO: - episode: 2507, ep_reward: 85.333, ep_step: 1000 2023-05-30 01:14:22 - SimpleLog - INFO: - episode: 2508, ep_reward: 86.340, ep_step: 1000 2023-05-30 01:14:24 - SimpleLog - INFO: - episode: 2509, ep_reward: 87.189, ep_step: 1000 2023-05-30 01:14:26 - SimpleLog - INFO: - episode: 2510, ep_reward: 143.669, ep_step: 1000 2023-05-30 01:14:28 - SimpleLog - INFO: - episode: 2511, ep_reward: 21.066, ep_step: 829 2023-05-30 01:14:30 - SimpleLog - INFO: - episode: 2512, ep_reward: 20.150, ep_step: 1000 2023-05-30 01:14:32 - SimpleLog - INFO: - episode: 2513, ep_reward: 11.181, ep_step: 812 2023-05-30 01:14:34 - SimpleLog - INFO: - episode: 2514, ep_reward: 4.142, ep_step: 902 2023-05-30 01:14:36 - SimpleLog - INFO: - episode: 2515, ep_reward: -56.430, ep_step: 1000 2023-05-30 01:14:39 - SimpleLog - INFO: - episode: 2516, ep_reward: 91.083, ep_step: 1000 2023-05-30 01:14:41 - SimpleLog - INFO: - episode: 2517, ep_reward: -55.872, ep_step: 1000 2023-05-30 01:14:43 - SimpleLog - INFO: - episode: 2518, ep_reward: 81.430, ep_step: 1000 2023-05-30 01:14:44 - SimpleLog - INFO: - episode: 2519, ep_reward: -97.813, ep_step: 225 2023-05-30 01:14:46 - SimpleLog - INFO: - episode: 2520, ep_reward: 9.325, ep_step: 961 2023-05-30 01:14:48 - SimpleLog - INFO: - episode: 2521, ep_reward: 16.593, ep_step: 1000 2023-05-30 01:14:50 - SimpleLog - INFO: - episode: 2522, ep_reward: 19.150, ep_step: 1000 2023-05-30 01:14:52 - SimpleLog - INFO: - episode: 2523, ep_reward: 4.359, ep_step: 803 2023-05-30 01:14:54 - SimpleLog - INFO: - episode: 2524, ep_reward: 114.363, ep_step: 1000 2023-05-30 01:14:56 - SimpleLog - INFO: - episode: 2525, ep_reward: 10.326, ep_step: 681 2023-05-30 01:14:58 - SimpleLog - INFO: - episode: 2526, ep_reward: -14.076, ep_step: 785 2023-05-30 01:15:00 - SimpleLog - INFO: - episode: 2527, ep_reward: 120.615, ep_step: 1000 2023-05-30 01:15:02 - SimpleLog - INFO: - episode: 2528, ep_reward: -30.587, ep_step: 918 2023-05-30 01:15:04 - SimpleLog - INFO: - episode: 2529, ep_reward: -8.433, ep_step: 870 2023-05-30 01:15:05 - SimpleLog - INFO: - episode: 2530, ep_reward: -39.711, ep_step: 522 2023-05-30 01:15:06 - SimpleLog - INFO: - episode: 2531, ep_reward: -34.760, ep_step: 423 2023-05-30 01:15:08 - SimpleLog - INFO: - episode: 2532, ep_reward: -22.704, ep_step: 863 2023-05-30 01:15:10 - SimpleLog - INFO: - episode: 2533, ep_reward: 92.475, ep_step: 1000 2023-05-30 01:15:12 - SimpleLog - INFO: - episode: 2534, ep_reward: 92.011, ep_step: 1000 2023-05-30 01:15:14 - SimpleLog - INFO: - episode: 2535, ep_reward: -12.241, ep_step: 613 2023-05-30 01:15:16 - SimpleLog - INFO: - episode: 2536, ep_reward: 150.802, ep_step: 1000 2023-05-30 01:15:18 - SimpleLog - INFO: - episode: 2537, ep_reward: -5.908, ep_step: 886 2023-05-30 01:15:20 - SimpleLog - INFO: - episode: 2538, ep_reward: 80.185, ep_step: 1000 2023-05-30 01:15:23 - SimpleLog - INFO: - episode: 2539, ep_reward: 93.258, ep_step: 1000 2023-05-30 01:15:24 - SimpleLog - INFO: - episode: 2540, ep_reward: 1.477, ep_step: 748 2023-05-30 01:15:27 - SimpleLog - INFO: - episode: 2541, ep_reward: 91.709, ep_step: 1000 2023-05-30 01:15:28 - SimpleLog - INFO: - episode: 2542, ep_reward: -46.185, ep_step: 577 2023-05-30 01:15:30 - SimpleLog - INFO: - episode: 2543, ep_reward: 25.131, ep_step: 1000 2023-05-30 01:15:32 - SimpleLog - INFO: - episode: 2544, ep_reward: 9.931, ep_step: 955 2023-05-30 01:15:34 - SimpleLog - INFO: - episode: 2545, ep_reward: 21.315, ep_step: 1000 2023-05-30 01:15:36 - SimpleLog - INFO: - episode: 2546, ep_reward: -58.654, ep_step: 614 2023-05-30 01:15:38 - SimpleLog - INFO: - episode: 2547, ep_reward: 25.016, ep_step: 1000 2023-05-30 01:15:40 - SimpleLog - INFO: - episode: 2548, ep_reward: 16.527, ep_step: 1000 2023-05-30 01:15:43 - SimpleLog - INFO: - episode: 2549, ep_reward: 14.971, ep_step: 1000 2023-05-30 01:15:45 - SimpleLog - INFO: - episode: 2550, ep_reward: 18.209, ep_step: 1000 2023-05-30 01:15:47 - SimpleLog - INFO: - episode: 2551, ep_reward: -22.927, ep_step: 1000 2023-05-30 01:15:49 - SimpleLog - INFO: - episode: 2552, ep_reward: 13.966, ep_step: 1000 2023-05-30 01:15:52 - SimpleLog - INFO: - episode: 2553, ep_reward: 21.513, ep_step: 1000 2023-05-30 01:15:54 - SimpleLog - INFO: - episode: 2554, ep_reward: 23.878, ep_step: 1000 2023-05-30 01:15:56 - SimpleLog - INFO: - episode: 2555, ep_reward: 86.989, ep_step: 1000 2023-05-30 01:15:58 - SimpleLog - INFO: - episode: 2556, ep_reward: 94.469, ep_step: 1000 2023-05-30 01:16:01 - SimpleLog - INFO: - episode: 2557, ep_reward: 92.837, ep_step: 1000 2023-05-30 01:16:03 - SimpleLog - INFO: - episode: 2558, ep_reward: 92.410, ep_step: 1000 2023-05-30 01:16:05 - SimpleLog - INFO: - episode: 2559, ep_reward: 12.409, ep_step: 1000 2023-05-30 01:16:06 - SimpleLog - INFO: - episode: 2560, ep_reward: -45.513, ep_step: 487 2023-05-30 01:16:08 - SimpleLog - INFO: - episode: 2561, ep_reward: 93.503, ep_step: 1000 2023-05-30 01:16:09 - SimpleLog - INFO: - episode: 2562, ep_reward: -59.756, ep_step: 395 2023-05-30 01:16:11 - SimpleLog - INFO: - episode: 2563, ep_reward: 14.709, ep_step: 1000 2023-05-30 01:16:12 - SimpleLog - INFO: - episode: 2564, ep_reward: -59.567, ep_step: 327 2023-05-30 01:16:14 - SimpleLog - INFO: - episode: 2565, ep_reward: 90.311, ep_step: 1000 2023-05-30 01:16:15 - SimpleLog - INFO: - episode: 2566, ep_reward: -76.481, ep_step: 353 2023-05-30 01:16:18 - SimpleLog - INFO: - episode: 2567, ep_reward: 85.550, ep_step: 1000 2023-05-30 01:16:19 - SimpleLog - INFO: - episode: 2568, ep_reward: -11.818, ep_step: 794 2023-05-30 01:16:20 - SimpleLog - INFO: - episode: 2569, ep_reward: -54.959, ep_step: 398 2023-05-30 01:16:22 - SimpleLog - INFO: - episode: 2570, ep_reward: 11.928, ep_step: 1000 2023-05-30 01:16:24 - SimpleLog - INFO: - episode: 2571, ep_reward: -46.479, ep_step: 475 2023-05-30 01:16:26 - SimpleLog - INFO: - episode: 2572, ep_reward: 86.754, ep_step: 1000 2023-05-30 01:16:26 - SimpleLog - INFO: - episode: 2573, ep_reward: -72.303, ep_step: 200 2023-05-30 01:16:30 - SimpleLog - INFO: - update_step: 1750000, online_eval_reward: 95.869 2023-05-30 01:16:30 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 95.869, save the best model! 2023-05-30 01:16:30 - SimpleLog - INFO: - episode: 2574, ep_reward: 87.708, ep_step: 1000 2023-05-30 01:16:32 - SimpleLog - INFO: - episode: 2575, ep_reward: 1.811, ep_step: 811 2023-05-30 01:16:34 - SimpleLog - INFO: - episode: 2576, ep_reward: 167.834, ep_step: 1000 2023-05-30 01:16:37 - SimpleLog - INFO: - episode: 2577, ep_reward: 93.042, ep_step: 1000 2023-05-30 01:16:38 - SimpleLog - INFO: - episode: 2578, ep_reward: -43.698, ep_step: 422 2023-05-30 01:16:39 - SimpleLog - INFO: - episode: 2579, ep_reward: -50.663, ep_step: 480 2023-05-30 01:16:40 - SimpleLog - INFO: - episode: 2580, ep_reward: 7.550, ep_step: 719 2023-05-30 01:16:42 - SimpleLog - INFO: - episode: 2581, ep_reward: 27.985, ep_step: 933 2023-05-30 01:16:45 - SimpleLog - INFO: - episode: 2582, ep_reward: 80.020, ep_step: 1000 2023-05-30 01:16:46 - SimpleLog - INFO: - episode: 2583, ep_reward: -14.551, ep_step: 629 2023-05-30 01:16:47 - SimpleLog - INFO: - episode: 2584, ep_reward: -41.053, ep_step: 405 2023-05-30 01:16:49 - SimpleLog - INFO: - episode: 2585, ep_reward: 30.780, ep_step: 910 2023-05-30 01:16:50 - SimpleLog - INFO: - episode: 2586, ep_reward: -43.206, ep_step: 416 2023-05-30 01:16:52 - SimpleLog - INFO: - episode: 2587, ep_reward: 10.588, ep_step: 687 2023-05-30 01:16:53 - SimpleLog - INFO: - episode: 2588, ep_reward: 11.924, ep_step: 650 2023-05-30 01:16:55 - SimpleLog - INFO: - episode: 2589, ep_reward: 94.591, ep_step: 1000 2023-05-30 01:16:57 - SimpleLog - INFO: - episode: 2590, ep_reward: 10.744, ep_step: 845 2023-05-30 01:16:59 - SimpleLog - INFO: - episode: 2591, ep_reward: 97.622, ep_step: 1000 2023-05-30 01:17:02 - SimpleLog - INFO: - episode: 2592, ep_reward: 85.655, ep_step: 1000 2023-05-30 01:17:04 - SimpleLog - INFO: - episode: 2593, ep_reward: 85.125, ep_step: 1000 2023-05-30 01:17:06 - SimpleLog - INFO: - episode: 2594, ep_reward: 24.483, ep_step: 1000 2023-05-30 01:17:08 - SimpleLog - INFO: - episode: 2595, ep_reward: 34.877, ep_step: 1000 2023-05-30 01:17:09 - SimpleLog - INFO: - episode: 2596, ep_reward: -90.350, ep_step: 152 2023-05-30 01:17:11 - SimpleLog - INFO: - episode: 2597, ep_reward: 33.425, ep_step: 1000 2023-05-30 01:17:13 - SimpleLog - INFO: - episode: 2598, ep_reward: 28.770, ep_step: 1000 2023-05-30 01:17:16 - SimpleLog - INFO: - episode: 2599, ep_reward: -23.039, ep_step: 1000 2023-05-30 01:17:18 - SimpleLog - INFO: - episode: 2600, ep_reward: 85.273, ep_step: 1000 2023-05-30 01:17:20 - SimpleLog - INFO: - episode: 2601, ep_reward: 166.991, ep_step: 1000 2023-05-30 01:17:22 - SimpleLog - INFO: - episode: 2602, ep_reward: 15.663, ep_step: 806 2023-05-30 01:17:23 - SimpleLog - INFO: - episode: 2603, ep_reward: -102.725, ep_step: 358 2023-05-30 01:17:25 - SimpleLog - INFO: - episode: 2604, ep_reward: 134.215, ep_step: 1000 2023-05-30 01:17:27 - SimpleLog - INFO: - episode: 2605, ep_reward: 3.487, ep_step: 818 2023-05-30 01:17:29 - SimpleLog - INFO: - episode: 2606, ep_reward: 83.234, ep_step: 1000 2023-05-30 01:17:31 - SimpleLog - INFO: - episode: 2607, ep_reward: 15.697, ep_step: 835 2023-05-30 01:17:32 - SimpleLog - INFO: - episode: 2608, ep_reward: -5.058, ep_step: 664 2023-05-30 01:17:35 - SimpleLog - INFO: - episode: 2609, ep_reward: 12.181, ep_step: 1000 2023-05-30 01:17:37 - SimpleLog - INFO: - episode: 2610, ep_reward: 1.425, ep_step: 938 2023-05-30 01:17:39 - SimpleLog - INFO: - episode: 2611, ep_reward: 94.976, ep_step: 1000 2023-05-30 01:17:41 - SimpleLog - INFO: - episode: 2612, ep_reward: 111.244, ep_step: 1000 2023-05-30 01:17:43 - SimpleLog - INFO: - episode: 2613, ep_reward: -10.536, ep_step: 621 2023-05-30 01:17:45 - SimpleLog - INFO: - episode: 2614, ep_reward: 83.130, ep_step: 1000 2023-05-30 01:17:47 - SimpleLog - INFO: - episode: 2615, ep_reward: -28.583, ep_step: 787 2023-05-30 01:17:49 - SimpleLog - INFO: - episode: 2616, ep_reward: -52.174, ep_step: 1000 2023-05-30 01:17:51 - SimpleLog - INFO: - episode: 2617, ep_reward: 83.181, ep_step: 1000 2023-05-30 01:17:53 - SimpleLog - INFO: - episode: 2618, ep_reward: 164.309, ep_step: 1000 2023-05-30 01:17:55 - SimpleLog - INFO: - episode: 2619, ep_reward: -38.525, ep_step: 579 2023-05-30 01:17:56 - SimpleLog - INFO: - episode: 2620, ep_reward: -51.315, ep_step: 476 2023-05-30 01:17:56 - SimpleLog - INFO: - episode: 2621, ep_reward: -106.140, ep_step: 272 2023-05-30 01:17:59 - SimpleLog - INFO: - episode: 2622, ep_reward: 96.130, ep_step: 1000 2023-05-30 01:18:01 - SimpleLog - INFO: - episode: 2623, ep_reward: 11.362, ep_step: 1000 2023-05-30 01:18:03 - SimpleLog - INFO: - episode: 2624, ep_reward: 97.494, ep_step: 1000 2023-05-30 01:18:06 - SimpleLog - INFO: - episode: 2625, ep_reward: 12.254, ep_step: 1000 2023-05-30 01:18:08 - SimpleLog - INFO: - episode: 2626, ep_reward: 84.546, ep_step: 1000 2023-05-30 01:18:09 - SimpleLog - INFO: - episode: 2627, ep_reward: -31.401, ep_step: 438 2023-05-30 01:18:10 - SimpleLog - INFO: - episode: 2628, ep_reward: -35.194, ep_step: 494 2023-05-30 01:18:12 - SimpleLog - INFO: - episode: 2629, ep_reward: 79.904, ep_step: 1000 2023-05-30 01:18:14 - SimpleLog - INFO: - episode: 2630, ep_reward: 186.249, ep_step: 1000 2023-05-30 01:18:17 - SimpleLog - INFO: - episode: 2631, ep_reward: 85.612, ep_step: 1000 2023-05-30 01:18:19 - SimpleLog - INFO: - episode: 2632, ep_reward: 163.950, ep_step: 1000 2023-05-30 01:18:21 - SimpleLog - INFO: - episode: 2633, ep_reward: 84.607, ep_step: 1000 2023-05-30 01:18:23 - SimpleLog - INFO: - episode: 2634, ep_reward: 93.666, ep_step: 1000 2023-05-30 01:18:26 - SimpleLog - INFO: - episode: 2635, ep_reward: 92.171, ep_step: 1000 2023-05-30 01:18:28 - SimpleLog - INFO: - episode: 2636, ep_reward: 190.291, ep_step: 1000 2023-05-30 01:18:30 - SimpleLog - INFO: - episode: 2637, ep_reward: 86.759, ep_step: 1000 2023-05-30 01:18:32 - SimpleLog - INFO: - episode: 2638, ep_reward: 20.891, ep_step: 1000 2023-05-30 01:18:35 - SimpleLog - INFO: - episode: 2639, ep_reward: 84.041, ep_step: 1000 2023-05-30 01:18:37 - SimpleLog - INFO: - episode: 2640, ep_reward: 146.627, ep_step: 1000 2023-05-30 01:18:37 - SimpleLog - INFO: - episode: 2641, ep_reward: -98.781, ep_step: 245 2023-05-30 01:18:38 - SimpleLog - INFO: - episode: 2642, ep_reward: -26.019, ep_step: 466 2023-05-30 01:18:40 - SimpleLog - INFO: - episode: 2643, ep_reward: -62.993, ep_step: 643 2023-05-30 01:18:42 - SimpleLog - INFO: - episode: 2644, ep_reward: 156.707, ep_step: 1000 2023-05-30 01:18:44 - SimpleLog - INFO: - episode: 2645, ep_reward: 160.329, ep_step: 1000 2023-05-30 01:18:47 - SimpleLog - INFO: - episode: 2646, ep_reward: 15.979, ep_step: 1000 2023-05-30 01:18:49 - SimpleLog - INFO: - episode: 2647, ep_reward: 38.731, ep_step: 1000 2023-05-30 01:18:51 - SimpleLog - INFO: - episode: 2648, ep_reward: 85.809, ep_step: 1000 2023-05-30 01:18:53 - SimpleLog - INFO: - episode: 2649, ep_reward: 177.258, ep_step: 1000 2023-05-30 01:18:55 - SimpleLog - INFO: - episode: 2650, ep_reward: -37.813, ep_step: 573 2023-05-30 01:18:57 - SimpleLog - INFO: - episode: 2651, ep_reward: -47.336, ep_step: 1000 2023-05-30 01:18:59 - SimpleLog - INFO: - episode: 2652, ep_reward: 12.581, ep_step: 887 2023-05-30 01:19:01 - SimpleLog - INFO: - episode: 2653, ep_reward: 78.508, ep_step: 1000 2023-05-30 01:19:03 - SimpleLog - INFO: - episode: 2654, ep_reward: 86.663, ep_step: 1000 2023-05-30 01:19:06 - SimpleLog - INFO: - episode: 2655, ep_reward: 84.813, ep_step: 1000 2023-05-30 01:19:09 - SimpleLog - INFO: - update_step: 1820000, online_eval_reward: 83.745 2023-05-30 01:19:10 - SimpleLog - INFO: - episode: 2656, ep_reward: 84.930, ep_step: 1000 2023-05-30 01:19:12 - SimpleLog - INFO: - episode: 2657, ep_reward: 90.043, ep_step: 1000 2023-05-30 01:19:14 - SimpleLog - INFO: - episode: 2658, ep_reward: 88.556, ep_step: 1000 2023-05-30 01:19:16 - SimpleLog - INFO: - episode: 2659, ep_reward: 28.424, ep_step: 962 2023-05-30 01:19:19 - SimpleLog - INFO: - episode: 2660, ep_reward: 15.626, ep_step: 1000 2023-05-30 01:19:19 - SimpleLog - INFO: - episode: 2661, ep_reward: -114.904, ep_step: 59 2023-05-30 01:19:21 - SimpleLog - INFO: - episode: 2662, ep_reward: 179.046, ep_step: 1000 2023-05-30 01:19:23 - SimpleLog - INFO: - episode: 2663, ep_reward: 85.226, ep_step: 1000 2023-05-30 01:19:26 - SimpleLog - INFO: - episode: 2664, ep_reward: 121.876, ep_step: 1000 2023-05-30 01:19:28 - SimpleLog - INFO: - episode: 2665, ep_reward: 87.539, ep_step: 1000 2023-05-30 01:19:30 - SimpleLog - INFO: - episode: 2666, ep_reward: 89.510, ep_step: 1000 2023-05-30 01:19:32 - SimpleLog - INFO: - episode: 2667, ep_reward: 25.354, ep_step: 763 2023-05-30 01:19:34 - SimpleLog - INFO: - episode: 2668, ep_reward: 20.579, ep_step: 1000 2023-05-30 01:19:35 - SimpleLog - INFO: - episode: 2669, ep_reward: -17.664, ep_step: 591 2023-05-30 01:19:38 - SimpleLog - INFO: - episode: 2670, ep_reward: 85.150, ep_step: 1000 2023-05-30 01:19:39 - SimpleLog - INFO: - episode: 2671, ep_reward: -29.263, ep_step: 469 2023-05-30 01:19:41 - SimpleLog - INFO: - episode: 2672, ep_reward: 121.642, ep_step: 1000 2023-05-30 01:19:43 - SimpleLog - INFO: - episode: 2673, ep_reward: 178.641, ep_step: 1000 2023-05-30 01:19:45 - SimpleLog - INFO: - episode: 2674, ep_reward: 22.965, ep_step: 1000 2023-05-30 01:19:47 - SimpleLog - INFO: - episode: 2675, ep_reward: 14.665, ep_step: 850 2023-05-30 01:19:50 - SimpleLog - INFO: - episode: 2676, ep_reward: 175.880, ep_step: 1000 2023-05-30 01:19:52 - SimpleLog - INFO: - episode: 2677, ep_reward: 90.758, ep_step: 1000 2023-05-30 01:19:54 - SimpleLog - INFO: - episode: 2678, ep_reward: 86.681, ep_step: 1000 2023-05-30 01:19:56 - SimpleLog - INFO: - episode: 2679, ep_reward: 90.116, ep_step: 1000 2023-05-30 01:19:59 - SimpleLog - INFO: - episode: 2680, ep_reward: 16.735, ep_step: 1000 2023-05-30 01:20:01 - SimpleLog - INFO: - episode: 2681, ep_reward: 92.308, ep_step: 1000 2023-05-30 01:20:03 - SimpleLog - INFO: - episode: 2682, ep_reward: 144.220, ep_step: 1000 2023-05-30 01:20:05 - SimpleLog - INFO: - episode: 2683, ep_reward: 91.920, ep_step: 1000 2023-05-30 01:20:08 - SimpleLog - INFO: - episode: 2684, ep_reward: 124.269, ep_step: 1000 2023-05-30 01:20:08 - SimpleLog - INFO: - episode: 2685, ep_reward: -118.920, ep_step: 101 2023-05-30 01:20:08 - SimpleLog - INFO: - episode: 2686, ep_reward: -125.641, ep_step: 180 2023-05-30 01:20:09 - SimpleLog - INFO: - episode: 2687, ep_reward: -54.363, ep_step: 450 2023-05-30 01:20:11 - SimpleLog - INFO: - episode: 2688, ep_reward: 139.940, ep_step: 1000 2023-05-30 01:20:14 - SimpleLog - INFO: - episode: 2689, ep_reward: 165.464, ep_step: 1000 2023-05-30 01:20:16 - SimpleLog - INFO: - episode: 2690, ep_reward: 24.411, ep_step: 1000 2023-05-30 01:20:18 - SimpleLog - INFO: - episode: 2691, ep_reward: 17.717, ep_step: 1000 2023-05-30 01:20:20 - SimpleLog - INFO: - episode: 2692, ep_reward: 131.043, ep_step: 1000 2023-05-30 01:20:23 - SimpleLog - INFO: - episode: 2693, ep_reward: 92.231, ep_step: 1000 2023-05-30 01:20:25 - SimpleLog - INFO: - episode: 2694, ep_reward: 99.999, ep_step: 1000 2023-05-30 01:20:27 - SimpleLog - INFO: - episode: 2695, ep_reward: 113.813, ep_step: 1000 2023-05-30 01:20:29 - SimpleLog - INFO: - episode: 2696, ep_reward: 89.016, ep_step: 1000 2023-05-30 01:20:32 - SimpleLog - INFO: - episode: 2697, ep_reward: 116.456, ep_step: 1000 2023-05-30 01:20:32 - SimpleLog - INFO: - episode: 2698, ep_reward: -107.114, ep_step: 76 2023-05-30 01:20:32 - SimpleLog - INFO: - episode: 2699, ep_reward: -106.567, ep_step: 78 2023-05-30 01:20:34 - SimpleLog - INFO: - episode: 2700, ep_reward: 13.363, ep_step: 1000 2023-05-30 01:20:37 - SimpleLog - INFO: - episode: 2701, ep_reward: 145.577, ep_step: 1000 2023-05-30 01:20:39 - SimpleLog - INFO: - episode: 2702, ep_reward: 87.827, ep_step: 1000 2023-05-30 01:20:41 - SimpleLog - INFO: - episode: 2703, ep_reward: 85.417, ep_step: 1000 2023-05-30 01:20:43 - SimpleLog - INFO: - episode: 2704, ep_reward: 94.954, ep_step: 1000 2023-05-30 01:20:46 - SimpleLog - INFO: - episode: 2705, ep_reward: 87.776, ep_step: 1000 2023-05-30 01:20:48 - SimpleLog - INFO: - episode: 2706, ep_reward: 88.966, ep_step: 1000 2023-05-30 01:20:50 - SimpleLog - INFO: - episode: 2707, ep_reward: 90.092, ep_step: 1000 2023-05-30 01:20:51 - SimpleLog - INFO: - episode: 2708, ep_reward: -52.735, ep_step: 465 2023-05-30 01:20:52 - SimpleLog - INFO: - episode: 2709, ep_reward: -63.056, ep_step: 406 2023-05-30 01:20:54 - SimpleLog - INFO: - episode: 2710, ep_reward: 94.565, ep_step: 1000 2023-05-30 01:20:56 - SimpleLog - INFO: - episode: 2711, ep_reward: 6.886, ep_step: 654 2023-05-30 01:20:58 - SimpleLog - INFO: - episode: 2712, ep_reward: 89.559, ep_step: 1000 2023-05-30 01:21:00 - SimpleLog - INFO: - episode: 2713, ep_reward: 18.776, ep_step: 1000 2023-05-30 01:21:02 - SimpleLog - INFO: - episode: 2714, ep_reward: -10.226, ep_step: 640 2023-05-30 01:21:03 - SimpleLog - INFO: - episode: 2715, ep_reward: -57.248, ep_step: 608 2023-05-30 01:21:05 - SimpleLog - INFO: - episode: 2716, ep_reward: 17.659, ep_step: 1000 2023-05-30 01:21:08 - SimpleLog - INFO: - episode: 2717, ep_reward: 150.835, ep_step: 1000 2023-05-30 01:21:10 - SimpleLog - INFO: - episode: 2718, ep_reward: 86.335, ep_step: 1000 2023-05-30 01:21:12 - SimpleLog - INFO: - episode: 2719, ep_reward: 87.477, ep_step: 1000 2023-05-30 01:21:14 - SimpleLog - INFO: - episode: 2720, ep_reward: 33.831, ep_step: 915 2023-05-30 01:21:16 - SimpleLog - INFO: - episode: 2721, ep_reward: 88.346, ep_step: 1000 2023-05-30 01:21:19 - SimpleLog - INFO: - episode: 2722, ep_reward: 154.655, ep_step: 1000 2023-05-30 01:21:21 - SimpleLog - INFO: - episode: 2723, ep_reward: 109.621, ep_step: 1000 2023-05-30 01:21:23 - SimpleLog - INFO: - episode: 2724, ep_reward: 87.157, ep_step: 1000 2023-05-30 01:21:25 - SimpleLog - INFO: - episode: 2725, ep_reward: 167.698, ep_step: 1000 2023-05-30 01:21:27 - SimpleLog - INFO: - episode: 2726, ep_reward: 10.659, ep_step: 873 2023-05-30 01:21:30 - SimpleLog - INFO: - episode: 2727, ep_reward: 169.111, ep_step: 1000 2023-05-30 01:21:32 - SimpleLog - INFO: - episode: 2728, ep_reward: 137.818, ep_step: 1000 2023-05-30 01:21:34 - SimpleLog - INFO: - episode: 2729, ep_reward: 19.992, ep_step: 783 2023-05-30 01:21:35 - SimpleLog - INFO: - episode: 2730, ep_reward: -61.729, ep_step: 427 2023-05-30 01:21:36 - SimpleLog - INFO: - episode: 2731, ep_reward: -38.528, ep_step: 586 2023-05-30 01:21:38 - SimpleLog - INFO: - episode: 2732, ep_reward: 149.729, ep_step: 1000 2023-05-30 01:21:40 - SimpleLog - INFO: - episode: 2733, ep_reward: -38.350, ep_step: 600 2023-05-30 01:21:42 - SimpleLog - INFO: - episode: 2734, ep_reward: 89.485, ep_step: 1000 2023-05-30 01:21:44 - SimpleLog - INFO: - episode: 2735, ep_reward: 147.390, ep_step: 1000 2023-05-30 01:21:46 - SimpleLog - INFO: - episode: 2736, ep_reward: 89.486, ep_step: 1000 2023-05-30 01:21:49 - SimpleLog - INFO: - update_step: 1890000, online_eval_reward: 159.462 2023-05-30 01:21:49 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 159.462, save the best model! 2023-05-30 01:21:51 - SimpleLog - INFO: - episode: 2737, ep_reward: 121.545, ep_step: 1000 2023-05-30 01:21:53 - SimpleLog - INFO: - episode: 2738, ep_reward: 148.474, ep_step: 1000 2023-05-30 01:21:55 - SimpleLog - INFO: - episode: 2739, ep_reward: -14.785, ep_step: 811 2023-05-30 01:21:57 - SimpleLog - INFO: - episode: 2740, ep_reward: 144.280, ep_step: 1000 2023-05-30 01:21:59 - SimpleLog - INFO: - episode: 2741, ep_reward: 141.419, ep_step: 1000 2023-05-30 01:22:01 - SimpleLog - INFO: - episode: 2742, ep_reward: 92.072, ep_step: 1000 2023-05-30 01:22:04 - SimpleLog - INFO: - episode: 2743, ep_reward: 153.471, ep_step: 1000 2023-05-30 01:22:06 - SimpleLog - INFO: - episode: 2744, ep_reward: 89.316, ep_step: 1000 2023-05-30 01:22:08 - SimpleLog - INFO: - episode: 2745, ep_reward: 90.688, ep_step: 1000 2023-05-30 01:22:10 - SimpleLog - INFO: - episode: 2746, ep_reward: 88.969, ep_step: 1000 2023-05-30 01:22:13 - SimpleLog - INFO: - episode: 2747, ep_reward: 163.269, ep_step: 1000 2023-05-30 01:22:13 - SimpleLog - INFO: - episode: 2748, ep_reward: -90.842, ep_step: 160 2023-05-30 01:22:15 - SimpleLog - INFO: - episode: 2749, ep_reward: 140.169, ep_step: 1000 2023-05-30 01:22:17 - SimpleLog - INFO: - episode: 2750, ep_reward: 90.792, ep_step: 1000 2023-05-30 01:22:20 - SimpleLog - INFO: - episode: 2751, ep_reward: 140.237, ep_step: 1000 2023-05-30 01:22:21 - SimpleLog - INFO: - episode: 2752, ep_reward: -71.774, ep_step: 546 2023-05-30 01:22:23 - SimpleLog - INFO: - episode: 2753, ep_reward: 148.204, ep_step: 1000 2023-05-30 01:22:25 - SimpleLog - INFO: - episode: 2754, ep_reward: 91.150, ep_step: 1000 2023-05-30 01:22:27 - SimpleLog - INFO: - episode: 2755, ep_reward: 8.070, ep_step: 942 2023-05-30 01:22:29 - SimpleLog - INFO: - episode: 2756, ep_reward: -34.701, ep_step: 578 2023-05-30 01:22:29 - SimpleLog - INFO: - episode: 2757, ep_reward: -86.535, ep_step: 189 2023-05-30 01:22:30 - SimpleLog - INFO: - episode: 2758, ep_reward: -37.624, ep_step: 579 2023-05-30 01:22:32 - SimpleLog - INFO: - episode: 2759, ep_reward: -31.387, ep_step: 500 2023-05-30 01:22:34 - SimpleLog - INFO: - episode: 2760, ep_reward: 13.992, ep_step: 931 2023-05-30 01:22:35 - SimpleLog - INFO: - episode: 2761, ep_reward: -40.534, ep_step: 607 2023-05-30 01:22:37 - SimpleLog - INFO: - episode: 2762, ep_reward: 8.926, ep_step: 715 2023-05-30 01:22:37 - SimpleLog - INFO: - episode: 2763, ep_reward: -85.737, ep_step: 174 2023-05-30 01:22:38 - SimpleLog - INFO: - episode: 2764, ep_reward: -31.800, ep_step: 539 2023-05-30 01:22:41 - SimpleLog - INFO: - episode: 2765, ep_reward: 16.778, ep_step: 1000 2023-05-30 01:22:43 - SimpleLog - INFO: - episode: 2766, ep_reward: 139.299, ep_step: 1000 2023-05-30 01:22:45 - SimpleLog - INFO: - episode: 2767, ep_reward: 4.780, ep_step: 767 2023-05-30 01:22:47 - SimpleLog - INFO: - episode: 2768, ep_reward: 34.165, ep_step: 934 2023-05-30 01:22:49 - SimpleLog - INFO: - episode: 2769, ep_reward: 127.403, ep_step: 1000 2023-05-30 01:22:50 - SimpleLog - INFO: - episode: 2770, ep_reward: -40.180, ep_step: 549 2023-05-30 01:22:52 - SimpleLog - INFO: - episode: 2771, ep_reward: 122.159, ep_step: 1000 2023-05-30 01:22:55 - SimpleLog - INFO: - episode: 2772, ep_reward: 39.804, ep_step: 1000 2023-05-30 01:22:56 - SimpleLog - INFO: - episode: 2773, ep_reward: -34.358, ep_step: 503 2023-05-30 01:22:58 - SimpleLog - INFO: - episode: 2774, ep_reward: 103.760, ep_step: 1000 2023-05-30 01:22:59 - SimpleLog - INFO: - episode: 2775, ep_reward: -77.584, ep_step: 592 2023-05-30 01:23:02 - SimpleLog - INFO: - episode: 2776, ep_reward: 102.121, ep_step: 1000 2023-05-30 01:23:03 - SimpleLog - INFO: - episode: 2777, ep_reward: -74.569, ep_step: 815 2023-05-30 01:23:04 - SimpleLog - INFO: - episode: 2778, ep_reward: -90.391, ep_step: 241 2023-05-30 01:23:06 - SimpleLog - INFO: - episode: 2779, ep_reward: 108.472, ep_step: 1000 2023-05-30 01:23:08 - SimpleLog - INFO: - episode: 2780, ep_reward: 88.981, ep_step: 1000 2023-05-30 01:23:10 - SimpleLog - INFO: - episode: 2781, ep_reward: -77.708, ep_step: 635 2023-05-30 01:23:10 - SimpleLog - INFO: - episode: 2782, ep_reward: -101.581, ep_step: 230 2023-05-30 01:23:11 - SimpleLog - INFO: - episode: 2783, ep_reward: -63.759, ep_step: 420 2023-05-30 01:23:14 - SimpleLog - INFO: - episode: 2784, ep_reward: 67.210, ep_step: 1000 2023-05-30 01:23:16 - SimpleLog - INFO: - episode: 2785, ep_reward: 105.325, ep_step: 1000 2023-05-30 01:23:18 - SimpleLog - INFO: - episode: 2786, ep_reward: 9.280, ep_step: 951 2023-05-30 01:23:20 - SimpleLog - INFO: - episode: 2787, ep_reward: 87.386, ep_step: 1000 2023-05-30 01:23:21 - SimpleLog - INFO: - episode: 2788, ep_reward: -130.972, ep_step: 293 2023-05-30 01:23:21 - SimpleLog - INFO: - episode: 2789, ep_reward: -93.301, ep_step: 205 2023-05-30 01:23:23 - SimpleLog - INFO: - episode: 2790, ep_reward: 20.106, ep_step: 1000 2023-05-30 01:23:26 - SimpleLog - INFO: - episode: 2791, ep_reward: 2.211, ep_step: 920 2023-05-30 01:23:28 - SimpleLog - INFO: - episode: 2792, ep_reward: 87.913, ep_step: 1000 2023-05-30 01:23:30 - SimpleLog - INFO: - episode: 2793, ep_reward: 110.480, ep_step: 1000 2023-05-30 01:23:31 - SimpleLog - INFO: - episode: 2794, ep_reward: -57.599, ep_step: 373 2023-05-30 01:23:33 - SimpleLog - INFO: - episode: 2795, ep_reward: 90.063, ep_step: 1000 2023-05-30 01:23:34 - SimpleLog - INFO: - episode: 2796, ep_reward: -92.269, ep_step: 186 2023-05-30 01:23:36 - SimpleLog - INFO: - episode: 2797, ep_reward: 121.161, ep_step: 1000 2023-05-30 01:23:38 - SimpleLog - INFO: - episode: 2798, ep_reward: 130.724, ep_step: 1000 2023-05-30 01:23:39 - SimpleLog - INFO: - episode: 2799, ep_reward: -59.445, ep_step: 645 2023-05-30 01:23:42 - SimpleLog - INFO: - episode: 2800, ep_reward: 152.498, ep_step: 1000 2023-05-30 01:23:44 - SimpleLog - INFO: - episode: 2801, ep_reward: 13.012, ep_step: 901 2023-05-30 01:23:44 - SimpleLog - INFO: - episode: 2802, ep_reward: -92.888, ep_step: 192 2023-05-30 01:23:45 - SimpleLog - INFO: - episode: 2803, ep_reward: -40.178, ep_step: 543 2023-05-30 01:23:48 - SimpleLog - INFO: - episode: 2804, ep_reward: 23.155, ep_step: 1000 2023-05-30 01:23:49 - SimpleLog - INFO: - episode: 2805, ep_reward: -61.721, ep_step: 609 2023-05-30 01:23:51 - SimpleLog - INFO: - episode: 2806, ep_reward: 130.297, ep_step: 1000 2023-05-30 01:23:53 - SimpleLog - INFO: - episode: 2807, ep_reward: -39.483, ep_step: 948 2023-05-30 01:23:56 - SimpleLog - INFO: - episode: 2808, ep_reward: 12.547, ep_step: 1000 2023-05-30 01:23:57 - SimpleLog - INFO: - episode: 2809, ep_reward: -52.248, ep_step: 577 2023-05-30 01:23:58 - SimpleLog - INFO: - episode: 2810, ep_reward: -77.504, ep_step: 656 2023-05-30 01:23:59 - SimpleLog - INFO: - episode: 2811, ep_reward: -59.473, ep_step: 431 2023-05-30 01:24:00 - SimpleLog - INFO: - episode: 2812, ep_reward: -59.927, ep_step: 432 2023-05-30 01:24:03 - SimpleLog - INFO: - episode: 2813, ep_reward: 86.045, ep_step: 1000 2023-05-30 01:24:04 - SimpleLog - INFO: - episode: 2814, ep_reward: -35.616, ep_step: 498 2023-05-30 01:24:06 - SimpleLog - INFO: - episode: 2815, ep_reward: 86.468, ep_step: 1000 2023-05-30 01:24:08 - SimpleLog - INFO: - episode: 2816, ep_reward: 16.332, ep_step: 1000 2023-05-30 01:24:10 - SimpleLog - INFO: - episode: 2817, ep_reward: 86.486, ep_step: 1000 2023-05-30 01:24:13 - SimpleLog - INFO: - episode: 2818, ep_reward: 19.361, ep_step: 942 2023-05-30 01:24:14 - SimpleLog - INFO: - episode: 2819, ep_reward: -42.298, ep_step: 611 2023-05-30 01:24:16 - SimpleLog - INFO: - episode: 2820, ep_reward: 99.094, ep_step: 1000 2023-05-30 01:24:18 - SimpleLog - INFO: - episode: 2821, ep_reward: 80.376, ep_step: 1000 2023-05-30 01:24:21 - SimpleLog - INFO: - episode: 2822, ep_reward: 92.173, ep_step: 1000 2023-05-30 01:24:23 - SimpleLog - INFO: - episode: 2823, ep_reward: 88.803, ep_step: 1000 2023-05-30 01:24:25 - SimpleLog - INFO: - episode: 2824, ep_reward: 85.529, ep_step: 1000 2023-05-30 01:24:28 - SimpleLog - INFO: - update_step: 1960000, online_eval_reward: 26.404 2023-05-30 01:24:29 - SimpleLog - INFO: - episode: 2825, ep_reward: 16.866, ep_step: 1000 2023-05-30 01:24:32 - SimpleLog - INFO: - episode: 2826, ep_reward: 18.974, ep_step: 1000 2023-05-30 01:24:34 - SimpleLog - INFO: - episode: 2827, ep_reward: 17.469, ep_step: 1000 2023-05-30 01:24:36 - SimpleLog - INFO: - episode: 2828, ep_reward: 82.783, ep_step: 1000 2023-05-30 01:24:38 - SimpleLog - INFO: - episode: 2829, ep_reward: 102.306, ep_step: 1000 2023-05-30 01:24:41 - SimpleLog - INFO: - episode: 2830, ep_reward: 95.297, ep_step: 1000 2023-05-30 01:24:41 - SimpleLog - INFO: - episode: 2831, ep_reward: -94.107, ep_step: 239 2023-05-30 01:24:43 - SimpleLog - INFO: - episode: 2832, ep_reward: 19.735, ep_step: 1000 2023-05-30 01:24:46 - SimpleLog - INFO: - episode: 2833, ep_reward: 78.639, ep_step: 1000 2023-05-30 01:24:48 - SimpleLog - INFO: - episode: 2834, ep_reward: 10.793, ep_step: 1000 2023-05-30 01:24:50 - SimpleLog - INFO: - episode: 2835, ep_reward: 21.773, ep_step: 1000 2023-05-30 01:24:52 - SimpleLog - INFO: - episode: 2836, ep_reward: 87.763, ep_step: 1000 2023-05-30 01:24:54 - SimpleLog - INFO: - episode: 2837, ep_reward: 21.380, ep_step: 1000 2023-05-30 01:24:57 - SimpleLog - INFO: - episode: 2838, ep_reward: 98.805, ep_step: 1000 2023-05-30 01:24:59 - SimpleLog - INFO: - episode: 2839, ep_reward: 99.191, ep_step: 1000 2023-05-30 01:25:01 - SimpleLog - INFO: - episode: 2840, ep_reward: 111.619, ep_step: 1000 2023-05-30 01:25:03 - SimpleLog - INFO: - episode: 2841, ep_reward: 86.451, ep_step: 1000 2023-05-30 01:25:06 - SimpleLog - INFO: - episode: 2842, ep_reward: 89.466, ep_step: 1000 2023-05-30 01:25:08 - SimpleLog - INFO: - episode: 2843, ep_reward: 86.634, ep_step: 1000 2023-05-30 01:25:10 - SimpleLog - INFO: - episode: 2844, ep_reward: 116.733, ep_step: 1000 2023-05-30 01:25:12 - SimpleLog - INFO: - episode: 2845, ep_reward: -16.582, ep_step: 1000 2023-05-30 01:25:14 - SimpleLog - INFO: - episode: 2846, ep_reward: -8.299, ep_step: 747 2023-05-30 01:25:16 - SimpleLog - INFO: - episode: 2847, ep_reward: 86.103, ep_step: 1000 2023-05-30 01:25:18 - SimpleLog - INFO: - episode: 2848, ep_reward: -6.926, ep_step: 653 2023-05-30 01:25:19 - SimpleLog - INFO: - episode: 2849, ep_reward: 6.531, ep_step: 738 2023-05-30 01:25:21 - SimpleLog - INFO: - episode: 2850, ep_reward: 2.867, ep_step: 730 2023-05-30 01:25:23 - SimpleLog - INFO: - episode: 2851, ep_reward: 89.921, ep_step: 1000 2023-05-30 01:25:26 - SimpleLog - INFO: - episode: 2852, ep_reward: 21.289, ep_step: 1000 2023-05-30 01:25:27 - SimpleLog - INFO: - episode: 2853, ep_reward: -113.770, ep_step: 535 2023-05-30 01:25:29 - SimpleLog - INFO: - episode: 2854, ep_reward: 18.629, ep_step: 1000 2023-05-30 01:25:31 - SimpleLog - INFO: - episode: 2855, ep_reward: 111.970, ep_step: 1000 2023-05-30 01:25:34 - SimpleLog - INFO: - episode: 2856, ep_reward: 2.549, ep_step: 956 2023-05-30 01:25:36 - SimpleLog - INFO: - episode: 2857, ep_reward: 83.220, ep_step: 1000 2023-05-30 01:25:38 - SimpleLog - INFO: - episode: 2858, ep_reward: 15.004, ep_step: 979 2023-05-30 01:25:40 - SimpleLog - INFO: - episode: 2859, ep_reward: 118.839, ep_step: 1000 2023-05-30 01:25:43 - SimpleLog - INFO: - episode: 2860, ep_reward: 86.877, ep_step: 1000 2023-05-30 01:25:45 - SimpleLog - INFO: - episode: 2861, ep_reward: 118.241, ep_step: 1000 2023-05-30 01:25:47 - SimpleLog - INFO: - episode: 2862, ep_reward: 115.889, ep_step: 1000 2023-05-30 01:25:49 - SimpleLog - INFO: - episode: 2863, ep_reward: 85.045, ep_step: 1000 2023-05-30 01:25:52 - SimpleLog - INFO: - episode: 2864, ep_reward: 13.875, ep_step: 995 2023-05-30 01:25:53 - SimpleLog - INFO: - episode: 2865, ep_reward: -63.112, ep_step: 519 2023-05-30 01:25:55 - SimpleLog - INFO: - episode: 2866, ep_reward: 119.391, ep_step: 1000 2023-05-30 01:25:57 - SimpleLog - INFO: - episode: 2867, ep_reward: 91.192, ep_step: 1000 2023-05-30 01:26:00 - SimpleLog - INFO: - episode: 2868, ep_reward: 115.031, ep_step: 1000 2023-05-30 01:26:02 - SimpleLog - INFO: - episode: 2869, ep_reward: 20.811, ep_step: 1000 2023-05-30 01:26:04 - SimpleLog - INFO: - episode: 2870, ep_reward: 130.563, ep_step: 1000 2023-05-30 01:26:06 - SimpleLog - INFO: - episode: 2871, ep_reward: 85.963, ep_step: 1000 2023-05-30 01:26:08 - SimpleLog - INFO: - episode: 2872, ep_reward: -46.132, ep_step: 526 2023-05-30 01:26:10 - SimpleLog - INFO: - episode: 2873, ep_reward: 97.434, ep_step: 1000 2023-05-30 01:26:12 - SimpleLog - INFO: - episode: 2874, ep_reward: 80.311, ep_step: 1000 2023-05-30 01:26:14 - SimpleLog - INFO: - episode: 2875, ep_reward: -13.773, ep_step: 897 2023-05-30 01:26:16 - SimpleLog - INFO: - episode: 2876, ep_reward: 88.750, ep_step: 1000 2023-05-30 01:26:19 - SimpleLog - INFO: - episode: 2877, ep_reward: 157.910, ep_step: 1000 2023-05-30 01:26:21 - SimpleLog - INFO: - episode: 2878, ep_reward: 97.140, ep_step: 1000 2023-05-30 01:26:23 - SimpleLog - INFO: - episode: 2879, ep_reward: 3.177, ep_step: 781 2023-05-30 01:26:25 - SimpleLog - INFO: - episode: 2880, ep_reward: 0.872, ep_step: 984 2023-05-30 01:26:27 - SimpleLog - INFO: - episode: 2881, ep_reward: 103.098, ep_step: 1000 2023-05-30 01:26:29 - SimpleLog - INFO: - episode: 2882, ep_reward: 87.704, ep_step: 1000 2023-05-30 01:26:31 - SimpleLog - INFO: - episode: 2883, ep_reward: 121.811, ep_step: 1000 2023-05-30 01:26:34 - SimpleLog - INFO: - episode: 2884, ep_reward: 110.785, ep_step: 1000 2023-05-30 01:26:36 - SimpleLog - INFO: - episode: 2885, ep_reward: 153.026, ep_step: 1000 2023-05-30 01:26:38 - SimpleLog - INFO: - episode: 2886, ep_reward: 104.289, ep_step: 1000 2023-05-30 01:26:40 - SimpleLog - INFO: - episode: 2887, ep_reward: 96.914, ep_step: 1000 2023-05-30 01:26:43 - SimpleLog - INFO: - episode: 2888, ep_reward: 95.232, ep_step: 1000 2023-05-30 01:26:45 - SimpleLog - INFO: - episode: 2889, ep_reward: 97.121, ep_step: 1000 2023-05-30 01:26:47 - SimpleLog - INFO: - episode: 2890, ep_reward: 95.733, ep_step: 1000 2023-05-30 01:26:49 - SimpleLog - INFO: - episode: 2891, ep_reward: 21.227, ep_step: 1000 2023-05-30 01:26:52 - SimpleLog - INFO: - episode: 2892, ep_reward: 96.726, ep_step: 1000 2023-05-30 01:26:54 - SimpleLog - INFO: - episode: 2893, ep_reward: 126.116, ep_step: 1000 2023-05-30 01:26:56 - SimpleLog - INFO: - episode: 2894, ep_reward: 82.233, ep_step: 1000 2023-05-30 01:26:58 - SimpleLog - INFO: - episode: 2895, ep_reward: 96.336, ep_step: 1000 2023-05-30 01:27:01 - SimpleLog - INFO: - episode: 2896, ep_reward: 84.318, ep_step: 1000 2023-05-30 01:27:03 - SimpleLog - INFO: - episode: 2897, ep_reward: 111.698, ep_step: 1000 2023-05-30 01:27:05 - SimpleLog - INFO: - episode: 2898, ep_reward: 93.826, ep_step: 1000 2023-05-30 01:27:07 - SimpleLog - INFO: - update_step: 2030000, online_eval_reward: 89.788 2023-05-30 01:27:09 - SimpleLog - INFO: - episode: 2899, ep_reward: 97.498, ep_step: 1000 2023-05-30 01:27:12 - SimpleLog - INFO: - episode: 2900, ep_reward: 21.983, ep_step: 1000 2023-05-30 01:27:14 - SimpleLog - INFO: - episode: 2901, ep_reward: 123.170, ep_step: 1000 2023-05-30 01:27:16 - SimpleLog - INFO: - episode: 2902, ep_reward: 119.572, ep_step: 1000 2023-05-30 01:27:18 - SimpleLog - INFO: - episode: 2903, ep_reward: 19.984, ep_step: 838 2023-05-30 01:27:20 - SimpleLog - INFO: - episode: 2904, ep_reward: 17.570, ep_step: 1000 2023-05-30 01:27:22 - SimpleLog - INFO: - episode: 2905, ep_reward: 95.730, ep_step: 1000 2023-05-30 01:27:25 - SimpleLog - INFO: - episode: 2906, ep_reward: 22.618, ep_step: 1000 2023-05-30 01:27:26 - SimpleLog - INFO: - episode: 2907, ep_reward: -64.025, ep_step: 450 2023-05-30 01:27:28 - SimpleLog - INFO: - episode: 2908, ep_reward: 122.587, ep_step: 1000 2023-05-30 01:27:30 - SimpleLog - INFO: - episode: 2909, ep_reward: 139.470, ep_step: 1000 2023-05-30 01:27:32 - SimpleLog - INFO: - episode: 2910, ep_reward: 125.554, ep_step: 1000 2023-05-30 01:27:35 - SimpleLog - INFO: - episode: 2911, ep_reward: 143.403, ep_step: 1000 2023-05-30 01:27:37 - SimpleLog - INFO: - episode: 2912, ep_reward: 131.811, ep_step: 1000 2023-05-30 01:27:39 - SimpleLog - INFO: - episode: 2913, ep_reward: 20.419, ep_step: 1000 2023-05-30 01:27:41 - SimpleLog - INFO: - episode: 2914, ep_reward: 140.233, ep_step: 1000 2023-05-30 01:27:44 - SimpleLog - INFO: - episode: 2915, ep_reward: 109.344, ep_step: 1000 2023-05-30 01:27:46 - SimpleLog - INFO: - episode: 2916, ep_reward: 139.616, ep_step: 1000 2023-05-30 01:27:48 - SimpleLog - INFO: - episode: 2917, ep_reward: 20.430, ep_step: 1000 2023-05-30 01:27:50 - SimpleLog - INFO: - episode: 2918, ep_reward: 22.417, ep_step: 1000 2023-05-30 01:27:53 - SimpleLog - INFO: - episode: 2919, ep_reward: 114.022, ep_step: 1000 2023-05-30 01:27:55 - SimpleLog - INFO: - episode: 2920, ep_reward: 23.356, ep_step: 1000 2023-05-30 01:27:57 - SimpleLog - INFO: - episode: 2921, ep_reward: 111.909, ep_step: 1000 2023-05-30 01:27:59 - SimpleLog - INFO: - episode: 2922, ep_reward: -148.929, ep_step: 632 2023-05-30 01:28:01 - SimpleLog - INFO: - episode: 2923, ep_reward: 108.241, ep_step: 1000 2023-05-30 01:28:03 - SimpleLog - INFO: - episode: 2924, ep_reward: 131.135, ep_step: 1000 2023-05-30 01:28:03 - SimpleLog - INFO: - episode: 2925, ep_reward: -100.562, ep_step: 122 2023-05-30 01:28:06 - SimpleLog - INFO: - episode: 2926, ep_reward: 95.234, ep_step: 1000 2023-05-30 01:28:08 - SimpleLog - INFO: - episode: 2927, ep_reward: 89.892, ep_step: 1000 2023-05-30 01:28:10 - SimpleLog - INFO: - episode: 2928, ep_reward: 97.608, ep_step: 1000 2023-05-30 01:28:12 - SimpleLog - INFO: - episode: 2929, ep_reward: 92.834, ep_step: 1000 2023-05-30 01:28:14 - SimpleLog - INFO: - episode: 2930, ep_reward: 9.581, ep_step: 924 2023-05-30 01:28:17 - SimpleLog - INFO: - episode: 2931, ep_reward: 25.395, ep_step: 1000 2023-05-30 01:28:19 - SimpleLog - INFO: - episode: 2932, ep_reward: 98.491, ep_step: 1000 2023-05-30 01:28:21 - SimpleLog - INFO: - episode: 2933, ep_reward: 92.935, ep_step: 1000 2023-05-30 01:28:23 - SimpleLog - INFO: - episode: 2934, ep_reward: 94.209, ep_step: 1000 2023-05-30 01:28:25 - SimpleLog - INFO: - episode: 2935, ep_reward: 15.564, ep_step: 862 2023-05-30 01:28:28 - SimpleLog - INFO: - episode: 2936, ep_reward: 94.339, ep_step: 1000 2023-05-30 01:28:30 - SimpleLog - INFO: - episode: 2937, ep_reward: 91.169, ep_step: 1000 2023-05-30 01:28:32 - SimpleLog - INFO: - episode: 2938, ep_reward: 12.963, ep_step: 950 2023-05-30 01:28:32 - SimpleLog - INFO: - episode: 2939, ep_reward: -88.736, ep_step: 185 2023-05-30 01:28:35 - SimpleLog - INFO: - episode: 2940, ep_reward: 98.014, ep_step: 1000 2023-05-30 01:28:37 - SimpleLog - INFO: - episode: 2941, ep_reward: 11.751, ep_step: 892 2023-05-30 01:28:39 - SimpleLog - INFO: - episode: 2942, ep_reward: 122.801, ep_step: 1000 2023-05-30 01:28:41 - SimpleLog - INFO: - episode: 2943, ep_reward: 10.368, ep_step: 980 2023-05-30 01:28:43 - SimpleLog - INFO: - episode: 2944, ep_reward: 24.511, ep_step: 1000 2023-05-30 01:28:46 - SimpleLog - INFO: - episode: 2945, ep_reward: 94.462, ep_step: 1000 2023-05-30 01:28:48 - SimpleLog - INFO: - episode: 2946, ep_reward: 12.358, ep_step: 1000 2023-05-30 01:28:50 - SimpleLog - INFO: - episode: 2947, ep_reward: 95.763, ep_step: 1000 2023-05-30 01:28:52 - SimpleLog - INFO: - episode: 2948, ep_reward: 135.214, ep_step: 1000 2023-05-30 01:28:54 - SimpleLog - INFO: - episode: 2949, ep_reward: -3.231, ep_step: 749 2023-05-30 01:28:56 - SimpleLog - INFO: - episode: 2950, ep_reward: -19.746, ep_step: 956 2023-05-30 01:28:58 - SimpleLog - INFO: - episode: 2951, ep_reward: 90.678, ep_step: 1000 2023-05-30 01:29:00 - SimpleLog - INFO: - episode: 2952, ep_reward: 12.126, ep_step: 913 2023-05-30 01:29:03 - SimpleLog - INFO: - episode: 2953, ep_reward: 97.876, ep_step: 1000 2023-05-30 01:29:05 - SimpleLog - INFO: - episode: 2954, ep_reward: 101.680, ep_step: 1000 2023-05-30 01:29:07 - SimpleLog - INFO: - episode: 2955, ep_reward: 14.177, ep_step: 878 2023-05-30 01:29:09 - SimpleLog - INFO: - episode: 2956, ep_reward: 9.135, ep_step: 938 2023-05-30 01:29:11 - SimpleLog - INFO: - episode: 2957, ep_reward: 98.135, ep_step: 1000 2023-05-30 01:29:12 - SimpleLog - INFO: - episode: 2958, ep_reward: -56.988, ep_step: 421 2023-05-30 01:29:14 - SimpleLog - INFO: - episode: 2959, ep_reward: 50.529, ep_step: 1000 2023-05-30 01:29:17 - SimpleLog - INFO: - episode: 2960, ep_reward: 19.313, ep_step: 1000 2023-05-30 01:29:17 - SimpleLog - INFO: - episode: 2961, ep_reward: -88.108, ep_step: 174 2023-05-30 01:29:19 - SimpleLog - INFO: - episode: 2962, ep_reward: 23.379, ep_step: 1000 2023-05-30 01:29:22 - SimpleLog - INFO: - episode: 2963, ep_reward: 90.937, ep_step: 1000 2023-05-30 01:29:24 - SimpleLog - INFO: - episode: 2964, ep_reward: 98.094, ep_step: 1000 2023-05-30 01:29:26 - SimpleLog - INFO: - episode: 2965, ep_reward: 15.019, ep_step: 908 2023-05-30 01:29:28 - SimpleLog - INFO: - episode: 2966, ep_reward: 109.514, ep_step: 1000 2023-05-30 01:29:30 - SimpleLog - INFO: - episode: 2967, ep_reward: 93.500, ep_step: 1000 2023-05-30 01:29:33 - SimpleLog - INFO: - episode: 2968, ep_reward: 132.263, ep_step: 1000 2023-05-30 01:29:35 - SimpleLog - INFO: - episode: 2969, ep_reward: 10.447, ep_step: 913 2023-05-30 01:29:35 - SimpleLog - INFO: - episode: 2970, ep_reward: -90.327, ep_step: 187 2023-05-30 01:29:37 - SimpleLog - INFO: - episode: 2971, ep_reward: 87.085, ep_step: 1000 2023-05-30 01:29:40 - SimpleLog - INFO: - episode: 2972, ep_reward: 122.133, ep_step: 1000 2023-05-30 01:29:42 - SimpleLog - INFO: - episode: 2973, ep_reward: 120.538, ep_step: 1000 2023-05-30 01:29:44 - SimpleLog - INFO: - episode: 2974, ep_reward: 105.197, ep_step: 1000 2023-05-30 01:29:47 - SimpleLog - INFO: - update_step: 2100000, online_eval_reward: 87.009 2023-05-30 01:29:48 - SimpleLog - INFO: - episode: 2975, ep_reward: 97.494, ep_step: 1000 2023-05-30 01:29:51 - SimpleLog - INFO: - episode: 2976, ep_reward: 95.130, ep_step: 1000 2023-05-30 01:29:53 - SimpleLog - INFO: - episode: 2977, ep_reward: 87.276, ep_step: 1000 2023-05-30 01:29:55 - SimpleLog - INFO: - episode: 2978, ep_reward: 7.114, ep_step: 982 2023-05-30 01:29:57 - SimpleLog - INFO: - episode: 2979, ep_reward: 124.960, ep_step: 1000 2023-05-30 01:29:59 - SimpleLog - INFO: - episode: 2980, ep_reward: 104.619, ep_step: 1000 2023-05-30 01:30:02 - SimpleLog - INFO: - episode: 2981, ep_reward: 94.378, ep_step: 1000 2023-05-30 01:30:04 - SimpleLog - INFO: - episode: 2982, ep_reward: 131.903, ep_step: 1000 2023-05-30 01:30:06 - SimpleLog - INFO: - episode: 2983, ep_reward: -17.077, ep_step: 739 2023-05-30 01:30:08 - SimpleLog - INFO: - episode: 2984, ep_reward: 96.424, ep_step: 1000 2023-05-30 01:30:09 - SimpleLog - INFO: - episode: 2985, ep_reward: -17.647, ep_step: 705 2023-05-30 01:30:11 - SimpleLog - INFO: - episode: 2986, ep_reward: -31.856, ep_step: 801 2023-05-30 01:30:13 - SimpleLog - INFO: - episode: 2987, ep_reward: -2.324, ep_step: 770 2023-05-30 01:30:15 - SimpleLog - INFO: - episode: 2988, ep_reward: 85.873, ep_step: 1000 2023-05-30 01:30:18 - SimpleLog - INFO: - episode: 2989, ep_reward: 118.223, ep_step: 1000 2023-05-30 01:30:20 - SimpleLog - INFO: - episode: 2990, ep_reward: 120.182, ep_step: 1000 2023-05-30 01:30:21 - SimpleLog - INFO: - episode: 2991, ep_reward: -53.739, ep_step: 489 2023-05-30 01:30:22 - SimpleLog - INFO: - episode: 2992, ep_reward: -13.298, ep_step: 712 2023-05-30 01:30:24 - SimpleLog - INFO: - episode: 2993, ep_reward: -3.881, ep_step: 773 2023-05-30 01:30:24 - SimpleLog - INFO: - episode: 2994, ep_reward: -119.071, ep_step: 81 2023-05-30 01:30:27 - SimpleLog - INFO: - episode: 2995, ep_reward: -100.344, ep_step: 978 2023-05-30 01:30:29 - SimpleLog - INFO: - episode: 2996, ep_reward: 96.136, ep_step: 1000 2023-05-30 01:30:30 - SimpleLog - INFO: - episode: 2997, ep_reward: -4.930, ep_step: 712 2023-05-30 01:30:33 - SimpleLog - INFO: - episode: 2998, ep_reward: 118.506, ep_step: 1000 2023-05-30 01:30:35 - SimpleLog - INFO: - episode: 2999, ep_reward: 118.668, ep_step: 1000 2023-05-30 01:30:37 - SimpleLog - INFO: - episode: 3000, ep_reward: 3.425, ep_step: 839 2023-05-30 01:30:39 - SimpleLog - INFO: - episode: 3001, ep_reward: 113.493, ep_step: 1000 2023-05-30 01:30:41 - SimpleLog - INFO: - episode: 3002, ep_reward: -8.152, ep_step: 723 2023-05-30 01:30:43 - SimpleLog - INFO: - episode: 3003, ep_reward: 39.385, ep_step: 997 2023-05-30 01:30:43 - SimpleLog - INFO: - episode: 3004, ep_reward: -119.569, ep_step: 133 2023-05-30 01:30:45 - SimpleLog - INFO: - episode: 3005, ep_reward: 115.030, ep_step: 1000 2023-05-30 01:30:48 - SimpleLog - INFO: - episode: 3006, ep_reward: 133.231, ep_step: 1000 2023-05-30 01:30:48 - SimpleLog - INFO: - episode: 3007, ep_reward: -90.978, ep_step: 199 2023-05-30 01:30:50 - SimpleLog - INFO: - episode: 3008, ep_reward: 124.592, ep_step: 1000 2023-05-30 01:30:53 - SimpleLog - INFO: - episode: 3009, ep_reward: 116.866, ep_step: 1000 2023-05-30 01:30:55 - SimpleLog - INFO: - episode: 3010, ep_reward: 116.883, ep_step: 1000 2023-05-30 01:30:57 - SimpleLog - INFO: - episode: 3011, ep_reward: 120.269, ep_step: 1000 2023-05-30 01:30:58 - SimpleLog - INFO: - episode: 3012, ep_reward: -36.058, ep_step: 528 2023-05-30 01:31:01 - SimpleLog - INFO: - episode: 3013, ep_reward: 114.280, ep_step: 1000 2023-05-30 01:31:03 - SimpleLog - INFO: - episode: 3014, ep_reward: 132.862, ep_step: 1000 2023-05-30 01:31:04 - SimpleLog - INFO: - episode: 3015, ep_reward: -54.301, ep_step: 555 2023-05-30 01:31:06 - SimpleLog - INFO: - episode: 3016, ep_reward: 127.539, ep_step: 1000 2023-05-30 01:31:08 - SimpleLog - INFO: - episode: 3017, ep_reward: 134.302, ep_step: 1000 2023-05-30 01:31:11 - SimpleLog - INFO: - episode: 3018, ep_reward: -82.613, ep_step: 1000 2023-05-30 01:31:11 - SimpleLog - INFO: - episode: 3019, ep_reward: -83.824, ep_step: 190 2023-05-30 01:31:13 - SimpleLog - INFO: - episode: 3020, ep_reward: 86.821, ep_step: 1000 2023-05-30 01:31:16 - SimpleLog - INFO: - episode: 3021, ep_reward: 135.303, ep_step: 1000 2023-05-30 01:31:18 - SimpleLog - INFO: - episode: 3022, ep_reward: 131.573, ep_step: 1000 2023-05-30 01:31:20 - SimpleLog - INFO: - episode: 3023, ep_reward: 135.666, ep_step: 1000 2023-05-30 01:31:22 - SimpleLog - INFO: - episode: 3024, ep_reward: 91.059, ep_step: 1000 2023-05-30 01:31:25 - SimpleLog - INFO: - episode: 3025, ep_reward: 105.696, ep_step: 1000 2023-05-30 01:31:27 - SimpleLog - INFO: - episode: 3026, ep_reward: 134.073, ep_step: 1000 2023-05-30 01:31:29 - SimpleLog - INFO: - episode: 3027, ep_reward: 91.819, ep_step: 1000 2023-05-30 01:31:31 - SimpleLog - INFO: - episode: 3028, ep_reward: 142.550, ep_step: 1000 2023-05-30 01:31:34 - SimpleLog - INFO: - episode: 3029, ep_reward: 9.814, ep_step: 1000 2023-05-30 01:31:36 - SimpleLog - INFO: - episode: 3030, ep_reward: 77.737, ep_step: 1000 2023-05-30 01:31:38 - SimpleLog - INFO: - episode: 3031, ep_reward: 142.816, ep_step: 1000 2023-05-30 01:31:40 - SimpleLog - INFO: - episode: 3032, ep_reward: 78.077, ep_step: 1000 2023-05-30 01:31:41 - SimpleLog - INFO: - episode: 3033, ep_reward: -59.970, ep_step: 468 2023-05-30 01:31:43 - SimpleLog - INFO: - episode: 3034, ep_reward: 0.876, ep_step: 799 2023-05-30 01:31:45 - SimpleLog - INFO: - episode: 3035, ep_reward: 75.774, ep_step: 1000 2023-05-30 01:31:48 - SimpleLog - INFO: - episode: 3036, ep_reward: 87.068, ep_step: 1000 2023-05-30 01:31:49 - SimpleLog - INFO: - episode: 3037, ep_reward: -47.247, ep_step: 458 2023-05-30 01:31:50 - SimpleLog - INFO: - episode: 3038, ep_reward: -126.027, ep_step: 605 2023-05-30 01:31:52 - SimpleLog - INFO: - episode: 3039, ep_reward: 119.124, ep_step: 1000 2023-05-30 01:31:55 - SimpleLog - INFO: - episode: 3040, ep_reward: 100.854, ep_step: 1000 2023-05-30 01:31:57 - SimpleLog - INFO: - episode: 3041, ep_reward: 116.561, ep_step: 1000 2023-05-30 01:31:59 - SimpleLog - INFO: - episode: 3042, ep_reward: 91.851, ep_step: 1000 2023-05-30 01:32:01 - SimpleLog - INFO: - episode: 3043, ep_reward: 107.044, ep_step: 1000 2023-05-30 01:32:03 - SimpleLog - INFO: - episode: 3044, ep_reward: 86.717, ep_step: 1000 2023-05-30 01:32:04 - SimpleLog - INFO: - episode: 3045, ep_reward: -89.774, ep_step: 189 2023-05-30 01:32:06 - SimpleLog - INFO: - episode: 3046, ep_reward: 134.649, ep_step: 1000 2023-05-30 01:32:08 - SimpleLog - INFO: - episode: 3047, ep_reward: 113.129, ep_step: 1000 2023-05-30 01:32:09 - SimpleLog - INFO: - episode: 3048, ep_reward: -57.035, ep_step: 442 2023-05-30 01:32:12 - SimpleLog - INFO: - episode: 3049, ep_reward: -4.421, ep_step: 943 2023-05-30 01:32:14 - SimpleLog - INFO: - episode: 3050, ep_reward: 125.601, ep_step: 1000 2023-05-30 01:32:16 - SimpleLog - INFO: - episode: 3051, ep_reward: 93.866, ep_step: 1000 2023-05-30 01:32:18 - SimpleLog - INFO: - episode: 3052, ep_reward: 112.965, ep_step: 1000 2023-05-30 01:32:21 - SimpleLog - INFO: - episode: 3053, ep_reward: 89.786, ep_step: 1000 2023-05-30 01:32:22 - SimpleLog - INFO: - episode: 3054, ep_reward: -7.358, ep_step: 883 2023-05-30 01:32:26 - SimpleLog - INFO: - update_step: 2170000, online_eval_reward: 9.815 2023-05-30 01:32:27 - SimpleLog - INFO: - episode: 3055, ep_reward: 95.109, ep_step: 1000 2023-05-30 01:32:27 - SimpleLog - INFO: - episode: 3056, ep_reward: -44.380, ep_step: 411 2023-05-30 01:32:30 - SimpleLog - INFO: - episode: 3057, ep_reward: 13.265, ep_step: 1000 2023-05-30 01:32:30 - SimpleLog - INFO: - episode: 3058, ep_reward: -87.473, ep_step: 181 2023-05-30 01:32:32 - SimpleLog - INFO: - episode: 3059, ep_reward: 108.851, ep_step: 1000 2023-05-30 01:32:35 - SimpleLog - INFO: - episode: 3060, ep_reward: 124.592, ep_step: 1000 2023-05-30 01:32:36 - SimpleLog - INFO: - episode: 3061, ep_reward: -54.282, ep_step: 550 2023-05-30 01:32:36 - SimpleLog - INFO: - episode: 3062, ep_reward: -91.576, ep_step: 190 2023-05-30 01:32:39 - SimpleLog - INFO: - episode: 3063, ep_reward: 110.395, ep_step: 1000 2023-05-30 01:32:41 - SimpleLog - INFO: - episode: 3064, ep_reward: 114.573, ep_step: 1000 2023-05-30 01:32:43 - SimpleLog - INFO: - episode: 3065, ep_reward: 115.855, ep_step: 1000 2023-05-30 01:32:45 - SimpleLog - INFO: - episode: 3066, ep_reward: 130.333, ep_step: 1000 2023-05-30 01:32:48 - SimpleLog - INFO: - episode: 3067, ep_reward: 129.439, ep_step: 1000 2023-05-30 01:32:49 - SimpleLog - INFO: - episode: 3068, ep_reward: -64.389, ep_step: 472 2023-05-30 01:32:50 - SimpleLog - INFO: - episode: 3069, ep_reward: -25.277, ep_step: 517 2023-05-30 01:32:52 - SimpleLog - INFO: - episode: 3070, ep_reward: 13.480, ep_step: 1000 2023-05-30 01:32:53 - SimpleLog - INFO: - episode: 3071, ep_reward: -82.847, ep_step: 207 2023-05-30 01:32:55 - SimpleLog - INFO: - episode: 3072, ep_reward: 83.793, ep_step: 1000 2023-05-30 01:32:57 - SimpleLog - INFO: - episode: 3073, ep_reward: 115.000, ep_step: 1000 2023-05-30 01:32:59 - SimpleLog - INFO: - episode: 3074, ep_reward: 108.350, ep_step: 1000 2023-05-30 01:33:01 - SimpleLog - INFO: - episode: 3075, ep_reward: -81.934, ep_step: 602 2023-05-30 01:33:02 - SimpleLog - INFO: - episode: 3076, ep_reward: -50.838, ep_step: 822 2023-05-30 01:33:05 - SimpleLog - INFO: - episode: 3077, ep_reward: 91.184, ep_step: 1000 2023-05-30 01:33:06 - SimpleLog - INFO: - episode: 3078, ep_reward: -8.292, ep_step: 659 2023-05-30 01:33:08 - SimpleLog - INFO: - episode: 3079, ep_reward: 137.414, ep_step: 1000 2023-05-30 01:33:10 - SimpleLog - INFO: - episode: 3080, ep_reward: 12.132, ep_step: 878 2023-05-30 01:33:13 - SimpleLog - INFO: - episode: 3081, ep_reward: 4.040, ep_step: 969 2023-05-30 01:33:15 - SimpleLog - INFO: - episode: 3082, ep_reward: 113.777, ep_step: 1000 2023-05-30 01:33:16 - SimpleLog - INFO: - episode: 3083, ep_reward: -12.554, ep_step: 704 2023-05-30 01:33:17 - SimpleLog - INFO: - episode: 3084, ep_reward: -58.420, ep_step: 429 2023-05-30 01:33:20 - SimpleLog - INFO: - episode: 3085, ep_reward: 19.835, ep_step: 1000 2023-05-30 01:33:22 - SimpleLog - INFO: - episode: 3086, ep_reward: 126.511, ep_step: 1000 2023-05-30 01:33:24 - SimpleLog - INFO: - episode: 3087, ep_reward: 114.284, ep_step: 1000 2023-05-30 01:33:26 - SimpleLog - INFO: - episode: 3088, ep_reward: -11.752, ep_step: 770 2023-05-30 01:33:28 - SimpleLog - INFO: - episode: 3089, ep_reward: 92.489, ep_step: 1000 2023-05-30 01:33:30 - SimpleLog - INFO: - episode: 3090, ep_reward: 22.941, ep_step: 1000 2023-05-30 01:33:33 - SimpleLog - INFO: - episode: 3091, ep_reward: 31.414, ep_step: 1000 2023-05-30 01:33:35 - SimpleLog - INFO: - episode: 3092, ep_reward: 118.954, ep_step: 1000 2023-05-30 01:33:37 - SimpleLog - INFO: - episode: 3093, ep_reward: 91.041, ep_step: 1000 2023-05-30 01:33:39 - SimpleLog - INFO: - episode: 3094, ep_reward: 106.082, ep_step: 1000 2023-05-30 01:33:42 - SimpleLog - INFO: - episode: 3095, ep_reward: 116.593, ep_step: 1000 2023-05-30 01:33:44 - SimpleLog - INFO: - episode: 3096, ep_reward: 107.725, ep_step: 1000 2023-05-30 01:33:46 - SimpleLog - INFO: - episode: 3097, ep_reward: 95.302, ep_step: 1000 2023-05-30 01:33:48 - SimpleLog - INFO: - episode: 3098, ep_reward: 107.878, ep_step: 1000 2023-05-30 01:33:51 - SimpleLog - INFO: - episode: 3099, ep_reward: 108.131, ep_step: 1000 2023-05-30 01:33:52 - SimpleLog - INFO: - episode: 3100, ep_reward: -64.743, ep_step: 562 2023-05-30 01:33:53 - SimpleLog - INFO: - episode: 3101, ep_reward: -77.655, ep_step: 657 2023-05-30 01:33:56 - SimpleLog - INFO: - episode: 3102, ep_reward: 118.047, ep_step: 1000 2023-05-30 01:33:58 - SimpleLog - INFO: - episode: 3103, ep_reward: 106.273, ep_step: 1000 2023-05-30 01:33:59 - SimpleLog - INFO: - episode: 3104, ep_reward: -26.840, ep_step: 575 2023-05-30 01:34:01 - SimpleLog - INFO: - episode: 3105, ep_reward: 132.863, ep_step: 1000 2023-05-30 01:34:04 - SimpleLog - INFO: - episode: 3106, ep_reward: 131.169, ep_step: 1000 2023-05-30 01:34:06 - SimpleLog - INFO: - episode: 3107, ep_reward: 97.043, ep_step: 1000 2023-05-30 01:34:08 - SimpleLog - INFO: - episode: 3108, ep_reward: 111.417, ep_step: 1000 2023-05-30 01:34:10 - SimpleLog - INFO: - episode: 3109, ep_reward: -10.838, ep_step: 923 2023-05-30 01:34:12 - SimpleLog - INFO: - episode: 3110, ep_reward: 131.232, ep_step: 1000 2023-05-30 01:34:15 - SimpleLog - INFO: - episode: 3111, ep_reward: 105.968, ep_step: 1000 2023-05-30 01:34:17 - SimpleLog - INFO: - episode: 3112, ep_reward: 102.605, ep_step: 1000 2023-05-30 01:34:19 - SimpleLog - INFO: - episode: 3113, ep_reward: 55.706, ep_step: 1000 2023-05-30 01:34:21 - SimpleLog - INFO: - episode: 3114, ep_reward: -34.595, ep_step: 1000 2023-05-30 01:34:24 - SimpleLog - INFO: - episode: 3115, ep_reward: 49.267, ep_step: 1000 2023-05-30 01:34:26 - SimpleLog - INFO: - episode: 3116, ep_reward: 97.599, ep_step: 1000 2023-05-30 01:34:28 - SimpleLog - INFO: - episode: 3117, ep_reward: 101.051, ep_step: 1000 2023-05-30 01:34:30 - SimpleLog - INFO: - episode: 3118, ep_reward: 97.446, ep_step: 1000 2023-05-30 01:34:33 - SimpleLog - INFO: - episode: 3119, ep_reward: 118.687, ep_step: 1000 2023-05-30 01:34:35 - SimpleLog - INFO: - episode: 3120, ep_reward: 86.012, ep_step: 1000 2023-05-30 01:34:37 - SimpleLog - INFO: - episode: 3121, ep_reward: 98.697, ep_step: 1000 2023-05-30 01:34:39 - SimpleLog - INFO: - episode: 3122, ep_reward: 106.808, ep_step: 1000 2023-05-30 01:34:42 - SimpleLog - INFO: - episode: 3123, ep_reward: 100.041, ep_step: 1000 2023-05-30 01:34:44 - SimpleLog - INFO: - episode: 3124, ep_reward: 108.204, ep_step: 1000 2023-05-30 01:34:46 - SimpleLog - INFO: - episode: 3125, ep_reward: 111.155, ep_step: 1000 2023-05-30 01:34:48 - SimpleLog - INFO: - episode: 3126, ep_reward: 98.968, ep_step: 1000 2023-05-30 01:34:51 - SimpleLog - INFO: - episode: 3127, ep_reward: 93.747, ep_step: 1000 2023-05-30 01:34:53 - SimpleLog - INFO: - episode: 3128, ep_reward: 85.063, ep_step: 1000 2023-05-30 01:34:54 - SimpleLog - INFO: - episode: 3129, ep_reward: -83.426, ep_step: 354 2023-05-30 01:34:54 - SimpleLog - INFO: - episode: 3130, ep_reward: -86.936, ep_step: 160 2023-05-30 01:34:56 - SimpleLog - INFO: - episode: 3131, ep_reward: 89.826, ep_step: 1000 2023-05-30 01:34:59 - SimpleLog - INFO: - episode: 3132, ep_reward: -39.877, ep_step: 1000 2023-05-30 01:35:01 - SimpleLog - INFO: - episode: 3133, ep_reward: 113.558, ep_step: 1000 2023-05-30 01:35:03 - SimpleLog - INFO: - episode: 3134, ep_reward: 82.481, ep_step: 1000 2023-05-30 01:35:05 - SimpleLog - INFO: - update_step: 2240000, online_eval_reward: 111.051 2023-05-30 01:35:07 - SimpleLog - INFO: - episode: 3135, ep_reward: 114.333, ep_step: 1000 2023-05-30 01:35:10 - SimpleLog - INFO: - episode: 3136, ep_reward: 108.013, ep_step: 1000 2023-05-30 01:35:12 - SimpleLog - INFO: - episode: 3137, ep_reward: 98.471, ep_step: 1000 2023-05-30 01:35:14 - SimpleLog - INFO: - episode: 3138, ep_reward: -10.756, ep_step: 1000 2023-05-30 01:35:16 - SimpleLog - INFO: - episode: 3139, ep_reward: 114.597, ep_step: 1000 2023-05-30 01:35:19 - SimpleLog - INFO: - episode: 3140, ep_reward: 114.366, ep_step: 1000 2023-05-30 01:35:21 - SimpleLog - INFO: - episode: 3141, ep_reward: 121.118, ep_step: 1000 2023-05-30 01:35:23 - SimpleLog - INFO: - episode: 3142, ep_reward: 108.058, ep_step: 1000 2023-05-30 01:35:25 - SimpleLog - INFO: - episode: 3143, ep_reward: 82.927, ep_step: 1000 2023-05-30 01:35:26 - SimpleLog - INFO: - episode: 3144, ep_reward: -87.764, ep_step: 145 2023-05-30 01:35:28 - SimpleLog - INFO: - episode: 3145, ep_reward: 55.714, ep_step: 1000 2023-05-30 01:35:30 - SimpleLog - INFO: - episode: 3146, ep_reward: 108.240, ep_step: 1000 2023-05-30 01:35:32 - SimpleLog - INFO: - episode: 3147, ep_reward: 98.678, ep_step: 1000 2023-05-30 01:35:35 - SimpleLog - INFO: - episode: 3148, ep_reward: 114.292, ep_step: 1000 2023-05-30 01:35:37 - SimpleLog - INFO: - episode: 3149, ep_reward: 91.813, ep_step: 1000 2023-05-30 01:35:37 - SimpleLog - INFO: - episode: 3150, ep_reward: -84.601, ep_step: 202 2023-05-30 01:35:39 - SimpleLog - INFO: - episode: 3151, ep_reward: -6.828, ep_step: 890 2023-05-30 01:35:41 - SimpleLog - INFO: - episode: 3152, ep_reward: -42.347, ep_step: 678 2023-05-30 01:35:42 - SimpleLog - INFO: - episode: 3153, ep_reward: -34.103, ep_step: 576 2023-05-30 01:35:44 - SimpleLog - INFO: - episode: 3154, ep_reward: 42.886, ep_step: 1000 2023-05-30 01:35:47 - SimpleLog - INFO: - episode: 3155, ep_reward: 116.109, ep_step: 1000 2023-05-30 01:35:48 - SimpleLog - INFO: - episode: 3156, ep_reward: -63.328, ep_step: 517 2023-05-30 01:35:49 - SimpleLog - INFO: - episode: 3157, ep_reward: -43.527, ep_step: 672 2023-05-30 01:35:51 - SimpleLog - INFO: - episode: 3158, ep_reward: 93.378, ep_step: 1000 2023-05-30 01:35:52 - SimpleLog - INFO: - episode: 3159, ep_reward: -100.888, ep_step: 212 2023-05-30 01:35:54 - SimpleLog - INFO: - episode: 3160, ep_reward: -51.358, ep_step: 1000 2023-05-30 01:35:56 - SimpleLog - INFO: - episode: 3161, ep_reward: -57.137, ep_step: 1000 2023-05-30 01:35:58 - SimpleLog - INFO: - episode: 3162, ep_reward: -144.990, ep_step: 564 2023-05-30 01:36:00 - SimpleLog - INFO: - episode: 3163, ep_reward: 89.286, ep_step: 1000 2023-05-30 01:36:01 - SimpleLog - INFO: - episode: 3164, ep_reward: -95.094, ep_step: 240 2023-05-30 01:36:02 - SimpleLog - INFO: - episode: 3165, ep_reward: -65.077, ep_step: 488 2023-05-30 01:36:04 - SimpleLog - INFO: - episode: 3166, ep_reward: 80.174, ep_step: 1000 2023-05-30 01:36:06 - SimpleLog - INFO: - episode: 3167, ep_reward: 88.589, ep_step: 1000 2023-05-30 01:36:07 - SimpleLog - INFO: - episode: 3168, ep_reward: -58.736, ep_step: 408 2023-05-30 01:36:09 - SimpleLog - INFO: - episode: 3169, ep_reward: -21.725, ep_step: 775 2023-05-30 01:36:10 - SimpleLog - INFO: - episode: 3170, ep_reward: -34.281, ep_step: 523 2023-05-30 01:36:12 - SimpleLog - INFO: - episode: 3171, ep_reward: 85.196, ep_step: 1000 2023-05-30 01:36:14 - SimpleLog - INFO: - episode: 3172, ep_reward: 82.144, ep_step: 1000 2023-05-30 01:36:17 - SimpleLog - INFO: - episode: 3173, ep_reward: 98.496, ep_step: 1000 2023-05-30 01:36:19 - SimpleLog - INFO: - episode: 3174, ep_reward: 14.864, ep_step: 1000 2023-05-30 01:36:21 - SimpleLog - INFO: - episode: 3175, ep_reward: 90.821, ep_step: 1000 2023-05-30 01:36:23 - SimpleLog - INFO: - episode: 3176, ep_reward: 115.232, ep_step: 1000 2023-05-30 01:36:26 - SimpleLog - INFO: - episode: 3177, ep_reward: 95.278, ep_step: 1000 2023-05-30 01:36:28 - SimpleLog - INFO: - episode: 3178, ep_reward: 109.657, ep_step: 1000 2023-05-30 01:36:30 - SimpleLog - INFO: - episode: 3179, ep_reward: 106.084, ep_step: 1000 2023-05-30 01:36:32 - SimpleLog - INFO: - episode: 3180, ep_reward: 96.848, ep_step: 1000 2023-05-30 01:36:34 - SimpleLog - INFO: - episode: 3181, ep_reward: -70.390, ep_step: 616 2023-05-30 01:36:36 - SimpleLog - INFO: - episode: 3182, ep_reward: 111.612, ep_step: 1000 2023-05-30 01:36:38 - SimpleLog - INFO: - episode: 3183, ep_reward: 5.016, ep_step: 1000 2023-05-30 01:36:41 - SimpleLog - INFO: - episode: 3184, ep_reward: 125.176, ep_step: 1000 2023-05-30 01:36:43 - SimpleLog - INFO: - episode: 3185, ep_reward: 114.101, ep_step: 1000 2023-05-30 01:36:45 - SimpleLog - INFO: - episode: 3186, ep_reward: 108.100, ep_step: 1000 2023-05-30 01:36:47 - SimpleLog - INFO: - episode: 3187, ep_reward: 128.997, ep_step: 1000 2023-05-30 01:36:50 - SimpleLog - INFO: - episode: 3188, ep_reward: 107.705, ep_step: 1000 2023-05-30 01:36:52 - SimpleLog - INFO: - episode: 3189, ep_reward: 7.901, ep_step: 930 2023-05-30 01:36:54 - SimpleLog - INFO: - episode: 3190, ep_reward: 3.362, ep_step: 999 2023-05-30 01:36:56 - SimpleLog - INFO: - episode: 3191, ep_reward: 123.138, ep_step: 1000 2023-05-30 01:36:59 - SimpleLog - INFO: - episode: 3192, ep_reward: 116.149, ep_step: 1000 2023-05-30 01:37:00 - SimpleLog - INFO: - episode: 3193, ep_reward: -11.789, ep_step: 876 2023-05-30 01:37:03 - SimpleLog - INFO: - episode: 3194, ep_reward: 111.843, ep_step: 1000 2023-05-30 01:37:05 - SimpleLog - INFO: - episode: 3195, ep_reward: 100.233, ep_step: 1000 2023-05-30 01:37:06 - SimpleLog - INFO: - episode: 3196, ep_reward: -51.142, ep_step: 529 2023-05-30 01:37:08 - SimpleLog - INFO: - episode: 3197, ep_reward: 94.166, ep_step: 1000 2023-05-30 01:37:10 - SimpleLog - INFO: - episode: 3198, ep_reward: -53.539, ep_step: 570 2023-05-30 01:37:12 - SimpleLog - INFO: - episode: 3199, ep_reward: 93.135, ep_step: 1000 2023-05-30 01:37:13 - SimpleLog - INFO: - episode: 3200, ep_reward: -55.916, ep_step: 529 2023-05-30 01:37:15 - SimpleLog - INFO: - episode: 3201, ep_reward: 109.691, ep_step: 1000 2023-05-30 01:37:18 - SimpleLog - INFO: - episode: 3202, ep_reward: 107.936, ep_step: 1000 2023-05-30 01:37:20 - SimpleLog - INFO: - episode: 3203, ep_reward: 113.392, ep_step: 1000 2023-05-30 01:37:22 - SimpleLog - INFO: - episode: 3204, ep_reward: 116.277, ep_step: 1000 2023-05-30 01:37:24 - SimpleLog - INFO: - episode: 3205, ep_reward: 71.533, ep_step: 1000 2023-05-30 01:37:27 - SimpleLog - INFO: - episode: 3206, ep_reward: 110.371, ep_step: 1000 2023-05-30 01:37:29 - SimpleLog - INFO: - episode: 3207, ep_reward: 118.622, ep_step: 1000 2023-05-30 01:37:31 - SimpleLog - INFO: - episode: 3208, ep_reward: 129.920, ep_step: 1000 2023-05-30 01:37:32 - SimpleLog - INFO: - episode: 3209, ep_reward: -50.848, ep_step: 549 2023-05-30 01:37:35 - SimpleLog - INFO: - episode: 3210, ep_reward: 117.233, ep_step: 1000 2023-05-30 01:37:37 - SimpleLog - INFO: - episode: 3211, ep_reward: -9.141, ep_step: 914 2023-05-30 01:37:38 - SimpleLog - INFO: - episode: 3212, ep_reward: -74.993, ep_step: 690 2023-05-30 01:37:41 - SimpleLog - INFO: - episode: 3213, ep_reward: 105.585, ep_step: 1000 2023-05-30 01:37:45 - SimpleLog - INFO: - update_step: 2310000, online_eval_reward: 116.691 2023-05-30 01:37:45 - SimpleLog - INFO: - episode: 3214, ep_reward: 80.456, ep_step: 1000 2023-05-30 01:37:47 - SimpleLog - INFO: - episode: 3215, ep_reward: -63.366, ep_step: 1000 2023-05-30 01:37:48 - SimpleLog - INFO: - episode: 3216, ep_reward: -42.065, ep_step: 607 2023-05-30 01:37:51 - SimpleLog - INFO: - episode: 3217, ep_reward: -64.207, ep_step: 1000 2023-05-30 01:37:52 - SimpleLog - INFO: - episode: 3218, ep_reward: -48.578, ep_step: 541 2023-05-30 01:37:54 - SimpleLog - INFO: - episode: 3219, ep_reward: 112.008, ep_step: 1000 2023-05-30 01:37:56 - SimpleLog - INFO: - episode: 3220, ep_reward: 104.516, ep_step: 1000 2023-05-30 01:37:59 - SimpleLog - INFO: - episode: 3221, ep_reward: -3.831, ep_step: 1000 2023-05-30 01:38:01 - SimpleLog - INFO: - episode: 3222, ep_reward: 106.429, ep_step: 1000 2023-05-30 01:38:03 - SimpleLog - INFO: - episode: 3223, ep_reward: 122.661, ep_step: 1000 2023-05-30 01:38:05 - SimpleLog - INFO: - episode: 3224, ep_reward: 69.168, ep_step: 1000 2023-05-30 01:38:07 - SimpleLog - INFO: - episode: 3225, ep_reward: -40.487, ep_step: 722 2023-05-30 01:38:09 - SimpleLog - INFO: - episode: 3226, ep_reward: 81.977, ep_step: 1000 2023-05-30 01:38:11 - SimpleLog - INFO: - episode: 3227, ep_reward: 106.963, ep_step: 1000 2023-05-30 01:38:14 - SimpleLog - INFO: - episode: 3228, ep_reward: 116.468, ep_step: 1000 2023-05-30 01:38:15 - SimpleLog - INFO: - episode: 3229, ep_reward: -18.152, ep_step: 769 2023-05-30 01:38:18 - SimpleLog - INFO: - episode: 3230, ep_reward: -53.311, ep_step: 1000 2023-05-30 01:38:20 - SimpleLog - INFO: - episode: 3231, ep_reward: -44.126, ep_step: 1000 2023-05-30 01:38:21 - SimpleLog - INFO: - episode: 3232, ep_reward: -38.845, ep_step: 575 2023-05-30 01:38:23 - SimpleLog - INFO: - episode: 3233, ep_reward: 124.022, ep_step: 1000 2023-05-30 01:38:26 - SimpleLog - INFO: - episode: 3234, ep_reward: 78.682, ep_step: 1000 2023-05-30 01:38:28 - SimpleLog - INFO: - episode: 3235, ep_reward: 117.886, ep_step: 1000 2023-05-30 01:38:30 - SimpleLog - INFO: - episode: 3236, ep_reward: 117.270, ep_step: 1000 2023-05-30 01:38:32 - SimpleLog - INFO: - episode: 3237, ep_reward: 110.611, ep_step: 1000 2023-05-30 01:38:35 - SimpleLog - INFO: - episode: 3238, ep_reward: 114.761, ep_step: 1000 2023-05-30 01:38:37 - SimpleLog - INFO: - episode: 3239, ep_reward: 3.369, ep_step: 1000 2023-05-30 01:38:39 - SimpleLog - INFO: - episode: 3240, ep_reward: 107.516, ep_step: 1000 2023-05-30 01:38:41 - SimpleLog - INFO: - episode: 3241, ep_reward: 115.771, ep_step: 1000 2023-05-30 01:38:42 - SimpleLog - INFO: - episode: 3242, ep_reward: -67.306, ep_step: 478 2023-05-30 01:38:45 - SimpleLog - INFO: - episode: 3243, ep_reward: 113.590, ep_step: 1000 2023-05-30 01:38:46 - SimpleLog - INFO: - episode: 3244, ep_reward: -5.799, ep_step: 820 2023-05-30 01:38:48 - SimpleLog - INFO: - episode: 3245, ep_reward: -16.643, ep_step: 726 2023-05-30 01:38:50 - SimpleLog - INFO: - episode: 3246, ep_reward: 110.883, ep_step: 1000 2023-05-30 01:38:53 - SimpleLog - INFO: - episode: 3247, ep_reward: 104.431, ep_step: 1000 2023-05-30 01:38:54 - SimpleLog - INFO: - episode: 3248, ep_reward: -56.592, ep_step: 499 2023-05-30 01:38:56 - SimpleLog - INFO: - episode: 3249, ep_reward: 112.170, ep_step: 1000 2023-05-30 01:38:58 - SimpleLog - INFO: - episode: 3250, ep_reward: 76.952, ep_step: 1000 2023-05-30 01:39:00 - SimpleLog - INFO: - episode: 3251, ep_reward: 133.179, ep_step: 1000 2023-05-30 01:39:03 - SimpleLog - INFO: - episode: 3252, ep_reward: 11.460, ep_step: 937 2023-05-30 01:39:05 - SimpleLog - INFO: - episode: 3253, ep_reward: 116.267, ep_step: 1000 2023-05-30 01:39:07 - SimpleLog - INFO: - episode: 3254, ep_reward: 89.423, ep_step: 1000 2023-05-30 01:39:09 - SimpleLog - INFO: - episode: 3255, ep_reward: 108.480, ep_step: 1000 2023-05-30 01:39:11 - SimpleLog - INFO: - episode: 3256, ep_reward: 112.963, ep_step: 1000 2023-05-30 01:39:14 - SimpleLog - INFO: - episode: 3257, ep_reward: 100.377, ep_step: 1000 2023-05-30 01:39:16 - SimpleLog - INFO: - episode: 3258, ep_reward: 107.945, ep_step: 1000 2023-05-30 01:39:18 - SimpleLog - INFO: - episode: 3259, ep_reward: 113.775, ep_step: 1000 2023-05-30 01:39:20 - SimpleLog - INFO: - episode: 3260, ep_reward: 77.917, ep_step: 1000 2023-05-30 01:39:23 - SimpleLog - INFO: - episode: 3261, ep_reward: 97.661, ep_step: 1000 2023-05-30 01:39:25 - SimpleLog - INFO: - episode: 3262, ep_reward: 114.637, ep_step: 1000 2023-05-30 01:39:27 - SimpleLog - INFO: - episode: 3263, ep_reward: 86.086, ep_step: 1000 2023-05-30 01:39:30 - SimpleLog - INFO: - episode: 3264, ep_reward: 105.808, ep_step: 1000 2023-05-30 01:39:32 - SimpleLog - INFO: - episode: 3265, ep_reward: 109.132, ep_step: 1000 2023-05-30 01:39:34 - SimpleLog - INFO: - episode: 3266, ep_reward: 81.739, ep_step: 1000 2023-05-30 01:39:36 - SimpleLog - INFO: - episode: 3267, ep_reward: 113.392, ep_step: 1000 2023-05-30 01:39:39 - SimpleLog - INFO: - episode: 3268, ep_reward: 113.971, ep_step: 1000 2023-05-30 01:39:41 - SimpleLog - INFO: - episode: 3269, ep_reward: 113.782, ep_step: 1000 2023-05-30 01:39:43 - SimpleLog - INFO: - episode: 3270, ep_reward: 100.001, ep_step: 1000 2023-05-30 01:39:45 - SimpleLog - INFO: - episode: 3271, ep_reward: 62.886, ep_step: 1000 2023-05-30 01:39:48 - SimpleLog - INFO: - episode: 3272, ep_reward: 121.982, ep_step: 1000 2023-05-30 01:39:50 - SimpleLog - INFO: - episode: 3273, ep_reward: 78.093, ep_step: 1000 2023-05-30 01:39:52 - SimpleLog - INFO: - episode: 3274, ep_reward: 91.703, ep_step: 1000 2023-05-30 01:39:54 - SimpleLog - INFO: - episode: 3275, ep_reward: 90.730, ep_step: 1000 2023-05-30 01:39:57 - SimpleLog - INFO: - episode: 3276, ep_reward: 114.906, ep_step: 1000 2023-05-30 01:39:59 - SimpleLog - INFO: - episode: 3277, ep_reward: 115.135, ep_step: 1000 2023-05-30 01:40:01 - SimpleLog - INFO: - episode: 3278, ep_reward: -21.177, ep_step: 1000 2023-05-30 01:40:03 - SimpleLog - INFO: - episode: 3279, ep_reward: 3.008, ep_step: 822 2023-05-30 01:40:05 - SimpleLog - INFO: - episode: 3280, ep_reward: 80.009, ep_step: 1000 2023-05-30 01:40:07 - SimpleLog - INFO: - episode: 3281, ep_reward: 87.915, ep_step: 1000 2023-05-30 01:40:10 - SimpleLog - INFO: - episode: 3282, ep_reward: 128.703, ep_step: 1000 2023-05-30 01:40:12 - SimpleLog - INFO: - episode: 3283, ep_reward: 134.022, ep_step: 1000 2023-05-30 01:40:13 - SimpleLog - INFO: - episode: 3284, ep_reward: -43.336, ep_step: 604 2023-05-30 01:40:15 - SimpleLog - INFO: - episode: 3285, ep_reward: -8.833, ep_step: 809 2023-05-30 01:40:17 - SimpleLog - INFO: - episode: 3286, ep_reward: 65.920, ep_step: 1000 2023-05-30 01:40:18 - SimpleLog - INFO: - episode: 3287, ep_reward: -141.679, ep_step: 212 2023-05-30 01:40:20 - SimpleLog - INFO: - episode: 3288, ep_reward: 82.431, ep_step: 1000 2023-05-30 01:40:22 - SimpleLog - INFO: - update_step: 2380000, online_eval_reward: -125.483 2023-05-30 01:40:23 - SimpleLog - INFO: - episode: 3289, ep_reward: 89.787, ep_step: 1000 2023-05-30 01:40:24 - SimpleLog - INFO: - episode: 3290, ep_reward: -57.545, ep_step: 647 2023-05-30 01:40:26 - SimpleLog - INFO: - episode: 3291, ep_reward: 57.936, ep_step: 1000 2023-05-30 01:40:29 - SimpleLog - INFO: - episode: 3292, ep_reward: 18.651, ep_step: 1000 2023-05-30 01:40:31 - SimpleLog - INFO: - episode: 3293, ep_reward: -44.619, ep_step: 961 2023-05-30 01:40:33 - SimpleLog - INFO: - episode: 3294, ep_reward: 79.784, ep_step: 1000 2023-05-30 01:40:35 - SimpleLog - INFO: - episode: 3295, ep_reward: 113.568, ep_step: 1000 2023-05-30 01:40:37 - SimpleLog - INFO: - episode: 3296, ep_reward: 28.004, ep_step: 1000 2023-05-30 01:40:40 - SimpleLog - INFO: - episode: 3297, ep_reward: 94.512, ep_step: 1000 2023-05-30 01:40:42 - SimpleLog - INFO: - episode: 3298, ep_reward: 102.329, ep_step: 1000 2023-05-30 01:40:44 - SimpleLog - INFO: - episode: 3299, ep_reward: 81.975, ep_step: 1000 2023-05-30 01:40:46 - SimpleLog - INFO: - episode: 3300, ep_reward: -4.194, ep_step: 883 2023-05-30 01:40:48 - SimpleLog - INFO: - episode: 3301, ep_reward: 114.671, ep_step: 1000 2023-05-30 01:40:50 - SimpleLog - INFO: - episode: 3302, ep_reward: -27.590, ep_step: 829 2023-05-30 01:40:53 - SimpleLog - INFO: - episode: 3303, ep_reward: 106.808, ep_step: 1000 2023-05-30 01:40:54 - SimpleLog - INFO: - episode: 3304, ep_reward: -14.157, ep_step: 757 2023-05-30 01:40:57 - SimpleLog - INFO: - episode: 3305, ep_reward: 111.856, ep_step: 1000 2023-05-30 01:40:59 - SimpleLog - INFO: - episode: 3306, ep_reward: 106.377, ep_step: 1000 2023-05-30 01:41:01 - SimpleLog - INFO: - episode: 3307, ep_reward: 101.687, ep_step: 1000 2023-05-30 01:41:03 - SimpleLog - INFO: - episode: 3308, ep_reward: 113.236, ep_step: 1000 2023-05-30 01:41:06 - SimpleLog - INFO: - episode: 3309, ep_reward: 102.419, ep_step: 1000 2023-05-30 01:41:08 - SimpleLog - INFO: - episode: 3310, ep_reward: 101.867, ep_step: 1000 2023-05-30 01:41:10 - SimpleLog - INFO: - episode: 3311, ep_reward: 79.590, ep_step: 1000 2023-05-30 01:41:12 - SimpleLog - INFO: - episode: 3312, ep_reward: 106.914, ep_step: 1000 2023-05-30 01:41:14 - SimpleLog - INFO: - episode: 3313, ep_reward: -21.435, ep_step: 766 2023-05-30 01:41:16 - SimpleLog - INFO: - episode: 3314, ep_reward: 77.397, ep_step: 1000 2023-05-30 01:41:19 - SimpleLog - INFO: - episode: 3315, ep_reward: 96.440, ep_step: 1000 2023-05-30 01:41:21 - SimpleLog - INFO: - episode: 3316, ep_reward: 106.409, ep_step: 1000 2023-05-30 01:41:23 - SimpleLog - INFO: - episode: 3317, ep_reward: 100.047, ep_step: 1000 2023-05-30 01:41:25 - SimpleLog - INFO: - episode: 3318, ep_reward: 10.740, ep_step: 1000 2023-05-30 01:41:27 - SimpleLog - INFO: - episode: 3319, ep_reward: 109.108, ep_step: 1000 2023-05-30 01:41:30 - SimpleLog - INFO: - episode: 3320, ep_reward: 96.534, ep_step: 1000 2023-05-30 01:41:32 - SimpleLog - INFO: - episode: 3321, ep_reward: 105.397, ep_step: 1000 2023-05-30 01:41:34 - SimpleLog - INFO: - episode: 3322, ep_reward: 125.923, ep_step: 1000 2023-05-30 01:41:35 - SimpleLog - INFO: - episode: 3323, ep_reward: -90.930, ep_step: 226 2023-05-30 01:41:37 - SimpleLog - INFO: - episode: 3324, ep_reward: 101.154, ep_step: 1000 2023-05-30 01:41:39 - SimpleLog - INFO: - episode: 3325, ep_reward: 102.401, ep_step: 1000 2023-05-30 01:41:42 - SimpleLog - INFO: - episode: 3326, ep_reward: 89.884, ep_step: 1000 2023-05-30 01:41:44 - SimpleLog - INFO: - episode: 3327, ep_reward: 4.854, ep_step: 1000 2023-05-30 01:41:46 - SimpleLog - INFO: - episode: 3328, ep_reward: 76.075, ep_step: 1000 2023-05-30 01:41:48 - SimpleLog - INFO: - episode: 3329, ep_reward: 80.878, ep_step: 1000 2023-05-30 01:41:51 - SimpleLog - INFO: - episode: 3330, ep_reward: 95.543, ep_step: 1000 2023-05-30 01:41:53 - SimpleLog - INFO: - episode: 3331, ep_reward: 109.237, ep_step: 1000 2023-05-30 01:41:55 - SimpleLog - INFO: - episode: 3332, ep_reward: 108.318, ep_step: 1000 2023-05-30 01:41:57 - SimpleLog - INFO: - episode: 3333, ep_reward: 100.846, ep_step: 1000 2023-05-30 01:42:00 - SimpleLog - INFO: - episode: 3334, ep_reward: 99.391, ep_step: 1000 2023-05-30 01:42:02 - SimpleLog - INFO: - episode: 3335, ep_reward: 109.758, ep_step: 1000 2023-05-30 01:42:04 - SimpleLog - INFO: - episode: 3336, ep_reward: 102.095, ep_step: 1000 2023-05-30 01:42:06 - SimpleLog - INFO: - episode: 3337, ep_reward: 103.704, ep_step: 1000 2023-05-30 01:42:09 - SimpleLog - INFO: - episode: 3338, ep_reward: 103.653, ep_step: 1000 2023-05-30 01:42:11 - SimpleLog - INFO: - episode: 3339, ep_reward: 100.686, ep_step: 1000 2023-05-30 01:42:13 - SimpleLog - INFO: - episode: 3340, ep_reward: 102.668, ep_step: 1000 2023-05-30 01:42:15 - SimpleLog - INFO: - episode: 3341, ep_reward: 98.846, ep_step: 1000 2023-05-30 01:42:17 - SimpleLog - INFO: - episode: 3342, ep_reward: 105.746, ep_step: 1000 2023-05-30 01:42:20 - SimpleLog - INFO: - episode: 3343, ep_reward: 121.200, ep_step: 1000 2023-05-30 01:42:22 - SimpleLog - INFO: - episode: 3344, ep_reward: 118.469, ep_step: 1000 2023-05-30 01:42:24 - SimpleLog - INFO: - episode: 3345, ep_reward: 137.707, ep_step: 1000 2023-05-30 01:42:26 - SimpleLog - INFO: - episode: 3346, ep_reward: 2.789, ep_step: 766 2023-05-30 01:42:28 - SimpleLog - INFO: - episode: 3347, ep_reward: 130.550, ep_step: 1000 2023-05-30 01:42:30 - SimpleLog - INFO: - episode: 3348, ep_reward: 115.530, ep_step: 1000 2023-05-30 01:42:33 - SimpleLog - INFO: - episode: 3349, ep_reward: 123.625, ep_step: 1000 2023-05-30 01:42:34 - SimpleLog - INFO: - episode: 3350, ep_reward: 5.159, ep_step: 706 2023-05-30 01:42:37 - SimpleLog - INFO: - episode: 3351, ep_reward: 106.810, ep_step: 1000 2023-05-30 01:42:39 - SimpleLog - INFO: - episode: 3352, ep_reward: 7.899, ep_step: 977 2023-05-30 01:42:40 - SimpleLog - INFO: - episode: 3353, ep_reward: -115.662, ep_step: 394 2023-05-30 01:42:42 - SimpleLog - INFO: - episode: 3354, ep_reward: 104.550, ep_step: 1000 2023-05-30 01:42:44 - SimpleLog - INFO: - episode: 3355, ep_reward: 115.863, ep_step: 1000 2023-05-30 01:42:46 - SimpleLog - INFO: - episode: 3356, ep_reward: 117.801, ep_step: 1000 2023-05-30 01:42:47 - SimpleLog - INFO: - episode: 3357, ep_reward: -101.528, ep_step: 78 2023-05-30 01:42:49 - SimpleLog - INFO: - episode: 3358, ep_reward: 110.915, ep_step: 1000 2023-05-30 01:42:49 - SimpleLog - INFO: - episode: 3359, ep_reward: -102.331, ep_step: 41 2023-05-30 01:42:49 - SimpleLog - INFO: - episode: 3360, ep_reward: -102.336, ep_step: 37 2023-05-30 01:42:49 - SimpleLog - INFO: - episode: 3361, ep_reward: -101.876, ep_step: 38 2023-05-30 01:42:49 - SimpleLog - INFO: - episode: 3362, ep_reward: -104.098, ep_step: 58 2023-05-30 01:42:51 - SimpleLog - INFO: - episode: 3363, ep_reward: -99.867, ep_step: 1000 2023-05-30 01:42:54 - SimpleLog - INFO: - episode: 3364, ep_reward: -113.993, ep_step: 1000 2023-05-30 01:42:56 - SimpleLog - INFO: - episode: 3365, ep_reward: -104.432, ep_step: 1000 2023-05-30 01:42:58 - SimpleLog - INFO: - episode: 3366, ep_reward: -103.305, ep_step: 1000 2023-05-30 01:43:02 - SimpleLog - INFO: - update_step: 2450000, online_eval_reward: -114.404 2023-05-30 01:43:02 - SimpleLog - INFO: - episode: 3367, ep_reward: -102.248, ep_step: 1000 2023-05-30 01:43:04 - SimpleLog - INFO: - episode: 3368, ep_reward: -141.463, ep_step: 512 2023-05-30 01:43:04 - SimpleLog - INFO: - episode: 3369, ep_reward: -103.947, ep_step: 101 2023-05-30 01:43:04 - SimpleLog - INFO: - episode: 3370, ep_reward: -96.241, ep_step: 114 2023-05-30 01:43:04 - SimpleLog - INFO: - episode: 3371, ep_reward: -100.452, ep_step: 117 2023-05-30 01:43:05 - SimpleLog - INFO: - episode: 3372, ep_reward: -131.297, ep_step: 138 2023-05-30 01:43:05 - SimpleLog - INFO: - episode: 3373, ep_reward: -136.166, ep_step: 176 2023-05-30 01:43:05 - SimpleLog - INFO: - episode: 3374, ep_reward: -121.124, ep_step: 84 2023-05-30 01:43:06 - SimpleLog - INFO: - episode: 3375, ep_reward: -136.199, ep_step: 167 2023-05-30 01:43:06 - SimpleLog - INFO: - episode: 3376, ep_reward: -123.593, ep_step: 103 2023-05-30 01:43:06 - SimpleLog - INFO: - episode: 3377, ep_reward: -120.177, ep_step: 79 2023-05-30 01:43:06 - SimpleLog - INFO: - episode: 3378, ep_reward: -125.055, ep_step: 131 2023-05-30 01:43:07 - SimpleLog - INFO: - episode: 3379, ep_reward: -125.634, ep_step: 109 2023-05-30 01:43:07 - SimpleLog - INFO: - episode: 3380, ep_reward: -125.696, ep_step: 119 2023-05-30 01:43:07 - SimpleLog - INFO: - episode: 3381, ep_reward: -116.177, ep_step: 145 2023-05-30 01:43:08 - SimpleLog - INFO: - episode: 3382, ep_reward: -116.441, ep_step: 192 2023-05-30 01:43:08 - SimpleLog - INFO: - episode: 3383, ep_reward: -122.408, ep_step: 114 2023-05-30 01:43:08 - SimpleLog - INFO: - episode: 3384, ep_reward: -123.310, ep_step: 82 2023-05-30 01:43:08 - SimpleLog - INFO: - episode: 3385, ep_reward: -125.716, ep_step: 91 2023-05-30 01:43:09 - SimpleLog - INFO: - episode: 3386, ep_reward: -140.189, ep_step: 123 2023-05-30 01:43:09 - SimpleLog - INFO: - episode: 3387, ep_reward: -148.165, ep_step: 203 2023-05-30 01:43:09 - SimpleLog - INFO: - episode: 3388, ep_reward: -136.916, ep_step: 172 2023-05-30 01:43:10 - SimpleLog - INFO: - episode: 3389, ep_reward: -132.023, ep_step: 143 2023-05-30 01:43:12 - SimpleLog - INFO: - episode: 3390, ep_reward: -73.274, ep_step: 1000 2023-05-30 01:43:13 - SimpleLog - INFO: - episode: 3391, ep_reward: -143.816, ep_step: 476 2023-05-30 01:43:14 - SimpleLog - INFO: - episode: 3392, ep_reward: -111.992, ep_step: 390 2023-05-30 01:43:15 - SimpleLog - INFO: - episode: 3393, ep_reward: -99.504, ep_step: 283 2023-05-30 01:43:15 - SimpleLog - INFO: - episode: 3394, ep_reward: -95.454, ep_step: 244 2023-05-30 01:43:17 - SimpleLog - INFO: - episode: 3395, ep_reward: -85.696, ep_step: 1000 2023-05-30 01:43:20 - SimpleLog - INFO: - episode: 3396, ep_reward: -82.923, ep_step: 1000 2023-05-30 01:43:22 - SimpleLog - INFO: - episode: 3397, ep_reward: -100.290, ep_step: 1000 2023-05-30 01:43:24 - SimpleLog - INFO: - episode: 3398, ep_reward: -72.962, ep_step: 1000 2023-05-30 01:43:26 - SimpleLog - INFO: - episode: 3399, ep_reward: -102.460, ep_step: 1000 2023-05-30 01:43:29 - SimpleLog - INFO: - episode: 3400, ep_reward: -87.713, ep_step: 1000 2023-05-30 01:43:29 - SimpleLog - INFO: - episode: 3401, ep_reward: -95.420, ep_step: 227 2023-05-30 01:43:32 - SimpleLog - INFO: - episode: 3402, ep_reward: -79.096, ep_step: 1000 2023-05-30 01:43:32 - SimpleLog - INFO: - episode: 3403, ep_reward: -93.887, ep_step: 208 2023-05-30 01:43:33 - SimpleLog - INFO: - episode: 3404, ep_reward: -85.639, ep_step: 308 2023-05-30 01:43:33 - SimpleLog - INFO: - episode: 3405, ep_reward: -92.929, ep_step: 144 2023-05-30 01:43:34 - SimpleLog - INFO: - episode: 3406, ep_reward: -94.542, ep_step: 243 2023-05-30 01:43:36 - SimpleLog - INFO: - episode: 3407, ep_reward: -50.388, ep_step: 1000 2023-05-30 01:43:38 - SimpleLog - INFO: - episode: 3408, ep_reward: -83.352, ep_step: 1000 2023-05-30 01:43:39 - SimpleLog - INFO: - episode: 3409, ep_reward: -88.383, ep_step: 182 2023-05-30 01:43:41 - SimpleLog - INFO: - episode: 3410, ep_reward: -83.726, ep_step: 1000 2023-05-30 01:43:41 - SimpleLog - INFO: - episode: 3411, ep_reward: -87.977, ep_step: 207 2023-05-30 01:43:42 - SimpleLog - INFO: - episode: 3412, ep_reward: -140.055, ep_step: 263 2023-05-30 01:43:44 - SimpleLog - INFO: - episode: 3413, ep_reward: -44.454, ep_step: 1000 2023-05-30 01:43:46 - SimpleLog - INFO: - episode: 3414, ep_reward: -44.782, ep_step: 1000 2023-05-30 01:43:47 - SimpleLog - INFO: - episode: 3415, ep_reward: -105.945, ep_step: 331 2023-05-30 01:43:49 - SimpleLog - INFO: - episode: 3416, ep_reward: -137.912, ep_step: 659 2023-05-30 01:43:51 - SimpleLog - INFO: - episode: 3417, ep_reward: -43.526, ep_step: 1000 2023-05-30 01:43:52 - SimpleLog - INFO: - episode: 3418, ep_reward: -144.604, ep_step: 455 2023-05-30 01:43:54 - SimpleLog - INFO: - episode: 3419, ep_reward: -9.816, ep_step: 1000 2023-05-30 01:43:57 - SimpleLog - INFO: - episode: 3420, ep_reward: 9.335, ep_step: 1000 2023-05-30 01:43:58 - SimpleLog - INFO: - episode: 3421, ep_reward: -137.319, ep_step: 494 2023-05-30 01:43:59 - SimpleLog - INFO: - episode: 3422, ep_reward: -89.934, ep_step: 821 2023-05-30 01:44:01 - SimpleLog - INFO: - episode: 3423, ep_reward: -88.448, ep_step: 777 2023-05-30 01:44:03 - SimpleLog - INFO: - episode: 3424, ep_reward: -85.767, ep_step: 743 2023-05-30 01:44:05 - SimpleLog - INFO: - episode: 3425, ep_reward: 29.435, ep_step: 1000 2023-05-30 01:44:07 - SimpleLog - INFO: - episode: 3426, ep_reward: 10.434, ep_step: 1000 2023-05-30 01:44:10 - SimpleLog - INFO: - episode: 3427, ep_reward: 36.923, ep_step: 1000 2023-05-30 01:44:12 - SimpleLog - INFO: - episode: 3428, ep_reward: -100.109, ep_step: 885 2023-05-30 01:44:13 - SimpleLog - INFO: - episode: 3429, ep_reward: -91.141, ep_step: 774 2023-05-30 01:44:16 - SimpleLog - INFO: - episode: 3430, ep_reward: 78.561, ep_step: 1000 2023-05-30 01:44:18 - SimpleLog - INFO: - episode: 3431, ep_reward: 7.097, ep_step: 1000 2023-05-30 01:44:20 - SimpleLog - INFO: - episode: 3432, ep_reward: 95.312, ep_step: 1000 2023-05-30 01:44:22 - SimpleLog - INFO: - episode: 3433, ep_reward: -75.188, ep_step: 568 2023-05-30 01:44:24 - SimpleLog - INFO: - episode: 3434, ep_reward: 84.136, ep_step: 1000 2023-05-30 01:44:25 - SimpleLog - INFO: - episode: 3435, ep_reward: -78.897, ep_step: 688 2023-05-30 01:44:27 - SimpleLog - INFO: - episode: 3436, ep_reward: -73.975, ep_step: 617 2023-05-30 01:44:29 - SimpleLog - INFO: - episode: 3437, ep_reward: 84.080, ep_step: 1000 2023-05-30 01:44:30 - SimpleLog - INFO: - episode: 3438, ep_reward: -64.522, ep_step: 549 2023-05-30 01:44:32 - SimpleLog - INFO: - episode: 3439, ep_reward: -67.334, ep_step: 553 2023-05-30 01:44:33 - SimpleLog - INFO: - episode: 3440, ep_reward: -90.806, ep_step: 792 2023-05-30 01:44:36 - SimpleLog - INFO: - episode: 3441, ep_reward: 93.661, ep_step: 1000 2023-05-30 01:44:38 - SimpleLog - INFO: - episode: 3442, ep_reward: 68.026, ep_step: 1000 2023-05-30 01:44:39 - SimpleLog - INFO: - episode: 3443, ep_reward: -122.823, ep_step: 632 2023-05-30 01:44:41 - SimpleLog - INFO: - episode: 3444, ep_reward: -7.384, ep_step: 963 2023-05-30 01:44:43 - SimpleLog - INFO: - episode: 3445, ep_reward: -73.525, ep_step: 655 2023-05-30 01:44:45 - SimpleLog - INFO: - episode: 3446, ep_reward: 16.110, ep_step: 1000 2023-05-30 01:44:47 - SimpleLog - INFO: - episode: 3447, ep_reward: -78.079, ep_step: 871 2023-05-30 01:44:47 - SimpleLog - INFO: - episode: 3448, ep_reward: -132.347, ep_step: 175 2023-05-30 01:44:49 - SimpleLog - INFO: - episode: 3449, ep_reward: -68.862, ep_step: 795 2023-05-30 01:44:51 - SimpleLog - INFO: - episode: 3450, ep_reward: -59.158, ep_step: 701 2023-05-30 01:44:51 - SimpleLog - INFO: - episode: 3451, ep_reward: -142.515, ep_step: 216 2023-05-30 01:44:53 - SimpleLog - INFO: - episode: 3452, ep_reward: -54.997, ep_step: 637 2023-05-30 01:44:53 - SimpleLog - INFO: - episode: 3453, ep_reward: -137.351, ep_step: 123 2023-05-30 01:44:55 - SimpleLog - INFO: - episode: 3454, ep_reward: -65.571, ep_step: 833 2023-05-30 01:44:57 - SimpleLog - INFO: - episode: 3455, ep_reward: -62.456, ep_step: 681 2023-05-30 01:44:57 - SimpleLog - INFO: - episode: 3456, ep_reward: -98.957, ep_step: 219 2023-05-30 01:44:59 - SimpleLog - INFO: - episode: 3457, ep_reward: -44.640, ep_step: 730 2023-05-30 01:45:00 - SimpleLog - INFO: - episode: 3458, ep_reward: -84.796, ep_step: 514 2023-05-30 01:45:00 - SimpleLog - INFO: - episode: 3459, ep_reward: -119.676, ep_step: 62 2023-05-30 01:45:00 - SimpleLog - INFO: - episode: 3460, ep_reward: -117.285, ep_step: 55 2023-05-30 01:45:02 - SimpleLog - INFO: - episode: 3461, ep_reward: -49.900, ep_step: 905 2023-05-30 01:45:03 - SimpleLog - INFO: - episode: 3462, ep_reward: -65.397, ep_step: 562 2023-05-30 01:45:05 - SimpleLog - INFO: - episode: 3463, ep_reward: -88.035, ep_step: 773 2023-05-30 01:45:07 - SimpleLog - INFO: - episode: 3464, ep_reward: 53.648, ep_step: 1000 2023-05-30 01:45:10 - SimpleLog - INFO: - episode: 3465, ep_reward: 44.151, ep_step: 1000 2023-05-30 01:45:11 - SimpleLog - INFO: - episode: 3466, ep_reward: -80.794, ep_step: 802 2023-05-30 01:45:14 - SimpleLog - INFO: - episode: 3467, ep_reward: -62.601, ep_step: 967 2023-05-30 01:45:14 - SimpleLog - INFO: - episode: 3468, ep_reward: -123.221, ep_step: 89 2023-05-30 01:45:15 - SimpleLog - INFO: - episode: 3469, ep_reward: -107.519, ep_step: 450 2023-05-30 01:45:15 - SimpleLog - INFO: - episode: 3470, ep_reward: -124.511, ep_step: 250 2023-05-30 01:45:18 - SimpleLog - INFO: - episode: 3471, ep_reward: 36.054, ep_step: 1000 2023-05-30 01:45:18 - SimpleLog - INFO: - episode: 3472, ep_reward: -118.030, ep_step: 255 2023-05-30 01:45:20 - SimpleLog - INFO: - episode: 3473, ep_reward: 9.315, ep_step: 1000 2023-05-30 01:45:23 - SimpleLog - INFO: - episode: 3474, ep_reward: 44.785, ep_step: 1000 2023-05-30 01:45:24 - SimpleLog - INFO: - episode: 3475, ep_reward: -131.100, ep_step: 460 2023-05-30 01:45:26 - SimpleLog - INFO: - episode: 3476, ep_reward: -44.158, ep_step: 1000 2023-05-30 01:45:28 - SimpleLog - INFO: - episode: 3477, ep_reward: -4.367, ep_step: 1000 2023-05-30 01:45:30 - SimpleLog - INFO: - episode: 3478, ep_reward: 20.001, ep_step: 1000 2023-05-30 01:45:32 - SimpleLog - INFO: - episode: 3479, ep_reward: -80.735, ep_step: 817 2023-05-30 01:45:34 - SimpleLog - INFO: - episode: 3480, ep_reward: -69.854, ep_step: 632 2023-05-30 01:45:36 - SimpleLog - INFO: - episode: 3481, ep_reward: -8.393, ep_step: 1000 2023-05-30 01:45:38 - SimpleLog - INFO: - episode: 3482, ep_reward: 46.659, ep_step: 1000 2023-05-30 01:45:42 - SimpleLog - INFO: - update_step: 2520000, online_eval_reward: -68.368 2023-05-30 01:45:42 - SimpleLog - INFO: - episode: 3483, ep_reward: -78.688, ep_step: 756 2023-05-30 01:45:44 - SimpleLog - INFO: - episode: 3484, ep_reward: 19.524, ep_step: 1000 2023-05-30 01:45:46 - SimpleLog - INFO: - episode: 3485, ep_reward: 20.505, ep_step: 1000 2023-05-30 01:45:48 - SimpleLog - INFO: - episode: 3486, ep_reward: -65.625, ep_step: 866 2023-05-30 01:45:50 - SimpleLog - INFO: - episode: 3487, ep_reward: 24.429, ep_step: 1000 2023-05-30 01:45:52 - SimpleLog - INFO: - episode: 3488, ep_reward: -68.326, ep_step: 890 2023-05-30 01:45:54 - SimpleLog - INFO: - episode: 3489, ep_reward: -60.811, ep_step: 576 2023-05-30 01:45:55 - SimpleLog - INFO: - episode: 3490, ep_reward: -56.400, ep_step: 756 2023-05-30 01:45:56 - SimpleLog - INFO: - episode: 3491, ep_reward: -91.136, ep_step: 351 2023-05-30 01:45:58 - SimpleLog - INFO: - episode: 3492, ep_reward: -70.212, ep_step: 783 2023-05-30 01:46:00 - SimpleLog - INFO: - episode: 3493, ep_reward: -57.601, ep_step: 714 2023-05-30 01:46:02 - SimpleLog - INFO: - episode: 3494, ep_reward: -69.386, ep_step: 885 2023-05-30 01:46:04 - SimpleLog - INFO: - episode: 3495, ep_reward: -65.290, ep_step: 830 2023-05-30 01:46:06 - SimpleLog - INFO: - episode: 3496, ep_reward: 22.847, ep_step: 1000 2023-05-30 01:46:08 - SimpleLog - INFO: - episode: 3497, ep_reward: -76.932, ep_step: 997 2023-05-30 01:46:10 - SimpleLog - INFO: - episode: 3498, ep_reward: 2.686, ep_step: 1000 2023-05-30 01:46:13 - SimpleLog - INFO: - episode: 3499, ep_reward: -20.478, ep_step: 1000 2023-05-30 01:46:15 - SimpleLog - INFO: - episode: 3500, ep_reward: 15.359, ep_step: 1000 2023-05-30 01:46:17 - SimpleLog - INFO: - episode: 3501, ep_reward: 23.850, ep_step: 1000 2023-05-30 01:46:19 - SimpleLog - INFO: - episode: 3502, ep_reward: 19.467, ep_step: 1000 2023-05-30 01:46:20 - SimpleLog - INFO: - episode: 3503, ep_reward: -108.419, ep_step: 315 2023-05-30 01:46:22 - SimpleLog - INFO: - episode: 3504, ep_reward: 18.682, ep_step: 1000 2023-05-30 01:46:23 - SimpleLog - INFO: - episode: 3505, ep_reward: -110.140, ep_step: 289 2023-05-30 01:46:25 - SimpleLog - INFO: - episode: 3506, ep_reward: 19.532, ep_step: 1000 2023-05-30 01:46:27 - SimpleLog - INFO: - episode: 3507, ep_reward: 45.067, ep_step: 1000 2023-05-30 01:46:30 - SimpleLog - INFO: - episode: 3508, ep_reward: 18.328, ep_step: 1000 2023-05-30 01:46:32 - SimpleLog - INFO: - episode: 3509, ep_reward: 19.928, ep_step: 1000 2023-05-30 01:46:34 - SimpleLog - INFO: - episode: 3510, ep_reward: -54.753, ep_step: 977 2023-05-30 01:46:36 - SimpleLog - INFO: - episode: 3511, ep_reward: 19.603, ep_step: 1000 2023-05-30 01:46:39 - SimpleLog - INFO: - episode: 3512, ep_reward: 17.985, ep_step: 1000 2023-05-30 01:46:41 - SimpleLog - INFO: - episode: 3513, ep_reward: 15.156, ep_step: 1000 2023-05-30 01:46:43 - SimpleLog - INFO: - episode: 3514, ep_reward: 16.563, ep_step: 1000 2023-05-30 01:46:45 - SimpleLog - INFO: - episode: 3515, ep_reward: 3.125, ep_step: 1000 2023-05-30 01:46:46 - SimpleLog - INFO: - episode: 3516, ep_reward: -103.976, ep_step: 374 2023-05-30 01:46:47 - SimpleLog - INFO: - episode: 3517, ep_reward: -116.538, ep_step: 203 2023-05-30 01:46:49 - SimpleLog - INFO: - episode: 3518, ep_reward: 38.594, ep_step: 1000 2023-05-30 01:46:51 - SimpleLog - INFO: - episode: 3519, ep_reward: 21.789, ep_step: 1000 2023-05-30 01:46:53 - SimpleLog - INFO: - episode: 3520, ep_reward: 16.653, ep_step: 1000 2023-05-30 01:46:56 - SimpleLog - INFO: - episode: 3521, ep_reward: 5.841, ep_step: 1000 2023-05-30 01:46:58 - SimpleLog - INFO: - episode: 3522, ep_reward: 42.898, ep_step: 1000 2023-05-30 01:47:00 - SimpleLog - INFO: - episode: 3523, ep_reward: -79.609, ep_step: 948 2023-05-30 01:47:02 - SimpleLog - INFO: - episode: 3524, ep_reward: 4.557, ep_step: 1000 2023-05-30 01:47:05 - SimpleLog - INFO: - episode: 3525, ep_reward: 2.970, ep_step: 1000 2023-05-30 01:47:07 - SimpleLog - INFO: - episode: 3526, ep_reward: 11.461, ep_step: 1000 2023-05-30 01:47:09 - SimpleLog - INFO: - episode: 3527, ep_reward: 5.385, ep_step: 1000 2023-05-30 01:47:11 - SimpleLog - INFO: - episode: 3528, ep_reward: -20.679, ep_step: 1000 2023-05-30 01:47:14 - SimpleLog - INFO: - episode: 3529, ep_reward: 3.930, ep_step: 1000 2023-05-30 01:47:16 - SimpleLog - INFO: - episode: 3530, ep_reward: 12.352, ep_step: 1000 2023-05-30 01:47:18 - SimpleLog - INFO: - episode: 3531, ep_reward: 9.477, ep_step: 1000 2023-05-30 01:47:20 - SimpleLog - INFO: - episode: 3532, ep_reward: 11.067, ep_step: 1000 2023-05-30 01:47:23 - SimpleLog - INFO: - episode: 3533, ep_reward: 14.437, ep_step: 1000 2023-05-30 01:47:25 - SimpleLog - INFO: - episode: 3534, ep_reward: 13.476, ep_step: 1000 2023-05-30 01:47:26 - SimpleLog - INFO: - episode: 3535, ep_reward: -82.891, ep_step: 323 2023-05-30 01:47:28 - SimpleLog - INFO: - episode: 3536, ep_reward: 12.606, ep_step: 1000 2023-05-30 01:47:30 - SimpleLog - INFO: - episode: 3537, ep_reward: 14.271, ep_step: 1000 2023-05-30 01:47:32 - SimpleLog - INFO: - episode: 3538, ep_reward: -48.747, ep_step: 968 2023-05-30 01:47:33 - SimpleLog - INFO: - episode: 3539, ep_reward: -85.267, ep_step: 295 2023-05-30 01:47:35 - SimpleLog - INFO: - episode: 3540, ep_reward: 14.092, ep_step: 1000 2023-05-30 01:47:37 - SimpleLog - INFO: - episode: 3541, ep_reward: -54.338, ep_step: 797 2023-05-30 01:47:39 - SimpleLog - INFO: - episode: 3542, ep_reward: -60.043, ep_step: 679 2023-05-30 01:47:41 - SimpleLog - INFO: - episode: 3543, ep_reward: 11.443, ep_step: 1000 2023-05-30 01:47:43 - SimpleLog - INFO: - episode: 3544, ep_reward: -51.135, ep_step: 903 2023-05-30 01:47:45 - SimpleLog - INFO: - episode: 3545, ep_reward: -92.276, ep_step: 818 2023-05-30 01:47:47 - SimpleLog - INFO: - episode: 3546, ep_reward: -102.274, ep_step: 972 2023-05-30 01:47:49 - SimpleLog - INFO: - episode: 3547, ep_reward: 14.060, ep_step: 1000 2023-05-30 01:47:52 - SimpleLog - INFO: - episode: 3548, ep_reward: 19.386, ep_step: 1000 2023-05-30 01:47:54 - SimpleLog - INFO: - episode: 3549, ep_reward: 16.373, ep_step: 1000 2023-05-30 01:47:56 - SimpleLog - INFO: - episode: 3550, ep_reward: 12.003, ep_step: 1000 2023-05-30 01:47:58 - SimpleLog - INFO: - episode: 3551, ep_reward: 63.510, ep_step: 1000 2023-05-30 01:48:01 - SimpleLog - INFO: - episode: 3552, ep_reward: 12.781, ep_step: 1000 2023-05-30 01:48:03 - SimpleLog - INFO: - episode: 3553, ep_reward: 51.273, ep_step: 1000 2023-05-30 01:48:05 - SimpleLog - INFO: - episode: 3554, ep_reward: 48.349, ep_step: 1000 2023-05-30 01:48:07 - SimpleLog - INFO: - episode: 3555, ep_reward: 48.009, ep_step: 1000 2023-05-30 01:48:10 - SimpleLog - INFO: - episode: 3556, ep_reward: 16.260, ep_step: 1000 2023-05-30 01:48:12 - SimpleLog - INFO: - episode: 3557, ep_reward: 49.117, ep_step: 1000 2023-05-30 01:48:14 - SimpleLog - INFO: - episode: 3558, ep_reward: 18.448, ep_step: 1000 2023-05-30 01:48:16 - SimpleLog - INFO: - episode: 3559, ep_reward: -92.239, ep_step: 689 2023-05-30 01:48:18 - SimpleLog - INFO: - episode: 3560, ep_reward: 14.716, ep_step: 1000 2023-05-30 01:48:22 - SimpleLog - INFO: - update_step: 2590000, online_eval_reward: 23.631 2023-05-30 01:48:22 - SimpleLog - INFO: - episode: 3561, ep_reward: 23.407, ep_step: 1000 2023-05-30 01:48:24 - SimpleLog - INFO: - episode: 3562, ep_reward: 21.286, ep_step: 1000 2023-05-30 01:48:27 - SimpleLog - INFO: - episode: 3563, ep_reward: 16.119, ep_step: 1000 2023-05-30 01:48:29 - SimpleLog - INFO: - episode: 3564, ep_reward: 18.323, ep_step: 1000 2023-05-30 01:48:31 - SimpleLog - INFO: - episode: 3565, ep_reward: 62.899, ep_step: 1000 2023-05-30 01:48:33 - SimpleLog - INFO: - episode: 3566, ep_reward: 83.625, ep_step: 1000 2023-05-30 01:48:36 - SimpleLog - INFO: - episode: 3567, ep_reward: 29.553, ep_step: 1000 2023-05-30 01:48:38 - SimpleLog - INFO: - episode: 3568, ep_reward: 62.798, ep_step: 1000 2023-05-30 01:48:40 - SimpleLog - INFO: - episode: 3569, ep_reward: 18.947, ep_step: 1000 2023-05-30 01:48:42 - SimpleLog - INFO: - episode: 3570, ep_reward: -21.216, ep_step: 943 2023-05-30 01:48:44 - SimpleLog - INFO: - episode: 3571, ep_reward: -21.709, ep_step: 899 2023-05-30 01:48:46 - SimpleLog - INFO: - episode: 3572, ep_reward: 63.747, ep_step: 1000 2023-05-30 01:48:49 - SimpleLog - INFO: - episode: 3573, ep_reward: 90.944, ep_step: 1000 2023-05-30 01:48:49 - SimpleLog - INFO: - episode: 3574, ep_reward: -91.215, ep_step: 158 2023-05-30 01:48:51 - SimpleLog - INFO: - episode: 3575, ep_reward: -27.567, ep_step: 907 2023-05-30 01:48:53 - SimpleLog - INFO: - episode: 3576, ep_reward: -26.816, ep_step: 952 2023-05-30 01:48:55 - SimpleLog - INFO: - episode: 3577, ep_reward: 80.390, ep_step: 1000 2023-05-30 01:48:58 - SimpleLog - INFO: - episode: 3578, ep_reward: 86.003, ep_step: 1000 2023-05-30 01:49:00 - SimpleLog - INFO: - episode: 3579, ep_reward: 88.028, ep_step: 1000 2023-05-30 01:49:02 - SimpleLog - INFO: - episode: 3580, ep_reward: 20.013, ep_step: 1000 2023-05-30 01:49:04 - SimpleLog - INFO: - episode: 3581, ep_reward: 18.767, ep_step: 1000 2023-05-30 01:49:07 - SimpleLog - INFO: - episode: 3582, ep_reward: 18.885, ep_step: 1000 2023-05-30 01:49:09 - SimpleLog - INFO: - episode: 3583, ep_reward: 22.416, ep_step: 1000 2023-05-30 01:49:11 - SimpleLog - INFO: - episode: 3584, ep_reward: 50.913, ep_step: 1000 2023-05-30 01:49:13 - SimpleLog - INFO: - episode: 3585, ep_reward: 22.523, ep_step: 1000 2023-05-30 01:49:16 - SimpleLog - INFO: - episode: 3586, ep_reward: 18.736, ep_step: 1000 2023-05-30 01:49:18 - SimpleLog - INFO: - episode: 3587, ep_reward: 14.136, ep_step: 1000 2023-05-30 01:49:20 - SimpleLog - INFO: - episode: 3588, ep_reward: -22.680, ep_step: 1000 2023-05-30 01:49:23 - SimpleLog - INFO: - episode: 3589, ep_reward: 5.326, ep_step: 1000 2023-05-30 01:49:25 - SimpleLog - INFO: - episode: 3590, ep_reward: 56.760, ep_step: 1000 2023-05-30 01:49:27 - SimpleLog - INFO: - episode: 3591, ep_reward: 21.907, ep_step: 1000 2023-05-30 01:49:29 - SimpleLog - INFO: - episode: 3592, ep_reward: 64.384, ep_step: 1000 2023-05-30 01:49:32 - SimpleLog - INFO: - episode: 3593, ep_reward: 73.840, ep_step: 1000 2023-05-30 01:49:33 - SimpleLog - INFO: - episode: 3594, ep_reward: -75.696, ep_step: 613 2023-05-30 01:49:35 - SimpleLog - INFO: - episode: 3595, ep_reward: 79.555, ep_step: 1000 2023-05-30 01:49:37 - SimpleLog - INFO: - episode: 3596, ep_reward: 79.682, ep_step: 1000 2023-05-30 01:49:40 - SimpleLog - INFO: - episode: 3597, ep_reward: -14.064, ep_step: 1000 2023-05-30 01:49:42 - SimpleLog - INFO: - episode: 3598, ep_reward: 79.343, ep_step: 1000 2023-05-30 01:49:44 - SimpleLog - INFO: - episode: 3599, ep_reward: 2.440, ep_step: 1000 2023-05-30 01:49:46 - SimpleLog - INFO: - episode: 3600, ep_reward: 83.068, ep_step: 1000 2023-05-30 01:49:49 - SimpleLog - INFO: - episode: 3601, ep_reward: 31.787, ep_step: 1000 2023-05-30 01:49:51 - SimpleLog - INFO: - episode: 3602, ep_reward: 83.692, ep_step: 1000 2023-05-30 01:49:52 - SimpleLog - INFO: - episode: 3603, ep_reward: -94.569, ep_step: 310 2023-05-30 01:49:54 - SimpleLog - INFO: - episode: 3604, ep_reward: -29.055, ep_step: 886 2023-05-30 01:49:56 - SimpleLog - INFO: - episode: 3605, ep_reward: 88.219, ep_step: 1000 2023-05-30 01:49:58 - SimpleLog - INFO: - episode: 3606, ep_reward: 87.706, ep_step: 1000 2023-05-30 01:50:00 - SimpleLog - INFO: - episode: 3607, ep_reward: 83.431, ep_step: 1000 2023-05-30 01:50:02 - SimpleLog - INFO: - episode: 3608, ep_reward: 86.592, ep_step: 1000 2023-05-30 01:50:05 - SimpleLog - INFO: - episode: 3609, ep_reward: 76.827, ep_step: 1000 2023-05-30 01:50:07 - SimpleLog - INFO: - episode: 3610, ep_reward: 90.997, ep_step: 1000 2023-05-30 01:50:09 - SimpleLog - INFO: - episode: 3611, ep_reward: 87.238, ep_step: 1000 2023-05-30 01:50:11 - SimpleLog - INFO: - episode: 3612, ep_reward: 97.395, ep_step: 1000 2023-05-30 01:50:14 - SimpleLog - INFO: - episode: 3613, ep_reward: 84.610, ep_step: 1000 2023-05-30 01:50:16 - SimpleLog - INFO: - episode: 3614, ep_reward: 93.497, ep_step: 1000 2023-05-30 01:50:18 - SimpleLog - INFO: - episode: 3615, ep_reward: 88.692, ep_step: 1000 2023-05-30 01:50:20 - SimpleLog - INFO: - episode: 3616, ep_reward: 96.781, ep_step: 1000 2023-05-30 01:50:23 - SimpleLog - INFO: - episode: 3617, ep_reward: 87.984, ep_step: 1000 2023-05-30 01:50:25 - SimpleLog - INFO: - episode: 3618, ep_reward: 83.255, ep_step: 1000 2023-05-30 01:50:27 - SimpleLog - INFO: - episode: 3619, ep_reward: 79.406, ep_step: 1000 2023-05-30 01:50:29 - SimpleLog - INFO: - episode: 3620, ep_reward: -94.632, ep_step: 776 2023-05-30 01:50:31 - SimpleLog - INFO: - episode: 3621, ep_reward: 94.262, ep_step: 1000 2023-05-30 01:50:33 - SimpleLog - INFO: - episode: 3622, ep_reward: 87.263, ep_step: 1000 2023-05-30 01:50:36 - SimpleLog - INFO: - episode: 3623, ep_reward: 101.049, ep_step: 1000 2023-05-30 01:50:38 - SimpleLog - INFO: - episode: 3624, ep_reward: 91.769, ep_step: 1000 2023-05-30 01:50:40 - SimpleLog - INFO: - episode: 3625, ep_reward: 86.123, ep_step: 1000 2023-05-30 01:50:42 - SimpleLog - INFO: - episode: 3626, ep_reward: 97.516, ep_step: 1000 2023-05-30 01:50:45 - SimpleLog - INFO: - episode: 3627, ep_reward: 115.046, ep_step: 1000 2023-05-30 01:50:47 - SimpleLog - INFO: - episode: 3628, ep_reward: 101.420, ep_step: 1000 2023-05-30 01:50:49 - SimpleLog - INFO: - episode: 3629, ep_reward: 73.905, ep_step: 1000 2023-05-30 01:50:51 - SimpleLog - INFO: - episode: 3630, ep_reward: 4.313, ep_step: 1000 2023-05-30 01:50:54 - SimpleLog - INFO: - episode: 3631, ep_reward: 80.946, ep_step: 1000 2023-05-30 01:50:55 - SimpleLog - INFO: - episode: 3632, ep_reward: -17.864, ep_step: 790 2023-05-30 01:50:58 - SimpleLog - INFO: - episode: 3633, ep_reward: 114.366, ep_step: 1000 2023-05-30 01:51:01 - SimpleLog - INFO: - update_step: 2660000, online_eval_reward: 88.329 2023-05-30 01:51:02 - SimpleLog - INFO: - episode: 3634, ep_reward: 12.501, ep_step: 951 2023-05-30 01:51:04 - SimpleLog - INFO: - episode: 3635, ep_reward: 96.161, ep_step: 1000 2023-05-30 01:51:05 - SimpleLog - INFO: - episode: 3636, ep_reward: -86.517, ep_step: 558 2023-05-30 01:51:07 - SimpleLog - INFO: - episode: 3637, ep_reward: 105.439, ep_step: 1000 2023-05-30 01:51:10 - SimpleLog - INFO: - episode: 3638, ep_reward: 114.882, ep_step: 1000 2023-05-30 01:51:12 - SimpleLog - INFO: - episode: 3639, ep_reward: 111.793, ep_step: 1000 2023-05-30 01:51:14 - SimpleLog - INFO: - episode: 3640, ep_reward: 104.288, ep_step: 1000 2023-05-30 01:51:16 - SimpleLog - INFO: - episode: 3641, ep_reward: 88.245, ep_step: 1000 2023-05-30 01:51:19 - SimpleLog - INFO: - episode: 3642, ep_reward: 94.524, ep_step: 1000 2023-05-30 01:51:21 - SimpleLog - INFO: - episode: 3643, ep_reward: 84.073, ep_step: 1000 2023-05-30 01:51:23 - SimpleLog - INFO: - episode: 3644, ep_reward: 85.519, ep_step: 1000 2023-05-30 01:51:24 - SimpleLog - INFO: - episode: 3645, ep_reward: -65.834, ep_step: 544 2023-05-30 01:51:27 - SimpleLog - INFO: - episode: 3646, ep_reward: 102.553, ep_step: 1000 2023-05-30 01:51:29 - SimpleLog - INFO: - episode: 3647, ep_reward: 111.557, ep_step: 1000 2023-05-30 01:51:31 - SimpleLog - INFO: - episode: 3648, ep_reward: -40.147, ep_step: 750 2023-05-30 01:51:33 - SimpleLog - INFO: - episode: 3649, ep_reward: 123.010, ep_step: 1000 2023-05-30 01:51:35 - SimpleLog - INFO: - episode: 3650, ep_reward: 115.465, ep_step: 1000 2023-05-30 01:51:37 - SimpleLog - INFO: - episode: 3651, ep_reward: 104.116, ep_step: 1000 2023-05-30 01:51:38 - SimpleLog - INFO: - episode: 3652, ep_reward: -93.916, ep_step: 209 2023-05-30 01:51:40 - SimpleLog - INFO: - episode: 3653, ep_reward: 94.094, ep_step: 1000 2023-05-30 01:51:42 - SimpleLog - INFO: - episode: 3654, ep_reward: 76.000, ep_step: 1000 2023-05-30 01:51:45 - SimpleLog - INFO: - episode: 3655, ep_reward: 59.838, ep_step: 1000 2023-05-30 01:51:47 - SimpleLog - INFO: - episode: 3656, ep_reward: 64.427, ep_step: 1000 2023-05-30 01:51:49 - SimpleLog - INFO: - episode: 3657, ep_reward: 82.047, ep_step: 1000 2023-05-30 01:51:51 - SimpleLog - INFO: - episode: 3658, ep_reward: 83.438, ep_step: 1000 2023-05-30 01:51:53 - SimpleLog - INFO: - episode: 3659, ep_reward: -78.185, ep_step: 596 2023-05-30 01:51:54 - SimpleLog - INFO: - episode: 3660, ep_reward: -86.293, ep_step: 699 2023-05-30 01:51:56 - SimpleLog - INFO: - episode: 3661, ep_reward: 85.743, ep_step: 1000 2023-05-30 01:51:59 - SimpleLog - INFO: - episode: 3662, ep_reward: 78.708, ep_step: 1000 2023-05-30 01:52:01 - SimpleLog - INFO: - episode: 3663, ep_reward: 20.598, ep_step: 1000 2023-05-30 01:52:03 - SimpleLog - INFO: - episode: 3664, ep_reward: 79.272, ep_step: 1000 2023-05-30 01:52:05 - SimpleLog - INFO: - episode: 3665, ep_reward: 76.974, ep_step: 1000 2023-05-30 01:52:08 - SimpleLog - INFO: - episode: 3666, ep_reward: 92.693, ep_step: 1000 2023-05-30 01:52:10 - SimpleLog - INFO: - episode: 3667, ep_reward: -20.128, ep_step: 826 2023-05-30 01:52:10 - SimpleLog - INFO: - episode: 3668, ep_reward: -110.062, ep_step: 52 2023-05-30 01:52:12 - SimpleLog - INFO: - episode: 3669, ep_reward: 106.850, ep_step: 1000 2023-05-30 01:52:13 - SimpleLog - INFO: - episode: 3670, ep_reward: -33.667, ep_step: 637 2023-05-30 01:52:16 - SimpleLog - INFO: - episode: 3671, ep_reward: 75.933, ep_step: 1000 2023-05-30 01:52:18 - SimpleLog - INFO: - episode: 3672, ep_reward: -18.109, ep_step: 872 2023-05-30 01:52:18 - SimpleLog - INFO: - episode: 3673, ep_reward: -101.689, ep_step: 81 2023-05-30 01:52:20 - SimpleLog - INFO: - episode: 3674, ep_reward: -5.279, ep_step: 903 2023-05-30 01:52:22 - SimpleLog - INFO: - episode: 3675, ep_reward: 81.535, ep_step: 1000 2023-05-30 01:52:24 - SimpleLog - INFO: - episode: 3676, ep_reward: 86.464, ep_step: 1000 2023-05-30 01:52:27 - SimpleLog - INFO: - episode: 3677, ep_reward: 103.730, ep_step: 1000 2023-05-30 01:52:29 - SimpleLog - INFO: - episode: 3678, ep_reward: 4.079, ep_step: 1000 2023-05-30 01:52:31 - SimpleLog - INFO: - episode: 3679, ep_reward: 116.558, ep_step: 1000 2023-05-30 01:52:33 - SimpleLog - INFO: - episode: 3680, ep_reward: 102.381, ep_step: 1000 2023-05-30 01:52:36 - SimpleLog - INFO: - episode: 3681, ep_reward: 112.843, ep_step: 1000 2023-05-30 01:52:38 - SimpleLog - INFO: - episode: 3682, ep_reward: 42.151, ep_step: 1000 2023-05-30 01:52:40 - SimpleLog - INFO: - episode: 3683, ep_reward: 103.090, ep_step: 1000 2023-05-30 01:52:43 - SimpleLog - INFO: - episode: 3684, ep_reward: 104.489, ep_step: 1000 2023-05-30 01:52:45 - SimpleLog - INFO: - episode: 3685, ep_reward: 63.051, ep_step: 1000 2023-05-30 01:52:47 - SimpleLog - INFO: - episode: 3686, ep_reward: 77.257, ep_step: 1000 2023-05-30 01:52:49 - SimpleLog - INFO: - episode: 3687, ep_reward: 9.980, ep_step: 1000 2023-05-30 01:52:52 - SimpleLog - INFO: - episode: 3688, ep_reward: -25.217, ep_step: 1000 2023-05-30 01:52:53 - SimpleLog - INFO: - episode: 3689, ep_reward: -101.379, ep_step: 738 2023-05-30 01:52:55 - SimpleLog - INFO: - episode: 3690, ep_reward: -7.336, ep_step: 1000 2023-05-30 01:52:58 - SimpleLog - INFO: - episode: 3691, ep_reward: -28.082, ep_step: 1000 2023-05-30 01:53:00 - SimpleLog - INFO: - episode: 3692, ep_reward: 51.293, ep_step: 1000 2023-05-30 01:53:02 - SimpleLog - INFO: - episode: 3693, ep_reward: 114.974, ep_step: 1000 2023-05-30 01:53:05 - SimpleLog - INFO: - episode: 3694, ep_reward: 5.518, ep_step: 1000 2023-05-30 01:53:05 - SimpleLog - INFO: - episode: 3695, ep_reward: -88.824, ep_step: 165 2023-05-30 01:53:07 - SimpleLog - INFO: - episode: 3696, ep_reward: 133.979, ep_step: 1000 2023-05-30 01:53:08 - SimpleLog - INFO: - episode: 3697, ep_reward: -33.414, ep_step: 511 2023-05-30 01:53:11 - SimpleLog - INFO: - episode: 3698, ep_reward: 143.008, ep_step: 1000 2023-05-30 01:53:12 - SimpleLog - INFO: - episode: 3699, ep_reward: -68.316, ep_step: 486 2023-05-30 01:53:14 - SimpleLog - INFO: - episode: 3700, ep_reward: 98.222, ep_step: 1000 2023-05-30 01:53:16 - SimpleLog - INFO: - episode: 3701, ep_reward: 115.236, ep_step: 1000 2023-05-30 01:53:17 - SimpleLog - INFO: - episode: 3702, ep_reward: -82.823, ep_step: 209 2023-05-30 01:53:19 - SimpleLog - INFO: - episode: 3703, ep_reward: 100.040, ep_step: 1000 2023-05-30 01:53:20 - SimpleLog - INFO: - episode: 3704, ep_reward: -63.771, ep_step: 431 2023-05-30 01:53:21 - SimpleLog - INFO: - episode: 3705, ep_reward: -48.204, ep_step: 452 2023-05-30 01:53:23 - SimpleLog - INFO: - episode: 3706, ep_reward: 137.480, ep_step: 1000 2023-05-30 01:53:25 - SimpleLog - INFO: - episode: 3707, ep_reward: 119.636, ep_step: 1000 2023-05-30 01:53:26 - SimpleLog - INFO: - episode: 3708, ep_reward: -65.522, ep_step: 425 2023-05-30 01:53:27 - SimpleLog - INFO: - episode: 3709, ep_reward: -93.319, ep_step: 200 2023-05-30 01:53:28 - SimpleLog - INFO: - episode: 3710, ep_reward: -27.909, ep_step: 700 2023-05-30 01:53:29 - SimpleLog - INFO: - episode: 3711, ep_reward: -59.455, ep_step: 425 2023-05-30 01:53:32 - SimpleLog - INFO: - episode: 3712, ep_reward: 111.063, ep_step: 1000 2023-05-30 01:53:34 - SimpleLog - INFO: - episode: 3713, ep_reward: 3.147, ep_step: 908 2023-05-30 01:53:36 - SimpleLog - INFO: - episode: 3714, ep_reward: 80.122, ep_step: 1000 2023-05-30 01:53:38 - SimpleLog - INFO: - episode: 3715, ep_reward: 71.092, ep_step: 1000 2023-05-30 01:53:40 - SimpleLog - INFO: - update_step: 2730000, online_eval_reward: 90.213 2023-05-30 01:53:42 - SimpleLog - INFO: - episode: 3716, ep_reward: -49.512, ep_step: 704 2023-05-30 01:53:44 - SimpleLog - INFO: - episode: 3717, ep_reward: 77.533, ep_step: 1000 2023-05-30 01:53:44 - SimpleLog - INFO: - episode: 3718, ep_reward: -113.512, ep_step: 85 2023-05-30 01:53:46 - SimpleLog - INFO: - episode: 3719, ep_reward: 75.689, ep_step: 1000 2023-05-30 01:53:48 - SimpleLog - INFO: - episode: 3720, ep_reward: -36.502, ep_step: 577 2023-05-30 01:53:49 - SimpleLog - INFO: - episode: 3721, ep_reward: -31.552, ep_step: 613 2023-05-30 01:53:50 - SimpleLog - INFO: - episode: 3722, ep_reward: -38.756, ep_step: 614 2023-05-30 01:53:52 - SimpleLog - INFO: - episode: 3723, ep_reward: -39.174, ep_step: 641 2023-05-30 01:53:54 - SimpleLog - INFO: - episode: 3724, ep_reward: 104.640, ep_step: 1000 2023-05-30 01:53:56 - SimpleLog - INFO: - episode: 3725, ep_reward: -41.280, ep_step: 984 2023-05-30 01:53:58 - SimpleLog - INFO: - episode: 3726, ep_reward: -32.661, ep_step: 610 2023-05-30 01:54:00 - SimpleLog - INFO: - episode: 3727, ep_reward: 109.692, ep_step: 1000 2023-05-30 01:54:02 - SimpleLog - INFO: - episode: 3728, ep_reward: 106.690, ep_step: 1000 2023-05-30 01:54:04 - SimpleLog - INFO: - episode: 3729, ep_reward: 99.823, ep_step: 1000 2023-05-30 01:54:07 - SimpleLog - INFO: - episode: 3730, ep_reward: 99.668, ep_step: 1000 2023-05-30 01:54:09 - SimpleLog - INFO: - episode: 3731, ep_reward: 104.735, ep_step: 1000 2023-05-30 01:54:11 - SimpleLog - INFO: - episode: 3732, ep_reward: 109.396, ep_step: 1000 2023-05-30 01:54:13 - SimpleLog - INFO: - episode: 3733, ep_reward: 96.765, ep_step: 1000 2023-05-30 01:54:16 - SimpleLog - INFO: - episode: 3734, ep_reward: 99.178, ep_step: 1000 2023-05-30 01:54:18 - SimpleLog - INFO: - episode: 3735, ep_reward: 80.280, ep_step: 1000 2023-05-30 01:54:20 - SimpleLog - INFO: - episode: 3736, ep_reward: 94.772, ep_step: 1000 2023-05-30 01:54:22 - SimpleLog - INFO: - episode: 3737, ep_reward: 83.462, ep_step: 1000 2023-05-30 01:54:25 - SimpleLog - INFO: - episode: 3738, ep_reward: 112.834, ep_step: 1000 2023-05-30 01:54:27 - SimpleLog - INFO: - episode: 3739, ep_reward: 83.938, ep_step: 1000 2023-05-30 01:54:29 - SimpleLog - INFO: - episode: 3740, ep_reward: 112.959, ep_step: 1000 2023-05-30 01:54:32 - SimpleLog - INFO: - episode: 3741, ep_reward: 86.440, ep_step: 1000 2023-05-30 01:54:33 - SimpleLog - INFO: - episode: 3742, ep_reward: -29.493, ep_step: 773 2023-05-30 01:54:36 - SimpleLog - INFO: - episode: 3743, ep_reward: 9.671, ep_step: 1000 2023-05-30 01:54:38 - SimpleLog - INFO: - episode: 3744, ep_reward: 84.471, ep_step: 1000 2023-05-30 01:54:40 - SimpleLog - INFO: - episode: 3745, ep_reward: 102.144, ep_step: 1000 2023-05-30 01:54:42 - SimpleLog - INFO: - episode: 3746, ep_reward: -33.238, ep_step: 876 2023-05-30 01:54:44 - SimpleLog - INFO: - episode: 3747, ep_reward: 57.579, ep_step: 1000 2023-05-30 01:54:46 - SimpleLog - INFO: - episode: 3748, ep_reward: -26.815, ep_step: 849 2023-05-30 01:54:48 - SimpleLog - INFO: - episode: 3749, ep_reward: 85.581, ep_step: 1000 2023-05-30 01:54:51 - SimpleLog - INFO: - episode: 3750, ep_reward: 96.264, ep_step: 1000 2023-05-30 01:54:53 - SimpleLog - INFO: - episode: 3751, ep_reward: 46.469, ep_step: 1000 2023-05-30 01:54:55 - SimpleLog - INFO: - episode: 3752, ep_reward: -30.173, ep_step: 1000 2023-05-30 01:54:57 - SimpleLog - INFO: - episode: 3753, ep_reward: -34.563, ep_step: 759 2023-05-30 01:54:59 - SimpleLog - INFO: - episode: 3754, ep_reward: 125.718, ep_step: 1000 2023-05-30 01:55:02 - SimpleLog - INFO: - episode: 3755, ep_reward: 99.148, ep_step: 1000 2023-05-30 01:55:04 - SimpleLog - INFO: - episode: 3756, ep_reward: -19.513, ep_step: 891 2023-05-30 01:55:06 - SimpleLog - INFO: - episode: 3757, ep_reward: -22.029, ep_step: 1000 2023-05-30 01:55:08 - SimpleLog - INFO: - episode: 3758, ep_reward: 49.741, ep_step: 1000 2023-05-30 01:55:10 - SimpleLog - INFO: - episode: 3759, ep_reward: -25.213, ep_step: 740 2023-05-30 01:55:12 - SimpleLog - INFO: - episode: 3760, ep_reward: -12.581, ep_step: 1000 2023-05-30 01:55:12 - SimpleLog - INFO: - episode: 3761, ep_reward: -73.098, ep_step: 243 2023-05-30 01:55:14 - SimpleLog - INFO: - episode: 3762, ep_reward: -7.161, ep_step: 819 2023-05-30 01:55:17 - SimpleLog - INFO: - episode: 3763, ep_reward: 84.030, ep_step: 1000 2023-05-30 01:55:19 - SimpleLog - INFO: - episode: 3764, ep_reward: 82.147, ep_step: 1000 2023-05-30 01:55:21 - SimpleLog - INFO: - episode: 3765, ep_reward: -6.639, ep_step: 946 2023-05-30 01:55:23 - SimpleLog - INFO: - episode: 3766, ep_reward: 112.439, ep_step: 1000 2023-05-30 01:55:25 - SimpleLog - INFO: - episode: 3767, ep_reward: 109.234, ep_step: 1000 2023-05-30 01:55:28 - SimpleLog - INFO: - episode: 3768, ep_reward: 114.589, ep_step: 1000 2023-05-30 01:55:30 - SimpleLog - INFO: - episode: 3769, ep_reward: 90.805, ep_step: 1000 2023-05-30 01:55:32 - SimpleLog - INFO: - episode: 3770, ep_reward: 115.647, ep_step: 1000 2023-05-30 01:55:34 - SimpleLog - INFO: - episode: 3771, ep_reward: 90.050, ep_step: 1000 2023-05-30 01:55:37 - SimpleLog - INFO: - episode: 3772, ep_reward: 101.411, ep_step: 1000 2023-05-30 01:55:39 - SimpleLog - INFO: - episode: 3773, ep_reward: 13.015, ep_step: 1000 2023-05-30 01:55:41 - SimpleLog - INFO: - episode: 3774, ep_reward: 111.206, ep_step: 1000 2023-05-30 01:55:43 - SimpleLog - INFO: - episode: 3775, ep_reward: -75.872, ep_step: 802 2023-05-30 01:55:45 - SimpleLog - INFO: - episode: 3776, ep_reward: 74.649, ep_step: 1000 2023-05-30 01:55:48 - SimpleLog - INFO: - episode: 3777, ep_reward: 110.455, ep_step: 1000 2023-05-30 01:55:50 - SimpleLog - INFO: - episode: 3778, ep_reward: 90.827, ep_step: 1000 2023-05-30 01:55:52 - SimpleLog - INFO: - episode: 3779, ep_reward: -23.133, ep_step: 855 2023-05-30 01:55:52 - SimpleLog - INFO: - episode: 3780, ep_reward: -113.039, ep_step: 97 2023-05-30 01:55:54 - SimpleLog - INFO: - episode: 3781, ep_reward: 94.092, ep_step: 1000 2023-05-30 01:55:56 - SimpleLog - INFO: - episode: 3782, ep_reward: -3.501, ep_step: 833 2023-05-30 01:55:58 - SimpleLog - INFO: - episode: 3783, ep_reward: 79.833, ep_step: 1000 2023-05-30 01:56:01 - SimpleLog - INFO: - episode: 3784, ep_reward: 82.606, ep_step: 1000 2023-05-30 01:56:03 - SimpleLog - INFO: - episode: 3785, ep_reward: -7.367, ep_step: 927 2023-05-30 01:56:05 - SimpleLog - INFO: - episode: 3786, ep_reward: -0.376, ep_step: 1000 2023-05-30 01:56:07 - SimpleLog - INFO: - episode: 3787, ep_reward: 87.757, ep_step: 1000 2023-05-30 01:56:09 - SimpleLog - INFO: - episode: 3788, ep_reward: -3.978, ep_step: 897 2023-05-30 01:56:10 - SimpleLog - INFO: - episode: 3789, ep_reward: -143.183, ep_step: 256 2023-05-30 01:56:12 - SimpleLog - INFO: - episode: 3790, ep_reward: 94.031, ep_step: 1000 2023-05-30 01:56:14 - SimpleLog - INFO: - episode: 3791, ep_reward: -5.048, ep_step: 947 2023-05-30 01:56:17 - SimpleLog - INFO: - episode: 3792, ep_reward: 79.675, ep_step: 1000 2023-05-30 01:56:21 - SimpleLog - INFO: - update_step: 2800000, online_eval_reward: 99.688 2023-05-30 01:56:21 - SimpleLog - INFO: - episode: 3793, ep_reward: -18.044, ep_step: 906 2023-05-30 01:56:23 - SimpleLog - INFO: - episode: 3794, ep_reward: 110.292, ep_step: 1000 2023-05-30 01:56:25 - SimpleLog - INFO: - episode: 3795, ep_reward: 95.043, ep_step: 1000 2023-05-30 01:56:27 - SimpleLog - INFO: - episode: 3796, ep_reward: 92.038, ep_step: 1000 2023-05-30 01:56:30 - SimpleLog - INFO: - episode: 3797, ep_reward: 109.483, ep_step: 1000 2023-05-30 01:56:32 - SimpleLog - INFO: - episode: 3798, ep_reward: 100.404, ep_step: 1000 2023-05-30 01:56:34 - SimpleLog - INFO: - episode: 3799, ep_reward: 90.606, ep_step: 1000 2023-05-30 01:56:34 - SimpleLog - INFO: - episode: 3800, ep_reward: -116.260, ep_step: 110 2023-05-30 01:56:37 - SimpleLog - INFO: - episode: 3801, ep_reward: 102.055, ep_step: 1000 2023-05-30 01:56:39 - SimpleLog - INFO: - episode: 3802, ep_reward: 99.595, ep_step: 1000 2023-05-30 01:56:41 - SimpleLog - INFO: - episode: 3803, ep_reward: 99.020, ep_step: 1000 2023-05-30 01:56:43 - SimpleLog - INFO: - episode: 3804, ep_reward: 106.618, ep_step: 1000 2023-05-30 01:56:46 - SimpleLog - INFO: - episode: 3805, ep_reward: 100.381, ep_step: 1000 2023-05-30 01:56:48 - SimpleLog - INFO: - episode: 3806, ep_reward: 98.369, ep_step: 1000 2023-05-30 01:56:50 - SimpleLog - INFO: - episode: 3807, ep_reward: -32.430, ep_step: 943 2023-05-30 01:56:51 - SimpleLog - INFO: - episode: 3808, ep_reward: -66.155, ep_step: 566 2023-05-30 01:56:54 - SimpleLog - INFO: - episode: 3809, ep_reward: 101.388, ep_step: 1000 2023-05-30 01:56:55 - SimpleLog - INFO: - episode: 3810, ep_reward: -41.139, ep_step: 646 2023-05-30 01:56:57 - SimpleLog - INFO: - episode: 3811, ep_reward: 94.031, ep_step: 1000 2023-05-30 01:56:59 - SimpleLog - INFO: - episode: 3812, ep_reward: 108.850, ep_step: 1000 2023-05-30 01:57:02 - SimpleLog - INFO: - episode: 3813, ep_reward: 111.916, ep_step: 1000 2023-05-30 01:57:04 - SimpleLog - INFO: - episode: 3814, ep_reward: 108.231, ep_step: 1000 2023-05-30 01:57:06 - SimpleLog - INFO: - episode: 3815, ep_reward: 98.404, ep_step: 1000 2023-05-30 01:57:08 - SimpleLog - INFO: - episode: 3816, ep_reward: 103.244, ep_step: 1000 2023-05-30 01:57:11 - SimpleLog - INFO: - episode: 3817, ep_reward: 102.348, ep_step: 1000 2023-05-30 01:57:13 - SimpleLog - INFO: - episode: 3818, ep_reward: 101.748, ep_step: 1000 2023-05-30 01:57:15 - SimpleLog - INFO: - episode: 3819, ep_reward: 83.095, ep_step: 1000 2023-05-30 01:57:17 - SimpleLog - INFO: - episode: 3820, ep_reward: 82.505, ep_step: 1000 2023-05-30 01:57:20 - SimpleLog - INFO: - episode: 3821, ep_reward: 16.975, ep_step: 1000 2023-05-30 01:57:21 - SimpleLog - INFO: - episode: 3822, ep_reward: -17.117, ep_step: 782 2023-05-30 01:57:23 - SimpleLog - INFO: - episode: 3823, ep_reward: -13.834, ep_step: 845 2023-05-30 01:57:26 - SimpleLog - INFO: - episode: 3824, ep_reward: 83.998, ep_step: 1000 2023-05-30 01:57:28 - SimpleLog - INFO: - episode: 3825, ep_reward: 72.058, ep_step: 1000 2023-05-30 01:57:30 - SimpleLog - INFO: - episode: 3826, ep_reward: 93.416, ep_step: 1000 2023-05-30 01:57:31 - SimpleLog - INFO: - episode: 3827, ep_reward: -99.009, ep_step: 188 2023-05-30 01:57:33 - SimpleLog - INFO: - episode: 3828, ep_reward: 82.661, ep_step: 1000 2023-05-30 01:57:35 - SimpleLog - INFO: - episode: 3829, ep_reward: -27.461, ep_step: 986 2023-05-30 01:57:37 - SimpleLog - INFO: - episode: 3830, ep_reward: 100.059, ep_step: 1000 2023-05-30 01:57:39 - SimpleLog - INFO: - episode: 3831, ep_reward: 94.077, ep_step: 1000 2023-05-30 01:57:42 - SimpleLog - INFO: - episode: 3832, ep_reward: 94.899, ep_step: 1000 2023-05-30 01:57:44 - SimpleLog - INFO: - episode: 3833, ep_reward: 76.884, ep_step: 1000 2023-05-30 01:57:46 - SimpleLog - INFO: - episode: 3834, ep_reward: 137.272, ep_step: 1000 2023-05-30 01:57:48 - SimpleLog - INFO: - episode: 3835, ep_reward: -14.938, ep_step: 738 2023-05-30 01:57:50 - SimpleLog - INFO: - episode: 3836, ep_reward: 86.214, ep_step: 1000 2023-05-30 01:57:52 - SimpleLog - INFO: - episode: 3837, ep_reward: 113.269, ep_step: 1000 2023-05-30 01:57:55 - SimpleLog - INFO: - episode: 3838, ep_reward: 119.980, ep_step: 1000 2023-05-30 01:57:57 - SimpleLog - INFO: - episode: 3839, ep_reward: 107.998, ep_step: 1000 2023-05-30 01:57:59 - SimpleLog - INFO: - episode: 3840, ep_reward: 89.262, ep_step: 1000 2023-05-30 01:58:01 - SimpleLog - INFO: - episode: 3841, ep_reward: 103.219, ep_step: 1000 2023-05-30 01:58:04 - SimpleLog - INFO: - episode: 3842, ep_reward: 95.239, ep_step: 1000 2023-05-30 01:58:06 - SimpleLog - INFO: - episode: 3843, ep_reward: -16.319, ep_step: 819 2023-05-30 01:58:08 - SimpleLog - INFO: - episode: 3844, ep_reward: -8.153, ep_step: 862 2023-05-30 01:58:09 - SimpleLog - INFO: - episode: 3845, ep_reward: -20.125, ep_step: 702 2023-05-30 01:58:11 - SimpleLog - INFO: - episode: 3846, ep_reward: -18.657, ep_step: 681 2023-05-30 01:58:13 - SimpleLog - INFO: - episode: 3847, ep_reward: 87.729, ep_step: 1000 2023-05-30 01:58:13 - SimpleLog - INFO: - episode: 3848, ep_reward: -102.073, ep_step: 99 2023-05-30 01:58:15 - SimpleLog - INFO: - episode: 3849, ep_reward: 110.758, ep_step: 1000 2023-05-30 01:58:18 - SimpleLog - INFO: - episode: 3850, ep_reward: 80.337, ep_step: 1000 2023-05-30 01:58:18 - SimpleLog - INFO: - episode: 3851, ep_reward: -123.994, ep_step: 140 2023-05-30 01:58:20 - SimpleLog - INFO: - episode: 3852, ep_reward: 110.411, ep_step: 1000 2023-05-30 01:58:22 - SimpleLog - INFO: - episode: 3853, ep_reward: 0.359, ep_step: 999 2023-05-30 01:58:25 - SimpleLog - INFO: - episode: 3854, ep_reward: 104.848, ep_step: 1000 2023-05-30 01:58:27 - SimpleLog - INFO: - episode: 3855, ep_reward: 106.125, ep_step: 1000 2023-05-30 01:58:29 - SimpleLog - INFO: - episode: 3856, ep_reward: 110.672, ep_step: 1000 2023-05-30 01:58:31 - SimpleLog - INFO: - episode: 3857, ep_reward: -27.910, ep_step: 1000 2023-05-30 01:58:34 - SimpleLog - INFO: - episode: 3858, ep_reward: 103.858, ep_step: 1000 2023-05-30 01:58:36 - SimpleLog - INFO: - episode: 3859, ep_reward: -14.102, ep_step: 827 2023-05-30 01:58:38 - SimpleLog - INFO: - episode: 3860, ep_reward: 83.473, ep_step: 1000 2023-05-30 01:58:40 - SimpleLog - INFO: - episode: 3861, ep_reward: 101.920, ep_step: 1000 2023-05-30 01:58:42 - SimpleLog - INFO: - episode: 3862, ep_reward: 98.425, ep_step: 1000 2023-05-30 01:58:44 - SimpleLog - INFO: - episode: 3863, ep_reward: -23.711, ep_step: 792 2023-05-30 01:58:46 - SimpleLog - INFO: - episode: 3864, ep_reward: 84.736, ep_step: 1000 2023-05-30 01:58:49 - SimpleLog - INFO: - episode: 3865, ep_reward: 96.663, ep_step: 1000 2023-05-30 01:58:51 - SimpleLog - INFO: - episode: 3866, ep_reward: 110.858, ep_step: 1000 2023-05-30 01:58:53 - SimpleLog - INFO: - episode: 3867, ep_reward: 111.042, ep_step: 1000 2023-05-30 01:58:55 - SimpleLog - INFO: - episode: 3868, ep_reward: 104.727, ep_step: 1000 2023-05-30 01:58:58 - SimpleLog - INFO: - episode: 3869, ep_reward: 100.846, ep_step: 1000 2023-05-30 01:59:00 - SimpleLog - INFO: - update_step: 2870000, online_eval_reward: 109.076 2023-05-30 01:59:01 - SimpleLog - INFO: - episode: 3870, ep_reward: -7.108, ep_step: 713 2023-05-30 01:59:02 - SimpleLog - INFO: - episode: 3871, ep_reward: -71.615, ep_step: 535 2023-05-30 01:59:04 - SimpleLog - INFO: - episode: 3872, ep_reward: 102.003, ep_step: 1000 2023-05-30 01:59:07 - SimpleLog - INFO: - episode: 3873, ep_reward: 110.667, ep_step: 1000 2023-05-30 01:59:09 - SimpleLog - INFO: - episode: 3874, ep_reward: 122.256, ep_step: 1000 2023-05-30 01:59:11 - SimpleLog - INFO: - episode: 3875, ep_reward: 110.023, ep_step: 1000 2023-05-30 01:59:14 - SimpleLog - INFO: - episode: 3876, ep_reward: -22.705, ep_step: 1000 2023-05-30 01:59:15 - SimpleLog - INFO: - episode: 3877, ep_reward: -4.272, ep_step: 780 2023-05-30 01:59:18 - SimpleLog - INFO: - episode: 3878, ep_reward: 113.582, ep_step: 1000 2023-05-30 01:59:20 - SimpleLog - INFO: - episode: 3879, ep_reward: 104.984, ep_step: 1000 2023-05-30 01:59:22 - SimpleLog - INFO: - episode: 3880, ep_reward: -27.381, ep_step: 1000 2023-05-30 01:59:24 - SimpleLog - INFO: - episode: 3881, ep_reward: 100.056, ep_step: 1000 2023-05-30 01:59:27 - SimpleLog - INFO: - episode: 3882, ep_reward: 88.604, ep_step: 1000 2023-05-30 01:59:29 - SimpleLog - INFO: - episode: 3883, ep_reward: 76.175, ep_step: 1000 2023-05-30 01:59:31 - SimpleLog - INFO: - episode: 3884, ep_reward: 95.103, ep_step: 1000 2023-05-30 01:59:33 - SimpleLog - INFO: - episode: 3885, ep_reward: 109.304, ep_step: 1000 2023-05-30 01:59:35 - SimpleLog - INFO: - episode: 3886, ep_reward: -36.775, ep_step: 817 2023-05-30 01:59:37 - SimpleLog - INFO: - episode: 3887, ep_reward: 104.763, ep_step: 1000 2023-05-30 01:59:39 - SimpleLog - INFO: - episode: 3888, ep_reward: -92.224, ep_step: 912 2023-05-30 01:59:42 - SimpleLog - INFO: - episode: 3889, ep_reward: 101.296, ep_step: 1000 2023-05-30 01:59:44 - SimpleLog - INFO: - episode: 3890, ep_reward: 110.871, ep_step: 1000 2023-05-30 01:59:46 - SimpleLog - INFO: - episode: 3891, ep_reward: 103.334, ep_step: 1000 2023-05-30 01:59:48 - SimpleLog - INFO: - episode: 3892, ep_reward: 108.668, ep_step: 1000 2023-05-30 01:59:51 - SimpleLog - INFO: - episode: 3893, ep_reward: 103.395, ep_step: 1000 2023-05-30 01:59:53 - SimpleLog - INFO: - episode: 3894, ep_reward: 118.745, ep_step: 1000 2023-05-30 01:59:55 - SimpleLog - INFO: - episode: 3895, ep_reward: 36.387, ep_step: 1000 2023-05-30 01:59:57 - SimpleLog - INFO: - episode: 3896, ep_reward: -74.075, ep_step: 625 2023-05-30 01:59:59 - SimpleLog - INFO: - episode: 3897, ep_reward: 74.995, ep_step: 1000 2023-05-30 02:00:01 - SimpleLog - INFO: - episode: 3898, ep_reward: 87.206, ep_step: 1000 2023-05-30 02:00:03 - SimpleLog - INFO: - episode: 3899, ep_reward: 101.276, ep_step: 1000 2023-05-30 02:00:06 - SimpleLog - INFO: - episode: 3900, ep_reward: 81.770, ep_step: 1000 2023-05-30 02:00:08 - SimpleLog - INFO: - episode: 3901, ep_reward: 109.066, ep_step: 1000 2023-05-30 02:00:10 - SimpleLog - INFO: - episode: 3902, ep_reward: 103.171, ep_step: 1000 2023-05-30 02:00:12 - SimpleLog - INFO: - episode: 3903, ep_reward: 99.397, ep_step: 1000 2023-05-30 02:00:15 - SimpleLog - INFO: - episode: 3904, ep_reward: 101.879, ep_step: 1000 2023-05-30 02:00:17 - SimpleLog - INFO: - episode: 3905, ep_reward: 108.034, ep_step: 1000 2023-05-30 02:00:19 - SimpleLog - INFO: - episode: 3906, ep_reward: 110.489, ep_step: 1000 2023-05-30 02:00:21 - SimpleLog - INFO: - episode: 3907, ep_reward: 105.214, ep_step: 1000 2023-05-30 02:00:24 - SimpleLog - INFO: - episode: 3908, ep_reward: 105.351, ep_step: 1000 2023-05-30 02:00:26 - SimpleLog - INFO: - episode: 3909, ep_reward: 115.117, ep_step: 1000 2023-05-30 02:00:28 - SimpleLog - INFO: - episode: 3910, ep_reward: 94.528, ep_step: 1000 2023-05-30 02:00:30 - SimpleLog - INFO: - episode: 3911, ep_reward: -1.507, ep_step: 841 2023-05-30 02:00:32 - SimpleLog - INFO: - episode: 3912, ep_reward: 113.993, ep_step: 1000 2023-05-30 02:00:34 - SimpleLog - INFO: - episode: 3913, ep_reward: 110.850, ep_step: 1000 2023-05-30 02:00:37 - SimpleLog - INFO: - episode: 3914, ep_reward: 109.153, ep_step: 1000 2023-05-30 02:00:39 - SimpleLog - INFO: - episode: 3915, ep_reward: 105.637, ep_step: 1000 2023-05-30 02:00:41 - SimpleLog - INFO: - episode: 3916, ep_reward: 102.073, ep_step: 1000 2023-05-30 02:00:43 - SimpleLog - INFO: - episode: 3917, ep_reward: -29.774, ep_step: 688 2023-05-30 02:00:45 - SimpleLog - INFO: - episode: 3918, ep_reward: 109.093, ep_step: 1000 2023-05-30 02:00:47 - SimpleLog - INFO: - episode: 3919, ep_reward: 112.629, ep_step: 1000 2023-05-30 02:00:49 - SimpleLog - INFO: - episode: 3920, ep_reward: 100.768, ep_step: 1000 2023-05-30 02:00:52 - SimpleLog - INFO: - episode: 3921, ep_reward: 111.272, ep_step: 1000 2023-05-30 02:00:54 - SimpleLog - INFO: - episode: 3922, ep_reward: 112.007, ep_step: 1000 2023-05-30 02:00:56 - SimpleLog - INFO: - episode: 3923, ep_reward: 112.064, ep_step: 1000 2023-05-30 02:00:58 - SimpleLog - INFO: - episode: 3924, ep_reward: 102.568, ep_step: 1000 2023-05-30 02:01:01 - SimpleLog - INFO: - episode: 3925, ep_reward: 109.376, ep_step: 1000 2023-05-30 02:01:03 - SimpleLog - INFO: - episode: 3926, ep_reward: 30.283, ep_step: 1000 2023-05-30 02:01:05 - SimpleLog - INFO: - episode: 3927, ep_reward: 84.407, ep_step: 1000 2023-05-30 02:01:07 - SimpleLog - INFO: - episode: 3928, ep_reward: 87.597, ep_step: 1000 2023-05-30 02:01:10 - SimpleLog - INFO: - episode: 3929, ep_reward: 114.713, ep_step: 1000 2023-05-30 02:01:12 - SimpleLog - INFO: - episode: 3930, ep_reward: 93.146, ep_step: 1000 2023-05-30 02:01:14 - SimpleLog - INFO: - episode: 3931, ep_reward: 98.038, ep_step: 1000 2023-05-30 02:01:16 - SimpleLog - INFO: - episode: 3932, ep_reward: 107.540, ep_step: 1000 2023-05-30 02:01:19 - SimpleLog - INFO: - episode: 3933, ep_reward: 110.417, ep_step: 1000 2023-05-30 02:01:21 - SimpleLog - INFO: - episode: 3934, ep_reward: 100.763, ep_step: 1000 2023-05-30 02:01:23 - SimpleLog - INFO: - episode: 3935, ep_reward: 101.452, ep_step: 1000 2023-05-30 02:01:25 - SimpleLog - INFO: - episode: 3936, ep_reward: 105.007, ep_step: 1000 2023-05-30 02:01:26 - SimpleLog - INFO: - episode: 3937, ep_reward: -101.774, ep_step: 171 2023-05-30 02:01:28 - SimpleLog - INFO: - episode: 3938, ep_reward: 79.385, ep_step: 1000 2023-05-30 02:01:30 - SimpleLog - INFO: - episode: 3939, ep_reward: 107.025, ep_step: 1000 2023-05-30 02:01:32 - SimpleLog - INFO: - episode: 3940, ep_reward: 93.904, ep_step: 1000 2023-05-30 02:01:35 - SimpleLog - INFO: - episode: 3941, ep_reward: 100.826, ep_step: 1000 2023-05-30 02:01:37 - SimpleLog - INFO: - episode: 3942, ep_reward: 80.136, ep_step: 1000 2023-05-30 02:01:39 - SimpleLog - INFO: - update_step: 2940000, online_eval_reward: 9.730 2023-05-30 02:01:41 - SimpleLog - INFO: - episode: 3943, ep_reward: 114.114, ep_step: 1000 2023-05-30 02:01:43 - SimpleLog - INFO: - episode: 3944, ep_reward: 7.117, ep_step: 952 2023-05-30 02:01:45 - SimpleLog - INFO: - episode: 3945, ep_reward: 1.757, ep_step: 820 2023-05-30 02:01:47 - SimpleLog - INFO: - episode: 3946, ep_reward: 109.829, ep_step: 1000 2023-05-30 02:01:49 - SimpleLog - INFO: - episode: 3947, ep_reward: 112.763, ep_step: 1000 2023-05-30 02:01:52 - SimpleLog - INFO: - episode: 3948, ep_reward: -1.157, ep_step: 977 2023-05-30 02:01:53 - SimpleLog - INFO: - episode: 3949, ep_reward: -41.949, ep_step: 597 2023-05-30 02:01:55 - SimpleLog - INFO: - episode: 3950, ep_reward: 0.206, ep_step: 807 2023-05-30 02:01:57 - SimpleLog - INFO: - episode: 3951, ep_reward: 5.288, ep_step: 976 2023-05-30 02:01:59 - SimpleLog - INFO: - episode: 3952, ep_reward: -5.481, ep_step: 835 2023-05-30 02:02:01 - SimpleLog - INFO: - episode: 3953, ep_reward: 89.527, ep_step: 1000 2023-05-30 02:02:03 - SimpleLog - INFO: - episode: 3954, ep_reward: 79.896, ep_step: 1000 2023-05-30 02:02:06 - SimpleLog - INFO: - episode: 3955, ep_reward: 94.669, ep_step: 1000 2023-05-30 02:02:08 - SimpleLog - INFO: - episode: 3956, ep_reward: 80.961, ep_step: 1000 2023-05-30 02:02:10 - SimpleLog - INFO: - episode: 3957, ep_reward: 78.055, ep_step: 1000 2023-05-30 02:02:12 - SimpleLog - INFO: - episode: 3958, ep_reward: -55.104, ep_step: 879 2023-05-30 02:02:14 - SimpleLog - INFO: - episode: 3959, ep_reward: 82.299, ep_step: 1000 2023-05-30 02:02:17 - SimpleLog - INFO: - episode: 3960, ep_reward: 77.515, ep_step: 1000 2023-05-30 02:02:19 - SimpleLog - INFO: - episode: 3961, ep_reward: 2.472, ep_step: 1000 2023-05-30 02:02:21 - SimpleLog - INFO: - episode: 3962, ep_reward: 72.265, ep_step: 1000 2023-05-30 02:02:23 - SimpleLog - INFO: - episode: 3963, ep_reward: 80.877, ep_step: 1000 2023-05-30 02:02:26 - SimpleLog - INFO: - episode: 3964, ep_reward: 74.048, ep_step: 1000 2023-05-30 02:02:28 - SimpleLog - INFO: - episode: 3965, ep_reward: 94.603, ep_step: 1000 2023-05-30 02:02:30 - SimpleLog - INFO: - episode: 3966, ep_reward: 107.421, ep_step: 1000 2023-05-30 02:02:32 - SimpleLog - INFO: - episode: 3967, ep_reward: 95.995, ep_step: 1000 2023-05-30 02:02:35 - SimpleLog - INFO: - episode: 3968, ep_reward: 110.309, ep_step: 1000 2023-05-30 02:02:37 - SimpleLog - INFO: - episode: 3969, ep_reward: 79.338, ep_step: 1000 2023-05-30 02:02:39 - SimpleLog - INFO: - episode: 3970, ep_reward: 115.617, ep_step: 1000 2023-05-30 02:02:41 - SimpleLog - INFO: - episode: 3971, ep_reward: 106.507, ep_step: 1000 2023-05-30 02:02:44 - SimpleLog - INFO: - episode: 3972, ep_reward: 82.302, ep_step: 1000 2023-05-30 02:02:46 - SimpleLog - INFO: - episode: 3973, ep_reward: 15.771, ep_step: 989 2023-05-30 02:02:48 - SimpleLog - INFO: - episode: 3974, ep_reward: 114.912, ep_step: 1000 2023-05-30 02:02:50 - SimpleLog - INFO: - episode: 3975, ep_reward: 112.921, ep_step: 1000 2023-05-30 02:02:53 - SimpleLog - INFO: - episode: 3976, ep_reward: 109.481, ep_step: 1000 2023-05-30 02:02:54 - SimpleLog - INFO: - episode: 3977, ep_reward: -22.453, ep_step: 774 2023-05-30 02:02:57 - SimpleLog - INFO: - episode: 3978, ep_reward: 4.823, ep_step: 970 2023-05-30 02:02:59 - SimpleLog - INFO: - episode: 3979, ep_reward: 84.655, ep_step: 1000 2023-05-30 02:03:01 - SimpleLog - INFO: - episode: 3980, ep_reward: 97.920, ep_step: 1000 2023-05-30 02:03:03 - SimpleLog - INFO: - episode: 3981, ep_reward: 0.815, ep_step: 973 2023-05-30 02:03:05 - SimpleLog - INFO: - episode: 3982, ep_reward: 108.792, ep_step: 1000 2023-05-30 02:03:08 - SimpleLog - INFO: - episode: 3983, ep_reward: 87.432, ep_step: 1000 2023-05-30 02:03:10 - SimpleLog - INFO: - episode: 3984, ep_reward: -34.201, ep_step: 949 2023-05-30 02:03:12 - SimpleLog - INFO: - episode: 3985, ep_reward: -15.081, ep_step: 700 2023-05-30 02:03:14 - SimpleLog - INFO: - episode: 3986, ep_reward: 108.507, ep_step: 1000 2023-05-30 02:03:16 - SimpleLog - INFO: - episode: 3987, ep_reward: 10.654, ep_step: 914 2023-05-30 02:03:18 - SimpleLog - INFO: - episode: 3988, ep_reward: 89.150, ep_step: 1000 2023-05-30 02:03:20 - SimpleLog - INFO: - episode: 3989, ep_reward: 95.697, ep_step: 1000 2023-05-30 02:03:23 - SimpleLog - INFO: - episode: 3990, ep_reward: 85.205, ep_step: 1000 2023-05-30 02:03:25 - SimpleLog - INFO: - episode: 3991, ep_reward: 97.518, ep_step: 1000 2023-05-30 02:03:27 - SimpleLog - INFO: - episode: 3992, ep_reward: 10.465, ep_step: 947 2023-05-30 02:03:29 - SimpleLog - INFO: - episode: 3993, ep_reward: 107.726, ep_step: 1000 2023-05-30 02:03:31 - SimpleLog - INFO: - episode: 3994, ep_reward: 7.248, ep_step: 958 2023-05-30 02:03:34 - SimpleLog - INFO: - episode: 3995, ep_reward: 125.658, ep_step: 1000 2023-05-30 02:03:35 - SimpleLog - INFO: - episode: 3996, ep_reward: -39.041, ep_step: 658 2023-05-30 02:03:37 - SimpleLog - INFO: - episode: 3997, ep_reward: 110.961, ep_step: 1000 2023-05-30 02:03:39 - SimpleLog - INFO: - episode: 3998, ep_reward: -86.023, ep_step: 673 2023-05-30 02:03:41 - SimpleLog - INFO: - episode: 3999, ep_reward: 111.163, ep_step: 1000 2023-05-30 02:03:43 - SimpleLog - INFO: - episode: 4000, ep_reward: 111.620, ep_step: 1000 2023-05-30 02:03:46 - SimpleLog - INFO: - episode: 4001, ep_reward: 106.904, ep_step: 1000 2023-05-30 02:03:48 - SimpleLog - INFO: - episode: 4002, ep_reward: 116.124, ep_step: 1000 2023-05-30 02:03:50 - SimpleLog - INFO: - episode: 4003, ep_reward: 111.232, ep_step: 1000 2023-05-30 02:03:52 - SimpleLog - INFO: - episode: 4004, ep_reward: 113.064, ep_step: 1000 2023-05-30 02:03:54 - SimpleLog - INFO: - episode: 4005, ep_reward: 106.259, ep_step: 1000 2023-05-30 02:03:57 - SimpleLog - INFO: - episode: 4006, ep_reward: 13.128, ep_step: 1000 2023-05-30 02:03:59 - SimpleLog - INFO: - episode: 4007, ep_reward: 110.002, ep_step: 1000 2023-05-30 02:04:00 - SimpleLog - INFO: - episode: 4008, ep_reward: -28.181, ep_step: 627 2023-05-30 02:04:03 - SimpleLog - INFO: - episode: 4009, ep_reward: 118.393, ep_step: 1000 2023-05-30 02:04:05 - SimpleLog - INFO: - episode: 4010, ep_reward: 108.771, ep_step: 1000 2023-05-30 02:04:07 - SimpleLog - INFO: - episode: 4011, ep_reward: 112.819, ep_step: 1000 2023-05-30 02:04:09 - SimpleLog - INFO: - episode: 4012, ep_reward: 86.338, ep_step: 1000 2023-05-30 02:04:12 - SimpleLog - INFO: - episode: 4013, ep_reward: 105.028, ep_step: 1000 2023-05-30 02:04:14 - SimpleLog - INFO: - episode: 4014, ep_reward: 101.850, ep_step: 1000 2023-05-30 02:04:16 - SimpleLog - INFO: - episode: 4015, ep_reward: 79.360, ep_step: 1000 2023-05-30 02:04:18 - SimpleLog - INFO: - update_step: 3010000, online_eval_reward: 37.629 2023-05-30 02:04:20 - SimpleLog - INFO: - episode: 4016, ep_reward: 33.360, ep_step: 1000 2023-05-30 02:04:22 - SimpleLog - INFO: - episode: 4017, ep_reward: 106.941, ep_step: 1000 2023-05-30 02:04:25 - SimpleLog - INFO: - episode: 4018, ep_reward: 105.394, ep_step: 1000 2023-05-30 02:04:27 - SimpleLog - INFO: - episode: 4019, ep_reward: 107.149, ep_step: 1000 2023-05-30 02:04:28 - SimpleLog - INFO: - episode: 4020, ep_reward: -100.855, ep_step: 620 2023-05-30 02:04:31 - SimpleLog - INFO: - episode: 4021, ep_reward: 113.520, ep_step: 1000 2023-05-30 02:04:33 - SimpleLog - INFO: - episode: 4022, ep_reward: 97.824, ep_step: 1000 2023-05-30 02:04:35 - SimpleLog - INFO: - episode: 4023, ep_reward: 96.296, ep_step: 1000 2023-05-30 02:04:37 - SimpleLog - INFO: - episode: 4024, ep_reward: 89.863, ep_step: 1000 2023-05-30 02:04:40 - SimpleLog - INFO: - episode: 4025, ep_reward: 108.064, ep_step: 1000 2023-05-30 02:04:42 - SimpleLog - INFO: - episode: 4026, ep_reward: 99.743, ep_step: 1000 2023-05-30 02:04:44 - SimpleLog - INFO: - episode: 4027, ep_reward: 111.607, ep_step: 1000 2023-05-30 02:04:46 - SimpleLog - INFO: - episode: 4028, ep_reward: 108.272, ep_step: 1000 2023-05-30 02:04:49 - SimpleLog - INFO: - episode: 4029, ep_reward: 99.950, ep_step: 1000 2023-05-30 02:04:51 - SimpleLog - INFO: - episode: 4030, ep_reward: 111.193, ep_step: 1000 2023-05-30 02:04:53 - SimpleLog - INFO: - episode: 4031, ep_reward: 115.643, ep_step: 1000 2023-05-30 02:04:55 - SimpleLog - INFO: - episode: 4032, ep_reward: 106.994, ep_step: 1000 2023-05-30 02:04:58 - SimpleLog - INFO: - episode: 4033, ep_reward: 96.601, ep_step: 1000 2023-05-30 02:05:00 - SimpleLog - INFO: - episode: 4034, ep_reward: 112.507, ep_step: 1000 2023-05-30 02:05:02 - SimpleLog - INFO: - episode: 4035, ep_reward: 105.838, ep_step: 1000 2023-05-30 02:05:04 - SimpleLog - INFO: - episode: 4036, ep_reward: 110.050, ep_step: 1000 2023-05-30 02:05:07 - SimpleLog - INFO: - episode: 4037, ep_reward: 103.653, ep_step: 1000 2023-05-30 02:05:09 - SimpleLog - INFO: - episode: 4038, ep_reward: 115.676, ep_step: 1000 2023-05-30 02:05:11 - SimpleLog - INFO: - episode: 4039, ep_reward: 117.810, ep_step: 1000 2023-05-30 02:05:13 - SimpleLog - INFO: - episode: 4040, ep_reward: 112.697, ep_step: 1000 2023-05-30 02:05:16 - SimpleLog - INFO: - episode: 4041, ep_reward: 108.443, ep_step: 1000 2023-05-30 02:05:18 - SimpleLog - INFO: - episode: 4042, ep_reward: 6.724, ep_step: 970 2023-05-30 02:05:20 - SimpleLog - INFO: - episode: 4043, ep_reward: 114.253, ep_step: 1000 2023-05-30 02:05:22 - SimpleLog - INFO: - episode: 4044, ep_reward: 105.957, ep_step: 1000 2023-05-30 02:05:24 - SimpleLog - INFO: - episode: 4045, ep_reward: -42.528, ep_step: 642 2023-05-30 02:05:26 - SimpleLog - INFO: - episode: 4046, ep_reward: 113.700, ep_step: 1000 2023-05-30 02:05:28 - SimpleLog - INFO: - episode: 4047, ep_reward: 105.983, ep_step: 1000 2023-05-30 02:05:31 - SimpleLog - INFO: - episode: 4048, ep_reward: -9.105, ep_step: 1000 2023-05-30 02:05:33 - SimpleLog - INFO: - episode: 4049, ep_reward: 15.617, ep_step: 971 2023-05-30 02:05:34 - SimpleLog - INFO: - episode: 4050, ep_reward: -41.649, ep_step: 638 2023-05-30 02:05:36 - SimpleLog - INFO: - episode: 4051, ep_reward: 3.395, ep_step: 984 2023-05-30 02:05:38 - SimpleLog - INFO: - episode: 4052, ep_reward: -69.599, ep_step: 565 2023-05-30 02:05:40 - SimpleLog - INFO: - episode: 4053, ep_reward: 120.366, ep_step: 1000 2023-05-30 02:05:42 - SimpleLog - INFO: - episode: 4054, ep_reward: 101.823, ep_step: 1000 2023-05-30 02:05:44 - SimpleLog - INFO: - episode: 4055, ep_reward: 109.752, ep_step: 1000 2023-05-30 02:05:47 - SimpleLog - INFO: - episode: 4056, ep_reward: 7.548, ep_step: 1000 2023-05-30 02:05:49 - SimpleLog - INFO: - episode: 4057, ep_reward: 113.303, ep_step: 1000 2023-05-30 02:05:49 - SimpleLog - INFO: - episode: 4058, ep_reward: -111.491, ep_step: 82 2023-05-30 02:05:51 - SimpleLog - INFO: - episode: 4059, ep_reward: 119.419, ep_step: 1000 2023-05-30 02:05:54 - SimpleLog - INFO: - episode: 4060, ep_reward: 5.278, ep_step: 950 2023-05-30 02:05:56 - SimpleLog - INFO: - episode: 4061, ep_reward: 114.405, ep_step: 1000 2023-05-30 02:05:57 - SimpleLog - INFO: - episode: 4062, ep_reward: -25.856, ep_step: 720 2023-05-30 02:05:59 - SimpleLog - INFO: - episode: 4063, ep_reward: -21.783, ep_step: 604 2023-05-30 02:06:00 - SimpleLog - INFO: - episode: 4064, ep_reward: -36.928, ep_step: 522 2023-05-30 02:06:02 - SimpleLog - INFO: - episode: 4065, ep_reward: 70.535, ep_step: 1000 2023-05-30 02:06:04 - SimpleLog - INFO: - episode: 4066, ep_reward: 5.976, ep_step: 931 2023-05-30 02:06:06 - SimpleLog - INFO: - episode: 4067, ep_reward: 1.033, ep_step: 911 2023-05-30 02:06:09 - SimpleLog - INFO: - episode: 4068, ep_reward: 129.321, ep_step: 1000 2023-05-30 02:06:11 - SimpleLog - INFO: - episode: 4069, ep_reward: 128.659, ep_step: 1000 2023-05-30 02:06:13 - SimpleLog - INFO: - episode: 4070, ep_reward: 108.332, ep_step: 1000 2023-05-30 02:06:15 - SimpleLog - INFO: - episode: 4071, ep_reward: 116.009, ep_step: 1000 2023-05-30 02:06:18 - SimpleLog - INFO: - episode: 4072, ep_reward: 118.248, ep_step: 1000 2023-05-30 02:06:20 - SimpleLog - INFO: - episode: 4073, ep_reward: 116.436, ep_step: 1000 2023-05-30 02:06:22 - SimpleLog - INFO: - episode: 4074, ep_reward: 130.647, ep_step: 1000 2023-05-30 02:06:24 - SimpleLog - INFO: - episode: 4075, ep_reward: 81.270, ep_step: 1000 2023-05-30 02:06:27 - SimpleLog - INFO: - episode: 4076, ep_reward: 90.867, ep_step: 1000 2023-05-30 02:06:28 - SimpleLog - INFO: - episode: 4077, ep_reward: -18.725, ep_step: 608 2023-05-30 02:06:30 - SimpleLog - INFO: - episode: 4078, ep_reward: 108.360, ep_step: 1000 2023-05-30 02:06:32 - SimpleLog - INFO: - episode: 4079, ep_reward: 129.218, ep_step: 1000 2023-05-30 02:06:34 - SimpleLog - INFO: - episode: 4080, ep_reward: -34.121, ep_step: 911 2023-05-30 02:06:37 - SimpleLog - INFO: - episode: 4081, ep_reward: 138.635, ep_step: 1000 2023-05-30 02:06:39 - SimpleLog - INFO: - episode: 4082, ep_reward: 130.552, ep_step: 1000 2023-05-30 02:06:40 - SimpleLog - INFO: - episode: 4083, ep_reward: -54.689, ep_step: 541 2023-05-30 02:06:42 - SimpleLog - INFO: - episode: 4084, ep_reward: 13.498, ep_step: 1000 2023-05-30 02:06:44 - SimpleLog - INFO: - episode: 4085, ep_reward: 1.192, ep_step: 633 2023-05-30 02:06:46 - SimpleLog - INFO: - episode: 4086, ep_reward: 79.064, ep_step: 1000 2023-05-30 02:06:48 - SimpleLog - INFO: - episode: 4087, ep_reward: -16.542, ep_step: 981 2023-05-30 02:06:51 - SimpleLog - INFO: - episode: 4088, ep_reward: 10.361, ep_step: 1000 2023-05-30 02:06:53 - SimpleLog - INFO: - episode: 4089, ep_reward: 12.954, ep_step: 1000 2023-05-30 02:06:54 - SimpleLog - INFO: - episode: 4090, ep_reward: -127.659, ep_step: 629 2023-05-30 02:06:57 - SimpleLog - INFO: - update_step: 3080000, online_eval_reward: 0.234 2023-05-30 02:06:58 - SimpleLog - INFO: - episode: 4091, ep_reward: 8.933, ep_step: 1000 2023-05-30 02:07:00 - SimpleLog - INFO: - episode: 4092, ep_reward: -5.851, ep_step: 718 2023-05-30 02:07:02 - SimpleLog - INFO: - episode: 4093, ep_reward: 123.620, ep_step: 1000 2023-05-30 02:07:04 - SimpleLog - INFO: - episode: 4094, ep_reward: 15.395, ep_step: 1000 2023-05-30 02:07:06 - SimpleLog - INFO: - episode: 4095, ep_reward: 84.920, ep_step: 1000 2023-05-30 02:07:09 - SimpleLog - INFO: - episode: 4096, ep_reward: 100.929, ep_step: 1000 2023-05-30 02:07:11 - SimpleLog - INFO: - episode: 4097, ep_reward: 9.157, ep_step: 1000 2023-05-30 02:07:13 - SimpleLog - INFO: - episode: 4098, ep_reward: 89.903, ep_step: 1000 2023-05-30 02:07:15 - SimpleLog - INFO: - episode: 4099, ep_reward: 152.809, ep_step: 1000 2023-05-30 02:07:18 - SimpleLog - INFO: - episode: 4100, ep_reward: 139.562, ep_step: 1000 2023-05-30 02:07:20 - SimpleLog - INFO: - episode: 4101, ep_reward: 133.921, ep_step: 1000 2023-05-30 02:07:22 - SimpleLog - INFO: - episode: 4102, ep_reward: 79.811, ep_step: 1000 2023-05-30 02:07:24 - SimpleLog - INFO: - episode: 4103, ep_reward: 119.397, ep_step: 1000 2023-05-30 02:07:27 - SimpleLog - INFO: - episode: 4104, ep_reward: 114.450, ep_step: 1000 2023-05-30 02:07:29 - SimpleLog - INFO: - episode: 4105, ep_reward: 116.052, ep_step: 1000 2023-05-30 02:07:31 - SimpleLog - INFO: - episode: 4106, ep_reward: 132.974, ep_step: 1000 2023-05-30 02:07:33 - SimpleLog - INFO: - episode: 4107, ep_reward: -4.849, ep_step: 873 2023-05-30 02:07:35 - SimpleLog - INFO: - episode: 4108, ep_reward: 127.366, ep_step: 1000 2023-05-30 02:07:38 - SimpleLog - INFO: - episode: 4109, ep_reward: 93.806, ep_step: 1000 2023-05-30 02:07:40 - SimpleLog - INFO: - episode: 4110, ep_reward: 88.861, ep_step: 1000 2023-05-30 02:07:42 - SimpleLog - INFO: - episode: 4111, ep_reward: 87.503, ep_step: 1000 2023-05-30 02:07:44 - SimpleLog - INFO: - episode: 4112, ep_reward: 80.821, ep_step: 1000 2023-05-30 02:07:46 - SimpleLog - INFO: - episode: 4113, ep_reward: 5.039, ep_step: 708 2023-05-30 02:07:48 - SimpleLog - INFO: - episode: 4114, ep_reward: 15.779, ep_step: 894 2023-05-30 02:07:50 - SimpleLog - INFO: - episode: 4115, ep_reward: 92.756, ep_step: 1000 2023-05-30 02:07:53 - SimpleLog - INFO: - episode: 4116, ep_reward: 110.819, ep_step: 1000 2023-05-30 02:07:55 - SimpleLog - INFO: - episode: 4117, ep_reward: 86.574, ep_step: 1000 2023-05-30 02:07:57 - SimpleLog - INFO: - episode: 4118, ep_reward: 148.384, ep_step: 1000 2023-05-30 02:07:59 - SimpleLog - INFO: - episode: 4119, ep_reward: 132.889, ep_step: 1000 2023-05-30 02:08:02 - SimpleLog - INFO: - episode: 4120, ep_reward: 149.688, ep_step: 1000 2023-05-30 02:08:04 - SimpleLog - INFO: - episode: 4121, ep_reward: 23.682, ep_step: 1000 2023-05-30 02:08:06 - SimpleLog - INFO: - episode: 4122, ep_reward: 84.316, ep_step: 1000 2023-05-30 02:08:08 - SimpleLog - INFO: - episode: 4123, ep_reward: 82.361, ep_step: 1000 2023-05-30 02:08:11 - SimpleLog - INFO: - episode: 4124, ep_reward: 101.429, ep_step: 1000 2023-05-30 02:08:12 - SimpleLog - INFO: - episode: 4125, ep_reward: -64.616, ep_step: 522 2023-05-30 02:08:14 - SimpleLog - INFO: - episode: 4126, ep_reward: 103.237, ep_step: 1000 2023-05-30 02:08:16 - SimpleLog - INFO: - episode: 4127, ep_reward: -6.527, ep_step: 789 2023-05-30 02:08:18 - SimpleLog - INFO: - episode: 4128, ep_reward: 91.270, ep_step: 1000 2023-05-30 02:08:20 - SimpleLog - INFO: - episode: 4129, ep_reward: 91.462, ep_step: 1000 2023-05-30 02:08:23 - SimpleLog - INFO: - episode: 4130, ep_reward: 17.281, ep_step: 1000 2023-05-30 02:08:25 - SimpleLog - INFO: - episode: 4131, ep_reward: 136.720, ep_step: 1000 2023-05-30 02:08:27 - SimpleLog - INFO: - episode: 4132, ep_reward: 86.717, ep_step: 1000 2023-05-30 02:08:29 - SimpleLog - INFO: - episode: 4133, ep_reward: 88.999, ep_step: 1000 2023-05-30 02:08:32 - SimpleLog - INFO: - episode: 4134, ep_reward: 109.764, ep_step: 1000 2023-05-30 02:08:34 - SimpleLog - INFO: - episode: 4135, ep_reward: 109.754, ep_step: 1000 2023-05-30 02:08:36 - SimpleLog - INFO: - episode: 4136, ep_reward: 92.671, ep_step: 1000 2023-05-30 02:08:38 - SimpleLog - INFO: - episode: 4137, ep_reward: -43.668, ep_step: 613 2023-05-30 02:08:40 - SimpleLog - INFO: - episode: 4138, ep_reward: 91.286, ep_step: 1000 2023-05-30 02:08:42 - SimpleLog - INFO: - episode: 4139, ep_reward: 116.028, ep_step: 1000 2023-05-30 02:08:44 - SimpleLog - INFO: - episode: 4140, ep_reward: 81.143, ep_step: 1000 2023-05-30 02:08:47 - SimpleLog - INFO: - episode: 4141, ep_reward: 81.665, ep_step: 1000 2023-05-30 02:08:47 - SimpleLog - INFO: - episode: 4142, ep_reward: -116.197, ep_step: 82 2023-05-30 02:08:47 - SimpleLog - INFO: - episode: 4143, ep_reward: -117.085, ep_step: 81 2023-05-30 02:08:49 - SimpleLog - INFO: - episode: 4144, ep_reward: -22.709, ep_step: 939 2023-05-30 02:08:51 - SimpleLog - INFO: - episode: 4145, ep_reward: 87.090, ep_step: 1000 2023-05-30 02:08:54 - SimpleLog - INFO: - episode: 4146, ep_reward: 34.606, ep_step: 1000 2023-05-30 02:08:56 - SimpleLog - INFO: - episode: 4147, ep_reward: 88.606, ep_step: 1000 2023-05-30 02:08:58 - SimpleLog - INFO: - episode: 4148, ep_reward: 114.340, ep_step: 1000 2023-05-30 02:09:00 - SimpleLog - INFO: - episode: 4149, ep_reward: 100.115, ep_step: 1000 2023-05-30 02:09:03 - SimpleLog - INFO: - episode: 4150, ep_reward: 120.237, ep_step: 1000 2023-05-30 02:09:04 - SimpleLog - INFO: - episode: 4151, ep_reward: 4.436, ep_step: 780 2023-05-30 02:09:06 - SimpleLog - INFO: - episode: 4152, ep_reward: 90.455, ep_step: 1000 2023-05-30 02:09:09 - SimpleLog - INFO: - episode: 4153, ep_reward: 105.409, ep_step: 1000 2023-05-30 02:09:11 - SimpleLog - INFO: - episode: 4154, ep_reward: 107.459, ep_step: 1000 2023-05-30 02:09:13 - SimpleLog - INFO: - episode: 4155, ep_reward: 108.099, ep_step: 1000 2023-05-30 02:09:15 - SimpleLog - INFO: - episode: 4156, ep_reward: -18.027, ep_step: 938 2023-05-30 02:09:18 - SimpleLog - INFO: - episode: 4157, ep_reward: 114.872, ep_step: 1000 2023-05-30 02:09:20 - SimpleLog - INFO: - episode: 4158, ep_reward: 95.437, ep_step: 1000 2023-05-30 02:09:22 - SimpleLog - INFO: - episode: 4159, ep_reward: 47.554, ep_step: 1000 2023-05-30 02:09:24 - SimpleLog - INFO: - episode: 4160, ep_reward: 106.342, ep_step: 1000 2023-05-30 02:09:27 - SimpleLog - INFO: - episode: 4161, ep_reward: 127.747, ep_step: 1000 2023-05-30 02:09:29 - SimpleLog - INFO: - episode: 4162, ep_reward: 130.265, ep_step: 1000 2023-05-30 02:09:31 - SimpleLog - INFO: - episode: 4163, ep_reward: 101.587, ep_step: 1000 2023-05-30 02:09:33 - SimpleLog - INFO: - episode: 4164, ep_reward: 132.004, ep_step: 1000 2023-05-30 02:09:37 - SimpleLog - INFO: - update_step: 3150000, online_eval_reward: 151.050 2023-05-30 02:09:38 - SimpleLog - INFO: - episode: 4165, ep_reward: 126.057, ep_step: 1000 2023-05-30 02:09:39 - SimpleLog - INFO: - episode: 4166, ep_reward: -9.578, ep_step: 640 2023-05-30 02:09:41 - SimpleLog - INFO: - episode: 4167, ep_reward: 138.225, ep_step: 1000 2023-05-30 02:09:43 - SimpleLog - INFO: - episode: 4168, ep_reward: 115.315, ep_step: 1000 2023-05-30 02:09:45 - SimpleLog - INFO: - episode: 4169, ep_reward: -1.267, ep_step: 823 2023-05-30 02:09:47 - SimpleLog - INFO: - episode: 4170, ep_reward: 6.016, ep_step: 702 2023-05-30 02:09:48 - SimpleLog - INFO: - episode: 4171, ep_reward: 2.362, ep_step: 664 2023-05-30 02:09:51 - SimpleLog - INFO: - episode: 4172, ep_reward: 135.304, ep_step: 1000 2023-05-30 02:09:52 - SimpleLog - INFO: - episode: 4173, ep_reward: -40.573, ep_step: 569 2023-05-30 02:09:54 - SimpleLog - INFO: - episode: 4174, ep_reward: 130.667, ep_step: 1000 2023-05-30 02:09:56 - SimpleLog - INFO: - episode: 4175, ep_reward: 1.750, ep_step: 770 2023-05-30 02:09:58 - SimpleLog - INFO: - episode: 4176, ep_reward: 127.755, ep_step: 1000 2023-05-30 02:10:00 - SimpleLog - INFO: - episode: 4177, ep_reward: 103.109, ep_step: 1000 2023-05-30 02:10:03 - SimpleLog - INFO: - episode: 4178, ep_reward: 132.144, ep_step: 1000 2023-05-30 02:10:03 - SimpleLog - INFO: - episode: 4179, ep_reward: -114.308, ep_step: 90 2023-05-30 02:10:05 - SimpleLog - INFO: - episode: 4180, ep_reward: 97.605, ep_step: 1000 2023-05-30 02:10:05 - SimpleLog - INFO: - episode: 4181, ep_reward: -117.765, ep_step: 109 2023-05-30 02:10:07 - SimpleLog - INFO: - episode: 4182, ep_reward: -17.917, ep_step: 789 2023-05-30 02:10:07 - SimpleLog - INFO: - episode: 4183, ep_reward: -112.401, ep_step: 78 2023-05-30 02:10:08 - SimpleLog - INFO: - episode: 4184, ep_reward: -117.709, ep_step: 88 2023-05-30 02:10:10 - SimpleLog - INFO: - episode: 4185, ep_reward: 111.337, ep_step: 1000 2023-05-30 02:10:12 - SimpleLog - INFO: - episode: 4186, ep_reward: -6.312, ep_step: 966 2023-05-30 02:10:14 - SimpleLog - INFO: - episode: 4187, ep_reward: 75.014, ep_step: 1000 2023-05-30 02:10:16 - SimpleLog - INFO: - episode: 4188, ep_reward: 98.715, ep_step: 1000 2023-05-30 02:10:19 - SimpleLog - INFO: - episode: 4189, ep_reward: 81.791, ep_step: 1000 2023-05-30 02:10:19 - SimpleLog - INFO: - episode: 4190, ep_reward: -101.183, ep_step: 190 2023-05-30 02:10:21 - SimpleLog - INFO: - episode: 4191, ep_reward: -5.248, ep_step: 978 2023-05-30 02:10:24 - SimpleLog - INFO: - episode: 4192, ep_reward: 95.311, ep_step: 1000 2023-05-30 02:10:25 - SimpleLog - INFO: - episode: 4193, ep_reward: -64.706, ep_step: 732 2023-05-30 02:10:27 - SimpleLog - INFO: - episode: 4194, ep_reward: 124.354, ep_step: 1000 2023-05-30 02:10:30 - SimpleLog - INFO: - episode: 4195, ep_reward: 85.881, ep_step: 1000 2023-05-30 02:10:32 - SimpleLog - INFO: - episode: 4196, ep_reward: -10.967, ep_step: 794 2023-05-30 02:10:33 - SimpleLog - INFO: - episode: 4197, ep_reward: -43.266, ep_step: 610 2023-05-30 02:10:34 - SimpleLog - INFO: - episode: 4198, ep_reward: -38.309, ep_step: 550 2023-05-30 02:10:36 - SimpleLog - INFO: - episode: 4199, ep_reward: 73.004, ep_step: 1000 2023-05-30 02:10:37 - SimpleLog - INFO: - episode: 4200, ep_reward: -108.823, ep_step: 342 2023-05-30 02:10:39 - SimpleLog - INFO: - episode: 4201, ep_reward: 98.498, ep_step: 1000 2023-05-30 02:10:42 - SimpleLog - INFO: - episode: 4202, ep_reward: 134.946, ep_step: 1000 2023-05-30 02:10:44 - SimpleLog - INFO: - episode: 4203, ep_reward: 109.258, ep_step: 1000 2023-05-30 02:10:46 - SimpleLog - INFO: - episode: 4204, ep_reward: 132.198, ep_step: 1000 2023-05-30 02:10:48 - SimpleLog - INFO: - episode: 4205, ep_reward: 105.364, ep_step: 1000 2023-05-30 02:10:51 - SimpleLog - INFO: - episode: 4206, ep_reward: 120.661, ep_step: 1000 2023-05-30 02:10:53 - SimpleLog - INFO: - episode: 4207, ep_reward: 112.059, ep_step: 1000 2023-05-30 02:10:55 - SimpleLog - INFO: - episode: 4208, ep_reward: 120.306, ep_step: 1000 2023-05-30 02:10:57 - SimpleLog - INFO: - episode: 4209, ep_reward: 120.971, ep_step: 1000 2023-05-30 02:10:59 - SimpleLog - INFO: - episode: 4210, ep_reward: -116.034, ep_step: 607 2023-05-30 02:11:01 - SimpleLog - INFO: - episode: 4211, ep_reward: 122.316, ep_step: 1000 2023-05-30 02:11:03 - SimpleLog - INFO: - episode: 4212, ep_reward: 109.986, ep_step: 1000 2023-05-30 02:11:05 - SimpleLog - INFO: - episode: 4213, ep_reward: -32.900, ep_step: 599 2023-05-30 02:11:07 - SimpleLog - INFO: - episode: 4214, ep_reward: 10.915, ep_step: 893 2023-05-30 02:11:09 - SimpleLog - INFO: - episode: 4215, ep_reward: 130.857, ep_step: 1000 2023-05-30 02:11:11 - SimpleLog - INFO: - episode: 4216, ep_reward: 114.027, ep_step: 1000 2023-05-30 02:11:13 - SimpleLog - INFO: - episode: 4217, ep_reward: 75.491, ep_step: 1000 2023-05-30 02:11:16 - SimpleLog - INFO: - episode: 4218, ep_reward: 87.366, ep_step: 1000 2023-05-30 02:11:18 - SimpleLog - INFO: - episode: 4219, ep_reward: 106.066, ep_step: 1000 2023-05-30 02:11:20 - SimpleLog - INFO: - episode: 4220, ep_reward: 136.460, ep_step: 1000 2023-05-30 02:11:22 - SimpleLog - INFO: - episode: 4221, ep_reward: 135.191, ep_step: 1000 2023-05-30 02:11:25 - SimpleLog - INFO: - episode: 4222, ep_reward: 106.547, ep_step: 1000 2023-05-30 02:11:26 - SimpleLog - INFO: - episode: 4223, ep_reward: -79.395, ep_step: 557 2023-05-30 02:11:28 - SimpleLog - INFO: - episode: 4224, ep_reward: 96.289, ep_step: 1000 2023-05-30 02:11:30 - SimpleLog - INFO: - episode: 4225, ep_reward: 90.443, ep_step: 1000 2023-05-30 02:11:32 - SimpleLog - INFO: - episode: 4226, ep_reward: -91.229, ep_step: 493 2023-05-30 02:11:33 - SimpleLog - INFO: - episode: 4227, ep_reward: -56.214, ep_step: 454 2023-05-30 02:11:35 - SimpleLog - INFO: - episode: 4228, ep_reward: 129.397, ep_step: 1000 2023-05-30 02:11:37 - SimpleLog - INFO: - episode: 4229, ep_reward: 111.686, ep_step: 1000 2023-05-30 02:11:37 - SimpleLog - INFO: - episode: 4230, ep_reward: -112.282, ep_step: 167 2023-05-30 02:11:40 - SimpleLog - INFO: - episode: 4231, ep_reward: 84.681, ep_step: 1000 2023-05-30 02:11:42 - SimpleLog - INFO: - episode: 4232, ep_reward: 138.727, ep_step: 1000 2023-05-30 02:11:44 - SimpleLog - INFO: - episode: 4233, ep_reward: 93.215, ep_step: 1000 2023-05-30 02:11:46 - SimpleLog - INFO: - episode: 4234, ep_reward: 88.014, ep_step: 1000 2023-05-30 02:11:49 - SimpleLog - INFO: - episode: 4235, ep_reward: 87.511, ep_step: 1000 2023-05-30 02:11:50 - SimpleLog - INFO: - episode: 4236, ep_reward: -5.005, ep_step: 687 2023-05-30 02:11:53 - SimpleLog - INFO: - episode: 4237, ep_reward: 148.439, ep_step: 1000 2023-05-30 02:11:55 - SimpleLog - INFO: - episode: 4238, ep_reward: 84.548, ep_step: 1000 2023-05-30 02:11:57 - SimpleLog - INFO: - episode: 4239, ep_reward: 119.698, ep_step: 1000 2023-05-30 02:11:59 - SimpleLog - INFO: - episode: 4240, ep_reward: 5.879, ep_step: 1000 2023-05-30 02:12:02 - SimpleLog - INFO: - episode: 4241, ep_reward: 126.407, ep_step: 1000 2023-05-30 02:12:04 - SimpleLog - INFO: - episode: 4242, ep_reward: 86.229, ep_step: 1000 2023-05-30 02:12:06 - SimpleLog - INFO: - episode: 4243, ep_reward: 88.451, ep_step: 1000 2023-05-30 02:12:08 - SimpleLog - INFO: - episode: 4244, ep_reward: 145.728, ep_step: 1000 2023-05-30 02:12:11 - SimpleLog - INFO: - episode: 4245, ep_reward: 90.211, ep_step: 1000 2023-05-30 02:12:13 - SimpleLog - INFO: - episode: 4246, ep_reward: 0.899, ep_step: 980 2023-05-30 02:12:17 - SimpleLog - INFO: - update_step: 3220000, online_eval_reward: 80.039 2023-05-30 02:12:17 - SimpleLog - INFO: - episode: 4247, ep_reward: 9.459, ep_step: 1000 2023-05-30 02:12:19 - SimpleLog - INFO: - episode: 4248, ep_reward: 3.747, ep_step: 1000 2023-05-30 02:12:21 - SimpleLog - INFO: - episode: 4249, ep_reward: 97.535, ep_step: 1000 2023-05-30 02:12:24 - SimpleLog - INFO: - episode: 4250, ep_reward: 103.388, ep_step: 1000 2023-05-30 02:12:26 - SimpleLog - INFO: - episode: 4251, ep_reward: 139.543, ep_step: 1000 2023-05-30 02:12:28 - SimpleLog - INFO: - episode: 4252, ep_reward: 142.091, ep_step: 1000 2023-05-30 02:12:30 - SimpleLog - INFO: - episode: 4253, ep_reward: 110.662, ep_step: 1000 2023-05-30 02:12:33 - SimpleLog - INFO: - episode: 4254, ep_reward: 124.859, ep_step: 1000 2023-05-30 02:12:35 - SimpleLog - INFO: - episode: 4255, ep_reward: 104.747, ep_step: 1000 2023-05-30 02:12:37 - SimpleLog - INFO: - episode: 4256, ep_reward: 5.922, ep_step: 911 2023-05-30 02:12:39 - SimpleLog - INFO: - episode: 4257, ep_reward: 139.819, ep_step: 1000 2023-05-30 02:12:41 - SimpleLog - INFO: - episode: 4258, ep_reward: 125.071, ep_step: 1000 2023-05-30 02:12:43 - SimpleLog - INFO: - episode: 4259, ep_reward: -22.004, ep_step: 654 2023-05-30 02:12:45 - SimpleLog - INFO: - episode: 4260, ep_reward: 128.356, ep_step: 1000 2023-05-30 02:12:47 - SimpleLog - INFO: - episode: 4261, ep_reward: 8.673, ep_step: 957 2023-05-30 02:12:49 - SimpleLog - INFO: - episode: 4262, ep_reward: -33.379, ep_step: 891 2023-05-30 02:12:51 - SimpleLog - INFO: - episode: 4263, ep_reward: 2.195, ep_step: 933 2023-05-30 02:12:53 - SimpleLog - INFO: - episode: 4264, ep_reward: -29.145, ep_step: 860 2023-05-30 02:12:56 - SimpleLog - INFO: - episode: 4265, ep_reward: 92.738, ep_step: 1000 2023-05-30 02:12:57 - SimpleLog - INFO: - episode: 4266, ep_reward: -72.569, ep_step: 533 2023-05-30 02:12:59 - SimpleLog - INFO: - episode: 4267, ep_reward: 10.871, ep_step: 980 2023-05-30 02:13:01 - SimpleLog - INFO: - episode: 4268, ep_reward: 124.952, ep_step: 1000 2023-05-30 02:13:03 - SimpleLog - INFO: - episode: 4269, ep_reward: -8.296, ep_step: 899 2023-05-30 02:13:05 - SimpleLog - INFO: - episode: 4270, ep_reward: 112.783, ep_step: 1000 2023-05-30 02:13:08 - SimpleLog - INFO: - episode: 4271, ep_reward: 109.741, ep_step: 1000 2023-05-30 02:13:10 - SimpleLog - INFO: - episode: 4272, ep_reward: 126.223, ep_step: 1000 2023-05-30 02:13:12 - SimpleLog - INFO: - episode: 4273, ep_reward: 122.724, ep_step: 1000 2023-05-30 02:13:14 - SimpleLog - INFO: - episode: 4274, ep_reward: 102.306, ep_step: 1000 2023-05-30 02:13:16 - SimpleLog - INFO: - episode: 4275, ep_reward: -2.776, ep_step: 789 2023-05-30 02:13:18 - SimpleLog - INFO: - episode: 4276, ep_reward: -23.366, ep_step: 794 2023-05-30 02:13:20 - SimpleLog - INFO: - episode: 4277, ep_reward: 129.371, ep_step: 1000 2023-05-30 02:13:22 - SimpleLog - INFO: - episode: 4278, ep_reward: 105.705, ep_step: 1000 2023-05-30 02:13:25 - SimpleLog - INFO: - episode: 4279, ep_reward: 100.199, ep_step: 1000 2023-05-30 02:13:26 - SimpleLog - INFO: - episode: 4280, ep_reward: -20.457, ep_step: 743 2023-05-30 02:13:28 - SimpleLog - INFO: - episode: 4281, ep_reward: -34.931, ep_step: 777 2023-05-30 02:13:30 - SimpleLog - INFO: - episode: 4282, ep_reward: -31.976, ep_step: 654 2023-05-30 02:13:31 - SimpleLog - INFO: - episode: 4283, ep_reward: -20.606, ep_step: 688 2023-05-30 02:13:33 - SimpleLog - INFO: - episode: 4284, ep_reward: 111.498, ep_step: 1000 2023-05-30 02:13:36 - SimpleLog - INFO: - episode: 4285, ep_reward: 6.584, ep_step: 964 2023-05-30 02:13:38 - SimpleLog - INFO: - episode: 4286, ep_reward: 7.082, ep_step: 942 2023-05-30 02:13:40 - SimpleLog - INFO: - episode: 4287, ep_reward: 129.232, ep_step: 1000 2023-05-30 02:13:42 - SimpleLog - INFO: - episode: 4288, ep_reward: 40.239, ep_step: 1000 2023-05-30 02:13:44 - SimpleLog - INFO: - episode: 4289, ep_reward: 138.735, ep_step: 1000 2023-05-30 02:13:45 - SimpleLog - INFO: - episode: 4290, ep_reward: -48.659, ep_step: 418 2023-05-30 02:13:48 - SimpleLog - INFO: - episode: 4291, ep_reward: 116.565, ep_step: 1000 2023-05-30 02:13:50 - SimpleLog - INFO: - episode: 4292, ep_reward: 134.765, ep_step: 1000 2023-05-30 02:13:52 - SimpleLog - INFO: - episode: 4293, ep_reward: 88.456, ep_step: 1000 2023-05-30 02:13:54 - SimpleLog - INFO: - episode: 4294, ep_reward: 71.787, ep_step: 1000 2023-05-30 02:13:57 - SimpleLog - INFO: - episode: 4295, ep_reward: 107.071, ep_step: 1000 2023-05-30 02:13:58 - SimpleLog - INFO: - episode: 4296, ep_reward: -12.715, ep_step: 771 2023-05-30 02:14:01 - SimpleLog - INFO: - episode: 4297, ep_reward: 120.603, ep_step: 1000 2023-05-30 02:14:03 - SimpleLog - INFO: - episode: 4298, ep_reward: 142.175, ep_step: 1000 2023-05-30 02:14:05 - SimpleLog - INFO: - episode: 4299, ep_reward: 7.832, ep_step: 923 2023-05-30 02:14:07 - SimpleLog - INFO: - episode: 4300, ep_reward: 125.205, ep_step: 1000 2023-05-30 02:14:09 - SimpleLog - INFO: - episode: 4301, ep_reward: 114.802, ep_step: 1000 2023-05-30 02:14:11 - SimpleLog - INFO: - episode: 4302, ep_reward: -6.587, ep_step: 688 2023-05-30 02:14:13 - SimpleLog - INFO: - episode: 4303, ep_reward: 112.548, ep_step: 1000 2023-05-30 02:14:15 - SimpleLog - INFO: - episode: 4304, ep_reward: 113.776, ep_step: 1000 2023-05-30 02:14:17 - SimpleLog - INFO: - episode: 4305, ep_reward: -33.280, ep_step: 601 2023-05-30 02:14:19 - SimpleLog - INFO: - episode: 4306, ep_reward: 124.175, ep_step: 1000 2023-05-30 02:14:21 - SimpleLog - INFO: - episode: 4307, ep_reward: -6.697, ep_step: 684 2023-05-30 02:14:22 - SimpleLog - INFO: - episode: 4308, ep_reward: 12.163, ep_step: 706 2023-05-30 02:14:24 - SimpleLog - INFO: - episode: 4309, ep_reward: 128.768, ep_step: 1000 2023-05-30 02:14:27 - SimpleLog - INFO: - episode: 4310, ep_reward: 120.234, ep_step: 1000 2023-05-30 02:14:29 - SimpleLog - INFO: - episode: 4311, ep_reward: 110.655, ep_step: 1000 2023-05-30 02:14:31 - SimpleLog - INFO: - episode: 4312, ep_reward: 124.967, ep_step: 1000 2023-05-30 02:14:33 - SimpleLog - INFO: - episode: 4313, ep_reward: 126.096, ep_step: 1000 2023-05-30 02:14:36 - SimpleLog - INFO: - episode: 4314, ep_reward: 122.519, ep_step: 1000 2023-05-30 02:14:38 - SimpleLog - INFO: - episode: 4315, ep_reward: 138.248, ep_step: 1000 2023-05-30 02:14:40 - SimpleLog - INFO: - episode: 4316, ep_reward: 13.763, ep_step: 1000 2023-05-30 02:14:42 - SimpleLog - INFO: - episode: 4317, ep_reward: 134.345, ep_step: 1000 2023-05-30 02:14:45 - SimpleLog - INFO: - episode: 4318, ep_reward: 117.995, ep_step: 1000 2023-05-30 02:14:47 - SimpleLog - INFO: - episode: 4319, ep_reward: 128.356, ep_step: 1000 2023-05-30 02:14:49 - SimpleLog - INFO: - episode: 4320, ep_reward: 114.479, ep_step: 1000 2023-05-30 02:14:50 - SimpleLog - INFO: - episode: 4321, ep_reward: -59.929, ep_step: 532 2023-05-30 02:14:53 - SimpleLog - INFO: - episode: 4322, ep_reward: 140.166, ep_step: 1000 2023-05-30 02:14:56 - SimpleLog - INFO: - update_step: 3290000, online_eval_reward: 112.966 2023-05-30 02:14:57 - SimpleLog - INFO: - episode: 4323, ep_reward: 14.735, ep_step: 1000 2023-05-30 02:14:59 - SimpleLog - INFO: - episode: 4324, ep_reward: 110.886, ep_step: 1000 2023-05-30 02:15:01 - SimpleLog - INFO: - episode: 4325, ep_reward: 8.629, ep_step: 952 2023-05-30 02:15:03 - SimpleLog - INFO: - episode: 4326, ep_reward: 127.121, ep_step: 1000 2023-05-30 02:15:06 - SimpleLog - INFO: - episode: 4327, ep_reward: 140.116, ep_step: 1000 2023-05-30 02:15:08 - SimpleLog - INFO: - episode: 4328, ep_reward: 111.407, ep_step: 1000 2023-05-30 02:15:10 - SimpleLog - INFO: - episode: 4329, ep_reward: 86.681, ep_step: 1000 2023-05-30 02:15:11 - SimpleLog - INFO: - episode: 4330, ep_reward: -27.760, ep_step: 617 2023-05-30 02:15:14 - SimpleLog - INFO: - episode: 4331, ep_reward: 85.970, ep_step: 1000 2023-05-30 02:15:16 - SimpleLog - INFO: - episode: 4332, ep_reward: 116.386, ep_step: 1000 2023-05-30 02:15:18 - SimpleLog - INFO: - episode: 4333, ep_reward: 109.398, ep_step: 1000 2023-05-30 02:15:20 - SimpleLog - INFO: - episode: 4334, ep_reward: 119.663, ep_step: 1000 2023-05-30 02:15:23 - SimpleLog - INFO: - episode: 4335, ep_reward: 138.539, ep_step: 1000 2023-05-30 02:15:24 - SimpleLog - INFO: - episode: 4336, ep_reward: -65.180, ep_step: 495 2023-05-30 02:15:26 - SimpleLog - INFO: - episode: 4337, ep_reward: 113.909, ep_step: 1000 2023-05-30 02:15:28 - SimpleLog - INFO: - episode: 4338, ep_reward: 115.426, ep_step: 1000 2023-05-30 02:15:30 - SimpleLog - INFO: - episode: 4339, ep_reward: 114.209, ep_step: 1000 2023-05-30 02:15:33 - SimpleLog - INFO: - episode: 4340, ep_reward: 126.389, ep_step: 1000 2023-05-30 02:15:35 - SimpleLog - INFO: - episode: 4341, ep_reward: 113.333, ep_step: 1000 2023-05-30 02:15:37 - SimpleLog - INFO: - episode: 4342, ep_reward: -27.914, ep_step: 754 2023-05-30 02:15:39 - SimpleLog - INFO: - episode: 4343, ep_reward: 84.779, ep_step: 1000 2023-05-30 02:15:40 - SimpleLog - INFO: - episode: 4344, ep_reward: -30.577, ep_step: 532 2023-05-30 02:15:42 - SimpleLog - INFO: - episode: 4345, ep_reward: 88.631, ep_step: 1000 2023-05-30 02:15:45 - SimpleLog - INFO: - episode: 4346, ep_reward: 77.526, ep_step: 1000 2023-05-30 02:15:47 - SimpleLog - INFO: - episode: 4347, ep_reward: 90.638, ep_step: 1000 2023-05-30 02:15:49 - SimpleLog - INFO: - episode: 4348, ep_reward: 119.537, ep_step: 1000 2023-05-30 02:15:51 - SimpleLog - INFO: - episode: 4349, ep_reward: 125.105, ep_step: 1000 2023-05-30 02:15:54 - SimpleLog - INFO: - episode: 4350, ep_reward: 101.929, ep_step: 1000 2023-05-30 02:15:56 - SimpleLog - INFO: - episode: 4351, ep_reward: 145.008, ep_step: 1000 2023-05-30 02:15:58 - SimpleLog - INFO: - episode: 4352, ep_reward: 6.084, ep_step: 1000 2023-05-30 02:16:00 - SimpleLog - INFO: - episode: 4353, ep_reward: 125.832, ep_step: 1000 2023-05-30 02:16:03 - SimpleLog - INFO: - episode: 4354, ep_reward: 80.504, ep_step: 1000 2023-05-30 02:16:05 - SimpleLog - INFO: - episode: 4355, ep_reward: 118.311, ep_step: 1000 2023-05-30 02:16:07 - SimpleLog - INFO: - episode: 4356, ep_reward: 14.572, ep_step: 902 2023-05-30 02:16:09 - SimpleLog - INFO: - episode: 4357, ep_reward: 16.384, ep_step: 809 2023-05-30 02:16:11 - SimpleLog - INFO: - episode: 4358, ep_reward: 115.753, ep_step: 1000 2023-05-30 02:16:13 - SimpleLog - INFO: - episode: 4359, ep_reward: 132.725, ep_step: 1000 2023-05-30 02:16:15 - SimpleLog - INFO: - episode: 4360, ep_reward: 136.724, ep_step: 1000 2023-05-30 02:16:17 - SimpleLog - INFO: - episode: 4361, ep_reward: 1.146, ep_step: 821 2023-05-30 02:16:19 - SimpleLog - INFO: - episode: 4362, ep_reward: 139.190, ep_step: 1000 2023-05-30 02:16:22 - SimpleLog - INFO: - episode: 4363, ep_reward: 137.439, ep_step: 1000 2023-05-30 02:16:24 - SimpleLog - INFO: - episode: 4364, ep_reward: 3.482, ep_step: 802 2023-05-30 02:16:26 - SimpleLog - INFO: - episode: 4365, ep_reward: 140.735, ep_step: 1000 2023-05-30 02:16:28 - SimpleLog - INFO: - episode: 4366, ep_reward: 129.842, ep_step: 1000 2023-05-30 02:16:30 - SimpleLog - INFO: - episode: 4367, ep_reward: 151.775, ep_step: 1000 2023-05-30 02:16:32 - SimpleLog - INFO: - episode: 4368, ep_reward: 5.944, ep_step: 766 2023-05-30 02:16:34 - SimpleLog - INFO: - episode: 4369, ep_reward: 127.311, ep_step: 1000 2023-05-30 02:16:36 - SimpleLog - INFO: - episode: 4370, ep_reward: -22.302, ep_step: 586 2023-05-30 02:16:38 - SimpleLog - INFO: - episode: 4371, ep_reward: 132.204, ep_step: 1000 2023-05-30 02:16:40 - SimpleLog - INFO: - episode: 4372, ep_reward: 141.979, ep_step: 1000 2023-05-30 02:16:42 - SimpleLog - INFO: - episode: 4373, ep_reward: 139.402, ep_step: 1000 2023-05-30 02:16:45 - SimpleLog - INFO: - episode: 4374, ep_reward: 13.643, ep_step: 1000 2023-05-30 02:16:47 - SimpleLog - INFO: - episode: 4375, ep_reward: 139.001, ep_step: 1000 2023-05-30 02:16:49 - SimpleLog - INFO: - episode: 4376, ep_reward: 4.027, ep_step: 943 2023-05-30 02:16:51 - SimpleLog - INFO: - episode: 4377, ep_reward: 113.846, ep_step: 1000 2023-05-30 02:16:53 - SimpleLog - INFO: - episode: 4378, ep_reward: -16.441, ep_step: 854 2023-05-30 02:16:55 - SimpleLog - INFO: - episode: 4379, ep_reward: 7.396, ep_step: 1000 2023-05-30 02:16:58 - SimpleLog - INFO: - episode: 4380, ep_reward: 125.121, ep_step: 1000 2023-05-30 02:17:00 - SimpleLog - INFO: - episode: 4381, ep_reward: 109.164, ep_step: 1000 2023-05-30 02:17:02 - SimpleLog - INFO: - episode: 4382, ep_reward: 12.428, ep_step: 850 2023-05-30 02:17:03 - SimpleLog - INFO: - episode: 4383, ep_reward: 1.676, ep_step: 706 2023-05-30 02:17:05 - SimpleLog - INFO: - episode: 4384, ep_reward: 9.525, ep_step: 915 2023-05-30 02:17:08 - SimpleLog - INFO: - episode: 4385, ep_reward: 129.872, ep_step: 1000 2023-05-30 02:17:10 - SimpleLog - INFO: - episode: 4386, ep_reward: 119.629, ep_step: 1000 2023-05-30 02:17:12 - SimpleLog - INFO: - episode: 4387, ep_reward: 5.753, ep_step: 897 2023-05-30 02:17:14 - SimpleLog - INFO: - episode: 4388, ep_reward: 109.105, ep_step: 1000 2023-05-30 02:17:16 - SimpleLog - INFO: - episode: 4389, ep_reward: -5.145, ep_step: 979 2023-05-30 02:17:19 - SimpleLog - INFO: - episode: 4390, ep_reward: 99.099, ep_step: 1000 2023-05-30 02:17:21 - SimpleLog - INFO: - episode: 4391, ep_reward: 27.974, ep_step: 950 2023-05-30 02:17:23 - SimpleLog - INFO: - episode: 4392, ep_reward: 83.326, ep_step: 1000 2023-05-30 02:17:25 - SimpleLog - INFO: - episode: 4393, ep_reward: 151.674, ep_step: 1000 2023-05-30 02:17:28 - SimpleLog - INFO: - episode: 4394, ep_reward: 82.092, ep_step: 1000 2023-05-30 02:17:29 - SimpleLog - INFO: - episode: 4395, ep_reward: 28.581, ep_step: 802 2023-05-30 02:17:32 - SimpleLog - INFO: - episode: 4396, ep_reward: 133.444, ep_step: 1000 2023-05-30 02:17:35 - SimpleLog - INFO: - update_step: 3360000, online_eval_reward: 158.401 2023-05-30 02:17:36 - SimpleLog - INFO: - episode: 4397, ep_reward: 140.990, ep_step: 1000 2023-05-30 02:17:37 - SimpleLog - INFO: - episode: 4398, ep_reward: -56.665, ep_step: 396 2023-05-30 02:17:39 - SimpleLog - INFO: - episode: 4399, ep_reward: 149.175, ep_step: 1000 2023-05-30 02:17:41 - SimpleLog - INFO: - episode: 4400, ep_reward: 135.715, ep_step: 1000 2023-05-30 02:17:43 - SimpleLog - INFO: - episode: 4401, ep_reward: 145.598, ep_step: 1000 2023-05-30 02:17:46 - SimpleLog - INFO: - episode: 4402, ep_reward: 162.137, ep_step: 1000 2023-05-30 02:17:47 - SimpleLog - INFO: - episode: 4403, ep_reward: -47.757, ep_step: 489 2023-05-30 02:17:49 - SimpleLog - INFO: - episode: 4404, ep_reward: 140.695, ep_step: 1000 2023-05-30 02:17:51 - SimpleLog - INFO: - episode: 4405, ep_reward: 140.186, ep_step: 1000 2023-05-30 02:17:53 - SimpleLog - INFO: - episode: 4406, ep_reward: 22.109, ep_step: 985 2023-05-30 02:17:56 - SimpleLog - INFO: - episode: 4407, ep_reward: 128.306, ep_step: 1000 2023-05-30 02:17:58 - SimpleLog - INFO: - episode: 4408, ep_reward: 139.245, ep_step: 1000 2023-05-30 02:18:00 - SimpleLog - INFO: - episode: 4409, ep_reward: 135.669, ep_step: 1000 2023-05-30 02:18:02 - SimpleLog - INFO: - episode: 4410, ep_reward: -0.927, ep_step: 961 2023-05-30 02:18:05 - SimpleLog - INFO: - episode: 4411, ep_reward: 87.127, ep_step: 1000 2023-05-30 02:18:07 - SimpleLog - INFO: - episode: 4412, ep_reward: 137.512, ep_step: 1000 2023-05-30 02:18:09 - SimpleLog - INFO: - episode: 4413, ep_reward: 12.426, ep_step: 1000 2023-05-30 02:18:11 - SimpleLog - INFO: - episode: 4414, ep_reward: 114.220, ep_step: 1000 2023-05-30 02:18:14 - SimpleLog - INFO: - episode: 4415, ep_reward: 136.366, ep_step: 1000 2023-05-30 02:18:16 - SimpleLog - INFO: - episode: 4416, ep_reward: 137.604, ep_step: 1000 2023-05-30 02:18:18 - SimpleLog - INFO: - episode: 4417, ep_reward: 101.354, ep_step: 1000 2023-05-30 02:18:20 - SimpleLog - INFO: - episode: 4418, ep_reward: 80.363, ep_step: 1000 2023-05-30 02:18:23 - SimpleLog - INFO: - episode: 4419, ep_reward: 136.705, ep_step: 1000 2023-05-30 02:18:25 - SimpleLog - INFO: - episode: 4420, ep_reward: 82.605, ep_step: 1000 2023-05-30 02:18:27 - SimpleLog - INFO: - episode: 4421, ep_reward: 15.208, ep_step: 1000 2023-05-30 02:18:30 - SimpleLog - INFO: - episode: 4422, ep_reward: 148.066, ep_step: 1000 2023-05-30 02:18:31 - SimpleLog - INFO: - episode: 4423, ep_reward: 12.834, ep_step: 789 2023-05-30 02:18:34 - SimpleLog - INFO: - episode: 4424, ep_reward: 164.046, ep_step: 1000 2023-05-30 02:18:34 - SimpleLog - INFO: - episode: 4425, ep_reward: -70.450, ep_step: 321 2023-05-30 02:18:37 - SimpleLog - INFO: - episode: 4426, ep_reward: 164.938, ep_step: 1000 2023-05-30 02:18:39 - SimpleLog - INFO: - episode: 4427, ep_reward: 79.102, ep_step: 1000 2023-05-30 02:18:41 - SimpleLog - INFO: - episode: 4428, ep_reward: 130.738, ep_step: 1000 2023-05-30 02:18:43 - SimpleLog - INFO: - episode: 4429, ep_reward: 82.520, ep_step: 1000 2023-05-30 02:18:46 - SimpleLog - INFO: - episode: 4430, ep_reward: 150.459, ep_step: 1000 2023-05-30 02:18:48 - SimpleLog - INFO: - episode: 4431, ep_reward: 165.746, ep_step: 1000 2023-05-30 02:18:50 - SimpleLog - INFO: - episode: 4432, ep_reward: 80.090, ep_step: 1000 2023-05-30 02:18:52 - SimpleLog - INFO: - episode: 4433, ep_reward: 134.865, ep_step: 1000 2023-05-30 02:18:55 - SimpleLog - INFO: - episode: 4434, ep_reward: 9.882, ep_step: 1000 2023-05-30 02:18:57 - SimpleLog - INFO: - episode: 4435, ep_reward: 7.745, ep_step: 1000 2023-05-30 02:18:59 - SimpleLog - INFO: - episode: 4436, ep_reward: 177.916, ep_step: 1000 2023-05-30 02:19:01 - SimpleLog - INFO: - episode: 4437, ep_reward: 139.522, ep_step: 1000 2023-05-30 02:19:04 - SimpleLog - INFO: - episode: 4438, ep_reward: 165.745, ep_step: 1000 2023-05-30 02:19:05 - SimpleLog - INFO: - episode: 4439, ep_reward: -38.178, ep_step: 509 2023-05-30 02:19:07 - SimpleLog - INFO: - episode: 4440, ep_reward: 166.176, ep_step: 1000 2023-05-30 02:19:09 - SimpleLog - INFO: - episode: 4441, ep_reward: 122.434, ep_step: 1000 2023-05-30 02:19:11 - SimpleLog - INFO: - episode: 4442, ep_reward: 124.279, ep_step: 1000 2023-05-30 02:19:14 - SimpleLog - INFO: - episode: 4443, ep_reward: 148.086, ep_step: 1000 2023-05-30 02:19:16 - SimpleLog - INFO: - episode: 4444, ep_reward: 84.474, ep_step: 1000 2023-05-30 02:19:18 - SimpleLog - INFO: - episode: 4445, ep_reward: 150.325, ep_step: 1000 2023-05-30 02:19:20 - SimpleLog - INFO: - episode: 4446, ep_reward: 19.819, ep_step: 782 2023-05-30 02:19:22 - SimpleLog - INFO: - episode: 4447, ep_reward: 179.108, ep_step: 1000 2023-05-30 02:19:25 - SimpleLog - INFO: - episode: 4448, ep_reward: 164.154, ep_step: 1000 2023-05-30 02:19:27 - SimpleLog - INFO: - episode: 4449, ep_reward: 145.650, ep_step: 1000 2023-05-30 02:19:29 - SimpleLog - INFO: - episode: 4450, ep_reward: 6.602, ep_step: 857 2023-05-30 02:19:31 - SimpleLog - INFO: - episode: 4451, ep_reward: 162.023, ep_step: 1000 2023-05-30 02:19:33 - SimpleLog - INFO: - episode: 4452, ep_reward: 157.364, ep_step: 1000 2023-05-30 02:19:35 - SimpleLog - INFO: - episode: 4453, ep_reward: 10.012, ep_step: 616 2023-05-30 02:19:36 - SimpleLog - INFO: - episode: 4454, ep_reward: -27.860, ep_step: 449 2023-05-30 02:19:37 - SimpleLog - INFO: - episode: 4455, ep_reward: 23.545, ep_step: 759 2023-05-30 02:19:39 - SimpleLog - INFO: - episode: 4456, ep_reward: 16.921, ep_step: 807 2023-05-30 02:19:40 - SimpleLog - INFO: - episode: 4457, ep_reward: -28.923, ep_step: 536 2023-05-30 02:19:42 - SimpleLog - INFO: - episode: 4458, ep_reward: 8.913, ep_step: 871 2023-05-30 02:19:45 - SimpleLog - INFO: - episode: 4459, ep_reward: 143.018, ep_step: 1000 2023-05-30 02:19:47 - SimpleLog - INFO: - episode: 4460, ep_reward: 5.350, ep_step: 950 2023-05-30 02:19:48 - SimpleLog - INFO: - episode: 4461, ep_reward: -77.509, ep_step: 638 2023-05-30 02:19:50 - SimpleLog - INFO: - episode: 4462, ep_reward: 116.780, ep_step: 1000 2023-05-30 02:19:53 - SimpleLog - INFO: - episode: 4463, ep_reward: 148.145, ep_step: 1000 2023-05-30 02:19:55 - SimpleLog - INFO: - episode: 4464, ep_reward: -9.546, ep_step: 936 2023-05-30 02:19:57 - SimpleLog - INFO: - episode: 4465, ep_reward: 151.226, ep_step: 1000 2023-05-30 02:19:59 - SimpleLog - INFO: - episode: 4466, ep_reward: 169.796, ep_step: 1000 2023-05-30 02:20:01 - SimpleLog - INFO: - episode: 4467, ep_reward: 163.327, ep_step: 1000 2023-05-30 02:20:04 - SimpleLog - INFO: - episode: 4468, ep_reward: 143.692, ep_step: 1000 2023-05-30 02:20:06 - SimpleLog - INFO: - episode: 4469, ep_reward: 138.596, ep_step: 1000 2023-05-30 02:20:08 - SimpleLog - INFO: - episode: 4470, ep_reward: 134.604, ep_step: 1000 2023-05-30 02:20:10 - SimpleLog - INFO: - episode: 4471, ep_reward: 167.546, ep_step: 1000 2023-05-30 02:20:13 - SimpleLog - INFO: - episode: 4472, ep_reward: 152.736, ep_step: 1000 2023-05-30 02:20:15 - SimpleLog - INFO: - update_step: 3430000, online_eval_reward: 100.704 2023-05-30 02:20:17 - SimpleLog - INFO: - episode: 4473, ep_reward: 109.003, ep_step: 1000 2023-05-30 02:20:19 - SimpleLog - INFO: - episode: 4474, ep_reward: 142.991, ep_step: 1000 2023-05-30 02:20:21 - SimpleLog - INFO: - episode: 4475, ep_reward: 23.605, ep_step: 742 2023-05-30 02:20:22 - SimpleLog - INFO: - episode: 4476, ep_reward: -0.401, ep_step: 647 2023-05-30 02:20:24 - SimpleLog - INFO: - episode: 4477, ep_reward: 149.109, ep_step: 1000 2023-05-30 02:20:27 - SimpleLog - INFO: - episode: 4478, ep_reward: 142.547, ep_step: 1000 2023-05-30 02:20:29 - SimpleLog - INFO: - episode: 4479, ep_reward: 172.307, ep_step: 1000 2023-05-30 02:20:31 - SimpleLog - INFO: - episode: 4480, ep_reward: 164.213, ep_step: 1000 2023-05-30 02:20:33 - SimpleLog - INFO: - episode: 4481, ep_reward: 164.576, ep_step: 1000 2023-05-30 02:20:36 - SimpleLog - INFO: - episode: 4482, ep_reward: -28.661, ep_step: 950 2023-05-30 02:20:37 - SimpleLog - INFO: - episode: 4483, ep_reward: -31.488, ep_step: 475 2023-05-30 02:20:39 - SimpleLog - INFO: - episode: 4484, ep_reward: 154.421, ep_step: 1000 2023-05-30 02:20:41 - SimpleLog - INFO: - episode: 4485, ep_reward: 85.817, ep_step: 1000 2023-05-30 02:20:43 - SimpleLog - INFO: - episode: 4486, ep_reward: 8.393, ep_step: 659 2023-05-30 02:20:45 - SimpleLog - INFO: - episode: 4487, ep_reward: 152.260, ep_step: 1000 2023-05-30 02:20:47 - SimpleLog - INFO: - episode: 4488, ep_reward: 90.612, ep_step: 1000 2023-05-30 02:20:49 - SimpleLog - INFO: - episode: 4489, ep_reward: 111.664, ep_step: 1000 2023-05-30 02:20:51 - SimpleLog - INFO: - episode: 4490, ep_reward: -29.140, ep_step: 932 2023-05-30 02:20:53 - SimpleLog - INFO: - episode: 4491, ep_reward: 10.569, ep_step: 692 2023-05-30 02:20:55 - SimpleLog - INFO: - episode: 4492, ep_reward: 155.404, ep_step: 1000 2023-05-30 02:20:56 - SimpleLog - INFO: - episode: 4493, ep_reward: -78.406, ep_step: 299 2023-05-30 02:20:58 - SimpleLog - INFO: - episode: 4494, ep_reward: 132.132, ep_step: 1000 2023-05-30 02:21:00 - SimpleLog - INFO: - episode: 4495, ep_reward: 141.991, ep_step: 1000 2023-05-30 02:21:03 - SimpleLog - INFO: - episode: 4496, ep_reward: 126.956, ep_step: 1000 2023-05-30 02:21:05 - SimpleLog - INFO: - episode: 4497, ep_reward: 141.041, ep_step: 1000 2023-05-30 02:21:07 - SimpleLog - INFO: - episode: 4498, ep_reward: 132.037, ep_step: 1000 2023-05-30 02:21:09 - SimpleLog - INFO: - episode: 4499, ep_reward: 132.826, ep_step: 1000 2023-05-30 02:21:12 - SimpleLog - INFO: - episode: 4500, ep_reward: 144.411, ep_step: 1000 2023-05-30 02:21:14 - SimpleLog - INFO: - episode: 4501, ep_reward: 107.116, ep_step: 1000 2023-05-30 02:21:16 - SimpleLog - INFO: - episode: 4502, ep_reward: -0.058, ep_step: 817 2023-05-30 02:21:18 - SimpleLog - INFO: - episode: 4503, ep_reward: 139.922, ep_step: 1000 2023-05-30 02:21:20 - SimpleLog - INFO: - episode: 4504, ep_reward: -7.524, ep_step: 815 2023-05-30 02:21:22 - SimpleLog - INFO: - episode: 4505, ep_reward: 151.867, ep_step: 1000 2023-05-30 02:21:24 - SimpleLog - INFO: - episode: 4506, ep_reward: 131.517, ep_step: 1000 2023-05-30 02:21:27 - SimpleLog - INFO: - episode: 4507, ep_reward: 102.885, ep_step: 1000 2023-05-30 02:21:28 - SimpleLog - INFO: - episode: 4508, ep_reward: -0.353, ep_step: 624 2023-05-30 02:21:30 - SimpleLog - INFO: - episode: 4509, ep_reward: 137.136, ep_step: 1000 2023-05-30 02:21:32 - SimpleLog - INFO: - episode: 4510, ep_reward: 140.331, ep_step: 1000 2023-05-30 02:21:35 - SimpleLog - INFO: - episode: 4511, ep_reward: 179.372, ep_step: 1000 2023-05-30 02:21:36 - SimpleLog - INFO: - episode: 4512, ep_reward: 14.660, ep_step: 637 2023-05-30 02:21:37 - SimpleLog - INFO: - episode: 4513, ep_reward: -25.758, ep_step: 528 2023-05-30 02:21:40 - SimpleLog - INFO: - episode: 4514, ep_reward: 160.344, ep_step: 1000 2023-05-30 02:21:42 - SimpleLog - INFO: - episode: 4515, ep_reward: 129.789, ep_step: 1000 2023-05-30 02:21:44 - SimpleLog - INFO: - episode: 4516, ep_reward: 92.513, ep_step: 1000 2023-05-30 02:21:45 - SimpleLog - INFO: - episode: 4517, ep_reward: -0.131, ep_step: 590 2023-05-30 02:21:48 - SimpleLog - INFO: - episode: 4518, ep_reward: 179.881, ep_step: 1000 2023-05-30 02:21:50 - SimpleLog - INFO: - episode: 4519, ep_reward: 136.745, ep_step: 1000 2023-05-30 02:21:52 - SimpleLog - INFO: - episode: 4520, ep_reward: 123.994, ep_step: 1000 2023-05-30 02:21:54 - SimpleLog - INFO: - episode: 4521, ep_reward: 29.441, ep_step: 796 2023-05-30 02:21:56 - SimpleLog - INFO: - episode: 4522, ep_reward: 174.234, ep_step: 1000 2023-05-30 02:21:59 - SimpleLog - INFO: - episode: 4523, ep_reward: 106.891, ep_step: 1000 2023-05-30 02:22:01 - SimpleLog - INFO: - episode: 4524, ep_reward: 156.783, ep_step: 1000 2023-05-30 02:22:02 - SimpleLog - INFO: - episode: 4525, ep_reward: 8.086, ep_step: 751 2023-05-30 02:22:05 - SimpleLog - INFO: - episode: 4526, ep_reward: 129.924, ep_step: 1000 2023-05-30 02:22:07 - SimpleLog - INFO: - episode: 4527, ep_reward: 139.058, ep_step: 1000 2023-05-30 02:22:09 - SimpleLog - INFO: - episode: 4528, ep_reward: 141.150, ep_step: 1000 2023-05-30 02:22:12 - SimpleLog - INFO: - episode: 4529, ep_reward: 152.673, ep_step: 1000 2023-05-30 02:22:14 - SimpleLog - INFO: - episode: 4530, ep_reward: 148.044, ep_step: 1000 2023-05-30 02:22:16 - SimpleLog - INFO: - episode: 4531, ep_reward: 153.098, ep_step: 1000 2023-05-30 02:22:18 - SimpleLog - INFO: - episode: 4532, ep_reward: 140.497, ep_step: 1000 2023-05-30 02:22:20 - SimpleLog - INFO: - episode: 4533, ep_reward: 20.615, ep_step: 858 2023-05-30 02:22:22 - SimpleLog - INFO: - episode: 4534, ep_reward: 7.471, ep_step: 703 2023-05-30 02:22:24 - SimpleLog - INFO: - episode: 4535, ep_reward: 166.216, ep_step: 1000 2023-05-30 02:22:26 - SimpleLog - INFO: - episode: 4536, ep_reward: 131.671, ep_step: 1000 2023-05-30 02:22:28 - SimpleLog - INFO: - episode: 4537, ep_reward: 129.496, ep_step: 1000 2023-05-30 02:22:31 - SimpleLog - INFO: - episode: 4538, ep_reward: 161.890, ep_step: 1000 2023-05-30 02:22:32 - SimpleLog - INFO: - episode: 4539, ep_reward: -23.004, ep_step: 747 2023-05-30 02:22:35 - SimpleLog - INFO: - episode: 4540, ep_reward: 77.990, ep_step: 1000 2023-05-30 02:22:37 - SimpleLog - INFO: - episode: 4541, ep_reward: 129.410, ep_step: 1000 2023-05-30 02:22:39 - SimpleLog - INFO: - episode: 4542, ep_reward: 173.352, ep_step: 1000 2023-05-30 02:22:41 - SimpleLog - INFO: - episode: 4543, ep_reward: 158.431, ep_step: 1000 2023-05-30 02:22:44 - SimpleLog - INFO: - episode: 4544, ep_reward: 177.552, ep_step: 1000 2023-05-30 02:22:46 - SimpleLog - INFO: - episode: 4545, ep_reward: 139.034, ep_step: 1000 2023-05-30 02:22:48 - SimpleLog - INFO: - episode: 4546, ep_reward: 160.610, ep_step: 1000 2023-05-30 02:22:50 - SimpleLog - INFO: - episode: 4547, ep_reward: 187.042, ep_step: 1000 2023-05-30 02:22:54 - SimpleLog - INFO: - update_step: 3500000, online_eval_reward: 146.393 2023-05-30 02:22:55 - SimpleLog - INFO: - episode: 4548, ep_reward: 126.954, ep_step: 1000 2023-05-30 02:22:57 - SimpleLog - INFO: - episode: 4549, ep_reward: 149.518, ep_step: 1000 2023-05-30 02:22:59 - SimpleLog - INFO: - episode: 4550, ep_reward: 150.796, ep_step: 1000 2023-05-30 02:23:01 - SimpleLog - INFO: - episode: 4551, ep_reward: 140.656, ep_step: 1000 2023-05-30 02:23:04 - SimpleLog - INFO: - episode: 4552, ep_reward: 169.710, ep_step: 1000 2023-05-30 02:23:05 - SimpleLog - INFO: - episode: 4553, ep_reward: 1.045, ep_step: 750 2023-05-30 02:23:08 - SimpleLog - INFO: - episode: 4554, ep_reward: 149.721, ep_step: 1000 2023-05-30 02:23:10 - SimpleLog - INFO: - episode: 4555, ep_reward: 147.411, ep_step: 1000 2023-05-30 02:23:12 - SimpleLog - INFO: - episode: 4556, ep_reward: 108.147, ep_step: 1000 2023-05-30 02:23:14 - SimpleLog - INFO: - episode: 4557, ep_reward: 90.704, ep_step: 1000 2023-05-30 02:23:17 - SimpleLog - INFO: - episode: 4558, ep_reward: 136.661, ep_step: 1000 2023-05-30 02:23:19 - SimpleLog - INFO: - episode: 4559, ep_reward: 132.267, ep_step: 1000 2023-05-30 02:23:20 - SimpleLog - INFO: - episode: 4560, ep_reward: -53.042, ep_step: 388 2023-05-30 02:23:22 - SimpleLog - INFO: - episode: 4561, ep_reward: 135.505, ep_step: 1000 2023-05-30 02:23:24 - SimpleLog - INFO: - episode: 4562, ep_reward: 135.008, ep_step: 1000 2023-05-30 02:23:26 - SimpleLog - INFO: - episode: 4563, ep_reward: -2.130, ep_step: 696 2023-05-30 02:23:28 - SimpleLog - INFO: - episode: 4564, ep_reward: 158.406, ep_step: 1000 2023-05-30 02:23:30 - SimpleLog - INFO: - episode: 4565, ep_reward: 81.834, ep_step: 1000 2023-05-30 02:23:32 - SimpleLog - INFO: - episode: 4566, ep_reward: 139.741, ep_step: 1000 2023-05-30 02:23:34 - SimpleLog - INFO: - episode: 4567, ep_reward: 8.699, ep_step: 832 2023-05-30 02:23:37 - SimpleLog - INFO: - episode: 4568, ep_reward: 142.253, ep_step: 1000 2023-05-30 02:23:38 - SimpleLog - INFO: - episode: 4569, ep_reward: 7.832, ep_step: 684 2023-05-30 02:23:40 - SimpleLog - INFO: - episode: 4570, ep_reward: 124.985, ep_step: 1000 2023-05-30 02:23:43 - SimpleLog - INFO: - episode: 4571, ep_reward: 144.083, ep_step: 1000 2023-05-30 02:23:45 - SimpleLog - INFO: - episode: 4572, ep_reward: 142.153, ep_step: 1000 2023-05-30 02:23:47 - SimpleLog - INFO: - episode: 4573, ep_reward: 132.528, ep_step: 1000 2023-05-30 02:23:49 - SimpleLog - INFO: - episode: 4574, ep_reward: 83.473, ep_step: 1000 2023-05-30 02:23:52 - SimpleLog - INFO: - episode: 4575, ep_reward: 86.354, ep_step: 1000 2023-05-30 02:23:54 - SimpleLog - INFO: - episode: 4576, ep_reward: 87.530, ep_step: 1000 2023-05-30 02:23:56 - SimpleLog - INFO: - episode: 4577, ep_reward: 142.075, ep_step: 1000 2023-05-30 02:23:58 - SimpleLog - INFO: - episode: 4578, ep_reward: 127.158, ep_step: 1000 2023-05-30 02:24:01 - SimpleLog - INFO: - episode: 4579, ep_reward: 102.378, ep_step: 1000 2023-05-30 02:24:03 - SimpleLog - INFO: - episode: 4580, ep_reward: 132.841, ep_step: 1000 2023-05-30 02:24:05 - SimpleLog - INFO: - episode: 4581, ep_reward: 108.514, ep_step: 1000 2023-05-30 02:24:07 - SimpleLog - INFO: - episode: 4582, ep_reward: 132.354, ep_step: 1000 2023-05-30 02:24:10 - SimpleLog - INFO: - episode: 4583, ep_reward: 118.154, ep_step: 1000 2023-05-30 02:24:12 - SimpleLog - INFO: - episode: 4584, ep_reward: 131.397, ep_step: 1000 2023-05-30 02:24:14 - SimpleLog - INFO: - episode: 4585, ep_reward: 135.247, ep_step: 1000 2023-05-30 02:24:16 - SimpleLog - INFO: - episode: 4586, ep_reward: 123.894, ep_step: 1000 2023-05-30 02:24:19 - SimpleLog - INFO: - episode: 4587, ep_reward: 116.877, ep_step: 1000 2023-05-30 02:24:21 - SimpleLog - INFO: - episode: 4588, ep_reward: 150.894, ep_step: 1000 2023-05-30 02:24:23 - SimpleLog - INFO: - episode: 4589, ep_reward: 18.327, ep_step: 793 2023-05-30 02:24:25 - SimpleLog - INFO: - episode: 4590, ep_reward: 124.357, ep_step: 1000 2023-05-30 02:24:27 - SimpleLog - INFO: - episode: 4591, ep_reward: 158.228, ep_step: 1000 2023-05-30 02:24:29 - SimpleLog - INFO: - episode: 4592, ep_reward: -20.365, ep_step: 692 2023-05-30 02:24:31 - SimpleLog - INFO: - episode: 4593, ep_reward: 177.365, ep_step: 1000 2023-05-30 02:24:33 - SimpleLog - INFO: - episode: 4594, ep_reward: 157.143, ep_step: 1000 2023-05-30 02:24:35 - SimpleLog - INFO: - episode: 4595, ep_reward: 132.259, ep_step: 1000 2023-05-30 02:24:37 - SimpleLog - INFO: - episode: 4596, ep_reward: -4.484, ep_step: 587 2023-05-30 02:24:37 - SimpleLog - INFO: - episode: 4597, ep_reward: -85.230, ep_step: 174 2023-05-30 02:24:39 - SimpleLog - INFO: - episode: 4598, ep_reward: 125.530, ep_step: 1000 2023-05-30 02:24:41 - SimpleLog - INFO: - episode: 4599, ep_reward: -7.108, ep_step: 600 2023-05-30 02:24:43 - SimpleLog - INFO: - episode: 4600, ep_reward: 170.518, ep_step: 1000 2023-05-30 02:24:44 - SimpleLog - INFO: - episode: 4601, ep_reward: -9.530, ep_step: 619 2023-05-30 02:24:47 - SimpleLog - INFO: - episode: 4602, ep_reward: 149.070, ep_step: 1000 2023-05-30 02:24:49 - SimpleLog - INFO: - episode: 4603, ep_reward: 3.827, ep_step: 915 2023-05-30 02:24:50 - SimpleLog - INFO: - episode: 4604, ep_reward: -35.563, ep_step: 592 2023-05-30 02:24:51 - SimpleLog - INFO: - episode: 4605, ep_reward: -76.805, ep_step: 631 2023-05-30 02:24:53 - SimpleLog - INFO: - episode: 4606, ep_reward: 16.179, ep_step: 795 2023-05-30 02:24:56 - SimpleLog - INFO: - episode: 4607, ep_reward: 170.208, ep_step: 1000 2023-05-30 02:24:56 - SimpleLog - INFO: - episode: 4608, ep_reward: -75.218, ep_step: 186 2023-05-30 02:24:57 - SimpleLog - INFO: - episode: 4609, ep_reward: -61.507, ep_step: 341 2023-05-30 02:24:58 - SimpleLog - INFO: - episode: 4610, ep_reward: -33.554, ep_step: 544 2023-05-30 02:25:00 - SimpleLog - INFO: - episode: 4611, ep_reward: 149.101, ep_step: 1000 2023-05-30 02:25:02 - SimpleLog - INFO: - episode: 4612, ep_reward: -88.243, ep_step: 762 2023-05-30 02:25:04 - SimpleLog - INFO: - episode: 4613, ep_reward: 159.368, ep_step: 1000 2023-05-30 02:25:06 - SimpleLog - INFO: - episode: 4614, ep_reward: -16.640, ep_step: 893 2023-05-30 02:25:08 - SimpleLog - INFO: - episode: 4615, ep_reward: 147.868, ep_step: 1000 2023-05-30 02:25:11 - SimpleLog - INFO: - episode: 4616, ep_reward: 103.117, ep_step: 1000 2023-05-30 02:25:13 - SimpleLog - INFO: - episode: 4617, ep_reward: 142.079, ep_step: 1000 2023-05-30 02:25:15 - SimpleLog - INFO: - episode: 4618, ep_reward: 134.197, ep_step: 1000 2023-05-30 02:25:17 - SimpleLog - INFO: - episode: 4619, ep_reward: 19.743, ep_step: 737 2023-05-30 02:25:19 - SimpleLog - INFO: - episode: 4620, ep_reward: 173.855, ep_step: 1000 2023-05-30 02:25:21 - SimpleLog - INFO: - episode: 4621, ep_reward: 144.327, ep_step: 1000 2023-05-30 02:25:24 - SimpleLog - INFO: - episode: 4622, ep_reward: 72.485, ep_step: 1000 2023-05-30 02:25:26 - SimpleLog - INFO: - episode: 4623, ep_reward: 174.252, ep_step: 1000 2023-05-30 02:25:28 - SimpleLog - INFO: - episode: 4624, ep_reward: 153.722, ep_step: 1000 2023-05-30 02:25:30 - SimpleLog - INFO: - episode: 4625, ep_reward: 144.957, ep_step: 1000 2023-05-30 02:25:33 - SimpleLog - INFO: - update_step: 3570000, online_eval_reward: 1.203 2023-05-30 02:25:34 - SimpleLog - INFO: - episode: 4626, ep_reward: 156.435, ep_step: 1000 2023-05-30 02:25:36 - SimpleLog - INFO: - episode: 4627, ep_reward: 164.707, ep_step: 1000 2023-05-30 02:25:38 - SimpleLog - INFO: - episode: 4628, ep_reward: 131.305, ep_step: 1000 2023-05-30 02:25:41 - SimpleLog - INFO: - episode: 4629, ep_reward: 150.797, ep_step: 1000 2023-05-30 02:25:43 - SimpleLog - INFO: - episode: 4630, ep_reward: 138.900, ep_step: 1000 2023-05-30 02:25:45 - SimpleLog - INFO: - episode: 4631, ep_reward: 76.188, ep_step: 1000 2023-05-30 02:25:47 - SimpleLog - INFO: - episode: 4632, ep_reward: 160.482, ep_step: 1000 2023-05-30 02:25:48 - SimpleLog - INFO: - episode: 4633, ep_reward: -59.784, ep_step: 427 2023-05-30 02:25:51 - SimpleLog - INFO: - episode: 4634, ep_reward: 148.659, ep_step: 1000 2023-05-30 02:25:53 - SimpleLog - INFO: - episode: 4635, ep_reward: 140.672, ep_step: 1000 2023-05-30 02:25:55 - SimpleLog - INFO: - episode: 4636, ep_reward: 135.021, ep_step: 1000 2023-05-30 02:25:57 - SimpleLog - INFO: - episode: 4637, ep_reward: 151.958, ep_step: 1000 2023-05-30 02:26:00 - SimpleLog - INFO: - episode: 4638, ep_reward: 119.017, ep_step: 1000 2023-05-30 02:26:02 - SimpleLog - INFO: - episode: 4639, ep_reward: 159.531, ep_step: 1000 2023-05-30 02:26:04 - SimpleLog - INFO: - episode: 4640, ep_reward: 103.574, ep_step: 1000 2023-05-30 02:26:06 - SimpleLog - INFO: - episode: 4641, ep_reward: 142.637, ep_step: 1000 2023-05-30 02:26:09 - SimpleLog - INFO: - episode: 4642, ep_reward: 158.335, ep_step: 1000 2023-05-30 02:26:11 - SimpleLog - INFO: - episode: 4643, ep_reward: 151.591, ep_step: 1000 2023-05-30 02:26:13 - SimpleLog - INFO: - episode: 4644, ep_reward: 160.157, ep_step: 1000 2023-05-30 02:26:14 - SimpleLog - INFO: - episode: 4645, ep_reward: -14.107, ep_step: 528 2023-05-30 02:26:17 - SimpleLog - INFO: - episode: 4646, ep_reward: 163.010, ep_step: 1000 2023-05-30 02:26:19 - SimpleLog - INFO: - episode: 4647, ep_reward: 106.601, ep_step: 1000 2023-05-30 02:26:21 - SimpleLog - INFO: - episode: 4648, ep_reward: 167.249, ep_step: 1000 2023-05-30 02:26:23 - SimpleLog - INFO: - episode: 4649, ep_reward: 139.554, ep_step: 1000 2023-05-30 02:26:25 - SimpleLog - INFO: - episode: 4650, ep_reward: 12.133, ep_step: 820 2023-05-30 02:26:27 - SimpleLog - INFO: - episode: 4651, ep_reward: 43.827, ep_step: 914 2023-05-30 02:26:29 - SimpleLog - INFO: - episode: 4652, ep_reward: 162.614, ep_step: 1000 2023-05-30 02:26:31 - SimpleLog - INFO: - episode: 4653, ep_reward: 6.315, ep_step: 720 2023-05-30 02:26:32 - SimpleLog - INFO: - episode: 4654, ep_reward: -36.973, ep_step: 527 2023-05-30 02:26:34 - SimpleLog - INFO: - episode: 4655, ep_reward: 22.081, ep_step: 970 2023-05-30 02:26:37 - SimpleLog - INFO: - episode: 4656, ep_reward: 129.066, ep_step: 1000 2023-05-30 02:26:39 - SimpleLog - INFO: - episode: 4657, ep_reward: 137.751, ep_step: 1000 2023-05-30 02:26:41 - SimpleLog - INFO: - episode: 4658, ep_reward: 137.730, ep_step: 1000 2023-05-30 02:26:43 - SimpleLog - INFO: - episode: 4659, ep_reward: 139.293, ep_step: 1000 2023-05-30 02:26:46 - SimpleLog - INFO: - episode: 4660, ep_reward: 132.452, ep_step: 1000 2023-05-30 02:26:48 - SimpleLog - INFO: - episode: 4661, ep_reward: 132.461, ep_step: 1000 2023-05-30 02:26:50 - SimpleLog - INFO: - episode: 4662, ep_reward: 149.896, ep_step: 1000 2023-05-30 02:26:52 - SimpleLog - INFO: - episode: 4663, ep_reward: 147.980, ep_step: 1000 2023-05-30 02:26:54 - SimpleLog - INFO: - episode: 4664, ep_reward: -73.481, ep_step: 582 2023-05-30 02:26:55 - SimpleLog - INFO: - episode: 4665, ep_reward: -68.706, ep_step: 525 2023-05-30 02:26:57 - SimpleLog - INFO: - episode: 4666, ep_reward: 160.875, ep_step: 1000 2023-05-30 02:26:59 - SimpleLog - INFO: - episode: 4667, ep_reward: 76.380, ep_step: 1000 2023-05-30 02:27:00 - SimpleLog - INFO: - episode: 4668, ep_reward: -37.324, ep_step: 520 2023-05-30 02:27:03 - SimpleLog - INFO: - episode: 4669, ep_reward: 164.507, ep_step: 1000 2023-05-30 02:27:04 - SimpleLog - INFO: - episode: 4670, ep_reward: -70.268, ep_step: 549 2023-05-30 02:27:06 - SimpleLog - INFO: - episode: 4671, ep_reward: 166.307, ep_step: 1000 2023-05-30 02:27:08 - SimpleLog - INFO: - episode: 4672, ep_reward: 134.701, ep_step: 1000 2023-05-30 02:27:10 - SimpleLog - INFO: - episode: 4673, ep_reward: -69.786, ep_step: 534 2023-05-30 02:27:11 - SimpleLog - INFO: - episode: 4674, ep_reward: -46.922, ep_step: 479 2023-05-30 02:27:12 - SimpleLog - INFO: - episode: 4675, ep_reward: 4.639, ep_step: 770 2023-05-30 02:27:15 - SimpleLog - INFO: - episode: 4676, ep_reward: 132.483, ep_step: 1000 2023-05-30 02:27:15 - SimpleLog - INFO: - episode: 4677, ep_reward: -118.039, ep_step: 78 2023-05-30 02:27:16 - SimpleLog - INFO: - episode: 4678, ep_reward: -16.224, ep_step: 604 2023-05-30 02:27:17 - SimpleLog - INFO: - episode: 4679, ep_reward: -64.841, ep_step: 500 2023-05-30 02:27:20 - SimpleLog - INFO: - episode: 4680, ep_reward: 142.431, ep_step: 1000 2023-05-30 02:27:22 - SimpleLog - INFO: - episode: 4681, ep_reward: 145.263, ep_step: 1000 2023-05-30 02:27:24 - SimpleLog - INFO: - episode: 4682, ep_reward: 149.035, ep_step: 1000 2023-05-30 02:27:25 - SimpleLog - INFO: - episode: 4683, ep_reward: -92.520, ep_step: 166 2023-05-30 02:27:27 - SimpleLog - INFO: - episode: 4684, ep_reward: 165.294, ep_step: 1000 2023-05-30 02:27:29 - SimpleLog - INFO: - episode: 4685, ep_reward: 98.506, ep_step: 1000 2023-05-30 02:27:31 - SimpleLog - INFO: - episode: 4686, ep_reward: 150.222, ep_step: 1000 2023-05-30 02:27:34 - SimpleLog - INFO: - episode: 4687, ep_reward: 145.673, ep_step: 1000 2023-05-30 02:27:36 - SimpleLog - INFO: - episode: 4688, ep_reward: 133.302, ep_step: 1000 2023-05-30 02:27:38 - SimpleLog - INFO: - episode: 4689, ep_reward: 98.639, ep_step: 1000 2023-05-30 02:27:40 - SimpleLog - INFO: - episode: 4690, ep_reward: 108.095, ep_step: 1000 2023-05-30 02:27:43 - SimpleLog - INFO: - episode: 4691, ep_reward: 117.532, ep_step: 1000 2023-05-30 02:27:45 - SimpleLog - INFO: - episode: 4692, ep_reward: 141.449, ep_step: 1000 2023-05-30 02:27:47 - SimpleLog - INFO: - episode: 4693, ep_reward: 152.302, ep_step: 1000 2023-05-30 02:27:49 - SimpleLog - INFO: - episode: 4694, ep_reward: 7.559, ep_step: 725 2023-05-30 02:27:50 - SimpleLog - INFO: - episode: 4695, ep_reward: -70.643, ep_step: 356 2023-05-30 02:27:51 - SimpleLog - INFO: - episode: 4696, ep_reward: -49.341, ep_step: 488 2023-05-30 02:27:53 - SimpleLog - INFO: - episode: 4697, ep_reward: 143.915, ep_step: 1000 2023-05-30 02:27:55 - SimpleLog - INFO: - episode: 4698, ep_reward: 5.185, ep_step: 941 2023-05-30 02:27:57 - SimpleLog - INFO: - episode: 4699, ep_reward: 149.845, ep_step: 1000 2023-05-30 02:28:00 - SimpleLog - INFO: - episode: 4700, ep_reward: 170.054, ep_step: 1000 2023-05-30 02:28:02 - SimpleLog - INFO: - episode: 4701, ep_reward: 155.999, ep_step: 1000 2023-05-30 02:28:03 - SimpleLog - INFO: - episode: 4702, ep_reward: 5.198, ep_step: 677 2023-05-30 02:28:04 - SimpleLog - INFO: - episode: 4703, ep_reward: -51.313, ep_step: 513 2023-05-30 02:28:06 - SimpleLog - INFO: - episode: 4704, ep_reward: -52.030, ep_step: 535 2023-05-30 02:28:07 - SimpleLog - INFO: - episode: 4705, ep_reward: -12.787, ep_step: 814 2023-05-30 02:28:08 - SimpleLog - INFO: - episode: 4706, ep_reward: -60.121, ep_step: 283 2023-05-30 02:28:10 - SimpleLog - INFO: - episode: 4707, ep_reward: 171.816, ep_step: 1000 2023-05-30 02:28:12 - SimpleLog - INFO: - update_step: 3640000, online_eval_reward: 181.204 2023-05-30 02:28:12 - SimpleLog - INFO: - current update step obtain a better online_eval_reward: 181.204, save the best model! 2023-05-30 02:28:13 - SimpleLog - INFO: - episode: 4708, ep_reward: -42.978, ep_step: 374 2023-05-30 02:28:15 - SimpleLog - INFO: - episode: 4709, ep_reward: 120.540, ep_step: 1000 2023-05-30 02:28:16 - SimpleLog - INFO: - episode: 4710, ep_reward: -65.340, ep_step: 295 2023-05-30 02:28:18 - SimpleLog - INFO: - episode: 4711, ep_reward: 186.884, ep_step: 1000 2023-05-30 02:28:21 - SimpleLog - INFO: - episode: 4712, ep_reward: 183.803, ep_step: 1000 2023-05-30 02:28:21 - SimpleLog - INFO: - episode: 4713, ep_reward: -86.176, ep_step: 154 2023-05-30 02:28:23 - SimpleLog - INFO: - episode: 4714, ep_reward: 155.480, ep_step: 1000 2023-05-30 02:28:25 - SimpleLog - INFO: - episode: 4715, ep_reward: 163.843, ep_step: 1000 2023-05-30 02:28:27 - SimpleLog - INFO: - episode: 4716, ep_reward: 6.542, ep_step: 605 2023-05-30 02:28:29 - SimpleLog - INFO: - episode: 4717, ep_reward: 147.262, ep_step: 1000 2023-05-30 02:28:31 - SimpleLog - INFO: - episode: 4718, ep_reward: 141.223, ep_step: 1000 2023-05-30 02:28:34 - SimpleLog - INFO: - episode: 4719, ep_reward: 153.116, ep_step: 1000 2023-05-30 02:28:36 - SimpleLog - INFO: - episode: 4720, ep_reward: 118.629, ep_step: 1000 2023-05-30 02:28:38 - SimpleLog - INFO: - episode: 4721, ep_reward: 129.206, ep_step: 1000 2023-05-30 02:28:40 - SimpleLog - INFO: - episode: 4722, ep_reward: 2.877, ep_step: 732 2023-05-30 02:28:42 - SimpleLog - INFO: - episode: 4723, ep_reward: -0.885, ep_step: 805 2023-05-30 02:28:42 - SimpleLog - INFO: - episode: 4724, ep_reward: -53.109, ep_step: 379 2023-05-30 02:28:45 - SimpleLog - INFO: - episode: 4725, ep_reward: 95.362, ep_step: 1000 2023-05-30 02:28:47 - SimpleLog - INFO: - episode: 4726, ep_reward: 166.915, ep_step: 1000 2023-05-30 02:28:48 - SimpleLog - INFO: - episode: 4727, ep_reward: -63.963, ep_step: 414 2023-05-30 02:28:50 - SimpleLog - INFO: - episode: 4728, ep_reward: 157.135, ep_step: 1000 2023-05-30 02:28:52 - SimpleLog - INFO: - episode: 4729, ep_reward: 113.101, ep_step: 1000 2023-05-30 02:28:53 - SimpleLog - INFO: - episode: 4730, ep_reward: -45.387, ep_step: 457 2023-05-30 02:28:56 - SimpleLog - INFO: - episode: 4731, ep_reward: 128.673, ep_step: 1000 2023-05-30 02:28:58 - SimpleLog - INFO: - episode: 4732, ep_reward: 107.029, ep_step: 1000 2023-05-30 02:29:00 - SimpleLog - INFO: - episode: 4733, ep_reward: 126.966, ep_step: 1000 2023-05-30 02:29:01 - SimpleLog - INFO: - episode: 4734, ep_reward: -54.479, ep_step: 370 2023-05-30 02:29:03 - SimpleLog - INFO: - episode: 4735, ep_reward: 38.183, ep_step: 905 2023-05-30 02:29:05 - SimpleLog - INFO: - episode: 4736, ep_reward: 102.580, ep_step: 1000 2023-05-30 02:29:06 - SimpleLog - INFO: - episode: 4737, ep_reward: -51.240, ep_step: 505 2023-05-30 02:29:07 - SimpleLog - INFO: - episode: 4738, ep_reward: -84.679, ep_step: 170 2023-05-30 02:29:09 - SimpleLog - INFO: - episode: 4739, ep_reward: -10.540, ep_step: 891 2023-05-30 02:29:10 - SimpleLog - INFO: - episode: 4740, ep_reward: -51.623, ep_step: 507 2023-05-30 02:29:11 - SimpleLog - INFO: - episode: 4741, ep_reward: -104.223, ep_step: 228 2023-05-30 02:29:13 - SimpleLog - INFO: - episode: 4742, ep_reward: 135.726, ep_step: 1000 2023-05-30 02:29:15 - SimpleLog - INFO: - episode: 4743, ep_reward: 127.065, ep_step: 1000 2023-05-30 02:29:17 - SimpleLog - INFO: - episode: 4744, ep_reward: 149.759, ep_step: 1000 2023-05-30 02:29:20 - SimpleLog - INFO: - episode: 4745, ep_reward: 170.455, ep_step: 1000 2023-05-30 02:29:22 - SimpleLog - INFO: - episode: 4746, ep_reward: 145.911, ep_step: 1000 2023-05-30 02:29:24 - SimpleLog - INFO: - episode: 4747, ep_reward: 107.012, ep_step: 1000 2023-05-30 02:29:26 - SimpleLog - INFO: - episode: 4748, ep_reward: 145.113, ep_step: 1000 2023-05-30 02:29:28 - SimpleLog - INFO: - episode: 4749, ep_reward: -56.243, ep_step: 577 2023-05-30 02:29:30 - SimpleLog - INFO: - episode: 4750, ep_reward: 165.947, ep_step: 1000 2023-05-30 02:29:32 - SimpleLog - INFO: - episode: 4751, ep_reward: 140.238, ep_step: 1000 2023-05-30 02:29:34 - SimpleLog - INFO: - episode: 4752, ep_reward: 139.478, ep_step: 1000 2023-05-30 02:29:37 - SimpleLog - INFO: - episode: 4753, ep_reward: 130.687, ep_step: 1000 2023-05-30 02:29:39 - SimpleLog - INFO: - episode: 4754, ep_reward: -0.016, ep_step: 814 2023-05-30 02:29:39 - SimpleLog - INFO: - episode: 4755, ep_reward: -56.782, ep_step: 407 2023-05-30 02:29:42 - SimpleLog - INFO: - episode: 4756, ep_reward: -13.566, ep_step: 967 2023-05-30 02:29:43 - SimpleLog - INFO: - episode: 4757, ep_reward: -2.869, ep_step: 585 2023-05-30 02:29:44 - SimpleLog - INFO: - episode: 4758, ep_reward: 0.930, ep_step: 603 2023-05-30 02:29:46 - SimpleLog - INFO: - episode: 4759, ep_reward: 2.735, ep_step: 607 2023-05-30 02:29:48 - SimpleLog - INFO: - episode: 4760, ep_reward: 181.882, ep_step: 1000 2023-05-30 02:29:50 - SimpleLog - INFO: - episode: 4761, ep_reward: 182.752, ep_step: 1000 2023-05-30 02:29:52 - SimpleLog - INFO: - episode: 4762, ep_reward: 182.612, ep_step: 1000 2023-05-30 02:29:55 - SimpleLog - INFO: - episode: 4763, ep_reward: 176.400, ep_step: 1000 2023-05-30 02:29:55 - SimpleLog - INFO: - episode: 4764, ep_reward: -69.129, ep_step: 279 2023-05-30 02:29:58 - SimpleLog - INFO: - episode: 4765, ep_reward: 167.620, ep_step: 1000 2023-05-30 02:30:00 - SimpleLog - INFO: - episode: 4766, ep_reward: 180.378, ep_step: 1000 2023-05-30 02:30:02 - SimpleLog - INFO: - episode: 4767, ep_reward: 183.363, ep_step: 1000 2023-05-30 02:30:04 - SimpleLog - INFO: - episode: 4768, ep_reward: 168.304, ep_step: 1000 2023-05-30 02:30:06 - SimpleLog - INFO: - episode: 4769, ep_reward: 129.947, ep_step: 1000 2023-05-30 02:30:09 - SimpleLog - INFO: - episode: 4770, ep_reward: 177.245, ep_step: 1000 2023-05-30 02:30:10 - SimpleLog - INFO: - episode: 4771, ep_reward: -78.226, ep_step: 606 2023-05-30 02:30:12 - SimpleLog - INFO: - episode: 4772, ep_reward: 81.557, ep_step: 1000 2023-05-30 02:30:15 - SimpleLog - INFO: - episode: 4773, ep_reward: 130.884, ep_step: 1000 2023-05-30 02:30:17 - SimpleLog - INFO: - episode: 4774, ep_reward: 140.778, ep_step: 1000 2023-05-30 02:30:19 - SimpleLog - INFO: - episode: 4775, ep_reward: 139.268, ep_step: 1000 2023-05-30 02:30:21 - SimpleLog - INFO: - episode: 4776, ep_reward: 143.312, ep_step: 1000 2023-05-30 02:30:24 - SimpleLog - INFO: - episode: 4777, ep_reward: 139.471, ep_step: 1000 2023-05-30 02:30:24 - SimpleLog - INFO: - episode: 4778, ep_reward: -134.980, ep_step: 139 2023-05-30 02:30:26 - SimpleLog - INFO: - episode: 4779, ep_reward: 165.729, ep_step: 1000 2023-05-30 02:30:28 - SimpleLog - INFO: - episode: 4780, ep_reward: 162.321, ep_step: 1000 2023-05-30 02:30:31 - SimpleLog - INFO: - episode: 4781, ep_reward: 141.541, ep_step: 1000 2023-05-30 02:30:32 - SimpleLog - INFO: - episode: 4782, ep_reward: -45.027, ep_step: 451 2023-05-30 02:30:34 - SimpleLog - INFO: - episode: 4783, ep_reward: 84.459, ep_step: 1000 2023-05-30 02:30:36 - SimpleLog - INFO: - episode: 4784, ep_reward: 131.166, ep_step: 1000 2023-05-30 02:30:38 - SimpleLog - INFO: - episode: 4785, ep_reward: 77.134, ep_step: 1000 2023-05-30 02:30:41 - SimpleLog - INFO: - episode: 4786, ep_reward: 166.336, ep_step: 1000 2023-05-30 02:30:43 - SimpleLog - INFO: - episode: 4787, ep_reward: 137.096, ep_step: 1000 2023-05-30 02:30:45 - SimpleLog - INFO: - episode: 4788, ep_reward: 155.027, ep_step: 1000 2023-05-30 02:30:46 - SimpleLog - INFO: - episode: 4789, ep_reward: -59.618, ep_step: 407 2023-05-30 02:30:48 - SimpleLog - INFO: - episode: 4790, ep_reward: 142.153, ep_step: 1000 2023-05-30 02:30:52 - SimpleLog - INFO: - update_step: 3710000, online_eval_reward: 170.166 2023-05-30 02:30:52 - SimpleLog - INFO: - episode: 4791, ep_reward: 160.723, ep_step: 1000 2023-05-30 02:30:55 - SimpleLog - INFO: - episode: 4792, ep_reward: 171.920, ep_step: 1000 2023-05-30 02:30:57 - SimpleLog - INFO: - episode: 4793, ep_reward: 82.303, ep_step: 1000 2023-05-30 02:30:59 - SimpleLog - INFO: - episode: 4794, ep_reward: -0.162, ep_step: 893 2023-05-30 02:31:01 - SimpleLog - INFO: - episode: 4795, ep_reward: 168.586, ep_step: 1000 2023-05-30 02:31:03 - SimpleLog - INFO: - episode: 4796, ep_reward: 167.966, ep_step: 1000 2023-05-30 02:31:04 - SimpleLog - INFO: - episode: 4797, ep_reward: -73.494, ep_step: 344 2023-05-30 02:31:07 - SimpleLog - INFO: - episode: 4798, ep_reward: 97.730, ep_step: 1000 2023-05-30 02:31:08 - SimpleLog - INFO: - episode: 4799, ep_reward: -74.635, ep_step: 441 2023-05-30 02:31:08 - SimpleLog - INFO: - episode: 4800, ep_reward: -90.939, ep_step: 145 2023-05-30 02:31:10 - SimpleLog - INFO: - episode: 4801, ep_reward: 157.343, ep_step: 1000 2023-05-30 02:31:11 - SimpleLog - INFO: - episode: 4802, ep_reward: -1.563, ep_step: 591 2023-05-30 02:31:14 - SimpleLog - INFO: - episode: 4803, ep_reward: 158.491, ep_step: 1000 2023-05-30 02:31:15 - SimpleLog - INFO: - episode: 4804, ep_reward: -47.458, ep_step: 478 2023-05-30 02:31:15 - SimpleLog - INFO: - episode: 4805, ep_reward: -97.281, ep_step: 217 2023-05-30 02:31:18 - SimpleLog - INFO: - episode: 4806, ep_reward: 166.685, ep_step: 1000 2023-05-30 02:31:20 - SimpleLog - INFO: - episode: 4807, ep_reward: 2.635, ep_step: 1000 2023-05-30 02:31:22 - SimpleLog - INFO: - episode: 4808, ep_reward: 172.383, ep_step: 1000 2023-05-30 02:31:24 - SimpleLog - INFO: - episode: 4809, ep_reward: 131.494, ep_step: 1000 2023-05-30 02:31:27 - SimpleLog - INFO: - episode: 4810, ep_reward: 159.409, ep_step: 1000 2023-05-30 02:31:29 - SimpleLog - INFO: - episode: 4811, ep_reward: 153.424, ep_step: 1000 2023-05-30 02:31:31 - SimpleLog - INFO: - episode: 4812, ep_reward: 158.735, ep_step: 1000 2023-05-30 02:31:33 - SimpleLog - INFO: - episode: 4813, ep_reward: 155.645, ep_step: 1000 2023-05-30 02:31:34 - SimpleLog - INFO: - episode: 4814, ep_reward: -105.800, ep_step: 333 2023-05-30 02:31:34 - SimpleLog - INFO: - episode: 4815, ep_reward: -92.136, ep_step: 174 2023-05-30 02:31:37 - SimpleLog - INFO: - episode: 4816, ep_reward: 146.728, ep_step: 1000 2023-05-30 02:31:37 - SimpleLog - INFO: - episode: 4817, ep_reward: -75.446, ep_step: 279 2023-05-30 02:31:40 - SimpleLog - INFO: - episode: 4818, ep_reward: 132.715, ep_step: 1000 2023-05-30 02:31:42 - SimpleLog - INFO: - episode: 4819, ep_reward: 139.667, ep_step: 1000 2023-05-30 02:31:44 - SimpleLog - INFO: - episode: 4820, ep_reward: 160.755, ep_step: 1000 2023-05-30 02:31:46 - SimpleLog - INFO: - episode: 4821, ep_reward: 131.690, ep_step: 1000 2023-05-30 02:31:47 - SimpleLog - INFO: - episode: 4822, ep_reward: -102.581, ep_step: 196 2023-05-30 02:31:49 - SimpleLog - INFO: - episode: 4823, ep_reward: 163.234, ep_step: 1000 2023-05-30 02:31:51 - SimpleLog - INFO: - episode: 4824, ep_reward: -0.079, ep_step: 660 2023-05-30 02:31:53 - SimpleLog - INFO: - episode: 4825, ep_reward: 134.534, ep_step: 1000 2023-05-30 02:31:55 - SimpleLog - INFO: - episode: 4826, ep_reward: 140.752, ep_step: 1000 2023-05-30 02:31:57 - SimpleLog - INFO: - episode: 4827, ep_reward: 2.507, ep_step: 803 2023-05-30 02:31:59 - SimpleLog - INFO: - episode: 4828, ep_reward: -13.658, ep_step: 736 2023-05-30 02:32:01 - SimpleLog - INFO: - episode: 4829, ep_reward: 127.472, ep_step: 1000 2023-05-30 02:32:03 - SimpleLog - INFO: - episode: 4830, ep_reward: 180.912, ep_step: 1000 2023-05-30 02:32:05 - SimpleLog - INFO: - episode: 4831, ep_reward: -7.923, ep_step: 680 2023-05-30 02:32:07 - SimpleLog - INFO: - episode: 4832, ep_reward: 134.951, ep_step: 1000 2023-05-30 02:32:09 - SimpleLog - INFO: - episode: 4833, ep_reward: 80.294, ep_step: 1000 2023-05-30 02:32:11 - SimpleLog - INFO: - episode: 4834, ep_reward: 116.765, ep_step: 1000 2023-05-30 02:32:13 - SimpleLog - INFO: - episode: 4835, ep_reward: 6.666, ep_step: 734 2023-05-30 02:32:15 - SimpleLog - INFO: - episode: 4836, ep_reward: -15.009, ep_step: 763 2023-05-30 02:32:17 - SimpleLog - INFO: - episode: 4837, ep_reward: 138.161, ep_step: 1000 2023-05-30 02:32:19 - SimpleLog - INFO: - episode: 4838, ep_reward: 158.911, ep_step: 1000 2023-05-30 02:32:21 - SimpleLog - INFO: - episode: 4839, ep_reward: -3.779, ep_step: 857 2023-05-30 02:32:23 - SimpleLog - INFO: - episode: 4840, ep_reward: 3.724, ep_step: 886 2023-05-30 02:32:25 - SimpleLog - INFO: - episode: 4841, ep_reward: -34.608, ep_step: 620 2023-05-30 02:32:27 - SimpleLog - INFO: - episode: 4842, ep_reward: 6.774, ep_step: 970 2023-05-30 02:32:29 - SimpleLog - INFO: - episode: 4843, ep_reward: 168.491, ep_step: 1000 2023-05-30 02:32:31 - SimpleLog - INFO: - episode: 4844, ep_reward: 100.676, ep_step: 1000 2023-05-30 02:32:33 - SimpleLog - INFO: - episode: 4845, ep_reward: 148.856, ep_step: 1000 2023-05-30 02:32:36 - SimpleLog - INFO: - episode: 4846, ep_reward: 110.235, ep_step: 1000 2023-05-30 02:32:38 - SimpleLog - INFO: - episode: 4847, ep_reward: 140.354, ep_step: 1000 2023-05-30 02:32:40 - SimpleLog - INFO: - episode: 4848, ep_reward: 136.224, ep_step: 1000 2023-05-30 02:32:43 - SimpleLog - INFO: - episode: 4849, ep_reward: 161.206, ep_step: 1000 2023-05-30 02:32:45 - SimpleLog - INFO: - episode: 4850, ep_reward: 158.928, ep_step: 1000 2023-05-30 02:32:46 - SimpleLog - INFO: - episode: 4851, ep_reward: 3.388, ep_step: 734 2023-05-30 02:32:49 - SimpleLog - INFO: - episode: 4852, ep_reward: 162.684, ep_step: 1000 2023-05-30 02:32:51 - SimpleLog - INFO: - episode: 4853, ep_reward: 16.916, ep_step: 976 2023-05-30 02:32:53 - SimpleLog - INFO: - episode: 4854, ep_reward: 0.361, ep_step: 769 2023-05-30 02:32:55 - SimpleLog - INFO: - episode: 4855, ep_reward: 150.419, ep_step: 1000 2023-05-30 02:32:57 - SimpleLog - INFO: - episode: 4856, ep_reward: 155.156, ep_step: 1000 2023-05-30 02:32:59 - SimpleLog - INFO: - episode: 4857, ep_reward: 73.591, ep_step: 1000 2023-05-30 02:33:02 - SimpleLog - INFO: - episode: 4858, ep_reward: 182.460, ep_step: 1000 2023-05-30 02:33:03 - SimpleLog - INFO: - episode: 4859, ep_reward: -15.334, ep_step: 663 2023-05-30 02:33:05 - SimpleLog - INFO: - episode: 4860, ep_reward: 1.311, ep_step: 879 2023-05-30 02:33:07 - SimpleLog - INFO: - episode: 4861, ep_reward: -33.009, ep_step: 658 2023-05-30 02:33:09 - SimpleLog - INFO: - episode: 4862, ep_reward: 3.387, ep_step: 867 2023-05-30 02:33:11 - SimpleLog - INFO: - episode: 4863, ep_reward: 16.508, ep_step: 993 2023-05-30 02:33:13 - SimpleLog - INFO: - episode: 4864, ep_reward: 132.550, ep_step: 1000 2023-05-30 02:33:15 - SimpleLog - INFO: - episode: 4865, ep_reward: 152.744, ep_step: 1000 2023-05-30 02:33:17 - SimpleLog - INFO: - episode: 4866, ep_reward: 22.263, ep_step: 766 2023-05-30 02:33:18 - SimpleLog - INFO: - episode: 4867, ep_reward: -35.343, ep_step: 468 2023-05-30 02:33:19 - SimpleLog - INFO: - episode: 4868, ep_reward: -31.382, ep_step: 460 2023-05-30 02:33:22 - SimpleLog - INFO: - episode: 4869, ep_reward: 143.084, ep_step: 1000 2023-05-30 02:33:22 - SimpleLog - INFO: - episode: 4870, ep_reward: -52.660, ep_step: 388 2023-05-30 02:33:23 - SimpleLog - INFO: - episode: 4871, ep_reward: -54.579, ep_step: 405 2023-05-30 02:33:26 - SimpleLog - INFO: - episode: 4872, ep_reward: 151.543, ep_step: 1000 2023-05-30 02:33:28 - SimpleLog - INFO: - episode: 4873, ep_reward: 156.716, ep_step: 1000 2023-05-30 02:33:29 - SimpleLog - INFO: - episode: 4874, ep_reward: 4.435, ep_step: 680 2023-05-30 02:33:32 - SimpleLog - INFO: - update_step: 3780000, online_eval_reward: 170.899 2023-05-30 02:33:33 - SimpleLog - INFO: - episode: 4875, ep_reward: 5.260, ep_step: 702 2023-05-30 02:33:35 - SimpleLog - INFO: - episode: 4876, ep_reward: 163.874, ep_step: 1000 2023-05-30 02:33:37 - SimpleLog - INFO: - episode: 4877, ep_reward: 176.245, ep_step: 1000 2023-05-30 02:33:40 - SimpleLog - INFO: - episode: 4878, ep_reward: 136.779, ep_step: 1000 2023-05-30 02:33:42 - SimpleLog - INFO: - episode: 4879, ep_reward: 159.533, ep_step: 1000 2023-05-30 02:33:44 - SimpleLog - INFO: - episode: 4880, ep_reward: 161.770, ep_step: 1000 2023-05-30 02:33:46 - SimpleLog - INFO: - episode: 4881, ep_reward: 165.127, ep_step: 1000 2023-05-30 02:33:47 - SimpleLog - INFO: - episode: 4882, ep_reward: -46.918, ep_step: 456 2023-05-30 02:33:49 - SimpleLog - INFO: - episode: 4883, ep_reward: -13.040, ep_step: 874 2023-05-30 02:33:52 - SimpleLog - INFO: - episode: 4884, ep_reward: 76.863, ep_step: 1000 2023-05-30 02:33:53 - SimpleLog - INFO: - episode: 4885, ep_reward: -65.379, ep_step: 454 2023-05-30 02:33:55 - SimpleLog - INFO: - episode: 4886, ep_reward: 165.238, ep_step: 1000 2023-05-30 02:33:57 - SimpleLog - INFO: - episode: 4887, ep_reward: 12.388, ep_step: 785 2023-05-30 02:33:59 - SimpleLog - INFO: - episode: 4888, ep_reward: 150.575, ep_step: 1000 2023-05-30 02:34:00 - SimpleLog - INFO: - episode: 4889, ep_reward: -65.478, ep_step: 358 2023-05-30 02:34:01 - SimpleLog - INFO: - episode: 4890, ep_reward: -66.671, ep_step: 490 2023-05-30 02:34:03 - SimpleLog - INFO: - episode: 4891, ep_reward: 103.833, ep_step: 1000 2023-05-30 02:34:05 - SimpleLog - INFO: - episode: 4892, ep_reward: 145.508, ep_step: 1000 2023-05-30 02:34:08 - SimpleLog - INFO: - episode: 4893, ep_reward: 15.753, ep_step: 963 2023-05-30 02:34:08 - SimpleLog - INFO: - episode: 4894, ep_reward: -101.278, ep_step: 67 2023-05-30 02:34:08 - SimpleLog - INFO: - episode: 4895, ep_reward: -104.177, ep_step: 60 2023-05-30 02:34:08 - SimpleLog - INFO: - episode: 4896, ep_reward: -105.427, ep_step: 55 2023-05-30 02:34:08 - SimpleLog - INFO: - episode: 4897, ep_reward: -103.232, ep_step: 54 2023-05-30 02:34:08 - SimpleLog - INFO: - episode: 4898, ep_reward: -104.537, ep_step: 60 2023-05-30 02:34:08 - SimpleLog - INFO: - episode: 4899, ep_reward: -102.439, ep_step: 50 2023-05-30 02:34:09 - SimpleLog - INFO: - episode: 4900, ep_reward: -102.805, ep_step: 56 2023-05-30 02:34:09 - SimpleLog - INFO: - episode: 4901, ep_reward: -102.320, ep_step: 57 2023-05-30 02:34:09 - SimpleLog - INFO: - episode: 4902, ep_reward: -102.758, ep_step: 68 2023-05-30 02:34:09 - SimpleLog - INFO: - episode: 4903, ep_reward: -102.074, ep_step: 56 2023-05-30 02:34:09 - SimpleLog - INFO: - episode: 4904, ep_reward: -100.509, ep_step: 49 2023-05-30 02:34:09 - SimpleLog - INFO: - episode: 4905, ep_reward: -102.885, ep_step: 58 2023-05-30 02:34:09 - SimpleLog - INFO: - episode: 4906, ep_reward: -104.068, ep_step: 47 2023-05-30 02:34:09 - SimpleLog - INFO: - episode: 4907, ep_reward: -106.438, ep_step: 62 2023-05-30 02:34:10 - SimpleLog - INFO: - episode: 4908, ep_reward: -103.060, ep_step: 57 2023-05-30 02:34:10 - SimpleLog - INFO: - episode: 4909, ep_reward: -104.842, ep_step: 58 2023-05-30 02:34:10 - SimpleLog - INFO: - episode: 4910, ep_reward: -103.809, ep_step: 62 2023-05-30 02:34:10 - SimpleLog - INFO: - episode: 4911, ep_reward: -103.053, ep_step: 57 2023-05-30 02:34:10 - SimpleLog - INFO: - episode: 4912, ep_reward: -110.675, ep_step: 79 2023-05-30 02:34:10 - SimpleLog - INFO: - episode: 4913, ep_reward: -105.133, ep_step: 57 2023-05-30 02:34:10 - SimpleLog - INFO: - episode: 4914, ep_reward: -105.104, ep_step: 62 2023-05-30 02:34:11 - SimpleLog - INFO: - episode: 4915, ep_reward: -105.556, ep_step: 69 2023-05-30 02:34:11 - SimpleLog - INFO: - episode: 4916, ep_reward: -103.707, ep_step: 75 2023-05-30 02:34:11 - SimpleLog - INFO: - episode: 4917, ep_reward: -104.989, ep_step: 65 2023-05-30 02:34:11 - SimpleLog - INFO: - episode: 4918, ep_reward: -104.979, ep_step: 63 2023-05-30 02:34:11 - SimpleLog - INFO: - episode: 4919, ep_reward: -105.809, ep_step: 61 2023-05-30 02:34:11 - SimpleLog - INFO: - episode: 4920, ep_reward: -104.773, ep_step: 62 2023-05-30 02:34:11 - SimpleLog - INFO: - episode: 4921, ep_reward: -106.845, ep_step: 64 2023-05-30 02:34:12 - SimpleLog - INFO: - episode: 4922, ep_reward: -109.224, ep_step: 68 2023-05-30 02:34:14 - SimpleLog - INFO: - episode: 4923, ep_reward: -103.713, ep_step: 1000 2023-05-30 02:34:14 - SimpleLog - INFO: - episode: 4924, ep_reward: -102.941, ep_step: 66 2023-05-30 02:34:14 - SimpleLog - INFO: - episode: 4925, ep_reward: -101.781, ep_step: 65 2023-05-30 02:34:14 - SimpleLog - INFO: - episode: 4926, ep_reward: -103.844, ep_step: 65 2023-05-30 02:34:15 - SimpleLog - INFO: - episode: 4927, ep_reward: -105.282, ep_step: 65 2023-05-30 02:34:15 - SimpleLog - INFO: - episode: 4928, ep_reward: -101.354, ep_step: 59 2023-05-30 02:34:15 - SimpleLog - INFO: - episode: 4929, ep_reward: -104.518, ep_step: 64 2023-05-30 02:34:15 - SimpleLog - INFO: - episode: 4930, ep_reward: -106.609, ep_step: 85 2023-05-30 02:34:15 - SimpleLog - INFO: - episode: 4931, ep_reward: -108.444, ep_step: 78 2023-05-30 02:34:15 - SimpleLog - INFO: - episode: 4932, ep_reward: -101.298, ep_step: 65 2023-05-30 02:34:15 - SimpleLog - INFO: - episode: 4933, ep_reward: -102.929, ep_step: 68 2023-05-30 02:34:16 - SimpleLog - INFO: - episode: 4934, ep_reward: -108.486, ep_step: 80 2023-05-30 02:34:16 - SimpleLog - INFO: - episode: 4935, ep_reward: -110.227, ep_step: 83 2023-05-30 02:34:16 - SimpleLog - INFO: - episode: 4936, ep_reward: -104.321, ep_step: 74 2023-05-30 02:34:16 - SimpleLog - INFO: - episode: 4937, ep_reward: -103.876, ep_step: 62 2023-05-30 02:34:16 - SimpleLog - INFO: - episode: 4938, ep_reward: -110.718, ep_step: 96 2023-05-30 02:34:17 - SimpleLog - INFO: - episode: 4939, ep_reward: -104.785, ep_step: 72 2023-05-30 02:34:17 - SimpleLog - INFO: - episode: 4940, ep_reward: -108.818, ep_step: 87 2023-05-30 02:34:17 - SimpleLog - INFO: - episode: 4941, ep_reward: -104.345, ep_step: 83 2023-05-30 02:34:17 - SimpleLog - INFO: - episode: 4942, ep_reward: -111.583, ep_step: 92 2023-05-30 02:34:17 - SimpleLog - INFO: - episode: 4943, ep_reward: -105.479, ep_step: 88 2023-05-30 02:34:18 - SimpleLog - INFO: - episode: 4944, ep_reward: -107.906, ep_step: 89 2023-05-30 02:34:18 - SimpleLog - INFO: - episode: 4945, ep_reward: -101.260, ep_step: 71 2023-05-30 02:34:18 - SimpleLog - INFO: - episode: 4946, ep_reward: -103.855, ep_step: 72 2023-05-30 02:34:18 - SimpleLog - INFO: - episode: 4947, ep_reward: -102.208, ep_step: 83 2023-05-30 02:34:18 - SimpleLog - INFO: - episode: 4948, ep_reward: -103.294, ep_step: 78 2023-05-30 02:34:18 - SimpleLog - INFO: - episode: 4949, ep_reward: -105.070, ep_step: 88 2023-05-30 02:34:19 - SimpleLog - INFO: - episode: 4950, ep_reward: -112.559, ep_step: 100 2023-05-30 02:34:19 - SimpleLog - INFO: - episode: 4951, ep_reward: -104.126, ep_step: 87 2023-05-30 02:34:19 - SimpleLog - INFO: - episode: 4952, ep_reward: -106.100, ep_step: 93 2023-05-30 02:34:19 - SimpleLog - INFO: - episode: 4953, ep_reward: -102.995, ep_step: 75 2023-05-30 02:34:20 - SimpleLog - INFO: - episode: 4954, ep_reward: -112.026, ep_step: 101 2023-05-30 02:34:20 - SimpleLog - INFO: - episode: 4955, ep_reward: -103.175, ep_step: 78 2023-05-30 02:34:20 - SimpleLog - INFO: - episode: 4956, ep_reward: -101.914, ep_step: 79 2023-05-30 02:34:20 - SimpleLog - INFO: - episode: 4957, ep_reward: -105.200, ep_step: 76 2023-05-30 02:34:20 - SimpleLog - INFO: - episode: 4958, ep_reward: -102.922, ep_step: 98 2023-05-30 02:34:21 - SimpleLog - INFO: - episode: 4959, ep_reward: -127.233, ep_step: 123 2023-05-30 02:34:21 - SimpleLog - INFO: - episode: 4960, ep_reward: -104.106, ep_step: 108 2023-05-30 02:34:21 - SimpleLog - INFO: - episode: 4961, ep_reward: -107.156, ep_step: 107 2023-05-30 02:34:21 - SimpleLog - INFO: - episode: 4962, ep_reward: -104.803, ep_step: 89 2023-05-30 02:34:21 - SimpleLog - INFO: - episode: 4963, ep_reward: -128.446, ep_step: 92 2023-05-30 02:34:22 - SimpleLog - INFO: - episode: 4964, ep_reward: -125.285, ep_step: 106 2023-05-30 02:34:22 - SimpleLog - INFO: - episode: 4965, ep_reward: -123.133, ep_step: 88 2023-05-30 02:34:22 - SimpleLog - INFO: - episode: 4966, ep_reward: -113.193, ep_step: 139 2023-05-30 02:34:22 - SimpleLog - INFO: - episode: 4967, ep_reward: -104.032, ep_step: 86 2023-05-30 02:34:23 - SimpleLog - INFO: - episode: 4968, ep_reward: -103.106, ep_step: 87 2023-05-30 02:34:23 - SimpleLog - INFO: - episode: 4969, ep_reward: -107.069, ep_step: 116 2023-05-30 02:34:23 - SimpleLog - INFO: - episode: 4970, ep_reward: -102.020, ep_step: 76 2023-05-30 02:34:23 - SimpleLog - INFO: - episode: 4971, ep_reward: -109.801, ep_step: 181 2023-05-30 02:34:24 - SimpleLog - INFO: - episode: 4972, ep_reward: -104.214, ep_step: 82 2023-05-30 02:34:24 - SimpleLog - INFO: - episode: 4973, ep_reward: -104.512, ep_step: 75 2023-05-30 02:34:24 - SimpleLog - INFO: - episode: 4974, ep_reward: -101.853, ep_step: 82 2023-05-30 02:34:24 - SimpleLog - INFO: - episode: 4975, ep_reward: -127.447, ep_step: 92 2023-05-30 02:34:24 - SimpleLog - INFO: - episode: 4976, ep_reward: -126.386, ep_step: 112 2023-05-30 02:34:25 - SimpleLog - INFO: - episode: 4977, ep_reward: -118.291, ep_step: 63 2023-05-30 02:34:25 - SimpleLog - INFO: - episode: 4978, ep_reward: -143.426, ep_step: 255 2023-05-30 02:34:26 - SimpleLog - INFO: - episode: 4979, ep_reward: -111.284, ep_step: 155 2023-05-30 02:34:27 - SimpleLog - INFO: - episode: 4980, ep_reward: -92.357, ep_step: 496 2023-05-30 02:34:27 - SimpleLog - INFO: - episode: 4981, ep_reward: -121.749, ep_step: 107 2023-05-30 02:34:27 - SimpleLog - INFO: - episode: 4982, ep_reward: -107.219, ep_step: 142 2023-05-30 02:34:28 - SimpleLog - INFO: - episode: 4983, ep_reward: -52.177, ep_step: 532 2023-05-30 02:34:29 - SimpleLog - INFO: - episode: 4984, ep_reward: -75.490, ep_step: 459 2023-05-30 02:34:30 - SimpleLog - INFO: - episode: 4985, ep_reward: -78.788, ep_step: 417 2023-05-30 02:34:32 - SimpleLog - INFO: - episode: 4986, ep_reward: -64.450, ep_step: 825 2023-05-30 02:34:33 - SimpleLog - INFO: - episode: 4987, ep_reward: -71.042, ep_step: 438 2023-05-30 02:34:34 - SimpleLog - INFO: - episode: 4988, ep_reward: -78.191, ep_step: 486 2023-05-30 02:34:35 - SimpleLog - INFO: - episode: 4989, ep_reward: -113.357, ep_step: 78 2023-05-30 02:34:36 - SimpleLog - INFO: - episode: 4990, ep_reward: -74.833, ep_step: 453 2023-05-30 02:34:36 - SimpleLog - INFO: - episode: 4991, ep_reward: -92.924, ep_step: 233 2023-05-30 02:34:37 - SimpleLog - INFO: - episode: 4992, ep_reward: -71.500, ep_step: 409 2023-05-30 02:34:38 - SimpleLog - INFO: - episode: 4993, ep_reward: -79.528, ep_step: 531 2023-05-30 02:34:39 - SimpleLog - INFO: - episode: 4994, ep_reward: -66.917, ep_step: 406 2023-05-30 02:34:40 - SimpleLog - INFO: - episode: 4995, ep_reward: -67.366, ep_step: 420 2023-05-30 02:34:41 - SimpleLog - INFO: - episode: 4996, ep_reward: -90.695, ep_step: 223 2023-05-30 02:34:42 - SimpleLog - INFO: - episode: 4997, ep_reward: -83.538, ep_step: 680 2023-05-30 02:34:43 - SimpleLog - INFO: - episode: 4998, ep_reward: -85.986, ep_step: 350 2023-05-30 02:34:44 - SimpleLog - INFO: - episode: 4999, ep_reward: -73.125, ep_step: 387 2023-05-30 02:34:45 - SimpleLog - INFO: - episode: 5000, ep_reward: -110.402, ep_step: 669 2023-05-30 02:34:47 - SimpleLog - INFO: - episode: 5001, ep_reward: -46.413, ep_step: 688 2023-05-30 02:34:49 - SimpleLog - INFO: - episode: 5002, ep_reward: 4.240, ep_step: 1000 2023-05-30 02:34:51 - SimpleLog - INFO: - episode: 5003, ep_reward: 150.970, ep_step: 1000 2023-05-30 02:34:53 - SimpleLog - INFO: - episode: 5004, ep_reward: -59.249, ep_step: 674 2023-05-30 02:34:55 - SimpleLog - INFO: - episode: 5005, ep_reward: 91.404, ep_step: 1000 2023-05-30 02:34:57 - SimpleLog - INFO: - episode: 5006, ep_reward: 107.193, ep_step: 1000 2023-05-30 02:35:00 - SimpleLog - INFO: - episode: 5007, ep_reward: 103.840, ep_step: 1000 2023-05-30 02:35:02 - SimpleLog - INFO: - episode: 5008, ep_reward: 121.202, ep_step: 1000 2023-05-30 02:35:02 - SimpleLog - INFO: - episode: 5009, ep_reward: -117.137, ep_step: 114 2023-05-30 02:35:04 - SimpleLog - INFO: - episode: 5010, ep_reward: 134.076, ep_step: 1000 2023-05-30 02:35:05 - SimpleLog - INFO: - episode: 5011, ep_reward: -61.787, ep_step: 361 2023-05-30 02:35:07 - SimpleLog - INFO: - episode: 5012, ep_reward: 9.045, ep_step: 868 2023-05-30 02:35:08 - SimpleLog - INFO: - episode: 5013, ep_reward: -61.430, ep_step: 460 2023-05-30 02:35:10 - SimpleLog - INFO: - episode: 5014, ep_reward: 159.155, ep_step: 1000 2023-05-30 02:35:13 - SimpleLog - INFO: - episode: 5015, ep_reward: 34.519, ep_step: 982 2023-05-30 02:35:15 - SimpleLog - INFO: - episode: 5016, ep_reward: 119.712, ep_step: 1000 2023-05-30 02:35:16 - SimpleLog - INFO: - episode: 5017, ep_reward: -64.793, ep_step: 388 2023-05-30 02:35:18 - SimpleLog - INFO: - episode: 5018, ep_reward: 150.499, ep_step: 1000 2023-05-30 02:35:20 - SimpleLog - INFO: - episode: 5019, ep_reward: 1.820, ep_step: 1000 2023-05-30 02:35:22 - SimpleLog - INFO: - episode: 5020, ep_reward: 162.510, ep_step: 1000 2023-05-30 02:35:25 - SimpleLog - INFO: - episode: 5021, ep_reward: 70.167, ep_step: 1000 2023-05-30 02:35:27 - SimpleLog - INFO: - episode: 5022, ep_reward: 147.426, ep_step: 1000 2023-05-30 02:35:29 - SimpleLog - INFO: - episode: 5023, ep_reward: -0.392, ep_step: 1000 2023-05-30 02:35:31 - SimpleLog - INFO: - episode: 5024, ep_reward: -3.699, ep_step: 918 2023-05-30 02:35:34 - SimpleLog - INFO: - episode: 5025, ep_reward: -0.156, ep_step: 1000 2023-05-30 02:35:35 - SimpleLog - INFO: - episode: 5026, ep_reward: -5.882, ep_step: 847 2023-05-30 02:35:38 - SimpleLog - INFO: - episode: 5027, ep_reward: 119.041, ep_step: 1000 2023-05-30 02:35:40 - SimpleLog - INFO: - episode: 5028, ep_reward: 81.496, ep_step: 1000 2023-05-30 02:35:41 - SimpleLog - INFO: - episode: 5029, ep_reward: -55.637, ep_step: 549 2023-05-30 02:35:43 - SimpleLog - INFO: - episode: 5030, ep_reward: 1.174, ep_step: 1000 2023-05-30 02:35:45 - SimpleLog - INFO: - episode: 5031, ep_reward: -2.019, ep_step: 915 2023-05-30 02:35:48 - SimpleLog - INFO: - episode: 5032, ep_reward: 108.416, ep_step: 1000 2023-05-30 02:35:50 - SimpleLog - INFO: - episode: 5033, ep_reward: 114.616, ep_step: 1000 2023-05-30 02:35:52 - SimpleLog - INFO: - episode: 5034, ep_reward: 140.344, ep_step: 1000 2023-05-30 02:35:54 - SimpleLog - INFO: - episode: 5035, ep_reward: 137.882, ep_step: 1000 2023-05-30 02:35:56 - SimpleLog - INFO: - episode: 5036, ep_reward: -28.440, ep_step: 697 2023-05-30 02:35:58 - SimpleLog - INFO: - episode: 5037, ep_reward: 80.441, ep_step: 1000 2023-05-30 02:36:00 - SimpleLog - INFO: - episode: 5038, ep_reward: 76.540, ep_step: 1000 2023-05-30 02:36:03 - SimpleLog - INFO: - episode: 5039, ep_reward: 151.574, ep_step: 1000 2023-05-30 02:36:04 - SimpleLog - INFO: - episode: 5040, ep_reward: -31.999, ep_step: 629 2023-05-30 02:36:06 - SimpleLog - INFO: - episode: 5041, ep_reward: 79.046, ep_step: 1000 2023-05-30 02:36:08 - SimpleLog - INFO: - episode: 5042, ep_reward: -1.641, ep_step: 862 2023-05-30 02:36:10 - SimpleLog - INFO: - update_step: 3850000, online_eval_reward: -68.799 2023-05-30 02:36:11 - SimpleLog - INFO: - episode: 5043, ep_reward: 81.237, ep_step: 1000 2023-05-30 02:36:13 - SimpleLog - INFO: - episode: 5044, ep_reward: -35.580, ep_step: 860 2023-05-30 02:36:15 - SimpleLog - INFO: - episode: 5045, ep_reward: 135.441, ep_step: 1000 2023-05-30 02:36:18 - SimpleLog - INFO: - episode: 5046, ep_reward: 133.255, ep_step: 1000 2023-05-30 02:36:20 - SimpleLog - INFO: - episode: 5047, ep_reward: 1.547, ep_step: 848 2023-05-30 02:36:22 - SimpleLog - INFO: - episode: 5048, ep_reward: 156.813, ep_step: 1000 2023-05-30 02:36:24 - SimpleLog - INFO: - episode: 5049, ep_reward: 129.746, ep_step: 1000 2023-05-30 02:36:26 - SimpleLog - INFO: - episode: 5050, ep_reward: 145.458, ep_step: 1000 2023-05-30 02:36:29 - SimpleLog - INFO: - episode: 5051, ep_reward: 108.377, ep_step: 1000 2023-05-30 02:36:31 - SimpleLog - INFO: - episode: 5052, ep_reward: 111.631, ep_step: 1000 2023-05-30 02:36:33 - SimpleLog - INFO: - episode: 5053, ep_reward: 111.782, ep_step: 1000 2023-05-30 02:36:35 - SimpleLog - INFO: - episode: 5054, ep_reward: 140.346, ep_step: 1000 2023-05-30 02:36:38 - SimpleLog - INFO: - episode: 5055, ep_reward: 142.574, ep_step: 1000 2023-05-30 02:36:40 - SimpleLog - INFO: - episode: 5056, ep_reward: 104.135, ep_step: 1000 2023-05-30 02:36:42 - SimpleLog - INFO: - episode: 5057, ep_reward: 114.559, ep_step: 1000 2023-05-30 02:36:44 - SimpleLog - INFO: - episode: 5058, ep_reward: 132.217, ep_step: 1000 2023-05-30 02:36:47 - SimpleLog - INFO: - episode: 5059, ep_reward: 111.809, ep_step: 1000 2023-05-30 02:36:49 - SimpleLog - INFO: - episode: 5060, ep_reward: 150.066, ep_step: 1000 2023-05-30 02:36:51 - SimpleLog - INFO: - episode: 5061, ep_reward: 148.768, ep_step: 1000 2023-05-30 02:36:53 - SimpleLog - INFO: - episode: 5062, ep_reward: 117.003, ep_step: 1000 2023-05-30 02:36:56 - SimpleLog - INFO: - episode: 5063, ep_reward: 140.160, ep_step: 1000 2023-05-30 02:36:58 - SimpleLog - INFO: - episode: 5064, ep_reward: 131.552, ep_step: 1000 2023-05-30 02:37:00 - SimpleLog - INFO: - episode: 5065, ep_reward: 104.140, ep_step: 1000 2023-05-30 02:37:02 - SimpleLog - INFO: - episode: 5066, ep_reward: 19.934, ep_step: 985 2023-05-30 02:37:04 - SimpleLog - INFO: - episode: 5067, ep_reward: -7.101, ep_step: 665 2023-05-30 02:37:06 - SimpleLog - INFO: - episode: 5068, ep_reward: 142.706, ep_step: 1000 2023-05-30 02:37:08 - SimpleLog - INFO: - episode: 5069, ep_reward: -6.194, ep_step: 699 2023-05-30 02:37:10 - SimpleLog - INFO: - episode: 5070, ep_reward: 138.878, ep_step: 1000 2023-05-30 02:37:12 - SimpleLog - INFO: - episode: 5071, ep_reward: 115.432, ep_step: 1000 2023-05-30 02:37:13 - SimpleLog - INFO: - episode: 5072, ep_reward: -42.195, ep_step: 513 2023-05-30 02:37:15 - SimpleLog - INFO: - episode: 5073, ep_reward: -17.928, ep_step: 655 2023-05-30 02:37:17 - SimpleLog - INFO: - episode: 5074, ep_reward: -34.167, ep_step: 849 2023-05-30 02:37:18 - SimpleLog - INFO: - episode: 5075, ep_reward: -46.516, ep_step: 533 2023-05-30 02:37:20 - SimpleLog - INFO: - episode: 5076, ep_reward: 3.149, ep_step: 729 2023-05-30 02:37:21 - SimpleLog - INFO: - episode: 5077, ep_reward: -59.035, ep_step: 413 2023-05-30 02:37:23 - SimpleLog - INFO: - episode: 5078, ep_reward: 140.068, ep_step: 1000 2023-05-30 02:37:25 - SimpleLog - INFO: - episode: 5079, ep_reward: 118.142, ep_step: 1000 2023-05-30 02:37:27 - SimpleLog - INFO: - episode: 5080, ep_reward: -32.559, ep_step: 601 2023-05-30 02:37:29 - SimpleLog - INFO: - episode: 5081, ep_reward: 129.292, ep_step: 1000 2023-05-30 02:37:30 - SimpleLog - INFO: - episode: 5082, ep_reward: -35.673, ep_step: 705 2023-05-30 02:37:33 - SimpleLog - INFO: - episode: 5083, ep_reward: 112.241, ep_step: 1000 2023-05-30 02:37:35 - SimpleLog - INFO: - episode: 5084, ep_reward: -2.876, ep_step: 987 2023-05-30 02:37:37 - SimpleLog - INFO: - episode: 5085, ep_reward: 124.810, ep_step: 1000 2023-05-30 02:37:39 - SimpleLog - INFO: - episode: 5086, ep_reward: 119.743, ep_step: 1000 2023-05-30 02:37:42 - SimpleLog - INFO: - episode: 5087, ep_reward: 129.219, ep_step: 1000 2023-05-30 02:37:44 - SimpleLog - INFO: - episode: 5088, ep_reward: 133.536, ep_step: 1000 2023-05-30 02:37:46 - SimpleLog - INFO: - episode: 5089, ep_reward: 125.392, ep_step: 1000 2023-05-30 02:37:48 - SimpleLog - INFO: - episode: 5090, ep_reward: 139.191, ep_step: 1000 2023-05-30 02:37:51 - SimpleLog - INFO: - episode: 5091, ep_reward: 123.746, ep_step: 1000 2023-05-30 02:37:53 - SimpleLog - INFO: - episode: 5092, ep_reward: 145.048, ep_step: 1000 2023-05-30 02:37:55 - SimpleLog - INFO: - episode: 5093, ep_reward: 146.454, ep_step: 1000 2023-05-30 02:37:58 - SimpleLog - INFO: - episode: 5094, ep_reward: 157.022, ep_step: 1000 2023-05-30 02:38:00 - SimpleLog - INFO: - episode: 5095, ep_reward: 153.290, ep_step: 1000 2023-05-30 02:38:02 - SimpleLog - INFO: - episode: 5096, ep_reward: 166.770, ep_step: 1000 2023-05-30 02:38:04 - SimpleLog - INFO: - episode: 5097, ep_reward: 170.460, ep_step: 1000 2023-05-30 02:38:07 - SimpleLog - INFO: - episode: 5098, ep_reward: 134.555, ep_step: 1000 2023-05-30 02:38:09 - SimpleLog - INFO: - episode: 5099, ep_reward: 162.520, ep_step: 1000 2023-05-30 02:38:11 - SimpleLog - INFO: - episode: 5100, ep_reward: 137.853, ep_step: 1000 2023-05-30 02:38:13 - SimpleLog - INFO: - episode: 5101, ep_reward: 152.127, ep_step: 1000 2023-05-30 02:38:14 - SimpleLog - INFO: - episode: 5102, ep_reward: -54.916, ep_step: 334 2023-05-30 02:38:15 - SimpleLog - INFO: - episode: 5103, ep_reward: -20.966, ep_step: 638 2023-05-30 02:38:18 - SimpleLog - INFO: - episode: 5104, ep_reward: 155.216, ep_step: 1000 2023-05-30 02:38:20 - SimpleLog - INFO: - episode: 5105, ep_reward: 148.008, ep_step: 1000 2023-05-30 02:38:22 - SimpleLog - INFO: - episode: 5106, ep_reward: 152.385, ep_step: 1000 2023-05-30 02:38:24 - SimpleLog - INFO: - episode: 5107, ep_reward: 45.976, ep_step: 970 2023-05-30 02:38:27 - SimpleLog - INFO: - episode: 5108, ep_reward: 143.766, ep_step: 1000 2023-05-30 02:38:29 - SimpleLog - INFO: - episode: 5109, ep_reward: 129.218, ep_step: 1000 2023-05-30 02:38:31 - SimpleLog - INFO: - episode: 5110, ep_reward: 149.653, ep_step: 1000 2023-05-30 02:38:33 - SimpleLog - INFO: - episode: 5111, ep_reward: 27.355, ep_step: 871 2023-05-30 02:38:35 - SimpleLog - INFO: - episode: 5112, ep_reward: 114.718, ep_step: 1000 2023-05-30 02:38:38 - SimpleLog - INFO: - episode: 5113, ep_reward: 141.290, ep_step: 1000 2023-05-30 02:38:40 - SimpleLog - INFO: - episode: 5114, ep_reward: 164.053, ep_step: 1000 2023-05-30 02:38:41 - SimpleLog - INFO: - episode: 5115, ep_reward: 11.098, ep_step: 611 2023-05-30 02:38:44 - SimpleLog - INFO: - episode: 5116, ep_reward: 133.864, ep_step: 1000 2023-05-30 02:38:46 - SimpleLog - INFO: - episode: 5117, ep_reward: 158.662, ep_step: 1000 2023-05-30 02:38:48 - SimpleLog - INFO: - episode: 5118, ep_reward: 144.868, ep_step: 1000 2023-05-30 02:38:50 - SimpleLog - INFO: - update_step: 3920000, online_eval_reward: 163.486 2023-05-30 02:38:52 - SimpleLog - INFO: - episode: 5119, ep_reward: -41.312, ep_step: 961 2023-05-30 02:38:54 - SimpleLog - INFO: - episode: 5120, ep_reward: 147.565, ep_step: 1000 2023-05-30 02:38:57 - SimpleLog - INFO: - episode: 5121, ep_reward: 166.856, ep_step: 1000 2023-05-30 02:38:59 - SimpleLog - INFO: - episode: 5122, ep_reward: -9.247, ep_step: 898 2023-05-30 02:39:01 - SimpleLog - INFO: - episode: 5123, ep_reward: 141.196, ep_step: 1000 2023-05-30 02:39:03 - SimpleLog - INFO: - episode: 5124, ep_reward: 122.483, ep_step: 1000 2023-05-30 02:39:05 - SimpleLog - INFO: - episode: 5125, ep_reward: 140.161, ep_step: 1000 2023-05-30 02:39:08 - SimpleLog - INFO: - episode: 5126, ep_reward: 150.378, ep_step: 1000 2023-05-30 02:39:10 - SimpleLog - INFO: - episode: 5127, ep_reward: 156.799, ep_step: 1000 2023-05-30 02:39:12 - SimpleLog - INFO: - episode: 5128, ep_reward: 172.760, ep_step: 1000 2023-05-30 02:39:14 - SimpleLog - INFO: - episode: 5129, ep_reward: 160.104, ep_step: 1000 2023-05-30 02:39:17 - SimpleLog - INFO: - episode: 5130, ep_reward: 79.973, ep_step: 1000 2023-05-30 02:39:19 - SimpleLog - INFO: - episode: 5131, ep_reward: 7.881, ep_step: 964 2023-05-30 02:39:20 - SimpleLog - INFO: - episode: 5132, ep_reward: -38.094, ep_step: 480 2023-05-30 02:39:21 - SimpleLog - INFO: - episode: 5133, ep_reward: -44.298, ep_step: 538 2023-05-30 02:39:23 - SimpleLog - INFO: - episode: 5134, ep_reward: -47.693, ep_step: 578 2023-05-30 02:39:25 - SimpleLog - INFO: - episode: 5135, ep_reward: 136.393, ep_step: 1000 2023-05-30 02:39:27 - SimpleLog - INFO: - episode: 5136, ep_reward: 163.391, ep_step: 1000 2023-05-30 02:39:29 - SimpleLog - INFO: - episode: 5137, ep_reward: 9.255, ep_step: 765 2023-05-30 02:39:31 - SimpleLog - INFO: - episode: 5138, ep_reward: 80.275, ep_step: 1000 2023-05-30 02:39:33 - SimpleLog - INFO: - episode: 5139, ep_reward: 141.901, ep_step: 1000 2023-05-30 02:39:35 - SimpleLog - INFO: - episode: 5140, ep_reward: 22.099, ep_step: 751 2023-05-30 02:39:37 - SimpleLog - INFO: - episode: 5141, ep_reward: 39.604, ep_step: 897 2023-05-30 02:39:39 - SimpleLog - INFO: - episode: 5142, ep_reward: 5.274, ep_step: 729 2023-05-30 02:39:41 - SimpleLog - INFO: - episode: 5143, ep_reward: 176.380, ep_step: 1000 2023-05-30 02:39:42 - SimpleLog - INFO: - episode: 5144, ep_reward: 5.239, ep_step: 544 2023-05-30 02:39:44 - SimpleLog - INFO: - episode: 5145, ep_reward: 172.153, ep_step: 1000 2023-05-30 02:39:45 - SimpleLog - INFO: - episode: 5146, ep_reward: -94.797, ep_step: 147 2023-05-30 02:39:47 - SimpleLog - INFO: - episode: 5147, ep_reward: 155.924, ep_step: 1000 2023-05-30 02:39:49 - SimpleLog - INFO: - episode: 5148, ep_reward: 167.100, ep_step: 1000 2023-05-30 02:39:51 - SimpleLog - INFO: - episode: 5149, ep_reward: 137.097, ep_step: 1000 2023-05-30 02:39:53 - SimpleLog - INFO: - episode: 5150, ep_reward: -15.328, ep_step: 925 2023-05-30 02:39:56 - SimpleLog - INFO: - episode: 5151, ep_reward: 168.693, ep_step: 1000 2023-05-30 02:39:57 - SimpleLog - INFO: - episode: 5152, ep_reward: -71.032, ep_step: 460 2023-05-30 02:39:59 - SimpleLog - INFO: - episode: 5153, ep_reward: 140.664, ep_step: 1000 2023-05-30 02:40:01 - SimpleLog - INFO: - episode: 5154, ep_reward: 154.394, ep_step: 1000 2023-05-30 02:40:03 - SimpleLog - INFO: - episode: 5155, ep_reward: 163.962, ep_step: 1000 2023-05-30 02:40:06 - SimpleLog - INFO: - episode: 5156, ep_reward: 138.662, ep_step: 1000 2023-05-30 02:40:08 - SimpleLog - INFO: - episode: 5157, ep_reward: 145.962, ep_step: 1000 2023-05-30 02:40:10 - SimpleLog - INFO: - episode: 5158, ep_reward: 133.611, ep_step: 1000 2023-05-30 02:40:12 - SimpleLog - INFO: - episode: 5159, ep_reward: 133.471, ep_step: 1000 2023-05-30 02:40:15 - SimpleLog - INFO: - episode: 5160, ep_reward: 134.929, ep_step: 1000 2023-05-30 02:40:17 - SimpleLog - INFO: - episode: 5161, ep_reward: 70.811, ep_step: 1000 2023-05-30 02:40:19 - SimpleLog - INFO: - episode: 5162, ep_reward: 145.441, ep_step: 1000 2023-05-30 02:40:21 - SimpleLog - INFO: - episode: 5163, ep_reward: 124.414, ep_step: 1000 2023-05-30 02:40:24 - SimpleLog - INFO: - episode: 5164, ep_reward: 152.736, ep_step: 1000 2023-05-30 02:40:26 - SimpleLog - INFO: - episode: 5165, ep_reward: 77.844, ep_step: 1000 2023-05-30 02:40:28 - SimpleLog - INFO: - episode: 5166, ep_reward: 83.215, ep_step: 1000 2023-05-30 02:40:30 - SimpleLog - INFO: - episode: 5167, ep_reward: 109.624, ep_step: 1000 2023-05-30 02:40:33 - SimpleLog - INFO: - episode: 5168, ep_reward: 35.828, ep_step: 911 2023-05-30 02:40:35 - SimpleLog - INFO: - episode: 5169, ep_reward: 70.482, ep_step: 1000 2023-05-30 02:40:37 - SimpleLog - INFO: - episode: 5170, ep_reward: 107.839, ep_step: 1000 2023-05-30 02:40:39 - SimpleLog - INFO: - episode: 5171, ep_reward: 152.018, ep_step: 1000 2023-05-30 02:40:42 - SimpleLog - INFO: - episode: 5172, ep_reward: 139.697, ep_step: 1000 2023-05-30 02:40:44 - SimpleLog - INFO: - episode: 5173, ep_reward: 69.239, ep_step: 1000 2023-05-30 02:40:46 - SimpleLog - INFO: - episode: 5174, ep_reward: 79.047, ep_step: 1000 2023-05-30 02:40:48 - SimpleLog - INFO: - episode: 5175, ep_reward: 76.494, ep_step: 1000 2023-05-30 02:40:50 - SimpleLog - INFO: - episode: 5176, ep_reward: -24.762, ep_step: 886 2023-05-30 02:40:53 - SimpleLog - INFO: - episode: 5177, ep_reward: 125.420, ep_step: 1000 2023-05-30 02:40:55 - SimpleLog - INFO: - episode: 5178, ep_reward: 149.552, ep_step: 1000 2023-05-30 02:40:57 - SimpleLog - INFO: - episode: 5179, ep_reward: 136.028, ep_step: 1000 2023-05-30 02:40:58 - SimpleLog - INFO: - episode: 5180, ep_reward: -43.993, ep_step: 507 2023-05-30 02:41:00 - SimpleLog - INFO: - episode: 5181, ep_reward: 161.014, ep_step: 1000 2023-05-30 02:41:02 - SimpleLog - INFO: - episode: 5182, ep_reward: -50.744, ep_step: 579 2023-05-30 02:41:04 - SimpleLog - INFO: - episode: 5183, ep_reward: 137.000, ep_step: 1000 2023-05-30 02:41:06 - SimpleLog - INFO: - episode: 5184, ep_reward: 79.559, ep_step: 1000 2023-05-30 02:41:08 - SimpleLog - INFO: - episode: 5185, ep_reward: 114.047, ep_step: 1000 2023-05-30 02:41:10 - SimpleLog - INFO: - episode: 5186, ep_reward: 14.987, ep_step: 840 2023-05-30 02:41:13 - SimpleLog - INFO: - episode: 5187, ep_reward: 132.182, ep_step: 1000 2023-05-30 02:41:15 - SimpleLog - INFO: - episode: 5188, ep_reward: 125.151, ep_step: 1000 2023-05-30 02:41:17 - SimpleLog - INFO: - episode: 5189, ep_reward: 140.565, ep_step: 1000 2023-05-30 02:41:19 - SimpleLog - INFO: - episode: 5190, ep_reward: 133.398, ep_step: 1000 2023-05-30 02:41:22 - SimpleLog - INFO: - episode: 5191, ep_reward: 130.887, ep_step: 1000 2023-05-30 02:41:24 - SimpleLog - INFO: - episode: 5192, ep_reward: 145.459, ep_step: 1000 2023-05-30 02:41:26 - SimpleLog - INFO: - episode: 5193, ep_reward: 139.671, ep_step: 1000 2023-05-30 02:41:30 - SimpleLog - INFO: - update_step: 3990000, online_eval_reward: 124.327 2023-05-30 02:41:30 - SimpleLog - INFO: - episode: 5194, ep_reward: 125.944, ep_step: 1000 2023-05-30 02:41:32 - SimpleLog - INFO: - episode: 5195, ep_reward: -1.489, ep_step: 673 2023-05-30 02:41:34 - SimpleLog - INFO: - episode: 5196, ep_reward: 138.267, ep_step: 1000 2023-05-30 02:41:36 - SimpleLog - INFO: - episode: 5197, ep_reward: 123.722, ep_step: 1000 2023-05-30 02:41:38 - SimpleLog - INFO: - episode: 5198, ep_reward: -3.315, ep_step: 923 2023-05-30 02:41:40 - SimpleLog - INFO: - episode: 5199, ep_reward: 134.625, ep_step: 1000 2023-05-30 02:41:42 - SimpleLog - INFO: - episode: 5200, ep_reward: -7.840, ep_step: 686 2023-05-30 02:41:44 - SimpleLog - INFO: - episode: 5201, ep_reward: -50.147, ep_step: 744 2023-05-30 02:41:46 - SimpleLog - INFO: - episode: 5202, ep_reward: 22.737, ep_step: 843 2023-05-30 02:41:48 - SimpleLog - INFO: - episode: 5203, ep_reward: 146.930, ep_step: 1000 2023-05-30 02:41:50 - SimpleLog - INFO: - episode: 5204, ep_reward: 139.432, ep_step: 1000 2023-05-30 02:41:50 - SimpleLog - INFO: - episode: 5205, ep_reward: -139.479, ep_step: 170 2023-05-30 02:41:53 - SimpleLog - INFO: - episode: 5206, ep_reward: 76.318, ep_step: 1000 2023-05-30 02:41:55 - SimpleLog - INFO: - episode: 5207, ep_reward: 160.345, ep_step: 1000 2023-05-30 02:41:57 - SimpleLog - INFO: - episode: 5208, ep_reward: 139.559, ep_step: 1000 2023-05-30 02:41:59 - SimpleLog - INFO: - episode: 5209, ep_reward: 168.445, ep_step: 1000 2023-05-30 02:42:01 - SimpleLog - INFO: - episode: 5210, ep_reward: 7.563, ep_step: 703 2023-05-30 02:42:03 - SimpleLog - INFO: - episode: 5211, ep_reward: 82.818, ep_step: 1000 2023-05-30 02:42:05 - SimpleLog - INFO: - episode: 5212, ep_reward: 148.309, ep_step: 1000 2023-05-30 02:42:08 - SimpleLog - INFO: - episode: 5213, ep_reward: 141.068, ep_step: 1000 2023-05-30 02:42:10 - SimpleLog - INFO: - episode: 5214, ep_reward: 157.111, ep_step: 1000 2023-05-30 02:42:12 - SimpleLog - INFO: - episode: 5215, ep_reward: 150.893, ep_step: 1000 2023-05-30 02:42:14 - SimpleLog - INFO: - episode: 5216, ep_reward: 146.060, ep_step: 1000 2023-05-30 02:42:17 - SimpleLog - INFO: - episode: 5217, ep_reward: 158.231, ep_step: 1000 2023-05-30 02:42:19 - SimpleLog - INFO: - episode: 5218, ep_reward: 179.989, ep_step: 1000 2023-05-30 02:42:21 - SimpleLog - INFO: - episode: 5219, ep_reward: 137.925, ep_step: 1000 2023-05-30 02:42:23 - SimpleLog - INFO: - episode: 5220, ep_reward: 138.376, ep_step: 1000 2023-05-30 02:42:25 - SimpleLog - INFO: - episode: 5221, ep_reward: -9.243, ep_step: 571 2023-05-30 02:42:27 - SimpleLog - INFO: - episode: 5222, ep_reward: 164.021, ep_step: 1000 2023-05-30 02:42:28 - SimpleLog - INFO: - episode: 5223, ep_reward: -15.749, ep_step: 651 2023-05-30 02:42:31 - SimpleLog - INFO: - episode: 5224, ep_reward: 131.964, ep_step: 1000 2023-05-30 02:42:33 - SimpleLog - INFO: - episode: 5225, ep_reward: 174.717, ep_step: 1000 2023-05-30 02:42:35 - SimpleLog - INFO: - episode: 5226, ep_reward: 156.078, ep_step: 1000 2023-05-30 02:42:37 - SimpleLog - INFO: - episode: 5227, ep_reward: 173.125, ep_step: 1000 2023-05-30 02:42:40 - SimpleLog - INFO: - episode: 5228, ep_reward: 168.917, ep_step: 1000 2023-05-30 02:42:42 - SimpleLog - INFO: - episode: 5229, ep_reward: 174.344, ep_step: 1000 2023-05-30 02:42:44 - SimpleLog - INFO: - episode: 5230, ep_reward: 161.295, ep_step: 1000 2023-05-30 02:42:46 - SimpleLog - INFO: - episode: 5231, ep_reward: 129.921, ep_step: 1000 2023-05-30 02:42:48 - SimpleLog - INFO: - episode: 5232, ep_reward: -42.541, ep_step: 498 2023-05-30 02:42:50 - SimpleLog - INFO: - episode: 5233, ep_reward: -48.346, ep_step: 1000 2023-05-30 02:42:52 - SimpleLog - INFO: - episode: 5234, ep_reward: 141.104, ep_step: 1000 2023-05-30 02:42:53 - SimpleLog - INFO: - episode: 5235, ep_reward: -43.755, ep_step: 554 2023-05-30 02:42:55 - SimpleLog - INFO: - episode: 5236, ep_reward: 11.456, ep_step: 813 2023-05-30 02:42:57 - SimpleLog - INFO: - episode: 5237, ep_reward: 2.286, ep_step: 966 2023-05-30 02:42:59 - SimpleLog - INFO: - episode: 5238, ep_reward: -28.813, ep_step: 842 2023-05-30 02:43:01 - SimpleLog - INFO: - episode: 5239, ep_reward: -24.155, ep_step: 668 2023-05-30 02:43:02 - SimpleLog - INFO: - episode: 5240, ep_reward: -40.898, ep_step: 596 2023-05-30 02:43:02 - SimpleLog - INFO: - episode: 5241, ep_reward: -106.828, ep_step: 93 2023-05-30 02:43:03 - SimpleLog - INFO: - episode: 5242, ep_reward: -39.618, ep_step: 499 2023-05-30 02:43:06 - SimpleLog - INFO: - episode: 5243, ep_reward: 2.006, ep_step: 1000 2023-05-30 02:43:08 - SimpleLog - INFO: - episode: 5244, ep_reward: 8.403, ep_step: 837 2023-05-30 02:43:10 - SimpleLog - INFO: - episode: 5245, ep_reward: 87.819, ep_step: 1000 2023-05-30 02:43:12 - SimpleLog - INFO: - episode: 5246, ep_reward: 8.829, ep_step: 1000 2023-05-30 02:43:14 - SimpleLog - INFO: - episode: 5247, ep_reward: 133.421, ep_step: 1000 2023-05-30 02:43:17 - SimpleLog - INFO: - episode: 5248, ep_reward: 131.768, ep_step: 1000 2023-05-30 02:43:19 - SimpleLog - INFO: - episode: 5249, ep_reward: 140.219, ep_step: 1000 2023-05-30 02:43:21 - SimpleLog - INFO: - episode: 5250, ep_reward: 137.234, ep_step: 1000 2023-05-30 02:43:23 - SimpleLog - INFO: - episode: 5251, ep_reward: 121.481, ep_step: 1000 2023-05-30 02:43:25 - SimpleLog - INFO: - episode: 5252, ep_reward: -6.230, ep_step: 745 2023-05-30 02:43:27 - SimpleLog - INFO: - episode: 5253, ep_reward: -22.245, ep_step: 729 2023-05-30 02:43:29 - SimpleLog - INFO: - episode: 5254, ep_reward: 128.440, ep_step: 1000 2023-05-30 02:43:31 - SimpleLog - INFO: - episode: 5255, ep_reward: 116.565, ep_step: 1000 2023-05-30 02:43:33 - SimpleLog - INFO: - episode: 5256, ep_reward: 153.545, ep_step: 1000 2023-05-30 02:43:36 - SimpleLog - INFO: - episode: 5257, ep_reward: 159.866, ep_step: 1000 2023-05-30 02:43:38 - SimpleLog - INFO: - episode: 5258, ep_reward: 130.329, ep_step: 1000 2023-05-30 02:43:40 - SimpleLog - INFO: - episode: 5259, ep_reward: 138.510, ep_step: 1000 2023-05-30 02:43:42 - SimpleLog - INFO: - episode: 5260, ep_reward: 109.805, ep_step: 1000 2023-05-30 02:43:45 - SimpleLog - INFO: - episode: 5261, ep_reward: 139.133, ep_step: 1000 2023-05-30 02:43:47 - SimpleLog - INFO: - episode: 5262, ep_reward: 125.661, ep_step: 1000 2023-05-30 02:43:49 - SimpleLog - INFO: - episode: 5263, ep_reward: 129.098, ep_step: 1000 2023-05-30 02:43:51 - SimpleLog - INFO: - episode: 5264, ep_reward: 104.825, ep_step: 1000 2023-05-30 02:43:54 - SimpleLog - INFO: - episode: 5265, ep_reward: 4.318, ep_step: 1000 2023-05-30 02:43:56 - SimpleLog - INFO: - episode: 5266, ep_reward: 78.856, ep_step: 1000 2023-05-30 02:43:57 - SimpleLog - INFO: - episode: 5267, ep_reward: -72.469, ep_step: 528 2023-05-30 02:43:59 - SimpleLog - INFO: - episode: 5268, ep_reward: 102.554, ep_step: 1000 2023-05-30 02:44:02 - SimpleLog - INFO: - episode: 5269, ep_reward: 114.090, ep_step: 1000 2023-05-30 02:44:04 - SimpleLog - INFO: - episode: 5270, ep_reward: 87.742, ep_step: 1000 2023-05-30 02:44:06 - SimpleLog - INFO: - episode: 5271, ep_reward: 80.160, ep_step: 1000 2023-05-30 02:44:09 - SimpleLog - INFO: - update_step: 4060000, online_eval_reward: 8.097 2023-05-30 02:44:10 - SimpleLog - INFO: - episode: 5272, ep_reward: 138.191, ep_step: 1000 2023-05-30 02:44:13 - SimpleLog - INFO: - episode: 5273, ep_reward: 4.870, ep_step: 1000 2023-05-30 02:44:15 - SimpleLog - INFO: - episode: 5274, ep_reward: 153.677, ep_step: 1000 2023-05-30 02:44:17 - SimpleLog - INFO: - episode: 5275, ep_reward: 132.967, ep_step: 1000 2023-05-30 02:44:19 - SimpleLog - INFO: - episode: 5276, ep_reward: 2.098, ep_step: 1000 2023-05-30 02:44:21 - SimpleLog - INFO: - episode: 5277, ep_reward: -5.508, ep_step: 794 2023-05-30 02:44:23 - SimpleLog - INFO: - episode: 5278, ep_reward: -35.199, ep_step: 744 2023-05-30 02:44:25 - SimpleLog - INFO: - episode: 5279, ep_reward: 101.193, ep_step: 1000 2023-05-30 02:44:27 - SimpleLog - INFO: - episode: 5280, ep_reward: 146.402, ep_step: 1000 2023-05-30 02:44:30 - SimpleLog - INFO: - episode: 5281, ep_reward: 137.051, ep_step: 1000 2023-05-30 02:44:32 - SimpleLog - INFO: - episode: 5282, ep_reward: 144.108, ep_step: 1000 2023-05-30 02:44:34 - SimpleLog - INFO: - episode: 5283, ep_reward: 141.850, ep_step: 1000 2023-05-30 02:44:36 - SimpleLog - INFO: - episode: 5284, ep_reward: 16.372, ep_step: 838 2023-05-30 02:44:38 - SimpleLog - INFO: - episode: 5285, ep_reward: 151.717, ep_step: 1000 2023-05-30 02:44:41 - SimpleLog - INFO: - episode: 5286, ep_reward: 162.027, ep_step: 1000 2023-05-30 02:44:43 - SimpleLog - INFO: - episode: 5287, ep_reward: -5.779, ep_step: 860 2023-05-30 02:44:44 - SimpleLog - INFO: - episode: 5288, ep_reward: 4.805, ep_step: 699 2023-05-30 02:44:46 - SimpleLog - INFO: - episode: 5289, ep_reward: 151.260, ep_step: 1000 2023-05-30 02:44:49 - SimpleLog - INFO: - episode: 5290, ep_reward: 149.330, ep_step: 1000 2023-05-30 02:44:51 - SimpleLog - INFO: - episode: 5291, ep_reward: 131.058, ep_step: 1000 2023-05-30 02:44:53 - SimpleLog - INFO: - episode: 5292, ep_reward: 146.673, ep_step: 1000 2023-05-30 02:44:55 - SimpleLog - INFO: - episode: 5293, ep_reward: 111.961, ep_step: 1000 2023-05-30 02:44:57 - SimpleLog - INFO: - episode: 5294, ep_reward: -8.493, ep_step: 860 2023-05-30 02:45:00 - SimpleLog - INFO: - episode: 5295, ep_reward: 122.901, ep_step: 1000 2023-05-30 02:45:02 - SimpleLog - INFO: - episode: 5296, ep_reward: 135.688, ep_step: 1000 2023-05-30 02:45:04 - SimpleLog - INFO: - episode: 5297, ep_reward: 119.143, ep_step: 1000 2023-05-30 02:45:06 - SimpleLog - INFO: - episode: 5298, ep_reward: 148.156, ep_step: 1000 2023-05-30 02:45:09 - SimpleLog - INFO: - episode: 5299, ep_reward: 118.688, ep_step: 1000 2023-05-30 02:45:11 - SimpleLog - INFO: - episode: 5300, ep_reward: 136.290, ep_step: 1000 2023-05-30 02:45:13 - SimpleLog - INFO: - episode: 5301, ep_reward: 139.487, ep_step: 1000 2023-05-30 02:45:15 - SimpleLog - INFO: - episode: 5302, ep_reward: 128.639, ep_step: 1000 2023-05-30 02:45:18 - SimpleLog - INFO: - episode: 5303, ep_reward: 137.196, ep_step: 1000 2023-05-30 02:45:20 - SimpleLog - INFO: - episode: 5304, ep_reward: 134.582, ep_step: 1000 2023-05-30 02:45:22 - SimpleLog - INFO: - episode: 5305, ep_reward: 14.137, ep_step: 838 2023-05-30 02:45:24 - SimpleLog - INFO: - episode: 5306, ep_reward: 131.360, ep_step: 1000 2023-05-30 02:45:26 - SimpleLog - INFO: - episode: 5307, ep_reward: 131.305, ep_step: 1000 2023-05-30 02:45:28 - SimpleLog - INFO: - episode: 5308, ep_reward: 137.190, ep_step: 1000 2023-05-30 02:45:31 - SimpleLog - INFO: - episode: 5309, ep_reward: 86.202, ep_step: 1000 2023-05-30 02:45:33 - SimpleLog - INFO: - episode: 5310, ep_reward: 121.655, ep_step: 1000 2023-05-30 02:45:35 - SimpleLog - INFO: - episode: 5311, ep_reward: 128.045, ep_step: 1000 2023-05-30 02:45:37 - SimpleLog - INFO: - episode: 5312, ep_reward: -21.940, ep_step: 909 2023-05-30 02:45:39 - SimpleLog - INFO: - episode: 5313, ep_reward: 142.287, ep_step: 1000 2023-05-30 02:45:42 - SimpleLog - INFO: - episode: 5314, ep_reward: 126.355, ep_step: 1000 2023-05-30 02:45:44 - SimpleLog - INFO: - episode: 5315, ep_reward: 108.992, ep_step: 1000 2023-05-30 02:45:46 - SimpleLog - INFO: - episode: 5316, ep_reward: 121.720, ep_step: 1000 2023-05-30 02:45:48 - SimpleLog - INFO: - episode: 5317, ep_reward: 130.771, ep_step: 1000 2023-05-30 02:45:50 - SimpleLog - INFO: - episode: 5318, ep_reward: -88.520, ep_step: 709 2023-05-30 02:45:52 - SimpleLog - INFO: - episode: 5319, ep_reward: 127.522, ep_step: 1000 2023-05-30 02:45:55 - SimpleLog - INFO: - episode: 5320, ep_reward: 134.505, ep_step: 1000 2023-05-30 02:45:57 - SimpleLog - INFO: - episode: 5321, ep_reward: 125.000, ep_step: 1000 2023-05-30 02:45:59 - SimpleLog - INFO: - episode: 5322, ep_reward: -35.570, ep_step: 895 2023-05-30 02:46:01 - SimpleLog - INFO: - episode: 5323, ep_reward: 137.308, ep_step: 1000 2023-05-30 02:46:03 - SimpleLog - INFO: - episode: 5324, ep_reward: 138.159, ep_step: 1000 2023-05-30 02:46:05 - SimpleLog - INFO: - episode: 5325, ep_reward: 150.897, ep_step: 1000 2023-05-30 02:46:08 - SimpleLog - INFO: - episode: 5326, ep_reward: 163.732, ep_step: 1000 2023-05-30 02:46:10 - SimpleLog - INFO: - episode: 5327, ep_reward: 141.636, ep_step: 1000 2023-05-30 02:46:12 - SimpleLog - INFO: - episode: 5328, ep_reward: 171.923, ep_step: 1000 2023-05-30 02:46:14 - SimpleLog - INFO: - episode: 5329, ep_reward: 142.572, ep_step: 1000 2023-05-30 02:46:17 - SimpleLog - INFO: - episode: 5330, ep_reward: 178.624, ep_step: 1000 2023-05-30 02:46:19 - SimpleLog - INFO: - episode: 5331, ep_reward: 166.171, ep_step: 1000 2023-05-30 02:46:21 - SimpleLog - INFO: - episode: 5332, ep_reward: 131.090, ep_step: 1000 2023-05-30 02:46:23 - SimpleLog - INFO: - episode: 5333, ep_reward: 142.949, ep_step: 1000 2023-05-30 02:46:26 - SimpleLog - INFO: - episode: 5334, ep_reward: 72.849, ep_step: 1000 2023-05-30 02:46:28 - SimpleLog - INFO: - episode: 5335, ep_reward: 167.785, ep_step: 1000 2023-05-30 02:46:30 - SimpleLog - INFO: - episode: 5336, ep_reward: 149.839, ep_step: 1000 2023-05-30 02:46:33 - SimpleLog - INFO: - episode: 5337, ep_reward: 171.151, ep_step: 1000 2023-05-30 02:46:35 - SimpleLog - INFO: - episode: 5338, ep_reward: 170.127, ep_step: 1000 2023-05-30 02:46:37 - SimpleLog - INFO: - episode: 5339, ep_reward: 157.943, ep_step: 1000 2023-05-30 02:46:39 - SimpleLog - INFO: - episode: 5340, ep_reward: 13.161, ep_step: 799 2023-05-30 02:46:41 - SimpleLog - INFO: - episode: 5341, ep_reward: 171.675, ep_step: 1000 2023-05-30 02:46:43 - SimpleLog - INFO: - episode: 5342, ep_reward: 154.319, ep_step: 1000 2023-05-30 02:46:45 - SimpleLog - INFO: - episode: 5343, ep_reward: -43.463, ep_step: 522 2023-05-30 02:46:47 - SimpleLog - INFO: - episode: 5344, ep_reward: 121.238, ep_step: 1000 2023-05-30 02:46:49 - SimpleLog - INFO: - update_step: 4130000, online_eval_reward: 178.075 2023-05-30 02:46:51 - SimpleLog - INFO: - episode: 5345, ep_reward: 159.492, ep_step: 1000 2023-05-30 02:46:53 - SimpleLog - INFO: - episode: 5346, ep_reward: 77.626, ep_step: 1000 2023-05-30 02:46:55 - SimpleLog - INFO: - episode: 5347, ep_reward: 69.609, ep_step: 1000 2023-05-30 02:46:58 - SimpleLog - INFO: - episode: 5348, ep_reward: 173.213, ep_step: 1000 2023-05-30 02:47:00 - SimpleLog - INFO: - episode: 5349, ep_reward: 151.994, ep_step: 1000 2023-05-30 02:47:02 - SimpleLog - INFO: - episode: 5350, ep_reward: 8.763, ep_step: 888 2023-05-30 02:47:04 - SimpleLog - INFO: - episode: 5351, ep_reward: 24.437, ep_step: 958 2023-05-30 02:47:06 - SimpleLog - INFO: - episode: 5352, ep_reward: 152.150, ep_step: 1000 2023-05-30 02:47:09 - SimpleLog - INFO: - episode: 5353, ep_reward: 156.208, ep_step: 1000 2023-05-30 02:47:11 - SimpleLog - INFO: - episode: 5354, ep_reward: 147.725, ep_step: 1000 2023-05-30 02:47:12 - SimpleLog - INFO: - episode: 5355, ep_reward: -72.157, ep_step: 629 2023-05-30 02:47:14 - SimpleLog - INFO: - episode: 5356, ep_reward: 71.070, ep_step: 1000 2023-05-30 02:47:17 - SimpleLog - INFO: - episode: 5357, ep_reward: 133.488, ep_step: 1000 2023-05-30 02:47:19 - SimpleLog - INFO: - episode: 5358, ep_reward: 154.685, ep_step: 1000 2023-05-30 02:47:21 - SimpleLog - INFO: - episode: 5359, ep_reward: -16.315, ep_step: 752 2023-05-30 02:47:23 - SimpleLog - INFO: - episode: 5360, ep_reward: 18.243, ep_step: 968 2023-05-30 02:47:25 - SimpleLog - INFO: - episode: 5361, ep_reward: 4.597, ep_step: 869 2023-05-30 02:47:27 - SimpleLog - INFO: - episode: 5362, ep_reward: 170.946, ep_step: 1000 2023-05-30 02:47:29 - SimpleLog - INFO: - episode: 5363, ep_reward: 137.051, ep_step: 1000 2023-05-30 02:47:32 - SimpleLog - INFO: - episode: 5364, ep_reward: 82.321, ep_step: 1000 2023-05-30 02:47:34 - SimpleLog - INFO: - episode: 5365, ep_reward: 79.822, ep_step: 1000 2023-05-30 02:47:36 - SimpleLog - INFO: - episode: 5366, ep_reward: 134.533, ep_step: 1000 2023-05-30 02:47:38 - SimpleLog - INFO: - episode: 5367, ep_reward: 136.205, ep_step: 1000 2023-05-30 02:47:41 - SimpleLog - INFO: - episode: 5368, ep_reward: 178.845, ep_step: 1000 2023-05-30 02:47:42 - SimpleLog - INFO: - episode: 5369, ep_reward: 12.052, ep_step: 687 2023-05-30 02:47:44 - SimpleLog - INFO: - episode: 5370, ep_reward: 145.443, ep_step: 1000 2023-05-30 02:47:47 - SimpleLog - INFO: - episode: 5371, ep_reward: 20.946, ep_step: 950 2023-05-30 02:47:49 - SimpleLog - INFO: - episode: 5372, ep_reward: 140.725, ep_step: 1000 2023-05-30 02:47:51 - SimpleLog - INFO: - episode: 5373, ep_reward: 120.229, ep_step: 1000 2023-05-30 02:47:53 - SimpleLog - INFO: - episode: 5374, ep_reward: 142.447, ep_step: 1000 2023-05-30 02:47:56 - SimpleLog - INFO: - episode: 5375, ep_reward: 171.216, ep_step: 1000 2023-05-30 02:47:58 - SimpleLog - INFO: - episode: 5376, ep_reward: 155.641, ep_step: 1000 2023-05-30 02:47:59 - SimpleLog - INFO: - episode: 5377, ep_reward: -76.147, ep_step: 510 2023-05-30 02:48:01 - SimpleLog - INFO: - episode: 5378, ep_reward: 86.986, ep_step: 1000 2023-05-30 02:48:03 - SimpleLog - INFO: - episode: 5379, ep_reward: -8.802, ep_step: 743 2023-05-30 02:48:05 - SimpleLog - INFO: - episode: 5380, ep_reward: 143.182, ep_step: 1000 2023-05-30 02:48:07 - SimpleLog - INFO: - episode: 5381, ep_reward: 5.564, ep_step: 839 2023-05-30 02:48:09 - SimpleLog - INFO: - episode: 5382, ep_reward: 130.110, ep_step: 1000 2023-05-30 02:48:11 - SimpleLog - INFO: - episode: 5383, ep_reward: 132.170, ep_step: 1000 2023-05-30 02:48:14 - SimpleLog - INFO: - episode: 5384, ep_reward: 152.558, ep_step: 1000 2023-05-30 02:48:15 - SimpleLog - INFO: - episode: 5385, ep_reward: 11.323, ep_step: 762 2023-05-30 02:48:18 - SimpleLog - INFO: - episode: 5386, ep_reward: 171.540, ep_step: 1000 2023-05-30 02:48:20 - SimpleLog - INFO: - episode: 5387, ep_reward: 4.387, ep_step: 963 2023-05-30 02:48:22 - SimpleLog - INFO: - episode: 5388, ep_reward: 144.366, ep_step: 1000 2023-05-30 02:48:24 - SimpleLog - INFO: - episode: 5389, ep_reward: 132.768, ep_step: 1000 2023-05-30 02:48:27 - SimpleLog - INFO: - episode: 5390, ep_reward: 168.914, ep_step: 1000 2023-05-30 02:48:29 - SimpleLog - INFO: - episode: 5391, ep_reward: 82.777, ep_step: 1000 2023-05-30 02:48:29 - SimpleLog - INFO: - episode: 5392, ep_reward: -101.067, ep_step: 185 2023-05-30 02:48:32 - SimpleLog - INFO: - episode: 5393, ep_reward: 148.247, ep_step: 1000 2023-05-30 02:48:34 - SimpleLog - INFO: - episode: 5394, ep_reward: 138.548, ep_step: 1000 2023-05-30 02:48:35 - SimpleLog - INFO: - episode: 5395, ep_reward: -7.680, ep_step: 725 2023-05-30 02:48:37 - SimpleLog - INFO: - episode: 5396, ep_reward: -56.046, ep_step: 610 2023-05-30 02:48:37 - SimpleLog - INFO: - episode: 5397, ep_reward: -93.750, ep_step: 244 2023-05-30 02:48:40 - SimpleLog - INFO: - episode: 5398, ep_reward: 126.282, ep_step: 1000 2023-05-30 02:48:42 - SimpleLog - INFO: - episode: 5399, ep_reward: 138.748, ep_step: 1000 2023-05-30 02:48:44 - SimpleLog - INFO: - episode: 5400, ep_reward: 63.255, ep_step: 1000 2023-05-30 02:48:46 - SimpleLog - INFO: - episode: 5401, ep_reward: 149.165, ep_step: 1000 2023-05-30 02:48:48 - SimpleLog - INFO: - episode: 5402, ep_reward: -20.310, ep_step: 825 2023-05-30 02:48:50 - SimpleLog - INFO: - episode: 5403, ep_reward: 122.005, ep_step: 1000 2023-05-30 02:48:53 - SimpleLog - INFO: - episode: 5404, ep_reward: 147.687, ep_step: 1000 2023-05-30 02:48:55 - SimpleLog - INFO: - episode: 5405, ep_reward: -6.247, ep_step: 794 2023-05-30 02:48:57 - SimpleLog - INFO: - episode: 5406, ep_reward: -13.681, ep_step: 968 2023-05-30 02:48:59 - SimpleLog - INFO: - episode: 5407, ep_reward: 129.809, ep_step: 1000 2023-05-30 02:49:01 - SimpleLog - INFO: - episode: 5408, ep_reward: 2.566, ep_step: 1000 2023-05-30 02:49:03 - SimpleLog - INFO: - episode: 5409, ep_reward: 165.219, ep_step: 1000 2023-05-30 02:49:06 - SimpleLog - INFO: - episode: 5410, ep_reward: -1.304, ep_step: 990 2023-05-30 02:49:08 - SimpleLog - INFO: - episode: 5411, ep_reward: 114.705, ep_step: 1000 2023-05-30 02:49:10 - SimpleLog - INFO: - episode: 5412, ep_reward: 142.564, ep_step: 1000 2023-05-30 02:49:12 - SimpleLog - INFO: - episode: 5413, ep_reward: 106.692, ep_step: 1000 2023-05-30 02:49:15 - SimpleLog - INFO: - episode: 5414, ep_reward: 169.031, ep_step: 1000 2023-05-30 02:49:17 - SimpleLog - INFO: - episode: 5415, ep_reward: 151.757, ep_step: 1000 2023-05-30 02:49:19 - SimpleLog - INFO: - episode: 5416, ep_reward: 142.320, ep_step: 1000 2023-05-30 02:49:21 - SimpleLog - INFO: - episode: 5417, ep_reward: 2.695, ep_step: 920 2023-05-30 02:49:23 - SimpleLog - INFO: - episode: 5418, ep_reward: 11.870, ep_step: 823 2023-05-30 02:49:25 - SimpleLog - INFO: - episode: 5419, ep_reward: 135.253, ep_step: 1000 2023-05-30 02:49:28 - SimpleLog - INFO: - update_step: 4200000, online_eval_reward: 145.357 2023-05-30 02:49:30 - SimpleLog - INFO: - episode: 5420, ep_reward: 167.456, ep_step: 1000 2023-05-30 02:49:32 - SimpleLog - INFO: - episode: 5421, ep_reward: 133.532, ep_step: 1000 2023-05-30 02:49:33 - SimpleLog - INFO: - episode: 5422, ep_reward: -88.406, ep_step: 514 2023-05-30 02:49:35 - SimpleLog - INFO: - episode: 5423, ep_reward: 4.494, ep_step: 949 2023-05-30 02:49:37 - SimpleLog - INFO: - episode: 5424, ep_reward: -65.525, ep_step: 1000 2023-05-30 02:49:40 - SimpleLog - INFO: - episode: 5425, ep_reward: 71.814, ep_step: 1000 2023-05-30 02:49:42 - SimpleLog - INFO: - episode: 5426, ep_reward: 15.020, ep_step: 838 2023-05-30 02:49:44 - SimpleLog - INFO: - episode: 5427, ep_reward: 76.899, ep_step: 1000 2023-05-30 02:49:45 - SimpleLog - INFO: - episode: 5428, ep_reward: -32.954, ep_step: 462 2023-05-30 02:49:46 - SimpleLog - INFO: - episode: 5429, ep_reward: -33.564, ep_step: 488 2023-05-30 02:49:48 - SimpleLog - INFO: - episode: 5430, ep_reward: 5.144, ep_step: 867 2023-05-30 02:49:50 - SimpleLog - INFO: - episode: 5431, ep_reward: 163.752, ep_step: 1000 2023-05-30 02:49:53 - SimpleLog - INFO: - episode: 5432, ep_reward: 67.998, ep_step: 1000 2023-05-30 02:49:55 - SimpleLog - INFO: - episode: 5433, ep_reward: 122.538, ep_step: 1000 2023-05-30 02:49:57 - SimpleLog - INFO: - episode: 5434, ep_reward: 70.393, ep_step: 1000 2023-05-30 02:49:59 - SimpleLog - INFO: - episode: 5435, ep_reward: 72.659, ep_step: 1000 2023-05-30 02:50:02 - SimpleLog - INFO: - episode: 5436, ep_reward: 152.343, ep_step: 1000 2023-05-30 02:50:02 - SimpleLog - INFO: - episode: 5437, ep_reward: -79.953, ep_step: 364 2023-05-30 02:50:03 - SimpleLog - INFO: - episode: 5438, ep_reward: -38.999, ep_step: 453 2023-05-30 02:50:06 - SimpleLog - INFO: - episode: 5439, ep_reward: 140.523, ep_step: 1000 2023-05-30 02:50:08 - SimpleLog - INFO: - episode: 5440, ep_reward: 62.885, ep_step: 1000 2023-05-30 02:50:10 - SimpleLog - INFO: - episode: 5441, ep_reward: 4.905, ep_step: 856 2023-05-30 02:50:12 - SimpleLog - INFO: - episode: 5442, ep_reward: 70.018, ep_step: 1000 2023-05-30 02:50:14 - SimpleLog - INFO: - episode: 5443, ep_reward: 48.051, ep_step: 1000 2023-05-30 02:50:17 - SimpleLog - INFO: - episode: 5444, ep_reward: 105.491, ep_step: 1000 2023-05-30 02:50:19 - SimpleLog - INFO: - episode: 5445, ep_reward: 134.725, ep_step: 1000 2023-05-30 02:50:21 - SimpleLog - INFO: - episode: 5446, ep_reward: 149.228, ep_step: 1000 2023-05-30 02:50:23 - SimpleLog - INFO: - episode: 5447, ep_reward: -45.253, ep_step: 935 2023-05-30 02:50:25 - SimpleLog - INFO: - episode: 5448, ep_reward: 8.877, ep_step: 870 2023-05-30 02:50:27 - SimpleLog - INFO: - episode: 5449, ep_reward: 16.104, ep_step: 834 2023-05-30 02:50:29 - SimpleLog - INFO: - episode: 5450, ep_reward: 65.009, ep_step: 1000 2023-05-30 02:50:32 - SimpleLog - INFO: - episode: 5451, ep_reward: 145.090, ep_step: 1000 2023-05-30 02:50:34 - SimpleLog - INFO: - episode: 5452, ep_reward: 107.621, ep_step: 1000 2023-05-30 02:50:36 - SimpleLog - INFO: - episode: 5453, ep_reward: 92.652, ep_step: 1000 2023-05-30 02:50:38 - SimpleLog - INFO: - episode: 5454, ep_reward: 120.238, ep_step: 1000 2023-05-30 02:50:41 - SimpleLog - INFO: - episode: 5455, ep_reward: 90.240, ep_step: 1000 2023-05-30 02:50:43 - SimpleLog - INFO: - episode: 5456, ep_reward: 93.315, ep_step: 1000 2023-05-30 02:50:45 - SimpleLog - INFO: - episode: 5457, ep_reward: 99.256, ep_step: 1000 2023-05-30 02:50:48 - SimpleLog - INFO: - episode: 5458, ep_reward: 114.079, ep_step: 1000 2023-05-30 02:50:50 - SimpleLog - INFO: - episode: 5459, ep_reward: 113.722, ep_step: 1000 2023-05-30 02:50:52 - SimpleLog - INFO: - episode: 5460, ep_reward: -11.757, ep_step: 898 2023-05-30 02:50:52 - SimpleLog - INFO: - episode: 5461, ep_reward: -118.842, ep_step: 149 2023-05-30 02:50:54 - SimpleLog - INFO: - episode: 5462, ep_reward: 135.401, ep_step: 1000 2023-05-30 02:50:56 - SimpleLog - INFO: - episode: 5463, ep_reward: -19.780, ep_step: 910 2023-05-30 02:50:59 - SimpleLog - INFO: - episode: 5464, ep_reward: 150.047, ep_step: 1000 2023-05-30 02:51:01 - SimpleLog - INFO: - episode: 5465, ep_reward: -8.451, ep_step: 853 2023-05-30 02:51:03 - SimpleLog - INFO: - episode: 5466, ep_reward: 147.442, ep_step: 1000 2023-05-30 02:51:05 - SimpleLog - INFO: - episode: 5467, ep_reward: 128.666, ep_step: 1000 2023-05-30 02:51:07 - SimpleLog - INFO: - episode: 5468, ep_reward: 126.240, ep_step: 1000 2023-05-30 02:51:10 - SimpleLog - INFO: - episode: 5469, ep_reward: 152.593, ep_step: 1000 2023-05-30 02:51:12 - SimpleLog - INFO: - episode: 5470, ep_reward: 80.235, ep_step: 1000 2023-05-30 02:51:14 - SimpleLog - INFO: - episode: 5471, ep_reward: 65.805, ep_step: 1000 2023-05-30 02:51:16 - SimpleLog - INFO: - episode: 5472, ep_reward: 103.266, ep_step: 1000 2023-05-30 02:51:19 - SimpleLog - INFO: - episode: 5473, ep_reward: 138.475, ep_step: 1000 2023-05-30 02:51:21 - SimpleLog - INFO: - episode: 5474, ep_reward: 132.027, ep_step: 1000 2023-05-30 02:51:22 - SimpleLog - INFO: - episode: 5475, ep_reward: -49.630, ep_step: 682 2023-05-30 02:51:24 - SimpleLog - INFO: - episode: 5476, ep_reward: -17.310, ep_step: 830 2023-05-30 02:51:26 - SimpleLog - INFO: - episode: 5477, ep_reward: -13.298, ep_step: 773 2023-05-30 02:51:28 - SimpleLog - INFO: - episode: 5478, ep_reward: 151.613, ep_step: 1000 2023-05-30 02:51:31 - SimpleLog - INFO: - episode: 5479, ep_reward: 130.746, ep_step: 1000 2023-05-30 02:51:33 - SimpleLog - INFO: - episode: 5480, ep_reward: 139.138, ep_step: 1000 2023-05-30 02:51:35 - SimpleLog - INFO: - episode: 5481, ep_reward: 106.046, ep_step: 1000 2023-05-30 02:51:37 - SimpleLog - INFO: - episode: 5482, ep_reward: 139.629, ep_step: 1000 2023-05-30 02:51:40 - SimpleLog - INFO: - episode: 5483, ep_reward: 112.105, ep_step: 1000 2023-05-30 02:51:42 - SimpleLog - INFO: - episode: 5484, ep_reward: 83.793, ep_step: 1000 2023-05-30 02:51:44 - SimpleLog - INFO: - episode: 5485, ep_reward: 66.264, ep_step: 1000 2023-05-30 02:51:46 - SimpleLog - INFO: - episode: 5486, ep_reward: 100.636, ep_step: 1000 2023-05-30 02:51:47 - SimpleLog - INFO: - episode: 5487, ep_reward: -117.549, ep_step: 134 2023-05-30 02:51:49 - SimpleLog - INFO: - episode: 5488, ep_reward: 131.012, ep_step: 1000 2023-05-30 02:51:51 - SimpleLog - INFO: - episode: 5489, ep_reward: 140.503, ep_step: 1000 2023-05-30 02:51:53 - SimpleLog - INFO: - episode: 5490, ep_reward: 148.027, ep_step: 1000 2023-05-30 02:51:56 - SimpleLog - INFO: - episode: 5491, ep_reward: 135.550, ep_step: 1000 2023-05-30 02:51:58 - SimpleLog - INFO: - episode: 5492, ep_reward: 79.104, ep_step: 1000 2023-05-30 02:52:00 - SimpleLog - INFO: - episode: 5493, ep_reward: 5.303, ep_step: 868 2023-05-30 02:52:02 - SimpleLog - INFO: - episode: 5494, ep_reward: 92.700, ep_step: 1000 2023-05-30 02:52:04 - SimpleLog - INFO: - episode: 5495, ep_reward: 31.591, ep_step: 968 2023-05-30 02:52:08 - SimpleLog - INFO: - update_step: 4270000, online_eval_reward: 28.808 2023-05-30 02:52:08 - SimpleLog - INFO: - episode: 5496, ep_reward: 9.580, ep_step: 1000 2023-05-30 02:52:10 - SimpleLog - INFO: - episode: 5497, ep_reward: 4.644, ep_step: 1000 2023-05-30 02:52:13 - SimpleLog - INFO: - episode: 5498, ep_reward: -2.862, ep_step: 922 2023-05-30 02:52:15 - SimpleLog - INFO: - episode: 5499, ep_reward: 127.025, ep_step: 1000 2023-05-30 02:52:17 - SimpleLog - INFO: - episode: 5500, ep_reward: 85.731, ep_step: 1000 2023-05-30 02:52:18 - SimpleLog - INFO: - episode: 5501, ep_reward: -36.151, ep_step: 524 2023-05-30 02:52:21 - SimpleLog - INFO: - episode: 5502, ep_reward: 135.531, ep_step: 1000 2023-05-30 02:52:23 - SimpleLog - INFO: - episode: 5503, ep_reward: 160.808, ep_step: 1000 2023-05-30 02:52:25 - SimpleLog - INFO: - episode: 5504, ep_reward: 131.399, ep_step: 1000 2023-05-30 02:52:27 - SimpleLog - INFO: - episode: 5505, ep_reward: 132.693, ep_step: 1000 2023-05-30 02:52:30 - SimpleLog - INFO: - episode: 5506, ep_reward: 3.886, ep_step: 1000 2023-05-30 02:52:32 - SimpleLog - INFO: - episode: 5507, ep_reward: 133.127, ep_step: 1000 2023-05-30 02:52:34 - SimpleLog - INFO: - episode: 5508, ep_reward: -20.204, ep_step: 739 2023-05-30 02:52:36 - SimpleLog - INFO: - episode: 5509, ep_reward: 131.690, ep_step: 1000 2023-05-30 02:52:38 - SimpleLog - INFO: - episode: 5510, ep_reward: 131.391, ep_step: 1000 2023-05-30 02:52:40 - SimpleLog - INFO: - episode: 5511, ep_reward: 148.627, ep_step: 1000 2023-05-30 02:52:41 - SimpleLog - INFO: - episode: 5512, ep_reward: -71.269, ep_step: 423 2023-05-30 02:52:42 - SimpleLog - INFO: - episode: 5513, ep_reward: -53.193, ep_step: 425 2023-05-30 02:52:44 - SimpleLog - INFO: - episode: 5514, ep_reward: 1.768, ep_step: 758 2023-05-30 02:52:46 - SimpleLog - INFO: - episode: 5515, ep_reward: 114.555, ep_step: 1000 2023-05-30 02:52:48 - SimpleLog - INFO: - episode: 5516, ep_reward: 109.438, ep_step: 1000 2023-05-30 02:52:50 - SimpleLog - INFO: - episode: 5517, ep_reward: -1.193, ep_step: 880 2023-05-30 02:52:51 - SimpleLog - INFO: - episode: 5518, ep_reward: -105.390, ep_step: 301 2023-05-30 02:52:53 - SimpleLog - INFO: - episode: 5519, ep_reward: -10.234, ep_step: 905 2023-05-30 02:52:55 - SimpleLog - INFO: - episode: 5520, ep_reward: 131.561, ep_step: 1000 2023-05-30 02:52:56 - SimpleLog - INFO: - episode: 5521, ep_reward: -114.656, ep_step: 82 2023-05-30 02:52:58 - SimpleLog - INFO: - episode: 5522, ep_reward: 109.461, ep_step: 1000 2023-05-30 02:53:00 - SimpleLog - INFO: - episode: 5523, ep_reward: -1.917, ep_step: 746 2023-05-30 02:53:02 - SimpleLog - INFO: - episode: 5524, ep_reward: 116.544, ep_step: 1000 2023-05-30 02:53:04 - SimpleLog - INFO: - episode: 5525, ep_reward: 121.707, ep_step: 1000 2023-05-30 02:53:04 - SimpleLog - INFO: - episode: 5526, ep_reward: -121.161, ep_step: 75 2023-05-30 02:53:04 - SimpleLog - INFO: - episode: 5527, ep_reward: -120.849, ep_step: 67 2023-05-30 02:53:07 - SimpleLog - INFO: - episode: 5528, ep_reward: 113.049, ep_step: 1000 2023-05-30 02:53:09 - SimpleLog - INFO: - episode: 5529, ep_reward: 85.234, ep_step: 1000 2023-05-30 02:53:11 - SimpleLog - INFO: - episode: 5530, ep_reward: 137.500, ep_step: 1000 2023-05-30 02:53:14 - SimpleLog - INFO: - episode: 5531, ep_reward: 132.249, ep_step: 1000 2023-05-30 02:53:16 - SimpleLog - INFO: - episode: 5532, ep_reward: 115.214, ep_step: 1000 2023-05-30 02:53:18 - SimpleLog - INFO: - episode: 5533, ep_reward: 113.525, ep_step: 1000 2023-05-30 02:53:20 - SimpleLog - INFO: - episode: 5534, ep_reward: 109.010, ep_step: 1000 2023-05-30 02:53:23 - SimpleLog - INFO: - episode: 5535, ep_reward: 38.194, ep_step: 1000 2023-05-30 02:53:25 - SimpleLog - INFO: - episode: 5536, ep_reward: 129.385, ep_step: 1000 2023-05-30 02:53:27 - SimpleLog - INFO: - episode: 5537, ep_reward: 89.796, ep_step: 1000 2023-05-30 02:53:29 - SimpleLog - INFO: - episode: 5538, ep_reward: 105.261, ep_step: 1000 2023-05-30 02:53:32 - SimpleLog - INFO: - episode: 5539, ep_reward: 132.894, ep_step: 1000 2023-05-30 02:53:32 - SimpleLog - INFO: - episode: 5540, ep_reward: -93.855, ep_step: 284 2023-05-30 02:53:35 - SimpleLog - INFO: - episode: 5541, ep_reward: 117.055, ep_step: 1000 2023-05-30 02:53:37 - SimpleLog - INFO: - episode: 5542, ep_reward: 134.584, ep_step: 1000 2023-05-30 02:53:39 - SimpleLog - INFO: - episode: 5543, ep_reward: 143.038, ep_step: 1000 2023-05-30 02:53:41 - SimpleLog - INFO: - episode: 5544, ep_reward: 5.544, ep_step: 696 2023-05-30 02:53:43 - SimpleLog - INFO: - episode: 5545, ep_reward: 151.484, ep_step: 1000 2023-05-30 02:53:45 - SimpleLog - INFO: - episode: 5546, ep_reward: -39.395, ep_step: 924 2023-05-30 02:53:47 - SimpleLog - INFO: - episode: 5547, ep_reward: 117.838, ep_step: 1000 2023-05-30 02:53:50 - SimpleLog - INFO: - episode: 5548, ep_reward: 145.420, ep_step: 1000 2023-05-30 02:53:52 - SimpleLog - INFO: - episode: 5549, ep_reward: 85.616, ep_step: 1000 2023-05-30 02:53:54 - SimpleLog - INFO: - episode: 5550, ep_reward: 146.630, ep_step: 1000 2023-05-30 02:53:56 - SimpleLog - INFO: - episode: 5551, ep_reward: 127.898, ep_step: 1000 2023-05-30 02:53:59 - SimpleLog - INFO: - episode: 5552, ep_reward: 115.885, ep_step: 1000 2023-05-30 02:54:00 - SimpleLog - INFO: - episode: 5553, ep_reward: -36.895, ep_step: 702 2023-05-30 02:54:03 - SimpleLog - INFO: - episode: 5554, ep_reward: 126.529, ep_step: 1000 2023-05-30 02:54:05 - SimpleLog - INFO: - episode: 5555, ep_reward: 152.595, ep_step: 1000 2023-05-30 02:54:07 - SimpleLog - INFO: - episode: 5556, ep_reward: 144.240, ep_step: 1000 2023-05-30 02:54:09 - SimpleLog - INFO: - episode: 5557, ep_reward: 148.234, ep_step: 1000 2023-05-30 02:54:12 - SimpleLog - INFO: - episode: 5558, ep_reward: 147.042, ep_step: 1000 2023-05-30 02:54:14 - SimpleLog - INFO: - episode: 5559, ep_reward: -32.494, ep_step: 862 2023-05-30 02:54:16 - SimpleLog - INFO: - episode: 5560, ep_reward: 133.974, ep_step: 1000 2023-05-30 02:54:18 - SimpleLog - INFO: - episode: 5561, ep_reward: 160.803, ep_step: 1000 2023-05-30 02:54:20 - SimpleLog - INFO: - episode: 5562, ep_reward: 0.986, ep_step: 766 2023-05-30 02:54:22 - SimpleLog - INFO: - episode: 5563, ep_reward: 145.845, ep_step: 1000 2023-05-30 02:54:24 - SimpleLog - INFO: - episode: 5564, ep_reward: 154.411, ep_step: 1000 2023-05-30 02:54:27 - SimpleLog - INFO: - episode: 5565, ep_reward: 162.775, ep_step: 1000 2023-05-30 02:54:29 - SimpleLog - INFO: - episode: 5566, ep_reward: 172.057, ep_step: 1000 2023-05-30 02:54:31 - SimpleLog - INFO: - episode: 5567, ep_reward: -22.626, ep_step: 949 2023-05-30 02:54:33 - SimpleLog - INFO: - episode: 5568, ep_reward: 153.484, ep_step: 1000 2023-05-30 02:54:35 - SimpleLog - INFO: - episode: 5569, ep_reward: 91.986, ep_step: 1000 2023-05-30 02:54:38 - SimpleLog - INFO: - episode: 5570, ep_reward: 111.793, ep_step: 1000 2023-05-30 02:54:39 - SimpleLog - INFO: - episode: 5571, ep_reward: -61.178, ep_step: 488 2023-05-30 02:54:41 - SimpleLog - INFO: - episode: 5572, ep_reward: 136.244, ep_step: 1000 2023-05-30 02:54:43 - SimpleLog - INFO: - episode: 5573, ep_reward: 161.432, ep_step: 1000 2023-05-30 02:54:44 - SimpleLog - INFO: - episode: 5574, ep_reward: -72.985, ep_step: 296 2023-05-30 02:54:46 - SimpleLog - INFO: - episode: 5575, ep_reward: 1.656, ep_step: 855 2023-05-30 02:54:48 - SimpleLog - INFO: - update_step: 4340000, online_eval_reward: 181.188 2023-05-30 02:54:49 - SimpleLog - INFO: - episode: 5576, ep_reward: -72.349, ep_step: 457 2023-05-30 02:54:51 - SimpleLog - INFO: - episode: 5577, ep_reward: 185.697, ep_step: 1000 2023-05-30 02:54:53 - SimpleLog - INFO: - episode: 5578, ep_reward: 147.640, ep_step: 1000 2023-05-30 02:54:56 - SimpleLog - INFO: - episode: 5579, ep_reward: 154.537, ep_step: 1000 2023-05-30 02:54:58 - SimpleLog - INFO: - episode: 5580, ep_reward: 165.225, ep_step: 1000 2023-05-30 02:55:00 - SimpleLog - INFO: - episode: 5581, ep_reward: -10.156, ep_step: 807 2023-05-30 02:55:02 - SimpleLog - INFO: - episode: 5582, ep_reward: 150.523, ep_step: 1000 2023-05-30 02:55:04 - SimpleLog - INFO: - episode: 5583, ep_reward: -19.282, ep_step: 716 2023-05-30 02:55:06 - SimpleLog - INFO: - episode: 5584, ep_reward: 141.206, ep_step: 1000 2023-05-30 02:55:08 - SimpleLog - INFO: - episode: 5585, ep_reward: 92.262, ep_step: 1000 2023-05-30 02:55:10 - SimpleLog - INFO: - episode: 5586, ep_reward: 137.711, ep_step: 1000 2023-05-30 02:55:13 - SimpleLog - INFO: - episode: 5587, ep_reward: 126.704, ep_step: 1000 2023-05-30 02:55:15 - SimpleLog - INFO: - episode: 5588, ep_reward: 138.011, ep_step: 1000 2023-05-30 02:55:17 - SimpleLog - INFO: - episode: 5589, ep_reward: 159.076, ep_step: 1000 2023-05-30 02:55:19 - SimpleLog - INFO: - episode: 5590, ep_reward: 149.084, ep_step: 1000 2023-05-30 02:55:22 - SimpleLog - INFO: - episode: 5591, ep_reward: 78.088, ep_step: 1000 2023-05-30 02:55:24 - SimpleLog - INFO: - episode: 5592, ep_reward: 145.870, ep_step: 1000 2023-05-30 02:55:26 - SimpleLog - INFO: - episode: 5593, ep_reward: 143.564, ep_step: 1000 2023-05-30 02:55:27 - SimpleLog - INFO: - episode: 5594, ep_reward: -101.846, ep_step: 211 2023-05-30 02:55:29 - SimpleLog - INFO: - episode: 5595, ep_reward: 1.775, ep_step: 898 2023-05-30 02:55:31 - SimpleLog - INFO: - episode: 5596, ep_reward: 164.277, ep_step: 1000 2023-05-30 02:55:33 - SimpleLog - INFO: - episode: 5597, ep_reward: 163.939, ep_step: 1000 2023-05-30 02:55:35 - SimpleLog - INFO: - episode: 5598, ep_reward: 141.854, ep_step: 1000 2023-05-30 02:55:37 - SimpleLog - INFO: - episode: 5599, ep_reward: 3.908, ep_step: 690 2023-05-30 02:55:39 - SimpleLog - INFO: - episode: 5600, ep_reward: 158.165, ep_step: 1000 2023-05-30 02:55:41 - SimpleLog - INFO: - episode: 5601, ep_reward: -51.254, ep_step: 770 2023-05-30 02:55:43 - SimpleLog - INFO: - episode: 5602, ep_reward: 80.785, ep_step: 1000 2023-05-30 02:55:45 - SimpleLog - INFO: - episode: 5603, ep_reward: 156.928, ep_step: 1000 2023-05-30 02:55:48 - SimpleLog - INFO: - episode: 5604, ep_reward: 173.051, ep_step: 1000 2023-05-30 02:55:50 - SimpleLog - INFO: - episode: 5605, ep_reward: 156.445, ep_step: 1000 2023-05-30 02:55:52 - SimpleLog - INFO: - episode: 5606, ep_reward: 157.081, ep_step: 1000 2023-05-30 02:55:54 - SimpleLog - INFO: - episode: 5607, ep_reward: 155.735, ep_step: 1000 2023-05-30 02:55:57 - SimpleLog - INFO: - episode: 5608, ep_reward: 163.347, ep_step: 1000 2023-05-30 02:55:59 - SimpleLog - INFO: - episode: 5609, ep_reward: 5.210, ep_step: 815 2023-05-30 02:56:01 - SimpleLog - INFO: - episode: 5610, ep_reward: 123.343, ep_step: 1000 2023-05-30 02:56:03 - SimpleLog - INFO: - episode: 5611, ep_reward: 4.857, ep_step: 773 2023-05-30 02:56:05 - SimpleLog - INFO: - episode: 5612, ep_reward: 160.015, ep_step: 1000 2023-05-30 02:56:07 - SimpleLog - INFO: - episode: 5613, ep_reward: 152.921, ep_step: 1000 2023-05-30 02:56:09 - SimpleLog - INFO: - episode: 5614, ep_reward: 19.034, ep_step: 823 2023-05-30 02:56:11 - SimpleLog - INFO: - episode: 5615, ep_reward: 161.552, ep_step: 1000 2023-05-30 02:56:13 - SimpleLog - INFO: - episode: 5616, ep_reward: 155.451, ep_step: 1000 2023-05-30 02:56:16 - SimpleLog - INFO: - episode: 5617, ep_reward: 157.200, ep_step: 1000 2023-05-30 02:56:18 - SimpleLog - INFO: - episode: 5618, ep_reward: 159.062, ep_step: 1000 2023-05-30 02:56:20 - SimpleLog - INFO: - episode: 5619, ep_reward: 152.972, ep_step: 1000 2023-05-30 02:56:22 - SimpleLog - INFO: - episode: 5620, ep_reward: 146.833, ep_step: 1000 2023-05-30 02:56:25 - SimpleLog - INFO: - episode: 5621, ep_reward: 137.291, ep_step: 1000 2023-05-30 02:56:27 - SimpleLog - INFO: - episode: 5622, ep_reward: 121.184, ep_step: 1000 2023-05-30 02:56:29 - SimpleLog - INFO: - episode: 5623, ep_reward: 142.314, ep_step: 1000 2023-05-30 02:56:31 - SimpleLog - INFO: - episode: 5624, ep_reward: 138.940, ep_step: 1000 2023-05-30 02:56:34 - SimpleLog - INFO: - episode: 5625, ep_reward: 164.858, ep_step: 1000 2023-05-30 02:56:36 - SimpleLog - INFO: - episode: 5626, ep_reward: 143.011, ep_step: 1000 2023-05-30 02:56:38 - SimpleLog - INFO: - episode: 5627, ep_reward: 123.220, ep_step: 1000 2023-05-30 02:56:40 - SimpleLog - INFO: - episode: 5628, ep_reward: 140.261, ep_step: 1000 2023-05-30 02:56:43 - SimpleLog - INFO: - episode: 5629, ep_reward: 117.853, ep_step: 1000 2023-05-30 02:56:45 - SimpleLog - INFO: - episode: 5630, ep_reward: 87.138, ep_step: 1000 2023-05-30 02:56:45 - SimpleLog - INFO: - episode: 5631, ep_reward: -100.612, ep_step: 258 2023-05-30 02:56:46 - SimpleLog - INFO: - episode: 5632, ep_reward: -102.115, ep_step: 262 2023-05-30 02:56:48 - SimpleLog - INFO: - episode: 5633, ep_reward: 134.590, ep_step: 1000 2023-05-30 02:56:50 - SimpleLog - INFO: - episode: 5634, ep_reward: 144.250, ep_step: 1000 2023-05-30 02:56:51 - SimpleLog - INFO: - episode: 5635, ep_reward: -68.087, ep_step: 470 2023-05-30 02:56:54 - SimpleLog - INFO: - episode: 5636, ep_reward: 148.828, ep_step: 1000 2023-05-30 02:56:56 - SimpleLog - INFO: - episode: 5637, ep_reward: 165.844, ep_step: 1000 2023-05-30 02:56:58 - SimpleLog - INFO: - episode: 5638, ep_reward: 129.205, ep_step: 1000 2023-05-30 02:57:00 - SimpleLog - INFO: - episode: 5639, ep_reward: 82.043, ep_step: 1000 2023-05-30 02:57:03 - SimpleLog - INFO: - episode: 5640, ep_reward: 130.773, ep_step: 1000 2023-05-30 02:57:05 - SimpleLog - INFO: - episode: 5641, ep_reward: 115.430, ep_step: 1000 2023-05-30 02:57:07 - SimpleLog - INFO: - episode: 5642, ep_reward: 131.812, ep_step: 1000 2023-05-30 02:57:09 - SimpleLog - INFO: - episode: 5643, ep_reward: 5.958, ep_step: 691 2023-05-30 02:57:10 - SimpleLog - INFO: - episode: 5644, ep_reward: 7.072, ep_step: 632 2023-05-30 02:57:12 - SimpleLog - INFO: - episode: 5645, ep_reward: 3.459, ep_step: 1000 2023-05-30 02:57:15 - SimpleLog - INFO: - episode: 5646, ep_reward: 82.175, ep_step: 1000 2023-05-30 02:57:15 - SimpleLog - INFO: - episode: 5647, ep_reward: -64.935, ep_step: 324 2023-05-30 02:57:18 - SimpleLog - INFO: - episode: 5648, ep_reward: -0.446, ep_step: 926 2023-05-30 02:57:20 - SimpleLog - INFO: - episode: 5649, ep_reward: 127.974, ep_step: 1000 2023-05-30 02:57:22 - SimpleLog - INFO: - episode: 5650, ep_reward: 133.188, ep_step: 1000 2023-05-30 02:57:23 - SimpleLog - INFO: - episode: 5651, ep_reward: -26.397, ep_step: 575 2023-05-30 02:57:26 - SimpleLog - INFO: - episode: 5652, ep_reward: 5.407, ep_step: 1000 2023-05-30 02:57:27 - SimpleLog - INFO: - update_step: 4410000, online_eval_reward: 3.510 2023-05-30 02:57:29 - SimpleLog - INFO: - episode: 5653, ep_reward: 125.481, ep_step: 1000 2023-05-30 02:57:31 - SimpleLog - INFO: - episode: 5654, ep_reward: 137.600, ep_step: 1000 2023-05-30 02:57:34 - SimpleLog - INFO: - episode: 5655, ep_reward: 139.340, ep_step: 1000 2023-05-30 02:57:36 - SimpleLog - INFO: - episode: 5656, ep_reward: 165.007, ep_step: 1000 2023-05-30 02:57:38 - SimpleLog - INFO: - episode: 5657, ep_reward: -2.606, ep_step: 762 2023-05-30 02:57:40 - SimpleLog - INFO: - episode: 5658, ep_reward: 133.418, ep_step: 1000 2023-05-30 02:57:42 - SimpleLog - INFO: - episode: 5659, ep_reward: 128.777, ep_step: 1000 2023-05-30 02:57:44 - SimpleLog - INFO: - episode: 5660, ep_reward: 142.067, ep_step: 1000 2023-05-30 02:57:47 - SimpleLog - INFO: - episode: 5661, ep_reward: 139.213, ep_step: 1000 2023-05-30 02:57:49 - SimpleLog - INFO: - episode: 5662, ep_reward: 140.184, ep_step: 1000 2023-05-30 02:57:51 - SimpleLog - INFO: - episode: 5663, ep_reward: 131.399, ep_step: 1000 2023-05-30 02:57:53 - SimpleLog - INFO: - episode: 5664, ep_reward: 132.415, ep_step: 1000 2023-05-30 02:57:55 - SimpleLog - INFO: - episode: 5665, ep_reward: -4.194, ep_step: 716 2023-05-30 02:57:57 - SimpleLog - INFO: - episode: 5666, ep_reward: 116.768, ep_step: 1000 2023-05-30 02:57:59 - SimpleLog - INFO: - episode: 5667, ep_reward: 131.872, ep_step: 1000 2023-05-30 02:58:02 - SimpleLog - INFO: - episode: 5668, ep_reward: 139.467, ep_step: 1000 2023-05-30 02:58:04 - SimpleLog - INFO: - episode: 5669, ep_reward: 132.575, ep_step: 1000 2023-05-30 02:58:06 - SimpleLog - INFO: - episode: 5670, ep_reward: 132.001, ep_step: 1000 2023-05-30 02:58:08 - SimpleLog - INFO: - episode: 5671, ep_reward: 138.865, ep_step: 1000 2023-05-30 02:58:11 - SimpleLog - INFO: - episode: 5672, ep_reward: 133.175, ep_step: 1000 2023-05-30 02:58:13 - SimpleLog - INFO: - episode: 5673, ep_reward: 141.939, ep_step: 1000 2023-05-30 02:58:15 - SimpleLog - INFO: - episode: 5674, ep_reward: 80.144, ep_step: 1000 2023-05-30 02:58:17 - SimpleLog - INFO: - episode: 5675, ep_reward: -1.161, ep_step: 938 2023-05-30 02:58:19 - SimpleLog - INFO: - episode: 5676, ep_reward: -22.797, ep_step: 734 2023-05-30 02:58:21 - SimpleLog - INFO: - episode: 5677, ep_reward: 6.357, ep_step: 838 2023-05-30 02:58:23 - SimpleLog - INFO: - episode: 5678, ep_reward: 155.020, ep_step: 1000 2023-05-30 02:58:24 - SimpleLog - INFO: - episode: 5679, ep_reward: -156.889, ep_step: 556 2023-05-30 02:58:27 - SimpleLog - INFO: - episode: 5680, ep_reward: -15.276, ep_step: 947 2023-05-30 02:58:28 - SimpleLog - INFO: - episode: 5681, ep_reward: -0.582, ep_step: 694 2023-05-30 02:58:30 - SimpleLog - INFO: - episode: 5682, ep_reward: 151.620, ep_step: 1000 2023-05-30 02:58:32 - SimpleLog - INFO: - episode: 5683, ep_reward: -0.501, ep_step: 711 2023-05-30 02:58:34 - SimpleLog - INFO: - episode: 5684, ep_reward: 173.913, ep_step: 1000 2023-05-30 02:58:36 - SimpleLog - INFO: - episode: 5685, ep_reward: 91.459, ep_step: 1000 2023-05-30 02:58:39 - SimpleLog - INFO: - episode: 5686, ep_reward: 178.046, ep_step: 1000 2023-05-30 02:58:41 - SimpleLog - INFO: - episode: 5687, ep_reward: 6.603, ep_step: 833 2023-05-30 02:58:43 - SimpleLog - INFO: - episode: 5688, ep_reward: -69.267, ep_step: 1000 2023-05-30 02:58:44 - SimpleLog - INFO: - episode: 5689, ep_reward: -30.604, ep_step: 480 2023-05-30 02:58:45 - SimpleLog - INFO: - episode: 5690, ep_reward: 4.278, ep_step: 667 2023-05-30 02:58:48 - SimpleLog - INFO: - episode: 5691, ep_reward: 123.491, ep_step: 1000 2023-05-30 02:58:49 - SimpleLog - INFO: - episode: 5692, ep_reward: -6.277, ep_step: 707 2023-05-30 02:58:51 - SimpleLog - INFO: - episode: 5693, ep_reward: 17.021, ep_step: 777 2023-05-30 02:58:53 - SimpleLog - INFO: - episode: 5694, ep_reward: -10.008, ep_step: 928 2023-05-30 02:58:55 - SimpleLog - INFO: - episode: 5695, ep_reward: 148.827, ep_step: 1000 2023-05-30 02:58:57 - SimpleLog - INFO: - episode: 5696, ep_reward: -43.424, ep_step: 570 2023-05-30 02:58:59 - SimpleLog - INFO: - episode: 5697, ep_reward: 119.022, ep_step: 1000 2023-05-30 02:59:01 - SimpleLog - INFO: - episode: 5698, ep_reward: 132.616, ep_step: 1000 2023-05-30 02:59:04 - SimpleLog - INFO: - episode: 5699, ep_reward: 139.792, ep_step: 1000 2023-05-30 02:59:06 - SimpleLog - INFO: - episode: 5700, ep_reward: 135.493, ep_step: 1000 2023-05-30 02:59:08 - SimpleLog - INFO: - episode: 5701, ep_reward: 15.218, ep_step: 784 2023-05-30 02:59:10 - SimpleLog - INFO: - episode: 5702, ep_reward: 125.956, ep_step: 1000 2023-05-30 02:59:12 - SimpleLog - INFO: - episode: 5703, ep_reward: 84.345, ep_step: 1000 2023-05-30 02:59:14 - SimpleLog - INFO: - episode: 5704, ep_reward: 142.737, ep_step: 1000 2023-05-30 02:59:17 - SimpleLog - INFO: - episode: 5705, ep_reward: 136.397, ep_step: 1000 2023-05-30 02:59:19 - SimpleLog - INFO: - episode: 5706, ep_reward: 132.947, ep_step: 1000 2023-05-30 02:59:21 - SimpleLog - INFO: - episode: 5707, ep_reward: 108.593, ep_step: 1000 2023-05-30 02:59:23 - SimpleLog - INFO: - episode: 5708, ep_reward: 157.708, ep_step: 1000 2023-05-30 02:59:26 - SimpleLog - INFO: - episode: 5709, ep_reward: 140.156, ep_step: 1000 2023-05-30 02:59:28 - SimpleLog - INFO: - episode: 5710, ep_reward: 138.329, ep_step: 1000 2023-05-30 02:59:30 - SimpleLog - INFO: - episode: 5711, ep_reward: 149.715, ep_step: 1000 2023-05-30 02:59:32 - SimpleLog - INFO: - episode: 5712, ep_reward: 154.511, ep_step: 1000 2023-05-30 02:59:35 - SimpleLog - INFO: - episode: 5713, ep_reward: 105.625, ep_step: 1000 2023-05-30 02:59:36 - SimpleLog - INFO: - episode: 5714, ep_reward: 12.612, ep_step: 795 2023-05-30 02:59:38 - SimpleLog - INFO: - episode: 5715, ep_reward: 10.243, ep_step: 802 2023-05-30 02:59:40 - SimpleLog - INFO: - episode: 5716, ep_reward: 6.455, ep_step: 807 2023-05-30 02:59:42 - SimpleLog - INFO: - episode: 5717, ep_reward: -1.922, ep_step: 773 2023-05-30 02:59:44 - SimpleLog - INFO: - episode: 5718, ep_reward: 170.950, ep_step: 1000 2023-05-30 02:59:46 - SimpleLog - INFO: - episode: 5719, ep_reward: -1.450, ep_step: 794 2023-05-30 02:59:48 - SimpleLog - INFO: - episode: 5720, ep_reward: 149.160, ep_step: 1000 2023-05-30 02:59:50 - SimpleLog - INFO: - episode: 5721, ep_reward: 152.936, ep_step: 1000 2023-05-30 02:59:53 - SimpleLog - INFO: - episode: 5722, ep_reward: 140.985, ep_step: 1000 2023-05-30 02:59:54 - SimpleLog - INFO: - episode: 5723, ep_reward: 14.749, ep_step: 717 2023-05-30 02:59:56 - SimpleLog - INFO: - episode: 5724, ep_reward: -2.172, ep_step: 790 2023-05-30 02:59:58 - SimpleLog - INFO: - episode: 5725, ep_reward: 11.710, ep_step: 758 2023-05-30 02:59:59 - SimpleLog - INFO: - episode: 5726, ep_reward: 8.593, ep_step: 805 2023-05-30 03:00:02 - SimpleLog - INFO: - episode: 5727, ep_reward: 160.092, ep_step: 1000 2023-05-30 03:00:04 - SimpleLog - INFO: - episode: 5728, ep_reward: 130.459, ep_step: 1000 2023-05-30 03:00:07 - SimpleLog - INFO: - update_step: 4480000, online_eval_reward: 137.228 2023-05-30 03:00:08 - SimpleLog - INFO: - episode: 5729, ep_reward: 158.907, ep_step: 1000 2023-05-30 03:00:10 - SimpleLog - INFO: - episode: 5730, ep_reward: 132.281, ep_step: 1000 2023-05-30 03:00:13 - SimpleLog - INFO: - episode: 5731, ep_reward: 134.580, ep_step: 1000 2023-05-30 03:00:15 - SimpleLog - INFO: - episode: 5732, ep_reward: 144.287, ep_step: 1000 2023-05-30 03:00:17 - SimpleLog - INFO: - episode: 5733, ep_reward: 144.076, ep_step: 1000 2023-05-30 03:00:19 - SimpleLog - INFO: - episode: 5734, ep_reward: 104.874, ep_step: 1000 2023-05-30 03:00:22 - SimpleLog - INFO: - episode: 5735, ep_reward: 89.877, ep_step: 1000 2023-05-30 03:00:24 - SimpleLog - INFO: - episode: 5736, ep_reward: 82.282, ep_step: 1000 2023-05-30 03:00:26 - SimpleLog - INFO: - episode: 5737, ep_reward: 135.432, ep_step: 1000 2023-05-30 03:00:28 - SimpleLog - INFO: - episode: 5738, ep_reward: 140.648, ep_step: 1000 2023-05-30 03:00:31 - SimpleLog - INFO: - episode: 5739, ep_reward: 155.521, ep_step: 1000 2023-05-30 03:00:33 - SimpleLog - INFO: - episode: 5740, ep_reward: 76.535, ep_step: 1000 2023-05-30 03:00:35 - SimpleLog - INFO: - episode: 5741, ep_reward: 157.186, ep_step: 1000 2023-05-30 03:00:37 - SimpleLog - INFO: - episode: 5742, ep_reward: 151.275, ep_step: 1000 2023-05-30 03:00:40 - SimpleLog - INFO: - episode: 5743, ep_reward: 168.433, ep_step: 1000 2023-05-30 03:00:42 - SimpleLog - INFO: - episode: 5744, ep_reward: 78.334, ep_step: 1000 2023-05-30 03:00:44 - SimpleLog - INFO: - episode: 5745, ep_reward: 151.371, ep_step: 1000 2023-05-30 03:00:46 - SimpleLog - INFO: - episode: 5746, ep_reward: 87.961, ep_step: 1000 2023-05-30 03:00:49 - SimpleLog - INFO: - episode: 5747, ep_reward: 164.604, ep_step: 1000 2023-05-30 03:00:51 - SimpleLog - INFO: - episode: 5748, ep_reward: 129.977, ep_step: 1000 2023-05-30 03:00:53 - SimpleLog - INFO: - episode: 5749, ep_reward: 146.847, ep_step: 1000 2023-05-30 03:00:55 - SimpleLog - INFO: - episode: 5750, ep_reward: 101.485, ep_step: 1000 2023-05-30 03:00:57 - SimpleLog - INFO: - episode: 5751, ep_reward: 10.050, ep_step: 800 2023-05-30 03:00:59 - SimpleLog - INFO: - episode: 5752, ep_reward: 75.871, ep_step: 1000 2023-05-30 03:01:02 - SimpleLog - INFO: - episode: 5753, ep_reward: 154.534, ep_step: 1000 2023-05-30 03:01:04 - SimpleLog - INFO: - episode: 5754, ep_reward: 147.316, ep_step: 1000 2023-05-30 03:01:06 - SimpleLog - INFO: - episode: 5755, ep_reward: 155.028, ep_step: 1000 2023-05-30 03:01:08 - SimpleLog - INFO: - episode: 5756, ep_reward: 127.630, ep_step: 1000 2023-05-30 03:01:11 - SimpleLog - INFO: - episode: 5757, ep_reward: 79.289, ep_step: 1000 2023-05-30 03:01:13 - SimpleLog - INFO: - episode: 5758, ep_reward: 155.617, ep_step: 1000 2023-05-30 03:01:14 - SimpleLog - INFO: - episode: 5759, ep_reward: 6.035, ep_step: 711 2023-05-30 03:01:17 - SimpleLog - INFO: - episode: 5760, ep_reward: 137.733, ep_step: 1000 2023-05-30 03:01:19 - SimpleLog - INFO: - episode: 5761, ep_reward: 122.972, ep_step: 1000 2023-05-30 03:01:21 - SimpleLog - INFO: - episode: 5762, ep_reward: 6.070, ep_step: 913 2023-05-30 03:01:23 - SimpleLog - INFO: - episode: 5763, ep_reward: 149.230, ep_step: 1000 2023-05-30 03:01:25 - SimpleLog - INFO: - episode: 5764, ep_reward: 0.973, ep_step: 879 2023-05-30 03:01:27 - SimpleLog - INFO: - episode: 5765, ep_reward: 4.371, ep_step: 875 2023-05-30 03:01:29 - SimpleLog - INFO: - episode: 5766, ep_reward: -3.039, ep_step: 975 2023-05-30 03:01:32 - SimpleLog - INFO: - episode: 5767, ep_reward: 144.120, ep_step: 1000 2023-05-30 03:01:34 - SimpleLog - INFO: - episode: 5768, ep_reward: 150.867, ep_step: 1000 2023-05-30 03:01:36 - SimpleLog - INFO: - episode: 5769, ep_reward: 132.260, ep_step: 1000 2023-05-30 03:01:38 - SimpleLog - INFO: - episode: 5770, ep_reward: 149.854, ep_step: 1000 2023-05-30 03:01:40 - SimpleLog - INFO: - episode: 5771, ep_reward: -8.341, ep_step: 854 2023-05-30 03:01:43 - SimpleLog - INFO: - episode: 5772, ep_reward: 154.339, ep_step: 1000 2023-05-30 03:01:45 - SimpleLog - INFO: - episode: 5773, ep_reward: 156.505, ep_step: 1000 2023-05-30 03:01:47 - SimpleLog - INFO: - episode: 5774, ep_reward: 153.721, ep_step: 1000 2023-05-30 03:01:49 - SimpleLog - INFO: - episode: 5775, ep_reward: 140.784, ep_step: 1000 2023-05-30 03:01:52 - SimpleLog - INFO: - episode: 5776, ep_reward: 145.243, ep_step: 1000 2023-05-30 03:01:54 - SimpleLog - INFO: - episode: 5777, ep_reward: 127.104, ep_step: 1000 2023-05-30 03:01:56 - SimpleLog - INFO: - episode: 5778, ep_reward: 161.993, ep_step: 1000 2023-05-30 03:01:58 - SimpleLog - INFO: - episode: 5779, ep_reward: 162.721, ep_step: 1000 2023-05-30 03:02:01 - SimpleLog - INFO: - episode: 5780, ep_reward: 139.243, ep_step: 1000 2023-05-30 03:02:03 - SimpleLog - INFO: - episode: 5781, ep_reward: 138.499, ep_step: 1000 2023-05-30 03:02:05 - SimpleLog - INFO: - episode: 5782, ep_reward: 154.696, ep_step: 1000 2023-05-30 03:02:07 - SimpleLog - INFO: - episode: 5783, ep_reward: 162.849, ep_step: 1000 2023-05-30 03:02:10 - SimpleLog - INFO: - episode: 5784, ep_reward: 168.621, ep_step: 1000 2023-05-30 03:02:12 - SimpleLog - INFO: - episode: 5785, ep_reward: 122.831, ep_step: 1000 2023-05-30 03:02:14 - SimpleLog - INFO: - episode: 5786, ep_reward: 127.718, ep_step: 1000 2023-05-30 03:02:17 - SimpleLog - INFO: - episode: 5787, ep_reward: 166.402, ep_step: 1000 2023-05-30 03:02:19 - SimpleLog - INFO: - episode: 5788, ep_reward: 150.733, ep_step: 1000 2023-05-30 03:02:21 - SimpleLog - INFO: - episode: 5789, ep_reward: 103.891, ep_step: 1000 2023-05-30 03:02:23 - SimpleLog - INFO: - episode: 5790, ep_reward: -18.409, ep_step: 811 2023-05-30 03:02:25 - SimpleLog - INFO: - episode: 5791, ep_reward: 171.011, ep_step: 1000 2023-05-30 03:02:27 - SimpleLog - INFO: - episode: 5792, ep_reward: 106.801, ep_step: 1000 2023-05-30 03:02:30 - SimpleLog - INFO: - episode: 5793, ep_reward: 158.933, ep_step: 1000 2023-05-30 03:02:32 - SimpleLog - INFO: - episode: 5794, ep_reward: 152.341, ep_step: 1000 2023-05-30 03:02:34 - SimpleLog - INFO: - episode: 5795, ep_reward: 79.932, ep_step: 1000 2023-05-30 03:02:36 - SimpleLog - INFO: - episode: 5796, ep_reward: 163.073, ep_step: 1000 2023-05-30 03:02:38 - SimpleLog - INFO: - episode: 5797, ep_reward: 8.201, ep_step: 704 2023-05-30 03:02:40 - SimpleLog - INFO: - episode: 5798, ep_reward: 155.950, ep_step: 1000 2023-05-30 03:02:42 - SimpleLog - INFO: - episode: 5799, ep_reward: 148.215, ep_step: 1000 2023-05-30 03:02:47 - SimpleLog - INFO: - update_step: 4550000, online_eval_reward: 153.160 2023-05-30 03:02:47 - SimpleLog - INFO: - episode: 5800, ep_reward: 167.306, ep_step: 1000 2023-05-30 03:02:49 - SimpleLog - INFO: - episode: 5801, ep_reward: 176.438, ep_step: 1000 2023-05-30 03:02:51 - SimpleLog - INFO: - episode: 5802, ep_reward: 168.341, ep_step: 1000 2023-05-30 03:02:53 - SimpleLog - INFO: - episode: 5803, ep_reward: 124.522, ep_step: 1000 2023-05-30 03:02:56 - SimpleLog - INFO: - episode: 5804, ep_reward: 115.317, ep_step: 1000 2023-05-30 03:02:58 - SimpleLog - INFO: - episode: 5805, ep_reward: 169.128, ep_step: 1000 2023-05-30 03:03:00 - SimpleLog - INFO: - episode: 5806, ep_reward: -19.373, ep_step: 749 2023-05-30 03:03:01 - SimpleLog - INFO: - episode: 5807, ep_reward: -71.207, ep_step: 429 2023-05-30 03:03:03 - SimpleLog - INFO: - episode: 5808, ep_reward: 169.940, ep_step: 1000 2023-05-30 03:03:03 - SimpleLog - INFO: - episode: 5809, ep_reward: -88.793, ep_step: 159 2023-05-30 03:03:05 - SimpleLog - INFO: - episode: 5810, ep_reward: 7.066, ep_step: 1000 2023-05-30 03:03:08 - SimpleLog - INFO: - episode: 5811, ep_reward: 161.043, ep_step: 1000 2023-05-30 03:03:10 - SimpleLog - INFO: - episode: 5812, ep_reward: 156.134, ep_step: 1000 2023-05-30 03:03:12 - SimpleLog - INFO: - episode: 5813, ep_reward: 162.432, ep_step: 1000 2023-05-30 03:03:14 - SimpleLog - INFO: - episode: 5814, ep_reward: 7.617, ep_step: 698 2023-05-30 03:03:16 - SimpleLog - INFO: - episode: 5815, ep_reward: 172.528, ep_step: 1000 2023-05-30 03:03:18 - SimpleLog - INFO: - episode: 5816, ep_reward: 144.204, ep_step: 1000 2023-05-30 03:03:20 - SimpleLog - INFO: - episode: 5817, ep_reward: 139.064, ep_step: 1000 2023-05-30 03:03:22 - SimpleLog - INFO: - episode: 5818, ep_reward: -16.996, ep_step: 727 2023-05-30 03:03:24 - SimpleLog - INFO: - episode: 5819, ep_reward: 158.485, ep_step: 1000 2023-05-30 03:03:27 - SimpleLog - INFO: - episode: 5820, ep_reward: 154.317, ep_step: 1000 2023-05-30 03:03:27 - SimpleLog - INFO: - episode: 5821, ep_reward: -83.851, ep_step: 223 2023-05-30 03:03:28 - SimpleLog - INFO: - episode: 5822, ep_reward: -31.779, ep_step: 516 2023-05-30 03:03:31 - SimpleLog - INFO: - episode: 5823, ep_reward: 109.295, ep_step: 1000 2023-05-30 03:03:32 - SimpleLog - INFO: - episode: 5824, ep_reward: -5.365, ep_step: 752 2023-05-30 03:03:34 - SimpleLog - INFO: - episode: 5825, ep_reward: 2.252, ep_step: 1000 2023-05-30 03:03:36 - SimpleLog - INFO: - episode: 5826, ep_reward: -36.239, ep_step: 460 2023-05-30 03:03:38 - SimpleLog - INFO: - episode: 5827, ep_reward: 171.009, ep_step: 1000 2023-05-30 03:03:39 - SimpleLog - INFO: - episode: 5828, ep_reward: -36.004, ep_step: 454 2023-05-30 03:03:41 - SimpleLog - INFO: - episode: 5829, ep_reward: 48.822, ep_step: 896 2023-05-30 03:03:43 - SimpleLog - INFO: - episode: 5830, ep_reward: 149.316, ep_step: 1000 2023-05-30 03:03:45 - SimpleLog - INFO: - episode: 5831, ep_reward: 162.274, ep_step: 1000 2023-05-30 03:03:47 - SimpleLog - INFO: - episode: 5832, ep_reward: 162.437, ep_step: 1000 2023-05-30 03:03:50 - SimpleLog - INFO: - episode: 5833, ep_reward: 168.799, ep_step: 1000 2023-05-30 03:03:52 - SimpleLog - INFO: - episode: 5834, ep_reward: 167.689, ep_step: 1000 2023-05-30 03:03:54 - SimpleLog - INFO: - episode: 5835, ep_reward: 168.607, ep_step: 1000 2023-05-30 03:03:56 - SimpleLog - INFO: - episode: 5836, ep_reward: 4.980, ep_step: 863 2023-05-30 03:03:58 - SimpleLog - INFO: - episode: 5837, ep_reward: 176.573, ep_step: 1000 2023-05-30 03:04:01 - SimpleLog - INFO: - episode: 5838, ep_reward: 154.783, ep_step: 1000 2023-05-30 03:04:03 - SimpleLog - INFO: - episode: 5839, ep_reward: 156.695, ep_step: 1000 2023-05-30 03:04:05 - SimpleLog - INFO: - episode: 5840, ep_reward: 109.253, ep_step: 1000 2023-05-30 03:04:05 - SimpleLog - INFO: - episode: 5841, ep_reward: -118.550, ep_step: 68 2023-05-30 03:04:08 - SimpleLog - INFO: - episode: 5842, ep_reward: 142.010, ep_step: 1000 2023-05-30 03:04:10 - SimpleLog - INFO: - episode: 5843, ep_reward: 173.097, ep_step: 1000 2023-05-30 03:04:12 - SimpleLog - INFO: - episode: 5844, ep_reward: 161.388, ep_step: 1000 2023-05-30 03:04:14 - SimpleLog - INFO: - episode: 5845, ep_reward: 147.858, ep_step: 1000 2023-05-30 03:04:16 - SimpleLog - INFO: - episode: 5846, ep_reward: 16.277, ep_step: 749 2023-05-30 03:04:16 - SimpleLog - INFO: - episode: 5847, ep_reward: -107.820, ep_step: 170 2023-05-30 03:04:17 - SimpleLog - INFO: - episode: 5848, ep_reward: -104.418, ep_step: 163 2023-05-30 03:04:19 - SimpleLog - INFO: - episode: 5849, ep_reward: 164.581, ep_step: 1000 2023-05-30 03:04:21 - SimpleLog - INFO: - episode: 5850, ep_reward: 160.044, ep_step: 1000 2023-05-30 03:04:24 - SimpleLog - INFO: - episode: 5851, ep_reward: 158.447, ep_step: 1000 2023-05-30 03:04:26 - SimpleLog - INFO: - episode: 5852, ep_reward: 172.474, ep_step: 1000 2023-05-30 03:04:28 - SimpleLog - INFO: - episode: 5853, ep_reward: -0.062, ep_step: 886 2023-05-30 03:04:30 - SimpleLog - INFO: - episode: 5854, ep_reward: 173.358, ep_step: 1000 2023-05-30 03:04:32 - SimpleLog - INFO: - episode: 5855, ep_reward: 175.744, ep_step: 1000 2023-05-30 03:04:35 - SimpleLog - INFO: - episode: 5856, ep_reward: 146.957, ep_step: 1000 2023-05-30 03:04:37 - SimpleLog - INFO: - episode: 5857, ep_reward: 170.902, ep_step: 1000 2023-05-30 03:04:39 - SimpleLog - INFO: - episode: 5858, ep_reward: -3.304, ep_step: 1000 2023-05-30 03:04:41 - SimpleLog - INFO: - episode: 5859, ep_reward: 159.069, ep_step: 1000 2023-05-30 03:04:44 - SimpleLog - INFO: - episode: 5860, ep_reward: 80.419, ep_step: 1000 2023-05-30 03:04:46 - SimpleLog - INFO: - episode: 5861, ep_reward: 165.500, ep_step: 1000 2023-05-30 03:04:48 - SimpleLog - INFO: - episode: 5862, ep_reward: 159.860, ep_step: 1000 2023-05-30 03:04:50 - SimpleLog - INFO: - episode: 5863, ep_reward: 159.379, ep_step: 1000 2023-05-30 03:04:53 - SimpleLog - INFO: - episode: 5864, ep_reward: 136.021, ep_step: 1000 2023-05-30 03:04:55 - SimpleLog - INFO: - episode: 5865, ep_reward: 1.728, ep_step: 896 2023-05-30 03:04:57 - SimpleLog - INFO: - episode: 5866, ep_reward: -7.467, ep_step: 1000 2023-05-30 03:04:58 - SimpleLog - INFO: - episode: 5867, ep_reward: -71.584, ep_step: 410 2023-05-30 03:05:00 - SimpleLog - INFO: - episode: 5868, ep_reward: 158.951, ep_step: 1000 2023-05-30 03:05:01 - SimpleLog - INFO: - episode: 5869, ep_reward: -69.114, ep_step: 422 2023-05-30 03:05:02 - SimpleLog - INFO: - episode: 5870, ep_reward: -72.546, ep_step: 403 2023-05-30 03:05:04 - SimpleLog - INFO: - episode: 5871, ep_reward: 146.854, ep_step: 1000 2023-05-30 03:05:07 - SimpleLog - INFO: - episode: 5872, ep_reward: 102.119, ep_step: 1000 2023-05-30 03:05:09 - SimpleLog - INFO: - episode: 5873, ep_reward: 126.249, ep_step: 1000 2023-05-30 03:05:11 - SimpleLog - INFO: - episode: 5874, ep_reward: 171.223, ep_step: 1000 2023-05-30 03:05:13 - SimpleLog - INFO: - episode: 5875, ep_reward: 173.539, ep_step: 1000 2023-05-30 03:05:16 - SimpleLog - INFO: - episode: 5876, ep_reward: 102.090, ep_step: 1000 2023-05-30 03:05:18 - SimpleLog - INFO: - episode: 5877, ep_reward: 3.401, ep_step: 902 2023-05-30 03:05:19 - SimpleLog - INFO: - episode: 5878, ep_reward: -70.063, ep_step: 362 2023-05-30 03:05:21 - SimpleLog - INFO: - episode: 5879, ep_reward: 173.944, ep_step: 1000 2023-05-30 03:05:23 - SimpleLog - INFO: - episode: 5880, ep_reward: 166.055, ep_step: 1000 2023-05-30 03:05:26 - SimpleLog - INFO: - update_step: 4620000, online_eval_reward: 22.876 2023-05-30 03:05:27 - SimpleLog - INFO: - episode: 5881, ep_reward: 161.108, ep_step: 1000 2023-05-30 03:05:29 - SimpleLog - INFO: - episode: 5882, ep_reward: 113.798, ep_step: 1000 2023-05-30 03:05:32 - SimpleLog - INFO: - episode: 5883, ep_reward: 168.625, ep_step: 1000 2023-05-30 03:05:33 - SimpleLog - INFO: - episode: 5884, ep_reward: -10.458, ep_step: 556 2023-05-30 03:05:35 - SimpleLog - INFO: - episode: 5885, ep_reward: -1.421, ep_step: 934 2023-05-30 03:05:37 - SimpleLog - INFO: - episode: 5886, ep_reward: 155.210, ep_step: 1000 2023-05-30 03:05:39 - SimpleLog - INFO: - episode: 5887, ep_reward: 100.538, ep_step: 1000 2023-05-30 03:05:42 - SimpleLog - INFO: - episode: 5888, ep_reward: 80.383, ep_step: 1000 2023-05-30 03:05:44 - SimpleLog - INFO: - episode: 5889, ep_reward: -30.875, ep_step: 783 2023-05-30 03:05:46 - SimpleLog - INFO: - episode: 5890, ep_reward: 169.464, ep_step: 1000 2023-05-30 03:05:48 - SimpleLog - INFO: - episode: 5891, ep_reward: 161.300, ep_step: 1000 2023-05-30 03:05:50 - SimpleLog - INFO: - episode: 5892, ep_reward: 97.199, ep_step: 1000 2023-05-30 03:05:53 - SimpleLog - INFO: - episode: 5893, ep_reward: 163.615, ep_step: 1000 2023-05-30 03:05:55 - SimpleLog - INFO: - episode: 5894, ep_reward: 154.525, ep_step: 1000 2023-05-30 03:05:57 - SimpleLog - INFO: - episode: 5895, ep_reward: 178.584, ep_step: 1000 2023-05-30 03:05:58 - SimpleLog - INFO: - episode: 5896, ep_reward: -77.345, ep_step: 477 2023-05-30 03:06:00 - SimpleLog - INFO: - episode: 5897, ep_reward: 178.906, ep_step: 1000 2023-05-30 03:06:03 - SimpleLog - INFO: - episode: 5898, ep_reward: 176.474, ep_step: 1000 2023-05-30 03:06:05 - SimpleLog - INFO: - episode: 5899, ep_reward: 178.559, ep_step: 1000 2023-05-30 03:06:07 - SimpleLog - INFO: - episode: 5900, ep_reward: 96.382, ep_step: 1000 2023-05-30 03:06:08 - SimpleLog - INFO: - episode: 5901, ep_reward: -27.684, ep_step: 563 2023-05-30 03:06:11 - SimpleLog - INFO: - episode: 5902, ep_reward: 146.766, ep_step: 1000 2023-05-30 03:06:13 - SimpleLog - INFO: - episode: 5903, ep_reward: 100.709, ep_step: 1000 2023-05-30 03:06:15 - SimpleLog - INFO: - episode: 5904, ep_reward: 160.062, ep_step: 1000 2023-05-30 03:06:18 - SimpleLog - INFO: - episode: 5905, ep_reward: 156.184, ep_step: 1000 2023-05-30 03:06:20 - SimpleLog - INFO: - episode: 5906, ep_reward: 116.730, ep_step: 1000 2023-05-30 03:06:22 - SimpleLog - INFO: - episode: 5907, ep_reward: 67.248, ep_step: 1000 2023-05-30 03:06:24 - SimpleLog - INFO: - episode: 5908, ep_reward: 163.679, ep_step: 1000 2023-05-30 03:06:27 - SimpleLog - INFO: - episode: 5909, ep_reward: 127.394, ep_step: 1000 2023-05-30 03:06:29 - SimpleLog - INFO: - episode: 5910, ep_reward: 160.785, ep_step: 1000 2023-05-30 03:06:31 - SimpleLog - INFO: - episode: 5911, ep_reward: 174.748, ep_step: 1000 2023-05-30 03:06:33 - SimpleLog - INFO: - episode: 5912, ep_reward: 16.663, ep_step: 716 2023-05-30 03:06:34 - SimpleLog - INFO: - episode: 5913, ep_reward: -32.582, ep_step: 488 2023-05-30 03:06:35 - SimpleLog - INFO: - episode: 5914, ep_reward: -34.007, ep_step: 472 2023-05-30 03:06:37 - SimpleLog - INFO: - episode: 5915, ep_reward: 181.717, ep_step: 1000 2023-05-30 03:06:40 - SimpleLog - INFO: - episode: 5916, ep_reward: 161.022, ep_step: 1000 2023-05-30 03:06:42 - SimpleLog - INFO: - episode: 5917, ep_reward: 160.995, ep_step: 1000 2023-05-30 03:06:44 - SimpleLog - INFO: - episode: 5918, ep_reward: 29.211, ep_step: 973 2023-05-30 03:06:46 - SimpleLog - INFO: - episode: 5919, ep_reward: 159.965, ep_step: 1000 2023-05-30 03:06:48 - SimpleLog - INFO: - episode: 5920, ep_reward: -4.615, ep_step: 781 2023-05-30 03:06:50 - SimpleLog - INFO: - episode: 5921, ep_reward: 147.321, ep_step: 1000 2023-05-30 03:06:53 - SimpleLog - INFO: - episode: 5922, ep_reward: 149.348, ep_step: 1000 2023-05-30 03:06:55 - SimpleLog - INFO: - episode: 5923, ep_reward: 148.882, ep_step: 1000 2023-05-30 03:06:56 - SimpleLog - INFO: - episode: 5924, ep_reward: -35.177, ep_step: 477 2023-05-30 03:06:58 - SimpleLog - INFO: - episode: 5925, ep_reward: 34.054, ep_step: 970 2023-05-30 03:06:59 - SimpleLog - INFO: - episode: 5926, ep_reward: -69.857, ep_step: 516 2023-05-30 03:07:02 - SimpleLog - INFO: - episode: 5927, ep_reward: 79.705, ep_step: 1000 2023-05-30 03:07:04 - SimpleLog - INFO: - episode: 5928, ep_reward: 171.694, ep_step: 1000 2023-05-30 03:07:05 - SimpleLog - INFO: - episode: 5929, ep_reward: -6.893, ep_step: 724 2023-05-30 03:07:08 - SimpleLog - INFO: - episode: 5930, ep_reward: 132.770, ep_step: 1000 2023-05-30 03:07:10 - SimpleLog - INFO: - episode: 5931, ep_reward: 156.120, ep_step: 1000 2023-05-30 03:07:12 - SimpleLog - INFO: - episode: 5932, ep_reward: 174.898, ep_step: 1000 2023-05-30 03:07:14 - SimpleLog - INFO: - episode: 5933, ep_reward: -25.849, ep_step: 872 2023-05-30 03:07:16 - SimpleLog - INFO: - episode: 5934, ep_reward: 159.891, ep_step: 1000 2023-05-30 03:07:19 - SimpleLog - INFO: - episode: 5935, ep_reward: 146.082, ep_step: 1000 2023-05-30 03:07:21 - SimpleLog - INFO: - episode: 5936, ep_reward: -0.524, ep_step: 1000 2023-05-30 03:07:23 - SimpleLog - INFO: - episode: 5937, ep_reward: 84.527, ep_step: 1000 2023-05-30 03:07:26 - SimpleLog - INFO: - episode: 5938, ep_reward: 132.802, ep_step: 1000 2023-05-30 03:07:28 - SimpleLog - INFO: - episode: 5939, ep_reward: 122.204, ep_step: 1000 2023-05-30 03:07:29 - SimpleLog - INFO: - episode: 5940, ep_reward: -25.801, ep_step: 494 2023-05-30 03:07:31 - SimpleLog - INFO: - episode: 5941, ep_reward: 4.434, ep_step: 749 2023-05-30 03:07:33 - SimpleLog - INFO: - episode: 5942, ep_reward: 80.012, ep_step: 1000 2023-05-30 03:07:35 - SimpleLog - INFO: - episode: 5943, ep_reward: 101.140, ep_step: 1000 2023-05-30 03:07:37 - SimpleLog - INFO: - episode: 5944, ep_reward: 157.974, ep_step: 1000 2023-05-30 03:07:38 - SimpleLog - INFO: - episode: 5945, ep_reward: -69.820, ep_step: 524 2023-05-30 03:07:41 - SimpleLog - INFO: - episode: 5946, ep_reward: 78.391, ep_step: 1000 2023-05-30 03:07:43 - SimpleLog - INFO: - episode: 5947, ep_reward: 128.853, ep_step: 1000 2023-05-30 03:07:45 - SimpleLog - INFO: - episode: 5948, ep_reward: 168.603, ep_step: 1000 2023-05-30 03:07:47 - SimpleLog - INFO: - episode: 5949, ep_reward: 167.653, ep_step: 1000 2023-05-30 03:07:49 - SimpleLog - INFO: - episode: 5950, ep_reward: 13.631, ep_step: 811 2023-05-30 03:07:50 - SimpleLog - INFO: - episode: 5951, ep_reward: -33.729, ep_step: 502 2023-05-30 03:07:53 - SimpleLog - INFO: - episode: 5952, ep_reward: 161.947, ep_step: 1000 2023-05-30 03:07:55 - SimpleLog - INFO: - episode: 5953, ep_reward: 139.927, ep_step: 1000 2023-05-30 03:07:57 - SimpleLog - INFO: - episode: 5954, ep_reward: 163.343, ep_step: 1000 2023-05-30 03:07:59 - SimpleLog - INFO: - episode: 5955, ep_reward: 171.894, ep_step: 1000 2023-05-30 03:08:02 - SimpleLog - INFO: - episode: 5956, ep_reward: 158.471, ep_step: 1000 2023-05-30 03:08:04 - SimpleLog - INFO: - episode: 5957, ep_reward: 155.597, ep_step: 1000 2023-05-30 03:08:06 - SimpleLog - INFO: - update_step: 4690000, online_eval_reward: 173.925 2023-05-30 03:08:08 - SimpleLog - INFO: - episode: 5958, ep_reward: 126.245, ep_step: 1000 2023-05-30 03:08:10 - SimpleLog - INFO: - episode: 5959, ep_reward: 143.737, ep_step: 1000 2023-05-30 03:08:13 - SimpleLog - INFO: - episode: 5960, ep_reward: 140.536, ep_step: 1000 2023-05-30 03:08:15 - SimpleLog - INFO: - episode: 5961, ep_reward: 148.621, ep_step: 1000 2023-05-30 03:08:17 - SimpleLog - INFO: - episode: 5962, ep_reward: 172.656, ep_step: 1000 2023-05-30 03:08:19 - SimpleLog - INFO: - episode: 5963, ep_reward: 173.326, ep_step: 1000 2023-05-30 03:08:22 - SimpleLog - INFO: - episode: 5964, ep_reward: 160.695, ep_step: 1000 2023-05-30 03:08:24 - SimpleLog - INFO: - episode: 5965, ep_reward: 171.189, ep_step: 1000 2023-05-30 03:08:26 - SimpleLog - INFO: - episode: 5966, ep_reward: -4.178, ep_step: 861 2023-05-30 03:08:28 - SimpleLog - INFO: - episode: 5967, ep_reward: 166.554, ep_step: 1000 2023-05-30 03:08:30 - SimpleLog - INFO: - episode: 5968, ep_reward: 140.110, ep_step: 1000 2023-05-30 03:08:32 - SimpleLog - INFO: - episode: 5969, ep_reward: 141.339, ep_step: 1000 2023-05-30 03:08:35 - SimpleLog - INFO: - episode: 5970, ep_reward: 162.028, ep_step: 1000 2023-05-30 03:08:37 - SimpleLog - INFO: - episode: 5971, ep_reward: 146.602, ep_step: 1000 2023-05-30 03:08:39 - SimpleLog - INFO: - episode: 5972, ep_reward: 164.563, ep_step: 1000 2023-05-30 03:08:41 - SimpleLog - INFO: - episode: 5973, ep_reward: 162.884, ep_step: 1000 2023-05-30 03:08:44 - SimpleLog - INFO: - episode: 5974, ep_reward: 103.295, ep_step: 1000 2023-05-30 03:08:46 - SimpleLog - INFO: - episode: 5975, ep_reward: 166.604, ep_step: 1000 2023-05-30 03:08:48 - SimpleLog - INFO: - episode: 5976, ep_reward: 185.442, ep_step: 1000 2023-05-30 03:08:50 - SimpleLog - INFO: - episode: 5977, ep_reward: 158.762, ep_step: 1000 2023-05-30 03:08:53 - SimpleLog - INFO: - episode: 5978, ep_reward: 173.990, ep_step: 1000 2023-05-30 03:08:55 - SimpleLog - INFO: - episode: 5979, ep_reward: 151.417, ep_step: 1000 2023-05-30 03:08:57 - SimpleLog - INFO: - episode: 5980, ep_reward: 174.439, ep_step: 1000 2023-05-30 03:08:59 - SimpleLog - INFO: - episode: 5981, ep_reward: 150.608, ep_step: 1000 2023-05-30 03:09:02 - SimpleLog - INFO: - episode: 5982, ep_reward: 142.519, ep_step: 1000 2023-05-30 03:09:04 - SimpleLog - INFO: - episode: 5983, ep_reward: 162.851, ep_step: 1000 2023-05-30 03:09:06 - SimpleLog - INFO: - episode: 5984, ep_reward: 148.017, ep_step: 1000 2023-05-30 03:09:08 - SimpleLog - INFO: - episode: 5985, ep_reward: 13.130, ep_step: 815 2023-05-30 03:09:10 - SimpleLog - INFO: - episode: 5986, ep_reward: 151.387, ep_step: 1000 2023-05-30 03:09:12 - SimpleLog - INFO: - episode: 5987, ep_reward: 163.483, ep_step: 1000 2023-05-30 03:09:15 - SimpleLog - INFO: - episode: 5988, ep_reward: 101.044, ep_step: 1000 2023-05-30 03:09:17 - SimpleLog - INFO: - episode: 5989, ep_reward: 87.377, ep_step: 1000 2023-05-30 03:09:19 - SimpleLog - INFO: - episode: 5990, ep_reward: 157.011, ep_step: 1000 2023-05-30 03:09:21 - SimpleLog - INFO: - episode: 5991, ep_reward: 155.037, ep_step: 1000 2023-05-30 03:09:24 - SimpleLog - INFO: - episode: 5992, ep_reward: 161.908, ep_step: 1000 2023-05-30 03:09:26 - SimpleLog - INFO: - episode: 5993, ep_reward: -31.715, ep_step: 822 2023-05-30 03:09:28 - SimpleLog - INFO: - episode: 5994, ep_reward: 138.113, ep_step: 1000 2023-05-30 03:09:30 - SimpleLog - INFO: - episode: 5995, ep_reward: 121.569, ep_step: 1000 2023-05-30 03:09:32 - SimpleLog - INFO: - episode: 5996, ep_reward: 157.423, ep_step: 1000 2023-05-30 03:09:35 - SimpleLog - INFO: - episode: 5997, ep_reward: 162.922, ep_step: 1000 2023-05-30 03:09:37 - SimpleLog - INFO: - episode: 5998, ep_reward: -8.667, ep_step: 952 2023-05-30 03:09:39 - SimpleLog - INFO: - episode: 5999, ep_reward: 103.900, ep_step: 1000 2023-05-30 03:09:41 - SimpleLog - INFO: - episode: 6000, ep_reward: 126.575, ep_step: 1000 2023-05-30 03:09:43 - SimpleLog - INFO: - episode: 6001, ep_reward: 123.621, ep_step: 1000 2023-05-30 03:09:46 - SimpleLog - INFO: - episode: 6002, ep_reward: 68.536, ep_step: 1000 2023-05-30 03:09:48 - SimpleLog - INFO: - episode: 6003, ep_reward: -20.577, ep_step: 790 2023-05-30 03:09:49 - SimpleLog - INFO: - episode: 6004, ep_reward: -26.412, ep_step: 586 2023-05-30 03:09:50 - SimpleLog - INFO: - episode: 6005, ep_reward: -12.759, ep_step: 655 2023-05-30 03:09:52 - SimpleLog - INFO: - episode: 6006, ep_reward: -8.416, ep_step: 733 2023-05-30 03:09:54 - SimpleLog - INFO: - episode: 6007, ep_reward: 156.146, ep_step: 1000 2023-05-30 03:09:56 - SimpleLog - INFO: - episode: 6008, ep_reward: 168.732, ep_step: 1000 2023-05-30 03:09:58 - SimpleLog - INFO: - episode: 6009, ep_reward: 9.071, ep_step: 699 2023-05-30 03:10:00 - SimpleLog - INFO: - episode: 6010, ep_reward: 166.195, ep_step: 1000 2023-05-30 03:10:02 - SimpleLog - INFO: - episode: 6011, ep_reward: 183.810, ep_step: 1000 2023-05-30 03:10:05 - SimpleLog - INFO: - episode: 6012, ep_reward: 157.939, ep_step: 1000 2023-05-30 03:10:06 - SimpleLog - INFO: - episode: 6013, ep_reward: 5.413, ep_step: 683 2023-05-30 03:10:08 - SimpleLog - INFO: - episode: 6014, ep_reward: 84.808, ep_step: 1000 2023-05-30 03:10:11 - SimpleLog - INFO: - episode: 6015, ep_reward: 154.434, ep_step: 1000 2023-05-30 03:10:13 - SimpleLog - INFO: - episode: 6016, ep_reward: 166.932, ep_step: 1000 2023-05-30 03:10:15 - SimpleLog - INFO: - episode: 6017, ep_reward: -1.831, ep_step: 710 2023-05-30 03:10:17 - SimpleLog - INFO: - episode: 6018, ep_reward: 162.700, ep_step: 1000 2023-05-30 03:10:17 - SimpleLog - INFO: - episode: 6019, ep_reward: -83.384, ep_step: 201 2023-05-30 03:10:19 - SimpleLog - INFO: - episode: 6020, ep_reward: -15.833, ep_step: 850 2023-05-30 03:10:21 - SimpleLog - INFO: - episode: 6021, ep_reward: 151.169, ep_step: 1000 2023-05-30 03:10:24 - SimpleLog - INFO: - episode: 6022, ep_reward: 146.389, ep_step: 1000 2023-05-30 03:10:25 - SimpleLog - INFO: - episode: 6023, ep_reward: -4.285, ep_step: 671 2023-05-30 03:10:27 - SimpleLog - INFO: - episode: 6024, ep_reward: 155.068, ep_step: 1000 2023-05-30 03:10:30 - SimpleLog - INFO: - episode: 6025, ep_reward: 82.954, ep_step: 1000 2023-05-30 03:10:32 - SimpleLog - INFO: - episode: 6026, ep_reward: 166.796, ep_step: 1000 2023-05-30 03:10:34 - SimpleLog - INFO: - episode: 6027, ep_reward: 165.840, ep_step: 1000 2023-05-30 03:10:36 - SimpleLog - INFO: - episode: 6028, ep_reward: 9.322, ep_step: 704 2023-05-30 03:10:38 - SimpleLog - INFO: - episode: 6029, ep_reward: 159.891, ep_step: 1000 2023-05-30 03:10:39 - SimpleLog - INFO: - episode: 6030, ep_reward: -46.085, ep_step: 475 2023-05-30 03:10:41 - SimpleLog - INFO: - episode: 6031, ep_reward: 128.305, ep_step: 1000 2023-05-30 03:10:44 - SimpleLog - INFO: - episode: 6032, ep_reward: 148.468, ep_step: 1000 2023-05-30 03:10:46 - SimpleLog - INFO: - update_step: 4760000, online_eval_reward: 149.704 2023-05-30 03:10:48 - SimpleLog - INFO: - episode: 6033, ep_reward: 141.957, ep_step: 1000 2023-05-30 03:10:50 - SimpleLog - INFO: - episode: 6034, ep_reward: 160.043, ep_step: 1000 2023-05-30 03:10:52 - SimpleLog - INFO: - episode: 6035, ep_reward: 16.181, ep_step: 846 2023-05-30 03:10:54 - SimpleLog - INFO: - episode: 6036, ep_reward: 2.984, ep_step: 786 2023-05-30 03:10:56 - SimpleLog - INFO: - episode: 6037, ep_reward: 138.408, ep_step: 1000 2023-05-30 03:10:58 - SimpleLog - INFO: - episode: 6038, ep_reward: 159.244, ep_step: 1000 2023-05-30 03:11:00 - SimpleLog - INFO: - episode: 6039, ep_reward: 147.006, ep_step: 1000 2023-05-30 03:11:03 - SimpleLog - INFO: - episode: 6040, ep_reward: -23.818, ep_step: 951 2023-05-30 03:11:05 - SimpleLog - INFO: - episode: 6041, ep_reward: 135.122, ep_step: 1000 2023-05-30 03:11:07 - SimpleLog - INFO: - episode: 6042, ep_reward: 86.058, ep_step: 1000 2023-05-30 03:11:09 - SimpleLog - INFO: - episode: 6043, ep_reward: 4.059, ep_step: 1000 2023-05-30 03:11:11 - SimpleLog - INFO: - episode: 6044, ep_reward: -26.762, ep_step: 525 2023-05-30 03:11:13 - SimpleLog - INFO: - episode: 6045, ep_reward: 126.237, ep_step: 1000 2023-05-30 03:11:15 - SimpleLog - INFO: - episode: 6046, ep_reward: 133.548, ep_step: 1000 2023-05-30 03:11:17 - SimpleLog - INFO: - episode: 6047, ep_reward: 109.121, ep_step: 1000 2023-05-30 03:11:20 - SimpleLog - INFO: - episode: 6048, ep_reward: 87.517, ep_step: 1000 2023-05-30 03:11:21 - SimpleLog - INFO: - episode: 6049, ep_reward: -64.009, ep_step: 456 2023-05-30 03:11:23 - SimpleLog - INFO: - episode: 6050, ep_reward: 113.570, ep_step: 1000 2023-05-30 03:11:25 - SimpleLog - INFO: - episode: 6051, ep_reward: 99.642, ep_step: 1000 2023-05-30 03:11:27 - SimpleLog - INFO: - episode: 6052, ep_reward: 111.672, ep_step: 1000 2023-05-30 03:11:30 - SimpleLog - INFO: - episode: 6053, ep_reward: 100.682, ep_step: 1000 2023-05-30 03:11:32 - SimpleLog - INFO: - episode: 6054, ep_reward: 113.174, ep_step: 1000 2023-05-30 03:11:34 - SimpleLog - INFO: - episode: 6055, ep_reward: 113.995, ep_step: 1000 2023-05-30 03:11:36 - SimpleLog - INFO: - episode: 6056, ep_reward: 112.699, ep_step: 1000 2023-05-30 03:11:39 - SimpleLog - INFO: - episode: 6057, ep_reward: 99.021, ep_step: 1000 2023-05-30 03:11:39 - SimpleLog - INFO: - episode: 6058, ep_reward: -110.222, ep_step: 42 2023-05-30 03:11:41 - SimpleLog - INFO: - episode: 6059, ep_reward: 102.693, ep_step: 1000 2023-05-30 03:11:43 - SimpleLog - INFO: - episode: 6060, ep_reward: 97.019, ep_step: 1000 2023-05-30 03:11:45 - SimpleLog - INFO: - episode: 6061, ep_reward: 120.233, ep_step: 1000 2023-05-30 03:11:48 - SimpleLog - INFO: - episode: 6062, ep_reward: 124.799, ep_step: 1000 2023-05-30 03:11:50 - SimpleLog - INFO: - episode: 6063, ep_reward: 128.024, ep_step: 1000 2023-05-30 03:11:52 - SimpleLog - INFO: - episode: 6064, ep_reward: 101.891, ep_step: 1000 2023-05-30 03:11:54 - SimpleLog - INFO: - episode: 6065, ep_reward: 111.081, ep_step: 1000 2023-05-30 03:11:57 - SimpleLog - INFO: - episode: 6066, ep_reward: 143.185, ep_step: 1000 2023-05-30 03:11:59 - SimpleLog - INFO: - episode: 6067, ep_reward: 124.766, ep_step: 1000 2023-05-30 03:12:01 - SimpleLog - INFO: - episode: 6068, ep_reward: 134.881, ep_step: 1000 2023-05-30 03:12:02 - SimpleLog - INFO: - episode: 6069, ep_reward: -64.357, ep_step: 453 2023-05-30 03:12:04 - SimpleLog - INFO: - episode: 6070, ep_reward: 136.719, ep_step: 1000 2023-05-30 03:12:07 - SimpleLog - INFO: - episode: 6071, ep_reward: 132.606, ep_step: 1000 2023-05-30 03:12:09 - SimpleLog - INFO: - episode: 6072, ep_reward: 144.663, ep_step: 1000 2023-05-30 03:12:11 - SimpleLog - INFO: - episode: 6073, ep_reward: 144.551, ep_step: 1000 2023-05-30 03:12:13 - SimpleLog - INFO: - episode: 6074, ep_reward: 174.403, ep_step: 1000 2023-05-30 03:12:16 - SimpleLog - INFO: - episode: 6075, ep_reward: 154.062, ep_step: 1000 2023-05-30 03:12:16 - SimpleLog - INFO: - episode: 6076, ep_reward: -81.099, ep_step: 311 2023-05-30 03:12:19 - SimpleLog - INFO: - episode: 6077, ep_reward: 143.727, ep_step: 1000 2023-05-30 03:12:21 - SimpleLog - INFO: - episode: 6078, ep_reward: 141.013, ep_step: 1000 2023-05-30 03:12:23 - SimpleLog - INFO: - episode: 6079, ep_reward: 166.713, ep_step: 1000 2023-05-30 03:12:25 - SimpleLog - INFO: - episode: 6080, ep_reward: 86.014, ep_step: 1000 2023-05-30 03:12:28 - SimpleLog - INFO: - episode: 6081, ep_reward: 167.694, ep_step: 1000 2023-05-30 03:12:30 - SimpleLog - INFO: - episode: 6082, ep_reward: 161.271, ep_step: 1000 2023-05-30 03:12:32 - SimpleLog - INFO: - episode: 6083, ep_reward: 140.693, ep_step: 1000 2023-05-30 03:12:34 - SimpleLog - INFO: - episode: 6084, ep_reward: 156.361, ep_step: 1000 2023-05-30 03:12:37 - SimpleLog - INFO: - episode: 6085, ep_reward: 157.839, ep_step: 1000 2023-05-30 03:12:38 - SimpleLog - INFO: - episode: 6086, ep_reward: 13.472, ep_step: 784 2023-05-30 03:12:41 - SimpleLog - INFO: - episode: 6087, ep_reward: 140.891, ep_step: 1000 2023-05-30 03:12:43 - SimpleLog - INFO: - episode: 6088, ep_reward: 129.770, ep_step: 1000 2023-05-30 03:12:45 - SimpleLog - INFO: - episode: 6089, ep_reward: 135.594, ep_step: 1000 2023-05-30 03:12:47 - SimpleLog - INFO: - episode: 6090, ep_reward: 144.377, ep_step: 1000 2023-05-30 03:12:50 - SimpleLog - INFO: - episode: 6091, ep_reward: 154.957, ep_step: 1000 2023-05-30 03:12:52 - SimpleLog - INFO: - episode: 6092, ep_reward: 162.295, ep_step: 1000 2023-05-30 03:12:54 - SimpleLog - INFO: - episode: 6093, ep_reward: 138.313, ep_step: 1000 2023-05-30 03:12:55 - SimpleLog - INFO: - episode: 6094, ep_reward: -40.208, ep_step: 488 2023-05-30 03:12:56 - SimpleLog - INFO: - episode: 6095, ep_reward: -73.150, ep_step: 442 2023-05-30 03:12:59 - SimpleLog - INFO: - episode: 6096, ep_reward: 137.294, ep_step: 1000 2023-05-30 03:13:00 - SimpleLog - INFO: - episode: 6097, ep_reward: 18.951, ep_step: 780 2023-05-30 03:13:01 - SimpleLog - INFO: - episode: 6098, ep_reward: -35.436, ep_step: 538 2023-05-30 03:13:04 - SimpleLog - INFO: - episode: 6099, ep_reward: 8.717, ep_step: 993 2023-05-30 03:13:04 - SimpleLog - INFO: - episode: 6100, ep_reward: -112.975, ep_step: 129 2023-05-30 03:13:06 - SimpleLog - INFO: - episode: 6101, ep_reward: 143.843, ep_step: 1000 2023-05-30 03:13:08 - SimpleLog - INFO: - episode: 6102, ep_reward: 15.433, ep_step: 891 2023-05-30 03:13:08 - SimpleLog - INFO: - episode: 6103, ep_reward: -124.306, ep_step: 91 2023-05-30 03:13:11 - SimpleLog - INFO: - episode: 6104, ep_reward: 11.194, ep_step: 985 2023-05-30 03:13:13 - SimpleLog - INFO: - episode: 6105, ep_reward: 159.832, ep_step: 1000 2023-05-30 03:13:15 - SimpleLog - INFO: - episode: 6106, ep_reward: 137.735, ep_step: 1000 2023-05-30 03:13:17 - SimpleLog - INFO: - episode: 6107, ep_reward: 114.840, ep_step: 1000 2023-05-30 03:13:20 - SimpleLog - INFO: - episode: 6108, ep_reward: 160.597, ep_step: 1000 2023-05-30 03:13:22 - SimpleLog - INFO: - episode: 6109, ep_reward: 149.137, ep_step: 1000 2023-05-30 03:13:25 - SimpleLog - INFO: - update_step: 4830000, online_eval_reward: 9.123 2023-05-30 03:13:26 - SimpleLog - INFO: - episode: 6110, ep_reward: 25.147, ep_step: 1000 2023-05-30 03:13:28 - SimpleLog - INFO: - episode: 6111, ep_reward: 130.143, ep_step: 1000 2023-05-30 03:13:30 - SimpleLog - INFO: - episode: 6112, ep_reward: 6.918, ep_step: 927 2023-05-30 03:13:33 - SimpleLog - INFO: - episode: 6113, ep_reward: 134.759, ep_step: 1000 2023-05-30 03:13:35 - SimpleLog - INFO: - episode: 6114, ep_reward: 141.827, ep_step: 1000 2023-05-30 03:13:37 - SimpleLog - INFO: - episode: 6115, ep_reward: 133.934, ep_step: 1000 2023-05-30 03:13:39 - SimpleLog - INFO: - episode: 6116, ep_reward: 149.130, ep_step: 1000 2023-05-30 03:13:42 - SimpleLog - INFO: - episode: 6117, ep_reward: 132.483, ep_step: 1000 2023-05-30 03:13:44 - SimpleLog - INFO: - episode: 6118, ep_reward: 145.769, ep_step: 1000 2023-05-30 03:13:46 - SimpleLog - INFO: - episode: 6119, ep_reward: 4.300, ep_step: 925 2023-05-30 03:13:48 - SimpleLog - INFO: - episode: 6120, ep_reward: 130.573, ep_step: 1000 2023-05-30 03:13:50 - SimpleLog - INFO: - episode: 6121, ep_reward: 22.044, ep_step: 876 2023-05-30 03:13:53 - SimpleLog - INFO: - episode: 6122, ep_reward: 149.059, ep_step: 1000 2023-05-30 03:13:55 - SimpleLog - INFO: - episode: 6123, ep_reward: 85.067, ep_step: 1000 2023-05-30 03:13:57 - SimpleLog - INFO: - episode: 6124, ep_reward: 144.885, ep_step: 1000 2023-05-30 03:13:59 - SimpleLog - INFO: - episode: 6125, ep_reward: 151.691, ep_step: 1000 2023-05-30 03:14:02 - SimpleLog - INFO: - episode: 6126, ep_reward: 143.548, ep_step: 1000 2023-05-30 03:14:04 - SimpleLog - INFO: - episode: 6127, ep_reward: 141.327, ep_step: 1000 2023-05-30 03:14:06 - SimpleLog - INFO: - episode: 6128, ep_reward: 155.208, ep_step: 1000 2023-05-30 03:14:09 - SimpleLog - INFO: - episode: 6129, ep_reward: 148.802, ep_step: 1000 2023-05-30 03:14:11 - SimpleLog - INFO: - episode: 6130, ep_reward: 153.693, ep_step: 1000 2023-05-30 03:14:13 - SimpleLog - INFO: - episode: 6131, ep_reward: 121.316, ep_step: 1000 2023-05-30 03:14:15 - SimpleLog - INFO: - episode: 6132, ep_reward: 150.685, ep_step: 1000 2023-05-30 03:14:18 - SimpleLog - INFO: - episode: 6133, ep_reward: 137.656, ep_step: 1000 2023-05-30 03:14:20 - SimpleLog - INFO: - episode: 6134, ep_reward: 138.490, ep_step: 1000 2023-05-30 03:14:22 - SimpleLog - INFO: - episode: 6135, ep_reward: 154.938, ep_step: 1000 2023-05-30 03:14:24 - SimpleLog - INFO: - episode: 6136, ep_reward: 111.267, ep_step: 1000 2023-05-30 03:14:27 - SimpleLog - INFO: - episode: 6137, ep_reward: 156.071, ep_step: 1000 2023-05-30 03:14:29 - SimpleLog - INFO: - episode: 6138, ep_reward: 86.105, ep_step: 1000 2023-05-30 03:14:31 - SimpleLog - INFO: - episode: 6139, ep_reward: 1.357, ep_step: 977 2023-05-30 03:14:33 - SimpleLog - INFO: - episode: 6140, ep_reward: 101.954, ep_step: 1000 2023-05-30 03:14:36 - SimpleLog - INFO: - episode: 6141, ep_reward: 152.570, ep_step: 1000 2023-05-30 03:14:38 - SimpleLog - INFO: - episode: 6142, ep_reward: 149.540, ep_step: 1000 2023-05-30 03:14:40 - SimpleLog - INFO: - episode: 6143, ep_reward: 126.684, ep_step: 1000 2023-05-30 03:14:42 - SimpleLog - INFO: - episode: 6144, ep_reward: 8.822, ep_step: 930 2023-05-30 03:14:45 - SimpleLog - INFO: - episode: 6145, ep_reward: 155.427, ep_step: 1000 2023-05-30 03:14:47 - SimpleLog - INFO: - episode: 6146, ep_reward: 159.270, ep_step: 1000 2023-05-30 03:14:49 - SimpleLog - INFO: - episode: 6147, ep_reward: 188.622, ep_step: 1000 2023-05-30 03:14:51 - SimpleLog - INFO: - episode: 6148, ep_reward: 167.252, ep_step: 1000 2023-05-30 03:14:54 - SimpleLog - INFO: - episode: 6149, ep_reward: 157.208, ep_step: 1000 2023-05-30 03:14:54 - SimpleLog - INFO: - episode: 6150, ep_reward: -56.714, ep_step: 364 2023-05-30 03:14:56 - SimpleLog - INFO: - episode: 6151, ep_reward: -22.960, ep_step: 709 2023-05-30 03:14:58 - SimpleLog - INFO: - episode: 6152, ep_reward: 150.518, ep_step: 1000 2023-05-30 03:15:00 - SimpleLog - INFO: - episode: 6153, ep_reward: 150.342, ep_step: 1000 2023-05-30 03:15:03 - SimpleLog - INFO: - episode: 6154, ep_reward: 150.978, ep_step: 1000 2023-05-30 03:15:05 - SimpleLog - INFO: - episode: 6155, ep_reward: 170.365, ep_step: 1000 2023-05-30 03:15:07 - SimpleLog - INFO: - episode: 6156, ep_reward: 180.538, ep_step: 1000 2023-05-30 03:15:09 - SimpleLog - INFO: - episode: 6157, ep_reward: 170.792, ep_step: 1000 2023-05-30 03:15:12 - SimpleLog - INFO: - episode: 6158, ep_reward: 135.349, ep_step: 1000 2023-05-30 03:15:13 - SimpleLog - INFO: - episode: 6159, ep_reward: 7.494, ep_step: 729 2023-05-30 03:15:16 - SimpleLog - INFO: - episode: 6160, ep_reward: 153.722, ep_step: 1000 2023-05-30 03:15:16 - SimpleLog - INFO: - episode: 6161, ep_reward: -95.884, ep_step: 214 2023-05-30 03:15:18 - SimpleLog - INFO: - episode: 6162, ep_reward: 168.885, ep_step: 1000 2023-05-30 03:15:20 - SimpleLog - INFO: - episode: 6163, ep_reward: -21.591, ep_step: 740 2023-05-30 03:15:20 - SimpleLog - INFO: - episode: 6164, ep_reward: -95.696, ep_step: 215 2023-05-30 03:15:21 - SimpleLog - INFO: - episode: 6165, ep_reward: -93.881, ep_step: 152 2023-05-30 03:15:23 - SimpleLog - INFO: - episode: 6166, ep_reward: 168.951, ep_step: 1000 2023-05-30 03:15:25 - SimpleLog - INFO: - episode: 6167, ep_reward: 112.381, ep_step: 1000 2023-05-30 03:15:28 - SimpleLog - INFO: - episode: 6168, ep_reward: 148.978, ep_step: 1000 2023-05-30 03:15:30 - SimpleLog - INFO: - episode: 6169, ep_reward: 161.463, ep_step: 1000 2023-05-30 03:15:32 - SimpleLog - INFO: - episode: 6170, ep_reward: 163.472, ep_step: 1000 2023-05-30 03:15:34 - SimpleLog - INFO: - episode: 6171, ep_reward: 164.988, ep_step: 1000 2023-05-30 03:15:37 - SimpleLog - INFO: - episode: 6172, ep_reward: 164.735, ep_step: 1000 2023-05-30 03:15:39 - SimpleLog - INFO: - episode: 6173, ep_reward: 86.069, ep_step: 1000 2023-05-30 03:15:41 - SimpleLog - INFO: - episode: 6174, ep_reward: 157.853, ep_step: 1000 2023-05-30 03:15:42 - SimpleLog - INFO: - episode: 6175, ep_reward: -42.795, ep_step: 565 2023-05-30 03:15:45 - SimpleLog - INFO: - episode: 6176, ep_reward: 164.126, ep_step: 1000 2023-05-30 03:15:47 - SimpleLog - INFO: - episode: 6177, ep_reward: 151.671, ep_step: 1000 2023-05-30 03:15:47 - SimpleLog - INFO: - episode: 6178, ep_reward: -130.462, ep_step: 98 2023-05-30 03:15:47 - SimpleLog - INFO: - episode: 6179, ep_reward: -132.366, ep_step: 117 2023-05-30 03:15:50 - SimpleLog - INFO: - episode: 6180, ep_reward: 108.522, ep_step: 1000 2023-05-30 03:15:51 - SimpleLog - INFO: - episode: 6181, ep_reward: 12.177, ep_step: 848 2023-05-30 03:15:54 - SimpleLog - INFO: - episode: 6182, ep_reward: 121.863, ep_step: 1000 2023-05-30 03:15:56 - SimpleLog - INFO: - episode: 6183, ep_reward: 179.049, ep_step: 1000 2023-05-30 03:15:58 - SimpleLog - INFO: - episode: 6184, ep_reward: 149.046, ep_step: 1000 2023-05-30 03:16:00 - SimpleLog - INFO: - episode: 6185, ep_reward: 149.108, ep_step: 1000 2023-05-30 03:16:03 - SimpleLog - INFO: - episode: 6186, ep_reward: 154.259, ep_step: 1000 2023-05-30 03:16:05 - SimpleLog - INFO: - update_step: 4900000, online_eval_reward: 18.619 2023-05-30 03:16:07 - SimpleLog - INFO: - episode: 6187, ep_reward: 164.436, ep_step: 1000 2023-05-30 03:16:07 - SimpleLog - INFO: - episode: 6188, ep_reward: -89.983, ep_step: 174 2023-05-30 03:16:09 - SimpleLog - INFO: - episode: 6189, ep_reward: 148.767, ep_step: 1000 2023-05-30 03:16:10 - SimpleLog - INFO: - episode: 6190, ep_reward: -65.873, ep_step: 461 2023-05-30 03:16:13 - SimpleLog - INFO: - episode: 6191, ep_reward: 7.798, ep_step: 1000 2023-05-30 03:16:15 - SimpleLog - INFO: - episode: 6192, ep_reward: 161.002, ep_step: 1000 2023-05-30 03:16:17 - SimpleLog - INFO: - episode: 6193, ep_reward: 4.682, ep_step: 849 2023-05-30 03:16:19 - SimpleLog - INFO: - episode: 6194, ep_reward: 159.363, ep_step: 1000 2023-05-30 03:16:21 - SimpleLog - INFO: - episode: 6195, ep_reward: 153.481, ep_step: 1000 2023-05-30 03:16:24 - SimpleLog - INFO: - episode: 6196, ep_reward: 153.672, ep_step: 1000 2023-05-30 03:16:26 - SimpleLog - INFO: - episode: 6197, ep_reward: 140.910, ep_step: 1000 2023-05-30 03:16:28 - SimpleLog - INFO: - episode: 6198, ep_reward: 151.757, ep_step: 1000 2023-05-30 03:16:30 - SimpleLog - INFO: - episode: 6199, ep_reward: 107.150, ep_step: 1000 2023-05-30 03:16:33 - SimpleLog - INFO: - episode: 6200, ep_reward: 152.446, ep_step: 1000 2023-05-30 03:16:35 - SimpleLog - INFO: - episode: 6201, ep_reward: 76.759, ep_step: 1000 2023-05-30 03:16:37 - SimpleLog - INFO: - episode: 6202, ep_reward: 155.438, ep_step: 1000 2023-05-30 03:16:39 - SimpleLog - INFO: - episode: 6203, ep_reward: 106.117, ep_step: 1000 2023-05-30 03:16:42 - SimpleLog - INFO: - episode: 6204, ep_reward: 174.015, ep_step: 1000 2023-05-30 03:16:44 - SimpleLog - INFO: - episode: 6205, ep_reward: 148.173, ep_step: 1000 2023-05-30 03:16:46 - SimpleLog - INFO: - episode: 6206, ep_reward: 160.564, ep_step: 1000 2023-05-30 03:16:48 - SimpleLog - INFO: - episode: 6207, ep_reward: 163.222, ep_step: 1000 2023-05-30 03:16:51 - SimpleLog - INFO: - episode: 6208, ep_reward: 164.358, ep_step: 1000 2023-05-30 03:16:53 - SimpleLog - INFO: - episode: 6209, ep_reward: 164.521, ep_step: 1000 2023-05-30 03:16:55 - SimpleLog - INFO: - episode: 6210, ep_reward: 3.712, ep_step: 875 2023-05-30 03:16:57 - SimpleLog - INFO: - episode: 6211, ep_reward: 177.098, ep_step: 1000 2023-05-30 03:16:59 - SimpleLog - INFO: - episode: 6212, ep_reward: 184.847, ep_step: 1000 2023-05-30 03:17:02 - SimpleLog - INFO: - episode: 6213, ep_reward: 139.498, ep_step: 1000 2023-05-30 03:17:04 - SimpleLog - INFO: - episode: 6214, ep_reward: 3.662, ep_step: 897 2023-05-30 03:17:05 - SimpleLog - INFO: - episode: 6215, ep_reward: 4.943, ep_step: 792 2023-05-30 03:17:08 - SimpleLog - INFO: - episode: 6216, ep_reward: 73.805, ep_step: 1000 2023-05-30 03:17:09 - SimpleLog - INFO: - episode: 6217, ep_reward: -44.410, ep_step: 650 2023-05-30 03:17:11 - SimpleLog - INFO: - episode: 6218, ep_reward: -30.993, ep_step: 842 2023-05-30 03:17:13 - SimpleLog - INFO: - episode: 6219, ep_reward: 5.362, ep_step: 789 2023-05-30 03:17:15 - SimpleLog - INFO: - episode: 6220, ep_reward: 7.158, ep_step: 808 2023-05-30 03:17:17 - SimpleLog - INFO: - episode: 6221, ep_reward: 3.425, ep_step: 917 2023-05-30 03:17:18 - SimpleLog - INFO: - episode: 6222, ep_reward: -75.137, ep_step: 459 2023-05-30 03:17:20 - SimpleLog - INFO: - episode: 6223, ep_reward: -0.001, ep_step: 885 2023-05-30 03:17:22 - SimpleLog - INFO: - episode: 6224, ep_reward: 72.759, ep_step: 1000 2023-05-30 03:17:24 - SimpleLog - INFO: - episode: 6225, ep_reward: 138.836, ep_step: 1000 2023-05-30 03:17:27 - SimpleLog - INFO: - episode: 6226, ep_reward: 139.867, ep_step: 1000 2023-05-30 03:17:28 - SimpleLog - INFO: - episode: 6227, ep_reward: 8.058, ep_step: 868 2023-05-30 03:17:30 - SimpleLog - INFO: - episode: 6228, ep_reward: -44.918, ep_step: 552 2023-05-30 03:17:32 - SimpleLog - INFO: - episode: 6229, ep_reward: 87.940, ep_step: 1000 2023-05-30 03:17:34 - SimpleLog - INFO: - episode: 6230, ep_reward: 77.547, ep_step: 1000 2023-05-30 03:17:36 - SimpleLog - INFO: - episode: 6231, ep_reward: 4.939, ep_step: 897 2023-05-30 03:17:38 - SimpleLog - INFO: - episode: 6232, ep_reward: 138.728, ep_step: 1000 2023-05-30 03:17:41 - SimpleLog - INFO: - episode: 6233, ep_reward: 179.751, ep_step: 1000 2023-05-30 03:17:43 - SimpleLog - INFO: - episode: 6234, ep_reward: 108.934, ep_step: 1000 2023-05-30 03:17:45 - SimpleLog - INFO: - episode: 6235, ep_reward: 145.990, ep_step: 1000 2023-05-30 03:17:48 - SimpleLog - INFO: - episode: 6236, ep_reward: 151.755, ep_step: 1000 2023-05-30 03:17:50 - SimpleLog - INFO: - episode: 6237, ep_reward: 150.057, ep_step: 1000 2023-05-30 03:17:52 - SimpleLog - INFO: - episode: 6238, ep_reward: 84.521, ep_step: 1000 2023-05-30 03:17:54 - SimpleLog - INFO: - episode: 6239, ep_reward: 88.178, ep_step: 1000 2023-05-30 03:17:57 - SimpleLog - INFO: - episode: 6240, ep_reward: 5.860, ep_step: 1000 2023-05-30 03:17:59 - SimpleLog - INFO: - episode: 6241, ep_reward: 78.491, ep_step: 1000 2023-05-30 03:18:01 - SimpleLog - INFO: - episode: 6242, ep_reward: 82.005, ep_step: 1000 2023-05-30 03:18:03 - SimpleLog - INFO: - episode: 6243, ep_reward: 109.856, ep_step: 1000 2023-05-30 03:18:06 - SimpleLog - INFO: - episode: 6244, ep_reward: 155.250, ep_step: 1000 2023-05-30 03:18:08 - SimpleLog - INFO: - episode: 6245, ep_reward: 84.525, ep_step: 1000 2023-05-30 03:18:10 - SimpleLog - INFO: - episode: 6246, ep_reward: 125.209, ep_step: 1000 2023-05-30 03:18:12 - SimpleLog - INFO: - episode: 6247, ep_reward: 148.524, ep_step: 1000 2023-05-30 03:18:14 - SimpleLog - INFO: - episode: 6248, ep_reward: 153.649, ep_step: 1000 2023-05-30 03:18:17 - SimpleLog - INFO: - episode: 6249, ep_reward: 105.591, ep_step: 1000 2023-05-30 03:18:19 - SimpleLog - INFO: - episode: 6250, ep_reward: 145.383, ep_step: 1000 2023-05-30 03:18:21 - SimpleLog - INFO: - episode: 6251, ep_reward: 77.264, ep_step: 1000 2023-05-30 03:18:23 - SimpleLog - INFO: - episode: 6252, ep_reward: 146.235, ep_step: 1000 2023-05-30 03:18:26 - SimpleLog - INFO: - episode: 6253, ep_reward: 76.726, ep_step: 1000 2023-05-30 03:18:28 - SimpleLog - INFO: - episode: 6254, ep_reward: 161.544, ep_step: 1000 2023-05-30 03:18:30 - SimpleLog - INFO: - episode: 6255, ep_reward: -0.036, ep_step: 961 2023-05-30 03:18:32 - SimpleLog - INFO: - episode: 6256, ep_reward: 74.248, ep_step: 1000 2023-05-30 03:18:35 - SimpleLog - INFO: - episode: 6257, ep_reward: 140.041, ep_step: 1000 2023-05-30 03:18:37 - SimpleLog - INFO: - episode: 6258, ep_reward: 172.187, ep_step: 1000 2023-05-30 03:18:39 - SimpleLog - INFO: - episode: 6259, ep_reward: 158.341, ep_step: 1000 2023-05-30 03:18:41 - SimpleLog - INFO: - episode: 6260, ep_reward: 148.310, ep_step: 1000 2023-05-30 03:18:44 - SimpleLog - INFO: - update_step: 4970000, online_eval_reward: 160.065 2023-05-30 03:18:46 - SimpleLog - INFO: - episode: 6261, ep_reward: 138.913, ep_step: 1000 2023-05-30 03:18:46 - SimpleLog - INFO: - episode: 6262, ep_reward: -65.099, ep_step: 390 2023-05-30 03:18:48 - SimpleLog - INFO: - episode: 6263, ep_reward: -32.942, ep_step: 646 2023-05-30 03:18:50 - SimpleLog - INFO: - episode: 6264, ep_reward: 126.867, ep_step: 1000 2023-05-30 03:18:52 - SimpleLog - INFO: - episode: 6265, ep_reward: 145.322, ep_step: 1000 2023-05-30 03:18:55 - SimpleLog - INFO: - episode: 6266, ep_reward: 92.800, ep_step: 1000 2023-05-30 03:18:55 - SimpleLog - INFO: - episode: 6267, ep_reward: -83.463, ep_step: 208 2023-05-30 03:18:57 - SimpleLog - INFO: - episode: 6268, ep_reward: 125.673, ep_step: 1000 2023-05-30 03:18:59 - SimpleLog - INFO: - episode: 6269, ep_reward: -17.936, ep_step: 633 2023-05-30 03:19:01 - SimpleLog - INFO: - episode: 6270, ep_reward: 114.932, ep_step: 1000 2023-05-30 03:19:03 - SimpleLog - INFO: - episode: 6271, ep_reward: 163.066, ep_step: 1000 2023-05-30 03:19:05 - SimpleLog - INFO: - episode: 6272, ep_reward: 140.508, ep_step: 1000 2023-05-30 03:19:08 - SimpleLog - INFO: - episode: 6273, ep_reward: 168.383, ep_step: 1000 2023-05-30 03:19:10 - SimpleLog - INFO: - episode: 6274, ep_reward: 157.750, ep_step: 1000 2023-05-30 03:19:12 - SimpleLog - INFO: - episode: 6275, ep_reward: 106.884, ep_step: 1000 2023-05-30 03:19:14 - SimpleLog - INFO: - episode: 6276, ep_reward: 40.181, ep_step: 978 2023-05-30 03:19:17 - SimpleLog - INFO: - episode: 6277, ep_reward: 138.001, ep_step: 1000 2023-05-30 03:19:18 - SimpleLog - INFO: - episode: 6278, ep_reward: -79.629, ep_step: 398 2023-05-30 03:19:20 - SimpleLog - INFO: - episode: 6279, ep_reward: 168.577, ep_step: 1000 2023-05-30 03:19:22 - SimpleLog - INFO: - episode: 6280, ep_reward: 160.642, ep_step: 1000 2023-05-30 03:19:24 - SimpleLog - INFO: - episode: 6281, ep_reward: 156.520, ep_step: 1000 2023-05-30 03:19:27 - SimpleLog - INFO: - episode: 6282, ep_reward: 163.411, ep_step: 1000 2023-05-30 03:19:29 - SimpleLog - INFO: - episode: 6283, ep_reward: 136.382, ep_step: 1000 2023-05-30 03:19:31 - SimpleLog - INFO: - episode: 6284, ep_reward: 132.427, ep_step: 1000 2023-05-30 03:19:33 - SimpleLog - INFO: - episode: 6285, ep_reward: 132.983, ep_step: 1000 2023-05-30 03:19:34 - SimpleLog - INFO: - episode: 6286, ep_reward: -70.927, ep_step: 450 2023-05-30 03:19:37 - SimpleLog - INFO: - episode: 6287, ep_reward: 157.703, ep_step: 1000 2023-05-30 03:19:39 - SimpleLog - INFO: - episode: 6288, ep_reward: 135.155, ep_step: 1000 2023-05-30 03:19:41 - SimpleLog - INFO: - episode: 6289, ep_reward: 134.406, ep_step: 1000 2023-05-30 03:19:43 - SimpleLog - INFO: - episode: 6290, ep_reward: 158.086, ep_step: 1000 2023-05-30 03:19:46 - SimpleLog - INFO: - episode: 6291, ep_reward: 160.572, ep_step: 1000 2023-05-30 03:19:48 - SimpleLog - INFO: - episode: 6292, ep_reward: 147.210, ep_step: 1000 2023-05-30 03:19:50 - SimpleLog - INFO: - episode: 6293, ep_reward: 146.716, ep_step: 1000 2023-05-30 03:19:52 - SimpleLog - INFO: - episode: 6294, ep_reward: 147.790, ep_step: 1000 2023-05-30 03:19:55 - SimpleLog - INFO: - episode: 6295, ep_reward: 137.516, ep_step: 1000 2023-05-30 03:19:57 - SimpleLog - INFO: - episode: 6296, ep_reward: 169.169, ep_step: 1000 2023-05-30 03:19:59 - SimpleLog - INFO: - episode: 6297, ep_reward: 11.621, ep_step: 885 2023-05-30 03:20:01 - SimpleLog - INFO: - episode: 6298, ep_reward: 12.553, ep_step: 893 2023-05-30 03:20:03 - SimpleLog - INFO: - episode: 6299, ep_reward: 84.898, ep_step: 1000 2023-05-30 03:20:04 - SimpleLog - INFO: - episode: 6300, ep_reward: -76.449, ep_step: 464 2023-05-30 03:20:05 - SimpleLog - INFO: - episode: 6301, ep_reward: -34.405, ep_step: 520 2023-05-30 03:20:08 - SimpleLog - INFO: - episode: 6302, ep_reward: 147.365, ep_step: 1000 2023-05-30 03:20:10 - SimpleLog - INFO: - episode: 6303, ep_reward: 181.256, ep_step: 1000 2023-05-30 03:20:12 - SimpleLog - INFO: - episode: 6304, ep_reward: 95.209, ep_step: 1000 2023-05-30 03:20:14 - SimpleLog - INFO: - episode: 6305, ep_reward: 169.562, ep_step: 1000 2023-05-30 03:20:16 - SimpleLog - INFO: - episode: 6306, ep_reward: -16.689, ep_step: 762 2023-05-30 03:20:17 - SimpleLog - INFO: - episode: 6307, ep_reward: -37.857, ep_step: 504 2023-05-30 03:20:20 - SimpleLog - INFO: - episode: 6308, ep_reward: 159.278, ep_step: 1000 2023-05-30 03:20:22 - SimpleLog - INFO: - episode: 6309, ep_reward: 162.228, ep_step: 1000 2023-05-30 03:20:24 - SimpleLog - INFO: - episode: 6310, ep_reward: 148.940, ep_step: 1000 2023-05-30 03:20:25 - SimpleLog - INFO: - episode: 6311, ep_reward: -52.383, ep_step: 571 2023-05-30 03:20:28 - SimpleLog - INFO: - episode: 6312, ep_reward: 141.674, ep_step: 1000 2023-05-30 03:20:30 - SimpleLog - INFO: - episode: 6313, ep_reward: 152.390, ep_step: 1000 2023-05-30 03:20:32 - SimpleLog - INFO: - episode: 6314, ep_reward: 0.373, ep_step: 934 2023-05-30 03:20:34 - SimpleLog - INFO: - episode: 6315, ep_reward: 110.245, ep_step: 1000 2023-05-30 03:20:36 - SimpleLog - INFO: - episode: 6316, ep_reward: 100.169, ep_step: 1000 2023-05-30 03:20:39 - SimpleLog - INFO: - episode: 6317, ep_reward: 146.500, ep_step: 1000 2023-05-30 03:20:41 - SimpleLog - INFO: - episode: 6318, ep_reward: 148.034, ep_step: 1000 2023-05-30 03:20:43 - SimpleLog - INFO: - episode: 6319, ep_reward: 141.371, ep_step: 1000 2023-05-30 03:20:46 - SimpleLog - INFO: - episode: 6320, ep_reward: 77.069, ep_step: 1000 2023-05-30 03:20:48 - SimpleLog - INFO: - episode: 6321, ep_reward: 157.383, ep_step: 1000 2023-05-30 03:20:50 - SimpleLog - INFO: - episode: 6322, ep_reward: 167.063, ep_step: 1000 2023-05-30 03:20:52 - SimpleLog - INFO: - episode: 6323, ep_reward: 10.880, ep_step: 811 2023-05-30 03:20:54 - SimpleLog - INFO: - episode: 6324, ep_reward: 162.491, ep_step: 1000 2023-05-30 03:20:55 - SimpleLog - INFO: - episode: 6325, ep_reward: -99.246, ep_step: 214 2023-05-30 03:20:56 - SimpleLog - INFO: - episode: 6326, ep_reward: 5.970, ep_step: 849 2023-05-30 03:20:59 - SimpleLog - INFO: - episode: 6327, ep_reward: 84.010, ep_step: 1000 2023-05-30 03:21:01 - SimpleLog - INFO: - episode: 6328, ep_reward: 153.021, ep_step: 1000 2023-05-30 03:21:03 - SimpleLog - INFO: - episode: 6329, ep_reward: 148.350, ep_step: 1000 2023-05-30 03:21:06 - SimpleLog - INFO: - episode: 6330, ep_reward: 151.640, ep_step: 1000 2023-05-30 03:21:08 - SimpleLog - INFO: - episode: 6331, ep_reward: 178.856, ep_step: 1000 2023-05-30 03:21:09 - SimpleLog - INFO: - episode: 6332, ep_reward: 17.144, ep_step: 715 2023-05-30 03:21:12 - SimpleLog - INFO: - episode: 6333, ep_reward: 154.890, ep_step: 1000 2023-05-30 03:21:14 - SimpleLog - INFO: - episode: 6334, ep_reward: 1.957, ep_step: 851 2023-05-30 03:21:16 - SimpleLog - INFO: - episode: 6335, ep_reward: 87.463, ep_step: 1000 2023-05-30 03:21:18 - SimpleLog - INFO: - episode: 6336, ep_reward: 155.689, ep_step: 1000 2023-05-30 03:21:20 - SimpleLog - INFO: - episode: 6337, ep_reward: 165.312, ep_step: 1000 2023-05-30 03:21:22 - SimpleLog - INFO: - episode: 6338, ep_reward: 6.514, ep_step: 719 2023-05-30 03:21:24 - SimpleLog - INFO: - update_step: 5040000, online_eval_reward: 174.346 2023-05-30 03:21:26 - SimpleLog - INFO: - episode: 6339, ep_reward: 165.572, ep_step: 1000 2023-05-30 03:21:28 - SimpleLog - INFO: - episode: 6340, ep_reward: 118.794, ep_step: 1000 2023-05-30 03:21:30 - SimpleLog - INFO: - episode: 6341, ep_reward: 1.760, ep_step: 608 2023-05-30 03:21:31 - SimpleLog - INFO: - episode: 6342, ep_reward: -37.290, ep_step: 526 2023-05-30 03:21:33 - SimpleLog - INFO: - episode: 6343, ep_reward: -5.042, ep_step: 751 2023-05-30 03:21:35 - SimpleLog - INFO: - episode: 6344, ep_reward: 152.202, ep_step: 1000 2023-05-30 03:21:37 - SimpleLog - INFO: - episode: 6345, ep_reward: 150.151, ep_step: 1000 2023-05-30 03:21:39 - SimpleLog - INFO: - episode: 6346, ep_reward: 92.973, ep_step: 1000 2023-05-30 03:21:42 - SimpleLog - INFO: - episode: 6347, ep_reward: 94.429, ep_step: 1000 2023-05-30 03:21:44 - SimpleLog - INFO: - episode: 6348, ep_reward: 145.882, ep_step: 1000 2023-05-30 03:21:46 - SimpleLog - INFO: - episode: 6349, ep_reward: 143.434, ep_step: 1000 2023-05-30 03:21:48 - SimpleLog - INFO: - episode: 6350, ep_reward: 132.301, ep_step: 1000 2023-05-30 03:21:50 - SimpleLog - INFO: - episode: 6351, ep_reward: 1.668, ep_step: 784 2023-05-30 03:21:51 - SimpleLog - INFO: - episode: 6352, ep_reward: -97.074, ep_step: 247 2023-05-30 03:21:53 - SimpleLog - INFO: - episode: 6353, ep_reward: 121.172, ep_step: 1000 2023-05-30 03:21:55 - SimpleLog - INFO: - episode: 6354, ep_reward: 75.028, ep_step: 1000 2023-05-30 03:21:57 - SimpleLog - INFO: - episode: 6355, ep_reward: 15.103, ep_step: 872 2023-05-30 03:21:59 - SimpleLog - INFO: - episode: 6356, ep_reward: 106.594, ep_step: 1000 2023-05-30 03:22:02 - SimpleLog - INFO: - episode: 6357, ep_reward: 133.000, ep_step: 1000 2023-05-30 03:22:04 - SimpleLog - INFO: - episode: 6358, ep_reward: 104.196, ep_step: 1000 2023-05-30 03:22:06 - SimpleLog - INFO: - episode: 6359, ep_reward: 149.248, ep_step: 1000 2023-05-30 03:22:08 - SimpleLog - INFO: - episode: 6360, ep_reward: 146.973, ep_step: 1000 2023-05-30 03:22:11 - SimpleLog - INFO: - episode: 6361, ep_reward: 121.382, ep_step: 1000 2023-05-30 03:22:13 - SimpleLog - INFO: - episode: 6362, ep_reward: 97.058, ep_step: 1000 2023-05-30 03:22:15 - SimpleLog - INFO: - episode: 6363, ep_reward: 137.537, ep_step: 1000 2023-05-30 03:22:17 - SimpleLog - INFO: - episode: 6364, ep_reward: 138.560, ep_step: 1000 2023-05-30 03:22:20 - SimpleLog - INFO: - episode: 6365, ep_reward: 141.965, ep_step: 1000 2023-05-30 03:22:22 - SimpleLog - INFO: - episode: 6366, ep_reward: 97.666, ep_step: 1000 2023-05-30 03:22:24 - SimpleLog - INFO: - episode: 6367, ep_reward: 134.792, ep_step: 1000 2023-05-30 03:22:27 - SimpleLog - INFO: - episode: 6368, ep_reward: 105.964, ep_step: 1000 2023-05-30 03:22:29 - SimpleLog - INFO: - episode: 6369, ep_reward: 133.217, ep_step: 1000 2023-05-30 03:22:31 - SimpleLog - INFO: - episode: 6370, ep_reward: 129.668, ep_step: 1000 2023-05-30 03:22:33 - SimpleLog - INFO: - episode: 6371, ep_reward: 140.280, ep_step: 1000 2023-05-30 03:22:36 - SimpleLog - INFO: - episode: 6372, ep_reward: 148.551, ep_step: 1000 2023-05-30 03:22:38 - SimpleLog - INFO: - episode: 6373, ep_reward: 135.852, ep_step: 1000 2023-05-30 03:22:40 - SimpleLog - INFO: - episode: 6374, ep_reward: -1.416, ep_step: 958 2023-05-30 03:22:42 - SimpleLog - INFO: - episode: 6375, ep_reward: 134.635, ep_step: 1000 2023-05-30 03:22:44 - SimpleLog - INFO: - episode: 6376, ep_reward: 140.277, ep_step: 1000 2023-05-30 03:22:47 - SimpleLog - INFO: - episode: 6377, ep_reward: 133.065, ep_step: 1000 2023-05-30 03:22:49 - SimpleLog - INFO: - episode: 6378, ep_reward: 75.688, ep_step: 1000 2023-05-30 03:22:51 - SimpleLog - INFO: - episode: 6379, ep_reward: 163.310, ep_step: 1000 2023-05-30 03:22:53 - SimpleLog - INFO: - episode: 6380, ep_reward: 154.745, ep_step: 1000 2023-05-30 03:22:56 - SimpleLog - INFO: - episode: 6381, ep_reward: 153.182, ep_step: 1000 2023-05-30 03:22:58 - SimpleLog - INFO: - episode: 6382, ep_reward: 136.223, ep_step: 1000 2023-05-30 03:23:00 - SimpleLog - INFO: - episode: 6383, ep_reward: 136.207, ep_step: 1000 2023-05-30 03:23:03 - SimpleLog - INFO: - episode: 6384, ep_reward: 139.536, ep_step: 1000 2023-05-30 03:23:05 - SimpleLog - INFO: - episode: 6385, ep_reward: 40.482, ep_step: 958 2023-05-30 03:23:07 - SimpleLog - INFO: - episode: 6386, ep_reward: 144.378, ep_step: 1000 2023-05-30 03:23:09 - SimpleLog - INFO: - episode: 6387, ep_reward: 163.681, ep_step: 1000 2023-05-30 03:23:11 - SimpleLog - INFO: - episode: 6388, ep_reward: 150.879, ep_step: 1000 2023-05-30 03:23:14 - SimpleLog - INFO: - episode: 6389, ep_reward: 171.416, ep_step: 1000 2023-05-30 03:23:16 - SimpleLog - INFO: - episode: 6390, ep_reward: 19.499, ep_step: 794 2023-05-30 03:23:18 - SimpleLog - INFO: - episode: 6391, ep_reward: 149.252, ep_step: 1000 2023-05-30 03:23:20 - SimpleLog - INFO: - episode: 6392, ep_reward: 143.813, ep_step: 1000 2023-05-30 03:23:22 - SimpleLog - INFO: - episode: 6393, ep_reward: 40.794, ep_step: 948 2023-05-30 03:23:24 - SimpleLog - INFO: - episode: 6394, ep_reward: 111.347, ep_step: 1000 2023-05-30 03:23:27 - SimpleLog - INFO: - episode: 6395, ep_reward: 153.184, ep_step: 1000 2023-05-30 03:23:29 - SimpleLog - INFO: - episode: 6396, ep_reward: 19.055, ep_step: 835 2023-05-30 03:23:31 - SimpleLog - INFO: - episode: 6397, ep_reward: 143.154, ep_step: 1000 2023-05-30 03:23:33 - SimpleLog - INFO: - episode: 6398, ep_reward: 24.317, ep_step: 951 2023-05-30 03:23:35 - SimpleLog - INFO: - episode: 6399, ep_reward: 151.958, ep_step: 1000 2023-05-30 03:23:37 - SimpleLog - INFO: - episode: 6400, ep_reward: 136.886, ep_step: 1000 2023-05-30 03:23:40 - SimpleLog - INFO: - episode: 6401, ep_reward: 154.190, ep_step: 1000 2023-05-30 03:23:42 - SimpleLog - INFO: - episode: 6402, ep_reward: 134.423, ep_step: 1000 2023-05-30 03:23:44 - SimpleLog - INFO: - episode: 6403, ep_reward: 133.304, ep_step: 1000 2023-05-30 03:23:46 - SimpleLog - INFO: - episode: 6404, ep_reward: 103.075, ep_step: 1000 2023-05-30 03:23:49 - SimpleLog - INFO: - episode: 6405, ep_reward: 132.256, ep_step: 1000 2023-05-30 03:23:50 - SimpleLog - INFO: - episode: 6406, ep_reward: 15.674, ep_step: 741 2023-05-30 03:23:52 - SimpleLog - INFO: - episode: 6407, ep_reward: 3.225, ep_step: 916 2023-05-30 03:23:55 - SimpleLog - INFO: - episode: 6408, ep_reward: 151.238, ep_step: 1000 2023-05-30 03:23:57 - SimpleLog - INFO: - episode: 6409, ep_reward: 161.116, ep_step: 1000 2023-05-30 03:23:59 - SimpleLog - INFO: - episode: 6410, ep_reward: 100.886, ep_step: 1000 2023-05-30 03:24:01 - SimpleLog - INFO: - episode: 6411, ep_reward: 155.308, ep_step: 1000 2023-05-30 03:24:04 - SimpleLog - INFO: - update_step: 5110000, online_eval_reward: 148.990 2023-05-30 03:24:05 - SimpleLog - INFO: - episode: 6412, ep_reward: 153.127, ep_step: 1000 2023-05-30 03:24:06 - SimpleLog - INFO: - episode: 6413, ep_reward: -111.047, ep_step: 49 2023-05-30 03:24:06 - SimpleLog - INFO: - episode: 6414, ep_reward: -108.734, ep_step: 192 2023-05-30 03:24:08 - SimpleLog - INFO: - episode: 6415, ep_reward: 12.089, ep_step: 839 2023-05-30 03:24:10 - SimpleLog - INFO: - episode: 6416, ep_reward: -0.259, ep_step: 962 2023-05-30 03:24:10 - SimpleLog - INFO: - episode: 6417, ep_reward: -131.775, ep_step: 148 2023-05-30 03:24:13 - SimpleLog - INFO: - episode: 6418, ep_reward: 157.061, ep_step: 1000 2023-05-30 03:24:13 - SimpleLog - INFO: - episode: 6419, ep_reward: -129.090, ep_step: 122 2023-05-30 03:24:13 - SimpleLog - INFO: - episode: 6420, ep_reward: -127.761, ep_step: 147 2023-05-30 03:24:16 - SimpleLog - INFO: - episode: 6421, ep_reward: 138.508, ep_step: 1000 2023-05-30 03:24:18 - SimpleLog - INFO: - episode: 6422, ep_reward: 131.494, ep_step: 1000 2023-05-30 03:24:18 - SimpleLog - INFO: - episode: 6423, ep_reward: -128.394, ep_step: 126 2023-05-30 03:24:18 - SimpleLog - INFO: - episode: 6424, ep_reward: -131.998, ep_step: 126 2023-05-30 03:24:19 - SimpleLog - INFO: - episode: 6425, ep_reward: -126.594, ep_step: 106 2023-05-30 03:24:21 - SimpleLog - INFO: - episode: 6426, ep_reward: 161.099, ep_step: 1000 2023-05-30 03:24:23 - SimpleLog - INFO: - episode: 6427, ep_reward: 146.298, ep_step: 1000 2023-05-30 03:24:25 - SimpleLog - INFO: - episode: 6428, ep_reward: 154.005, ep_step: 1000 2023-05-30 03:24:28 - SimpleLog - INFO: - episode: 6429, ep_reward: 172.467, ep_step: 1000 2023-05-30 03:24:30 - SimpleLog - INFO: - episode: 6430, ep_reward: 178.223, ep_step: 1000 2023-05-30 03:24:31 - SimpleLog - INFO: - episode: 6431, ep_reward: -66.124, ep_step: 445 2023-05-30 03:24:33 - SimpleLog - INFO: - episode: 6432, ep_reward: 155.649, ep_step: 1000 2023-05-30 03:24:35 - SimpleLog - INFO: - episode: 6433, ep_reward: 159.638, ep_step: 1000 2023-05-30 03:24:38 - SimpleLog - INFO: - episode: 6434, ep_reward: 166.512, ep_step: 1000 2023-05-30 03:24:40 - SimpleLog - INFO: - episode: 6435, ep_reward: 155.817, ep_step: 1000 2023-05-30 03:24:42 - SimpleLog - INFO: - episode: 6436, ep_reward: 177.453, ep_step: 1000 2023-05-30 03:24:44 - SimpleLog - INFO: - episode: 6437, ep_reward: 162.215, ep_step: 1000 2023-05-30 03:24:47 - SimpleLog - INFO: - episode: 6438, ep_reward: 166.533, ep_step: 1000 2023-05-30 03:24:49 - SimpleLog - INFO: - episode: 6439, ep_reward: 145.459, ep_step: 1000 2023-05-30 03:24:51 - SimpleLog - INFO: - episode: 6440, ep_reward: 135.237, ep_step: 1000 2023-05-30 03:24:52 - SimpleLog - INFO: - episode: 6441, ep_reward: -103.445, ep_step: 302 2023-05-30 03:24:52 - SimpleLog - INFO: - episode: 6442, ep_reward: -97.416, ep_step: 200 2023-05-30 03:24:55 - SimpleLog - INFO: - episode: 6443, ep_reward: 149.448, ep_step: 1000 2023-05-30 03:24:57 - SimpleLog - INFO: - episode: 6444, ep_reward: 137.662, ep_step: 1000 2023-05-30 03:24:59 - SimpleLog - INFO: - episode: 6445, ep_reward: 145.958, ep_step: 1000 2023-05-30 03:25:01 - SimpleLog - INFO: - episode: 6446, ep_reward: 128.306, ep_step: 1000 2023-05-30 03:25:04 - SimpleLog - INFO: - episode: 6447, ep_reward: 149.920, ep_step: 1000 2023-05-30 03:25:05 - SimpleLog - INFO: - episode: 6448, ep_reward: -77.041, ep_step: 554 2023-05-30 03:25:06 - SimpleLog - INFO: - episode: 6449, ep_reward: -2.381, ep_step: 702 2023-05-30 03:25:09 - SimpleLog - INFO: - episode: 6450, ep_reward: 146.942, ep_step: 1000 2023-05-30 03:25:11 - SimpleLog - INFO: - episode: 6451, ep_reward: 144.845, ep_step: 1000 2023-05-30 03:25:13 - SimpleLog - INFO: - episode: 6452, ep_reward: 16.698, ep_step: 809 2023-05-30 03:25:15 - SimpleLog - INFO: - episode: 6453, ep_reward: 80.667, ep_step: 1000 2023-05-30 03:25:15 - SimpleLog - INFO: - episode: 6454, ep_reward: -122.790, ep_step: 97 2023-05-30 03:25:18 - SimpleLog - INFO: - episode: 6455, ep_reward: 141.509, ep_step: 1000 2023-05-30 03:25:20 - SimpleLog - INFO: - episode: 6456, ep_reward: 150.632, ep_step: 1000 2023-05-30 03:25:22 - SimpleLog - INFO: - episode: 6457, ep_reward: 114.521, ep_step: 1000 2023-05-30 03:25:24 - SimpleLog - INFO: - episode: 6458, ep_reward: 15.096, ep_step: 742 2023-05-30 03:25:26 - SimpleLog - INFO: - episode: 6459, ep_reward: 155.638, ep_step: 1000 2023-05-30 03:25:28 - SimpleLog - INFO: - episode: 6460, ep_reward: 158.507, ep_step: 1000 2023-05-30 03:25:30 - SimpleLog - INFO: - episode: 6461, ep_reward: 28.213, ep_step: 955 2023-05-30 03:25:33 - SimpleLog - INFO: - episode: 6462, ep_reward: 159.898, ep_step: 1000 2023-05-30 03:25:34 - SimpleLog - INFO: - episode: 6463, ep_reward: 20.999, ep_step: 789 2023-05-30 03:25:37 - SimpleLog - INFO: - episode: 6464, ep_reward: 137.907, ep_step: 1000 2023-05-30 03:25:39 - SimpleLog - INFO: - episode: 6465, ep_reward: 145.934, ep_step: 1000 2023-05-30 03:25:41 - SimpleLog - INFO: - episode: 6466, ep_reward: 142.634, ep_step: 1000 2023-05-30 03:25:43 - SimpleLog - INFO: - episode: 6467, ep_reward: 145.168, ep_step: 1000 2023-05-30 03:25:46 - SimpleLog - INFO: - episode: 6468, ep_reward: 97.195, ep_step: 1000 2023-05-30 03:25:48 - SimpleLog - INFO: - episode: 6469, ep_reward: 133.537, ep_step: 1000 2023-05-30 03:25:50 - SimpleLog - INFO: - episode: 6470, ep_reward: 148.446, ep_step: 1000 2023-05-30 03:25:53 - SimpleLog - INFO: - episode: 6471, ep_reward: 153.670, ep_step: 1000 2023-05-30 03:25:55 - SimpleLog - INFO: - episode: 6472, ep_reward: 137.500, ep_step: 1000 2023-05-30 03:25:57 - SimpleLog - INFO: - episode: 6473, ep_reward: 143.463, ep_step: 1000 2023-05-30 03:25:59 - SimpleLog - INFO: - episode: 6474, ep_reward: 177.126, ep_step: 1000 2023-05-30 03:26:01 - SimpleLog - INFO: - episode: 6475, ep_reward: 6.292, ep_step: 718 2023-05-30 03:26:03 - SimpleLog - INFO: - episode: 6476, ep_reward: 164.638, ep_step: 1000 2023-05-30 03:26:05 - SimpleLog - INFO: - episode: 6477, ep_reward: 167.895, ep_step: 1000 2023-05-30 03:26:08 - SimpleLog - INFO: - episode: 6478, ep_reward: 146.382, ep_step: 1000 2023-05-30 03:26:10 - SimpleLog - INFO: - episode: 6479, ep_reward: 159.954, ep_step: 1000 2023-05-30 03:26:11 - SimpleLog - INFO: - episode: 6480, ep_reward: -40.207, ep_step: 518 2023-05-30 03:26:13 - SimpleLog - INFO: - episode: 6481, ep_reward: 138.520, ep_step: 1000 2023-05-30 03:26:15 - SimpleLog - INFO: - episode: 6482, ep_reward: 11.266, ep_step: 858 2023-05-30 03:26:17 - SimpleLog - INFO: - episode: 6483, ep_reward: -3.172, ep_step: 980 2023-05-30 03:26:20 - SimpleLog - INFO: - episode: 6484, ep_reward: 157.193, ep_step: 1000 2023-05-30 03:26:22 - SimpleLog - INFO: - episode: 6485, ep_reward: 140.914, ep_step: 1000 2023-05-30 03:26:24 - SimpleLog - INFO: - episode: 6486, ep_reward: 4.178, ep_step: 903 2023-05-30 03:26:26 - SimpleLog - INFO: - episode: 6487, ep_reward: 84.014, ep_step: 1000 2023-05-30 03:26:28 - SimpleLog - INFO: - episode: 6488, ep_reward: 126.924, ep_step: 1000 2023-05-30 03:26:31 - SimpleLog - INFO: - episode: 6489, ep_reward: 125.333, ep_step: 1000 2023-05-30 03:26:33 - SimpleLog - INFO: - episode: 6490, ep_reward: 7.389, ep_step: 889 2023-05-30 03:26:34 - SimpleLog - INFO: - episode: 6491, ep_reward: -44.260, ep_step: 487 2023-05-30 03:26:36 - SimpleLog - INFO: - episode: 6492, ep_reward: -5.931, ep_step: 909 2023-05-30 03:26:38 - SimpleLog - INFO: - episode: 6493, ep_reward: 143.663, ep_step: 1000 2023-05-30 03:26:40 - SimpleLog - INFO: - episode: 6494, ep_reward: 135.882, ep_step: 1000 2023-05-30 03:26:43 - SimpleLog - INFO: - update_step: 5180000, online_eval_reward: 57.659 2023-05-30 03:26:44 - SimpleLog - INFO: - episode: 6495, ep_reward: 107.832, ep_step: 1000 2023-05-30 03:26:46 - SimpleLog - INFO: - episode: 6496, ep_reward: 1.497, ep_step: 1000 2023-05-30 03:26:49 - SimpleLog - INFO: - episode: 6497, ep_reward: 165.285, ep_step: 1000 2023-05-30 03:26:51 - SimpleLog - INFO: - episode: 6498, ep_reward: 130.568, ep_step: 1000 2023-05-30 03:26:53 - SimpleLog - INFO: - episode: 6499, ep_reward: 162.830, ep_step: 1000 2023-05-30 03:26:55 - SimpleLog - INFO: - episode: 6500, ep_reward: 13.828, ep_step: 983 2023-05-30 03:26:58 - SimpleLog - INFO: - episode: 6501, ep_reward: 128.410, ep_step: 1000 2023-05-30 03:27:00 - SimpleLog - INFO: - episode: 6502, ep_reward: 143.827, ep_step: 1000 2023-05-30 03:27:02 - SimpleLog - INFO: - episode: 6503, ep_reward: 131.873, ep_step: 1000 2023-05-30 03:27:05 - SimpleLog - INFO: - episode: 6504, ep_reward: 162.275, ep_step: 1000 2023-05-30 03:27:07 - SimpleLog - INFO: - episode: 6505, ep_reward: 160.167, ep_step: 1000 2023-05-30 03:27:09 - SimpleLog - INFO: - episode: 6506, ep_reward: 164.143, ep_step: 1000 2023-05-30 03:27:11 - SimpleLog - INFO: - episode: 6507, ep_reward: 169.689, ep_step: 1000 2023-05-30 03:27:14 - SimpleLog - INFO: - episode: 6508, ep_reward: 150.461, ep_step: 1000 2023-05-30 03:27:16 - SimpleLog - INFO: - episode: 6509, ep_reward: 84.206, ep_step: 1000 2023-05-30 03:27:18 - SimpleLog - INFO: - episode: 6510, ep_reward: 180.103, ep_step: 1000 2023-05-30 03:27:20 - SimpleLog - INFO: - episode: 6511, ep_reward: 165.457, ep_step: 1000 2023-05-30 03:27:23 - SimpleLog - INFO: - episode: 6512, ep_reward: 180.104, ep_step: 1000 2023-05-30 03:27:25 - SimpleLog - INFO: - episode: 6513, ep_reward: 150.249, ep_step: 1000 2023-05-30 03:27:27 - SimpleLog - INFO: - episode: 6514, ep_reward: 83.257, ep_step: 1000 2023-05-30 03:27:29 - SimpleLog - INFO: - episode: 6515, ep_reward: 22.003, ep_step: 750 2023-05-30 03:27:31 - SimpleLog - INFO: - episode: 6516, ep_reward: -38.197, ep_step: 867 2023-05-30 03:27:33 - SimpleLog - INFO: - episode: 6517, ep_reward: 157.723, ep_step: 1000 2023-05-30 03:27:34 - SimpleLog - INFO: - episode: 6518, ep_reward: -51.914, ep_step: 494 2023-05-30 03:27:36 - SimpleLog - INFO: - episode: 6519, ep_reward: -23.812, ep_step: 761 2023-05-30 03:27:38 - SimpleLog - INFO: - episode: 6520, ep_reward: 144.643, ep_step: 1000 2023-05-30 03:27:39 - SimpleLog - INFO: - episode: 6521, ep_reward: -42.462, ep_step: 444 2023-05-30 03:27:41 - SimpleLog - INFO: - episode: 6522, ep_reward: 168.018, ep_step: 1000 2023-05-30 03:27:44 - SimpleLog - INFO: - episode: 6523, ep_reward: 177.367, ep_step: 1000 2023-05-30 03:27:45 - SimpleLog - INFO: - episode: 6524, ep_reward: -45.891, ep_step: 423 2023-05-30 03:27:47 - SimpleLog - INFO: - episode: 6525, ep_reward: 150.051, ep_step: 1000 2023-05-30 03:27:49 - SimpleLog - INFO: - episode: 6526, ep_reward: 125.586, ep_step: 1000 2023-05-30 03:27:51 - SimpleLog - INFO: - episode: 6527, ep_reward: 0.388, ep_step: 959 2023-05-30 03:27:53 - SimpleLog - INFO: - episode: 6528, ep_reward: 1.185, ep_step: 679 2023-05-30 03:27:55 - SimpleLog - INFO: - episode: 6529, ep_reward: 150.943, ep_step: 1000 2023-05-30 03:27:57 - SimpleLog - INFO: - episode: 6530, ep_reward: 165.003, ep_step: 1000 2023-05-30 03:27:59 - SimpleLog - INFO: - episode: 6531, ep_reward: 17.792, ep_step: 804 2023-05-30 03:28:02 - SimpleLog - INFO: - episode: 6532, ep_reward: 101.506, ep_step: 1000 2023-05-30 03:28:04 - SimpleLog - INFO: - episode: 6533, ep_reward: 121.638, ep_step: 1000 2023-05-30 03:28:06 - SimpleLog - INFO: - episode: 6534, ep_reward: 168.644, ep_step: 1000 2023-05-30 03:28:08 - SimpleLog - INFO: - episode: 6535, ep_reward: 156.575, ep_step: 1000 2023-05-30 03:28:11 - SimpleLog - INFO: - episode: 6536, ep_reward: 83.464, ep_step: 1000 2023-05-30 03:28:13 - SimpleLog - INFO: - episode: 6537, ep_reward: 161.497, ep_step: 1000 2023-05-30 03:28:14 - SimpleLog - INFO: - episode: 6538, ep_reward: 1.520, ep_step: 707 2023-05-30 03:28:17 - SimpleLog - INFO: - episode: 6539, ep_reward: 170.228, ep_step: 1000 2023-05-30 03:28:19 - SimpleLog - INFO: - episode: 6540, ep_reward: 162.867, ep_step: 1000 2023-05-30 03:28:20 - SimpleLog - INFO: - episode: 6541, ep_reward: 23.407, ep_step: 712 2023-05-30 03:28:23 - SimpleLog - INFO: - episode: 6542, ep_reward: 81.444, ep_step: 1000 2023-05-30 03:28:25 - SimpleLog - INFO: - episode: 6543, ep_reward: 182.431, ep_step: 1000 2023-05-30 03:28:27 - SimpleLog - INFO: - episode: 6544, ep_reward: 156.461, ep_step: 1000 2023-05-30 03:28:29 - SimpleLog - INFO: - episode: 6545, ep_reward: 5.170, ep_step: 904 2023-05-30 03:28:31 - SimpleLog - INFO: - episode: 6546, ep_reward: 142.613, ep_step: 1000 2023-05-30 03:28:34 - SimpleLog - INFO: - episode: 6547, ep_reward: 12.753, ep_step: 933 2023-05-30 03:28:36 - SimpleLog - INFO: - episode: 6548, ep_reward: 4.275, ep_step: 914 2023-05-30 03:28:37 - SimpleLog - INFO: - episode: 6549, ep_reward: -2.503, ep_step: 628 2023-05-30 03:28:39 - SimpleLog - INFO: - episode: 6550, ep_reward: 13.387, ep_step: 777 2023-05-30 03:28:41 - SimpleLog - INFO: - episode: 6551, ep_reward: 139.972, ep_step: 1000 2023-05-30 03:28:43 - SimpleLog - INFO: - episode: 6552, ep_reward: 159.763, ep_step: 1000 2023-05-30 03:28:46 - SimpleLog - INFO: - episode: 6553, ep_reward: 114.045, ep_step: 1000 2023-05-30 03:28:48 - SimpleLog - INFO: - episode: 6554, ep_reward: 119.418, ep_step: 1000 2023-05-30 03:28:50 - SimpleLog - INFO: - episode: 6555, ep_reward: 142.347, ep_step: 1000 2023-05-30 03:28:52 - SimpleLog - INFO: - episode: 6556, ep_reward: 133.199, ep_step: 1000 2023-05-30 03:28:55 - SimpleLog - INFO: - episode: 6557, ep_reward: 130.908, ep_step: 1000 2023-05-30 03:28:57 - SimpleLog - INFO: - episode: 6558, ep_reward: 10.560, ep_step: 890 2023-05-30 03:28:59 - SimpleLog - INFO: - episode: 6559, ep_reward: 123.078, ep_step: 1000 2023-05-30 03:29:01 - SimpleLog - INFO: - episode: 6560, ep_reward: 131.645, ep_step: 1000 2023-05-30 03:29:03 - SimpleLog - INFO: - episode: 6561, ep_reward: 135.392, ep_step: 1000 2023-05-30 03:29:06 - SimpleLog - INFO: - episode: 6562, ep_reward: 150.664, ep_step: 1000 2023-05-30 03:29:08 - SimpleLog - INFO: - episode: 6563, ep_reward: 132.727, ep_step: 1000 2023-05-30 03:29:10 - SimpleLog - INFO: - episode: 6564, ep_reward: 164.597, ep_step: 1000 2023-05-30 03:29:12 - SimpleLog - INFO: - episode: 6565, ep_reward: 139.419, ep_step: 1000 2023-05-30 03:29:14 - SimpleLog - INFO: - episode: 6566, ep_reward: -34.133, ep_step: 539 2023-05-30 03:29:16 - SimpleLog - INFO: - episode: 6567, ep_reward: 98.832, ep_step: 1000 2023-05-30 03:29:18 - SimpleLog - INFO: - episode: 6568, ep_reward: 161.012, ep_step: 1000 2023-05-30 03:29:20 - SimpleLog - INFO: - episode: 6569, ep_reward: 138.838, ep_step: 1000 2023-05-30 03:29:21 - SimpleLog - INFO: - episode: 6570, ep_reward: -91.996, ep_step: 198 2023-05-30 03:29:23 - SimpleLog - INFO: - update_step: 5250000, online_eval_reward: 87.573 2023-05-30 03:29:25 - SimpleLog - INFO: - episode: 6571, ep_reward: 131.260, ep_step: 1000 2023-05-30 03:29:27 - SimpleLog - INFO: - episode: 6572, ep_reward: 141.526, ep_step: 1000 2023-05-30 03:29:27 - SimpleLog - INFO: - episode: 6573, ep_reward: -97.860, ep_step: 271 2023-05-30 03:29:30 - SimpleLog - INFO: - episode: 6574, ep_reward: 3.496, ep_step: 945 2023-05-30 03:29:32 - SimpleLog - INFO: - episode: 6575, ep_reward: 154.768, ep_step: 1000 2023-05-30 03:29:34 - SimpleLog - INFO: - episode: 6576, ep_reward: 124.846, ep_step: 1000 2023-05-30 03:29:36 - SimpleLog - INFO: - episode: 6577, ep_reward: -23.590, ep_step: 781 2023-05-30 03:29:38 - SimpleLog - INFO: - episode: 6578, ep_reward: 155.529, ep_step: 1000 2023-05-30 03:29:40 - SimpleLog - INFO: - episode: 6579, ep_reward: 145.942, ep_step: 1000 2023-05-30 03:29:43 - SimpleLog - INFO: - episode: 6580, ep_reward: 133.137, ep_step: 1000 2023-05-30 03:29:45 - SimpleLog - INFO: - episode: 6581, ep_reward: 123.864, ep_step: 1000 2023-05-30 03:29:47 - SimpleLog - INFO: - episode: 6582, ep_reward: 153.801, ep_step: 1000 2023-05-30 03:29:49 - SimpleLog - INFO: - episode: 6583, ep_reward: 158.574, ep_step: 1000 2023-05-30 03:29:52 - SimpleLog - INFO: - episode: 6584, ep_reward: 172.540, ep_step: 1000 2023-05-30 03:29:54 - SimpleLog - INFO: - episode: 6585, ep_reward: 161.524, ep_step: 1000 2023-05-30 03:29:56 - SimpleLog - INFO: - episode: 6586, ep_reward: 151.334, ep_step: 1000 2023-05-30 03:29:58 - SimpleLog - INFO: - episode: 6587, ep_reward: 156.727, ep_step: 1000 2023-05-30 03:29:59 - SimpleLog - INFO: - episode: 6588, ep_reward: -43.845, ep_step: 422 2023-05-30 03:30:01 - SimpleLog - INFO: - episode: 6589, ep_reward: 161.732, ep_step: 1000 2023-05-30 03:30:04 - SimpleLog - INFO: - episode: 6590, ep_reward: 165.594, ep_step: 1000 2023-05-30 03:30:06 - SimpleLog - INFO: - episode: 6591, ep_reward: 80.599, ep_step: 1000 2023-05-30 03:30:08 - SimpleLog - INFO: - episode: 6592, ep_reward: 146.362, ep_step: 1000 2023-05-30 03:30:10 - SimpleLog - INFO: - episode: 6593, ep_reward: 151.818, ep_step: 1000 2023-05-30 03:30:13 - SimpleLog - INFO: - episode: 6594, ep_reward: 152.639, ep_step: 1000 2023-05-30 03:30:15 - SimpleLog - INFO: - episode: 6595, ep_reward: 150.397, ep_step: 1000 2023-05-30 03:30:17 - SimpleLog - INFO: - episode: 6596, ep_reward: 23.938, ep_step: 747 2023-05-30 03:30:19 - SimpleLog - INFO: - episode: 6597, ep_reward: 135.560, ep_step: 1000 2023-05-30 03:30:21 - SimpleLog - INFO: - episode: 6598, ep_reward: 177.262, ep_step: 1000 2023-05-30 03:30:23 - SimpleLog - INFO: - episode: 6599, ep_reward: 168.284, ep_step: 1000 2023-05-30 03:30:26 - SimpleLog - INFO: - episode: 6600, ep_reward: 179.522, ep_step: 1000 2023-05-30 03:30:28 - SimpleLog - INFO: - episode: 6601, ep_reward: 147.013, ep_step: 1000 2023-05-30 03:30:30 - SimpleLog - INFO: - episode: 6602, ep_reward: 154.732, ep_step: 1000 2023-05-30 03:30:32 - SimpleLog - INFO: - episode: 6603, ep_reward: 106.808, ep_step: 1000 2023-05-30 03:30:35 - SimpleLog - INFO: - episode: 6604, ep_reward: 154.698, ep_step: 1000 2023-05-30 03:30:37 - SimpleLog - INFO: - episode: 6605, ep_reward: 152.534, ep_step: 1000 2023-05-30 03:30:39 - SimpleLog - INFO: - episode: 6606, ep_reward: 140.219, ep_step: 1000 2023-05-30 03:30:41 - SimpleLog - INFO: - episode: 6607, ep_reward: 176.604, ep_step: 1000 2023-05-30 03:30:43 - SimpleLog - INFO: - episode: 6608, ep_reward: 154.500, ep_step: 1000 2023-05-30 03:30:46 - SimpleLog - INFO: - episode: 6609, ep_reward: 138.518, ep_step: 1000 2023-05-30 03:30:48 - SimpleLog - INFO: - episode: 6610, ep_reward: 162.460, ep_step: 1000 2023-05-30 03:30:50 - SimpleLog - INFO: - episode: 6611, ep_reward: 136.303, ep_step: 1000 2023-05-30 03:30:52 - SimpleLog - INFO: - episode: 6612, ep_reward: 158.217, ep_step: 1000 2023-05-30 03:30:55 - SimpleLog - INFO: - episode: 6613, ep_reward: 155.092, ep_step: 1000 2023-05-30 03:30:57 - SimpleLog - INFO: - episode: 6614, ep_reward: 147.074, ep_step: 1000 2023-05-30 03:30:59 - SimpleLog - INFO: - episode: 6615, ep_reward: 151.853, ep_step: 1000 2023-05-30 03:31:01 - SimpleLog - INFO: - episode: 6616, ep_reward: 168.058, ep_step: 1000 2023-05-30 03:31:04 - SimpleLog - INFO: - episode: 6617, ep_reward: 157.415, ep_step: 1000 2023-05-30 03:31:06 - SimpleLog - INFO: - episode: 6618, ep_reward: 87.018, ep_step: 1000 2023-05-30 03:31:08 - SimpleLog - INFO: - episode: 6619, ep_reward: 19.566, ep_step: 758 2023-05-30 03:31:10 - SimpleLog - INFO: - episode: 6620, ep_reward: 151.898, ep_step: 1000 2023-05-30 03:31:12 - SimpleLog - INFO: - episode: 6621, ep_reward: -0.568, ep_step: 795 2023-05-30 03:31:14 - SimpleLog - INFO: - episode: 6622, ep_reward: 179.497, ep_step: 1000 2023-05-30 03:31:16 - SimpleLog - INFO: - episode: 6623, ep_reward: 12.608, ep_step: 954 2023-05-30 03:31:18 - SimpleLog - INFO: - episode: 6624, ep_reward: 188.494, ep_step: 1000 2023-05-30 03:31:20 - SimpleLog - INFO: - episode: 6625, ep_reward: 7.984, ep_step: 644 2023-05-30 03:31:22 - SimpleLog - INFO: - episode: 6626, ep_reward: 180.916, ep_step: 1000 2023-05-30 03:31:24 - SimpleLog - INFO: - episode: 6627, ep_reward: 178.645, ep_step: 1000 2023-05-30 03:31:26 - SimpleLog - INFO: - episode: 6628, ep_reward: 160.326, ep_step: 1000 2023-05-30 03:31:29 - SimpleLog - INFO: - episode: 6629, ep_reward: 127.057, ep_step: 1000 2023-05-30 03:31:31 - SimpleLog - INFO: - episode: 6630, ep_reward: 177.596, ep_step: 1000 2023-05-30 03:31:33 - SimpleLog - INFO: - episode: 6631, ep_reward: 168.777, ep_step: 1000 2023-05-30 03:31:35 - SimpleLog - INFO: - episode: 6632, ep_reward: 167.110, ep_step: 1000 2023-05-30 03:31:37 - SimpleLog - INFO: - episode: 6633, ep_reward: 15.189, ep_step: 794 2023-05-30 03:31:38 - SimpleLog - INFO: - episode: 6634, ep_reward: -99.079, ep_step: 259 2023-05-30 03:31:40 - SimpleLog - INFO: - episode: 6635, ep_reward: 136.101, ep_step: 1000 2023-05-30 03:31:42 - SimpleLog - INFO: - episode: 6636, ep_reward: 176.052, ep_step: 1000 2023-05-30 03:31:44 - SimpleLog - INFO: - episode: 6637, ep_reward: 179.646, ep_step: 1000 2023-05-30 03:31:47 - SimpleLog - INFO: - episode: 6638, ep_reward: 157.639, ep_step: 1000 2023-05-30 03:31:49 - SimpleLog - INFO: - episode: 6639, ep_reward: 159.792, ep_step: 1000 2023-05-30 03:31:51 - SimpleLog - INFO: - episode: 6640, ep_reward: 165.227, ep_step: 1000 2023-05-30 03:31:53 - SimpleLog - INFO: - episode: 6641, ep_reward: 147.361, ep_step: 1000 2023-05-30 03:31:56 - SimpleLog - INFO: - episode: 6642, ep_reward: 137.320, ep_step: 1000 2023-05-30 03:31:57 - SimpleLog - INFO: - episode: 6643, ep_reward: 12.099, ep_step: 732 2023-05-30 03:31:57 - SimpleLog - INFO: - episode: 6644, ep_reward: -113.222, ep_step: 67 2023-05-30 03:32:01 - SimpleLog - INFO: - update_step: 5320000, online_eval_reward: 63.602 2023-05-30 03:32:01 - SimpleLog - INFO: - episode: 6645, ep_reward: 140.087, ep_step: 1000 2023-05-30 03:32:04 - SimpleLog - INFO: - episode: 6646, ep_reward: -2.339, ep_step: 924 2023-05-30 03:32:06 - SimpleLog - INFO: - episode: 6647, ep_reward: 138.925, ep_step: 1000 2023-05-30 03:32:07 - SimpleLog - INFO: - episode: 6648, ep_reward: -103.673, ep_step: 329 2023-05-30 03:32:09 - SimpleLog - INFO: - episode: 6649, ep_reward: 145.138, ep_step: 1000 2023-05-30 03:32:09 - SimpleLog - INFO: - episode: 6650, ep_reward: -142.300, ep_step: 198 2023-05-30 03:32:09 - SimpleLog - INFO: - episode: 6651, ep_reward: -110.132, ep_step: 83 2023-05-30 03:32:10 - SimpleLog - INFO: - episode: 6652, ep_reward: -121.768, ep_step: 84 2023-05-30 03:32:10 - SimpleLog - INFO: - episode: 6653, ep_reward: -133.382, ep_step: 138 2023-05-30 03:32:10 - SimpleLog - INFO: - episode: 6654, ep_reward: -133.701, ep_step: 144 2023-05-30 03:32:12 - SimpleLog - INFO: - episode: 6655, ep_reward: 113.817, ep_step: 1000 2023-05-30 03:32:14 - SimpleLog - INFO: - episode: 6656, ep_reward: 12.299, ep_step: 794 2023-05-30 03:32:15 - SimpleLog - INFO: - episode: 6657, ep_reward: -70.797, ep_step: 363 2023-05-30 03:32:17 - SimpleLog - INFO: - episode: 6658, ep_reward: 139.737, ep_step: 1000 2023-05-30 03:32:19 - SimpleLog - INFO: - episode: 6659, ep_reward: -14.925, ep_step: 713 2023-05-30 03:32:20 - SimpleLog - INFO: - episode: 6660, ep_reward: -87.510, ep_step: 422 2023-05-30 03:32:21 - SimpleLog - INFO: - episode: 6661, ep_reward: -68.938, ep_step: 491 2023-05-30 03:32:23 - SimpleLog - INFO: - episode: 6662, ep_reward: -19.089, ep_step: 772 2023-05-30 03:32:24 - SimpleLog - INFO: - episode: 6663, ep_reward: -56.542, ep_step: 736 2023-05-30 03:32:25 - SimpleLog - INFO: - episode: 6664, ep_reward: -70.582, ep_step: 479 2023-05-30 03:32:28 - SimpleLog - INFO: - episode: 6665, ep_reward: 148.593, ep_step: 1000 2023-05-30 03:32:30 - SimpleLog - INFO: - episode: 6666, ep_reward: -1.071, ep_step: 1000 2023-05-30 03:32:32 - SimpleLog - INFO: - episode: 6667, ep_reward: 118.873, ep_step: 1000 2023-05-30 03:32:34 - SimpleLog - INFO: - episode: 6668, ep_reward: -44.684, ep_step: 601 2023-05-30 03:32:36 - SimpleLog - INFO: - episode: 6669, ep_reward: 142.342, ep_step: 1000 2023-05-30 03:32:38 - SimpleLog - INFO: - episode: 6670, ep_reward: 77.105, ep_step: 1000 2023-05-30 03:32:39 - SimpleLog - INFO: - episode: 6671, ep_reward: -60.950, ep_step: 356 2023-05-30 03:32:41 - SimpleLog - INFO: - episode: 6672, ep_reward: -11.403, ep_step: 941 2023-05-30 03:32:42 - SimpleLog - INFO: - episode: 6673, ep_reward: -49.795, ep_step: 482 2023-05-30 03:32:44 - SimpleLog - INFO: - episode: 6674, ep_reward: 148.998, ep_step: 1000 2023-05-30 03:32:46 - SimpleLog - INFO: - episode: 6675, ep_reward: -9.837, ep_step: 871 2023-05-30 03:32:48 - SimpleLog - INFO: - episode: 6676, ep_reward: 19.151, ep_step: 927 2023-05-30 03:32:51 - SimpleLog - INFO: - episode: 6677, ep_reward: 177.917, ep_step: 1000 2023-05-30 03:32:53 - SimpleLog - INFO: - episode: 6678, ep_reward: 130.031, ep_step: 1000 2023-05-30 03:32:55 - SimpleLog - INFO: - episode: 6679, ep_reward: 154.455, ep_step: 1000 2023-05-30 03:32:56 - SimpleLog - INFO: - episode: 6680, ep_reward: -56.021, ep_step: 539 2023-05-30 03:32:59 - SimpleLog - INFO: - episode: 6681, ep_reward: 168.922, ep_step: 1000 2023-05-30 03:33:00 - SimpleLog - INFO: - episode: 6682, ep_reward: 5.469, ep_step: 701 2023-05-30 03:33:02 - SimpleLog - INFO: - episode: 6683, ep_reward: 143.972, ep_step: 1000 2023-05-30 03:33:05 - SimpleLog - INFO: - episode: 6684, ep_reward: 108.916, ep_step: 1000 2023-05-30 03:33:07 - SimpleLog - INFO: - episode: 6685, ep_reward: 112.145, ep_step: 1000 2023-05-30 03:33:07 - SimpleLog - INFO: - episode: 6686, ep_reward: -95.636, ep_step: 293 2023-05-30 03:33:09 - SimpleLog - INFO: - episode: 6687, ep_reward: -6.892, ep_step: 865 2023-05-30 03:33:12 - SimpleLog - INFO: - episode: 6688, ep_reward: -14.471, ep_step: 975 2023-05-30 03:33:14 - SimpleLog - INFO: - episode: 6689, ep_reward: 2.059, ep_step: 1000 2023-05-30 03:33:16 - SimpleLog - INFO: - episode: 6690, ep_reward: 72.245, ep_step: 1000 2023-05-30 03:33:18 - SimpleLog - INFO: - episode: 6691, ep_reward: -5.107, ep_step: 794 2023-05-30 03:33:20 - SimpleLog - INFO: - episode: 6692, ep_reward: -2.742, ep_step: 819 2023-05-30 03:33:20 - SimpleLog - INFO: - episode: 6693, ep_reward: -115.937, ep_step: 138 2023-05-30 03:33:22 - SimpleLog - INFO: - episode: 6694, ep_reward: 80.788, ep_step: 1000 2023-05-30 03:33:24 - SimpleLog - INFO: - episode: 6695, ep_reward: -19.507, ep_step: 622 2023-05-30 03:33:25 - SimpleLog - INFO: - episode: 6696, ep_reward: -35.032, ep_step: 456 2023-05-30 03:33:27 - SimpleLog - INFO: - episode: 6697, ep_reward: 17.151, ep_step: 848 2023-05-30 03:33:29 - SimpleLog - INFO: - episode: 6698, ep_reward: 143.233, ep_step: 1000 2023-05-30 03:33:30 - SimpleLog - INFO: - episode: 6699, ep_reward: -42.787, ep_step: 519 2023-05-30 03:33:32 - SimpleLog - INFO: - episode: 6700, ep_reward: 166.036, ep_step: 1000 2023-05-30 03:33:35 - SimpleLog - INFO: - episode: 6701, ep_reward: 126.619, ep_step: 1000 2023-05-30 03:33:37 - SimpleLog - INFO: - episode: 6702, ep_reward: 10.832, ep_step: 1000 2023-05-30 03:33:39 - SimpleLog - INFO: - episode: 6703, ep_reward: -3.589, ep_step: 1000 2023-05-30 03:33:41 - SimpleLog - INFO: - episode: 6704, ep_reward: 12.256, ep_step: 1000 2023-05-30 03:33:44 - SimpleLog - INFO: - episode: 6705, ep_reward: 83.976, ep_step: 1000 2023-05-30 03:33:46 - SimpleLog - INFO: - episode: 6706, ep_reward: 117.496, ep_step: 1000 2023-05-30 03:33:48 - SimpleLog - INFO: - episode: 6707, ep_reward: 119.667, ep_step: 1000 2023-05-30 03:33:50 - SimpleLog - INFO: - episode: 6708, ep_reward: 13.207, ep_step: 1000 2023-05-30 03:33:53 - SimpleLog - INFO: - episode: 6709, ep_reward: 137.589, ep_step: 1000 2023-05-30 03:33:55 - SimpleLog - INFO: - episode: 6710, ep_reward: 85.862, ep_step: 1000 2023-05-30 03:33:57 - SimpleLog - INFO: - episode: 6711, ep_reward: 72.668, ep_step: 1000 2023-05-30 03:33:59 - SimpleLog - INFO: - episode: 6712, ep_reward: -85.387, ep_step: 1000 2023-05-30 03:34:02 - SimpleLog - INFO: - episode: 6713, ep_reward: 16.436, ep_step: 1000 2023-05-30 03:34:04 - SimpleLog - INFO: - episode: 6714, ep_reward: 155.871, ep_step: 1000 2023-05-30 03:34:06 - SimpleLog - INFO: - episode: 6715, ep_reward: 117.775, ep_step: 1000 2023-05-30 03:34:08 - SimpleLog - INFO: - episode: 6716, ep_reward: 137.714, ep_step: 1000 2023-05-30 03:34:11 - SimpleLog - INFO: - episode: 6717, ep_reward: 127.871, ep_step: 1000 2023-05-30 03:34:13 - SimpleLog - INFO: - episode: 6718, ep_reward: 136.043, ep_step: 1000 2023-05-30 03:34:14 - SimpleLog - INFO: - episode: 6719, ep_reward: -57.654, ep_step: 386 2023-05-30 03:34:16 - SimpleLog - INFO: - episode: 6720, ep_reward: 123.875, ep_step: 1000 2023-05-30 03:34:18 - SimpleLog - INFO: - episode: 6721, ep_reward: 154.827, ep_step: 1000 2023-05-30 03:34:21 - SimpleLog - INFO: - episode: 6722, ep_reward: 157.160, ep_step: 1000 2023-05-30 03:34:23 - SimpleLog - INFO: - episode: 6723, ep_reward: 0.257, ep_step: 884 2023-05-30 03:34:25 - SimpleLog - INFO: - episode: 6724, ep_reward: 147.283, ep_step: 1000 2023-05-30 03:34:27 - SimpleLog - INFO: - episode: 6725, ep_reward: 131.262, ep_step: 1000 2023-05-30 03:34:29 - SimpleLog - INFO: - episode: 6726, ep_reward: 18.120, ep_step: 967 2023-05-30 03:34:32 - SimpleLog - INFO: - episode: 6727, ep_reward: 78.131, ep_step: 1000 2023-05-30 03:34:34 - SimpleLog - INFO: - episode: 6728, ep_reward: 167.409, ep_step: 1000 2023-05-30 03:34:35 - SimpleLog - INFO: - episode: 6729, ep_reward: -2.855, ep_step: 626 2023-05-30 03:34:35 - SimpleLog - INFO: - episode: 6730, ep_reward: -113.082, ep_step: 62 2023-05-30 03:34:36 - SimpleLog - INFO: - episode: 6731, ep_reward: -127.419, ep_step: 130 2023-05-30 03:34:38 - SimpleLog - INFO: - episode: 6732, ep_reward: 120.761, ep_step: 1000 2023-05-30 03:34:41 - SimpleLog - INFO: - update_step: 5390000, online_eval_reward: 158.999 2023-05-30 03:34:42 - SimpleLog - INFO: - episode: 6733, ep_reward: 117.692, ep_step: 1000 2023-05-30 03:34:44 - SimpleLog - INFO: - episode: 6734, ep_reward: 3.109, ep_step: 934 2023-05-30 03:34:46 - SimpleLog - INFO: - episode: 6735, ep_reward: -7.378, ep_step: 788 2023-05-30 03:34:48 - SimpleLog - INFO: - episode: 6736, ep_reward: 121.221, ep_step: 1000 2023-05-30 03:34:49 - SimpleLog - INFO: - episode: 6737, ep_reward: -93.537, ep_step: 174 2023-05-30 03:34:51 - SimpleLog - INFO: - episode: 6738, ep_reward: 148.447, ep_step: 1000 2023-05-30 03:34:53 - SimpleLog - INFO: - episode: 6739, ep_reward: 15.100, ep_step: 1000 2023-05-30 03:34:55 - SimpleLog - INFO: - episode: 6740, ep_reward: 141.945, ep_step: 1000 2023-05-30 03:34:57 - SimpleLog - INFO: - episode: 6741, ep_reward: -5.554, ep_step: 717 2023-05-30 03:34:59 - SimpleLog - INFO: - episode: 6742, ep_reward: 157.937, ep_step: 1000 2023-05-30 03:35:01 - SimpleLog - INFO: - episode: 6743, ep_reward: 112.323, ep_step: 1000 2023-05-30 03:35:04 - SimpleLog - INFO: - episode: 6744, ep_reward: 152.722, ep_step: 1000 2023-05-30 03:35:06 - SimpleLog - INFO: - episode: 6745, ep_reward: 156.744, ep_step: 1000 2023-05-30 03:35:08 - SimpleLog - INFO: - episode: 6746, ep_reward: 134.661, ep_step: 1000 2023-05-30 03:35:10 - SimpleLog - INFO: - episode: 6747, ep_reward: -16.351, ep_step: 705 2023-05-30 03:35:12 - SimpleLog - INFO: - episode: 6748, ep_reward: 76.017, ep_step: 1000 2023-05-30 03:35:13 - SimpleLog - INFO: - episode: 6749, ep_reward: -45.315, ep_step: 436 2023-05-30 03:35:15 - SimpleLog - INFO: - episode: 6750, ep_reward: -0.487, ep_step: 781 2023-05-30 03:35:17 - SimpleLog - INFO: - episode: 6751, ep_reward: 178.898, ep_step: 1000 2023-05-30 03:35:19 - SimpleLog - INFO: - episode: 6752, ep_reward: 156.673, ep_step: 1000 2023-05-30 03:35:21 - SimpleLog - INFO: - episode: 6753, ep_reward: 168.341, ep_step: 1000 2023-05-30 03:35:23 - SimpleLog - INFO: - episode: 6754, ep_reward: -0.518, ep_step: 930 2023-05-30 03:35:26 - SimpleLog - INFO: - episode: 6755, ep_reward: 84.587, ep_step: 1000 2023-05-30 03:35:27 - SimpleLog - INFO: - episode: 6756, ep_reward: -48.939, ep_step: 445 2023-05-30 03:35:28 - SimpleLog - INFO: - episode: 6757, ep_reward: -77.300, ep_step: 516 2023-05-30 03:35:30 - SimpleLog - INFO: - episode: 6758, ep_reward: 79.823, ep_step: 1000 2023-05-30 03:35:32 - SimpleLog - INFO: - episode: 6759, ep_reward: 76.178, ep_step: 1000 2023-05-30 03:35:35 - SimpleLog - INFO: - episode: 6760, ep_reward: 78.525, ep_step: 1000 2023-05-30 03:35:37 - SimpleLog - INFO: - episode: 6761, ep_reward: 117.650, ep_step: 1000 2023-05-30 03:35:39 - SimpleLog - INFO: - episode: 6762, ep_reward: 82.314, ep_step: 1000 2023-05-30 03:35:41 - SimpleLog - INFO: - episode: 6763, ep_reward: -23.641, ep_step: 1000 2023-05-30 03:35:43 - SimpleLog - INFO: - episode: 6764, ep_reward: -31.327, ep_step: 626 2023-05-30 03:35:45 - SimpleLog - INFO: - episode: 6765, ep_reward: 131.980, ep_step: 1000 2023-05-30 03:35:47 - SimpleLog - INFO: - episode: 6766, ep_reward: 111.837, ep_step: 1000 2023-05-30 03:35:49 - SimpleLog - INFO: - episode: 6767, ep_reward: -46.070, ep_step: 939 2023-05-30 03:35:52 - SimpleLog - INFO: - episode: 6768, ep_reward: 113.411, ep_step: 1000 2023-05-30 03:35:54 - SimpleLog - INFO: - episode: 6769, ep_reward: -41.794, ep_step: 976 2023-05-30 03:35:56 - SimpleLog - INFO: - episode: 6770, ep_reward: 100.441, ep_step: 1000 2023-05-30 03:35:58 - SimpleLog - INFO: - episode: 6771, ep_reward: 79.451, ep_step: 1000 2023-05-30 03:36:01 - SimpleLog - INFO: - episode: 6772, ep_reward: 80.338, ep_step: 1000 2023-05-30 03:36:03 - SimpleLog - INFO: - episode: 6773, ep_reward: 79.312, ep_step: 1000 2023-05-30 03:36:05 - SimpleLog - INFO: - episode: 6774, ep_reward: 117.925, ep_step: 1000 2023-05-30 03:36:06 - SimpleLog - INFO: - episode: 6775, ep_reward: -78.645, ep_step: 495 2023-05-30 03:36:08 - SimpleLog - INFO: - episode: 6776, ep_reward: 51.606, ep_step: 1000 2023-05-30 03:36:11 - SimpleLog - INFO: - episode: 6777, ep_reward: 80.398, ep_step: 1000 2023-05-30 03:36:13 - SimpleLog - INFO: - episode: 6778, ep_reward: 76.860, ep_step: 1000 2023-05-30 03:36:14 - SimpleLog - INFO: - episode: 6779, ep_reward: -52.830, ep_step: 579 2023-05-30 03:36:16 - SimpleLog - INFO: - episode: 6780, ep_reward: 4.786, ep_step: 909 2023-05-30 03:36:18 - SimpleLog - INFO: - episode: 6781, ep_reward: 133.301, ep_step: 1000 2023-05-30 03:36:21 - SimpleLog - INFO: - episode: 6782, ep_reward: 140.192, ep_step: 1000 2023-05-30 03:36:23 - SimpleLog - INFO: - episode: 6783, ep_reward: 118.315, ep_step: 1000 2023-05-30 03:36:25 - SimpleLog - INFO: - episode: 6784, ep_reward: 5.921, ep_step: 790 2023-05-30 03:36:27 - SimpleLog - INFO: - episode: 6785, ep_reward: 138.223, ep_step: 1000 2023-05-30 03:36:29 - SimpleLog - INFO: - episode: 6786, ep_reward: 120.407, ep_step: 1000 2023-05-30 03:36:31 - SimpleLog - INFO: - episode: 6787, ep_reward: -25.924, ep_step: 652 2023-05-30 03:36:32 - SimpleLog - INFO: - episode: 6788, ep_reward: -34.978, ep_step: 744 2023-05-30 03:36:35 - SimpleLog - INFO: - episode: 6789, ep_reward: 130.986, ep_step: 1000 2023-05-30 03:36:35 - SimpleLog - INFO: - episode: 6790, ep_reward: -133.397, ep_step: 221 2023-05-30 03:36:37 - SimpleLog - INFO: - episode: 6791, ep_reward: 91.473, ep_step: 1000 2023-05-30 03:36:38 - SimpleLog - INFO: - episode: 6792, ep_reward: -129.845, ep_step: 125 2023-05-30 03:36:40 - SimpleLog - INFO: - episode: 6793, ep_reward: 119.089, ep_step: 1000 2023-05-30 03:36:41 - SimpleLog - INFO: - episode: 6794, ep_reward: -60.826, ep_step: 467 2023-05-30 03:36:42 - SimpleLog - INFO: - episode: 6795, ep_reward: -34.167, ep_step: 534 2023-05-30 03:36:44 - SimpleLog - INFO: - episode: 6796, ep_reward: 5.688, ep_step: 773 2023-05-30 03:36:44 - SimpleLog - INFO: - episode: 6797, ep_reward: -71.175, ep_step: 237 2023-05-30 03:36:47 - SimpleLog - INFO: - episode: 6798, ep_reward: 113.277, ep_step: 1000 2023-05-30 03:36:49 - SimpleLog - INFO: - episode: 6799, ep_reward: 108.030, ep_step: 1000 2023-05-30 03:36:50 - SimpleLog - INFO: - episode: 6800, ep_reward: -74.344, ep_step: 298 2023-05-30 03:36:52 - SimpleLog - INFO: - episode: 6801, ep_reward: 156.325, ep_step: 1000 2023-05-30 03:36:54 - SimpleLog - INFO: - episode: 6802, ep_reward: 111.915, ep_step: 1000 2023-05-30 03:36:56 - SimpleLog - INFO: - episode: 6803, ep_reward: 101.863, ep_step: 1000 2023-05-30 03:36:59 - SimpleLog - INFO: - episode: 6804, ep_reward: 133.306, ep_step: 1000 2023-05-30 03:37:01 - SimpleLog - INFO: - episode: 6805, ep_reward: 154.940, ep_step: 1000 2023-05-30 03:37:03 - SimpleLog - INFO: - episode: 6806, ep_reward: 131.064, ep_step: 1000 2023-05-30 03:37:05 - SimpleLog - INFO: - episode: 6807, ep_reward: 135.388, ep_step: 1000 2023-05-30 03:37:08 - SimpleLog - INFO: - episode: 6808, ep_reward: 135.850, ep_step: 1000 2023-05-30 03:37:10 - SimpleLog - INFO: - episode: 6809, ep_reward: 133.167, ep_step: 1000 2023-05-30 03:37:12 - SimpleLog - INFO: - episode: 6810, ep_reward: 150.519, ep_step: 1000 2023-05-30 03:37:14 - SimpleLog - INFO: - episode: 6811, ep_reward: -93.692, ep_step: 1000 2023-05-30 03:37:17 - SimpleLog - INFO: - episode: 6812, ep_reward: 139.524, ep_step: 1000 2023-05-30 03:37:20 - SimpleLog - INFO: - update_step: 5460000, online_eval_reward: 146.783 2023-05-30 03:37:21 - SimpleLog - INFO: - episode: 6813, ep_reward: 5.733, ep_step: 1000 2023-05-30 03:37:23 - SimpleLog - INFO: - episode: 6814, ep_reward: 161.103, ep_step: 1000 2023-05-30 03:37:25 - SimpleLog - INFO: - episode: 6815, ep_reward: 136.345, ep_step: 1000 2023-05-30 03:37:27 - SimpleLog - INFO: - episode: 6816, ep_reward: 124.273, ep_step: 1000 2023-05-30 03:37:29 - SimpleLog - INFO: - episode: 6817, ep_reward: -96.348, ep_step: 785 2023-05-30 03:37:31 - SimpleLog - INFO: - episode: 6818, ep_reward: 108.641, ep_step: 1000 2023-05-30 03:37:34 - SimpleLog - INFO: - episode: 6819, ep_reward: 134.599, ep_step: 1000 2023-05-30 03:37:36 - SimpleLog - INFO: - episode: 6820, ep_reward: 135.175, ep_step: 1000 2023-05-30 03:37:38 - SimpleLog - INFO: - episode: 6821, ep_reward: 10.940, ep_step: 1000 2023-05-30 03:37:39 - SimpleLog - INFO: - episode: 6822, ep_reward: -88.874, ep_step: 169 2023-05-30 03:37:39 - SimpleLog - INFO: - episode: 6823, ep_reward: -112.525, ep_step: 166 2023-05-30 03:37:39 - SimpleLog - INFO: - episode: 6824, ep_reward: -90.877, ep_step: 173 2023-05-30 03:37:40 - SimpleLog - INFO: - episode: 6825, ep_reward: -90.688, ep_step: 180 2023-05-30 03:37:40 - SimpleLog - INFO: - episode: 6826, ep_reward: -88.994, ep_step: 173 2023-05-30 03:37:41 - SimpleLog - INFO: - episode: 6827, ep_reward: -89.434, ep_step: 171 2023-05-30 03:37:41 - SimpleLog - INFO: - episode: 6828, ep_reward: -90.810, ep_step: 179 2023-05-30 03:37:41 - SimpleLog - INFO: - episode: 6829, ep_reward: -90.835, ep_step: 194 2023-05-30 03:37:42 - SimpleLog - INFO: - episode: 6830, ep_reward: -89.815, ep_step: 175 2023-05-30 03:37:42 - SimpleLog - INFO: - episode: 6831, ep_reward: -88.421, ep_step: 179 2023-05-30 03:37:43 - SimpleLog - INFO: - episode: 6832, ep_reward: -89.935, ep_step: 182 2023-05-30 03:37:43 - SimpleLog - INFO: - episode: 6833, ep_reward: -86.816, ep_step: 176 2023-05-30 03:37:43 - SimpleLog - INFO: - episode: 6834, ep_reward: -89.656, ep_step: 184 2023-05-30 03:37:44 - SimpleLog - INFO: - episode: 6835, ep_reward: -93.976, ep_step: 193 2023-05-30 03:37:44 - SimpleLog - INFO: - episode: 6836, ep_reward: -89.810, ep_step: 235 2023-05-30 03:37:45 - SimpleLog - INFO: - episode: 6837, ep_reward: -92.119, ep_step: 234 2023-05-30 03:37:45 - SimpleLog - INFO: - episode: 6838, ep_reward: -90.444, ep_step: 180 2023-05-30 03:37:46 - SimpleLog - INFO: - episode: 6839, ep_reward: -106.967, ep_step: 149 2023-05-30 03:37:46 - SimpleLog - INFO: - episode: 6840, ep_reward: -106.402, ep_step: 67 2023-05-30 03:37:46 - SimpleLog - INFO: - episode: 6841, ep_reward: -117.513, ep_step: 91 2023-05-30 03:37:46 - SimpleLog - INFO: - episode: 6842, ep_reward: -107.445, ep_step: 61 2023-05-30 03:37:46 - SimpleLog - INFO: - episode: 6843, ep_reward: -107.880, ep_step: 73 2023-05-30 03:37:47 - SimpleLog - INFO: - episode: 6844, ep_reward: -107.404, ep_step: 78 2023-05-30 03:37:47 - SimpleLog - INFO: - episode: 6845, ep_reward: -108.484, ep_step: 73 2023-05-30 03:37:47 - SimpleLog - INFO: - episode: 6846, ep_reward: -107.902, ep_step: 89 2023-05-30 03:37:47 - SimpleLog - INFO: - episode: 6847, ep_reward: -107.405, ep_step: 75 2023-05-30 03:37:47 - SimpleLog - INFO: - episode: 6848, ep_reward: -107.200, ep_step: 83 2023-05-30 03:37:47 - SimpleLog - INFO: - episode: 6849, ep_reward: -105.803, ep_step: 77 2023-05-30 03:37:48 - SimpleLog - INFO: - episode: 6850, ep_reward: -105.661, ep_step: 95 2023-05-30 03:37:48 - SimpleLog - INFO: - episode: 6851, ep_reward: -107.490, ep_step: 68 2023-05-30 03:37:48 - SimpleLog - INFO: - episode: 6852, ep_reward: -102.738, ep_step: 109 2023-05-30 03:37:48 - SimpleLog - INFO: - episode: 6853, ep_reward: -106.486, ep_step: 109 2023-05-30 03:37:49 - SimpleLog - INFO: - episode: 6854, ep_reward: -104.421, ep_step: 120 2023-05-30 03:37:49 - SimpleLog - INFO: - episode: 6855, ep_reward: -122.049, ep_step: 233 2023-05-30 03:37:50 - SimpleLog - INFO: - episode: 6856, ep_reward: -120.493, ep_step: 153 2023-05-30 03:37:50 - SimpleLog - INFO: - episode: 6857, ep_reward: -109.403, ep_step: 278 2023-05-30 03:37:52 - SimpleLog - INFO: - episode: 6858, ep_reward: -87.151, ep_step: 1000 2023-05-30 03:37:53 - SimpleLog - INFO: - episode: 6859, ep_reward: -125.172, ep_step: 123 2023-05-30 03:37:53 - SimpleLog - INFO: - episode: 6860, ep_reward: -108.255, ep_step: 266 2023-05-30 03:37:55 - SimpleLog - INFO: - episode: 6861, ep_reward: -94.482, ep_step: 1000 2023-05-30 03:37:56 - SimpleLog - INFO: - episode: 6862, ep_reward: -119.494, ep_step: 431 2023-05-30 03:37:57 - SimpleLog - INFO: - episode: 6863, ep_reward: -129.850, ep_step: 199 2023-05-30 03:37:58 - SimpleLog - INFO: - episode: 6864, ep_reward: -136.886, ep_step: 533 2023-05-30 03:37:59 - SimpleLog - INFO: - episode: 6865, ep_reward: -126.554, ep_step: 411 2023-05-30 03:38:00 - SimpleLog - INFO: - episode: 6866, ep_reward: -144.538, ep_step: 307 2023-05-30 03:38:00 - SimpleLog - INFO: - episode: 6867, ep_reward: -142.163, ep_step: 285 2023-05-30 03:38:01 - SimpleLog - INFO: - episode: 6868, ep_reward: -143.175, ep_step: 290 2023-05-30 03:38:02 - SimpleLog - INFO: - episode: 6869, ep_reward: -122.511, ep_step: 237 2023-05-30 03:38:02 - SimpleLog - INFO: - episode: 6870, ep_reward: -132.621, ep_step: 325 2023-05-30 03:38:03 - SimpleLog - INFO: - episode: 6871, ep_reward: -143.481, ep_step: 295 2023-05-30 03:38:04 - SimpleLog - INFO: - episode: 6872, ep_reward: -121.661, ep_step: 397 2023-05-30 03:38:05 - SimpleLog - INFO: - episode: 6873, ep_reward: -115.244, ep_step: 359 2023-05-30 03:38:05 - SimpleLog - INFO: - episode: 6874, ep_reward: -145.100, ep_step: 297 2023-05-30 03:38:06 - SimpleLog - INFO: - episode: 6875, ep_reward: -124.161, ep_step: 154 2023-05-30 03:38:07 - SimpleLog - INFO: - episode: 6876, ep_reward: -129.838, ep_step: 521 2023-05-30 03:38:07 - SimpleLog - INFO: - episode: 6877, ep_reward: -102.001, ep_step: 295 2023-05-30 03:38:09 - SimpleLog - INFO: - episode: 6878, ep_reward: -108.559, ep_step: 618 2023-05-30 03:38:11 - SimpleLog - INFO: - episode: 6879, ep_reward: -110.421, ep_step: 987 2023-05-30 03:38:13 - SimpleLog - INFO: - episode: 6880, ep_reward: -92.493, ep_step: 641 2023-05-30 03:38:13 - SimpleLog - INFO: - episode: 6881, ep_reward: -98.988, ep_step: 225 2023-05-30 03:38:15 - SimpleLog - INFO: - episode: 6882, ep_reward: 6.034, ep_step: 1000 2023-05-30 03:38:17 - SimpleLog - INFO: - episode: 6883, ep_reward: -52.372, ep_step: 572 2023-05-30 03:38:18 - SimpleLog - INFO: - episode: 6884, ep_reward: -47.123, ep_step: 509 2023-05-30 03:38:20 - SimpleLog - INFO: - episode: 6885, ep_reward: 79.602, ep_step: 1000 2023-05-30 03:38:22 - SimpleLog - INFO: - episode: 6886, ep_reward: 0.229, ep_step: 1000 2023-05-30 03:38:25 - SimpleLog - INFO: - episode: 6887, ep_reward: -12.496, ep_step: 1000 2023-05-30 03:38:25 - SimpleLog - INFO: - episode: 6888, ep_reward: -49.411, ep_step: 423 2023-05-30 03:38:28 - SimpleLog - INFO: - episode: 6889, ep_reward: 2.485, ep_step: 1000 2023-05-30 03:38:30 - SimpleLog - INFO: - episode: 6890, ep_reward: 3.218, ep_step: 1000 2023-05-30 03:38:32 - SimpleLog - INFO: - episode: 6891, ep_reward: -0.128, ep_step: 1000 2023-05-30 03:38:34 - SimpleLog - INFO: - episode: 6892, ep_reward: 79.340, ep_step: 1000 2023-05-30 03:38:37 - SimpleLog - INFO: - episode: 6893, ep_reward: 5.001, ep_step: 1000 2023-05-30 03:38:39 - SimpleLog - INFO: - episode: 6894, ep_reward: 129.784, ep_step: 1000 2023-05-30 03:38:41 - SimpleLog - INFO: - episode: 6895, ep_reward: 79.376, ep_step: 1000 2023-05-30 03:38:43 - SimpleLog - INFO: - episode: 6896, ep_reward: 35.262, ep_step: 1000 2023-05-30 03:38:46 - SimpleLog - INFO: - episode: 6897, ep_reward: 59.007, ep_step: 1000 2023-05-30 03:38:47 - SimpleLog - INFO: - episode: 6898, ep_reward: -6.294, ep_step: 665 2023-05-30 03:38:49 - SimpleLog - INFO: - episode: 6899, ep_reward: 77.960, ep_step: 1000 2023-05-30 03:38:52 - SimpleLog - INFO: - episode: 6900, ep_reward: -1.847, ep_step: 992 2023-05-30 03:38:54 - SimpleLog - INFO: - episode: 6901, ep_reward: 101.605, ep_step: 1000 2023-05-30 03:38:56 - SimpleLog - INFO: - episode: 6902, ep_reward: 125.300, ep_step: 1000 2023-05-30 03:38:58 - SimpleLog - INFO: - episode: 6903, ep_reward: 1.791, ep_step: 929 2023-05-30 03:39:00 - SimpleLog - INFO: - episode: 6904, ep_reward: 10.313, ep_step: 870 2023-05-30 03:39:01 - SimpleLog - INFO: - episode: 6905, ep_reward: -56.341, ep_step: 461 2023-05-30 03:39:04 - SimpleLog - INFO: - episode: 6906, ep_reward: 69.770, ep_step: 1000 2023-05-30 03:39:06 - SimpleLog - INFO: - episode: 6907, ep_reward: 80.843, ep_step: 1000 2023-05-30 03:39:08 - SimpleLog - INFO: - episode: 6908, ep_reward: 70.799, ep_step: 1000 2023-05-30 03:39:10 - SimpleLog - INFO: - episode: 6909, ep_reward: -7.281, ep_step: 789 2023-05-30 03:39:12 - SimpleLog - INFO: - episode: 6910, ep_reward: 82.779, ep_step: 1000 2023-05-30 03:39:14 - SimpleLog - INFO: - episode: 6911, ep_reward: 68.304, ep_step: 1000 2023-05-30 03:39:15 - SimpleLog - INFO: - episode: 6912, ep_reward: -99.550, ep_step: 206 2023-05-30 03:39:16 - SimpleLog - INFO: - episode: 6913, ep_reward: -49.678, ep_step: 608 2023-05-30 03:39:18 - SimpleLog - INFO: - episode: 6914, ep_reward: 108.476, ep_step: 1000 2023-05-30 03:39:21 - SimpleLog - INFO: - episode: 6915, ep_reward: 109.167, ep_step: 1000 2023-05-30 03:39:23 - SimpleLog - INFO: - episode: 6916, ep_reward: 68.996, ep_step: 1000 2023-05-30 03:39:25 - SimpleLog - INFO: - episode: 6917, ep_reward: -0.174, ep_step: 726 2023-05-30 03:39:27 - SimpleLog - INFO: - episode: 6918, ep_reward: -2.130, ep_step: 990 2023-05-30 03:39:29 - SimpleLog - INFO: - episode: 6919, ep_reward: -9.532, ep_step: 900 2023-05-30 03:39:31 - SimpleLog - INFO: - episode: 6920, ep_reward: 115.880, ep_step: 1000 2023-05-30 03:39:33 - SimpleLog - INFO: - episode: 6921, ep_reward: 80.448, ep_step: 1000 2023-05-30 03:39:35 - SimpleLog - INFO: - episode: 6922, ep_reward: -76.107, ep_step: 728 2023-05-30 03:39:37 - SimpleLog - INFO: - episode: 6923, ep_reward: 124.814, ep_step: 1000 2023-05-30 03:39:40 - SimpleLog - INFO: - episode: 6924, ep_reward: 90.902, ep_step: 1000 2023-05-30 03:39:42 - SimpleLog - INFO: - episode: 6925, ep_reward: 62.217, ep_step: 1000 2023-05-30 03:39:42 - SimpleLog - INFO: - episode: 6926, ep_reward: -101.056, ep_step: 200 2023-05-30 03:39:44 - SimpleLog - INFO: - episode: 6927, ep_reward: -3.847, ep_step: 972 2023-05-30 03:39:47 - SimpleLog - INFO: - episode: 6928, ep_reward: 82.865, ep_step: 1000 2023-05-30 03:39:49 - SimpleLog - INFO: - episode: 6929, ep_reward: 42.130, ep_step: 1000 2023-05-30 03:39:49 - SimpleLog - INFO: - episode: 6930, ep_reward: -115.794, ep_step: 235 2023-05-30 03:39:52 - SimpleLog - INFO: - episode: 6931, ep_reward: 68.005, ep_step: 1000 2023-05-30 03:39:54 - SimpleLog - INFO: - episode: 6932, ep_reward: -9.338, ep_step: 1000 2023-05-30 03:39:56 - SimpleLog - INFO: - episode: 6933, ep_reward: 20.336, ep_step: 1000 2023-05-30 03:39:57 - SimpleLog - INFO: - episode: 6934, ep_reward: -109.683, ep_step: 219 2023-05-30 03:40:00 - SimpleLog - INFO: - update_step: 5530000, online_eval_reward: 81.482 2023-05-30 03:40:01 - SimpleLog - INFO: - episode: 6935, ep_reward: 77.115, ep_step: 1000 2023-05-30 03:40:03 - SimpleLog - INFO: - episode: 6936, ep_reward: 34.274, ep_step: 1000 2023-05-30 03:40:05 - SimpleLog - INFO: - episode: 6937, ep_reward: -44.698, ep_step: 1000 2023-05-30 03:40:08 - SimpleLog - INFO: - episode: 6938, ep_reward: 84.850, ep_step: 1000 2023-05-30 03:40:10 - SimpleLog - INFO: - episode: 6939, ep_reward: -0.848, ep_step: 1000 2023-05-30 03:40:12 - SimpleLog - INFO: - episode: 6940, ep_reward: 91.275, ep_step: 1000 2023-05-30 03:40:14 - SimpleLog - INFO: - episode: 6941, ep_reward: 90.165, ep_step: 1000 2023-05-30 03:40:17 - SimpleLog - INFO: - episode: 6942, ep_reward: 101.974, ep_step: 1000 2023-05-30 03:40:19 - SimpleLog - INFO: - episode: 6943, ep_reward: 74.887, ep_step: 1000 2023-05-30 03:40:21 - SimpleLog - INFO: - episode: 6944, ep_reward: 104.921, ep_step: 1000 2023-05-30 03:40:22 - SimpleLog - INFO: - episode: 6945, ep_reward: -93.534, ep_step: 200 2023-05-30 03:40:24 - SimpleLog - INFO: - episode: 6946, ep_reward: 84.615, ep_step: 1000 2023-05-30 03:40:26 - SimpleLog - INFO: - episode: 6947, ep_reward: 112.801, ep_step: 1000 2023-05-30 03:40:28 - SimpleLog - INFO: - episode: 6948, ep_reward: 106.540, ep_step: 1000 2023-05-30 03:40:31 - SimpleLog - INFO: - episode: 6949, ep_reward: 38.716, ep_step: 1000 2023-05-30 03:40:33 - SimpleLog - INFO: - episode: 6950, ep_reward: 73.294, ep_step: 1000 2023-05-30 03:40:35 - SimpleLog - INFO: - episode: 6951, ep_reward: 77.073, ep_step: 1000 2023-05-30 03:40:37 - SimpleLog - INFO: - episode: 6952, ep_reward: 98.122, ep_step: 1000 2023-05-30 03:40:40 - SimpleLog - INFO: - episode: 6953, ep_reward: 107.147, ep_step: 1000 2023-05-30 03:40:41 - SimpleLog - INFO: - episode: 6954, ep_reward: -57.327, ep_step: 671 2023-05-30 03:40:43 - SimpleLog - INFO: - episode: 6955, ep_reward: 92.392, ep_step: 1000 2023-05-30 03:40:44 - SimpleLog - INFO: - episode: 6956, ep_reward: -113.881, ep_step: 289 2023-05-30 03:40:46 - SimpleLog - INFO: - episode: 6957, ep_reward: 93.574, ep_step: 1000 2023-05-30 03:40:49 - SimpleLog - INFO: - episode: 6958, ep_reward: 87.804, ep_step: 1000 2023-05-30 03:40:51 - SimpleLog - INFO: - episode: 6959, ep_reward: 79.261, ep_step: 1000 2023-05-30 03:40:53 - SimpleLog - INFO: - episode: 6960, ep_reward: -3.840, ep_step: 925 2023-05-30 03:40:55 - SimpleLog - INFO: - episode: 6961, ep_reward: -10.376, ep_step: 859 2023-05-30 03:40:57 - SimpleLog - INFO: - episode: 6962, ep_reward: -39.536, ep_step: 983 2023-05-30 03:40:59 - SimpleLog - INFO: - episode: 6963, ep_reward: 117.302, ep_step: 1000 2023-05-30 03:41:02 - SimpleLog - INFO: - episode: 6964, ep_reward: 57.621, ep_step: 1000 2023-05-30 03:41:04 - SimpleLog - INFO: - episode: 6965, ep_reward: -19.139, ep_step: 1000 2023-05-30 03:41:06 - SimpleLog - INFO: - episode: 6966, ep_reward: 108.334, ep_step: 1000 2023-05-30 03:41:08 - SimpleLog - INFO: - episode: 6967, ep_reward: 64.929, ep_step: 1000 2023-05-30 03:41:11 - SimpleLog - INFO: - episode: 6968, ep_reward: 95.483, ep_step: 1000 2023-05-30 03:41:13 - SimpleLog - INFO: - episode: 6969, ep_reward: -2.281, ep_step: 1000 2023-05-30 03:41:15 - SimpleLog - INFO: - episode: 6970, ep_reward: 3.915, ep_step: 1000 2023-05-30 03:41:17 - SimpleLog - INFO: - episode: 6971, ep_reward: 84.550, ep_step: 1000 2023-05-30 03:41:20 - SimpleLog - INFO: - episode: 6972, ep_reward: -0.371, ep_step: 1000 2023-05-30 03:41:22 - SimpleLog - INFO: - episode: 6973, ep_reward: 80.491, ep_step: 1000 2023-05-30 03:41:24 - SimpleLog - INFO: - episode: 6974, ep_reward: -40.991, ep_step: 1000 2023-05-30 03:41:26 - SimpleLog - INFO: - episode: 6975, ep_reward: -4.931, ep_step: 1000 2023-05-30 03:41:29 - SimpleLog - INFO: - episode: 6976, ep_reward: 6.153, ep_step: 1000 2023-05-30 03:41:31 - SimpleLog - INFO: - episode: 6977, ep_reward: -1.143, ep_step: 1000 2023-05-30 03:41:33 - SimpleLog - INFO: - episode: 6978, ep_reward: 2.807, ep_step: 1000 2023-05-30 03:41:35 - SimpleLog - INFO: - episode: 6979, ep_reward: 4.476, ep_step: 1000 2023-05-30 03:41:38 - SimpleLog - INFO: - episode: 6980, ep_reward: -0.825, ep_step: 1000 2023-05-30 03:41:40 - SimpleLog - INFO: - episode: 6981, ep_reward: 1.276, ep_step: 1000 2023-05-30 03:41:42 - SimpleLog - INFO: - episode: 6982, ep_reward: 4.795, ep_step: 1000 2023-05-30 03:41:44 - SimpleLog - INFO: - episode: 6983, ep_reward: 0.377, ep_step: 1000 2023-05-30 03:41:47 - SimpleLog - INFO: - episode: 6984, ep_reward: 11.745, ep_step: 1000 2023-05-30 03:41:48 - SimpleLog - INFO: - episode: 6985, ep_reward: -70.803, ep_step: 664 2023-05-30 03:41:50 - SimpleLog - INFO: - episode: 6986, ep_reward: 1.296, ep_step: 1000 2023-05-30 03:41:52 - SimpleLog - INFO: - episode: 6987, ep_reward: 2.922, ep_step: 1000 2023-05-30 03:41:55 - SimpleLog - INFO: - episode: 6988, ep_reward: 0.906, ep_step: 1000 2023-05-30 03:41:57 - SimpleLog - INFO: - episode: 6989, ep_reward: 10.666, ep_step: 1000 2023-05-30 03:41:59 - SimpleLog - INFO: - episode: 6990, ep_reward: 74.703, ep_step: 1000 2023-05-30 03:42:00 - SimpleLog - INFO: - episode: 6991, ep_reward: -77.538, ep_step: 341 2023-05-30 03:42:02 - SimpleLog - INFO: - episode: 6992, ep_reward: -0.617, ep_step: 1000 2023-05-30 03:42:04 - SimpleLog - INFO: - episode: 6993, ep_reward: 90.809, ep_step: 1000 2023-05-30 03:42:07 - SimpleLog - INFO: - episode: 6994, ep_reward: 74.462, ep_step: 1000 2023-05-30 03:42:09 - SimpleLog - INFO: - episode: 6995, ep_reward: 0.801, ep_step: 1000 2023-05-30 03:42:11 - SimpleLog - INFO: - episode: 6996, ep_reward: 94.633, ep_step: 1000 2023-05-30 03:42:14 - SimpleLog - INFO: - episode: 6997, ep_reward: 101.834, ep_step: 1000 2023-05-30 03:42:16 - SimpleLog - INFO: - episode: 6998, ep_reward: 64.941, ep_step: 1000 2023-05-30 03:42:18 - SimpleLog - INFO: - episode: 6999, ep_reward: 3.400, ep_step: 999 2023-05-30 03:42:20 - SimpleLog - INFO: - episode: 7000, ep_reward: 93.559, ep_step: 1000 2023-05-30 03:42:23 - SimpleLog - INFO: - episode: 7001, ep_reward: 91.914, ep_step: 1000 2023-05-30 03:42:25 - SimpleLog - INFO: - episode: 7002, ep_reward: 99.998, ep_step: 1000 2023-05-30 03:42:27 - SimpleLog - INFO: - episode: 7003, ep_reward: -3.434, ep_step: 977 2023-05-30 03:42:29 - SimpleLog - INFO: - episode: 7004, ep_reward: 4.664, ep_step: 1000 2023-05-30 03:42:31 - SimpleLog - INFO: - episode: 7005, ep_reward: 1.482, ep_step: 966 2023-05-30 03:42:34 - SimpleLog - INFO: - episode: 7006, ep_reward: 0.036, ep_step: 979 2023-05-30 03:42:36 - SimpleLog - INFO: - episode: 7007, ep_reward: 0.721, ep_step: 943 2023-05-30 03:42:39 - SimpleLog - INFO: - update_step: 5600000, online_eval_reward: -20.861 2023-05-30 03:42:40 - SimpleLog - INFO: - episode: 7008, ep_reward: -3.667, ep_step: 958 2023-05-30 03:42:42 - SimpleLog - INFO: - episode: 7009, ep_reward: 68.062, ep_step: 1000 2023-05-30 03:42:44 - SimpleLog - INFO: - episode: 7010, ep_reward: 116.446, ep_step: 1000 2023-05-30 03:42:46 - SimpleLog - INFO: - episode: 7011, ep_reward: -22.059, ep_step: 716 2023-05-30 03:42:47 - SimpleLog - INFO: - episode: 7012, ep_reward: -105.923, ep_step: 326 2023-05-30 03:42:49 - SimpleLog - INFO: - episode: 7013, ep_reward: 68.747, ep_step: 1000 2023-05-30 03:42:51 - SimpleLog - INFO: - episode: 7014, ep_reward: 146.873, ep_step: 1000 2023-05-30 03:42:53 - SimpleLog - INFO: - episode: 7015, ep_reward: -1.458, ep_step: 890 2023-05-30 03:42:55 - SimpleLog - INFO: - episode: 7016, ep_reward: 106.500, ep_step: 1000 2023-05-30 03:42:58 - SimpleLog - INFO: - episode: 7017, ep_reward: 130.631, ep_step: 1000 2023-05-30 03:42:59 - SimpleLog - INFO: - episode: 7018, ep_reward: -39.332, ep_step: 509 2023-05-30 03:43:01 - SimpleLog - INFO: - episode: 7019, ep_reward: 124.266, ep_step: 1000 2023-05-30 03:43:03 - SimpleLog - INFO: - episode: 7020, ep_reward: 69.580, ep_step: 1000 2023-05-30 03:43:05 - SimpleLog - INFO: - episode: 7021, ep_reward: 75.535, ep_step: 1000 2023-05-30 03:43:07 - SimpleLog - INFO: - episode: 7022, ep_reward: -6.414, ep_step: 851 2023-05-30 03:43:10 - SimpleLog - INFO: - episode: 7023, ep_reward: 83.536, ep_step: 1000 2023-05-30 03:43:10 - SimpleLog - INFO: - episode: 7024, ep_reward: -130.562, ep_step: 178 2023-05-30 03:43:12 - SimpleLog - INFO: - episode: 7025, ep_reward: -81.638, ep_step: 1000 2023-05-30 03:43:14 - SimpleLog - INFO: - episode: 7026, ep_reward: -27.813, ep_step: 1000 2023-05-30 03:43:15 - SimpleLog - INFO: - episode: 7027, ep_reward: -120.390, ep_step: 114 2023-05-30 03:43:17 - SimpleLog - INFO: - episode: 7028, ep_reward: 72.558, ep_step: 1000 2023-05-30 03:43:19 - SimpleLog - INFO: - episode: 7029, ep_reward: 140.083, ep_step: 1000 2023-05-30 03:43:22 - SimpleLog - INFO: - episode: 7030, ep_reward: -24.209, ep_step: 1000 2023-05-30 03:43:24 - SimpleLog - INFO: - episode: 7031, ep_reward: 131.143, ep_step: 1000 2023-05-30 03:43:26 - SimpleLog - INFO: - episode: 7032, ep_reward: 106.783, ep_step: 1000 2023-05-30 03:43:28 - SimpleLog - INFO: - episode: 7033, ep_reward: -2.943, ep_step: 942 2023-05-30 03:43:30 - SimpleLog - INFO: - episode: 7034, ep_reward: 129.283, ep_step: 1000 2023-05-30 03:43:33 - SimpleLog - INFO: - episode: 7035, ep_reward: 118.067, ep_step: 1000 2023-05-30 03:43:35 - SimpleLog - INFO: - episode: 7036, ep_reward: 107.199, ep_step: 1000 2023-05-30 03:43:36 - SimpleLog - INFO: - episode: 7037, ep_reward: -57.593, ep_step: 692 2023-05-30 03:43:38 - SimpleLog - INFO: - episode: 7038, ep_reward: -49.043, ep_step: 662 2023-05-30 03:43:39 - SimpleLog - INFO: - episode: 7039, ep_reward: -64.638, ep_step: 491 2023-05-30 03:43:40 - SimpleLog - INFO: - episode: 7040, ep_reward: -42.225, ep_step: 537 2023-05-30 03:43:41 - SimpleLog - INFO: - episode: 7041, ep_reward: -81.329, ep_step: 353 2023-05-30 03:43:42 - SimpleLog - INFO: - episode: 7042, ep_reward: -84.486, ep_step: 365 2023-05-30 03:43:42 - SimpleLog - INFO: - episode: 7043, ep_reward: -113.348, ep_step: 103 2023-05-30 03:43:42 - SimpleLog - INFO: - episode: 7044, ep_reward: -129.326, ep_step: 129 2023-05-30 03:43:44 - SimpleLog - INFO: - episode: 7045, ep_reward: 0.056, ep_step: 814 2023-05-30 03:43:47 - SimpleLog - INFO: - episode: 7046, ep_reward: -91.937, ep_step: 1000 2023-05-30 03:43:47 - SimpleLog - INFO: - episode: 7047, ep_reward: -143.373, ep_step: 317 2023-05-30 03:43:50 - SimpleLog - INFO: - episode: 7048, ep_reward: -109.187, ep_step: 1000 2023-05-30 03:43:52 - SimpleLog - INFO: - episode: 7049, ep_reward: -96.057, ep_step: 1000 2023-05-30 03:43:54 - SimpleLog - INFO: - episode: 7050, ep_reward: -79.551, ep_step: 1000 2023-05-30 03:43:54 - SimpleLog - INFO: - episode: 7051, ep_reward: -110.887, ep_step: 115 2023-05-30 03:43:57 - SimpleLog - INFO: - episode: 7052, ep_reward: -90.855, ep_step: 1000 2023-05-30 03:43:59 - SimpleLog - INFO: - episode: 7053, ep_reward: -91.647, ep_step: 1000 2023-05-30 03:44:01 - SimpleLog - INFO: - episode: 7054, ep_reward: -94.304, ep_step: 1000 2023-05-30 03:44:03 - SimpleLog - INFO: - episode: 7055, ep_reward: -92.169, ep_step: 1000 2023-05-30 03:44:04 - SimpleLog - INFO: - episode: 7056, ep_reward: -134.129, ep_step: 239 2023-05-30 03:44:06 - SimpleLog - INFO: - episode: 7057, ep_reward: -94.211, ep_step: 1000 2023-05-30 03:44:06 - SimpleLog - INFO: - episode: 7058, ep_reward: -141.573, ep_step: 140 2023-05-30 03:44:09 - SimpleLog - INFO: - episode: 7059, ep_reward: -105.567, ep_step: 1000 2023-05-30 03:44:09 - SimpleLog - INFO: - episode: 7060, ep_reward: -133.763, ep_step: 106 2023-05-30 03:44:10 - SimpleLog - INFO: - episode: 7061, ep_reward: -159.767, ep_step: 325 2023-05-30 03:44:10 - SimpleLog - INFO: - episode: 7062, ep_reward: -120.076, ep_step: 303 2023-05-30 03:44:11 - SimpleLog - INFO: - episode: 7063, ep_reward: -105.259, ep_step: 186 2023-05-30 03:44:11 - SimpleLog - INFO: - episode: 7064, ep_reward: -129.303, ep_step: 97 2023-05-30 03:44:11 - SimpleLog - INFO: - episode: 7065, ep_reward: -137.478, ep_step: 133 2023-05-30 03:44:14 - SimpleLog - INFO: - episode: 7066, ep_reward: -91.616, ep_step: 1000 2023-05-30 03:44:14 - SimpleLog - INFO: - episode: 7067, ep_reward: -147.052, ep_step: 285 2023-05-30 03:44:15 - SimpleLog - INFO: - episode: 7068, ep_reward: -120.131, ep_step: 329 2023-05-30 03:44:16 - SimpleLog - INFO: - episode: 7069, ep_reward: -162.146, ep_step: 453 2023-05-30 03:44:16 - SimpleLog - INFO: - episode: 7070, ep_reward: -100.644, ep_step: 184 2023-05-30 03:44:17 - SimpleLog - INFO: - episode: 7071, ep_reward: -104.590, ep_step: 220 2023-05-30 03:44:17 - SimpleLog - INFO: - episode: 7072, ep_reward: -103.473, ep_step: 217 2023-05-30 03:44:18 - SimpleLog - INFO: - episode: 7073, ep_reward: -105.005, ep_step: 178 2023-05-30 03:44:18 - SimpleLog - INFO: - episode: 7074, ep_reward: -104.166, ep_step: 199 2023-05-30 03:44:19 - SimpleLog - INFO: - episode: 7075, ep_reward: -105.228, ep_step: 205 2023-05-30 03:44:19 - SimpleLog - INFO: - episode: 7076, ep_reward: -112.841, ep_step: 208 2023-05-30 03:44:20 - SimpleLog - INFO: - episode: 7077, ep_reward: -106.923, ep_step: 259 2023-05-30 03:44:20 - SimpleLog - INFO: - episode: 7078, ep_reward: -105.984, ep_step: 224 2023-05-30 03:44:21 - SimpleLog - INFO: - episode: 7079, ep_reward: -104.528, ep_step: 209 2023-05-30 03:44:21 - SimpleLog - INFO: - episode: 7080, ep_reward: -104.875, ep_step: 239 2023-05-30 03:44:22 - SimpleLog - INFO: - episode: 7081, ep_reward: -103.658, ep_step: 232 2023-05-30 03:44:22 - SimpleLog - INFO: - episode: 7082, ep_reward: -111.944, ep_step: 262 2023-05-30 03:44:25 - SimpleLog - INFO: - episode: 7083, ep_reward: -87.618, ep_step: 1000 2023-05-30 03:44:27 - SimpleLog - INFO: - episode: 7084, ep_reward: -87.773, ep_step: 1000 2023-05-30 03:44:29 - SimpleLog - INFO: - episode: 7085, ep_reward: -105.897, ep_step: 1000 2023-05-30 03:44:31 - SimpleLog - INFO: - episode: 7086, ep_reward: -96.300, ep_step: 1000 2023-05-30 03:44:34 - SimpleLog - INFO: - episode: 7087, ep_reward: -96.738, ep_step: 1000 2023-05-30 03:44:36 - SimpleLog - INFO: - episode: 7088, ep_reward: -90.872, ep_step: 1000 2023-05-30 03:44:38 - SimpleLog - INFO: - episode: 7089, ep_reward: -81.290, ep_step: 1000 2023-05-30 03:44:40 - SimpleLog - INFO: - episode: 7090, ep_reward: -85.432, ep_step: 1000 2023-05-30 03:44:43 - SimpleLog - INFO: - episode: 7091, ep_reward: -82.005, ep_step: 1000 2023-05-30 03:44:45 - SimpleLog - INFO: - episode: 7092, ep_reward: -81.643, ep_step: 1000 2023-05-30 03:44:47 - SimpleLog - INFO: - episode: 7093, ep_reward: -80.501, ep_step: 1000 2023-05-30 03:44:49 - SimpleLog - INFO: - episode: 7094, ep_reward: -70.688, ep_step: 1000 2023-05-30 03:44:52 - SimpleLog - INFO: - episode: 7095, ep_reward: -81.551, ep_step: 1000 2023-05-30 03:44:54 - SimpleLog - INFO: - episode: 7096, ep_reward: -79.151, ep_step: 1000 2023-05-30 03:44:56 - SimpleLog - INFO: - episode: 7097, ep_reward: -81.614, ep_step: 1000 2023-05-30 03:44:58 - SimpleLog - INFO: - episode: 7098, ep_reward: -80.376, ep_step: 1000 2023-05-30 03:45:01 - SimpleLog - INFO: - episode: 7099, ep_reward: -78.199, ep_step: 1000 2023-05-30 03:45:03 - SimpleLog - INFO: - episode: 7100, ep_reward: -80.243, ep_step: 1000 2023-05-30 03:45:03 - SimpleLog - INFO: - episode: 7101, ep_reward: -97.443, ep_step: 211 2023-05-30 03:45:06 - SimpleLog - INFO: - episode: 7102, ep_reward: -76.664, ep_step: 1000 2023-05-30 03:45:06 - SimpleLog - INFO: - episode: 7103, ep_reward: -109.816, ep_step: 329 2023-05-30 03:45:07 - SimpleLog - INFO: - episode: 7104, ep_reward: -104.651, ep_step: 202 2023-05-30 03:45:07 - SimpleLog - INFO: - episode: 7105, ep_reward: -98.867, ep_step: 246 2023-05-30 03:45:09 - SimpleLog - INFO: - episode: 7106, ep_reward: -128.250, ep_step: 554 2023-05-30 03:45:10 - SimpleLog - INFO: - episode: 7107, ep_reward: -135.920, ep_step: 601 2023-05-30 03:45:11 - SimpleLog - INFO: - episode: 7108, ep_reward: -74.216, ep_step: 494 2023-05-30 03:45:11 - SimpleLog - INFO: - episode: 7109, ep_reward: -112.965, ep_step: 82 2023-05-30 03:45:13 - SimpleLog - INFO: - episode: 7110, ep_reward: -76.554, ep_step: 505 2023-05-30 03:45:14 - SimpleLog - INFO: - episode: 7111, ep_reward: -89.172, ep_step: 446 2023-05-30 03:45:14 - SimpleLog - INFO: - episode: 7112, ep_reward: -112.561, ep_step: 333 2023-05-30 03:45:15 - SimpleLog - INFO: - episode: 7113, ep_reward: -111.228, ep_step: 165 2023-05-30 03:45:17 - SimpleLog - INFO: - episode: 7114, ep_reward: -1.345, ep_step: 1000 2023-05-30 03:45:19 - SimpleLog - INFO: - update_step: 5670000, online_eval_reward: -9.263 2023-05-30 03:45:20 - SimpleLog - INFO: - episode: 7115, ep_reward: -82.221, ep_step: 324 2023-05-30 03:45:22 - SimpleLog - INFO: - episode: 7116, ep_reward: -4.353, ep_step: 1000 2023-05-30 03:45:24 - SimpleLog - INFO: - episode: 7117, ep_reward: -2.773, ep_step: 1000 2023-05-30 03:45:26 - SimpleLog - INFO: - episode: 7118, ep_reward: -55.483, ep_step: 705 2023-05-30 03:45:28 - SimpleLog - INFO: - episode: 7119, ep_reward: 3.614, ep_step: 1000 2023-05-30 03:45:30 - SimpleLog - INFO: - episode: 7120, ep_reward: -8.154, ep_step: 1000 2023-05-30 03:45:32 - SimpleLog - INFO: - episode: 7121, ep_reward: -5.716, ep_step: 1000 2023-05-30 03:45:33 - SimpleLog - INFO: - episode: 7122, ep_reward: -124.048, ep_step: 74 2023-05-30 03:45:35 - SimpleLog - INFO: - episode: 7123, ep_reward: 4.949, ep_step: 1000 2023-05-30 03:45:36 - SimpleLog - INFO: - episode: 7124, ep_reward: -62.227, ep_step: 728 2023-05-30 03:45:37 - SimpleLog - INFO: - episode: 7125, ep_reward: -129.320, ep_step: 105 2023-05-30 03:45:39 - SimpleLog - INFO: - episode: 7126, ep_reward: 74.655, ep_step: 1000 2023-05-30 03:45:41 - SimpleLog - INFO: - episode: 7127, ep_reward: 13.682, ep_step: 1000 2023-05-30 03:45:43 - SimpleLog - INFO: - episode: 7128, ep_reward: 72.989, ep_step: 1000 2023-05-30 03:45:45 - SimpleLog - INFO: - episode: 7129, ep_reward: -55.736, ep_step: 657 2023-05-30 03:45:47 - SimpleLog - INFO: - episode: 7130, ep_reward: 68.097, ep_step: 1000 2023-05-30 03:45:49 - SimpleLog - INFO: - episode: 7131, ep_reward: 6.642, ep_step: 1000 2023-05-30 03:45:52 - SimpleLog - INFO: - episode: 7132, ep_reward: 1.585, ep_step: 1000 2023-05-30 03:45:54 - SimpleLog - INFO: - episode: 7133, ep_reward: 79.229, ep_step: 1000 2023-05-30 03:45:56 - SimpleLog - INFO: - episode: 7134, ep_reward: 79.785, ep_step: 1000 2023-05-30 03:45:58 - SimpleLog - INFO: - episode: 7135, ep_reward: 82.781, ep_step: 1000 2023-05-30 03:46:00 - SimpleLog - INFO: - episode: 7136, ep_reward: -62.024, ep_step: 627 2023-05-30 03:46:01 - SimpleLog - INFO: - episode: 7137, ep_reward: -61.886, ep_step: 651 2023-05-30 03:46:04 - SimpleLog - INFO: - episode: 7138, ep_reward: 70.110, ep_step: 1000 2023-05-30 03:46:06 - SimpleLog - INFO: - episode: 7139, ep_reward: -48.791, ep_step: 1000 2023-05-30 03:46:08 - SimpleLog - INFO: - episode: 7140, ep_reward: -0.596, ep_step: 991 2023-05-30 03:46:10 - SimpleLog - INFO: - episode: 7141, ep_reward: 83.334, ep_step: 1000 2023-05-30 03:46:13 - SimpleLog - INFO: - episode: 7142, ep_reward: 65.532, ep_step: 1000 2023-05-30 03:46:15 - SimpleLog - INFO: - episode: 7143, ep_reward: 5.873, ep_step: 1000 2023-05-30 03:46:17 - SimpleLog - INFO: - episode: 7144, ep_reward: 73.885, ep_step: 1000 2023-05-30 03:46:19 - SimpleLog - INFO: - episode: 7145, ep_reward: -67.565, ep_step: 688 2023-05-30 03:46:21 - SimpleLog - INFO: - episode: 7146, ep_reward: 7.780, ep_step: 1000 2023-05-30 03:46:23 - SimpleLog - INFO: - episode: 7147, ep_reward: 96.757, ep_step: 1000 2023-05-30 03:46:25 - SimpleLog - INFO: - episode: 7148, ep_reward: 82.736, ep_step: 1000 2023-05-30 03:46:27 - SimpleLog - INFO: - episode: 7149, ep_reward: -76.150, ep_step: 801 2023-05-30 03:46:29 - SimpleLog - INFO: - episode: 7150, ep_reward: 28.465, ep_step: 1000 2023-05-30 03:46:32 - SimpleLog - INFO: - episode: 7151, ep_reward: 7.651, ep_step: 1000 2023-05-30 03:46:33 - SimpleLog - INFO: - episode: 7152, ep_reward: -53.021, ep_step: 525 2023-05-30 03:46:33 - SimpleLog - INFO: - episode: 7153, ep_reward: -93.816, ep_step: 227 2023-05-30 03:46:36 - SimpleLog - INFO: - episode: 7154, ep_reward: -3.210, ep_step: 1000 2023-05-30 03:46:38 - SimpleLog - INFO: - episode: 7155, ep_reward: 93.033, ep_step: 1000 2023-05-30 03:46:40 - SimpleLog - INFO: - episode: 7156, ep_reward: -2.350, ep_step: 993 2023-05-30 03:46:42 - SimpleLog - INFO: - episode: 7157, ep_reward: 95.022, ep_step: 1000 2023-05-30 03:46:44 - SimpleLog - INFO: - episode: 7158, ep_reward: -24.871, ep_step: 769 2023-05-30 03:46:45 - SimpleLog - INFO: - episode: 7159, ep_reward: -44.340, ep_step: 553 2023-05-30 03:46:48 - SimpleLog - INFO: - episode: 7160, ep_reward: 85.041, ep_step: 1000 2023-05-30 03:46:50 - SimpleLog - INFO: - episode: 7161, ep_reward: 81.709, ep_step: 1000 2023-05-30 03:46:51 - SimpleLog - INFO: - episode: 7162, ep_reward: -60.477, ep_step: 600 2023-05-30 03:46:53 - SimpleLog - INFO: - episode: 7163, ep_reward: 68.293, ep_step: 1000 2023-05-30 03:46:56 - SimpleLog - INFO: - episode: 7164, ep_reward: 8.198, ep_step: 1000 2023-05-30 03:46:58 - SimpleLog - INFO: - episode: 7165, ep_reward: -1.595, ep_step: 1000 2023-05-30 03:47:00 - SimpleLog - INFO: - episode: 7166, ep_reward: 78.393, ep_step: 1000 2023-05-30 03:47:02 - SimpleLog - INFO: - episode: 7167, ep_reward: 85.207, ep_step: 1000 2023-05-30 03:47:05 - SimpleLog - INFO: - episode: 7168, ep_reward: 66.444, ep_step: 1000 2023-05-30 03:47:07 - SimpleLog - INFO: - episode: 7169, ep_reward: 9.699, ep_step: 1000 2023-05-30 03:47:09 - SimpleLog - INFO: - episode: 7170, ep_reward: 85.390, ep_step: 1000 2023-05-30 03:47:11 - SimpleLog - INFO: - episode: 7171, ep_reward: 85.813, ep_step: 1000 2023-05-30 03:47:14 - SimpleLog - INFO: - episode: 7172, ep_reward: 71.711, ep_step: 1000 2023-05-30 03:47:16 - SimpleLog - INFO: - episode: 7173, ep_reward: 78.920, ep_step: 1000 2023-05-30 03:47:17 - SimpleLog - INFO: - episode: 7174, ep_reward: -59.275, ep_step: 584 2023-05-30 03:47:20 - SimpleLog - INFO: - episode: 7175, ep_reward: 53.868, ep_step: 1000 2023-05-30 03:47:22 - SimpleLog - INFO: - episode: 7176, ep_reward: 78.331, ep_step: 1000 2023-05-30 03:47:24 - SimpleLog - INFO: - episode: 7177, ep_reward: -35.015, ep_step: 915 2023-05-30 03:47:26 - SimpleLog - INFO: - episode: 7178, ep_reward: 103.419, ep_step: 1000 2023-05-30 03:47:28 - SimpleLog - INFO: - episode: 7179, ep_reward: -4.354, ep_step: 937 2023-05-30 03:47:30 - SimpleLog - INFO: - episode: 7180, ep_reward: -34.565, ep_step: 970 2023-05-30 03:47:33 - SimpleLog - INFO: - episode: 7181, ep_reward: 12.052, ep_step: 1000 2023-05-30 03:47:35 - SimpleLog - INFO: - episode: 7182, ep_reward: 86.365, ep_step: 1000 2023-05-30 03:47:37 - SimpleLog - INFO: - episode: 7183, ep_reward: 11.891, ep_step: 1000 2023-05-30 03:47:39 - SimpleLog - INFO: - episode: 7184, ep_reward: 13.532, ep_step: 1000 2023-05-30 03:47:41 - SimpleLog - INFO: - episode: 7185, ep_reward: -78.085, ep_step: 539 2023-05-30 03:47:43 - SimpleLog - INFO: - episode: 7186, ep_reward: 11.473, ep_step: 1000 2023-05-30 03:47:45 - SimpleLog - INFO: - episode: 7187, ep_reward: 107.219, ep_step: 1000 2023-05-30 03:47:47 - SimpleLog - INFO: - episode: 7188, ep_reward: 86.831, ep_step: 1000 2023-05-30 03:47:50 - SimpleLog - INFO: - episode: 7189, ep_reward: 97.207, ep_step: 1000 2023-05-30 03:47:52 - SimpleLog - INFO: - episode: 7190, ep_reward: 104.537, ep_step: 1000 2023-05-30 03:47:54 - SimpleLog - INFO: - episode: 7191, ep_reward: 92.504, ep_step: 1000 2023-05-30 03:47:55 - SimpleLog - INFO: - episode: 7192, ep_reward: -85.568, ep_step: 269 2023-05-30 03:47:58 - SimpleLog - INFO: - update_step: 5740000, online_eval_reward: -19.634 2023-05-30 03:47:59 - SimpleLog - INFO: - episode: 7193, ep_reward: 99.774, ep_step: 1000 2023-05-30 03:48:01 - SimpleLog - INFO: - episode: 7194, ep_reward: 108.830, ep_step: 1000 2023-05-30 03:48:03 - SimpleLog - INFO: - episode: 7195, ep_reward: -48.854, ep_step: 1000 2023-05-30 03:48:06 - SimpleLog - INFO: - episode: 7196, ep_reward: 20.891, ep_step: 1000 2023-05-30 03:48:07 - SimpleLog - INFO: - episode: 7197, ep_reward: -45.132, ep_step: 743 2023-05-30 03:48:10 - SimpleLog - INFO: - episode: 7198, ep_reward: 94.409, ep_step: 1000 2023-05-30 03:48:10 - SimpleLog - INFO: - episode: 7199, ep_reward: -88.795, ep_step: 149 2023-05-30 03:48:12 - SimpleLog - INFO: - episode: 7200, ep_reward: 85.746, ep_step: 1000 2023-05-30 03:48:14 - SimpleLog - INFO: - episode: 7201, ep_reward: -41.085, ep_step: 1000 2023-05-30 03:48:17 - SimpleLog - INFO: - episode: 7202, ep_reward: -49.725, ep_step: 1000 2023-05-30 03:48:18 - SimpleLog - INFO: - episode: 7203, ep_reward: -53.742, ep_step: 560 2023-05-30 03:48:20 - SimpleLog - INFO: - episode: 7204, ep_reward: 84.527, ep_step: 1000 2023-05-30 03:48:22 - SimpleLog - INFO: - episode: 7205, ep_reward: -7.587, ep_step: 923 2023-05-30 03:48:24 - SimpleLog - INFO: - episode: 7206, ep_reward: 94.335, ep_step: 1000 2023-05-30 03:48:26 - SimpleLog - INFO: - episode: 7207, ep_reward: -50.513, ep_step: 619 2023-05-30 03:48:28 - SimpleLog - INFO: - episode: 7208, ep_reward: 106.256, ep_step: 1000 2023-05-30 03:48:30 - SimpleLog - INFO: - episode: 7209, ep_reward: 108.210, ep_step: 1000 2023-05-30 03:48:33 - SimpleLog - INFO: - episode: 7210, ep_reward: 115.130, ep_step: 1000 2023-05-30 03:48:35 - SimpleLog - INFO: - episode: 7211, ep_reward: 1.610, ep_step: 886 2023-05-30 03:48:37 - SimpleLog - INFO: - episode: 7212, ep_reward: 112.621, ep_step: 1000 2023-05-30 03:48:39 - SimpleLog - INFO: - episode: 7213, ep_reward: 107.623, ep_step: 1000 2023-05-30 03:48:41 - SimpleLog - INFO: - episode: 7214, ep_reward: 100.360, ep_step: 1000 2023-05-30 03:48:44 - SimpleLog - INFO: - episode: 7215, ep_reward: 108.997, ep_step: 1000 2023-05-30 03:48:46 - SimpleLog - INFO: - episode: 7216, ep_reward: 96.668, ep_step: 1000 2023-05-30 03:48:48 - SimpleLog - INFO: - episode: 7217, ep_reward: 0.068, ep_step: 917 2023-05-30 03:48:50 - SimpleLog - INFO: - episode: 7218, ep_reward: 95.852, ep_step: 1000 2023-05-30 03:48:52 - SimpleLog - INFO: - episode: 7219, ep_reward: 8.756, ep_step: 1000 2023-05-30 03:48:55 - SimpleLog - INFO: - episode: 7220, ep_reward: 98.674, ep_step: 1000 2023-05-30 03:48:57 - SimpleLog - INFO: - episode: 7221, ep_reward: 5.272, ep_step: 1000 2023-05-30 03:48:59 - SimpleLog - INFO: - episode: 7222, ep_reward: 82.838, ep_step: 1000 2023-05-30 03:49:01 - SimpleLog - INFO: - episode: 7223, ep_reward: 101.689, ep_step: 1000 2023-05-30 03:49:04 - SimpleLog - INFO: - episode: 7224, ep_reward: -48.300, ep_step: 1000 2023-05-30 03:49:06 - SimpleLog - INFO: - episode: 7225, ep_reward: 108.545, ep_step: 1000 2023-05-30 03:49:08 - SimpleLog - INFO: - episode: 7226, ep_reward: 106.597, ep_step: 1000 2023-05-30 03:49:10 - SimpleLog - INFO: - episode: 7227, ep_reward: 102.132, ep_step: 1000 2023-05-30 03:49:13 - SimpleLog - INFO: - episode: 7228, ep_reward: 108.145, ep_step: 1000 2023-05-30 03:49:13 - SimpleLog - INFO: - episode: 7229, ep_reward: -106.828, ep_step: 61 2023-05-30 03:49:13 - SimpleLog - INFO: - episode: 7230, ep_reward: -108.446, ep_step: 63 2023-05-30 03:49:15 - SimpleLog - INFO: - episode: 7231, ep_reward: -63.014, ep_step: 1000 2023-05-30 03:49:17 - SimpleLog - INFO: - episode: 7232, ep_reward: -58.259, ep_step: 1000 2023-05-30 03:49:19 - SimpleLog - INFO: - episode: 7233, ep_reward: 4.075, ep_step: 865 2023-05-30 03:49:22 - SimpleLog - INFO: - episode: 7234, ep_reward: 106.542, ep_step: 1000 2023-05-30 03:49:24 - SimpleLog - INFO: - episode: 7235, ep_reward: 6.577, ep_step: 904 2023-05-30 03:49:25 - SimpleLog - INFO: - episode: 7236, ep_reward: 12.239, ep_step: 808 2023-05-30 03:49:26 - SimpleLog - INFO: - episode: 7237, ep_reward: -95.951, ep_step: 148 2023-05-30 03:49:28 - SimpleLog - INFO: - episode: 7238, ep_reward: 103.699, ep_step: 1000 2023-05-30 03:49:30 - SimpleLog - INFO: - episode: 7239, ep_reward: 116.395, ep_step: 1000 2023-05-30 03:49:33 - SimpleLog - INFO: - episode: 7240, ep_reward: 96.830, ep_step: 1000 2023-05-30 03:49:34 - SimpleLog - INFO: - episode: 7241, ep_reward: -62.277, ep_step: 407 2023-05-30 03:49:35 - SimpleLog - INFO: - episode: 7242, ep_reward: -40.782, ep_step: 559 2023-05-30 03:49:36 - SimpleLog - INFO: - episode: 7243, ep_reward: -34.838, ep_step: 591 2023-05-30 03:49:38 - SimpleLog - INFO: - episode: 7244, ep_reward: -83.241, ep_step: 638 2023-05-30 03:49:40 - SimpleLog - INFO: - episode: 7245, ep_reward: 110.576, ep_step: 1000 2023-05-30 03:49:42 - SimpleLog - INFO: - episode: 7246, ep_reward: 17.844, ep_step: 948 2023-05-30 03:49:43 - SimpleLog - INFO: - episode: 7247, ep_reward: -48.580, ep_step: 588 2023-05-30 03:49:46 - SimpleLog - INFO: - episode: 7248, ep_reward: 72.388, ep_step: 1000 2023-05-30 03:49:48 - SimpleLog - INFO: - episode: 7249, ep_reward: 110.381, ep_step: 1000 2023-05-30 03:49:50 - SimpleLog - INFO: - episode: 7250, ep_reward: 7.183, ep_step: 922 2023-05-30 03:49:52 - SimpleLog - INFO: - episode: 7251, ep_reward: 117.884, ep_step: 1000 2023-05-30 03:49:54 - SimpleLog - INFO: - episode: 7252, ep_reward: 98.257, ep_step: 1000 2023-05-30 03:49:57 - SimpleLog - INFO: - episode: 7253, ep_reward: 99.741, ep_step: 1000 2023-05-30 03:49:59 - SimpleLog - INFO: - episode: 7254, ep_reward: 83.551, ep_step: 1000 2023-05-30 03:50:01 - SimpleLog - INFO: - episode: 7255, ep_reward: 95.274, ep_step: 1000 2023-05-30 03:50:03 - SimpleLog - INFO: - episode: 7256, ep_reward: 99.970, ep_step: 1000 2023-05-30 03:50:06 - SimpleLog - INFO: - episode: 7257, ep_reward: 102.105, ep_step: 1000 2023-05-30 03:50:08 - SimpleLog - INFO: - episode: 7258, ep_reward: 109.153, ep_step: 1000 2023-05-30 03:50:10 - SimpleLog - INFO: - episode: 7259, ep_reward: 71.976, ep_step: 1000 2023-05-30 03:50:12 - SimpleLog - INFO: - episode: 7260, ep_reward: 72.902, ep_step: 1000 2023-05-30 03:50:15 - SimpleLog - INFO: - episode: 7261, ep_reward: 86.196, ep_step: 1000 2023-05-30 03:50:17 - SimpleLog - INFO: - episode: 7262, ep_reward: 96.769, ep_step: 1000 2023-05-30 03:50:19 - SimpleLog - INFO: - episode: 7263, ep_reward: 87.890, ep_step: 1000 2023-05-30 03:50:21 - SimpleLog - INFO: - episode: 7264, ep_reward: 110.489, ep_step: 1000 2023-05-30 03:50:24 - SimpleLog - INFO: - episode: 7265, ep_reward: 105.436, ep_step: 1000 2023-05-30 03:50:26 - SimpleLog - INFO: - episode: 7266, ep_reward: -6.188, ep_step: 998 2023-05-30 03:50:28 - SimpleLog - INFO: - episode: 7267, ep_reward: 106.162, ep_step: 1000 2023-05-30 03:50:30 - SimpleLog - INFO: - episode: 7268, ep_reward: 104.049, ep_step: 1000 2023-05-30 03:50:32 - SimpleLog - INFO: - episode: 7269, ep_reward: 89.258, ep_step: 1000 2023-05-30 03:50:35 - SimpleLog - INFO: - episode: 7270, ep_reward: 96.695, ep_step: 1000 2023-05-30 03:50:38 - SimpleLog - INFO: - update_step: 5810000, online_eval_reward: 3.532 2023-05-30 03:50:39 - SimpleLog - INFO: - episode: 7271, ep_reward: 104.908, ep_step: 1000 2023-05-30 03:50:41 - SimpleLog - INFO: - episode: 7272, ep_reward: 103.784, ep_step: 1000 2023-05-30 03:50:43 - SimpleLog - INFO: - episode: 7273, ep_reward: 89.513, ep_step: 1000 2023-05-30 03:50:46 - SimpleLog - INFO: - episode: 7274, ep_reward: 94.240, ep_step: 1000 2023-05-30 03:50:47 - SimpleLog - INFO: - episode: 7275, ep_reward: -77.192, ep_step: 563 2023-05-30 03:50:49 - SimpleLog - INFO: - episode: 7276, ep_reward: -52.343, ep_step: 1000 2023-05-30 03:50:51 - SimpleLog - INFO: - episode: 7277, ep_reward: 91.077, ep_step: 1000 2023-05-30 03:50:54 - SimpleLog - INFO: - episode: 7278, ep_reward: 96.651, ep_step: 1000 2023-05-30 03:50:56 - SimpleLog - INFO: - episode: 7279, ep_reward: 8.192, ep_step: 898 2023-05-30 03:50:57 - SimpleLog - INFO: - episode: 7280, ep_reward: -54.614, ep_step: 749 2023-05-30 03:51:00 - SimpleLog - INFO: - episode: 7281, ep_reward: -50.748, ep_step: 1000 2023-05-30 03:51:02 - SimpleLog - INFO: - episode: 7282, ep_reward: 98.837, ep_step: 1000 2023-05-30 03:51:04 - SimpleLog - INFO: - episode: 7283, ep_reward: 106.300, ep_step: 1000 2023-05-30 03:51:06 - SimpleLog - INFO: - episode: 7284, ep_reward: 101.691, ep_step: 1000 2023-05-30 03:51:08 - SimpleLog - INFO: - episode: 7285, ep_reward: -23.200, ep_step: 773 2023-05-30 03:51:10 - SimpleLog - INFO: - episode: 7286, ep_reward: -8.215, ep_step: 773 2023-05-30 03:51:12 - SimpleLog - INFO: - episode: 7287, ep_reward: 85.975, ep_step: 1000 2023-05-30 03:51:14 - SimpleLog - INFO: - episode: 7288, ep_reward: 7.688, ep_step: 897 2023-05-30 03:51:16 - SimpleLog - INFO: - episode: 7289, ep_reward: 81.384, ep_step: 1000 2023-05-30 03:51:19 - SimpleLog - INFO: - episode: 7290, ep_reward: -44.747, ep_step: 1000 2023-05-30 03:51:21 - SimpleLog - INFO: - episode: 7291, ep_reward: -36.379, ep_step: 1000 2023-05-30 03:51:23 - SimpleLog - INFO: - episode: 7292, ep_reward: 109.271, ep_step: 1000 2023-05-30 03:51:24 - SimpleLog - INFO: - episode: 7293, ep_reward: -75.076, ep_step: 499 2023-05-30 03:51:26 - SimpleLog - INFO: - episode: 7294, ep_reward: 71.544, ep_step: 1000 2023-05-30 03:51:29 - SimpleLog - INFO: - episode: 7295, ep_reward: 110.569, ep_step: 1000 2023-05-30 03:51:31 - SimpleLog - INFO: - episode: 7296, ep_reward: -30.018, ep_step: 1000 2023-05-30 03:51:32 - SimpleLog - INFO: - episode: 7297, ep_reward: -46.605, ep_step: 581 2023-05-30 03:51:34 - SimpleLog - INFO: - episode: 7298, ep_reward: 107.453, ep_step: 1000 2023-05-30 03:51:36 - SimpleLog - INFO: - episode: 7299, ep_reward: -50.943, ep_step: 647 2023-05-30 03:51:38 - SimpleLog - INFO: - episode: 7300, ep_reward: -88.787, ep_step: 1000 2023-05-30 03:51:39 - SimpleLog - INFO: - episode: 7301, ep_reward: -115.156, ep_step: 148 2023-05-30 03:51:41 - SimpleLog - INFO: - episode: 7302, ep_reward: 107.332, ep_step: 1000 2023-05-30 03:51:43 - SimpleLog - INFO: - episode: 7303, ep_reward: 0.160, ep_step: 1000 2023-05-30 03:51:45 - SimpleLog - INFO: - episode: 7304, ep_reward: 112.629, ep_step: 1000 2023-05-30 03:51:48 - SimpleLog - INFO: - episode: 7305, ep_reward: 108.543, ep_step: 1000 2023-05-30 03:51:50 - SimpleLog - INFO: - episode: 7306, ep_reward: 75.750, ep_step: 1000 2023-05-30 03:51:50 - SimpleLog - INFO: - episode: 7307, ep_reward: -100.441, ep_step: 208 2023-05-30 03:51:52 - SimpleLog - INFO: - episode: 7308, ep_reward: -85.174, ep_step: 889 2023-05-30 03:51:54 - SimpleLog - INFO: - episode: 7309, ep_reward: -85.464, ep_step: 714 2023-05-30 03:51:55 - SimpleLog - INFO: - episode: 7310, ep_reward: -131.757, ep_step: 571 2023-05-30 03:51:55 - SimpleLog - INFO: - episode: 7311, ep_reward: -110.807, ep_step: 81 2023-05-30 03:51:56 - SimpleLog - INFO: - episode: 7312, ep_reward: -112.632, ep_step: 98 2023-05-30 03:51:56 - SimpleLog - INFO: - episode: 7313, ep_reward: -101.345, ep_step: 231 2023-05-30 03:51:56 - SimpleLog - INFO: - episode: 7314, ep_reward: -110.958, ep_step: 65 2023-05-30 03:51:57 - SimpleLog - INFO: - episode: 7315, ep_reward: -111.116, ep_step: 287 2023-05-30 03:51:57 - SimpleLog - INFO: - episode: 7316, ep_reward: -108.607, ep_step: 129 2023-05-30 03:51:59 - SimpleLog - INFO: - episode: 7317, ep_reward: -101.156, ep_step: 894 2023-05-30 03:52:01 - SimpleLog - INFO: - episode: 7318, ep_reward: -81.633, ep_step: 606 2023-05-30 03:52:02 - SimpleLog - INFO: - episode: 7319, ep_reward: -82.099, ep_step: 608 2023-05-30 03:52:02 - SimpleLog - INFO: - episode: 7320, ep_reward: -112.413, ep_step: 44 2023-05-30 03:52:02 - SimpleLog - INFO: - episode: 7321, ep_reward: -113.125, ep_step: 42 2023-05-30 03:52:02 - SimpleLog - INFO: - episode: 7322, ep_reward: -111.305, ep_step: 63 2023-05-30 03:52:02 - SimpleLog - INFO: - episode: 7323, ep_reward: -112.956, ep_step: 41 2023-05-30 03:52:05 - SimpleLog - INFO: - episode: 7324, ep_reward: 87.770, ep_step: 1000 2023-05-30 03:52:07 - SimpleLog - INFO: - episode: 7325, ep_reward: -77.048, ep_step: 1000 2023-05-30 03:52:08 - SimpleLog - INFO: - episode: 7326, ep_reward: -79.029, ep_step: 650 2023-05-30 03:52:10 - SimpleLog - INFO: - episode: 7327, ep_reward: -74.973, ep_step: 612 2023-05-30 03:52:12 - SimpleLog - INFO: - episode: 7328, ep_reward: -24.972, ep_step: 1000 2023-05-30 03:52:14 - SimpleLog - INFO: - episode: 7329, ep_reward: -51.868, ep_step: 832 2023-05-30 03:52:16 - SimpleLog - INFO: - episode: 7330, ep_reward: 83.808, ep_step: 1000 2023-05-30 03:52:18 - SimpleLog - INFO: - episode: 7331, ep_reward: 88.492, ep_step: 1000 2023-05-30 03:52:21 - SimpleLog - INFO: - episode: 7332, ep_reward: 9.282, ep_step: 1000 2023-05-30 03:52:23 - SimpleLog - INFO: - episode: 7333, ep_reward: -37.931, ep_step: 1000 2023-05-30 03:52:24 - SimpleLog - INFO: - episode: 7334, ep_reward: -70.459, ep_step: 568 2023-05-30 03:52:26 - SimpleLog - INFO: - episode: 7335, ep_reward: 109.233, ep_step: 1000 2023-05-30 03:52:28 - SimpleLog - INFO: - episode: 7336, ep_reward: -21.643, ep_step: 879 2023-05-30 03:52:31 - SimpleLog - INFO: - episode: 7337, ep_reward: 92.067, ep_step: 1000 2023-05-30 03:52:32 - SimpleLog - INFO: - episode: 7338, ep_reward: -38.915, ep_step: 782 2023-05-30 03:52:35 - SimpleLog - INFO: - episode: 7339, ep_reward: 96.507, ep_step: 1000 2023-05-30 03:52:37 - SimpleLog - INFO: - episode: 7340, ep_reward: 88.312, ep_step: 1000 2023-05-30 03:52:38 - SimpleLog - INFO: - episode: 7341, ep_reward: -73.708, ep_step: 539 2023-05-30 03:52:40 - SimpleLog - INFO: - episode: 7342, ep_reward: 98.695, ep_step: 1000 2023-05-30 03:52:43 - SimpleLog - INFO: - episode: 7343, ep_reward: 82.181, ep_step: 1000 2023-05-30 03:52:45 - SimpleLog - INFO: - episode: 7344, ep_reward: 81.758, ep_step: 1000 2023-05-30 03:52:46 - SimpleLog - INFO: - episode: 7345, ep_reward: -50.252, ep_step: 690 2023-05-30 03:52:49 - SimpleLog - INFO: - episode: 7346, ep_reward: 53.610, ep_step: 1000 2023-05-30 03:52:51 - SimpleLog - INFO: - episode: 7347, ep_reward: 59.497, ep_step: 1000 2023-05-30 03:52:53 - SimpleLog - INFO: - episode: 7348, ep_reward: 7.005, ep_step: 856 2023-05-30 03:52:55 - SimpleLog - INFO: - episode: 7349, ep_reward: 106.938, ep_step: 1000 2023-05-30 03:52:57 - SimpleLog - INFO: - episode: 7350, ep_reward: 104.022, ep_step: 1000 2023-05-30 03:53:00 - SimpleLog - INFO: - episode: 7351, ep_reward: 82.441, ep_step: 1000 2023-05-30 03:53:01 - SimpleLog - INFO: - episode: 7352, ep_reward: -3.764, ep_step: 792 2023-05-30 03:53:04 - SimpleLog - INFO: - episode: 7353, ep_reward: 105.324, ep_step: 1000 2023-05-30 03:53:06 - SimpleLog - INFO: - episode: 7354, ep_reward: 99.191, ep_step: 1000 2023-05-30 03:53:08 - SimpleLog - INFO: - episode: 7355, ep_reward: 79.553, ep_step: 1000 2023-05-30 03:53:10 - SimpleLog - INFO: - episode: 7356, ep_reward: 93.276, ep_step: 1000 2023-05-30 03:53:13 - SimpleLog - INFO: - episode: 7357, ep_reward: 64.755, ep_step: 1000 2023-05-30 03:53:14 - SimpleLog - INFO: - episode: 7358, ep_reward: -0.228, ep_step: 788 2023-05-30 03:53:17 - SimpleLog - INFO: - update_step: 5880000, online_eval_reward: 26.911 2023-05-30 03:53:17 - SimpleLog - INFO: - episode: 7359, ep_reward: -57.307, ep_step: 546 2023-05-30 03:53:20 - SimpleLog - INFO: - episode: 7360, ep_reward: 111.280, ep_step: 1000 2023-05-30 03:53:22 - SimpleLog - INFO: - episode: 7361, ep_reward: 98.534, ep_step: 1000 2023-05-30 03:53:24 - SimpleLog - INFO: - episode: 7362, ep_reward: -17.617, ep_step: 846 2023-05-30 03:53:26 - SimpleLog - INFO: - episode: 7363, ep_reward: 96.055, ep_step: 1000 2023-05-30 03:53:28 - SimpleLog - INFO: - episode: 7364, ep_reward: 76.713, ep_step: 1000 2023-05-30 03:53:31 - SimpleLog - INFO: - episode: 7365, ep_reward: 121.504, ep_step: 1000 2023-05-30 03:53:33 - SimpleLog - INFO: - episode: 7366, ep_reward: 0.354, ep_step: 925 2023-05-30 03:53:34 - SimpleLog - INFO: - episode: 7367, ep_reward: -14.668, ep_step: 708 2023-05-30 03:53:37 - SimpleLog - INFO: - episode: 7368, ep_reward: 109.416, ep_step: 1000 2023-05-30 03:53:39 - SimpleLog - INFO: - episode: 7369, ep_reward: 123.680, ep_step: 1000 2023-05-30 03:53:41 - SimpleLog - INFO: - episode: 7370, ep_reward: 89.434, ep_step: 1000 2023-05-30 03:53:43 - SimpleLog - INFO: - episode: 7371, ep_reward: 81.539, ep_step: 1000 2023-05-30 03:53:46 - SimpleLog - INFO: - episode: 7372, ep_reward: 111.691, ep_step: 1000 2023-05-30 03:53:48 - SimpleLog - INFO: - episode: 7373, ep_reward: 116.782, ep_step: 1000 2023-05-30 03:53:50 - SimpleLog - INFO: - episode: 7374, ep_reward: 8.060, ep_step: 994 2023-05-30 03:53:52 - SimpleLog - INFO: - episode: 7375, ep_reward: 105.667, ep_step: 1000 2023-05-30 03:53:54 - SimpleLog - INFO: - episode: 7376, ep_reward: 92.752, ep_step: 1000 2023-05-30 03:53:57 - SimpleLog - INFO: - episode: 7377, ep_reward: 105.661, ep_step: 1000 2023-05-30 03:53:59 - SimpleLog - INFO: - episode: 7378, ep_reward: 108.668, ep_step: 1000 2023-05-30 03:54:01 - SimpleLog - INFO: - episode: 7379, ep_reward: 101.246, ep_step: 1000 2023-05-30 03:54:03 - SimpleLog - INFO: - episode: 7380, ep_reward: 5.063, ep_step: 978 2023-05-30 03:54:06 - SimpleLog - INFO: - episode: 7381, ep_reward: 108.231, ep_step: 1000 2023-05-30 03:54:08 - SimpleLog - INFO: - episode: 7382, ep_reward: 100.145, ep_step: 1000 2023-05-30 03:54:10 - SimpleLog - INFO: - episode: 7383, ep_reward: 110.090, ep_step: 1000 2023-05-30 03:54:12 - SimpleLog - INFO: - episode: 7384, ep_reward: 106.183, ep_step: 1000 2023-05-30 03:54:15 - SimpleLog - INFO: - episode: 7385, ep_reward: 113.193, ep_step: 1000 2023-05-30 03:54:17 - SimpleLog - INFO: - episode: 7386, ep_reward: 116.507, ep_step: 1000 2023-05-30 03:54:19 - SimpleLog - INFO: - episode: 7387, ep_reward: 103.744, ep_step: 1000 2023-05-30 03:54:21 - SimpleLog - INFO: - episode: 7388, ep_reward: 107.338, ep_step: 1000 2023-05-30 03:54:24 - SimpleLog - INFO: - episode: 7389, ep_reward: 108.614, ep_step: 1000 2023-05-30 03:54:26 - SimpleLog - INFO: - episode: 7390, ep_reward: 107.218, ep_step: 1000 2023-05-30 03:54:28 - SimpleLog - INFO: - episode: 7391, ep_reward: 0.250, ep_step: 973 2023-05-30 03:54:30 - SimpleLog - INFO: - episode: 7392, ep_reward: -0.116, ep_step: 978 2023-05-30 03:54:33 - SimpleLog - INFO: - episode: 7393, ep_reward: 107.439, ep_step: 1000 2023-05-30 03:54:35 - SimpleLog - INFO: - episode: 7394, ep_reward: 2.292, ep_step: 813 2023-05-30 03:54:37 - SimpleLog - INFO: - episode: 7395, ep_reward: 119.290, ep_step: 1000 2023-05-30 03:54:39 - SimpleLog - INFO: - episode: 7396, ep_reward: 8.313, ep_step: 1000 2023-05-30 03:54:41 - SimpleLog - INFO: - episode: 7397, ep_reward: 7.608, ep_step: 855 2023-05-30 03:54:43 - SimpleLog - INFO: - episode: 7398, ep_reward: 111.033, ep_step: 1000 2023-05-30 03:54:45 - SimpleLog - INFO: - episode: 7399, ep_reward: -0.776, ep_step: 965 2023-05-30 03:54:48 - SimpleLog - INFO: - episode: 7400, ep_reward: 107.885, ep_step: 1000 2023-05-30 03:54:50 - SimpleLog - INFO: - episode: 7401, ep_reward: 128.313, ep_step: 1000 2023-05-30 03:54:52 - SimpleLog - INFO: - episode: 7402, ep_reward: 101.862, ep_step: 1000 2023-05-30 03:54:54 - SimpleLog - INFO: - episode: 7403, ep_reward: 73.681, ep_step: 1000 2023-05-30 03:54:55 - SimpleLog - INFO: - episode: 7404, ep_reward: -82.989, ep_step: 361 2023-05-30 03:54:58 - SimpleLog - INFO: - episode: 7405, ep_reward: 108.569, ep_step: 1000 2023-05-30 03:55:00 - SimpleLog - INFO: - episode: 7406, ep_reward: -23.084, ep_step: 1000 2023-05-30 03:55:01 - SimpleLog - INFO: - episode: 7407, ep_reward: -85.903, ep_step: 280 2023-05-30 03:55:03 - SimpleLog - INFO: - episode: 7408, ep_reward: -0.316, ep_step: 945 2023-05-30 03:55:05 - SimpleLog - INFO: - episode: 7409, ep_reward: 105.723, ep_step: 1000 2023-05-30 03:55:07 - SimpleLog - INFO: - episode: 7410, ep_reward: 128.580, ep_step: 1000 2023-05-30 03:55:08 - SimpleLog - INFO: - episode: 7411, ep_reward: -96.803, ep_step: 270 2023-05-30 03:55:10 - SimpleLog - INFO: - episode: 7412, ep_reward: 17.257, ep_step: 866 2023-05-30 03:55:10 - SimpleLog - INFO: - episode: 7413, ep_reward: -123.382, ep_step: 139 2023-05-30 03:55:12 - SimpleLog - INFO: - episode: 7414, ep_reward: -3.284, ep_step: 986 2023-05-30 03:55:15 - SimpleLog - INFO: - episode: 7415, ep_reward: 103.826, ep_step: 1000 2023-05-30 03:55:16 - SimpleLog - INFO: - episode: 7416, ep_reward: -46.662, ep_step: 535 2023-05-30 03:55:18 - SimpleLog - INFO: - episode: 7417, ep_reward: -60.149, ep_step: 1000 2023-05-30 03:55:20 - SimpleLog - INFO: - episode: 7418, ep_reward: 131.943, ep_step: 1000 2023-05-30 03:55:22 - SimpleLog - INFO: - episode: 7419, ep_reward: 14.214, ep_step: 863 2023-05-30 03:55:25 - SimpleLog - INFO: - episode: 7420, ep_reward: 137.930, ep_step: 1000 2023-05-30 03:55:26 - SimpleLog - INFO: - episode: 7421, ep_reward: 3.428, ep_step: 859 2023-05-30 03:55:29 - SimpleLog - INFO: - episode: 7422, ep_reward: 84.841, ep_step: 1000 2023-05-30 03:55:30 - SimpleLog - INFO: - episode: 7423, ep_reward: -85.567, ep_step: 695 2023-05-30 03:55:32 - SimpleLog - INFO: - episode: 7424, ep_reward: -33.162, ep_step: 549 2023-05-30 03:55:34 - SimpleLog - INFO: - episode: 7425, ep_reward: 66.680, ep_step: 1000 2023-05-30 03:55:35 - SimpleLog - INFO: - episode: 7426, ep_reward: -36.389, ep_step: 550 2023-05-30 03:55:37 - SimpleLog - INFO: - episode: 7427, ep_reward: 13.945, ep_step: 917 2023-05-30 03:55:39 - SimpleLog - INFO: - episode: 7428, ep_reward: -42.225, ep_step: 678 2023-05-30 03:55:41 - SimpleLog - INFO: - episode: 7429, ep_reward: 1.173, ep_step: 903 2023-05-30 03:55:43 - SimpleLog - INFO: - episode: 7430, ep_reward: -40.095, ep_step: 935 2023-05-30 03:55:45 - SimpleLog - INFO: - episode: 7431, ep_reward: 108.255, ep_step: 1000 2023-05-30 03:55:47 - SimpleLog - INFO: - episode: 7432, ep_reward: -4.238, ep_step: 995 2023-05-30 03:55:49 - SimpleLog - INFO: - episode: 7433, ep_reward: -1.823, ep_step: 965 2023-05-30 03:55:52 - SimpleLog - INFO: - episode: 7434, ep_reward: 72.177, ep_step: 1000 2023-05-30 03:55:54 - SimpleLog - INFO: - episode: 7435, ep_reward: 107.444, ep_step: 1000 2023-05-30 03:55:57 - SimpleLog - INFO: - update_step: 5950000, online_eval_reward: 77.015 2023-05-30 03:55:58 - SimpleLog - INFO: - episode: 7436, ep_reward: 129.528, ep_step: 1000 2023-05-30 03:56:00 - SimpleLog - INFO: - episode: 7437, ep_reward: 78.253, ep_step: 1000 2023-05-30 03:56:03 - SimpleLog - INFO: - episode: 7438, ep_reward: 98.865, ep_step: 1000 2023-05-30 03:56:05 - SimpleLog - INFO: - episode: 7439, ep_reward: 104.995, ep_step: 1000 2023-05-30 03:56:07 - SimpleLog - INFO: - episode: 7440, ep_reward: 71.435, ep_step: 1000 2023-05-30 03:56:09 - SimpleLog - INFO: - episode: 7441, ep_reward: 126.914, ep_step: 1000 2023-05-30 03:56:12 - SimpleLog - INFO: - episode: 7442, ep_reward: 113.858, ep_step: 1000 2023-05-30 03:56:14 - SimpleLog - INFO: - episode: 7443, ep_reward: 114.689, ep_step: 1000 2023-05-30 03:56:16 - SimpleLog - INFO: - episode: 7444, ep_reward: -0.326, ep_step: 1000 2023-05-30 03:56:19 - SimpleLog - INFO: - episode: 7445, ep_reward: -5.325, ep_step: 1000 2023-05-30 03:56:21 - SimpleLog - INFO: - episode: 7446, ep_reward: 17.353, ep_step: 905 2023-05-30 03:56:23 - SimpleLog - INFO: - episode: 7447, ep_reward: -5.001, ep_step: 934 2023-05-30 03:56:24 - SimpleLog - INFO: - episode: 7448, ep_reward: -26.754, ep_step: 619 2023-05-30 03:56:26 - SimpleLog - INFO: - episode: 7449, ep_reward: 132.456, ep_step: 1000 2023-05-30 03:56:29 - SimpleLog - INFO: - episode: 7450, ep_reward: 115.534, ep_step: 1000 2023-05-30 03:56:30 - SimpleLog - INFO: - episode: 7451, ep_reward: -56.968, ep_step: 395 2023-05-30 03:56:32 - SimpleLog - INFO: - episode: 7452, ep_reward: 121.805, ep_step: 1000 2023-05-30 03:56:34 - SimpleLog - INFO: - episode: 7453, ep_reward: 2.230, ep_step: 964 2023-05-30 03:56:36 - SimpleLog - INFO: - episode: 7454, ep_reward: -8.417, ep_step: 1000 2023-05-30 03:56:39 - SimpleLog - INFO: - episode: 7455, ep_reward: 127.393, ep_step: 1000 2023-05-30 03:56:41 - SimpleLog - INFO: - episode: 7456, ep_reward: 103.681, ep_step: 1000 2023-05-30 03:56:43 - SimpleLog - INFO: - episode: 7457, ep_reward: -51.619, ep_step: 1000 2023-05-30 03:56:45 - SimpleLog - INFO: - episode: 7458, ep_reward: 108.140, ep_step: 1000 2023-05-30 03:56:48 - SimpleLog - INFO: - episode: 7459, ep_reward: 111.704, ep_step: 1000 2023-05-30 03:56:50 - SimpleLog - INFO: - episode: 7460, ep_reward: 123.619, ep_step: 1000 2023-05-30 03:56:52 - SimpleLog - INFO: - episode: 7461, ep_reward: 67.098, ep_step: 1000 2023-05-30 03:56:54 - SimpleLog - INFO: - episode: 7462, ep_reward: -43.077, ep_step: 711 2023-05-30 03:56:56 - SimpleLog - INFO: - episode: 7463, ep_reward: 70.791, ep_step: 1000 2023-05-30 03:56:58 - SimpleLog - INFO: - episode: 7464, ep_reward: 11.217, ep_step: 866 2023-05-30 03:56:59 - SimpleLog - INFO: - episode: 7465, ep_reward: -84.936, ep_step: 246 2023-05-30 03:57:00 - SimpleLog - INFO: - episode: 7466, ep_reward: -45.909, ep_step: 721 2023-05-30 03:57:02 - SimpleLog - INFO: - episode: 7467, ep_reward: 122.613, ep_step: 1000 2023-05-30 03:57:05 - SimpleLog - INFO: - episode: 7468, ep_reward: -2.462, ep_step: 1000 2023-05-30 03:57:07 - SimpleLog - INFO: - episode: 7469, ep_reward: 2.638, ep_step: 882 2023-05-30 03:57:09 - SimpleLog - INFO: - episode: 7470, ep_reward: 117.981, ep_step: 1000 2023-05-30 03:57:10 - SimpleLog - INFO: - episode: 7471, ep_reward: -57.035, ep_step: 487 2023-05-30 03:57:12 - SimpleLog - INFO: - episode: 7472, ep_reward: 81.968, ep_step: 1000 2023-05-30 03:57:14 - SimpleLog - INFO: - episode: 7473, ep_reward: -57.049, ep_step: 628 2023-05-30 03:57:16 - SimpleLog - INFO: - episode: 7474, ep_reward: -39.111, ep_step: 945 2023-05-30 03:57:18 - SimpleLog - INFO: - episode: 7475, ep_reward: -50.530, ep_step: 1000 2023-05-30 03:57:20 - SimpleLog - INFO: - episode: 7476, ep_reward: 70.010, ep_step: 1000 2023-05-30 03:57:22 - SimpleLog - INFO: - episode: 7477, ep_reward: -41.400, ep_step: 764 2023-05-30 03:57:23 - SimpleLog - INFO: - episode: 7478, ep_reward: -50.653, ep_step: 475 2023-05-30 03:57:25 - SimpleLog - INFO: - episode: 7479, ep_reward: -43.370, ep_step: 764 2023-05-30 03:57:27 - SimpleLog - INFO: - episode: 7480, ep_reward: 124.201, ep_step: 1000 2023-05-30 03:57:29 - SimpleLog - INFO: - episode: 7481, ep_reward: -44.356, ep_step: 800 2023-05-30 03:57:30 - SimpleLog - INFO: - episode: 7482, ep_reward: -102.082, ep_step: 243 2023-05-30 03:57:32 - SimpleLog - INFO: - episode: 7483, ep_reward: 57.979, ep_step: 1000 2023-05-30 03:57:32 - SimpleLog - INFO: - episode: 7484, ep_reward: -113.972, ep_step: 127 2023-05-30 03:57:32 - SimpleLog - INFO: - episode: 7485, ep_reward: -115.804, ep_step: 108 2023-05-30 03:57:35 - SimpleLog - INFO: - episode: 7486, ep_reward: -56.482, ep_step: 1000 2023-05-30 03:57:37 - SimpleLog - INFO: - episode: 7487, ep_reward: 71.596, ep_step: 1000 2023-05-30 03:57:38 - SimpleLog - INFO: - episode: 7488, ep_reward: -86.519, ep_step: 433 2023-05-30 03:57:39 - SimpleLog - INFO: - episode: 7489, ep_reward: -48.824, ep_step: 480 2023-05-30 03:57:41 - SimpleLog - INFO: - episode: 7490, ep_reward: 7.090, ep_step: 950 2023-05-30 03:57:43 - SimpleLog - INFO: - episode: 7491, ep_reward: -11.414, ep_step: 604 2023-05-30 03:57:44 - SimpleLog - INFO: - episode: 7492, ep_reward: -43.498, ep_step: 412 2023-05-30 03:57:45 - SimpleLog - INFO: - episode: 7493, ep_reward: -13.548, ep_step: 565 2023-05-30 03:57:47 - SimpleLog - INFO: - episode: 7494, ep_reward: 16.552, ep_step: 918 2023-05-30 03:57:49 - SimpleLog - INFO: - episode: 7495, ep_reward: -7.045, ep_step: 1000 2023-05-30 03:57:50 - SimpleLog - INFO: - episode: 7496, ep_reward: -48.121, ep_step: 442 2023-05-30 03:57:51 - SimpleLog - INFO: - episode: 7497, ep_reward: -51.263, ep_step: 485 2023-05-30 03:57:52 - SimpleLog - INFO: - episode: 7498, ep_reward: -10.544, ep_step: 571 2023-05-30 03:57:55 - SimpleLog - INFO: - episode: 7499, ep_reward: 131.572, ep_step: 1000 2023-05-30 03:57:56 - SimpleLog - INFO: - episode: 7500, ep_reward: -108.472, ep_step: 659 2023-05-30 03:57:57 - SimpleLog - INFO: - episode: 7501, ep_reward: -50.996, ep_step: 444 2023-05-30 03:57:59 - SimpleLog - INFO: - episode: 7502, ep_reward: 127.124, ep_step: 1000 2023-05-30 03:58:01 - SimpleLog - INFO: - episode: 7503, ep_reward: 0.874, ep_step: 773 2023-05-30 03:58:02 - SimpleLog - INFO: - episode: 7504, ep_reward: -54.816, ep_step: 535 2023-05-30 03:58:05 - SimpleLog - INFO: - episode: 7505, ep_reward: 110.093, ep_step: 1000 2023-05-30 03:58:07 - SimpleLog - INFO: - episode: 7506, ep_reward: 115.495, ep_step: 1000 2023-05-30 03:58:09 - SimpleLog - INFO: - episode: 7507, ep_reward: -38.161, ep_step: 1000 2023-05-30 03:58:11 - SimpleLog - INFO: - episode: 7508, ep_reward: 2.686, ep_step: 759 2023-05-30 03:58:12 - SimpleLog - INFO: - episode: 7509, ep_reward: -74.068, ep_step: 294 2023-05-30 03:58:14 - SimpleLog - INFO: - episode: 7510, ep_reward: -2.548, ep_step: 871 2023-05-30 03:58:16 - SimpleLog - INFO: - episode: 7511, ep_reward: 118.622, ep_step: 1000 2023-05-30 03:58:17 - SimpleLog - INFO: - episode: 7512, ep_reward: -46.605, ep_step: 604 2023-05-30 03:58:19 - SimpleLog - INFO: - episode: 7513, ep_reward: 104.223, ep_step: 1000 2023-05-30 03:58:22 - SimpleLog - INFO: - episode: 7514, ep_reward: 103.862, ep_step: 1000 2023-05-30 03:58:24 - SimpleLog - INFO: - episode: 7515, ep_reward: -42.740, ep_step: 1000 2023-05-30 03:58:26 - SimpleLog - INFO: - episode: 7516, ep_reward: -59.423, ep_step: 1000 2023-05-30 03:58:28 - SimpleLog - INFO: - episode: 7517, ep_reward: -37.799, ep_step: 1000 2023-05-30 03:58:31 - SimpleLog - INFO: - episode: 7518, ep_reward: -6.911, ep_step: 941 2023-05-30 03:58:33 - SimpleLog - INFO: - episode: 7519, ep_reward: -1.616, ep_step: 923 2023-05-30 03:58:35 - SimpleLog - INFO: - episode: 7520, ep_reward: 108.791, ep_step: 1000 2023-05-30 03:58:37 - SimpleLog - INFO: - update_step: 6020000, online_eval_reward: 11.527 2023-05-30 03:58:39 - SimpleLog - INFO: - episode: 7521, ep_reward: 5.360, ep_step: 848 2023-05-30 03:58:41 - SimpleLog - INFO: - episode: 7522, ep_reward: 89.883, ep_step: 1000 2023-05-30 03:58:43 - SimpleLog - INFO: - episode: 7523, ep_reward: 123.079, ep_step: 1000 2023-05-30 03:58:45 - SimpleLog - INFO: - episode: 7524, ep_reward: 102.901, ep_step: 1000 2023-05-30 03:58:46 - SimpleLog - INFO: - episode: 7525, ep_reward: -54.213, ep_step: 468 2023-05-30 03:58:49 - SimpleLog - INFO: - episode: 7526, ep_reward: 114.213, ep_step: 1000 2023-05-30 03:58:51 - SimpleLog - INFO: - episode: 7527, ep_reward: -63.121, ep_step: 1000 2023-05-30 03:58:53 - SimpleLog - INFO: - episode: 7528, ep_reward: 110.252, ep_step: 1000 2023-05-30 03:58:55 - SimpleLog - INFO: - episode: 7529, ep_reward: 100.942, ep_step: 1000 2023-05-30 03:58:57 - SimpleLog - INFO: - episode: 7530, ep_reward: -117.874, ep_step: 714 2023-05-30 03:58:59 - SimpleLog - INFO: - episode: 7531, ep_reward: 103.875, ep_step: 1000 2023-05-30 03:59:02 - SimpleLog - INFO: - episode: 7532, ep_reward: 119.556, ep_step: 1000 2023-05-30 03:59:04 - SimpleLog - INFO: - episode: 7533, ep_reward: 68.332, ep_step: 1000 2023-05-30 03:59:06 - SimpleLog - INFO: - episode: 7534, ep_reward: 101.220, ep_step: 1000 2023-05-30 03:59:08 - SimpleLog - INFO: - episode: 7535, ep_reward: 56.003, ep_step: 1000 2023-05-30 03:59:11 - SimpleLog - INFO: - episode: 7536, ep_reward: 108.326, ep_step: 1000 2023-05-30 03:59:13 - SimpleLog - INFO: - episode: 7537, ep_reward: 114.607, ep_step: 1000 2023-05-30 03:59:15 - SimpleLog - INFO: - episode: 7538, ep_reward: 15.627, ep_step: 1000 2023-05-30 03:59:17 - SimpleLog - INFO: - episode: 7539, ep_reward: 107.891, ep_step: 1000 2023-05-30 03:59:20 - SimpleLog - INFO: - episode: 7540, ep_reward: 122.880, ep_step: 1000 2023-05-30 03:59:22 - SimpleLog - INFO: - episode: 7541, ep_reward: 136.081, ep_step: 1000 2023-05-30 03:59:24 - SimpleLog - INFO: - episode: 7542, ep_reward: -30.690, ep_step: 1000 2023-05-30 03:59:26 - SimpleLog - INFO: - episode: 7543, ep_reward: 88.519, ep_step: 1000 2023-05-30 03:59:29 - SimpleLog - INFO: - episode: 7544, ep_reward: 138.758, ep_step: 1000 2023-05-30 03:59:30 - SimpleLog - INFO: - episode: 7545, ep_reward: -34.933, ep_step: 617 2023-05-30 03:59:32 - SimpleLog - INFO: - episode: 7546, ep_reward: 104.771, ep_step: 1000 2023-05-30 03:59:35 - SimpleLog - INFO: - episode: 7547, ep_reward: 97.066, ep_step: 1000 2023-05-30 03:59:37 - SimpleLog - INFO: - episode: 7548, ep_reward: 71.583, ep_step: 1000 2023-05-30 03:59:38 - SimpleLog - INFO: - episode: 7549, ep_reward: -38.234, ep_step: 570 2023-05-30 03:59:39 - SimpleLog - INFO: - episode: 7550, ep_reward: -102.442, ep_step: 214 2023-05-30 03:59:41 - SimpleLog - INFO: - episode: 7551, ep_reward: 81.474, ep_step: 1000 2023-05-30 03:59:43 - SimpleLog - INFO: - episode: 7552, ep_reward: 78.346, ep_step: 1000 2023-05-30 03:59:45 - SimpleLog - INFO: - episode: 7553, ep_reward: -95.035, ep_step: 766 2023-05-30 03:59:47 - SimpleLog - INFO: - episode: 7554, ep_reward: 1.429, ep_step: 1000 2023-05-30 03:59:49 - SimpleLog - INFO: - episode: 7555, ep_reward: -0.340, ep_step: 1000 2023-05-30 03:59:50 - SimpleLog - INFO: - episode: 7556, ep_reward: -78.246, ep_step: 383 2023-05-30 03:59:51 - SimpleLog - INFO: - episode: 7557, ep_reward: -72.170, ep_step: 266 2023-05-30 03:59:52 - SimpleLog - INFO: - episode: 7558, ep_reward: -78.038, ep_step: 333 2023-05-30 03:59:52 - SimpleLog - INFO: - episode: 7559, ep_reward: -69.516, ep_step: 289 2023-05-30 03:59:54 - SimpleLog - INFO: - episode: 7560, ep_reward: -67.877, ep_step: 624 2023-05-30 03:59:54 - SimpleLog - INFO: - episode: 7561, ep_reward: -65.380, ep_step: 323 2023-05-30 03:59:55 - SimpleLog - INFO: - episode: 7562, ep_reward: -67.790, ep_step: 299 2023-05-30 03:59:56 - SimpleLog - INFO: - episode: 7563, ep_reward: -76.969, ep_step: 611 2023-05-30 03:59:59 - SimpleLog - INFO: - episode: 7564, ep_reward: 0.309, ep_step: 1000 2023-05-30 04:00:01 - SimpleLog - INFO: - episode: 7565, ep_reward: 3.616, ep_step: 1000 2023-05-30 04:00:02 - SimpleLog - INFO: - episode: 7566, ep_reward: -75.449, ep_step: 297 2023-05-30 04:00:02 - SimpleLog - INFO: - episode: 7567, ep_reward: -83.302, ep_step: 209 2023-05-30 04:00:04 - SimpleLog - INFO: - episode: 7568, ep_reward: 2.866, ep_step: 1000 2023-05-30 04:00:05 - SimpleLog - INFO: - episode: 7569, ep_reward: -47.703, ep_step: 473 2023-05-30 04:00:08 - SimpleLog - INFO: - episode: 7570, ep_reward: 5.103, ep_step: 1000 2023-05-30 04:00:10 - SimpleLog - INFO: - episode: 7571, ep_reward: 6.809, ep_step: 1000 2023-05-30 04:00:11 - SimpleLog - INFO: - episode: 7572, ep_reward: -114.132, ep_step: 401 2023-05-30 04:00:13 - SimpleLog - INFO: - episode: 7573, ep_reward: 98.861, ep_step: 1000 2023-05-30 04:00:15 - SimpleLog - INFO: - episode: 7574, ep_reward: 90.761, ep_step: 1000 2023-05-30 04:00:18 - SimpleLog - INFO: - episode: 7575, ep_reward: 98.989, ep_step: 1000 2023-05-30 04:00:20 - SimpleLog - INFO: - episode: 7576, ep_reward: 90.313, ep_step: 1000 2023-05-30 04:00:22 - SimpleLog - INFO: - episode: 7577, ep_reward: 82.486, ep_step: 1000 2023-05-30 04:00:24 - SimpleLog - INFO: - episode: 7578, ep_reward: 93.961, ep_step: 1000 2023-05-30 04:00:27 - SimpleLog - INFO: - episode: 7579, ep_reward: 109.821, ep_step: 1000 2023-05-30 04:00:29 - SimpleLog - INFO: - episode: 7580, ep_reward: 102.835, ep_step: 1000 2023-05-30 04:00:31 - SimpleLog - INFO: - episode: 7581, ep_reward: 86.912, ep_step: 1000 2023-05-30 04:00:33 - SimpleLog - INFO: - episode: 7582, ep_reward: 107.804, ep_step: 1000 2023-05-30 04:00:36 - SimpleLog - INFO: - episode: 7583, ep_reward: 78.548, ep_step: 1000 2023-05-30 04:00:38 - SimpleLog - INFO: - episode: 7584, ep_reward: 106.632, ep_step: 1000 2023-05-30 04:00:40 - SimpleLog - INFO: - episode: 7585, ep_reward: 125.760, ep_step: 1000 2023-05-30 04:00:42 - SimpleLog - INFO: - episode: 7586, ep_reward: 135.022, ep_step: 1000 2023-05-30 04:00:45 - SimpleLog - INFO: - episode: 7587, ep_reward: 4.415, ep_step: 905 2023-05-30 04:00:47 - SimpleLog - INFO: - episode: 7588, ep_reward: 123.737, ep_step: 1000 2023-05-30 04:00:49 - SimpleLog - INFO: - episode: 7589, ep_reward: 121.105, ep_step: 1000 2023-05-30 04:00:51 - SimpleLog - INFO: - episode: 7590, ep_reward: 120.615, ep_step: 1000 2023-05-30 04:00:54 - SimpleLog - INFO: - episode: 7591, ep_reward: 111.342, ep_step: 1000 2023-05-30 04:00:56 - SimpleLog - INFO: - episode: 7592, ep_reward: 127.163, ep_step: 1000 2023-05-30 04:00:58 - SimpleLog - INFO: - episode: 7593, ep_reward: 56.544, ep_step: 1000 2023-05-30 04:01:00 - SimpleLog - INFO: - episode: 7594, ep_reward: 131.401, ep_step: 1000 2023-05-30 04:01:03 - SimpleLog - INFO: - episode: 7595, ep_reward: 109.111, ep_step: 1000 2023-05-30 04:01:05 - SimpleLog - INFO: - episode: 7596, ep_reward: 126.669, ep_step: 1000 2023-05-30 04:01:07 - SimpleLog - INFO: - episode: 7597, ep_reward: 137.035, ep_step: 1000 2023-05-30 04:01:09 - SimpleLog - INFO: - episode: 7598, ep_reward: 108.154, ep_step: 1000 2023-05-30 04:01:12 - SimpleLog - INFO: - episode: 7599, ep_reward: 78.839, ep_step: 1000 2023-05-30 04:01:14 - SimpleLog - INFO: - episode: 7600, ep_reward: 154.396, ep_step: 1000 2023-05-30 04:01:17 - SimpleLog - INFO: - update_step: 6090000, online_eval_reward: 66.330 2023-05-30 04:01:18 - SimpleLog - INFO: - episode: 7601, ep_reward: -2.460, ep_step: 1000 2023-05-30 04:01:20 - SimpleLog - INFO: - episode: 7602, ep_reward: 78.014, ep_step: 1000 2023-05-30 04:01:23 - SimpleLog - INFO: - episode: 7603, ep_reward: 74.973, ep_step: 1000 2023-05-30 04:01:24 - SimpleLog - INFO: - episode: 7604, ep_reward: -35.110, ep_step: 479 2023-05-30 04:01:25 - SimpleLog - INFO: - episode: 7605, ep_reward: -39.554, ep_step: 481 2023-05-30 04:01:27 - SimpleLog - INFO: - episode: 7606, ep_reward: 115.875, ep_step: 1000 2023-05-30 04:01:29 - SimpleLog - INFO: - episode: 7607, ep_reward: 57.646, ep_step: 1000 2023-05-30 04:01:32 - SimpleLog - INFO: - episode: 7608, ep_reward: -4.305, ep_step: 1000 2023-05-30 04:01:34 - SimpleLog - INFO: - episode: 7609, ep_reward: 1.701, ep_step: 1000 2023-05-30 04:01:36 - SimpleLog - INFO: - episode: 7610, ep_reward: 106.483, ep_step: 1000 2023-05-30 04:01:38 - SimpleLog - INFO: - episode: 7611, ep_reward: 81.024, ep_step: 1000 2023-05-30 04:01:41 - SimpleLog - INFO: - episode: 7612, ep_reward: 129.022, ep_step: 1000 2023-05-30 04:01:42 - SimpleLog - INFO: - episode: 7613, ep_reward: -63.728, ep_step: 656 2023-05-30 04:01:45 - SimpleLog - INFO: - episode: 7614, ep_reward: 135.784, ep_step: 1000 2023-05-30 04:01:47 - SimpleLog - INFO: - episode: 7615, ep_reward: -101.084, ep_step: 985 2023-05-30 04:01:49 - SimpleLog - INFO: - episode: 7616, ep_reward: -35.266, ep_step: 1000 2023-05-30 04:01:51 - SimpleLog - INFO: - episode: 7617, ep_reward: 135.707, ep_step: 1000 2023-05-30 04:01:54 - SimpleLog - INFO: - episode: 7618, ep_reward: 159.361, ep_step: 1000 2023-05-30 04:01:56 - SimpleLog - INFO: - episode: 7619, ep_reward: -8.370, ep_step: 940 2023-05-30 04:01:57 - SimpleLog - INFO: - episode: 7620, ep_reward: -37.235, ep_step: 741 2023-05-30 04:01:58 - SimpleLog - INFO: - episode: 7621, ep_reward: -103.824, ep_step: 83 2023-05-30 04:01:58 - SimpleLog - INFO: - episode: 7622, ep_reward: -98.171, ep_step: 97 2023-05-30 04:01:58 - SimpleLog - INFO: - episode: 7623, ep_reward: -99.034, ep_step: 96 2023-05-30 04:01:58 - SimpleLog - INFO: - episode: 7624, ep_reward: -105.099, ep_step: 80 2023-05-30 04:01:58 - SimpleLog - INFO: - episode: 7625, ep_reward: -102.445, ep_step: 68 2023-05-30 04:01:58 - SimpleLog - INFO: - episode: 7626, ep_reward: -102.780, ep_step: 78 2023-05-30 04:01:59 - SimpleLog - INFO: - episode: 7627, ep_reward: -102.275, ep_step: 69 2023-05-30 04:01:59 - SimpleLog - INFO: - episode: 7628, ep_reward: -104.944, ep_step: 75 2023-05-30 04:01:59 - SimpleLog - INFO: - episode: 7629, ep_reward: -109.287, ep_step: 81 2023-05-30 04:01:59 - SimpleLog - INFO: - episode: 7630, ep_reward: -106.908, ep_step: 82 2023-05-30 04:01:59 - SimpleLog - INFO: - episode: 7631, ep_reward: -105.784, ep_step: 82 2023-05-30 04:02:00 - SimpleLog - INFO: - episode: 7632, ep_reward: -105.319, ep_step: 70 2023-05-30 04:02:00 - SimpleLog - INFO: - episode: 7633, ep_reward: -103.697, ep_step: 73 2023-05-30 04:02:00 - SimpleLog - INFO: - episode: 7634, ep_reward: -105.128, ep_step: 78 2023-05-30 04:02:00 - SimpleLog - INFO: - episode: 7635, ep_reward: -106.152, ep_step: 75 2023-05-30 04:02:00 - SimpleLog - INFO: - episode: 7636, ep_reward: -105.183, ep_step: 67 2023-05-30 04:02:00 - SimpleLog - INFO: - episode: 7637, ep_reward: -106.438, ep_step: 75 2023-05-30 04:02:01 - SimpleLog - INFO: - episode: 7638, ep_reward: -105.606, ep_step: 74 2023-05-30 04:02:01 - SimpleLog - INFO: - episode: 7639, ep_reward: -105.791, ep_step: 82 2023-05-30 04:02:01 - SimpleLog - INFO: - episode: 7640, ep_reward: -103.140, ep_step: 64 2023-05-30 04:02:02 - SimpleLog - INFO: - episode: 7641, ep_reward: 3.528, ep_step: 650 2023-05-30 04:02:04 - SimpleLog - INFO: - episode: 7642, ep_reward: 2.518, ep_step: 928 2023-05-30 04:02:07 - SimpleLog - INFO: - episode: 7643, ep_reward: 161.023, ep_step: 1000 2023-05-30 04:02:09 - SimpleLog - INFO: - episode: 7644, ep_reward: 94.692, ep_step: 1000 2023-05-30 04:02:11 - SimpleLog - INFO: - episode: 7645, ep_reward: 3.141, ep_step: 855 2023-05-30 04:02:13 - SimpleLog - INFO: - episode: 7646, ep_reward: 68.991, ep_step: 1000 2023-05-30 04:02:15 - SimpleLog - INFO: - episode: 7647, ep_reward: 1.412, ep_step: 990 2023-05-30 04:02:18 - SimpleLog - INFO: - episode: 7648, ep_reward: 127.783, ep_step: 1000 2023-05-30 04:02:20 - SimpleLog - INFO: - episode: 7649, ep_reward: 79.058, ep_step: 1000 2023-05-30 04:02:22 - SimpleLog - INFO: - episode: 7650, ep_reward: 139.053, ep_step: 1000 2023-05-30 04:02:24 - SimpleLog - INFO: - episode: 7651, ep_reward: -5.374, ep_step: 880 2023-05-30 04:02:26 - SimpleLog - INFO: - episode: 7652, ep_reward: 139.205, ep_step: 1000 2023-05-30 04:02:29 - SimpleLog - INFO: - episode: 7653, ep_reward: 126.905, ep_step: 1000 2023-05-30 04:02:30 - SimpleLog - INFO: - episode: 7654, ep_reward: 8.787, ep_step: 870 2023-05-30 04:02:33 - SimpleLog - INFO: - episode: 7655, ep_reward: 116.551, ep_step: 1000 2023-05-30 04:02:35 - SimpleLog - INFO: - episode: 7656, ep_reward: 138.298, ep_step: 1000 2023-05-30 04:02:37 - SimpleLog - INFO: - episode: 7657, ep_reward: 73.387, ep_step: 1000 2023-05-30 04:02:39 - SimpleLog - INFO: - episode: 7658, ep_reward: 72.638, ep_step: 1000 2023-05-30 04:02:41 - SimpleLog - INFO: - episode: 7659, ep_reward: -3.958, ep_step: 779 2023-05-30 04:02:42 - SimpleLog - INFO: - episode: 7660, ep_reward: -77.074, ep_step: 273 2023-05-30 04:02:44 - SimpleLog - INFO: - episode: 7661, ep_reward: 77.137, ep_step: 1000 2023-05-30 04:02:46 - SimpleLog - INFO: - episode: 7662, ep_reward: 70.118, ep_step: 1000 2023-05-30 04:02:49 - SimpleLog - INFO: - episode: 7663, ep_reward: 140.620, ep_step: 1000 2023-05-30 04:02:50 - SimpleLog - INFO: - episode: 7664, ep_reward: -126.089, ep_step: 446 2023-05-30 04:02:50 - SimpleLog - INFO: - episode: 7665, ep_reward: -80.810, ep_step: 356 2023-05-30 04:02:53 - SimpleLog - INFO: - episode: 7666, ep_reward: 78.022, ep_step: 1000 2023-05-30 04:02:55 - SimpleLog - INFO: - episode: 7667, ep_reward: -8.511, ep_step: 983 2023-05-30 04:02:57 - SimpleLog - INFO: - episode: 7668, ep_reward: 134.617, ep_step: 1000 2023-05-30 04:02:59 - SimpleLog - INFO: - episode: 7669, ep_reward: 106.382, ep_step: 1000 2023-05-30 04:03:00 - SimpleLog - INFO: - episode: 7670, ep_reward: -112.422, ep_step: 392 2023-05-30 04:03:02 - SimpleLog - INFO: - episode: 7671, ep_reward: 140.735, ep_step: 1000 2023-05-30 04:03:05 - SimpleLog - INFO: - episode: 7672, ep_reward: 131.593, ep_step: 1000 2023-05-30 04:03:07 - SimpleLog - INFO: - episode: 7673, ep_reward: 103.941, ep_step: 1000 2023-05-30 04:03:09 - SimpleLog - INFO: - episode: 7674, ep_reward: 78.073, ep_step: 1000 2023-05-30 04:03:11 - SimpleLog - INFO: - episode: 7675, ep_reward: -116.601, ep_step: 836 2023-05-30 04:03:13 - SimpleLog - INFO: - episode: 7676, ep_reward: 124.154, ep_step: 1000 2023-05-30 04:03:16 - SimpleLog - INFO: - episode: 7677, ep_reward: 119.109, ep_step: 1000 2023-05-30 04:03:18 - SimpleLog - INFO: - episode: 7678, ep_reward: 119.555, ep_step: 1000 2023-05-30 04:03:20 - SimpleLog - INFO: - episode: 7679, ep_reward: 3.413, ep_step: 951 2023-05-30 04:03:22 - SimpleLog - INFO: - episode: 7680, ep_reward: 130.899, ep_step: 1000 2023-05-30 04:03:23 - SimpleLog - INFO: - episode: 7681, ep_reward: -40.586, ep_step: 542 2023-05-30 04:03:26 - SimpleLog - INFO: - episode: 7682, ep_reward: 116.550, ep_step: 1000 2023-05-30 04:03:28 - SimpleLog - INFO: - episode: 7683, ep_reward: 119.379, ep_step: 1000 2023-05-30 04:03:30 - SimpleLog - INFO: - episode: 7684, ep_reward: 136.765, ep_step: 1000 2023-05-30 04:03:32 - SimpleLog - INFO: - episode: 7685, ep_reward: 140.639, ep_step: 1000 2023-05-30 04:03:35 - SimpleLog - INFO: - episode: 7686, ep_reward: 118.993, ep_step: 1000 2023-05-30 04:03:37 - SimpleLog - INFO: - episode: 7687, ep_reward: 136.396, ep_step: 1000 2023-05-30 04:03:39 - SimpleLog - INFO: - episode: 7688, ep_reward: -17.168, ep_step: 980 2023-05-30 04:03:41 - SimpleLog - INFO: - episode: 7689, ep_reward: -15.896, ep_step: 975 2023-05-30 04:03:42 - SimpleLog - INFO: - episode: 7690, ep_reward: -102.985, ep_step: 247 2023-05-30 04:03:44 - SimpleLog - INFO: - episode: 7691, ep_reward: 138.357, ep_step: 1000 2023-05-30 04:03:46 - SimpleLog - INFO: - episode: 7692, ep_reward: 86.614, ep_step: 1000 2023-05-30 04:03:49 - SimpleLog - INFO: - episode: 7693, ep_reward: 126.771, ep_step: 1000 2023-05-30 04:03:51 - SimpleLog - INFO: - episode: 7694, ep_reward: 102.477, ep_step: 1000 2023-05-30 04:03:53 - SimpleLog - INFO: - episode: 7695, ep_reward: -2.476, ep_step: 931 2023-05-30 04:03:55 - SimpleLog - INFO: - episode: 7696, ep_reward: 129.682, ep_step: 1000 2023-05-30 04:03:57 - SimpleLog - INFO: - update_step: 6160000, online_eval_reward: 117.186 2023-05-30 04:03:59 - SimpleLog - INFO: - episode: 7697, ep_reward: 127.363, ep_step: 1000 2023-05-30 04:04:02 - SimpleLog - INFO: - episode: 7698, ep_reward: 105.076, ep_step: 1000 2023-05-30 04:04:04 - SimpleLog - INFO: - episode: 7699, ep_reward: 130.559, ep_step: 1000 2023-05-30 04:04:06 - SimpleLog - INFO: - episode: 7700, ep_reward: 112.603, ep_step: 1000 2023-05-30 04:04:08 - SimpleLog - INFO: - episode: 7701, ep_reward: 128.327, ep_step: 1000 2023-05-30 04:04:10 - SimpleLog - INFO: - episode: 7702, ep_reward: 104.713, ep_step: 1000 2023-05-30 04:04:12 - SimpleLog - INFO: - episode: 7703, ep_reward: 8.574, ep_step: 786 2023-05-30 04:04:14 - SimpleLog - INFO: - episode: 7704, ep_reward: 78.028, ep_step: 1000 2023-05-30 04:04:17 - SimpleLog - INFO: - episode: 7705, ep_reward: 107.325, ep_step: 1000 2023-05-30 04:04:19 - SimpleLog - INFO: - episode: 7706, ep_reward: -31.137, ep_step: 827 2023-05-30 04:04:20 - SimpleLog - INFO: - episode: 7707, ep_reward: -25.911, ep_step: 797 2023-05-30 04:04:23 - SimpleLog - INFO: - episode: 7708, ep_reward: 144.281, ep_step: 1000 2023-05-30 04:04:25 - SimpleLog - INFO: - episode: 7709, ep_reward: 138.246, ep_step: 1000 2023-05-30 04:04:27 - SimpleLog - INFO: - episode: 7710, ep_reward: 130.425, ep_step: 1000 2023-05-30 04:04:29 - SimpleLog - INFO: - episode: 7711, ep_reward: 132.733, ep_step: 1000 2023-05-30 04:04:32 - SimpleLog - INFO: - episode: 7712, ep_reward: 79.105, ep_step: 1000 2023-05-30 04:04:34 - SimpleLog - INFO: - episode: 7713, ep_reward: 119.344, ep_step: 1000 2023-05-30 04:04:36 - SimpleLog - INFO: - episode: 7714, ep_reward: 104.365, ep_step: 1000 2023-05-30 04:04:38 - SimpleLog - INFO: - episode: 7715, ep_reward: 123.606, ep_step: 1000 2023-05-30 04:04:41 - SimpleLog - INFO: - episode: 7716, ep_reward: 134.552, ep_step: 1000 2023-05-30 04:04:43 - SimpleLog - INFO: - episode: 7717, ep_reward: 137.650, ep_step: 1000 2023-05-30 04:04:45 - SimpleLog - INFO: - episode: 7718, ep_reward: -1.388, ep_step: 818 2023-05-30 04:04:47 - SimpleLog - INFO: - episode: 7719, ep_reward: 121.534, ep_step: 1000 2023-05-30 04:04:49 - SimpleLog - INFO: - episode: 7720, ep_reward: 138.585, ep_step: 1000 2023-05-30 04:04:51 - SimpleLog - INFO: - episode: 7721, ep_reward: 126.667, ep_step: 1000 2023-05-30 04:04:54 - SimpleLog - INFO: - episode: 7722, ep_reward: 78.989, ep_step: 1000 2023-05-30 04:04:56 - SimpleLog - INFO: - episode: 7723, ep_reward: 78.952, ep_step: 1000 2023-05-30 04:04:58 - SimpleLog - INFO: - episode: 7724, ep_reward: 101.170, ep_step: 1000 2023-05-30 04:05:01 - SimpleLog - INFO: - episode: 7725, ep_reward: 145.876, ep_step: 1000 2023-05-30 04:05:03 - SimpleLog - INFO: - episode: 7726, ep_reward: 102.287, ep_step: 1000 2023-05-30 04:05:05 - SimpleLog - INFO: - episode: 7727, ep_reward: 132.263, ep_step: 1000 2023-05-30 04:05:07 - SimpleLog - INFO: - episode: 7728, ep_reward: 127.916, ep_step: 1000 2023-05-30 04:05:09 - SimpleLog - INFO: - episode: 7729, ep_reward: -30.187, ep_step: 922 2023-05-30 04:05:12 - SimpleLog - INFO: - episode: 7730, ep_reward: 93.421, ep_step: 1000 2023-05-30 04:05:13 - SimpleLog - INFO: - episode: 7731, ep_reward: 12.418, ep_step: 836 2023-05-30 04:05:16 - SimpleLog - INFO: - episode: 7732, ep_reward: 140.347, ep_step: 1000 2023-05-30 04:05:18 - SimpleLog - INFO: - episode: 7733, ep_reward: 127.424, ep_step: 1000 2023-05-30 04:05:20 - SimpleLog - INFO: - episode: 7734, ep_reward: 7.933, ep_step: 829 2023-05-30 04:05:22 - SimpleLog - INFO: - episode: 7735, ep_reward: 12.319, ep_step: 1000 2023-05-30 04:05:24 - SimpleLog - INFO: - episode: 7736, ep_reward: -2.688, ep_step: 944 2023-05-30 04:05:26 - SimpleLog - INFO: - episode: 7737, ep_reward: 131.774, ep_step: 1000 2023-05-30 04:05:29 - SimpleLog - INFO: - episode: 7738, ep_reward: 110.018, ep_step: 1000 2023-05-30 04:05:31 - SimpleLog - INFO: - episode: 7739, ep_reward: 104.244, ep_step: 1000 2023-05-30 04:05:33 - SimpleLog - INFO: - episode: 7740, ep_reward: 1.918, ep_step: 779 2023-05-30 04:05:35 - SimpleLog - INFO: - episode: 7741, ep_reward: 9.155, ep_step: 946 2023-05-30 04:05:37 - SimpleLog - INFO: - episode: 7742, ep_reward: 130.815, ep_step: 1000 2023-05-30 04:05:39 - SimpleLog - INFO: - episode: 7743, ep_reward: 2.149, ep_step: 906 2023-05-30 04:05:41 - SimpleLog - INFO: - episode: 7744, ep_reward: 122.557, ep_step: 1000 2023-05-30 04:05:44 - SimpleLog - INFO: - episode: 7745, ep_reward: 123.763, ep_step: 1000 2023-05-30 04:05:46 - SimpleLog - INFO: - episode: 7746, ep_reward: 133.127, ep_step: 1000 2023-05-30 04:05:48 - SimpleLog - INFO: - episode: 7747, ep_reward: 79.538, ep_step: 1000 2023-05-30 04:05:50 - SimpleLog - INFO: - episode: 7748, ep_reward: 119.559, ep_step: 1000 2023-05-30 04:05:53 - SimpleLog - INFO: - episode: 7749, ep_reward: 115.922, ep_step: 1000 2023-05-30 04:05:55 - SimpleLog - INFO: - episode: 7750, ep_reward: 139.679, ep_step: 1000 2023-05-30 04:05:57 - SimpleLog - INFO: - episode: 7751, ep_reward: 129.589, ep_step: 1000 2023-05-30 04:05:59 - SimpleLog - INFO: - episode: 7752, ep_reward: 13.899, ep_step: 940 2023-05-30 04:06:02 - SimpleLog - INFO: - episode: 7753, ep_reward: 124.706, ep_step: 1000 2023-05-30 04:06:04 - SimpleLog - INFO: - episode: 7754, ep_reward: 137.314, ep_step: 1000 2023-05-30 04:06:06 - SimpleLog - INFO: - episode: 7755, ep_reward: 121.184, ep_step: 1000 2023-05-30 04:06:08 - SimpleLog - INFO: - episode: 7756, ep_reward: 132.192, ep_step: 1000 2023-05-30 04:06:11 - SimpleLog - INFO: - episode: 7757, ep_reward: 123.877, ep_step: 1000 2023-05-30 04:06:13 - SimpleLog - INFO: - episode: 7758, ep_reward: 134.779, ep_step: 1000 2023-05-30 04:06:15 - SimpleLog - INFO: - episode: 7759, ep_reward: 123.772, ep_step: 1000 2023-05-30 04:06:17 - SimpleLog - INFO: - episode: 7760, ep_reward: 104.738, ep_step: 1000 2023-05-30 04:06:20 - SimpleLog - INFO: - episode: 7761, ep_reward: 132.875, ep_step: 1000 2023-05-30 04:06:22 - SimpleLog - INFO: - episode: 7762, ep_reward: 134.584, ep_step: 1000 2023-05-30 04:06:24 - SimpleLog - INFO: - episode: 7763, ep_reward: 111.899, ep_step: 1000 2023-05-30 04:06:26 - SimpleLog - INFO: - episode: 7764, ep_reward: 109.396, ep_step: 1000 2023-05-30 04:06:29 - SimpleLog - INFO: - episode: 7765, ep_reward: 105.105, ep_step: 1000 2023-05-30 04:06:31 - SimpleLog - INFO: - episode: 7766, ep_reward: 136.648, ep_step: 1000 2023-05-30 04:06:33 - SimpleLog - INFO: - episode: 7767, ep_reward: 85.227, ep_step: 1000 2023-05-30 04:06:37 - SimpleLog - INFO: - update_step: 6230000, online_eval_reward: 108.891 2023-05-30 04:06:37 - SimpleLog - INFO: - episode: 7768, ep_reward: 121.738, ep_step: 1000 2023-05-30 04:06:40 - SimpleLog - INFO: - episode: 7769, ep_reward: 118.114, ep_step: 1000 2023-05-30 04:06:42 - SimpleLog - INFO: - episode: 7770, ep_reward: 18.944, ep_step: 898 2023-05-30 04:06:44 - SimpleLog - INFO: - episode: 7771, ep_reward: 114.396, ep_step: 1000 2023-05-30 04:06:46 - SimpleLog - INFO: - episode: 7772, ep_reward: 130.926, ep_step: 1000 2023-05-30 04:06:48 - SimpleLog - INFO: - episode: 7773, ep_reward: 127.494, ep_step: 1000 2023-05-30 04:06:51 - SimpleLog - INFO: - episode: 7774, ep_reward: 129.244, ep_step: 1000 2023-05-30 04:06:53 - SimpleLog - INFO: - episode: 7775, ep_reward: 118.263, ep_step: 1000 2023-05-30 04:06:55 - SimpleLog - INFO: - episode: 7776, ep_reward: 108.428, ep_step: 1000 2023-05-30 04:06:57 - SimpleLog - INFO: - episode: 7777, ep_reward: 132.122, ep_step: 1000 2023-05-30 04:07:00 - SimpleLog - INFO: - episode: 7778, ep_reward: 124.415, ep_step: 1000 2023-05-30 04:07:02 - SimpleLog - INFO: - episode: 7779, ep_reward: 128.364, ep_step: 1000 2023-05-30 04:07:04 - SimpleLog - INFO: - episode: 7780, ep_reward: 132.434, ep_step: 1000 2023-05-30 04:07:06 - SimpleLog - INFO: - episode: 7781, ep_reward: 68.899, ep_step: 1000 2023-05-30 04:07:09 - SimpleLog - INFO: - episode: 7782, ep_reward: 3.995, ep_step: 1000 2023-05-30 04:07:09 - SimpleLog - INFO: - episode: 7783, ep_reward: -138.618, ep_step: 152 2023-05-30 04:07:11 - SimpleLog - INFO: - episode: 7784, ep_reward: 125.521, ep_step: 1000 2023-05-30 04:07:14 - SimpleLog - INFO: - episode: 7785, ep_reward: 129.053, ep_step: 1000 2023-05-30 04:07:16 - SimpleLog - INFO: - episode: 7786, ep_reward: 69.742, ep_step: 1000 2023-05-30 04:07:16 - SimpleLog - INFO: - episode: 7787, ep_reward: -132.732, ep_step: 202 2023-05-30 04:07:18 - SimpleLog - INFO: - episode: 7788, ep_reward: 1.100, ep_step: 772 2023-05-30 04:07:20 - SimpleLog - INFO: - episode: 7789, ep_reward: 131.495, ep_step: 1000 2023-05-30 04:07:22 - SimpleLog - INFO: - episode: 7790, ep_reward: 114.444, ep_step: 1000 2023-05-30 04:07:25 - SimpleLog - INFO: - episode: 7791, ep_reward: 128.263, ep_step: 1000 2023-05-30 04:07:27 - SimpleLog - INFO: - episode: 7792, ep_reward: 114.347, ep_step: 1000 2023-05-30 04:07:29 - SimpleLog - INFO: - episode: 7793, ep_reward: 105.492, ep_step: 1000 2023-05-30 04:07:31 - SimpleLog - INFO: - episode: 7794, ep_reward: 130.854, ep_step: 1000 2023-05-30 04:07:34 - SimpleLog - INFO: - episode: 7795, ep_reward: 144.504, ep_step: 1000 2023-05-30 04:07:34 - SimpleLog - INFO: - episode: 7796, ep_reward: -133.925, ep_step: 154 2023-05-30 04:07:35 - SimpleLog - INFO: - episode: 7797, ep_reward: -180.205, ep_step: 642 2023-05-30 04:07:37 - SimpleLog - INFO: - episode: 7798, ep_reward: -15.463, ep_step: 814 2023-05-30 04:07:40 - SimpleLog - INFO: - episode: 7799, ep_reward: 120.623, ep_step: 1000 2023-05-30 04:07:42 - SimpleLog - INFO: - episode: 7800, ep_reward: 114.103, ep_step: 1000 2023-05-30 04:07:42 - SimpleLog - INFO: - episode: 7801, ep_reward: -126.092, ep_step: 102 2023-05-30 04:07:44 - SimpleLog - INFO: - episode: 7802, ep_reward: 125.728, ep_step: 1000 2023-05-30 04:07:46 - SimpleLog - INFO: - episode: 7803, ep_reward: 125.693, ep_step: 1000 2023-05-30 04:07:49 - SimpleLog - INFO: - episode: 7804, ep_reward: 95.666, ep_step: 1000 2023-05-30 04:07:51 - SimpleLog - INFO: - episode: 7805, ep_reward: 132.337, ep_step: 1000 2023-05-30 04:07:53 - SimpleLog - INFO: - episode: 7806, ep_reward: 132.813, ep_step: 1000 2023-05-30 04:07:55 - SimpleLog - INFO: - episode: 7807, ep_reward: 3.646, ep_step: 882 2023-05-30 04:07:57 - SimpleLog - INFO: - episode: 7808, ep_reward: 123.193, ep_step: 1000 2023-05-30 04:08:00 - SimpleLog - INFO: - episode: 7809, ep_reward: 0.694, ep_step: 1000 2023-05-30 04:08:02 - SimpleLog - INFO: - episode: 7810, ep_reward: 119.479, ep_step: 1000 2023-05-30 04:08:04 - SimpleLog - INFO: - episode: 7811, ep_reward: -21.197, ep_step: 775 2023-05-30 04:08:05 - SimpleLog - INFO: - episode: 7812, ep_reward: -26.162, ep_step: 631 2023-05-30 04:08:07 - SimpleLog - INFO: - episode: 7813, ep_reward: 137.810, ep_step: 1000 2023-05-30 04:08:10 - SimpleLog - INFO: - episode: 7814, ep_reward: 146.572, ep_step: 1000 2023-05-30 04:08:12 - SimpleLog - INFO: - episode: 7815, ep_reward: 64.895, ep_step: 1000 2023-05-30 04:08:14 - SimpleLog - INFO: - episode: 7816, ep_reward: 109.437, ep_step: 1000 2023-05-30 04:08:16 - SimpleLog - INFO: - episode: 7817, ep_reward: 108.143, ep_step: 1000 2023-05-30 04:08:19 - SimpleLog - INFO: - episode: 7818, ep_reward: 88.247, ep_step: 1000 2023-05-30 04:08:21 - SimpleLog - INFO: - episode: 7819, ep_reward: 76.908, ep_step: 1000 2023-05-30 04:08:23 - SimpleLog - INFO: - episode: 7820, ep_reward: 17.246, ep_step: 808 2023-05-30 04:08:25 - SimpleLog - INFO: - episode: 7821, ep_reward: 81.028, ep_step: 1000 2023-05-30 04:08:27 - SimpleLog - INFO: - episode: 7822, ep_reward: -32.346, ep_step: 820 2023-05-30 04:08:28 - SimpleLog - INFO: - episode: 7823, ep_reward: -91.402, ep_step: 437 2023-05-30 04:08:30 - SimpleLog - INFO: - episode: 7824, ep_reward: 132.747, ep_step: 1000 2023-05-30 04:08:32 - SimpleLog - INFO: - episode: 7825, ep_reward: 128.579, ep_step: 1000 2023-05-30 04:08:34 - SimpleLog - INFO: - episode: 7826, ep_reward: 83.614, ep_step: 1000 2023-05-30 04:08:35 - SimpleLog - INFO: - episode: 7827, ep_reward: -106.575, ep_step: 203 2023-05-30 04:08:37 - SimpleLog - INFO: - episode: 7828, ep_reward: 81.159, ep_step: 1000 2023-05-30 04:08:39 - SimpleLog - INFO: - episode: 7829, ep_reward: 21.355, ep_step: 1000 2023-05-30 04:08:42 - SimpleLog - INFO: - episode: 7830, ep_reward: 4.480, ep_step: 1000 2023-05-30 04:08:44 - SimpleLog - INFO: - episode: 7831, ep_reward: 94.784, ep_step: 1000 2023-05-30 04:08:46 - SimpleLog - INFO: - episode: 7832, ep_reward: -6.456, ep_step: 985 2023-05-30 04:08:48 - SimpleLog - INFO: - episode: 7833, ep_reward: 91.537, ep_step: 1000 2023-05-30 04:08:51 - SimpleLog - INFO: - episode: 7834, ep_reward: 81.836, ep_step: 1000 2023-05-30 04:08:52 - SimpleLog - INFO: - episode: 7835, ep_reward: -110.854, ep_step: 430 2023-05-30 04:08:54 - SimpleLog - INFO: - episode: 7836, ep_reward: 126.268, ep_step: 1000 2023-05-30 04:08:56 - SimpleLog - INFO: - episode: 7837, ep_reward: 100.369, ep_step: 1000 2023-05-30 04:08:58 - SimpleLog - INFO: - episode: 7838, ep_reward: -14.209, ep_step: 921 2023-05-30 04:09:00 - SimpleLog - INFO: - episode: 7839, ep_reward: -20.943, ep_step: 770 2023-05-30 04:09:00 - SimpleLog - INFO: - episode: 7840, ep_reward: -128.451, ep_step: 118 2023-05-30 04:09:01 - SimpleLog - INFO: - episode: 7841, ep_reward: -83.215, ep_step: 417 2023-05-30 04:09:01 - SimpleLog - INFO: - episode: 7842, ep_reward: -124.458, ep_step: 89 2023-05-30 04:09:03 - SimpleLog - INFO: - episode: 7843, ep_reward: -59.199, ep_step: 561 2023-05-30 04:09:04 - SimpleLog - INFO: - episode: 7844, ep_reward: -118.888, ep_step: 547 2023-05-30 04:09:04 - SimpleLog - INFO: - episode: 7845, ep_reward: -116.728, ep_step: 280 2023-05-30 04:09:05 - SimpleLog - INFO: - episode: 7846, ep_reward: -83.417, ep_step: 275 2023-05-30 04:09:06 - SimpleLog - INFO: - episode: 7847, ep_reward: -117.847, ep_step: 222 2023-05-30 04:09:06 - SimpleLog - INFO: - episode: 7848, ep_reward: -112.211, ep_step: 203 2023-05-30 04:09:07 - SimpleLog - INFO: - episode: 7849, ep_reward: -100.604, ep_step: 367 2023-05-30 04:09:08 - SimpleLog - INFO: - episode: 7850, ep_reward: -129.554, ep_step: 285 2023-05-30 04:09:08 - SimpleLog - INFO: - episode: 7851, ep_reward: -121.696, ep_step: 95 2023-05-30 04:09:08 - SimpleLog - INFO: - episode: 7852, ep_reward: -101.332, ep_step: 182 2023-05-30 04:09:08 - SimpleLog - INFO: - episode: 7853, ep_reward: -128.802, ep_step: 137 2023-05-30 04:09:09 - SimpleLog - INFO: - episode: 7854, ep_reward: -125.633, ep_step: 96 2023-05-30 04:09:09 - SimpleLog - INFO: - episode: 7855, ep_reward: -125.640, ep_step: 181 2023-05-30 04:09:10 - SimpleLog - INFO: - episode: 7856, ep_reward: -141.465, ep_step: 373 2023-05-30 04:09:10 - SimpleLog - INFO: - episode: 7857, ep_reward: -132.934, ep_step: 107 2023-05-30 04:09:10 - SimpleLog - INFO: - episode: 7858, ep_reward: -129.750, ep_step: 99 2023-05-30 04:09:11 - SimpleLog - INFO: - episode: 7859, ep_reward: -120.635, ep_step: 64 2023-05-30 04:09:11 - SimpleLog - INFO: - episode: 7860, ep_reward: -123.652, ep_step: 75 2023-05-30 04:09:11 - SimpleLog - INFO: - episode: 7861, ep_reward: -124.446, ep_step: 82 2023-05-30 04:09:11 - SimpleLog - INFO: - episode: 7862, ep_reward: -133.467, ep_step: 157 2023-05-30 04:09:11 - SimpleLog - INFO: - episode: 7863, ep_reward: -120.911, ep_step: 65 2023-05-30 04:09:12 - SimpleLog - INFO: - episode: 7864, ep_reward: -140.631, ep_step: 278 2023-05-30 04:09:12 - SimpleLog - INFO: - episode: 7865, ep_reward: -123.459, ep_step: 99 2023-05-30 04:09:13 - SimpleLog - INFO: - episode: 7866, ep_reward: -143.203, ep_step: 338 2023-05-30 04:09:13 - SimpleLog - INFO: - episode: 7867, ep_reward: -130.299, ep_step: 123 2023-05-30 04:09:14 - SimpleLog - INFO: - episode: 7868, ep_reward: -128.294, ep_step: 140 2023-05-30 04:09:15 - SimpleLog - INFO: - update_step: 6300000, online_eval_reward: -135.734 2023-05-30 04:09:15 - SimpleLog - INFO: - episode: 7869, ep_reward: -145.136, ep_step: 300 2023-05-30 04:09:15 - SimpleLog - INFO: - episode: 7870, ep_reward: -140.346, ep_step: 298 2023-05-30 04:09:16 - SimpleLog - INFO: - episode: 7871, ep_reward: -128.247, ep_step: 130 2023-05-30 04:09:16 - SimpleLog - INFO: - episode: 7872, ep_reward: -115.740, ep_step: 57 2023-05-30 04:09:16 - SimpleLog - INFO: - episode: 7873, ep_reward: -117.522, ep_step: 65 2023-05-30 04:09:16 - SimpleLog - INFO: - episode: 7874, ep_reward: -128.875, ep_step: 145 2023-05-30 04:09:18 - SimpleLog - INFO: - episode: 7875, ep_reward: -86.758, ep_step: 1000 2023-05-30 04:09:19 - SimpleLog - INFO: - episode: 7876, ep_reward: -131.960, ep_step: 111 2023-05-30 04:09:19 - SimpleLog - INFO: - episode: 7877, ep_reward: -135.850, ep_step: 137 2023-05-30 04:09:19 - SimpleLog - INFO: - episode: 7878, ep_reward: -127.487, ep_step: 84 2023-05-30 04:09:19 - SimpleLog - INFO: - episode: 7879, ep_reward: -128.973, ep_step: 89 2023-05-30 04:09:20 - SimpleLog - INFO: - episode: 7880, ep_reward: -125.213, ep_step: 73 2023-05-30 04:09:20 - SimpleLog - INFO: - episode: 7881, ep_reward: -135.225, ep_step: 117 2023-05-30 04:09:20 - SimpleLog - INFO: - episode: 7882, ep_reward: -132.988, ep_step: 101 2023-05-30 04:09:20 - SimpleLog - INFO: - episode: 7883, ep_reward: -116.136, ep_step: 55 2023-05-30 04:09:22 - SimpleLog - INFO: - episode: 7884, ep_reward: -75.397, ep_step: 1000 2023-05-30 04:09:23 - SimpleLog - INFO: - episode: 7885, ep_reward: -125.744, ep_step: 108 2023-05-30 04:09:25 - SimpleLog - INFO: - episode: 7886, ep_reward: -77.585, ep_step: 1000 2023-05-30 04:09:25 - SimpleLog - INFO: - episode: 7887, ep_reward: -121.510, ep_step: 80 2023-05-30 04:09:27 - SimpleLog - INFO: - episode: 7888, ep_reward: -166.612, ep_step: 721 2023-05-30 04:09:27 - SimpleLog - INFO: - episode: 7889, ep_reward: -117.202, ep_step: 62 2023-05-30 04:09:29 - SimpleLog - INFO: - episode: 7890, ep_reward: -156.568, ep_step: 718 2023-05-30 04:09:30 - SimpleLog - INFO: - episode: 7891, ep_reward: -166.143, ep_step: 686 2023-05-30 04:09:32 - SimpleLog - INFO: - episode: 7892, ep_reward: -173.391, ep_step: 838 2023-05-30 04:09:33 - SimpleLog - INFO: - episode: 7893, ep_reward: -136.887, ep_step: 270 2023-05-30 04:09:33 - SimpleLog - INFO: - episode: 7894, ep_reward: -129.006, ep_step: 147 2023-05-30 04:09:34 - SimpleLog - INFO: - episode: 7895, ep_reward: -146.775, ep_step: 368 2023-05-30 04:09:35 - SimpleLog - INFO: - episode: 7896, ep_reward: -161.516, ep_step: 714 2023-05-30 04:09:36 - SimpleLog - INFO: - episode: 7897, ep_reward: -115.111, ep_step: 366 2023-05-30 04:09:38 - SimpleLog - INFO: - episode: 7898, ep_reward: -99.068, ep_step: 801 2023-05-30 04:09:40 - SimpleLog - INFO: - episode: 7899, ep_reward: -34.254, ep_step: 1000 2023-05-30 04:09:42 - SimpleLog - INFO: - episode: 7900, ep_reward: -4.587, ep_step: 1000 2023-05-30 04:09:45 - SimpleLog - INFO: - episode: 7901, ep_reward: -38.293, ep_step: 1000 2023-05-30 04:09:47 - SimpleLog - INFO: - episode: 7902, ep_reward: -146.159, ep_step: 806 2023-05-30 04:09:49 - SimpleLog - INFO: - episode: 7903, ep_reward: -16.340, ep_step: 1000 2023-05-30 04:09:50 - SimpleLog - INFO: - episode: 7904, ep_reward: -126.837, ep_step: 573 2023-05-30 04:09:51 - SimpleLog - INFO: - episode: 7905, ep_reward: -96.142, ep_step: 472 2023-05-30 04:09:53 - SimpleLog - INFO: - episode: 7906, ep_reward: -140.922, ep_step: 775 2023-05-30 04:09:55 - SimpleLog - INFO: - episode: 7907, ep_reward: 7.658, ep_step: 1000 2023-05-30 04:09:57 - SimpleLog - INFO: - episode: 7908, ep_reward: 7.804, ep_step: 1000 2023-05-30 04:10:00 - SimpleLog - INFO: - episode: 7909, ep_reward: 10.800, ep_step: 1000 2023-05-30 04:10:02 - SimpleLog - INFO: - episode: 7910, ep_reward: 14.734, ep_step: 1000 2023-05-30 04:10:04 - SimpleLog - INFO: - episode: 7911, ep_reward: 6.041, ep_step: 1000 2023-05-30 04:10:06 - SimpleLog - INFO: - episode: 7912, ep_reward: 7.828, ep_step: 1000 2023-05-30 04:10:09 - SimpleLog - INFO: - episode: 7913, ep_reward: 9.057, ep_step: 1000 2023-05-30 04:10:11 - SimpleLog - INFO: - episode: 7914, ep_reward: -0.709, ep_step: 1000 2023-05-30 04:10:13 - SimpleLog - INFO: - episode: 7915, ep_reward: -42.802, ep_step: 673 2023-05-30 04:10:15 - SimpleLog - INFO: - episode: 7916, ep_reward: 4.906, ep_step: 1000 2023-05-30 04:10:17 - SimpleLog - INFO: - episode: 7917, ep_reward: 1.496, ep_step: 1000 2023-05-30 04:10:19 - SimpleLog - INFO: - episode: 7918, ep_reward: 80.211, ep_step: 1000 2023-05-30 04:10:22 - SimpleLog - INFO: - episode: 7919, ep_reward: 6.182, ep_step: 1000 2023-05-30 04:10:24 - SimpleLog - INFO: - episode: 7920, ep_reward: 7.286, ep_step: 1000 2023-05-30 04:10:25 - SimpleLog - INFO: - episode: 7921, ep_reward: -39.717, ep_step: 620 2023-05-30 04:10:28 - SimpleLog - INFO: - episode: 7922, ep_reward: -22.300, ep_step: 1000 2023-05-30 04:10:30 - SimpleLog - INFO: - episode: 7923, ep_reward: 4.833, ep_step: 1000 2023-05-30 04:10:32 - SimpleLog - INFO: - episode: 7924, ep_reward: 4.371, ep_step: 1000 2023-05-30 04:10:33 - SimpleLog - INFO: - episode: 7925, ep_reward: -60.486, ep_step: 510 2023-05-30 04:10:33 - SimpleLog - INFO: - episode: 7926, ep_reward: -102.966, ep_step: 75 2023-05-30 04:10:34 - SimpleLog - INFO: - episode: 7927, ep_reward: -61.148, ep_step: 497 2023-05-30 04:10:37 - SimpleLog - INFO: - episode: 7928, ep_reward: 0.435, ep_step: 1000 2023-05-30 04:10:38 - SimpleLog - INFO: - episode: 7929, ep_reward: -50.365, ep_step: 693 2023-05-30 04:10:38 - SimpleLog - INFO: - episode: 7930, ep_reward: -100.290, ep_step: 65 2023-05-30 04:10:40 - SimpleLog - INFO: - episode: 7931, ep_reward: -58.645, ep_step: 825 2023-05-30 04:10:43 - SimpleLog - INFO: - episode: 7932, ep_reward: 78.624, ep_step: 1000 2023-05-30 04:10:44 - SimpleLog - INFO: - episode: 7933, ep_reward: -36.161, ep_step: 606 2023-05-30 04:10:46 - SimpleLog - INFO: - episode: 7934, ep_reward: 7.620, ep_step: 881 2023-05-30 04:10:47 - SimpleLog - INFO: - episode: 7935, ep_reward: -36.257, ep_step: 571 2023-05-30 04:10:48 - SimpleLog - INFO: - episode: 7936, ep_reward: -37.234, ep_step: 566 2023-05-30 04:10:51 - SimpleLog - INFO: - episode: 7937, ep_reward: 1.507, ep_step: 931 2023-05-30 04:10:53 - SimpleLog - INFO: - episode: 7938, ep_reward: 4.915, ep_step: 1000 2023-05-30 04:10:55 - SimpleLog - INFO: - episode: 7939, ep_reward: 11.784, ep_step: 870 2023-05-30 04:10:57 - SimpleLog - INFO: - episode: 7940, ep_reward: -1.881, ep_step: 949 2023-05-30 04:10:59 - SimpleLog - INFO: - episode: 7941, ep_reward: 1.199, ep_step: 887 2023-05-30 04:11:01 - SimpleLog - INFO: - episode: 7942, ep_reward: 91.383, ep_step: 1000 2023-05-30 04:11:02 - SimpleLog - INFO: - episode: 7943, ep_reward: -77.468, ep_step: 521 2023-05-30 04:11:05 - SimpleLog - INFO: - episode: 7944, ep_reward: 102.373, ep_step: 1000 2023-05-30 04:11:07 - SimpleLog - INFO: - episode: 7945, ep_reward: 93.224, ep_step: 1000 2023-05-30 04:11:09 - SimpleLog - INFO: - episode: 7946, ep_reward: 7.293, ep_step: 918 2023-05-30 04:11:11 - SimpleLog - INFO: - episode: 7947, ep_reward: 109.802, ep_step: 1000 2023-05-30 04:11:14 - SimpleLog - INFO: - episode: 7948, ep_reward: 104.125, ep_step: 1000 2023-05-30 04:11:16 - SimpleLog - INFO: - episode: 7949, ep_reward: 83.238, ep_step: 1000 2023-05-30 04:11:18 - SimpleLog - INFO: - episode: 7950, ep_reward: 85.964, ep_step: 1000 2023-05-30 04:11:20 - SimpleLog - INFO: - episode: 7951, ep_reward: 94.254, ep_step: 1000 2023-05-30 04:11:23 - SimpleLog - INFO: - episode: 7952, ep_reward: 76.796, ep_step: 1000 2023-05-30 04:11:25 - SimpleLog - INFO: - episode: 7953, ep_reward: 67.203, ep_step: 1000 2023-05-30 04:11:27 - SimpleLog - INFO: - episode: 7954, ep_reward: 89.202, ep_step: 1000 2023-05-30 04:11:27 - SimpleLog - INFO: - episode: 7955, ep_reward: -100.440, ep_step: 123 2023-05-30 04:11:27 - SimpleLog - INFO: - episode: 7956, ep_reward: -108.135, ep_step: 61 2023-05-30 04:11:28 - SimpleLog - INFO: - episode: 7957, ep_reward: -97.982, ep_step: 113 2023-05-30 04:11:30 - SimpleLog - INFO: - episode: 7958, ep_reward: 102.315, ep_step: 1000 2023-05-30 04:11:31 - SimpleLog - INFO: - episode: 7959, ep_reward: -125.907, ep_step: 288 2023-05-30 04:11:31 - SimpleLog - INFO: - episode: 7960, ep_reward: -106.028, ep_step: 181 2023-05-30 04:11:31 - SimpleLog - INFO: - episode: 7961, ep_reward: -100.954, ep_step: 97 2023-05-30 04:11:32 - SimpleLog - INFO: - episode: 7962, ep_reward: -97.425, ep_step: 143 2023-05-30 04:11:34 - SimpleLog - INFO: - episode: 7963, ep_reward: 105.468, ep_step: 1000 2023-05-30 04:11:36 - SimpleLog - INFO: - episode: 7964, ep_reward: 84.588, ep_step: 1000 2023-05-30 04:11:38 - SimpleLog - INFO: - episode: 7965, ep_reward: -101.279, ep_step: 1000 2023-05-30 04:11:39 - SimpleLog - INFO: - episode: 7966, ep_reward: -106.970, ep_step: 212 2023-05-30 04:11:41 - SimpleLog - INFO: - episode: 7967, ep_reward: 9.921, ep_step: 1000 2023-05-30 04:11:42 - SimpleLog - INFO: - episode: 7968, ep_reward: -178.954, ep_step: 604 2023-05-30 04:11:43 - SimpleLog - INFO: - episode: 7969, ep_reward: -130.236, ep_step: 479 2023-05-30 04:11:44 - SimpleLog - INFO: - episode: 7970, ep_reward: -112.353, ep_step: 199 2023-05-30 04:11:44 - SimpleLog - INFO: - episode: 7971, ep_reward: -116.816, ep_step: 252 2023-05-30 04:11:47 - SimpleLog - INFO: - episode: 7972, ep_reward: -0.272, ep_step: 1000 2023-05-30 04:11:49 - SimpleLog - INFO: - episode: 7973, ep_reward: 7.221, ep_step: 1000 2023-05-30 04:11:51 - SimpleLog - INFO: - episode: 7974, ep_reward: 83.718, ep_step: 1000 2023-05-30 04:11:54 - SimpleLog - INFO: - update_step: 6370000, online_eval_reward: 46.466 2023-05-30 04:11:55 - SimpleLog - INFO: - episode: 7975, ep_reward: 7.109, ep_step: 1000 2023-05-30 04:11:58 - SimpleLog - INFO: - episode: 7976, ep_reward: 5.518, ep_step: 1000 2023-05-30 04:12:00 - SimpleLog - INFO: - episode: 7977, ep_reward: 83.913, ep_step: 1000 2023-05-30 04:12:02 - SimpleLog - INFO: - episode: 7978, ep_reward: 5.503, ep_step: 1000 2023-05-30 04:12:04 - SimpleLog - INFO: - episode: 7979, ep_reward: 5.624, ep_step: 1000 2023-05-30 04:12:07 - SimpleLog - INFO: - episode: 7980, ep_reward: 68.402, ep_step: 1000 2023-05-30 04:12:09 - SimpleLog - INFO: - episode: 7981, ep_reward: 0.223, ep_step: 950 2023-05-30 04:12:11 - SimpleLog - INFO: - episode: 7982, ep_reward: 96.017, ep_step: 1000 2023-05-30 04:12:13 - SimpleLog - INFO: - episode: 7983, ep_reward: 80.800, ep_step: 1000 2023-05-30 04:12:16 - SimpleLog - INFO: - episode: 7984, ep_reward: 79.227, ep_step: 1000 2023-05-30 04:12:16 - SimpleLog - INFO: - episode: 7985, ep_reward: -88.414, ep_step: 408 2023-05-30 04:12:18 - SimpleLog - INFO: - episode: 7986, ep_reward: -86.501, ep_step: 718 2023-05-30 04:12:20 - SimpleLog - INFO: - episode: 7987, ep_reward: 6.913, ep_step: 1000 2023-05-30 04:12:23 - SimpleLog - INFO: - episode: 7988, ep_reward: 10.313, ep_step: 1000 2023-05-30 04:12:25 - SimpleLog - INFO: - episode: 7989, ep_reward: 107.217, ep_step: 1000 2023-05-30 04:12:26 - SimpleLog - INFO: - episode: 7990, ep_reward: -142.390, ep_step: 700 2023-05-30 04:12:28 - SimpleLog - INFO: - episode: 7991, ep_reward: -65.143, ep_step: 535 2023-05-30 04:12:30 - SimpleLog - INFO: - episode: 7992, ep_reward: 80.513, ep_step: 1000 2023-05-30 04:12:32 - SimpleLog - INFO: - episode: 7993, ep_reward: -91.169, ep_step: 808 2023-05-30 04:12:33 - SimpleLog - INFO: - episode: 7994, ep_reward: -57.094, ep_step: 504 2023-05-30 04:12:35 - SimpleLog - INFO: - episode: 7995, ep_reward: 78.903, ep_step: 1000 2023-05-30 04:12:37 - SimpleLog - INFO: - episode: 7996, ep_reward: 102.917, ep_step: 1000 2023-05-30 04:12:39 - SimpleLog - INFO: - episode: 7997, ep_reward: -66.108, ep_step: 524 2023-05-30 04:12:41 - SimpleLog - INFO: - episode: 7998, ep_reward: 82.162, ep_step: 1000 2023-05-30 04:12:43 - SimpleLog - INFO: - episode: 7999, ep_reward: 79.100, ep_step: 1000 2023-05-30 04:12:45 - SimpleLog - INFO: - episode: 8000, ep_reward: -0.437, ep_step: 968 2023-05-30 04:12:47 - SimpleLog - INFO: - episode: 8001, ep_reward: 77.557, ep_step: 1000 2023-05-30 04:12:50 - SimpleLog - INFO: - episode: 8002, ep_reward: 96.481, ep_step: 1000 2023-05-30 04:12:52 - SimpleLog - INFO: - episode: 8003, ep_reward: 79.497, ep_step: 1000 2023-05-30 04:12:54 - SimpleLog - INFO: - episode: 8004, ep_reward: 103.919, ep_step: 1000 2023-05-30 04:12:57 - SimpleLog - INFO: - episode: 8005, ep_reward: 126.237, ep_step: 1000 2023-05-30 04:12:59 - SimpleLog - INFO: - episode: 8006, ep_reward: 92.488, ep_step: 1000 2023-05-30 04:13:01 - SimpleLog - INFO: - episode: 8007, ep_reward: 12.216, ep_step: 1000 2023-05-30 04:13:03 - SimpleLog - INFO: - episode: 8008, ep_reward: -3.770, ep_step: 946 2023-05-30 04:13:05 - SimpleLog - INFO: - episode: 8009, ep_reward: -21.086, ep_step: 887 2023-05-30 04:13:07 - SimpleLog - INFO: - episode: 8010, ep_reward: 94.109, ep_step: 1000 2023-05-30 04:13:10 - SimpleLog - INFO: - episode: 8011, ep_reward: 83.225, ep_step: 1000 2023-05-30 04:13:12 - SimpleLog - INFO: - episode: 8012, ep_reward: 13.467, ep_step: 1000 2023-05-30 04:13:14 - SimpleLog - INFO: - episode: 8013, ep_reward: 103.112, ep_step: 1000 2023-05-30 04:13:16 - SimpleLog - INFO: - episode: 8014, ep_reward: 103.179, ep_step: 1000 2023-05-30 04:13:19 - SimpleLog - INFO: - episode: 8015, ep_reward: 113.900, ep_step: 1000 2023-05-30 04:13:21 - SimpleLog - INFO: - episode: 8016, ep_reward: 96.683, ep_step: 1000 2023-05-30 04:13:23 - SimpleLog - INFO: - episode: 8017, ep_reward: 107.642, ep_step: 1000 2023-05-30 04:13:25 - SimpleLog - INFO: - episode: 8018, ep_reward: 5.135, ep_step: 988 2023-05-30 04:13:28 - SimpleLog - INFO: - episode: 8019, ep_reward: 104.727, ep_step: 1000 2023-05-30 04:13:30 - SimpleLog - INFO: - episode: 8020, ep_reward: 117.278, ep_step: 1000 2023-05-30 04:13:32 - SimpleLog - INFO: - episode: 8021, ep_reward: 103.477, ep_step: 1000 2023-05-30 04:13:34 - SimpleLog - INFO: - episode: 8022, ep_reward: -29.082, ep_step: 724 2023-05-30 04:13:36 - SimpleLog - INFO: - episode: 8023, ep_reward: 79.458, ep_step: 1000 2023-05-30 04:13:38 - SimpleLog - INFO: - episode: 8024, ep_reward: 108.683, ep_step: 1000 2023-05-30 04:13:40 - SimpleLog - INFO: - episode: 8025, ep_reward: 99.016, ep_step: 1000 2023-05-30 04:13:43 - SimpleLog - INFO: - episode: 8026, ep_reward: 4.686, ep_step: 1000 2023-05-30 04:13:45 - SimpleLog - INFO: - episode: 8027, ep_reward: 123.019, ep_step: 1000 2023-05-30 04:13:47 - SimpleLog - INFO: - episode: 8028, ep_reward: 8.860, ep_step: 1000 2023-05-30 04:13:49 - SimpleLog - INFO: - episode: 8029, ep_reward: 7.003, ep_step: 1000 2023-05-30 04:13:52 - SimpleLog - INFO: - episode: 8030, ep_reward: 120.951, ep_step: 1000 2023-05-30 04:13:54 - SimpleLog - INFO: - episode: 8031, ep_reward: 108.444, ep_step: 1000 2023-05-30 04:13:56 - SimpleLog - INFO: - episode: 8032, ep_reward: 111.293, ep_step: 1000 2023-05-30 04:13:58 - SimpleLog - INFO: - episode: 8033, ep_reward: 139.434, ep_step: 1000 2023-05-30 04:14:01 - SimpleLog - INFO: - episode: 8034, ep_reward: 7.105, ep_step: 1000 2023-05-30 04:14:01 - SimpleLog - INFO: - episode: 8035, ep_reward: -121.084, ep_step: 248 2023-05-30 04:14:03 - SimpleLog - INFO: - episode: 8036, ep_reward: 73.401, ep_step: 1000 2023-05-30 04:14:04 - SimpleLog - INFO: - episode: 8037, ep_reward: -132.390, ep_step: 79 2023-05-30 04:14:04 - SimpleLog - INFO: - episode: 8038, ep_reward: -126.995, ep_step: 81 2023-05-30 04:14:06 - SimpleLog - INFO: - episode: 8039, ep_reward: 103.486, ep_step: 1000 2023-05-30 04:14:08 - SimpleLog - INFO: - episode: 8040, ep_reward: 96.246, ep_step: 1000 2023-05-30 04:14:09 - SimpleLog - INFO: - episode: 8041, ep_reward: -106.214, ep_step: 77 2023-05-30 04:14:09 - SimpleLog - INFO: - episode: 8042, ep_reward: -101.751, ep_step: 89 2023-05-30 04:14:09 - SimpleLog - INFO: - episode: 8043, ep_reward: -97.773, ep_step: 179 2023-05-30 04:14:09 - SimpleLog - INFO: - episode: 8044, ep_reward: -99.410, ep_step: 81 2023-05-30 04:14:10 - SimpleLog - INFO: - episode: 8045, ep_reward: -72.401, ep_step: 322 2023-05-30 04:14:12 - SimpleLog - INFO: - episode: 8046, ep_reward: 81.995, ep_step: 1000 2023-05-30 04:14:12 - SimpleLog - INFO: - episode: 8047, ep_reward: -104.269, ep_step: 83 2023-05-30 04:14:15 - SimpleLog - INFO: - episode: 8048, ep_reward: 86.536, ep_step: 1000 2023-05-30 04:14:15 - SimpleLog - INFO: - episode: 8049, ep_reward: -106.953, ep_step: 284 2023-05-30 04:14:16 - SimpleLog - INFO: - episode: 8050, ep_reward: -102.335, ep_step: 238 2023-05-30 04:14:18 - SimpleLog - INFO: - episode: 8051, ep_reward: 81.170, ep_step: 1000 2023-05-30 04:14:18 - SimpleLog - INFO: - episode: 8052, ep_reward: -102.234, ep_step: 124 2023-05-30 04:14:19 - SimpleLog - INFO: - episode: 8053, ep_reward: -113.297, ep_step: 361 2023-05-30 04:14:20 - SimpleLog - INFO: - episode: 8054, ep_reward: -102.045, ep_step: 279 2023-05-30 04:14:20 - SimpleLog - INFO: - episode: 8055, ep_reward: -99.189, ep_step: 220 2023-05-30 04:14:21 - SimpleLog - INFO: - episode: 8056, ep_reward: -95.377, ep_step: 185 2023-05-30 04:14:21 - SimpleLog - INFO: - episode: 8057, ep_reward: -100.568, ep_step: 77 2023-05-30 04:14:21 - SimpleLog - INFO: - episode: 8058, ep_reward: -98.716, ep_step: 182 2023-05-30 04:14:22 - SimpleLog - INFO: - episode: 8059, ep_reward: -103.706, ep_step: 243 2023-05-30 04:14:24 - SimpleLog - INFO: - episode: 8060, ep_reward: 14.979, ep_step: 1000 2023-05-30 04:14:26 - SimpleLog - INFO: - episode: 8061, ep_reward: -4.379, ep_step: 1000 2023-05-30 04:14:29 - SimpleLog - INFO: - episode: 8062, ep_reward: 79.612, ep_step: 1000 2023-05-30 04:14:31 - SimpleLog - INFO: - episode: 8063, ep_reward: -4.526, ep_step: 1000 2023-05-30 04:14:34 - SimpleLog - INFO: - update_step: 6440000, online_eval_reward: -60.982 2023-05-30 04:14:35 - SimpleLog - INFO: - episode: 8064, ep_reward: 80.992, ep_step: 1000 2023-05-30 04:14:38 - SimpleLog - INFO: - episode: 8065, ep_reward: 16.135, ep_step: 1000 2023-05-30 04:14:39 - SimpleLog - INFO: - episode: 8066, ep_reward: -94.689, ep_step: 832 2023-05-30 04:14:42 - SimpleLog - INFO: - episode: 8067, ep_reward: 100.680, ep_step: 1000 2023-05-30 04:14:44 - SimpleLog - INFO: - episode: 8068, ep_reward: 68.073, ep_step: 1000 2023-05-30 04:14:46 - SimpleLog - INFO: - episode: 8069, ep_reward: 15.806, ep_step: 1000 2023-05-30 04:14:48 - SimpleLog - INFO: - episode: 8070, ep_reward: 16.447, ep_step: 1000 2023-05-30 04:14:51 - SimpleLog - INFO: - episode: 8071, ep_reward: 15.558, ep_step: 1000 2023-05-30 04:14:53 - SimpleLog - INFO: - episode: 8072, ep_reward: 91.562, ep_step: 1000 2023-05-30 04:14:55 - SimpleLog - INFO: - episode: 8073, ep_reward: 130.337, ep_step: 1000 2023-05-30 04:14:56 - SimpleLog - INFO: - episode: 8074, ep_reward: -51.387, ep_step: 464 2023-05-30 04:14:59 - SimpleLog - INFO: - episode: 8075, ep_reward: 111.442, ep_step: 1000 2023-05-30 04:15:01 - SimpleLog - INFO: - episode: 8076, ep_reward: 96.504, ep_step: 1000 2023-05-30 04:15:03 - SimpleLog - INFO: - episode: 8077, ep_reward: 21.456, ep_step: 955 2023-05-30 04:15:05 - SimpleLog - INFO: - episode: 8078, ep_reward: 122.939, ep_step: 1000 2023-05-30 04:15:08 - SimpleLog - INFO: - episode: 8079, ep_reward: 132.123, ep_step: 1000 2023-05-30 04:15:10 - SimpleLog - INFO: - episode: 8080, ep_reward: 119.308, ep_step: 1000 2023-05-30 04:15:11 - SimpleLog - INFO: - episode: 8081, ep_reward: -55.152, ep_step: 477 2023-05-30 04:15:13 - SimpleLog - INFO: - episode: 8082, ep_reward: 130.056, ep_step: 1000 2023-05-30 04:15:15 - SimpleLog - INFO: - episode: 8083, ep_reward: 115.042, ep_step: 1000 2023-05-30 04:15:18 - SimpleLog - INFO: - episode: 8084, ep_reward: 79.397, ep_step: 1000 2023-05-30 04:15:20 - SimpleLog - INFO: - episode: 8085, ep_reward: 42.864, ep_step: 1000 2023-05-30 04:15:22 - SimpleLog - INFO: - episode: 8086, ep_reward: 110.020, ep_step: 1000 2023-05-30 04:15:24 - SimpleLog - INFO: - episode: 8087, ep_reward: -3.910, ep_step: 970 2023-05-30 04:15:27 - SimpleLog - INFO: - episode: 8088, ep_reward: 112.724, ep_step: 1000 2023-05-30 04:15:29 - SimpleLog - INFO: - episode: 8089, ep_reward: 112.173, ep_step: 1000 2023-05-30 04:15:31 - SimpleLog - INFO: - episode: 8090, ep_reward: 12.649, ep_step: 1000 2023-05-30 04:15:33 - SimpleLog - INFO: - episode: 8091, ep_reward: 100.558, ep_step: 1000 2023-05-30 04:15:36 - SimpleLog - INFO: - episode: 8092, ep_reward: 109.665, ep_step: 1000 2023-05-30 04:15:38 - SimpleLog - INFO: - episode: 8093, ep_reward: 108.730, ep_step: 1000 2023-05-30 04:15:38 - SimpleLog - INFO: - episode: 8094, ep_reward: -126.390, ep_step: 158 2023-05-30 04:15:41 - SimpleLog - INFO: - episode: 8095, ep_reward: 99.113, ep_step: 1000 2023-05-30 04:15:43 - SimpleLog - INFO: - episode: 8096, ep_reward: 99.490, ep_step: 1000 2023-05-30 04:15:45 - SimpleLog - INFO: - episode: 8097, ep_reward: 89.131, ep_step: 1000 2023-05-30 04:15:47 - SimpleLog - INFO: - episode: 8098, ep_reward: 101.779, ep_step: 1000 2023-05-30 04:15:50 - SimpleLog - INFO: - episode: 8099, ep_reward: 106.603, ep_step: 1000 2023-05-30 04:15:52 - SimpleLog - INFO: - episode: 8100, ep_reward: 100.318, ep_step: 1000 2023-05-30 04:15:54 - SimpleLog - INFO: - episode: 8101, ep_reward: 118.200, ep_step: 1000 2023-05-30 04:15:56 - SimpleLog - INFO: - episode: 8102, ep_reward: 102.114, ep_step: 1000 2023-05-30 04:15:59 - SimpleLog - INFO: - episode: 8103, ep_reward: 111.308, ep_step: 1000 2023-05-30 04:16:01 - SimpleLog - INFO: - episode: 8104, ep_reward: 126.494, ep_step: 1000 2023-05-30 04:16:03 - SimpleLog - INFO: - episode: 8105, ep_reward: 98.725, ep_step: 1000 2023-05-30 04:16:05 - SimpleLog - INFO: - episode: 8106, ep_reward: 81.848, ep_step: 1000 2023-05-30 04:16:08 - SimpleLog - INFO: - episode: 8107, ep_reward: 105.539, ep_step: 1000 2023-05-30 04:16:10 - SimpleLog - INFO: - episode: 8108, ep_reward: 6.612, ep_step: 1000 2023-05-30 04:16:12 - SimpleLog - INFO: - episode: 8109, ep_reward: 99.464, ep_step: 1000 2023-05-30 04:16:14 - SimpleLog - INFO: - episode: 8110, ep_reward: 4.030, ep_step: 994 2023-05-30 04:16:17 - SimpleLog - INFO: - episode: 8111, ep_reward: 82.111, ep_step: 1000 2023-05-30 04:16:19 - SimpleLog - INFO: - episode: 8112, ep_reward: -42.053, ep_step: 942 2023-05-30 04:16:21 - SimpleLog - INFO: - episode: 8113, ep_reward: 89.357, ep_step: 1000 2023-05-30 04:16:23 - SimpleLog - INFO: - episode: 8114, ep_reward: 75.437, ep_step: 1000 2023-05-30 04:16:25 - SimpleLog - INFO: - episode: 8115, ep_reward: -45.742, ep_step: 1000 2023-05-30 04:16:28 - SimpleLog - INFO: - episode: 8116, ep_reward: -1.547, ep_step: 979 2023-05-30 04:16:30 - SimpleLog - INFO: - episode: 8117, ep_reward: 74.557, ep_step: 1000 2023-05-30 04:16:32 - SimpleLog - INFO: - episode: 8118, ep_reward: -25.153, ep_step: 825 2023-05-30 04:16:34 - SimpleLog - INFO: - episode: 8119, ep_reward: 108.805, ep_step: 1000 2023-05-30 04:16:36 - SimpleLog - INFO: - episode: 8120, ep_reward: 87.881, ep_step: 1000 2023-05-30 04:16:38 - SimpleLog - INFO: - episode: 8121, ep_reward: 113.732, ep_step: 1000 2023-05-30 04:16:40 - SimpleLog - INFO: - episode: 8122, ep_reward: -86.086, ep_step: 603 2023-05-30 04:16:42 - SimpleLog - INFO: - episode: 8123, ep_reward: 12.583, ep_step: 1000 2023-05-30 04:16:44 - SimpleLog - INFO: - episode: 8124, ep_reward: 16.515, ep_step: 1000 2023-05-30 04:16:47 - SimpleLog - INFO: - episode: 8125, ep_reward: 97.973, ep_step: 1000 2023-05-30 04:16:49 - SimpleLog - INFO: - episode: 8126, ep_reward: 115.431, ep_step: 1000 2023-05-30 04:16:51 - SimpleLog - INFO: - episode: 8127, ep_reward: 20.067, ep_step: 1000 2023-05-30 04:16:53 - SimpleLog - INFO: - episode: 8128, ep_reward: 85.692, ep_step: 1000 2023-05-30 04:16:56 - SimpleLog - INFO: - episode: 8129, ep_reward: 84.651, ep_step: 1000 2023-05-30 04:16:58 - SimpleLog - INFO: - episode: 8130, ep_reward: 80.827, ep_step: 1000 2023-05-30 04:17:00 - SimpleLog - INFO: - episode: 8131, ep_reward: 16.540, ep_step: 1000 2023-05-30 04:17:02 - SimpleLog - INFO: - episode: 8132, ep_reward: -38.069, ep_step: 739 2023-05-30 04:17:04 - SimpleLog - INFO: - episode: 8133, ep_reward: 98.238, ep_step: 1000 2023-05-30 04:17:06 - SimpleLog - INFO: - episode: 8134, ep_reward: 19.401, ep_step: 1000 2023-05-30 04:17:09 - SimpleLog - INFO: - episode: 8135, ep_reward: 19.036, ep_step: 1000 2023-05-30 04:17:11 - SimpleLog - INFO: - episode: 8136, ep_reward: 112.460, ep_step: 1000 2023-05-30 04:17:14 - SimpleLog - INFO: - update_step: 6510000, online_eval_reward: 27.522 2023-05-30 04:17:15 - SimpleLog - INFO: - episode: 8137, ep_reward: -38.760, ep_step: 1000 2023-05-30 04:17:17 - SimpleLog - INFO: - episode: 8138, ep_reward: 113.819, ep_step: 1000 2023-05-30 04:17:19 - SimpleLog - INFO: - episode: 8139, ep_reward: 111.303, ep_step: 1000 2023-05-30 04:17:22 - SimpleLog - INFO: - episode: 8140, ep_reward: 88.325, ep_step: 1000 2023-05-30 04:17:24 - SimpleLog - INFO: - episode: 8141, ep_reward: 87.142, ep_step: 1000 2023-05-30 04:17:26 - SimpleLog - INFO: - episode: 8142, ep_reward: 76.697, ep_step: 1000 2023-05-30 04:17:28 - SimpleLog - INFO: - episode: 8143, ep_reward: 74.670, ep_step: 1000 2023-05-30 04:17:31 - SimpleLog - INFO: - episode: 8144, ep_reward: 117.035, ep_step: 1000 2023-05-30 04:17:33 - SimpleLog - INFO: - episode: 8145, ep_reward: 109.803, ep_step: 1000 2023-05-30 04:17:35 - SimpleLog - INFO: - episode: 8146, ep_reward: 124.972, ep_step: 1000 2023-05-30 04:17:37 - SimpleLog - INFO: - episode: 8147, ep_reward: 104.777, ep_step: 1000 2023-05-30 04:17:40 - SimpleLog - INFO: - episode: 8148, ep_reward: 77.653, ep_step: 1000 2023-05-30 04:17:42 - SimpleLog - INFO: - episode: 8149, ep_reward: 125.935, ep_step: 1000 2023-05-30 04:17:44 - SimpleLog - INFO: - episode: 8150, ep_reward: 106.792, ep_step: 1000 2023-05-30 04:17:46 - SimpleLog - INFO: - episode: 8151, ep_reward: 16.093, ep_step: 1000 2023-05-30 04:17:47 - SimpleLog - INFO: - episode: 8152, ep_reward: -87.722, ep_step: 471 2023-05-30 04:17:50 - SimpleLog - INFO: - episode: 8153, ep_reward: 120.006, ep_step: 1000 2023-05-30 04:17:52 - SimpleLog - INFO: - episode: 8154, ep_reward: 117.601, ep_step: 1000 2023-05-30 04:17:53 - SimpleLog - INFO: - episode: 8155, ep_reward: -59.789, ep_step: 575 2023-05-30 04:17:56 - SimpleLog - INFO: - episode: 8156, ep_reward: 114.365, ep_step: 1000 2023-05-30 04:17:58 - SimpleLog - INFO: - episode: 8157, ep_reward: 107.236, ep_step: 1000 2023-05-30 04:18:00 - SimpleLog - INFO: - episode: 8158, ep_reward: 110.080, ep_step: 1000 2023-05-30 04:18:02 - SimpleLog - INFO: - episode: 8159, ep_reward: 110.125, ep_step: 1000 2023-05-30 04:18:05 - SimpleLog - INFO: - episode: 8160, ep_reward: 102.948, ep_step: 1000 2023-05-30 04:18:07 - SimpleLog - INFO: - episode: 8161, ep_reward: 122.257, ep_step: 1000 2023-05-30 04:18:09 - SimpleLog - INFO: - episode: 8162, ep_reward: 105.492, ep_step: 1000 2023-05-30 04:18:11 - SimpleLog - INFO: - episode: 8163, ep_reward: 104.913, ep_step: 1000 2023-05-30 04:18:14 - SimpleLog - INFO: - episode: 8164, ep_reward: 133.675, ep_step: 1000 2023-05-30 04:18:16 - SimpleLog - INFO: - episode: 8165, ep_reward: 112.073, ep_step: 1000 2023-05-30 04:18:18 - SimpleLog - INFO: - episode: 8166, ep_reward: 109.877, ep_step: 1000 2023-05-30 04:18:20 - SimpleLog - INFO: - episode: 8167, ep_reward: 111.244, ep_step: 1000 2023-05-30 04:18:23 - SimpleLog - INFO: - episode: 8168, ep_reward: 112.365, ep_step: 1000 2023-05-30 04:18:25 - SimpleLog - INFO: - episode: 8169, ep_reward: 20.247, ep_step: 1000 2023-05-30 04:18:27 - SimpleLog - INFO: - episode: 8170, ep_reward: 108.699, ep_step: 1000 2023-05-30 04:18:29 - SimpleLog - INFO: - episode: 8171, ep_reward: 112.575, ep_step: 1000 2023-05-30 04:18:32 - SimpleLog - INFO: - episode: 8172, ep_reward: 3.998, ep_step: 981 2023-05-30 04:18:34 - SimpleLog - INFO: - episode: 8173, ep_reward: 109.742, ep_step: 1000 2023-05-30 04:18:36 - SimpleLog - INFO: - episode: 8174, ep_reward: 8.184, ep_step: 1000 2023-05-30 04:18:37 - SimpleLog - INFO: - episode: 8175, ep_reward: -71.116, ep_step: 521 2023-05-30 04:18:40 - SimpleLog - INFO: - episode: 8176, ep_reward: 13.232, ep_step: 1000 2023-05-30 04:18:42 - SimpleLog - INFO: - episode: 8177, ep_reward: 5.083, ep_step: 1000 2023-05-30 04:18:44 - SimpleLog - INFO: - episode: 8178, ep_reward: 117.819, ep_step: 1000 2023-05-30 04:18:46 - SimpleLog - INFO: - episode: 8179, ep_reward: 112.111, ep_step: 1000 2023-05-30 04:18:48 - SimpleLog - INFO: - episode: 8180, ep_reward: -131.385, ep_step: 586 2023-05-30 04:18:50 - SimpleLog - INFO: - episode: 8181, ep_reward: 128.073, ep_step: 1000 2023-05-30 04:18:52 - SimpleLog - INFO: - episode: 8182, ep_reward: 13.642, ep_step: 1000 2023-05-30 04:18:54 - SimpleLog - INFO: - episode: 8183, ep_reward: 100.748, ep_step: 1000 2023-05-30 04:18:57 - SimpleLog - INFO: - episode: 8184, ep_reward: 45.115, ep_step: 1000 2023-05-30 04:18:59 - SimpleLog - INFO: - episode: 8185, ep_reward: 112.968, ep_step: 1000 2023-05-30 04:19:01 - SimpleLog - INFO: - episode: 8186, ep_reward: 127.978, ep_step: 1000 2023-05-30 04:19:04 - SimpleLog - INFO: - episode: 8187, ep_reward: 130.973, ep_step: 1000 2023-05-30 04:19:05 - SimpleLog - INFO: - episode: 8188, ep_reward: -39.410, ep_step: 624 2023-05-30 04:19:07 - SimpleLog - INFO: - episode: 8189, ep_reward: 139.989, ep_step: 1000 2023-05-30 04:19:09 - SimpleLog - INFO: - episode: 8190, ep_reward: 36.239, ep_step: 1000 2023-05-30 04:19:12 - SimpleLog - INFO: - episode: 8191, ep_reward: 130.353, ep_step: 1000 2023-05-30 04:19:14 - SimpleLog - INFO: - episode: 8192, ep_reward: 118.220, ep_step: 1000 2023-05-30 04:19:16 - SimpleLog - INFO: - episode: 8193, ep_reward: -8.430, ep_step: 820 2023-05-30 04:19:18 - SimpleLog - INFO: - episode: 8194, ep_reward: 6.435, ep_step: 965 2023-05-30 04:19:20 - SimpleLog - INFO: - episode: 8195, ep_reward: -5.162, ep_step: 878 2023-05-30 04:19:22 - SimpleLog - INFO: - episode: 8196, ep_reward: 139.297, ep_step: 1000 2023-05-30 04:19:25 - SimpleLog - INFO: - episode: 8197, ep_reward: 120.582, ep_step: 1000 2023-05-30 04:19:27 - SimpleLog - INFO: - episode: 8198, ep_reward: 135.841, ep_step: 1000 2023-05-30 04:19:29 - SimpleLog - INFO: - episode: 8199, ep_reward: 139.486, ep_step: 1000 2023-05-30 04:19:31 - SimpleLog - INFO: - episode: 8200, ep_reward: 138.436, ep_step: 1000 2023-05-30 04:19:34 - SimpleLog - INFO: - episode: 8201, ep_reward: 132.326, ep_step: 1000 2023-05-30 04:19:35 - SimpleLog - INFO: - episode: 8202, ep_reward: 12.600, ep_step: 828 2023-05-30 04:19:38 - SimpleLog - INFO: - episode: 8203, ep_reward: 86.549, ep_step: 1000 2023-05-30 04:19:40 - SimpleLog - INFO: - episode: 8204, ep_reward: 137.277, ep_step: 1000 2023-05-30 04:19:42 - SimpleLog - INFO: - episode: 8205, ep_reward: 84.703, ep_step: 1000 2023-05-30 04:19:44 - SimpleLog - INFO: - episode: 8206, ep_reward: 119.610, ep_step: 1000 2023-05-30 04:19:47 - SimpleLog - INFO: - episode: 8207, ep_reward: 121.773, ep_step: 1000 2023-05-30 04:19:49 - SimpleLog - INFO: - episode: 8208, ep_reward: -27.682, ep_step: 845 2023-05-30 04:19:49 - SimpleLog - INFO: - episode: 8209, ep_reward: -121.410, ep_step: 71 2023-05-30 04:19:49 - SimpleLog - INFO: - episode: 8210, ep_reward: -123.982, ep_step: 64 2023-05-30 04:19:49 - SimpleLog - INFO: - episode: 8211, ep_reward: -124.176, ep_step: 73 2023-05-30 04:19:49 - SimpleLog - INFO: - episode: 8212, ep_reward: -126.275, ep_step: 71 2023-05-30 04:19:51 - SimpleLog - INFO: - episode: 8213, ep_reward: 112.436, ep_step: 1000 2023-05-30 04:19:54 - SimpleLog - INFO: - update_step: 6580000, online_eval_reward: -56.401 2023-05-30 04:19:56 - SimpleLog - INFO: - episode: 8214, ep_reward: 86.187, ep_step: 1000 2023-05-30 04:19:58 - SimpleLog - INFO: - episode: 8215, ep_reward: 72.681, ep_step: 1000 2023-05-30 04:20:00 - SimpleLog - INFO: - episode: 8216, ep_reward: 82.965, ep_step: 1000 2023-05-30 04:20:02 - SimpleLog - INFO: - episode: 8217, ep_reward: 71.259, ep_step: 1000 2023-05-30 04:20:03 - SimpleLog - INFO: - episode: 8218, ep_reward: -144.362, ep_step: 307 2023-05-30 04:20:05 - SimpleLog - INFO: - episode: 8219, ep_reward: 8.536, ep_step: 1000 2023-05-30 04:20:06 - SimpleLog - INFO: - episode: 8220, ep_reward: -74.043, ep_step: 386 2023-05-30 04:20:08 - SimpleLog - INFO: - episode: 8221, ep_reward: 82.307, ep_step: 1000 2023-05-30 04:20:11 - SimpleLog - INFO: - episode: 8222, ep_reward: 114.489, ep_step: 1000 2023-05-30 04:20:13 - SimpleLog - INFO: - episode: 8223, ep_reward: 92.192, ep_step: 1000 2023-05-30 04:20:15 - SimpleLog - INFO: - episode: 8224, ep_reward: 132.653, ep_step: 1000 2023-05-30 04:20:17 - SimpleLog - INFO: - episode: 8225, ep_reward: -49.710, ep_step: 1000 2023-05-30 04:20:20 - SimpleLog - INFO: - episode: 8226, ep_reward: -34.257, ep_step: 1000 2023-05-30 04:20:22 - SimpleLog - INFO: - episode: 8227, ep_reward: 14.478, ep_step: 1000 2023-05-30 04:20:24 - SimpleLog - INFO: - episode: 8228, ep_reward: 113.859, ep_step: 1000 2023-05-30 04:20:25 - SimpleLog - INFO: - episode: 8229, ep_reward: -148.929, ep_step: 504 2023-05-30 04:20:28 - SimpleLog - INFO: - episode: 8230, ep_reward: 2.680, ep_step: 1000 2023-05-30 04:20:30 - SimpleLog - INFO: - episode: 8231, ep_reward: 109.470, ep_step: 1000 2023-05-30 04:20:32 - SimpleLog - INFO: - episode: 8232, ep_reward: 119.602, ep_step: 1000 2023-05-30 04:20:34 - SimpleLog - INFO: - episode: 8233, ep_reward: 118.292, ep_step: 1000 2023-05-30 04:20:37 - SimpleLog - INFO: - episode: 8234, ep_reward: 120.330, ep_step: 1000 2023-05-30 04:20:39 - SimpleLog - INFO: - episode: 8235, ep_reward: 127.763, ep_step: 1000 2023-05-30 04:20:41 - SimpleLog - INFO: - episode: 8236, ep_reward: 121.024, ep_step: 1000 2023-05-30 04:20:43 - SimpleLog - INFO: - episode: 8237, ep_reward: 117.048, ep_step: 1000 2023-05-30 04:20:45 - SimpleLog - INFO: - episode: 8238, ep_reward: -79.640, ep_step: 548 2023-05-30 04:20:47 - SimpleLog - INFO: - episode: 8239, ep_reward: 110.382, ep_step: 1000 2023-05-30 04:20:49 - SimpleLog - INFO: - episode: 8240, ep_reward: 117.767, ep_step: 1000 2023-05-30 04:20:50 - SimpleLog - INFO: - episode: 8241, ep_reward: -131.517, ep_step: 230 2023-05-30 04:20:52 - SimpleLog - INFO: - episode: 8242, ep_reward: 119.989, ep_step: 1000 2023-05-30 04:20:54 - SimpleLog - INFO: - episode: 8243, ep_reward: -104.757, ep_step: 1000 2023-05-30 04:20:57 - SimpleLog - INFO: - episode: 8244, ep_reward: 134.487, ep_step: 1000 2023-05-30 04:20:59 - SimpleLog - INFO: - episode: 8245, ep_reward: 95.832, ep_step: 1000 2023-05-30 04:21:01 - SimpleLog - INFO: - episode: 8246, ep_reward: 119.690, ep_step: 1000 2023-05-30 04:21:03 - SimpleLog - INFO: - episode: 8247, ep_reward: 10.093, ep_step: 1000 2023-05-30 04:21:06 - SimpleLog - INFO: - episode: 8248, ep_reward: 10.085, ep_step: 1000 2023-05-30 04:21:08 - SimpleLog - INFO: - episode: 8249, ep_reward: 112.065, ep_step: 1000 2023-05-30 04:21:10 - SimpleLog - INFO: - episode: 8250, ep_reward: 102.587, ep_step: 1000 2023-05-30 04:21:12 - SimpleLog - INFO: - episode: 8251, ep_reward: 4.854, ep_step: 869 2023-05-30 04:21:14 - SimpleLog - INFO: - episode: 8252, ep_reward: 2.799, ep_step: 997 2023-05-30 04:21:17 - SimpleLog - INFO: - episode: 8253, ep_reward: 94.504, ep_step: 1000 2023-05-30 04:21:19 - SimpleLog - INFO: - episode: 8254, ep_reward: 84.778, ep_step: 1000 2023-05-30 04:21:21 - SimpleLog - INFO: - episode: 8255, ep_reward: 105.883, ep_step: 1000 2023-05-30 04:21:23 - SimpleLog - INFO: - episode: 8256, ep_reward: 132.636, ep_step: 1000 2023-05-30 04:21:26 - SimpleLog - INFO: - episode: 8257, ep_reward: 76.338, ep_step: 1000 2023-05-30 04:21:28 - SimpleLog - INFO: - episode: 8258, ep_reward: 21.023, ep_step: 863 2023-05-30 04:21:30 - SimpleLog - INFO: - episode: 8259, ep_reward: 137.809, ep_step: 1000 2023-05-30 04:21:32 - SimpleLog - INFO: - episode: 8260, ep_reward: 139.071, ep_step: 1000 2023-05-30 04:21:34 - SimpleLog - INFO: - episode: 8261, ep_reward: -88.468, ep_step: 1000 2023-05-30 04:21:36 - SimpleLog - INFO: - episode: 8262, ep_reward: -38.364, ep_step: 543 2023-05-30 04:21:37 - SimpleLog - INFO: - episode: 8263, ep_reward: -5.354, ep_step: 769 2023-05-30 04:21:39 - SimpleLog - INFO: - episode: 8264, ep_reward: -38.380, ep_step: 593 2023-05-30 04:21:39 - SimpleLog - INFO: - episode: 8265, ep_reward: -118.613, ep_step: 85 2023-05-30 04:21:41 - SimpleLog - INFO: - episode: 8266, ep_reward: -1.386, ep_step: 951 2023-05-30 04:21:43 - SimpleLog - INFO: - episode: 8267, ep_reward: 107.159, ep_step: 1000 2023-05-30 04:21:44 - SimpleLog - INFO: - episode: 8268, ep_reward: -96.858, ep_step: 227 2023-05-30 04:21:46 - SimpleLog - INFO: - episode: 8269, ep_reward: 111.638, ep_step: 1000 2023-05-30 04:21:48 - SimpleLog - INFO: - episode: 8270, ep_reward: 98.575, ep_step: 1000 2023-05-30 04:21:51 - SimpleLog - INFO: - episode: 8271, ep_reward: 87.043, ep_step: 1000 2023-05-30 04:21:53 - SimpleLog - INFO: - episode: 8272, ep_reward: 6.537, ep_step: 898 2023-05-30 04:21:55 - SimpleLog - INFO: - episode: 8273, ep_reward: -38.381, ep_step: 992 2023-05-30 04:21:57 - SimpleLog - INFO: - episode: 8274, ep_reward: 36.320, ep_step: 985 2023-05-30 04:21:58 - SimpleLog - INFO: - episode: 8275, ep_reward: -119.310, ep_step: 501 2023-05-30 04:22:00 - SimpleLog - INFO: - episode: 8276, ep_reward: 105.742, ep_step: 1000 2023-05-30 04:22:03 - SimpleLog - INFO: - episode: 8277, ep_reward: 85.279, ep_step: 1000 2023-05-30 04:22:05 - SimpleLog - INFO: - episode: 8278, ep_reward: 109.714, ep_step: 1000 2023-05-30 04:22:07 - SimpleLog - INFO: - episode: 8279, ep_reward: 89.086, ep_step: 1000 2023-05-30 04:22:09 - SimpleLog - INFO: - episode: 8280, ep_reward: 125.542, ep_step: 1000 2023-05-30 04:22:12 - SimpleLog - INFO: - episode: 8281, ep_reward: 90.827, ep_step: 1000 2023-05-30 04:22:14 - SimpleLog - INFO: - episode: 8282, ep_reward: 108.346, ep_step: 1000 2023-05-30 04:22:16 - SimpleLog - INFO: - episode: 8283, ep_reward: -11.302, ep_step: 889 2023-05-30 04:22:18 - SimpleLog - INFO: - episode: 8284, ep_reward: 88.132, ep_step: 1000 2023-05-30 04:22:20 - SimpleLog - INFO: - episode: 8285, ep_reward: 76.152, ep_step: 1000 2023-05-30 04:22:23 - SimpleLog - INFO: - episode: 8286, ep_reward: 41.101, ep_step: 1000 2023-05-30 04:22:25 - SimpleLog - INFO: - episode: 8287, ep_reward: 87.832, ep_step: 1000 2023-05-30 04:22:27 - SimpleLog - INFO: - episode: 8288, ep_reward: 127.928, ep_step: 1000 2023-05-30 04:22:29 - SimpleLog - INFO: - episode: 8289, ep_reward: 69.471, ep_step: 1000 2023-05-30 04:22:32 - SimpleLog - INFO: - episode: 8290, ep_reward: 110.659, ep_step: 1000 2023-05-30 04:22:34 - SimpleLog - INFO: - update_step: 6650000, online_eval_reward: 91.705 2023-05-30 04:22:36 - SimpleLog - INFO: - episode: 8291, ep_reward: 121.481, ep_step: 1000 2023-05-30 04:22:38 - SimpleLog - INFO: - episode: 8292, ep_reward: -12.449, ep_step: 908 2023-05-30 04:22:40 - SimpleLog - INFO: - episode: 8293, ep_reward: 76.819, ep_step: 1000 2023-05-30 04:22:42 - SimpleLog - INFO: - episode: 8294, ep_reward: 75.086, ep_step: 1000 2023-05-30 04:22:43 - SimpleLog - INFO: - episode: 8295, ep_reward: -93.912, ep_step: 395 2023-05-30 04:22:45 - SimpleLog - INFO: - episode: 8296, ep_reward: -6.699, ep_step: 884 2023-05-30 04:22:47 - SimpleLog - INFO: - episode: 8297, ep_reward: 6.020, ep_step: 1000 2023-05-30 04:22:50 - SimpleLog - INFO: - episode: 8298, ep_reward: 73.433, ep_step: 1000 2023-05-30 04:22:51 - SimpleLog - INFO: - episode: 8299, ep_reward: -3.617, ep_step: 691 2023-05-30 04:22:53 - SimpleLog - INFO: - episode: 8300, ep_reward: -13.586, ep_step: 764 2023-05-30 04:22:55 - SimpleLog - INFO: - episode: 8301, ep_reward: -29.270, ep_step: 835 2023-05-30 04:22:57 - SimpleLog - INFO: - episode: 8302, ep_reward: 84.749, ep_step: 1000 2023-05-30 04:22:59 - SimpleLog - INFO: - episode: 8303, ep_reward: 140.634, ep_step: 1000 2023-05-30 04:23:02 - SimpleLog - INFO: - episode: 8304, ep_reward: 118.657, ep_step: 1000 2023-05-30 04:23:03 - SimpleLog - INFO: - episode: 8305, ep_reward: 0.544, ep_step: 782 2023-05-30 04:23:05 - SimpleLog - INFO: - episode: 8306, ep_reward: -55.134, ep_step: 541 2023-05-30 04:23:07 - SimpleLog - INFO: - episode: 8307, ep_reward: 84.632, ep_step: 1000 2023-05-30 04:23:09 - SimpleLog - INFO: - episode: 8308, ep_reward: 121.816, ep_step: 1000 2023-05-30 04:23:11 - SimpleLog - INFO: - episode: 8309, ep_reward: 86.313, ep_step: 1000 2023-05-30 04:23:13 - SimpleLog - INFO: - episode: 8310, ep_reward: -3.001, ep_step: 759 2023-05-30 04:23:15 - SimpleLog - INFO: - episode: 8311, ep_reward: 116.148, ep_step: 1000 2023-05-30 04:23:17 - SimpleLog - INFO: - episode: 8312, ep_reward: 12.125, ep_step: 914 2023-05-30 04:23:20 - SimpleLog - INFO: - episode: 8313, ep_reward: -51.889, ep_step: 1000 2023-05-30 04:23:22 - SimpleLog - INFO: - episode: 8314, ep_reward: -44.144, ep_step: 931 2023-05-30 04:23:24 - SimpleLog - INFO: - episode: 8315, ep_reward: 106.528, ep_step: 1000 2023-05-30 04:23:26 - SimpleLog - INFO: - episode: 8316, ep_reward: 104.048, ep_step: 1000 2023-05-30 04:23:28 - SimpleLog - INFO: - episode: 8317, ep_reward: 84.247, ep_step: 1000 2023-05-30 04:23:31 - SimpleLog - INFO: - episode: 8318, ep_reward: 87.538, ep_step: 1000 2023-05-30 04:23:33 - SimpleLog - INFO: - episode: 8319, ep_reward: 82.230, ep_step: 1000 2023-05-30 04:23:35 - SimpleLog - INFO: - episode: 8320, ep_reward: 133.643, ep_step: 1000 2023-05-30 04:23:37 - SimpleLog - INFO: - episode: 8321, ep_reward: 72.613, ep_step: 1000 2023-05-30 04:23:40 - SimpleLog - INFO: - episode: 8322, ep_reward: -49.157, ep_step: 1000 2023-05-30 04:23:42 - SimpleLog - INFO: - episode: 8323, ep_reward: 133.051, ep_step: 1000 2023-05-30 04:23:44 - SimpleLog - INFO: - episode: 8324, ep_reward: 2.543, ep_step: 953 2023-05-30 04:23:46 - SimpleLog - INFO: - episode: 8325, ep_reward: 123.532, ep_step: 1000 2023-05-30 04:23:49 - SimpleLog - INFO: - episode: 8326, ep_reward: 3.703, ep_step: 978 2023-05-30 04:23:51 - SimpleLog - INFO: - episode: 8327, ep_reward: 84.868, ep_step: 1000 2023-05-30 04:23:53 - SimpleLog - INFO: - episode: 8328, ep_reward: 86.055, ep_step: 1000 2023-05-30 04:23:55 - SimpleLog - INFO: - episode: 8329, ep_reward: 77.637, ep_step: 1000 2023-05-30 04:23:58 - SimpleLog - INFO: - episode: 8330, ep_reward: 76.091, ep_step: 1000 2023-05-30 04:24:00 - SimpleLog - INFO: - episode: 8331, ep_reward: 81.306, ep_step: 1000 2023-05-30 04:24:02 - SimpleLog - INFO: - episode: 8332, ep_reward: 102.522, ep_step: 1000 2023-05-30 04:24:04 - SimpleLog - INFO: - episode: 8333, ep_reward: 110.107, ep_step: 1000 2023-05-30 04:24:07 - SimpleLog - INFO: - episode: 8334, ep_reward: 107.662, ep_step: 1000 2023-05-30 04:24:09 - SimpleLog - INFO: - episode: 8335, ep_reward: 11.807, ep_step: 1000 2023-05-30 04:24:11 - SimpleLog - INFO: - episode: 8336, ep_reward: 95.426, ep_step: 1000 2023-05-30 04:24:13 - SimpleLog - INFO: - episode: 8337, ep_reward: 102.358, ep_step: 1000 2023-05-30 04:24:16 - SimpleLog - INFO: - episode: 8338, ep_reward: 110.097, ep_step: 1000 2023-05-30 04:24:18 - SimpleLog - INFO: - episode: 8339, ep_reward: 102.520, ep_step: 1000 2023-05-30 04:24:20 - SimpleLog - INFO: - episode: 8340, ep_reward: 10.884, ep_step: 1000 2023-05-30 04:24:22 - SimpleLog - INFO: - episode: 8341, ep_reward: 114.016, ep_step: 1000 2023-05-30 04:24:23 - SimpleLog - INFO: - episode: 8342, ep_reward: -89.390, ep_step: 286 2023-05-30 04:24:25 - SimpleLog - INFO: - episode: 8343, ep_reward: 116.369, ep_step: 1000 2023-05-30 04:24:27 - SimpleLog - INFO: - episode: 8344, ep_reward: -55.635, ep_step: 1000 2023-05-30 04:24:30 - SimpleLog - INFO: - episode: 8345, ep_reward: 10.937, ep_step: 948 2023-05-30 04:24:30 - SimpleLog - INFO: - episode: 8346, ep_reward: -100.285, ep_step: 378 2023-05-30 04:24:33 - SimpleLog - INFO: - episode: 8347, ep_reward: 82.265, ep_step: 1000 2023-05-30 04:24:35 - SimpleLog - INFO: - episode: 8348, ep_reward: 86.203, ep_step: 1000 2023-05-30 04:24:37 - SimpleLog - INFO: - episode: 8349, ep_reward: 5.191, ep_step: 923 2023-05-30 04:24:39 - SimpleLog - INFO: - episode: 8350, ep_reward: 132.859, ep_step: 1000 2023-05-30 04:24:42 - SimpleLog - INFO: - episode: 8351, ep_reward: 81.783, ep_step: 1000 2023-05-30 04:24:44 - SimpleLog - INFO: - episode: 8352, ep_reward: 8.159, ep_step: 1000 2023-05-30 04:24:46 - SimpleLog - INFO: - episode: 8353, ep_reward: 79.816, ep_step: 1000 2023-05-30 04:24:48 - SimpleLog - INFO: - episode: 8354, ep_reward: -3.676, ep_step: 917 2023-05-30 04:24:50 - SimpleLog - INFO: - episode: 8355, ep_reward: 1.245, ep_step: 1000 2023-05-30 04:24:51 - SimpleLog - INFO: - episode: 8356, ep_reward: -101.725, ep_step: 193 2023-05-30 04:24:52 - SimpleLog - INFO: - episode: 8357, ep_reward: -24.985, ep_step: 676 2023-05-30 04:24:53 - SimpleLog - INFO: - episode: 8358, ep_reward: -98.900, ep_step: 445 2023-05-30 04:24:56 - SimpleLog - INFO: - episode: 8359, ep_reward: 169.298, ep_step: 1000 2023-05-30 04:24:57 - SimpleLog - INFO: - episode: 8360, ep_reward: -114.631, ep_step: 425 2023-05-30 04:24:59 - SimpleLog - INFO: - episode: 8361, ep_reward: 153.330, ep_step: 1000 2023-05-30 04:24:59 - SimpleLog - INFO: - episode: 8362, ep_reward: -100.777, ep_step: 200 2023-05-30 04:25:01 - SimpleLog - INFO: - episode: 8363, ep_reward: 8.882, ep_step: 806 2023-05-30 04:25:03 - SimpleLog - INFO: - episode: 8364, ep_reward: 16.997, ep_step: 821 2023-05-30 04:25:03 - SimpleLog - INFO: - episode: 8365, ep_reward: -119.893, ep_step: 115 2023-05-30 04:25:05 - SimpleLog - INFO: - episode: 8366, ep_reward: 111.422, ep_step: 1000 2023-05-30 04:25:08 - SimpleLog - INFO: - episode: 8367, ep_reward: 9.343, ep_step: 1000 2023-05-30 04:25:10 - SimpleLog - INFO: - episode: 8368, ep_reward: 9.053, ep_step: 1000 2023-05-30 04:25:13 - SimpleLog - INFO: - update_step: 6720000, online_eval_reward: -1.930 2023-05-30 04:25:14 - SimpleLog - INFO: - episode: 8369, ep_reward: 142.037, ep_step: 1000 2023-05-30 04:25:16 - SimpleLog - INFO: - episode: 8370, ep_reward: 131.402, ep_step: 1000 2023-05-30 04:25:18 - SimpleLog - INFO: - episode: 8371, ep_reward: 105.359, ep_step: 1000 2023-05-30 04:25:19 - SimpleLog - INFO: - episode: 8372, ep_reward: -61.756, ep_step: 414 2023-05-30 04:25:21 - SimpleLog - INFO: - episode: 8373, ep_reward: 101.686, ep_step: 1000 2023-05-30 04:25:23 - SimpleLog - INFO: - episode: 8374, ep_reward: 103.917, ep_step: 1000 2023-05-30 04:25:25 - SimpleLog - INFO: - episode: 8375, ep_reward: 12.340, ep_step: 845 2023-05-30 04:25:28 - SimpleLog - INFO: - episode: 8376, ep_reward: 103.477, ep_step: 1000 2023-05-30 04:25:30 - SimpleLog - INFO: - episode: 8377, ep_reward: 109.902, ep_step: 1000 2023-05-30 04:25:32 - SimpleLog - INFO: - episode: 8378, ep_reward: 5.735, ep_step: 913 2023-05-30 04:25:34 - SimpleLog - INFO: - episode: 8379, ep_reward: 113.209, ep_step: 1000 2023-05-30 04:25:36 - SimpleLog - INFO: - episode: 8380, ep_reward: 100.350, ep_step: 1000 2023-05-30 04:25:39 - SimpleLog - INFO: - episode: 8381, ep_reward: 57.121, ep_step: 1000 2023-05-30 04:25:40 - SimpleLog - INFO: - episode: 8382, ep_reward: -74.771, ep_step: 520 2023-05-30 04:25:42 - SimpleLog - INFO: - episode: 8383, ep_reward: 130.077, ep_step: 1000 2023-05-30 04:25:44 - SimpleLog - INFO: - episode: 8384, ep_reward: 17.001, ep_step: 822 2023-05-30 04:25:46 - SimpleLog - INFO: - episode: 8385, ep_reward: 15.706, ep_step: 774 2023-05-30 04:25:47 - SimpleLog - INFO: - episode: 8386, ep_reward: -26.687, ep_step: 661 2023-05-30 04:25:49 - SimpleLog - INFO: - episode: 8387, ep_reward: 127.009, ep_step: 1000 2023-05-30 04:25:52 - SimpleLog - INFO: - episode: 8388, ep_reward: 141.955, ep_step: 1000 2023-05-30 04:25:54 - SimpleLog - INFO: - episode: 8389, ep_reward: 128.485, ep_step: 1000 2023-05-30 04:25:56 - SimpleLog - INFO: - episode: 8390, ep_reward: 17.041, ep_step: 931 2023-05-30 04:25:58 - SimpleLog - INFO: - episode: 8391, ep_reward: 139.810, ep_step: 1000 2023-05-30 04:26:01 - SimpleLog - INFO: - episode: 8392, ep_reward: 120.024, ep_step: 1000 2023-05-30 04:26:03 - SimpleLog - INFO: - episode: 8393, ep_reward: 91.184, ep_step: 1000 2023-05-30 04:26:05 - SimpleLog - INFO: - episode: 8394, ep_reward: 128.107, ep_step: 1000 2023-05-30 04:26:07 - SimpleLog - INFO: - episode: 8395, ep_reward: 75.655, ep_step: 1000 2023-05-30 04:26:10 - SimpleLog - INFO: - episode: 8396, ep_reward: 83.379, ep_step: 1000 2023-05-30 04:26:12 - SimpleLog - INFO: - episode: 8397, ep_reward: 61.306, ep_step: 1000 2023-05-30 04:26:14 - SimpleLog - INFO: - episode: 8398, ep_reward: 81.535, ep_step: 1000 2023-05-30 04:26:16 - SimpleLog - INFO: - episode: 8399, ep_reward: 133.666, ep_step: 1000 2023-05-30 04:26:19 - SimpleLog - INFO: - episode: 8400, ep_reward: 111.854, ep_step: 1000 2023-05-30 04:26:21 - SimpleLog - INFO: - episode: 8401, ep_reward: 85.910, ep_step: 1000 2023-05-30 04:26:23 - SimpleLog - INFO: - episode: 8402, ep_reward: 107.012, ep_step: 1000 2023-05-30 04:26:24 - SimpleLog - INFO: - episode: 8403, ep_reward: -95.084, ep_step: 234 2023-05-30 04:26:26 - SimpleLog - INFO: - episode: 8404, ep_reward: 107.280, ep_step: 1000 2023-05-30 04:26:28 - SimpleLog - INFO: - episode: 8405, ep_reward: 104.459, ep_step: 1000 2023-05-30 04:26:30 - SimpleLog - INFO: - episode: 8406, ep_reward: -3.125, ep_step: 1000 2023-05-30 04:26:33 - SimpleLog - INFO: - episode: 8407, ep_reward: 112.494, ep_step: 1000 2023-05-30 04:26:35 - SimpleLog - INFO: - episode: 8408, ep_reward: 133.433, ep_step: 1000 2023-05-30 04:26:37 - SimpleLog - INFO: - episode: 8409, ep_reward: -19.447, ep_step: 975 2023-05-30 04:26:39 - SimpleLog - INFO: - episode: 8410, ep_reward: 133.581, ep_step: 1000 2023-05-30 04:26:41 - SimpleLog - INFO: - episode: 8411, ep_reward: 7.484, ep_step: 846 2023-05-30 04:26:43 - SimpleLog - INFO: - episode: 8412, ep_reward: 7.053, ep_step: 954 2023-05-30 04:26:45 - SimpleLog - INFO: - episode: 8413, ep_reward: 9.579, ep_step: 862 2023-05-30 04:26:48 - SimpleLog - INFO: - episode: 8414, ep_reward: 148.257, ep_step: 1000 2023-05-30 04:26:50 - SimpleLog - INFO: - episode: 8415, ep_reward: 120.488, ep_step: 1000 2023-05-30 04:26:52 - SimpleLog - INFO: - episode: 8416, ep_reward: 138.359, ep_step: 1000 2023-05-30 04:26:54 - SimpleLog - INFO: - episode: 8417, ep_reward: 127.157, ep_step: 1000 2023-05-30 04:26:56 - SimpleLog - INFO: - episode: 8418, ep_reward: -5.107, ep_step: 850 2023-05-30 04:26:58 - SimpleLog - INFO: - episode: 8419, ep_reward: 16.788, ep_step: 833 2023-05-30 04:27:00 - SimpleLog - INFO: - episode: 8420, ep_reward: 6.240, ep_step: 1000 2023-05-30 04:27:02 - SimpleLog - INFO: - episode: 8421, ep_reward: -32.853, ep_step: 877 2023-05-30 04:27:04 - SimpleLog - INFO: - episode: 8422, ep_reward: -8.561, ep_step: 856 2023-05-30 04:27:06 - SimpleLog - INFO: - episode: 8423, ep_reward: -11.360, ep_step: 913 2023-05-30 04:27:08 - SimpleLog - INFO: - episode: 8424, ep_reward: 86.179, ep_step: 1000 2023-05-30 04:27:11 - SimpleLog - INFO: - episode: 8425, ep_reward: 128.591, ep_step: 1000 2023-05-30 04:27:13 - SimpleLog - INFO: - episode: 8426, ep_reward: 134.339, ep_step: 1000 2023-05-30 04:27:15 - SimpleLog - INFO: - episode: 8427, ep_reward: 146.924, ep_step: 1000 2023-05-30 04:27:17 - SimpleLog - INFO: - episode: 8428, ep_reward: -12.236, ep_step: 876 2023-05-30 04:27:19 - SimpleLog - INFO: - episode: 8429, ep_reward: 138.167, ep_step: 1000 2023-05-30 04:27:21 - SimpleLog - INFO: - episode: 8430, ep_reward: -7.034, ep_step: 779 2023-05-30 04:27:23 - SimpleLog - INFO: - episode: 8431, ep_reward: 84.208, ep_step: 1000 2023-05-30 04:27:26 - SimpleLog - INFO: - episode: 8432, ep_reward: -33.718, ep_step: 1000 2023-05-30 04:27:28 - SimpleLog - INFO: - episode: 8433, ep_reward: 130.227, ep_step: 1000 2023-05-30 04:27:30 - SimpleLog - INFO: - episode: 8434, ep_reward: 78.953, ep_step: 1000 2023-05-30 04:27:32 - SimpleLog - INFO: - episode: 8435, ep_reward: 80.371, ep_step: 1000 2023-05-30 04:27:35 - SimpleLog - INFO: - episode: 8436, ep_reward: 80.875, ep_step: 1000 2023-05-30 04:27:37 - SimpleLog - INFO: - episode: 8437, ep_reward: 4.529, ep_step: 1000 2023-05-30 04:27:39 - SimpleLog - INFO: - episode: 8438, ep_reward: 68.573, ep_step: 1000 2023-05-30 04:27:42 - SimpleLog - INFO: - episode: 8439, ep_reward: 78.732, ep_step: 1000 2023-05-30 04:27:42 - SimpleLog - INFO: - episode: 8440, ep_reward: -75.168, ep_step: 428 2023-05-30 04:27:45 - SimpleLog - INFO: - episode: 8441, ep_reward: 81.737, ep_step: 1000 2023-05-30 04:27:47 - SimpleLog - INFO: - episode: 8442, ep_reward: 65.933, ep_step: 1000 2023-05-30 04:27:49 - SimpleLog - INFO: - episode: 8443, ep_reward: 80.200, ep_step: 1000 2023-05-30 04:27:52 - SimpleLog - INFO: - update_step: 6790000, online_eval_reward: 122.881 2023-05-30 04:27:53 - SimpleLog - INFO: - episode: 8444, ep_reward: -87.025, ep_step: 1000 2023-05-30 04:27:56 - SimpleLog - INFO: - episode: 8445, ep_reward: 158.942, ep_step: 1000 2023-05-30 04:27:58 - SimpleLog - INFO: - episode: 8446, ep_reward: 76.030, ep_step: 1000 2023-05-30 04:28:00 - SimpleLog - INFO: - episode: 8447, ep_reward: 147.007, ep_step: 1000 2023-05-30 04:28:02 - SimpleLog - INFO: - episode: 8448, ep_reward: 133.956, ep_step: 1000 2023-05-30 04:28:04 - SimpleLog - INFO: - episode: 8449, ep_reward: 16.363, ep_step: 825 2023-05-30 04:28:07 - SimpleLog - INFO: - episode: 8450, ep_reward: 133.001, ep_step: 1000 2023-05-30 04:28:09 - SimpleLog - INFO: - episode: 8451, ep_reward: 151.904, ep_step: 1000 2023-05-30 04:28:09 - SimpleLog - INFO: - episode: 8452, ep_reward: -68.929, ep_step: 312 2023-05-30 04:28:10 - SimpleLog - INFO: - episode: 8453, ep_reward: -67.428, ep_step: 313 2023-05-30 04:28:12 - SimpleLog - INFO: - episode: 8454, ep_reward: 146.388, ep_step: 1000 2023-05-30 04:28:14 - SimpleLog - INFO: - episode: 8455, ep_reward: -10.055, ep_step: 810 2023-05-30 04:28:16 - SimpleLog - INFO: - episode: 8456, ep_reward: 11.805, ep_step: 856 2023-05-30 04:28:18 - SimpleLog - INFO: - episode: 8457, ep_reward: 140.130, ep_step: 1000 2023-05-30 04:28:21 - SimpleLog - INFO: - episode: 8458, ep_reward: 148.434, ep_step: 1000 2023-05-30 04:28:23 - SimpleLog - INFO: - episode: 8459, ep_reward: 137.235, ep_step: 1000 2023-05-30 04:28:25 - SimpleLog - INFO: - episode: 8460, ep_reward: 141.159, ep_step: 1000 2023-05-30 04:28:26 - SimpleLog - INFO: - episode: 8461, ep_reward: -71.908, ep_step: 311 2023-05-30 04:28:28 - SimpleLog - INFO: - episode: 8462, ep_reward: 9.445, ep_step: 883 2023-05-30 04:28:30 - SimpleLog - INFO: - episode: 8463, ep_reward: 4.418, ep_step: 769 2023-05-30 04:28:31 - SimpleLog - INFO: - episode: 8464, ep_reward: -1.534, ep_step: 770 2023-05-30 04:28:34 - SimpleLog - INFO: - episode: 8465, ep_reward: 137.938, ep_step: 1000 2023-05-30 04:28:36 - SimpleLog - INFO: - episode: 8466, ep_reward: 104.114, ep_step: 1000 2023-05-30 04:28:38 - SimpleLog - INFO: - episode: 8467, ep_reward: 5.940, ep_step: 1000 2023-05-30 04:28:40 - SimpleLog - INFO: - episode: 8468, ep_reward: 131.295, ep_step: 1000 2023-05-30 04:28:42 - SimpleLog - INFO: - episode: 8469, ep_reward: -4.692, ep_step: 899 2023-05-30 04:28:45 - SimpleLog - INFO: - episode: 8470, ep_reward: 103.031, ep_step: 1000 2023-05-30 04:28:45 - SimpleLog - INFO: - episode: 8471, ep_reward: -74.448, ep_step: 313 2023-05-30 04:28:48 - SimpleLog - INFO: - episode: 8472, ep_reward: 74.312, ep_step: 1000 2023-05-30 04:28:49 - SimpleLog - INFO: - episode: 8473, ep_reward: -22.105, ep_step: 789 2023-05-30 04:28:52 - SimpleLog - INFO: - episode: 8474, ep_reward: 132.177, ep_step: 1000 2023-05-30 04:28:53 - SimpleLog - INFO: - episode: 8475, ep_reward: -41.003, ep_step: 628 2023-05-30 04:28:55 - SimpleLog - INFO: - episode: 8476, ep_reward: 134.554, ep_step: 1000 2023-05-30 04:28:58 - SimpleLog - INFO: - episode: 8477, ep_reward: 111.653, ep_step: 1000 2023-05-30 04:29:00 - SimpleLog - INFO: - episode: 8478, ep_reward: 140.578, ep_step: 1000 2023-05-30 04:29:02 - SimpleLog - INFO: - episode: 8479, ep_reward: 132.428, ep_step: 1000 2023-05-30 04:29:04 - SimpleLog - INFO: - episode: 8480, ep_reward: 145.114, ep_step: 1000 2023-05-30 04:29:06 - SimpleLog - INFO: - episode: 8481, ep_reward: 6.575, ep_step: 852 2023-05-30 04:29:09 - SimpleLog - INFO: - episode: 8482, ep_reward: 141.716, ep_step: 1000 2023-05-30 04:29:11 - SimpleLog - INFO: - episode: 8483, ep_reward: 135.408, ep_step: 1000 2023-05-30 04:29:13 - SimpleLog - INFO: - episode: 8484, ep_reward: 111.870, ep_step: 1000 2023-05-30 04:29:15 - SimpleLog - INFO: - episode: 8485, ep_reward: 126.330, ep_step: 1000 2023-05-30 04:29:18 - SimpleLog - INFO: - episode: 8486, ep_reward: 138.389, ep_step: 1000 2023-05-30 04:29:20 - SimpleLog - INFO: - episode: 8487, ep_reward: 0.774, ep_step: 921 2023-05-30 04:29:22 - SimpleLog - INFO: - episode: 8488, ep_reward: 130.616, ep_step: 1000 2023-05-30 04:29:24 - SimpleLog - INFO: - episode: 8489, ep_reward: 104.728, ep_step: 1000 2023-05-30 04:29:26 - SimpleLog - INFO: - episode: 8490, ep_reward: 133.799, ep_step: 1000 2023-05-30 04:29:29 - SimpleLog - INFO: - episode: 8491, ep_reward: 8.313, ep_step: 1000 2023-05-30 04:29:31 - SimpleLog - INFO: - episode: 8492, ep_reward: 149.312, ep_step: 1000 2023-05-30 04:29:33 - SimpleLog - INFO: - episode: 8493, ep_reward: 11.675, ep_step: 1000 2023-05-30 04:29:35 - SimpleLog - INFO: - episode: 8494, ep_reward: 120.703, ep_step: 1000 2023-05-30 04:29:38 - SimpleLog - INFO: - episode: 8495, ep_reward: 109.956, ep_step: 1000 2023-05-30 04:29:40 - SimpleLog - INFO: - episode: 8496, ep_reward: 80.727, ep_step: 1000 2023-05-30 04:29:42 - SimpleLog - INFO: - episode: 8497, ep_reward: 120.296, ep_step: 1000 2023-05-30 04:29:44 - SimpleLog - INFO: - episode: 8498, ep_reward: 119.592, ep_step: 1000 2023-05-30 04:29:47 - SimpleLog - INFO: - episode: 8499, ep_reward: 124.709, ep_step: 1000 2023-05-30 04:29:49 - SimpleLog - INFO: - episode: 8500, ep_reward: 133.221, ep_step: 1000 2023-05-30 04:29:51 - SimpleLog - INFO: - episode: 8501, ep_reward: 22.983, ep_step: 996 2023-05-30 04:29:53 - SimpleLog - INFO: - episode: 8502, ep_reward: 131.417, ep_step: 1000 2023-05-30 04:29:56 - SimpleLog - INFO: - episode: 8503, ep_reward: 82.392, ep_step: 1000 2023-05-30 04:29:58 - SimpleLog - INFO: - episode: 8504, ep_reward: 129.429, ep_step: 1000 2023-05-30 04:30:00 - SimpleLog - INFO: - episode: 8505, ep_reward: 100.136, ep_step: 1000 2023-05-30 04:30:02 - SimpleLog - INFO: - episode: 8506, ep_reward: -3.317, ep_step: 682 2023-05-30 04:30:04 - SimpleLog - INFO: - episode: 8507, ep_reward: 153.676, ep_step: 1000 2023-05-30 04:30:06 - SimpleLog - INFO: - episode: 8508, ep_reward: 139.684, ep_step: 1000 2023-05-30 04:30:09 - SimpleLog - INFO: - episode: 8509, ep_reward: 166.830, ep_step: 1000 2023-05-30 04:30:11 - SimpleLog - INFO: - episode: 8510, ep_reward: 126.793, ep_step: 1000 2023-05-30 04:30:13 - SimpleLog - INFO: - episode: 8511, ep_reward: 147.293, ep_step: 1000 2023-05-30 04:30:15 - SimpleLog - INFO: - episode: 8512, ep_reward: 156.993, ep_step: 1000 2023-05-30 04:30:18 - SimpleLog - INFO: - episode: 8513, ep_reward: 168.959, ep_step: 1000 2023-05-30 04:30:20 - SimpleLog - INFO: - episode: 8514, ep_reward: 152.117, ep_step: 1000 2023-05-30 04:30:20 - SimpleLog - INFO: - episode: 8515, ep_reward: -89.806, ep_step: 161 2023-05-30 04:30:22 - SimpleLog - INFO: - episode: 8516, ep_reward: 147.032, ep_step: 1000 2023-05-30 04:30:25 - SimpleLog - INFO: - episode: 8517, ep_reward: 0.628, ep_step: 954 2023-05-30 04:30:25 - SimpleLog - INFO: - episode: 8518, ep_reward: -54.528, ep_step: 364 2023-05-30 04:30:28 - SimpleLog - INFO: - episode: 8519, ep_reward: 108.794, ep_step: 1000 2023-05-30 04:30:32 - SimpleLog - INFO: - update_step: 6860000, online_eval_reward: 153.026 2023-05-30 04:30:32 - SimpleLog - INFO: - episode: 8520, ep_reward: 69.715, ep_step: 1000 2023-05-30 04:30:34 - SimpleLog - INFO: - episode: 8521, ep_reward: 138.138, ep_step: 1000 2023-05-30 04:30:36 - SimpleLog - INFO: - episode: 8522, ep_reward: 138.936, ep_step: 1000 2023-05-30 04:30:39 - SimpleLog - INFO: - episode: 8523, ep_reward: 146.455, ep_step: 1000 2023-05-30 04:30:41 - SimpleLog - INFO: - episode: 8524, ep_reward: 154.169, ep_step: 1000 2023-05-30 04:30:43 - SimpleLog - INFO: - episode: 8525, ep_reward: 137.339, ep_step: 1000 2023-05-30 04:30:45 - SimpleLog - INFO: - episode: 8526, ep_reward: 147.791, ep_step: 1000 2023-05-30 04:30:48 - SimpleLog - INFO: - episode: 8527, ep_reward: 141.315, ep_step: 1000 2023-05-30 04:30:50 - SimpleLog - INFO: - episode: 8528, ep_reward: 78.498, ep_step: 1000 2023-05-30 04:30:52 - SimpleLog - INFO: - episode: 8529, ep_reward: 140.466, ep_step: 1000 2023-05-30 04:30:54 - SimpleLog - INFO: - episode: 8530, ep_reward: 104.782, ep_step: 1000 2023-05-30 04:30:57 - SimpleLog - INFO: - episode: 8531, ep_reward: 147.520, ep_step: 1000 2023-05-30 04:30:59 - SimpleLog - INFO: - episode: 8532, ep_reward: 132.083, ep_step: 1000 2023-05-30 04:31:01 - SimpleLog - INFO: - episode: 8533, ep_reward: -0.099, ep_step: 823 2023-05-30 04:31:03 - SimpleLog - INFO: - episode: 8534, ep_reward: 119.109, ep_step: 1000 2023-05-30 04:31:05 - SimpleLog - INFO: - episode: 8535, ep_reward: 152.854, ep_step: 1000 2023-05-30 04:31:07 - SimpleLog - INFO: - episode: 8536, ep_reward: 139.326, ep_step: 1000 2023-05-30 04:31:10 - SimpleLog - INFO: - episode: 8537, ep_reward: 135.690, ep_step: 1000 2023-05-30 04:31:12 - SimpleLog - INFO: - episode: 8538, ep_reward: 131.429, ep_step: 1000 2023-05-30 04:31:14 - SimpleLog - INFO: - episode: 8539, ep_reward: -3.825, ep_step: 958 2023-05-30 04:31:16 - SimpleLog - INFO: - episode: 8540, ep_reward: 110.291, ep_step: 1000 2023-05-30 04:31:19 - SimpleLog - INFO: - episode: 8541, ep_reward: 147.539, ep_step: 1000 2023-05-30 04:31:21 - SimpleLog - INFO: - episode: 8542, ep_reward: 148.881, ep_step: 1000 2023-05-30 04:31:23 - SimpleLog - INFO: - episode: 8543, ep_reward: 154.893, ep_step: 1000 2023-05-30 04:31:25 - SimpleLog - INFO: - episode: 8544, ep_reward: 138.981, ep_step: 1000 2023-05-30 04:31:28 - SimpleLog - INFO: - episode: 8545, ep_reward: 151.448, ep_step: 1000 2023-05-30 04:31:30 - SimpleLog - INFO: - episode: 8546, ep_reward: 130.232, ep_step: 1000 2023-05-30 04:31:32 - SimpleLog - INFO: - episode: 8547, ep_reward: 137.304, ep_step: 1000 2023-05-30 04:31:34 - SimpleLog - INFO: - episode: 8548, ep_reward: 165.503, ep_step: 1000 2023-05-30 04:31:36 - SimpleLog - INFO: - episode: 8549, ep_reward: 148.751, ep_step: 1000 2023-05-30 04:31:39 - SimpleLog - INFO: - episode: 8550, ep_reward: 86.435, ep_step: 1000 2023-05-30 04:31:41 - SimpleLog - INFO: - episode: 8551, ep_reward: 81.336, ep_step: 1000 2023-05-30 04:31:43 - SimpleLog - INFO: - episode: 8552, ep_reward: 77.779, ep_step: 1000 2023-05-30 04:31:45 - SimpleLog - INFO: - episode: 8553, ep_reward: 161.491, ep_step: 1000 2023-05-30 04:31:48 - SimpleLog - INFO: - episode: 8554, ep_reward: 78.419, ep_step: 1000 2023-05-30 04:31:50 - SimpleLog - INFO: - episode: 8555, ep_reward: 167.570, ep_step: 1000 2023-05-30 04:31:52 - SimpleLog - INFO: - episode: 8556, ep_reward: 141.854, ep_step: 1000 2023-05-30 04:31:54 - SimpleLog - INFO: - episode: 8557, ep_reward: 154.872, ep_step: 1000 2023-05-30 04:31:56 - SimpleLog - INFO: - episode: 8558, ep_reward: -31.806, ep_step: 489 2023-05-30 04:31:58 - SimpleLog - INFO: - episode: 8559, ep_reward: 79.442, ep_step: 1000 2023-05-30 04:32:00 - SimpleLog - INFO: - episode: 8560, ep_reward: 161.087, ep_step: 1000 2023-05-30 04:32:02 - SimpleLog - INFO: - episode: 8561, ep_reward: 169.519, ep_step: 1000 2023-05-30 04:32:03 - SimpleLog - INFO: - episode: 8562, ep_reward: -36.583, ep_step: 534 2023-05-30 04:32:06 - SimpleLog - INFO: - episode: 8563, ep_reward: 82.102, ep_step: 1000 2023-05-30 04:32:08 - SimpleLog - INFO: - episode: 8564, ep_reward: 118.033, ep_step: 1000 2023-05-30 04:32:10 - SimpleLog - INFO: - episode: 8565, ep_reward: 101.724, ep_step: 1000 2023-05-30 04:32:12 - SimpleLog - INFO: - episode: 8566, ep_reward: 13.647, ep_step: 841 2023-05-30 04:32:14 - SimpleLog - INFO: - episode: 8567, ep_reward: 13.901, ep_step: 806 2023-05-30 04:32:16 - SimpleLog - INFO: - episode: 8568, ep_reward: 132.559, ep_step: 1000 2023-05-30 04:32:18 - SimpleLog - INFO: - episode: 8569, ep_reward: 136.523, ep_step: 1000 2023-05-30 04:32:21 - SimpleLog - INFO: - episode: 8570, ep_reward: 142.150, ep_step: 1000 2023-05-30 04:32:23 - SimpleLog - INFO: - episode: 8571, ep_reward: 144.477, ep_step: 1000 2023-05-30 04:32:25 - SimpleLog - INFO: - episode: 8572, ep_reward: 111.248, ep_step: 1000 2023-05-30 04:32:27 - SimpleLog - INFO: - episode: 8573, ep_reward: 123.249, ep_step: 1000 2023-05-30 04:32:30 - SimpleLog - INFO: - episode: 8574, ep_reward: 142.783, ep_step: 1000 2023-05-30 04:32:32 - SimpleLog - INFO: - episode: 8575, ep_reward: 138.743, ep_step: 1000 2023-05-30 04:32:34 - SimpleLog - INFO: - episode: 8576, ep_reward: 143.183, ep_step: 1000 2023-05-30 04:32:36 - SimpleLog - INFO: - episode: 8577, ep_reward: 158.428, ep_step: 1000 2023-05-30 04:32:39 - SimpleLog - INFO: - episode: 8578, ep_reward: 142.069, ep_step: 1000 2023-05-30 04:32:41 - SimpleLog - INFO: - episode: 8579, ep_reward: 140.953, ep_step: 1000 2023-05-30 04:32:43 - SimpleLog - INFO: - episode: 8580, ep_reward: 149.863, ep_step: 1000 2023-05-30 04:32:45 - SimpleLog - INFO: - episode: 8581, ep_reward: 155.516, ep_step: 1000 2023-05-30 04:32:48 - SimpleLog - INFO: - episode: 8582, ep_reward: 135.317, ep_step: 1000 2023-05-30 04:32:50 - SimpleLog - INFO: - episode: 8583, ep_reward: 148.401, ep_step: 1000 2023-05-30 04:32:50 - SimpleLog - INFO: - episode: 8584, ep_reward: -111.758, ep_step: 66 2023-05-30 04:32:52 - SimpleLog - INFO: - episode: 8585, ep_reward: 153.381, ep_step: 1000 2023-05-30 04:32:53 - SimpleLog - INFO: - episode: 8586, ep_reward: -68.907, ep_step: 300 2023-05-30 04:32:55 - SimpleLog - INFO: - episode: 8587, ep_reward: 150.052, ep_step: 1000 2023-05-30 04:32:57 - SimpleLog - INFO: - episode: 8588, ep_reward: 123.956, ep_step: 1000 2023-05-30 04:32:58 - SimpleLog - INFO: - episode: 8589, ep_reward: -84.056, ep_step: 186 2023-05-30 04:32:58 - SimpleLog - INFO: - episode: 8590, ep_reward: -69.959, ep_step: 288 2023-05-30 04:33:01 - SimpleLog - INFO: - episode: 8591, ep_reward: 139.495, ep_step: 1000 2023-05-30 04:33:03 - SimpleLog - INFO: - episode: 8592, ep_reward: 140.858, ep_step: 1000 2023-05-30 04:33:05 - SimpleLog - INFO: - episode: 8593, ep_reward: 141.966, ep_step: 1000 2023-05-30 04:33:07 - SimpleLog - INFO: - episode: 8594, ep_reward: 152.289, ep_step: 1000 2023-05-30 04:33:11 - SimpleLog - INFO: - update_step: 6930000, online_eval_reward: 160.137 2023-05-30 04:33:12 - SimpleLog - INFO: - episode: 8595, ep_reward: 141.118, ep_step: 1000 2023-05-30 04:33:14 - SimpleLog - INFO: - episode: 8596, ep_reward: 136.109, ep_step: 1000 2023-05-30 04:33:16 - SimpleLog - INFO: - episode: 8597, ep_reward: 1.860, ep_step: 755 2023-05-30 04:33:16 - SimpleLog - INFO: - episode: 8598, ep_reward: -65.563, ep_step: 343 2023-05-30 04:33:19 - SimpleLog - INFO: - episode: 8599, ep_reward: 77.975, ep_step: 1000 2023-05-30 04:33:20 - SimpleLog - INFO: - episode: 8600, ep_reward: -0.249, ep_step: 714 2023-05-30 04:33:22 - SimpleLog - INFO: - episode: 8601, ep_reward: 126.827, ep_step: 1000 2023-05-30 04:33:25 - SimpleLog - INFO: - episode: 8602, ep_reward: 160.412, ep_step: 1000 2023-05-30 04:33:25 - SimpleLog - INFO: - episode: 8603, ep_reward: -75.130, ep_step: 338 2023-05-30 04:33:28 - SimpleLog - INFO: - episode: 8604, ep_reward: 155.579, ep_step: 1000 2023-05-30 04:33:30 - SimpleLog - INFO: - episode: 8605, ep_reward: 140.389, ep_step: 1000 2023-05-30 04:33:32 - SimpleLog - INFO: - episode: 8606, ep_reward: -2.867, ep_step: 1000 2023-05-30 04:33:34 - SimpleLog - INFO: - episode: 8607, ep_reward: 143.879, ep_step: 1000 2023-05-30 04:33:37 - SimpleLog - INFO: - episode: 8608, ep_reward: 142.460, ep_step: 1000 2023-05-30 04:33:39 - SimpleLog - INFO: - episode: 8609, ep_reward: 149.877, ep_step: 1000 2023-05-30 04:33:41 - SimpleLog - INFO: - episode: 8610, ep_reward: 137.837, ep_step: 1000 2023-05-30 04:33:43 - SimpleLog - INFO: - episode: 8611, ep_reward: 142.958, ep_step: 1000 2023-05-30 04:33:46 - SimpleLog - INFO: - episode: 8612, ep_reward: 147.935, ep_step: 1000 2023-05-30 04:33:48 - SimpleLog - INFO: - episode: 8613, ep_reward: 146.210, ep_step: 1000 2023-05-30 04:33:50 - SimpleLog - INFO: - episode: 8614, ep_reward: 151.678, ep_step: 1000 2023-05-30 04:33:52 - SimpleLog - INFO: - episode: 8615, ep_reward: 14.916, ep_step: 809 2023-05-30 04:33:54 - SimpleLog - INFO: - episode: 8616, ep_reward: 105.502, ep_step: 1000 2023-05-30 04:33:57 - SimpleLog - INFO: - episode: 8617, ep_reward: 4.492, ep_step: 1000 2023-05-30 04:33:59 - SimpleLog - INFO: - episode: 8618, ep_reward: 140.695, ep_step: 1000 2023-05-30 04:34:01 - SimpleLog - INFO: - episode: 8619, ep_reward: 147.833, ep_step: 1000 2023-05-30 04:34:03 - SimpleLog - INFO: - episode: 8620, ep_reward: 147.655, ep_step: 1000 2023-05-30 04:34:06 - SimpleLog - INFO: - episode: 8621, ep_reward: 149.480, ep_step: 1000 2023-05-30 04:34:08 - SimpleLog - INFO: - episode: 8622, ep_reward: 161.797, ep_step: 1000 2023-05-30 04:34:10 - SimpleLog - INFO: - episode: 8623, ep_reward: 148.658, ep_step: 1000 2023-05-30 04:34:12 - SimpleLog - INFO: - episode: 8624, ep_reward: 110.315, ep_step: 1000 2023-05-30 04:34:15 - SimpleLog - INFO: - episode: 8625, ep_reward: 157.997, ep_step: 1000 2023-05-30 04:34:17 - SimpleLog - INFO: - episode: 8626, ep_reward: 157.624, ep_step: 1000 2023-05-30 04:34:19 - SimpleLog - INFO: - episode: 8627, ep_reward: 157.492, ep_step: 1000 2023-05-30 04:34:21 - SimpleLog - INFO: - episode: 8628, ep_reward: 103.581, ep_step: 1000 2023-05-30 04:34:24 - SimpleLog - INFO: - episode: 8629, ep_reward: 165.247, ep_step: 1000 2023-05-30 04:34:26 - SimpleLog - INFO: - episode: 8630, ep_reward: 103.194, ep_step: 1000 2023-05-30 04:34:27 - SimpleLog - INFO: - episode: 8631, ep_reward: -53.526, ep_step: 450 2023-05-30 04:34:29 - SimpleLog - INFO: - episode: 8632, ep_reward: 167.903, ep_step: 1000 2023-05-30 04:34:31 - SimpleLog - INFO: - episode: 8633, ep_reward: 12.465, ep_step: 816 2023-05-30 04:34:33 - SimpleLog - INFO: - episode: 8634, ep_reward: 152.229, ep_step: 1000 2023-05-30 04:34:35 - SimpleLog - INFO: - episode: 8635, ep_reward: 139.449, ep_step: 1000 2023-05-30 04:34:38 - SimpleLog - INFO: - episode: 8636, ep_reward: 175.545, ep_step: 1000 2023-05-30 04:34:40 - SimpleLog - INFO: - episode: 8637, ep_reward: 164.020, ep_step: 1000 2023-05-30 04:34:42 - SimpleLog - INFO: - episode: 8638, ep_reward: 166.209, ep_step: 1000 2023-05-30 04:34:44 - SimpleLog - INFO: - episode: 8639, ep_reward: 110.971, ep_step: 1000 2023-05-30 04:34:47 - SimpleLog - INFO: - episode: 8640, ep_reward: 169.399, ep_step: 1000 2023-05-30 04:34:49 - SimpleLog - INFO: - episode: 8641, ep_reward: 153.817, ep_step: 1000 2023-05-30 04:34:51 - SimpleLog - INFO: - episode: 8642, ep_reward: 47.003, ep_step: 901 2023-05-30 04:34:53 - SimpleLog - INFO: - episode: 8643, ep_reward: 6.406, ep_step: 1000 2023-05-30 04:34:55 - SimpleLog - INFO: - episode: 8644, ep_reward: 174.651, ep_step: 1000 2023-05-30 04:34:58 - SimpleLog - INFO: - episode: 8645, ep_reward: 173.359, ep_step: 1000 2023-05-30 04:35:00 - SimpleLog - INFO: - episode: 8646, ep_reward: -2.102, ep_step: 990 2023-05-30 04:35:02 - SimpleLog - INFO: - episode: 8647, ep_reward: 148.073, ep_step: 1000 2023-05-30 04:35:04 - SimpleLog - INFO: - episode: 8648, ep_reward: 140.754, ep_step: 1000 2023-05-30 04:35:07 - SimpleLog - INFO: - episode: 8649, ep_reward: 159.867, ep_step: 1000 2023-05-30 04:35:09 - SimpleLog - INFO: - episode: 8650, ep_reward: 163.836, ep_step: 1000 2023-05-30 04:35:11 - SimpleLog - INFO: - episode: 8651, ep_reward: 156.242, ep_step: 1000 2023-05-30 04:35:13 - SimpleLog - INFO: - episode: 8652, ep_reward: 142.784, ep_step: 1000 2023-05-30 04:35:16 - SimpleLog - INFO: - episode: 8653, ep_reward: 136.915, ep_step: 1000 2023-05-30 04:35:18 - SimpleLog - INFO: - episode: 8654, ep_reward: 144.219, ep_step: 1000 2023-05-30 04:35:20 - SimpleLog - INFO: - episode: 8655, ep_reward: -34.894, ep_step: 846 2023-05-30 04:35:22 - SimpleLog - INFO: - episode: 8656, ep_reward: 149.894, ep_step: 1000 2023-05-30 04:35:24 - SimpleLog - INFO: - episode: 8657, ep_reward: 21.635, ep_step: 779 2023-05-30 04:35:26 - SimpleLog - INFO: - episode: 8658, ep_reward: 159.616, ep_step: 1000 2023-05-30 04:35:28 - SimpleLog - INFO: - episode: 8659, ep_reward: -10.470, ep_step: 840 2023-05-30 04:35:29 - SimpleLog - INFO: - episode: 8660, ep_reward: -57.492, ep_step: 434 2023-05-30 04:35:31 - SimpleLog - INFO: - episode: 8661, ep_reward: 153.322, ep_step: 1000 2023-05-30 04:35:33 - SimpleLog - INFO: - episode: 8662, ep_reward: 71.001, ep_step: 1000 2023-05-30 04:35:35 - SimpleLog - INFO: - episode: 8663, ep_reward: -18.254, ep_step: 810 2023-05-30 04:35:37 - SimpleLog - INFO: - episode: 8664, ep_reward: 149.835, ep_step: 1000 2023-05-30 04:35:38 - SimpleLog - INFO: - episode: 8665, ep_reward: -54.467, ep_step: 339 2023-05-30 04:35:40 - SimpleLog - INFO: - episode: 8666, ep_reward: 85.263, ep_step: 1000 2023-05-30 04:35:43 - SimpleLog - INFO: - episode: 8667, ep_reward: 79.486, ep_step: 1000 2023-05-30 04:35:45 - SimpleLog - INFO: - episode: 8668, ep_reward: 165.217, ep_step: 1000 2023-05-30 04:35:47 - SimpleLog - INFO: - episode: 8669, ep_reward: 147.886, ep_step: 1000 2023-05-30 04:35:50 - SimpleLog - INFO: - update_step: 7000000, online_eval_reward: 111.023 2023-05-30 04:35:51 - SimpleLog - INFO: - episode: 8670, ep_reward: 136.199, ep_step: 1000 2023-05-30 04:35:54 - SimpleLog - INFO: - episode: 8671, ep_reward: 158.136, ep_step: 1000 2023-05-30 04:35:56 - SimpleLog - INFO: - episode: 8672, ep_reward: 139.956, ep_step: 1000 2023-05-30 04:35:58 - SimpleLog - INFO: - episode: 8673, ep_reward: 148.422, ep_step: 1000 2023-05-30 04:36:00 - SimpleLog - INFO: - episode: 8674, ep_reward: 154.569, ep_step: 1000 2023-05-30 04:36:03 - SimpleLog - INFO: - episode: 8675, ep_reward: 147.968, ep_step: 1000 2023-05-30 04:36:05 - SimpleLog - INFO: - episode: 8676, ep_reward: 146.742, ep_step: 1000 2023-05-30 04:36:07 - SimpleLog - INFO: - episode: 8677, ep_reward: 157.395, ep_step: 1000 2023-05-30 04:36:09 - SimpleLog - INFO: - episode: 8678, ep_reward: -66.857, ep_step: 1000 2023-05-30 04:36:12 - SimpleLog - INFO: - episode: 8679, ep_reward: -18.456, ep_step: 986 2023-05-30 04:36:14 - SimpleLog - INFO: - episode: 8680, ep_reward: 140.597, ep_step: 1000 2023-05-30 04:36:16 - SimpleLog - INFO: - episode: 8681, ep_reward: 164.204, ep_step: 1000 2023-05-30 04:36:18 - SimpleLog - INFO: - episode: 8682, ep_reward: 20.098, ep_step: 796 2023-05-30 04:36:20 - SimpleLog - INFO: - episode: 8683, ep_reward: 145.836, ep_step: 1000 2023-05-30 04:36:22 - SimpleLog - INFO: - episode: 8684, ep_reward: 155.708, ep_step: 1000 2023-05-30 04:36:25 - SimpleLog - INFO: - episode: 8685, ep_reward: 170.935, ep_step: 1000 2023-05-30 04:36:27 - SimpleLog - INFO: - episode: 8686, ep_reward: 0.571, ep_step: 1000 2023-05-30 04:36:29 - SimpleLog - INFO: - episode: 8687, ep_reward: -85.888, ep_step: 1000 2023-05-30 04:36:31 - SimpleLog - INFO: - episode: 8688, ep_reward: 70.872, ep_step: 1000 2023-05-30 04:36:34 - SimpleLog - INFO: - episode: 8689, ep_reward: 81.708, ep_step: 1000 2023-05-30 04:36:36 - SimpleLog - INFO: - episode: 8690, ep_reward: -81.287, ep_step: 1000 2023-05-30 04:36:38 - SimpleLog - INFO: - episode: 8691, ep_reward: 73.411, ep_step: 1000 2023-05-30 04:36:40 - SimpleLog - INFO: - episode: 8692, ep_reward: 0.027, ep_step: 1000 2023-05-30 04:36:42 - SimpleLog - INFO: - episode: 8693, ep_reward: -82.883, ep_step: 1000 2023-05-30 04:36:45 - SimpleLog - INFO: - episode: 8694, ep_reward: -85.034, ep_step: 1000 2023-05-30 04:36:47 - SimpleLog - INFO: - episode: 8695, ep_reward: -82.724, ep_step: 1000 2023-05-30 04:36:49 - SimpleLog - INFO: - episode: 8696, ep_reward: 5.697, ep_step: 1000 2023-05-30 04:36:52 - SimpleLog - INFO: - episode: 8697, ep_reward: -87.299, ep_step: 1000 2023-05-30 04:36:54 - SimpleLog - INFO: - episode: 8698, ep_reward: 4.725, ep_step: 1000 2023-05-30 04:36:56 - SimpleLog - INFO: - episode: 8699, ep_reward: -91.759, ep_step: 1000 2023-05-30 04:36:58 - SimpleLog - INFO: - episode: 8700, ep_reward: -80.300, ep_step: 1000 2023-05-30 04:37:01 - SimpleLog - INFO: - episode: 8701, ep_reward: 79.415, ep_step: 1000 2023-05-30 04:37:03 - SimpleLog - INFO: - episode: 8702, ep_reward: -99.881, ep_step: 1000 2023-05-30 04:37:05 - SimpleLog - INFO: - episode: 8703, ep_reward: 5.659, ep_step: 1000 2023-05-30 04:37:07 - SimpleLog - INFO: - episode: 8704, ep_reward: -36.639, ep_step: 1000 2023-05-30 04:37:10 - SimpleLog - INFO: - episode: 8705, ep_reward: -36.015, ep_step: 1000 2023-05-30 04:37:12 - SimpleLog - INFO: - episode: 8706, ep_reward: -37.864, ep_step: 1000 2023-05-30 04:37:13 - SimpleLog - INFO: - episode: 8707, ep_reward: -96.939, ep_step: 536 2023-05-30 04:37:15 - SimpleLog - INFO: - episode: 8708, ep_reward: -98.448, ep_step: 1000 2023-05-30 04:37:17 - SimpleLog - INFO: - episode: 8709, ep_reward: 3.077, ep_step: 1000 2023-05-30 04:37:20 - SimpleLog - INFO: - episode: 8710, ep_reward: 4.839, ep_step: 1000 2023-05-30 04:37:22 - SimpleLog - INFO: - episode: 8711, ep_reward: -21.608, ep_step: 1000 2023-05-30 04:37:24 - SimpleLog - INFO: - episode: 8712, ep_reward: -52.720, ep_step: 1000 2023-05-30 04:37:25 - SimpleLog - INFO: - episode: 8713, ep_reward: -74.018, ep_step: 537 2023-05-30 04:37:27 - SimpleLog - INFO: - episode: 8714, ep_reward: -83.133, ep_step: 629 2023-05-30 04:37:29 - SimpleLog - INFO: - episode: 8715, ep_reward: 2.473, ep_step: 1000 2023-05-30 04:37:30 - SimpleLog - INFO: - episode: 8716, ep_reward: -66.112, ep_step: 393 2023-05-30 04:37:32 - SimpleLog - INFO: - episode: 8717, ep_reward: 4.607, ep_step: 1000 2023-05-30 04:37:34 - SimpleLog - INFO: - episode: 8718, ep_reward: -77.247, ep_step: 553 2023-05-30 04:37:35 - SimpleLog - INFO: - episode: 8719, ep_reward: -50.743, ep_step: 536 2023-05-30 04:37:36 - SimpleLog - INFO: - episode: 8720, ep_reward: -73.499, ep_step: 539 2023-05-30 04:37:38 - SimpleLog - INFO: - episode: 8721, ep_reward: 69.741, ep_step: 1000 2023-05-30 04:37:40 - SimpleLog - INFO: - episode: 8722, ep_reward: -91.010, ep_step: 704 2023-05-30 04:37:42 - SimpleLog - INFO: - episode: 8723, ep_reward: -34.202, ep_step: 909 2023-05-30 04:37:43 - SimpleLog - INFO: - episode: 8724, ep_reward: -71.604, ep_step: 550 2023-05-30 04:37:45 - SimpleLog - INFO: - episode: 8725, ep_reward: -35.986, ep_step: 921 2023-05-30 04:37:47 - SimpleLog - INFO: - episode: 8726, ep_reward: -38.044, ep_step: 921 2023-05-30 04:37:48 - SimpleLog - INFO: - episode: 8727, ep_reward: -136.896, ep_step: 520 2023-05-30 04:37:51 - SimpleLog - INFO: - episode: 8728, ep_reward: -76.628, ep_step: 1000 2023-05-30 04:37:52 - SimpleLog - INFO: - episode: 8729, ep_reward: -85.735, ep_step: 573 2023-05-30 04:37:54 - SimpleLog - INFO: - episode: 8730, ep_reward: 97.514, ep_step: 1000 2023-05-30 04:37:55 - SimpleLog - INFO: - episode: 8731, ep_reward: -109.327, ep_step: 212 2023-05-30 04:37:56 - SimpleLog - INFO: - episode: 8732, ep_reward: -90.514, ep_step: 673 2023-05-30 04:37:57 - SimpleLog - INFO: - episode: 8733, ep_reward: -106.747, ep_step: 178 2023-05-30 04:37:59 - SimpleLog - INFO: - episode: 8734, ep_reward: -43.174, ep_step: 977 2023-05-30 04:38:01 - SimpleLog - INFO: - episode: 8735, ep_reward: -30.842, ep_step: 854 2023-05-30 04:38:03 - SimpleLog - INFO: - episode: 8736, ep_reward: 143.146, ep_step: 1000 2023-05-30 04:38:04 - SimpleLog - INFO: - episode: 8737, ep_reward: -62.001, ep_step: 415 2023-05-30 04:38:06 - SimpleLog - INFO: - episode: 8738, ep_reward: 18.034, ep_step: 800 2023-05-30 04:38:07 - SimpleLog - INFO: - episode: 8739, ep_reward: -30.812, ep_step: 818 2023-05-30 04:38:09 - SimpleLog - INFO: - episode: 8740, ep_reward: -42.912, ep_step: 504 2023-05-30 04:38:10 - SimpleLog - INFO: - episode: 8741, ep_reward: -41.410, ep_step: 551 2023-05-30 04:38:11 - SimpleLog - INFO: - episode: 8742, ep_reward: -105.396, ep_step: 376 2023-05-30 04:38:12 - SimpleLog - INFO: - episode: 8743, ep_reward: -64.191, ep_step: 429 2023-05-30 04:38:13 - SimpleLog - INFO: - episode: 8744, ep_reward: -31.827, ep_step: 818 2023-05-30 04:38:15 - SimpleLog - INFO: - episode: 8745, ep_reward: -64.134, ep_step: 472 2023-05-30 04:38:17 - SimpleLog - INFO: - episode: 8746, ep_reward: 95.953, ep_step: 1000 2023-05-30 04:38:18 - SimpleLog - INFO: - episode: 8747, ep_reward: -30.190, ep_step: 696 2023-05-30 04:38:21 - SimpleLog - INFO: - episode: 8748, ep_reward: 67.498, ep_step: 1000 2023-05-30 04:38:23 - SimpleLog - INFO: - episode: 8749, ep_reward: 156.133, ep_step: 1000 2023-05-30 04:38:25 - SimpleLog - INFO: - episode: 8750, ep_reward: 117.436, ep_step: 1000 2023-05-30 04:38:27 - SimpleLog - INFO: - episode: 8751, ep_reward: -50.683, ep_step: 642 2023-05-30 04:38:30 - SimpleLog - INFO: - update_step: 7070000, online_eval_reward: 108.702 2023-05-30 04:38:30 - SimpleLog - INFO: - episode: 8752, ep_reward: -36.699, ep_step: 787 2023-05-30 04:38:32 - SimpleLog - INFO: - episode: 8753, ep_reward: 67.620, ep_step: 1000 2023-05-30 04:38:35 - SimpleLog - INFO: - episode: 8754, ep_reward: 75.587, ep_step: 1000 2023-05-30 04:38:37 - SimpleLog - INFO: - episode: 8755, ep_reward: 109.898, ep_step: 1000 2023-05-30 04:38:39 - SimpleLog - INFO: - episode: 8756, ep_reward: 79.165, ep_step: 1000 2023-05-30 04:38:41 - SimpleLog - INFO: - episode: 8757, ep_reward: -0.882, ep_step: 832 2023-05-30 04:38:43 - SimpleLog - INFO: - episode: 8758, ep_reward: 6.818, ep_step: 1000 2023-05-30 04:38:46 - SimpleLog - INFO: - episode: 8759, ep_reward: 88.561, ep_step: 1000 2023-05-30 04:38:48 - SimpleLog - INFO: - episode: 8760, ep_reward: 77.643, ep_step: 1000 2023-05-30 04:38:49 - SimpleLog - INFO: - episode: 8761, ep_reward: -78.376, ep_step: 394 2023-05-30 04:38:50 - SimpleLog - INFO: - episode: 8762, ep_reward: -63.083, ep_step: 392 2023-05-30 04:38:52 - SimpleLog - INFO: - episode: 8763, ep_reward: 75.339, ep_step: 1000 2023-05-30 04:38:54 - SimpleLog - INFO: - episode: 8764, ep_reward: 110.601, ep_step: 1000 2023-05-30 04:38:56 - SimpleLog - INFO: - episode: 8765, ep_reward: 81.894, ep_step: 1000 2023-05-30 04:38:59 - SimpleLog - INFO: - episode: 8766, ep_reward: 87.161, ep_step: 1000 2023-05-30 04:39:01 - SimpleLog - INFO: - episode: 8767, ep_reward: 114.508, ep_step: 1000 2023-05-30 04:39:03 - SimpleLog - INFO: - episode: 8768, ep_reward: 65.122, ep_step: 1000 2023-05-30 04:39:05 - SimpleLog - INFO: - episode: 8769, ep_reward: 87.201, ep_step: 1000 2023-05-30 04:39:08 - SimpleLog - INFO: - episode: 8770, ep_reward: 80.751, ep_step: 1000 2023-05-30 04:39:10 - SimpleLog - INFO: - episode: 8771, ep_reward: 100.710, ep_step: 1000 2023-05-30 04:39:12 - SimpleLog - INFO: - episode: 8772, ep_reward: 112.674, ep_step: 1000 2023-05-30 04:39:14 - SimpleLog - INFO: - episode: 8773, ep_reward: 109.586, ep_step: 1000 2023-05-30 04:39:17 - SimpleLog - INFO: - episode: 8774, ep_reward: 110.267, ep_step: 1000 2023-05-30 04:39:19 - SimpleLog - INFO: - episode: 8775, ep_reward: 13.440, ep_step: 900 2023-05-30 04:39:21 - SimpleLog - INFO: - episode: 8776, ep_reward: 113.041, ep_step: 1000 2023-05-30 04:39:23 - SimpleLog - INFO: - episode: 8777, ep_reward: 92.740, ep_step: 1000 2023-05-30 04:39:25 - SimpleLog - INFO: - episode: 8778, ep_reward: 60.110, ep_step: 1000 2023-05-30 04:39:28 - SimpleLog - INFO: - episode: 8779, ep_reward: 7.875, ep_step: 1000 2023-05-30 04:39:30 - SimpleLog - INFO: - episode: 8780, ep_reward: 83.745, ep_step: 1000 2023-05-30 04:39:32 - SimpleLog - INFO: - episode: 8781, ep_reward: 97.887, ep_step: 1000 2023-05-30 04:39:34 - SimpleLog - INFO: - episode: 8782, ep_reward: 92.801, ep_step: 1000 2023-05-30 04:39:37 - SimpleLog - INFO: - episode: 8783, ep_reward: 1.185, ep_step: 963 2023-05-30 04:39:39 - SimpleLog - INFO: - episode: 8784, ep_reward: 107.806, ep_step: 1000 2023-05-30 04:39:41 - SimpleLog - INFO: - episode: 8785, ep_reward: -95.299, ep_step: 810 2023-05-30 04:39:43 - SimpleLog - INFO: - episode: 8786, ep_reward: 93.085, ep_step: 1000 2023-05-30 04:39:45 - SimpleLog - INFO: - episode: 8787, ep_reward: 102.399, ep_step: 1000 2023-05-30 04:39:47 - SimpleLog - INFO: - episode: 8788, ep_reward: 81.994, ep_step: 1000 2023-05-30 04:39:50 - SimpleLog - INFO: - episode: 8789, ep_reward: 85.508, ep_step: 1000 2023-05-30 04:39:52 - SimpleLog - INFO: - episode: 8790, ep_reward: 89.082, ep_step: 1000 2023-05-30 04:39:54 - SimpleLog - INFO: - episode: 8791, ep_reward: 88.674, ep_step: 1000 2023-05-30 04:39:56 - SimpleLog - INFO: - episode: 8792, ep_reward: 5.960, ep_step: 981 2023-05-30 04:39:59 - SimpleLog - INFO: - episode: 8793, ep_reward: 84.144, ep_step: 1000 2023-05-30 04:40:01 - SimpleLog - INFO: - episode: 8794, ep_reward: 0.815, ep_step: 951 2023-05-30 04:40:03 - SimpleLog - INFO: - episode: 8795, ep_reward: 85.693, ep_step: 1000 2023-05-30 04:40:05 - SimpleLog - INFO: - episode: 8796, ep_reward: 87.914, ep_step: 1000 2023-05-30 04:40:08 - SimpleLog - INFO: - episode: 8797, ep_reward: 108.887, ep_step: 1000 2023-05-30 04:40:10 - SimpleLog - INFO: - episode: 8798, ep_reward: 82.182, ep_step: 1000 2023-05-30 04:40:11 - SimpleLog - INFO: - episode: 8799, ep_reward: -40.309, ep_step: 578 2023-05-30 04:40:13 - SimpleLog - INFO: - episode: 8800, ep_reward: 52.864, ep_step: 1000 2023-05-30 04:40:16 - SimpleLog - INFO: - episode: 8801, ep_reward: 107.954, ep_step: 1000 2023-05-30 04:40:17 - SimpleLog - INFO: - episode: 8802, ep_reward: -38.794, ep_step: 518 2023-05-30 04:40:19 - SimpleLog - INFO: - episode: 8803, ep_reward: 84.952, ep_step: 1000 2023-05-30 04:40:21 - SimpleLog - INFO: - episode: 8804, ep_reward: 85.234, ep_step: 1000 2023-05-30 04:40:24 - SimpleLog - INFO: - episode: 8805, ep_reward: 107.513, ep_step: 1000 2023-05-30 04:40:26 - SimpleLog - INFO: - episode: 8806, ep_reward: 82.944, ep_step: 1000 2023-05-30 04:40:27 - SimpleLog - INFO: - episode: 8807, ep_reward: -52.202, ep_step: 534 2023-05-30 04:40:29 - SimpleLog - INFO: - episode: 8808, ep_reward: 79.710, ep_step: 1000 2023-05-30 04:40:32 - SimpleLog - INFO: - episode: 8809, ep_reward: 7.402, ep_step: 1000 2023-05-30 04:40:34 - SimpleLog - INFO: - episode: 8810, ep_reward: 12.093, ep_step: 1000 2023-05-30 04:40:36 - SimpleLog - INFO: - episode: 8811, ep_reward: 108.762, ep_step: 1000 2023-05-30 04:40:38 - SimpleLog - INFO: - episode: 8812, ep_reward: 103.717, ep_step: 1000 2023-05-30 04:40:41 - SimpleLog - INFO: - episode: 8813, ep_reward: 6.308, ep_step: 996 2023-05-30 04:40:43 - SimpleLog - INFO: - episode: 8814, ep_reward: 78.919, ep_step: 1000 2023-05-30 04:40:45 - SimpleLog - INFO: - episode: 8815, ep_reward: 14.003, ep_step: 957 2023-05-30 04:40:47 - SimpleLog - INFO: - episode: 8816, ep_reward: -71.136, ep_step: 837 2023-05-30 04:40:49 - SimpleLog - INFO: - episode: 8817, ep_reward: 95.140, ep_step: 1000 2023-05-30 04:40:51 - SimpleLog - INFO: - episode: 8818, ep_reward: 15.403, ep_step: 1000 2023-05-30 04:40:53 - SimpleLog - INFO: - episode: 8819, ep_reward: 15.959, ep_step: 926 2023-05-30 04:40:56 - SimpleLog - INFO: - episode: 8820, ep_reward: 119.133, ep_step: 1000 2023-05-30 04:40:58 - SimpleLog - INFO: - episode: 8821, ep_reward: 0.324, ep_step: 980 2023-05-30 04:41:00 - SimpleLog - INFO: - episode: 8822, ep_reward: -102.717, ep_step: 868 2023-05-30 04:41:01 - SimpleLog - INFO: - episode: 8823, ep_reward: -53.120, ep_step: 659 2023-05-30 04:41:04 - SimpleLog - INFO: - episode: 8824, ep_reward: 10.201, ep_step: 1000 2023-05-30 04:41:06 - SimpleLog - INFO: - episode: 8825, ep_reward: 104.428, ep_step: 1000 2023-05-30 04:41:09 - SimpleLog - INFO: - update_step: 7140000, online_eval_reward: 7.930 2023-05-30 04:41:10 - SimpleLog - INFO: - episode: 8826, ep_reward: 114.541, ep_step: 1000 2023-05-30 04:41:12 - SimpleLog - INFO: - episode: 8827, ep_reward: 58.589, ep_step: 1000 2023-05-30 04:41:15 - SimpleLog - INFO: - episode: 8828, ep_reward: 102.412, ep_step: 1000 2023-05-30 04:41:17 - SimpleLog - INFO: - episode: 8829, ep_reward: 107.092, ep_step: 1000 2023-05-30 04:41:19 - SimpleLog - INFO: - episode: 8830, ep_reward: 5.159, ep_step: 921 2023-05-30 04:41:21 - SimpleLog - INFO: - episode: 8831, ep_reward: 3.345, ep_step: 998 2023-05-30 04:41:23 - SimpleLog - INFO: - episode: 8832, ep_reward: 78.058, ep_step: 1000 2023-05-30 04:41:26 - SimpleLog - INFO: - episode: 8833, ep_reward: 96.903, ep_step: 1000 2023-05-30 04:41:28 - SimpleLog - INFO: - episode: 8834, ep_reward: 107.534, ep_step: 1000 2023-05-30 04:41:30 - SimpleLog - INFO: - episode: 8835, ep_reward: 99.819, ep_step: 1000 2023-05-30 04:41:32 - SimpleLog - INFO: - episode: 8836, ep_reward: 99.326, ep_step: 1000 2023-05-30 04:41:35 - SimpleLog - INFO: - episode: 8837, ep_reward: 106.072, ep_step: 1000 2023-05-30 04:41:37 - SimpleLog - INFO: - episode: 8838, ep_reward: 103.610, ep_step: 1000 2023-05-30 04:41:39 - SimpleLog - INFO: - episode: 8839, ep_reward: 119.486, ep_step: 1000 2023-05-30 04:41:40 - SimpleLog - INFO: - episode: 8840, ep_reward: -57.679, ep_step: 537 2023-05-30 04:41:43 - SimpleLog - INFO: - episode: 8841, ep_reward: 82.634, ep_step: 1000 2023-05-30 04:41:45 - SimpleLog - INFO: - episode: 8842, ep_reward: 126.190, ep_step: 1000 2023-05-30 04:41:47 - SimpleLog - INFO: - episode: 8843, ep_reward: 86.346, ep_step: 1000 2023-05-30 04:41:49 - SimpleLog - INFO: - episode: 8844, ep_reward: 84.175, ep_step: 1000 2023-05-30 04:41:52 - SimpleLog - INFO: - episode: 8845, ep_reward: 82.695, ep_step: 1000 2023-05-30 04:41:54 - SimpleLog - INFO: - episode: 8846, ep_reward: 84.234, ep_step: 1000 2023-05-30 04:41:56 - SimpleLog - INFO: - episode: 8847, ep_reward: 7.256, ep_step: 963 2023-05-30 04:41:58 - SimpleLog - INFO: - episode: 8848, ep_reward: 146.799, ep_step: 1000 2023-05-30 04:42:00 - SimpleLog - INFO: - episode: 8849, ep_reward: 75.455, ep_step: 1000 2023-05-30 04:42:03 - SimpleLog - INFO: - episode: 8850, ep_reward: 133.032, ep_step: 1000 2023-05-30 04:42:05 - SimpleLog - INFO: - episode: 8851, ep_reward: 138.014, ep_step: 1000 2023-05-30 04:42:07 - SimpleLog - INFO: - episode: 8852, ep_reward: 72.944, ep_step: 1000 2023-05-30 04:42:09 - SimpleLog - INFO: - episode: 8853, ep_reward: 15.071, ep_step: 900 2023-05-30 04:42:11 - SimpleLog - INFO: - episode: 8854, ep_reward: 135.157, ep_step: 1000 2023-05-30 04:42:13 - SimpleLog - INFO: - episode: 8855, ep_reward: -32.979, ep_step: 545 2023-05-30 04:42:13 - SimpleLog - INFO: - episode: 8856, ep_reward: -77.462, ep_step: 319 2023-05-30 04:42:16 - SimpleLog - INFO: - episode: 8857, ep_reward: 116.090, ep_step: 1000 2023-05-30 04:42:18 - SimpleLog - INFO: - episode: 8858, ep_reward: 126.322, ep_step: 1000 2023-05-30 04:42:20 - SimpleLog - INFO: - episode: 8859, ep_reward: 136.455, ep_step: 1000 2023-05-30 04:42:22 - SimpleLog - INFO: - episode: 8860, ep_reward: 83.686, ep_step: 1000 2023-05-30 04:42:25 - SimpleLog - INFO: - episode: 8861, ep_reward: 144.949, ep_step: 1000 2023-05-30 04:42:27 - SimpleLog - INFO: - episode: 8862, ep_reward: 145.986, ep_step: 1000 2023-05-30 04:42:29 - SimpleLog - INFO: - episode: 8863, ep_reward: 9.798, ep_step: 946 2023-05-30 04:42:31 - SimpleLog - INFO: - episode: 8864, ep_reward: 124.117, ep_step: 1000 2023-05-30 04:42:33 - SimpleLog - INFO: - episode: 8865, ep_reward: 108.336, ep_step: 1000 2023-05-30 04:42:36 - SimpleLog - INFO: - episode: 8866, ep_reward: 118.106, ep_step: 1000 2023-05-30 04:42:38 - SimpleLog - INFO: - episode: 8867, ep_reward: 138.351, ep_step: 1000 2023-05-30 04:42:40 - SimpleLog - INFO: - episode: 8868, ep_reward: 119.409, ep_step: 1000 2023-05-30 04:42:42 - SimpleLog - INFO: - episode: 8869, ep_reward: 107.127, ep_step: 1000 2023-05-30 04:42:43 - SimpleLog - INFO: - episode: 8870, ep_reward: -77.136, ep_step: 310 2023-05-30 04:42:45 - SimpleLog - INFO: - episode: 8871, ep_reward: 115.333, ep_step: 1000 2023-05-30 04:42:48 - SimpleLog - INFO: - episode: 8872, ep_reward: 116.239, ep_step: 1000 2023-05-30 04:42:50 - SimpleLog - INFO: - episode: 8873, ep_reward: 141.059, ep_step: 1000 2023-05-30 04:42:52 - SimpleLog - INFO: - episode: 8874, ep_reward: 113.741, ep_step: 1000 2023-05-30 04:42:54 - SimpleLog - INFO: - episode: 8875, ep_reward: 91.354, ep_step: 1000 2023-05-30 04:42:56 - SimpleLog - INFO: - episode: 8876, ep_reward: -5.576, ep_step: 819 2023-05-30 04:42:58 - SimpleLog - INFO: - episode: 8877, ep_reward: 85.890, ep_step: 1000 2023-05-30 04:43:01 - SimpleLog - INFO: - episode: 8878, ep_reward: 10.219, ep_step: 1000 2023-05-30 04:43:03 - SimpleLog - INFO: - episode: 8879, ep_reward: -98.951, ep_step: 1000 2023-05-30 04:43:05 - SimpleLog - INFO: - episode: 8880, ep_reward: 75.468, ep_step: 1000 2023-05-30 04:43:06 - SimpleLog - INFO: - episode: 8881, ep_reward: -53.310, ep_step: 525 2023-05-30 04:43:09 - SimpleLog - INFO: - episode: 8882, ep_reward: -90.575, ep_step: 1000 2023-05-30 04:43:11 - SimpleLog - INFO: - episode: 8883, ep_reward: 129.826, ep_step: 1000 2023-05-30 04:43:13 - SimpleLog - INFO: - episode: 8884, ep_reward: 127.914, ep_step: 1000 2023-05-30 04:43:15 - SimpleLog - INFO: - episode: 8885, ep_reward: 112.306, ep_step: 1000 2023-05-30 04:43:17 - SimpleLog - INFO: - episode: 8886, ep_reward: 11.735, ep_step: 824 2023-05-30 04:43:19 - SimpleLog - INFO: - episode: 8887, ep_reward: 0.302, ep_step: 1000 2023-05-30 04:43:21 - SimpleLog - INFO: - episode: 8888, ep_reward: 6.873, ep_step: 837 2023-05-30 04:43:24 - SimpleLog - INFO: - episode: 8889, ep_reward: 117.808, ep_step: 1000 2023-05-30 04:43:26 - SimpleLog - INFO: - episode: 8890, ep_reward: 106.261, ep_step: 1000 2023-05-30 04:43:28 - SimpleLog - INFO: - episode: 8891, ep_reward: 110.050, ep_step: 1000 2023-05-30 04:43:30 - SimpleLog - INFO: - episode: 8892, ep_reward: 3.938, ep_step: 859 2023-05-30 04:43:32 - SimpleLog - INFO: - episode: 8893, ep_reward: 9.457, ep_step: 854 2023-05-30 04:43:34 - SimpleLog - INFO: - episode: 8894, ep_reward: 97.028, ep_step: 1000 2023-05-30 04:43:36 - SimpleLog - INFO: - episode: 8895, ep_reward: -78.314, ep_step: 582 2023-05-30 04:43:38 - SimpleLog - INFO: - episode: 8896, ep_reward: 128.283, ep_step: 1000 2023-05-30 04:43:40 - SimpleLog - INFO: - episode: 8897, ep_reward: 133.967, ep_step: 1000 2023-05-30 04:43:42 - SimpleLog - INFO: - episode: 8898, ep_reward: 105.626, ep_step: 1000 2023-05-30 04:43:45 - SimpleLog - INFO: - episode: 8899, ep_reward: 145.528, ep_step: 1000 2023-05-30 04:43:48 - SimpleLog - INFO: - update_step: 7210000, online_eval_reward: 70.873 2023-05-30 04:43:49 - SimpleLog - INFO: - episode: 8900, ep_reward: 136.009, ep_step: 1000 2023-05-30 04:43:49 - SimpleLog - INFO: - episode: 8901, ep_reward: -99.272, ep_step: 217 2023-05-30 04:43:51 - SimpleLog - INFO: - episode: 8902, ep_reward: -8.693, ep_step: 855 2023-05-30 04:43:53 - SimpleLog - INFO: - episode: 8903, ep_reward: 7.437, ep_step: 900 2023-05-30 04:43:55 - SimpleLog - INFO: - episode: 8904, ep_reward: 3.660, ep_step: 1000 2023-05-30 04:43:57 - SimpleLog - INFO: - episode: 8905, ep_reward: -79.388, ep_step: 642 2023-05-30 04:43:59 - SimpleLog - INFO: - episode: 8906, ep_reward: 123.105, ep_step: 1000 2023-05-30 04:44:01 - SimpleLog - INFO: - episode: 8907, ep_reward: 127.777, ep_step: 1000 2023-05-30 04:44:04 - SimpleLog - INFO: - episode: 8908, ep_reward: 106.653, ep_step: 1000 2023-05-30 04:44:06 - SimpleLog - INFO: - episode: 8909, ep_reward: 105.164, ep_step: 1000 2023-05-30 04:44:08 - SimpleLog - INFO: - episode: 8910, ep_reward: -12.411, ep_step: 1000 2023-05-30 04:44:10 - SimpleLog - INFO: - episode: 8911, ep_reward: -59.596, ep_step: 616 2023-05-30 04:44:12 - SimpleLog - INFO: - episode: 8912, ep_reward: 14.504, ep_step: 1000 2023-05-30 04:44:14 - SimpleLog - INFO: - episode: 8913, ep_reward: 17.942, ep_step: 1000 2023-05-30 04:44:16 - SimpleLog - INFO: - episode: 8914, ep_reward: 10.384, ep_step: 1000 2023-05-30 04:44:19 - SimpleLog - INFO: - episode: 8915, ep_reward: 14.119, ep_step: 1000 2023-05-30 04:44:20 - SimpleLog - INFO: - episode: 8916, ep_reward: -48.500, ep_step: 647 2023-05-30 04:44:22 - SimpleLog - INFO: - episode: 8917, ep_reward: 88.356, ep_step: 1000 2023-05-30 04:44:25 - SimpleLog - INFO: - episode: 8918, ep_reward: 13.677, ep_step: 1000 2023-05-30 04:44:27 - SimpleLog - INFO: - episode: 8919, ep_reward: -107.797, ep_step: 869 2023-05-30 04:44:29 - SimpleLog - INFO: - episode: 8920, ep_reward: -7.909, ep_step: 893 2023-05-30 04:44:31 - SimpleLog - INFO: - episode: 8921, ep_reward: 86.387, ep_step: 1000 2023-05-30 04:44:33 - SimpleLog - INFO: - episode: 8922, ep_reward: 92.117, ep_step: 1000 2023-05-30 04:44:35 - SimpleLog - INFO: - episode: 8923, ep_reward: 100.506, ep_step: 1000 2023-05-30 04:44:38 - SimpleLog - INFO: - episode: 8924, ep_reward: 107.058, ep_step: 1000 2023-05-30 04:44:40 - SimpleLog - INFO: - episode: 8925, ep_reward: -34.743, ep_step: 928 2023-05-30 04:44:42 - SimpleLog - INFO: - episode: 8926, ep_reward: 114.626, ep_step: 1000 2023-05-30 04:44:44 - SimpleLog - INFO: - episode: 8927, ep_reward: 75.580, ep_step: 1000 2023-05-30 04:44:47 - SimpleLog - INFO: - episode: 8928, ep_reward: 133.921, ep_step: 1000 2023-05-30 04:44:49 - SimpleLog - INFO: - episode: 8929, ep_reward: 102.389, ep_step: 1000 2023-05-30 04:44:50 - SimpleLog - INFO: - episode: 8930, ep_reward: -10.359, ep_step: 743 2023-05-30 04:44:53 - SimpleLog - INFO: - episode: 8931, ep_reward: 122.236, ep_step: 1000 2023-05-30 04:44:55 - SimpleLog - INFO: - episode: 8932, ep_reward: 149.871, ep_step: 1000 2023-05-30 04:44:57 - SimpleLog - INFO: - episode: 8933, ep_reward: 7.614, ep_step: 866 2023-05-30 04:44:59 - SimpleLog - INFO: - episode: 8934, ep_reward: 6.811, ep_step: 907 2023-05-30 04:45:00 - SimpleLog - INFO: - episode: 8935, ep_reward: -43.423, ep_step: 556 2023-05-30 04:45:02 - SimpleLog - INFO: - episode: 8936, ep_reward: 98.319, ep_step: 1000 2023-05-30 04:45:05 - SimpleLog - INFO: - episode: 8937, ep_reward: 125.166, ep_step: 1000 2023-05-30 04:45:06 - SimpleLog - INFO: - episode: 8938, ep_reward: -40.133, ep_step: 578 2023-05-30 04:45:08 - SimpleLog - INFO: - episode: 8939, ep_reward: 102.929, ep_step: 1000 2023-05-30 04:45:11 - SimpleLog - INFO: - episode: 8940, ep_reward: 112.894, ep_step: 1000 2023-05-30 04:45:13 - SimpleLog - INFO: - episode: 8941, ep_reward: 97.590, ep_step: 1000 2023-05-30 04:45:14 - SimpleLog - INFO: - episode: 8942, ep_reward: -39.386, ep_step: 511 2023-05-30 04:45:16 - SimpleLog - INFO: - episode: 8943, ep_reward: 113.357, ep_step: 1000 2023-05-30 04:45:18 - SimpleLog - INFO: - episode: 8944, ep_reward: 96.176, ep_step: 1000 2023-05-30 04:45:21 - SimpleLog - INFO: - episode: 8945, ep_reward: 109.719, ep_step: 1000 2023-05-30 04:45:23 - SimpleLog - INFO: - episode: 8946, ep_reward: 128.787, ep_step: 1000 2023-05-30 04:45:25 - SimpleLog - INFO: - episode: 8947, ep_reward: 128.285, ep_step: 1000 2023-05-30 04:45:27 - SimpleLog - INFO: - episode: 8948, ep_reward: 113.620, ep_step: 1000 2023-05-30 04:45:30 - SimpleLog - INFO: - episode: 8949, ep_reward: 119.700, ep_step: 1000 2023-05-30 04:45:32 - SimpleLog - INFO: - episode: 8950, ep_reward: 102.398, ep_step: 1000 2023-05-30 04:45:34 - SimpleLog - INFO: - episode: 8951, ep_reward: 109.439, ep_step: 1000 2023-05-30 04:45:36 - SimpleLog - INFO: - episode: 8952, ep_reward: -21.076, ep_step: 884 2023-05-30 04:45:38 - SimpleLog - INFO: - episode: 8953, ep_reward: 102.612, ep_step: 1000 2023-05-30 04:45:41 - SimpleLog - INFO: - episode: 8954, ep_reward: 32.264, ep_step: 1000 2023-05-30 04:45:43 - SimpleLog - INFO: - episode: 8955, ep_reward: 126.439, ep_step: 1000 2023-05-30 04:45:45 - SimpleLog - INFO: - episode: 8956, ep_reward: 107.044, ep_step: 1000 2023-05-30 04:45:47 - SimpleLog - INFO: - episode: 8957, ep_reward: 128.241, ep_step: 1000 2023-05-30 04:45:50 - SimpleLog - INFO: - episode: 8958, ep_reward: 76.109, ep_step: 1000 2023-05-30 04:45:52 - SimpleLog - INFO: - episode: 8959, ep_reward: 102.654, ep_step: 1000 2023-05-30 04:45:54 - SimpleLog - INFO: - episode: 8960, ep_reward: 98.258, ep_step: 1000 2023-05-30 04:45:56 - SimpleLog - INFO: - episode: 8961, ep_reward: -28.958, ep_step: 849 2023-05-30 04:45:58 - SimpleLog - INFO: - episode: 8962, ep_reward: 126.836, ep_step: 1000 2023-05-30 04:46:01 - SimpleLog - INFO: - episode: 8963, ep_reward: 117.117, ep_step: 1000 2023-05-30 04:46:03 - SimpleLog - INFO: - episode: 8964, ep_reward: 110.135, ep_step: 1000 2023-05-30 04:46:05 - SimpleLog - INFO: - episode: 8965, ep_reward: 114.753, ep_step: 1000 2023-05-30 04:46:07 - SimpleLog - INFO: - episode: 8966, ep_reward: -4.697, ep_step: 984 2023-05-30 04:46:10 - SimpleLog - INFO: - episode: 8967, ep_reward: 80.221, ep_step: 1000 2023-05-30 04:46:12 - SimpleLog - INFO: - episode: 8968, ep_reward: 83.275, ep_step: 1000 2023-05-30 04:46:14 - SimpleLog - INFO: - episode: 8969, ep_reward: 117.155, ep_step: 1000 2023-05-30 04:46:16 - SimpleLog - INFO: - episode: 8970, ep_reward: 98.818, ep_step: 1000 2023-05-30 04:46:19 - SimpleLog - INFO: - episode: 8971, ep_reward: 90.819, ep_step: 1000 2023-05-30 04:46:20 - SimpleLog - INFO: - episode: 8972, ep_reward: -24.126, ep_step: 720 2023-05-30 04:46:22 - SimpleLog - INFO: - episode: 8973, ep_reward: 118.296, ep_step: 1000 2023-05-30 04:46:25 - SimpleLog - INFO: - episode: 8974, ep_reward: -0.271, ep_step: 906 2023-05-30 04:46:28 - SimpleLog - INFO: - update_step: 7280000, online_eval_reward: 73.184 2023-05-30 04:46:29 - SimpleLog - INFO: - episode: 8975, ep_reward: 109.590, ep_step: 1000 2023-05-30 04:46:30 - SimpleLog - INFO: - episode: 8976, ep_reward: -42.502, ep_step: 590 2023-05-30 04:46:32 - SimpleLog - INFO: - episode: 8977, ep_reward: 91.702, ep_step: 1000 2023-05-30 04:46:34 - SimpleLog - INFO: - episode: 8978, ep_reward: 121.411, ep_step: 1000 2023-05-30 04:46:37 - SimpleLog - INFO: - episode: 8979, ep_reward: 75.214, ep_step: 1000 2023-05-30 04:46:39 - SimpleLog - INFO: - episode: 8980, ep_reward: 128.488, ep_step: 1000 2023-05-30 04:46:41 - SimpleLog - INFO: - episode: 8981, ep_reward: 79.641, ep_step: 1000 2023-05-30 04:46:43 - SimpleLog - INFO: - episode: 8982, ep_reward: 1.606, ep_step: 929 2023-05-30 04:46:46 - SimpleLog - INFO: - episode: 8983, ep_reward: 120.232, ep_step: 1000 2023-05-30 04:46:48 - SimpleLog - INFO: - episode: 8984, ep_reward: 133.132, ep_step: 1000 2023-05-30 04:46:50 - SimpleLog - INFO: - episode: 8985, ep_reward: 73.946, ep_step: 1000 2023-05-30 04:46:52 - SimpleLog - INFO: - episode: 8986, ep_reward: 126.598, ep_step: 1000 2023-05-30 04:46:55 - SimpleLog - INFO: - episode: 8987, ep_reward: 114.212, ep_step: 1000 2023-05-30 04:46:57 - SimpleLog - INFO: - episode: 8988, ep_reward: 106.575, ep_step: 1000 2023-05-30 04:46:59 - SimpleLog - INFO: - episode: 8989, ep_reward: 101.169, ep_step: 1000 2023-05-30 04:47:01 - SimpleLog - INFO: - episode: 8990, ep_reward: 96.974, ep_step: 1000 2023-05-30 04:47:03 - SimpleLog - INFO: - episode: 8991, ep_reward: 110.983, ep_step: 1000 2023-05-30 04:47:06 - SimpleLog - INFO: - episode: 8992, ep_reward: 97.302, ep_step: 1000 2023-05-30 04:47:08 - SimpleLog - INFO: - episode: 8993, ep_reward: 93.891, ep_step: 1000 2023-05-30 04:47:10 - SimpleLog - INFO: - episode: 8994, ep_reward: -26.233, ep_step: 1000 2023-05-30 04:47:12 - SimpleLog - INFO: - episode: 8995, ep_reward: -75.410, ep_step: 1000 2023-05-30 04:47:15 - SimpleLog - INFO: - episode: 8996, ep_reward: 103.187, ep_step: 1000 2023-05-30 04:47:17 - SimpleLog - INFO: - episode: 8997, ep_reward: -42.762, ep_step: 982 2023-05-30 04:47:19 - SimpleLog - INFO: - episode: 8998, ep_reward: -80.818, ep_step: 1000 2023-05-30 04:47:21 - SimpleLog - INFO: - episode: 8999, ep_reward: -4.707, ep_step: 1000 2023-05-30 04:47:24 - SimpleLog - INFO: - episode: 9000, ep_reward: 9.313, ep_step: 1000 2023-05-30 04:47:24 - SimpleLog - INFO: - episode: 9001, ep_reward: -106.527, ep_step: 128 2023-05-30 04:47:26 - SimpleLog - INFO: - episode: 9002, ep_reward: -88.493, ep_step: 1000 2023-05-30 04:47:28 - SimpleLog - INFO: - episode: 9003, ep_reward: -96.316, ep_step: 1000 2023-05-30 04:47:31 - SimpleLog - INFO: - episode: 9004, ep_reward: -98.712, ep_step: 1000 2023-05-30 04:47:33 - SimpleLog - INFO: - episode: 9005, ep_reward: -93.435, ep_step: 1000 2023-05-30 04:47:35 - SimpleLog - INFO: - episode: 9006, ep_reward: -94.180, ep_step: 1000 2023-05-30 04:47:37 - SimpleLog - INFO: - episode: 9007, ep_reward: -86.599, ep_step: 1000 2023-05-30 04:47:40 - SimpleLog - INFO: - episode: 9008, ep_reward: -90.695, ep_step: 1000 2023-05-30 04:47:42 - SimpleLog - INFO: - episode: 9009, ep_reward: -90.982, ep_step: 1000 2023-05-30 04:47:44 - SimpleLog - INFO: - episode: 9010, ep_reward: -91.502, ep_step: 1000 2023-05-30 04:47:46 - SimpleLog - INFO: - episode: 9011, ep_reward: -86.585, ep_step: 1000 2023-05-30 04:47:49 - SimpleLog - INFO: - episode: 9012, ep_reward: -91.497, ep_step: 1000 2023-05-30 04:47:51 - SimpleLog - INFO: - episode: 9013, ep_reward: -87.541, ep_step: 1000 2023-05-30 04:47:53 - SimpleLog - INFO: - episode: 9014, ep_reward: -90.420, ep_step: 1000 2023-05-30 04:47:53 - SimpleLog - INFO: - episode: 9015, ep_reward: -107.582, ep_step: 100 2023-05-30 04:47:56 - SimpleLog - INFO: - episode: 9016, ep_reward: -85.917, ep_step: 1000 2023-05-30 04:47:58 - SimpleLog - INFO: - episode: 9017, ep_reward: -82.987, ep_step: 1000 2023-05-30 04:48:00 - SimpleLog - INFO: - episode: 9018, ep_reward: -83.171, ep_step: 1000 2023-05-30 04:48:01 - SimpleLog - INFO: - episode: 9019, ep_reward: -65.179, ep_step: 455 2023-05-30 04:48:02 - SimpleLog - INFO: - episode: 9020, ep_reward: -67.530, ep_step: 522 2023-05-30 04:48:03 - SimpleLog - INFO: - episode: 9021, ep_reward: -114.198, ep_step: 91 2023-05-30 04:48:03 - SimpleLog - INFO: - episode: 9022, ep_reward: -115.898, ep_step: 96 2023-05-30 04:48:03 - SimpleLog - INFO: - episode: 9023, ep_reward: -113.500, ep_step: 94 2023-05-30 04:48:05 - SimpleLog - INFO: - episode: 9024, ep_reward: -90.470, ep_step: 1000 2023-05-30 04:48:06 - SimpleLog - INFO: - episode: 9025, ep_reward: -153.922, ep_step: 349 2023-05-30 04:48:07 - SimpleLog - INFO: - episode: 9026, ep_reward: -142.779, ep_step: 242 2023-05-30 04:48:07 - SimpleLog - INFO: - episode: 9027, ep_reward: -154.250, ep_step: 366 2023-05-30 04:48:08 - SimpleLog - INFO: - episode: 9028, ep_reward: -142.782, ep_step: 227 2023-05-30 04:48:08 - SimpleLog - INFO: - episode: 9029, ep_reward: -140.217, ep_step: 214 2023-05-30 04:48:10 - SimpleLog - INFO: - episode: 9030, ep_reward: -200.077, ep_step: 891 2023-05-30 04:48:12 - SimpleLog - INFO: - episode: 9031, ep_reward: -169.909, ep_step: 545 2023-05-30 04:48:12 - SimpleLog - INFO: - episode: 9032, ep_reward: -147.634, ep_step: 300 2023-05-30 04:48:13 - SimpleLog - INFO: - episode: 9033, ep_reward: -159.629, ep_step: 417 2023-05-30 04:48:14 - SimpleLog - INFO: - episode: 9034, ep_reward: -141.671, ep_step: 227 2023-05-30 04:48:14 - SimpleLog - INFO: - episode: 9035, ep_reward: -143.965, ep_step: 256 2023-05-30 04:48:15 - SimpleLog - INFO: - episode: 9036, ep_reward: -146.893, ep_step: 272 2023-05-30 04:48:16 - SimpleLog - INFO: - episode: 9037, ep_reward: -155.354, ep_step: 375 2023-05-30 04:48:17 - SimpleLog - INFO: - episode: 9038, ep_reward: -153.204, ep_step: 364 2023-05-30 04:48:18 - SimpleLog - INFO: - episode: 9039, ep_reward: -172.143, ep_step: 588 2023-05-30 04:48:19 - SimpleLog - INFO: - episode: 9040, ep_reward: -171.000, ep_step: 576 2023-05-30 04:48:20 - SimpleLog - INFO: - episode: 9041, ep_reward: -160.510, ep_step: 469 2023-05-30 04:48:22 - SimpleLog - INFO: - episode: 9042, ep_reward: -167.665, ep_step: 551 2023-05-30 04:48:22 - SimpleLog - INFO: - episode: 9043, ep_reward: -118.030, ep_step: 156 2023-05-30 04:48:23 - SimpleLog - INFO: - episode: 9044, ep_reward: -158.050, ep_step: 438 2023-05-30 04:48:24 - SimpleLog - INFO: - episode: 9045, ep_reward: -157.183, ep_step: 431 2023-05-30 04:48:25 - SimpleLog - INFO: - episode: 9046, ep_reward: -163.010, ep_step: 488 2023-05-30 04:48:26 - SimpleLog - INFO: - episode: 9047, ep_reward: -173.777, ep_step: 613 2023-05-30 04:48:28 - SimpleLog - INFO: - episode: 9048, ep_reward: -179.413, ep_step: 674 2023-05-30 04:48:30 - SimpleLog - INFO: - episode: 9049, ep_reward: -195.174, ep_step: 864 2023-05-30 04:48:30 - SimpleLog - INFO: - episode: 9050, ep_reward: -122.034, ep_step: 142 2023-05-30 04:48:32 - SimpleLog - INFO: - episode: 9051, ep_reward: -88.787, ep_step: 1000 2023-05-30 04:48:33 - SimpleLog - INFO: - episode: 9052, ep_reward: -109.829, ep_step: 151 2023-05-30 04:48:33 - SimpleLog - INFO: - episode: 9053, ep_reward: -112.479, ep_step: 275 2023-05-30 04:48:34 - SimpleLog - INFO: - episode: 9054, ep_reward: -116.495, ep_step: 82 2023-05-30 04:48:34 - SimpleLog - INFO: - episode: 9055, ep_reward: -117.200, ep_step: 72 2023-05-30 04:48:34 - SimpleLog - INFO: - episode: 9056, ep_reward: -116.502, ep_step: 68 2023-05-30 04:48:36 - SimpleLog - INFO: - episode: 9057, ep_reward: -85.162, ep_step: 1000 2023-05-30 04:48:38 - SimpleLog - INFO: - episode: 9058, ep_reward: -42.781, ep_step: 1000 2023-05-30 04:48:39 - SimpleLog - INFO: - episode: 9059, ep_reward: -102.918, ep_step: 195 2023-05-30 04:48:39 - SimpleLog - INFO: - episode: 9060, ep_reward: -102.949, ep_step: 219 2023-05-30 04:48:40 - SimpleLog - INFO: - episode: 9061, ep_reward: -107.869, ep_step: 224 2023-05-30 04:48:40 - SimpleLog - INFO: - episode: 9062, ep_reward: -112.268, ep_step: 229 2023-05-30 04:48:43 - SimpleLog - INFO: - episode: 9063, ep_reward: -59.282, ep_step: 1000 2023-05-30 04:48:43 - SimpleLog - INFO: - episode: 9064, ep_reward: -106.541, ep_step: 219 2023-05-30 04:48:44 - SimpleLog - INFO: - episode: 9065, ep_reward: -108.784, ep_step: 251 2023-05-30 04:48:44 - SimpleLog - INFO: - episode: 9066, ep_reward: -112.131, ep_step: 234 2023-05-30 04:48:46 - SimpleLog - INFO: - episode: 9067, ep_reward: -61.936, ep_step: 1000 2023-05-30 04:48:47 - SimpleLog - INFO: - episode: 9068, ep_reward: -117.515, ep_step: 202 2023-05-30 04:48:47 - SimpleLog - INFO: - episode: 9069, ep_reward: -128.633, ep_step: 152 2023-05-30 04:48:48 - SimpleLog - INFO: - episode: 9070, ep_reward: -138.142, ep_step: 460 2023-05-30 04:48:49 - SimpleLog - INFO: - episode: 9071, ep_reward: -112.200, ep_step: 242 2023-05-30 04:48:50 - SimpleLog - INFO: - episode: 9072, ep_reward: -144.719, ep_step: 453 2023-05-30 04:48:51 - SimpleLog - INFO: - episode: 9073, ep_reward: -127.466, ep_step: 353 2023-05-30 04:48:52 - SimpleLog - INFO: - episode: 9074, ep_reward: -134.728, ep_step: 419 2023-05-30 04:48:53 - SimpleLog - INFO: - episode: 9075, ep_reward: -130.951, ep_step: 411 2023-05-30 04:48:53 - SimpleLog - INFO: - episode: 9076, ep_reward: -123.539, ep_step: 313 2023-05-30 04:48:53 - SimpleLog - INFO: - episode: 9077, ep_reward: -116.995, ep_step: 70 2023-05-30 04:48:54 - SimpleLog - INFO: - episode: 9078, ep_reward: -119.401, ep_step: 100 2023-05-30 04:48:55 - SimpleLog - INFO: - episode: 9079, ep_reward: -131.704, ep_step: 620 2023-05-30 04:48:55 - SimpleLog - INFO: - episode: 9080, ep_reward: -123.683, ep_step: 94 2023-05-30 04:48:56 - SimpleLog - INFO: - episode: 9081, ep_reward: -125.290, ep_step: 378 2023-05-30 04:48:56 - SimpleLog - INFO: - episode: 9082, ep_reward: -129.497, ep_step: 144 2023-05-30 04:48:57 - SimpleLog - INFO: - episode: 9083, ep_reward: -129.277, ep_step: 123 2023-05-30 04:48:57 - SimpleLog - INFO: - episode: 9084, ep_reward: -125.949, ep_step: 114 2023-05-30 04:48:58 - SimpleLog - INFO: - episode: 9085, ep_reward: -126.747, ep_step: 330 2023-05-30 04:48:58 - SimpleLog - INFO: - episode: 9086, ep_reward: -124.775, ep_step: 126 2023-05-30 04:48:58 - SimpleLog - INFO: - episode: 9087, ep_reward: -123.403, ep_step: 119 2023-05-30 04:48:59 - SimpleLog - INFO: - episode: 9088, ep_reward: -117.502, ep_step: 245 2023-05-30 04:48:59 - SimpleLog - INFO: - episode: 9089, ep_reward: -116.868, ep_step: 234 2023-05-30 04:49:00 - SimpleLog - INFO: - episode: 9090, ep_reward: -134.685, ep_step: 424 2023-05-30 04:49:01 - SimpleLog - INFO: - episode: 9091, ep_reward: -128.147, ep_step: 137 2023-05-30 04:49:01 - SimpleLog - INFO: - episode: 9092, ep_reward: -128.448, ep_step: 229 2023-05-30 04:49:02 - SimpleLog - INFO: - episode: 9093, ep_reward: -137.228, ep_step: 292 2023-05-30 04:49:04 - SimpleLog - INFO: - episode: 9094, ep_reward: -85.447, ep_step: 1000 2023-05-30 04:49:07 - SimpleLog - INFO: - update_step: 7350000, online_eval_reward: -117.357 2023-05-30 04:49:08 - SimpleLog - INFO: - episode: 9095, ep_reward: -79.587, ep_step: 1000 2023-05-30 04:49:08 - SimpleLog - INFO: - episode: 9096, ep_reward: -132.500, ep_step: 142 2023-05-30 04:49:09 - SimpleLog - INFO: - episode: 9097, ep_reward: -129.337, ep_step: 156 2023-05-30 04:49:11 - SimpleLog - INFO: - episode: 9098, ep_reward: -94.307, ep_step: 1000 2023-05-30 04:49:13 - SimpleLog - INFO: - episode: 9099, ep_reward: -53.516, ep_step: 1000 2023-05-30 04:49:13 - SimpleLog - INFO: - episode: 9100, ep_reward: -120.713, ep_step: 108 2023-05-30 04:49:13 - SimpleLog - INFO: - episode: 9101, ep_reward: -113.582, ep_step: 68 2023-05-30 04:49:16 - SimpleLog - INFO: - episode: 9102, ep_reward: -57.239, ep_step: 1000 2023-05-30 04:49:18 - SimpleLog - INFO: - episode: 9103, ep_reward: -55.662, ep_step: 1000 2023-05-30 04:49:19 - SimpleLog - INFO: - episode: 9104, ep_reward: -108.718, ep_step: 219 2023-05-30 04:49:21 - SimpleLog - INFO: - episode: 9105, ep_reward: -110.178, ep_step: 945 2023-05-30 04:49:23 - SimpleLog - INFO: - episode: 9106, ep_reward: -67.018, ep_step: 1000 2023-05-30 04:49:23 - SimpleLog - INFO: - episode: 9107, ep_reward: -125.207, ep_step: 102 2023-05-30 04:49:24 - SimpleLog - INFO: - episode: 9108, ep_reward: -115.125, ep_step: 292 2023-05-30 04:49:24 - SimpleLog - INFO: - episode: 9109, ep_reward: -120.804, ep_step: 81 2023-05-30 04:49:24 - SimpleLog - INFO: - episode: 9110, ep_reward: -102.823, ep_step: 137 2023-05-30 04:49:25 - SimpleLog - INFO: - episode: 9111, ep_reward: -132.949, ep_step: 355 2023-05-30 04:49:27 - SimpleLog - INFO: - episode: 9112, ep_reward: -66.660, ep_step: 1000 2023-05-30 04:49:30 - SimpleLog - INFO: - episode: 9113, ep_reward: 12.711, ep_step: 1000 2023-05-30 04:49:30 - SimpleLog - INFO: - episode: 9114, ep_reward: -108.787, ep_step: 186 2023-05-30 04:49:32 - SimpleLog - INFO: - episode: 9115, ep_reward: -64.009, ep_step: 1000 2023-05-30 04:49:35 - SimpleLog - INFO: - episode: 9116, ep_reward: 36.717, ep_step: 1000 2023-05-30 04:49:36 - SimpleLog - INFO: - episode: 9117, ep_reward: -99.394, ep_step: 742 2023-05-30 04:49:38 - SimpleLog - INFO: - episode: 9118, ep_reward: -38.069, ep_step: 1000 2023-05-30 04:49:41 - SimpleLog - INFO: - episode: 9119, ep_reward: 39.959, ep_step: 1000 2023-05-30 04:49:43 - SimpleLog - INFO: - episode: 9120, ep_reward: -55.656, ep_step: 956 2023-05-30 04:49:44 - SimpleLog - INFO: - episode: 9121, ep_reward: -94.489, ep_step: 622 2023-05-30 04:49:46 - SimpleLog - INFO: - episode: 9122, ep_reward: -49.140, ep_step: 802 2023-05-30 04:49:48 - SimpleLog - INFO: - episode: 9123, ep_reward: 0.220, ep_step: 1000 2023-05-30 04:49:50 - SimpleLog - INFO: - episode: 9124, ep_reward: -0.228, ep_step: 1000 2023-05-30 04:49:53 - SimpleLog - INFO: - episode: 9125, ep_reward: 25.700, ep_step: 1000 2023-05-30 04:49:55 - SimpleLog - INFO: - episode: 9126, ep_reward: 4.284, ep_step: 1000 2023-05-30 04:49:57 - SimpleLog - INFO: - episode: 9127, ep_reward: -48.061, ep_step: 972 2023-05-30 04:49:59 - SimpleLog - INFO: - episode: 9128, ep_reward: 19.104, ep_step: 1000 2023-05-30 04:50:02 - SimpleLog - INFO: - episode: 9129, ep_reward: 10.269, ep_step: 1000 2023-05-30 04:50:04 - SimpleLog - INFO: - episode: 9130, ep_reward: 6.144, ep_step: 1000 2023-05-30 04:50:06 - SimpleLog - INFO: - episode: 9131, ep_reward: -49.711, ep_step: 1000 2023-05-30 04:50:08 - SimpleLog - INFO: - episode: 9132, ep_reward: -84.351, ep_step: 627 2023-05-30 04:50:10 - SimpleLog - INFO: - episode: 9133, ep_reward: 73.866, ep_step: 1000 2023-05-30 04:50:12 - SimpleLog - INFO: - episode: 9134, ep_reward: 4.678, ep_step: 1000 2023-05-30 04:50:14 - SimpleLog - INFO: - episode: 9135, ep_reward: 11.674, ep_step: 1000 2023-05-30 04:50:16 - SimpleLog - INFO: - episode: 9136, ep_reward: -77.545, ep_step: 511 2023-05-30 04:50:18 - SimpleLog - INFO: - episode: 9137, ep_reward: 6.036, ep_step: 1000 2023-05-30 04:50:20 - SimpleLog - INFO: - episode: 9138, ep_reward: 2.544, ep_step: 1000 2023-05-30 04:50:22 - SimpleLog - INFO: - episode: 9139, ep_reward: 5.150, ep_step: 1000 2023-05-30 04:50:23 - SimpleLog - INFO: - episode: 9140, ep_reward: -132.304, ep_step: 202 2023-05-30 04:50:25 - SimpleLog - INFO: - episode: 9141, ep_reward: -76.396, ep_step: 1000 2023-05-30 04:50:27 - SimpleLog - INFO: - episode: 9142, ep_reward: -77.904, ep_step: 1000 2023-05-30 04:50:30 - SimpleLog - INFO: - episode: 9143, ep_reward: -7.151, ep_step: 1000 2023-05-30 04:50:32 - SimpleLog - INFO: - episode: 9144, ep_reward: 6.346, ep_step: 1000 2023-05-30 04:50:33 - SimpleLog - INFO: - episode: 9145, ep_reward: -184.710, ep_step: 729 2023-05-30 04:50:36 - SimpleLog - INFO: - episode: 9146, ep_reward: -87.840, ep_step: 1000 2023-05-30 04:50:38 - SimpleLog - INFO: - episode: 9147, ep_reward: -99.160, ep_step: 1000 2023-05-30 04:50:40 - SimpleLog - INFO: - episode: 9148, ep_reward: -189.825, ep_step: 784 2023-05-30 04:50:41 - SimpleLog - INFO: - episode: 9149, ep_reward: -173.658, ep_step: 582 2023-05-30 04:50:42 - SimpleLog - INFO: - episode: 9150, ep_reward: -151.691, ep_step: 328 2023-05-30 04:50:43 - SimpleLog - INFO: - episode: 9151, ep_reward: -177.979, ep_step: 615 2023-05-30 04:50:44 - SimpleLog - INFO: - episode: 9152, ep_reward: -144.424, ep_step: 244 2023-05-30 04:50:45 - SimpleLog - INFO: - episode: 9153, ep_reward: -179.084, ep_step: 653 2023-05-30 04:50:46 - SimpleLog - INFO: - episode: 9154, ep_reward: -145.721, ep_step: 249 2023-05-30 04:50:47 - SimpleLog - INFO: - episode: 9155, ep_reward: -164.765, ep_step: 478 2023-05-30 04:50:48 - SimpleLog - INFO: - episode: 9156, ep_reward: -183.551, ep_step: 714 2023-05-30 04:50:51 - SimpleLog - INFO: - episode: 9157, ep_reward: -89.288, ep_step: 1000 2023-05-30 04:50:53 - SimpleLog - INFO: - episode: 9158, ep_reward: -89.414, ep_step: 1000 2023-05-30 04:50:55 - SimpleLog - INFO: - episode: 9159, ep_reward: -78.734, ep_step: 1000 2023-05-30 04:50:57 - SimpleLog - INFO: - episode: 9160, ep_reward: -74.327, ep_step: 1000 2023-05-30 04:50:59 - SimpleLog - INFO: - episode: 9161, ep_reward: -111.367, ep_step: 659 2023-05-30 04:51:01 - SimpleLog - INFO: - episode: 9162, ep_reward: -72.679, ep_step: 1000 2023-05-30 04:51:03 - SimpleLog - INFO: - episode: 9163, ep_reward: -147.620, ep_step: 944 2023-05-30 04:51:04 - SimpleLog - INFO: - episode: 9164, ep_reward: -102.769, ep_step: 191 2023-05-30 04:51:06 - SimpleLog - INFO: - episode: 9165, ep_reward: -67.771, ep_step: 1000 2023-05-30 04:51:07 - SimpleLog - INFO: - episode: 9166, ep_reward: -115.745, ep_step: 299 2023-05-30 04:51:08 - SimpleLog - INFO: - episode: 9167, ep_reward: -106.701, ep_step: 470 2023-05-30 04:51:08 - SimpleLog - INFO: - episode: 9168, ep_reward: -111.604, ep_step: 265 2023-05-30 04:51:10 - SimpleLog - INFO: - episode: 9169, ep_reward: -5.314, ep_step: 1000 2023-05-30 04:51:11 - SimpleLog - INFO: - episode: 9170, ep_reward: -114.037, ep_step: 371 2023-05-30 04:51:13 - SimpleLog - INFO: - episode: 9171, ep_reward: -83.660, ep_step: 563 2023-05-30 04:51:13 - SimpleLog - INFO: - episode: 9172, ep_reward: -113.196, ep_step: 348 2023-05-30 04:51:16 - SimpleLog - INFO: - episode: 9173, ep_reward: 14.094, ep_step: 1000 2023-05-30 04:51:18 - SimpleLog - INFO: - episode: 9174, ep_reward: 13.536, ep_step: 1000 2023-05-30 04:51:18 - SimpleLog - INFO: - episode: 9175, ep_reward: -98.955, ep_step: 183 2023-05-30 04:51:19 - SimpleLog - INFO: - episode: 9176, ep_reward: -96.582, ep_step: 426 2023-05-30 04:51:21 - SimpleLog - INFO: - episode: 9177, ep_reward: -86.233, ep_step: 595 2023-05-30 04:51:21 - SimpleLog - INFO: - episode: 9178, ep_reward: -122.901, ep_step: 334 2023-05-30 04:51:24 - SimpleLog - INFO: - episode: 9179, ep_reward: 10.326, ep_step: 1000 2023-05-30 04:51:24 - SimpleLog - INFO: - episode: 9180, ep_reward: -103.222, ep_step: 217 2023-05-30 04:51:26 - SimpleLog - INFO: - episode: 9181, ep_reward: 9.455, ep_step: 1000 2023-05-30 04:51:27 - SimpleLog - INFO: - episode: 9182, ep_reward: -87.013, ep_step: 384 2023-05-30 04:51:28 - SimpleLog - INFO: - episode: 9183, ep_reward: -95.735, ep_step: 432 2023-05-30 04:51:29 - SimpleLog - INFO: - episode: 9184, ep_reward: -102.395, ep_step: 259 2023-05-30 04:51:31 - SimpleLog - INFO: - episode: 9185, ep_reward: -40.036, ep_step: 1000 2023-05-30 04:51:33 - SimpleLog - INFO: - episode: 9186, ep_reward: -54.465, ep_step: 1000 2023-05-30 04:51:35 - SimpleLog - INFO: - episode: 9187, ep_reward: -101.197, ep_step: 802 2023-05-30 04:51:37 - SimpleLog - INFO: - episode: 9188, ep_reward: -48.889, ep_step: 1000 2023-05-30 04:51:40 - SimpleLog - INFO: - episode: 9189, ep_reward: -48.559, ep_step: 1000 2023-05-30 04:51:42 - SimpleLog - INFO: - episode: 9190, ep_reward: -93.977, ep_step: 934 2023-05-30 04:51:44 - SimpleLog - INFO: - episode: 9191, ep_reward: -96.310, ep_step: 888 2023-05-30 04:51:47 - SimpleLog - INFO: - update_step: 7420000, online_eval_reward: 4.702 2023-05-30 04:51:48 - SimpleLog - INFO: - episode: 9192, ep_reward: -15.971, ep_step: 1000 2023-05-30 04:51:50 - SimpleLog - INFO: - episode: 9193, ep_reward: -77.616, ep_step: 746 2023-05-30 04:51:52 - SimpleLog - INFO: - episode: 9194, ep_reward: 3.062, ep_step: 1000 2023-05-30 04:51:54 - SimpleLog - INFO: - episode: 9195, ep_reward: 2.359, ep_step: 1000 2023-05-30 04:51:56 - SimpleLog - INFO: - episode: 9196, ep_reward: 19.608, ep_step: 1000 2023-05-30 04:51:57 - SimpleLog - INFO: - episode: 9197, ep_reward: -117.128, ep_step: 77 2023-05-30 04:51:59 - SimpleLog - INFO: - episode: 9198, ep_reward: 54.791, ep_step: 1000 2023-05-30 04:52:01 - SimpleLog - INFO: - episode: 9199, ep_reward: -66.944, ep_step: 1000 2023-05-30 04:52:03 - SimpleLog - INFO: - episode: 9200, ep_reward: 7.333, ep_step: 1000 2023-05-30 04:52:06 - SimpleLog - INFO: - episode: 9201, ep_reward: 52.389, ep_step: 1000 2023-05-30 04:52:06 - SimpleLog - INFO: - episode: 9202, ep_reward: -113.281, ep_step: 114 2023-05-30 04:52:08 - SimpleLog - INFO: - episode: 9203, ep_reward: 3.380, ep_step: 1000 2023-05-30 04:52:10 - SimpleLog - INFO: - episode: 9204, ep_reward: -63.414, ep_step: 1000 2023-05-30 04:52:13 - SimpleLog - INFO: - episode: 9205, ep_reward: -36.840, ep_step: 1000 2023-05-30 04:52:15 - SimpleLog - INFO: - episode: 9206, ep_reward: 4.737, ep_step: 1000 2023-05-30 04:52:17 - SimpleLog - INFO: - episode: 9207, ep_reward: 3.230, ep_step: 1000 2023-05-30 04:52:19 - SimpleLog - INFO: - episode: 9208, ep_reward: 5.758, ep_step: 1000 2023-05-30 04:52:21 - SimpleLog - INFO: - episode: 9209, ep_reward: 5.629, ep_step: 1000 2023-05-30 04:52:24 - SimpleLog - INFO: - episode: 9210, ep_reward: 17.827, ep_step: 1000 2023-05-30 04:52:26 - SimpleLog - INFO: - episode: 9211, ep_reward: -12.484, ep_step: 1000 2023-05-30 04:52:28 - SimpleLog - INFO: - episode: 9212, ep_reward: 5.092, ep_step: 1000 2023-05-30 04:52:29 - SimpleLog - INFO: - episode: 9213, ep_reward: -92.786, ep_step: 238 2023-05-30 04:52:31 - SimpleLog - INFO: - episode: 9214, ep_reward: 9.826, ep_step: 1000 2023-05-30 04:52:33 - SimpleLog - INFO: - episode: 9215, ep_reward: 72.891, ep_step: 1000 2023-05-30 04:52:36 - SimpleLog - INFO: - episode: 9216, ep_reward: 23.373, ep_step: 1000 2023-05-30 04:52:38 - SimpleLog - INFO: - episode: 9217, ep_reward: 1.505, ep_step: 1000 2023-05-30 04:52:40 - SimpleLog - INFO: - episode: 9218, ep_reward: 1.132, ep_step: 1000 2023-05-30 04:52:41 - SimpleLog - INFO: - episode: 9219, ep_reward: -62.126, ep_step: 610 2023-05-30 04:52:44 - SimpleLog - INFO: - episode: 9220, ep_reward: 63.669, ep_step: 1000 2023-05-30 04:52:46 - SimpleLog - INFO: - episode: 9221, ep_reward: 75.171, ep_step: 1000 2023-05-30 04:52:48 - SimpleLog - INFO: - episode: 9222, ep_reward: 65.894, ep_step: 1000 2023-05-30 04:52:50 - SimpleLog - INFO: - episode: 9223, ep_reward: -7.461, ep_step: 1000 2023-05-30 04:52:52 - SimpleLog - INFO: - episode: 9224, ep_reward: -76.186, ep_step: 841 2023-05-30 04:52:55 - SimpleLog - INFO: - episode: 9225, ep_reward: -36.731, ep_step: 1000 2023-05-30 04:52:57 - SimpleLog - INFO: - episode: 9226, ep_reward: -66.307, ep_step: 1000 2023-05-30 04:52:59 - SimpleLog - INFO: - episode: 9227, ep_reward: 11.703, ep_step: 1000 2023-05-30 04:53:01 - SimpleLog - INFO: - episode: 9228, ep_reward: -42.402, ep_step: 1000 2023-05-30 04:53:04 - SimpleLog - INFO: - episode: 9229, ep_reward: -29.966, ep_step: 1000 2023-05-30 04:53:04 - SimpleLog - INFO: - episode: 9230, ep_reward: -97.389, ep_step: 201 2023-05-30 04:53:06 - SimpleLog - INFO: - episode: 9231, ep_reward: 66.366, ep_step: 1000 2023-05-30 04:53:09 - SimpleLog - INFO: - episode: 9232, ep_reward: 95.572, ep_step: 1000 2023-05-30 04:53:11 - SimpleLog - INFO: - episode: 9233, ep_reward: -13.169, ep_step: 1000 2023-05-30 04:53:13 - SimpleLog - INFO: - episode: 9234, ep_reward: -1.104, ep_step: 1000 2023-05-30 04:53:15 - SimpleLog - INFO: - episode: 9235, ep_reward: -2.739, ep_step: 1000 2023-05-30 04:53:18 - SimpleLog - INFO: - episode: 9236, ep_reward: -0.826, ep_step: 1000 2023-05-30 04:53:20 - SimpleLog - INFO: - episode: 9237, ep_reward: -2.669, ep_step: 1000 2023-05-30 04:53:22 - SimpleLog - INFO: - episode: 9238, ep_reward: -24.040, ep_step: 1000 2023-05-30 04:53:25 - SimpleLog - INFO: - episode: 9239, ep_reward: -5.097, ep_step: 1000 2023-05-30 04:53:27 - SimpleLog - INFO: - episode: 9240, ep_reward: -67.208, ep_step: 1000 2023-05-30 04:53:29 - SimpleLog - INFO: - episode: 9241, ep_reward: -64.359, ep_step: 1000 2023-05-30 04:53:31 - SimpleLog - INFO: - episode: 9242, ep_reward: -83.041, ep_step: 1000 2023-05-30 04:53:34 - SimpleLog - INFO: - episode: 9243, ep_reward: -5.876, ep_step: 1000 2023-05-30 04:53:34 - SimpleLog - INFO: - episode: 9244, ep_reward: -113.731, ep_step: 55 2023-05-30 04:53:36 - SimpleLog - INFO: - episode: 9245, ep_reward: -79.135, ep_step: 1000 2023-05-30 04:53:36 - SimpleLog - INFO: - episode: 9246, ep_reward: -118.218, ep_step: 68 2023-05-30 04:53:36 - SimpleLog - INFO: - episode: 9247, ep_reward: -114.537, ep_step: 73 2023-05-30 04:53:37 - SimpleLog - INFO: - episode: 9248, ep_reward: -117.301, ep_step: 64 2023-05-30 04:53:37 - SimpleLog - INFO: - episode: 9249, ep_reward: -122.063, ep_step: 108 2023-05-30 04:53:37 - SimpleLog - INFO: - episode: 9250, ep_reward: -115.186, ep_step: 115 2023-05-30 04:53:37 - SimpleLog - INFO: - episode: 9251, ep_reward: -129.359, ep_step: 173 2023-05-30 04:53:38 - SimpleLog - INFO: - episode: 9252, ep_reward: -123.462, ep_step: 99 2023-05-30 04:53:38 - SimpleLog - INFO: - episode: 9253, ep_reward: -139.406, ep_step: 210 2023-05-30 04:53:38 - SimpleLog - INFO: - episode: 9254, ep_reward: -124.681, ep_step: 111 2023-05-30 04:53:41 - SimpleLog - INFO: - episode: 9255, ep_reward: -84.570, ep_step: 1000 2023-05-30 04:53:42 - SimpleLog - INFO: - episode: 9256, ep_reward: -158.307, ep_step: 540 2023-05-30 04:53:44 - SimpleLog - INFO: - episode: 9257, ep_reward: -176.539, ep_step: 755 2023-05-30 04:53:44 - SimpleLog - INFO: - episode: 9258, ep_reward: -131.516, ep_step: 187 2023-05-30 04:53:45 - SimpleLog - INFO: - episode: 9259, ep_reward: -140.298, ep_step: 291 2023-05-30 04:53:47 - SimpleLog - INFO: - episode: 9260, ep_reward: -95.837, ep_step: 1000 2023-05-30 04:53:47 - SimpleLog - INFO: - episode: 9261, ep_reward: -111.865, ep_step: 55 2023-05-30 04:53:48 - SimpleLog - INFO: - episode: 9262, ep_reward: -148.867, ep_step: 272 2023-05-30 04:53:49 - SimpleLog - INFO: - episode: 9263, ep_reward: -191.213, ep_step: 772 2023-05-30 04:53:50 - SimpleLog - INFO: - episode: 9264, ep_reward: -150.394, ep_step: 332 2023-05-30 04:53:51 - SimpleLog - INFO: - episode: 9265, ep_reward: -163.691, ep_step: 454 2023-05-30 04:53:52 - SimpleLog - INFO: - episode: 9266, ep_reward: -158.309, ep_step: 437 2023-05-30 04:53:53 - SimpleLog - INFO: - episode: 9267, ep_reward: -149.209, ep_step: 329 2023-05-30 04:53:54 - SimpleLog - INFO: - episode: 9268, ep_reward: -151.371, ep_step: 365 2023-05-30 04:53:55 - SimpleLog - INFO: - episode: 9269, ep_reward: -151.226, ep_step: 367 2023-05-30 04:53:56 - SimpleLog - INFO: - episode: 9270, ep_reward: -155.861, ep_step: 427 2023-05-30 04:53:56 - SimpleLog - INFO: - episode: 9271, ep_reward: -145.610, ep_step: 302 2023-05-30 04:53:58 - SimpleLog - INFO: - episode: 9272, ep_reward: -177.399, ep_step: 707 2023-05-30 04:54:00 - SimpleLog - INFO: - episode: 9273, ep_reward: -94.935, ep_step: 1000 2023-05-30 04:54:01 - SimpleLog - INFO: - episode: 9274, ep_reward: -169.720, ep_step: 604 2023-05-30 04:54:04 - SimpleLog - INFO: - episode: 9275, ep_reward: -90.552, ep_step: 1000 2023-05-30 04:54:06 - SimpleLog - INFO: - episode: 9276, ep_reward: -189.628, ep_step: 872 2023-05-30 04:54:07 - SimpleLog - INFO: - episode: 9277, ep_reward: -161.797, ep_step: 513 2023-05-30 04:54:07 - SimpleLog - INFO: - episode: 9278, ep_reward: -118.112, ep_step: 89 2023-05-30 04:54:07 - SimpleLog - INFO: - episode: 9279, ep_reward: -117.413, ep_step: 64 2023-05-30 04:54:07 - SimpleLog - INFO: - episode: 9280, ep_reward: -122.388, ep_step: 83 2023-05-30 04:54:08 - SimpleLog - INFO: - episode: 9281, ep_reward: -118.869, ep_step: 84 2023-05-30 04:54:08 - SimpleLog - INFO: - episode: 9282, ep_reward: -113.060, ep_step: 63 2023-05-30 04:54:10 - SimpleLog - INFO: - episode: 9283, ep_reward: -93.324, ep_step: 1000 2023-05-30 04:54:10 - SimpleLog - INFO: - episode: 9284, ep_reward: -114.961, ep_step: 72 2023-05-30 04:54:10 - SimpleLog - INFO: - episode: 9285, ep_reward: -116.531, ep_step: 79 2023-05-30 04:54:13 - SimpleLog - INFO: - episode: 9286, ep_reward: -93.049, ep_step: 1000 2023-05-30 04:54:14 - SimpleLog - INFO: - episode: 9287, ep_reward: -174.546, ep_step: 691 2023-05-30 04:54:15 - SimpleLog - INFO: - episode: 9288, ep_reward: -160.868, ep_step: 516 2023-05-30 04:54:17 - SimpleLog - INFO: - episode: 9289, ep_reward: -176.156, ep_step: 696 2023-05-30 04:54:17 - SimpleLog - INFO: - episode: 9290, ep_reward: -132.678, ep_step: 243 2023-05-30 04:54:19 - SimpleLog - INFO: - episode: 9291, ep_reward: -165.449, ep_step: 566 2023-05-30 04:54:20 - SimpleLog - INFO: - episode: 9292, ep_reward: -154.571, ep_step: 433 2023-05-30 04:54:20 - SimpleLog - INFO: - episode: 9293, ep_reward: -127.915, ep_step: 149 2023-05-30 04:54:22 - SimpleLog - INFO: - episode: 9294, ep_reward: -193.033, ep_step: 919 2023-05-30 04:54:24 - SimpleLog - INFO: - episode: 9295, ep_reward: -183.753, ep_step: 783 2023-05-30 04:54:25 - SimpleLog - INFO: - episode: 9296, ep_reward: -169.503, ep_step: 596 2023-05-30 04:54:27 - SimpleLog - INFO: - update_step: 7490000, online_eval_reward: -91.413 2023-05-30 04:54:27 - SimpleLog - INFO: - episode: 9297, ep_reward: -113.665, ep_step: 56 2023-05-30 04:54:29 - SimpleLog - INFO: - episode: 9298, ep_reward: -90.019, ep_step: 1000 2023-05-30 04:54:30 - SimpleLog - INFO: - episode: 9299, ep_reward: -163.752, ep_step: 506 2023-05-30 04:54:32 - SimpleLog - INFO: - episode: 9300, ep_reward: -198.150, ep_step: 934 2023-05-30 04:54:33 - SimpleLog - INFO: - episode: 9301, ep_reward: -112.405, ep_step: 195 2023-05-30 04:54:34 - SimpleLog - INFO: - episode: 9302, ep_reward: -171.359, ep_step: 602 2023-05-30 04:54:34 - SimpleLog - INFO: - episode: 9303, ep_reward: -137.027, ep_step: 177 2023-05-30 04:54:36 - SimpleLog - INFO: - episode: 9304, ep_reward: -201.973, ep_step: 941 2023-05-30 04:54:38 - SimpleLog - INFO: - episode: 9305, ep_reward: -170.158, ep_step: 575 2023-05-30 04:54:39 - SimpleLog - INFO: - episode: 9306, ep_reward: -165.326, ep_step: 526 2023-05-30 04:54:40 - SimpleLog - INFO: - episode: 9307, ep_reward: -173.834, ep_step: 625 2023-05-30 04:54:41 - SimpleLog - INFO: - episode: 9308, ep_reward: -134.675, ep_step: 164 2023-05-30 04:54:42 - SimpleLog - INFO: - episode: 9309, ep_reward: -173.893, ep_step: 628 2023-05-30 04:54:42 - SimpleLog - INFO: - episode: 9310, ep_reward: -121.500, ep_step: 83 2023-05-30 04:54:43 - SimpleLog - INFO: - episode: 9311, ep_reward: -130.663, ep_step: 140 2023-05-30 04:54:45 - SimpleLog - INFO: - episode: 9312, ep_reward: -196.708, ep_step: 901 2023-05-30 04:54:47 - SimpleLog - INFO: - episode: 9313, ep_reward: -98.446, ep_step: 1000 2023-05-30 04:54:48 - SimpleLog - INFO: - episode: 9314, ep_reward: -162.749, ep_step: 496 2023-05-30 04:54:48 - SimpleLog - INFO: - episode: 9315, ep_reward: -139.528, ep_step: 174 2023-05-30 04:54:51 - SimpleLog - INFO: - episode: 9316, ep_reward: -92.334, ep_step: 1000 2023-05-30 04:54:51 - SimpleLog - INFO: - episode: 9317, ep_reward: -107.331, ep_step: 50 2023-05-30 04:54:51 - SimpleLog - INFO: - episode: 9318, ep_reward: -108.910, ep_step: 40 2023-05-30 04:54:51 - SimpleLog - INFO: - episode: 9319, ep_reward: -116.213, ep_step: 69 2023-05-30 04:54:51 - SimpleLog - INFO: - episode: 9320, ep_reward: -112.284, ep_step: 82 2023-05-30 04:54:51 - SimpleLog - INFO: - episode: 9321, ep_reward: -102.186, ep_step: 78 2023-05-30 04:54:52 - SimpleLog - INFO: - episode: 9322, ep_reward: -102.780, ep_step: 71 2023-05-30 04:54:52 - SimpleLog - INFO: - episode: 9323, ep_reward: -100.596, ep_step: 76 2023-05-30 04:54:54 - SimpleLog - INFO: - episode: 9324, ep_reward: -83.535, ep_step: 1000 2023-05-30 04:54:54 - SimpleLog - INFO: - episode: 9325, ep_reward: -124.100, ep_step: 83 2023-05-30 04:54:57 - SimpleLog - INFO: - episode: 9326, ep_reward: -79.966, ep_step: 1000 2023-05-30 04:54:59 - SimpleLog - INFO: - episode: 9327, ep_reward: -84.702, ep_step: 1000 2023-05-30 04:55:01 - SimpleLog - INFO: - episode: 9328, ep_reward: -78.497, ep_step: 1000 2023-05-30 04:55:01 - SimpleLog - INFO: - episode: 9329, ep_reward: -130.702, ep_step: 171 2023-05-30 04:55:02 - SimpleLog - INFO: - episode: 9330, ep_reward: -85.771, ep_step: 216 2023-05-30 04:55:03 - SimpleLog - INFO: - episode: 9331, ep_reward: -82.183, ep_step: 285 2023-05-30 04:55:03 - SimpleLog - INFO: - episode: 9332, ep_reward: -87.716, ep_step: 275 2023-05-30 04:55:05 - SimpleLog - INFO: - episode: 9333, ep_reward: -78.254, ep_step: 1000 2023-05-30 04:55:08 - SimpleLog - INFO: - episode: 9334, ep_reward: -76.991, ep_step: 1000 2023-05-30 04:55:10 - SimpleLog - INFO: - episode: 9335, ep_reward: -89.875, ep_step: 1000 2023-05-30 04:55:12 - SimpleLog - INFO: - episode: 9336, ep_reward: -89.657, ep_step: 1000 2023-05-30 04:55:14 - SimpleLog - INFO: - episode: 9337, ep_reward: -91.177, ep_step: 1000 2023-05-30 04:55:17 - SimpleLog - INFO: - episode: 9338, ep_reward: -91.936, ep_step: 1000 2023-05-30 04:55:19 - SimpleLog - INFO: - episode: 9339, ep_reward: -86.487, ep_step: 1000 2023-05-30 04:55:21 - SimpleLog - INFO: - episode: 9340, ep_reward: -96.513, ep_step: 1000 2023-05-30 04:55:24 - SimpleLog - INFO: - episode: 9341, ep_reward: -85.575, ep_step: 1000 2023-05-30 04:55:24 - SimpleLog - INFO: - episode: 9342, ep_reward: -126.222, ep_step: 149 2023-05-30 04:55:26 - SimpleLog - INFO: - episode: 9343, ep_reward: -74.099, ep_step: 1000 2023-05-30 04:55:28 - SimpleLog - INFO: - episode: 9344, ep_reward: -80.730, ep_step: 1000 2023-05-30 04:55:31 - SimpleLog - INFO: - episode: 9345, ep_reward: -82.039, ep_step: 1000 2023-05-30 04:55:33 - SimpleLog - INFO: - episode: 9346, ep_reward: -95.156, ep_step: 1000 2023-05-30 04:55:35 - SimpleLog - INFO: - episode: 9347, ep_reward: -80.706, ep_step: 1000 2023-05-30 04:55:37 - SimpleLog - INFO: - episode: 9348, ep_reward: -92.396, ep_step: 1000 2023-05-30 04:55:40 - SimpleLog - INFO: - episode: 9349, ep_reward: -91.209, ep_step: 1000 2023-05-30 04:55:42 - SimpleLog - INFO: - episode: 9350, ep_reward: -81.185, ep_step: 1000 2023-05-30 04:55:44 - SimpleLog - INFO: - episode: 9351, ep_reward: -78.959, ep_step: 1000 2023-05-30 04:55:46 - SimpleLog - INFO: - episode: 9352, ep_reward: -198.907, ep_step: 844 2023-05-30 04:55:47 - SimpleLog - INFO: - episode: 9353, ep_reward: -175.365, ep_step: 605 2023-05-30 04:55:50 - SimpleLog - INFO: - episode: 9354, ep_reward: -93.298, ep_step: 1000 2023-05-30 04:55:52 - SimpleLog - INFO: - episode: 9355, ep_reward: -77.476, ep_step: 1000 2023-05-30 04:55:54 - SimpleLog - INFO: - episode: 9356, ep_reward: -83.626, ep_step: 1000 2023-05-30 04:55:56 - SimpleLog - INFO: - episode: 9357, ep_reward: -76.534, ep_step: 1000 2023-05-30 04:55:59 - SimpleLog - INFO: - episode: 9358, ep_reward: -73.076, ep_step: 1000 2023-05-30 04:55:59 - SimpleLog - INFO: - episode: 9359, ep_reward: -104.072, ep_step: 147 2023-05-30 04:56:01 - SimpleLog - INFO: - episode: 9360, ep_reward: -163.458, ep_step: 712 2023-05-30 04:56:03 - SimpleLog - INFO: - episode: 9361, ep_reward: -91.113, ep_step: 1000 2023-05-30 04:56:05 - SimpleLog - INFO: - episode: 9362, ep_reward: -76.774, ep_step: 1000 2023-05-30 04:56:07 - SimpleLog - INFO: - episode: 9363, ep_reward: -73.105, ep_step: 1000 2023-05-30 04:56:08 - SimpleLog - INFO: - episode: 9364, ep_reward: -117.666, ep_step: 102 2023-05-30 04:56:10 - SimpleLog - INFO: - episode: 9365, ep_reward: -79.162, ep_step: 1000 2023-05-30 04:56:10 - SimpleLog - INFO: - episode: 9366, ep_reward: -118.757, ep_step: 130 2023-05-30 04:56:11 - SimpleLog - INFO: - episode: 9367, ep_reward: -116.975, ep_step: 167 2023-05-30 04:56:13 - SimpleLog - INFO: - episode: 9368, ep_reward: -70.821, ep_step: 1000 2023-05-30 04:56:13 - SimpleLog - INFO: - episode: 9369, ep_reward: -142.792, ep_step: 250 2023-05-30 04:56:16 - SimpleLog - INFO: - episode: 9370, ep_reward: -73.408, ep_step: 1000 2023-05-30 04:56:18 - SimpleLog - INFO: - episode: 9371, ep_reward: -77.406, ep_step: 1000 2023-05-30 04:56:20 - SimpleLog - INFO: - episode: 9372, ep_reward: -84.907, ep_step: 1000 2023-05-30 04:56:20 - SimpleLog - INFO: - episode: 9373, ep_reward: -112.453, ep_step: 167 2023-05-30 04:56:23 - SimpleLog - INFO: - episode: 9374, ep_reward: -74.913, ep_step: 1000 2023-05-30 04:56:25 - SimpleLog - INFO: - episode: 9375, ep_reward: -76.584, ep_step: 1000 2023-05-30 04:56:27 - SimpleLog - INFO: - episode: 9376, ep_reward: -72.334, ep_step: 1000 2023-05-30 04:56:27 - SimpleLog - INFO: - episode: 9377, ep_reward: -120.336, ep_step: 120 2023-05-30 04:56:28 - SimpleLog - INFO: - episode: 9378, ep_reward: -125.469, ep_step: 350 2023-05-30 04:56:29 - SimpleLog - INFO: - episode: 9379, ep_reward: -113.620, ep_step: 288 2023-05-30 04:56:31 - SimpleLog - INFO: - episode: 9380, ep_reward: -68.700, ep_step: 1000 2023-05-30 04:56:33 - SimpleLog - INFO: - episode: 9381, ep_reward: -68.230, ep_step: 1000 2023-05-30 04:56:36 - SimpleLog - INFO: - episode: 9382, ep_reward: -69.266, ep_step: 1000 2023-05-30 04:56:38 - SimpleLog - INFO: - episode: 9383, ep_reward: -68.021, ep_step: 1000 2023-05-30 04:56:40 - SimpleLog - INFO: - episode: 9384, ep_reward: -68.537, ep_step: 1000 2023-05-30 04:56:42 - SimpleLog - INFO: - episode: 9385, ep_reward: -74.373, ep_step: 1000 2023-05-30 04:56:43 - SimpleLog - INFO: - episode: 9386, ep_reward: -132.189, ep_step: 438 2023-05-30 04:56:46 - SimpleLog - INFO: - episode: 9387, ep_reward: -68.245, ep_step: 1000 2023-05-30 04:56:48 - SimpleLog - INFO: - episode: 9388, ep_reward: -72.923, ep_step: 1000 2023-05-30 04:56:48 - SimpleLog - INFO: - episode: 9389, ep_reward: -104.273, ep_step: 220 2023-05-30 04:56:51 - SimpleLog - INFO: - episode: 9390, ep_reward: -68.588, ep_step: 1000 2023-05-30 04:56:53 - SimpleLog - INFO: - episode: 9391, ep_reward: -78.112, ep_step: 1000 2023-05-30 04:56:53 - SimpleLog - INFO: - episode: 9392, ep_reward: -113.603, ep_step: 190 2023-05-30 04:56:56 - SimpleLog - INFO: - episode: 9393, ep_reward: -84.988, ep_step: 1000 2023-05-30 04:56:56 - SimpleLog - INFO: - episode: 9394, ep_reward: -114.016, ep_step: 256 2023-05-30 04:56:57 - SimpleLog - INFO: - episode: 9395, ep_reward: -122.436, ep_step: 155 2023-05-30 04:56:59 - SimpleLog - INFO: - episode: 9396, ep_reward: -84.546, ep_step: 1000 2023-05-30 04:57:00 - SimpleLog - INFO: - episode: 9397, ep_reward: -118.726, ep_step: 335 2023-05-30 04:57:00 - SimpleLog - INFO: - episode: 9398, ep_reward: -109.413, ep_step: 157 2023-05-30 04:57:00 - SimpleLog - INFO: - episode: 9399, ep_reward: -114.445, ep_step: 155 2023-05-30 04:57:03 - SimpleLog - INFO: - episode: 9400, ep_reward: -79.644, ep_step: 1000 2023-05-30 04:57:06 - SimpleLog - INFO: - update_step: 7560000, online_eval_reward: -90.121 2023-05-30 04:57:07 - SimpleLog - INFO: - episode: 9401, ep_reward: -79.257, ep_step: 1000 2023-05-30 04:57:09 - SimpleLog - INFO: - episode: 9402, ep_reward: -53.830, ep_step: 1000 2023-05-30 04:57:11 - SimpleLog - INFO: - episode: 9403, ep_reward: -70.763, ep_step: 1000 2023-05-30 04:57:12 - SimpleLog - INFO: - episode: 9404, ep_reward: -120.107, ep_step: 454 2023-05-30 04:57:15 - SimpleLog - INFO: - episode: 9405, ep_reward: -75.726, ep_step: 1000 2023-05-30 04:57:15 - SimpleLog - INFO: - episode: 9406, ep_reward: -103.318, ep_step: 132 2023-05-30 04:57:16 - SimpleLog - INFO: - episode: 9407, ep_reward: -140.387, ep_step: 622 2023-05-30 04:57:19 - SimpleLog - INFO: - episode: 9408, ep_reward: -76.736, ep_step: 1000 2023-05-30 04:57:21 - SimpleLog - INFO: - episode: 9409, ep_reward: -80.945, ep_step: 1000 2023-05-30 04:57:21 - SimpleLog - INFO: - episode: 9410, ep_reward: -107.295, ep_step: 122 2023-05-30 04:57:23 - SimpleLog - INFO: - episode: 9411, ep_reward: -80.780, ep_step: 1000 2023-05-30 04:57:24 - SimpleLog - INFO: - episode: 9412, ep_reward: -107.302, ep_step: 147 2023-05-30 04:57:24 - SimpleLog - INFO: - episode: 9413, ep_reward: -107.093, ep_step: 117 2023-05-30 04:57:24 - SimpleLog - INFO: - episode: 9414, ep_reward: -105.423, ep_step: 168 2023-05-30 04:57:25 - SimpleLog - INFO: - episode: 9415, ep_reward: -114.284, ep_step: 141 2023-05-30 04:57:25 - SimpleLog - INFO: - episode: 9416, ep_reward: -106.642, ep_step: 197 2023-05-30 04:57:27 - SimpleLog - INFO: - episode: 9417, ep_reward: -78.973, ep_step: 1000 2023-05-30 04:57:30 - SimpleLog - INFO: - episode: 9418, ep_reward: -78.712, ep_step: 1000 2023-05-30 04:57:32 - SimpleLog - INFO: - episode: 9419, ep_reward: -77.196, ep_step: 1000 2023-05-30 04:57:34 - SimpleLog - INFO: - episode: 9420, ep_reward: -69.470, ep_step: 1000 2023-05-30 04:57:36 - SimpleLog - INFO: - episode: 9421, ep_reward: -66.426, ep_step: 1000 2023-05-30 04:57:39 - SimpleLog - INFO: - episode: 9422, ep_reward: -74.853, ep_step: 1000 2023-05-30 04:57:41 - SimpleLog - INFO: - episode: 9423, ep_reward: -28.000, ep_step: 1000 2023-05-30 04:57:41 - SimpleLog - INFO: - episode: 9424, ep_reward: -115.747, ep_step: 66 2023-05-30 04:57:41 - SimpleLog - INFO: - episode: 9425, ep_reward: -115.343, ep_step: 191 2023-05-30 04:57:44 - SimpleLog - INFO: - episode: 9426, ep_reward: -74.148, ep_step: 1000 2023-05-30 04:57:46 - SimpleLog - INFO: - episode: 9427, ep_reward: -73.070, ep_step: 1000 2023-05-30 04:57:47 - SimpleLog - INFO: - episode: 9428, ep_reward: -103.017, ep_step: 458 2023-05-30 04:57:48 - SimpleLog - INFO: - episode: 9429, ep_reward: -102.003, ep_step: 340 2023-05-30 04:57:48 - SimpleLog - INFO: - episode: 9430, ep_reward: -102.446, ep_step: 321 2023-05-30 04:57:49 - SimpleLog - INFO: - episode: 9431, ep_reward: -108.561, ep_step: 405 2023-05-30 04:57:50 - SimpleLog - INFO: - episode: 9432, ep_reward: -88.964, ep_step: 214 2023-05-30 04:57:52 - SimpleLog - INFO: - episode: 9433, ep_reward: -38.383, ep_step: 1000 2023-05-30 04:57:53 - SimpleLog - INFO: - episode: 9434, ep_reward: -95.859, ep_step: 249 2023-05-30 04:57:55 - SimpleLog - INFO: - episode: 9435, ep_reward: -56.528, ep_step: 1000 2023-05-30 04:57:55 - SimpleLog - INFO: - episode: 9436, ep_reward: -113.358, ep_step: 69 2023-05-30 04:57:55 - SimpleLog - INFO: - episode: 9437, ep_reward: -111.924, ep_step: 64 2023-05-30 04:57:57 - SimpleLog - INFO: - episode: 9438, ep_reward: -59.503, ep_step: 1000 2023-05-30 04:58:00 - SimpleLog - INFO: - episode: 9439, ep_reward: -26.473, ep_step: 1000 2023-05-30 04:58:02 - SimpleLog - INFO: - episode: 9440, ep_reward: -8.633, ep_step: 1000 2023-05-30 04:58:04 - SimpleLog - INFO: - episode: 9441, ep_reward: -53.855, ep_step: 1000 2023-05-30 04:58:06 - SimpleLog - INFO: - episode: 9442, ep_reward: -8.318, ep_step: 1000 2023-05-30 04:58:09 - SimpleLog - INFO: - episode: 9443, ep_reward: -50.244, ep_step: 1000 2023-05-30 04:58:11 - SimpleLog - INFO: - episode: 9444, ep_reward: -51.472, ep_step: 1000 2023-05-30 04:58:13 - SimpleLog - INFO: - episode: 9445, ep_reward: -9.008, ep_step: 1000 2023-05-30 04:58:15 - SimpleLog - INFO: - episode: 9446, ep_reward: 0.371, ep_step: 1000 2023-05-30 04:58:18 - SimpleLog - INFO: - episode: 9447, ep_reward: -63.081, ep_step: 1000 2023-05-30 04:58:20 - SimpleLog - INFO: - episode: 9448, ep_reward: -60.372, ep_step: 1000 2023-05-30 04:58:22 - SimpleLog - INFO: - episode: 9449, ep_reward: -50.916, ep_step: 1000 2023-05-30 04:58:24 - SimpleLog - INFO: - episode: 9450, ep_reward: -56.345, ep_step: 1000 2023-05-30 04:58:27 - SimpleLog - INFO: - episode: 9451, ep_reward: -75.780, ep_step: 1000 2023-05-30 04:58:29 - SimpleLog - INFO: - episode: 9452, ep_reward: -59.071, ep_step: 1000 2023-05-30 04:58:31 - SimpleLog - INFO: - episode: 9453, ep_reward: -64.873, ep_step: 1000 2023-05-30 04:58:33 - SimpleLog - INFO: - episode: 9454, ep_reward: -62.409, ep_step: 1000 2023-05-30 04:58:36 - SimpleLog - INFO: - episode: 9455, ep_reward: -63.821, ep_step: 1000 2023-05-30 04:58:38 - SimpleLog - INFO: - episode: 9456, ep_reward: -74.056, ep_step: 1000 2023-05-30 04:58:40 - SimpleLog - INFO: - episode: 9457, ep_reward: -61.200, ep_step: 1000 2023-05-30 04:58:43 - SimpleLog - INFO: - episode: 9458, ep_reward: -54.482, ep_step: 1000 2023-05-30 04:58:45 - SimpleLog - INFO: - episode: 9459, ep_reward: -67.429, ep_step: 1000 2023-05-30 04:58:46 - SimpleLog - INFO: - episode: 9460, ep_reward: -113.725, ep_step: 377 2023-05-30 04:58:48 - SimpleLog - INFO: - episode: 9461, ep_reward: -63.454, ep_step: 1000 2023-05-30 04:58:50 - SimpleLog - INFO: - episode: 9462, ep_reward: -54.232, ep_step: 1000 2023-05-30 04:58:52 - SimpleLog - INFO: - episode: 9463, ep_reward: -63.221, ep_step: 1000 2023-05-30 04:58:55 - SimpleLog - INFO: - episode: 9464, ep_reward: -62.954, ep_step: 1000 2023-05-30 04:58:57 - SimpleLog - INFO: - episode: 9465, ep_reward: -61.980, ep_step: 1000 2023-05-30 04:58:59 - SimpleLog - INFO: - episode: 9466, ep_reward: -62.556, ep_step: 1000 2023-05-30 04:59:01 - SimpleLog - INFO: - episode: 9467, ep_reward: -67.768, ep_step: 1000 2023-05-30 04:59:04 - SimpleLog - INFO: - episode: 9468, ep_reward: -68.996, ep_step: 1000 2023-05-30 04:59:06 - SimpleLog - INFO: - episode: 9469, ep_reward: -68.146, ep_step: 1000 2023-05-30 04:59:08 - SimpleLog - INFO: - episode: 9470, ep_reward: -68.452, ep_step: 1000 2023-05-30 04:59:10 - SimpleLog - INFO: - episode: 9471, ep_reward: -75.358, ep_step: 1000 2023-05-30 04:59:13 - SimpleLog - INFO: - episode: 9472, ep_reward: -95.157, ep_step: 1000 2023-05-30 04:59:15 - SimpleLog - INFO: - episode: 9473, ep_reward: -85.702, ep_step: 1000 2023-05-30 04:59:17 - SimpleLog - INFO: - episode: 9474, ep_reward: -84.071, ep_step: 1000 2023-05-30 04:59:19 - SimpleLog - INFO: - episode: 9475, ep_reward: -98.338, ep_step: 1000 2023-05-30 04:59:20 - SimpleLog - INFO: - episode: 9476, ep_reward: -138.428, ep_step: 349 2023-05-30 04:59:21 - SimpleLog - INFO: - episode: 9477, ep_reward: -105.976, ep_step: 217 2023-05-30 04:59:23 - SimpleLog - INFO: - episode: 9478, ep_reward: -59.523, ep_step: 1000 2023-05-30 04:59:23 - SimpleLog - INFO: - episode: 9479, ep_reward: -120.842, ep_step: 78 2023-05-30 04:59:25 - SimpleLog - INFO: - episode: 9480, ep_reward: -57.642, ep_step: 1000 2023-05-30 04:59:26 - SimpleLog - INFO: - episode: 9481, ep_reward: -124.095, ep_step: 134 2023-05-30 04:59:28 - SimpleLog - INFO: - episode: 9482, ep_reward: -91.036, ep_step: 1000 2023-05-30 04:59:28 - SimpleLog - INFO: - episode: 9483, ep_reward: -119.155, ep_step: 81 2023-05-30 04:59:28 - SimpleLog - INFO: - episode: 9484, ep_reward: -123.714, ep_step: 82 2023-05-30 04:59:31 - SimpleLog - INFO: - episode: 9485, ep_reward: -73.954, ep_step: 1000 2023-05-30 04:59:33 - SimpleLog - INFO: - episode: 9486, ep_reward: -89.463, ep_step: 1000 2023-05-30 04:59:35 - SimpleLog - INFO: - episode: 9487, ep_reward: -72.014, ep_step: 1000 2023-05-30 04:59:38 - SimpleLog - INFO: - episode: 9488, ep_reward: -66.387, ep_step: 1000 2023-05-30 04:59:40 - SimpleLog - INFO: - episode: 9489, ep_reward: -66.918, ep_step: 1000 2023-05-30 04:59:42 - SimpleLog - INFO: - episode: 9490, ep_reward: -101.289, ep_step: 1000 2023-05-30 04:59:44 - SimpleLog - INFO: - episode: 9491, ep_reward: -66.250, ep_step: 1000 2023-05-30 04:59:46 - SimpleLog - INFO: - update_step: 7630000, online_eval_reward: -113.338 2023-05-30 04:59:48 - SimpleLog - INFO: - episode: 9492, ep_reward: -65.024, ep_step: 1000 2023-05-30 04:59:51 - SimpleLog - INFO: - episode: 9493, ep_reward: -64.427, ep_step: 1000 2023-05-30 04:59:53 - SimpleLog - INFO: - episode: 9494, ep_reward: -91.096, ep_step: 1000 2023-05-30 04:59:55 - SimpleLog - INFO: - episode: 9495, ep_reward: -63.003, ep_step: 1000 2023-05-30 04:59:58 - SimpleLog - INFO: - episode: 9496, ep_reward: -66.826, ep_step: 1000 2023-05-30 04:59:58 - SimpleLog - INFO: - episode: 9497, ep_reward: -119.284, ep_step: 168 2023-05-30 05:00:00 - SimpleLog - INFO: - episode: 9498, ep_reward: -42.198, ep_step: 1000 2023-05-30 05:00:02 - SimpleLog - INFO: - episode: 9499, ep_reward: -42.165, ep_step: 1000 2023-05-30 05:00:05 - SimpleLog - INFO: - episode: 9500, ep_reward: -42.351, ep_step: 1000 2023-05-30 05:00:07 - SimpleLog - INFO: - episode: 9501, ep_reward: -37.110, ep_step: 1000 2023-05-30 05:00:09 - SimpleLog - INFO: - episode: 9502, ep_reward: -40.091, ep_step: 1000 2023-05-30 05:00:12 - SimpleLog - INFO: - episode: 9503, ep_reward: -36.834, ep_step: 1000 2023-05-30 05:00:12 - SimpleLog - INFO: - episode: 9504, ep_reward: -108.481, ep_step: 66 2023-05-30 05:00:12 - SimpleLog - INFO: - episode: 9505, ep_reward: -108.678, ep_step: 62 2023-05-30 05:00:14 - SimpleLog - INFO: - episode: 9506, ep_reward: -46.801, ep_step: 1000 2023-05-30 05:00:14 - SimpleLog - INFO: - episode: 9507, ep_reward: -109.010, ep_step: 48 2023-05-30 05:00:14 - SimpleLog - INFO: - episode: 9508, ep_reward: -108.726, ep_step: 77 2023-05-30 05:00:15 - SimpleLog - INFO: - episode: 9509, ep_reward: -108.666, ep_step: 69 2023-05-30 05:00:15 - SimpleLog - INFO: - episode: 9510, ep_reward: -109.139, ep_step: 67 2023-05-30 05:00:17 - SimpleLog - INFO: - episode: 9511, ep_reward: -81.064, ep_step: 1000 2023-05-30 05:00:19 - SimpleLog - INFO: - episode: 9512, ep_reward: -76.070, ep_step: 1000 2023-05-30 05:00:21 - SimpleLog - INFO: - episode: 9513, ep_reward: -43.984, ep_step: 1000 2023-05-30 05:00:22 - SimpleLog - INFO: - episode: 9514, ep_reward: -105.850, ep_step: 107 2023-05-30 05:00:24 - SimpleLog - INFO: - episode: 9515, ep_reward: -66.910, ep_step: 1000 2023-05-30 05:00:26 - SimpleLog - INFO: - episode: 9516, ep_reward: -74.138, ep_step: 1000 2023-05-30 05:00:29 - SimpleLog - INFO: - episode: 9517, ep_reward: -86.520, ep_step: 1000 2023-05-30 05:00:31 - SimpleLog - INFO: - episode: 9518, ep_reward: -59.325, ep_step: 1000 2023-05-30 05:00:33 - SimpleLog - INFO: - episode: 9519, ep_reward: -72.475, ep_step: 1000 2023-05-30 05:00:35 - SimpleLog - INFO: - episode: 9520, ep_reward: -63.780, ep_step: 1000 2023-05-30 05:00:38 - SimpleLog - INFO: - episode: 9521, ep_reward: -69.838, ep_step: 1000 2023-05-30 05:00:40 - SimpleLog - INFO: - episode: 9522, ep_reward: -68.232, ep_step: 1000 2023-05-30 05:00:42 - SimpleLog - INFO: - episode: 9523, ep_reward: -68.104, ep_step: 1000 2023-05-30 05:00:44 - SimpleLog - INFO: - episode: 9524, ep_reward: -68.229, ep_step: 1000 2023-05-30 05:00:47 - SimpleLog - INFO: - episode: 9525, ep_reward: -65.728, ep_step: 1000 2023-05-30 05:00:49 - SimpleLog - INFO: - episode: 9526, ep_reward: -64.359, ep_step: 1000 2023-05-30 05:00:51 - SimpleLog - INFO: - episode: 9527, ep_reward: -67.332, ep_step: 1000 2023-05-30 05:00:54 - SimpleLog - INFO: - episode: 9528, ep_reward: -68.175, ep_step: 1000 2023-05-30 05:00:56 - SimpleLog - INFO: - episode: 9529, ep_reward: -86.900, ep_step: 1000 2023-05-30 05:00:58 - SimpleLog - INFO: - episode: 9530, ep_reward: -84.889, ep_step: 1000 2023-05-30 05:01:00 - SimpleLog - INFO: - episode: 9531, ep_reward: -65.179, ep_step: 1000 2023-05-30 05:01:03 - SimpleLog - INFO: - episode: 9532, ep_reward: -78.730, ep_step: 1000 2023-05-30 05:01:05 - SimpleLog - INFO: - episode: 9533, ep_reward: -70.815, ep_step: 1000 2023-05-30 05:01:06 - SimpleLog - INFO: - episode: 9534, ep_reward: -163.952, ep_step: 558 2023-05-30 05:01:09 - SimpleLog - INFO: - episode: 9535, ep_reward: -70.477, ep_step: 1000 2023-05-30 05:01:09 - SimpleLog - INFO: - episode: 9536, ep_reward: -135.642, ep_step: 156 2023-05-30 05:01:11 - SimpleLog - INFO: - episode: 9537, ep_reward: -70.119, ep_step: 1000 2023-05-30 05:01:12 - SimpleLog - INFO: - episode: 9538, ep_reward: -107.979, ep_step: 331 2023-05-30 05:01:12 - SimpleLog - INFO: - episode: 9539, ep_reward: -130.989, ep_step: 121 2023-05-30 05:01:14 - SimpleLog - INFO: - episode: 9540, ep_reward: -63.062, ep_step: 1000 2023-05-30 05:01:17 - SimpleLog - INFO: - episode: 9541, ep_reward: -63.843, ep_step: 1000 2023-05-30 05:01:19 - SimpleLog - INFO: - episode: 9542, ep_reward: -63.710, ep_step: 1000 2023-05-30 05:01:21 - SimpleLog - INFO: - episode: 9543, ep_reward: -61.382, ep_step: 1000 2023-05-30 05:01:24 - SimpleLog - INFO: - episode: 9544, ep_reward: -62.055, ep_step: 1000 2023-05-30 05:01:26 - SimpleLog - INFO: - episode: 9545, ep_reward: -62.174, ep_step: 1000 2023-05-30 05:01:28 - SimpleLog - INFO: - episode: 9546, ep_reward: -61.231, ep_step: 1000 2023-05-30 05:01:30 - SimpleLog - INFO: - episode: 9547, ep_reward: -85.908, ep_step: 1000 2023-05-30 05:01:33 - SimpleLog - INFO: - episode: 9548, ep_reward: -64.122, ep_step: 1000 2023-05-30 05:01:35 - SimpleLog - INFO: - episode: 9549, ep_reward: -62.024, ep_step: 1000 2023-05-30 05:01:37 - SimpleLog - INFO: - episode: 9550, ep_reward: -62.377, ep_step: 1000 2023-05-30 05:01:39 - SimpleLog - INFO: - episode: 9551, ep_reward: -63.265, ep_step: 1000 2023-05-30 05:01:42 - SimpleLog - INFO: - episode: 9552, ep_reward: -63.638, ep_step: 1000 2023-05-30 05:01:44 - SimpleLog - INFO: - episode: 9553, ep_reward: -64.051, ep_step: 1000 2023-05-30 05:01:46 - SimpleLog - INFO: - episode: 9554, ep_reward: -64.069, ep_step: 1000 2023-05-30 05:01:49 - SimpleLog - INFO: - episode: 9555, ep_reward: -64.752, ep_step: 1000 2023-05-30 05:01:51 - SimpleLog - INFO: - episode: 9556, ep_reward: -65.656, ep_step: 1000 2023-05-30 05:01:53 - SimpleLog - INFO: - episode: 9557, ep_reward: -66.755, ep_step: 1000 2023-05-30 05:01:55 - SimpleLog - INFO: - episode: 9558, ep_reward: -67.232, ep_step: 1000 2023-05-30 05:01:58 - SimpleLog - INFO: - episode: 9559, ep_reward: -67.800, ep_step: 1000 2023-05-30 05:01:58 - SimpleLog - INFO: - episode: 9560, ep_reward: -149.155, ep_step: 343 2023-05-30 05:02:01 - SimpleLog - INFO: - episode: 9561, ep_reward: -65.763, ep_step: 1000 2023-05-30 05:02:03 - SimpleLog - INFO: - episode: 9562, ep_reward: -62.307, ep_step: 1000 2023-05-30 05:02:05 - SimpleLog - INFO: - episode: 9563, ep_reward: -64.120, ep_step: 1000 2023-05-30 05:02:07 - SimpleLog - INFO: - episode: 9564, ep_reward: -65.062, ep_step: 1000 2023-05-30 05:02:10 - SimpleLog - INFO: - episode: 9565, ep_reward: -64.907, ep_step: 1000 2023-05-30 05:02:12 - SimpleLog - INFO: - episode: 9566, ep_reward: -66.076, ep_step: 1000 2023-05-30 05:02:14 - SimpleLog - INFO: - episode: 9567, ep_reward: -66.479, ep_step: 1000 2023-05-30 05:02:16 - SimpleLog - INFO: - episode: 9568, ep_reward: -67.592, ep_step: 1000 2023-05-30 05:02:19 - SimpleLog - INFO: - episode: 9569, ep_reward: -66.883, ep_step: 1000 2023-05-30 05:02:21 - SimpleLog - INFO: - episode: 9570, ep_reward: -64.005, ep_step: 1000 2023-05-30 05:02:23 - SimpleLog - INFO: - episode: 9571, ep_reward: -62.385, ep_step: 1000 2023-05-30 05:02:27 - SimpleLog - INFO: - update_step: 7700000, online_eval_reward: -60.153 2023-05-30 05:02:27 - SimpleLog - INFO: - episode: 9572, ep_reward: -63.962, ep_step: 1000 2023-05-30 05:02:30 - SimpleLog - INFO: - episode: 9573, ep_reward: -61.696, ep_step: 1000 2023-05-30 05:02:32 - SimpleLog - INFO: - episode: 9574, ep_reward: -65.183, ep_step: 1000 2023-05-30 05:02:34 - SimpleLog - INFO: - episode: 9575, ep_reward: -64.951, ep_step: 1000 2023-05-30 05:02:36 - SimpleLog - INFO: - episode: 9576, ep_reward: -64.540, ep_step: 1000 2023-05-30 05:02:39 - SimpleLog - INFO: - episode: 9577, ep_reward: -63.752, ep_step: 1000 2023-05-30 05:02:41 - SimpleLog - INFO: - episode: 9578, ep_reward: -62.479, ep_step: 1000 2023-05-30 05:02:43 - SimpleLog - INFO: - episode: 9579, ep_reward: -62.101, ep_step: 1000 2023-05-30 05:02:45 - SimpleLog - INFO: - episode: 9580, ep_reward: -61.398, ep_step: 1000 2023-05-30 05:02:48 - SimpleLog - INFO: - episode: 9581, ep_reward: -61.562, ep_step: 1000 2023-05-30 05:02:50 - SimpleLog - INFO: - episode: 9582, ep_reward: -61.987, ep_step: 1000 2023-05-30 05:02:50 - SimpleLog - INFO: - episode: 9583, ep_reward: -128.339, ep_step: 105 2023-05-30 05:02:53 - SimpleLog - INFO: - episode: 9584, ep_reward: -60.841, ep_step: 1000 2023-05-30 05:02:55 - SimpleLog - INFO: - episode: 9585, ep_reward: -61.003, ep_step: 1000 2023-05-30 05:02:57 - SimpleLog - INFO: - episode: 9586, ep_reward: -56.158, ep_step: 1000 2023-05-30 05:02:59 - SimpleLog - INFO: - episode: 9587, ep_reward: -67.888, ep_step: 1000 2023-05-30 05:03:02 - SimpleLog - INFO: - episode: 9588, ep_reward: -62.321, ep_step: 1000 2023-05-30 05:03:04 - SimpleLog - INFO: - episode: 9589, ep_reward: -68.864, ep_step: 1000 2023-05-30 05:03:06 - SimpleLog - INFO: - episode: 9590, ep_reward: -70.855, ep_step: 1000 2023-05-30 05:03:08 - SimpleLog - INFO: - episode: 9591, ep_reward: -68.817, ep_step: 1000 2023-05-30 05:03:11 - SimpleLog - INFO: - episode: 9592, ep_reward: -62.517, ep_step: 1000 2023-05-30 05:03:13 - SimpleLog - INFO: - episode: 9593, ep_reward: -65.356, ep_step: 1000 2023-05-30 05:03:15 - SimpleLog - INFO: - episode: 9594, ep_reward: -54.160, ep_step: 1000 2023-05-30 05:03:17 - SimpleLog - INFO: - episode: 9595, ep_reward: -63.383, ep_step: 1000 2023-05-30 05:03:20 - SimpleLog - INFO: - episode: 9596, ep_reward: -59.968, ep_step: 1000 2023-05-30 05:03:22 - SimpleLog - INFO: - episode: 9597, ep_reward: -60.513, ep_step: 1000 2023-05-30 05:03:24 - SimpleLog - INFO: - episode: 9598, ep_reward: -62.495, ep_step: 1000 2023-05-30 05:03:27 - SimpleLog - INFO: - episode: 9599, ep_reward: -62.579, ep_step: 1000 2023-05-30 05:03:29 - SimpleLog - INFO: - episode: 9600, ep_reward: -63.734, ep_step: 1000 2023-05-30 05:03:31 - SimpleLog - INFO: - episode: 9601, ep_reward: -64.691, ep_step: 1000 2023-05-30 05:03:33 - SimpleLog - INFO: - episode: 9602, ep_reward: -63.965, ep_step: 1000 2023-05-30 05:03:34 - SimpleLog - INFO: - episode: 9603, ep_reward: -118.529, ep_step: 130 2023-05-30 05:03:34 - SimpleLog - INFO: - episode: 9604, ep_reward: -121.094, ep_step: 161 2023-05-30 05:03:36 - SimpleLog - INFO: - episode: 9605, ep_reward: -66.771, ep_step: 1000 2023-05-30 05:03:39 - SimpleLog - INFO: - episode: 9606, ep_reward: -78.313, ep_step: 1000 2023-05-30 05:03:41 - SimpleLog - INFO: - episode: 9607, ep_reward: -63.891, ep_step: 1000 2023-05-30 05:03:43 - SimpleLog - INFO: - episode: 9608, ep_reward: -64.343, ep_step: 1000 2023-05-30 05:03:45 - SimpleLog - INFO: - episode: 9609, ep_reward: -76.473, ep_step: 1000 2023-05-30 05:03:48 - SimpleLog - INFO: - episode: 9610, ep_reward: -74.009, ep_step: 1000 2023-05-30 05:03:50 - SimpleLog - INFO: - episode: 9611, ep_reward: -63.974, ep_step: 1000 2023-05-30 05:03:52 - SimpleLog - INFO: - episode: 9612, ep_reward: -69.394, ep_step: 1000 2023-05-30 05:03:54 - SimpleLog - INFO: - episode: 9613, ep_reward: -72.124, ep_step: 1000 2023-05-30 05:03:57 - SimpleLog - INFO: - episode: 9614, ep_reward: -64.611, ep_step: 1000 2023-05-30 05:03:59 - SimpleLog - INFO: - episode: 9615, ep_reward: -59.334, ep_step: 1000 2023-05-30 05:04:01 - SimpleLog - INFO: - episode: 9616, ep_reward: -69.391, ep_step: 1000 2023-05-30 05:04:04 - SimpleLog - INFO: - episode: 9617, ep_reward: -64.693, ep_step: 1000 2023-05-30 05:04:06 - SimpleLog - INFO: - episode: 9618, ep_reward: -87.790, ep_step: 1000 2023-05-30 05:04:06 - SimpleLog - INFO: - episode: 9619, ep_reward: -119.883, ep_step: 94 2023-05-30 05:04:08 - SimpleLog - INFO: - episode: 9620, ep_reward: -63.591, ep_step: 1000 2023-05-30 05:04:10 - SimpleLog - INFO: - episode: 9621, ep_reward: -78.698, ep_step: 1000 2023-05-30 05:04:11 - SimpleLog - INFO: - episode: 9622, ep_reward: -110.117, ep_step: 251 2023-05-30 05:04:11 - SimpleLog - INFO: - episode: 9623, ep_reward: -126.310, ep_step: 150 2023-05-30 05:04:14 - SimpleLog - INFO: - episode: 9624, ep_reward: -61.815, ep_step: 1000 2023-05-30 05:04:14 - SimpleLog - INFO: - episode: 9625, ep_reward: -104.327, ep_step: 163 2023-05-30 05:04:16 - SimpleLog - INFO: - episode: 9626, ep_reward: -58.963, ep_step: 1000 2023-05-30 05:04:19 - SimpleLog - INFO: - episode: 9627, ep_reward: -61.967, ep_step: 1000 2023-05-30 05:04:21 - SimpleLog - INFO: - episode: 9628, ep_reward: -98.080, ep_step: 1000 2023-05-30 05:04:23 - SimpleLog - INFO: - episode: 9629, ep_reward: -94.235, ep_step: 1000 2023-05-30 05:04:25 - SimpleLog - INFO: - episode: 9630, ep_reward: -94.145, ep_step: 1000 2023-05-30 05:04:28 - SimpleLog - INFO: - episode: 9631, ep_reward: -67.012, ep_step: 1000 2023-05-30 05:04:28 - SimpleLog - INFO: - episode: 9632, ep_reward: -104.128, ep_step: 349 2023-05-30 05:04:31 - SimpleLog - INFO: - episode: 9633, ep_reward: -66.397, ep_step: 1000 2023-05-30 05:04:33 - SimpleLog - INFO: - episode: 9634, ep_reward: -48.627, ep_step: 1000 2023-05-30 05:04:35 - SimpleLog - INFO: - episode: 9635, ep_reward: -78.817, ep_step: 1000 2023-05-30 05:04:37 - SimpleLog - INFO: - episode: 9636, ep_reward: -70.946, ep_step: 1000 2023-05-30 05:04:38 - SimpleLog - INFO: - episode: 9637, ep_reward: -128.732, ep_step: 101 2023-05-30 05:04:40 - SimpleLog - INFO: - episode: 9638, ep_reward: -68.096, ep_step: 1000 2023-05-30 05:04:40 - SimpleLog - INFO: - episode: 9639, ep_reward: -117.386, ep_step: 117 2023-05-30 05:04:43 - SimpleLog - INFO: - episode: 9640, ep_reward: -66.802, ep_step: 1000 2023-05-30 05:04:43 - SimpleLog - INFO: - episode: 9641, ep_reward: -123.422, ep_step: 80 2023-05-30 05:04:45 - SimpleLog - INFO: - episode: 9642, ep_reward: -66.117, ep_step: 1000 2023-05-30 05:04:45 - SimpleLog - INFO: - episode: 9643, ep_reward: -121.605, ep_step: 137 2023-05-30 05:04:46 - SimpleLog - INFO: - episode: 9644, ep_reward: -126.763, ep_step: 255 2023-05-30 05:04:48 - SimpleLog - INFO: - episode: 9645, ep_reward: -62.198, ep_step: 1000 2023-05-30 05:04:50 - SimpleLog - INFO: - episode: 9646, ep_reward: -60.869, ep_step: 1000 2023-05-30 05:04:53 - SimpleLog - INFO: - episode: 9647, ep_reward: -56.951, ep_step: 1000 2023-05-30 05:04:55 - SimpleLog - INFO: - episode: 9648, ep_reward: -58.681, ep_step: 1000 2023-05-30 05:04:55 - SimpleLog - INFO: - episode: 9649, ep_reward: -116.654, ep_step: 157 2023-05-30 05:04:57 - SimpleLog - INFO: - episode: 9650, ep_reward: -65.518, ep_step: 1000 2023-05-30 05:05:00 - SimpleLog - INFO: - episode: 9651, ep_reward: -57.327, ep_step: 1000 2023-05-30 05:05:02 - SimpleLog - INFO: - episode: 9652, ep_reward: -68.134, ep_step: 1000 2023-05-30 05:05:03 - SimpleLog - INFO: - episode: 9653, ep_reward: -131.901, ep_step: 357 2023-05-30 05:05:03 - SimpleLog - INFO: - episode: 9654, ep_reward: -143.656, ep_step: 283 2023-05-30 05:05:06 - SimpleLog - INFO: - episode: 9655, ep_reward: -65.238, ep_step: 1000 2023-05-30 05:05:08 - SimpleLog - INFO: - update_step: 7770000, online_eval_reward: -52.078 2023-05-30 05:05:10 - SimpleLog - INFO: - episode: 9656, ep_reward: -62.546, ep_step: 1000 2023-05-30 05:05:12 - SimpleLog - INFO: - episode: 9657, ep_reward: -60.504, ep_step: 1000 2023-05-30 05:05:14 - SimpleLog - INFO: - episode: 9658, ep_reward: -60.611, ep_step: 1000 2023-05-30 05:05:17 - SimpleLog - INFO: - episode: 9659, ep_reward: -64.723, ep_step: 1000 2023-05-30 05:05:19 - SimpleLog - INFO: - episode: 9660, ep_reward: -58.303, ep_step: 1000 2023-05-30 05:05:21 - SimpleLog - INFO: - episode: 9661, ep_reward: -57.399, ep_step: 1000 2023-05-30 05:05:23 - SimpleLog - INFO: - episode: 9662, ep_reward: -54.749, ep_step: 1000 2023-05-30 05:05:26 - SimpleLog - INFO: - episode: 9663, ep_reward: -54.290, ep_step: 1000 2023-05-30 05:05:28 - SimpleLog - INFO: - episode: 9664, ep_reward: -58.041, ep_step: 1000 2023-05-30 05:05:30 - SimpleLog - INFO: - episode: 9665, ep_reward: -56.878, ep_step: 1000 2023-05-30 05:05:32 - SimpleLog - INFO: - episode: 9666, ep_reward: -62.312, ep_step: 1000 2023-05-30 05:05:35 - SimpleLog - INFO: - episode: 9667, ep_reward: -168.035, ep_step: 971 2023-05-30 05:05:37 - SimpleLog - INFO: - episode: 9668, ep_reward: -67.956, ep_step: 1000 2023-05-30 05:05:39 - SimpleLog - INFO: - episode: 9669, ep_reward: -75.938, ep_step: 1000 2023-05-30 05:05:41 - SimpleLog - INFO: - episode: 9670, ep_reward: -172.069, ep_step: 979 2023-05-30 05:05:43 - SimpleLog - INFO: - episode: 9671, ep_reward: -163.177, ep_step: 967 2023-05-30 05:05:46 - SimpleLog - INFO: - episode: 9672, ep_reward: -64.357, ep_step: 1000 2023-05-30 05:05:48 - SimpleLog - INFO: - episode: 9673, ep_reward: -63.254, ep_step: 1000 2023-05-30 05:05:50 - SimpleLog - INFO: - episode: 9674, ep_reward: -60.428, ep_step: 1000 2023-05-30 05:05:52 - SimpleLog - INFO: - episode: 9675, ep_reward: -63.118, ep_step: 1000 2023-05-30 05:05:55 - SimpleLog - INFO: - episode: 9676, ep_reward: -83.397, ep_step: 1000 2023-05-30 05:05:57 - SimpleLog - INFO: - episode: 9677, ep_reward: -59.375, ep_step: 1000 2023-05-30 05:05:59 - SimpleLog - INFO: - episode: 9678, ep_reward: -74.898, ep_step: 1000 2023-05-30 05:06:02 - SimpleLog - INFO: - episode: 9679, ep_reward: -69.980, ep_step: 1000 2023-05-30 05:06:04 - SimpleLog - INFO: - episode: 9680, ep_reward: -62.641, ep_step: 1000 2023-05-30 05:06:06 - SimpleLog - INFO: - episode: 9681, ep_reward: -65.219, ep_step: 1000 2023-05-30 05:06:08 - SimpleLog - INFO: - episode: 9682, ep_reward: -56.763, ep_step: 1000 2023-05-30 05:06:11 - SimpleLog - INFO: - episode: 9683, ep_reward: -64.290, ep_step: 1000 2023-05-30 05:06:13 - SimpleLog - INFO: - episode: 9684, ep_reward: -53.364, ep_step: 1000 2023-05-30 05:06:15 - SimpleLog - INFO: - episode: 9685, ep_reward: -63.076, ep_step: 1000 2023-05-30 05:06:17 - SimpleLog - INFO: - episode: 9686, ep_reward: -59.520, ep_step: 1000 2023-05-30 05:06:20 - SimpleLog - INFO: - episode: 9687, ep_reward: -60.822, ep_step: 1000 2023-05-30 05:06:22 - SimpleLog - INFO: - episode: 9688, ep_reward: -56.432, ep_step: 1000 2023-05-30 05:06:24 - SimpleLog - INFO: - episode: 9689, ep_reward: -62.796, ep_step: 1000 2023-05-30 05:06:26 - SimpleLog - INFO: - episode: 9690, ep_reward: -69.439, ep_step: 1000 2023-05-30 05:06:29 - SimpleLog - INFO: - episode: 9691, ep_reward: -66.726, ep_step: 1000 2023-05-30 05:06:31 - SimpleLog - INFO: - episode: 9692, ep_reward: -63.420, ep_step: 1000 2023-05-30 05:06:33 - SimpleLog - INFO: - episode: 9693, ep_reward: -94.361, ep_step: 1000 2023-05-30 05:06:35 - SimpleLog - INFO: - episode: 9694, ep_reward: -81.018, ep_step: 1000 2023-05-30 05:06:38 - SimpleLog - INFO: - episode: 9695, ep_reward: -84.980, ep_step: 1000 2023-05-30 05:06:40 - SimpleLog - INFO: - episode: 9696, ep_reward: -63.089, ep_step: 1000 2023-05-30 05:06:42 - SimpleLog - INFO: - episode: 9697, ep_reward: -62.242, ep_step: 1000 2023-05-30 05:06:44 - SimpleLog - INFO: - episode: 9698, ep_reward: -64.202, ep_step: 1000 2023-05-30 05:06:47 - SimpleLog - INFO: - episode: 9699, ep_reward: -58.541, ep_step: 1000 2023-05-30 05:06:49 - SimpleLog - INFO: - episode: 9700, ep_reward: -68.907, ep_step: 1000 2023-05-30 05:06:51 - SimpleLog - INFO: - episode: 9701, ep_reward: -30.579, ep_step: 1000 2023-05-30 05:06:53 - SimpleLog - INFO: - episode: 9702, ep_reward: -69.567, ep_step: 1000 2023-05-30 05:06:56 - SimpleLog - INFO: - episode: 9703, ep_reward: -69.453, ep_step: 1000 2023-05-30 05:06:58 - SimpleLog - INFO: - episode: 9704, ep_reward: -66.406, ep_step: 1000 2023-05-30 05:07:00 - SimpleLog - INFO: - episode: 9705, ep_reward: -63.184, ep_step: 1000 2023-05-30 05:07:03 - SimpleLog - INFO: - episode: 9706, ep_reward: -60.833, ep_step: 1000 2023-05-30 05:07:05 - SimpleLog - INFO: - episode: 9707, ep_reward: -59.823, ep_step: 1000 2023-05-30 05:07:07 - SimpleLog - INFO: - episode: 9708, ep_reward: -57.614, ep_step: 1000 2023-05-30 05:07:08 - SimpleLog - INFO: - episode: 9709, ep_reward: -100.305, ep_step: 201 2023-05-30 05:07:10 - SimpleLog - INFO: - episode: 9710, ep_reward: -57.529, ep_step: 1000 2023-05-30 05:07:12 - SimpleLog - INFO: - episode: 9711, ep_reward: -57.937, ep_step: 1000 2023-05-30 05:07:14 - SimpleLog - INFO: - episode: 9712, ep_reward: -58.576, ep_step: 1000 2023-05-30 05:07:17 - SimpleLog - INFO: - episode: 9713, ep_reward: -62.542, ep_step: 1000 2023-05-30 05:07:19 - SimpleLog - INFO: - episode: 9714, ep_reward: -56.890, ep_step: 1000 2023-05-30 05:07:21 - SimpleLog - INFO: - episode: 9715, ep_reward: -51.338, ep_step: 1000 2023-05-30 05:07:23 - SimpleLog - INFO: - episode: 9716, ep_reward: -55.566, ep_step: 1000 2023-05-30 05:07:26 - SimpleLog - INFO: - episode: 9717, ep_reward: -59.250, ep_step: 1000 2023-05-30 05:07:28 - SimpleLog - INFO: - episode: 9718, ep_reward: -50.813, ep_step: 1000 2023-05-30 05:07:30 - SimpleLog - INFO: - episode: 9719, ep_reward: -53.730, ep_step: 1000 2023-05-30 05:07:32 - SimpleLog - INFO: - episode: 9720, ep_reward: -52.375, ep_step: 1000 2023-05-30 05:07:35 - SimpleLog - INFO: - episode: 9721, ep_reward: -56.754, ep_step: 1000 2023-05-30 05:07:37 - SimpleLog - INFO: - episode: 9722, ep_reward: -56.522, ep_step: 1000 2023-05-30 05:07:39 - SimpleLog - INFO: - episode: 9723, ep_reward: -58.818, ep_step: 1000 2023-05-30 05:07:41 - SimpleLog - INFO: - episode: 9724, ep_reward: -55.601, ep_step: 1000 2023-05-30 05:07:44 - SimpleLog - INFO: - episode: 9725, ep_reward: -55.293, ep_step: 1000 2023-05-30 05:07:48 - SimpleLog - INFO: - update_step: 7840000, online_eval_reward: -45.908 2023-05-30 05:07:48 - SimpleLog - INFO: - episode: 9726, ep_reward: -54.560, ep_step: 1000 2023-05-30 05:07:50 - SimpleLog - INFO: - episode: 9727, ep_reward: -53.524, ep_step: 1000 2023-05-30 05:07:52 - SimpleLog - INFO: - episode: 9728, ep_reward: -53.792, ep_step: 1000 2023-05-30 05:07:55 - SimpleLog - INFO: - episode: 9729, ep_reward: -55.936, ep_step: 1000 2023-05-30 05:07:57 - SimpleLog - INFO: - episode: 9730, ep_reward: -55.968, ep_step: 1000 2023-05-30 05:07:59 - SimpleLog - INFO: - episode: 9731, ep_reward: -56.889, ep_step: 1000 2023-05-30 05:08:01 - SimpleLog - INFO: - episode: 9732, ep_reward: -54.510, ep_step: 1000 2023-05-30 05:08:04 - SimpleLog - INFO: - episode: 9733, ep_reward: -55.530, ep_step: 1000 2023-05-30 05:08:06 - SimpleLog - INFO: - episode: 9734, ep_reward: -52.736, ep_step: 1000 2023-05-30 05:08:08 - SimpleLog - INFO: - episode: 9735, ep_reward: -49.681, ep_step: 1000 2023-05-30 05:08:10 - SimpleLog - INFO: - episode: 9736, ep_reward: -52.122, ep_step: 1000 2023-05-30 05:08:13 - SimpleLog - INFO: - episode: 9737, ep_reward: -46.629, ep_step: 1000 2023-05-30 05:08:15 - SimpleLog - INFO: - episode: 9738, ep_reward: -41.217, ep_step: 1000 2023-05-30 05:08:17 - SimpleLog - INFO: - episode: 9739, ep_reward: -47.871, ep_step: 1000 2023-05-30 05:08:19 - SimpleLog - INFO: - episode: 9740, ep_reward: -47.945, ep_step: 1000 2023-05-30 05:08:22 - SimpleLog - INFO: - episode: 9741, ep_reward: -48.912, ep_step: 1000 2023-05-30 05:08:24 - SimpleLog - INFO: - episode: 9742, ep_reward: -36.825, ep_step: 1000 2023-05-30 05:08:26 - SimpleLog - INFO: - episode: 9743, ep_reward: -39.895, ep_step: 1000 2023-05-30 05:08:28 - SimpleLog - INFO: - episode: 9744, ep_reward: -48.345, ep_step: 1000 2023-05-30 05:08:31 - SimpleLog - INFO: - episode: 9745, ep_reward: -49.147, ep_step: 1000 2023-05-30 05:08:33 - SimpleLog - INFO: - episode: 9746, ep_reward: -51.228, ep_step: 1000 2023-05-30 05:08:35 - SimpleLog - INFO: - episode: 9747, ep_reward: -49.868, ep_step: 1000 2023-05-30 05:08:38 - SimpleLog - INFO: - episode: 9748, ep_reward: -48.252, ep_step: 1000 2023-05-30 05:08:40 - SimpleLog - INFO: - episode: 9749, ep_reward: -50.907, ep_step: 1000 2023-05-30 05:08:42 - SimpleLog - INFO: - episode: 9750, ep_reward: -50.473, ep_step: 1000 2023-05-30 05:08:44 - SimpleLog - INFO: - episode: 9751, ep_reward: -49.149, ep_step: 1000 2023-05-30 05:08:47 - SimpleLog - INFO: - episode: 9752, ep_reward: -34.985, ep_step: 1000 2023-05-30 05:08:49 - SimpleLog - INFO: - episode: 9753, ep_reward: -34.196, ep_step: 1000 2023-05-30 05:08:49 - SimpleLog - INFO: - episode: 9754, ep_reward: -104.429, ep_step: 261 2023-05-30 05:08:52 - SimpleLog - INFO: - episode: 9755, ep_reward: -79.948, ep_step: 1000 2023-05-30 05:08:54 - SimpleLog - INFO: - episode: 9756, ep_reward: -50.463, ep_step: 1000 2023-05-30 05:08:56 - SimpleLog - INFO: - episode: 9757, ep_reward: -39.887, ep_step: 1000 2023-05-30 05:08:58 - SimpleLog - INFO: - episode: 9758, ep_reward: -50.083, ep_step: 1000 2023-05-30 05:09:01 - SimpleLog - INFO: - episode: 9759, ep_reward: -32.447, ep_step: 1000 2023-05-30 05:09:03 - SimpleLog - INFO: - episode: 9760, ep_reward: -43.378, ep_step: 1000 2023-05-30 05:09:05 - SimpleLog - INFO: - episode: 9761, ep_reward: -50.435, ep_step: 1000 2023-05-30 05:09:06 - SimpleLog - INFO: - episode: 9762, ep_reward: -94.608, ep_step: 185 2023-05-30 05:09:08 - SimpleLog - INFO: - episode: 9763, ep_reward: -49.638, ep_step: 1000 2023-05-30 05:09:10 - SimpleLog - INFO: - episode: 9764, ep_reward: -37.669, ep_step: 1000 2023-05-30 05:09:12 - SimpleLog - INFO: - episode: 9765, ep_reward: -36.843, ep_step: 1000 2023-05-30 05:09:15 - SimpleLog - INFO: - episode: 9766, ep_reward: -35.021, ep_step: 1000 2023-05-30 05:09:17 - SimpleLog - INFO: - episode: 9767, ep_reward: -40.899, ep_step: 1000 2023-05-30 05:09:19 - SimpleLog - INFO: - episode: 9768, ep_reward: -50.676, ep_step: 1000 2023-05-30 05:09:22 - SimpleLog - INFO: - episode: 9769, ep_reward: -86.499, ep_step: 1000 2023-05-30 05:09:24 - SimpleLog - INFO: - episode: 9770, ep_reward: -50.058, ep_step: 1000 2023-05-30 05:09:26 - SimpleLog - INFO: - episode: 9771, ep_reward: -47.323, ep_step: 1000 2023-05-30 05:09:28 - SimpleLog - INFO: - episode: 9772, ep_reward: -38.552, ep_step: 1000 2023-05-30 05:09:31 - SimpleLog - INFO: - episode: 9773, ep_reward: -38.282, ep_step: 1000 2023-05-30 05:09:33 - SimpleLog - INFO: - episode: 9774, ep_reward: -37.490, ep_step: 1000 2023-05-30 05:09:35 - SimpleLog - INFO: - episode: 9775, ep_reward: -36.429, ep_step: 1000 2023-05-30 05:09:37 - SimpleLog - INFO: - episode: 9776, ep_reward: -49.463, ep_step: 1000 2023-05-30 05:09:40 - SimpleLog - INFO: - episode: 9777, ep_reward: -37.259, ep_step: 1000 2023-05-30 05:09:42 - SimpleLog - INFO: - episode: 9778, ep_reward: -38.974, ep_step: 1000 2023-05-30 05:09:44 - SimpleLog - INFO: - episode: 9779, ep_reward: -56.128, ep_step: 1000 2023-05-30 05:09:46 - SimpleLog - INFO: - episode: 9780, ep_reward: -40.281, ep_step: 1000 2023-05-30 05:09:49 - SimpleLog - INFO: - episode: 9781, ep_reward: -51.770, ep_step: 1000 2023-05-30 05:09:51 - SimpleLog - INFO: - episode: 9782, ep_reward: -49.016, ep_step: 1000 2023-05-30 05:09:53 - SimpleLog - INFO: - episode: 9783, ep_reward: -46.961, ep_step: 1000 2023-05-30 05:09:55 - SimpleLog - INFO: - episode: 9784, ep_reward: -48.592, ep_step: 1000 2023-05-30 05:09:58 - SimpleLog - INFO: - episode: 9785, ep_reward: -47.466, ep_step: 1000 2023-05-30 05:10:00 - SimpleLog - INFO: - episode: 9786, ep_reward: -48.282, ep_step: 1000 2023-05-30 05:10:02 - SimpleLog - INFO: - episode: 9787, ep_reward: -46.320, ep_step: 1000 2023-05-30 05:10:04 - SimpleLog - INFO: - episode: 9788, ep_reward: -47.615, ep_step: 1000 2023-05-30 05:10:07 - SimpleLog - INFO: - episode: 9789, ep_reward: -50.417, ep_step: 1000 2023-05-30 05:10:09 - SimpleLog - INFO: - episode: 9790, ep_reward: -48.291, ep_step: 1000 2023-05-30 05:10:11 - SimpleLog - INFO: - episode: 9791, ep_reward: -48.422, ep_step: 1000 2023-05-30 05:10:13 - SimpleLog - INFO: - episode: 9792, ep_reward: -46.220, ep_step: 1000 2023-05-30 05:10:16 - SimpleLog - INFO: - episode: 9793, ep_reward: -46.863, ep_step: 1000 2023-05-30 05:10:18 - SimpleLog - INFO: - episode: 9794, ep_reward: -47.488, ep_step: 1000 2023-05-30 05:10:20 - SimpleLog - INFO: - episode: 9795, ep_reward: -45.263, ep_step: 1000 2023-05-30 05:10:23 - SimpleLog - INFO: - episode: 9796, ep_reward: -47.170, ep_step: 1000 2023-05-30 05:10:25 - SimpleLog - INFO: - episode: 9797, ep_reward: -46.554, ep_step: 1000 2023-05-30 05:10:28 - SimpleLog - INFO: - update_step: 7910000, online_eval_reward: -22.295 2023-05-30 05:10:29 - SimpleLog - INFO: - episode: 9798, ep_reward: -44.847, ep_step: 1000 2023-05-30 05:10:31 - SimpleLog - INFO: - episode: 9799, ep_reward: -43.600, ep_step: 1000 2023-05-30 05:10:33 - SimpleLog - INFO: - episode: 9800, ep_reward: -60.049, ep_step: 1000 2023-05-30 05:10:36 - SimpleLog - INFO: - episode: 9801, ep_reward: -87.138, ep_step: 1000 2023-05-30 05:10:38 - SimpleLog - INFO: - episode: 9802, ep_reward: -47.392, ep_step: 1000 2023-05-30 05:10:40 - SimpleLog - INFO: - episode: 9803, ep_reward: -96.525, ep_step: 1000 2023-05-30 05:10:43 - SimpleLog - INFO: - episode: 9804, ep_reward: -46.858, ep_step: 1000 2023-05-30 05:10:45 - SimpleLog - INFO: - episode: 9805, ep_reward: -44.029, ep_step: 1000 2023-05-30 05:10:47 - SimpleLog - INFO: - episode: 9806, ep_reward: -45.092, ep_step: 1000 2023-05-30 05:10:49 - SimpleLog - INFO: - episode: 9807, ep_reward: -60.127, ep_step: 1000 2023-05-30 05:10:52 - SimpleLog - INFO: - episode: 9808, ep_reward: -48.453, ep_step: 1000 2023-05-30 05:10:54 - SimpleLog - INFO: - episode: 9809, ep_reward: -54.477, ep_step: 1000 2023-05-30 05:10:56 - SimpleLog - INFO: - episode: 9810, ep_reward: -55.843, ep_step: 1000 2023-05-30 05:10:58 - SimpleLog - INFO: - episode: 9811, ep_reward: -40.286, ep_step: 1000 2023-05-30 05:11:01 - SimpleLog - INFO: - episode: 9812, ep_reward: -40.692, ep_step: 1000 2023-05-30 05:11:03 - SimpleLog - INFO: - episode: 9813, ep_reward: -47.272, ep_step: 1000 2023-05-30 05:11:05 - SimpleLog - INFO: - episode: 9814, ep_reward: -38.757, ep_step: 1000 2023-05-30 05:11:07 - SimpleLog - INFO: - episode: 9815, ep_reward: -40.641, ep_step: 1000 2023-05-30 05:11:10 - SimpleLog - INFO: - episode: 9816, ep_reward: -39.362, ep_step: 1000 2023-05-30 05:11:12 - SimpleLog - INFO: - episode: 9817, ep_reward: -36.117, ep_step: 1000 2023-05-30 05:11:14 - SimpleLog - INFO: - episode: 9818, ep_reward: -34.918, ep_step: 1000 2023-05-30 05:11:16 - SimpleLog - INFO: - episode: 9819, ep_reward: -45.502, ep_step: 1000 2023-05-30 05:11:19 - SimpleLog - INFO: - episode: 9820, ep_reward: -45.049, ep_step: 1000 2023-05-30 05:11:21 - SimpleLog - INFO: - episode: 9821, ep_reward: -37.882, ep_step: 1000 2023-05-30 05:11:23 - SimpleLog - INFO: - episode: 9822, ep_reward: -46.181, ep_step: 1000 2023-05-30 05:11:25 - SimpleLog - INFO: - episode: 9823, ep_reward: -42.608, ep_step: 1000 2023-05-30 05:11:28 - SimpleLog - INFO: - episode: 9824, ep_reward: -51.466, ep_step: 1000 2023-05-30 05:11:30 - SimpleLog - INFO: - episode: 9825, ep_reward: -38.357, ep_step: 1000 2023-05-30 05:11:32 - SimpleLog - INFO: - episode: 9826, ep_reward: -40.791, ep_step: 1000 2023-05-30 05:11:35 - SimpleLog - INFO: - episode: 9827, ep_reward: -42.357, ep_step: 1000 2023-05-30 05:11:37 - SimpleLog - INFO: - episode: 9828, ep_reward: -38.526, ep_step: 1000 2023-05-30 05:11:39 - SimpleLog - INFO: - episode: 9829, ep_reward: -52.057, ep_step: 1000 2023-05-30 05:11:41 - SimpleLog - INFO: - episode: 9830, ep_reward: -43.015, ep_step: 1000 2023-05-30 05:11:44 - SimpleLog - INFO: - episode: 9831, ep_reward: -40.245, ep_step: 1000 2023-05-30 05:11:46 - SimpleLog - INFO: - episode: 9832, ep_reward: -42.589, ep_step: 1000 2023-05-30 05:11:48 - SimpleLog - INFO: - episode: 9833, ep_reward: -50.882, ep_step: 1000 2023-05-30 05:11:50 - SimpleLog - INFO: - episode: 9834, ep_reward: -38.570, ep_step: 1000 2023-05-30 05:11:53 - SimpleLog - INFO: - episode: 9835, ep_reward: -36.272, ep_step: 1000 2023-05-30 05:11:55 - SimpleLog - INFO: - episode: 9836, ep_reward: -33.880, ep_step: 1000 2023-05-30 05:11:57 - SimpleLog - INFO: - episode: 9837, ep_reward: -53.572, ep_step: 1000 2023-05-30 05:11:59 - SimpleLog - INFO: - episode: 9838, ep_reward: -24.008, ep_step: 1000 2023-05-30 05:12:02 - SimpleLog - INFO: - episode: 9839, ep_reward: -44.860, ep_step: 1000 2023-05-30 05:12:04 - SimpleLog - INFO: - episode: 9840, ep_reward: -40.915, ep_step: 1000 2023-05-30 05:12:06 - SimpleLog - INFO: - episode: 9841, ep_reward: -38.543, ep_step: 1000 2023-05-30 05:12:08 - SimpleLog - INFO: - episode: 9842, ep_reward: -61.157, ep_step: 1000 2023-05-30 05:12:11 - SimpleLog - INFO: - episode: 9843, ep_reward: -37.882, ep_step: 1000 2023-05-30 05:12:13 - SimpleLog - INFO: - episode: 9844, ep_reward: -38.397, ep_step: 1000 2023-05-30 05:12:15 - SimpleLog - INFO: - episode: 9845, ep_reward: -194.129, ep_step: 786 2023-05-30 05:12:17 - SimpleLog - INFO: - episode: 9846, ep_reward: -88.057, ep_step: 1000 2023-05-30 05:12:19 - SimpleLog - INFO: - episode: 9847, ep_reward: -40.161, ep_step: 1000 2023-05-30 05:12:22 - SimpleLog - INFO: - episode: 9848, ep_reward: -40.663, ep_step: 1000 2023-05-30 05:12:24 - SimpleLog - INFO: - episode: 9849, ep_reward: -41.691, ep_step: 1000 2023-05-30 05:12:26 - SimpleLog - INFO: - episode: 9850, ep_reward: -54.723, ep_step: 1000 2023-05-30 05:12:28 - SimpleLog - INFO: - episode: 9851, ep_reward: -39.196, ep_step: 1000 2023-05-30 05:12:31 - SimpleLog - INFO: - episode: 9852, ep_reward: -38.385, ep_step: 1000 2023-05-30 05:12:33 - SimpleLog - INFO: - episode: 9853, ep_reward: -51.205, ep_step: 1000 2023-05-30 05:12:35 - SimpleLog - INFO: - episode: 9854, ep_reward: -49.306, ep_step: 1000 2023-05-30 05:12:37 - SimpleLog - INFO: - episode: 9855, ep_reward: -60.102, ep_step: 1000 2023-05-30 05:12:39 - SimpleLog - INFO: - episode: 9856, ep_reward: -147.799, ep_step: 835 2023-05-30 05:12:41 - SimpleLog - INFO: - episode: 9857, ep_reward: -134.811, ep_step: 640 2023-05-30 05:12:43 - SimpleLog - INFO: - episode: 9858, ep_reward: -40.352, ep_step: 1000 2023-05-30 05:12:45 - SimpleLog - INFO: - episode: 9859, ep_reward: -36.606, ep_step: 1000 2023-05-30 05:12:46 - SimpleLog - INFO: - episode: 9860, ep_reward: -121.215, ep_step: 428 2023-05-30 05:12:49 - SimpleLog - INFO: - episode: 9861, ep_reward: -61.087, ep_step: 1000 2023-05-30 05:12:51 - SimpleLog - INFO: - episode: 9862, ep_reward: -38.160, ep_step: 1000 2023-05-30 05:12:53 - SimpleLog - INFO: - episode: 9863, ep_reward: -40.315, ep_step: 1000 2023-05-30 05:12:55 - SimpleLog - INFO: - episode: 9864, ep_reward: -56.911, ep_step: 1000 2023-05-30 05:12:58 - SimpleLog - INFO: - episode: 9865, ep_reward: -40.926, ep_step: 1000 2023-05-30 05:13:00 - SimpleLog - INFO: - episode: 9866, ep_reward: -42.651, ep_step: 1000 2023-05-30 05:13:02 - SimpleLog - INFO: - episode: 9867, ep_reward: -57.841, ep_step: 1000 2023-05-30 05:13:04 - SimpleLog - INFO: - episode: 9868, ep_reward: -133.388, ep_step: 764 2023-05-30 05:13:04 - SimpleLog - INFO: - episode: 9869, ep_reward: -111.959, ep_step: 72 2023-05-30 05:13:08 - SimpleLog - INFO: - update_step: 7980000, online_eval_reward: -59.720 2023-05-30 05:13:08 - SimpleLog - INFO: - episode: 9870, ep_reward: -46.242, ep_step: 1000 2023-05-30 05:13:11 - SimpleLog - INFO: - episode: 9871, ep_reward: -20.392, ep_step: 1000 2023-05-30 05:13:13 - SimpleLog - INFO: - episode: 9872, ep_reward: -41.004, ep_step: 1000 2023-05-30 05:13:14 - SimpleLog - INFO: - episode: 9873, ep_reward: -151.407, ep_step: 537 2023-05-30 05:13:16 - SimpleLog - INFO: - episode: 9874, ep_reward: -42.309, ep_step: 1000 2023-05-30 05:13:17 - SimpleLog - INFO: - episode: 9875, ep_reward: -123.158, ep_step: 255 2023-05-30 05:13:17 - SimpleLog - INFO: - episode: 9876, ep_reward: -131.248, ep_step: 180 2023-05-30 05:13:19 - SimpleLog - INFO: - episode: 9877, ep_reward: -62.214, ep_step: 1000 2023-05-30 05:13:22 - SimpleLog - INFO: - episode: 9878, ep_reward: -42.821, ep_step: 1000 2023-05-30 05:13:24 - SimpleLog - INFO: - episode: 9879, ep_reward: -37.466, ep_step: 1000 2023-05-30 05:13:26 - SimpleLog - INFO: - episode: 9880, ep_reward: -37.230, ep_step: 1000 2023-05-30 05:13:29 - SimpleLog - INFO: - episode: 9881, ep_reward: -36.854, ep_step: 1000 2023-05-30 05:13:31 - SimpleLog - INFO: - episode: 9882, ep_reward: -37.813, ep_step: 1000 2023-05-30 05:13:33 - SimpleLog - INFO: - episode: 9883, ep_reward: -40.171, ep_step: 1000 2023-05-30 05:13:35 - SimpleLog - INFO: - episode: 9884, ep_reward: -40.562, ep_step: 1000 2023-05-30 05:13:38 - SimpleLog - INFO: - episode: 9885, ep_reward: -59.223, ep_step: 1000 2023-05-30 05:13:40 - SimpleLog - INFO: - episode: 9886, ep_reward: -46.921, ep_step: 1000 2023-05-30 05:13:42 - SimpleLog - INFO: - episode: 9887, ep_reward: -62.492, ep_step: 1000 2023-05-30 05:13:44 - SimpleLog - INFO: - episode: 9888, ep_reward: -60.678, ep_step: 1000 2023-05-30 05:13:47 - SimpleLog - INFO: - episode: 9889, ep_reward: -59.230, ep_step: 1000 2023-05-30 05:13:49 - SimpleLog - INFO: - episode: 9890, ep_reward: -60.191, ep_step: 1000 2023-05-30 05:13:51 - SimpleLog - INFO: - episode: 9891, ep_reward: -59.706, ep_step: 1000 2023-05-30 05:13:53 - SimpleLog - INFO: - episode: 9892, ep_reward: -59.564, ep_step: 1000 2023-05-30 05:13:56 - SimpleLog - INFO: - episode: 9893, ep_reward: -47.249, ep_step: 1000 2023-05-30 05:13:58 - SimpleLog - INFO: - episode: 9894, ep_reward: -58.325, ep_step: 1000 2023-05-30 05:14:00 - SimpleLog - INFO: - episode: 9895, ep_reward: -56.990, ep_step: 1000 2023-05-30 05:14:03 - SimpleLog - INFO: - episode: 9896, ep_reward: -54.753, ep_step: 1000 2023-05-30 05:14:05 - SimpleLog - INFO: - episode: 9897, ep_reward: -53.279, ep_step: 1000 2023-05-30 05:14:07 - SimpleLog - INFO: - episode: 9898, ep_reward: -36.087, ep_step: 1000 2023-05-30 05:14:09 - SimpleLog - INFO: - episode: 9899, ep_reward: -50.986, ep_step: 1000 2023-05-30 05:14:12 - SimpleLog - INFO: - episode: 9900, ep_reward: -54.423, ep_step: 1000 2023-05-30 05:14:14 - SimpleLog - INFO: - episode: 9901, ep_reward: -54.454, ep_step: 1000 2023-05-30 05:14:16 - SimpleLog - INFO: - episode: 9902, ep_reward: -56.156, ep_step: 1000 2023-05-30 05:14:18 - SimpleLog - INFO: - episode: 9903, ep_reward: -50.639, ep_step: 1000 2023-05-30 05:14:21 - SimpleLog - INFO: - episode: 9904, ep_reward: -30.809, ep_step: 1000 2023-05-30 05:14:23 - SimpleLog - INFO: - episode: 9905, ep_reward: -48.673, ep_step: 1000 2023-05-30 05:14:25 - SimpleLog - INFO: - episode: 9906, ep_reward: -36.875, ep_step: 1000 2023-05-30 05:14:28 - SimpleLog - INFO: - episode: 9907, ep_reward: -35.554, ep_step: 1000 2023-05-30 05:14:30 - SimpleLog - INFO: - episode: 9908, ep_reward: -35.910, ep_step: 1000 2023-05-30 05:14:32 - SimpleLog - INFO: - episode: 9909, ep_reward: -37.304, ep_step: 1000 2023-05-30 05:14:34 - SimpleLog - INFO: - episode: 9910, ep_reward: -53.360, ep_step: 1000 2023-05-30 05:14:37 - SimpleLog - INFO: - episode: 9911, ep_reward: -36.146, ep_step: 1000 2023-05-30 05:14:39 - SimpleLog - INFO: - episode: 9912, ep_reward: -55.306, ep_step: 1000 2023-05-30 05:14:41 - SimpleLog - INFO: - episode: 9913, ep_reward: -55.053, ep_step: 1000 2023-05-30 05:14:43 - SimpleLog - INFO: - episode: 9914, ep_reward: -56.583, ep_step: 1000 2023-05-30 05:14:46 - SimpleLog - INFO: - episode: 9915, ep_reward: -41.138, ep_step: 1000 2023-05-30 05:14:48 - SimpleLog - INFO: - episode: 9916, ep_reward: -39.820, ep_step: 1000 2023-05-30 05:14:50 - SimpleLog - INFO: - episode: 9917, ep_reward: -54.253, ep_step: 1000 2023-05-30 05:14:52 - SimpleLog - INFO: - episode: 9918, ep_reward: -50.951, ep_step: 1000 2023-05-30 05:14:55 - SimpleLog - INFO: - episode: 9919, ep_reward: -52.345, ep_step: 1000 2023-05-30 05:14:57 - SimpleLog - INFO: - episode: 9920, ep_reward: -38.639, ep_step: 1000 2023-05-30 05:14:59 - SimpleLog - INFO: - episode: 9921, ep_reward: -40.226, ep_step: 1000 2023-05-30 05:15:01 - SimpleLog - INFO: - episode: 9922, ep_reward: -60.579, ep_step: 1000 2023-05-30 05:15:04 - SimpleLog - INFO: - episode: 9923, ep_reward: -41.609, ep_step: 1000 2023-05-30 05:15:06 - SimpleLog - INFO: - episode: 9924, ep_reward: -41.333, ep_step: 1000 2023-05-30 05:15:08 - SimpleLog - INFO: - episode: 9925, ep_reward: -59.898, ep_step: 1000 2023-05-30 05:15:10 - SimpleLog - INFO: - episode: 9926, ep_reward: -59.001, ep_step: 1000 2023-05-30 05:15:11 - SimpleLog - INFO: - episode: 9927, ep_reward: -112.051, ep_step: 320 2023-05-30 05:15:13 - SimpleLog - INFO: - episode: 9928, ep_reward: -45.965, ep_step: 1000 2023-05-30 05:15:16 - SimpleLog - INFO: - episode: 9929, ep_reward: -61.388, ep_step: 1000 2023-05-30 05:15:18 - SimpleLog - INFO: - episode: 9930, ep_reward: -43.067, ep_step: 1000 2023-05-30 05:15:20 - SimpleLog - INFO: - episode: 9931, ep_reward: -56.604, ep_step: 1000 2023-05-30 05:15:22 - SimpleLog - INFO: - episode: 9932, ep_reward: -55.601, ep_step: 1000 2023-05-30 05:15:25 - SimpleLog - INFO: - episode: 9933, ep_reward: -56.261, ep_step: 1000 2023-05-30 05:15:27 - SimpleLog - INFO: - episode: 9934, ep_reward: -55.372, ep_step: 1000 2023-05-30 05:15:29 - SimpleLog - INFO: - episode: 9935, ep_reward: -43.179, ep_step: 1000 2023-05-30 05:15:31 - SimpleLog - INFO: - episode: 9936, ep_reward: -53.553, ep_step: 1000 2023-05-30 05:15:34 - SimpleLog - INFO: - episode: 9937, ep_reward: -54.351, ep_step: 1000 2023-05-30 05:15:36 - SimpleLog - INFO: - episode: 9938, ep_reward: -55.243, ep_step: 1000 2023-05-30 05:15:38 - SimpleLog - INFO: - episode: 9939, ep_reward: -52.097, ep_step: 1000 2023-05-30 05:15:40 - SimpleLog - INFO: - episode: 9940, ep_reward: -53.319, ep_step: 1000 2023-05-30 05:15:43 - SimpleLog - INFO: - episode: 9941, ep_reward: -42.527, ep_step: 1000 2023-05-30 05:15:45 - SimpleLog - INFO: - episode: 9942, ep_reward: -55.914, ep_step: 1000 2023-05-30 05:15:47 - SimpleLog - INFO: - update_step: 8050000, online_eval_reward: -119.320 2023-05-30 05:15:48 - SimpleLog - INFO: - episode: 9943, ep_reward: -55.464, ep_step: 1000 2023-05-30 05:15:48 - SimpleLog - INFO: - episode: 9944, ep_reward: -125.214, ep_step: 262 2023-05-30 05:15:49 - SimpleLog - INFO: - episode: 9945, ep_reward: -118.649, ep_step: 235 2023-05-30 05:15:51 - SimpleLog - INFO: - episode: 9946, ep_reward: -44.797, ep_step: 1000 2023-05-30 05:15:53 - SimpleLog - INFO: - episode: 9947, ep_reward: -59.622, ep_step: 1000 2023-05-30 05:15:56 - SimpleLog - INFO: - episode: 9948, ep_reward: -46.541, ep_step: 1000 2023-05-30 05:15:58 - SimpleLog - INFO: - episode: 9949, ep_reward: -45.585, ep_step: 1000 2023-05-30 05:16:00 - SimpleLog - INFO: - episode: 9950, ep_reward: -61.701, ep_step: 1000 2023-05-30 05:16:02 - SimpleLog - INFO: - episode: 9951, ep_reward: -45.515, ep_step: 1000 2023-05-30 05:16:03 - SimpleLog - INFO: - episode: 9952, ep_reward: -132.576, ep_step: 223 2023-05-30 05:16:05 - SimpleLog - INFO: - episode: 9953, ep_reward: -59.936, ep_step: 1000 2023-05-30 05:16:07 - SimpleLog - INFO: - episode: 9954, ep_reward: -61.300, ep_step: 1000 2023-05-30 05:16:10 - SimpleLog - INFO: - episode: 9955, ep_reward: -102.042, ep_step: 1000 2023-05-30 05:16:12 - SimpleLog - INFO: - episode: 9956, ep_reward: -102.332, ep_step: 1000 2023-05-30 05:16:13 - SimpleLog - INFO: - episode: 9957, ep_reward: -160.224, ep_step: 589 2023-05-30 05:16:13 - SimpleLog - INFO: - episode: 9958, ep_reward: -109.394, ep_step: 71 2023-05-30 05:16:14 - SimpleLog - INFO: - episode: 9959, ep_reward: -107.892, ep_step: 74 2023-05-30 05:16:14 - SimpleLog - INFO: - episode: 9960, ep_reward: -107.853, ep_step: 72 2023-05-30 05:16:16 - SimpleLog - INFO: - episode: 9961, ep_reward: -86.781, ep_step: 1000 2023-05-30 05:16:18 - SimpleLog - INFO: - episode: 9962, ep_reward: -86.186, ep_step: 1000 2023-05-30 05:16:20 - SimpleLog - INFO: - episode: 9963, ep_reward: -89.075, ep_step: 1000 2023-05-30 05:16:23 - SimpleLog - INFO: - episode: 9964, ep_reward: -83.872, ep_step: 1000 2023-05-30 05:16:25 - SimpleLog - INFO: - episode: 9965, ep_reward: -94.360, ep_step: 1000 2023-05-30 05:16:27 - SimpleLog - INFO: - episode: 9966, ep_reward: -89.083, ep_step: 1000 2023-05-30 05:16:29 - SimpleLog - INFO: - episode: 9967, ep_reward: -94.082, ep_step: 1000 2023-05-30 05:16:32 - SimpleLog - INFO: - episode: 9968, ep_reward: -89.144, ep_step: 1000 2023-05-30 05:16:34 - SimpleLog - INFO: - episode: 9969, ep_reward: -88.496, ep_step: 1000 2023-05-30 05:16:36 - SimpleLog - INFO: - episode: 9970, ep_reward: -87.820, ep_step: 1000 2023-05-30 05:16:38 - SimpleLog - INFO: - episode: 9971, ep_reward: -87.742, ep_step: 1000 2023-05-30 05:16:41 - SimpleLog - INFO: - episode: 9972, ep_reward: -87.596, ep_step: 1000 2023-05-30 05:16:43 - SimpleLog - INFO: - episode: 9973, ep_reward: -85.458, ep_step: 1000 2023-05-30 05:16:45 - SimpleLog - INFO: - episode: 9974, ep_reward: -82.128, ep_step: 1000 2023-05-30 05:16:47 - SimpleLog - INFO: - episode: 9975, ep_reward: -83.798, ep_step: 1000 2023-05-30 05:16:50 - SimpleLog - INFO: - episode: 9976, ep_reward: -93.003, ep_step: 1000 2023-05-30 05:16:52 - SimpleLog - INFO: - episode: 9977, ep_reward: -102.705, ep_step: 1000 2023-05-30 05:16:54 - SimpleLog - INFO: - episode: 9978, ep_reward: -102.630, ep_step: 1000 2023-05-30 05:16:54 - SimpleLog - INFO: - episode: 9979, ep_reward: -106.513, ep_step: 88 2023-05-30 05:16:54 - SimpleLog - INFO: - episode: 9980, ep_reward: -106.599, ep_step: 94 2023-05-30 05:16:57 - SimpleLog - INFO: - episode: 9981, ep_reward: -81.085, ep_step: 1000 2023-05-30 05:16:59 - SimpleLog - INFO: - episode: 9982, ep_reward: -72.775, ep_step: 1000 2023-05-30 05:17:01 - SimpleLog - INFO: - episode: 9983, ep_reward: -78.584, ep_step: 1000 2023-05-30 05:17:04 - SimpleLog - INFO: - episode: 9984, ep_reward: -84.096, ep_step: 1000 2023-05-30 05:17:06 - SimpleLog - INFO: - episode: 9985, ep_reward: -74.236, ep_step: 1000 2023-05-30 05:17:08 - SimpleLog - INFO: - episode: 9986, ep_reward: -96.592, ep_step: 1000 2023-05-30 05:17:10 - SimpleLog - INFO: - episode: 9987, ep_reward: -81.476, ep_step: 1000 2023-05-30 05:17:10 - SimpleLog - INFO: - episode: 9988, ep_reward: -108.998, ep_step: 56 2023-05-30 05:17:11 - SimpleLog - INFO: - episode: 9989, ep_reward: -108.274, ep_step: 55 2023-05-30 05:17:11 - SimpleLog - INFO: - episode: 9990, ep_reward: -111.559, ep_step: 74 2023-05-30 05:17:11 - SimpleLog - INFO: - episode: 9991, ep_reward: -109.101, ep_step: 53 2023-05-30 05:17:13 - SimpleLog - INFO: - episode: 9992, ep_reward: -83.073, ep_step: 1000 2023-05-30 05:17:13 - SimpleLog - INFO: - episode: 9993, ep_reward: -118.120, ep_step: 89 2023-05-30 05:17:13 - SimpleLog - INFO: - episode: 9994, ep_reward: -108.579, ep_step: 50 2023-05-30 05:17:14 - SimpleLog - INFO: - episode: 9995, ep_reward: -106.076, ep_step: 176 2023-05-30 05:17:14 - SimpleLog - INFO: - episode: 9996, ep_reward: -112.690, ep_step: 210 2023-05-30 05:17:15 - SimpleLog - INFO: - episode: 9997, ep_reward: -119.130, ep_step: 125 2023-05-30 05:17:17 - SimpleLog - INFO: - episode: 9998, ep_reward: -82.084, ep_step: 1000 2023-05-30 05:17:19 - SimpleLog - INFO: - episode: 9999, ep_reward: -72.503, ep_step: 1000 2023-05-30 05:17:19 - SimpleLog - INFO: - Finish training! total time consumed: 18475.74s