2022-12-06 17:32:22 - r - INFO: - Hyperparameters: 2022-12-06 17:32:22 - r - INFO: - ================================================================================ 2022-12-06 17:32:22 - r - INFO: - Name Value Type 2022-12-06 17:32:22 - r - INFO: - env_name CartPole-v1 2022-12-06 17:32:22 - r - INFO: - new_step_api 1 2022-12-06 17:32:22 - r - INFO: - wrapper None 2022-12-06 17:32:22 - r - INFO: - render 0 2022-12-06 17:32:22 - r - INFO: - algo_name PPO 2022-12-06 17:32:22 - r - INFO: - mode collect 2022-12-06 17:32:22 - r - INFO: - seed 1 2022-12-06 17:32:22 - r - INFO: - device cuda 2022-12-06 17:32:22 - r - INFO: - train_eps 200 2022-12-06 17:32:22 - r - INFO: - test_eps 10 2022-12-06 17:32:22 - r - INFO: - eval_eps 10 2022-12-06 17:32:22 - r - INFO: - eval_per_episode 5 2022-12-06 17:32:22 - r - INFO: - max_steps 200 2022-12-06 17:32:22 - r - INFO: - load_checkpoint 1 2022-12-06 17:32:22 - r - INFO: - load_path Train_CartPole-v1_PPO_20221206-171046 2022-12-06 17:32:22 - r - INFO: - show_fig 0 2022-12-06 17:32:22 - r - INFO: - save_fig 1 2022-12-06 17:32:22 - r - INFO: - collect_eps 100 2022-12-06 17:32:22 - r - INFO: - min_reward 195 2022-12-06 17:32:22 - r - INFO: - continuous 0 2022-12-06 17:32:22 - r - INFO: - gamma 0.99 2022-12-06 17:32:22 - r - INFO: - k_epochs 4 2022-12-06 17:32:22 - r - INFO: - actor_lr 0.0003 2022-12-06 17:32:22 - r - INFO: - critic_lr 0.0003 2022-12-06 17:32:22 - r - INFO: - eps_clip 0.2 2022-12-06 17:32:22 - r - INFO: - entropy_coef 0.01 2022-12-06 17:32:22 - r - INFO: - update_freq 100 2022-12-06 17:32:22 - r - INFO: - actor_hidden_dim 256 2022-12-06 17:32:22 - r - INFO: - critic_hidden_dim 256 2022-12-06 17:32:22 - r - INFO: - ================================================================================ 2022-12-06 17:32:23 - r - INFO: - n_states: 4, n_actions: 2 2022-12-06 17:32:24 - r - INFO: - Start collecting! 2022-12-06 17:32:24 - r - INFO: - Env: CartPole-v1, Algorithm: PPO, Device: cuda 2022-12-06 17:32:35 - r - INFO: - trajectories 1 collected, reward 199.0 2022-12-06 17:32:53 - r - INFO: - trajectories 2 collected, reward 200.0 2022-12-06 17:33:06 - r - INFO: - trajectories 3 collected, reward 200.0 2022-12-06 17:33:36 - r - INFO: - trajectories 4 collected, reward 197.0 2022-12-06 17:34:15 - r - INFO: - trajectories 5 collected, reward 196.0 2022-12-06 17:34:29 - r - INFO: - trajectories 6 collected, reward 200.0 2022-12-06 17:34:36 - r - INFO: - trajectories 7 collected, reward 200.0 2022-12-06 17:34:54 - r - INFO: - trajectories 8 collected, reward 198.0 2022-12-06 17:35:42 - r - INFO: - trajectories 9 collected, reward 200.0 2022-12-06 17:35:56 - r - INFO: - trajectories 10 collected, reward 197.0 2022-12-06 17:36:04 - r - INFO: - trajectories 11 collected, reward 197.0 2022-12-06 17:36:10 - r - INFO: - trajectories 12 collected, reward 200.0 2022-12-06 17:36:17 - r - INFO: - trajectories 13 collected, reward 196.0 2022-12-06 17:37:15 - r - INFO: - trajectories 14 collected, reward 196.0 2022-12-06 17:38:00 - r - INFO: - trajectories 15 collected, reward 199.0 2022-12-06 17:38:06 - r - INFO: - trajectories 16 collected, reward 197.0 2022-12-06 17:38:22 - r - INFO: - trajectories 17 collected, reward 196.0 2022-12-06 17:38:41 - r - INFO: - trajectories 18 collected, reward 198.0 2022-12-06 17:39:07 - r - INFO: - trajectories 19 collected, reward 200.0 2022-12-06 17:39:29 - r - INFO: - trajectories 20 collected, reward 197.0 2022-12-06 17:39:38 - r - INFO: - trajectories 21 collected, reward 200.0 2022-12-06 17:40:10 - r - INFO: - trajectories 22 collected, reward 198.0 2022-12-06 17:40:20 - r - INFO: - trajectories 23 collected, reward 195.0 2022-12-06 17:40:35 - r - INFO: - trajectories 24 collected, reward 199.0 2022-12-06 17:40:53 - r - INFO: - trajectories 25 collected, reward 200.0 2022-12-06 17:41:00 - r - INFO: - trajectories 26 collected, reward 198.0 2022-12-06 17:41:53 - r - INFO: - trajectories 27 collected, reward 199.0 2022-12-06 17:41:57 - r - INFO: - trajectories 28 collected, reward 200.0 2022-12-06 17:42:46 - r - INFO: - trajectories 29 collected, reward 196.0 2022-12-06 17:43:18 - r - INFO: - trajectories 30 collected, reward 198.0 2022-12-06 17:43:32 - r - INFO: - trajectories 31 collected, reward 200.0 2022-12-06 17:43:52 - r - INFO: - trajectories 32 collected, reward 198.0 2022-12-06 17:44:21 - r - INFO: - trajectories 33 collected, reward 200.0 2022-12-06 17:44:26 - r - INFO: - trajectories 34 collected, reward 199.0 2022-12-06 17:44:47 - r - INFO: - trajectories 35 collected, reward 199.0 2022-12-06 17:45:12 - r - INFO: - trajectories 36 collected, reward 196.0 2022-12-06 17:45:23 - r - INFO: - trajectories 37 collected, reward 200.0 2022-12-06 17:45:26 - r - INFO: - trajectories 38 collected, reward 196.0 2022-12-06 17:45:39 - r - INFO: - trajectories 39 collected, reward 195.0 2022-12-06 17:45:42 - r - INFO: - trajectories 40 collected, reward 196.0 2022-12-06 17:45:48 - r - INFO: - trajectories 41 collected, reward 197.0 2022-12-06 17:45:59 - r - INFO: - trajectories 42 collected, reward 200.0 2022-12-06 17:46:20 - r - INFO: - trajectories 43 collected, reward 197.0 2022-12-06 17:46:22 - r - INFO: - trajectories 44 collected, reward 198.0 2022-12-06 17:46:25 - r - INFO: - trajectories 45 collected, reward 200.0 2022-12-06 17:46:59 - r - INFO: - trajectories 46 collected, reward 197.0 2022-12-06 17:47:17 - r - INFO: - trajectories 47 collected, reward 199.0 2022-12-06 17:47:23 - r - INFO: - trajectories 48 collected, reward 198.0 2022-12-06 17:47:30 - r - INFO: - trajectories 49 collected, reward 195.0 2022-12-06 17:47:47 - r - INFO: - trajectories 50 collected, reward 198.0 2022-12-06 17:47:59 - r - INFO: - trajectories 51 collected, reward 195.0 2022-12-06 17:48:02 - r - INFO: - trajectories 52 collected, reward 196.0 2022-12-06 17:48:16 - r - INFO: - trajectories 53 collected, reward 200.0 2022-12-06 17:48:50 - r - INFO: - trajectories 54 collected, reward 198.0 2022-12-06 17:48:55 - r - INFO: - trajectories 55 collected, reward 199.0 2022-12-06 17:49:35 - r - INFO: - trajectories 56 collected, reward 195.0 2022-12-06 17:49:46 - r - INFO: - trajectories 57 collected, reward 200.0 2022-12-06 17:50:07 - r - INFO: - trajectories 58 collected, reward 198.0 2022-12-06 17:50:50 - r - INFO: - trajectories 59 collected, reward 195.0 2022-12-06 17:50:55 - r - INFO: - trajectories 60 collected, reward 198.0 2022-12-06 17:51:43 - r - INFO: - trajectories 61 collected, reward 196.0 2022-12-06 17:52:22 - r - INFO: - trajectories 62 collected, reward 200.0 2022-12-06 17:52:24 - r - INFO: - trajectories 63 collected, reward 196.0 2022-12-06 17:52:24 - r - INFO: - trajectories 64 collected, reward 195.0 2022-12-06 17:52:39 - r - INFO: - trajectories 65 collected, reward 200.0 2022-12-06 17:53:54 - r - INFO: - trajectories 66 collected, reward 200.0 2022-12-06 17:53:58 - r - INFO: - trajectories 67 collected, reward 198.0 2022-12-06 17:54:16 - r - INFO: - trajectories 68 collected, reward 195.0 2022-12-06 17:55:00 - r - INFO: - trajectories 69 collected, reward 199.0 2022-12-06 17:55:02 - r - INFO: - trajectories 70 collected, reward 199.0 2022-12-06 17:55:42 - r - INFO: - trajectories 71 collected, reward 196.0 2022-12-06 17:56:12 - r - INFO: - trajectories 72 collected, reward 199.0 2022-12-06 17:57:33 - r - INFO: - trajectories 73 collected, reward 199.0 2022-12-06 17:57:44 - r - INFO: - trajectories 74 collected, reward 196.0 2022-12-06 17:58:08 - r - INFO: - trajectories 75 collected, reward 196.0 2022-12-06 17:58:14 - r - INFO: - trajectories 76 collected, reward 198.0 2022-12-06 17:58:24 - r - INFO: - trajectories 77 collected, reward 196.0 2022-12-06 17:59:53 - r - INFO: - trajectories 78 collected, reward 200.0 2022-12-06 18:00:34 - r - INFO: - trajectories 79 collected, reward 199.0 2022-12-06 18:00:52 - r - INFO: - trajectories 80 collected, reward 199.0 2022-12-06 18:01:03 - r - INFO: - trajectories 81 collected, reward 199.0 2022-12-06 18:01:05 - r - INFO: - trajectories 82 collected, reward 195.0 2022-12-06 18:01:13 - r - INFO: - trajectories 83 collected, reward 197.0 2022-12-06 18:01:26 - r - INFO: - trajectories 84 collected, reward 196.0 2022-12-06 18:02:51 - r - INFO: - trajectories 85 collected, reward 200.0 2022-12-06 18:03:01 - r - INFO: - trajectories 86 collected, reward 195.0 2022-12-06 18:03:02 - r - INFO: - trajectories 87 collected, reward 196.0 2022-12-06 18:03:46 - r - INFO: - trajectories 88 collected, reward 195.0 2022-12-06 18:05:09 - r - INFO: - trajectories 89 collected, reward 200.0 2022-12-06 18:05:26 - r - INFO: - trajectories 90 collected, reward 198.0 2022-12-06 18:06:06 - r - INFO: - trajectories 91 collected, reward 195.0 2022-12-06 18:07:49 - r - INFO: - trajectories 92 collected, reward 199.0 2022-12-06 18:07:55 - r - INFO: - trajectories 93 collected, reward 195.0 2022-12-06 18:08:36 - r - INFO: - trajectories 94 collected, reward 195.0 2022-12-06 18:08:37 - r - INFO: - trajectories 95 collected, reward 199.0 2022-12-06 18:09:36 - r - INFO: - trajectories 96 collected, reward 200.0 2022-12-06 18:10:16 - r - INFO: - trajectories 97 collected, reward 198.0 2022-12-06 18:10:16 - r - INFO: - trajectories 98 collected, reward 198.0 2022-12-06 18:10:17 - r - INFO: - trajectories 99 collected, reward 195.0 2022-12-06 18:11:23 - r - INFO: - trajectories 100 collected, reward 197.0 2022-12-06 18:11:23 - r - INFO: - trajectories saved to F:\rl-tutorials\joyrl/tasks/Collect_CartPole-v1_PPO_20221206-173222/traj/ 2022-12-06 18:11:23 - r - INFO: - Finish collecting!