2023-03-29 23:43:19 - r - INFO: - Hyperparameters:
2023-03-29 23:43:19 - r - INFO: - ================================================================================
2023-03-29 23:43:19 - r - INFO: -         Name        	       Value        	        Type        
2023-03-29 23:43:19 - r - INFO: -       env_name      	deep-sea-treasure-v0	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -     new_step_api    	         1          	   <class 'bool'>   
2023-03-29 23:43:19 - r - INFO: -       wrapper       	        None        	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -        render       	         0          	   <class 'bool'>   
2023-03-29 23:43:19 - r - INFO: -      algo_name      	    MO-QLearning    	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -         mode        	       train        	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -         seed        	         0          	   <class 'int'>    
2023-03-29 23:43:19 - r - INFO: -        device       	        cpu         	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -      train_eps      	        100         	   <class 'int'>    
2023-03-29 23:43:19 - r - INFO: -       test_eps      	         20         	   <class 'int'>    
2023-03-29 23:43:19 - r - INFO: -       eval_eps      	         10         	   <class 'int'>    
2023-03-29 23:43:19 - r - INFO: -   eval_per_episode  	         5          	   <class 'int'>    
2023-03-29 23:43:19 - r - INFO: -      max_steps      	        200         	   <class 'int'>    
2023-03-29 23:43:19 - r - INFO: -   load_checkpoint   	         0          	   <class 'bool'>   
2023-03-29 23:43:19 - r - INFO: -      load_path      	       tasks        	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -       show_fig      	         0          	   <class 'bool'>   
2023-03-29 23:43:19 - r - INFO: -       save_fig      	         1          	   <class 'bool'>   
2023-03-29 23:43:19 - r - INFO: -    epsilon_start    	        0.95        	  <class 'float'>   
2023-03-29 23:43:19 - r - INFO: -     epsilon_end     	        0.01        	  <class 'float'>   
2023-03-29 23:43:19 - r - INFO: -    epsilon_decay    	        300         	   <class 'int'>    
2023-03-29 23:43:19 - r - INFO: -        gamma        	        0.9         	  <class 'float'>   
2023-03-29 23:43:19 - r - INFO: -          lr         	        0.1         	  <class 'float'>   
2023-03-29 23:43:19 - r - INFO: -       weights       	     [0.5, 0.5]     	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -       task_dir      	E:\LAMDA\ongoing work\joyrl\joyrl-offline/tasks/Train_deep-sea-treasure-v0_MO-QLearning_20230329-234319	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -       res_dir       	E:\LAMDA\ongoing work\joyrl\joyrl-offline/tasks/Train_deep-sea-treasure-v0_MO-QLearning_20230329-234319/results	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -       log_dir       	E:\LAMDA\ongoing work\joyrl\joyrl-offline/tasks/Train_deep-sea-treasure-v0_MO-QLearning_20230329-234319/logs	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -       traj_dir      	E:\LAMDA\ongoing work\joyrl\joyrl-offline/tasks/Train_deep-sea-treasure-v0_MO-QLearning_20230329-234319/traj	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: -        tb_dir       	E:\LAMDA\ongoing work\joyrl\joyrl-offline/tasks/Train_deep-sea-treasure-v0_MO-QLearning_20230329-234319/tb_logs	   <class 'str'>    
2023-03-29 23:43:19 - r - INFO: - ================================================================================
2023-03-29 23:43:19 - r - INFO: - n_states: 2, n_actions: 4
2023-03-29 23:43:19 - r - INFO: - Start training!
2023-03-29 23:43:19 - r - INFO: - Env: deep-sea-treasure-v0, Algorithm: MO-QLearning, Device: cpu
2023-03-29 23:43:19 - r - INFO: - Episode: 1/100, Reward: 16.100, -131.000, Step: 131
2023-03-29 23:43:19 - r - INFO: - Episode: 2/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 3/100, Reward: 0.700, -4.000, Step: 4
2023-03-29 23:43:19 - r - INFO: - Episode: 4/100, Reward: 0.700, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Episode: 5/100, Reward: 8.200, -15.000, Step: 15
2023-03-29 23:43:19 - r - INFO: - Current episode 5 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 6/100, Reward: 0.700, -4.000, Step: 4
2023-03-29 23:43:19 - r - INFO: - Episode: 7/100, Reward: 0.700, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Episode: 8/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 9/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 10/100, Reward: 8.200, -5.000, Step: 5
2023-03-29 23:43:19 - r - INFO: - Current episode 10 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 11/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 12/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 13/100, Reward: 8.200, -5.000, Step: 5
2023-03-29 23:43:19 - r - INFO: - Episode: 14/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 15/100, Reward: 0.700, -5.000, Step: 5
2023-03-29 23:43:19 - r - INFO: - Current episode 15 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 16/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 17/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 18/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 19/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 20/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 20 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 21/100, Reward: 8.200, -6.000, Step: 6
2023-03-29 23:43:19 - r - INFO: - Episode: 22/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 23/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 24/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 25/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 25 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 26/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 27/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 28/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 29/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 30/100, Reward: 8.200, -4.000, Step: 4
2023-03-29 23:43:19 - r - INFO: - Current episode 30 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 31/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 32/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 33/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 34/100, Reward: 8.200, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Episode: 35/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Current episode 35 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 36/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 37/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 38/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 39/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 40/100, Reward: 8.200, -4.000, Step: 4
2023-03-29 23:43:19 - r - INFO: - Current episode 40 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 41/100, Reward: 0.700, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Episode: 42/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 43/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 44/100, Reward: 0.700, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Episode: 45/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 45 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 46/100, Reward: 8.200, -7.000, Step: 7
2023-03-29 23:43:19 - r - INFO: - Episode: 47/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 48/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 49/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 50/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Current episode 50 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 51/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 52/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 53/100, Reward: 8.200, -25.000, Step: 25
2023-03-29 23:43:19 - r - INFO: - Episode: 54/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 55/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Current episode 55 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 56/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 57/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 58/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 59/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 60/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 60 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 61/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 62/100, Reward: 8.200, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Episode: 63/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 64/100, Reward: 8.200, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Episode: 65/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 65 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 66/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 67/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 68/100, Reward: 0.700, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Episode: 69/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 70/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 70 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 71/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 72/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 73/100, Reward: 8.200, -6.000, Step: 6
2023-03-29 23:43:19 - r - INFO: - Episode: 74/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 75/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 75 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 76/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 77/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 78/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 79/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 80/100, Reward: 8.200, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Current episode 80 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 81/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 82/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 83/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 84/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 85/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 85 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 86/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Episode: 87/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 88/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 89/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 90/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Current episode 90 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 91/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 92/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 93/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 94/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 95/100, Reward: 0.700, -3.000, Step: 3
2023-03-29 23:43:19 - r - INFO: - Current episode 95 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Episode: 96/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 97/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 98/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 99/100, Reward: 0.700, -1.000, Step: 1
2023-03-29 23:43:19 - r - INFO: - Episode: 100/100, Reward: 0.700, -2.000, Step: 2
2023-03-29 23:43:19 - r - INFO: - Current episode 100 has the best eval reward: 0.700
2023-03-29 23:43:19 - r - INFO: - Finish training!