Upload PPO LunarLander-v2 trained agent

README.md

@@ -16,7 +16,7 @@ model-index:
       type: LunarLander-v2
     metrics:
     - type: mean_reward
-      value: 278.15 +/- 19.11
+      value: 285.10 +/- 13.63
       name: mean_reward
       verified: false
 ---
@@ -25,137 +25,13 @@ model-index:
 This is a trained model of a **PPO** agent playing **LunarLander-v2**
 using the [stable-baselines3 library](https://github.com/DLR-RM/stable-baselines3).
 
-## Install dependencies and create a virtual screen 🔽
+## Usage (with Stable-baselines3)
+TODO: Add your code
 
-```python
-# Virtual display
-from pyvirtualdisplay import Display
-
-virtual_display = Display(visible=0, size=(1400, 900))
-virtual_display.start()
-```
-## Import the packages 📦
-```python
-import gym
-
-from huggingface_sb3 import load_from_hub, package_to_hub, push_to_hub
-from huggingface_hub import notebook_login # To log to our Hugging Face account to be able to upload models to the Hub.
-
-from stable_baselines3 import PPO
-from stable_baselines3.common.evaluation import evaluate_policy
-from stable_baselines3.common.env_util import make_vec_env
-```
-## Understand what is Gym and how it works 🤖
-```python
-import gym
-
-# First, we create our environment called LunarLander-v2
-env = gym.make("LunarLander-v2")
-
-# Then we reset this environment
-observation = env.reset()
-
-for _ in range(20):
-  # Take a random action
-  action = env.action_space.sample()
-  print("Action taken:", action)
-
-  # Do this action in the environment and get
-  # next_state, reward, done and info
-  observation, reward, done, info = env.step(action)
-  
-  # If the game is done (in our case we land, crashed or timeout)
-  if done:
-      # Reset the environment
-      print("Environment is reset")
-      observation = env.reset()
-```
-## Create the LunarLander environment 🌛 and understand how it works
-```python
-# We create our environment with gym.make("<name_of_the_environment>")
-env = gym.make("LunarLander-v2")
-env.reset()
-print("_____OBSERVATION SPACE_____ \n")
-print("Observation Space Shape", env.observation_space.shape)
-print("Sample observation", env.observation_space.sample()) # Get a random observation
-print("\n _____ACTION SPACE_____ \n")
-print("Action Space Shape", env.action_space.n)
-print("Action Space Sample", env.action_space.sample()) # Take a random action
-
-# Create the environment
-env = make_vec_env('LunarLander-v2', n_envs=16)
-```
-## Create the Model 🤖
-```python
-# We added some parameters to accelerate the training
-model = PPO(
-    policy = 'MlpPolicy',
-    env = env,
-    n_steps = 1024,
-    batch_size = 64,
-    n_epochs = 4,
-    gamma = 0.999,
-    gae_lambda = 0.98,
-    ent_coef = 0.01,
-    verbose=1)
-model_name = "ppo-LunarLander-v2"
-```
-
-## Train the PPO agent 🏃
-```python
-# Train it for 1,000,000 timesteps
-model.learn(total_timesteps=3000000)
-# Save the model
-model.save(model_name)
-```
 
-## Evaluate the agent 📈
 ```python
-#load the model
-model = model.load("/content/ppo-LunarLander-v2.zip")
+from stable_baselines3 import ...
+from huggingface_sb3 import load_from_hub
 
-eval_env = gym.make("LunarLander-v2")
-mean_reward, std_reward = evaluate_policy(model, eval_env, n_eval_episodes=10, deterministic=True)
-print(f"mean_reward={mean_reward:.2f} +/- {std_reward}")
-```
-## Publish our trained model on the Hub 🔥
-```python
-notebook_login()
-!git config --global credential.helper store
+...
 ```
-```python
-import gym
-
-from stable_baselines3 import PPO
-from stable_baselines3.common.vec_env import DummyVecEnv
-from stable_baselines3.common.env_util import make_vec_env
-
-from huggingface_sb3 import package_to_hub
-
-# PLACE the variables you've just defined two cells above
-# Define the name of the environment
-env_id = "LunarLander-v2"
-
-# TODO: Define the model architecture we used
-model_architecture = "PPO"
-
-## Define a repo_id
-## repo_id is the id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name} for instance ThomasSimonini/ppo-LunarLander-v2
-## CHANGE WITH YOUR REPO ID
-repo_id = "vicfeuga/ppo-LunarLander-v2" # Change with your repo id, you can't push with mine 😄
-
-## Define the commit message
-commit_message = "Upload PPO LunarLander-v2 trained agent"
-
-# Create the evaluation env
-eval_env = DummyVecEnv([lambda: gym.make(env_id)])
-
-# PLACE the package_to_hub function you've just filled here
-package_to_hub(model=model, # Our trained model
-               model_name=model_name, # The name of our trained model 
-               model_architecture=model_architecture, # The model architecture we used: in our case PPO
-               env_id=env_id, # Name of the environment
-               eval_env=eval_env, # Evaluation Environment
-               repo_id=repo_id, # id of the model repository from the Hugging Face Hub (repo_id = {organization}/{repo_name} for instance ThomasSimonini/ppo-LunarLander-v2
-               commit_message=commit_message)
-```

config.json

@@ -1 +1 @@
-{"policy_class": {":type:": "<class 'abc.ABCMeta'>", ":serialized:": "gAWVOwAAAAAAAACMIXN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi5wb2xpY2llc5SMEUFjdG9yQ3JpdGljUG9saWN5lJOULg==", "__module__": "stable_baselines3.common.policies", "__doc__": "\n    Policy class for actor-critic algorithms (has both policy and value prediction).\n    Used by A2C, PPO and the likes.\n\n    :param observation_space: Observation space\n    :param action_space: Action space\n    :param lr_schedule: Learning rate schedule (could be constant)\n    :param net_arch: The specification of the policy and value networks.\n    :param activation_fn: Activation function\n    :param ortho_init: Whether to use or not orthogonal initialization\n    :param use_sde: Whether to use State Dependent Exploration or not\n    :param log_std_init: Initial value for the log standard deviation\n    :param full_std: Whether to use (n_features x n_actions) parameters\n        for the std instead of only (n_features,) when using gSDE\n    :param sde_net_arch: Network architecture for extracting features\n        when using gSDE. If None, the latent features from the policy will be used.\n        Pass an empty list to use the states as features.\n    :param use_expln: Use ``expln()`` function instead of ``exp()`` to ensure\n        a positive standard deviation (cf paper). It allows to keep variance\n        above zero and prevent it from growing too fast. In practice, ``exp()`` is usually enough.\n    :param squash_output: Whether to squash the output using a tanh function,\n        this allows to ensure boundaries when using gSDE.\n    :param features_extractor_class: Features extractor to use.\n    :param features_extractor_kwargs: Keyword arguments\n        to pass to the features extractor.\n    :param normalize_images: Whether to normalize images or not,\n         dividing by 255.0 (True by default)\n    :param optimizer_class: The optimizer to use,\n        ``th.optim.Adam`` by default\n    :param optimizer_kwargs: Additional keyword arguments,\n        excluding the learning rate, to pass to the optimizer\n    ", "__init__": "<function ActorCriticPolicy.__init__ at 0x7f455c197040>", "_get_constructor_parameters": "<function ActorCriticPolicy._get_constructor_parameters at 0x7f455c1970d0>", "reset_noise": "<function ActorCriticPolicy.reset_noise at 0x7f455c197160>", "_build_mlp_extractor": "<function ActorCriticPolicy._build_mlp_extractor at 0x7f455c1971f0>", "_build": "<function ActorCriticPolicy._build at 0x7f455c197280>", "forward": "<function ActorCriticPolicy.forward at 0x7f455c197310>", "_get_action_dist_from_latent": "<function ActorCriticPolicy._get_action_dist_from_latent at 0x7f455c1973a0>", "_predict": "<function ActorCriticPolicy._predict at 0x7f455c197430>", "evaluate_actions": "<function ActorCriticPolicy.evaluate_actions at 0x7f455c1974c0>", "get_distribution": "<function ActorCriticPolicy.get_distribution at 0x7f455c197550>", "predict_values": "<function ActorCriticPolicy.predict_values at 0x7f455c1975e0>", "__abstractmethods__": "frozenset()", "_abc_impl": "<_abc_data object at 0x7f455c195120>"}, "verbose": 1, "policy_kwargs": {}, "observation_space": {":type:": "<class 'gym.spaces.box.Box'>", ":serialized:": "gAWVnwEAAAAAAACMDmd5bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lGgFk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMBl9zaGFwZZRLCIWUjANsb3eUjBJudW1weS5jb3JlLm51bWVyaWOUjAtfZnJvbWJ1ZmZlcpSTlCiWIAAAAAAAAAAAAID/AACA/wAAgP8AAID/AACA/wAAgP8AAID/AACA/5RoCksIhZSMAUOUdJRSlIwEaGlnaJRoEiiWIAAAAAAAAAAAAIB/AACAfwAAgH8AAIB/AACAfwAAgH8AAIB/AACAf5RoCksIhZRoFXSUUpSMDWJvdW5kZWRfYmVsb3eUaBIolggAAAAAAAAAAAAAAAAAAACUaAeMAmIxlImIh5RSlChLA4wBfJROTk5K/////0r/////SwB0lGJLCIWUaBV0lFKUjA1ib3VuZGVkX2Fib3ZllGgSKJYIAAAAAAAAAAAAAAAAAAAAlGghSwiFlGgVdJRSlIwKX25wX3JhbmRvbZROdWIu", "dtype": "float32", "_shape": [8], "low": "[-inf -inf -inf -inf -inf -inf -inf -inf]", "high": "[inf inf inf inf inf inf inf inf]", "bounded_below": "[False False False False False False False False]", "bounded_above": "[False False False False False False False False]", "_np_random": null}, "action_space": {":type:": "<class 'gym.spaces.discrete.Discrete'>", ":serialized:": "gAWVggAAAAAAAACME2d5bS5zcGFjZXMuZGlzY3JldGWUjAhEaXNjcmV0ZZSTlCmBlH2UKIwBbpRLBIwGX3NoYXBllCmMBWR0eXBllIwFbnVtcHmUaAeTlIwCaTiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYowKX25wX3JhbmRvbZROdWIu", "n": 4, "_shape": [], "dtype": "int64", "_np_random": null}, "n_envs": 16, "num_timesteps": 3014656, "_total_timesteps": 3000000, "_num_timesteps_at_start": 0, "seed": null, "action_noise": null, "start_time": 1673263818381712278, "learning_rate": 0.0003, "tensorboard_log": null, "lr_schedule": {":type:": "<class 'function'>", ":serialized:": "gAWVwwIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZSMBGZ1bmOUS4BDAgABlIwDdmFslIWUKXSUUpR9lCiMC19fcGFja2FnZV9flIwYc3RhYmxlX2Jhc2VsaW5lczMuY29tbW9ulIwIX19uYW1lX1+UjB5zdGFibGVfYmFzZWxpbmVzMy5jb21tb24udXRpbHOUjAhfX2ZpbGVfX5SMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFmgNjAxfX3F1YWxuYW1lX1+UjBljb25zdGFudF9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UjA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoF4wHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/M6kqMFUyYYWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="}, "_last_obs": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVdQIAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYAAgAAAAAAAObkIj0pmF26fL2GNuBedzGM4z051nijtQAAgD8AAIA/5pb2vQfjXD/hr7K9s6LYvqzQEb5uuYo9AAAAAAAAAAAAYO07RqhPP/zvlr3288y+yI4VvDatdb0AAAAAAAAAAJoZkLl7HIS6MpyRst6ierBAqec6qxEMMwAAgD8AAIA/TZfxPa0pLz9Aq3e+p0DOvjPjyjzwTYe+AAAAAAAAAADmjG+93eMOPhw3ET1q2p6+Oph3uwNf87wAAAAAAAAAAGbAO7z4ueE9XuQzPSNDpb4fnfa9QiThvAAAAAAAAAAAGgDCPSkkcboaFSC4IoQXs05FJrtIkDs3AAAAAAAAAAAAAKk5ZUNaPuERxDsyeru+/eKNvRrZlz0AAAAAAAAAAAAVurz6e1U+aUm1vN09rr5c06i9M6KkPAAAAAAAAAAAra0vPrhrWz+C1Re+D/rAvnEaXz7fzEu+AAAAAAAAAABNA+Y9cDiTP55MnD6DFtK+A6SJPvX4iD4AAAAAAAAAAFtKmr6kw4g/2UvIvUQ3877Kyf6+EKBsPQAAAAAAAAAAjbQ/PhIBoz8OABM/XA/XvjCUwj4gou0+AAAAAAAAAADNg4S8Ug63u7p8i7vsXpE88pkbPVBBdr0AAIA/AACAP42ksj0zWAo/FCqYvoIot77uhdm9wEVcvQAAAAAAAAAAlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjSUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYksQSwiGlIwBQ5R0lFKULg=="}, "_last_episode_starts": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVgwAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACUjAVudW1weZSMBWR0eXBllJOUjAJiMZSJiIeUUpQoSwOMAXyUTk5OSv////9K/////0sAdJRiSxCFlIwBQ5R0lFKULg=="}, "_last_original_obs": null, "_episode_num": 0, "use_sde": false, "sde_sample_freq": -1, "_current_progress_remaining": -0.004885333333333408, "ep_info_buffer": {":type:": "<class 'collections.deque'>", ":serialized:": "gAWVKxAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMIlYJuL6lZckCUhpRSlIwBbJRL4YwBdJRHQKUsRGViWmh1fZQoaAZoCWgPQwh/bJIfMThxQJSGlFKUaBVL3WgWR0ClLE562OQydX2UKGgGaAloD0MICYuKOF2XcUCUhpRSlGgVS91oFkdApSyVBUrCnHV9lChoBmgJaA9DCGe610k9OnBAlIaUUpRoFUvraBZHQKUs3RXOnl51fZQoaAZoCWgPQwhzR//LtcJsQJSGlFKUaBVL8WgWR0ClLNwCr92pdX2UKGgGaAloD0MIjnQGRt6TcUCUhpRSlGgVTRYBaBZHQKUtAdJ8OTd1fZQoaAZoCWgPQwhh/Z/DfLhyQJSGlFKUaBVNBwFoFkdApS0CJQ+EAnV9lChoBmgJaA9DCGsMOiH0XW9AlIaUUpRoFUvhaBZHQKUtOWom5Ud1fZQoaAZoCWgPQwjH8xlQb3RvQJSGlFKUaBVL/GgWR0ClLV/ffoA5dX2UKGgGaAloD0MI3EjZIil1cUCUhpRSlGgVS+JoFkdApS2Cjvd/KHV9lChoBmgJaA9DCGkB2lazxG9AlIaUUpRoFUv9aBZHQKUtpwtrbg11fZQoaAZoCWgPQwjRQCybOTFwQJSGlFKUaBVL2mgWR0ClLdf6O5rhdX2UKGgGaAloD0MIe6Lrwk9ocECUhpRSlGgVS+toFkdApS3cHt4RmXV9lChoBmgJaA9DCGb1DreD4XFAlIaUUpRoFU0OAWgWR0ClLe53cHnmdX2UKGgGaAloD0MI/Z/DfHnBP0CUhpRSlGgVS8BoFkdApS4/smfGuXV9lChoBmgJaA9DCO27IvhfpW9AlIaUUpRoFUv2aBZHQKUusaEzwc51fZQoaAZoCWgPQwhcrKjBtHVyQJSGlFKUaBVL3GgWR0ClLsmDDjzadX2UKGgGaAloD0MIcayL2+j+cUCUhpRSlGgVS+FoFkdApS7k63iJf3V9lChoBmgJaA9DCJrtCn0wVHJAlIaUUpRoFUvbaBZHQKUvYWdmQKd1fZQoaAZoCWgPQwhbs5WXvEByQJSGlFKUaBVL92gWR0ClL3Hn2ZiNdX2UKGgGaAloD0MIxR9FnfnlckCUhpRSlGgVS+BoFkdApS+dI065oXV9lChoBmgJaA9DCHqJsUy/dHFAlIaUUpRoFUvSaBZHQKUvqs8PnSx1fZQoaAZoCWgPQwjPo+L/Tn5xQJSGlFKUaBVL5GgWR0ClL6r74zrNdX2UKGgGaAloD0MIovFEEGfkckCUhpRSlGgVS/ZoFkdApS+7P0I1L3V9lChoBmgJaA9DCGE41zCDTnFAlIaUUpRoFUveaBZHQKUv7vE0iyJ1fZQoaAZoCWgPQwi3e7lPjtNwQJSGlFKUaBVL2mgWR0ClMChX8wYcdX2UKGgGaAloD0MIahSSzKpqckCUhpRSlGgVS+hoFkdApTArsOXmeXV9lChoBmgJaA9DCBtn0xGAr3FAlIaUUpRoFUvhaBZHQKUwiH2ys0Z1fZQoaAZoCWgPQwhmEvWCTwFyQJSGlFKUaBVNAAFoFkdApTDHkaMrE3V9lChoBmgJaA9DCETbMXWXI3FAlIaUUpRoFU0CAWgWR0ClMNE9t/FzdX2UKGgGaAloD0MI9BlQb0bQcECUhpRSlGgVS/doFkdApTEdbA1vVHV9lChoBmgJaA9DCK+YEd4e2m5AlIaUUpRoFUvkaBZHQKUxaan75211fZQoaAZoCWgPQwg2rKksCt1xQJSGlFKUaBVL9GgWR0ClMYX8n/kvdX2UKGgGaAloD0MITkUqjC1MckCUhpRSlGgVS/RoFkdApTGyJCSid3V9lChoBmgJaA9DCAUYlj+f/nFAlIaUUpRoFUvYaBZHQKUx1zbN8md1fZQoaAZoCWgPQwhqiCr82VVwQJSGlFKUaBVL3GgWR0ClOxRh2GIsdX2UKGgGaAloD0MIrRQCuUR9cUCUhpRSlGgVS9poFkdApTs0A1ejVXV9lChoBmgJaA9DCOblsPvOnXJAlIaUUpRoFUvYaBZHQKU7T7Qb+991fZQoaAZoCWgPQwhEaW/wBZRxQJSGlFKUaBVL42gWR0ClO2kWqLjxdX2UKGgGaAloD0MI9FDbhtF6cUCUhpRSlGgVS/BoFkdApTvjuMMqjXV9lChoBmgJaA9DCJsEb0hjHXBAlIaUUpRoFUvkaBZHQKU8Ao+fRNR1fZQoaAZoCWgPQwhckZigBsVuQJSGlFKUaBVL9mgWR0ClPDGOlwcYdX2UKGgGaAloD0MI4/p3faa4cUCUhpRSlGgVS91oFkdApTybodMj/3V9lChoBmgJaA9DCNEhcCRQyW5AlIaUUpRoFUv2aBZHQKU8mt7rs0J1fZQoaAZoCWgPQwiY9s391eJwQJSGlFKUaBVL/GgWR0ClPOlZ5iVjdX2UKGgGaAloD0MIqB5pcJtMcUCUhpRSlGgVS+FoFkdApTz1SOzY3HV9lChoBmgJaA9DCHycacK2LXNAlIaUUpRoFUvXaBZHQKU9IR7qptJ1fZQoaAZoCWgPQwjtuOF3081wQJSGlFKUaBVL7GgWR0ClPX1LJ0W/dX2UKGgGaAloD0MIHjaRmUvLcUCUhpRSlGgVS+FoFkdApT2KQV9F4XV9lChoBmgJaA9DCNP02QFXWnJAlIaUUpRoFUvhaBZHQKU9xQnhKlJ1fZQoaAZoCWgPQwhypZ4FIdFwQJSGlFKUaBVL6mgWR0ClPcU/W1+idX2UKGgGaAloD0MIk/5eCs9NcECUhpRSlGgVS+toFkdApT4AmkWRBHV9lChoBmgJaA9DCNrk8ElnpnFAlIaUUpRoFUvkaBZHQKU+EXKKYRd1fZQoaAZoCWgPQwiz6nO1FadwQJSGlFKUaBVL9GgWR0ClPizGxUvPdX2UKGgGaAloD0MIjPhOzDqUcUCUhpRSlGgVS/JoFkdApT6rY02tMnV9lChoBmgJaA9DCOEKKNRT/XBAlIaUUpRoFU0CAWgWR0ClPv/NqxkedX2UKGgGaAloD0MI96sA3+3tb0CUhpRSlGgVTQABaBZHQKU/MqBmPHV1fZQoaAZoCWgPQwikObLySxdxQJSGlFKUaBVL72gWR0ClP3E9ECvHdX2UKGgGaAloD0MIzxWlhOA7bkCUhpRSlGgVS/RoFkdApT+AFFDv3XV9lChoBmgJaA9DCAnh0cYRGXFAlIaUUpRoFUvkaBZHQKU/pdcB2fV1fZQoaAZoCWgPQwg2c0hqoZdyQJSGlFKUaBVL7WgWR0ClP89bX6IndX2UKGgGaAloD0MIG2K85lWucECUhpRSlGgVS+FoFkdApT/a6asp5XV9lChoBmgJaA9DCLcMOEtJvHBAlIaUUpRoFUvnaBZHQKVAT4C6pYN1fZQoaAZoCWgPQwh0CYfeIspwQJSGlFKUaBVL+mgWR0ClQIAAIY3vdX2UKGgGaAloD0MIGVWGcXfocECUhpRSlGgVS+JoFkdApUCB5LRKH3V9lChoBmgJaA9DCOaUgJiE7XFAlIaUUpRoFU0AAWgWR0ClQOBomG/OdX2UKGgGaAloD0MIx549l6nObkCUhpRSlGgVS99oFkdApUDuFxn3+XV9lChoBmgJaA9DCCLElbN3xW9AlIaUUpRoFUvzaBZHQKVA9/kvK2d1fZQoaAZoCWgPQwiV8loJ3dZxQJSGlFKUaBVL+WgWR0ClQRhG6PKddX2UKGgGaAloD0MIkUWaeIdzcECUhpRSlGgVS+RoFkdApUFtC/oJRnV9lChoBmgJaA9DCBoziXrBinJAlIaUUpRoFUvjaBZHQKVBurOqvNh1fZQoaAZoCWgPQwhHIF7Xb8hwQJSGlFKUaBVL5WgWR0ClQe4i5d4WdX2UKGgGaAloD0MI203wTZPncUCUhpRSlGgVS+RoFkdApUIg5HVf/nV9lChoBmgJaA9DCCdok8NnEnFAlIaUUpRoFUvdaBZHQKVCanivPkd1fZQoaAZoCWgPQwjW/s72aPNwQJSGlFKUaBVL9mgWR0ClQmpyhi9adX2UKGgGaAloD0MIk4rG2t+NcUCUhpRSlGgVS+1oFkdApUJz41xbS3V9lChoBmgJaA9DCMWp1sJsG3FAlIaUUpRoFUvtaBZHQKVCqIpH7P91fZQoaAZoCWgPQwjFAIkm0LZvQJSGlFKUaBVL5WgWR0ClQwVjAi3YdX2UKGgGaAloD0MIoUyjycU2b0CUhpRSlGgVS+9oFkdApUNWdy1eB3V9lChoBmgJaA9DCBH8byW7qHBAlIaUUpRoFUvyaBZHQKVDXky1uzh1fZQoaAZoCWgPQwhXPsvzIAJyQJSGlFKUaBVL42gWR0ClQ41G0/nodX2UKGgGaAloD0MI5llJKz5xcECUhpRSlGgVS+VoFkdApUOggTyrgnV9lChoBmgJaA9DCPAyw0ZZ725AlIaUUpRoFUvxaBZHQKVDzdD6WPd1fZQoaAZoCWgPQwjYLJeNTs1xQJSGlFKUaBVL8WgWR0ClQ/BWo3rEdX2UKGgGaAloD0MI3bQZp+G5cUCUhpRSlGgVS+poFkdApUQ1mDlHSXV9lChoBmgJaA9DCEGADB07pnFAlIaUUpRoFUvtaBZHQKVEi1c+qzZ1fZQoaAZoCWgPQwj0+/7Ni/pnQJSGlFKUaBVN6ANoFkdApUSV8eCCjHV9lChoBmgJaA9DCGSxTSpavXBAlIaUUpRoFUvXaBZHQKVEryWAwwl1fZQoaAZoCWgPQwi5GW7Ap8ZyQJSGlFKUaBVL1WgWR0ClRSW+fywwdX2UKGgGaAloD0MIfhzNkRVZcUCUhpRSlGgVTQABaBZHQKVFcC5Etul1fZQoaAZoCWgPQwhVMgBU8bNwQJSGlFKUaBVNAwFoFkdApUWD4WUKRnV9lChoBmgJaA9DCAOTG0XWNnNAlIaUUpRoFU0NAWgWR0ClRZmqxTsIdX2UKGgGaAloD0MIXfsCeuGTcECUhpRSlGgVS9RoFkdApUXcifQKKHV9lChoBmgJaA9DCNkngGIkGXJAlIaUUpRoFUvsaBZHQKVGMNEPUa11fZQoaAZoCWgPQwhLPQtCuVpzQJSGlFKUaBVNEQFoFkdApUZPggow23V9lChoBmgJaA9DCDrMlxdgmXBAlIaUUpRoFUvpaBZHQKVGWy/KyOd1fZQoaAZoCWgPQwg8aHbd28RwQJSGlFKUaBVL8mgWR0ClRox+BpYcdX2UKGgGaAloD0MIZjIcz+dac0CUhpRSlGgVS9loFkdApUaU78vVVnV9lChoBmgJaA9DCOxP4nMnv3NAlIaUUpRoFUvfaBZHQKVG8/8EV351fZQoaAZoCWgPQwirmEo/Yc5vQJSGlFKUaBVNAgFoFkdApUb0OG0u2HVlLg=="}, "ep_success_buffer": {":type:": "<class 'collections.deque'>", ":serialized:": "gAWVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="}, "_n_updates": 736, "n_steps": 1024, "gamma": 0.999, "gae_lambda": 0.98, "ent_coef": 0.01, "vf_coef": 0.5, "max_grad_norm": 0.5, "batch_size": 64, "n_epochs": 4, "clip_range": {":type:": "<class 'function'>", ":serialized:": "gAWVwwIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZSMBGZ1bmOUS4BDAgABlIwDdmFslIWUKXSUUpR9lCiMC19fcGFja2FnZV9flIwYc3RhYmxlX2Jhc2VsaW5lczMuY29tbW9ulIwIX19uYW1lX1+UjB5zdGFibGVfYmFzZWxpbmVzMy5jb21tb24udXRpbHOUjAhfX2ZpbGVfX5SMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFmgNjAxfX3F1YWxuYW1lX1+UjBljb25zdGFudF9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UjA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoF4wHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/yZmZmZmZmoWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="}, "clip_range_vf": null, "normalize_advantage": true, "target_kl": null, "system_info": {"OS": "Linux-5.10.147+-x86_64-with-glibc2.27 #1 SMP Sat Dec 10 16:00:40 UTC 2022", "Python": "3.8.16", "Stable-Baselines3": "1.6.2", "PyTorch": "1.13.0+cu116", "GPU Enabled": "True", "Numpy": "1.21.6", "Gym": "0.21.0"}}
+{"policy_class": {":type:": "<class 'abc.ABCMeta'>", ":serialized:": "gAWVOwAAAAAAAACMIXN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi5wb2xpY2llc5SMEUFjdG9yQ3JpdGljUG9saWN5lJOULg==", "__module__": "stable_baselines3.common.policies", "__doc__": "\n    Policy class for actor-critic algorithms (has both policy and value prediction).\n    Used by A2C, PPO and the likes.\n\n    :param observation_space: Observation space\n    :param action_space: Action space\n    :param lr_schedule: Learning rate schedule (could be constant)\n    :param net_arch: The specification of the policy and value networks.\n    :param activation_fn: Activation function\n    :param ortho_init: Whether to use or not orthogonal initialization\n    :param use_sde: Whether to use State Dependent Exploration or not\n    :param log_std_init: Initial value for the log standard deviation\n    :param full_std: Whether to use (n_features x n_actions) parameters\n        for the std instead of only (n_features,) when using gSDE\n    :param sde_net_arch: Network architecture for extracting features\n        when using gSDE. If None, the latent features from the policy will be used.\n        Pass an empty list to use the states as features.\n    :param use_expln: Use ``expln()`` function instead of ``exp()`` to ensure\n        a positive standard deviation (cf paper). It allows to keep variance\n        above zero and prevent it from growing too fast. In practice, ``exp()`` is usually enough.\n    :param squash_output: Whether to squash the output using a tanh function,\n        this allows to ensure boundaries when using gSDE.\n    :param features_extractor_class: Features extractor to use.\n    :param features_extractor_kwargs: Keyword arguments\n        to pass to the features extractor.\n    :param normalize_images: Whether to normalize images or not,\n         dividing by 255.0 (True by default)\n    :param optimizer_class: The optimizer to use,\n        ``th.optim.Adam`` by default\n    :param optimizer_kwargs: Additional keyword arguments,\n        excluding the learning rate, to pass to the optimizer\n    ", "__init__": "<function ActorCriticPolicy.__init__ at 0x7f1b1fcb5940>", "_get_constructor_parameters": "<function ActorCriticPolicy._get_constructor_parameters at 0x7f1b1fcb59d0>", "reset_noise": "<function ActorCriticPolicy.reset_noise at 0x7f1b1fcb5a60>", "_build_mlp_extractor": "<function ActorCriticPolicy._build_mlp_extractor at 0x7f1b1fcb5af0>", "_build": "<function ActorCriticPolicy._build at 0x7f1b1fcb5b80>", "forward": "<function ActorCriticPolicy.forward at 0x7f1b1fcb5c10>", "_get_action_dist_from_latent": "<function ActorCriticPolicy._get_action_dist_from_latent at 0x7f1b1fcb5ca0>", "_predict": "<function ActorCriticPolicy._predict at 0x7f1b1fcb5d30>", "evaluate_actions": "<function ActorCriticPolicy.evaluate_actions at 0x7f1b1fcb5dc0>", "get_distribution": "<function ActorCriticPolicy.get_distribution at 0x7f1b1fcb5e50>", "predict_values": "<function ActorCriticPolicy.predict_values at 0x7f1b1fcb5ee0>", "__abstractmethods__": "frozenset()", "_abc_impl": "<_abc_data object at 0x7f1b1fc9bde0>"}, "verbose": 1, "policy_kwargs": {}, "observation_space": {":type:": "<class 'gym.spaces.box.Box'>", ":serialized:": "gAWVnwEAAAAAAACMDmd5bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lGgFk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMBl9zaGFwZZRLCIWUjANsb3eUjBJudW1weS5jb3JlLm51bWVyaWOUjAtfZnJvbWJ1ZmZlcpSTlCiWIAAAAAAAAAAAAID/AACA/wAAgP8AAID/AACA/wAAgP8AAID/AACA/5RoCksIhZSMAUOUdJRSlIwEaGlnaJRoEiiWIAAAAAAAAAAAAIB/AACAfwAAgH8AAIB/AACAfwAAgH8AAIB/AACAf5RoCksIhZRoFXSUUpSMDWJvdW5kZWRfYmVsb3eUaBIolggAAAAAAAAAAAAAAAAAAACUaAeMAmIxlImIh5RSlChLA4wBfJROTk5K/////0r/////SwB0lGJLCIWUaBV0lFKUjA1ib3VuZGVkX2Fib3ZllGgSKJYIAAAAAAAAAAAAAAAAAAAAlGghSwiFlGgVdJRSlIwKX25wX3JhbmRvbZROdWIu", "dtype": "float32", "_shape": [8], "low": "[-inf -inf -inf -inf -inf -inf -inf -inf]", "high": "[inf inf inf inf inf inf inf inf]", "bounded_below": "[False False False False False False False False]", "bounded_above": "[False False False False False False False False]", "_np_random": null}, "action_space": {":type:": "<class 'gym.spaces.discrete.Discrete'>", ":serialized:": "gAWVggAAAAAAAACME2d5bS5zcGFjZXMuZGlzY3JldGWUjAhEaXNjcmV0ZZSTlCmBlH2UKIwBbpRLBIwGX3NoYXBllCmMBWR0eXBllIwFbnVtcHmUaAeTlIwCaTiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYowKX25wX3JhbmRvbZROdWIu", "n": 4, "_shape": [], "dtype": "int64", "_np_random": null}, "n_envs": 30, "num_timesteps": 3010560, "_total_timesteps": 3000000, "_num_timesteps_at_start": 0, "seed": null, "action_noise": null, "start_time": 1673340816052036269, "learning_rate": 0.0003, "tensorboard_log": null, "lr_schedule": {":type:": "<class 'function'>", ":serialized:": "gAWVwwIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZSMBGZ1bmOUS4BDAgABlIwDdmFslIWUKXSUUpR9lCiMC19fcGFja2FnZV9flIwYc3RhYmxlX2Jhc2VsaW5lczMuY29tbW9ulIwIX19uYW1lX1+UjB5zdGFibGVfYmFzZWxpbmVzMy5jb21tb24udXRpbHOUjAhfX2ZpbGVfX5SMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFmgNjAxfX3F1YWxuYW1lX1+UjBljb25zdGFudF9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UjA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoF4wHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/M6kqMFUyYYWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="}, "_last_obs": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVNQQAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJbAAwAAAAAAAGBFCj6b4ik/A4D9PYOERr/s/2w+QCWIPQAAAAAAAAAAZjwNPSkof7qisQgz7RE4MRU/TTu/Qp6zAACAPwAAgD+ab1A8FJCDugDAczo1REw1tsE5OgMpjLkAAIA/AACAP5ppJjspaGm6LlEOOhBjlDWVI7g6ELqHNAAAgD8AAIA/gKUTvnxHgz72J88++E0Hv7IaX7rPi8c+AAAAAAAAAAAA+tM96bsLPu23j74hIJC+/AxYvYZIJr4AAAAAAAAAAHNvtb189aM/RX3jvguFDL9i4Au+lrufvgAAAAAAAAAAmlmWPGxd9Ltb1Ay+xEHoPGhKPz2qrr+9AACAPwAAgD+apRq8hfvcua4wwbU1SrawTTSbu4KIADUAAIA/AACAP/C0hz7UzJ8/5sYNPzHRAr9RWeQ+6ORWPgAAAAAAAAAAZkbXuymUYbrdwe47h3ZNNZHYKbsf1EQ0AACAPwAAgD9mmpe8exCKuhqaITRtsYEv1IEGu+88nLMAAIA/AACAP5rmkrzO56u842GCPiqPPr7/qPa9QnM6vgAAgD8AAAAA02QbPh9lfz4z2Ny+Wh3svnn81jywa1++AAAAAAAAAADNz9G8XCMuuv7VTTy0doc8bw3Duv6zbj0AAIA/AACAP82sPDy4Fqc/wvYLPoTSLb/VCQU8J3w5PQAAAAAAAAAAmuFIPJJlhT+zGe888gBfvyu/aDxdQO28AAAAAAAAAAAa43s9ew6JuubzLbjebyWz27fsOd5wSjcAAIA/AACAP6ZsOL5undM+K1ykPr7cH79m8i2+1uKHPgAAAAAAAAAAM5ZXPY+2dLqIa7s2CXaFMSlu3ro+G9y1AACAPwAAgD9N/ys+v0chPlhi3r4XkM2+I564PRXweb4AAAAAAAAAAAB03byPllW6J3uuOR3xS7SlTKW6GoZzswAAgD8AAIA/rUgTPod7cD9IAqo+JrAmvwISkj7MhjU+AAAAAAAAAACaPog8lCOzOwLnS761zjO+3LO1u5giaj8AAAAAAAAAAFNzMD4FSTY+MMzCvoGh076zTUs99qMqvgAAAAAAAAAAAESgPQCIsz4etCa+lHHyvudGGT1qX2u9AAAAAAAAAABmfK48+yWgvL2uJryQkfI8pAnwvW5zST0AAIA/AACAPw0YXD49Owe90gISO+8Qs7nGmWq+s4qDugAAgD8AAIA/5tO7PcUGkj8mAJg+tNJBv8lwKz6YQAs+AAAAAAAAAABmvn69gzUWvKYDRD3UcUU9W0J6PTo4Er4AAIA/AACAP5SMBW51bXB5lIwFZHR5cGWUk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGJLHksIhpSMAUOUdJRSlC4="}, "_last_episode_starts": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVkQAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYeAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAJSMBW51bXB5lIwFZHR5cGWUk5SMAmIxlImIh5RSlChLA4wBfJROTk5K/////0r/////SwB0lGJLHoWUjAFDlHSUUpQu"}, "_last_original_obs": null, "_episode_num": 0, "use_sde": false, "sde_sample_freq": -1, "_current_progress_remaining": -0.0035199999999999676, "ep_info_buffer": {":type:": "<class 'collections.deque'>", ":serialized:": "gAWVMhAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMIg7709idVckCUhpRSlIwBbJRLyYwBdJRHQKoG2DgZTAF1fZQoaAZoCWgPQwgtCrsoenlxQJSGlFKUaBVL0WgWR0CqBxY5DJEIdX2UKGgGaAloD0MIKZXwhF41c0CUhpRSlGgVTS0BaBZHQKoHG5ZKWcB1fZQoaAZoCWgPQwglkBK79sNzQJSGlFKUaBVL/2gWR0CqB1yB06o3dX2UKGgGaAloD0MIf8LZrWVjcUCUhpRSlGgVS8NoFkdAqgeBIFvAGnV9lChoBmgJaA9DCBTsv84NQHFAlIaUUpRoFUvWaBZHQKoHpRDTjNp1fZQoaAZoCWgPQwhcrn5sEupwQJSGlFKUaBVLpmgWR0CqB9Mrd30PdX2UKGgGaAloD0MIfa62Yr+bcECUhpRSlGgVS7RoFkdAqgfqz5XU6XV9lChoBmgJaA9DCHIYzF+hlHJAlIaUUpRoFUvKaBZHQKoISTyJ9Ap1fZQoaAZoCWgPQwjk9ssna/VyQJSGlFKUaBVLl2gWR0CqCFat9x6wdX2UKGgGaAloD0MI+yDLgomncUCUhpRSlGgVS8poFkdAqgiC++M6zXV9lChoBmgJaA9DCMVU+gmnxHNAlIaUUpRoFUvGaBZHQKoIxoIv8Il1fZQoaAZoCWgPQwixGeCCbJ1xQJSGlFKUaBVLxGgWR0CqCNS+6Ae8dX2UKGgGaAloD0MIqtbCLHRRc0CUhpRSlGgVS+poFkdAqgjNHDrJKnV9lChoBmgJaA9DCEn0Mopl43JAlIaUUpRoFUvYaBZHQKoJCMPz4Dd1fZQoaAZoCWgPQwiDpiVWBq5wQJSGlFKUaBVLrGgWR0CqCStYr8R+dX2UKGgGaAloD0MIdv9YiA4lcUCUhpRSlGgVTUkBaBZHQKoJcTRplBh1fZQoaAZoCWgPQwiYM9sVehtyQJSGlFKUaBVLxGgWR0CqCcC1y/9HdX2UKGgGaAloD0MIbvqzHylCc0CUhpRSlGgVS+doFkdAqgndxffGdnV9lChoBmgJaA9DCBPyQc+mvnFAlIaUUpRoFUu8aBZHQKoJ7RMN+b51fZQoaAZoCWgPQwh/3H75JM1wQJSGlFKUaBVLuGgWR0CqCigAIY3vdX2UKGgGaAloD0MIaOvgYC8DdECUhpRSlGgVS9toFkdAqgpDJIUah3V9lChoBmgJaA9DCGJJufucaXFAlIaUUpRoFU0wAWgWR0CqCkhlDneSdX2UKGgGaAloD0MIs+xJYPOrcECUhpRSlGgVS8JoFkdAqgpO1KGtZHV9lChoBmgJaA9DCF69iozONnJAlIaUUpRoFU0MAWgWR0CqCk2vjfeldX2UKGgGaAloD0MIPZl/9A07c0CUhpRSlGgVTQ4BaBZHQKoKXtShrWR1fZQoaAZoCWgPQwjSOqqaIDxyQJSGlFKUaBVNJQFoFkdAqgsG1KGtZHV9lChoBmgJaA9DCFch5SdVZnBAlIaUUpRoFUvdaBZHQKoLKny/bj91fZQoaAZoCWgPQwjCEg8oG19zQJSGlFKUaBVLuGgWR0CqCzB/Aj6fdX2UKGgGaAloD0MIQl96+3PIc0CUhpRSlGgVS7xoFkdAqgtCr3j+73V9lChoBmgJaA9DCHo57L4j7HFAlIaUUpRoFUvaaBZHQKoLgqVhTfl1fZQoaAZoCWgPQwjVPEfk+8FzQJSGlFKUaBVL9mgWR0CqC4JQ+EAYdX2UKGgGaAloD0MICwithy9DPkCUhpRSlGgVS1toFkdAqgve2/i5u3V9lChoBmgJaA9DCIcXRKQmg3NAlIaUUpRoFUvxaBZHQKoMMJ53Tux1fZQoaAZoCWgPQwgV5GcjFytxQJSGlFKUaBVLuWgWR0CqDKRy4nWrdX2UKGgGaAloD0MIjh6/tykOckCUhpRSlGgVS+ZoFkdAqgzDh3qzJXV9lChoBmgJaA9DCEz+J3831nNAlIaUUpRoFUvhaBZHQKoM1fJmukl1fZQoaAZoCWgPQwgX9N4YQrFxQJSGlFKUaBVLqWgWR0CqDPX5vcagdX2UKGgGaAloD0MI3pBGBQ7LcECUhpRSlGgVS75oFkdAqg0bnRsuWnV9lChoBmgJaA9DCJ4oCYk03XJAlIaUUpRoFUvraBZHQKoNLkTYdyV1fZQoaAZoCWgPQwjxKQDGs41zQJSGlFKUaBVLwmgWR0CqDSwnx8UmdX2UKGgGaAloD0MIGvm84qlPc0CUhpRSlGgVS8ZoFkdAqg14dXDFZXV9lChoBmgJaA9DCAJjfQMTbnNAlIaUUpRoFUvxaBZHQKoNt2q1gIB1fZQoaAZoCWgPQwh4YtaLIapyQJSGlFKUaBVLnGgWR0CqDctZ/0/XdX2UKGgGaAloD0MIP/89eO3Lb0CUhpRSlGgVTQgBaBZHQKoOU+ajN6h1fZQoaAZoCWgPQwj2RUJbTkBxQJSGlFKUaBVL22gWR0CqDmtCJGe+dX2UKGgGaAloD0MI4gFlU65JckCUhpRSlGgVS8loFkdAqg69A5aNdnV9lChoBmgJaA9DCDNQGf/+d3FAlIaUUpRoFUvMaBZHQKoO9CFbmlt1fZQoaAZoCWgPQwglehnFclBwQJSGlFKUaBVLzWgWR0CqDwTpgTh6dX2UKGgGaAloD0MIBMjQsQPhcUCUhpRSlGgVS9FoFkdAqg8y86FM7HV9lChoBmgJaA9DCAhyUMKM7HFAlIaUUpRoFUu9aBZHQKoPevW6K+B1fZQoaAZoCWgPQwiYMQVrXAJyQJSGlFKUaBVLuWgWR0CqD/S1eBxxdX2UKGgGaAloD0MIeeV620wNc0CUhpRSlGgVS6VoFkdAqhA+zByjpXV9lChoBmgJaA9DCKDCEaTSMnFAlIaUUpRoFUvaaBZHQKoQhOIInjR1fZQoaAZoCWgPQwhfJLTlnCZxQJSGlFKUaBVNRwFoFkdAqhB/FirksHV9lChoBmgJaA9DCN3u5T75uHBAlIaUUpRoFUu+aBZHQKoQiMnZ00Z1fZQoaAZoCWgPQwjg929eHO9zQJSGlFKUaBVL5GgWR0CqELWmxdIHdX2UKGgGaAloD0MIXDl7ZzQ9ckCUhpRSlGgVTS4BaBZHQKoRJ8F6iTN1fZQoaAZoCWgPQwh4CyQofhtxQJSGlFKUaBVNPgFoFkdAqhFfezlcQnV9lChoBmgJaA9DCMRg/gpZw3FAlIaUUpRoFU04AWgWR0CqEcg2Ifr9dX2UKGgGaAloD0MIfEYiNELQcECUhpRSlGgVS7ZoFkdAqhHN+7UXpHV9lChoBmgJaA9DCEzChTxCC3JAlIaUUpRoFUvKaBZHQKoR4S0Sh8J1fZQoaAZoCWgPQwiO6nQgq5NwQJSGlFKUaBVLrmgWR0CqEfJ0wJw9dX2UKGgGaAloD0MIlKXW+w0HcUCUhpRSlGgVS71oFkdAqhH8Of/WD3V9lChoBmgJaA9DCIrNx7VhVXNAlIaUUpRoFUvnaBZHQKoSeIkZ75V1fZQoaAZoCWgPQwgtlbcj3DdzQJSGlFKUaBVN1wFoFkdAqhKFyYG+snV9lChoBmgJaA9DCHCaPjtgH3BAlIaUUpRoFUu8aBZHQKoSiwt8NQV1fZQoaAZoCWgPQwg1KQXd3nxyQJSGlFKUaBVLsWgWR0CqEwDU3GXHdX2UKGgGaAloD0MIcaq1MAv+cUCUhpRSlGgVS+9oFkdAqhMsK/mDDnV9lChoBmgJaA9DCKTjamTXSXJAlIaUUpRoFUu5aBZHQKoTh3g1m8N1fZQoaAZoCWgPQwiu8ZnsX3tyQJSGlFKUaBVNVQFoFkdAqhORkAggYHV9lChoBmgJaA9DCDhpGhRNfHNAlIaUUpRoFU0IAWgWR0CqE6M8YAKfdX2UKGgGaAloD0MIQFBu27dWckCUhpRSlGgVTSEBaBZHQKoUIvkili11fZQoaAZoCWgPQwhR3sfR3CxyQJSGlFKUaBVNbAFoFkdAqhSNIwudw3V9lChoBmgJaA9DCFPL1voiqHJAlIaUUpRoFUvaaBZHQKoUlQAMlTp1fZQoaAZoCWgPQwiVuflGNNdxQJSGlFKUaBVL2mgWR0CqFRjIJZ4fdX2UKGgGaAloD0MIhpLJqV2YckCUhpRSlGgVS+RoFkdAqhUUPvrnknV9lChoBmgJaA9DCPRwAtOpAHNAlIaUUpRoFUu4aBZHQKoVQEkjX4F1fZQoaAZoCWgPQwi0WfW5GlhxQJSGlFKUaBVLuWgWR0CqFT3VCojwdX2UKGgGaAloD0MI1NNH4M9zcECUhpRSlGgVS7xoFkdAqhYxC+lCTnV9lChoBmgJaA9DCHehuU6jp3NAlIaUUpRoFU0xAWgWR0CqFjYxk/bCdX2UKGgGaAloD0MI7s1vmOgCcECUhpRSlGgVS+ZoFkdAqhZuUOd5IHV9lChoBmgJaA9DCA7bFmV2HXNAlIaUUpRoFUvpaBZHQKoWraFEiMZ1fZQoaAZoCWgPQwjL2TujLbdyQJSGlFKUaBVL/2gWR0CqFsz7di2EdX2UKGgGaAloD0MIdFyN7AqqckCUhpRSlGgVTWABaBZHQKoW1Pa+N991fZQoaAZoCWgPQwhb0HtjyLdyQJSGlFKUaBVLmWgWR0CqFuviDM/ydX2UKGgGaAloD0MIcZNRZVitcECUhpRSlGgVS81oFkdAqhb+ukk8inV9lChoBmgJaA9DCNriGp/JE3JAlIaUUpRoFUvXaBZHQKoXRF6zE751fZQoaAZoCWgPQwgS+S6lrm9xQJSGlFKUaBVLumgWR0CqF0o91U2ldX2UKGgGaAloD0MILQWk/U+JcECUhpRSlGgVS8FoFkdAqhdkHWz4UXV9lChoBmgJaA9DCLudfeXB/nJAlIaUUpRoFU2SAmgWR0CqF3G8ujASdX2UKGgGaAloD0MI0jb+RKUJckCUhpRSlGgVS+FoFkdAqheRVyWAw3V9lChoBmgJaA9DCDYf14YKYHJAlIaUUpRoFUvIaBZHQKoXmCmMwUR1fZQoaAZoCWgPQwjqBgq8EzhxQJSGlFKUaBVNSwFoFkdAqhhUfaHsTnV9lChoBmgJaA9DCJj75CjARHNAlIaUUpRoFU0EAWgWR0CqGHzQmeDndX2UKGgGaAloD0MI0zHnGXuwc0CUhpRSlGgVS9BoFkdAqhi65NGmUHV9lChoBmgJaA9DCP9eCg8ar29AlIaUUpRoFUuvaBZHQKoY6GHHmzV1fZQoaAZoCWgPQwgh6j4AaX5yQJSGlFKUaBVL12gWR0CqGQCJGe+VdX2UKGgGaAloD0MIUFPL1rpocECUhpRSlGgVS+VoFkdAqhlNUp/gBXV9lChoBmgJaA9DCBL7BFCMFEFAlIaUUpRoFUtvaBZHQKoZhBsyi251ZS4="}, "ep_success_buffer": {":type:": "<class 'collections.deque'>", ":serialized:": "gAWVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="}, "_n_updates": 980, "n_steps": 1024, "gamma": 0.9999, "gae_lambda": 0.98, "ent_coef": 0.01, "vf_coef": 0.5, "max_grad_norm": 0.5, "batch_size": 64, "n_epochs": 10, "clip_range": {":type:": "<class 'function'>", ":serialized:": "gAWVwwIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZSMBGZ1bmOUS4BDAgABlIwDdmFslIWUKXSUUpR9lCiMC19fcGFja2FnZV9flIwYc3RhYmxlX2Jhc2VsaW5lczMuY29tbW9ulIwIX19uYW1lX1+UjB5zdGFibGVfYmFzZWxpbmVzMy5jb21tb24udXRpbHOUjAhfX2ZpbGVfX5SMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFmgNjAxfX3F1YWxuYW1lX1+UjBljb25zdGFudF9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UjA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoF4wHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/yZmZmZmZmoWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="}, "clip_range_vf": null, "normalize_advantage": true, "target_kl": null, "system_info": {"OS": "Linux-5.10.147+-x86_64-with-glibc2.27 #1 SMP Sat Dec 10 16:00:40 UTC 2022", "Python": "3.8.16", "Stable-Baselines3": "1.6.2", "PyTorch": "1.13.0+cu116", "GPU Enabled": "False", "Numpy": "1.21.6", "Gym": "0.21.0"}}

ppo-LunarLander-v2.zip

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c796c899364275bfcede5e6e6961b3dc5b437622fe3b6e9527bca4676045fec5
-size 147234
+oid sha256:f622c3810362412d39dfe03290e22f5e7600b0d3eccaed9277c3b4e5a3364157
+size 147218

ppo-LunarLander-v2/data

@@ -4,19 +4,19 @@
         ":serialized:": "gAWVOwAAAAAAAACMIXN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi5wb2xpY2llc5SMEUFjdG9yQ3JpdGljUG9saWN5lJOULg==",
         "__module__": "stable_baselines3.common.policies",
         "__doc__": "\n    Policy class for actor-critic algorithms (has both policy and value prediction).\n    Used by A2C, PPO and the likes.\n\n    :param observation_space: Observation space\n    :param action_space: Action space\n    :param lr_schedule: Learning rate schedule (could be constant)\n    :param net_arch: The specification of the policy and value networks.\n    :param activation_fn: Activation function\n    :param ortho_init: Whether to use or not orthogonal initialization\n    :param use_sde: Whether to use State Dependent Exploration or not\n    :param log_std_init: Initial value for the log standard deviation\n    :param full_std: Whether to use (n_features x n_actions) parameters\n        for the std instead of only (n_features,) when using gSDE\n    :param sde_net_arch: Network architecture for extracting features\n        when using gSDE. If None, the latent features from the policy will be used.\n        Pass an empty list to use the states as features.\n    :param use_expln: Use ``expln()`` function instead of ``exp()`` to ensure\n        a positive standard deviation (cf paper). It allows to keep variance\n        above zero and prevent it from growing too fast. In practice, ``exp()`` is usually enough.\n    :param squash_output: Whether to squash the output using a tanh function,\n        this allows to ensure boundaries when using gSDE.\n    :param features_extractor_class: Features extractor to use.\n    :param features_extractor_kwargs: Keyword arguments\n        to pass to the features extractor.\n    :param normalize_images: Whether to normalize images or not,\n         dividing by 255.0 (True by default)\n    :param optimizer_class: The optimizer to use,\n        ``th.optim.Adam`` by default\n    :param optimizer_kwargs: Additional keyword arguments,\n        excluding the learning rate, to pass to the optimizer\n    ",
-        "__init__": "<function ActorCriticPolicy.__init__ at 0x7f455c197040>",
-        "_get_constructor_parameters": "<function ActorCriticPolicy._get_constructor_parameters at 0x7f455c1970d0>",
-        "reset_noise": "<function ActorCriticPolicy.reset_noise at 0x7f455c197160>",
-        "_build_mlp_extractor": "<function ActorCriticPolicy._build_mlp_extractor at 0x7f455c1971f0>",
-        "_build": "<function ActorCriticPolicy._build at 0x7f455c197280>",
-        "forward": "<function ActorCriticPolicy.forward at 0x7f455c197310>",
-        "_get_action_dist_from_latent": "<function ActorCriticPolicy._get_action_dist_from_latent at 0x7f455c1973a0>",
-        "_predict": "<function ActorCriticPolicy._predict at 0x7f455c197430>",
-        "evaluate_actions": "<function ActorCriticPolicy.evaluate_actions at 0x7f455c1974c0>",
-        "get_distribution": "<function ActorCriticPolicy.get_distribution at 0x7f455c197550>",
-        "predict_values": "<function ActorCriticPolicy.predict_values at 0x7f455c1975e0>",
+        "__init__": "<function ActorCriticPolicy.__init__ at 0x7f1b1fcb5940>",
+        "_get_constructor_parameters": "<function ActorCriticPolicy._get_constructor_parameters at 0x7f1b1fcb59d0>",
+        "reset_noise": "<function ActorCriticPolicy.reset_noise at 0x7f1b1fcb5a60>",
+        "_build_mlp_extractor": "<function ActorCriticPolicy._build_mlp_extractor at 0x7f1b1fcb5af0>",
+        "_build": "<function ActorCriticPolicy._build at 0x7f1b1fcb5b80>",
+        "forward": "<function ActorCriticPolicy.forward at 0x7f1b1fcb5c10>",
+        "_get_action_dist_from_latent": "<function ActorCriticPolicy._get_action_dist_from_latent at 0x7f1b1fcb5ca0>",
+        "_predict": "<function ActorCriticPolicy._predict at 0x7f1b1fcb5d30>",
+        "evaluate_actions": "<function ActorCriticPolicy.evaluate_actions at 0x7f1b1fcb5dc0>",
+        "get_distribution": "<function ActorCriticPolicy.get_distribution at 0x7f1b1fcb5e50>",
+        "predict_values": "<function ActorCriticPolicy.predict_values at 0x7f1b1fcb5ee0>",
         "__abstractmethods__": "frozenset()",
-        "_abc_impl": "<_abc_data object at 0x7f455c195120>"
+        "_abc_impl": "<_abc_data object at 0x7f1b1fc9bde0>"
     },
     "verbose": 1,
     "policy_kwargs": {},
@@ -41,13 +41,13 @@
         "dtype": "int64",
         "_np_random": null
     },
-    "n_envs": 16,
-    "num_timesteps": 3014656,
+    "n_envs": 30,
+    "num_timesteps": 3010560,
     "_total_timesteps": 3000000,
     "_num_timesteps_at_start": 0,
     "seed": null,
     "action_noise": null,
-    "start_time": 1673263818381712278,
+    "start_time": 1673340816052036269,
     "learning_rate": 0.0003,
     "tensorboard_log": null,
     "lr_schedule": {
@@ -56,34 +56,34 @@
     },
     "_last_obs": {
         ":type:": "<class 'numpy.ndarray'>",
-        ":serialized:": "gAWVdQIAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYAAgAAAAAAAObkIj0pmF26fL2GNuBedzGM4z051nijtQAAgD8AAIA/5pb2vQfjXD/hr7K9s6LYvqzQEb5uuYo9AAAAAAAAAAAAYO07RqhPP/zvlr3288y+yI4VvDatdb0AAAAAAAAAAJoZkLl7HIS6MpyRst6ierBAqec6qxEMMwAAgD8AAIA/TZfxPa0pLz9Aq3e+p0DOvjPjyjzwTYe+AAAAAAAAAADmjG+93eMOPhw3ET1q2p6+Oph3uwNf87wAAAAAAAAAAGbAO7z4ueE9XuQzPSNDpb4fnfa9QiThvAAAAAAAAAAAGgDCPSkkcboaFSC4IoQXs05FJrtIkDs3AAAAAAAAAAAAAKk5ZUNaPuERxDsyeru+/eKNvRrZlz0AAAAAAAAAAAAVurz6e1U+aUm1vN09rr5c06i9M6KkPAAAAAAAAAAAra0vPrhrWz+C1Re+D/rAvnEaXz7fzEu+AAAAAAAAAABNA+Y9cDiTP55MnD6DFtK+A6SJPvX4iD4AAAAAAAAAAFtKmr6kw4g/2UvIvUQ3877Kyf6+EKBsPQAAAAAAAAAAjbQ/PhIBoz8OABM/XA/XvjCUwj4gou0+AAAAAAAAAADNg4S8Ug63u7p8i7vsXpE88pkbPVBBdr0AAIA/AACAP42ksj0zWAo/FCqYvoIot77uhdm9wEVcvQAAAAAAAAAAlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjSUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYksQSwiGlIwBQ5R0lFKULg=="
+        ":serialized:": "gAWVNQQAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJbAAwAAAAAAAGBFCj6b4ik/A4D9PYOERr/s/2w+QCWIPQAAAAAAAAAAZjwNPSkof7qisQgz7RE4MRU/TTu/Qp6zAACAPwAAgD+ab1A8FJCDugDAczo1REw1tsE5OgMpjLkAAIA/AACAP5ppJjspaGm6LlEOOhBjlDWVI7g6ELqHNAAAgD8AAIA/gKUTvnxHgz72J88++E0Hv7IaX7rPi8c+AAAAAAAAAAAA+tM96bsLPu23j74hIJC+/AxYvYZIJr4AAAAAAAAAAHNvtb189aM/RX3jvguFDL9i4Au+lrufvgAAAAAAAAAAmlmWPGxd9Ltb1Ay+xEHoPGhKPz2qrr+9AACAPwAAgD+apRq8hfvcua4wwbU1SrawTTSbu4KIADUAAIA/AACAP/C0hz7UzJ8/5sYNPzHRAr9RWeQ+6ORWPgAAAAAAAAAAZkbXuymUYbrdwe47h3ZNNZHYKbsf1EQ0AACAPwAAgD9mmpe8exCKuhqaITRtsYEv1IEGu+88nLMAAIA/AACAP5rmkrzO56u842GCPiqPPr7/qPa9QnM6vgAAgD8AAAAA02QbPh9lfz4z2Ny+Wh3svnn81jywa1++AAAAAAAAAADNz9G8XCMuuv7VTTy0doc8bw3Duv6zbj0AAIA/AACAP82sPDy4Fqc/wvYLPoTSLb/VCQU8J3w5PQAAAAAAAAAAmuFIPJJlhT+zGe888gBfvyu/aDxdQO28AAAAAAAAAAAa43s9ew6JuubzLbjebyWz27fsOd5wSjcAAIA/AACAP6ZsOL5undM+K1ykPr7cH79m8i2+1uKHPgAAAAAAAAAAM5ZXPY+2dLqIa7s2CXaFMSlu3ro+G9y1AACAPwAAgD9N/ys+v0chPlhi3r4XkM2+I564PRXweb4AAAAAAAAAAAB03byPllW6J3uuOR3xS7SlTKW6GoZzswAAgD8AAIA/rUgTPod7cD9IAqo+JrAmvwISkj7MhjU+AAAAAAAAAACaPog8lCOzOwLnS761zjO+3LO1u5giaj8AAAAAAAAAAFNzMD4FSTY+MMzCvoGh076zTUs99qMqvgAAAAAAAAAAAESgPQCIsz4etCa+lHHyvudGGT1qX2u9AAAAAAAAAABmfK48+yWgvL2uJryQkfI8pAnwvW5zST0AAIA/AACAPw0YXD49Owe90gISO+8Qs7nGmWq+s4qDugAAgD8AAIA/5tO7PcUGkj8mAJg+tNJBv8lwKz6YQAs+AAAAAAAAAABmvn69gzUWvKYDRD3UcUU9W0J6PTo4Er4AAIA/AACAP5SMBW51bXB5lIwFZHR5cGWUk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGJLHksIhpSMAUOUdJRSlC4="
     },
     "_last_episode_starts": {
         ":type:": "<class 'numpy.ndarray'>",
-        ":serialized:": "gAWVgwAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACUjAVudW1weZSMBWR0eXBllJOUjAJiMZSJiIeUUpQoSwOMAXyUTk5OSv////9K/////0sAdJRiSxCFlIwBQ5R0lFKULg=="
+        ":serialized:": "gAWVkQAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYeAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAJSMBW51bXB5lIwFZHR5cGWUk5SMAmIxlImIh5RSlChLA4wBfJROTk5K/////0r/////SwB0lGJLHoWUjAFDlHSUUpQu"
     },
     "_last_original_obs": null,
     "_episode_num": 0,
     "use_sde": false,
     "sde_sample_freq": -1,
-    "_current_progress_remaining": -0.004885333333333408,
+    "_current_progress_remaining": -0.0035199999999999676,
     "ep_info_buffer": {
         ":type:": "<class 'collections.deque'>",
-        ":serialized:": "gAWVKxAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMIlYJuL6lZckCUhpRSlIwBbJRL4YwBdJRHQKUsRGViWmh1fZQoaAZoCWgPQwh/bJIfMThxQJSGlFKUaBVL3WgWR0ClLE562OQydX2UKGgGaAloD0MICYuKOF2XcUCUhpRSlGgVS91oFkdApSyVBUrCnHV9lChoBmgJaA9DCGe610k9OnBAlIaUUpRoFUvraBZHQKUs3RXOnl51fZQoaAZoCWgPQwhzR//LtcJsQJSGlFKUaBVL8WgWR0ClLNwCr92pdX2UKGgGaAloD0MIjnQGRt6TcUCUhpRSlGgVTRYBaBZHQKUtAdJ8OTd1fZQoaAZoCWgPQwhh/Z/DfLhyQJSGlFKUaBVNBwFoFkdApS0CJQ+EAnV9lChoBmgJaA9DCGsMOiH0XW9AlIaUUpRoFUvhaBZHQKUtOWom5Ud1fZQoaAZoCWgPQwjH8xlQb3RvQJSGlFKUaBVL/GgWR0ClLV/ffoA5dX2UKGgGaAloD0MI3EjZIil1cUCUhpRSlGgVS+JoFkdApS2Cjvd/KHV9lChoBmgJaA9DCGkB2lazxG9AlIaUUpRoFUv9aBZHQKUtpwtrbg11fZQoaAZoCWgPQwjRQCybOTFwQJSGlFKUaBVL2mgWR0ClLdf6O5rhdX2UKGgGaAloD0MIe6Lrwk9ocECUhpRSlGgVS+toFkdApS3cHt4RmXV9lChoBmgJaA9DCGb1DreD4XFAlIaUUpRoFU0OAWgWR0ClLe53cHnmdX2UKGgGaAloD0MI/Z/DfHnBP0CUhpRSlGgVS8BoFkdApS4/smfGuXV9lChoBmgJaA9DCO27IvhfpW9AlIaUUpRoFUv2aBZHQKUusaEzwc51fZQoaAZoCWgPQwhcrKjBtHVyQJSGlFKUaBVL3GgWR0ClLsmDDjzadX2UKGgGaAloD0MIcayL2+j+cUCUhpRSlGgVS+FoFkdApS7k63iJf3V9lChoBmgJaA9DCJrtCn0wVHJAlIaUUpRoFUvbaBZHQKUvYWdmQKd1fZQoaAZoCWgPQwhbs5WXvEByQJSGlFKUaBVL92gWR0ClL3Hn2ZiNdX2UKGgGaAloD0MIxR9FnfnlckCUhpRSlGgVS+BoFkdApS+dI065oXV9lChoBmgJaA9DCHqJsUy/dHFAlIaUUpRoFUvSaBZHQKUvqs8PnSx1fZQoaAZoCWgPQwjPo+L/Tn5xQJSGlFKUaBVL5GgWR0ClL6r74zrNdX2UKGgGaAloD0MIovFEEGfkckCUhpRSlGgVS/ZoFkdApS+7P0I1L3V9lChoBmgJaA9DCGE41zCDTnFAlIaUUpRoFUveaBZHQKUv7vE0iyJ1fZQoaAZoCWgPQwi3e7lPjtNwQJSGlFKUaBVL2mgWR0ClMChX8wYcdX2UKGgGaAloD0MIahSSzKpqckCUhpRSlGgVS+hoFkdApTArsOXmeXV9lChoBmgJaA9DCBtn0xGAr3FAlIaUUpRoFUvhaBZHQKUwiH2ys0Z1fZQoaAZoCWgPQwhmEvWCTwFyQJSGlFKUaBVNAAFoFkdApTDHkaMrE3V9lChoBmgJaA9DCETbMXWXI3FAlIaUUpRoFU0CAWgWR0ClMNE9t/FzdX2UKGgGaAloD0MI9BlQb0bQcECUhpRSlGgVS/doFkdApTEdbA1vVHV9lChoBmgJaA9DCK+YEd4e2m5AlIaUUpRoFUvkaBZHQKUxaan75211fZQoaAZoCWgPQwg2rKksCt1xQJSGlFKUaBVL9GgWR0ClMYX8n/kvdX2UKGgGaAloD0MITkUqjC1MckCUhpRSlGgVS/RoFkdApTGyJCSid3V9lChoBmgJaA9DCAUYlj+f/nFAlIaUUpRoFUvYaBZHQKUx1zbN8md1fZQoaAZoCWgPQwhqiCr82VVwQJSGlFKUaBVL3GgWR0ClOxRh2GIsdX2UKGgGaAloD0MIrRQCuUR9cUCUhpRSlGgVS9poFkdApTs0A1ejVXV9lChoBmgJaA9DCOblsPvOnXJAlIaUUpRoFUvYaBZHQKU7T7Qb+991fZQoaAZoCWgPQwhEaW/wBZRxQJSGlFKUaBVL42gWR0ClO2kWqLjxdX2UKGgGaAloD0MI9FDbhtF6cUCUhpRSlGgVS/BoFkdApTvjuMMqjXV9lChoBmgJaA9DCJsEb0hjHXBAlIaUUpRoFUvkaBZHQKU8Ao+fRNR1fZQoaAZoCWgPQwhckZigBsVuQJSGlFKUaBVL9mgWR0ClPDGOlwcYdX2UKGgGaAloD0MI4/p3faa4cUCUhpRSlGgVS91oFkdApTybodMj/3V9lChoBmgJaA9DCNEhcCRQyW5AlIaUUpRoFUv2aBZHQKU8mt7rs0J1fZQoaAZoCWgPQwiY9s391eJwQJSGlFKUaBVL/GgWR0ClPOlZ5iVjdX2UKGgGaAloD0MIqB5pcJtMcUCUhpRSlGgVS+FoFkdApTz1SOzY3HV9lChoBmgJaA9DCHycacK2LXNAlIaUUpRoFUvXaBZHQKU9IR7qptJ1fZQoaAZoCWgPQwjtuOF3081wQJSGlFKUaBVL7GgWR0ClPX1LJ0W/dX2UKGgGaAloD0MIHjaRmUvLcUCUhpRSlGgVS+FoFkdApT2KQV9F4XV9lChoBmgJaA9DCNP02QFXWnJAlIaUUpRoFUvhaBZHQKU9xQnhKlJ1fZQoaAZoCWgPQwhypZ4FIdFwQJSGlFKUaBVL6mgWR0ClPcU/W1+idX2UKGgGaAloD0MIk/5eCs9NcECUhpRSlGgVS+toFkdApT4AmkWRBHV9lChoBmgJaA9DCNrk8ElnpnFAlIaUUpRoFUvkaBZHQKU+EXKKYRd1fZQoaAZoCWgPQwiz6nO1FadwQJSGlFKUaBVL9GgWR0ClPizGxUvPdX2UKGgGaAloD0MIjPhOzDqUcUCUhpRSlGgVS/JoFkdApT6rY02tMnV9lChoBmgJaA9DCOEKKNRT/XBAlIaUUpRoFU0CAWgWR0ClPv/NqxkedX2UKGgGaAloD0MI96sA3+3tb0CUhpRSlGgVTQABaBZHQKU/MqBmPHV1fZQoaAZoCWgPQwikObLySxdxQJSGlFKUaBVL72gWR0ClP3E9ECvHdX2UKGgGaAloD0MIzxWlhOA7bkCUhpRSlGgVS/RoFkdApT+AFFDv3XV9lChoBmgJaA9DCAnh0cYRGXFAlIaUUpRoFUvkaBZHQKU/pdcB2fV1fZQoaAZoCWgPQwg2c0hqoZdyQJSGlFKUaBVL7WgWR0ClP89bX6IndX2UKGgGaAloD0MIG2K85lWucECUhpRSlGgVS+FoFkdApT/a6asp5XV9lChoBmgJaA9DCLcMOEtJvHBAlIaUUpRoFUvnaBZHQKVAT4C6pYN1fZQoaAZoCWgPQwh0CYfeIspwQJSGlFKUaBVL+mgWR0ClQIAAIY3vdX2UKGgGaAloD0MIGVWGcXfocECUhpRSlGgVS+JoFkdApUCB5LRKH3V9lChoBmgJaA9DCOaUgJiE7XFAlIaUUpRoFU0AAWgWR0ClQOBomG/OdX2UKGgGaAloD0MIx549l6nObkCUhpRSlGgVS99oFkdApUDuFxn3+XV9lChoBmgJaA9DCCLElbN3xW9AlIaUUpRoFUvzaBZHQKVA9/kvK2d1fZQoaAZoCWgPQwiV8loJ3dZxQJSGlFKUaBVL+WgWR0ClQRhG6PKddX2UKGgGaAloD0MIkUWaeIdzcECUhpRSlGgVS+RoFkdApUFtC/oJRnV9lChoBmgJaA9DCBoziXrBinJAlIaUUpRoFUvjaBZHQKVBurOqvNh1fZQoaAZoCWgPQwhHIF7Xb8hwQJSGlFKUaBVL5WgWR0ClQe4i5d4WdX2UKGgGaAloD0MI203wTZPncUCUhpRSlGgVS+RoFkdApUIg5HVf/nV9lChoBmgJaA9DCCdok8NnEnFAlIaUUpRoFUvdaBZHQKVCanivPkd1fZQoaAZoCWgPQwjW/s72aPNwQJSGlFKUaBVL9mgWR0ClQmpyhi9adX2UKGgGaAloD0MIk4rG2t+NcUCUhpRSlGgVS+1oFkdApUJz41xbS3V9lChoBmgJaA9DCMWp1sJsG3FAlIaUUpRoFUvtaBZHQKVCqIpH7P91fZQoaAZoCWgPQwjFAIkm0LZvQJSGlFKUaBVL5WgWR0ClQwVjAi3YdX2UKGgGaAloD0MIoUyjycU2b0CUhpRSlGgVS+9oFkdApUNWdy1eB3V9lChoBmgJaA9DCBH8byW7qHBAlIaUUpRoFUvyaBZHQKVDXky1uzh1fZQoaAZoCWgPQwhXPsvzIAJyQJSGlFKUaBVL42gWR0ClQ41G0/nodX2UKGgGaAloD0MI5llJKz5xcECUhpRSlGgVS+VoFkdApUOggTyrgnV9lChoBmgJaA9DCPAyw0ZZ725AlIaUUpRoFUvxaBZHQKVDzdD6WPd1fZQoaAZoCWgPQwjYLJeNTs1xQJSGlFKUaBVL8WgWR0ClQ/BWo3rEdX2UKGgGaAloD0MI3bQZp+G5cUCUhpRSlGgVS+poFkdApUQ1mDlHSXV9lChoBmgJaA9DCEGADB07pnFAlIaUUpRoFUvtaBZHQKVEi1c+qzZ1fZQoaAZoCWgPQwj0+/7Ni/pnQJSGlFKUaBVN6ANoFkdApUSV8eCCjHV9lChoBmgJaA9DCGSxTSpavXBAlIaUUpRoFUvXaBZHQKVEryWAwwl1fZQoaAZoCWgPQwi5GW7Ap8ZyQJSGlFKUaBVL1WgWR0ClRSW+fywwdX2UKGgGaAloD0MIfhzNkRVZcUCUhpRSlGgVTQABaBZHQKVFcC5Etul1fZQoaAZoCWgPQwhVMgBU8bNwQJSGlFKUaBVNAwFoFkdApUWD4WUKRnV9lChoBmgJaA9DCAOTG0XWNnNAlIaUUpRoFU0NAWgWR0ClRZmqxTsIdX2UKGgGaAloD0MIXfsCeuGTcECUhpRSlGgVS9RoFkdApUXcifQKKHV9lChoBmgJaA9DCNkngGIkGXJAlIaUUpRoFUvsaBZHQKVGMNEPUa11fZQoaAZoCWgPQwhLPQtCuVpzQJSGlFKUaBVNEQFoFkdApUZPggow23V9lChoBmgJaA9DCDrMlxdgmXBAlIaUUpRoFUvpaBZHQKVGWy/KyOd1fZQoaAZoCWgPQwg8aHbd28RwQJSGlFKUaBVL8mgWR0ClRox+BpYcdX2UKGgGaAloD0MIZjIcz+dac0CUhpRSlGgVS9loFkdApUaU78vVVnV9lChoBmgJaA9DCOxP4nMnv3NAlIaUUpRoFUvfaBZHQKVG8/8EV351fZQoaAZoCWgPQwirmEo/Yc5vQJSGlFKUaBVNAgFoFkdApUb0OG0u2HVlLg=="
+        ":serialized:": "gAWVMhAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMIg7709idVckCUhpRSlIwBbJRLyYwBdJRHQKoG2DgZTAF1fZQoaAZoCWgPQwgtCrsoenlxQJSGlFKUaBVL0WgWR0CqBxY5DJEIdX2UKGgGaAloD0MIKZXwhF41c0CUhpRSlGgVTS0BaBZHQKoHG5ZKWcB1fZQoaAZoCWgPQwglkBK79sNzQJSGlFKUaBVL/2gWR0CqB1yB06o3dX2UKGgGaAloD0MIf8LZrWVjcUCUhpRSlGgVS8NoFkdAqgeBIFvAGnV9lChoBmgJaA9DCBTsv84NQHFAlIaUUpRoFUvWaBZHQKoHpRDTjNp1fZQoaAZoCWgPQwhcrn5sEupwQJSGlFKUaBVLpmgWR0CqB9Mrd30PdX2UKGgGaAloD0MIfa62Yr+bcECUhpRSlGgVS7RoFkdAqgfqz5XU6XV9lChoBmgJaA9DCHIYzF+hlHJAlIaUUpRoFUvKaBZHQKoISTyJ9Ap1fZQoaAZoCWgPQwjk9ssna/VyQJSGlFKUaBVLl2gWR0CqCFat9x6wdX2UKGgGaAloD0MI+yDLgomncUCUhpRSlGgVS8poFkdAqgiC++M6zXV9lChoBmgJaA9DCMVU+gmnxHNAlIaUUpRoFUvGaBZHQKoIxoIv8Il1fZQoaAZoCWgPQwixGeCCbJ1xQJSGlFKUaBVLxGgWR0CqCNS+6Ae8dX2UKGgGaAloD0MIqtbCLHRRc0CUhpRSlGgVS+poFkdAqgjNHDrJKnV9lChoBmgJaA9DCEn0Mopl43JAlIaUUpRoFUvYaBZHQKoJCMPz4Dd1fZQoaAZoCWgPQwiDpiVWBq5wQJSGlFKUaBVLrGgWR0CqCStYr8R+dX2UKGgGaAloD0MIdv9YiA4lcUCUhpRSlGgVTUkBaBZHQKoJcTRplBh1fZQoaAZoCWgPQwiYM9sVehtyQJSGlFKUaBVLxGgWR0CqCcC1y/9HdX2UKGgGaAloD0MIbvqzHylCc0CUhpRSlGgVS+doFkdAqgndxffGdnV9lChoBmgJaA9DCBPyQc+mvnFAlIaUUpRoFUu8aBZHQKoJ7RMN+b51fZQoaAZoCWgPQwh/3H75JM1wQJSGlFKUaBVLuGgWR0CqCigAIY3vdX2UKGgGaAloD0MIaOvgYC8DdECUhpRSlGgVS9toFkdAqgpDJIUah3V9lChoBmgJaA9DCGJJufucaXFAlIaUUpRoFU0wAWgWR0CqCkhlDneSdX2UKGgGaAloD0MIs+xJYPOrcECUhpRSlGgVS8JoFkdAqgpO1KGtZHV9lChoBmgJaA9DCF69iozONnJAlIaUUpRoFU0MAWgWR0CqCk2vjfeldX2UKGgGaAloD0MIPZl/9A07c0CUhpRSlGgVTQ4BaBZHQKoKXtShrWR1fZQoaAZoCWgPQwjSOqqaIDxyQJSGlFKUaBVNJQFoFkdAqgsG1KGtZHV9lChoBmgJaA9DCFch5SdVZnBAlIaUUpRoFUvdaBZHQKoLKny/bj91fZQoaAZoCWgPQwjCEg8oG19zQJSGlFKUaBVLuGgWR0CqCzB/Aj6fdX2UKGgGaAloD0MIQl96+3PIc0CUhpRSlGgVS7xoFkdAqgtCr3j+73V9lChoBmgJaA9DCHo57L4j7HFAlIaUUpRoFUvaaBZHQKoLgqVhTfl1fZQoaAZoCWgPQwjVPEfk+8FzQJSGlFKUaBVL9mgWR0CqC4JQ+EAYdX2UKGgGaAloD0MICwithy9DPkCUhpRSlGgVS1toFkdAqgve2/i5u3V9lChoBmgJaA9DCIcXRKQmg3NAlIaUUpRoFUvxaBZHQKoMMJ53Tux1fZQoaAZoCWgPQwgV5GcjFytxQJSGlFKUaBVLuWgWR0CqDKRy4nWrdX2UKGgGaAloD0MIjh6/tykOckCUhpRSlGgVS+ZoFkdAqgzDh3qzJXV9lChoBmgJaA9DCEz+J3831nNAlIaUUpRoFUvhaBZHQKoM1fJmukl1fZQoaAZoCWgPQwgX9N4YQrFxQJSGlFKUaBVLqWgWR0CqDPX5vcagdX2UKGgGaAloD0MI3pBGBQ7LcECUhpRSlGgVS75oFkdAqg0bnRsuWnV9lChoBmgJaA9DCJ4oCYk03XJAlIaUUpRoFUvraBZHQKoNLkTYdyV1fZQoaAZoCWgPQwjxKQDGs41zQJSGlFKUaBVLwmgWR0CqDSwnx8UmdX2UKGgGaAloD0MIGvm84qlPc0CUhpRSlGgVS8ZoFkdAqg14dXDFZXV9lChoBmgJaA9DCAJjfQMTbnNAlIaUUpRoFUvxaBZHQKoNt2q1gIB1fZQoaAZoCWgPQwh4YtaLIapyQJSGlFKUaBVLnGgWR0CqDctZ/0/XdX2UKGgGaAloD0MIP/89eO3Lb0CUhpRSlGgVTQgBaBZHQKoOU+ajN6h1fZQoaAZoCWgPQwj2RUJbTkBxQJSGlFKUaBVL22gWR0CqDmtCJGe+dX2UKGgGaAloD0MI4gFlU65JckCUhpRSlGgVS8loFkdAqg69A5aNdnV9lChoBmgJaA9DCDNQGf/+d3FAlIaUUpRoFUvMaBZHQKoO9CFbmlt1fZQoaAZoCWgPQwglehnFclBwQJSGlFKUaBVLzWgWR0CqDwTpgTh6dX2UKGgGaAloD0MIBMjQsQPhcUCUhpRSlGgVS9FoFkdAqg8y86FM7HV9lChoBmgJaA9DCAhyUMKM7HFAlIaUUpRoFUu9aBZHQKoPevW6K+B1fZQoaAZoCWgPQwiYMQVrXAJyQJSGlFKUaBVLuWgWR0CqD/S1eBxxdX2UKGgGaAloD0MIeeV620wNc0CUhpRSlGgVS6VoFkdAqhA+zByjpXV9lChoBmgJaA9DCKDCEaTSMnFAlIaUUpRoFUvaaBZHQKoQhOIInjR1fZQoaAZoCWgPQwhfJLTlnCZxQJSGlFKUaBVNRwFoFkdAqhB/FirksHV9lChoBmgJaA9DCN3u5T75uHBAlIaUUpRoFUu+aBZHQKoQiMnZ00Z1fZQoaAZoCWgPQwjg929eHO9zQJSGlFKUaBVL5GgWR0CqELWmxdIHdX2UKGgGaAloD0MIXDl7ZzQ9ckCUhpRSlGgVTS4BaBZHQKoRJ8F6iTN1fZQoaAZoCWgPQwh4CyQofhtxQJSGlFKUaBVNPgFoFkdAqhFfezlcQnV9lChoBmgJaA9DCMRg/gpZw3FAlIaUUpRoFU04AWgWR0CqEcg2Ifr9dX2UKGgGaAloD0MIfEYiNELQcECUhpRSlGgVS7ZoFkdAqhHN+7UXpHV9lChoBmgJaA9DCEzChTxCC3JAlIaUUpRoFUvKaBZHQKoR4S0Sh8J1fZQoaAZoCWgPQwiO6nQgq5NwQJSGlFKUaBVLrmgWR0CqEfJ0wJw9dX2UKGgGaAloD0MIlKXW+w0HcUCUhpRSlGgVS71oFkdAqhH8Of/WD3V9lChoBmgJaA9DCIrNx7VhVXNAlIaUUpRoFUvnaBZHQKoSeIkZ75V1fZQoaAZoCWgPQwgtlbcj3DdzQJSGlFKUaBVN1wFoFkdAqhKFyYG+snV9lChoBmgJaA9DCHCaPjtgH3BAlIaUUpRoFUu8aBZHQKoSiwt8NQV1fZQoaAZoCWgPQwg1KQXd3nxyQJSGlFKUaBVLsWgWR0CqEwDU3GXHdX2UKGgGaAloD0MIcaq1MAv+cUCUhpRSlGgVS+9oFkdAqhMsK/mDDnV9lChoBmgJaA9DCKTjamTXSXJAlIaUUpRoFUu5aBZHQKoTh3g1m8N1fZQoaAZoCWgPQwiu8ZnsX3tyQJSGlFKUaBVNVQFoFkdAqhORkAggYHV9lChoBmgJaA9DCDhpGhRNfHNAlIaUUpRoFU0IAWgWR0CqE6M8YAKfdX2UKGgGaAloD0MIQFBu27dWckCUhpRSlGgVTSEBaBZHQKoUIvkili11fZQoaAZoCWgPQwhR3sfR3CxyQJSGlFKUaBVNbAFoFkdAqhSNIwudw3V9lChoBmgJaA9DCFPL1voiqHJAlIaUUpRoFUvaaBZHQKoUlQAMlTp1fZQoaAZoCWgPQwiVuflGNNdxQJSGlFKUaBVL2mgWR0CqFRjIJZ4fdX2UKGgGaAloD0MIhpLJqV2YckCUhpRSlGgVS+RoFkdAqhUUPvrnknV9lChoBmgJaA9DCPRwAtOpAHNAlIaUUpRoFUu4aBZHQKoVQEkjX4F1fZQoaAZoCWgPQwi0WfW5GlhxQJSGlFKUaBVLuWgWR0CqFT3VCojwdX2UKGgGaAloD0MI1NNH4M9zcECUhpRSlGgVS7xoFkdAqhYxC+lCTnV9lChoBmgJaA9DCHehuU6jp3NAlIaUUpRoFU0xAWgWR0CqFjYxk/bCdX2UKGgGaAloD0MI7s1vmOgCcECUhpRSlGgVS+ZoFkdAqhZuUOd5IHV9lChoBmgJaA9DCA7bFmV2HXNAlIaUUpRoFUvpaBZHQKoWraFEiMZ1fZQoaAZoCWgPQwjL2TujLbdyQJSGlFKUaBVL/2gWR0CqFsz7di2EdX2UKGgGaAloD0MIdFyN7AqqckCUhpRSlGgVTWABaBZHQKoW1Pa+N991fZQoaAZoCWgPQwhb0HtjyLdyQJSGlFKUaBVLmWgWR0CqFuviDM/ydX2UKGgGaAloD0MIcZNRZVitcECUhpRSlGgVS81oFkdAqhb+ukk8inV9lChoBmgJaA9DCNriGp/JE3JAlIaUUpRoFUvXaBZHQKoXRF6zE751fZQoaAZoCWgPQwgS+S6lrm9xQJSGlFKUaBVLumgWR0CqF0o91U2ldX2UKGgGaAloD0MILQWk/U+JcECUhpRSlGgVS8FoFkdAqhdkHWz4UXV9lChoBmgJaA9DCLudfeXB/nJAlIaUUpRoFU2SAmgWR0CqF3G8ujASdX2UKGgGaAloD0MI0jb+RKUJckCUhpRSlGgVS+FoFkdAqheRVyWAw3V9lChoBmgJaA9DCDYf14YKYHJAlIaUUpRoFUvIaBZHQKoXmCmMwUR1fZQoaAZoCWgPQwjqBgq8EzhxQJSGlFKUaBVNSwFoFkdAqhhUfaHsTnV9lChoBmgJaA9DCJj75CjARHNAlIaUUpRoFU0EAWgWR0CqGHzQmeDndX2UKGgGaAloD0MI0zHnGXuwc0CUhpRSlGgVS9BoFkdAqhi65NGmUHV9lChoBmgJaA9DCP9eCg8ar29AlIaUUpRoFUuvaBZHQKoY6GHHmzV1fZQoaAZoCWgPQwgh6j4AaX5yQJSGlFKUaBVL12gWR0CqGQCJGe+VdX2UKGgGaAloD0MIUFPL1rpocECUhpRSlGgVS+VoFkdAqhlNUp/gBXV9lChoBmgJaA9DCBL7BFCMFEFAlIaUUpRoFUtvaBZHQKoZhBsyi251ZS4="
     },
     "ep_success_buffer": {
         ":type:": "<class 'collections.deque'>",
         ":serialized:": "gAWVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="
     },
-    "_n_updates": 736,
+    "_n_updates": 980,
     "n_steps": 1024,
-    "gamma": 0.999,
+    "gamma": 0.9999,
     "gae_lambda": 0.98,
     "ent_coef": 0.01,
     "vf_coef": 0.5,
     "max_grad_norm": 0.5,
     "batch_size": 64,
-    "n_epochs": 4,
+    "n_epochs": 10,
     "clip_range": {
         ":type:": "<class 'function'>",
         ":serialized:": "gAWVwwIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZSMBGZ1bmOUS4BDAgABlIwDdmFslIWUKXSUUpR9lCiMC19fcGFja2FnZV9flIwYc3RhYmxlX2Jhc2VsaW5lczMuY29tbW9ulIwIX19uYW1lX1+UjB5zdGFibGVfYmFzZWxpbmVzMy5jb21tb24udXRpbHOUjAhfX2ZpbGVfX5SMSC91c3IvbG9jYWwvbGliL3B5dGhvbjMuOC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlIWUdJRSlIwcY2xvdWRwaWNrbGUuY2xvdWRwaWNrbGVfZmFzdJSMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFmgNjAxfX3F1YWxuYW1lX1+UjBljb25zdGFudF9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UjA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoF4wHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/yZmZmZmZmoWUUpSFlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="

ppo-LunarLander-v2/policy.optimizer.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:df431381982aa2b81b2e3a0e633be78020353c9c494e27c5a0c1facff0c33816
-size 88057
+oid sha256:684e318929a5103e4cef804633dbe27417fad785c2437d6fea50d6f7f1af662e
+size 87545

ppo-LunarLander-v2/policy.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d011c905f5fe758b1c5e4abf9ae9f87f8a0ace15afc2424285b44ddd94b647b5
-size 43201
+oid sha256:9e873cfd7cfcaadcce4bc86d0d35fdce12e2ccecaba5d91c497c3ab18e532d2d
+size 43073

ppo-LunarLander-v2/system_info.txt

@@ -2,6 +2,6 @@ OS: Linux-5.10.147+-x86_64-with-glibc2.27 #1 SMP Sat Dec 10 16:00:40 UTC 2022
 Python: 3.8.16
 Stable-Baselines3: 1.6.2
 PyTorch: 1.13.0+cu116
-GPU Enabled: True
+GPU Enabled: False
 Numpy: 1.21.6
 Gym: 0.21.0

results.json

@@ -1 +1 @@
-{"mean_reward": 278.1533869959842, "std_reward": 19.11126033184433, "is_deterministic": true, "n_eval_episodes": 10, "eval_datetime": "2023-01-09T12:47:20.726808"}
+{"mean_reward": 285.1019196116637, "std_reward": 13.631565452346784, "is_deterministic": true, "n_eval_episodes": 10, "eval_datetime": "2023-01-10T09:54:26.667412"}