File size: 2,280 Bytes
a471d9b 0d8b86a a471d9b 0d8b86a a471d9b 6a46663 0d8b86a d57e89d 0d8b86a d57e89d 0d8b86a d57e89d 0d8b86a d57e89d 0d8b86a d57e89d 0d8b86a d57e89d a471d9b d57e89d a471d9b d57e89d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
library_name: stable-baselines3
tags:
- PandaReachDense-v3
- deep-reinforcement-learning
- reinforcement-learning
- stable-baselines3
model-index:
- name: A2C
results:
- task:
type: reinforcement-learning
name: reinforcement-learning
dataset:
name: PandaReachDense-v3
type: PandaReachDense-v3
metrics:
- type: mean_reward
value: -0.24 +/- 0.09
name: mean_reward
verified: false
---
# **A2C** Agent playing **PandaReachDense-v3**
## General information about the project:
This is a trained model of a **A2C** agent playing **PandaReachDense-v3**
using the [stable-baselines3 library](https://github.com/DLR-RM/stable-baselines3). It controls a robotic arm to pick up balls.
### What I did to improve the model:
Manually tuned hyperparameters by adding "learning_rate=0.0007, n_steps=5, gamma=0.99, gae_lambda=0.95" to the A2C model.
```
model = A2C(policy = "MultiInputPolicy",
env = env,
learning_rate=0.0007,
n_steps=5,
gamma=0.99,
gae_lambda=0.95,
verbose=1)
```
## Links to relevant resources such as tutorials.
Reinforcement Learning Tips and Tricks: https://stable-baselines3.readthedocs.io/en/master/guide/rl_tips.html
A Github Training Framework : https://github.com/DLR-RM/rl-baselines3-zoo
Poe (GPT-4): Showed me how to use Optuna to do automated hyperparameter optimization, but I was still understanding how it worked and couldn't get it to run properly.
```
import optuna
from stable_baselines3 import A2C
from stable_baselines3.common.env_util import make_vec_env
def optimize_agent(trial):
learning_rate = trial.suggest_loguniform('learning_rate', 1e-5, 1)
gamma = trial.suggest_uniform('gamma', 0.8, 0.9999)
gae_lambda = trial.suggest_uniform('gae_lambda', 0.8, 0.99)
n_steps = trial.suggest_int('n_steps', 5, 20)
model = A2C('MlpPolicy', env, verbose=0, learning_rate=learning_rate, gamma=gamma, gae_lambda=gae_lambda, n_steps=n_steps)
model.learn(total_timesteps=5000)
rewards = sum(model.rollout_buffer.rewards)
return rewards
study = optuna.create_study(direction='maximize')
study.optimize(optimize_agent, n_trials=100)
print('Best hyperparameters:', study.best_params)
``` |