|
--- |
|
library_name: skrl |
|
tags: |
|
- deep-reinforcement-learning |
|
- reinforcement-learning |
|
- skrl |
|
model-index: |
|
- name: PPO |
|
results: |
|
- metrics: |
|
- type: mean_reward |
|
value: -8.89 +/- 10.3 |
|
name: Total reward (mean) |
|
task: |
|
type: reinforcement-learning |
|
name: reinforcement-learning |
|
dataset: |
|
name: IsaacGymEnvs-FactoryTaskNutBoltScrew |
|
type: IsaacGymEnvs-FactoryTaskNutBoltScrew |
|
--- |
|
|
|
<!-- --- |
|
torch: -21.51 +/- 14.99 |
|
jax: -35.77 +/- 0.39 |
|
numpy: -8.89 +/- 10.3 |
|
--- --> |
|
|
|
# IsaacGymEnvs-FactoryTaskNutBoltScrew-PPO |
|
|
|
Trained agent for [NVIDIA Isaac Gym Preview](https://github.com/NVIDIA-Omniverse/IsaacGymEnvs) environments. |
|
|
|
- **Task:** FactoryTaskNutBoltScrew |
|
- **Agent:** [PPO](https://skrl.readthedocs.io/en/latest/api/agents/ppo.html) |
|
|
|
# Usage (with skrl) |
|
|
|
Note: Visit the skrl [Examples](https://skrl.readthedocs.io/en/latest/intro/examples.html) section to access the scripts. |
|
|
|
* PyTorch |
|
|
|
```python |
|
from skrl.utils.huggingface import download_model_from_huggingface |
|
|
|
# assuming that there is an agent named `agent` |
|
path = download_model_from_huggingface("skrl/IsaacGymEnvs-FactoryTaskNutBoltScrew-PPO", filename="agent.pt") |
|
agent.load(path) |
|
``` |
|
|
|
* JAX |
|
|
|
```python |
|
from skrl.utils.huggingface import download_model_from_huggingface |
|
|
|
# assuming that there is an agent named `agent` |
|
path = download_model_from_huggingface("skrl/IsaacGymEnvs-FactoryTaskNutBoltScrew-PPO", filename="agent.pickle") |
|
agent.load(path) |
|
``` |
|
|
|
# Hyperparameters |
|
|
|
Note: Undefined parameters keep their values by default. |
|
|
|
```python |
|
# https://skrl.readthedocs.io/en/latest/api/agents/ppo.html#configuration-and-hyperparameters |
|
cfg = PPO_DEFAULT_CONFIG.copy() |
|
cfg["rollouts"] = 128 # memory_size |
|
cfg["learning_epochs"] = 8 |
|
cfg["mini_batches"] = 32 # 128 * 128 / 512 |
|
cfg["discount_factor"] = 0.99 |
|
cfg["lambda"] = 0.95 |
|
cfg["learning_rate"] = 1e-4 |
|
cfg["random_timesteps"] = 0 |
|
cfg["learning_starts"] = 0 |
|
cfg["grad_norm_clip"] = 0 |
|
cfg["ratio_clip"] = 0.2 |
|
cfg["value_clip"] = 0.2 |
|
cfg["clip_predicted_values"] = True |
|
cfg["entropy_loss_scale"] = 0.0 |
|
cfg["value_loss_scale"] = 1.0 |
|
cfg["kl_threshold"] = 0.016 |
|
cfg["rewards_shaper"] = None |
|
cfg["state_preprocessor"] = RunningStandardScaler |
|
cfg["state_preprocessor_kwargs"] = {"size": env.observation_space, "device": device} |
|
cfg["value_preprocessor"] = RunningStandardScaler |
|
cfg["value_preprocessor_kwargs"] = {"size": 1, "device": device} |
|
``` |
|
|
