Repository: RoboVerseOrg/RoboVerse
Pull request #164: [fix] isaacgym humanoid training pipeline 
Author: login: WeisonWEileen
Created: 2025-05-05T19:14:16Z | Merged: 2025-05-11T19:33:00Z

1. fix velocity and angular velocity reset bug in function `set_states` 
2. fix compatibility between gripper and none gripper joint loading in function `_load_robot_assets()` 
3. fix isaacgym humanoid training pipeline, tested on `humanoid_bench:Stand` task and update docs
4. add `sim_params` and `control_type` config in `scenario` which could be overided by `task`
5. add none-actionable object joint action placeholder in effort control mode 


Diff:
diff --git a/docs/source/_static/standard_output/humanoid_bench/humanoid_bench_rl_isaacgym.png b/docs/source/_static/standard_output/humanoid_bench/humanoid_bench_rl_isaacgym.png
new file mode 100644
index 000000000..c115e8469
Binary files /dev/null and b/docs/source/_static/standard_output/humanoid_bench/humanoid_bench_rl_isaacgym.png differ
diff --git a/docs/source/_static/standard_output/humanoid_bench/humanoid_bench_rl_isaacgym_curve.png b/docs/source/_static/standard_output/humanoid_bench/humanoid_bench_rl_isaacgym_curve.png
new file mode 100644
index 000000000..68363de3f
Binary files /dev/null and b/docs/source/_static/standard_output/humanoid_bench/humanoid_bench_rl_isaacgym_curve.png differ
diff --git a/docs/source/roboverse_learn/reinforcement_learning/humanoidbench_rl.md b/docs/source/roboverse_learn/reinforcement_learning/humanoidbench_rl.md
index d161b33dd..24e11a588 100644
--- a/docs/source/roboverse_learn/reinforcement_learning/humanoidbench_rl.md
+++ b/docs/source/roboverse_learn/reinforcement_learning/humanoidbench_rl.md
@@ -39,6 +39,19 @@ wandb login
     ```bash
     python roboverse_learn/humanoidbench_rl/train_sb3.py isaacgym
     ```
+    After training around 4~6k steps, you can see result like this
+<div style="display: flex; flex-wrap: wrap; justify-content: space-between; gap: 10px;">
+    <div style="display: flex; justify-content: space-between; width: 100%; margin-bottom: 20px;">
+        <div style="width: 48%; text-align: center;">
+            <img src="https://roboverse.wiki/_static/standard_output/humanoid_bench/humanoid_bench_rl_isaacgym.png" alt="IsaacGym Training" style="width: 48%;">
+            <!-- <p style="margin-top: 5px;">Isaac Gym</p> -->
+        </div>
+        <div style="width: 48%; text-align: center;">
+            <img src="https://roboverse.wiki/_static/standard_output/humanoid_bench/humanoid_bench_rl_isaacgym_curve.png" alt="IsaacGym Training Curve" style="width: 48%;">
+            <!-- <p style="margin-top: 5px;">Isaac Lab</p> -->
+        </div>
+    </div>
+</div>
 
 - IsaacLab:
 
diff --git a/metasim/cfg/control.py b/metasim/cfg/control.py
new file mode 100644
index 000000000..cb10b1056
--- /dev/null
+++ b/metasim/cfg/control.py
@@ -0,0 +1,17 @@
+"""Configuration classes for control."""
+
+from __future__ import annotations
+
+from metasim.utils import configclass
+
+
+@configclass
+class ControlCfg:
+    """high level control parameters cfg.
+
+    This class defines the higher level for the control configuration.
+    """
+
+    action_scale: float = 1.0  # to scale the actions
+    torque_limit_scale: float = 1.0  # scale it down can ensure safety
+    action_offset: bool = False  # set true if target position = action * action_scale + default position
diff --git a/metasim/cfg/robots/base_robot_cfg.py b/metasim/cfg/robots/base_robot_cfg.py
index 9b540c8b2..f4f0c78a4 100644
--- a/metasim/cfg/robots/base_robot_cfg.py
+++ b/metasim/cfg/robots/base_robot_cfg.py
@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 from dataclasses import MISSING
+from typing import Literal
 
 from metasim.cfg.objects import ArticulationObjCfg
 from metasim.utils import configclass
@@ -35,6 +36,10 @@ class BaseRobotCfg(ArticulationObjCfg):
     default_joint_positions: dict[str, float] = {}
     default_position: tuple[float, float, float] = (0.0, 0.0, 0.0)
     default_orientation: tuple[float, float, float, float] = (1.0, 0.0, 0.0, 0.0)  # w, x, y, z
+    control_type: dict[
+        str, Literal["position", "effort"]
+    ] = {}  # TODO support more controltype, for example, velocity control. Note that effort means use manual pd position controller to get torque and set torque using isaacgym API.
+
     """
     Joint limits in the format of `{joint_name: (lower_limit, upper_limit)}`.
     Note that different simulators may have different order of joints, so you should not use the order in this dict!
diff --git a/metasim/cfg/robots/franka_cfg.py b/metasim/cfg/robots/franka_cfg.py
index 6c3cc7fbe..567b17d42 100644
--- a/metasim/cfg/robots/franka_cfg.py
+++ b/metasim/cfg/robots/franka_cfg.py
@@ -1,5 +1,7 @@
 from __future__ import annotations
 
+from typing import Literal
+
 from metasim.utils import configclass
 
 from .base_robot_cfg import BaseActuatorCfg, BaseRobotCfg
@@ -57,6 +59,17 @@ class FrankaCfg(BaseRobotCfg):
         "panda_finger_joint1": 0.04,
         "panda_finger_joint2": 0.04,
     }
+    control_type: dict[str, Literal["position", "effort"]] = {
+        "panda_joint1": "position",
+        "panda_joint2": "position",
+        "panda_joint3": "position",
+        "panda_joint4": "position",
+        "panda_joint5": "position",
+        "panda_joint6": "position",
+        "panda_joint7": "position",
+        "panda_finger_joint1": "position",
+        "panda_finger_joint2": "position",
+    }
 
     # TODO: Make it more elegant
     gripper_release_q = [0.04, 0.04]
diff --git a/metasim/cfg/robots/h1_cfg.py b/metasim/cfg/robots/h1_cfg.py
index 7f910d51e..a57d5c911 100644
--- a/metasim/cfg/robots/h1_cfg.py
+++ b/metasim/cfg/robots/h1_cfg.py
@@ -1,5 +1,7 @@
 from __future__ import annotations
 
+from typing import Literal
+
 from metasim.utils import configclass
 
 from .base_robot_cfg import BaseActuatorCfg, BaseRobotCfg
@@ -19,25 +21,25 @@ class H1Cfg(BaseRobotCfg):
     collapse_fixed_joints: bool = True
 
     actuators: dict[str, BaseActuatorCfg] = {
-        "left_hip_yaw": BaseActuatorCfg(),
-        "left_hip_roll": BaseActuatorCfg(),
-        "left_hip_pitch": BaseActuatorCfg(),
-        "left_knee": BaseActuatorCfg(),
-        "left_ankle": BaseActuatorCfg(),
-        "right_hip_yaw": BaseActuatorCfg(),
-        "right_hip_roll": BaseActuatorCfg(),
-        "right_hip_pitch": BaseActuatorCfg(),
-        "right_knee": BaseActuatorCfg(),
-        "right_ankle": BaseActuatorCfg(),
-        "torso": BaseActuatorCfg(),
-        "left_shoulder_pitch": BaseActuatorCfg(),
-        "left_shoulder_roll": BaseActuatorCfg(),
-        "left_shoulder_yaw": BaseActuatorCfg(),
-        "left_elbow": BaseActuatorCfg(),
-        "right_shoulder_pitch": BaseActuatorCfg(),
-        "right_shoulder_roll": BaseActuatorCfg(),
-        "right_shoulder_yaw": BaseActuatorCfg(),
-        "right_elbow": BaseActuatorCfg(),
+        "left_hip_yaw": BaseActuatorCfg(stiffness=200, damping=5),
+        "left_hip_roll": BaseActuatorCfg(stiffness=200, damping=5),
+        "left_hip_pitch": BaseActuatorCfg(stiffness=200, damping=5),
+        "left_knee": BaseActuatorCfg(stiffness=300, damping=6),
+        "left_ankle": BaseActuatorCfg(stiffness=40, damping=2),
+        "right_hip_yaw": BaseActuatorCfg(stiffness=200, damping=5),
+        "right_hip_roll": BaseActuatorCfg(stiffness=200, damping=5),
+        "right_hip_pitch": BaseActuatorCfg(stiffness=200, damping=5),
+        "right_knee": BaseActuatorCfg(stiffness=300, damping=6),
+        "right_ankle": BaseActuatorCfg(stiffness=40, damping=2),
+        "torso": BaseActuatorCfg(stiffness=300, damping=6),
+        "left_shoulder_pitch": BaseActuatorCfg(stiffness=100, damping=2),
+        "left_shoulder_roll": BaseActuatorCfg(stiffness=100, damping=2),
+        "left_shoulder_yaw": BaseActuatorCfg(stiffness=100, damping=2),
+        "left_elbow": BaseActuatorCfg(stiffness=100, damping=2),
+        "right_shoulder_pitch": BaseActuatorCfg(stiffness=100, damping=2),
+        "right_shoulder_roll": BaseActuatorCfg(stiffness=100, damping=2),
+        "right_shoulder_yaw": BaseActuatorCfg(stiffness=100, damping=2),
+        "right_elbow": BaseActuatorCfg(stiffness=100, damping=2),
     }
     joint_limits: dict[str, tuple[float, float]] = {
         "left_hip_yaw": (-0.43, 0.43),
@@ -60,3 +62,47 @@ class H1Cfg(BaseRobotCfg):
         "right_shoulder_yaw": (-4.45, 1.3),
         "right_elbow": (-1.25, 2.61),
     }
+
+    default_joint_positions: dict[str, float] = {  # = target angles [rad] when action = 0.0
+        "left_hip_yaw": 0.0,
+        "left_hip_roll": 0.0,
+        "left_hip_pitch": -0.4,
+        "left_knee": 0.8,
+        "left_ankle": -0.4,
+        "right_hip_yaw": 0.0,
+        "right_hip_roll": 0.0,
+        "right_hip_pitch": -0.4,
+        "right_knee": 0.8,
+        "right_ankle": -0.4,
+        "torso": 0.0,
+        "left_shoulder_pitch": 0.0,
+        "left_shoulder_roll": 0,
+        "left_shoulder_yaw": 0.0,
+        "left_elbow": 0.0,
+        "right_shoulder_pitch": 0.0,
+        "right_shoulder_roll": 0.0,
+        "right_shoulder_yaw": 0.0,
+        "right_elbow": 0.0,
+    }
+
+    control_type: dict[str, Literal["pos", "effort"]] = {
+        "left_hip_yaw": "effort",
+        "left_hip_roll": "effort",
+        "left_hip_pitch": "effort",
+        "left_knee": "effort",
+        "left_ankle": "effort",
+        "right_hip_yaw": "effort",
+        "right_hip_roll": "effort",
+        "right_hip_pitch": "effort",
+        "right_knee": "effort",
+        "right_ankle": "effort",
+        "torso": "effort",
+        "left_shoulder_pitch": "effort",
+        "left_shoulder_roll": "effort",
+        "left_shoulder_yaw": "effort",
+        "left_elbow": "effort",
+        "right_shoulder_pitch": "effort",
+        "right_shoulder_roll": "effort",
+        "right_shoulder_yaw": "effort",
+        "right_elbow": "effort",
+    }
diff --git a/metasim/cfg/scenario.py b/metasim/cfg/scenario.py
index d83dca1b2..fc7dea498 100644
--- a/metasim/cfg/scenario.py
+++ b/metasim/cfg/scenario.py
@@ -12,6 +12,7 @@
 from metasim.utils.setup_util import get_robot, get_scene, get_task
 
 from .checkers import BaseChecker, EmptyChecker
+from .control import ControlCfg
 from .lights import BaseLightCfg, CylinderLightCfg, DistantLightCfg
 from .objects import BaseObjCfg
 from .randomization import RandomizationCfg
@@ -19,6 +20,7 @@
 from .robots.base_robot_cfg import BaseRobotCfg
 from .scenes.base_scene_cfg import SceneCfg
 from .sensors import BaseCameraCfg, BaseSensorCfg, PinholeCameraCfg
+from .simulator_params import SimParamCfg
 from .tasks.base_task_cfg import BaseTaskCfg
 
 
@@ -38,6 +40,8 @@ class ScenarioCfg:
     checker: BaseChecker = EmptyChecker()
     render: RenderCfg = RenderCfg()
     random: RandomizationCfg = RandomizationCfg()
+    sim_params: SimParamCfg = SimParamCfg()
+    control: ControlCfg = ControlCfg()
 
     ## Handlers
     sim: Literal["isaaclab", "isaacgym", "pyrep", "pybullet", "sapien", "mujoco"] = "isaaclab"
@@ -76,5 +80,9 @@ def __post_init__(self):
         if isinstance(self.scene, str):
             self.scene = get_scene(self.scene)
 
-        ### Check and download all the paths
+        ### Simulator parameters overvide by task
+        self.sim_params = self.task.sim_params if self.task is not None else self.sim_params
+        ### Control parameters  overvide by task
+        self.control = self.task.control if self.task is not None else self.control
+
         FileDownloader(self).do_it()
diff --git a/metasim/cfg/simulator_params.py b/metasim/cfg/simulator_params.py
new file mode 100644
index 000000000..4a93574ba
--- /dev/null
+++ b/metasim/cfg/simulator_params.py
@@ -0,0 +1,33 @@
+"""Configuration classes for simulator parameters."""
+
+from __future__ import annotations
+
+from metasim.utils import configclass
+
+
+@configclass
+class SimParamCfg:
+    """Simulation parameters cfg.
+
+    This class defines the parameters for the simulator.
+    It is important to ensure that each task is configured with appropriate simulation
+    parameters to avoid divergence or unexpected results.
+
+    Reference for IsaacGym: https://docs.robotsfan.com/isaacgym/api/python/struct_py.html#isaacgym.gymapi.PhysXParams
+    """
+
+    # TODO: currently only support isaacgym; add compatibility cfgs for other simulators.
+    timestep: float = 1.0 / 60.0
+    substeps: int = 2
+    contact_offset: float = 0.001
+    num_position_iterations: int = 8
+    num_velocity_iterations: int = 1
+    bounce_threshold_velocity: float = 0.2
+    friction_offset_threshold: float = 0.001
+    friction_correlation_distance: float = 0.0005
+    replace_cylinder_with_capsule: bool = False
+    solver_type: int = 1
+    rest_offset: float = 0.0
+    num_threads: int = 0
+    use_gpu_pipeline: bool = True
+    use_gpu: bool = True
diff --git a/metasim/cfg/tasks/base_task_cfg.py b/metasim/cfg/tasks/base_task_cfg.py
index 04625509a..5cd22cd8c 100644
--- a/metasim/cfg/tasks/base_task_cfg.py
+++ b/metasim/cfg/tasks/base_task_cfg.py
@@ -8,7 +8,9 @@
 import torch
 
 from metasim.cfg.checkers import BaseChecker
+from metasim.cfg.control import ControlCfg
 from metasim.cfg.objects import BaseObjCfg
+from metasim.cfg.simulator_params import SimParamCfg
 from metasim.constants import BenchmarkType, TaskType
 from metasim.types import EnvState
 from metasim.utils import configclass
@@ -29,6 +31,7 @@ class BaseTaskCfg:
         can_tabletop: Whether the task can be tabletop.
         reward_functions: The list of reward functions.
         reward_weights: The list of reward weights.
+        sim_params: The simulation params.
     """
 
     decimation: int = 3
@@ -41,6 +44,8 @@ class BaseTaskCfg:
     can_tabletop: bool = False
     reward_functions: list[callable[[list[EnvState], str | None], torch.FloatTensor]] = MISSING
     reward_weights: list[float] = MISSING
+    sim_params: SimParamCfg = SimParamCfg()
+    control: ControlCfg = ControlCfg()
 
 
 @configclass
diff --git a/metasim/cfg/tasks/humanoidbench/base_cfg.py b/metasim/cfg/tasks/humanoidbench/base_cfg.py
index 6d9540b83..01a1dd7c3 100644
--- a/metasim/cfg/tasks/humanoidbench/base_cfg.py
+++ b/metasim/cfg/tasks/humanoidbench/base_cfg.py
@@ -9,7 +9,8 @@
 from loguru import logger as log
 from rich.logging import RichHandler
 
-from metasim.cfg.tasks.base_task_cfg import BaseRLTaskCfg
+from metasim.cfg.control import ControlCfg
+from metasim.cfg.tasks.base_task_cfg import BaseRLTaskCfg, SimParamCfg
 from metasim.constants import BenchmarkType, TaskType
 from metasim.types import EnvState
 from metasim.utils import configclass, humanoid_reward_util
@@ -47,6 +48,15 @@ class HumanoidTaskCfg(BaseRLTaskCfg):
     episode_length = 800  # TODO: may change
     objects = []
     reward_weights = [1.0]
+    sim_params = SimParamCfg(
+        timestep=0.001,
+        contact_offset=0.01,
+        num_position_iterations=8,
+        num_velocity_iterations=0,
+        bounce_threshold_velocity=0.5,
+        replace_cylinder_with_capsule=True,
+    )
+    control = ControlCfg(action_scale=0.5, action_offset=True, torque_limit_scale=0.85)
 
     @staticmethod
     def humanoid_obs_flatten_func(envstates: list[EnvState]) -> torch.Tensor:
diff --git a/metasim/sim/isaacgym/isaacgym.py b/metasim/sim/isaacgym/isaacgym.py
index 453bde4aa..7f3ed813d 100644
--- a/metasim/sim/isaacgym/isaacgym.py
+++ b/metasim/sim/isaacgym/isaacgym.py
@@ -53,9 +53,28 @@ def __init__(self, scenario: ScenarioCfg):
         # environment related tensor indices
         self._env_rigid_body_global_indices: list = []  # 2 dim list: list in list, each list contains global indices of each env
 
-        self._root_states: torch.Tensor | None = None  # will update after refresh
-        self._dof_states: torch.Tensor | None = None  # will update after refresh
-        self._rigid_body_states: torch.Tensor | None = None  # will update after refresh
+        # will update after refresh
+        self._root_states: torch.Tensor | None = None
+        self._dof_states: torch.Tensor | None = None
+        self._rigid_body_states: torch.Tensor | None = None
+        self._robot_dof_state: torch.Tensor | None = None
+
+        # control related
+        self._robot_num_dof: int  # number of robot dof
+        self._obj_num_dof: int = 0  # number of object dof
+        self._actions: torch.Tensor | None = None
+        self._action_scale: torch.Tensor | None = (
+            None  # for configuration: desire_pos = action_scale * action + default_pos
+        )
+        self._default_dof_pos: torch.Tensor | None = (
+            None  # for the configuration: desire_pos = action_scale * action + default_pos
+        )
+        self._action_offset: bool = False  # for configuration: desire_pos = action_scale * action + default_pos
+        self._p_gains: torch.Tensor | None = None  # parameter for PD controller in for pd effort control
+        self._d_gains: torch.Tensor | None = None
+        self._torque_limits: torch.Tensor | None = None
+        self._pos_ctrl_dof_dix = []  # joint index in dof state, built-in position control mode
+        self._manual_pd_on: bool = False  # turn on maunual pd controller if effort joint exist
 
     def launch(self) -> None:
         ## IsaacGym Initialization
@@ -68,26 +87,29 @@ def launch(self) -> None:
         self._root_states = gymtorch.wrap_tensor(self.gym.acquire_actor_root_state_tensor(self.sim))
         self._dof_states = gymtorch.wrap_tensor(self.gym.acquire_dof_state_tensor(self.sim))
         self._rigid_body_states = gymtorch.wrap_tensor(self.gym.acquire_rigid_body_state_tensor(self.sim))
+        self._robot_dof_state = self._dof_states.view(self._num_envs, -1, 2)[:, self._obj_num_dof :]
 
     def _init_gym(self) -> None:
         physics_engine = gymapi.SIM_PHYSX
         self.gym = gymapi.acquire_gym()
         # configure sim
+        # TODO move more params into sim_params cfg
         sim_params = gymapi.SimParams()
         sim_params.up_axis = gymapi.UP_AXIS_Z
         sim_params.gravity = gymapi.Vec3(0.0, 0.0, -9.8)
-        sim_params.dt = 1.0 / 60.0
-        sim_params.substeps = 2
-        sim_params.use_gpu_pipeline = True
-        sim_params.physx.solver_type = 1
-        sim_params.physx.num_position_iterations = 8
-        sim_params.physx.num_velocity_iterations = 1
-        sim_params.physx.rest_offset = 0.0
-        sim_params.physx.contact_offset = 0.001
-        sim_params.physx.friction_offset_threshold = 0.001
-        sim_params.physx.friction_correlation_distance = 0.0005
-        sim_params.physx.num_threads = 0
-        sim_params.physx.use_gpu = True
+        sim_params.dt = self.scenario.sim_params.timestep
+        sim_params.substeps = self.scenario.sim_params.substeps
+        sim_params.use_gpu_pipeline = self.scenario.sim_params.use_gpu_pipeline
+        sim_params.physx.solver_type = self.scenario.sim_params.solver_type
+        sim_params.physx.num_position_iterations = self.scenario.sim_params.num_position_iterations
+        sim_params.physx.num_velocity_iterations = self.scenario.sim_params.num_velocity_iterations
+        sim_params.physx.rest_offset = self.scenario.sim_params.rest_offset
+        sim_params.physx.contact_offset = self.scenario.sim_params.contact_offset
+        sim_params.physx.friction_offset_threshold = self.scenario.sim_params.friction_offset_threshold
+        sim_params.physx.friction_correlation_distance = self.scenario.sim_params.friction_correlation_distance
+        sim_params.physx.num_threads = self.scenario.sim_params.num_threads
+        sim_params.physx.use_gpu = self.scenario.sim_params.use_gpu
+        sim_params.physx.bounce_threshold_velocity = self.scenario.sim_params.bounce_threshold_velocity
 
         compute_device_id = 0
         graphics_device_id = 0
@@ -197,6 +219,7 @@ def _load_object_asset(self, object: BaseObjCfg) -> None:
 
         asset_link_dict = self.gym.get_asset_rigid_body_dict(asset)
         self._asset_dict_dict[object.name] = asset_link_dict
+        self._obj_num_dof += self.gym.get_asset_dof_count(asset)
         return asset
 
     def _load_robot_assets(self) -> None:
@@ -208,36 +231,93 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_NONE
+        # Defaults are set to free movement and will be updated based on the configuration in actuator_cfg below.
+        asset_options.replace_cylinder_with_capsule = self.scenario.sim_params.replace_cylinder_with_capsule
         robot_asset = self.gym.load_asset(self.sim, asset_root, robot_asset_file, asset_options)
         # configure robot dofs
+        robot_num_dofs = self.gym.get_asset_dof_count(robot_asset)
+        self._robot_num_dof = robot_num_dofs
+
+        self._action_scale = torch.tensor(self.scenario.control.action_scale, device=self.device)
+        self._action_offset = self.scenario.control.action_offset
+
+        self._p_gains = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+        self._d_gains = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+        self._torque_limits = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+
         robot_dof_props = self.gym.get_asset_dof_properties(robot_asset)
+
         robot_lower_limits = robot_dof_props["lower"]
         robot_upper_limits = robot_dof_props["upper"]
-        robot_ranges = robot_upper_limits - robot_lower_limits
         robot_mids = 0.3 * (robot_upper_limits + robot_lower_limits)
-        robot_dof_props["driveMode"][:7].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][:7].fill(400.0)
-        robot_dof_props["damping"][:7].fill(40.0)
+        num_actions = 0
+        default_dof_pos = []
+        self._manual_pd_on = any(mode == "effort" for mode in self._robot.control_type.values())
+
+        dof_names = self.gym.get_asset_dof_names(robot_asset)
+        for i, dof_name in enumerate(dof_names):
+            # get config
+            i_actuator_cfg = self._robot.actuators[dof_name]
+            i_stiffness = i_actuator_cfg.stiffness
+            i_damping = i_actuator_cfg.damping
+            i_control_mode = (
+                self._robot.control_type[dof_name] if self._robot.control_type[dof_name] is not None else "position"
+            )
 
-        # grippers
-        robot_dof_props["driveMode"][7:].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][7:].fill(800.0)
-        robot_dof_props["damping"][7:].fill(40.0)
+            # task default position from cfg if exist, otherwise use 0.3*(uppper + lower) as default
+            if not i_actuator_cfg.is_ee:
+                default_dof_pos_i = (
+                    self._robot.default_joint_positions[dof_name]
+                    if self._robot.default_joint_positions[dof_name] is not None
+                    else robot_mids[i]
+                )
+                default_dof_pos.append(default_dof_pos_i)
+            # for end effector, always use open as default position
+            else:
+                default_dof_pos.append(robot_upper_limits[i])
+
+            # pd control effort mode
+            if i_control_mode == "effort":
+                self._p_gains[:, i] = i_stiffness
+                self._d_gains[:, i] = i_damping
+                torque_limit = (
+                    i_actuator_cfg.torque_limit
+                    if i_actuator_cfg.torque_limit is not None
+                    else torch.tensor(robot_dof_props["effort"][i], dtype=torch.float, device=self.device)
+                )
+                self._torque_limits[:, i] = self.scenario.control.torque_limit_scale * torque_limit
+                robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_EFFORT
+                robot_dof_props["stiffness"][i] = 0.0
+                robot_dof_props["damping"][i] = 0.0
+
+            # built-in position mode
+            elif i_control_mode == "position":
+                robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_POS
+                robot_dof_props["stiffness"][i] = i_stiffness
+                robot_dof_props["damping"][i] = i_damping
+                self._pos_ctrl_dof_dix.append(i + self._obj_num_dof)
+            else:
+                log.error(f"Unknown actuator control mode: {i_control_mode}, only support effort and position")
+                raise ValueError
 
-        robot_num_dofs = self.gym.get_asset_dof_count(robot_asset)
-        default_dof_pos = np.zeros(robot_num_dofs, dtype=np.float32)
-        default_dof_pos[:7] = robot_mids[:7]
-        # grippers open
-        default_dof_pos[7:] = robot_upper_limits[7:]
+            if i_actuator_cfg.actionable:
+                num_actions += 1
 
-        default_dof_state = np.zeros(robot_num_dofs, gymapi.DofState.dtype)
-        default_dof_state["pos"] = default_dof_pos
+        self._default_dof_pos = torch.tensor(default_dof_pos, device=self.device).unsqueeze(0)
+        self.actions = torch.zeros([self._num_envs, num_actions], device=self.device)
 
         # # get link index of panda hand, which we will use as end effector
         self._robot_link_dict = self.gym.get_asset_rigid_body_dict(robot_asset)
         self._robot_joint_dict = self.gym.get_asset_dof_dict(robot_asset)
 
-        return robot_asset, robot_dof_props, default_dof_state, default_dof_pos
+        return robot_asset, robot_dof_props
 
     def _make_envs(
         self,
@@ -259,7 +339,7 @@ def _make_envs(
 
         # get object and robot asset
         obj_assets_list = [self._load_object_asset(obj) for obj in self.objects]
-        robot_asset, robot_dof_props, default_dof_state, default_dof_pos = self._load_robot_assets()
+        robot_asset, robot_dof_props = self._load_robot_assets()
 
         #### Joint Info ####
         for art_obj_name, art_obj_joint_dict in self._articulated_joint_dict_dict.items():
@@ -432,6 +512,7 @@ def get_states(self, env_ids: list[int] | None = None) -> list[EnvState]:
                 )
             object_states[obj.name] = state
 
+        # FIXME some RL task need joint state as dof_pos - default_dof_pos, not absolute dof_pos. see https://github.com/leggedrobotics/legged_gym/blob/17847702f90d8227cd31cce9c920aa53a739a09a/legged_gym/envs/base/legged_robot.py#L216 for further details
         robot_states = {}
         for obj_id, robot in enumerate([self.robot]):
             joint_reindex = self.get_joint_reindex(robot.name)
@@ -444,7 +525,7 @@ def get_states(self, env_ids: list[int] | None = None) -> list[EnvState]:
                 joint_vel=self._dof_states.view(self.num_envs, -1, 2)[:, joint_reindex, 1],
                 joint_pos_target=None,  # TODO
                 joint_vel_target=None,  # TODO
-                joint_effort_target=None,  # TODO
+                joint_effort_target=self._effort if self._manual_pd_on else None,
             )
             robot_states[robot.name] = state
 
@@ -483,7 +564,7 @@ def _get_action_array_all(self, actions: list[Action]):
 
     def set_dof_targets(self, obj_name: str, actions: list[Action]):
         self._actions_cache = actions
-        action_input = torch.zeros_like(self._dof_states[:, 0].clone())
+        action_input = torch.zeros_like(self._dof_states[:, 0])
         action_array_all = self._get_action_array_all(actions)
         robot_dim = action_array_all.shape[1]
 
@@ -498,7 +579,17 @@ def set_dof_targets(self, obj_name: str, actions: list[Action]):
         ]
         action_input[robot_dim_index] = action_array_all.float().to(self.device).reshape(-1)
 
-        self.gym.set_dof_position_target_tensor(self.sim, gymtorch.unwrap_tensor(action_input))
+        # if any effort joint exist, set pd controller's target position for later effort calculation
+        if self._manual_pd_on:
+            actions_reshape = action_input.view(self._num_envs, self._obj_num_dof + self._robot_num_dof)
+            self.actions = actions_reshape[:, self._obj_num_dof :]
+            # and set position target for position actuator if any exist
+            if len(self._pos_ctrl_dof_dix) > 0:
+                self.gym.set_dof_position_target_tensor(self.sim, gymtorch.unwrap_tensor(action_input))
+
+        # directly set position target
+        else:
+            self.gym.set_dof_position_target_tensor(self.sim, gymtorch.unwrap_tensor(action_input))
 
     def refresh_render(self) -> None:
         # Step the physics
@@ -508,19 +599,31 @@ def refresh_render(self) -> None:
         # Refresh cameras and viewer
         self.gym.step_graphics(self.sim)
         self.gym.render_all_camera_sensors(self.sim)
+        # TODO add keyboard callback(mostly likely push v) to stop rendering in render mode
         if not self.headless:
             self.gym.draw_viewer(self.viewer, self.sim, False)
 
-    def simulate(self) -> None:
-        # Step the physics
-        for _ in range(self.scenario.decimation):
+    def _simulate_one_physics_step(self, action):
+        # for pd control joints by effort api, update torque and step the physics
+        if self._manual_pd_on:
+            self._apply_pd_control(action)
+            self.gym.simulate(self.sim)
+            self.gym.fetch_results(self.sim, True)
+            self.gym.refresh_dof_state_tensor(self.sim)
+        # for position control joints, just step the physics
+        else:
             self.gym.simulate(self.sim)
             self.gym.fetch_results(self.sim, True)
 
+    def simulate(self) -> None:
+        # Step the physics
+        for _ in range(self.scenario.decimation):
+            self._simulate_one_physics_step(self.actions)
         # Refresh tensors
+        if not self._manual_pd_on:
+            self.gym.refresh_dof_state_tensor(self.sim)
         self.gym.refresh_rigid_body_state_tensor(self.sim)
         self.gym.refresh_actor_root_state_tensor(self.sim)
-        self.gym.refresh_dof_state_tensor(self.sim)
         self.gym.refresh_jacobian_tensors(self.sim)
         self.gym.refresh_mass_matrix_tensors(self.sim)
 
@@ -532,6 +635,34 @@ def simulate(self) -> None:
 
         # self.gym.sync_frame_time(self.sim)
 
+    def _compute_effort(self, actions):
+        """Compute effort from actions"""
+        # scale the actions (generally output from policy)
+        action_scaled = self._action_scale * actions
+        robot_dof_pos = self._robot_dof_state[..., 0]
+        robot_dof_vel = self._robot_dof_state[..., 1]
+        if self._action_offset:
+            _effort = (
+                self._p_gains * (action_scaled + self._default_dof_pos - robot_dof_pos) - self._d_gains * robot_dof_vel
+            )
+        else:
+            _effort = self._p_gains * (action_scaled - robot_dof_pos) - self._d_gains * robot_dof_vel
+        self._effort = torch.clip(_effort, -self._torque_limits, self._torque_limits)
+        effort = self._effort.to(torch.float32)
+        return effort
+
+    def _apply_pd_control(self, actions):
+        """
+        Compute torque using pd controller for effort actuator and set torque.
+        """
+        effort = self._compute_effort(actions)
+
+        # NOTE: effort passed set_dof_actuation_force_tensor() must have the same dimension as the number of DOFs, even if some DOFs are not actionable.
+        if self._obj_num_dof > 0:
+            obj_force_placeholder = torch.zeros((self._num_envs, self._obj_num_dof), device=self.device)
+            effort = torch.cat((obj_force_placeholder, effort), dim=1)
+        self.gym.set_dof_actuation_force_tensor(self.sim, gymtorch.unwrap_tensor(effort))
+
     def set_states(self, states: list[EnvState], env_ids: list[int] | None = None):
         ## Support setting status only for specified env_ids
         if env_ids is None:
@@ -608,6 +739,9 @@ def set_states(self, states: list[EnvState], env_ids: list[int] | None = None):
         self.gym.refresh_jacobian_tensors(self.sim)
         self.gym.refresh_mass_matrix_tensors(self.sim)
 
+        # reset all env_id action to default
+        self.actions[env_ids] = 0.0
+
     def _set_actor_root_state(self, position_list, rotation_list, env_ids):
         new_root_states = self._root_states.clone()
 
@@ -622,6 +756,7 @@ def _set_actor_root_state(self, position_list, rotation_list, env_ids):
                 new_root_states[actor_idx, 3:7] = torch.tensor(
                     rotation_list[i][j], dtype=torch.float32, device=self.device
                 )
+                new_root_states[actor_idx, 7:13] = torch.zeros(6, dtype=torch.float32, device=self.device)
 
         # Get the actor indices to update
         actor_indices = []
diff --git a/roboverse_learn/humanoidbench_rl/configs/isaacgym.yaml b/roboverse_learn/humanoidbench_rl/configs/isaacgym.yaml
index a2429c9d9..9de929720 100644
--- a/roboverse_learn/humanoidbench_rl/configs/isaacgym.yaml
+++ b/roboverse_learn/humanoidbench_rl/configs/isaacgym.yaml
@@ -4,8 +4,8 @@
 sim: isaacgym
 robot: h1
 task: humanoidbench:Stand
-decimation: 1
-num_envs: 1024
+decimation: 4
+num_envs: 128
 add_table: false
 
 # Training settings
@@ -31,6 +31,8 @@ lr_schedule_type: linear  # Optional: linear, constant, cosine
 final_lr_fraction: 0.1    # Optional: final learning rate is 10% of initial learning rate
 
 # Wandb settings
-use_wandb: false
+use_wandb: true
 wandb_project: humanoidbench_rl_training
-wandb_entity: <wandb entity>
+
+# visualization
+headless: false
diff --git a/roboverse_learn/humanoidbench_rl/train_sb3.py b/roboverse_learn/humanoidbench_rl/train_sb3.py
index dc5623dee..3bb202043 100644
--- a/roboverse_learn/humanoidbench_rl/train_sb3.py
+++ b/roboverse_learn/humanoidbench_rl/train_sb3.py
@@ -122,7 +122,7 @@ def main():
         try_add_table=config.get("add_table", False),
         sim=config.get("sim"),
         num_envs=config.get("num_envs", 1),
-        headless=True if config.get("train_or_eval") == "train" else False,
+        headless=config.get("headless", True),
         cameras=[],
     )
 
diff --git a/roboverse_learn/humanoidbench_rl/wrapper_sb3.py b/roboverse_learn/humanoidbench_rl/wrapper_sb3.py
index ed78f9811..4087ee8b4 100644
--- a/roboverse_learn/humanoidbench_rl/wrapper_sb3.py
+++ b/roboverse_learn/humanoidbench_rl/wrapper_sb3.py
@@ -37,6 +37,7 @@ def __init__(self, scenario: ScenarioCfg):
         else:
             self.init_states = self.init_states[: self.num_envs]
 
+        # FIXME action limit differs with joint limit in locomotion configuration(desire pos = scale*action + default pos)
         # Set up action space based on robot joint limits
         joint_limits = self.robot.joint_limits
         action_low = []


Reviews:
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-05T21:29:28Z
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-05T22:56:26Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T18:56:49Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T18:58:22Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T19:05:36Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T19:17:05Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T20:18:01Z
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T21:16:00Z
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T21:16:45Z
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T21:17:20Z
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T21:18:25Z
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-06T21:19:58Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-07T06:32:02Z
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T06:43:59Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T07:17:59Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T07:25:20Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T10:00:21Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T14:20:10Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T17:05:42Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T17:15:50Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-08T17:34:28Z
user: login: Fisher-Wang
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-09T09:42:44Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-09T19:14:05Z
user: login: geng-haoran
body: Generally good. See my comment before.
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-09T19:38:50Z
user: login: WeisonWEileen
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-10T07:32:01Z

Review comments:
diff_hunk: @@ -68,26 +82,29 @@ def launch(self) -> None:
         self._root_states = gymtorch.wrap_tensor(self.gym.acquire_actor_root_state_tensor(self.sim))
         self._dof_states = gymtorch.wrap_tensor(self.gym.acquire_dof_state_tensor(self.sim))
         self._rigid_body_states = gymtorch.wrap_tensor(self.gym.acquire_rigid_body_state_tensor(self.sim))
+        self._robot_dof_state = self._dof_states.view(self._num_envs, -1, 2)[:, : self._robot_num_dof]
 
     def _init_gym(self) -> None:
         physics_engine = gymapi.SIM_PHYSX
         self.gym = gymapi.acquire_gym()
         # configure sim
+        # TODO move more params into sim_params cfg
         sim_params = gymapi.SimParams()
         sim_params.up_axis = gymapi.UP_AXIS_Z
         sim_params.gravity = gymapi.Vec3(0.0, 0.0, -9.8)
-        sim_params.dt = 1.0 / 60.0
+        sim_params.dt = self.task.sim_params.timestep
         sim_params.substeps = 2
         sim_params.use_gpu_pipeline = True
         sim_params.physx.solver_type = 1
-        sim_params.physx.num_position_iterations = 8
-        sim_params.physx.num_velocity_iterations = 1
+        sim_params.physx.num_position_iterations = self.task.sim_params.num_position_iterations
+        sim_params.physx.num_velocity_iterations = self.task.sim_params.num_velocity_iterations
         sim_params.physx.rest_offset = 0.0
-        sim_params.physx.contact_offset = 0.001
+        sim_params.physx.contact_offset = self.task.sim_params.contact_offset
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: These params should read from scenario instead of task; you can also add TODOs to leave it as future work
created_at: 2025-05-05T21:10:45Z
author_association: COLLABORATOR
diff_hunk: @@ -68,26 +82,29 @@ def launch(self) -> None:
         self._root_states = gymtorch.wrap_tensor(self.gym.acquire_actor_root_state_tensor(self.sim))
         self._dof_states = gymtorch.wrap_tensor(self.gym.acquire_dof_state_tensor(self.sim))
         self._rigid_body_states = gymtorch.wrap_tensor(self.gym.acquire_rigid_body_state_tensor(self.sim))
+        self._robot_dof_state = self._dof_states.view(self._num_envs, -1, 2)[:, : self._robot_num_dof]
 
     def _init_gym(self) -> None:
         physics_engine = gymapi.SIM_PHYSX
         self.gym = gymapi.acquire_gym()
         # configure sim
+        # TODO move more params into sim_params cfg
         sim_params = gymapi.SimParams()
         sim_params.up_axis = gymapi.UP_AXIS_Z
         sim_params.gravity = gymapi.Vec3(0.0, 0.0, -9.8)
-        sim_params.dt = 1.0 / 60.0
+        sim_params.dt = self.task.sim_params.timestep
         sim_params.substeps = 2
         sim_params.use_gpu_pipeline = True
         sim_params.physx.solver_type = 1
-        sim_params.physx.num_position_iterations = 8
-        sim_params.physx.num_velocity_iterations = 1
+        sim_params.physx.num_position_iterations = self.task.sim_params.num_position_iterations
+        sim_params.physx.num_velocity_iterations = self.task.sim_params.num_velocity_iterations
         sim_params.physx.rest_offset = 0.0
-        sim_params.physx.contact_offset = 0.001
+        sim_params.physx.contact_offset = self.task.sim_params.contact_offset
         sim_params.physx.friction_offset_threshold = 0.001
         sim_params.physx.friction_correlation_distance = 0.0005
         sim_params.physx.num_threads = 0
         sim_params.physx.use_gpu = True
+        sim_params.physx.bounce_threshold_velocity = 0.5
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: What does this do? Better to add comments and link to official doc 

In [isaacgym doc](https://docs.robotsfan.com/isaacgym/programming/tuning.html) it says
```
bounce_threshold_velocity - A contact with a relative velocity below this will not bounce. Default is 0.2 m/s
...
bounce_threshold_velocity - relative velocity required to trigger a bounce. The default is 0.2 m/s, which is some cases could be pretty high. Reducing could give a more natural bouncing behaviour, for example in case of bouncing balls.
```

So it seems not normal when this parameter is higher than default value?
created_at: 2025-05-05T21:13:57Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +226,81 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_EFFORT
+        asset_options.replace_cylinder_with_capsule = True
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: What does this do? Better to add comments and link to official doc.

In the doc it says
```
propertyreplace_cylinder_with_capsule
flag to replace Cylinders with capsules for additional performance.
```
Does this have any known side effects?

created_at: 2025-05-05T21:16:58Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +226,81 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_EFFORT
+        asset_options.replace_cylinder_with_capsule = True
         robot_asset = self.gym.load_asset(self.sim, asset_root, robot_asset_file, asset_options)
         # configure robot dofs
+        robot_num_dofs = self.gym.get_asset_dof_count(robot_asset)
+        self._robot_num_dof = robot_num_dofs
+
+        # TODO move pd controller into controller api
+        # pd control params
+        self._p_gains = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+        self._d_gains = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+        self._torque_limits = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+
+        # TODO combability for passive joint
+
         robot_dof_props = self.gym.get_asset_dof_properties(robot_asset)
         robot_lower_limits = robot_dof_props["lower"]
         robot_upper_limits = robot_dof_props["upper"]
-        robot_ranges = robot_upper_limits - robot_lower_limits
         robot_mids = 0.3 * (robot_upper_limits + robot_lower_limits)
-        robot_dof_props["driveMode"][:7].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][:7].fill(400.0)
-        robot_dof_props["damping"][:7].fill(40.0)
 
-        # grippers
-        robot_dof_props["driveMode"][7:].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][7:].fill(800.0)
-        robot_dof_props["damping"][7:].fill(40.0)
+        num_actions = 0
+        default_dof_pos = []
+        dof_names = self.gym.get_asset_dof_names(robot_asset)
+
+        for i in range(len(dof_names)):
+            # use pd controller to get force/torque
+            actuator_cfg = self._robot.actuators[dof_names[i]]
+            if not actuator_cfg.is_ee:
+                default_dof_pos_i = (
+                    self._robot.default_joint_positions[dof_names[i]]
+                    if self._robot.default_joint_positions[dof_names[i]] is not None
+                    else robot_mids[i]
+                )
+                default_dof_pos.append(default_dof_pos_i)
+                self._p_gains[:, i] = actuator_cfg.stiffness
+                self._d_gains[:, i] = actuator_cfg.damping
+                self._torque_limits[:, i] = 0.85 * (
+                    actuator_cfg.torque_limit
+                    if actuator_cfg.torque_limit is not None
+                    else torch.tensor(robot_dof_props["effort"][i], dtype=torch.float, device=self.device)
+                )
+                if self.effort_mode:
+                    robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_EFFORT
+                    robot_dof_props["stiffness"][i] = 0.0
+                    robot_dof_props["damping"][i] = 0.0
+                else:
+                    robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_POS
+                    robot_dof_props["stiffness"][i] = 400.0
+                    robot_dof_props["damping"][i] = 40.0
+            # for end effector joint, use built-in pos contorl
+            else:
+                # grippers
+                default_dof_pos.append(robot_upper_limits[i])
+                robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_POS
+                robot_dof_props["stiffness"][i] = 800.0
+                robot_dof_props["damping"][i] = 40.0
+                self._gripper_dof_dix.append(i)
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: Why pos control for ee joints?
created_at: 2025-05-05T21:18:34Z
author_association: COLLABORATOR
diff_hunk: @@ -56,6 +57,19 @@ def __init__(self, scenario: ScenarioCfg):
         self._root_states: torch.Tensor | None = None  # will update after refresh
         self._dof_states: torch.Tensor | None = None  # will update after refresh
         self._rigid_body_states: torch.Tensor | None = None  # will update after refresh
+        self._robot_dof_state: torch.Tensor | None = None
+
+        self._robot_num_dof: int  # TODO robots which have passive joint may not compatible
+        self._obj_num_dof: int = 0
+        self._actions: torch.Tensor | None = None
+        self._p_gains: torch.Tensor | None = None
+        self._d_gains: torch.Tensor | None = None
+        self._torque_limits: torch.Tensor | None = None
+        self._default_dof_pos: torch.Tensor | None = None
+        self._gripper_dof_dix = []
+        self.effort_mode = (
+            True if scenario.task.task_type == TaskType.LOCOMOTION else False
+        )  # torque for reward computation
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: Can we use effort mode for all tasks? It relates to what does the built-in position control do - whether it is a pd control which translate target pos to effort.
created_at: 2025-05-05T21:22:21Z
author_association: COLLABORATOR
diff_hunk: @@ -532,6 +604,27 @@ def simulate(self) -> None:
 
         # self.gym.sync_frame_time(self.sim)
 
+    def _apply_action(self):
+        """
+        Compute torque using pd controller for effort actuator and set desire postion for position actuator given action.
+        TODO: 1. integrate controller api
+              1. move action scaled factor into config]
+        """
+        action_scaled = torch.tensor(0.5, device=self.device) * self.actions
+        # effort actions
+        dof_pos = self._robot_dof_state[..., 0]
+        dof_vel = self._robot_dof_state[..., 1]
+        _torques = self._p_gains * (action_scaled + self._default_dof_pos - dof_pos) - self._d_gains * dof_vel
+        self._torques = torch.clip(_torques, -self._torque_limits, self._torque_limits)
+        torques = self._torques.to(torch.float32)
+        self.gym.set_dof_actuation_force_tensor(self.sim, gymtorch.unwrap_tensor(torques))
+
+        # pos actions
+        pos_action = torch.zeros_like(self._torques, device=self.device, dtype=torch.float32)
+        for i in self._gripper_dof_dix:
+            pos_action[:, i] = self.actions[:, i]
+        self.gym.set_dof_position_target_tensor(self.sim, gymtorch.unwrap_tensor(pos_action))
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: Should target position and effort be set together?

According to [humanoid-gym](https://github.com/roboterax/humanoid-gym/blob/main/humanoid/scripts/sim2sim.py#L158), only effort is set when using pd control
created_at: 2025-05-05T21:28:13Z
author_association: COLLABORATOR
diff_hunk: @@ -31,6 +31,10 @@ lr_schedule_type: linear  # Optional: linear, constant, cosine
 final_lr_fraction: 0.1    # Optional: final learning rate is 10% of initial learning rate
 
 # Wandb settings
-use_wandb: false
+use_wandb: true
+# use_wandb: false
 wandb_project: humanoidbench_rl_training
-wandb_entity: <wandb entity>
+wandb_entity: weisonweileen-southern-university-of-science-technology
path: roboverse_learn/humanoidbench_rl/configs/isaacgym.yaml
user: login: Fisher-Wang
body: Why not 
```suggestion

```

so that wandb would use the logined user name automatically
created_at: 2025-05-05T22:56:26Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +226,81 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_EFFORT
+        asset_options.replace_cylinder_with_capsule = True
         robot_asset = self.gym.load_asset(self.sim, asset_root, robot_asset_file, asset_options)
         # configure robot dofs
+        robot_num_dofs = self.gym.get_asset_dof_count(robot_asset)
+        self._robot_num_dof = robot_num_dofs
+
+        # TODO move pd controller into controller api
+        # pd control params
+        self._p_gains = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+        self._d_gains = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+        self._torque_limits = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+
+        # TODO combability for passive joint
+
         robot_dof_props = self.gym.get_asset_dof_properties(robot_asset)
         robot_lower_limits = robot_dof_props["lower"]
         robot_upper_limits = robot_dof_props["upper"]
-        robot_ranges = robot_upper_limits - robot_lower_limits
         robot_mids = 0.3 * (robot_upper_limits + robot_lower_limits)
-        robot_dof_props["driveMode"][:7].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][:7].fill(400.0)
-        robot_dof_props["damping"][:7].fill(40.0)
 
-        # grippers
-        robot_dof_props["driveMode"][7:].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][7:].fill(800.0)
-        robot_dof_props["damping"][7:].fill(40.0)
+        num_actions = 0
+        default_dof_pos = []
+        dof_names = self.gym.get_asset_dof_names(robot_asset)
+
+        for i in range(len(dof_names)):
+            # use pd controller to get force/torque
+            actuator_cfg = self._robot.actuators[dof_names[i]]
+            if not actuator_cfg.is_ee:
+                default_dof_pos_i = (
+                    self._robot.default_joint_positions[dof_names[i]]
+                    if self._robot.default_joint_positions[dof_names[i]] is not None
+                    else robot_mids[i]
+                )
+                default_dof_pos.append(default_dof_pos_i)
+                self._p_gains[:, i] = actuator_cfg.stiffness
+                self._d_gains[:, i] = actuator_cfg.damping
+                self._torque_limits[:, i] = 0.85 * (
+                    actuator_cfg.torque_limit
+                    if actuator_cfg.torque_limit is not None
+                    else torch.tensor(robot_dof_props["effort"][i], dtype=torch.float, device=self.device)
+                )
+                if self.effort_mode:
+                    robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_EFFORT
+                    robot_dof_props["stiffness"][i] = 0.0
+                    robot_dof_props["damping"][i] = 0.0
+                else:
+                    robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_POS
+                    robot_dof_props["stiffness"][i] = 400.0
+                    robot_dof_props["damping"][i] = 40.0
+            # for end effector joint, use built-in pos contorl
+            else:
+                # grippers
+                default_dof_pos.append(robot_upper_limits[i])
+                robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_POS
+                robot_dof_props["stiffness"][i] = 800.0
+                robot_dof_props["damping"][i] = 40.0
+                self._gripper_dof_dix.append(i)
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: because I check isaacgym_preview_4_package  [demo](https://gist.github.com/DanielTakeshi/2e6a11dc0af1ba26b50857f6ae8ff386) and [IsaacGymEnvs](https://github.com/isaac-sim/IsaacGymEnvs/blob/aeed298638a1f7b5421b38f5f3cc2d1079b6d9c3/isaacgymenvs/tasks/franka_cube_stack.py#L248) , both of them use pos control for ee joints, maybe that 's better way to simulate prismatic joint?
created_at: 2025-05-06T18:56:49Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +226,81 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_EFFORT
+        asset_options.replace_cylinder_with_capsule = True
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: oh, sorry. i will move to the simulator params config settings and provide a default value. 
created_at: 2025-05-06T18:58:22Z
author_association: COLLABORATOR
diff_hunk: @@ -68,26 +82,29 @@ def launch(self) -> None:
         self._root_states = gymtorch.wrap_tensor(self.gym.acquire_actor_root_state_tensor(self.sim))
         self._dof_states = gymtorch.wrap_tensor(self.gym.acquire_dof_state_tensor(self.sim))
         self._rigid_body_states = gymtorch.wrap_tensor(self.gym.acquire_rigid_body_state_tensor(self.sim))
+        self._robot_dof_state = self._dof_states.view(self._num_envs, -1, 2)[:, : self._robot_num_dof]
 
     def _init_gym(self) -> None:
         physics_engine = gymapi.SIM_PHYSX
         self.gym = gymapi.acquire_gym()
         # configure sim
+        # TODO move more params into sim_params cfg
         sim_params = gymapi.SimParams()
         sim_params.up_axis = gymapi.UP_AXIS_Z
         sim_params.gravity = gymapi.Vec3(0.0, 0.0, -9.8)
-        sim_params.dt = 1.0 / 60.0
+        sim_params.dt = self.task.sim_params.timestep
         sim_params.substeps = 2
         sim_params.use_gpu_pipeline = True
         sim_params.physx.solver_type = 1
-        sim_params.physx.num_position_iterations = 8
-        sim_params.physx.num_velocity_iterations = 1
+        sim_params.physx.num_position_iterations = self.task.sim_params.num_position_iterations
+        sim_params.physx.num_velocity_iterations = self.task.sim_params.num_velocity_iterations
         sim_params.physx.rest_offset = 0.0
-        sim_params.physx.contact_offset = 0.001
+        sim_params.physx.contact_offset = self.task.sim_params.contact_offset
         sim_params.physx.friction_offset_threshold = 0.001
         sim_params.physx.friction_correlation_distance = 0.0005
         sim_params.physx.num_threads = 0
         sim_params.physx.use_gpu = True
+        sim_params.physx.bounce_threshold_velocity = 0.5
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: I set as [legged_gym](https://github.com/leggedrobotics/legged_gym/blob/17847702f90d8227cd31cce9c920aa53a739a09a/legged_gym/envs/base/legged_robot_config.py#L196) do. I will move it to sim param and  set default value back to 0.2
created_at: 2025-05-06T19:05:36Z
author_association: COLLABORATOR
diff_hunk: @@ -68,26 +82,29 @@ def launch(self) -> None:
         self._root_states = gymtorch.wrap_tensor(self.gym.acquire_actor_root_state_tensor(self.sim))
         self._dof_states = gymtorch.wrap_tensor(self.gym.acquire_dof_state_tensor(self.sim))
         self._rigid_body_states = gymtorch.wrap_tensor(self.gym.acquire_rigid_body_state_tensor(self.sim))
+        self._robot_dof_state = self._dof_states.view(self._num_envs, -1, 2)[:, : self._robot_num_dof]
 
     def _init_gym(self) -> None:
         physics_engine = gymapi.SIM_PHYSX
         self.gym = gymapi.acquire_gym()
         # configure sim
+        # TODO move more params into sim_params cfg
         sim_params = gymapi.SimParams()
         sim_params.up_axis = gymapi.UP_AXIS_Z
         sim_params.gravity = gymapi.Vec3(0.0, 0.0, -9.8)
-        sim_params.dt = 1.0 / 60.0
+        sim_params.dt = self.task.sim_params.timestep
         sim_params.substeps = 2
         sim_params.use_gpu_pipeline = True
         sim_params.physx.solver_type = 1
-        sim_params.physx.num_position_iterations = 8
-        sim_params.physx.num_velocity_iterations = 1
+        sim_params.physx.num_position_iterations = self.task.sim_params.num_position_iterations
+        sim_params.physx.num_velocity_iterations = self.task.sim_params.num_velocity_iterations
         sim_params.physx.rest_offset = 0.0
-        sim_params.physx.contact_offset = 0.001
+        sim_params.physx.contact_offset = self.task.sim_params.contact_offset
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: as in [IsaacGymEnvs](https://github.com/isaac-sim/IsaacGymEnvs/tree/main/isaacgymenvs/cfg/train) (see `sim:` in `.yaml`), simulator params vary across tasks,  would it be better to declare them individually for each task ？
created_at: 2025-05-06T19:17:05Z
author_association: COLLABORATOR
diff_hunk: @@ -56,6 +57,19 @@ def __init__(self, scenario: ScenarioCfg):
         self._root_states: torch.Tensor | None = None  # will update after refresh
         self._dof_states: torch.Tensor | None = None  # will update after refresh
         self._rigid_body_states: torch.Tensor | None = None  # will update after refresh
+        self._robot_dof_state: torch.Tensor | None = None
+
+        self._robot_num_dof: int  # TODO robots which have passive joint may not compatible
+        self._obj_num_dof: int = 0
+        self._actions: torch.Tensor | None = None
+        self._p_gains: torch.Tensor | None = None
+        self._d_gains: torch.Tensor | None = None
+        self._torque_limits: torch.Tensor | None = None
+        self._default_dof_pos: torch.Tensor | None = None
+        self._gripper_dof_dix = []
+        self.effort_mode = (
+            True if scenario.task.task_type == TaskType.LOCOMOTION else False
+        )  # torque for reward computation
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: relavent [issue1](https://forums.developer.nvidia.com/t/get-torques-in-position-control-mode/209415/21)  and [issue2](https://forums.developer.nvidia.com/t/different-performance-between-effort-control-and-position-control/220666). built-in position control is not just a pd controller, we cannot get the torque precisely. I think if we need to torque for reward computation(for example, in locomotion task), just use effortmode and manual pd control. In some simpler case, for example, replay trajectory, we can use built-in control mode. For those none-actuator joint, we can use `gymapi.DOF_MODE_NONE`
created_at: 2025-05-06T20:18:01Z
author_association: COLLABORATOR
diff_hunk: @@ -68,26 +82,29 @@ def launch(self) -> None:
         self._root_states = gymtorch.wrap_tensor(self.gym.acquire_actor_root_state_tensor(self.sim))
         self._dof_states = gymtorch.wrap_tensor(self.gym.acquire_dof_state_tensor(self.sim))
         self._rigid_body_states = gymtorch.wrap_tensor(self.gym.acquire_rigid_body_state_tensor(self.sim))
+        self._robot_dof_state = self._dof_states.view(self._num_envs, -1, 2)[:, : self._robot_num_dof]
 
     def _init_gym(self) -> None:
         physics_engine = gymapi.SIM_PHYSX
         self.gym = gymapi.acquire_gym()
         # configure sim
+        # TODO move more params into sim_params cfg
         sim_params = gymapi.SimParams()
         sim_params.up_axis = gymapi.UP_AXIS_Z
         sim_params.gravity = gymapi.Vec3(0.0, 0.0, -9.8)
-        sim_params.dt = 1.0 / 60.0
+        sim_params.dt = self.task.sim_params.timestep
         sim_params.substeps = 2
         sim_params.use_gpu_pipeline = True
         sim_params.physx.solver_type = 1
-        sim_params.physx.num_position_iterations = 8
-        sim_params.physx.num_velocity_iterations = 1
+        sim_params.physx.num_position_iterations = self.task.sim_params.num_position_iterations
+        sim_params.physx.num_velocity_iterations = self.task.sim_params.num_velocity_iterations
         sim_params.physx.rest_offset = 0.0
-        sim_params.physx.contact_offset = 0.001
+        sim_params.physx.contact_offset = self.task.sim_params.contact_offset
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: Yeah, I mean, the current design principle is that, scenario is the config contains all the information, and a task may pre-define some of them. So during initialization, scenario config is updated (overwrited) by the task config, and simulator handler read from scenario, so that task specification is also passed to simulator handlers.
created_at: 2025-05-06T21:16:00Z
author_association: COLLABORATOR
diff_hunk: @@ -68,26 +82,29 @@ def launch(self) -> None:
         self._root_states = gymtorch.wrap_tensor(self.gym.acquire_actor_root_state_tensor(self.sim))
         self._dof_states = gymtorch.wrap_tensor(self.gym.acquire_dof_state_tensor(self.sim))
         self._rigid_body_states = gymtorch.wrap_tensor(self.gym.acquire_rigid_body_state_tensor(self.sim))
+        self._robot_dof_state = self._dof_states.view(self._num_envs, -1, 2)[:, : self._robot_num_dof]
 
     def _init_gym(self) -> None:
         physics_engine = gymapi.SIM_PHYSX
         self.gym = gymapi.acquire_gym()
         # configure sim
+        # TODO move more params into sim_params cfg
         sim_params = gymapi.SimParams()
         sim_params.up_axis = gymapi.UP_AXIS_Z
         sim_params.gravity = gymapi.Vec3(0.0, 0.0, -9.8)
-        sim_params.dt = 1.0 / 60.0
+        sim_params.dt = self.task.sim_params.timestep
         sim_params.substeps = 2
         sim_params.use_gpu_pipeline = True
         sim_params.physx.solver_type = 1
-        sim_params.physx.num_position_iterations = 8
-        sim_params.physx.num_velocity_iterations = 1
+        sim_params.physx.num_position_iterations = self.task.sim_params.num_position_iterations
+        sim_params.physx.num_velocity_iterations = self.task.sim_params.num_velocity_iterations
         sim_params.physx.rest_offset = 0.0
-        sim_params.physx.contact_offset = 0.001
+        sim_params.physx.contact_offset = self.task.sim_params.contact_offset
         sim_params.physx.friction_offset_threshold = 0.001
         sim_params.physx.friction_correlation_distance = 0.0005
         sim_params.physx.num_threads = 0
         sim_params.physx.use_gpu = True
+        sim_params.physx.bounce_threshold_velocity = 0.5
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: Sounds good
created_at: 2025-05-06T21:16:44Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +226,81 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_EFFORT
+        asset_options.replace_cylinder_with_capsule = True
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: No worry, just want to double check ;)
created_at: 2025-05-06T21:17:20Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +226,81 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_EFFORT
+        asset_options.replace_cylinder_with_capsule = True
         robot_asset = self.gym.load_asset(self.sim, asset_root, robot_asset_file, asset_options)
         # configure robot dofs
+        robot_num_dofs = self.gym.get_asset_dof_count(robot_asset)
+        self._robot_num_dof = robot_num_dofs
+
+        # TODO move pd controller into controller api
+        # pd control params
+        self._p_gains = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+        self._d_gains = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+        self._torque_limits = torch.zeros(
+            self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+        )
+
+        # TODO combability for passive joint
+
         robot_dof_props = self.gym.get_asset_dof_properties(robot_asset)
         robot_lower_limits = robot_dof_props["lower"]
         robot_upper_limits = robot_dof_props["upper"]
-        robot_ranges = robot_upper_limits - robot_lower_limits
         robot_mids = 0.3 * (robot_upper_limits + robot_lower_limits)
-        robot_dof_props["driveMode"][:7].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][:7].fill(400.0)
-        robot_dof_props["damping"][:7].fill(40.0)
 
-        # grippers
-        robot_dof_props["driveMode"][7:].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][7:].fill(800.0)
-        robot_dof_props["damping"][7:].fill(40.0)
+        num_actions = 0
+        default_dof_pos = []
+        dof_names = self.gym.get_asset_dof_names(robot_asset)
+
+        for i in range(len(dof_names)):
+            # use pd controller to get force/torque
+            actuator_cfg = self._robot.actuators[dof_names[i]]
+            if not actuator_cfg.is_ee:
+                default_dof_pos_i = (
+                    self._robot.default_joint_positions[dof_names[i]]
+                    if self._robot.default_joint_positions[dof_names[i]] is not None
+                    else robot_mids[i]
+                )
+                default_dof_pos.append(default_dof_pos_i)
+                self._p_gains[:, i] = actuator_cfg.stiffness
+                self._d_gains[:, i] = actuator_cfg.damping
+                self._torque_limits[:, i] = 0.85 * (
+                    actuator_cfg.torque_limit
+                    if actuator_cfg.torque_limit is not None
+                    else torch.tensor(robot_dof_props["effort"][i], dtype=torch.float, device=self.device)
+                )
+                if self.effort_mode:
+                    robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_EFFORT
+                    robot_dof_props["stiffness"][i] = 0.0
+                    robot_dof_props["damping"][i] = 0.0
+                else:
+                    robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_POS
+                    robot_dof_props["stiffness"][i] = 400.0
+                    robot_dof_props["damping"][i] = 40.0
+            # for end effector joint, use built-in pos contorl
+            else:
+                # grippers
+                default_dof_pos.append(robot_upper_limits[i])
+                robot_dof_props["driveMode"][i] = gymapi.DOF_MODE_POS
+                robot_dof_props["stiffness"][i] = 800.0
+                robot_dof_props["damping"][i] = 40.0
+                self._gripper_dof_dix.append(i)
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: interesting. Could you also check whether effort control will succeed?
created_at: 2025-05-06T21:18:25Z
author_association: COLLABORATOR
diff_hunk: @@ -56,6 +57,19 @@ def __init__(self, scenario: ScenarioCfg):
         self._root_states: torch.Tensor | None = None  # will update after refresh
         self._dof_states: torch.Tensor | None = None  # will update after refresh
         self._rigid_body_states: torch.Tensor | None = None  # will update after refresh
+        self._robot_dof_state: torch.Tensor | None = None
+
+        self._robot_num_dof: int  # TODO robots which have passive joint may not compatible
+        self._obj_num_dof: int = 0
+        self._actions: torch.Tensor | None = None
+        self._p_gains: torch.Tensor | None = None
+        self._d_gains: torch.Tensor | None = None
+        self._torque_limits: torch.Tensor | None = None
+        self._default_dof_pos: torch.Tensor | None = None
+        self._gripper_dof_dix = []
+        self.effort_mode = (
+            True if scenario.task.task_type == TaskType.LOCOMOTION else False
+        )  # torque for reward computation
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: I see, that makes sense. Maybe better to read from scenario config to decide which control mode to use. You can leave that part to me
created_at: 2025-05-06T21:19:58Z
author_association: COLLABORATOR
diff_hunk: @@ -68,26 +82,29 @@ def launch(self) -> None:
         self._root_states = gymtorch.wrap_tensor(self.gym.acquire_actor_root_state_tensor(self.sim))
         self._dof_states = gymtorch.wrap_tensor(self.gym.acquire_dof_state_tensor(self.sim))
         self._rigid_body_states = gymtorch.wrap_tensor(self.gym.acquire_rigid_body_state_tensor(self.sim))
+        self._robot_dof_state = self._dof_states.view(self._num_envs, -1, 2)[:, : self._robot_num_dof]
 
     def _init_gym(self) -> None:
         physics_engine = gymapi.SIM_PHYSX
         self.gym = gymapi.acquire_gym()
         # configure sim
+        # TODO move more params into sim_params cfg
         sim_params = gymapi.SimParams()
         sim_params.up_axis = gymapi.UP_AXIS_Z
         sim_params.gravity = gymapi.Vec3(0.0, 0.0, -9.8)
-        sim_params.dt = 1.0 / 60.0
+        sim_params.dt = self.task.sim_params.timestep
         sim_params.substeps = 2
         sim_params.use_gpu_pipeline = True
         sim_params.physx.solver_type = 1
-        sim_params.physx.num_position_iterations = 8
-        sim_params.physx.num_velocity_iterations = 1
+        sim_params.physx.num_position_iterations = self.task.sim_params.num_position_iterations
+        sim_params.physx.num_velocity_iterations = self.task.sim_params.num_velocity_iterations
         sim_params.physx.rest_offset = 0.0
-        sim_params.physx.contact_offset = 0.001
+        sim_params.physx.contact_offset = self.task.sim_params.contact_offset
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: sounds great, i will implement it
created_at: 2025-05-07T06:32:02Z
author_association: COLLABORATOR
diff_hunk: @@ -14,6 +14,7 @@ class BaseActuatorCfg:
     stiffness: float | None = None
     torque_limit: float | None = None
     actionable: bool = True
+    torque_limit: float | None = None
path: metasim/cfg/robots/base_robot_cfg.py
user: login: Fisher-Wang
body: This seems redundant to Line 15
created_at: 2025-05-08T05:57:01Z
author_association: COLLABORATOR
diff_hunk: @@ -41,6 +43,9 @@ class BaseTaskCfg:
     can_tabletop: bool = False
     reward_functions: list[callable[[list[EnvState], str | None], torch.FloatTensor]] = MISSING
     reward_weights: list[float] = MISSING
+    sim_params: SimParamCfg = SimParamCfg()
+    control_type: str = None  # TODO provide a basic control type
path: metasim/cfg/tasks/base_task_cfg.py
user: login: Fisher-Wang
body: Be consistent with scenario definition?
```suggestion
    control_type: Literal["pos", "effort"] = "pos"
```
created_at: 2025-05-08T05:59:25Z
author_association: COLLABORATOR
diff_hunk: @@ -26,3 +26,4 @@ class CloseBoxCfg(RLBenchTaskCfg):
         mode="le",
         radian_threshold=-14 / 180 * math.pi,
     )
+    control_type = "pos"
path: metasim/cfg/tasks/rlbench/close_box_cfg.py
user: login: Fisher-Wang
body: Old tasks use pos control by default, as they did in the past, no need to explicitly set it? 
created_at: 2025-05-08T06:01:01Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +226,81 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_EFFORT
+        asset_options.replace_cylinder_with_capsule = True
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: As discussed previously, `replace_cylinder_with_capsule` should also read from sim_params?
created_at: 2025-05-08T06:02:10Z
author_association: COLLABORATOR
diff_hunk: @@ -508,19 +585,23 @@ def refresh_render(self) -> None:
         # Refresh cameras and viewer
         self.gym.step_graphics(self.sim)
         self.gym.render_all_camera_sensors(self.sim)
+        # TODO add keyboard callback(mostly likely push v) to stop rendering in render mode
         if not self.headless:
             self.gym.draw_viewer(self.viewer, self.sim, False)
 
     def simulate(self) -> None:
         # Step the physics
         for _ in range(self.scenario.decimation):
+            # if effort mode, use pd controller to get torque
+            if self._effort_mode:
+                self._apply_action()
             self.gym.simulate(self.sim)
             self.gym.fetch_results(self.sim, True)
+            self.gym.refresh_dof_state_tensor(self.sim)
 
         # Refresh tensors
         self.gym.refresh_rigid_body_state_tensor(self.sim)
         self.gym.refresh_actor_root_state_tensor(self.sim)
-        self.gym.refresh_dof_state_tensor(self.sim)
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: According to official doc, "The function refresh_dof_state_tensor populates the tensor with the latest data from the simulation", so no need to put it inside the loop?
created_at: 2025-05-08T06:05:26Z
author_association: COLLABORATOR
diff_hunk: @@ -532,6 +613,35 @@ def simulate(self) -> None:
 
         # self.gym.sync_frame_time(self.sim)
 
+    def _apply_action(self):
+        """
+        Compute torque using pd controller for effort actuator and set desire postion for position actuator given action.
+        TODO: 1. integrate controller api
+              2. check whether replay demo works when using effort mode
+        """
+        # action_scaled = self._action_scale * self.actions[:, self._obj_num_dof :]
+        action_scaled = self._action_scale * self.actions
+        # effort actions
+        robot_dof_pos = self._robot_dof_state[..., 0]
+        robot_dof_vel = self._robot_dof_state[..., 1]
+        # FIXME  self._default_dof_pos. now default is RL locomotion configuration, set it to 0 for
+        _effort = (
+            self._p_gains * (action_scaled + self._default_dof_pos - robot_dof_pos) - self._d_gains * robot_dof_vel
+        )
+        self._effort = torch.clip(_effort, -self._torque_limits, self._torque_limits)
+        effort = self._effort.to(torch.float32)
+        if self._obj_num_dof > 0:
+            obj_force_placeholder = torch.zeros((self._num_envs, self._obj_num_dof), device=self.device)
+            effort = torch.cat((obj_force_placeholder, effort), dim=1)
+        self.gym.set_dof_actuation_force_tensor(self.sim, gymtorch.unwrap_tensor(effort))
+
+        # always use built-in position controller for gripper.
+        # referece:https://github.com/isaac-sim/IsaacGymEnvs/blob/main/isaacgymenvs/tasks/franka_cube_stack.py
+        pos_action = torch.zeros_like(effort, device=self.device, dtype=torch.float32)
+        for i in self._gripper_dof_dix:
+            pos_action[:, i] = self.actions[:, i]
+        self.gym.set_dof_position_target_tensor(self.sim, gymtorch.unwrap_tensor(pos_action))
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: Should target position and effort be set together?

According to [humanoid-gym](https://github.com/roboterax/humanoid-gym/blob/main/humanoid/scripts/sim2sim.py#L158), only effort is set when using pd control
created_at: 2025-05-08T06:07:08Z
author_association: COLLABORATOR
diff_hunk: @@ -498,7 +569,13 @@ def set_dof_targets(self, obj_name: str, actions: list[Action]):
         ]
         action_input[robot_dim_index] = action_array_all.float().to(self.device).reshape(-1)
 
-        self.gym.set_dof_position_target_tensor(self.sim, gymtorch.unwrap_tensor(action_input))
+        if self._effort_mode:
+            #
+            self.actions = action_input.view(self._num_envs, self._robot_num_dof + self._obj_num_dof)[
+                :, self._obj_num_dof :
+            ]
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: At this part `set_dof_actuation_force_tensor` should be called?
created_at: 2025-05-08T06:08:43Z
author_association: COLLABORATOR
diff_hunk: @@ -508,19 +585,23 @@ def refresh_render(self) -> None:
         # Refresh cameras and viewer
         self.gym.step_graphics(self.sim)
         self.gym.render_all_camera_sensors(self.sim)
+        # TODO add keyboard callback(mostly likely push v) to stop rendering in render mode
         if not self.headless:
             self.gym.draw_viewer(self.viewer, self.sim, False)
 
     def simulate(self) -> None:
         # Step the physics
         for _ in range(self.scenario.decimation):
+            # if effort mode, use pd controller to get torque
+            if self._effort_mode:
+                self._apply_action()
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: This seems not correct? `simulate` shouldn't apply any action?
created_at: 2025-05-08T06:10:57Z
author_association: COLLABORATOR
diff_hunk: @@ -532,6 +613,35 @@ def simulate(self) -> None:
 
         # self.gym.sync_frame_time(self.sim)
 
+    def _apply_action(self):
+        """
+        Compute torque using pd controller for effort actuator and set desire postion for position actuator given action.
+        TODO: 1. integrate controller api
+              2. check whether replay demo works when using effort mode
+        """
+        # action_scaled = self._action_scale * self.actions[:, self._obj_num_dof :]
+        action_scaled = self._action_scale * self.actions
+        # effort actions
+        robot_dof_pos = self._robot_dof_state[..., 0]
+        robot_dof_vel = self._robot_dof_state[..., 1]
+        # FIXME  self._default_dof_pos. now default is RL locomotion configuration, set it to 0 for
+        _effort = (
+            self._p_gains * (action_scaled + self._default_dof_pos - robot_dof_pos) - self._d_gains * robot_dof_vel
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: Why `+ self._default_dof_pos`?
created_at: 2025-05-08T06:31:21Z
author_association: COLLABORATOR
diff_hunk: @@ -508,19 +585,23 @@ def refresh_render(self) -> None:
         # Refresh cameras and viewer
         self.gym.step_graphics(self.sim)
         self.gym.render_all_camera_sensors(self.sim)
+        # TODO add keyboard callback(mostly likely push v) to stop rendering in render mode
         if not self.headless:
             self.gym.draw_viewer(self.viewer, self.sim, False)
 
     def simulate(self) -> None:
         # Step the physics
         for _ in range(self.scenario.decimation):
+            # if effort mode, use pd controller to get torque
+            if self._effort_mode:
+                self._apply_action()
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: Oh it seems correct. But this implementation is highly inter-dependent. So I highly don't recommend writing in this way since it is vulnerable to bug. Here is why:
- `simulate()` depends on `_apply_action()`
- `_apply_action()` depends on a global `self.actions`. `self.actions` is only updated in `self.set_dof_target()`, so it is fine.
- `_apply_action()` also depends on `self._robot_dof_state`, but what if `self._robot_dof_state` is not updated to latest?
We should avoid pass values through global variable (class attributes is another form of global variable, especially for a huge class). Instead, try passing values through function/method arguments. Meanwhile, try keeping class attributes unchanged once initialized
created_at: 2025-05-08T06:42:47Z
author_association: COLLABORATOR
diff_hunk: @@ -532,6 +604,27 @@ def simulate(self) -> None:
 
         # self.gym.sync_frame_time(self.sim)
 
+    def _apply_action(self):
+        """
+        Compute torque using pd controller for effort actuator and set desire postion for position actuator given action.
+        TODO: 1. integrate controller api
+              1. move action scaled factor into config]
+        """
+        action_scaled = torch.tensor(0.5, device=self.device) * self.actions
+        # effort actions
+        dof_pos = self._robot_dof_state[..., 0]
+        dof_vel = self._robot_dof_state[..., 1]
+        _torques = self._p_gains * (action_scaled + self._default_dof_pos - dof_pos) - self._d_gains * dof_vel
+        self._torques = torch.clip(_torques, -self._torque_limits, self._torque_limits)
+        torques = self._torques.to(torch.float32)
+        self.gym.set_dof_actuation_force_tensor(self.sim, gymtorch.unwrap_tensor(torques))
+
+        # pos actions
+        pos_action = torch.zeros_like(self._torques, device=self.device, dtype=torch.float32)
+        for i in self._gripper_dof_dix:
+            pos_action[:, i] = self.actions[:, i]
+        self.gym.set_dof_position_target_tensor(self.sim, gymtorch.unwrap_tensor(pos_action))
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: They This is to ensure compatibility when using effort mode for controlling the joints robot arm while still allowing control the grippe by built-in position mode, they don't interfere with each other. See [official implementation](https://gist.github.com/DanielTakeshi/2e6a11dc0af1ba26b50857f6ae8ff386), now I  have added the reference link in code annotation.
created_at: 2025-05-08T07:17:59Z
author_association: COLLABORATOR
diff_hunk: @@ -26,3 +26,4 @@ class CloseBoxCfg(RLBenchTaskCfg):
         mode="le",
         radian_threshold=-14 / 180 * math.pi,
     )
+    control_type = "pos"
path: metasim/cfg/tasks/rlbench/close_box_cfg.py
user: login: WeisonWEileen
body: that's correct—I'll remove it
created_at: 2025-05-08T07:25:20Z
author_association: COLLABORATOR
diff_hunk: @@ -508,19 +585,23 @@ def refresh_render(self) -> None:
         # Refresh cameras and viewer
         self.gym.step_graphics(self.sim)
         self.gym.render_all_camera_sensors(self.sim)
+        # TODO add keyboard callback(mostly likely push v) to stop rendering in render mode
         if not self.headless:
             self.gym.draw_viewer(self.viewer, self.sim, False)
 
     def simulate(self) -> None:
         # Step the physics
         for _ in range(self.scenario.decimation):
+            # if effort mode, use pd controller to get torque
+            if self._effort_mode:
+                self._apply_action()
             self.gym.simulate(self.sim)
             self.gym.fetch_results(self.sim, True)
+            self.gym.refresh_dof_state_tensor(self.sim)
 
         # Refresh tensors
         self.gym.refresh_rigid_body_state_tensor(self.sim)
         self.gym.refresh_actor_root_state_tensor(self.sim)
-        self.gym.refresh_dof_state_tensor(self.sim)
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: In manual PD control mode (effort model), the DOF state supplies the joint's current position and velocity - which are inputs for the PD controller in the `for` loop. 
created_at: 2025-05-08T10:00:21Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +226,81 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_EFFORT
+        asset_options.replace_cylinder_with_capsule = True
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: that's correct. move `replace_cylinder_with_capsule` to the sim_params. I can set `default_dof_drive_mode` as `gymapi.DOF_MODE_NONE`, freely move joint, it will be update when reading cfg in actuator cfg.
created_at: 2025-05-08T14:20:10Z
author_association: COLLABORATOR
diff_hunk: @@ -508,19 +585,23 @@ def refresh_render(self) -> None:
         # Refresh cameras and viewer
         self.gym.step_graphics(self.sim)
         self.gym.render_all_camera_sensors(self.sim)
+        # TODO add keyboard callback(mostly likely push v) to stop rendering in render mode
         if not self.headless:
             self.gym.draw_viewer(self.viewer, self.sim, False)
 
     def simulate(self) -> None:
         # Step the physics
         for _ in range(self.scenario.decimation):
+            # if effort mode, use pd controller to get torque
+            if self._effort_mode:
+                self._apply_action()
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: Got it, that makes sense. I was referring to [legged_gym](https://github.com/leggedrobotics/legged_gym/blob/17847702f90d8227cd31cce9c920aa53a739a09a/legged_gym/envs/base/legged_robot.py#L353). now it's improved.
created_at: 2025-05-08T17:05:41Z
author_association: COLLABORATOR
diff_hunk: @@ -532,6 +613,35 @@ def simulate(self) -> None:
 
         # self.gym.sync_frame_time(self.sim)
 
+    def _apply_action(self):
+        """
+        Compute torque using pd controller for effort actuator and set desire postion for position actuator given action.
+        TODO: 1. integrate controller api
+              2. check whether replay demo works when using effort mode
+        """
+        # action_scaled = self._action_scale * self.actions[:, self._obj_num_dof :]
+        action_scaled = self._action_scale * self.actions
+        # effort actions
+        robot_dof_pos = self._robot_dof_state[..., 0]
+        robot_dof_vel = self._robot_dof_state[..., 1]
+        # FIXME  self._default_dof_pos. now default is RL locomotion configuration, set it to 0 for
+        _effort = (
+            self._p_gains * (action_scaled + self._default_dof_pos - robot_dof_pos) - self._d_gains * robot_dof_vel
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: this a general action space configuration for rl locomotion task, see implemtation in [legged_gym](https://github.com/leggedrobotics/legged_gym/blob/17847702f90d8227cd31cce9c920aa53a739a09a/legged_gym/envs/base/legged_robot.py#L368). This configuration will encourage the agent to explore around default pos. Compatibility with other task (say, replay demonstrations, which do not need `+ self._default_dof_pos`)  is fixed now.
created_at: 2025-05-08T17:15:50Z
author_association: COLLABORATOR
diff_hunk: @@ -14,6 +14,7 @@ class BaseActuatorCfg:
     stiffness: float | None = None
     torque_limit: float | None = None
     actionable: bool = True
+    torque_limit: float | None = None
path: metasim/cfg/robots/base_robot_cfg.py
user: login: WeisonWEileen
body: that's correct

created_at: 2025-05-08T17:34:28Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +227,88 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_NONE
+        # Defaults are set to free movement and will be updated based on the configuration in actuator_cfg below.
+        asset_options.replace_cylinder_with_capsule = self.scenario.sim_params.replace_cylinder_with_capsule
         robot_asset = self.gym.load_asset(self.sim, asset_root, robot_asset_file, asset_options)
         # configure robot dofs
+        robot_num_dofs = self.gym.get_asset_dof_count(robot_asset)
+        self._robot_num_dof = robot_num_dofs
+
+        self._action_scale = torch.tensor(self.scenario.action_scale, device=self.device)
+        self._action_offset = self.scenario.action_offset
+        self._effort_mode = (
+            True if self.scenario.control_type == "effort" else False
+        )  # FIXME acuation mode type not complete for all task
+        if self._effort_mode:
+            # manually pd control params
+            self._p_gains = torch.zeros(
+                self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+            )
+            self._d_gains = torch.zeros(
+                self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+            )
+            self._torque_limits = torch.zeros(
+                self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+            )
+
+        # TODO combability for passive joint
+
         robot_dof_props = self.gym.get_asset_dof_properties(robot_asset)
         robot_lower_limits = robot_dof_props["lower"]
         robot_upper_limits = robot_dof_props["upper"]
-        robot_ranges = robot_upper_limits - robot_lower_limits
         robot_mids = 0.3 * (robot_upper_limits + robot_lower_limits)
-        robot_dof_props["driveMode"][:7].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][:7].fill(400.0)
-        robot_dof_props["damping"][:7].fill(40.0)
 
-        # grippers
-        robot_dof_props["driveMode"][7:].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][7:].fill(800.0)
-        robot_dof_props["damping"][7:].fill(40.0)
+        num_actions = 0
+        default_dof_pos = []
+        dof_names = self.gym.get_asset_dof_names(robot_asset)
+
+        for i in range(len(dof_names)):
+            actuator_cfg = self._robot.actuators[dof_names[i]]
+            if not actuator_cfg.is_ee:
+                default_dof_pos_i = (
+                    self._robot.default_joint_positions[dof_names[i]]
+                    if self._robot.default_joint_positions[dof_names[i]] is not None
+                    else robot_mids[i]
+                )
+                default_dof_pos.append(default_dof_pos_i)
+                if self._effort_mode:
+                    # use pd controller to get force/torque
+                    self._p_gains[:, i] = actuator_cfg.stiffness
+                    self._d_gains[:, i] = actuator_cfg.damping
+                    # TODO add 0.85 into config
path: metasim/sim/isaacgym/isaacgym.py
user: login: Fisher-Wang
body: What's the problem with original scale?
created_at: 2025-05-09T00:01:44Z
author_association: COLLABORATOR
diff_hunk: @@ -208,36 +227,88 @@ def _load_robot_assets(self) -> None:
         asset_options.disable_gravity = not self.robot.enabled_gravity
         asset_options.flip_visual_attachments = self.robot.isaacgym_flip_visual_attachments
         asset_options.collapse_fixed_joints = self.robot.collapse_fixed_joints
+        asset_options.default_dof_drive_mode = gymapi.DOF_MODE_NONE
+        # Defaults are set to free movement and will be updated based on the configuration in actuator_cfg below.
+        asset_options.replace_cylinder_with_capsule = self.scenario.sim_params.replace_cylinder_with_capsule
         robot_asset = self.gym.load_asset(self.sim, asset_root, robot_asset_file, asset_options)
         # configure robot dofs
+        robot_num_dofs = self.gym.get_asset_dof_count(robot_asset)
+        self._robot_num_dof = robot_num_dofs
+
+        self._action_scale = torch.tensor(self.scenario.action_scale, device=self.device)
+        self._action_offset = self.scenario.action_offset
+        self._effort_mode = (
+            True if self.scenario.control_type == "effort" else False
+        )  # FIXME acuation mode type not complete for all task
+        if self._effort_mode:
+            # manually pd control params
+            self._p_gains = torch.zeros(
+                self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+            )
+            self._d_gains = torch.zeros(
+                self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+            )
+            self._torque_limits = torch.zeros(
+                self._num_envs, robot_num_dofs, dtype=torch.float, device=self.device, requires_grad=False
+            )
+
+        # TODO combability for passive joint
+
         robot_dof_props = self.gym.get_asset_dof_properties(robot_asset)
         robot_lower_limits = robot_dof_props["lower"]
         robot_upper_limits = robot_dof_props["upper"]
-        robot_ranges = robot_upper_limits - robot_lower_limits
         robot_mids = 0.3 * (robot_upper_limits + robot_lower_limits)
-        robot_dof_props["driveMode"][:7].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][:7].fill(400.0)
-        robot_dof_props["damping"][:7].fill(40.0)
 
-        # grippers
-        robot_dof_props["driveMode"][7:].fill(gymapi.DOF_MODE_POS)
-        robot_dof_props["stiffness"][7:].fill(800.0)
-        robot_dof_props["damping"][7:].fill(40.0)
+        num_actions = 0
+        default_dof_pos = []
+        dof_names = self.gym.get_asset_dof_names(robot_asset)
+
+        for i in range(len(dof_names)):
+            actuator_cfg = self._robot.actuators[dof_names[i]]
+            if not actuator_cfg.is_ee:
+                default_dof_pos_i = (
+                    self._robot.default_joint_positions[dof_names[i]]
+                    if self._robot.default_joint_positions[dof_names[i]] is not None
+                    else robot_mids[i]
+                )
+                default_dof_pos.append(default_dof_pos_i)
+                if self._effort_mode:
+                    # use pd controller to get force/torque
+                    self._p_gains[:, i] = actuator_cfg.stiffness
+                    self._d_gains[:, i] = actuator_cfg.damping
+                    # TODO add 0.85 into config
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: scale down the torque limit is for safety, especially for those task requring sim2real, see implemtation in [humanoidgym](https://github.com/roboterax/humanoid-gym/blob/69e7c9d420161cd63d6a0fe3cdf219c9a8a1bc80/humanoid/envs/custom/humanoid_config.py#L54)
created_at: 2025-05-09T19:14:05Z
author_association: COLLABORATOR
diff_hunk: @@ -498,7 +569,13 @@ def set_dof_targets(self, obj_name: str, actions: list[Action]):
         ]
         action_input[robot_dim_index] = action_array_all.float().to(self.device).reshape(-1)
 
-        self.gym.set_dof_position_target_tensor(self.sim, gymtorch.unwrap_tensor(action_input))
+        if self._effort_mode:
+            #
+            self.actions = action_input.view(self._num_envs, self._robot_num_dof + self._obj_num_dof)[
+                :, self._obj_num_dof :
+            ]
path: metasim/sim/isaacgym/isaacgym.py
user: login: WeisonWEileen
body: it will be called at `simulate()`
created_at: 2025-05-10T07:32:01Z
author_association: COLLABORATOR

PR comments:
user: login: cloudflare-workers-and-pages[bot]
created_at: 2025-05-05T19:27:33Z
body: ## Deploying roboverse-release with &nbsp;<a href="https://pages.dev"><img alt="Cloudflare Pages" src="https://user-images.githubusercontent.com/23264/106598434-9e719e00-654f-11eb-9e59-6167043cfa01.png" width="16"></a> &nbsp;Cloudflare Pages

<table><tr><td><strong>Latest commit:</strong> </td><td>
<code>a5f8996</code>
</td></tr>
<tr><td><strong>Status:</strong></td><td>&nbsp;✅&nbsp; Deploy successful!</td></tr>
<tr><td><strong>Preview URL:</strong></td><td>
<a href='https://ab1e6f12.roboverse-release.pages.dev'>https://ab1e6f12.roboverse-release.pages.dev</a>
</td></tr>
<tr><td><strong>Branch Preview URL:</strong></td><td>
<a href='https://dev-humanoid.roboverse-release.pages.dev'>https://dev-humanoid.roboverse-release.pages.dev</a>
</td></tr>
</table>

[View logs](https://dash.cloudflare.com/39c910255a28356ea8ac6368a2848703/pages/view/roboverse-release/ab1e6f12-6ec8-4bb9-9cdc-b4d398cf94f5)

author_association: NONE
user: login: Fisher-Wang
created_at: 2025-05-05T22:55:04Z
body: When running
```
python roboverse_learn/humanoidbench_rl/train_sb3.py isaacgym
```

I encounter the following error
```
Traceback (most recent call last):
  File "roboverse_learn/humanoidbench_rl/train_sb3.py", line 280, in <module>
    main()
  File "roboverse_learn/humanoidbench_rl/train_sb3.py", line 117, in main
    from metasim.cfg.scenario import ScenarioCfg
  File "/home/fs/cod/RoboVerse/metasim/cfg/scenario.py", line 13, in <module>
    from metasim.utils.setup_util import get_robot, get_scene, get_task
  File "/home/fs/cod/RoboVerse/metasim/utils/setup_util.py", line 12, in <module>
    from metasim.cfg.tasks.base_task_cfg import BaseTaskCfg
  File "/home/fs/cod/RoboVerse/metasim/cfg/tasks/__init__.py", line 9, in <module>
    from .base_task_cfg import BaseTaskCfg
  File "/home/fs/cod/RoboVerse/metasim/cfg/tasks/base_task_cfg.py", line 16, in <module>
    from .sim_param_cfg import SimParamCfg
ModuleNotFoundError: No module named 'metasim.cfg.tasks.sim_param_cfg'
```

Please also include your implementation of `sim_param_cfg` for the sake of testing
author_association: COLLABORATOR
user: login: Fisher-Wang
created_at: 2025-05-09T09:43:19Z
body: @geng-haoran Could you also take a look at this since it's highly related to isaacgym?
author_association: COLLABORATOR
user: login: geng-haoran
created_at: 2025-05-09T19:35:25Z
body: I think it is generally good. One thing I found is that, seems the selection of control mode is all qpos or all effort. But sometimes it might be different, for example, franka, arm is qpos and gripper is effort.
author_association: COLLABORATOR
user: login: geng-haoran
created_at: 2025-05-09T22:49:47Z
body: When I run "python roboverse_learn/humanoidbench_rl/train_sb3.py isaacgym"
It shows: ModuleNotFoundError: No module named 'metasim.cfg.control'
author_association: COLLABORATOR
user: login: WeisonWEileen
created_at: 2025-05-10T05:27:01Z
body: > When I run "python roboverse_learn/humanoidbench_rl/train_sb3.py isaacgym" It shows: ModuleNotFoundError: No module named 'metasim.cfg.control'

that's a commit caused by commit mistake. now it's fixed.
author_association: COLLABORATOR
user: login: WeisonWEileen
created_at: 2025-05-10T07:35:44Z
body: > I think it is generally good. One thing I found is that, seems the selection of control mode is all qpos or all effort. But sometimes it might be different, for example, franka, arm is qpos and gripper is effort.

What if we modularize the gripper configuration of the robot? For example, modularizing the gripper section in Franka's robot configuration file, then assigning control parameters to it independently? 
author_association: COLLABORATOR
user: login: WeisonWEileen
created_at: 2025-05-10T08:56:30Z
body: > > I think it is generally good. One thing I found is that, seems the selection of control mode is all qpos or all effort. But sometimes it might be different, for example, franka, arm is qpos and gripper is effort.
> 
> What if we modularize the gripper configuration of the robot? For example, modularizing the gripper section in Franka's robot configuration file, then assigning control parameters to it independently?

maybe it can be listed as TODOs
author_association: COLLABORATOR
user: login: geng-haoran
created_at: 2025-05-10T09:31:31Z
body: sounds good!
author_association: COLLABORATOR
user: login: Fisher-Wang
created_at: 2025-05-12T22:09:49Z
body: This PR also resolved #2 
author_association: COLLABORATOR