SkillBlender RL#

We provide implementent SkillBlender into our framework.

RL algorithm: PPO by rsl_rl v1.0.2

RL learning framework: hierarchical RL

Simulator: IsaacGym

Installation#

pip install -e roboverse_learn/rl/rsl_rl

Training#

  • IssacGym:

    python3 roboverse_learn/skillblender_rl/train_skillblender.py --task "skillblender:Walking" --sim "isaacgym" --num_envs 1024 --robot "h1_wrist" --use_wandb

    after training around a few minuts for task skillblender:Walking and skillblender:Stepping, you can see like this. To speed up training, click the IsaacGym viewer and press V to stop rendering.

Task list#

4 Goal-Conditional Skills

  • Walking

  • Squatting

  • Stepping

  • Reaching

8 Loco-Manipulation Tasks

  • FarReach

  • ButtonPress

  • CabinetClose

  • FootballShoot

  • BoxPush

  • PackageLift

  • BoxTransfer

  • PackageCarry

Robots supports#

  • h1

  • g1

  • h1_2

Todos#

  • ground type selection

  • pushing robot

  • sim2sim

How to add new Task#

  1. Create a new wrapper.py in , add reward function define your reward functions in reward_fun_cfg.py, check whether the current states is enough for reward computation. If not, parse your state as follow:

    def _parse_NEW_STATES(self, envstate):
    """NEWSTATES PARSEING..."""

def _parse_state_for_reward(self, envstate):
    super()._parse_state_for_reward(self, envstate):
    _parse_NEW_STATES(self, envstate)

  2. Implemented _compute_observation()

    • fill obs and privelidged_obs.

    • modified _post_physics_step to reset variables you defined with reset_env_idx

  3. Add Cfg for your task metasim/cfg/tasks/skillblender

References and Acknowledgements#

We implement SkillBlender based on and inspired by the following projects: