Package: x-magical
x-magicalx-magicalis a benchmark extension ofMAGICALspecifically geared towardscross-embodiment imitation. The tasks still provide the Demo/Test structure that allows one to evaluate how well imitation or reward learning techniques can generalize the demonstrator's intent to substantially different deployment settings, but there's an added axis of variation focusing on how well these techniques can adapt to systematic embodiment gaps between the demonstrator and the learner. This is a challenging problem, as different embodiments are likely to use unique and suitable strategies that allow them to make progress on a task.Embodiments in anx-magicaltask must still learn the same set of general skills like 2D perception and manipulation, but they are specifically designed such that they solve the task in different ways due to differences in their end-effector, shapes, dynamics, etc. For example, in the sweeping task, some agents can sweep all debris in one motion while others need to sweep them one at a time. These differences in execution speeds and state-action trajectories pose challenges for current LfD techniques, and the ability to generalize across embodiments is precisely what this benchmark evaluates.x-magicalis under active development - stay tuned for more tasks and embodiments!Tasks, Embodiments and VariantsEach task inx-magicalcan be instantiated with a particularembodiment, which changes the nature of the robotic agent. Additionally, the task can be instantiated in a particularvariant, which changes one or more semantic aspects the environment. Both axes of variation are meant to evaluate combinatorial generalization. We list the task-embodiment pairings below, with a picture of the initial state of theDemovariant:TaskDescriptionSweepToTop: The agent must sweep all three debris to the goal zone shaded in pink.Embodiments:Gripper,Shortstick,MediumStick,Longstick.Variants: all exceptJitter.Here is a description (source) of what each variant modifies:VariantDescriptionDemoThe default variant with no randomization,i.e.the same initial state acrossreset().JitterThe rotations and orientations of all objects, and the size of goal regions, are jittered by up to 5% of the maximum range.LayoutPositions and rotations of all objects are completely randomized. The definition of what constitutes an "object" is task-dependent,i.e.some tasks might not randomize the pose of the robotic agent, just the pushable shapes.ColorThe color of blocks and goal regions is randomized, subject to task-specific constraints.ShapeThe shape of pushable blocks is randomized, again subject to task-specific constraints.DynamicsThe mass and friction of objects are randomized.AllAll applicable randomizations are applied.Usagex-magicalenvironments are available in the Gym registry and can be constructed via string specifiers that take on the form<task>-<embodiment>-<observation_space>-<view_mode>-<variant>-v0, where:task: The name of the desired task. See above for the full list of available tasks.embodiment: The embodiment to use for the robotic agent. See above for the list of supported embodiments per task.observation_space: Whether to use pixel or state-based observations. All environments support pixel observations but they may not necessarily provide state-based observation spaces.view_mode: Whether to use an allocentric or egocentric agent view.variant: The variant of the task to use. See above for the full list of variants.For example, here's a short code snippet that illustrates this usage:importgymimportxmagical# This must be called before making any Gym envs.xmagical.register_envs()# List all available environments.print(xmagical.ALL_REGISTERED_ENVS)# Create a demo variant for the SweepToTop task with a gripper agent.env=gym.make('SweepToTop-Gripper-Pixels-Allo-Demo-v0')obs=env.reset()print(obs.shape)# (384, 384, 3)env.render(mode='human')env.close()# Now create a test variant of this task with a shortstick agent,# an egocentric view and a state-based observation space.env=gym.make('SweepToTop-Shortstick-State-Ego-TestLayout-v0')init_obs=env.reset()print(init_obs.shape)# (16,)env.close()Installationx-magicalrequires Python 3.8 or higher. We recommend using anAnacondaenvironment for installation. You can create one with the following:condacreate-nxmagicalpython=3.8
condaactivatexmagicalInstalling PyPI releasepipinstallx-magicalInstalling from sourceClone the repository and install in editable mode:gitclonehttps://github.com/kevinzakka/x-magical.gitcdx-magical
pipinstall-rrequirements.txt
pipinstall-e.ContributingIf you'd like to contribute to this project, you should install the extra development dependencies as follows:pipinstall-e.[dev]AcknowledgmentsA big thank you to Sam Toyer, the developer ofMAGICAL, for the valuable help and discussions he provided during the development of this benchmark. Please consider citing MAGICAL if you find this repository useful:@inproceedings{toyer2020magical,
  author    ={Sam Toyer and Rohin Shah and Andrew Critch and Stuart Russell},
  title     ={The{MAGICAL}Benchmark for Robust Imitation},
  booktitle ={Advances in Neural Information Processing Systems},
  year      ={2020}}Additionally, we'd like to thank Brent Yi for fruitful technical discussions and various debugging sessions.