Package: il-datasets
IL DatasetsHi, welcome to the Imitation Learning (IL) Datasets.
Something that always bothered me a lot was how difficult it was to find good weights for an expert, trying to create a dataset for different state-of-the-art methods, and also having to run all methods due to no common datasets.
For these reasons, I've created this repository in an effort to make it more accessible for researchers to create datasets using experts from theHugging Face.
IL-Datasets provides teacher weights for different environments, a multi-threading solution for creating datasets faster, datasets for a set of environments (check the bottom of this document to see which environments are already released), and a benchmark for common imitation learning methods.
We hope that by releasing these features, we can make the barrier to learning and researching imitation learning more accessible.This project is under development. If you are interested in helping, feel free to contactme.RequirementsThe project supports Python versions3.8~3.11.
All requirements for theimitation_datasetspackage are listed inrequirements.txt. These requirements are required by the package and are installed together with theIL-Datasets.
For requirements to use thebenchmarkpackage, use both theimitation_datasetsrequirements and the ones listed inbenchmark.txt.
Development requirements are listed atdev.txt. We do not recommend using these dependencies outside development. They use an outdated version from gymv0.21.0to test theGymWrapperclass.InstallThe package is available on PyPi:# Stable versionpipinstallil-datasetsBut if you prefer, you can install it from the source.gitclonehttps://github.com/NathanGavenski/IL-Datasets.gitcdIL-Datasets
pipinstall-e.How does it work?This project also works with multithreading, which should accelerate the dataset creation.
It consists of oneControllerclass, which requires two different functions to work: (i) aenjoyfunction (for the agent to play and record an episode); and a (ii)collatefunction (for putting all episodes together).Theenjoyfunction will receive 3 parameters and return 1:"""Args:path (str): where the episode is going to be recordedexperiment (Context): A class for recording all information (if you don't want to use `print` - keeping the console clear)expert (Policy): A model based on the [StableBaselines3](https://stable-baselines3.readthedocs.io/en/master/) `BaseAlgorithm`.Returns:status (bool): Whether it was successful or not"""Obs: To use the model you can callpredict, the policy class already has the correct form of using it (a.k.a., how the StableBaselines3 uses).Thecollatefunction will receive 2 parameters and return 1:"""Args:path (str): Where it should save the final datasetepisodes  (list[str]): A list of paths for each fileReturns:status (bool): Whether it was successful or not"""Default functionsTheimitation_datasetspackage also comes with a set of default functions, so you don't need to implement aenjoyand acollatefunction in every project.
The resulting dataset will be aNpzFilewith the following data:"""Data:obs (list[list[float]): gym environment observation. Size [steps, observations space].actions (list[float]): agent action. Size [steps, action] (1 if single action, n if multiple actions).rewards (list[int]): reward from the action with the observations (e.g., r(obs, action)). Size [steps, ].episode_returns (list[float]): accumulated reward for each episode. Size [number of peisodes, ].episode_starts (list[bool]): whether the episode started at the current observation. Size [steps, ]."""A small functional example of how to use the given functions:# python <script> --game cartpole --threads 4 --episodes 1000 --mode allfromimitation_datasets.functionsimportbaseline_enjoy,baseline_collatefromimitation_datasets.controllerimportControllerfromimitation_datasets.argsimportget_argsargs=get_args()controller=Controller(baseline_enjoy,baseline_collate,args.episodes,args.threads)controller.start(args)This script will use the pre-registeredCartPole-v1environment with the HuggingFace weights and create ateacher.npzdataset file in./dataset/cartpole/teacher.npz.Registered environmentsIL-Datasets comes with some already registered weights from HuggingFace.
To check which environments are already registered, check under thesrc.imitation_datasets.registersfolder.Registering new expertsIf you would like to add new experts locally, you can call theExpertsclass. It uses the following structure:"""Args:identifier (str): Name for calling the expert (e.g., cartpole).Policy (Policy): a dataclass with:name (str): Gym environment namerepo_id (str): HuggingFace repo identificationfilename (str): HuggingFace weights file namethreshold (float): How much reward should the episode accumulate to be considered goodalgo (BaseAlgorithm): The class from StableBaselines3"""If not using StableBaselines, you can load aPolicyand not call theload()function (which downloads weights from HuggingFace).
Moreover, the expert has to have apredictfunction that receives:"""Args:obs (Tensor): current environment statestate (Tensor): Model's internal statedeterministic (bool): if it should explore or not."""DatasetsThe IL-Datasets also come with a default PyTorch dataset, calledBaselineDataset. It uses the pattern set by thebaseline_collatefunction, and it allows the use of HuggingFace datasets created by thebaseline_to_huggingfacefunction.
The dataset list for benchmarking is under development, so to check all new versions, you can visit our collection onHuggingFace.To use the Baseline dataset, you can use a local file:fromsrc.imitation_datasets.datasetimportBaselineDatasetBaselineDataset(f"./dataset/cartpole/teacher.npz")Or a HuggingFace path:fromsrc.imitation_datasets.datasetimportBaselineDatasetBaselineDataset(f"NathanGavenski/CartPole-v1",source="huggingface")Finally, the dataset allows for fewer episodes and splitting forevaluationandtrain.fromsrc.imitation_datasets.datasetimportBaselineDatasetdataset_train=BaselineDataset(f"NathanGavenski/CartPole-v1",source="huggingface",n_episodes=100)dataset_eval=BaselineDataset(f"NathanGavenski/CartPole-v1",source="huggingface",n_episodes=100,split="eval")BenchmarkLast but not least, IL-Datasets comes with its own benchmarking.
It uses IL methods on already published datasets to provide consistent results for researchers who also use our datasets.
Currently, we support:AlgorithmImplementationBenchmarkBehavioural Cloningbenchmark.methods.bc✅Behavioural Cloning from Observationbenchmark.methods.bco✅However, our plan is to implement more state-of-the-art methods.You can check the current benchmark results atbenchmark_results.md.This repository is under developmentHere is a list of the upcoming releases:Create actual documentationBenchmark methodsBehavioural CloningBehavioural Cloning from ObservationImitating Latent Policies from ObservationAugmented Behavioural Cloning from ObservationImitating Unkown Policies via ExplorationGenerative Adversarial Imitation LearningGenerative Adversarial Imitation Learning from ObservationOff-Policy Imitation Learning from ObservationsModel-Based Imitation Learning From Observation AloneBenchmark environmentsCartPole-v1MountainCar-v0Acrobot-v1LunarLander-v2Ant-v3Hopper-v3HalfCheetah-v3Walker-v3Humanoid-v3Swimmer-v3Although there are a lot of environments and methods to go through, this repository will be considered done once the documentation and the installation of the benchmarks are done. We don't have a plan for releases for environments and methods yet.If you like this repository, be sure to check my other projects:Development-basedAn easy to use Wrapper for TensorboardA watcher for python to facilitate development of small projectsAcademicSelf-Supervised Adversarial Imitation Learning (IJCNN)How Resilient are Imitation Learning Methods to Sub-Optimal Experts? (BRACIS)Self-supervised imitation learning from observation (MSc dissertation)Imitating Unknown Policies via Exploration (BMVC)Augmented behavioral cloning from observation (IJCNN)