Package: bbrl
BBRLbbrl- A Flexible and Simple Library for Reinforcement Learning deriving from SaLinABBRL stands for "BlackBoard Reinforcement Learning". Initially, this library was a fork ofthe SaLinA library.
But SaLinA is a general model for sequential learning whereas BBRL is dedicated to RL, thus it focuses on a subset of SaLinA.
Morevover, BBRL is designed for education purpose (in particular, to teach various RL algorithms, concepts and phenomena).
Thus the fork slowly drifted away from SaLinA and became independent after a few months, even if some parts of the code are still inherited from SaLinA.TL;DR.bbrlis a lightweight library extending PyTorch modules for developpingReinforcement LearningmodelsIt supports simultaneously training with AutoReset on multiple environmentsIt works on multiple CPUs and GPUsCitingbbrlBBRL being inspired from SaLinA, please use this bibtex if you want to cite BBRL in your publications:Link to the paper:SaLinA: Sequential Learning of Agents@misc{salina,
        author = {Ludovic Denoyer, Alfredo de la Fuente, Song Duong, Jean-Baptiste Gaya, Pierre-Alexandre Kamienny, Daniel H. Thompson},
        title = {SaLinA: Sequential Learning of Agents},
        year = {2021},
        publisher = {Arxiv},
        howpublished = {\url{https://github.com/facebookresearch/salina}},
    }Quick StartJust clone the repopip install -e .NewsMay-June 2023:Integrated the use of gymnasium. Turned google colab notebooks into colab compatible jupyter notebooks. Refactored all the notebooks.August 2022:Major updates of the notebook-based documentationMay 2022:First commit of the BBRL repositoryMarch 2022:Forked SaLinA and started to modify the modelDocumentationMain differences to SaLinABBRL only contains core classes to implement RL algorithms.Because both notations coexist in the literature, the GymAgent classes support the case where doing action $a_t$ in state $s_t$ results in reward $r_t$, and the case where it results in reward $r_{t+1}$.Some output string were corrected, some variable were renamed and some comments were improved to favor code readability.A few small bugs in SaLinA were fixed:The replay buffer was rejecting samples that did not fit inside when the number of added samples was beyond the limit. This has been corrected to implement the standard FIFO behavior of replay buffer.When using autoreset=True and no replay buffer, transitions from an episode to the next were considered as standard steps in an episode. We added a mechanism to properly filter them out, using an additionalget_transitions()function in the Workspace class.Starting with notebooksThere are two kinds of notebooks:some are just for reading, they explain the main concepts and design choices behind BBRLsome are designed for learning about RL algorithms and practicing with BBRL. Ideally, these notebooks should be studied in the provided order.Explanatory notebooksGetting started: TheBBRL modelExplanations about usingautoreset=FalseExplanations about usingautoreset=TrueDifferent Reward Indexing SchemesDealing with Time LimitsCoding your first RL algorithms with BBRLMost of the notebooks below can be run under jupyter notebook as well as under Google colaboratory. In any case, download it on your disk and run it with your favorite notebook environment.A simple agent-environment interaction codeBuilding a simple Neural RL agent in interaction with a gym environmentCoding DQN using autoreset=FalseCoding DQN using autoreset=TrueCoding DPPG and TD3 using autoreset=TrueCoding basic Policy Gradient algorithms and REINFORCEA basic version of the A2C algorithmA more advanced version of the A2C algorithmThe KL penalty version of PPOThe clipped version of PPOCoding SACCoding TQCLearning RL withbbrlin your favorite coding environmentHave a look at thebbrl_exampleslibrary.Code Documentation:Generated with pdocDevelopmentSeecontributingDependenciesbbrlutilizespytorch,hydrafor configuring experiments, andgymorgymnasiumfor reinforcement learning algorithms.Licensebbrlis released under the MIT license. SeeLICENSEfor additional details about it.