Package: ludus
LudusLudus is a reinforcement learning library for expediating development of RL applications and research. Ludus is still in an alpha development stage, so new algorithms and models are continuously being added with the growth of the framework. You can find many state-of-the-art algorithms implemented and ready to use. Additionly, the platform has ready to go integration with popular training environments like OpenAI'sgym. The high level API of Ludus combined with easily accesible and well documented, open source code allows for both efficiency and control.Getting StartedThese instructions will help you quickly get Ludus up and running, ready for RL application. Ludus is built on Python 3, and there is no gaurantee it will work with Python 2.PrerequisitesLudus requires several libraries to get started. Anaconda 3 is recommended as it contains most of the required libraries, as well as many related libraries that may be useful. Nearly all prerequisites are installed when doing the standard pip installation as described below. The exeption is Tensorflow. WhileGPU enabled Tensorflow is strongly recommendedover the CPU version, it is not required. You can find a guide to installing GPU enabled Tensorflowhere, and CPU Tensorflowhere.InstallingThe recommended method of installation is with the command,pip install ludus.Alternatively, the package can be installed by cloning the repository and running,pip setup.py installin the root directory of the project.Your First Ludus AgentTo get started, thevpg_cartpole examplesteps through the creation and training process for a simple agent. It is recommended that you use the notebook as an initial testing ground, and a template for other agents.In Ludus, the process of creating an intelligent agent can be divided into 3 major steps. Performing the 1st step, and then repeating steps 2 and 3 in a training loop is the typical program flow:Creating input networksDepending on the type of trainer you wish to use to train your agent, a variety of different input neural networks may be required. The simplest form of this in Vanilla Policy Gradient (VPG /VPGTrainer), which requires only one network that maps observations to actions. Other, more complex methods like Proximal Policy Optimization (PPO /PPOTrainer) require two networks, one choosing actions and another estimating state values. If you are not familiar with these concepts, It is recommended that you thoroughly looking through the examples, as they are consice and easy to work with.Environment simulation & data gatheringAnEnvControllerinstance is created and used to gather data from the environment. By adujsting then_threadsargument to an integer greater than 1, you can run multiple environments in parallel. Thesim_episodesfunction is used to simualte the environment and gather data. Because Ludus handles environment simulation and data collection for you, custom environments can easily be integrated into the environment so long as they conform to a specific format (more on thishere). Once data has been gathered through an instance ofEnvController, retrieving the data can be done with aget_data()call to the instance.TrainingTraining one epoch on the data is as simple as callingnetwork.train(ec.get_data()), where network is an instance of a child ofBaseTrainer(likeVPGTrainer), andecis an instance ofEnvController. After aget_data()call, the training data memory buffer will be reset, unless otherwise in the function parameters.Using a Custom Game EnvironemntWhile this feature is supported and easy to implement, the documentation is not yet complete. For the time being, it is recommended that you examine OpenAI'sgym. Creating an environment with the samereset,step, and initialization functions (the same input arguments and return values) will work with the Ludus framework.Built Withnumpy- Efficient mathematical operationsopencv2- Image manipulation in 2D environmentsTensorflow- Creating and training neural networksgym- Envronments for training agentsDocumentationFurther, more in depth documentation is in the works, although not ready quite.AuthorsEdan Meyer(ejmejm) is currently the lead and only developer for the project.LicenseThis project is licensed under the MIT License. Please see the attached license file for more details.