Package: rewards
rewardsA low code sdk for crearing custom environments and deep RL agents.Installation[linux]Installingrewardsis easy in linux. First clone the repository by runninggitclonehttps://github.com/rewards/rewards.gitOne cloned go to the repository and make suremakeis installed. If not installed just run:sudoaptinstallcmakeOnce done, now create a new virtual environment and install dependencies. You can achieve
this by running the following:makevirtualenv
makeinstallThis should install all the dependencies and our sdkrewards:v1.0.0.[windows]For installation in windows, it's also simple. All you have to do is just clone the repository same as before. Then create a new virtual environment.virtualenv.venvLoad the virtual environment.\venv\Scripts\ActivateNow go to the repository and install all the dependencies and therewardss package.pipinstall-rrequirements.txt
pythonsetup.pyinstallGetting startedrewardsis mainly made for two important reasons.First we want to make learning reinforcement learning easy, by introducing this low code framework. So that folks do not need to spend more time in making environments or other stuff. All they can focus is on creating different agents, models and expeiment with them.We want to make it as interactive and begginer friendly as possible. So we are also introducingrewards-platformwhich where we gamified the experience of learning RL.If playing games can be fun and competitive then why not RL? Hence withrewards-platformandrewardsyou can host and join ongoing competition and learn RL with your friends.NOTE: Our coming enterprise version is mainly focussed to build the same but for RL/Robotics based
companies where we want to ensure that their focus lies more on the research rather creating environments and other configurations.Take a look on how to get started with a sample experimentCurrently this version ofrewardsonly supports a single game and environment. That iscar-race. We will be adding support for more environments (including gym, unity, and custom environments) very soon.So let's go ahead and see how to get started with a sample experiment.fromrewardsimportworkflowconfigs=workflow.WorkFlowConfigurations(EXPERIMENT_NAME="Exp 3",MODE="training",LAYER_CONFIG=[[5,64],[64,3]])flow=workflow.RLWorkFlow(configs)flow.run_episodes()First you call our sdk'sworkflowmodule. The workflow module helps us toCreate Environments and configure environmentsCreate models and configure themRun the whole experiment and log all the resultsAll at one place. We first get started with writing our own configuration usingconfigs=workflow.WorkFlowConfigurations(EXPERIMENT_NAME="Exp 3",MODE="training",LAYER_CONFIG=[[5,64],[64,3]])Here is the table of configuration and what they meansConfiguration NameTYPEWhat it doesDefault valueOptionsEXPERIMENT_NAMEstrIt tells what is the name of the experiment. The name of the experiment will be logged inside user's weights and biases projects dashbord.sample RL experimentany stringENVIRONMENT_NAMEstrIt states the name of the environment.rewards:v1.0.0only supports one environment for now and that iscar-race.car-raceNULLENVIRONMENT_WORLDintAccording to our convention we keep some environments for training and some for testing (which are unseen). At one point of time, you can only train your agent on one single train environment.10/1/2MODEstrThis tells us which mode the agent is been running i.e. either in train or test mode.trainingtraining/testingCONTROL SPEEDfloatFor our car environment user can set the control speed of the car environment.0.05(0 - 1]TRAIN_SPEEDintFor our car environment user can set the control speed of the car environment.1001 - 100SCREEN_SIZETupleThe size of the pygame window.(800, 700)User' choiceLRfloatLearning rate0.01User' choiceLOSSstrLoss function namemsemse , rmse, maeOPTIMIZERstrOptimizer nameadamadam, rmsprop, adagradGAMMAfloatHyper parametergammevalue0.990 - 1EPSILONfloatHyper parameterepsilonvalue0.990 - 1LAYER_CONFIGList[List[int]]This expects a list of list. Where the inner list will have only two values [input neurons, output neurons]. This configuration will help us to build the neural network for our agent. The first value for the current environment must be 3.[[5, 64], [64, 3]]Here user can add more values but the values5in the first and3in the last must be fixed for this current environment that we are supporting. Example:[[5, ...], [..., ...], ...., [..., 3]],Where...can be any value. We recommend to keep it between (1 - 256)CHECKPOINT_FOLDER_PATHstrThe model checkpoint path from where it should be loaded. This can be eitherNonethen it will auto create a checkpoint path and store all the checkpoints there else it will save the models on the folder mentioned if exists../saved_modelsUser's choiceCHECKPOINT_MODEL_NAMEstrThe name of the model name. This can be either named by the user or by default it will create the model name asmodel_{<latest_date_and_time>}_.pth.model_2023-04-07 16:10:36.366395_.pth(This is just an example)User's choiceREWARD_FUNCTIONCallableUsers are expected to write some reward function (Callable) and then have to use this reward function for agent's training.def default_reward_function(props): if props["isAlive"]: return 1 return 0User's choicesome important parametersisAliverepresents whether the car is alive or not. So on that basis we can penalize our agent.obsThe car's radar's oberservations values. (more on documentation)rotationVelCar's rotational velocity value (more on documentation)So above is a quick overview of how to use different reward configurations. Now once configuration part is done, load those configuration toRLWorkFlow()and run for a single episodes.After this you are ready to run the above code:fromrewardsimportworkflowconfigs=workflow.WorkFlowConfigurations(EXPERIMENT_NAME="Exp 3",MODE="training",LAYER_CONFIG=[[5,64],[64,3]])flow=workflow.RLWorkFlow(configs)flow.run_episodes()Here you will be able to see the game, and a very nice dashboard with all the runs and configurations and nice graphs. Stay tuned withrewards.aifor further updates, documentation and examples.