Package: skinner
skinnerSkinner, a new framework of reinforcement learning by PythonIt is built for the beginner of RL.It is under development, the APIs are not designed perfectly, but runs stably. For grid worlds, it is mature enough.Enjoyskinner!RequrementsgymnumpyDownloaddownload from github, or pypi by pip commandpip install skinner.DesignWe consider theobserver design pattern. The env and agents in it observe each other generally. The agents observe the env to how to act and got the reward, env observe the agents and other objects to render the viewer and record the information.Featureso easyUseQuick startrundemo.pyin examples. There are other examples:demo1.py, demo2.py.Also, one could watch animations inbilibiliExamplesThe author make 3 examples. users are suggested to review the codes. Define objects inobjects.py, define new envs insimple_grid.pythen write a demonstration programming in a script (seedemo.py).Define envsIf you just want to build a simple env, then the following is an option, a grid world.#!/usr/bin/env python3# -*- coding: utf-8 -*-"""Demo of RLAn env with some traps and a gold."""fromskinnerimport*fromgym.envs.classic_controlimportrenderingfromobjectsimport*classMyGridWorld(GridMaze,SingleAgentEnv):"""Grid worldA robot playing the grid world, tries to find the golden (yellow circle), meanwhileit has to avoid of the traps(black circles)Extends:GridMaze: grid world with wallsSingleAgentEnv: there is only one agent"""# configure the env# get the positions of the objects (done automatically)CHARGER=...TRAPS=...DEATHTRAPS=...GOLD=...def__init__(self,*args,**kwargs):super(MyGridWorld,self).__init__(*args,**kwargs)self.add_walls(conf['walls'])self.add_objects((*traps,*deathtraps,charger,gold))# Define the condition when the demo of rl will stop.defis_terminal(self):returnself.agent.positioninself.DEATHTRAPSorself.agent.position==self.GOLDorself.agent.power<=0defis_successful(self):returnself.agent.position==self.GOLD# Following methods are not necessary, that only for recording the process of rldefpost_process(self):ifself.is_successful():self.history['n_steps'].append(self.agent.n_steps)else:self.history['n_steps'].append(self.max_steps)self.history['reward'].append(self.agent.total_reward)self.agent.post_process()defpre_process(self):self.history['n_steps']=[]self.history['reward']=[]defend_process(self):importpandasaspddata=pd.DataFrame(self.history)data.to_csv('history.csv')Configure env and its objectsseeconf.yamlfor an example. The object classes would be defined inobjects.py.# Grid Maze:# n_cols * n_rows: size of the maze, the number of squares# edge: the length of the edge of each square# walls: the positions of walls as the components of the environment## number of gridsn_cols:7n_rows:7## size of every gridedge:80## positions of wallswalls:!!set{!!python/tuple[2,6],!!python/tuple[3,6],...!!python/tuple[4,2]}## objects in environment (excluding the agent)## traps, not terminaltraps:!!python/object:objects.ObjectGroupname:'traps'members:-!!python/object:objects.Trapposition:!!python/tuple[3,5]color:[1,0.5,0]size:30-!!python/object:objects.Trapposition:!!python/tuple[1,3]color:[1,0.5,0]size:30-!!python/object:objects.Trapposition:!!python/tuple[7,1]color:[1,0.5,0]size:30## deathtraps, terminaldeathtraps:!!python/object:objects.ObjectGroupname:'deathtraps'members:-!!python/object:objects.DeathTrapposition:!!python/tuple[6,5]color:[.8,0,0.5]size:35-!!python/object:objects.DeathTrapposition:!!python/tuple[2,1]color:[.8,0,0.5]size:35## gold, terminalgold:!!python/object:objects.Goldname:'gold'position:!!python/tuple[7,7]color:[1,0.8,0]size:30Define objectsthe shape of object (circle by default)the method to plot (don't override it, if the shape is simple)class_Object(Object):props=('name','position','color','size')default_position=(0,0)# set default value to help you reducing the codes when creating an objectclassGold(_Object):defdraw(self,viewer):'''this method is the most direct to determine how to plot the objectYou should define the shape and coordinate'''...classCharger(_Object):defcreate_shape(self):'''redefine the shape, here we define a squre with edges length of 40.The default shape is a circle'''a=20self.shape=rendering.make_polygon([(-a,-a),(a,-a),(a,a),(-a,a)])self.shape.set_color(*self.color)Define agentstransition function $f(s,a)$reward function $r(s,a,s')$fromskinnerimport*classMyRobot(StandardAgent):actions=Discrete(4)# define the shapesize=30color=(0.8,0.6,0.4)def_reset(self):# define the initial state...def_next_state(self,state,action):"""transition function: s, a -> s'"""...def_get_reward(self,state0,action,state1):"""reward function: s,a,s'->r"""...# define parametersagent=MyRobot(alpha=0.3,gamma=0.9)Examplecodessee scripts inexamplesresultsCommemorationIn memory ofB. F. Skinner(1904-1990), a great American psychologist. The RL is mainly inspired by his behaviorism. There are many contributors in the history of behaviorist psychology, he may be the most famous one.