rlgw 0.0.16
Deterministic 2D grid-world for testing reinforcement learning algorithms
License: Apache2.0

# RL-GridWorld
Deterministic 2D grid-world for testing reinforcement learning algorithms.

## Installation

<code> pip install rlgw </code>

## Play

*play game in a pygame window - control using numpad keys(1 to 9)*

<code>
import rlgw.game as game

game.play()
</code>


## Grid-World

A grid-world is a 2D grid of cells. 


![Grid-World](https://github.com/Nelson-iitp/public/raw/main/_temp/a.png)


* An agent/player is initially placed on one of the cells called the *Starting Position*.

* Agent's goal is to move to the *Final Position* by taking a series of actions which be:
	* XX = No Move
	* NN = Move North
	* EE = Move East
	* WW = Move West
	* SS = Move South
	* NE = Move North-East
	* NW = Move North-West
	* SE = Move South-East
	* SW = Move South-West
	
* Actions avaialbe in the environment can be custom defined by choosing from above listed actions
	
* Whenever the agent takes an action, it will move to a new cell and collect the reward marked on that cell.

* Rewards can be both negative and positive. The agent's goal is to move in such a way that maximizes the cumulative reward overtime.

* When the agents collects an reward from a cell, the reward of the cell is replaced by a default 'EMPTY' value (usually -1). So, the next time agents visits this cell (in the same episode) it will get the EMPTY reward.

* The Final Position has a special reward (usually 0).

* The agent stays in the same cell if it tries to move past the boundary (and hence collects EMPTY reward).

![Grid-World](https://github.com/Nelson-iitp/public/raw/main/_temp/b.jpg)
	

## Notes

* The grid-world is deterministic.
* The number of moves that the Agent can make in one episode is upper bounded (to avoid looping).

<hr>

# Creating Custom Environment

* rlgw has only two submodules:
	* model.py - defines core classes **WORLD** and **ENV** which can be used to simulate any grid-world
	* game.py - wrapper for simulation in a pygame window (usually for testing agent's policy after learning)

* To create new environments:

<pre>
<code>

# 0. import WORLD & ENV classes
from rlgw.model import WORLD, ENV 

# 1. Create a WORLD object using WORLD.get_world(...) function, use help(WORLD.get_world)
world = WORLD.get_world(
	R=5,            # Rows
	C=8,            # Cols
	SP=(0,0),       # Starting Position
	EP=(1,5),       # Final Position
	RNDSP=True,     # Randomize start position on reset() ?
	RNDEP=True,     # Randomize end position on reset() ?
	EMPTY=-1,       # Empty reward
	RRN=[-5,-5],    # Range of Negative Rewards
	RRP=[5,5],      # Range of Positive Rewards
	RD=0.5,         # Reward Density (prob of a cell not being empty)
	PRD=0.5,        # Positive Reward Density (prob that a non-empty cell has positive reward)
	ER=0,           # Special End Reward
	MAXTS=5*8*2,    # Maximum steps per episode (should be at least 2x number of cells)
	SID=None,       # Initial State distribution  
	name="some.world")


# 2. Choose allowed actions <--- can choose from ('XX, 'NN','EE','WW','SS', 'NE', 'NW', 'SE', 'SW')
actions = ('NN','EE','WW','SS') # actual actions are integers - so the order here matters

# 3. Penalty can be Imposed by using Penalty=(True, p);  
#     ... where a pentaly of p times the EMPTY reward value is imposed on reward
penalty = (False, 0),  #<--- if True, imposes penalty on agent for staying in the same cell

# 4. Select reset type
always_reinit = True #<--- if True, always choose a new random initial state from distribution

# 5. Now use the WORLD object and other params to create an ENV object 
env = ENV(world = world, actions=actions, penalty=penalty, always_reinit=always_reinit, name="some.env")  

# 6. Visulaize
env.render(mode=1)	# use mode=0 for text-based render

# 7. Read environment state 
state = env.get_state()
print('state:',state)

# 8. Step through environment
action = env.action_space.sample()
next_state, reward, done, info = env.step(action)

# 9. Visulaize again
env.render(mode=1)	# use mode=0 for text-based render

# 10. Reset when episode complete (or before starting new episode)
state = env.reset()

</code>
</pre>

# Playing Custom Environment

* To play in a pygame window:

<pre>
<code>
from rlgw.game import GAME
import threading

# create game using 'env' created before
game = GAME(env=env, print_scores=True)
	
# play game in a new thread
th = threading.Thread(target=game.main)
th.start()
th.join()
print('Done!')
	
</code>
</pre>


## Information

* Author: Nelson Sharma
* mailto: nelson_2121CS07@iitp.ac.in

<hr>