Hello, this is my question Description
-------------------------------Welcome to the third assignment for this course. The goal of this assignment is to
acquire experience in defining and solving a reinforcement learning (RL) environment,
following Gym standards.
The assignment consists of three parts. The first part focuses on defining an
environment that is based on a Markov decision process (MDP). In the second part, we
will apply a tabular method SARSA to solve an environment that was previously
defined. In the third part, we apply the Q-learning method to solve a grid-world
environment.
Part I: Define an RL Environment [30 points]
In this part, we will define a grid-world reinforcement learning environment as an MDP.
While building an RL environment, you need to define possible states, actions, rewards
and other parameters.
STEPS:
1. Choose a scenario for your grid world. You are welcome to use the
visualizationdemo as a reference to visualize it.
An example of idea for RL environment:
• Theme: Lawnmower Grid World with
batteries as positive rewards and rocks as
negative rewards.
• States: {S1 = (0,0), S2 = (0,1), S3 = (0,2),
S4 = (0,3), S5 = (1,0), S6 = (1,1), S7 =
(1,2), S8 = (1,3), S9 = (2,0), S10 = (2,1),
S11 = (2,2), S12 = (2,3), S13 = (3,0), S14 =
(3,1), S15 = (3,2), S16 = (3,3)}
• Actions: {Up, Down, Right, Left}
• Rewards: {-5, -6, +5, +6}
• Objective: Reach the goal state with
maximum reward
2. Define an RL environment following the scenario that you chose.
Environment requirements:
• Min number of states: 12
• Min number of actions: 4
• Min number of rewards: 4
Environment definition should follow the OpenAI Gym structure, which includes
thebasic methods. You can use the “Defining RL env” demo as a base code.
def init:
# Initializes the class
# Define action and observation space
def step:
# Executes one timestep within the environment
# Input to the function is an action
def reset:
# Resets the state of the environment to an initial state
def render:
# Visualizes the environment
# Any form like vector representation or visualizing
usingmatplotlib is sufficient
3. Run a random agent for at least 10 timesteps to show that the environment logic
is defined correctly. Print the current state, chosen action, reward and return your
grid world visualization for each step.
In your report for Part I:
1. Describe the environment that you defined. Provide a set of actions, states,
rewards, main objective, etc.
2. Provide visualization of your environment.
3. Safety in AI: Write a brief review (∼ 5 sentences) explaining how you ensure the
safety of your environment. E.g. how do you ensure that the agent chooses only
actions that are allowed, that agent is navigating within defined state-space, etc -------------------This is my answer -------------------------------------------------
import numpy as np
import gym
from gym import spaces
import matplotlib.pyplot as plt

class LawnmowerGridWorld(gym.Env):
def init(self):
# Define action and observation space
self.observation_space = spaces.Discrete(16)
self.action_space = spaces.Discrete(4)
self.state_matrix = np.zeros((4, 4))

# Define rewards
self.rewards = {-5: ‘Rock’, -6: ‘Rock’, 5: ‘Battery’, 6: ‘Battery’}

# Randomly initialize rewardable slots
for _ in range(4): # 4 rewards
x, y = np.random.randint(0, 4, 2)
reward = np.random.choice(list(self.rewards.keys()))
self.state_matrix[x, y] = reward

self.agent_pos = (0, 0)
self.goal = (3, 3)

def step(self, action):
x, y = self.agent_pos

if action == 0: # Up
x = max(0, x - 1)
elif action == 1: # Down
x = min(3, x + 1)
elif action == 2: # Right
y = min(3, y + 1)
else: # Left
y = max(0, y - 1)

self.agent_pos = (x, y)
reward = self.state_matrix[x, y]
self.state_matrix[x, y] = 0 # Remove the reward after collecting

done = (x, y) == self.goal

return self.agent_pos, reward, done, {}

def reset(self):
self.agent_pos = (0, 0)
return self.agent_pos

def render(self):
grid = np.zeros_like(self.state_matrix)
for x in range(4):
for y in range(4):
if self.state_matrix[x, y] != 0:
grid[x, y] = list(self.rewards.keys()).index(self.state_matrix[x, y]) + 1

grid[self.goal] = 9
grid[self.agent_pos] = 8

plt.imshow(grid, cmap=‘viridis’, extent=(-0.5, 3.5, 3.5, -0.5))
plt.xticks(range(4))
plt.yticks(range(4))
plt.grid(True, which=‘both’, linestyle=‘-’, linewidth=0.5)

plt.show()
return grid

# Test Environment
env = LawnmowerGridWorld()
state = env.reset()
print(‘Initial State:’, state)
env.render()

# Run Random Agent
for i in range(10):
action = np.random.choice([0, 1, 2, 3])
state, reward, done, _ = env.step(action)
print(‘After step {}: State={}, Action={}, Reward={}, Done={}’.format(i+1, state, action, reward, done))
env.render()

if done:
break--------------------------------Now, can you make change my scenario of grid world and give me the code ---------------

env.close()