Part I: Define an RL Environment [30 points]
In this part, we will define a grid-world reinforcement learning environment as an MDP.
While building an RL environment, you need to define possible states, actions, rewards
and other parameters.
STEPS:
1. Choose a scenario for your grid world. You are welcome to use the
visualizationdemo as a reference to visualize it.
An example of idea for RL environment:
• Theme: Lawnmower Grid World with
batteries as positive rewards and rocks as
negative rewards.
• States: {S1 = (0,0), S2 = (0,1), S3 = (0,2),
S4 = (0,3), S5 = (1,0), S6 = (1,1), S7 =
(1,2), S8 = (1,3), S9 = (2,0), S10 = (2,1),
S11 = (2,2), S12 = (2,3), S13 = (3,0), S14 =
(3,1), S15 = (3,2), S16 = (3,3)}
• Actions: {Up, Down, Right, Left}
• Rewards: {-5, -6, +5, +6}
• Objective: Reach the goal state with
maximum reward
2. Define an RL environment following the scenario that you chose.
Environment requirements:
• Min number of states: 12
• Min number of actions: 4
• Min number of rewards: 4
Environment definition should follow the OpenAI Gym structure, which includes
thebasic methods. You can use the “Defining RL env” demo as a base code.
def __init__:
# Initializes the class
# Define action and observation space
def step:
# Executes one timestep within the environment
# Input to the function is an action
def reset:
# Resets the state of the environment to an initial state
def render:
# Visualizes the environment
# Any form like vector representation or visualizing
usingmatplotlib is sufficient
3. Run a random agent for at least 10 timesteps to show that the environment logic
is defined correctly. Print the current state, chosen action, reward and return your
grid world visualization for each step.This is my code for part 1 ----------------import numpy as np
import gym
from gym import spaces
import matplotlib.pyplot as plt

class LawnmowerGridWorld(gym.Env):
    def __init__(self):
        # Define action and observation space
        self.observation_space = spaces.Discrete(16)
        self.action_space = spaces.Discrete(4)
        self.state_matrix = np.zeros((4, 4))

        # Define rewards
        self.rewards = {-5: 'Rock', -6: 'Rock', 5: 'Battery', 6: 'Battery'}

        # Randomly initialize rewardable slots
        for _ in range(4): # 4 rewards
            x, y = np.random.randint(0, 4, 2)
            reward = np.random.choice(list(self.rewards.keys()))
            self.state_matrix[x, y] = reward

        self.agent_pos = (0, 0)
        self.goal = (3, 3)

    def step(self, action):
        x, y = self.agent_pos

        if action == 0:  # Up
            x = max(0, x - 1)
        elif action == 1:  # Down
            x = min(3, x + 1)
        elif action == 2:  # Right
            y = min(3, y + 1)
        else:  # Left
            y = max(0, y - 1)

        self.agent_pos = (x, y)
        reward = self.state_matrix[x, y]
        self.state_matrix[x, y] = 0  # Remove the reward after collecting

        done = (x, y) == self.goal

        return self.agent_pos, reward, done, {}

    def reset(self):
        self.agent_pos = (0, 0)
        return self.agent_pos

    def render(self):
        grid = np.zeros_like(self.state_matrix)
        for x in range(4):
            for y in range(4):
                if self.state_matrix[x, y] != 0:
                    grid[x, y] = list(self.rewards.keys()).index(self.state_matrix[x, y]) + 1

        grid[self.goal] = 9
        grid[self.agent_pos] = 8

        plt.imshow(grid, cmap='viridis', extent=(-0.5, 3.5, 3.5, -0.5))
        plt.xticks(range(4))
        plt.yticks(range(4))
        plt.grid(True, which='both', linestyle='-', linewidth=0.5)

        plt.show()
        return grid

# Test Environment
env = LawnmowerGridWorld()
state = env.reset()
print('Initial State:', state)
env.render()

# Run Random Agent
for i in range(10):
    action = np.random.choice([0, 1, 2, 3])
    state, reward, done, _ = env.step(action)
    print('After step {}: State={}, Action={}, Reward={}, Done={}'.format(i+1, state, action, reward, done))
    env.render()

    if done:
        break

env.close()---------------------------------------I would like to see the environment I defined, give me code for the same,