Repository: Farama-Foundation/MO-Gymnasium (language: python)

Issue:
Add doc/pydoc to all environments



Discussion / hints:

Fix patch:
diff --git a/README.md b/README.md
index de7d39cd..636f41eb 100644
--- a/README.md
+++ b/README.md
@@ -73,14 +73,14 @@ You can also check more examples in this colab notebook!
 | `water-reservoir-v0`                                                                                                                                                 | Continuous / Continuous             | `[cost_flooding, deficit_water]`                              | A Water reservoir environment. The agent executes a continuous action, corresponding to the amount of water released by the dam. From [Pianosi et al. 2013](https://iwaponline.com/jh/article/15/2/258/3425/Tree-based-fitted-Q-iteration-for-multi-objective).                                            |
 | `mo-mountaincar-v0` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/mo-mountaincar.png" width="200px">               | Continuous / Discrete               | `[time_penalty, reverse_penalty, forward_penalty]`            | Classic Mountain Car env, but with extra penalties for the forward and reverse actions. From [Vamplew et al. 2011](https://www.researchgate.net/publication/220343783_Empirical_evaluation_methods_for_multiobjective_reinforcement_learning_algorithms).                                                  |
 | `mo-mountaincarcontinuous-v0` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/mo-mountaincar.png" width="200px">     | Continuous / Continuous             | `[time_penalty, fuel_consumption_penalty]`                    | Continuous Mountain Car env, but with penalties for fuel consumption.                                                                                                                                                                                                                                      |
-| `mo-lunar-lander-v2` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/lunarlander.png" width="200px">                 | Continuous / Discrete or Continuous | `[landed, shaped_reward, main_engine_fuel, side_engine_fuel]` | MO version of the "LunarLander-v2" environment. Objectives defined similarly as in [Hung et al. 2022](https://openreview.net/forum?id=AwWaBXLIJE).                                                                                                                                                         |
+| `mo-lunar-lander-v2` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/lunarlander.png" width="200px">                 | Continuous / Discrete or Continuous | `[landed, shaped_reward, main_engine_fuel, side_engine_fuel]` | MO version of the `LunarLander-v2` [environment](https://gymnasium.farama.org/environments/box2d/lunar_lander/). Objectives defined similarly as in [Hung et al. 2022](https://openreview.net/forum?id=AwWaBXLIJE).                                                                                                                                                         |
 | `mo-reacher-v0` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/reacher.png" width="200px">                          | Continuous / Discrete               | `[target_1, target_2, target_3, target_4]`                    | [:warning: PyBullet support is limited.] Reacher robot from [PyBullet](https://github.com/benelot/pybullet-gym/blob/ec9e87459dd76d92fe3e59ee4417e5a665504f62/pybulletgym/envs/roboschool/robots/manipulators/reacher.py), but there are 4 different target positions. From [Alegre et al. 2022](https://proceedings.mlr.press/v162/alegre22a.html). |
 | `mo-reacher-v4` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/reacher-mujoco.png" width="200px">                   | Continuous / Discrete               | `[target_1, target_2, target_3, target_4]`                    | Mujoco version of `mo-reacher-v0`, based on `Reacher-v4` [environment](https://gymnasium.farama.org/environments/mujoco/reacher/). |
 | `minecart-v0` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/minecart.png" width="200px">                           | Continuous or Image / Discrete      | `[ore1, ore2, fuel]`                                          | Agent must collect two types of ores and minimize fuel consumption. From [Abels et al. 2019](https://arxiv.org/abs/1809.07803v2).                                                                                                                                                                          |
 | `mo-highway-v0` and `mo-highway-fast-v0` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/highway.png" width="200px"> | Continuous / Discrete               | `[speed, right_lane, collision]`                              | The agent's objective is to reach a high speed while avoiding collisions with neighbouring vehicles and staying on the rightest lane. From [highway-env](https://github.com/eleurent/highway-env).                                                                                                         |
 | `mo-supermario-v0` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/mario.png" width="200px">                         | Image / Discrete                    | `[x_pos, time, death, coin, enemy]`                           | [:warning: SuperMarioBrosEnv support is limited.] Multi-objective version of [SuperMarioBrosEnv](https://github.com/Kautenja/gym-super-mario-bros). Objectives are defined similarly as in [Yang et al. 2019](https://arxiv.org/pdf/1908.08342.pdf).                                                                                                         |
-| `mo-halfcheetah-v4` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/cheetah.png" width="200px">                      | Continuous / Continuous             | `[velocity, energy]`                                          | Multi-objective version of [HalfCheetah-v4](https://www.gymlibrary.ml/environments/mujoco/half_cheetah/) env. Similar to [Xu et al. 2020](https://github.com/mit-gfx/PGMORL).                                                                                                                              |
-| `mo-hopper-v4` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/hopper.png" width="200px">                            | Continuous / Continuous             | `[velocity, height, energy]`                                  | Multi-objective version of [Hopper-v4](https://www.gymlibrary.ml/environments/mujoco/hopper/) env.                                                                                                                                                                                                         |
+| `mo-halfcheetah-v4` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/cheetah.png" width="200px">                      | Continuous / Continuous             | `[velocity, energy]`                                          | Multi-objective version of [HalfCheetah-v4](https://gymnasium.farama.org/environments/mujoco/half_cheetah/) env. Similar to [Xu et al. 2020](https://github.com/mit-gfx/PGMORL).                                                                                                                              |
+| `mo-hopper-v4` <br><img src="https://raw.githubusercontent.com/Farama-Foundation/MO-Gymnasium/main/screenshots/hopper.png" width="200px">                            | Continuous / Continuous             | `[velocity, height, energy]`                                  | Multi-objective version of [Hopper-v4](https://gymnasium.farama.org/environments/mujoco/hopper/) env.                                                                                                                                                                                                         |
 
 <!-- end environments -->
 
diff --git a/mo_gymnasium/envs/breakable_bottles/breakable_bottles.py b/mo_gymnasium/envs/breakable_bottles/breakable_bottles.py
index 1766ae7b..10f0034d 100644
--- a/mo_gymnasium/envs/breakable_bottles/breakable_bottles.py
+++ b/mo_gymnasium/envs/breakable_bottles/breakable_bottles.py
@@ -6,6 +6,49 @@
 
 
 class BreakableBottles(Env):
+    """
+    ## Description
+    This environment implements the problems UnbreakableBottles and BreakableBottles defined in Section 4.1.2 of the paper
+    "Potential-based multiobjective reinforcement learning approaches to low-impact agents for AI safety"
+    (https://www.sciencedirect.com/science/article/pii/S0952197621000336).
+
+    ## Action Space
+    The action space is a discrete space with 3 actions:
+    - 0: move left
+    - 1: move right
+    - 2: pick up a bottle
+
+    ## Observation Space
+    The observation space is a dictionary with 4 keys:
+    - location: the current location of the agent
+    - bottles_carrying: the number of bottles the agent is currently carrying (0, 1 or 2)
+    - bottles_delivered: the number of bottles the agent has delivered (0 or 1)
+    - bottles_dropped: for each location, a boolean flag indicating if that location currently contains a bottle
+
+    ## Reward Space
+    The reward space has 3 dimensions:
+    - time penalty: -1 for each time step
+    - bottle reward: bottle_reward for each bottle delivered
+    - potential: While carrying multiple bottles there is a small probability of dropping them. A potential-based penalty is applied for bottles left on the ground.
+
+    ## Starting State
+    The agent starts at location 0, carrying no bottles, having delivered no bottles and having dropped no bottles.
+
+    ## Episode Termination
+    The episode terminates when the agent has delivered 2 bottles.
+
+    ## Arguments
+    - size: the number of locations in the environment
+    - prob_drop: the probability of dropping a bottle while carrying 2 bottles
+    - time_penalty: the time penalty for each time step
+    - bottle_reward: the reward for delivering a bottle
+    - unbreakable_bottles: if True, a bottle which is dropped in a location can be picked up again (so the outcome of dropping a bottle is reversible),
+    otherwise a dropped bottle cannot be picked up.
+
+    ## Credits
+    This environment was originally a contribution of Robert Klassert
+    """
+
     metadata = {"render_modes": ["human"]}
 
     # actions
@@ -28,7 +71,7 @@ def __init__(
         self.prob_drop = prob_drop
         self.time_penalty = time_penalty
         self.bottle_reward = bottle_reward
-        self.unbreakable_bottles = False
+        self.unbreakable_bottles = unbreakable_bottles
 
         # properties
         self.num_objectives = 3
@@ -65,7 +108,11 @@ def step(self, action):
 
         if action == self.LEFT and self.location > 0:
             # execute bottle drop, if agent is carrying at least two and current location is 1, 2 or 3
-            if self.location in range(1, self.size - 1) and self.bottles_carrying > 1 and np.random.random() < self.prob_drop:
+            if (
+                self.location in range(1, self.size - 1)
+                and self.bottles_carrying > 1
+                and self.np_random.random() < self.prob_drop
+            ):
                 self.bottles_carrying -= 1
                 self.bottles_dropped[self.location - 1] += 1
 
@@ -74,7 +121,11 @@ def step(self, action):
 
         elif action == self.RIGHT and self.location < self.size - 1:
             # execute bottle drop, if agent is carrying at least two and current location is 1, 2 or 3
-            if self.location in range(1, self.size - 1) and self.bottles_carrying > 1 and np.random.random() < self.prob_drop:
+            if (
+                self.location in range(1, self.size - 1)
+                and self.bottles_carrying > 1
+                and self.np_random.random() < self.prob_drop
+            ):
                 self.bottles_carrying -= 1
                 self.bottles_dropped[self.location - 1] += 1
 
@@ -117,10 +168,9 @@ def step(self, action):
         # sum_t(r2_t) = 0 -> no impact
         reward[2] = self.potential(observation) - old_potential
 
-        info = {}
         if self.render_mode == "human":
             self.render()
-        return observation, reward, terminal, False, info
+        return observation, reward, terminal, False, {}
 
     def reset(self, *, seed: Optional[int] = None, options: Optional[dict] = None):
         super().reset(seed=seed)
diff --git a/mo_gymnasium/envs/continuous_mountain_car/continuous_mountain_car.py b/mo_gymnasium/envs/continuous_mountain_car/continuous_mountain_car.py
index a237aed5..491f902d 100644
--- a/mo_gymnasium/envs/continuous_mountain_car/continuous_mountain_car.py
+++ b/mo_gymnasium/envs/continuous_mountain_car/continuous_mountain_car.py
@@ -9,6 +9,17 @@
 
 
 class MOContinuousMountainCar(Continuous_MountainCarEnv):
+    """
+    A continuous version of the MountainCar environment, where the goal is to reach the top of the mountain.
+
+    See https://gymnasium.farama.org/environments/classic_control/mountain_car_continuous/ for more information.
+
+    ## Reward space:
+    The reward space is a 2D vector containing the time penalty and the fuel reward.
+    - time penalty: -1.0 for each time step
+    - fuel reward: -||action||^2 , i.e. the negative of the norm of the action vector
+    """
+
     def __init__(self, render_mode: Optional[str] = None, goal_velocity=0):
         super().__init__(render_mode, goal_velocity)
 
diff --git a/mo_gymnasium/envs/deep_sea_treasure/deep_sea_treasure.py b/mo_gymnasium/envs/deep_sea_treasure/deep_sea_treasure.py
index a163ea9a..313cdbcd 100644
--- a/mo_gymnasium/envs/deep_sea_treasure/deep_sea_treasure.py
+++ b/mo_gymnasium/envs/deep_sea_treasure/deep_sea_treasure.py
@@ -43,11 +43,37 @@
 
 
 class DeepSeaTreasure(gym.Env):
-    """Deep Sea Treasure environment
+    """
+    ## Description
+    The Deep Sea Treasure environment is classic MORL problem in which the agent controls a submarine in a 2D grid world.
+
+    ## Observation Space
+    The observation space is a 2D discrete box with values in [0, 10] for the x and y coordinates of the submarine.
+
+    ## Action Space
+    The actions is a discrete space where:
+    - 0: up
+    - 1: down
+    - 2: left
+    - 3: right
+
+    ## Reward Space
+    The reward is 2-dimensional:
+    - time penalty: -1 at each time step
+    - treasure value: the value of the treasure at the current position
+
+    ## Starting State
+    The starting state is always the same: (0, 0)
+
+    ## Episode Termination
+    The episode terminates when the agent reaches a treasure.
 
-    Adapted from: https://github.com/RunzheYang/MORL
+    ## Arguments
+    - dst_map: the map of the deep sea treasure. Default is the convex map from Yang et al. (2019).
+    - float_state: if True, the state is a 2D continuous box with values in [0.0, 1.0] for the x and y coordinates of the submarine.
 
-    CCS weights: [1,0], [0.7,0.3], [0.67,0.33], [0.6,0.4], [0.56,0.44], [0.52,0.48], [0.5,0.5], [0.4,0.6], [0.3,0.7], [0, 1]
+    ## Credits
+    The code was adapted from: https://github.com/RunzheYang/MORL
     """
 
     metadata = {"render_modes": ["human", "rgb_array"], "render_fps": 4}
diff --git a/mo_gymnasium/envs/fishwood/fishwood.py b/mo_gymnasium/envs/fishwood/fishwood.py
index f6c441f9..3ed00b76 100644
--- a/mo_gymnasium/envs/fishwood/fishwood.py
+++ b/mo_gymnasium/envs/fishwood/fishwood.py
@@ -1,4 +1,3 @@
-import random
 from typing import Optional
 
 import gymnasium as gym
@@ -7,7 +6,42 @@
 
 
 class FishWood(gym.Env):
-    metadata = {"render_modes": ["ansi"]}
+    """
+    ## Description
+    The FishWood environment is a simple MORL problem in which the agent controls a fisherman which can either fish or go collect wood.
+    From "Multi-objective Reinforcement Learning for the Expected Utility of the Return"
+    https://www.researchgate.net/publication/328718263_Multi-objective_Reinforcement_Learning_for_the_Expected_Utility_of_the_Return
+
+    ## Observation Space
+    The observation space is a discrete space with two states:
+    - 0: fishing
+    - 1: in the woods
+
+    ## Action Space
+    The actions is a discrete space where:
+    - 0: go fishing
+    - 1: go collect wood
+
+    ## Reward Space
+    The reward is 2-dimensional:
+    - 0: +1 if agent is in the woods, with woodproba probability, and 0 otherwise
+    - 1: +1 if the agent is fishing, with fishproba probability, and 0 otherwise
+
+    ## Starting State
+    Agent starts in the woods
+
+    ## Termination
+    The episode ends after MAX_TS=200 steps
+
+    ## Arguments
+    - fishproba: probability of catching a fish when fishing
+    - woodproba: probability of collecting wood when in the woods
+
+    ## Credits
+    Code provided by Denis Steckelmacher
+    """
+
+    metadata = {"render_modes": ["human"]}
     FISH = 0
     WOOD = 1
     MAX_TS = 200
@@ -23,7 +57,7 @@ def __init__(self, render_mode: Optional[str] = None, fishproba=0.1, woodproba=0
         # 2 objectives, amount of fish and amount of wood
         self.reward_space = spaces.Box(low=np.array([0, 0]), high=np.array([1.0, 1.0]), dtype=np.float32)
 
-        self._state = self.reset()
+        self._state = self.WOOD
 
     def reset(self, seed=None, **kwargs):
         super().reset(seed=seed)
@@ -36,25 +70,28 @@ def reset(self, seed=None, **kwargs):
         return self._state, {}
 
     def render(self):
-        if self._state == self.WOOD:
-            return f"t={self._timestep}, in wood."
-        else:
-            return f"t={self._timestep}, fishing"
+        if self.render_mode == "human":
+            if self._state == self.WOOD:
+                return f"t={self._timestep}, in wood."
+            else:
+                return f"t={self._timestep}, fishing"
 
     def step(self, action):
         # Obtain a resource from the current state
-        rewards = np.zeros((2,))
+        rewards = np.zeros((2,), dtype=np.float32)
 
-        if self._state == self.WOOD and random.random() < self._woodproba:
+        if self._state == self.WOOD and self.np_random.random() < self._woodproba:
             rewards[self.WOOD] = 1.0
-        elif self._state == self.FISH and random.random() < self._fishproba:
+        elif self._state == self.FISH and self.np_random.random() < self._fishproba:
             rewards[self.FISH] = 1.0
 
         # Execute the action
         self._state = action
         self._timestep += 1
 
-        return self._state, rewards, self._timestep == self.MAX_TS, False, {}
+        if self.render_mode == "human":
+            self.render()
+        return self._state, rewards, self._timestep == self.MAX_TS, self._timestep == self.MAX_TS, {}
 
 
 if __name__ == "__main__":
diff --git a/mo_gymnasium/envs/four_room/four_room.py b/mo_gymnasium/envs/four_room/four_room.py
index 84944cbc..8087fb0f 100644
--- a/mo_gymnasium/envs/four_room/four_room.py
+++ b/mo_gymnasium/envs/four_room/four_room.py
@@ -32,15 +32,38 @@
 
 class FourRoom(gym.Env):
     """
+    ## Description
     A discretized version of the gridworld environment introduced in [1]. Here, an agent learns to
     collect shapes with positive reward, while avoid those with negative reward, and then travel to a fixed goal.
     The gridworld is split into four rooms separated by walls with passage-ways.
 
-    # Code adaptaed from: https://github.com/mike-gimelfarb/deep-successor-features-for-transfer/blob/main/source/tasks/gridworld.py
-
     References
     ----------
     [1] Barreto, André, et al. "Successor Features for Transfer in Reinforcement Learning." NIPS. 2017.
+
+    ## Observation Space
+    The observation contains the 2D position of the agent in the gridworld, plus a binary vector indicating which items were collected.
+
+    ## Action Space
+    The action space is discrete with 4 actions: left, up, right, down.
+
+    ## Reward Space
+    The reward is a 3-dimensional vector with the following components:
+    - +1 if collected a blue square, else 0
+    - +1 if collected a green triangle, else 0
+    - +1 if collected a red circle, else 0
+
+    ## Starting State
+    The agent starts in the lower left of the map.
+
+    ## Episode Termination
+    The episode terminates when the agent reaches the goal state, G.
+
+    ## Arguments
+    - maze: Array containing the gridworld map. See MAZE for an example.
+
+    ## Credits
+    Code adaptaed from: https://github.com/mike-gimelfarb/deep-successor-features-for-transfer/blob/main/source/tasks/gridworld.py
     """
 
     LEFT, UP, RIGHT, DOWN = 0, 1, 2, 3
diff --git a/mo_gymnasium/envs/fruit_tree/fruit_tree.py b/mo_gymnasium/envs/fruit_tree/fruit_tree.py
index 4da63385..fc2368bb 100644
--- a/mo_gymnasium/envs/fruit_tree/fruit_tree.py
+++ b/mo_gymnasium/envs/fruit_tree/fruit_tree.py
@@ -239,6 +239,28 @@
 
 
 class FruitTreeEnv(gym.Env):
+    """
+    ## Description
+
+    Full binary tree of depth d=5,6 or 7. Every leaf contains a fruit with a value for the nutrients Protein, Carbs, Fats, Vitamins, Minerals and Water.
+    From [Yang et al. 2019](https://arxiv.org/pdf/1908.08342.pdf).
+
+    ## Observation Space
+    Discrete space of size 2^d-1, where d is the depth of the tree.
+
+    ## Action Space
+    The agent can chose to go left or right at every node. The action space is therefore a discrete space of size 2.
+
+    ## Reward Space
+    Each leaf node contains a 6-dimensional vector containing the nutrients of the fruit. The agent receives a reward for each nutrient it collects.
+
+    ## Starting State
+    The agent starts at the root node (0, 0).
+
+    ## Episode Termination
+    The episode terminates when the agent reaches a leaf node.
+    """
+
     def __init__(self, depth=6):
         assert depth in [5, 6, 7], "Depth must be 5, 6 or 7."
         self.reward_dim = 6
diff --git a/mo_gymnasium/envs/highway/highway.py b/mo_gymnasium/envs/highway/highway.py
index 26f7956b..e8c0c9f3 100644
--- a/mo_gymnasium/envs/highway/highway.py
+++ b/mo_gymnasium/envs/highway/highway.py
@@ -16,7 +16,18 @@
 
 
 class MOHighwayEnv(HighwayEnv):
-    """A multi-objective version of the HighwayEnv environment."""
+    """
+    ## Description
+    Multi-objective version of the HighwayEnv environment.
+
+    See https://github.com/eleurent/highway-env for more information.
+
+    ## Reward Space
+    The reward is 3-dimensional:
+    - 0: high speed reward
+    - 1: right lane reward
+    - 2: collision reward
+    """
 
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
diff --git a/mo_gymnasium/envs/lunar_lander/lunar_lander.py b/mo_gymnasium/envs/lunar_lander/lunar_lander.py
index 45283c93..fbf1651f 100644
--- a/mo_gymnasium/envs/lunar_lander/lunar_lander.py
+++ b/mo_gymnasium/envs/lunar_lander/lunar_lander.py
@@ -17,6 +17,20 @@
 
 
 class MOLunarLander(LunarLander):
+    """
+    ## Description
+    Multi-objective version of the LunarLander environment.
+
+    See https://gymnasium.farama.org/environments/box2d/lunar_lander/ for more information.
+
+    ## Reward Space
+    The reward is 4-dimensional:
+    - 0: -100 if crash, +100 if lands successfully
+    - 1: Shaping reward
+    - 2: Fuel cost (main engine)
+    - 3: Fuel cost (side engine)
+    """
+
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
 
diff --git a/mo_gymnasium/envs/mario/mario.py b/mo_gymnasium/envs/mario/mario.py
index 16448416..532b7a83 100644
--- a/mo_gymnasium/envs/mario/mario.py
+++ b/mo_gymnasium/envs/mario/mario.py
@@ -17,6 +17,23 @@
 
 
 class MOSuperMarioBros(SuperMarioBrosEnv):
+    """
+    ## Description
+    Multi-objective version of the SuperMarioBro environment.
+
+    See https://github.com/Kautenja/gym-super-mario-bros for more information.
+
+    ## Reward Space
+    The reward is a 5-dimensional vector:
+    - 0: How far Mario moved in the x position
+    - 1: Time penalty for how much time has passed between two time steps
+    - 2: -25 if Mario died, 0 otherwise
+    - 3: +100 if Mario collected coins, else 0
+    - 4: Points for killing an enemy
+
+    ## Episode Termination
+    The episode terminates when Mario dies or reaches the flag.
+    """
 
     metadata = {"render_modes": ["human", "rgb_array"], "render_fps": 60}
 
diff --git a/mo_gymnasium/envs/minecart/minecart.py b/mo_gymnasium/envs/minecart/minecart.py
index e85b43ff..4dfda734 100644
--- a/mo_gymnasium/envs/minecart/minecart.py
+++ b/mo_gymnasium/envs/minecart/minecart.py
@@ -86,92 +86,44 @@
 MINE_IMG = str(Path(__file__).parent.absolute()) + "/assets/mine.png"
 
 
-class Mine:
-    """Class representing an individual Mine"""
-
-    def __init__(self, ore_cnt, x, y):
-        self.distributions = [scipy.stats.norm(np.random.random(), np.random.random()) for _ in range(ore_cnt)]
-        self.pos = np.array((x, y))
-
-    def distance(self, cart):
-        return mag(cart.pos - self.pos)
-
-    def mineable(self, cart):
-        return self.distance(cart) <= MINE_RADIUS * MINE_SCALE * CART_SCALE
-
-    def mine(self):
-        """Generates collected resources according to the mine's random
-        distribution
-
-        Returns:
-            list -- list of collected resources
-        """
-        return [max(0.0, dist.rvs()) for dist in self.distributions]
-
-    def distribution_means(self):
-        """
-        Computes the mean of the truncated normal distributions
-        """
-        means = np.zeros(len(self.distributions))
-
-        for i, dist in enumerate(self.distributions):
-            mean, std = dist.mean(), dist.std()
-            means[i] = truncated_mean(mean, std, 0, float("inf"))
-            if np.isnan(means[i]):
-                means[i] = 0
-        return means
-
-
-class Cart:
-    """Class representing the actual minecart"""
-
-    def __init__(self, ore_cnt):
-        self.ore_cnt = ore_cnt
-        self.pos = np.array([HOME_X, HOME_Y])
-        self.speed = 0
-        self.angle = 45
-        self.content = np.zeros(self.ore_cnt)
-        self.departed = False  # Keep track of whether the agent has left the base
-
-    def accelerate(self, acceleration):
-        self.speed = clip(self.speed + acceleration, 0, MAX_SPEED)
-
-    def rotate(self, rotation):
-        self.angle = (self.angle + rotation) % 360
-
-    def step(self):
-        """
-        Update cart's position, taking the current speed into account
-        Colliding with a border at anything but a straight angle will cause
-        cart to "slide" along the wall.
-        """
-        pre = np.copy(self.pos)
-        if self.speed < EPS_SPEED:
-            return False
-        x_velocity = self.speed * math.cos(self.angle * math.pi / 180)
-        y_velocity = self.speed * math.sin(self.angle * math.pi / 180)
-        x, y = self.pos
-        if y != 0 and y != 1 and (y_velocity > 0 + EPS_SPEED or y_velocity < 0 - EPS_SPEED):
-            if x == 1 and x_velocity > 0:
-                self.angle += math.copysign(ROTATION, y_velocity)
-            if x == 0 and x_velocity < 0:
-                self.angle -= math.copysign(ROTATION, y_velocity)
-        if x != 0 and x != 1 and (x_velocity > 0 + EPS_SPEED or x_velocity < 0 - EPS_SPEED):
-            if y == 1 and y_velocity > 0:
-                self.angle -= math.copysign(ROTATION, x_velocity)
-
-            if y == 0 and y_velocity < 0:
-                self.angle += math.copysign(ROTATION, x_velocity)
-
-        self.pos[0] = clip(x + x_velocity, 0, 1)
-        self.pos[1] = clip(y + y_velocity, 0, 1)
-        self.speed = mag(pre - self.pos)
-
-        return True
-
-
 class Minecart(gym.Env):
-    """Minecart environment"""
+    """
+    ## Description
+    Agent must collect two types of ores and minimize fuel consumption.
+    From Abels et al. 2019 https://arxiv.org/abs/1809.07803v2.
+
+    ## Observation Space
+    The observation is a 7-dimensional vector containing the following information:
+    - 2D position of the cart
+    - Speed of the cart
+    - sin and cos of the cart's orientation
+    - porcentage of the capacity of the cart filled
+    If image_observation is True, the observation is a 3D image of the environment.
+
+    ## Action Space
+    The action space is a discrete space with 6 actions:
+    - 0: Mine
+    - 1: Left
+    - 2: Right
+    - 3: Accelerate
+    - 4: Brake
+    - 5: None
+
+    ## Reward Space
+    The reward is a 3D vector:
+    - 0: Quantity of the first minerium that was retrieved to the base (sparse)
+    - 1: Quantity of the second minerium that was retrieved to the base (sparse)
+    - 2: Fuel consumed (dense)
+
+    ## Starting State
+    The cart starts at the base on the upper left corner of the map.
+
+    ## Episode Termination
+    The episode ends when the cart returns to the base.
+
+    ## Credits
+    The code was refactored from https://github.com/axelabels/DynMORL
+    """
 
     metadata = {"render_modes": ["rgb_array", "human"], "render_fps": FPS}
 
@@ -733,6 +685,90 @@ def __deepcopy__(self, memo):
         return this_copy
 
 
+class Mine:
+    """Class representing an individual Mine"""
+
+    def __init__(self, ore_cnt, x, y):
+        self.distributions = [scipy.stats.norm(np.random.random(), np.random.random()) for _ in range(ore_cnt)]
+        self.pos = np.array((x, y))
+
+    def distance(self, cart):
+        return mag(cart.pos - self.pos)
+
+    def mineable(self, cart):
+        return self.distance(cart) <= MINE_RADIUS * MINE_SCALE * CART_SCALE
+
+    def mine(self):
+        """Generates collected resources according to the mine's random
+        distribution
+
+        Returns:
+            list -- list of collected resources
+        """
+        return [max(0.0, dist.rvs()) for dist in self.distributions]
+
+    def distribution_means(self):
+        """
+        Computes the mean of the truncated normal distributions
+        """
+        means = np.zeros(len(self.distributions))
+
+        for i, dist in enumerate(self.distributions):
+            mean, std = dist.mean(), dist.std()
+            means[i] = truncated_mean(mean, std, 0, float("inf"))
+            if np.isnan(means[i]):
+                means[i] = 0
+        return means
+
+
+class Cart:
+    """Class representing the actual minecart"""
+
+    def __init__(self, ore_cnt):
+        self.ore_cnt = ore_cnt
+        self.pos = np.array([HOME_X, HOME_Y])
+        self.speed = 0
+        self.angle = 45
+        self.content = np.zeros(self.ore_cnt)
+        self.departed = False  # Keep track of whether the agent has left the base
+
+    def accelerate(self, acceleration):
+        self.speed = clip(self.speed + acceleration, 0, MAX_SPEED)
+
+    def rotate(self, rotation):
+        self.angle = (self.angle + rotation) % 360
+
+    def step(self):
+        """
+        Update cart's position, taking the current speed into account
+        Colliding with a border at anything but a straight angle will cause
+        cart to "slide" along the wall.
+        """
+        pre = np.copy(self.pos)
+        if self.speed < EPS_SPEED:
+            return False
+        x_velocity = self.speed * math.cos(self.angle * math.pi / 180)
+        y_velocity = self.speed * math.sin(self.angle * math.pi / 180)
+        x, y = self.pos
+        if y != 0 and y != 1 and (y_velocity > 0 + EPS_SPEED or y_velocity < 0 - EPS_SPEED):
+            if x == 1 and x_velocity > 0:
+                self.angle += math.copysign(ROTATION, y_velocity)
+            if x == 0 and x_velocity < 0:
+                self.angle -= math.copysign(ROTATION, y_velocity)
+        if x != 0 and x != 1 and (x_velocity > 0 + EPS_SPEED or x_velocity < 0 - EPS_SPEED):
+            if y == 1 and y_velocity > 0:
+                self.angle -= math.copysign(ROTATION, x_velocity)
+
+            if y == 0 and y_velocity < 0:
+                self.angle += math.copysign(ROTATION, x_velocity)
+
+        self.pos[0] = clip(x + x_velocity, 0, 1)
+        self.pos[1] = clip(y + y_velocity, 0, 1)
+        self.speed = mag(pre - self.pos)
+
+        return True
+
+
 def compute_angle(p0, p1, p2):
     v0 = np.array(p0) - np.array(p1)
     v1 = np.array(p2) - np.array(p1)
diff --git a/mo_gymnasium/envs/mountain_car/mountain_car.py b/mo_gymnasium/envs/mountain_car/mountain_car.py
index 8387256e..fc93e527 100644
--- a/mo_gymnasium/envs/mountain_car/mountain_car.py
+++ b/mo_gymnasium/envs/mountain_car/mountain_car.py
@@ -7,6 +7,18 @@
 
 
 class MOMountainCar(MountainCarEnv):
+    """
+    A multi-objective version of the MountainCar environment, where the goal is to reach the top of the mountain.
+
+    See https://gymnasium.farama.org/environments/classic_control/mountain_car_continuous/ for more information.
+
+    ## Reward space:
+    The reward space is a 3D vector containing the time penalty, and penalties for reversing and going forward.
+    - time penalty: -1.0 for each time step
+    - reverse penalty: -1.0 for each time step the actuin is 0 (reverse)
+    - forward penalty: -1.0 for each time step the action is 2 (forward)
+    """
+
     def __init__(self, render_mode: Optional[str] = None, goal_velocity=0):
         super().__init__(render_mode, goal_velocity)
 
diff --git a/mo_gymnasium/envs/mujoco/half_cheetah.py b/mo_gymnasium/envs/mujoco/half_cheetah.py
index 06afd159..3e2ae6ba 100644
--- a/mo_gymnasium/envs/mujoco/half_cheetah.py
+++ b/mo_gymnasium/envs/mujoco/half_cheetah.py
@@ -4,6 +4,18 @@
 
 
 class MOHalfCheehtahEnv(HalfCheetahEnv):
+    """
+    ## Description
+    Multi-objective version of the HalfCheetahEnv environment.
+
+    See https://gymnasium.farama.org/environments/mujoco/half_cheetah/ for more information.
+
+    ## Reward Space
+    The reward is 2-dimensional:
+    - 0: Reward for running forward
+    - 1: Control cost of the action
+    """
+
     def __init__(self, **kwargs):
         super().__init__(**kwargs)
         self.reward_space = Box(low=-np.inf, high=np.inf, shape=(2,))
diff --git a/mo_gymnasium/envs/mujoco/hopper.py b/mo_gymnasium/envs/mujoco/hopper.py
index 45adcdb4..07174c13 100644
--- a/mo_gymnasium/envs/mujoco/hopper.py
+++ b/mo_gymnasium/envs/mujoco/hopper.py
@@ -4,6 +4,20 @@
 
 
 class MOHopperEnv(HopperEnv):
+    """
+    ## Description
+    Multi-objective version of the HopperEnv environment.
+
+    See https://gymnasium.farama.org/environments/mujoco/hopper/ for more information.
+
+    ## Reward Space
+    The reward is 3-dimensional:
+    - 0: Reward for going forward on the x-axis
+    - 1: Reward for jumping high on the z-axis
+    - 2: Control cost of the action
+    If the cost_objective flag is set to False, the reward is 2-dimensional, and the cost is added to other objectives.
+    """
+
     def __init__(self, cost_objective=True, **kwargs):
         super().__init__(**kwargs)
         self.cost_objetive = cost_objective
diff --git a/mo_gymnasium/envs/mujoco/reacher.py b/mo_gymnasium/envs/mujoco/reacher.py
index 63f39209..81a97a6d 100644
--- a/mo_gymnasium/envs/mujoco/reacher.py
+++ b/mo_gymnasium/envs/mujoco/reacher.py
@@ -11,6 +11,26 @@
 
 
 class MOReacherEnv(ReacherEnv):
+    """
+    ## Description
+    Mujoco version of `mo-reacher-v0`, based on `Reacher-v4` environment (https://gymnasium.farama.org/environments/mujoco/reacher/).
+
+    ## Observation Space
+    The observation is 6-dimensional and contains:
+    - sin and cos of the angles of the central and elbow joints
+    - angular velocity of the central and elbow joints
+
+    ## Action Space
+    The action space is discrete and contains the 3^2=9 possible actions based on applying positive (+1), negative (-1) or zero (0) torque to each of the two joints.
+
+    ## Reward Space
+    The reward is 4-dimensional and is defined based on the distance of the tip of the arm and the four target locations.
+    For each i={1,2,3,4} it is computed as:
+    ```math
+        r_i = 1  - 4 * || finger_tip_coord - target_i ||^2
+    ```
+    """
+
     def __init__(self, **kwargs):
         utils.EzPickle.__init__(self, **kwargs)
         self.observation_space = Box(low=-np.inf, high=np.inf, shape=(6,), dtype=np.float64)
diff --git a/mo_gymnasium/envs/resource_gathering/resource_gathering.py b/mo_gymnasium/envs/resource_gathering/resource_gathering.py
index c66eb624..193c9953 100644
--- a/mo_gymnasium/envs/resource_gathering/resource_gathering.py
+++ b/mo_gymnasium/envs/resource_gathering/resource_gathering.py
@@ -8,10 +8,36 @@
 
 
 class ResourceGathering(gym.Env):
-    """Resource Gathering environment
-
+    """
+    ## Description
     From "Barrett, Leon & Narayanan, Srini. (2008). Learning all optimal policies with multiple criteria.
     Proceedings of the 25th International Conference on Machine Learning. 41-47. 10.1145/1390156.1390162."
+
+    ## Observation Space
+    The observation is discrete and consists of 4 elements:
+    - 0: The x coordinate of the agent
+    - 1: The y coordinate of the agent
+    - 2: Flag indicating if the agent collected the gold
+    - 3: Flag indicating if the agent collected the diamond
+
+    ## Action Space
+    The action is discrete and consists of 4 elements:
+    - 0: Move up
+    - 1: Move down
+    - 2: Move left
+    - 3: Move right
+
+    ## Reward Space
+    The reward is 3-dimensional:
+    - 0: +1 if returned home with gold, else 0
+    - 1: +1 if returned home with diamond, else 0
+    - 2: -1 if killed by an enemy, else 0
+
+    ## Starting State
+    The agent starts at the home position with no gold or diamond.
+
+    ## Episode Termination
+    The episode terminates when the agent returns home, or when the agent is killed by an enemy.
     """
 
     metadata = {"render_modes": ["human", "rgb_array"], "render_fps": 4}
@@ -153,7 +179,7 @@ def step(self, action):
         elif cell == "R2":
             self.has_gem = 1
         elif cell == "E1" or cell == "E2":
-            if np.random.random() < 0.1:
+            if self.np_random.random() < 0.1:
                 vec_reward[0] = -1.0
                 done = True
         elif cell == "H":
diff --git a/mo_gymnasium/envs/water_reservoir/dam_env.py b/mo_gymnasium/envs/water_reservoir/dam_env.py
index f14afb0d..a5717910 100644
--- a/mo_gymnasium/envs/water_reservoir/dam_env.py
+++ b/mo_gymnasium/envs/water_reservoir/dam_env.py
@@ -6,14 +6,26 @@
 
 
 class DamEnv(gym.Env):
-    """A Water reservoir environment.
+    """
+    ## Description
+    A Water reservoir environment.
     The agent executes a continuous action, corresponding to the amount of water
     released by the dam.
+
+    ## Observation Space
+    The observation is a float corresponding to the current level of the reservoir.
+
+    ## Action Space
+    The action is a float corresponding to the amount of water released by the dam.
+
+    ## Reward Space
     There are up to 4 rewards:
      - cost due to excess level wrt a flooding threshold (upstream)
      - deficit in the water supply wrt the water demand
      - deficit in hydroelectric supply wrt hydroelectric demand
      - cost due to excess level wrt a flooding threshold (downstream)
+
+     ## Credits
      Code from:
      https://gitlab.ai.vub.ac.be/mreymond/dam
      Ported from:


Test patch:
diff --git a/tests/test_envs.py b/tests/test_envs.py
index 115ba822..2686e996 100644
--- a/tests/test_envs.py
+++ b/tests/test_envs.py
@@ -98,8 +98,7 @@ def test_hopper():
 def test_breakable_bottles():
     env = gym.make("breakable-bottles-v0")
     env = LinearReward(env)
-    # skip because of bug in gym (see https://github.com/openai/gym/issues/2956)
-    # check_env(env)
+    check_env(env)
 
 
 def test_water_reservoir():