Repository: openai/gym

Issue:
Reset in GoalEnv never called
What is the purpose of the reset in GoalEnv base class?

https://github.com/openai/gym/blob/7006c7a182a991b9409c206c56abc121eaffafb5/gym/core.py#L169

If I derive my own env from `GoalEnv`, the base reset is never called - so it never checks observation_space.

Is there an example of when this is actually called?
BUG - KellyCoinflip episode should end when the agent reaches the maximum wealth.
# Bug description
The episode does not end when agent reaches the maximum wealth. This is not consistent with the information from the docstring. 

A Monte Carlo approach can also be used to illustrate the problem. The distribution of realized rewards obtained by an agent betting according to the Kelly criterion is incorrect. E.g. the docstring indicates such agent should earn $240 on average, and the bug leads to a much lower mean.

# To reproduce the bug
## Code
```
import gym 

env = gym.make('KellyCoinflip-v0')
env.seed(1)
env.reset()
done=False

while not done:
    action = int(env.wealth * 20)  # bet 20% of the wealth
    observation, reward, done, info = env.step(action)
    env.render()
```

## Output
```
Current wealth:  20.0 ; Rounds left:  299
[...]
Current wealth:  238.47 ; Rounds left:  192
Current wealth:  250.0 ; Rounds left:  191
Current wealth:  200.0 ; Rounds left:  190
[...]
```

## Expected output 
```
Current wealth:  20.0 ; Rounds left:  299
[...]
Current wealth:  238.47 ; Rounds left:  192
Current wealth:  250.0 ; Rounds left:  191
```

# Proposed modifications
I would like to work on a PR with the following modifications:
* fix the bug described above,
* refactor to reduce code duplication,
* rename some variables to follow PEP-8 guidelines. 


Discussion:
I suppose one must call:

    super(MyEnv, self).reset()
Nope, this doesn't work as I get `NotImplementedError`
Can you post a code example of this not working?  This sounds like just a normal python inheritance issue.
Yeah, I mean when you derive from this class, you have to override reset function. So that the GoalEnv reset function is never called. 
Oh you're saying that since `GoalEnv.reset()` calls `Env.reset()` this can't possibly work https://github.com/openai/gym/blob/7006c7a182a991b9409c206c56abc121eaffafb5/gym/core.py#L173

I think the easiest fix would be to only have the `isinstance` check in `GoalEnv.reset()` and remove the other code.
We could also merge `GoalEnv` into https://github.com/openai/gym/blob/e0f916a7d5e4b6afaee05a73ca3da508423d8cf2/gym/envs/robotics/robot_env.py#L16 since this appears to be the only use.
Thoughts @pzhokhov ?


Resolving patch:
diff --git a/gym/core.py b/gym/core.py
--- a/gym/core.py
+++ b/gym/core.py
@@ -166,11 +166,9 @@ def reset(self):
         # Enforce that each GoalEnv uses a Goal-compatible observation space.
         if not isinstance(self.observation_space, gym.spaces.Dict):
             raise error.Error('GoalEnv requires an observation space of type gym.spaces.Dict')
-        result = super(GoalEnv, self).reset()
         for key in ['observation', 'achieved_goal', 'desired_goal']:
-            if key not in result:
+            if key not in self.observation_space.spaces:
                 raise error.Error('GoalEnv requires the "{}" key to be part of the observation dictionary.'.format(key))
-        return result
 
     def compute_reward(self, achieved_goal, desired_goal, info):
         """Compute the step reward. This externalizes the reward function and makes
diff --git a/gym/envs/robotics/robot_env.py b/gym/envs/robotics/robot_env.py
--- a/gym/envs/robotics/robot_env.py
+++ b/gym/envs/robotics/robot_env.py
@@ -76,6 +76,7 @@ def reset(self):
         # Gimbel lock) or we may not achieve an initial condition (e.g. an object is within the hand).
         # In this case, we just keep randomizing until we eventually achieve a valid initial
         # configuration.
+        super(RobotEnv, self).reset()
         did_reset_sim = False
         while not did_reset_sim:
             did_reset_sim = self._reset_sim()


Test patch: