internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: openai/gym

Issue:
Why is there a restiction on shape in multi discrete?
https://github.com/openai/gym/blob/422c9c7bb3c3c5a756c3b12dfe99733bfbfe3920/gym/spaces/multi_discrete.py#L10

Why is this imposed?
Say one may need a high dimensional space like a (3,3,3) grid with discretised values for each element in the grid. 
It can be very easily circumvented by using np.random.random_sample and passing the shape
Is there some specific reason for doing this?

Also note this example provided in dict_space doesn't work currently.
https://github.com/openai/gym/blob/422c9c7bb3c3c5a756c3b12dfe99733bfbfe3920/gym/spaces/dict_space.py#L22




Discussion:

Resolving patch:
diff --git a/gym/spaces/multi_discrete.py b/gym/spaces/multi_discrete.py
--- a/gym/spaces/multi_discrete.py
+++ b/gym/spaces/multi_discrete.py
@@ -7,10 +7,9 @@ def __init__(self, nvec):
         nvec: vector of counts of each categorical variable
         """
         self.nvec = np.asarray(nvec, dtype=np.int32)
-        assert self.nvec.ndim == 1, 'nvec should be a 1d array (or list) of ints'
-        gym.Space.__init__(self, (self.nvec.size,), np.int8)
+        gym.Space.__init__(self, (self.nvec.shape,), np.int8)
     def sample(self):
-        return (gym.spaces.np_random.rand(self.nvec.size) * self.nvec).astype(self.dtype)
+        return (gym.spaces.np_random.random_sample(self.nvec.shape) * self.nvec).astype(self.dtype)
     def contains(self, x):
         return (0 <= x).all() and (x < self.nvec).all() and x.dtype.kind in 'ui'
     


Test patch:
Repository: openai/gym

Issue:
"TypeError: 'NoneType' object is not iterable" when running cem.py
Hi,

I simply run the examples/agents/cem.py demo script (from GitHub) like so:

```python3 examples/agents/cem.py```

after

```sudo -H pip install gym```

This is on Peppermint Linux. The demo runs fine, but in the end, there is this exception:

```
INFO: Finished writing results. You can upload them to the scoreboard via gym.upload('/tmp/cem-agent-results')
Exception ignored in: <bound method Viewer.__del__ of <gym.envs.classic_control.rendering.Viewer object at 0x7f506917c710>>
Traceback (most recent call last):
  File "/home/stefan/dev/gym-master/gym/envs/classic_control/rendering.py", line 143, in __del__
  File "/home/stefan/dev/gym-master/gym/envs/classic_control/rendering.py", line 62, in close
  File "/usr/local/lib/python3.5/dist-packages/pyglet/window/xlib/__init__.py", line 480, in close
  File "/usr/local/lib/python3.5/dist-packages/pyglet/gl/xlib.py", line 345, in destroy
  File "/usr/local/lib/python3.5/dist-packages/pyglet/gl/base.py", line 334, in destroy
  File "/usr/local/lib/python3.5/dist-packages/pyglet/gl/xlib.py", line 335, in detach
  File "/usr/local/lib/python3.5/dist-packages/pyglet/gl/lib.py", line 97, in errcheck
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 887, in _find_spec
TypeError: 'NoneType' object is not iterable
```



Discussion:
According to this issue https://github.com/openai/gym/issues/1094 you may not be calling `env.close()` at the end of your script.  If that doesn't work, please post a short script that reproduces the error so I can look at it.
As I said, I just run [the example from GitHub](https://github.com/openai/gym/blob/master/examples/agents/cem.py).

Resolving patch:
diff --git a/gym/wrappers/monitor.py b/gym/wrappers/monitor.py
--- a/gym/wrappers/monitor.py
+++ b/gym/wrappers/monitor.py
@@ -40,13 +40,6 @@ def reset(self, **kwargs):
 
         return observation
 
-    def close(self):
-        super(Monitor, self)._close()
-
-        # _monitor will not be set if super(Monitor, self).__init__ raises, this check prevents a confusing error message
-        if getattr(self, '_monitor', None):
-            self.close()
-
     def set_monitor_mode(self, mode):
         logger.info("Setting the monitor mode is deprecated and will be removed soon")
         self._set_mode(mode)
@@ -138,6 +131,12 @@ def _flush(self, force=False):
 
     def close(self):
         """Flush all monitor data to disk and close any open rending windows."""
+        super(Monitor, self).close()
+
+        # _monitor will not be set if super(Monitor, self).__init__ raises, this check prevents a confusing error message
+        if not hasattr(self, '_monitor'):
+            return
+
         if not self.enabled:
             return
         self.stats_recorder.close()


Test patch:
Repository: openai/gym

Issue:
2 x # TODO: depreciation warning in registration.py
`# TODO: Add the following deprecation warning after 2017/02/18`
and
`# TODO: Add the following deprecation warning after 2017/03/01`
(both unactioned) appear in gym/gym/envs/registration.py


Discussion:
@pzhokhov want to add those?

Resolving patch:
diff --git a/gym/envs/registration.py b/gym/envs/registration.py
--- a/gym/envs/registration.py
+++ b/gym/envs/registration.py
@@ -51,8 +51,7 @@ def __init__(self, id, entry_point=None, trials=100, reward_threshold=None, loca
         # BACKWARDS COMPAT 2017/1/18
         if tags.get('wrapper_config.TimeLimit.max_episode_steps'):
             max_episode_steps = tags.get('wrapper_config.TimeLimit.max_episode_steps')
-            # TODO: Add the following deprecation warning after 2017/02/18
-            # warnings.warn("DEPRECATION WARNING wrapper_config.TimeLimit has been deprecated. Replace any calls to `register(tags={'wrapper_config.TimeLimit.max_episode_steps': 200)}` with `register(max_episode_steps=200)`. This change was made 2017/1/31 and is included in gym version 0.8.0. If you are getting many of these warnings, you may need to update universe past version 0.21.3")
+            warnings.warn("DEPRECATION WARNING wrapper_config.TimeLimit has been deprecated. Replace any calls to `register(tags={'wrapper_config.TimeLimit.max_episode_steps': 200)}` with `register(max_episode_steps=200)`. This change was made 2017/1/31 and is included in gym version 0.8.0. If you are getting many of these warnings, you may need to update switch from universe 0.21.3 to retro (https://github.com/openai/retro)")
 
         tags['wrapper_config.TimeLimit.max_episode_steps'] = max_episode_steps
         ######
@@ -60,9 +59,7 @@ def __init__(self, id, entry_point=None, trials=100, reward_threshold=None, loca
         # BACKWARDS COMPAT 2017/1/31
         if timestep_limit is not None:
             max_episode_steps = timestep_limit
-            # TODO: Add the following deprecation warning after 2017/03/01
-            # warnings.warn("register(timestep_limit={}) is deprecated. Use register(max_episode_steps={}) instead.".format(timestep_limit, timestep_limit))
-        ######
+            warnings.warn("register(timestep_limit={}) is deprecated. Use register(max_episode_steps={}) instead.".format(timestep_limit, timestep_limit))
 
         self.max_episode_steps = max_episode_steps
         self.max_episode_seconds = max_episode_seconds


Test patch:
Repository: openai/gym

Issue:
why MultiDiscrete  dtype=int8 
MultiDiscrete  dtype is hardcoded as int8. 
Is this a bug ? What happens when  we need more than 256 category  
```
space = MultiDiscrete([7, 1024])
space.sample()
```

https://github.com/openai/gym/blob/e944885e3b31a10fb6973093b39ff7682ef3aa3d/gym/spaces/multi_discrete.py#L10


Discussion:
That is a little surprising, since normal discrete is int64, @pzhokhov should this be set to an int64?

Resolving patch:
diff --git a/gym/spaces/multi_discrete.py b/gym/spaces/multi_discrete.py
--- a/gym/spaces/multi_discrete.py
+++ b/gym/spaces/multi_discrete.py
@@ -1,4 +1,3 @@
-import gym
 import numpy as np
 from .space import Space
 
@@ -29,9 +28,9 @@ def __init__(self, nvec):
         nvec: vector of counts of each categorical variable
         """
         assert (np.array(nvec) > 0).all(), 'nvec (counts) have to be positive'
-        self.nvec = np.asarray(nvec, dtype=np.uint32)
+        self.nvec = np.asarray(nvec, dtype=np.int64)
 
-        super(MultiDiscrete, self).__init__(self.nvec.shape, np.uint32)
+        super(MultiDiscrete, self).__init__(self.nvec.shape, np.int64)
         self.np_random = np.random.RandomState()
 
     def seed(self, seed):
diff --git a/gym/spaces/space.py b/gym/spaces/space.py
--- a/gym/spaces/space.py
+++ b/gym/spaces/space.py
@@ -1,6 +1,3 @@
-import numpy as np
-
-
 class Space(object):
     """Defines the observation and action spaces, so you can write generic
     code that applies to any Env. For example, you can choose a random


Test patch:
Repository: openai/gym

Issue:
BUG - KellyCoinflip episode should end when the agent reaches the maximum wealth.
# Bug description
The episode does not end when agent reaches the maximum wealth. This is not consistent with the information from the docstring. 

A Monte Carlo approach can also be used to illustrate the problem. The distribution of realized rewards obtained by an agent betting according to the Kelly criterion is incorrect. E.g. the docstring indicates such agent should earn $240 on average, and the bug leads to a much lower mean.

# To reproduce the bug
## Code
```
import gym 

env = gym.make('KellyCoinflip-v0')
env.seed(1)
env.reset()
done=False

while not done:
    action = int(env.wealth * 20)  # bet 20% of the wealth
    observation, reward, done, info = env.step(action)
    env.render()
```

## Output
```
Current wealth:  20.0 ; Rounds left:  299
[...]
Current wealth:  238.47 ; Rounds left:  192
Current wealth:  250.0 ; Rounds left:  191
Current wealth:  200.0 ; Rounds left:  190
[...]
```

## Expected output 
```
Current wealth:  20.0 ; Rounds left:  299
[...]
Current wealth:  238.47 ; Rounds left:  192
Current wealth:  250.0 ; Rounds left:  191
```

# Proposed modifications
I would like to work on a PR with the following modifications:
* fix the bug described above,
* refactor to reduce code duplication,
* rename some variables to follow PEP-8 guidelines. 


Discussion:

Resolving patch:
diff --git a/gym/envs/toy_text/kellycoinflip.py b/gym/envs/toy_text/kellycoinflip.py
--- a/gym/envs/toy_text/kellycoinflip.py
+++ b/gym/envs/toy_text/kellycoinflip.py
@@ -177,7 +177,7 @@ def step(self, action):
             self.losses += 1
 
         done = self.wealth < 0.01 or self.wealth == self.max_wealth or not self.rounds
-        reward = self.max_wealth if done else 0.0
+        reward = self.wealth if done else 0.0
 
         return self._get_obs(), reward, done, {}
 


Test patch:
Repository: openai/gym

Issue:
[Question] Typo error in gym/core.py?
Greetings,

I'm trying to set up an environment from scratch and I noticed a potential error in the docstring of the step method of Env. Shouldn't the action argument come from the agent and not the environment( as stated in the docstring)? Or maybe I just understood something wrong.

Cheers


Discussion:
A bit of both. You are right in that the argument comes from the agent. However, it has to be an element of  within the action space (which is an instance of a gym/spaces class). The space itself is owned by the environment. As I understand it, this is what the docstring refers to.

So in short, yes the action is chosen by the agent, no the comment is correct. Though I do see where the confusion is coming from and it should probably be adjusted.
Yes the possibility space is within the Environment but it's not the environment itself picking which precise action to do. 

In that sense something along the lines of : 
" Action provided by the agent ( within the defined action_space)" 

Or something similar would be more helpful than insinuating that it is the environment itself that is choosing which action to perform.

Resolving patch:
diff --git a/gym/core.py b/gym/core.py
--- a/gym/core.py
+++ b/gym/core.py
@@ -47,7 +47,7 @@ def step(self, action):
         Accepts an action and returns a tuple (observation, reward, done, info).
 
         Args:
-            action (object): an action provided by the environment
+            action (object): an action provided by the agent
 
         Returns:
             observation (object): agent's observation of the current environment


Test patch:
Repository: openai/gym

Issue:
VectorEnv keeps process from exiting
The following script hangs forever:

```
from gym.vector import make

def main():
    venv = make("PongNoFrameskip-v4", num_envs=3)
    venv.reset()
    while True:
        obs, rews, dones, infos = venv.step(venv.action_space.sample())
        raise Exception("oh no")

if __name__ == "__main__":
    main()
```

Baselines has some setting to deal with this: https://github.com/openai/baselines/blob/master/baselines/common/vec_env/subproc_vec_env.py#L55


Discussion:
Possibly related, also hangs:

```
from gym.vector import make

venv = make("PongNoFrameskip-v4", num_envs=3)
```

Resolving patch:
diff --git a/gym/vector/async_vector_env.py b/gym/vector/async_vector_env.py
--- a/gym/vector/async_vector_env.py
+++ b/gym/vector/async_vector_env.py
@@ -97,7 +97,7 @@ def __init__(self, env_fns, observation_space=None, action_space=None,
                 self.parent_pipes.append(parent_pipe)
                 self.processes.append(process)
 
-                process.deamon = True
+                process.daemon = True
                 process.start()
                 child_pipe.close()
 


Test patch:
Repository: openai/gym

Issue:
Support wrapper transformations to VecEnv
@tristandeleu @pzhokhov @christopherhesse It might be rather convenient for the user experience to provide a list of wrapper transformations for the atomic environments when creating vectorized environment, e.g.

```python
transforms = [AtariPreprocessing, SignReward, ...]
env = gym.vector.make('Pong-v0', 16, True, transforms=transforms)
```
For additional arguments, the user is required to use `partial()` to define them within the transform list. So that each internal environment is wrapped according to the transformation list. 


Discussion:
I have no strong opinion against having an option to add wrappers in `gym.vector.make`. I'd leave that up to @pzhokhov and @christopherhesse. Regarding naming, we just have to keep in mind that every argument added to `make` is one that cannot be overwritten (as part of `**kwargs`) for the environment; there is already `num_envs` and `asynchronous` that cannot be overwritten.

For anything complex, I would encourage users to use `AsyncVectorEnv` (or `SyncVectorEnv`) directly, with their own custom `env_fns` functions.

Resolving patch:
diff --git a/gym/vector/__init__.py b/gym/vector/__init__.py
--- a/gym/vector/__init__.py
+++ b/gym/vector/__init__.py
@@ -1,10 +1,15 @@
+try:
+    from collections.abc import Iterable
+except ImportError:
+    Iterable = (tuple, list)
+
 from gym.vector.async_vector_env import AsyncVectorEnv
 from gym.vector.sync_vector_env import SyncVectorEnv
 from gym.vector.vector_env import VectorEnv
 
 __all__ = ['AsyncVectorEnv', 'SyncVectorEnv', 'VectorEnv', 'make']
 
-def make(id, num_envs=1, asynchronous=True, **kwargs):
+def make(id, num_envs=1, asynchronous=True, wrappers=None, **kwargs):
     """Create a vectorized environment from multiple copies of an environment,
     from its id
 
@@ -20,6 +25,10 @@ def make(id, num_envs=1, asynchronous=True, **kwargs):
         If `True`, wraps the environments in an `AsyncVectorEnv` (which uses 
         `multiprocessing` to run the environments in parallel). If `False`,
         wraps the environments in a `SyncVectorEnv`.
+        
+    wrappers : Callable or Iterable of Callables (default: `None`)
+        If not `None`, then apply the wrappers to each internal 
+        environment during creation. 
 
     Returns
     -------
@@ -38,6 +47,15 @@ def make(id, num_envs=1, asynchronous=True, **kwargs):
     """
     from gym.envs import make as make_
     def _make_env():
-        return make_(id, **kwargs)
+        env = make_(id, **kwargs)
+        if wrappers is not None:
+            if callable(wrappers):
+                env = wrappers(env)
+            elif isinstance(wrappers, Iterable) and all([callable(w) for w in wrappers]):
+                for wrapper in wrappers:
+                    env = wrapper(env)
+            else:
+                raise NotImplementedError
+        return env
     env_fns = [_make_env for _ in range(num_envs)]
     return AsyncVectorEnv(env_fns) if asynchronous else SyncVectorEnv(env_fns)


Test patch:
Repository: openai/gym

Issue:
Remove FireReset wrapper for atari environments



Discussion:

Resolving patch:
diff --git a/gym/wrappers/atari_preprocessing.py b/gym/wrappers/atari_preprocessing.py
--- a/gym/wrappers/atari_preprocessing.py
+++ b/gym/wrappers/atari_preprocessing.py
@@ -15,7 +15,6 @@ class AtariPreprocessing(gym.Wrapper):
     Specifically:
 
     * NoopReset: obtain initial state by taking random number of no-ops on reset. 
-    * FireReset: take action on reset for environments that are fixed until firing. 
     * Frame skipping: 4 by default
     * Max-pooling: most recent two observations
     * Termination signal when a life is lost: turned off by default. Not recommended by Machado et al. (2018).
@@ -105,12 +104,6 @@ def reset(self, **kwargs):
             if done:
                 self.env.reset(**kwargs)
 
-        # FireReset
-        action_meanings = self.env.unwrapped.get_action_meanings()
-        if action_meanings[1] == 'FIRE' and len(action_meanings) >= 3:
-            self.env.step(1)
-            self.env.step(2)
-
         self.lives = self.ale.lives()
         if self.grayscale_obs:
             self.ale.getScreenGrayscale(self.obs_buffer[0])


Test patch:
diff --git a/gym/wrappers/test_atari_preprocessing.py b/gym/wrappers/test_atari_preprocessing.py
--- a/gym/wrappers/test_atari_preprocessing.py
+++ b/gym/wrappers/test_atari_preprocessing.py
@@ -16,19 +16,21 @@ def test_atari_preprocessing_grayscale(env_fn):
     env1 = env_fn()
     env2 = AtariPreprocessing(env_fn(), screen_size=84, grayscale_obs=True, frame_skip=1, noop_max=0)
     env3 = AtariPreprocessing(env_fn(), screen_size=84, grayscale_obs=False, frame_skip=1, noop_max=0)
-    env1.reset()
-    # take these steps to imitate actions of FireReset logic
-    env1.step(1)
-    obs1 = env1.step(2)[0]
+    env1.seed(0)
+    env2.seed(0)
+    env3.seed(0)
+    obs1 = env1.reset()
     obs2 = env2.reset()
     obs3 = env3.reset()
     assert obs1.shape == (210, 160, 3)
     assert obs2.shape == (84, 84)
     assert obs3.shape == (84, 84, 3)
-    np.testing.assert_allclose(obs3, cv2.resize(obs1, (84, 84), interpolation=cv2.INTER_AREA))
+    assert np.allclose(obs3, cv2.resize(obs1, (84, 84), interpolation=cv2.INTER_AREA))
     obs3_gray = cv2.cvtColor(obs3, cv2.COLOR_RGB2GRAY)
     # the edges of the numbers do not render quite the same in the grayscale, so we ignore them
-    np.testing.assert_allclose(obs2[10:], obs3_gray[10:])
+    assert np.allclose(obs2[10:38], obs3_gray[10:38])
+    # the paddle also do not render quite the same
+    assert np.allclose(obs2[44:], obs3_gray[44:])
 
     env1.close()
     env2.close()
@@ -53,4 +55,4 @@ def test_atari_preprocessing_scale(env_fn):
                 assert (0 <= obs).all() and (obs <= max_obs).all(), 'Obs. must be in range [0,{}]'.format(max_obs)
                 step_i += 1
 
-            env.close()
\ No newline at end of file
+            env.close()
Repository: openai/gym

Issue:
Bug in PixelObservationWrapper 
Error log
```
    env = PixelObservationWrapper(env, pixels_only=True)
  File "/home/tsan/Desktop/gym/gym/wrappers/pixel_observation.py", line 89, in __init__
    pixels = self.env.render(**render_kwargs)
  File "/home/tsan/Desktop/gym/gym/core.py", line 233, in render
    return self.env.render(mode, **kwargs)
TypeError: render() got an unexpected keyword argument 'pixels'
```

Can be reproduced by running
```
import gym
from gym.wrappers.pixel_observation import PixelObservationWrapper  # pylint: disable=E0401

env = gym.make('Acrobot-v1')
env.reset()
env = PixelObservationWrapper(env, pixels_only=True)
env.step(0)
```


Discussion:
@zuoxingdong any thoughts?

Resolving patch:
diff --git a/gym/wrappers/pixel_observation.py b/gym/wrappers/pixel_observation.py
--- a/gym/wrappers/pixel_observation.py
+++ b/gym/wrappers/pixel_observation.py
@@ -86,7 +86,7 @@ def __init__(self,
 
         pixels_spaces = {}
         for pixel_key in pixel_keys:
-            pixels = self.env.render(**render_kwargs)
+            pixels = self.env.render(**render_kwargs[pixel_key])
 
             if np.issubdtype(pixels.dtype, np.integer):
                 low, high = (0, 255)


Test patch:
Repository: openai/gym

Issue:
Sampling Bug
Gym Version: 0.15.3
issue: Box samples numbers above the `high` parameter.

```
from gym.spaces import Box
observation_space = Box(low=-3, high=-1, shape=(9,), dtype='int')
print(observation_space.sample())
>> [ 0 -2  0 -2  0 -1  0 -2  0]
```
The current implementation samples float numbers from uniform distribution of [`low`, `high`] and then converts the resulting samples to desired `dtype`. This runs into the problem of sampling `low` parameter very rarely(and not uniformly) when `dtype` is `int`(as we are converting the floats back to int which results in ceil operation in case of negative numbers) i.e in the above example -3 is almost never sampled as most of the low sampled floats like -2.85, -2.9 get converted to -2.
https://github.com/openai/gym/blob/0cd9266d986d470ed9c0dd87a41cd680b65cfe1c/gym/spaces/box.py#L93-L97



Discussion:

Resolving patch:
diff --git a/gym/spaces/box.py b/gym/spaces/box.py
--- a/gym/spaces/box.py
+++ b/gym/spaces/box.py
@@ -93,6 +93,8 @@ def sample(self):
         sample[bounded] = self.np_random.uniform(low=self.low[bounded], 
                                             high=high[bounded],
                                             size=bounded[bounded].shape)
+        if self.dtype.kind == 'i':
+            sample = np.floor(sample)
 
         return sample.astype(self.dtype)
         


Test patch:
Repository: openai/gym

Issue:
Missing attrs in LazyFrames: dtype, shape
This is my own fault.. I managed to push the wrong branch in #1906 


Discussion:

Resolving patch:
diff --git a/gym/wrappers/frame_stack.py b/gym/wrappers/frame_stack.py
--- a/gym/wrappers/frame_stack.py
+++ b/gym/wrappers/frame_stack.py
@@ -22,11 +22,11 @@ class LazyFrames(object):
     __slots__ = ('frame_shape', 'dtype', 'shape', 'lz4_compress', '_frames')
 
     def __init__(self, frames, lz4_compress=False):
+        self.frame_shape = tuple(frames[0].shape)
+        self.shape = (len(frames),) + self.frame_shape
+        self.dtype = frames[0].dtype
         if lz4_compress:
             from lz4.block import compress
-            self.frame_shape = tuple(frames[0].shape)
-            self.dtype = frames[0].dtype
-            self.shape = (len(frames),) + self.frame_shape
             frames = [compress(frame) for frame in frames]
         self._frames = frames
         self.lz4_compress = lz4_compress


Test patch:
diff --git a/gym/wrappers/test_frame_stack.py b/gym/wrappers/test_frame_stack.py
--- a/gym/wrappers/test_frame_stack.py
+++ b/gym/wrappers/test_frame_stack.py
@@ -21,6 +21,7 @@ def test_frame_stack(env_id, num_stack, lz4_compress):
     shape = env.observation_space.shape
     env = FrameStack(env, num_stack, lz4_compress)
     assert env.observation_space.shape == (num_stack,) + shape
+    assert env.observation_space.dtype == env.env.observation_space.dtype
 
     obs = env.reset()
     obs = np.asarray(obs)
@@ -34,3 +35,6 @@ def test_frame_stack(env_id, num_stack, lz4_compress):
     for i in range(1, num_stack - 1):
         assert np.allclose(obs[i - 1], obs[i])
     assert not np.allclose(obs[-1], obs[-2])
+
+    obs, _, _, _ = env.step(env.action_space.sample())
+    assert len(obs) == num_stack