Repository: openai/gym

Issue:
CarRacing-v0 - Weird Camera Jerk Issue
I noticed something weird when training my model on carracing-v0. Whenever a jerky braking situation is given, the camera flips vertically occasionally. Here is a picture of the same:

![flipped car](https://user-images.githubusercontent.com/30774147/99491692-340a8200-2992-11eb-997a-05a3f15a487a.png)

The left window shows the flipped image, and the one on the right is the frame previous to the current frame. The frame after the flipped frame is back to normal.

I noticed that there are a few odd lines in the environment's source file:

https://github.com/openai/gym/blob/c4d0af393ef9fba641bd3ebbbf1f60d291c8475d/gym/envs/box2d/car_racing.py#L421-L423

After some prodding, I noticed that occasionally the velocity of the car becomes negative as a result of which the atan function returns a negative number. The issue seems to be solved by commenting out those lines.

I do not understand the advantage of making the camera move with higher velocities since this can sometimes allow an agent to learn a lot more long range information by just having rapid movements. I observed this in some of my runs!

@olegklimov - Would be great if you could comment on this.


Discussion:
I conducted a little experiment that confirms the sign switching problem reported by @praveenVnktsh 

Using this code 
```python
        if np.linalg.norm(vel) > 0.5:
            old_angle = ((angle + np.pi) % (2 * np.pi)) - np.pi  # normalize angle
            angle = math.atan2(vel[0], vel[1])
            if np.sign(old_angle) != np.sign(angle):
                print(f"old_angle = {old_angle}, new_angle = {angle}")
```
I then proceeded to steer the car recklessly until I see some angle flipping. Of course, near 0, pi and -pi, I see some angle flipping inherent from numerical approximations, but there's other instances where there's clearly a "wrong" angle flip

```
old_angle = 1.0416961908340454, new_angle = -1.2111190759830692
old_angle = 1.0416961908340454, new_angle = -1.2111190759830692
old_angle = 1.2610578536987305, new_angle = -1.1509918860429935
old_angle = 1.2610578536987305, new_angle = -1.1509918860429935
old_angle = 1.4788283109664917, new_angle = -1.1005596690570452
old_angle = 1.4788283109664917, new_angle = -1.1005596690570452
old_angle = 1.694751262664795, new_angle = -1.0476754974926226
old_angle = 1.694751262664795, new_angle = -1.0476754974926226
old_angle = 1.9067429304122925, new_angle = -0.9976619854163612
old_angle = 1.9067429304122925, new_angle = -0.9976619854163612
old_angle = 2.112739086151123, new_angle = -0.9519961141207258
```

The bug can also be visualized in the following gif:

<img src="https://user-images.githubusercontent.com/16105100/145725175-54f152f9-0924-4920-ad48-b3ee5d1c23ff.gif" width="50%" height="50%" />

If we comment out the lines suggested by @praveenVnktsh I cannot reproduce the visual jerk effect. In the following gif the angle seems consistent with the movement at all times

<img src="https://user-images.githubusercontent.com/16105100/145725255-a8a4461c-00c1-416c-90c0-8285c84f4d2d.gif" width="50%" height="50%" />

After playing around with this for a few minutes and without further context on why these lines were there in the first place, my recommendation would be to remove them. In a visual learning context, the current state of the environment is plain wrong and should be fixed

Resolving patch:
diff --git a/gym/envs/__init__.py b/gym/envs/__init__.py
--- a/gym/envs/__init__.py
+++ b/gym/envs/__init__.py
@@ -88,7 +88,7 @@
 )
 
 register(
-    id="CarRacing-v0",
+    id="CarRacing-v1",
     entry_point="gym.envs.box2d:CarRacing",
     max_episode_steps=1000,
     reward_threshold=900,
diff --git a/gym/envs/box2d/car_racing.py b/gym/envs/box2d/car_racing.py
--- a/gym/envs/box2d/car_racing.py
+++ b/gym/envs/box2d/car_racing.py
@@ -46,9 +46,10 @@
 
 
 class FrictionDetector(contactListener):
-    def __init__(self, env):
+    def __init__(self, env, lap_complete_percent):
         contactListener.__init__(self)
         self.env = env
+        self.lap_complete_percent = lap_complete_percent
 
     def BeginContact(self, contact):
         self._contact(contact, True)
@@ -81,6 +82,14 @@ def _contact(self, contact, begin):
                 tile.road_visited = True
                 self.env.reward += 1000.0 / len(self.env.track)
                 self.env.tile_visited_count += 1
+
+                # Lap is considered completed if enough % of the track was covered
+                if (
+                    tile.idx == 0
+                    and self.env.tile_visited_count / len(self.env.track)
+                    > self.lap_complete_percent
+                ):
+                    self.env_new_lap = True
         else:
             obj.tiles.remove(tile)
 
@@ -144,9 +153,9 @@ class CarRacing(gym.Env, EzPickle):
         "video.frames_per_second": FPS,
     }
 
-    def __init__(self, verbose=1):
+    def __init__(self, verbose=1, lap_complete_percent=0.95):
         EzPickle.__init__(self)
-        self.contactListener_keepref = FrictionDetector(self)
+        self.contactListener_keepref = FrictionDetector(self, lap_complete_percent)
         self.world = Box2D.b2World((0, 0), contactListener=self.contactListener_keepref)
         self.viewer = None
         self.invisible_state_window = None
@@ -156,6 +165,7 @@ def __init__(self, verbose=1):
         self.reward = 0.0
         self.prev_reward = 0.0
         self.verbose = verbose
+        self.new_lap = False
         self.fd_tile = fixtureDef(
             shape=polygonShape(vertices=[(0, 0), (1, 0), (1, -1), (0, -1)])
         )
@@ -336,6 +346,7 @@ def _create_track(self):
             t.color = [ROAD_COLOR[0] + c, ROAD_COLOR[1] + c, ROAD_COLOR[2] + c]
             t.road_visited = False
             t.road_friction = 1.0
+            t.idx = i
             t.fixtures[0].sensor = True
             self.road_poly.append(([road1_l, road1_r, road2_r, road2_l], t.color))
             self.road.append(t)
@@ -370,6 +381,7 @@ def reset(self, *, seed: Optional[int] = None, options: Optional[dict] = None):
         self.prev_reward = 0.0
         self.tile_visited_count = 0
         self.t = 0.0
+        self.new_lap = False
         self.road_poly = []
 
         while True:
@@ -406,7 +418,7 @@ def step(self, action):
             self.car.fuel_spent = 0.0
             step_reward = self.reward - self.prev_reward
             self.prev_reward = self.reward
-            if self.tile_visited_count == len(self.track):
+            if self.tile_visited_count == len(self.track) or self.new_lap:
                 done = True
             x, y = self.car.hull.position
             if abs(x) > PLAYFIELD or abs(y) > PLAYFIELD:
@@ -440,9 +452,6 @@ def render(self, mode="human"):
         scroll_x = self.car.hull.position[0]
         scroll_y = self.car.hull.position[1]
         angle = -self.car.hull.angle
-        vel = self.car.hull.linearVelocity
-        if np.linalg.norm(vel) > 0.5:
-            angle = math.atan2(vel[0], vel[1])
         self.transform.set_scale(zoom, zoom)
         self.transform.set_translation(
             WINDOW_W / 2


Test patch: