LongiControl: A New Reinforcement Learning Environment

Reinforcement Learning (RL) is a subfield of machine learning for artificially intelligent systems to solve a variety of complex problems [1].Recent years have seen a surge of applicative successes using RL to solve challenging games and smaller domain problems [2][3][4].These successes in RL have been achieved in part due to the strong collaborative effort by the RL community to work on common, open-sourced environment simulators such as OpenAI's Gym [5] that allow for expedited development and valid comparisons between different, state-of-art strategies.However many existing environments contain games rather than real-world problems.Only recent publications initiate the transition to application-oriented RL [6][7].In this contribution, we aim to bridge real-world motivated RL with easy accessibility inside a highly relevant problem: the longitudinal control of an autonomous vehicle.Autonomous driving is the future, but until autonomous vehicles find their way in the stochastic real world independently, there are still numerous problems to solve.The longitudinal control problem has various challenges.One example is the trade-off between conflicting goals of travel time minimization and energy consumption.They contradict each other because a fast driving vehicle leads to highenergy consumption and vice versa.Another challenge is compliance with speed limits.The stochastic speed limits represent legal speed limits as well as the inherent limitation of speed due to other road users.Compliance with these limits is challenging because exceeding the limits can have a positive effect on travel time and energy consumption.In the real world, however, exceeding speed limits can lead to accidents and is therefore unacceptable.As a result, the newly introduced environment is also suitable for testing RL safety algorithms.The LongiControl environment consists of a data based electric vehicle model and a single-lane track with stochastic speed restrictions.The state of the agent includes the actual speed, previous acceleration, current speed limit and at most the next two speed limits as long as they are within a visual range of 150m.The agent selects the acceleration of the vehicle and receives as a reward a combination of speed, energy consumption, jerk and a measure for speeding.Through the proposed RL environment, which is adapted to the OpenAi Gym standardization, we show that it is easy to prototype and implement state-of-art RL algorithms.Besides, the LongiControl environment is suitable for various examinations.In addition to the comparison of RL algorithms and the evaluation of safety algorithms, investigations in the area of Multi-Objective Reinforcement Learning are also possible.Further possible research objectives are the comparison with planning algorithms for known routes, investigation of the influence of model uncertainties and the consideration of very long-term objectives like arriving at a specific time.LongiControl is designed to enable the community to leverage the latest strategies of reinforcement learning to address a real-world and high-impact problem in the field of autonomous driving.