  Traditional autonomous vehicle pipelines that follow a modular approach have
been very successful in the past both in academia and industry, which has led
to autonomy deployed on road. Though this approach provides ease of
interpretation, its generalizability to unseen environments is limited and
hand-engineering of numerous parameters is required, especially in the
prediction and planning systems. Recently, deep reinforcement learning has been
shown to learn complex strategic games and perform challenging robotic tasks,
which provides an appealing framework for learning to drive. In this work, we
propose a deep reinforcement learning framework to learn optimal control policy
using waypoints and low-dimensional visual representations, also known as
affordances. We demonstrate that our agents when trained from scratch learn the
tasks of lane-following, driving around inter-sections as well as stopping in
front of other actors or traffic lights even in the dense traffic setting. We
note that our method achieves comparable or better performance than the
baseline methods on the original and NoCrash benchmarks on the CARLA simulator.
