  Imagine an autonomous robot vehicle driving in dense, possibly unregulated
urban traffic. To contend with an uncertain, interactive environment with many
traffic participants, the robot vehicle has to perform long-term planning in
order to drive effectively and approach human-level performance. Planning
explicitly over a long time horizon, however, incurs prohibitive computational
cost and is impractical under real-time constraints. To achieve real-time
performance for large-scale planning, this paper introduces Learning from Tree
Search for Driving (LeTS-Drive), which integrates planning and learning in a
close loop. LeTS-Drive learns a driving policy from a planner based on
sparsely-sampled tree search. It then guides online planning using this learned
policy for real-time vehicle control. These two steps are repeated to form a
close loop so that the planner and the learner inform each other and both
improve in synchrony. The entire algorithm evolves on its own in a
self-supervised manner, without explicit human efforts on data labeling. We
applied LeTS-Drive to autonomous driving in crowded urban environments in
simulation. Experimental results clearly show that LeTS-Drive outperforms
either planning or learning alone, as well as open-loop integration of planning
and learning.
