  Learned indices have been proposed to replace classic index structures like
B-Tree with machine learning (ML) models. They require to replace both the
indices and query processing algorithms currently deployed by the databases,
and such a radical departure is likely to encounter challenges and obstacles.
In contrast, we propose a fundamentally different way of using ML techniques to
improve on the query performance of the classic R-Tree without the need of
changing its structure or query processing algorithms. Specifically, we develop
reinforcement learning (RL) based models to decide how to choose a subtree for
insertion and how to split a node, instead of relying on hand-crafted heuristic
rules as R-Tree and its variants. Experiments on real and synthetic datasets
with up to 100 million spatial objects clearly show that our RL based index
outperforms R-Tree and its variants.
