  Electric Vehicle (EV) has become a preferable choice in the modern
transportation system due to its environmental and energy sustainability.
However, in many large cities, EV drivers often fail to find the proper spots
for charging, because of the limited charging infrastructures and the
spatiotemporally unbalanced charging demands. Indeed, the recent emergence of
deep reinforcement learning provides great potential to improve the charging
experience from various aspects over a long-term horizon. In this paper, we
propose a framework, named Multi-Agent Spatio-Temporal Reinforcement Learning
(Master), for intelligently recommending public accessible charging stations by
jointly considering various long-term spatiotemporal factors. Specifically, by
regarding each charging station as an individual agent, we formulate this
problem as a multi-objective multi-agent reinforcement learning task. We first
develop a multi-agent actor-critic framework with the centralized attentive
critic to coordinate the recommendation between geo-distributed agents.
Moreover, to quantify the influence of future potential charging competition,
we introduce a delayed access strategy to exploit the knowledge of future
charging competition during training. After that, to effectively optimize
multiple learning objectives, we extend the centralized attentive critic to
multi-critics and develop a dynamic gradient re-weighting strategy to
adaptively guide the optimization direction. Finally, extensive experiments on
two real-world datasets demonstrate that Master achieves the best comprehensive
performance compared with nine baseline approaches.
