## 1 Introduction
In Reference [22], by designing a marketing auction mechanism, a reinforcement learning algorithm was adopted to obtain an energy management strategy with minimized economic cost. A study of energy management in an office building with renewable energy resources was carried out in Reference [23], and an echo-state based reinforcement learning method was used to manage the output power of devices in an office building. In these studies, only electrical power management was involved and only economical cost was considered. However, as for multi-energy coupled EI, thermal power is involved so that environmental cost [24] needs to be taken into consideration in this paper. What is more, considering the physical entities, secure operation of the system is necessary to be discussed at the same time. Reinforcement learning method is adopted in this paper to schedule an energy-management-based energy routing path, and an artificial neural network is combined with reinforcement learning to avoid the curse of dimension. To summarize, the major contributions of this paper are:  
1. In an environment of multi-energy coupled EI, the concept of energy routing center (ERC) is proposed for the first time. As a core energy interaction entity, a two-layered structure is designed for ERC and their corresponding functions are depicted in this paper. In the physical layer, multi-energy conversion devices and connection ports allowing plug-and-play of multi-energy users are integrated. In the information layer, an energy routing unit is embedded to schedule energy management and routing design. The ERC provides users with a novel integrated multi-energy conversion node which improves the flexibility of energy interaction in EI. 2. Considering the physical connections among ERCs in EI, a multi-energy management-based optimal energy routing design problem considering operating cost, environmental cost, and security operation is researched in this paper. Specifically, in order to reduce the difficulty of control in reality, the voltages of ERC ports are managed to regulate power flow. Considering reality factors such as physical connection structure makes energy management problems adapt to the EI circumstances better. 3. Due to the fluctuations of renewable resources and users' demands, the topological relationship between load and source in multi-energy system varies frequently, therefore, reinforcement learning combined with artificial neural network (ANN) is adopted in the optimal energy routing design to form an energy routing path with high efficiency and lower costs. As a model free method, reinforcement learning does not rely on the priori knowledge of the environment which shows high efficiency. The remainder of the paper is organized as follows. Section 2 establishes a structure of the EI, which consists of multiple ERCs and identifies the inner structure and functions of the ERCs. Section 3 defines the connection weights of connection lines between ERCs on account of operating cost, environmental cost, and power transmission loss. In Section 4, an ANN-based Q-learning algorithm is adopted to solve energy-management-based energy routing design problems. Simulations were performed in Section 5 to validate the effectiveness of the proposed algorithm. Section 6 concludes this paper.