## 4 Spatiotemporal Rebalancing Pricing Algorithm
### Actor Module  
In Figure 1b, we demonstrate the framework of the actor module. The actor module is no longer a traditional MLP but introduces GRU [11], embedded models, and GCN modules [13; 14]. The input data of the actor module are the state in the environment, defined as a multidimensional vector in Section 3.3. In addition, we found that the more time information is contained, the more beneficial it is for the model to develop incentive strategies. When this conclusion maps to the real world, the state change in the bike network has a certain delay due to the bike journey with time span; it is not instantaneous. Facing the historical state sequence, we use the GRU module to extract relevant time characteristics.  
\[X_{t}=GRU(S_{t-K},S_{t-K+1},\ldots,S_{t}), \tag{13}\]  
where \(0\leq K\leq T\) indicates the number of historical stages.  
In addition, the context information in the bicycle environment, such as weather and temperature, is also related to time. We encode the context information and add it to the feature after extracting time information through the embedded model \(E\).  
\[\tilde{X}_{t}=X_{t}+E. \tag{14}\]  
The attributes of nodes that constitute the state vector change along with the relationship between nodes. The environment part of Figure 1a shows the partial node relationship diagram. Information that can be obtained from the diagram is \(N(v_{6})=\{v_{1},v_{4},v_{5}\}\) and \(N(v_{4})=\{v_{3},v_{7}\}\). The incentive policy of node \(v_{6}\) receives the influence of \(v_{1},v_{4},v_{5}\) and then the secondary effects of \(v_{2},v_{3},v_{5},v_{7}\). Therefore, the graph structure in the entire network eventually affects the incentive actions for each station. We define matrix \(A\) to represent the relationship between nodes in the network. \(\tilde{A}\) represents the relational matrix with self-edges, with \(\tilde{A}=A+I\), where \(I\) is the identity matrix. In graph theory, \(A\) represents the relationship between first-order neighbors, and \(A^{2}\) represents the relationship between second-order neighbors. The graph convolution network contains the information of matrix \(A\), so it can combine the characteristics of nearby nodes. Therefore, the formula of the graph convolution network is as follows:  
\[h^{(l)}=GCN(h^{(l-1)})=\sigma(Lh^{(l-1)}\Theta_{l}+b_{l})\in R^{N\times C^{ \prime}}, \tag{15}\]  
where \(L\) is the Laplacian matrix of \(A\) when \(h^{(l-1)}\in R^{N\times C}\) is the input of the \(l_{th}\) graph convolutional layer, \(h^{(l)}\) is its output, \(\Theta_{l}\in R^{C\times C^{\prime}}\), \(b_{l}\in R^{C^{\prime}}\) are learnable parameters, and \(\sigma\) represents the activation function, for example, tanh. \(h^{(0)}\) represents \(\tilde{X}_{t}\). Not only does graph convolution network extract the spatial characteristics of nodes, but it also reduces the difficulties caused by dimensions. Due to the limitation of matrix \(A\), the price (action) of every node is just related to several adjacent nodes rather than the whole nodes.  
Actor module contains GRU, the embedded model, and GCN. The output is the incentive strategy (_action_). When the difference between any two incentive strategies \(Ac\) and \(Ac^{\prime}\) is very small, the feedback gap from Bike Gym is not large during the model's learning process. Therefore, we add a small discrete function after the action value, preventing Bike Gym from falling into a dead cycle. The incentive strategy is inputted into the step function of Bike Gym.