## 2 Methodology
### Classification Block
#### 2.1.1 Transformer Encoder  
The Transformer is wholly based on the attention mechanism, which can simultaneously obtain global and weighted information. Moreover, its multi-head mechanism can map input features from different perspectives. Undoubtedly, its expression ability becomes stronger, which is able to better extract the deep features for time-series problems [16]. Inspired by its powerful ability, we apply Transformer for feature extraction to achieve better clustering results for station classification with the pure passenger flow data in this paper. Furthermore, Transformer is mainly composed of an encoder and a decoder. Our research focuses on the task of passenger flow feature extraction rather than natural language processing (NLP) tasks, so we only use the encoder. To ensure the effectiveness of station classification, we use more than one week of inflow and outflow data as the inputs. As shown in Figure 2, the inflow and outflow data will be sent into the Transformer encoder, which can be expressed as a matrix \(X_{S\_W,T}^{P}\) shown in Equation (1). \[X_{S\_W,T}^{P}=\left\{ \begin{array}{cccc}x_{1\_1,\_1,\_1}&x_{1\_1,\_1,\_2}^{ P}&&x_{1\_1,\_1,\_M}^{P}\\ x_{1\_2\_1,\_1}&x_{1\_2\_2,\_1}^{P}&&x_{1\_2,\_1,\_M}^{P}\\ \vdots&\vdots&\ddots&\vdots\\ x_{S\_W,\_1,\_1}^{P}&x_{S\_W,\_1,\_2}^{P}&\cdots&x_{S\_W,\_1,\_M}^{P}\\ \end{array} \right\} \tag{1}\]  
Figure 2: The framework of the proposed (Transformer-K-Means)-(ResNet-GCN-AttLSTM). \[I_{C}=X_{S\_W,T}^{Inflow}\Big{|}\Big{|}X_{S\_W,T}^{Outflow} \tag{2}\]  
where \(S\) is the number of stations in the whole network, \(W\) is the number of weeks, and \(M=DS\times days\) is the number of time intervals during a week. Daily sample data (\(DS\)) are the number of time intervals per day; days are the number of days during a week. T = \([t_{1},t_{2},\ldots,t_{M}]\) is a series of time intervals in a week. \(P\in[\mathit{Inflow},\mathit{Outflow}]\) refers to the inflow or outflow patterns. \(X_{S\_W,T}^{Inflow}\in\mathbb{R}^{(S\times W)\times(DS\times days)}\), \(X_{S\_W,T}^{Outflow}\in\mathbb{R}^{(S\times W)\times(DS\times days)}\) represent the inflow data and outflow data, respectively. Inflow and outflow data will be concatenated by column, which is the inputs in the Transformer encoder (\(I_{C}\in\mathbb{R}^{(S\times W)\times(2\times DS\times days)}\)) shown in Equation (2). 1. Normalization First, \(I_{C}\) is standardized as \((I_{C})_{SD}\) based on Equation (3). \[(I_{C})_{SD}=\frac{I_{C}-(I_{C})_{mean}}{(I_{C})_{std}}\] (3) where \((I_{C})_{mean}\) is the mean value of \(I_{C}\); \((I_{C})_{std}\) is the standard deviation of \(I_{C}\). \((I_{C})_{SD}\in\mathbb{R}^{(S\times W)\times(2\times DS\times days)}\) with two dimensions will be transformed to \((I_{C})_{SD\_Day}\in\mathbb{R}^{(S\times W)\times days\times(2\times DS)}\) and \((I_{C})_{SD\_Interval}\in\mathbb{R}^{(S\times W)\times DS\times(2\times days)}\) with three dimensions, respectively. 2. Positional Encoding After that, we follow the positional encoding in the original Transformer model [16] to realize the position encoding of \((I_{C})_{SD\_Day}\) and \((I_{C})_{SD\_Interval}\) as \(PE\Big{(}(I_{C})_{SD\_Day}\Big{)}\) and \(PE\big{(}(I_{C})_{SD\_Interval}\big{)}\) based on Equations (4) and (5), respectively. \[PE_{(pos,2i)}=sin(\frac{pos}{10000^{\frac{2i}{model}}})\ (i\in(0,\,1/d_{model},\,pos\in(1,\,S\times W))\] (4) \[PE_{(pos,2i+1)}=cos(\frac{pos}{10000^{\frac{2i}{model}}})\ (i\in(0,\,1/d_{model},\,pos\in(1,\,S\times W))\] (5) where \(d_{model}\) in Equations (4) and (5) is set as \(2\times DS\), \(2\times days\) in \((I_{C})_{SD\_Day}\) and \((I_{C})_{SD\_Interval}\), respectively. \(PE\Big{(}(I_{C})_{SD\_Day}\Big{)}\in\mathbb{R}^{(S\times W)\times d_{model} }(d_{model}=2\times DS)\), and \(PE\big{(}(I_{C})_{SD\_Interval}\big{)}\in\mathbb{R}^{(S\times W)\times d_{ model}}(d_{model}=2\times days)\) are two dimensions, respectively. 3. Multi-head Attention Take \(PE\Big{(}(I_{C})_{SD\_Day}\Big{)}\) for example, \(PE\big{(}(I_{C})_{SD\_Interval}\big{)}\) has the same processing.