## 3 Problem Statement and Preliminary Analysis
### Data Description  
Smart card data stores original transactions about metro ridership, where a transaction refers to either tap-in or tap-out of stations. Data was extracted from the AFC system of Shenzhen Metro Cooperation (Shenzhen Metro Cooperation: [[http://www.szmc.net/](http://www.szmc.net/)]([http://www.szmc.net/](http://www.szmc.net/))) from August to November in 2013, consisting of over 330 millions transactions in total with more than 2 million passengers per day. In 2013, there were five metro lines and 118 stations in Shenzhen. Table 2 lists the attributes of AFC transaction data used in this paper. As shown in Figure 1a, there are 7 days of incomplete data as highlighted in red: data for August 9 th (Fri) are totally missing, transactions on Aug 1 th (Thu), Sept 2 nd (Mon), Sept 3 rd (Tue), Sept 5 th (Thu) and Sept 9 th (Mon) are incomplete, and the number of transactions falls off drastically on Nov 12 (Tue) out of a sudden because of Typhoon Haiyan ([[https://en.wikipedia.org/wiki/Typhoon_Haiyan](https://en.wikipedia.org/wiki/Typhoon_Haiyan)]([https://en.wikipedia.org/wiki/Typhoon_Haiyan](https://en.wikipedia.org/wiki/Typhoon_Haiyan))). Note that it is reasonable to assume that the reduction of passenger flow at the beginning of September and October is due to the school season and the National Day holiday respectively. In order to eliminate \begin{table}
\begin{tabular}{l l} \hline \hline
**Notation** & **Explanation** \\ \hline \(D\) & Dataset \\ \(n\) & Number of objects in \(D\) \\ \(K\) & Number of clusters \\ \(\mathbf{\pi}=\{C_{1},C_{2},\ldots,C_{K}\}\) & Clustering (or partition) \\ \(C_{k},k=1,2,\ldots,K\) & Cluster with label \(k\) \\ \(c(k)\) & Centroid of cluster \(C_{k}\) \\ \(l(x_{i})\) & Cluster label of \(x_{i}\) \\ \(\mathcal{L}=\{l(x_{i})\},\forall x_{i}\in D\) & Cluster vector of members \\ \hline \hline \end{tabular}
\end{table} 
Table 1: List of notations. \begin{table}
\begin{tabular}{l l} \hline \hline
**Field** & **Description** \\ \hline _trans\_id_ & Identifies a transaction \\ _card\_id_ & Identifies a passenger \\ _line\_id_ & Identifies a metro line \\ _line\_name_ & Name of metro line \\ _station\_name_ & Name of metro station \\ _trans\_type_ & Indicates either in or out of station \\ _trans\_timeslamp_ & Datetime Timestamp of transaction \\ \hline \hline \end{tabular}
\end{table} 
Table 2: List of AFC data fields. influences of extreme values, we use the average volume over days of week respectively to deal with inaccuracy data issue, as shown in Figure 1b. Specifically, we preprocess incomplete data during given time window at given station using the average passenger flow volume over the same days of week during the same time window. For example, the volume at the station Huizhanzhongxin at 8:00 a.m. on 12 November (Tue) would be corrected by the average in-station volume at Huizhanzhongxin at 8:00 a.m. over all Tuesdays. In order to understand the passenger flow at metro stations, we construct Station Flow Data (SFD) from Shenzhen metro smart card data, which reflects streaming tap-in or tap-out passenger flow at stations. We denote hourly tap-in passenger flow data as SFD-in and hourly tap-out flow as SFD-out. Both SFD-in and SFD-out have 118 rows (stations) and 2928 columns (122 days*24 h). For each \(x_{j}\in\) SFD-in or SFD-out, \(j=1,2,\ldots,n\), \(n=118\), the assignment of \(x_{j}\) is denoted as \(l(x_{j})\), meaning which cluster \(x_{j}\) belongs to. Therefore, the objective is to discover accurate partitions for stations. Each data point in SFD matrix is normalized as  
\[x^{\prime}_{s,h}=\frac{x_{s,h}-\min<x>}{\max<x>-\min<x>}, \tag{1}\]  
where \(x_{s,h}\) denotes the passenger volume at \(h\)-th time slot of station \(s\), \(<x>\) means the vector of stations, and \(x^{\prime}_{s,h}\) is the normalized value within range \([0,1]\). Figure 1: Aggregated daily transactions in Shenzhen metro smart card data of all transactions over 4-month period, where the black line represents in-station volume, and the dashed blue line denotes out-station volume.