Angle-based Convolution Networks for Extracting Local Spatial Features
Abstract
Convolutional Neural Networks (CNNs) have been a powerful feature extracting method for various machine learning problems, but it is limited to a regular grid shape spatial locations, e g., pixels of images or GPS grids.
However, due to physical or privacy preserving problems, spatio-temporal data in the real world often consist of irregular spatial locations.
To overcome this limitation, we propose
Angle-based Convolutional Networks (ACNs) that leverages the local feature extraction function of CNNs and the graph formulation of Graph Convolutional
Neural Networks (GCNs).
Our method considers angles among spatial locations and introduces coefﬁcient weight for angle partitions that enables us to extract local features appears in spatio-temporal data without regular grids.
Introduction
Spatio-temporal data analysis has attracted considerable interest from a wide range of applications in which researchers tries to understand human activities in cities [8, 21], climate conditions [2, 23, 13], public health [16, 14] and animal behavior in natural environments [10].
In these analyses, we assume that spatio-temporal data is a multivariate time series that provides where and when each data was obtained.
Although spatio-temporal data inherit plenty of information about space and temporal domains, its spatial location often has an irregular grid shape.
For example, when we attempt to deploy a sensor network system in a ﬁeld, it is almost unfeasible to put sensors regularly due to physical and governmental problems.
The granularity of spatial locations is often limited to a certain range, such as an administrative area, for protecting the anonymity of individuals [1].
This property makes spatio-temporal data analysis more complicated than other applications assuming a regular grid shape such as pixels in images [4].
Convolutional Neural Networks (CNNs) [7] is a workhorse in machine learning and has intensively contributed to current improvements in spatio-temporal data analysis with regular spatial locations, such as GPS grids [17, 20, 13].
Traditionally, they have utilized 2-dimensional convolution layers of CNNs to extract local spatial features.
Intuitively, it consists of weight corresponding to speciﬁc locations of input data and learns local features by taking an inner product between its weight and data patches.
However, CNNs are limited to a regular grid location [19] and cannot apply to irregular spatial locations that almost always appears in spatio-temporal data analysis.
[Equation]
Graph Convolutional Networks (GCNs) [6] is a recently proposed method to consider a graph structure among data features.
GCNs can extract features from input data by taking an inner product a
[Figure]
a
[Figure]
a
[Figure]
π
[Figure] [Figure]
a
[Figure]
a
[Figure]
… π
[Figure] [Figure]
among an adjacent graph matrix and a linear transformation matrix.
Although GCNs have been successfully applied to various kinds of spatio-temporal data analysis [9, 18], it has no function to extract local features from the graph structure as CNNs does on grids.
Because it does not consider an angle between a spatial location and another location.
In this paper, we propose an Angle-based Convolutional Networks (ACNs) that leverages both the virtue of CNNs and GCNs to extract local features from spatio-temporal data with irregular spacial locations.
By extending the convolutional layer of CNNs to consider both angles and distances among spatial locations, we introduce an angle-based convolutional layer that divides 2 π radian into discrete partitions and assigns coefﬁcient weight w for speciﬁc partitions.
Then, from GPS information of spatial locations, we obtain nearest neighbors of a spatial location and extract local features by taking an inner product between input data, weights, and similarities among locations.
We illustrate our convolutional layer in Figures 1.
We conduct experiments for imputing block missing observations of real-world spatio-temporal data illustrated in 2 and show that denoising autoencoder [15] with our method improves the performances from that with GCNs and other baselines.
Angle-based Convolution Networks (ACNs)
In this section, we introduce an angle-based convolutional networks for analyzing spatio-temporal data with irregular spatial locations.
Let us consider a case where we have spatio-temporal data observed from P locations with T timestamps.
We represent this multivariate time series as a matrix
X ∈ R P × T .
Here, we introduce notations for representing spatial information of sensor locations.
We denote a similarity matrix
W ∈ R P × P between spatial locations whose elements w i,j ≥ 0 set to a positive value if a pair of locations ( i, j ) has a relationship, or set to 0 otherwise.
We split 2 π radian into N partitions and put coefﬁcient weight a ∈ R N whose n -th element a n is a weight for the n -th angle partition of [2 π ( n − 1) /N, 2 πn/N ] , see Figure 1 (c).
We additionally put a weight for a self-loop as a N +1 .
A bearing matrix
B ∈ R P × P indicates angles from the i -th location to the j -th location that means b i,j = bearing(latitude i , longtitude i , latitude j , longtitude j ) [3].
We use B ( a ) to denote a matrix whose ( i, j ) -th element was set to B ( a ) i,j = a n if b i,j ∈ [2 π ( n − 1) /N, 2 πn/N ] .
Its diagonal elements were set to a N +1 .
We deﬁne our angle-based convolutional layer as:
[Equation]
i
[Equation]
where E j is a set of neighbors of the i -th location.
We illustrate an example of an our convolutional layer in Figure 1 (c).
With a matrix representation, our convolutional layer can be represented as :
[Equation]
where ◦ is the Hadamard product of tensors and σ is a rectiﬁcation function, such as the sigmoid function or the ReLU functions.
For simplicity, the above deﬁnition only considers spatio-temporal data with a single mode.
It is straightforward to extend to any number of modalities.
Related works
With the above formulation, a convolutional layer of GCNs [6] can be deﬁned as:
[Equation]
where V ∈ R T × T (cid:48) is a linear transformation matrix for the temporal domain,
˜ W = W
+ I P is a modiﬁed adjacent matrix with diagonal elements, I P is the identity matrix, and D i,j = (cid:80) i,j W i,j .
Different from Eq.
(3), it does not put any learnable weights for the adjacent graph matrix.
Therefore, this network is not able to extract local spatial features but just obtain a weighted mean of neighbors.
Li et al.
[9] proposed the diffusion convolution layer to consider spatial dependencies by using bidirectional random walks.
However, they have not consider an angle based coefﬁcients and thus their goal is different from ours.
In CV, a convolution-like operation on graph signals was proposed for point cloud classiﬁcations [12].
Their application did not include spatio-temporal data analysis.
Thus the focus is different from ours.
Experiments
We conduct experiments for imputing block missing observations of spatio-temporal data as illustrated in Figure 2.
We employ trip record data sets provided by a bike sharing system 1 and NYC Taxi
Limousine commission 2 in New York City.
From the bike sharing data set, we selected P = 91 bike stations in Manhattan and counted the number of bikes returned at bike stations every hour from Jan.
2017 to Dec. 2017 ( T = 8760 ) that means X ∈ R 91 × 8569 .
We measured the road distance among all bike stations and constructed an adjacent matrix W by k-nearest-neighbors with k = 5 where weights were set to w i,j = 1 for simplicity.
From the taxi data set, we obtained the number of yellow and green cabs dropped their passenger at a taxi zone every hour from Jan. 2017 to Dec. 2017 and selected P = 66 zones in Manhattan.
Thus, we constructed a matrix X ∈ R 66 × 8760 .
We checked a road map of NYC and set w i,j = 1 if the i -th and j -th zones are connected by a road or w i,j =otherwise.
We randomly picked p % ( p = 0 .
3 , 0 . 5) of bike stations or taxi zones and used them as new locations meaning missing blocks.
We treated the rest as existing locations.
Although there are various kinds of NNs for missing value imputation problems [11], we simply employed the Denoising Auto Encoder (DAE) [15] in this paper.
We made a corrupted input data ˜
X for DAE by setting the values of missing blocks of X to 0 .
In training phase, we randomly set p % elements in exiting locations to 0 .
We constructed DAE models with our layer, GCN layer and Fully Connected layer (FC).
As a proposed method, we employ a simple network architecture that consists of angle convolutional layers with 16 ﬁlters, 1 × 1 convolutional layers, and a skip connection illustrated in Figure 3.
Since the weight of our proposed layer depends on only angle partitions, we trained our method with observed data from existing locations.
Then used the learned weight for imputing missing blocks where.
For DAE with other layers, we trained each model
[Figure]
[Table]
to minimize the mean squared error between observations of existing locations and its estimation min Θ (cid:80) p ∈ E P (cid:80) Tt =1 ( X p,t − g ( ˜ X p,t )) 2 ) , where E P is a set of exiting spatial locations, Θ is a parameter, and g is a function of DAE.
We used the ReLU function as the rectiﬁcation function in experiments.
We optimized all models with Adam [5] where the learning rate is set to 0 .
01 and the size of mini batch was set to 168 = 24 ∗ 7 .
The number of epoch was set to 100 .
For our method, the number of partitions was set to N = 4 .
For DAE with GCNs (GCNs), we used a network architecture consists of two GCN layers where
T (cid:48)
= 4 and T (cid:48)
= 168 .
For FC, we used two fully connected layer that project P from ( P → P/ 2) and ( P/ 2 → P/ 4) .
We also employed means of all observed values (Means) and means of observed values of kNNs (Means kNNs) as baselines.
We run experiments ﬁve times and measured Root Mean Squared Error (RMSE) for new locations.
We show the results in Table 1.
From the result, we conﬁrmed that our proposed method showed the best performances on p = 0 .and
p = 0 .
5 of both data sets.
GCNs placed second in all settings.
This result indicates the beneﬁts of utilizing a graph structure for spatio-temporal data analysis as reported in existing papers and suggests a beneﬁt of our proposed layer.
It worked better than just getting a weighted means of neighbor locations by considering angles among spatial locations.
Discussion
We introduced angle-based convolutional networks that enable us to extract local features from spatio-temporal data with irregular spatial locations.
We deﬁned an angle-based convolutional layer as an extension of CNNs and GCNs that leverages angles and similarities of spatial locations.
We demonstrated preliminary experiments for block missing value imputation problems with two spatiotemporal data sets.
We ﬁxed various hyperparameters in experiments.
Further evaluations with carefully tuned hyperparameters should be done to conﬁrm the difference between ours and exiting methods ﬁrmly.
Other graph-based missing value imputation, such as Kernelized Probabilistic Matrix
Factorization methods [22] should be employed as additional baselines.
Acknowledgements : This work was supported by JST CREST Grant Number JPMJPR18UG, Japan.