Angle-based Convolution Networks for Extracting Local Spatial Features
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Convolutional Neural Networks (CNNs) have been a powerful feature extractingmethod for various machine learning problems, but it is limited to a regular gridshape spatial locations, e.g., pixels of images or GPS grids.
However, due tophysical or privacy preserving problems, spatio-temporal data in the real worldoften consist of irregular spatial locations.
To overcome this limitation, we propose
Angle-based Convolutional Networks (ACNs) that leverages the local featureextraction function of CNNs and the graph formulation of Graph Convolutional
Neural Networks (GCNs).
Our method considers angles among spatial locationsand introduces coefﬁcient weight for angle partitions that enables us to extract localfeatures appears in spatio-temporal data without regular grids
Introduction
Spatio-temporal data analysis has attracted considerable interest from a wide range of applications inwhich researchers tries to understand human activities in cities [8, 21], climate conditions [2, 23, 13],public health [16, 14] and animal behavior in natural environments [12, 10].
In these analyses, weassume that spatio-temporal data is a multivariate time series that provides where and when each datawas obtained.
Although spatio-temporal data inherit plenty of information about space and temporaldomains, its spatial location often has an irregular grid shape.
For example, when we attempt todeploy a sensor network system in a ﬁeld, it is almost unfeasible to put sensors regularly due tophysical and governmental problems.
The granularity of spatial locations is often limited to a certainrange, such as an administrative area, for protecting the anonymity of individuals [1].
This propertymakes spatio-temporal data analysis more complicated than other applications assuming a regulargrid shape such as pixels in images [4].
Convolutional Neural Networks (CNNs) [7] is a workhorse in machine learning and has intensivelycontributed to current improvements in spatio-temporal data analysis with regular spatial locations,such as GPS grids [17, 20, 13].
Traditionally, they have utilized 2-dimensional convolution layersof CNNs to extract local spatial features.
Intuitively, it consists of weight corresponding to speciﬁclocations of input data and learns local features by taking an inner product between its weight anddata patches.
However, CNNs are limited to a regular grid location [19] and cannot apply to irregularspatial locations that almost always appears in spatio-temporal data analysis.
Graph Convolutional Networks (GCNs) [6] is a recently proposed method to consider a graphstructure among data features.
GCNs can extract features from input data by taking an inner productamong an adjacent graph matrix and a linear transformation matrix.
Although GCNs have beensuccessfully applied to various kinds of spatio-temporal data analysis [9, 18], it has no function toextract local features from the graph structure as CNNs does on grids.
Because it does not consideran angle between a spatial location and another location.
a
[Figure]
aa
[Figure]
π
[Figure] [Figure]
a
[Figure]
a
[Figure]
a
[Figure]
a
[Figure]
… π /
[Figure]
weight
[Figure] [Figure]
In this paper, we propose an Angle-based Convolutional Networks (ACNs) that leverages both thevirtue of CNNs and GCNs to extract local features from spatio-temporal data with irregular spaciallocations.
By extending the convolutional layer of CNNs to consider both angles and distances amongspatial locations, we introduce an angle-based convolutional layer that divides 2 π radian into discretepartitions and assigns coefﬁcient weight w for speciﬁc partitions.
Then, from GPS information ofspatial locations, we obtain nearest neighbors of a spatial location and extract local features by takingan inner product between input data, weights, and similarities among locations.
We illustrate ourconvolutional layer in Figures 1.
We conduct experiments for imputing block missing observations ofreal-world spatio-temporal data illustrated in 2 and show that denoising autoencoder [15] with ourmethod improves the performances from that with GCNs and other baselines.
Angle-based Convolution Networks (ACNs)
In this section, we introduce an angle-based convolutional networks for analyzing spatio-temporaldata with irregular spatial locations.
Let us consider a case where we have spatio-temporal dataobserved from P locations with T timestamps.
We represent this multivariate time series as a matrix
X ∈ R P × T .
Here, we introduce notations for representing spatial information of sensor locations.
We denote asimilarity matrix
W ∈ R P × P between spatial locations whose elements w i,j ≥ 0 set to a positivevalue if a pair of locations ( i, j ) has a relationship, or set to 0 otherwise.
We split 2 π radian into Npartitions and put coefﬁcient weight a ∈ R N whose n -th element a n is a weight for the n -th anglepartition of [2 π ( n − 1) /N, 2 πn/N ] , see Figure 1 (c).
We additionally put a weight for a self-loop asa N +1 .
A bearing matrix
B ∈ R P × P indicates angles from the i -th location to the j -th location thatmeans b i,j = bearing(latitude i , longtitude i , latitude j , longtitude j ) [3].
We use B ( a ) to denotea matrix whose ( i, j ) -th element was set to B ( a ) i,j = a n if b i,j ∈ [2 π ( n − 1) /N, 2 πn/N ] .
Its diagonal elements were set to a N +1 .
We deﬁne our angle-based convolutional layer as:
[Equation]
where E j is a set of neighbors of the i -th location.
We illustrate an example of an our convolutionallayer in Figure 1 (c).
With a matrix representation, our convolutional layer can be represented as :
[Equation]
where ◦ is the Hadamard product of tensors and σ is a rectiﬁcation function, such as the sigmoidfunction or the ReLU functions.
For simplicity, the above deﬁnition only considers spatio-temporaldata with a single mode.
It is straightforward to extend to any number of modalities.
Related works
With above formulation, GCNs [6] can be deﬁned as:
[Equation]
where V ∈ R T × T (cid:48) is a linear transformation matrix for the temporal domain,
˜ W = W
+ I P is anadjacent matrix with added unit diagonal elements, I P is the identity matrix, and D i,j = (cid:80) i,j W i,j .
Different from Eq.
(2), it does not put any learnable weights for the adjacent graph matrix.
Therefore,this network is not able to extract local speciﬁc features from spatio-temporal data.
Li et al.
[9]proposed the diffusion convolution layer to consider spatial dependencies by using bidirectionalrandom walks.
Their method successfully improved the prediction accuracy by joining their layer toa Recurrent Neural Networks (RNNs).
However, they have not consider a bearing based coefﬁcientsand thus their goal is different from ours.
Experiments
We conduct experiments for imputing block missing observations of spatio-temporal data as illustratedin Figure 2.
We employ trip record data sets provided by a bike sharing system 1 and NYC Taxi
Limousine commission 2 in New York City.
From the bike sharing data set, we selected P = 91 bikestations in Manhattan and obtained the number of bikes returned at bike stations every hour from
Jan. 2017 to Dec. 2017 ( T = 8659 ) that means X ∈ R 91 × 8569 .
We measured the road distanceamong all bike stations and constructed an adjacent matrix W by k-nearest-neighbors with k = 5where weights were set to w i,j = 1 for simplicity.
From the bike sharing data set, we obtained thenumber of yellow and green cabs dropped their passenger at a taxi zone every hour for the sametime range of the previous data set and selected P = 66 zones in Manhattan.
Thus, we constructed amatrix X ∈ R 66 × 8569 .
We checked whether a road map of NYC and set w i,j = 1 if the i -th and j -thzones are connected by a road or w i,j = 0 otherwise.
We randomly picked p % ( p = 0 .
3 , 0 . 5) of bikestations or taxi zones and used them as new locations meaning missing blocks.
We treated the rest asexisting locations.
Each row vectors of X p, : are normalized by dividing by its maximum value.
Although there are various kinds of NNs for missing value imputation problems [11], we simplyemployed the Denoising Auto Encoder (DAE) [15] in this paper.
We made a corrupted input data ˜
Xfor DAE by setting the values of missing blocks of X to 0 and also randomly set 1 − p % elements inexiting locations to 0 .
We constructed DAE models with our layer, GCN layer and Fully Connectedlayer (FC).
As a proposed method, we employ a simple network architecture that consists of angleconvolutional layers with 16 ﬁlters, 1 × 1 convolutional layers, and a skip connection illustratedin Figure 3.
Since the weight of our proposed layer depends on only angle partitions, we trainedour weights from observed data from existing locations.
Then used the learned weight for imputingmissing blocks where.
For DAE with other layers, we trained each model to minimize the meansquared error between existing locations and its estimation min Θ (cid:80) p ∈ E P (cid:80) T t =1 ( X p,t − g ( ˜ X p,t )) 2 ) .where
E P is a set of exiting spatial locations and Θ is a parameter for DAE and g is a function for
DAE.
We used the ReLU function as the rectiﬁcation function in experiments.
We optimized all
[Figure]
[Table]
models with Adam [5] where the learning rate is set to 0 .
01 and the size of mini batch was set to
= 24 ∗ 7 .
The number of separations was set to N = 4 .
For DAE with GCNs (GCNs), we useda network architecture consists of two GCN layers where T (cid:48)
= 4 and T (cid:48)
= 168 .
For FC, we usedtwo fully connected layer that project P from ( P → P/ 2) and ( P/ 2 → P/ 4) to extract low-rankstructures.
We also employed means of all observed values (Means) and means of observed values ofkNNs (Means kNNs) as baselines.
We run our experiments ﬁve times and measured the Root Mean
Squared Error (RMSE) for new locations.
We show the results in Table 1.
From the result, we conﬁrmed that our proposed method showed the best performances on p = 0 .
3and p = 0 .
5 of both data sets.
GCNs placed second in all settings.
This result indicates the beneﬁtsof utilizing a graph structure for spatio-temporal data analysis as reported in existing papers andsuggests a beneﬁt of our proposed layer.
It worked better than just getting a weighted means ofneighbor locations by considering angles among spatial locations.
Discussion
In this paper, we introduced angle-based convolutional networks that enable us to extract local featuresfrom spatio-temporal data with irregular spatial locations.
We deﬁned an angle-based convolutionallayer as an extension of CNNs and GCNs that leverages angles and similarities of spatial locations.
We demonstrated preliminary experiments for missing value imputation problems with two real-worldspatio-temporal data sets.
Since we ﬁxed various hyperparameters in experiments, further evaluationwith carefully tuned hyperparameter should be done to conﬁrm the difference between ours andexiting methods ﬁrmly.
Other graph-based missing value imputation methods such as Kernelized
Probabilistic Matrix Factorization [22] should be employed as additional baselines.
Moreover, forour method, we only employed 1 × 1 convolutional layer to combine extracted features with ourlayer.
This approach may limit the ﬂexibility of our layer.
Thus we are going to develop another layermore suitable to extract local spatial features.
For future works, we are going to extend our method tomultimodal settings and learn complex local relations among different modalities.
It should be animportant task to utilize RNNs for considering temporal dynamics .