Angle-based Convolution Networks for Extracting Local Spatial Features
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Convolutional Neural Networks (CNNs) have been a powerful feature extractingmethod for various machine learning problems, but it is limited to a regular gridshape spatial locations, e.g., pixels of images or GPS grids.
However, due tophysical or privacy preserving problems, spatio-temporal data in the real worldoften consist of irregular spatial locations.
To overcome this limitation, we propose
Angle-based Convolutional Networks (ACNs) that leverages the local featureextraction function of CNNs and the graph formulation of Graph Convolutional
Neural Networks (GCNs).
Our method considers angles among spatial locationsand introduces coefﬁcient weight for angle partitions that enables us to extract localfeatures appears in spatio-temporal data without regular grids
Introduction
Spatio-temporal data analysis has attracted considerable interest from a wide range of applications inwhich researchers tries to understand human activities in cities [8, 21], climate conditions [2, 23, 13],public health [16, 14] and animal behavior in natural environments [12, 10].
In these analyses, weassume that spatio-temporal data is a multivariate time series that provides where and when each datawas obtained.
Although spatio-temporal data inherit plenty of information about space and temporaldomains, its spatial location often has an irregular grid shape.
For example, when we attempt todeploy a sensor network system in a ﬁeld, it is almost unfeasible to put sensors regularly due tophysical and governmental problems.
The granularity of spatial locations is often limited to a certainrange, such as an administrative area, for protecting the anonymity of individuals [1].
This propertymakes spatio-temporal data analysis more complicated than other applications assuming a regulargrid shape such as pixels in images [4].
Convolutional Neural Networks (CNNs) [7] is a workhorse in machine learning and has intensivelycontributed to current improvements in spatio-temporal data analysis with regular spatial locations,such as GPS grids [17, 20, 13].
Traditionally, they have utilized 2-dimensional convolution layersof CNNs to extract local spatial features.
Intuitively, it consists of weight corresponding to speciﬁclocations of input data and learns local features by taking an inner product between its weight anddata patches.
However, CNNs are limited to a regular grid location [19] and cannot apply to irregularspatial locations that almost always appears in spatio-temporal data analysis.
Graph Convolutional Networks (GCNs) [6] is a recently proposed method to consider a graphstructure among data features.
GCNs can extract features from input data by taking an inner productamong an adjacent graph matrix and a linear transformation matrix.
Although GCNs have beensuccessfully applied to various kinds of spatio-temporal data analysis [9, 18], it has no function toextract local features from the graph structure as CNNs does on grids.
Because it does not consideran angle between a spatial location and another location.
a
[Figure]
aa
[Figure]
π
[Figure] [Figure]
a
[Figure]
a
[Figure]
a
[Figure]
a
[Figure]
… π /
[Figure]
weight
[Figure] [Figure]
In this paper, we propose an Angle-based Convolutional Networks (ACNs) that leverages both thevirtue of CNNs and GCNs to extract local features from spatio-temporal data with irregular spaciallocations.
By extending the convolutional layer of CNNs to consider both angles and distances amongspatial locations, we introduce an angle-based convolutional layer that divides 2 π radian into discretepartitions and assigns coefﬁcient weight w for speciﬁc partitions.
Then, from GPS information ofspatial locations, we obtain nearest neighbors of a spatial location and extract local features by takingan inner product between input data, weights, and similarities among locations.
We illustrate ourconvolutional layer in Figures 1.
We conduct experiments for imputing block missing observations ofreal-world spatio-temporal data illustrated in 2 and show that denoising autoencoder [15] with ourmethod improves the performances from that with GCNs and other baselines.
Angle-based Convolution Networks (ACNs)
In this section, we introduce an angle-based convolutional networks for analyzing spatio-temporaldata with irregular spatial locations.
Let us consider a case where we have spatio-temporal dataobserved from P locations with T timestamps.
We represent this multivariate time series as a matrix
X ∈ R P × T .
Here, we introduce notations for representing spatial information of sensor locations.
We denote asimilarity matrix
W ∈ R P × P between spatial locations whose elements w i,j ≥ 0 set to a positivevalue if a pair of locations ( i, j ) has a relationship, or set to 0 otherwise.
We split 2 π radian into Npartitions and put coefﬁcient weight a ∈ R N whose n -th element a n is a weight for the n -th anglepartition of [2 π ( n − 1) /N, 2 πn/N ] , see Figure 1 (c).
We additionally put a weight for a self-loop asa N +1 .
A bearing matrix
B ∈ R P × P indicates angles from the i -th location to the j -th location thatmeans b i,j = bearing(latitude i , longtitude i , latitude j , longtitude j ) [3].
We use B ( a ) to denotea matrix whose ( i, j ) -th element was set to B ( a ) i,j = a n if b i,j ∈ [2 π ( n − 1) /N, 2 πn/N ] .
Its diagonal elements were set to a N +1 .
We deﬁne our angle-based convolutional layer as:
[Equation]
where E j is a set of neighbors of the i -th location.
We illustrate an example of an our convolutionallayer in Figure 1 (c).
With a matrix representation, our convolutional layer can be represented as :
[Equation]
where ◦ is the Hadamard product of tensors and σ is a rectiﬁcation function, such as the sigmoidfunction or the ReLU functions.
For simplicity, the above deﬁnition only considers spatio-temporaldata with a single mode.
It is straightforward to extend to any number of modalities.
Related works
With the above formulation, a convolutional layer of GCNs [6] can be deﬁned as:
[Equation]
where V ∈ R T × T (cid:48) is a linear transformation matrix for the temporal domain,
˜ W = W
+ I P is amodiﬁed adjacent matrix with diagonal elements, I P is the identity matrix, and D i,j = (cid:80) i,j W i,j .
Different from Eq.
(2), it does not put any learnable weights for the adjacent graph matrix.
Therefore,this network is not able to extract local spatial features but just obtain a weighted mean of neighbors.
Li et al.
[9] proposed the diffusion convolution layer to consider spatial dependencies by usingbidirectional random walks.
Their method successfully improved the prediction accuracy by joiningtheir layer to a Recurrent Neural Networks (RNNs).
However, they have not consider an angle basedcoefﬁcients and thus their goal is different from ours.
Experiments
We conduct experiments for imputing block missing observations of spatio-temporal data as illustratedin Figure 2.
We employ trip record data sets provided by a bike sharing system 1 and NYC Taxi
Limousine commission 2 in New York City.
From the bike sharing data set, we selected P = 91 bikestations in Manhattan and counted the number of bikes returned at bike stations every hour from Jan.2017 to Dec. 2017 ( T = 8760 ) that means X ∈ R 91 × 8569 .
We measured the road distance amongall bike stations and constructed an adjacent matrix W by k-nearest-neighbors with k = 5 whereweights were set to w i,j = 1 for simplicity.
From the taxi data set, we obtained the number of yellowand green cabs dropped their passenger at a taxi zone every hour from Jan. 2017 to Dec. 2017 andselected P = 66 zones in Manhattan.
Thus, we constructed a matrix X ∈ R 66 × 8760 .
We checked aroad map of NYC and set w i,j = 1 if the i -th and j -th zones are connected by a road or w i,j = 0otherwise.
We randomly picked p % ( p = 0 .
3 , 0 . 5) of bike stations or taxi zones and used them asnew locations meaning missing blocks.
We treated the rest as existing locations.
Each row vectors of X p, : are divided by its maximum value.
Although there are various kinds of NNs for missing value imputation problems [11], we simplyemployed the Denoising Auto Encoder (DAE) [15] in this paper.
We made a corrupted input data˜ X for DAE by setting the values of missing blocks of X to 0 . In training phase, we randomlyset p % elements in exiting locations to 0 .
We constructed DAE models with our layer, GCN layerand Fully Connected layer (FC).
As a proposed method, we employ a simple network architecturethat consists of angle convolutional layers with 16 ﬁlters, 1 × 1 convolutional layers, and a skipconnection illustrated in Figure 3.
Since the weight of our proposed layer depends on only anglepartitions, we trained our method with observed data from existing locations.
Then used the learnedweight for imputing missing blocks where.
For DAE with other layers, we trained each modelto minimize the mean squared error between observations of existing locations and its estimationmin Θ (cid:80) p ∈ E P (cid:80)
Tt =1 ( X p,t − g ( ˜ X p,t )) 2 ) , where E P is a set of exiting spatial locations, Θ is aparameter, and g is a function of DAE. [Figure]
[Table]
We used the ReLU function as the rectiﬁcation function in experiments.
We optimized all modelswith Adam [5] where the learning rate is set to 0 .
01 and the size of mini batch was set [-to
-]to 168 = 24 ∗ 7 .
The number of epoch was set to 100 .
For our method, the number of partitions was set to N = 4 .
For DAE with GCNs (GCNs), we used a network architecture consists of two GCN layers where T (cid:48)
= 4 and T (cid:48)
= 168 .
For FC, we used two fully connected layer that project P from ( P → P/ 2)and ( P/ 2 → P/ 4) to extract low-rank embeddings.
We also employed means of all observed values(Means) and means of observed values of kNNs (Means kNNs) as baselines.
We run our experimentsﬁve times and measured the Root Mean Squared Error (RMSE) for new locations.
We show theresults in Table 1.
From the result, we conﬁrmed that our proposed method showed the best performances on p = 0 .
3and p = 0 .
5 of both data sets.
GCNs placed second in all settings.
This result indicates the beneﬁtsof utilizing a graph structure for spatio-temporal data analysis as reported in existing papers andsuggests a beneﬁt of our proposed layer.
It worked better than just getting a weighted means ofneighbor locations by considering angles among spatial locations.
Discussion
In this paper, we introduced angle-based convolutional networks that enable us to extract local featuresfrom spatio-temporal data with irregular spatial locations.
We deﬁned an angle-based convolutionallayer as an extension of CNNs and GCNs that leverages angles and similarities of spatial locations.
We demonstrated preliminary experiments for missing value imputation problems with two real-worldspatio-temporal data sets.
Since we ﬁxed various hyperparameters in experiments, further evaluationwith carefully tuned hyperparameter should be done to conﬁrm the difference between ours andexiting methods ﬁrmly.
Other graph-based missing value imputation methods such as Kernelized
Probabilistic Matrix Factorization [22] should be employed as additional baselines.
Moreover, forour method, we only employed 1 × 1 convolutional layer to combine extracted features with ourlayer.
This approach may limit the ﬂexibility of our layer.
Thus we are going to develop another layermore suitable to extract local spatial features.
For future works, we are going to extend our method tomultimodal settings and learn complex local relations among different modalities.
It should be animportant task to utilize RNNs for considering temporal dynamics .