2016 Intl IEEE Conferences on Ubiquitous Intelligence & Computing, Advanced and Trusted Computing, Scalable Computing
and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress

Less Annotation on Personalized Activity
Recognition Using Context Data
Lisha Hu∗†‡ , Yiqiang Chen∗† , Shuangquan Wang∗† , Jindong Wang∗†‡ , Jianfei Shen∗†‡ , Xinlong Jiang∗†‡ , Zhiqi Shen§∗ Beijing Key Laboratory of Mobile Computing and Pervasive Device, Beijing, China† Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China‡ University of Chinese Academy of Sciences, Beijing, China§ School of Computer Science and Engineering, Nanyang Technological University, Singapore

Email:{hulisha,yqchen,wangshuangquan,wangjindong,shenjianfei,jiangxinlong}@ict.ac.cn, zqshen@ntu.edu.sg

To deal with the problem, a lot of incremental learning[7][8][9] and transfer learning methods [10][11] have been
proposed in literatures to adapt AR models to new domains
or individuals. In this paper, we focus on the incremental
learning mechanism. Generally, incremental learning method
consists of two stages to learn a classiﬁcation model [7].
An initial model is created ﬁrstly based on a few labeledinstances as the training set, and then updated automatically
if new (labeled/unlabeled) instances come. According to thevarious types of instances (labeled, unlabeled, unlabeled withnew features/attributes) which could be utilized in the second
stage, incremental learning mechanism can be divided into
three categories: supervised [7], semi-supervised [8] and selfsupervised incremental learning [9].
It is unrealistic to utilize supervised incremental learning
mechanism to update an AR model consistently. Too many
user annotations are required to generate labeled instances
for updating the model, which obviously interrupts user’s
daily living. Gladly, both semi-supervised and self-supervised
incremental learning methods do not need user annotation.
However, for semi-supervised incremental learning mechanism, continuously learning unsupervised knowledge (learned
from unlabeled instances in the second stage) may degrade
its supervised knowledge (learned from labeled instances inthe ﬁrst stage)[9]. Self-supervised incremental learning, on
the other hand, also learns from new experiences (morefeatures within unlabeled instances) without, in a certain level,
degrading its supervised knowledge [9], which makes it morefeasible to personalized AR model adaptation. In this paper,
we concentrate on the challenges of constructing an AR modelin the self-supervised incremental learning mechanism.
In the ﬁrst stage of self-supervised incremental learning,a generic model is learned based on the data of motion
sensors (accelerometer in this paper) collected from several
users’ ADLs. In the second stage, other data sources besides
motion sensors are needed to extract new features for the
unlabeled instances of the target user. Recently, dynamic
contextual information has shown its superiority in recognizinguser’s ADLs effectively and unobtrusively. For example, thenumber and change rate of Bluetooth device scanned in user’s

Abstract—Miscellaneous mini-wearable devices (e.g. wristbands, smartwatches, armbands) have emerged in our life,
capable of recognizing activities of daily living, monitoring
health information and so on. Conventional activity recognition(AR) models deployed inside these devices are generic classiﬁers
learned ofﬂine from abundant data. Transferring generic model
to user-oriented model requires time-consuming human effort forannotations. To solve this problem, we propose SS-ARTMAP-AR,a self-supervised incremental learning AR model updated fromsurrounding information such as Bluetooth, Wi-Fi, GPS, GSM
data without user’s annotation effort. Experimental results showthat SS-ARTMAP-AR can gradually adapt individual users andbecome more incremental intelligence.
Index Terms—activity recognition, activities of daily living,
smartwatch, incremental learning.

I. I NTRODUCTION
Knowledge of activities of daily living (ADLs, e.g. walking,running, going upstairs/downstairs) can be learned by making
use of various wearable devices (e.g. wristband, smartwatch,armband [1][2][3]) worn or even embedded into clothes oraccessories [4], hence wearable activity recognition (AR)techniques promote the development of ubiquitous computing
applications such as context awareness, energy expenditureand personal healthcare. For example, wearable AR techniques
for healthcare improve user’s health conditions by collecting
information related to ADLs, analyzing it and returning the
feedback to the user.
An AR classiﬁcation model is usually learned ofﬂine basedon the data collected from several users. Once done, thismodel is embedded into the wearable devices permanently torecognize ADLs for all future unknown users. Normally, sucha generic static model may not well ﬁt for a speciﬁc user with
distinctive personalities in terms of wearing styles and ADLs.
For example, a model learned based on the data from thedominant wrist (e.g. right wrist [5][6]) may not well work for
the users wearing device on his/her non-dominant wrist (e.g.
left wrist), or having an opposite dominant wrist. Therefore,
it would be better if the model can adjust itself adaptively byconsidering the data of its single user to precisely grab his
personalities.
978-1-5090-2771-2/16 $31.00 © 2016 IEEE
DOI 10.1109/UIC-ATC-ScalCom-CBDCom-IoP-SmartWorld.2016.90

327

surroundings can help to identify whether he is taking the
subway or not [12][13]. As a consequence, features related tocontextual information of the target user are extracted as wellin the second stage of self-supervised incremental learning.
Our contributions are:
• A self-supervised incremental learning framework is proposed to construct the personalized AR model for thetarget user.
• Several AR models are constructed by automaticallymining its target user’s personalities, learning from theknowledge of nearby surroundings, and updating themselves accordingly.
• We propose to take advantage of the contextual information in user’s surroundings to help personalize a genericmodel. According to the consistent discovery of the target
user’s particular patterns in his daily life, the model
gradually suits its target user better.
• We validate our method on real-world AR datasets.
Experiments show that those AR models can recognize
user’s ADLs accurately based on only a few labeled andplenty of unlabeled data of the target users.
The remainder of this paper is structured as follows. Section
II discusses some related work. We present an overview of theself-supervised learning (SSL) and Self-supervised ARTMAP
model in Section III. After that, we give detailed description
about the real-world datasets and preprocess steps in Section
IV. We validate the performance of our approach in Section
V. In the end, we conclude the paper and discuss some future
extensions.

Fig. 1.

Self-supervised ARTMAP

decision tree model which captures knowledge from a previous
task [16].
B. Universality: generic/personalized model
Personalization of activity recognition has become a topic
of interest recently. Increasing works emerge in AR literature
focusing on the personalization problem [7][15][17][18]. To
solve the problem that a model cannot accurately recognize
activities of a speciﬁc user, TransEMDT (Transfer learning
EMbedded Decision Tree) is presented which integrates decision tree and k-means clustering algorithm for personalized
activity-recognition model adaptation [17]. Authors in [18]present “uWave” application for interaction based on personalized gestures and physical manipulations of a consumer electronic or mobile device. Unlike statistical methods, “uWave”
requires a single training sample for each gesture pattern andallows users to employ personalized gestures and physicalmanipulations [18].
Speciﬁcally, Support vector machine (SVM) is a popularsupervised learning model represented as a hyperplane which
separates the data from two classes with a maximal margin[19].Extreme learning machine (ELM) is fast single hidden
layer feedforward neural network for classiﬁcation and regression with the weights between input layer and hidden layer
are randomly assigned [20]. Semi-supervised and incremental
versions [21][22][23][24] of those traditional models havebeen proposed to deal with problems of unlabeled data fortraining or coming one by one or chunk by chunk. Some
representative supervised, semi-supervised and incremental
models above are compared with our model in Section V.

II. RELATED WORKIn this section, we provide a brief introduction of existing
AR works in literature from different perspectives.
A. Model: ofﬂine batch mode learning/online incremental
learning
Generic AR models are conventionally constructed ofﬂine
based on the data collected beforehand. Once completed, themodel remains unchanged regardless of whoever its user is orwhat kind of scenarios it is utilized. For instance, the Kung Fu
model in the wristband [14] is a static hidden Markov modeland could not adapt itself to a new user. The data distribution
is greatly different between varying users.
Recently, more and more researchers realize the phenomenon that the distribution of data is heavily affected byvarying users. The performance will degrade when the modeltrained on some users is used to a new user. To solve thisproblem, a number of online incremental learning models havebeen proposed. [7][15][16]. An Online Sequential Reduced
Kernel Extreme Learning Machine is applied to update theinitial model and adapt it to new users based on the recognition
results of high conﬁdence level [7].A lifelong learning framework from sensor data streams is constructed for predictinguser modeling, which continuously learns from their users andadapts the system through stream-based active learning [15].
New task can be learned semi-incrementally from a partial

III. S ELF - SUPERVISED LEARNING FRAMEWORK
A. Framework
A self-supervised incremental learning framework is proposed which consists of two stages to construct a user-oriented
AR model.See Fig. 1.
• In the ﬁrst stage, called “Model Initialization Stage”, ageneric model is learned based on the labeled motion
data from different activities of multiple people.
• In the second stage, called “Model Update Stage”, thegeneric model is growing to be a personalized model
based on the consistently incremental unlabeled data

328

Fig. 2.

Five steps for data collection and preprocessof a target user. Those data for model update containcharacterizations of both motion and context.
The generic model is in charge of grasping the crucial
locomotion knowledge of activities from the motion data ofpeople. Making the generic model as a starting point, weare trying to construct a user-oriented model by gathering
the individual personalities of our target-user as well as thecontextual information during his ADLs.
B. Activity Recognition Model
Adaptive Resonance Theory (ART) is a cognitive and neuraltheory solving the problem of how a machine can learnknowledge quickly from the new data without forgetting
previously learned knowledge. Early ART neural networks(e.g. Fuzzy ART [25], Gaussian ART [26]) are proposed for
unsupervised incremental learning, they incrementally learn a
set of templates called categories. ARTMAP [27], also known
as Predictive ART, is the neural network architecture forsupervised incremental learning problem. Each input activatesa single category node during both training and testing. When anode is ﬁrst activated during training, it is mapped to the classthe input belongs to. If the input activates a node in testing,the corresponding class mapped to this node is regarded as theprediction.
For those ART and ARTMAP networks, the input has
a ﬁxed number of features throughout training and testing.
Self-supervised ARTMAP [9], learn partial knowledge fromlabeled inputs in the ﬁrst stage, then enrich itself according
to unlabeled inputs containing additional novel features inthe second stage. In this paper, a self-supervised ARTMAP
activity recognition model (SS-ARTMAP-AR) is learned foreach user within the Self-supervised learning framework.

Fig. 3.

Devices and applications for data collection and user annotation

Data” (Fig. 3), which displays the data as well as saves themin the phone’s extra storage card.
Besides “Shimmer Data”, another two applications “Sensor
List” and “Activity Recorder” also run on the phone (Fig.
3). “Sensor List” is in charge of collecting the context datain surroundings, including the scanned Bluetooth devices,
nearby Wi-Fi Access Points (APs), GPS readings if any,
Global System for Mobile communication (GSM). Thoroughdescriptions of the attributes can be found in [29][30][31][32].
During the data collection process, users are required
to annotate the ground truth (activity) with the application “Activity Recorder” (Fig. 3). We get the ground truthactivityT ype, beginT ime, endT ime for each activity collected.
The sampling rate of the accelerometer is 102.4Hz andthe value ranges from −6g to +6g. The scanning processof Bluetooth devices involves an inquiry scan of about 12
seconds, followed by a page scan. Each scanned Bluetooth
device is returned in the form of a Broadcast service[29]. Dueto the lengthy time cost in starting and scanning proceduresof Bluetooth and Wi-Fi, it collects Bluetooth and Wi-Fi data
every 30 seconds. One GPS sample is collected per minuteowing to the long signal propagation distance between thesatellite and the user.Besides, we collect GSM data every 5
seconds.
12 distinct activities are collected by user “A” and “B”, and
9 activities by user “C” Table I. Basically, data of each activity
are collected for more than 1 hour. In total, we get 238 hours
of collected data.

IV. DATA COLLECTION AND PREPROCESSING
In order to validate above framework, data are collected andpreprocessed through 3 steps (Fig. 2). In the following paper,
we use the word “sample” to represent the raw sensor readingand “instance” to describe the data generated for training aclassiﬁer.
A. Data Collection
We collect data from three users, 2 postgraduate students(User “A” and User “B”) and 1 ofﬁce worker (User “C”) during
their daily life for more than 5 months in total. Each user isprovided with a Shimmer 2r device [28] containing a tri-axial
accelerometer and a Bluetooth module to communicate withan android smartphone paired with the Shimmer(Fig. 3). The
Shimmer is worn in user’s non-dominant (left) wrist, where theacceleration samples are collected and transmitted wirelessly
to the phone according to an Android application “Shimmer

B. Feature Extraction
1) Accelerometer data
For a window of acceleration samples (data of 5 seconds)
in each axis (X/Y /Z-axis), we extract fourteen features fromtime and frequency domains in a window of samples bysliding window mechanism (window size=500, step size=100).

329

TABLE I
T ESTING ACCURACY OF SS-ARTMAP-AR

Together we get forty-two features from three axes to generate
an acceleration instance.
2) Bluetooth data
For each scanned Bluetooth sample, the documentation forAndroid SDK has deﬁned its major class from 12 “Major
Device Classes”, as well as its minor class from several “Minor
Device Classes” in the context of the major class assignment[29]. Besides, we can also check the on-going bonding state(None, Bonding, Bonded) of the scanned Bluetooth sample[29]. In all, we extract 98 features with Boolean value (0/1)
consisting of 3 bonding states, 12 “Major Device Classes” and83 “Minor Device Classes” for each sample. For each of the
98 features above, we count the percent of samples in the scan
whose value is 1 as the feature value. Therefore, a Bluetoothinstance contains 100 features.
3) Wi-Fi data
For the Wi-Fi samples in one scan, we count the numberof samples with distinct AP address as the ﬁrst feature (“AP#”), and calculate the average signal strength as the second
feature (“Mean level”). The percent of samples with new
AP address (“ChangeRate APAddress”), capability (“ChangeRate capability”), and frequency (“ChangeRate frequency”)
are also collected as the features. In all, a Wi-Fi instance owns
5 features.
4) GPS and GSM data
Each GPS sample generates a GPS instance with longitude
and latitude as its features. Each GSM sample generates a
GSM instance with 2 features. One feature is the signal
strength “bsss”, the other feature “ChangeRate GSM” is a
Boolean value representing whether “cid” or “lac” is changed
or not compared with the last GSM sample.

ID

Activity

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

Walking
Bicycling
Climbing stairs
Taking elevator
Taking the subway
Having a meal
Taking a nap
Meeting
Shopping
Watching movie
Taking the bus
Riding on the back of a bicycle
Watching table tennis game
Taking a taxi
Fitness training
Having fun in KTV

A
0.73
0.97
0.83
0.72
0.51
0.81
0.87
0.37
0.78
0.90
/
0.99
0.29
/
/
/

Accuracy
BC
0.57 0.78
0.94 0.99
0.83 0.76
0.65 0.53
0.45 0.62
0.29 0.63
0.85
/
0.25
/
/
0.70
/
0.67
0.39 0.56
/
/
/
/
0.55
/
0.62
/
0.43
/

model has robust recognition ability on the ADLs. Besides,
SS-ARTMAP-AR model has an accuracy of more than 50% on
3 high-level semantic activities (“Taking a nap”, “Shopping”,“Watching a movie”) of 2 users, which is interesting sincethose activities are quite similar in nature but different in environmental circumstances. For instance, an activity of “Shopping” is mixed with some “Walking” and special gestures ofpicking up/put down some products. An accuracy of 99% isachieved on “Riding on the back of a bicycle” class of user“A”, which is even better than the Bicycling class. However,
SS-ARTMAP-AR also shows poor predictive ability on 3classes (“Meeting”, “Watching table tennis game”, “Having
fun in KTV”) owning to their ambiguous characteristics on
existing dataset.

C. Concatenation and Feature Selection
For all the instances of accelerometer/ Bluetooth/ Wi-Fi/
GPS/ GSM sensors, we concatenate their features according
to the timestamps of instances to create new instances.
In order to remove redundant features from the feature set,
we utilize the supervised attribute ﬁlter “Normalize Filter” inWeka [33] to select the optimal feature subset for each user.

B. Comparison with other approaches
In this section, we evaluate the performance of SSARTMAP-AR model and compare it with several representative supervised and semi-supervised methods (Fig. 4). Comparison methods are divided into two groups:“SVM group”
and “ELM group”. Four methods are contained in “SVM
group” and ﬁve in “ELM group”. Among them, “SVM *”[19] and “OSELM *” [23] are supervised incremental learning
methods; “S3VM *” [21] and “SSELM *” [22] are semisupervised incremental learning methods. “ Acc” and “ All”
represents instances for training and testing the model contains
only acceleration features and all kinds of features (featuresfrom acceleration, Bluetooth, WiFi, GPS and GSM) respectively.
For all the supervised incremental methods introduced
above,the feature sets of initial model generation and model update are the same(“ Acc” or “ All”). Differently,“FAOSELM” [24] is a specialized supervised incremental
method in which an initial model is trained based on acceleration features, and updated based on data with all features(acceleration, Bluetooth, Wi-Fi, GPS and GSM). “FAOSELM”
is very similar to SS-ARTMAP-AR model except that data formodel update are with labels for the former, and no labels is

V. R ESULTS
A. Accuracy
The performance of SS-ARTMAP-AR is validated on eachuser’s dataset. For simplicity, we randomly select 3 labeledinstances with acceleration features, 500 unlabeled instanceswith acceleration, Bluetooth, Wi-Fi, GPS, GSM features selected, from each class of the dataset. The labeled/unlabeledinstances are the training data in Model Initialization Stage/ Model Update Stage of SS-ARTMAP-AR. Another 300
instances from each class constitute the testing set. The
prediction accuracies of 3 generated SS-ARTMAP-AR models
on each corresponding testing set are listed in Table I.
Generally speaking, the accuracy varies on different users
and classes. For 4 typical daily activities (“Walking”, “Bicycling”, “Climbing stairs”, “Taking elevator”), accuracy exceeds
50% for all the 3 users, which means the SS-ARTMAP-AR

330

0.8
0.7

Accuracy

0.6
SVM_Acc
SVM_All
S3VM_Acc
S3VM_AllOSELM_Acc
OSELM_All
SSELM_Acc
SSELM_All
FAOSELM
SS-ARTMAP-AR

0.5
0.4
0.3
0.2
0.1

1

2

3

4

5

6

7

8

9 10 20 30 40 50 60 70 80 90 100200300400500600700800900

Instance # in Training Set

Fig. 5. Performance comparison of self-supervised ARTMAP-AR and semisupervised ARTMAP-AR(a) User A
0.8

C. Contribution of environmental features

0.7

Accuracy

0.6

0.4
0.3
0.2
0.1

In this section, we want to evaluate the contribution ofenvironmental features (Bluetooth, Wi-Fi, GPS, GSM features)to the overall accuracy of SS-ARTMAP-AR model. For thesake of comparison, we also train an SS-ARTMAP-AR modelwith labeled instances in Model Initialization Stage, andunlabeled instances in Model Update Stage, both of whichcontain acceleration features only. We call it “Semi-supervised
ARTMAP”, since it is generated in a Semi-supervised learning
framework. Given the same set of labeled instances in Model
Initialization Stage, their predictive accuracy on the same
testing set are compared (Fig. 5).
The accuracy increases for both. Obviously, the accuracyof Self-supervised ARTMAP goes faster and more stable thanSemi-supervised ARTMAP model owing to the environmental
Bluetooth, Wi-Fi, GPS, GSM features. Therefore, we canconclude that: environmental features, which contains a great
deal of personalized characteristics to the data of on-going
activities, can increase classiﬁcation performance of an activity
recognition model.

SVM_Acc
SVM_All
S3VM_Acc
S3VM_AllOSELM_Acc
OSELM_All
SSELM_Acc
SSELM_All
FAOSELM
SS-ARTMAP-AR

0.5

1

2

3

4

5

6

7

8

9 10 20 30 40 50 60 70 80 90 100200300400500600700800900

Instance # in Training Set(b) User B
0.9
0.8

Accuracy

0.7
SVM_Acc
SVM_All
S3VM_Acc
S3VM_AllOSELM_Acc
OSELM_All
SSELM_Acc
SSELM_All
FAOSELM
SS-ARTMAP-AR

0.6
0.5
0.4
0.3
0.2
0.1

1

2

3

4

5

6

7

8

9 10 20 30 40 50 60 70 80 90 100200300400500600700800900

Instance # in Training Set(c) User C
Fig. 4. Performance comparison of SS-ARTMAP-AR and supervised, semisupervised methods in “SVM group” and “ELM group”

D. Effect of number of labeled instance
It is necessary to analyze the effect of the number oflabeled instances on the recognition ability of SS-ARTMAPAR, which is really important in practical application since
the label of instances comes from costly and intrusive user
annotation. For the whole dataset of User “A”, we randomly
select 100 instances from each class to construct the testing
set. The number of labeled instances for training from eachclass varies from 1 to 20. For each labeled instance number,
we record the least number of unlabeled instances (see thehistogram in Fig. 6) it needs in Model Update Stage to achieve
an testing accuracy of more than 50% on the testing set. We
look for the “least number” from 10, 20, 30,... until the proper
one is found. The actual accuracy of SS-ARTMAP-AR model
learned on the basis of different size of labeled instances areshown in the upper part of Fig. 6. From the results we canconclude: if there are only 2 labeled instances (data of morethan 15 seconds) from each class are in Model Initialization
Stage, it needs at least 10 unlabeled instances in Model Update
Stage to achieve an accuracy of large than 50%. It shows
the fact that SS-ARTMAP-AR does not rely on massive user
annotations. Performance of SS-ARTMAP-AR does beneﬁt
from more labeled data, but only a few labeled data, e.g. withneeded for the latter. To compare the results of four methods(Fig. 4) in “SVM group”,“ELM group” on 3 users’ data we canconclude that: First, the performance of supervised methods
are better than semi-supervised methods, which means user
annotations are beneﬁcial in constructing efﬁcient AR models.
Second, “ All” methods attain higher accuracy than “ Acc”
methods, which illustrates the signiﬁcance of context featuresin classifying ADLs. Third, the performance of methods in“SVM group” is much clearer and more distinguishable than
“ELM group”.
Among all the methods in comparison, SS-ARTMAP-AR,“FAOSELM” and “SVM All” gain highest predictive abilityon the dataset of User “B” (Fig. 4(b)), which is signiﬁcantachievement for SS-ARTMAP-AR since large amounts ofannotations are avoided. For datasets of User “A” and “C”,
“FAOSELM” and “SVM All” are a little bit superior to SSARTMAP-AR (Fig. 4(a),(c)). Still, these three methods aremuch better than the rest.

331

[9] G. P. Amis and G. A. Carpenter, “Self-supervised artmap,” Neural
Networks, vol. 23, no. 2, pp. 265–282, 2010.
[10] S. J. Pan and Q. Yang, “A survey on transfer learning,” Knowledge andData Engineering, IEEE Transactions on, vol. 22, no. 10, pp. 1345–
1359, 2010.
[11] D. H. Hu and Q. Yang, “Transfer learning for activity recognition via
sensor mapping,” in IJCAI Proceedings-International Joint Conferenceon Artiﬁcial Intelligence, vol. 22, no. 3, 2011, p. 1962.
[12] Y. Chen, Z. Chen, J. Liu, D. H. Hu, and Q. Yang, “Surrounding context
and episode awareness using dynamic bluetooth data,” in Proceedingsof the 2012 ACM Conference on Ubiquitous Computing. ACM, 2012,
pp. 629–630.
[13] Z. Chen, Y. Chen, S. Wang, J. Liu, X. Gao, and A. T. Campbell, “Inferring social contextual behavior from bluetooth traces,” in Proceedingsof the 2013 ACM conference on Pervasive and ubiquitous computing
adjunct publication. ACM, 2013, pp. 267–270.
[14] G. S. Chambers, S. Venkatesh, G. A. West, and H. H. Bui, “Hierarchicalrecognition of intentional human gestures for sports video annotation,” inPattern Recognition, 2002. Proceedings. 16th International Conferenceon, vol. 2. IEEE, 2002, pp. 1082–1085.
[15] M. Fetter and T. Gross, “Lilolea framework for lifelong learning fromsensor data streams for predictive user modelling,” in Human-Centered
Software Engineering. Springer, 2014, pp. 126–143.
[16] J. W. Lee and C. Giraud-Carrier, “Transfer learning in decision trees,”in Neural Networks, 2007. IJCNN 2007. International Joint Conference
on. IEEE, 2007, pp. 726–731.
[17] Z. Zhao, Y. Chen, J. Liu, Z. Shen, and M. Liu, “Cross-people mobilephone based activity recognition,” in IJCAI, vol. 11. Citeseer, 2011,
pp. 2545–250.
[18] J. Liu, L. Zhong, J. Wickramasuriya, and V. Vasudevan, “uwave:
Accelerometer-based personalized gesture recognition and its applications,” Pervasive and Mobile Computing, vol. 5, no. 6, pp. 657–675,
2009.
[19] C. Cortes and V. Vapnik, “Support-vector networks,” Machine learning,
vol. 20, no. 3, pp. 273–297, 1995.
[20] G.-B. Huang, L. Chen, and C.-K. Siew, “Universal approximation using
incremental constructive feedforward networks with random hidden
nodes,” Neural Networks, IEEE Transactions on, vol. 17, no. 4, pp.
879–892, 2006.
[21] T. Joachims, “Transductive inference for text classiﬁcation using support
vector machines,” in ICML, vol. 99, 1999, pp. 200–209.
[22] G. Huang, S. Song, J. N. Gupta, and C. Wu, “Semi-supervised and unsupervised extreme learning machines,” Cybernetics, IEEE Transactions
on, vol. 44, no. 12, pp. 2405–2417, 2014.
[23] N.-Y. Liang, G.-B. Huang, P. Saratchandran, and N. Sundararajan, “Afast and accurate online sequential learning algorithm for feedforward
networks,” Neural Networks, IEEE Transactions on, vol. 17, no. 6, pp.
1411–1423, 2006.
[24] X. Jiang, J. Liu, Y. Chen, D. Liu, Y. Gu, and Z. Chen, “Feature
adaptive online sequential extreme learning machine for lifelong indoor
localization,” Neural Computing and Applications, vol. 27, no. 1, pp.
215–225, 2016.
[25] G. A. Carpenter, S. Grossberg, and D. B. Rosen, “Fuzzy art: Fast stablelearning and categorization of analog patterns by an adaptive resonancesystem,” Neural networks, vol. 4, no. 6, pp. 759–771, 1991.
[26] J. R. Williamson, “Gaussian artmap: A neural network for fast incremental learning of noisy multidimensional maps,” Neural networks, vol. 9,
no. 5, pp. 881–897, 1996.
[27] G. A. Carpenter, S. Grossberg, and J. H. Reynolds, “Artmap: Supervised
real-time learning and classiﬁcation of nonstationary data by a selforganizing neural network,” Neural networks, vol. 4, no. 5, pp. 565–588,
1991.
[28] http://www.shimmer-research.com/.
[29] http://developer.android.com/guide/topics/connectivity/bluetooth.html.
[30] http://developer.android.com/reference/android/net/wiﬁ/
ScanResult.html.
[31] http://developer.android.com/guide/topics/location/strategies.html.
[32] http://developer.android.com/reference/android/telephony/gsm/
GsmCellLocation.html.
[33] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H.
Witten, “The weka data mining software: an update,” ACM SIGKDD
explorations newsletter, vol. 11, no. 1, pp. 10–18, 2009.

Fig. 6. Performance comparison of SS-ARTMAP-AR with different numberof labeled instances in Model Initialization Stage

15 seconds user annotations for each activity (“activity-levelone-off annotations”), are still enough for high recognition
performance.
VI. C ONCLUSION
In this paper, we construct SS-ARTMAP-AR, a selfsupervised incremental learning model to recognize ADLs ofusers with mini-wearable devices. SS-ARTMAP-AR is ﬁrstly
initialized using a few motion data with relevant annotations,then updated incrementally by the motion and context data
without any user annotation effort. According to the consistent
discovery of regular patterns in daily life, SS-ARTMAP-AR
gradually adapts the user and make the device more intelligent.
In the future, we plan to actively select informative instancesin personalizing the generic activity recognition models formini-wearable device users.
ACKNOWLEDGMENT
This work is supported in part by Natural Science
Foundation of China under Grant No.61572471, 61572466,
61572004, 61472399, 61502456, Chinese Academy of Sciences Research Equipment Development Project under Grant
No.YZ201527, Science and Technology Planning Projectof Guangdong Province under Grant No.2015B010105001,
Innovation Project of Institute of Computing Technology
under Grant No.20156010, Beijing Natural Science Foundation under Grant No.4162059 and International Science& Technology Cooperation Program of China under Grant
No.2014DFG12750.
R EFERENCES[1] https://jawbone.com/up.
[2] https://getpebble.com/steel.
[3] http://www.bodymedia.com/Support-Help/BodyMedia-FIT-BW.
[4] L. Bao and S. S. Intille, “Activity recognition from user-annotatedacceleration data,” in Pervasive computing. Springer, 2004, pp. 1–17.
[5] F. Moiz, P. Natoo, R. Derakhshani, and W. D. Leon-Salas, “A comparative study of classiﬁcation methods for gesture recognition using a 3-axis
accelerometer,” in Neural Networks (IJCNN), The 2011 International
Joint Conference on. IEEE, 2011, pp. 2479–2486.
[6] H. Junker, O. Amft, P. Lukowicz, and G. Tröster, “Gesture spotting withbody-worn inertial sensors to detect user activities,” Pattern Recognition,
vol. 41, no. 6, pp. 2010–2024, 2008.
[7] W.-Y. Deng, Q.-H. Zheng, and Z.-M. Wang, “Cross-person activity
recognition using reduced kernel extreme learning machine,” Neural
Networks, vol. 53, pp. 1–7, 2014.
[8] J. Erman, A. Mahanti, M. Arlitt, I. Cohen, and C. Williamson, “Semisupervised network trafﬁc classiﬁcation,” in ACM SIGMETRICS Performance Evaluation Review, vol. 35, no. 1. ACM, 2007, pp. 369–370.

332