Journal of Robotics, Networking and Artificial Life, Vol. 3, No. 1 (June 2016), 24-27

Estimating Age on Twitter Using Self-Training Semi-Supervised SVM
Tatsuyuki Iju

Graduate School of Information Engineering, University of the Ryukyus
1 Senbaru, Nishihara-cho, Nakagami-gun, Okinawa, Japan
Satoshi Endo, Koji Yamada Naruaki Toma, Yuhei Akamine

School of Information Engineering, University of the Ryukyus
1 Senbaru, Nishihara-cho, Nakagami-gun, Okinawa, Jaapn
E-mail: k148582@ie.u-ryukyu.ac.jp Endo@ie.u-ryukyu.ac.jp
http://www.u-ryukyu.ac.jp/en/Abstract
The estimation methods for Twitter user’s attributes typically require a vast amount of labeled data. Therefore, anefficient way is to tag the unlabeled data and add it to the set. We applied the self-training SVM as a semisupervised method for age estimation and introduced Plat scaling as the unlabeled data selection criterion in theself-training process. We show how the performance of the self-training SVM varies when the amount of training
data and the selection criterion values are changed.
Keywords: Twitter, Age, Semi-supervised learning, Self-training, SVM, Plat scaling

1. Introduction
Nowadays, the use of Twitter as a social activity sensor
has become popular trendy. Although it is more
efficient to consider attribute differences such as user
age and gender for analysis, users rarely share theirpersonal information to the public. Therefore, a variety
of methods for estimating Twitter user’s attributes has
been studied[1][2][3]. However, these methods require avast amount of labeled data. Since collecting labeleddata is typically a high cost work, the estimation method
is efficient when unlabeled data is labeled and used asadditional data. We investigate a method for buildingclassifier by using self-training SVM, which is acombination of semi-supervised method, self-training
and SVM. In this study, we formulate age estimation asa binary classification problem where user is labeled asunder 30 or over 30. The method for user vectorization
is simple bag-of-words model and all the tweets treatedas data were Japanese tweets. In section 2, we describe

self-training SVMs. In section3, we describe theexperiments and present and analyze the results. Finally,
we describe our findings and the extension of this work.
2. Self-training SVM algorithm
We describe about the method of building our classifierby using a self-training SVM. Self-training is a simple
semi-supervised learning algorithm, with examples ofapplications started by Scudder[4]. The standard
approach for self-training is as follows.
i. By Using underlying learning algorithm, train aclassifier from the labeled data set.
ii. Label a part of the unlabeled data set according to
the classifier, and retrain it, with the newly labeleddata as an additional training set.
We construct our classifier for Twitter user’s age
estimation by using a self-training SVM, in which theunderlying learning algorithm is SVM. Furthermore, we
introduce Plat scaling as a criterion for selecting users

Published by Atlantis Press
Copyright: the authors
24

Tatsuyuki Iju,Satoshi Endo, Koji Yamada Naruaki Toma, Yuhei Akamine

appropriately from the unlabeled set in order to belabeled. It is expected that poor quality data is filteredout by that criterion. Plat scaling[5] is a method used formodeling a function that returns posterior probability𝑃(𝐴𝐴𝐴|𝑋) in which user 𝑋 is in the 𝐴𝐴𝐴 class,according to classifier’s the decision function.
The steps of the self-training SVM algorithm wheresample set {𝑋𝑖 , 𝑖 = 1, … , 𝑁𝐼 } and sample set {𝑋𝑢 , 𝑢 =1, … , 𝑁𝑈 } belongs to training set 𝐹𝐼 and unlabeled set
𝐹𝑈 , respectively.
i. Using 𝐹𝐼 , we train a SVM and obtain probability
𝑃(𝐴𝐴𝐴𝑙 |𝑋𝑢 ) ∈ {𝑃(𝐴𝐴𝐴𝑙 |𝑋𝑢 ), 𝑙 = 1, … , 𝑁𝐴 } thatsample 𝑋𝑢 belongs to 𝐴𝐴𝐴𝑙 by Plat scaling.
ii. Define 𝐹𝑢∗ containing all 𝑋𝑢 with at least one ofthe 𝑃(𝐴𝐴𝐴𝑙 |𝑋𝑢 ) ≥ 𝑡ℎ𝑟𝑟𝑟ℎ𝑜𝑜𝑜 . Furthermore,
Define sample in 𝐹𝑈 ∗ as 𝑋𝑢∗ .
iii. Define new𝐹𝐼 as 𝐹𝐼 = 𝐹𝐼 + 𝐹𝑈 ∗ . The label of 𝑋𝑢∗
is predicted as the 𝐴𝐴𝐴𝑙 for which 𝑃(𝐴𝐴𝐴𝑙 |𝑋𝑢∗ ) ishighest. Furthermore, Define new 𝐹𝑈 as 𝐹𝑈 =𝐹𝑈 − 𝐹𝑈 ∗ .
iv. Repeat i. ~ iii. until 𝐹𝑈∗ cannot be defined.
3.

3.1.

Experiments

Experimental contents and settings

We defined age as two classes, under 30 and over 30.
This definition is the same as Rao’s research[1]. wecarried out some experiments in order to evaluate theperformance of the Self-training SVM for the Twitteruser’s age estimation. By using self-training SVM, webuilt a classifier for each of the nine training set arrays(three sets of different number of users containing 76,
256 and 376 with three possible selection criterions of0.5, 0.7 and 0.9 respectively). Then, we measure theclassifier’s performance with the test set. Additionally,in order to provide the baselines for these classifiers, webuilt other classifiers by using normal SVM for each offour training set arrays and measured their performancethe same way, we did for the self-training SVM ones.
The performance measurement was done by the fivefold cross validation. The number of users in the test set
was 480 and the number of users in the unlabeled set
was 1200, In addition, the number of users from under
30 and over 30 classes were balanced in both thetraining and test sets. The way for user vectorization
was simple bag-of-words model, at the time of classifier
training and user’s age prediction. Prior to proceeding
the experiments, it was necessary to set hyper

parameters of SVM and features in a bag-of-words
representation. Therefore, we performed a grid-search,
set the kernel and cost parameters of SVM as linear and1000 respectively. As for the features, we set the 158
top-ranked words by 𝜒 2 score which appeared in user
tweets from the training set.
3.2.

Results

First, we describe about the results for the normal
SVMs. Table 1 shows the performance of the normal
SVM classifiers. From Table 1, we can assure that theclassifiers can improve their performance as the size ofthe set is increased, at least when the size of the given
training set is in the 76 to 376 range. According to thisresult, as well as for self-training SVM, it is expectedthat performance can be improved by utilizing
unlabeled set with the training set size within the samerange size. Additionally, regardless of the training set
size, it was observed that precision is high and recall is
low for the under 30 class and, oppositely the precision
is low and recall is high for the over 30 class. F measure
was better for the under 30 class than for over 30 one. Itis, basically, that the under 30 class user was easier topredict than the over 30 class one. As for the results
about self-training SVM, Table 2, 3, 4, and correspondto the prediction results from classifiers built from 76,
256 and 376 users training sets, respectively. The
classifier with the highest improvement from baseline
was the one with 76 users training set with a selection
criterion of 0.9 (improvement was 0.032 point for theover30 class recall). Classifier from the 376 users with aselection criterion of 0.9 was the second highest(improvement was 0.025 point for the over 30 classrecall). Although the recall improvement for over 30
class is remarkable, precision for the over 30 class andboth precision and recall for the under 30 class subtly
improve or diminish as the mean F measure merelyimproves. Table 5 shows the labeling accuracy for the76 users set self-training and ratio of the labeled set for
all the unlabeled set. Precision, F and mean F measures
for the over 30 class were not defined, because theentire unlabeled set is filtered and eventually labeled as
the under 30 class. We could observe better labelingaccuracy for higher selection criterions and, in contrast,
for bigger selection criterions the amount of labeled datafrom the unlabeled set is reduced. However, as indicatedby Table 2 through Table 4, performance was better fora selection criterion of 0.5 than 0.7 for all of classifiers

Published by Atlantis Press
Copyright: the authors
25

Estimating Age on Twitter

Table 1: Results for the normal SVM
Number of the users in training set

Age classPrecision
RecallF measure
MeanF measure76256376

Under 30
Over 30
0.613
0.792
0.882
0.441
0.723
0.563
0.643

Under 30
Over 30
0.638
0.825
0.896
0.492
0.745
0.616
0.680

Under 30
Over 30
0.647
0.834
0.898
0.509
0.752
0.632
0.692

Table 2: Results for the training set with 76 number of users
Unlabeled data selection criterion

Age classPrecision
RecallF measureMean
F measure

0.50.70.9

Under 30
Over 30
0.614
0.792
0.882
0.442
0.723
0.563
0.643

Under 30
Over 30
0.610
0.793
0.888
0.430
0.722
0.554
0.639

Under 30
Over 30
0.625
0.791
0.901
0.473
0.728
0.590
0.659

Table 3: Results for the training set with 256 number of users
Unlabeled data selection criterion

Age classPrecision
RecallF measureMean
F measure

0.50.70.9

Under 30
Over 30
0.638
0.825
0.896
0.492
0.745
0.616
0.680

Under 30
Over 30
0.630
0.818
0.895
0.473
0.739
0.599
0.669

Under 30
Over 30
0.645
0.808
0.876
0.512
0.742
0.629
0.686

Table 4: Results for the training set with 376 number of users
Unlabeled data selection criterion

Age classPrecision
RecallF measureMean
F measure

0.5

0.7

0.9

Under 30
Over 30
0.647
0.834
0.898
0.510
0.752
0.632
0.692

Under 30
Over 30
0.649
0.825
0.891
0.516
0.750
0.634
0.692

Under 30
Over 30
0.657
0.829
0.890
0.5340.756
0.650
0.703

excluding the 376 users training set, although
performance for a selection criterion of 0.9 was best for
all classifiers. This result is inconsistent with the factthat the labeling error is more frequent for selection
criterion of 0.5 than 0.7 as indicated in Table 5. For thatreason, it is implicated that users are easier to predict if
selected from a self-training process with a selection

criterion of 0.7 than a 0.5 one, since the classifier with aselection criterion of 0.7 is more strongly affected bylabeling error than a 0.5 one. In addition, it is inferable
that, for a selection criterion of 0.9, the labeling error
rate is so small that the classifier achieves in improving
its performance.

Published by Atlantis Press
Copyright: the authors
26

Tatsuyuki Iju,Satoshi Endo, Koji Yamada Naruaki Toma, Yuhei Akamine

Table 5: Results for the training set 76 number of users (labeling)
Unlabeled data selection criterion
0.5
Age classPrecision
RecallF measure
MeanF measure
Labeling rate

4.

Under 30
0.621
1.00.766

Over 3001.00.7

0.9

Under 30
Over 30
0.722
0.699
0.890
0.4250.796
0.524
0.660

Under 30Over 30
0.752
0.705
0.90
0.440
0.820
0.537
0.678

0.97

0.83

References

Conclusions

In order to evaluate self-training SVM for Twitter user’s
age estimation, we construct a classifier by for each of
the twelve training set arrays of (three sets of different
users containing 76, 256 and 376 with three possible
selection criterion of 0.5, 0.7 and 0.9 respectively). Then
we evaluate the performance of the classifiers with test
set. As a result, in the recall for the over 30 class, it wasobserved a 0.032 and 0.025 point of improvement frombaseline for training set with a size of 76 and 376
respectively with a selection criterion of 0.9. For future
works, we will investigate the relation between theselection criterion and performance of self-training
SVM as well as explore the way to improve them.

1. Rao, Delip, et al. "Classifying latent user attributes intwitter." Proceedings of the 2nd international workshop
on Search and mining user-generated contents. ACM,2010.
2. Burger, John D., et al. "Discriminating gender on
Twitter." Proceedings of the Conference on Empirical
Methods in Natural Language Processing. Associationfor Computational Linguistics, 2011.
3. Pennacchiotti, Marco, and Ana-Maria Popescu.
"Democrats, republicans and starbucks afficionados: userclassification in twitter." Proceedings of the 17th ACM
SIGKDD international conference on Knowledge
discovery and data mining. ACM, 2011.
4. Scudder, Henry J. "Probability of error of some adaptive
pattern-recognition machines." Information Theory, IEEE
Transactions on 11.3 (1965): 363-371.
5. Platt, John. "Probabilistic outputs for support vector
machines and comparisons to regularized likelihood
methods." Advances in large margin classifiers 10.3
(1999): 61-74.

Published by Atlantis Press
Copyright: the authors
27