DEIM Forum 2016 D1-1

Hashtag Sense Disambiguation Based on Content andTemporal Proximities
Minxin Zou†

Mizuho Iwaihara‡†早稲田大学大学院情報生産システム研究科〒808-0135 福岡県北九州市若松区ひびきの 1-15-2206
E-mail: †zoumx@toki.waseda.jp,‡iwaihara@waseda.jp

Abstract A twitter hashtag is a label or tag which makes it easier for users to find a message on a specifictopic. But in fact, free creation of hashtags leads to the situation such that a hashtag may have multiple
senses. In this paper, we propose a method to disambiguate hashtag senses according to release time andtweet contents. Our assumption is that in a specific time period, a sense of a hashtag and context words
around the sense becomes dominant. In addition, for one user, he/she may also use the same sense whencreating a hashtag. Then we construct a co-occurrence graph and perform community detection over graphsof different time periods. In the co-occurrence graph, nodes are hashtags and words in tweets, and edges
represent the relationship between two nodes. Edge weights are based on four types of relationships: twonodes co-occur in one tweet, two nodes released by the same user, two nodes which are retweeted, and two
nodes co-occur in an external document. A set of nodes which can represent a sense is extracted as acommunity. We use Wikipedia disambiguation lists to filter out and merge minor senses. When doing final
matching, we associate a hashtag to a specific community according to the release time of the tweet and its
contents.
Keyword Twitter, Hashtag, Sense Disambiguation, Proximity, co-occurrence graph

1. Introduction

Knowledge based approaches

Twitter is a popular online social networking service

This type of approach is based on informationthat enables users to send and read short 140-character

contained in structured knowledge bases. WordNet [2] ismessages called "tweets". Quite a large number of

one of the most commonly used knowledge bases in thispeople write tweets every day. From the official data,

type of approach.

there are 320 million monthly active users and 1 billion

Machine-learning based approaches

unique visits monthly to sites with embedded tweets.

This approach can be further classified into three

Special words in tweets starting with “#” are called

categories:

supervised,semi-supervised

hashtags. People use the hashtag symbol # at their will

unsupervised

approaches.

Supervised

before a relevant keyword or phrase in their tweets tousually require large amounts of annotated datasets,

categorize tweets and help them easily retrieved incalled training data. Then machine-learning techniques

Twitter Search. There is no restriction on how to create

are applied on training data to identify the sense of ahashtags. One of the results is that one hashtag may haveword. Unsupervised approaches do not require any

multiple senses. For example, “#Apple” may refer to thetraining data; they rely on unannotated corpus instead.

fruit, while it also may refer to the Apple Company.

Semi-supervised approaches take use of bootstrapping

andapproaches

The process of identifying the sense of a polysemic

techniques to enlarge the training data. They use a small

word is called Word Sense Disambiguation (WSD).

amount of annotated data first, called seed data, then a

There exist many different approaches to WSD [1],classifier trained on this small annotated data annotates

which can be classified into the following three broad

raw corpus and the new annotated data serves as new

categories:

seed data on which the classifier is retrained. All of theseDEIM Forum 2016 D1-1

three categories have their own merits and demerits.

judge examines the mapped results and determines

Hybrid approaches

whether matches are correct or wrong. Finally, we use

This type of approach combines the previous method

error rate to show the rate of the wrong matches. In ourto find the most appropriate sense of a given word.

result, average error rate changes from 0.05 to 0.3 overIn this paper, we propose a method based on content

time. In July, the error rate reaches the lowest value,

and temporal proximities to disambiguate the senses forwhile in January, the error rate gets the highest value. It

hashtags. In our previous work [5], we proposed co-could be explained as people use special words inoccurrence graphs to disambiguate senses. In a co-

January.

occurrence graph, nodes are hashtags and words intweets,

and

edges

representthe

The rest of the paper is organized as follows: In

co-occurrence

Section 2, we review related work focus on word sense

relationship between two nodes. Edge weights of thedisambiguation briefly. In Sections 3 and 4, thegraph are frequencies of co-occurrence in tweets in ourconstruction of our method is described. In Section 5, we

previous work. In this paper, we discuss temporal

perform experiments and show results. In Section 6, wechanges of senses, and improve edge weighting based on

evaluate the results. Finally, a conclusion and future

four types of relationships: 1) two nodes co-occur in one

work are presented in Section 7.

tweet, 2) two nodes released by the same user, 3) two
nodes which are retweeted, and 4) two nodes co-occur in

2. Related work

an external document. By considering these four factors,

Vincent D. Blondel [3] has proposed an algorithm to

we think we can better show the relationship between adetect communities. This algorithm first assigns ahashtag and the words related to it in comparison withdifferent community to each node of the graph. Thenour previous version. The graph is partitioned according

merge each node with its neighbors which can maximize

to a community detection algorithm based on modularity

a gain. Second, the communities found during the first

optimization [3]. For temporal proximity, we divide thephase are nodes in the new graph. Repeat the two phases

time axis into certain periods, and then apply content

iteratively and find the communities finally.

proximity on each period, to detect temporal changes ofhashtag senses. We are facing two challenges:

Another method uses community detection on cooccurrence graphs to do word sense induction. It uses a

Due to no restriction on usage of hashtags and words,

community detection algorithm called Link Clustering,people often use irregular words in tweets. These words

clustering edges, which is equivalent to grouping theare often influenced by popular events. One of ourword collocations to identify sense-specific contexts [4].

observations is that words in some communities can not

On the other hand, our approach reflects social-media

represent a sense.

aspects within tweets into co-occurrence graphs.

Because of the fast development of word senses, there
is no golden standard to create perfect time periods. Eachhashtag in different time periods often has different
popular senses.

3. Hashtag sense disambiguation
3.1 The outline of our system
Our system uses tweets at different time periods as our

In order to resolve these issues, first we consider

input, and executes preprocessing to collect the releasetransforming informal words to formal words, then wetime and contents of these tweets. Then we divide thesedetect communities and map to a sense inventory. In thistweets into several time intervals according to theirpaper, we choose Wikipedia Disambiguation Lists as ourrelease time. In each time interval, we generate a word

sense inventory.

list for each tweet. Each word and hashtag in the tweets

We choose a sense based on ranking of the weights ofwords occurring in the tweets. For evaluation, a humanare assigned as labels to nodes of the co-occurrenceDEIM Forum 2016 D1-1

TweetsTime split

Tweets at different time intervals

Preprocessing

Words, hashtags

Graphconstruction

Wikipedia senses

Mapping toWikipedia
Community detection
Co-occurrence graph

Communities
FilterUnknown senses

FilterFig. 1. System Overview

graph to be generated. Edges represent relationships

construction.

between two nodes. Edge weights are based on four

Table 1. Words or hashtags aroundtypes of relationships: 1) two nodes co-occur in one“#orange” at different time periods

tweet, 2) two nodes released by the same user, 3) two

t=2015-12-31

t=2015-11-15

nodes which are retweeted, and 4) two nodes co-occur in

#job

#job

an external document. The final weights are the linear#hiring#sunsetweighting of them. Communities are detected based on

#blue

#sky

the community detection algorithm proposed by Vincent

bowl

love

D. Blondel in 2008 [3]. Each community is assigned#yellow#colorwith a list of words which can represent this community.

#naturehealthThe overview is shown in Fig. 1.

sunset#juice3.2 Time split

#red

deal

#green

photo

close

#sun

careerarc

morning#trees#sunrisecotton#flower

opening

joseph#pink#fruit#ca

syracuse#brownale

love#countyreflection

shop

#sky

#carrot

Hashtags have temporal features, meaning that
hashtags are easily influenced by popular events. For
example, the sense of “#orange” is different at different
time periods. The left column belongs to interval“December”, while the right column belongs to interval“November”. Around 2015-12-31, many people may go
to watch the sunset, so they talk about sky more often.
But around 2015-11-15, although there are people
talking about sunset, topic “fruit” and “juice” are also
appearing in tweets (see Table 1). In order to improve theaccuracy of our system, according to different releasetime, we divide these tweets into several time intervals
by months. In each time interval, we perform graphDEIM Forum 2016 D1-1

3.3 Data processing
Tweets are written freely by different people, resultingthat words in tweets are often abbreviated or irregular.
For example, some people use “sta” to represent theword “station”. We apply the following steps to process
tweets in each time interval:
1) The tweet contents are transformed from informal
languages to formal languages according to a formalinformal list (constructed, samples are shown in Table 2)[6].

2) The contents in tweets are filtered according toa stop word list. 3) All the words in tweets aretransformed into their lowercase form.

4) POS (part-of-speech) tagging is applied to each word and hashtag,
to extract noun words.

Fig. 2. Community detection for #cell
retweeted, create an edge between their corresponding
nodes set the edge weight as a factor w3 times the

Table 2. Part of informal-formal list [6]number of the retweets between the two nodes.
4. After Step 3, if two nodes have an edge, thenInformal

Formalsay sorry

apologize

go up

increasego downdecrease

set upestablish

look atexamineblow up

explodedetermined by formula (1). In the formula, w1 is set as 1,

find out

discoverw2 is set as 0.8, w3 is set as 0.5, and wn is determined bymultiply their edge weight by wn, where wn is thesimilarity of the two nodes calculated by WordNet.
WordNet similarity can measure the semantic
similarity or relatedness between two words [2].
The final edge weights are the sum of the four
relationships. For each edge, the edge weight is

WordNet similarity value. Here, the constant factors on

3.4 Construction of co-occurrence graphthe four relationships are empirically determined.

We define a co-occurrence graph of tweets as G=(V,w = 𝑤1 + 𝑤2 + 𝑤3 + 𝑤𝑛

E), where V is the union of a hashtag set Vh and word set

After the co-occurrence graph is constructed, we

Vw. The nodes in Vh are assigned with larger weights

remove all the nodes with a frequency in tweets below

than nodes in Vw, because hashtags can be used totwo, since nodes with such a low frequency cannot

categorize tweets.

represent a sense.

Edge weights are based on the following four types ofrelationships:(1)

3.5 Community detection
In the community detection algorithm of [3], first each

1. If two words or hashtags appear in one tweet, thennode is assigned to a different community. Then, for eachcreate an edge between them and set the edge weight anode v, v is moved to its neighbors w of v which the gain

factor w1 times the number of the tweets in which theis maximum. This process is repeated until no further

two nodes co-occur.

improvement can be achieved. In the second phase, the2. If two words or hashtags in two tweets are released

nodes are the communities obtained from the first phase.

by an identical user, create an edge between their

The weights of edges are given by the sum of the weightcorresponding nodes. Set the edge weight as a factorof the links between nodes in the corresponding two

w2 times the number of common users.

communities. Repeat the two phases on this smaller

3. If two words or hashtags in two tweets are

DEIM Forum 2016 D1-1

graph until there is no more improvement and then aconsidering similarities between them.

Here, tweets

maximum of modularity is attained.

are often containing words that are casual, resulting in

After we obtain communities, we first need to filterno corresponding sense in Wikipedia. So we try toout communities whose sizes are small. The minimum

resolve senses that are having Wikipedia entries, and thesize of a community is calculated by formula (2). In thisremaining communities as unknown or new senses. We

formula, n is the number of words in this community,

do the mapping by the following steps:

and m is the number of hashtags in this community. 𝑊𝑤𝑖

1. Delete articles which are too short. Preprocess theis the weight of the word, and 𝑊ℎ𝑖 is the weight of thehashtag.

remaining articles.
2. Calculate TF-IDF value for each word of all the𝑛𝑚Size = ∑ 𝑊𝑤𝑖 + ∑ 𝑊ℎ𝑖
𝑖=1

articles.
(2)

𝑖=1

If the size of a community is lower than the threshold,we delete it because it cannot represent a sense well.

3. Calculate cosine similarity in formula (3) for each
community vector d1 and each Wikipedia word
vector d2, where the values in d1 are the nodeIn each community, we rank the words according toweights when constructing co-occurrence graph, andtheir weights, and the top 40 words are chosen as thethe values of d2 are the TF-IDF value calculated incontext words of the community.

Step 2.

3.6 Mapping to Wikipedia Disambiguation
Lists

dimension of d1 is 40. Here we also choose the top

Wikipedia has disambiguation lists for resolving
ambiguous article titles. Figure 3 shows a part of thedisambiguation list for “cell”.

As we mentioned in Section 3.5, the40 words in d2, so the dimension of d2 is 40.
sim(𝑑1 , 𝑑2 ) =⃗ (𝑑1 ) ∙ 𝑉⃗ (𝑑2 )𝑉
⃗ (𝑑1 )| ∙ |𝑉
⃗ (𝑑2 )|
|𝑉

(3)

4. After calculating cosine similarities, we obtain asimilarity score. The highest score is regarded as themost suitable match.
5. In the score obtained in Step 4, if all scores are lower
than a threshold, then drop this sense.
6. If two groups match to the same Wikipedia article,
merge them.
7. Repeat step 5 until there is no change.

Table 3. Mapping result of “#cell”

Fig. 3. An example of Wikipedia
disambiguation lists
Each Wikipedia sense on a Wikipedia disambiguation
list corresponds to an article. We can map between
hashtag context words and Wikipedia articles, byDEIM Forum 2016 D1-1

4. Experiments
4.1 Dataset

Table 4. Example of the senses of #train
at different time intervals

We collected tweets via Twitter Search API, andparsed according to their release time. We chose 10

Time

Senseshashtags, including “#apple”, “#banana”, “#bank”,“#book”, “#cell”, “#key”, “#orange”, “#rose”, “#season”
and “#train”. For each hashtag, we use this hashtag as asearch keyword to collect 3000 tweets in 2015. We
conduct experiments for each hashtag at each time

Train (noun)

201511-1 to2015-

peace, gym, love, travel,1

2

month. For example, 2015-1-1 to 2015-1-31 is a timeusing funny, interesting, beautiful words than formal

12-1 to2015-

brain,
1

workout,

performance,

fit,

gym,
station,railway, new, ride, service,

12-31

photographTrain (Training)likely to be retweeted, the user him/herself is also moreeasily, dvd, mma, system,

Train (noun)

2015-

words. Because by using these words, tweets are more2

likely to be followed.

soccer, potty, child, videos,program, pro, save, wrestlininterval, 2015-2-1 to 2015-2-28 is also a time interval.

We found the following phenomena:fit,

Train_(wrestler)

beginning of each month and end at the end of each

Word preference on twitter. In Twitter, people preferhealth,photogratime intervals. Each time interval starts from the

workout,

photograph, gain, london,

11-30

interval separately. In our experiment, we construct 12

An example is shown in Table 4.

Words

Because Wikipedia disambiguation lists are notcomplete, certain communities cannot be mapped to acorrect sense, even when it has a corresponding
Wikipedia article.

Fig. 4. Average error rate at each monthfollow, save, videos, body,radio, bag, walk, bike, door,mma

DEIM Forum 2016 D1-1

Fig. 5. Average number of communities in each month

Fig. 6. The number of each community in each month
Section 3.4. Then a human judge examines the mapped

5. Evaluation

results and determines whether matches are correct or

We evaluate our results based on the error rate. For

wrong. In our framework, the Wikipedia article which iseach tweet, we first allocate it to a time interval

related to a given graph community most will be chosen.

according to its release time. Then as described in

But

Section 3.5, we utilize context words to map graphcorresponding each other.

communities to articles listed in disambiguation lists.

examines two stages of validity: 1) mapping between a

Tweets are ranked by summing up all the weights ofhashtag and its community, and 2) mapping between awords occurring in the tweets, where the weight of each

community and its article.

word is the sum of the edge weights of the graphcommunity the word belongs to, as we mentioned incommunities

andarticlesarenotalwaysTherefore the human judge

An example is shown below:

Tweet Content: Cell #Phone Plans - NO Contracts,
NO#CreditChecks,NO

Bills,#Mobile-DEIM Forum 2016 D1-1

http://CellPhone-Plans.net- Low #Cell Prices -#CellPhone 2015-2-15.
Sense1: 928.8

that twitter users use words freely so that we have nogolden standard to match them. In order to solve thisSense2: 0.0

problem, we transform informal words to formal words.

This is a tweet containing hashtag “#cell”. The

This can improve the situation in some degrees. Our

number after a sense Id is the edge weights of theultimate goal is to detect senses that are emerging andmapped graph community. By doing the mapping we

not covered by Wikipedia or other knowledge bases, but

mentioned in Section 3.6, we can get a correspondence

popular between twitter users.

between Sense 1 and “Mobile Phone” in Wikipedia,

In the future, we will focus on finding a better

while Sense 2 has a correspondence with “biology” instandard to match words in tweets. Also, for dividing

Wikipedia.

tweets into different time intervals, we will find a better

Sense1 has a higher score than Sense 2, so

our algorithm chooses Sense 1.

The human judge

function to divide it.

examines correctness of the top-ranked mappings.
Here also has an error example:

7. Reference

Tweet Content: If you don't have a #KINDLE[1] R. Pandit, S. Kumar Naskar. A Memory Baseddownload the free app for #CELL #PC #TABLET

Approach to Word Sense Disambiguation in Bengali

http://amzn.to/1BtVqdO

Using k-NN Method. 2015 IEEE, p 383 – 386.

http://getBook.at/HTG100K#Howtouse #Twitter 2015-11-15
Sense1:0.0[2] T. Pedersen, S. Patwardhan, J. Michelizzi:

Sense2: 0.0

WordNet::Similarity - Measuring the Relatedness ofIn this example, the edge weights are all 0.0. But

Concepts. HLT-NAACL 2004.

obviously, “#cell” in this tweet should also be mapped to[3] V.D. Blondel, J.-L. Guillaume, R. Lambiotte, E.

“Mobile Phone”. So we regard this tweet as a wrong

Lefebvre.: Fast unfolding of communities in large

example.

networks J. Stat. Mech. Theory Exp. (10), 2008, pp.

We evaluate by the error rate as shown in (4). If the10008.

ratio of correct mappings become higher, the error rate

[4] D. Jurgens: Word Sense Induction by Community

will be lower.

Detection. Graph-based Methods for Natural Language

Error rate =𝑇ℎ𝑒 𝑤𝑟𝑜𝑛𝑔 𝑚𝑎𝑡𝑐ℎ𝑒𝑠(4)

𝐴𝑙𝑙 𝑚𝑎𝑡𝑐ℎ𝑒𝑠

In evaluating error rates, we choose 100 tweets foreach of the 10 benchmark hashtags.

Processing, 2011, pages 24–28.
[5] M. Wang, M. Iwaihara: Hashtag Sense Induction
Based on Co-Occurrence Graphs.

Proc. 17th Asia-

Figure 4 shows the

Pacific Web Conference (APWeb 2015), Lecture Noteaverage error rate of the evaluation, where the average isin Computer Science 93132, pp. 154-165, Sep. 2015.

taken over hashtags collected in each period. In Figure 4,[6]

the error rate in July is lowest, while in January it

informal-english/;becomes highest. It could be explained as people use

academic.com/2012/07/06/informal-formal-vocabulary-special words in January, such as “#festival” andfor-ielts/“#NewYear”. As a result, words become difficult to bemapped to Wikipedia disambiguation lists. Figure 5
shows the average number of communities in eachmonth, and Figure 6 shows the details of the Figure 5.

6. Conclusion and future work
In this paper, we proposed a method to disambiguate
hashtag senses based on contents, and we observed
temporal changes of these senses. The most challenge is

http://www.engvid.com/english-resource/formalhttp://ielts-