DEIM Forum 2016 A3-3

Finding "Similar" Concepts with Evidences across Different Feature Vector
SpacesYating ZHANG

Adam JATOWT and Katsumi TANAKA

Graduate School of Informatics, Kyoto University Yoshida Honmachi, Sakyo, Kyoto 606-8501, Japan
E-mail: {zhang, adam, tanaka}@ dl.kuis.kyoto-u.ac.jp
Abstract “The past is a foreign country: they do things differently there” is an often quoted opening sentence of a famousnovel by L. Hartley entitled “Go Between”. It emphasizes the well-known observation that it is often difficult to understandentities, concepts and their context in the past, especially, in the distant past. Even though some professionals, common sense orjust our intuition might suggest that certain entities may be similar across time, still, usually, we lack convincing and concrete
evidence to support similarity estimation. For example, it may not be immediately obvious why Walkman in 1980s is considered
to be a similar entity to iPod. However, we can easily understand their similarity when learning that both were dominant, portablemusic devices in their corresponding times. In this paper, we propose to automatically detect evidences to explain similarities
and differences between entities in different time periods. For a given input entity we first output the ranked list of candidate
counterparts and then we detect supporting evidence to explain why the results are similar or different to the input entity. The
evidences our method generates should be relevant to entities, cover their diverse and important aspects and allow for easycomparison.
Keyword Evidence Finding, Similarity Searching, Difference Searching, Entity Comparison

1. INTRODUCTION[49]. In such an approach, a series of questi ons can be

In the current fast-paced world, people tend to possess

answered, such as “what music device 30 years ago played

limited knowledge about things from the past. An averagesimilar role as iPod does nowadays?” or “who today’s

person typically knows only about events and entities

Beatles are?” However, answering “what” questions is nottaught at school or ones curated as collective memory - theenough for users to deeply understand and verify thehighly selective representation of the past maintained byprovided answers, such as to determine in which way amass media. Furthermore, besides many past entities andgiven temporal counterpart is similar to the queried entity

events, also the contexts of distant times remain unknown

or how different both of them are.

for many users who do not actively study the past . Yet,Considering the issues raised above, we focus in thismaintaining the knowledge and the memory of the past is

work on solving the “why similar” question which helps

definitely important and, therefore, various memory

explaining temporal counterparts by outputting “evidence”

institutions such as libraries and archives offer open accessterms for clarifying the similarity and differences between

to past documents they keep. However, searching within

the counterparts. For example, given a pair of entities (e.g.,such collections as well as understanding the retrieved

iPod and Walkman) considered as temporal counterparts,

content is hampered due to our limited knowledge ofthe system should return a set of evidences indicating (1)vocabulary used in the past and its meaning. Considering

similar concepts maintained over time (i.e., both entities

the limited knowledge of the past average users possess, it

are used to listen to music, both are designed to be awould be beneficial if the users were provided with som eportablemusic

device, and both utilize somekind of assistance when interacting with archival document

storagemedia

tocollections. Ideally, such assistance should empower themdifferences that can be used to d ifferentiate each other (i.e.,

to search and comprehend within the past collections as

iPod has large storage size enabling to store more

efficiently as they would in “present” collections such as

songs, iPod allows watching movies, iPod has a

the Web.

display panel to show the information about songs

In our previous work, we approached a research problem

store

songs);(2)significant

such as lyrics, singer name and song name).

of temporal counterpart search which returns semantically

The problem of similarity and difference detection forsimilar terms from the past to an input query from thetemporal counterparts is not trivial. The key difficultypresent time using large temporal document collections.

comes from the need for comparisons on the concept levelinstead on the literal word level, since the context tends toterm comparison across time in Section 4. Then, wechange much across time, especially, over longer distan ces.

introduce our approaches for similarity and difference

In other words, for any pair of temporal counterparts, theirdetection in Section 5. Section 6 describes the optimization

context can be quite different due to time passage. Many

way to organize the output. The experiment al results arecases that are in fact desired similar concepts between

illustrated in Section 7. We conclude the paper and list

temporal counterparts may still in fact remain but may not

future directions in Section 8.

be mentioned in either of the context. For example, iPodand Walkman both utilize storage media to store the songs,

2. RELATED WORK

but in their context, the concept, storage media, might not

Temporal Information Retrieval has become the subject

be explicitly mentioned. Instead, for iPod, MP3 isof multiple studies in the recent years [12]. Prior research

considered as a suitable storage media, and Walkman uses

focused on tasks such as time-aware document ranking

cassette to store music. Note that naturally it is possible[6,13,15,25,30], temporal organization of searc h resultsthat the concepts (e.g., listening to music, portable[2,3], query understanding [13,33,23], future in -formation

music device) shared by the temporal counterparts are

retrieval [4,19,21], analysis of how the word meanings

directly mentioned and thus can be easily detected bychange over time [20,27,28,34], or addition of context totaking the overlap of their context. Our system however

explain the past [46] and so on.

focuses on solving the former hard case where the concepts
are not directly mentioned in the context.

Among the above topics, time-driven change of the semantic meaning - an emerging topic of study within

Another challenge lies in the criteria necessary forhistorical linguistics [1,11,18,29] is relevant to this work.

selecting an informative set of evidences. First, it is

Several researchers employed computational methods forimportant to select important aspects of entities. There may

analyzing changes in word senses over time. Mihalcea et

be many similar and different points between the compared

al. [34] classified words to one of three past epochs based

entities, however, it is essential to extract only significant

on word contexts. Kim et al. [27] and Kulkarni et al. [28]aspects and eliminate the obvious ones (i.e., both of iPod

computed the degree of meaning change by applying neuraland Walkman are of rectangular shape). Secondly, thenetworks for word

system should output organized results to offer useful

different from the above approaches as we direct ly search

explanation. Since the output is in the form of a set offor corresponding terms across time, and, in our case,evidences, it should be constructed considering the

temporal counterparts can have different syntactic forms.

coverage and diversity of the selected items.

representation.

Our objective is

Certain works approached the problem of computing

In view of the challenges mentioned above, we firstterm similarity across time [5,22,24,45]. Kalurachchi et al.

propose an approach, which enables to compare terms in[22]

regards to their semantic meaning. In other words, we

temporally altering concepts by applying association rule

propose the comparison on a concept level, by bridging

mining, assuming that concepts referred by similar events

term representations from one vector space (e.g., on e

(verbs) are semantically related. Kahabua et al. [24]derived from the present documents) and those frominvestigated detection of the change of terms through theanother vector space (e.g., one built from the pastcomparison of temporal Wikipedia snapshots. Berberich etdocuments).

Terms

in

are

al. [5] approached the problem by introducing a HMM

represented

bydistributed

representation

model and measuring the across-time sematic similarity[35,36]. Next, we propose metho ds to detect similar

between two terms by co mparing the contexts captured

concepts maintained by the given t wo entities and tousing co-occurrence measures. Tahmasebi et al. [45]discover essential differences between them. Finally, we

improved that approach by, first, detecting the periods ofintroduce an optimization function to optimize the returnedname change and, then, by analyzing the contexts during

set of similar and dissimilar points by the criteria of their

the change periods to find the temporal co -references ofcoverage and diversity.

different names. Several important differences distinguish

the

both

the

vectorvectorspacesproposedtodiscoversemantically

identical

The rest of the paper is organized as follows. In the next

our work from those works. First, the previous works

section we review the related work. In Section 3 we

focused mainly on detecting changes in the names of theformally define the problem and describe its background.

same, single entity over time. For example, the objective

We next describe the method for term representation andwas to look for the previous name of Pope Benedict (i.e.,

Joseph Ratzinger) or the previous name of St. Petersburg

networks was originally proposed by [42]. Mikolov et al.

(i.e., Leningrad). Second, the above mentioned approaches[35,36] improved such representation by introducing Skip -relied on applying the co-occurrence statistics according togram model based on a simplified neural network

the intuition that if two terms share similar contexts, thenarchitecture for constructing vector representations ofthese terms are semantically similar. In our work, we dowords from unstructured text. Skip -gram model has several

not require the context to be literally same (i.e., having

advantages:

same surface forms of context terms) but to have the same

relationships; (2) it can easily scale to millio ns of words.

meaning.

After applying Skip-gram model, a m×p matrix is created

(1)

it

capturesprecisesemantic

word

Transfer Learning [39] is to some extent related to ourfrom the documents in one space (e.g., the documents inwork. It has been mainly used in tasks such as POS tagging

2000s), D(T 1 ), where m is the vocabulary size and p are the

[9],

rank

dimensions of feature vectors. Similarly, a n×q matrix is[10,16,48] and content-based retrieval [26]. The temporaltext

classification[7,32,47],

learning

toconstructed from the documents in another space (e.g., thecorrespondence problem can also be understood as a

documents exist in 1980s), D(T 2 ) (as shown in Fig. 2).

transfer learning as it is a search process that uses samples
in the base time for inferring correspondent instances inbase timetarget time(e.g. 1987-1991) (e.g. 2003-2007)

the target time. However, the difference is that we do nott

only consider the structural correspondence but we also

Training by using
Skip-gram modelutilize the semantic similarity across time.

3. BACKGROUND AND PROBLEM DEFITION

Training by using
Skip-gram model

In this section, we formally define the problem ofsimilarity/difference

searchbetween

two

Vector Space
Representation

potentiallyFig. 2. Creating word vector representations for two spaces.

similar entities.
PROBLEM STATEMENT. Given two potentially similar

4.2 Term Comparison across Vector Spaces

entity names, e 1 and e 2 where e 1 and e 2 exist in different

Our goal is to compare words in two vector spaces.

spaces T 1 and T 2 , respectively, the task is to find (1) the set

However, it is impossible to directly compare words in twoofdifferent

similarconcepts

theyshare,

Ssim(e 1 ,e 2 )=semantic

vectorspacesasthefeatures{w 0 ≈  0 ,…,w i =  i } where w i and  i exist in e 1 and e 2(dimensions) in both spaces have no direct correspondence

respectively; (2) the set of their differences, Sdiff(e 1 ,e 2 ) =between each other (as shown in Fig. 1). To solve this{w j (e 1 ),…,  j (e 2 )} where w j (e 1 ) exists in e 1 but not in e 2 ,problem, we train a transformation matrix to build the j (e 2 ) denotes the opposite.

connection between the two vector spaces. To better

DEFINITION 1 (SEMANTICAL SIMILARITY). If theimagine the transformation idea, the semantic spaces could

context of w is semantically similar to the context of  ,be compared to buildings. If we regard two semantic spaces

then w is semantically similar to  .

as two buildings, then, in order to map the components
from one building to ones in the other one, we need first to
Output

music

music

Sony

Apple
song

iPod

Walkman

song

wj

mp3

wi

j

i

<music, music><Apple, Sony><mp3, cassette>…
<wi, i>cassette

to each other. Afterwards, the rest of the components canbe mapped automatically by considering their relative
positions to the main frames of their building. So, in ourcase, having found the correspondence between the anchor

T2

T1know how the main frames of the two buildings correspondterms in the two semantic spaces, we can automatically*Context terms of a given entity derived from frequently co-occurring terms

map all other remaining terms relative to these anchors. Fig.

Fig. 1. Conceptual view of pair detection .

3 conceptually portrays this idea by showing that thecorrespondence of anchor terms enables mapping otherterms, such as iPod to Walkman so that the relative position

4. ACROSS-TIME TERM COMPARISON

between iPod and anchors in one space is similar to the

4.1 Term Representation
Distributed

representation

relative position between Walkman and the corresponding
of

words

using

neuralanchors in another space (only two dimensions are shownfor solving the regularized least squares problem ( γ equals

for simplicity).

to 0.02) with the regularization component added to

music

music

prevent overfitting.
iPod

Walkman

K

river

river

M  arg min  M  i  wi 2   M 2

animalanimalman

man

base time(e.g. 2003-2007)

M

2

2

(1)

i 1

target time(e.g. 1987-1991)

Fig. 3: Conceptual view of the across vector spaces
transformation by matching similar relative geometric positions
in each space .

5. SIMILARITYAND

DIFFERENCE

DETECTION
In In this section, we first discuss the criteria of a set of

For realizing the above described t ransformation, it isterms to be useful for indicating the actual similarities or

essential to first find good anchor terms (“main frames”)

differences of terms across time. Based on these criteria,

which can help to build the correspondence between any

we then propose several features to extract similarities

two semantic spaces. However, it is non -trivial to manually

between two given entities and their differences.

prepare large enough sets of anchor terms that would covervarious domains as well as would exist in any possible

5.1 Criteria for Selecting Evidences

combinations of the base and target time periods. We then

Similarity of entities. <w i ,  i > is a pair of terms where

rely here on an approximation procedure for automaticallyw i appears in the context of entity e 1 and  i occurs in theproviding anchor pairs. We select terms which (a) have thecontext of entity e 2 . <w i ,  i > explains how e 1 is similar tosame syntactic forms in the b ase and the target time periods,

e 2 . <w i ,  i > should have at least some of the followingand (b) which are frequent in both the time periods. Suchcharacteristics: (a) w i is highly relevant to e 1 while  i is

Common Frequent Terms ( CFTs) are then used as therelevant to e 2 ; (b) w i and  i should indicate similaranchor terms. One reason to choose CFTs as anchors is thatconcept; (c) the relation between w i and e 1 should befrequent terms tend to be “connected” with many othersimilar to the relation between  i and e 2 .

terms. Another is that frequent terms (e.g., sky, river,Difference between entities. w i is a context term ofmusic, cat) change their meanings over time only toentity e 1 , denoted as w i (e 1 ) such that it denotes a concept

small extent. The more frequently a word is used, theexisting in e 1 but not in e 2 . w i (e 1 ) can explain the difference

harder is to change its dominant meani ng (or the longerof e 1 from e 2 . Similarly,  i is a context term of entity e 2 ,time it takes for a word to undergo the meaning shift) as

denoted as  i (e 2 ) such that it represents a concept existing

the word is commonly used by many people. The

in e 2 but not in e 1 .  i (e 2 ) can then explain the difference ofphenomenon that words used commonly in everyday

e 2 from e 1 . w i (or  i ) should at least satisfy some of thelanguage had evolved more slowly than words used les sfollowing characteristics: (a) w i (or  i ) is highly relevant

frequently

languages

to e 1 (or e 2 ); (b) the concept behind w i (or  i ) is less likely

including English [31,40]. This assumption guarantees

to appear in e 1 (or e 2 ); (c) it is rare to find such relation ofrelatively good correspondence between the two frames

w i and e 1 in the context of e 2.

has

been

observed

inseveralthat would be “constructed” with the help of CFTs.
After determining the anchor terms, our task is to build
the correspondence between two semantic spaces byutilizing the set of prepared anchor terms. In particular, we
will train the transformation matrix to automatically map
dimensions of the base vector space to the ones in the target
vector space. Let us suppose there are K pairs of anchor
terms {(  1 , w 1 ),…,(  k , w k ,)} where  i is a anchor in one
space and w i is its counterpart anchor in another space. The
transformation matrix Μ is then found by minimizing thedifferences between Μ∙  i and w i (see Eq. 1). This is doneby ensuring that the sum of Euclidean 2 -norms between thetransformed query vectors and their counterparts is as
small as possible when using K anchor pairs. Eq.1 is used

5.2 Feature Estimation
We start to quantify the objectives listed above. Threefeatures can be generalized from the above high-level
criteria:

relevance,

intra-similarity,andrelational-

similarity. We define and estimate each feature as follows.
Relevance. Rel(w,e) is the strength of the relatedness
between a context term w and entity e. It is measured by
the multiplication of two conditional probabilities (see Eq.
2). The left side guarantees the context term w is relevantto the entity e while the right side gives more priority tothose terms which only co-occur with e. Intuitively, thecontext term which is unique to the entity will have a highrelevance score. For example, although the context termmusic frequently appears within the context of iPod, it

is not unique to iPod since music can co-occur withthe difference (S di ff ) between two entities.

many other entities. On the other hand, Apple is more

We define a good set S sim (or S diff ) as the one in which

relevant to iPod since both frequently co -occurred witheach item have high quality but is different from each other.

each other.

In other words, a good set of evidence should consider bothrel w, e   P( w | e)  P(e | w)quality and diversity within the set.

6.1 Quality of EvidenceP( w, e) P( w, e) P( w, e) 2P ( e)
P( w)
P( w)

(

(2)

max (nt )) 2  tDmax (nt ,e )
nw
n w,e

We define the quality of the item in S sim as the pair ofterms <w i ,  i > which should have high in relevance score,high in intra similarity and high in relational similarity (seetd ( e )Eqs. 6-7).

Note that Eq. 2 computes the relevance be tween eachquasim  wi , i   rel ( wi , i )  sim( wi , i )1

(6)

sim( wi , i )  simIntra( wi , i )  simR( wi , e1 ,  i , e2 )(7)context term of a given input entity, e.g., rel(w i ,e 1 ) andrel(  i ,e 2 ) estimate the relevance of context terms of entity
e 1 and e 2 , respectively. To apply this feature for detecting

On the other hand, the quality of the item in S di ff is given

the evidences to indicate the similarity between two

to a single term, w i (e 1 ), which must be high in relevance

entities, the relevance of each evidence, rel(<w i ,  i >), canbe computed as Eq. 3.

score to e 1 , low in the intra similarity with all the context
terms (  i ) in e 2 and low in the relation similarity (see Eq.

rel  wi , i    rel ( wi , e1 ,  i , e2 )

8).
(3)

 rel ( wi , e1 )  rel (i , e2 )

quadiff wi (e1 )   rel (wi , e1 )  (1  max sim( wi ,  j )1 j C

Intra-Similarity. This feature measures the similarity
between a pair of context terms <w i ,  i > where w i is the(8)

where, C is the set of context terms of e 2 . sim(<w i ,  i >) is

context term of entity e 1 and  i is the context term of entity

computed by Eq. 7.

e 2 . As discussed in Sec. 4, since w i and  i come from

6.3 Output Optimizationdifferent time periods (i.e., different vector spaces) to

As mentioned before, diversity is an important criteria

compare their similarity, it is essential to first transform

for a good set of evidences. In this section we propose an

the representation of a term (e.g., w i ) from one vector space

objective function to optimize the output evidences by(e.g., T 1 ) to another space (e.g., T 2 ). We then compare theconsidering both the quality and diversity of the returned

transformed

representation(e.g.,

M T w i )

with

the

representation of the term in T 2 (e.g.,  i ) by cosine
similarity. The intra-similarity of <w i ,  i > is computed byset.
1. The evidence should be relevant to its entity (relevance)e.g., wi is relevant to e1 and i is relevant to e2

Eq. 4.

simIntra( wi , i )  cosM  wi , i 
Relational-Similarity.

Besidesthe(4)

intra-similarity

discussed above which measures the semantic similarity
between w i and  i , the relational similarity estimates if therelation between w i and e 1 is corresponds to the relation
between  i and e 2 . To represent the relation between w i ande 1 , we take the difference of their vector representations ,w i −e 1 . The relational similarity is defined as below.

simR( wi , e1 ,   i , e 2 ) cosM  ( wi  e1 ), ( i  e 2 ) 

3.

The selected pairs ofevidences should be different
from each other (diversity)e.g., <wi, i> ≠ <wj, j>

2. wi and i should be similar to each other (similarity)e.g., wi ≈ i

Fig. 4: Conceptual view of the optimization process .

Eq. 9 is used for the process of finding good set. Eachtime we select a high quality item which, at the same time,should be as dissimilar as possible to the already selected(5)

items, until reaching a predefined size of returned(selected) set.

L  arg max{  qua( Ai )  (1   ) max (simInter ( Av , Ai ))}(9)simInter ( Av , Ai )  cos( wv , v ,  wi , i ) cos(wv , wi )  cos(v , i ) for S sim ,for S diff ,simInter ( Av , Ai )  cos(wv , wi )(10)

Ai U  S

6. EVIDENCE FINDING
Based on the features computed in Sec. 5, in this section,
we introduce our method to detect a set of evidencesindicating similarity (S si m ) and a set of evidences denoting

Av S

L is the size of the returned set. S denotes the selectedusing such queries when they are curious about any

set and A v is an item in S, while U−S indicates theevidences that can indicate similarity between two entities,

unselected set and A i is the item in U−S. Note that Eq. 9 iswhich are typically considered different. To create the test

a generic function which can be applied to optimize thesets we will utilize external resources including theevidences S si m when replacing qua(A i ) by Eq. 6 or to

Wikipedia, a Web search engine and several historical

optimize the evidences S dif f by substitution with Eq. 8.

textbooks. In total, 51 pairs of entities will be tested. All
the test pairs are going to be made available for others toexperiment with.

7. EXPERIMENTS

We are going to add the experimental results over the

7.1 Dataset

manual constructed test sets in the camera ready paper. As

For the experiments we use the New York Times
Annotated Corpus [43]. This dataset contains over 1.8
million newspaper articles published between 1987 and2007. We select two time periods [1987,1991] and[2002,2007] for testing our proposed methods on finding
similarity/difference evidences between two entities acrosstime (e.g., Walkman from [1987,1991] and iPod appearsin [2002,2007]). Each time period contains around half amillion news articles. We next train the model of
distributed vector representation separately for these twotime periods. The vocabulary size of the entire corpus is360k, while the vocabulary size of each time period isaround 300k.

benches for our research task. We plan then to resort tomanual construction of test sets. The test sets will containtwo entity names with two sets of evidences respectively
indicating similarity and difference between them. In orderto test the performance of the proposed methods overdifferent types of queries, we are going to categorize thetested pairs of entities into three groups: (A,A), (A,A’) and(A,A)

represents

thosequeries

where

two

components of query have the same surface form such as(Japan, Japan). Here the user ’s search intent might bedetecting the consistenc y and changes of the same entity
time(e.g.,

comparison

betweenJapan

in[2002,2007] and in [1987,1991] ). (A,A’) denotes queries
where two entities have different surface forms but are
similar in semantics, such as (iPod, Walkman). In ourprevious

work

[49],

we

named

them

as

temporalcounterparts. For the purpose of experiments, we will
select several temporal counterparts that we used before 1
as tested queries. Finally, (A,B) is the type of queries where
A and B are totally different entities, that is, both are
different in surface form and in semantic meaning (or there
is no correspondence between them). Users may search
1existing in [2002,2007] and e 2 is the entity in [1987,1991].

7.3 Evaluation Measures and Tested Methods
Evaluation Measures. We will use precision and recall
as the main measures for evaluating the returned set ofevidences.
Baselines. We are also going to prepare two baselines asfollows:(1)

Overlap approach (Overlap): this method detects

similarities by directly computing the overlap of thecontext of two queried entities and se lecting the most
extracted from non-overlapped terms with the condition

To the best of our knowledge, there are no standard test

across(e 1 , e 2 ) and their results in Table 1 (e 1 denotes an entity

relevant ones; the differences (e.g., w i (e 1 )) will be

7.2 Test Sets

(A,B).

for now, we show some of the examples of the test queries

http ://www.d l.kuis.k yoto -u.ac.jp/~ada m/te mpora lsearc h_s hort.t xt

that the terms (e.g., w i ) should be highly relevant to theircorresponding entity (e.g., e 1 ). We will test Overlap
approach to examine whether the distributed vector
representation and transformation are necessary.
(2)

Skip-gram Model without transformation (COM):

the purpose of including this baseline is to test thenecessity of transformation across vector spaces for
detecting good sets of words denoting similarities anddifferences.

Since

COM

also

uses

distributional

representation for capturing word semantics same as the
proposed methods do, the only difference is that the term
vector training process combines the documents from twotime periods. Thus the vocabularies in the two time periods
are represented within one vector space. Different from theseparate training process, the combin ed training will likely
lose the relative positions between the terms in each time
period. Also COM assumes that the terms existing in bothtime periods have exactly same meaning (or does notsubstantially change their meanings) since there is a uniquevector representation for each term in the combined vector
space.
Proposed Methods. We will test the proposed methodsmentioned in the previous sections. All of them use theneural network based term representation. Since ourmethods consider

severalfeatures(relevance,

8. CONCLUSTION AND FUTURE WORKIn this paper we have proposed a method for computing

intra-

similarities and differences between pair of entities acrosssimilarity and relational-similarity) in the experiments wetime. The objective is to help users understand how things

will not only compare our proposed methods with all theor concepts in the past differ from the ones existing atfeatures included, but we are also going to test thepresent. We have applied transformation matrix andeffectiveness of each feature by adjusting the weight ofconsidered several characteristics of ideal evidences ofeach feature and tracking performance variations.

similarities and differences such as relevance, similarity

7.4 Experimental Results

and diversity.

Table 1 displays the returned similarity evidences forIn the future we plan to conduct extensive experiments

few example queries. The returned size of evidence equals

to evaluate our method over diverse datasets and different

20 in the experiments.

lengths of time periods. In addition, we will design more[Theevaluations

overdifferent

metrics

andtheexhaustive approaches such as ones using clustering.

performance comparison between proposed meth ods andthe baseline methods will be included later.]Table 1. Example results of similarity evidences generated by the proposed methods where e 1 is the entity in the time [2002,2007] ande 2 is in [1987,1991] . In the experiments, the size of the returned evidence set equals 20.
(e 1 ,e 2 )(ipod,walkman)
1<apple,son y>
2<itunes,cassettes>
3<mp3,discman>
4<audio,audio>
5<gigabyte,milimeter>
6<sony,nintendo>
7<digital,digital>
8<nano,rechargeables>
9<portable,portable>
10<music,cassette>
11<macintosh,rom>
12<video,video>
13<zune,dat>
14<cd,cd>15<bluetooth,vhs>
16<microsoft,matsushita>
17<download,tape>18<podcasts,disks>
19<podcasting,stereo>
20<electronics,electronics>(apple,sony)(putin,yeltsin)(facebook,usenet)(japan,japan)<ipod,vhs><itunes,audio><macintosh,digital><mac,disk><brion,guber><gigabyte,millimeter><imac,vcr><iphone,dat><mp3,cd><ipods,video><nano,disks><pc,stereo><computer,tape><music,cassette><portable,compact><mini,walkman><digital,hdtv><intel,matsushita><computers,recorders><macs,cassettes><russian,mikhail><russia,soviet><yeltsin,gorbachev><soviet,coup><lugovoi,engver><kremlin,republics><treaty,anti><moscow,moscow><united,union><chechnya,russia><chechen,russian><oligarchs,communist><khodorkovsky,bakatin><russians,plotters><gorbachev,kravchuk><bush,bush><lehrer,schuster><nuclear,nuclear><ukraine,ukraine><stalin,reformers><myspace,unix><zuckerberg,stoll><networking,encryption><coulton,gemmel><cloyd,davida><xanga,cryptography><yahoo,workstations><boink,gilmore><semel,rotenberg><web,internet><google,computer><mail,mail><oleyourryk,menlo><silicon,electronic><ck,ipc><hali,proteus><sites,networks><users,users><comScore,networked><Murdoch,stahlman><koizumi,kaifu><US,US><yasukuni,hirohito><tokyo,tokyo><korea,korea><abe,yasuhiro><yen,yen><takenaka,noboru><fukui,takeshita><shimbun,shimbun><exports,imports><militarism,anti><trillion,surplus><whaling,whaling><nuclear,fsx><emperor,emperor><ministry,ministry><trade,trade><prime,prime><minister,minister>

REFERENCES[1] J. Aitchison. Language Change, Progress or Decay?
Cambridge University Press, 2001.
[2] O. Alonso, M. Gertz, R. Baeza-Yates. Clustering andexploring search results using timeline constructions.
In Proc. of CIKM, pp. 97-106, 2009.
[3] O. Alonso, K. Shiells. Timelines as summaries of
popular scheduled events. In Proc. of WWW, pp.
1037-1044, 2013.
[4] R. Baeza-Yates. Searching the future. In SIGIR
Workshop MF/IR, 2005.
[5] K. Berberich, S. J. Bedathur, M. Sozio and G. Weikum,
Bridging the Terminology Gap in Web Archive Search,
In Proc. of WebDB, 2009.
[6] K. Berberich, M. Vazirgiannis, G. Weikum. Time aware authority ranking. Internet Mathematics, 2(3):301-332, 2005.
[7] J. Blitzer, M. Dredze, and F. Pereira. Biographies,
Bollywood, Boom-boxes and Blenders: Domain
Adaption for Sentiment Classification. In Proc. of
ACL, pp. 440-447, 2007.

[8] E. Borovikov. A survey of modern optical character
recognition
techniques.
arXiv
preprint
arXiv:1412.4183, 2014.
[9] J. Blitzer, R. McDonald, and F. Pereira. Domain
adaption with structural correspondence learning. In
Proc. of EMNLP, pp. 120-128, 2006.
[10] P. Cai, W. Gao, A. Zhou et al. Relevant knowledge
helps in choosing right teacher: active query selection
for ranking adaptation. In Proc. of SIGIR, pp. 115 124, 2001.
[11] L. Campbell, Historical Linguistics, 2nd edition, MIT
Press, 2004.
[12] R. Campos, G. Dias, A. M. Jorge and A. Jatowt.
Survey of temporal information retrieval and related
applications. ACM Computing Surveys (CSUR),
47(2), 15, 2014.
[13] W. Dakka, L. Gravano, G. Ipeirotis. Answering
general time-sensitive queries. IEEE TKDE, 24(2):220-235, 2012.
[14] S. Deerwester et al., Improving Information Retrieval
with Latent Semantic Indexing, In Proc. of the 51st

Annual Meeting of the American Society for
Information Science, 25, pp. 36–40, 1988.
[15] L. Elsas, T. Dumais. Leveraging temporal dynamics
of document content in relevance ranking. In Proc. of
WSDM, pp.1-10, 2010.
[16] W. Gao, P. Cai, K.F. Wong et al. Learning to rank only
using training data from related domain. In Proc. of
SIGIR, pp. 162-169, 2010.
[17] Z.
Harris,Distributional
Structure,Word,10(23):146–162, 1954.
[18] G. Hughes, Words in Time: A Social History of the
English Vocabulary. Basil Blackwell, 1988.
[19] A. Jatowt A, C. M. Au Yeung. Extracting collective
expectations about the future from large text
collections. In Proc. of CIKM’11, pp. 1259 -1264.
[20] A. Jatowt and K. Duh. A framework for analyzing
semantic change of words across time. In Proc. of
JCDL, pp. 229-238, 2014.
[21] A. Jatowt, K. Kanazawa, S. Oyama, et al. Supporting
analysis of future-related information in newsarchives and the web. In Proc. of JCDL, pp. 115 -124,
2009.
[22] A. Kalurachchi, A. S. Varde, S. Bedathur, et al.,
Incorporating Terminology Evolution for Query
Translation in Text Retrieval with Association Rules,
In Proc. of CIKM, pp. 1789-1792, 2010.
[23] N. Kanhabua, K. Nørvåg. Determining time of queries
for re-ranking search results, Research and Advanced
Technology for Digital Libraries. Springer Berlin
Heidelberg, pp. 261-272, 2010.
[24] N. Kanhabua, K. Nørvåg, Exploiting Time -based
Synonyms in Searching Document Archives, In Proc.
of JCDL, pp. 79-88, 2010.
[25] N. Kanhabua, K. Nørvåg. A comparison of time -aware
ranking methods. In Proc. of SIGIR, pp. 1257 -1258,
2011.
[26] M. P. Kato, H. Ohshima and K. Tanaka. Content -based
Retrieval for Heterogeneous Domains: Domain
Adaption by Relative Aggregation Points. In Proc.
SIGIR, pp. 811-820, 2012.
[27] Y. Kim, Y-I. Chiu, K. Hanaki, D. Hegde and S. Petrov.
Temporal Analysis of Language through Neural
Language Models. In Proc. of ACL Workshop, pp. 61 65, 2014.
[28] V. Kulkarni, R. Al-Rfou, B. Perozzi, and S. Skiena.
Statistically Significant Detection of Linguistic
Change. In Proc. of WWW’15, pp. 625 -635.
[29] W. Labov. Principles of Linguistic Change, WileyBlackwell, 2010.
[30] X. Li, W. B. Croft. Time-based language models, In
Proc. of CIKM, pp. 469-475, 2003.
[31] E. Lieberman, J.-B. Michel, J. Jackson, T. Tang, M. A.
Nowak. Quantifying the evolutionary dynamics oflanguage. Nature, pp. 713-716, 2007.
[32] X. Ling, W. Dai, G. R. Xue, Q. Yang and Y. Yu.
Spectral domain-transfer learning. In Proc. of
SIGKDD, pp. 488-496, 2008.
[33] D. Metzler, R. Jones, F. Peng, et al. Improving search
relevance for implicitly temporal queries. In Proc. of
SIGIR, pp. 700-701, 2009.
[34] R. Mihalcea, and V. Nastase, Word Epoch
Disambiguation: Finding How Words Change OverTime. In Proc. of ACL, pp. 259 -263, 2012.

[35] T. Mikolov, K. Chen, G. Corrado and J. Dean.
Efficient Estimation of Word Representatio ns in
Vector Space. In ICLR Workshop, 2013.
[36] T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J.
Dean. Distributed Representation of Phrases andTheir Compositionality. In Proc. of NIPS, pp. 3111 3119, 2013.
[37] P. Norvig. Natural language corpus data. Beauti ful
Data, pp. 219-242, 2009. http://norvig.com/spell correct.html[38] H. Ohshima and K. Tanaka. High -speed Detection of
Ontological Knowledge and Bi-directional LexicoSyntactic Patterns from the Web.
Journal of
Software, 5(2): 195-205, 2010.
[39] S. Pan and Q. Yang. A survey on transfer learning.
IEEE Transactions on Knowledge and Data
Engineering, 22(10): 1345 -1359, 2010.
[40] M. Pargel, Q. D. Atkinson and A. Meade. Frequencyof word-use predicts rates of lexical evolution
throughout Indo-European history. Nature, 449 , 717720, 2007.
[41] M. W. C. Reynaert. Character confusion versus focus
word-based correction of spelling and OCR variants
in corpora. International Journal on Document
Analysis and Recognition, 14(2): 173-187, 2011.
[42] D. E. Rumelhart, G. E. Hinton, R.J. Willia ms.
Learning
internal
representations
byerror
propagation. California University, San Diego La
Jolla Inst. For Cognitive Science, 1985.
[43] E. Sandhaus. The New York Times Annotated Corpus
Overview. The New York Times Company, Research& Develop., pp. 1-22, 2008.
[44] M. Steinbach, G. Karypis, V. Kumar. A comparison of
document clustering techniques. In Proc. of KDD
workshop, 400(1): 525-526, 2000.
[45] N. Tahmasebi, G. Gossen, N. Kanhabua, H. Holzmann,
and T. Risse. NEER: An Unsupervised Method for
Named Entity Evolution Recognition, In Proc. ofColing, pp. 2553-2568, 2012.
[46] N. K. Tran, A. Ceroni, Kanhabua N, et al. Back to thepast: Supporting interpretations of forgotten stories
by time-aware re-contextualization. In Proc. of
WSDM, pp. 339-348, 2015.
[47] H. Wang, H. Huang, F. Nie, and C. Ding. Crosslanguage web page classification via dual knowledgetransfer using nonnegative matrix tri -factorization. In
Proc. of SIGIR, pp. 933-942, 2011.
[48] B. Wang, J. Tang, W. Fan, S. Chen, Z. Yang and Y.
Liu. Heterogeneous cross domain r anking in latent
space. In Proc. of CIKM’09, pp. 987 -996.
[49] Y. Zhang, A. Jatowt, S. S. Bhowmick and K. Tanaka.
Omnia Mutantur, Nihil Interit: Connecting Past withPresent by Finding Corresponding Terms across Time.
In Proc. of ACL, pp. 645-655, 2015.