DEIM Forum 2019 P2-123

Present Relatedness Estimation of Archival Documents using External
Knowledge Base
Mari SATO† , Adam JATOWT† , and Masatoshi YOSHIKAWA†† Graduate School of Informatics, Kyoto University Yoshida-honmachi, Sakyo-ku, Kyoto, 606-8501 Japan
E-mail: †msato@db.soc.i.kyoto-u.ac.jp, ††adam@dl.kuis.kyoto-u.ac.jp, †††yoshikawa@i.kyoto-u.ac.jp
Abstract Our society nowadays generates massive amounts of digital data which is being archived and made available for long time. In addition, many born analog documents are being digitized and included in online archives.
Although document archives are often used by professionals that often precisely know what they wish to search,when ordinary users search in document archives they often would like to find content which has some relatedness tothe present, for example, content related to popular present events or entities. However, search engines for document
archives do not consider such aspects and retrieve documents in the same way as search engines in non-temporalcollections. In this paper, we propose the notion of ”present relatedness” of archival documents - the concept which
can be included in retrieval mechanisms for document archives. In particular, we put focus on named entities indocuments as their representation and we estimate their relatedness to the present using knowledge bases.
Key words archival search, information retrieval, data mining, text mining

1 Introduction
Our society nowadays generates massive amounts of data
which is being archived and made available for long time (e.g.
（注 1）Internet Archive

might be more interested in past documents that are related
to this issue, rather than ones which have weak connection
and relation to the present.
In this paper, we propose the notion of ”present related-collecting web snapshots from Alexa).

ness” of archival documents, which indicates their related-

In addition, more and more analog documents are being dig-ness to the current times. We implement this idea in, particu-

itized and included in online document archives (e.g. The

lar considering document correspondence to popular present

Times Digital Archive（注 2） and ACL Anthology Reference

events or entities. We put focus on named entities in docu-（注 3）

Corpus). The continuous development of digital docu-

ments as their representation and we estimate their related-

ment archives allows users to learn about historical events

ness to the present using knowledge bases as well as contents

by searching and browsing old documents as well as read-similarity and temporal expressions in the documents. We

ing recent ones about those events. While query suggestion

then train the learning to rank model with documents col-and text indexing methods have been already studied in thelected from diﬀerent periods and then evaluate whether thecontext of archival search [1] [2] [3]tran15, relatively little hasmodel can eﬀectively rank documents from a certain time

been done about ranking for archival search.

period according to theirpresent relatedness .

Professional users such as historians or linguists typically

We show two examples of our goal. Figure 1 shows arti-

know what they wish to find when searching in or interact-

cles from late 1980s. Both tell about economic figures, yet

ing with archives and have good skills in locating relevant

the top article contains a mention of ”Donald Trump.” Hav-

documents. On the other hand, general users may have lessing these two documents, we can say that the top document

defined search intents and often would like to find content re-is more related to the present since Donald Trump is cur-lated to the present. For example, events, figures, or places

rent US president. Figure 2 shows articles from 2000s. Boththat are the background for present events. Such content

are related to ”France,” yet the bottom article tells about

can be not only more relevant and attractive to the users

protests. Having these two documents, we can say that thebut has also a good chance to be of higher utility. For exam-bottom document is more related to the present since Frenchple, a journalist writing an article on a certain present issue

protests are more popular topic than French cooking. Our（注 1）：https://archive.org/web/（注 2）：https://www.gale.com/intl/c/the-times-digital-archive（注 3）：http://acl-arc.comp.nus.edu.sg/

proposed approach aims to rank documents related to thepresent higher as output.
This paper is organized as follows. In section 2 we discuss

When it comes to merchandising, Donald J. Trump andMody Dioum could not be much farther apart. But they

documents. It is reported that a significant number of queries
on the Web more or less include temporal intent [10] [11]. By

agree on one thing: The holiday season was hard on Fifth

exploiting temporality in archived documents, it can lead to

Avenue street peddlers. And as the avenue stepped back tobetter understanding of old documents and better perfor-

normal yesterday, it appeared that the city’s recent crack-

mance of search engines. There have been research works indown on merchandise peddlers was still having an eﬀect.

the context of archival search recently. Zhang et al. proposed
methods to suggest corresponding entities across time / ge-

Hale Stores, has moved to coordinate its major businesses
by naming Ira Neimark, chairman and chief executive of
its Bergdorf Goodman subsidiary, to the additional post ofvice president of merchandise development for the NeimanMarcus Group.

ographical areas as eﬀective queries in archival search [1] [2].
Holzmann proposed an indexing method focusing on anchor
texts in a knowledge base [3]. Yet, still little has been done
about ranking archived documents with respect to user s
search intent. Often, this intent is imprecise, especially, when

Figure 1 Excerpt news articles from 1987 (top) [17] and 1988
(bottom)（注 5）.

ordinary users are considered. Yet one can assume that inmany cases documents related to present social issues are theones that the users would wish to find. We then introduce

Jean-Franois Revel, a prolific philosopher, writer and jour-the notion of present relatedness of archival documents andnalist who summoned the classical polemical weapons ofdevelop initial methods towards its eﬀective estimation. The

Voltaire and Montaigne, including humor, irony and surprise, to illuminate subjects from French cuisine to French
anti-Americanism, died on Saturday in Paris.

application of such method can be in the form of a integral
component within archival search engines which will let users
decide the level of

But Sarkozy, still reeling from massive transit strikes andpresent relatedness

they wish to havewhen searching for past documents.

student protests this month throughout France, is unlikely

As for the concept of present relatedness, the closest work

to use the current unrest as a vehicle to turn introspective

to ours is probably the one on estimating the importance ofor vent his rage too loudly at those he once called ”thugs.”

historical figures. Jatowt et al. found out decisive features

Figure 2 Excerpt news articles from 2006 (top) [19] and 2007(bottom) [20].

that determine the importance of historical figures using features extracted from link structure, visit logs and article content on Wikipedia [8]. While their work tried to predict therelated works. In section 3 we define what is present relatedness and we outline our research problem. In section 4 we
describe the features and two-stage learning of our method.
Then we describe our experiments in section 5 and provide

importance score for given historical figures, our work has
the objective of estimating how documents are related tothe present given query.

3 Problem Statement

an evaluation of the approach on test data in section 6. We
conclude the paper and mention future improvements in section 7.

2 Related Works
A basic search problem in information retrieval (IR) is toDef. 1. Present relatedness is a relatedness of past documents to the present.
Then we define our problem as follows: Given user query
q and document d published in time Tpast (Tpast ≪ Tnow ),the task is to estimate present relatedness score of d:estimate standard relevance score of d, given user query q

P R(d | q, Tpast , Tnow ). Tnow is an arbitrary time period.

and document d [5]. The purpose of our work is to extend

We set Tnow as [2014, 2018].

this problem to archival search.
User’s information needs for web search have been classified into three groups; informational, navigational, and

4 Proposed Method
4 1 Overview

transactional. Broader analyzed search query logs for theWe first represent documents by a range of features thatcurrent Web and found out that around a half of the queries

have high chance to capture their relatedness to the present.

are informational in conventional Web search [6]. On the

We will list the features along with the hypotheses that guideother hand, Costa and Silva analyzed search query logs fortheir choice below. Having document representation we thenan Web archive search engine and found out that naviga-

utilize learning to rank models for ranking documents. In

tional need was the most common [7].

particular, we train the model with documents from two dif-

One central element of navigational need is temporality offerent periods whose gap is about 10-15 years and then wepredict and evaluate whether the model can rank documents

where Tpresent is an arbitrary number of present year. We

from the middle of two ages according to their

set Tpresent as2014.

present

relatedness . In other words, we assume that very old docu-

4 2. 3 Popularity of entities

ments are on average less related to recent documents. While

We considered that when a named entity is not popular

it may not always be the case, this assumption holds for aenough, a reader has a lower probability to find it relevant

large number of documents. Thanks to this we do not needand by this find the document related to the present. Basedto manually annotate the data for preparing a training set

on this hypothesis, we exploited Wikipedia pageviews us-which would be quite costly. First we explain which features

ing Wikimedia REST API（注 8） as a convenient measure of

we propose to use to learn the relatedness to the present.

popularity of entities. For each entity ei we retrieved the

4 2 Entity-related Features

pageview counts pageview(ei ) of its article. We compute

4 2. 1 Importance of each entity in the document

entity popularity(ei ) by normalizing the pageviews with a

Entities which are just passing mentions in documents arewidth of six standard deviations of the pageviews s since itprobably of little significance to the task of present related-covers the 99.7% of the distribution, eliminating outliers as

ness estimation. We then need to select important entities

well (See Equation 2).

in a document to define later features. We used three simple approaches to find the most important entities in each

popularity(ei ) =document.
（ 1 ） We counted the frequency of each entity and normal-

ized with the frequency of the whole entities in a document.
（ 2 ） We figured important keywords which play the role

pageview(ei )6s(2)

4 2. 4 Connectedness to present knowledge graph
We considered that when a named entity has fewer connections in the present-oriented knowledge graph, it is lessof describing the document. We computed TextRank score

related to the present. Based on this hypothesis, we con-[12] for each entity using the implementation by [13].

structed（ 3 ） We figured the oﬀset of earliest entities appeared inpresent

knowledge graph G(V, E), where V isthe set of nodes representing DBpedia instances of not only
selected entities in the documents but also entities whicha document.
We then use the five most important entities in each doc-

have internal links with them, and E is the set of edges rep-ument to compute additional features.

resenting links between V . We constructed G from DBpedia

4 2. 2 Activity period of entities

Page Links dataset（注 9）. We compute the connectedness to

We considered that when a document contains a named

the present using the biased random walk theory. Note that

entity which is no longer active or no longer strongly re-

R is a vector containing node scores, M is an aperiodic tran-

membered, a reader has a lower probability to consider thesition matrix, α is a decay factor, and d is the static score

document to be related to users. Based on this hypothe-

distribution vector summing up to one.

sis, we exploited activity period of each entity. For each
entity we want to retrieve the associated years and timeR = (1 − α)M × R + αd
where

We extracted the DBpedia Linked Data representation with0d=
1/|d|

temporal information with properties identifying time in-

4 3 Content-related Features

tervals (e.g. birthDate/deathDate for figures, and found-

4 3. 1 Cosine Similarity

intervals.

First we collected properties whose data type
（注 6）

is date (xsd:date, xsd:dateTime, xsd:gYear etc.

). Then(3)(Tend < Tpresent )(otherwise)

For querying

We measured cosine similarity between each past docu-

properties and date values, we used the DBpedia SPARQL

ment and the representative collection of present documents.

endpoint（注 7）. For each document d, we define N Ed = {e1 ,We used tf-idf, word2vec, and doc2vec as vectorization meth-e2 , …, en } as the set of named entities in the document andods. We explain the details about the collection of present

AP ei = {Tstart , Tend } as the activity period extracted fromdocuments in the next section.

ingYear/dissolutionYear for organizations).

ei. The relatedness of an entity to the present rei is defined

4 3. 2 Cosine similarity (Transformed version)

whether the activity period has overlap with present or not:
0 (Tend < Tpresent )r ei =(1)1 (otherwise)

We also measured cosine similarity between each transformed past document and the large collection of present
documents. As for the transformation method, we used theone proposed in [1].

（注 6）：https://www.w3.org/TR/xmlschema11-2/（注 8）：https://wikimedia.org/api/rest v1/（注 7）：https://dbpedia.org/sparql（注 9）：https://wiki.dbpedia.org/downloads-2016-10#datasets

Figure 6 Learning to rank for prediction.

Figure 3 Cosine similarity between present documents.

5 Experiments
We sampled 50k articles from the New York Times Annotated Corpus, which were published from 1987 to 1991 and2003 to 2007 respectively as the collection of old documents.
We used them in the learning to rank process. We selected
10k articles from the same corpus, which were published from1996 to 1997 as the collection of test documents used in the
Figure 4 Cosine similarity between present documents and transformed past documents.

prediction and evaluation process. To extract entity-related
features, we first extracted named entities from each document using TextRazor API（注 11）. The API returns Wikipedia

4 4 Temporal Expressions

URL with confidence scores given input text. We kept en-

We considered that a when a named entity appeared of-tities with higher confidence score than 0.2. Among kept

ten together with mentions of past years yet it appears lessentities, we assigned importance scores and computed fea-with mentions of recent years, it is less related to the present.

tures for five most important entities. We summed up theBased on this hypothesis, we extracted years mentions fromfeature values for a document.

（注 10）

each document using HeidelTime temporal tagger. We

To compute content-related features, we used crawled ar-then figured oldest, median, and ealiest year and calcurated

ticles of New York Times from 2014 to 2018 as the collection

their diﬀerence between the year Tpresent [14].

of present documents.

4 5 Learning to Rank
Using features described in the previous subsections, we

As for ranking algorithm, we employed LambdaRank [15]in both processes using LightGBM library（注 12）.

We settrained the model to learn the present relatedness score ofnDCG as metrics. We included the whole documents fromdocuments. We first gave low score to older documents andboth old and new collections as a ranked list for a query sincehigh score to newer documents based on assumption that

we did not assume query this time. Next, using the trainednewer documents are more related to the present. We use

model, we rank documents from the test collection according

pairwise approach to train the model to tell which document

to their ”present relatedness”. To make evaluation suﬃcient,

is more related to the present (See Figure 4). We also use

we split the collection into genres of news article. We usedlistwise approach to rank documents in order of publication

142 genres which have more than ten articles. We treatedyear within collections.

a genre as one query and ranked documents with the samegenre.

6 Evaluation
For evaluation, we randomly selected six genres. For eachgenre, we got 1000 documents returned by Solr. Using the
Figure 5 Learning to rank for training.

trained model, we re-ranked and pooled top 5 documents.
Four annotators gave 1-4 evaluation whether the document

Using the trained model, we rank documents whose publi-

is present-related or not (1 and 2 are regarded as not relevant

cation dates are not overlapped with documents used in theand 3 and 4 are regarded as relevant). Then we evaluated thetraining process. We randomly selected the ranked lists forannotated result by Precision@1, 3, 5 and MAP(See Table

evaluation (See Figure 5).
（注 11）：https://www.textrazor.com/（注 12）：https://github.com/Microsoft/LightGBM/tree/master/examples/（注 10）：https://github.com/HeidelTime/heideltime

lambdarank1). As for baseline, we used randomized ranking and cosine
similarity. Pairwise approach performed best in Precision@1,
3, and MAP.
Table 1 Evaluation of annotated result. Random and Cosine
methods are baselines. Pairwise and Listwise methods
are proposed approaches.
Random Cosine Pairwise Listwise
Precision@1

0.50

0.50

0.60

0.50

Precision@3

0.39

0.53

0.64

0.53

Precision@5

0.42

0.54

0.52

0.46

MAP

0.61

0.63

0.76

0.64

7 Conclusion and Future WorkIn this paper, we proposed a method to estimate ”present
relatedness” of past documents stored in long-term archives
using external knowledge base. We put focus on named entities in the documents as their representation and we estimated their relatedness to the present using knowledge bases
as well as contents similarity and temporal expressions. As
a result, our proposed approach performed better than baselines.
For our future work, we would like to try to indicate whyhighly-ranked documents are related to the present and thus
by this provide explanation to users to help them better understand the returned results. At the same time, we would
like to degrade documents with named entities which are not
popular enough to be Wikipedia article. These unfamiliar
entities are overlooked in our approach.

References[1] Y. Zhang, A. Jatowt, and K. Tanaka,
Towards Understanding Word Embeddings: Automatically Explaining
Similarity of Terms , Proc. of IEEE International Conference on Big Data, pp. 823-832, 2016.
[2] Y. Zhang, A. Jatowt, and K. Tanaka, Is Tofu the Cheeseof Asia?: Searching for Corresponding Objects across Geographical Areas , Proc. of WWW, 2017.
[3] H. Holzmann, Nejdl, and A. Anand,
Exploring Web
Achives through Temporal Anchor Texts , Proc. ACM on
Web Science Conference, pp.289-298, 2017.
[4] N. K. Tran, A. Ceroni, N. Kanhabua, C. Niederée, ”Backto the Past: Supporting Interpretations of Forgotten Stories
by Time-aware Re-contextualization”, Proc. of ACM International Conference on Web Search and Data Mining, pp.
339-348, 2015.
[5] C. D. Manning, P. Raghavan, and H. Schtze, Introduction to Information Retrieval , Cambridge University Press,
New York, NY, USA, 2008.
[6] A. Broder, A Taxonomy of Web Search , Proc. of ACM
SIGIR forum, pp. 310, 2002.
[7] M. Costa and M. Silva, A Understanding the Information
Needs of Web Archive Users , Proc. of International Web
Archiving Workshop, 2010.
[8] A. Jatowt, D. Kawai and K. Tanaka,Predicting Importance of Historical Persons using Wikipedia , Proc. of

CIKM, pp. 1909-1912, 2016.
[9] C. Morbidoni and A. Cucchiarelli, A Bag-of-Entities Approach to Document Focus Time Estimation , Proc. of KDWEB, 2017.
[10] R. Jones, and F. Diaz, ”Temporal Profiles of Queries”, Proc.
of ACM Transactions on Information Systems, 2007.
[11] D. Metzler, R. Jones, F. Peng, and R. Zhang, ”Improving
Search Relevance for Implicitly Temporal Queries”, Proc.
of SIGIR, pp. 700-701, 2009.
[12] R. Mihalcea, P. Tarau, ”Textrank: Bringing Order into
Texts”, Proc. of EMNLP, pp. 404411, 2004.
[13] F. Barrios, F. López, L. Argerich, R. Wachenchauzer, ”Variations of the Similarity Function of TextRank for Automated Summarization”, Anales de las 44JAIIO. Jornadas
Argentinas de Informtica, Argentine Symposium on Artificial Intelligence, 2015.
[14] B. Fetahu, K. Markert, W. Nejdl, A. Anand, ”Finding News
Citations for Wikipedia”, Proc. of ACM International Conference on Information and Knowledge Management, pp.
337-346, 2016.
[15] CJC. Burges, ”From ranknet to lambdarank to lambdamart: An overview,” Learning pp. 23-581, 2010.
[16] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, T.
Y. Liu, ”LightGBM: A Highly Eﬃcient Gradient Boosting
Decision Tree”, Advances in Neural Information Processing
Systems, pp. 3146-3154, 2017.
[17] ”ANTI-PEDDLER DRIVE PLEASES FIFTH AVE. MERCHANTS”, The Nwe York Times, https://www.nytimes.
com/1987/01/06/nyregion/anti-peddler-drive-pleases-fifth-ave-merchants.html[18] ”CREDIT MARKETS; Neiman Shifts Key Executives”,
The New York Times, https://www.nytimes.com/1988/04/12/business/credit-markets-neiman-shifts-key-executives.html[19] ”J.-F. Revel, French Philosopher, Is Dead at 82,” The New
York Times, https://www.nytimes.com/2006/05/02/world/
europe/02revel.html, 2006.
[20] ”Paris suburb riots called ’a lot worse’ than in 2005,” The
New York Times, https://www.nytimes.com/2007/11/27/
world/europe/27iht-riots.4.8500200.html, 2006.