Towards Summarizing
the Web of Entities

contributors:

August 15, 2012
Thomas Hofmann
Director of Engineering
Search Ads Quality
Zurich, Google Switzerland
thofmann@google.com

Enrique Alfonseca
Yasemin Altun
Massimiliano Ciaramitia
Jean-Yves Delort
Katja Filippova
Ioannis Tsochantaridis

Outline
1

The Big Picture

2

World of Entities

3

Web of Entities

4

Event Detection

5

Event Summarization

6

Timeline View

1

TheBigPicture

How to Summarize "the Web"?
● Web = content creation "in the wild".
○ Multi-authored, multi-lingual, absence of editorial
oversight, huge diversity (quality, style, intent)● How can we summarize content on the Web? And
what for?
Separate what is talked about (referred to) fromwhat is said (propositional content)

High-Level Roadmap (more than this talk)
Annotate all Web content by references to a known"set of things" = entities (across languages, styles)
Create an entity index of the Web
Cluster entity mentions around common events
or based on their propositional content
Entity-centric summarization around memorable
events and aggregation of "what is said"

This Talk !
● Not primarily about advances in academic research● Present an engineering-centric view: longterm vision plus simplified end-to-end prototype
system that operates on Web-scale● Need to cut many corners...
● Leave a lot of work half-finished or undone (UGC,unsupervised parsing, relation extraction, ....)● Identify & contribute to complex problems on theway. Shift away from standard multi-document
summarization...

EnriqueAlfonseca

2WorldofEntities

Freebase by Metaweb @Google
Curated knowledge base with (~05/2011):● 22 M nodes (concepts, named entities, locs, ...)
● 400 M connections (typed relations)● Community-driven effort; larger internal baseCreative common
attribution licenseMQL
MQL: query language toquery entity graph

Schema part of graphrepresentation

cf. Google I/O talk

Semantic Web ?!
Entity-centric knowledge base is only one side ofthe effort to establish a "semantic Web"
Manual annotation (by authors, Web masters, publishers; seeschema.org) requires large collaborative effort (slow,conflicting interests, lack of comprehensiveness,...)

Enable automatic understanding of text
by identifying mentions of entities & concepts.
Grounding of mentions/references in curated'ontology' of things (vs. open domain extraction)

Entities in Web Search
query 2 entity
mapping3

WebofEntities

Entity Grounding
Step 1
Identify entity
candidates(names)Freebase mids

Entity Grounding
Step 2
Load known
entity relations
and create
entity graph for
document

Freebase relations

Entity Grounding
Step 3

Joint inference
over entity
graph todisambiguatereferences

Inferred entities

Reference Disambiguation Task
1. Observe a set of mentions
2. Produce list of scored candidate
entities using a 'local' model3. Rerank candidates using topic model
4. Combine the scores, and search thespace for the best set of entities
CoNLL Test

Hoffart et al
2011

LogisReg

Reranked
LogisReg

Precision

81.82

87.67

89.44

~70.00

64.83

66.41

Recall

F-1

75.45

74.54

76.22

Yasemin Altun
Massi Ciaramitia

Generative prior model
for sets of entities
CoNLL data annotated by
Weikum group @MPI

4NewsEvent
Detection

Towards Entity Timelines
Events influence what the Web says about entities.
Temporality matters, so use it as an organizing
principle!
Given an entity and a time period, provide asummary of the most memorable events
involving this entity.
Current "strawman" system:● Identify relevant news stories/clusters about the entity.
● Filter and rank news clusters in order of importance.
● Concisely summarize each news cluster● Visualize summaries in temporal order

Web Search is not enough ...

Have to use
left navigation
restrict to see
useful recentresults

Web News is not enough ...

Confusionaroundentities inGoogle News

Web News is not enough ...
Even with alittle help, no
good results,
since only
today's news.

Entity-Centric View on NewsEvent Clustering &Cluster Selection● Restrict to Google news corpus + entities● Piggyback on clustering of stories (hack)● Given an entity, score news clusters● [ A lot more work on event detection required... ]1. Event prominence. The size of the news cluster (in how
many articles the news was published?).
2. Entity topicality. The average topicality of the entity inthe articles in the cluster (derived from mentiones)
3. Entity prominence. The percentage of titles in the cluster
mentioning the entity.

5EventSummarizationNews Cluster Summarization
Given a set of news articles, generate an extractive
summary by:● "Extractive" multi-document summarization, plus:● Use entity as focus (vs. generic summary)● Headline selection for timeline view
● Sentence compression
State-of-the-art results
on academic corpora(TAC 2011)

Multi-document Summarization
TopicSum

Haghighi and Vanderwende, 2009
Jean-Yves Delort
Enrique Alfonseca● Augmented for update summarization● Integrate paraphrase scoring from MT● Entity-specific sentence weighting● Special attention to titles

Ioannis Tsochantaridis

Sentence Compression
Katja Filippova

Beyond extractive summarization: compression

Learn from title / firstsentence pairs.

Sentence Compression

6Time LineView

Entity Timeline Prototype

Some screenshots ....

7

WherefromHere?

Future Directions● More serious event detection & extraction. Tying
events to entities and their relations.
● Include UGC, e.g. by mapping/linking to newstories● Improve summarizer by better understanding of
propositional content● Developing a full-blown time-line view

THANK YOU!