EUDETECTOR: Leveraging Language
Model to Identify EU-Related NewsKoustav Rudra, L3S Research Center, Leibniz University Hannover
Koustav Rudra
Danny Tran, Leibniz University Hannover
L3SMiroslav
ResearchCenter,
Leibniz
Shaltev,
LeibnizCentreUniversity
for TropicalHannoverMarine Research (ZMT), Bremen

The 1st International Workshop on News Recommendation and Intelligence
April 14, Co-located with The Web Conference 2021

EU-News
EU-News: News that is connected/related to EU and has an influence on EU policies
On 9 June Deputy Prime Minister Krasimir Karakachanov expressed concern, saying that North Macedonia was onlyusing Bulgaria to enter the EU and NATO. [...] On 11 June the Bulgarian side of the commission flatly rejected theMacedonian proposal to honour [...] Gotse Delchev, a historical figure shared by the two countries, on 7 October. [...]

News is about film industry
Mention shooting locations

NONEU-News: Not related to EU or has no impact on EU[...] With many of the major festivals around the world — London, Ireland, Venice, Telluride, and Toronto — happening
during September and October, a well-received launch can play a crucial role in a film's quest for sales, distribution,
and awards. [...]

EU-News: Challenges in Detection• Presence of entities is not enough• Number of EU entities is necessary not a sufficient condition• Along with the entities, it is necessary to understand their interaction with each other andsurrounding context• Need to identify whether they have any connection/influence on EU

Dataset••••

Crawled news articles over a period of one month (November 2019)
Collected around 93K news articles
Annotate 5000 news (kappa = 0.77)
1662 EU news and 3338 non-EU news

• Interestingly, 67% of news contain atleast one EU entity

Model: EUDETECTORClassification Layer
Did not identify anyentities (EU) separately

Experiment
Divide train, validation, test set in 60, 20, 20 ratio
Test over 15 different folds

Baselines:• Cliff-Calvin System:• Identify EU entities using Spacy + Wikidata query• Mark it as EU event if #EU entities >= K
• Bi-LSTM model• Initiate with glove embeddings

EU-News: Performance
As K increases, precision
improves, recall drops
Language model maintains
a consistent balanceEU-News: Role of EntitiesData Type

ExampleOriginal Text

Angela Merkel is Germany chancellor. She met US presidentMask entities
Donald Trump in last year G7 summit* * is * chancellor. She met * president * * in last year G7summit.
Mask context
Angela Merkel * Germany * * * US * Donald Trump * * * **Text-COMP
Text-SUFF

Comprehensiveness = FEU(Original Text) - FEU(Text-COMP)
Sufficiency = FEU(Original Text) - FEU(Text-SUFF)

Entities are more
important than context

Comprehensiveness = 0.214
Sufficiency = 0.116

J. DeYoung, S. Jain, N. F. Rajani, E. Lehman, C. Xiong, R. Socher, B. C. Wallace. ERASER: A Benchmark to Evaluate
Rationalized NLP Models. ACL 2020

EUDETECTOR: Application
EUDETECTOR: http://eudetector.l3s.uni-hannover.deConclusion
Exploring context is important to identify EU-related news
Mere presence of entities does not ensure EU event
Application may be used as a building block in other systems

Acknowledgement: MIRROR, EU Project10

Thank You
Contact: rudra@l3s.de