SlowGuess/NAACL_2022_OCR huggingface dataset:
indices are used to represent each node, with mention level information ignored. Although RESIN is a state-of-the-art IE system, there still remains some errors in event temporal links. Therefore, we do human curation to remove obviously erroneous links. Below are some event-event link examples that we delete from the instance graphs: (DIE, INJURE), (ARRESTJAIDLETAIN, ATTACK), (ENDPOSITION, STARTPOSITION), (DIE, DIE), (DEFEAT, EXCHANGEBUYSELL), (SENTENCE, DIE), (ENDPOSITION, SENTENCE), (THREATENCOERCE, RELEASEPAROLE).
C Scientific Artifacts
C.1 RESIN Information Extraction System
Data License and Usage We obtain the code from the open-source information extraction system RESIN (Wen et al., 2021). We run the code at https://github.com/RESIN-KAIROS/RESIN-pipeline-public. The system is released for research purpose and is licensed under the GNU General Public License v3 or later. The system covers the general news domain and supports three languages, i.e., English, Spanish and Russian. It does not contain offensive content. Discussions about IE Quality The performance of each component is shown in Table 3 (Wen et al., 2021). Although IE graphs are noisy, schema induction can still benefit from it. It is because that the schema induction task aims to find the recurring patterns, which will still be preserved even in the noisy data.
C.2 IED Schema Induction Corpus
Data License and Usage We obtain the dataset from the state-of-the-art graph schema induction literature (Li et al., 2021). The dataset is released for research purpose and is licensed under the GNU General Public License v3 or later. The system covers the general news domain and is an English corpus.
Data Collection We utilize the news articles in the state-of-the-art graph schema induction literature (Li et al., 2021) in https://github.com/limanling/temporal-graph-schema.
We collect associated news articles concerning each complex event type, such as Car-bombing IED, using Wikipedia as a source. As a first step, we search candidate Wikipedia categories based on the name of the complex event type, and then dig deeper into each page to identify complex
Component	Benchmark	Metric	Score
Mention Extraction	En	Trigger	ACE+ERE	F1	64.1
Argument	ACE+ERE	F1	49.7
Relation	ACE+ERE	F1	49.5
Es	Trigger	ACE+ERE	F1	63.4
Argument	ACE+ERE	F1	46.0
Relation	ACE+ERE	F1	46.6
Document-level Argument Extraction	ACE	F1	66.7
RAMS	F1	48.6
Coreference Resolution	En	Entity	OntoNotes	CoNLL	92.4
Event	ACE	CoNLL	84.8
Es	Entity	SemEval 2010	CoNLL	67.6
Event	ERE-ES	CoNLL	81.0
Temporal Ordering	RoBERTa	MATRES	F1	78.8
MATRES-b	Acc.	89.6
Visual Event Extraction	Video M2E2	Acc.	70.0
Table 3: Performance (%) of each component. MATRES-b refers to MATRES binary classification that only considers BEFORE and AFTER relations.
events that belong to that category. Afterwards, we collect the reference news articles for each complex event, use these articles as the cluster of documents relating to the complex event, and perform IE to construct the instance graph. Using this Wikipedia-based data collection approach, we have been able to cover a wide range of scenarios, including most complex events that occur in human society, such as Disease outbreak and Disaster. Therefore, our schema induction method does not depend on manual work and is not limited to a specific complex event scenario.