Semi-connected Joint Entity Recognition and Relation Extraction of Contextual Entities in Family History Records
Authors: Daniel Segrera
Venue: Submitted to ICLR 2023 (2023)
Keywords: Information Extraction
NER
Relation Detection

Abstract: Entity extraction is an important step in document understanding. Higher accuracy entity extraction on fine-grained entities can be achieved by combining the utility of Named Entity Recognition (NER) and Relation Extraction (RE) models. In this paper, a semi-connected joint model is proposed that implements NER and Relation extraction. This joint model utilizes relations between entities to infer context-dependent fine-grain named entities in text corpora. The RE module is prevented from conveying information to the NER module which reduces the error accumulation during training. That improves on the fine-grained NER F1-score of existing state-of-the-art from .4753 to .8563 on our data. This provides the potential for further applications in historical document processing. These applications will enable automated searching of historical documents, such as those used in economics research and family history. 

Semi-connected Joint Entity Recognition and Relation Extraction of Contextual Entities in Family History Records
Anonymous Submission
Anonymous Institute
Abstract
connections with their ancestors. To facilitate their work they often automate the extraction of infor- mation en masse from historical documents. One part of this process is called entity extraction. For example, in family history, digital text is searched for particular entities. These entities include names of parents, names of children, birth dates, marriage dates, etc.
Entity extraction is an important step in doc- ument understanding. Higher accuracy en- tity extraction on fine-grained entities can be achieved by combining the utility of Named Entity Recognition (NER) and Relation Ex- traction (RE) models. In this paper, semi- connected joint model proposed that imple- ments NER and Relation extraction. This joint model utilizes relations between entities to in- fer context-dependee
These entities are extracted and compared each other to build family tree charts in pro- cess sometimes called indexing. To precisely in- dex historical documents it is not enough to have course-grained labels, such as name or date, but fine-graina labels, such as spouse name and mar- riage date, are necessary. Furthermore, these fine- grained labels often rely on the document's internal context.
fine -raig named entities in text corpora. The RE module is prevented from conveying information to the NER mod- ule which reduces the error accumulation dur- ing training. That improves on the fine- tne-grained NER Fi-score existing state-of-the-at from .4753 8563 our data. This provides the potential for further applications in historical document processing. These applications will enable automated searching of historical doc- uments, such as those used in economics re- search and family history.
Unfortunately, these entities are often fine- grained and contextual to relationships between entities within a record. These organizations do not have models that can accurately extract such context-dependent fine- fine-grained entities, so they in- stead extract important words coarsc sarseegrriied entities (such person, place, date) and manu- ally label them as fine-grained classifications, This problem is even more pronounced for records writ- ten in languages such as French with less labeled data.
1 Introduction
Named Entity Recognition (NER) also called en- tity extraction or entity identification is a natural language processing (NLP) technique that automat- ically identifies named entities (names, places or dates for example) in a text and classifies them into predefined categories. It is often sufficient to identify an entity as course grained entity like a name if the application is attempting to identify employees working for a company from a para- graph of text. Although this course grained entity recognition is sufficient for many applications, fine grained classification is necessary for family his- tory applications where it is necessary to know that relationship between different entities in addition deriving their classification.
2 Related Work
In 2018, Belkoulis revolutionized the field named entity recognition (NER) by conceptual- izing joint entity and relationship extraction as multi-hean selection problem (Bekoulis et al., 201 8). His paper demonstrated that relation extrac- tion
helpful tool to improve entity extrac- tion accuracy. His original model used bidirec- tional LSTM for sentence encoding. That same year, papers saw improved results using ELMo (Pe- ters et al., 2018) for sentence encoding (Sanh et al., 2019). After the introduction of BERT (Devlin
There are many companies and research organi- zations in the fields of family history and histori- cal document understanding. Family history work helps people learn about their heritage and form

---

has
Heaneil oe faat you
on
Aiaanddy To
ontant fembiih
etre no sommune jourd
Jean Baptiste Calin Sabetier (Father)
Mare Jnne Gue
damisile quartier Lapline nui a houres du motin de lu deciorant et de Marte Anns Gus not son upouse sons profassion agin du trunts da ans domiciles en oette commune suquel déclare voulsir donner her prénoms du hiltnnes
Juternee
Transcription on records into Family trees. Example of a handwritten birth record, written in French, which after transeriptionnusing handwriting recognition model) passed through our joint entity relation model in order identify the entities and their respective relations. Note that the transcription is not perfect it contains spelling mistakes, grammatical inconsistencies, etc that make it harder for the model to find correct entities and relations
Figure I:
et al., 2019) in 2019, several papers saw even better results when using BERT for sentence encoding instead of ELMo
with fine-grained or contextual entities. Most of these models can not find all the relations in a multi- sentence corpus because they can not map rela- tionship between two entities in separate sentences. However, cross-sentence relation extraction is nec- essary to find the nuanced relationships in family history records.
Since then, papers in the field have improved entity recognition accuracy by creating joint mod- els that use relation extraction. The ways these joint models are implemented vary. One approach is to have the joint model ask questions about the data (Li et al., 20ho et al., 2020). Another approach uses distantly supervised data augmen- tation to reduce the impact of negative labels the joint model (Xie al., 2021). Jue Wang's s pa- per sees improved relation extraction by filling an entity-relation table (Wang and Lu, 2020). Hier- archical relationship extraction particularly ef- fective at detecting hierarchical relationships (Han et al., 2018) (0118)(Taaaaa (Takanobu et al., 2Zhang et al.. 2021).
Problem Description
Existing research insufficiently performs the task of fine-grained entity extraction on contextual entities. Traditional methods may be able to identify names and dates, but are unable to identify Mother's Name vs Sisters Name the dates corresponding dif- ferent events in record. Much of the difficulty comes from the space between entities in a para- graph.
Combining entity recognition with relation ex- traction significantly improves the accuracy of con- textual fine fine-grained entity extraction for automatic indexing systems used by researchers performing automated historical document analysis. This con- tributes a novel solution to significantly reduce the manual annotation effort when indexing records without sacrificing recognition accuracy.
The most successful approaches for entity recog- nition insert markers into the sentences. This is done in models such as PURE (Zhong and Chen, 2021). These markers reduce the need for embed- dings which reduces memory needed and improves inference speed. Papers that use this approach are in the top 3 micro-Fl scores for both entity extrac- tion and relation extraction accuracy for the bench- mark datasets: CoNLL 2003, ACI20044, ACE2005, and SciERC (Ye et al., 2021).
The indexing of family history records requires NER model that identifies contextual fine-grained entities. However, is diffieult to train NER mod- els to do that on their own. Relation extraction can help find the context in the corpus by identifying relations between entities. A joint model is needed
The models from existing research perform well on benchmark datasets. However, they fail to per- form on more complicated datasets, such as ones

---

Relation extraction model both use cross entropy for their loss function in order improve training accuracy.
BERT
For each record, the pipeline labels entities with tokens (as seen in Example 1) and then pairs combinations of tokens as subject and object (as seen in Example 2). It then feeds them to the relation extraction module. For each pair, the module predicts the relation between the two entities and generates triplet tuple that represents the predicted relation. From those tokens the module classifies the entities and generates tuples that represent the entity classifications.. The tuples from the relation extraction module and entity extraction module can be used to infer the fine- fine-grained labels of the entities.
E1
E2
RE Classifier
NER Classifier
Figure The Joint Entity and Relation Extraction model. The tokens encoded by BERT, Entity (E1) and Entity (E22, are passed simultaneously through the NER and RE module end-to-end model preserve cross-sentence context and prevent the com- pounding error.
Example 1:
(ENTSTART=Name) Paul (ENTEND=Namee) and (ENTSTART=Name) James Thompson (ENTEND=Name) ran to the store to get milk for their parents in (ENTSTART=Month) March [ENTEND=Month]
that can both extract entities and the relationships between those entities. This kind of model more ac- curately extracts context-based fine- tne-grained entities in family history records.
Additionally, most real-world transcriptions from handwritten documents are full of errors. Common errors include incomplete transcriptions, transcriptions in the wrong language, incorrect la- beling, grammar mistakes, and spelling errors. The model built must be able to overcome these diffi- culties.
Example 2:
(SUBISTART=Name) Paul (SUBJEND=Name) and [OBISTART=Name] James Thompson [OBJEND=Name) ran to the store to get milk for their parents in March.
3.1.1 Data Augmentation
3.1 Methods
relatively hard augment textual data to train neural model-especially when there is a limited amount of sample data. Given that the data used was French records, many records were fairly simi- lar if not for grammatical differences. These differ- ences lead to the thought and creation of a context- free grammar where rules are mapped to different phrasings found in the sample records. The com- bination of many different rules, many being cursive rules, and comprehensive dictionaries for simple terms (like names) allows for high variation in creating a new record. Since each rule created based on real sample data, the generated records are coherent with small seemingly insignificant errors. Our approach for data augmentation was to cre- a grammar and templates that implemented the grammar. The grammar consists of both simple and complex rules. Each simple rule is linked dictionary of strings and, when parsed, chooses a random element from its dictionary. "SelfGiven- Name", "Surname", and "Profession" are all ex-
A joint entity-relation extraction model can be used to extract fine grained entities from transcriptions resulting from automated handwriting transcription. A language model is needed to encode tokens and obtain contextualized representations for each in- put token in the training dataset. The best language model for this is BERT. Most joint entity and rela- tion extraction models suffer from cascading error propagation. In this domain, emrors in course grain entity extraction can increase errors in the relation extraction phase of the algorithm. These errors can accumulate more rapidly if the two sides of the joint model are too dependent on each other. To prevent this, the BERT encoded tokens are si- multancously passed through the entity extraction module and the relation extraction module an end-to-end model (as illustrated in Figure 1). This approach preserves cross-sentence context and re- duces the opportunity for errors to compound in the pipeline. Additionally, the NER model and

---

Triai PERSSON
DIGI
SIEE
PROFESSION
son denicile
FIRST
TDe CeRP
YESTERDAY
FIRST NAVE DE
FIRST NAME
PROFESSION
PERSON
Gatt masser
TODWY WAl
PHRASE
NE
quader
NAME
conmune
YESTERDAY VAR
TODAY WAR
KEY
'dnn commune ddr
COMMUNNE PHRASE
COMMILEE
Dictonary
Not enough space for current rule's full grammar
VARI
TODAY WAR
String literal
SHORT
Direction to follow grammar
Points the current rule's grammar definiion
COMMUNE
DICT
MANY AULES
MANY RILES
Figure 3: Textual data with similarities be augmented through the of context-free grammar (CFG) and context templates. Creating rule trees (such as in the image above) for the grammar can be done by grouping together different phrases/terms
are encompassed by similar ones. The generated rules, along with the similar phrases/terms left untouched, can be combined to create the context templates. Once templates and rules exist, new textual data can be created. Adding rules increases data variation exponentially.
amples of simple rules. Complex rules, the other hand, are designed to vary sentence struc- ture. They eliminate the need create multiple templates which all have a similar record structure. Unlike simple rules, complex rules are not linked solely to a dictionary, but are recursively connected to other rules. Since one rule is linked to many other rules, diverse amount of data can be gener- ated. An example of one of these complex rules is the "Ne" rule. This rule could appear in the tem- plate in phrase like (English translation given) h...he presented child of the feminine sex born "NE" of the person speaking and... " Thhe flowchart in Figure 3 illustrates the complexity of this rule.
birth records given to Around 300 records were found, but only half of those records legible. Others contained various errors like non-completion, too many typos, and repetitive, randomly-place phrases which led to unusable records. The data augmenter should, theory, be able to generate the content of every legible birth record (excluding typos and small grammatical er- rors).
In some cases certain phrases need to cached for later use. For example, if in a marriage record the husband's surname is randomly attributed to "Dupont" then it is natural to expect his father's surname to be "Dupont" as well. A list of cache- able rules must be predefined for this to happen. Each time rule is compiled, it first needs check if it is cache-able rule. If so, checks if has already been eached and then uses what cached. If not, or the rule was not yet cached, will compile the rule and cache the result if it should be cached. Each time a new record is started, the cache is cleared.
Diversity within template can be obtained through the of these simple and complex rules, but if record structure considerably different then new template needs to be made. There are currently three birth record templates. The first contains the structure a father and mother present, the second template varies in that midwife is present and is the main speaker in the record, and the third template has no father present but includes the mother's parents.
record which has both
The generation of record from a template is relatively simple process. First, parse the con- tents the template to understand exactly what to change and what stays the same. This parsing
These templates and rules were generated through the careful consideration of the textual

---

process utilizes predefined set of tags. Each tag has a predefined order of operations that must be performed in order to successfully comply with the language structure and meaning. Each tag is ad- dressed one at a time and a string is eventually gen- erated through its associated rulc. Also, rules can be recursive, referencing other rules and tags. Ele- ments of the rules that are string literals stop here, while other rules linked dictionaries randomly sampled their associated dictionary. Once all the elements of a rule have been augmented/ransllttd, they are bubbled back up to the template where they are injected as a string into the output.
3.1.3 Relation Extraction
Initially, Relationship Extraction model was heavily inspired by the PURE model (Zhong and Chen, 2021). PURE works by adding markers to highlight the two objects within the sentence. These markers help the BERT part of the model extract context-dependent embeddings. As such, these markers reduce the need for embeddings which reduces memory needed and improves infer- ence speed. Then the model classifies relationships based on embeddings for those markers. Training with these markers has the advantage of maintain- ing the embeddings of other words, which helps the PURE model generalize to unseen words.
Since the records received were not perfect and had many errors, is necessary to introduce an- other process capable of inserting noise into the augmented data in an attempt to mimic the imper- fections of the received records. This noise is done character by character and consists of random inser- tions, deletions, and alterations. Even spacing can be removed between words or inserted into the mid- dle of word, and yet the correct tag is preserved. The noise can also easily be varied from 0 percent (perfect record) to 100 percent (not a character left unaltered).
However, the historical records differ largely from the records that PURE was trained on. Gener- ally, relationships need to be found between 10-13 entities in those records. Re-encoding the sentence and rumning it through BERT for each possible pair of entities would cause large computation overhead. Additionally, these records have a rela- tively constrained vocabulary, which reduced the generalization benefit of using markers. Due to these differences, the markers are dropped in the modified model.
The relationships are instead identified by con- catenating the embeddings of the first word in each entity in a pair and running them through lin- ear classifier. Dropping the markers also allows for sharing the BERT embeddings between the NER and RE steps, such that the raw text is passed through BERT once. The output of BERT fed through the two NER outputs mentioned above and the one RE output.
3.1.2 Named Entity Recognition
Named Entity Recognition is performed using the context-aware embeddings produced by BERT. Af- ter running the raw text through BERT, the embed- ding produced for each word is normalized. This normalization process prevents exploding gradi- ents.
The computation overhead of running this clas- sifier all possible pairs of words in the sentence is negligible compared the complexity of BERT itself. Due this difference, the model can con- sider all possible relationships for all entity types in an end-to-end fashion. The final confidence of a relationship tuple is the geometric mean of the confidence of the relationship with the confidence of the entities involved.
Each embedding is then run through two linear output layers. The first layer predicts BIO tags (Ramshaw and Marcus, 1999). BIO tagging is a common tagging format for tagging tokens in a chunking task in computational linguisties (ex. named-entity recognition). The B- prefix before a tag indicates that the tag is the beginning of a chunk, and an I- prefix before a tag indicates that the tag is inside a chunk. The B- tag is used only when a tag is followed by a tag of the same type without O tokens between them. An O tag indicates that token belongs no entity/chunk. The BIO tagging helps keep entities separate even if they are the same entity category. The last layer completes the process and predicts the entity type between Name, Date, Gender, Age, and a None class.
3.1.4 Extracting fine-grained entities
After the relations between entities are found in record, they can be converted into fine- fine-grained entities. By assigning one person as the primary subject of the record, these entity relationships can be used to convert the coarse-grained labels on entities (such as Name) into fine- ainengranned entities (such as FatherName).

---

Entity
Precision
Recall
F1
Relation
Precision
Recall
F1
Name
98.92
97.40
GenderOf
99.99
99.99
Year
98.62
98.62
AgeOf
96.33
98.13
Month
92.41
94.81
FatherOf
91.428
98.63
Day
94.12
93.02
MotherOf
97.26
97.10
Gender
99.99
99.99
SpouseOf
92.86
99.99
Age
95.52
99.99
BirthOf 99.99
99.99
Micro Avg.
98.49
97.25
97.87
MarriageOF 99.99
99.99
Micro Avg. 96.04
98.83
97.42
Table 1:
Entity recognition accuracy metrics on set of French birth and marriage dataset from 19th Century. is interesting note that precision and recall are more than 90% for all the entities. The Date entity has been further divided into year, birth and day and still its accuracy metries are not much compromised.
Table 2:
Relation recognition accuracy metrics on French birth and marriage dataset from the 19th Cen- trr.. interesting nole that precision and recall are more than 90% for all the relations. 'Fatherof" and 'Spouseof" have the lowest precision of still, they at 91.428 and 92.86.
For example, if "Ted" is the main subject of the record then:
Model
NER
Relation Micro F-1
Micro F-1
("Susan", SpouscOf, "Ted") (Mary. NaouseName) ("1830", BirthOf. -T ("1830",, Year) ("Thirty", AgoOr, "Ted") -> ("Thirty", SelfAge) ("Male", GenderOf, "Ted") ("Male", SelfGender)
Ours
97.9
97.4
PL-Marker
91.1
73.0
(Ye et al., 2021)
4 Experiments
PURE
90.9
69.4
(Zhong and Chen, 2021)
4.1 Setup
Table-Sequence
89.5
67.6
To verify the effectiveness of joint entity and re- lation extraction, comprehensive experiments are conducted on corpus of French birth records and marriage records. Our joint model uses the same training dataset and test dataset for both the NER component and relation extraction component. Us- ing data augmentation on the french records, a dataset of 400 artificial birth and marriage records is generated at the time of execution. This is used as the training dataset. The test dataset is the origi- nal 270 hand-labeled french birth records and mar- riage records. However, this test dataset comes with many complications. These include inaccu- rate labels, spelling mistakes, missing punctuation, myriads of categories and lengthy sentences. Also French has fewer existing language models and more complicated grammar rules, adding to the complications.
(Wang and Lu, 2020)
TriMF
87.6
66.5
(Shen et al., 2021)
TablERT (Ma et al., 2022)
88.0
66.1
Table 3: Our model performs better than the best mod- els in the field of NER (97.9%) and the field of Relation Extraction (97.4%).
micro Fl-score of approximately 97.87%. This is marked improvement over the benchmark model in Entity Recognition, the PL-MMarker model, which has micro Fl-score of 91.1% (Ye et al.. 2021). The higher accuracy of our entity recog- nition model mutually beneficial to the relation extraction component of our joint model. Of note is our entity extraction architecture is able to suc- cessfully train with relatively little training data and perform remarkably well. This also allows our model to train very quickly. One reason the entity extraction so good, use of BERT. Its ability to parse sentences bidirectionally allows for embeddings to be built bidirectionally.
All these anomalies make it harder for neural net- works to understand the data. Whereas, the bench- mark datasets (in relation extraction) have nearly perfect sentences in English with fewer entity and relation types. These qualities make the benchmark datascts less noisy than our French datasct.
4.2
Entity Extraction Results
4.3 Relation Extraction Results
As seen in Table 1, the rough-graines entity ex- traction component of our joint model has a total
The benchmark the field for relation extraction, the PL-Marker model, has a micro score of 73%

---

State-of-the-an Fine-grained Entity Extraction (FlairNER)
Our model (Joint NER-Relation Extraction)
Fine-Entity
Precision
Recall
F1
Fine-Entity
Precision
Recall
F1
SeltGender
98.04
99.99
Sel[Gender
99.99
96.15
SelfName
16.67
0.55
SelfName
89.29
97.40
FatherName
36.84
28.00
FatherName
81.25
90.28
MotherName
42.86
45.00
MotherName
89.74
92.11
NaouseName
41.74
47.52
SpouseName
75.86
84.62
S FatheNaaee
34.12
38. 16
Spouse FatherFateeeaaee
46.15
57.14
NotheNaeter
44.29
48.44
SoouseMotherNamee
56.00
58.33
OtherPersonName
3.80
88.97
OtherPerson
91.80
84,85
SelfAge
76.71
73.68
SelfAge
75.81
99.99
FutherAge
51.90
49.40
FutherAge
76.92
76.92
MotherAge
56.07
64.52
MotherAge
91.67
88.00
SpouseAge
83.87
72.22
SpouseAge
60.71
77:27
SpouseFatherAge
33.61
16.67
FpouseFate
36.35
38.75
SpouseMotherAge
35.71
35.71
SpouseMotherAge
20.00
33.33
BirthDay
72.73
77.78
BirthDay
66.67
99.99
BirthMonth
76.09
83.33
BirthMonth
66.67
99.99
Birth Year
90.34
81.12
Birth Year
99.99
99.99
MarriageDay
47.37
45.00
MarriageDay
99.99
99.99
MarriageMonth
53.33
30.00
MarriageMonth
57.143
99.99
Marriage Year
52.46
56.14
Marriage Year
50.00
99.99
Micro Avg.
65.29
37.36
47.53
Micro Avg.
83.98
87.35
85.63
Table 4: Comparison of Fine-grained Entity Extraction between a state-of-the-art NER model (FlairNER) and our proposed joint model. Our model stands as a clear winner with an score being 85.63 whereas the regular NER model's F1 score is 45.53. Note the huge improvement in important fine-graine entities like SelfName FatherName, MotherName, SpouseName, FatherAge, MotherAge. Also, there is a huge improvement in the recall for marriage dates and birth dates.
(Ye et al., 2021). Our model has achieved a RE mi- Cro FI score of approximately 97.42% (see Table 2). This is an exceptional improvement in relation extraction performance. Our model achieves this level of improvement due the nature of joint entity-relation extraction models where training entity extraction improves relation extraction and to lesser degree training relation extraction im- proves entity extraction. Much like the the Entity Extraction component of our model, the Relation Extraction of our model also trains quickly and ac- curately, with relatively little training data. These impressive results of the relation extraction module (in conjunction with the entity extraction module) clearly demonstrate the value our model has.
over the default method (vanilla NER) of indexing family history documents.
Conclusion
The model succeeded in indexing the fine-grained entities in family history records within an accept- able margin of accuracy. Results were validated by recording the model's NER micro Fl-score, rela- tion extraction micro F1-score, and the Fl score for each fine-grained entity type at the end of the pipeline. It then compared the NER micro F1-score and RE micro F1-score against the corresponding metrics for benchmark joint entity-relation extrac- tion papers.
The results our joint model are better than the benchmark models in the field, even though our data was much noisier than the data they used. As such our model contributes to the current literature on joint entity-relation extraction in terms of family history records.
4.4 Final Results
Next the predicted relationships and entities are used to infer the fine-grained entities. Where as the entity recognition module and relation extraction module avoid compounding errors by using end- to-end architecture, inferences made after training are very susceptible to compounding errors. This explains why certain fine-grained entities such as the age of the subject's spouse's mother have such low accuracy metrics; they are too sensitive to com- pounding errors. However, the final results in Table show the joint model is a marked improvement
References
Giannis Bekoulis, Johannes Deleu, Thomas Demeester. and Chris Develder. 2018. Joint entity recognition and relation extraction multi-head selection prob- lem. Expert Systems with Applications, 11::34 ArXiy: 1804.07847.

---

Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. 1Xiv::810004800 [cs]. ArXiv: 1810.04805.
Deming Ye, Yankai Lin, and Maosong Sun. 2021. Pack Together: Entity and Relation Extraction with Levitated Marker. arXiv:2109.0600 [cs]. ArXiv: 2109.06067.
Kai Zhang, Yuan Yao. Ruobing Xie. Xu Han, Zhiyuan Liu, Fen Lin, Leyu Lin, and Maosong Sun. 2021. Open hierarchical relation extraction. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics:
Xu Han, Pengfei Yu, Zhiyuan Liu, Maosong Sun, and Peng Li. 2018. Hierarchical Relation Extraction with Coarse-to-Fine tone Grained Attention. In Proceedings of the 2018 Conference on Empirical Methods in Nat- ural Language Processing, pages 2236-223 2245, Brus- sels. Belgium. Association for Computational Lin- guistics.
Human Language Technologies, pages 5682-5693, Online, Association for Computational Linguistics.
Tianyang Zhao, Zhao Yan, Yunbo Cao, and Zhoujun Li. 2020. Asking Fiffective and Diverse Questions: A Machine Reading Comprehension based Framework for Joint Entity-Relation Extraction. In Proceedings of the Twenty-Ninh International Joint Conference Artificial Intelligence, pages 3948-3955 Yoko- hama, Japan. International Joint Conferences on Ar- tificial Intelligence Organization.
Xiaoya Li. Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity- Relation Extraction as Multi-Turn Question Answer- ing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguisties, pages 1340-1350, Florence, Italy. Association for Compu- tational Linguistics.
Youmi Ma, Taisuya Hiraoka, and Naoaki Okazaki. 2022. Named Entity Recognition and Relation Ex- traction using Enhanced Table Filling by Contextu- alized Representations. Number: arXiv:2010.07522 arXiv:2010.07522 [cs].
Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 50-61, Online, Association for Computational Linguistics.
Matthew E. Peters, Mark Neumann, Mohit lyyer. Matt Gardner, Christopher Clark, Kenton Lec, and Luke Zettlemoyer. 2018. Deep contextualized word representations. 182.v:1802.05363 [ex]. ArXiv: 1802.05365.
L. A. Ramshaw and M. Marcus. 1999, Text Chanking Using Transformation-Buttt Learning, pages 157- 176. Springer Netherlands, Dordrecht.
Victor Sanh, Thomas Wolf, and Sebastian Ruder. 2019. A Hierarchical Multi-Task Approach for Learning Embeddings from Semantic Tasks. Proceedings of the AAAI Conference on Artificial Intelligence. 33:6949-6955
Yongliang Shen, Xinyin Ma, Yechun Tang, and Weim- ing Lu. 2021. A Singer--Sense Memory Flow Frame- work for Joint Entiiy and Relation Extraction. Num- ber: 1rXiv:2101.102. arXiv:2101. [cs].
Ryuichi Takanobu, Tianyang Zhang. Jiexi Liu, and Min- lie Huang. 2019. Á Hierarchical Framework for Relation Extraction with Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 33:7072-7079.
Jue Wang and Wei Lu. 2020. Two Better than One: Joint Entity and Relation Extraction with Table- Sequence Encoders. In Proceedings of the 2020 Con- ference Empirical Methods in Natural Language Processing (EMNLP), pages 1706-1721, Online. As- sociation for Computational Linguistics.
Chenhao Xie, Jiaqing Liang, Jingping Liu, Chengsong Huang, Wenhao Huang, and Yanghua Xiao. 2021. Revisiting the Negative Data of Distantly Supervised Relation Extraction. arXiv:2105 200.10155 [es]. ArXiv: 10158.


Reviews:
[{"review_id": "znO5W1Ec0Yy", "reviewer": "Reviewer_2SCp", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "No Review", "summary_of_the_review": "No Review", "strength_and_weaknesses": "No Review", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "yXlN48vZZ24", "reviewer": "Reviewer_AKDM", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes semi-connected joint entity recognition and relation extraction of contextual entities in family history records ER task. By jointly training NER and RE to improve the fine-grained NER task.", "summary_of_the_review": "The motivation is unclear. The novelty is so limited. The experiments are insufficient. The experimental results in Table 3 are not comparable and convincing.", "strength_and_weaknesses": "The motivation is unclear. \n\n1.There is no content about semi-connected. What's the meaning? And why use this structure?\n\n2.The authors said, \"Existing research insufficiently performs the task of fine-grained entity extraction on contextual entities.\" Please explain the contextual entities and fine-grained entities in detail. And give some examples of other models' results.\n\n3.The authors said most of these models could not find all the relations in a multisentence corpus because they can not map a relationship between two entities in separate sentences.  However, there are many cross-sentence relation extraction methods that should be referred.\n\n4.The reason for the usage of BERT is unclear.\n\n5.The authors refer to transcriptions, but there are no related contents, such as tools used and average precision.\n\n\nThe novelty is so limited.\n\n1.The NER and RE jointly learning is a common training strategy, which is not a novel solution. And Why is it more suitable for fine-grained entity extraction?\n\n2.The model structure is simple and common.\n\n\nThe experiments are insufficient.\n\n1.The authors design many rules (around 300), which is helpful for this task. However, it will weaken the learning ability of the model and become an engineering-based method. In addition, the authors should add ablation studies to show the effectiveness of these rules and the BERT basic model when changing other NN structures and removing these rules.\n\n2.For experiments, I seriously doubt that the authors don't run experiments on your datasets using other baseline models because these are easily copied from their paper's results (on the ACE2005 dataset). Therefore, the results in Table 3 are not convincing!\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "wJj6gjCQiq", "reviewer": "Reviewer_hDjQ", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "N/A", "summary_of_the_review": "This paper uses the ACL template instead of the ICLR one.", "strength_and_weaknesses": "N/A", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "iyDmttrYuM0", "reviewer": "Reviewer_oJAx", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "*Disclaimer: I am a vision person and an emergency reviewer*\n\nThis manuscript studies the problem of named entity recognition (similar to object detection) and relation extraction (similar to human object interaction detection) from French texts transcripted by off-the-shelf OCR models. The method firstly classifies text tokens into pre-defined entity categories then classifies relationships which is quite similar to the two-stage methods we used in human-object interaction detection. A data augmentation method based upon context free parse tree is proposed, but not ablated. Evaluations are done on a newly collected dataset (provided in the supp) and reports improved results on entity F1, relation F1 and combined F1. Codes are not provided or promised.", "summary_of_the_review": "I fail to understand the core methodology improvement, why evaluating on public datasets is not possible and where the large performance improvement comes from.", "strength_and_weaknesses": "Strengths:\n+ The authors collect a new dataset for French family tree parsing and release it.\n\nWeaknesses:\n- The manuscript is not prepared in ICLR template.\n- The central technique contribution is not clear to me. It seems that the baseline PURE is also a two-stage method and I fail to understand what 'semi-connected' means.\n- The evaluation is not done on public datasets like ACE or SciERC, which makes it difficult to understand where the improvements come from. And I cannot understand why the proposed method can outperform PURE by such a large margin in Table.3. A detailed ablation is recommended.\n- If the augmentation strategy based upon context free grammar is the key factor, why not evaluate it on public datasets? Is this feasible?\n- I have a layman question: I understand western languages are similar to each other, but is encoding French with a pre-trained BERT really reasonable? Correct me if this is a common practice.\n\nTypos:\n- course grain in several places\n- Much like the the Entity Extraction component of our model, there are two 'the'.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5gSevSR3ac1", "reviewer": "Reviewer_VUao", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims to extract \"contextual entities\" (or fine-grained entities as refered in the paper) from specialized documents, which is Family History Records written in French.\n\nFor that, the paper first proposes a heavily cutomized data augmentation method that consists of complex rules and dictionaries.\n\nThen a model for joint extraction of entities and relations is applied, which mostly follows an existing work PURE, only without markers.\nThe model predicts both entities and relations, based on which fine-grained entities can be identified. Actually, what refered to as fine-grained entites here are entities with adjective relations ahead, such as \"FatherAge\", \"MotherAge\", \"SpouseAge\" etc.", "summary_of_the_review": "The method is of little novelty; the experiments and comparison are not justifiedly conducted; the task is very specialized and might be of limited interest to the entire community; and the presentation is not well.\n\nAlso, the template is not even for ICLR, but ACL instead. I wonder whether this incurs desk rejection.\n\nTherefore, I recommand reject.", "strength_and_weaknesses": "Strength:\n\nThe paper addresses a new problem of entity recognition from Family History Records.\n\nComparisons between the proposed method and several related works are made.\n\n\nWeaknesses:\n\n1. the targeted task, although being newly discussed, is very specialized. I assume it has very limited application scenario and thus this paper provides very restricted reference or insights for most of the community. Besides, the dataset is also not publically available, from the paper I can't get sufficient understanding of this specific task.\n\n2. The proposed method is mostly based on PURE, as clarified in the paper, while the main novelty should be the data augmentation introduced in 3.1.1. This augmentation strategy consists of extremly sophisticated rules that relies on very specific understanding of the structure and content for Family History Records. Overall, I think the proposed method can hardly generalize to very related datasets or even standard benchmark like ACE04/05 etc.\n\n3. The experiments are not justified.\n\n   3.1 The author claims that the proposed method outperforms FlairNER by +38 absolute F1 (Table4), I very much doubt the justification of this conclusion. As two methods are applied with different pipelines. FlairNER directly treats each fine-grained entity as independent categary (I suppose), while the proposed method extract coarse-grained entity and relation, then assemble relation ahead of entity as fine-grained descriptions.\n\n   3.2 For results in Table 3, I also find the comparison hardly convincing, what if \"Ours\" method are not trained with the augmented data? Such ablation is lacked.\n\n   3.3 Besides, are all those model built upon standard BERT model? Why not use the French version as there already exists one[1]? Will this affects much of the reported results?\n\n4. The presentation need further refine.\n\n   4.1 Introduction section, for many times \"coarse\" are written into \"course\".\n\n   4.2 \"Hier- archical relationship extraction is particularly ef- fective at detecting hierarchical relationships\" This sentence does not make sense to me. And the followed citation should be included in shared parentness.\n\n   4.3 \"is considerably different then a new template needs to be made\" -> \"is considerably different than a new template needs to be made\"\n\n[1] Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, and Benoît Sagot. 2020. CamemBERT: a Tasty French Language Model. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7203–7219, Online. Association for Computational Linguistics.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject