A Co-guided Neural Network for Person Name Recognition in Academic Homepages
Anonymous submission
Abstract
Academic homepages are important channels for learning researchers’ proﬁles.
Knowing the person names in academic homepages is essential to the extraction of other entities such as contacts, publications, and biography.
Traditional NER models are trained on newswire corpora such as CoNLL-2003, which contain well-formed names in consistent and complete syntax.
However, academic homepages often contain text with incomplete syntax and names of various forms.
Few studies address person name recognition in this context.
To ﬁll this gap, we start with proposing a ﬁnegrained name annotation scheme.
This scheme further labels detailed name forms including ﬁrst, middle, or last names, and name words in full or name initials.
We then propose a
Co-guided Neural Network (CogNN) model to learn from homepages labelled with our ﬁne-grained annotations.
CogNN uses coattention mechanisms to co-guide two jointly trained neural networks, each focusing on different dimensions of the name forms.
It thus takes full advantage of our annotation scheme and can accurately recognise person names in academic homepages.
Experimental results on real datasets show that CogNN signiﬁcantly outperforms state-of-the-art NER models in extracting person names from academic homepages, while achieving comparable performance on a traditional NER dataset.
Introduction
Academic homepages are an important source for learning researchers’ proﬁles, including names, contact details, bibliography, working experience, and publications.
Among these, person names are basic and yet important.
Knowing the person names is essential to extracting other entities in academic homepages and also provides valuable insights on researcher collaboration networks.
We focus on extracting person names from academic homepages using detailed name form information, such as whether a token is a ﬁrst, middle, or last name, and whether the token is a full name word or a name initial.
Figure 1 shows an example of person name recognition in academic homepages.
Given the text content of an academic homepage, we aim to recognise all person names as highlighted in the example.
The challenges of recognising person names in academic homepages lie in the diversity of text and name forms.
The text in academic homepages is free-form and may have incomplete syntax.
For example, in Figure 1, the biography section consists of complete sentences while the students section is simply a table.
The person names may be in different forms as well.
Figure 1 contains well-formed full name of the researcher ‘ John
Doe ’ in the page header and abbreviated names in the publications section.
Further, the abbreviated names may have different abbreviation forms, e.g., ‘ B.B. Doe ’ vs. ‘ Doe, J. ’.
Recent NER models (Huang et al., 2015; Chiu and Nichols, 2016; Ma and Hovy, 2016) are trained on well-formed texts such as news articles.
These models do not solve our problem since news articles and academic homepages have substantially different underlying text distributions.
Figure 2 shows an example, where Stanford Named
Entity Tagger is applied to recognise the person names from a publication string in an academic homepage.
The tokens in italics are name tokens, while the tokens in bold are those recognised as name tokens.
We can see that not all the names have been recognised.
To address person name recognition in academic homepages, we propose a ﬁne-grained name annotation scheme that annotates detailed name forms including ﬁrst, middle, or last name, and a full name word or a name initial (cf.
Fig-
Kime C , Sakaki-Yumoto M , Goodrich L ,
Hayashi Y , Sami S , Derynck R , Asahi M , Panning B , Yamanaka S , Tomoda K ure 1).
Such detailed annotations offer richer training signals to NER models to learn the patterns of person names in free-form text.
However, detailed annotations also bring challenges because more label classes need to be learned.
To take advantage of the detailed name annotations and to address their challenges, we propose a Co-guided Neural Network (CogNN) model for person name recognition.
CogNN consists of two sub-neural networks (Bi-LSTM-CRF variants).
One of the sub-network focuses on predicting whether a token is a name token, while the other focuses on predicting the name form class of the token.
We add a co-attention layer to connect the two Bi-LSTM-CRF based sub-networks.
This way, the two sub-networks can share the learning signals to reinforce their label prediction conﬁdence.
The two sub-networks are trained simultaneously by minimising their total loss.
This paper makes the following contributions:
• New angle : We study person name recognition in academic homepages, for which we propose a ﬁne-grained annotation scheme that provides information on various forms of names.
• New dataset : We create a dataset of diverse academic homepages where the person names are fully annotated with detailed name forms.
This dataset will be released upon paper publication.
• New model :
We propose a Co-guided Neural Network (CogNN) model to recognise per- son names using the ﬁne-grained annotations.
It learns the different name form classes with two neural networks while fusing the learned signals through co-attention mechanism.
Experimental results show that CogNN outperforms state-ofthe-art NER models in the accuracy of extracting person names from academic homepages.
The rest of this paper is organised as follows.
Section 2 summarises related studies.
Section 3 describes our name annotation scheme and dataset.
Section 4 detials the proposed model.
Section 5 presents experimental results.
Section 6 concludes the paper.
Related Work
Named entity recognition (NER) aims to identify proper names in text and classify them into different types, such as person, organisation, and location (Nadeau and Sekine, 2007).
Neural NER models have shown excellent performance on long texts which follow strict syntactic rules, such as newswire and Wikipedia articles (Huang et al., 2015; Chiu and Nichols, 2016; Ma and Hovy, 2016).
However, these NER models are less attractive when applied to short texts which may not have consistent and complete syntax (Li et al., 2015; Dugas and Nichols, 2016).
Recent studies also consider user-generated short texts from social media platforms such as Twitter and Snapchat.
Since social media texts are usually posted with images, researchers propose to make use of both textual and visual context to recognise the named entities (Lu et al., 2018; Moon et al., 2018).
Such models are less relevant because academic homepages do not contain images consistently except for a photo of the page owner.
NER studies on academic homepages usually treat the text content of a webpage as a document, upon which traditional NER techniques are applied.
For example, Zhang et al.
(2018) use a Bi-LSTM-CRF based hierarchical model to extract all the publication strings from a given academic homepage.
Tang et al.
(2010) assume that the page owner’s full name is given and use a Treestructured CRF model to extract multiple types of entities from a given academic homepage.
This technique does not apply to our problem as we assume no pre-knowledge about the page owners.
Person names are often recognised together with other named entities, such as locations and organisations (Huang et al., 2015; Ma and Hovy, 2016; Chiu and Nichols, 2016).
There are a few studies focusing only on person names.
Dozier and Haschart (2000) extract the attorney and judge names in legal texts using a semantic parser.
Packer et al.(2010) focus on extracting name from noisy OCR data, which may include spelling errors and incomplete texts.
They combine rule based methods, the Maximum Entropy Markov
Model, and the CRF model using a simple votingbased ensemble.
Minkov et al.
(2005) extract person names from emails using the CRF model.
They design email speciﬁc structural features and exploit in-document repetition to improve the extraction accuracy.
Shaalan and Raza (2007), Elsebai et al.
(2009), Bidhend et al.
(2012), and
Aboaoga and Ab Aziz (2013) study person name recognition in Arabic.
They use rule-based methods based on gazetteer name lists and regular expressions.
To the best of our knowledge, no existing work has used detailed name forms for extracting person names.
Proposed Name Annotation Scheme
We ﬁrst present our name annotation scheme with detailed name forms and our HomeName dataset that is annotated under this scheme.
Detailed name form annotations are done to better capture the person name form features in free-form texts.
Both well-formed names written in full and various forms of abbreviated names may appear in academic homepages.
Annotating the name tokens with detailed forms offers more direct training signals to NER models to learn the patterns of person names.
This also allows an
NER model to be trained with fewer data.
Thus, unlike traditional NER datasets, which only label a name token with a PER (person) label, we further provide detailed name form information for each name token.
We label each name token using a three-dimensional annotation scheme:
• BIE : Begin , Inside , or End of name, indicating the position of a token in a person name, • FML : First , Middle , or Last name, indicating whether a name token is used as the ﬁrst, middle, or last name, and • FI : Full or Initial , indicating whether a name token is a full name word or an initial.
Using the three-dimensional annotation scheme above, we can describe the detailed name form of a name token.
For example, in Figure 1, ‘ John Doe ’ can be labelled as Begin First Full
End Last Full , while ‘ Johnny van der
Doe ’ can be labelled as Begin First Full Inside Last Full Inside Last Full End Last Full .
HomeName is a collection of academic homepages with person names fully annotated using the proposed annotation scheme.
We construct our dataset based on the HomePub dataset 1 .
The HomePub dataset contains 2,500 web pages from different universities and research institutes, among which 2,087 are academic homepages and 413 are non-academic homepages (such as staff directory pages).
We keep only the academic homepages, which are from 286 institutes.
We annotate all the person names in the academic homepages using our proposed name annotation scheme.
Each academic homepage is annotated by two annotators, with inter-annotator agreement measured at Cohens κ = 0 .
63 for names and κ = 0 .
41 for detailed name forms.
Disagreement is resolved by a third annotator.
We provide an annotation scheme, a semi-automatic annotation tool and a one-hour training to each annotator.
More details on the annotation are given in the supplementary material.
The ﬁne-grained annotations offer more direct training signals to NER models but also bring challenges because more label classes need to be learned.
Next, we present our CogNN model that takes advantages of the detailed name form annotations to recognise person names.
Proposed Model
Given a sequence of input tokens X , where X =
[ x 1 , x 2 , ..., x n ] and n is the length of the sequence, our aim is to predict for each token
x i whether it is a name token.
Our proposed model CogNN achieves this aim with the help of two Bi-LSTM-CRF based subnetworks: the name token network and the name form network , as illustrated in Figure 3.
The name token network focuses on predicting whether a token is part of a name (the BIE dimension), while the name form network focuses on predicting the detailed name form of the token ( FML or FI dimensions).
Co-attention layers are added to the Bi-LSTM-CRF based sub-networks, so that they can share the learning signals and reinforce the prediction conﬁdence.
The intuition here is that knowing whether a token is part of a name helps recognise the detailed name form class of the token, and vice versa.
For example, if a token is not considered as part of a name, then even if it is a word initial, it should not be labelled as a name initial.
In particular, we start with concatenating the word embedding and a letter case vector for an input token to produce its representation.
We feed the input under this representation into Bi-LSTM to learn its hidden representation matrix (Section 4.1).
Then, we use co-attention mechanism to coguide the two jointly trained sub-networks.
Our co-attention mechanism is designed to share the training signals between the two sub-networks.
It updates the importance of each token learned from the two sub-networks and records their correlations (Section 4.2).
The two sub-networks are trained simultaneously by minimising their total loss (Section 4.4).
Next, we detail each layer of our CogNN model.
Capture: Hidden Feature Extraction
Both the name token network (denoted as N Y ) and the name form network (denoted as N Y (cid:48) ) share the same layer structure.
They only differ in the target labels Y and Y (cid:48) .
Here, Y denotes the label sequence that records whether an input token is part of name, and Y (cid:48) denotes the label sequence that records the form class of each input token.
For simplicity, we focus on the name token network
N Y in the following discussion.
We start with concatenating the word embedding e i and letter case vector s i for an input token x i ∈ X to produce its vector representation.
We use GloVe (Pennington et al., 2014) computed on our HomeName corpus for the word embeddings e i .
The letter case vector
s i records the letter case information of x i , which is an important hint for recognising names.
For example, the ﬁrst letter of a name token is often in uppercase, and a name initial is often formed by an uppercase letter plus a dot.
Our letter case vector is a three-dimensional binary vector where each dimension represents: (i) the ﬁrst character in the token is in uppercase, (ii) all characters in the token are in uppercase, and (iii) any character in the token is in uppercase.
We then use Bi-LSTM (Dyer et al., 2015) to capture the hidden features from the input sequence.
The output hidden representation, denoted as h i , summarises the context information of x i in X .
Our hidden representation matrix H in N Y can be written as [ h 1 , h 2 , ..., h n ] , where h i ∈ R d and d is the number of dimensions of the hidden representation.
Similarly, H (cid:48) in N Y (cid:48) can be written as [ h (cid:48) 1 , h (cid:48) 2 , ..., h (cid:48) n ] .
4. Share: Co-attention Mechanism
Next, we share the learning signals in the hidden representation matrices H and H (cid:48) , and obtain new hidden representation matrices ˜ H and ˜ H (cid:48) for the two sub-networks, respectively.
Note that training the two sub-networks separately is suboptimal, since the underlying correlation among the name label dimensions is lost.
For example, a token recognised as Inside in N Y is more possible to be Middle in N Y (cid:48) .
To address this issue, we use co-attention to take the learning signals from two hidden representations into account by:
[Equation]
where W h and W h (cid:48) ∈ R k × d are trainable parameters, k is dimensionality of the parameters, ⊕ is the concatenating operation, and P ∈ R 2 k × n .
A related technique is used by Yang et al.
However, they only consider the token importance in a single hidden representation sequence of a document, while we consider the token importance in two hidden representation sequences simultaneously.
The co-attention distribution that records the importance of each token after examining two hidden representation sequences can be obtained as:
[Equation]
where W p ∈ R 1 × 2 k are trainable parameters and
A ∈ R n is an importance weight matrix.
The new hidden representation ˜ h i can be computed by:
[Equation]
We thus obtain the new hidden representation sequences ˜ H =
[ ˜ h 1 , ˜
h 2 , ..., ˜ h n ] and ˜ H (cid:48) =
[ ˜ h (cid:48) 1 , ˜ h (cid:48) 2 , ..., ˜ h (cid:48) n ] for the two sub-networks.
4. Output Layer
The new hidden representation sequences ˜ H = [ ˜ h 1 , ˜ h 2 , ..., ˜ h n ] is trained to produce a label sequence Y .
To enforce the structural correlations between labels, Y is passed to a CRF layer to learn the correlations of the labels in neighborhood. Let Y denotes the set of all possible label sequences for ˜ H . Then, the the probability of the label sequence Y for a given representation sequence ˜ H can be written as :
[Equation] Input : input the number of input tokens Output or do // Capture e = GloVe(
) s i = computeCaseVector ( x i ) h
i = Bi-LSTM( e i ⊕
s i ) h (cid:48) i =
Bi-LSTM( e i ⊕
[Equation]
i ) // Share a i =
getCoAttention ( h i , h (cid:48) i ) ˜ h
i = getGuided( h i , a i ) a (cid:48) i =
getCoAttention( h (cid:48) i , h i ) ˜ h
[Equation]
i = getGuided
[Equation]
h (cid:48)
i , a (cid:48) i ) 12 for i to N do // Predict [Equation] y [Equation]
=
[Equation]
i ) return y where ψ t ( y (cid:48) , y ; ˜ H ) is a potential function, W Y is a set of parameters that deﬁnes the weight vector and bias corresponding to label pair ( y (cid:48) , y ) .
Similarly, we can also compute p [Equation] Y [Equation]
4. Joint Training
The remaining question is how to train two networks simultaneously to produce label sequences
Y and Y (cid:48) .
We achieve this by joint optimisation.
Speciﬁcally, we train the CogNN model end-toend by minimising loss L , which is the sum of the loss of the two sub-networks:
[Equation]
where L ( W Y ) and L ( W Y ) are the negative loglikelihood of the ground truth label sequences ˆ Y and ˆ Y (cid:48) for the input sequences respectively, which are computed by:
[Equation]
Experimental Study
We evaluate our proposed CogNN model on our HomeName dataset and the CoNLL- NER shared task dataset (Tjong Kim Sang and
De Meulder, 2003).
Recall ( R ), Precision ( P ) and F1-scores ( F ) are used to measure the performance.
The experiments are run with an NVIDIA
GeForce GTX 1080 GPU.
We verify the following two aspects of our approach:
1) Effectiveness.
We compare the CogNN model with baseline NER models and variants of the CogNN model on extracting person names from academic homepages (Section 5.1).
2) Applicability.
We further explore whether our model and labelling scheme can improve person name recognition on news articles (Section 5.2).
5. Effectiveness on Academic Homepages
We ﬁrst study the performance of CogNN on academic homepages.
Dataset and annotations We use the HomeName dataset with the proposed detailed name form annotation scheme (Section 3).
We use 1,677 homepages for training and developing, and 410 homepages for testing.
Models and hyperparameters Four models are tested:
• CRF : The Stanford NER system (Finkel et al., 2005) with parameter tuned.
• Bi-LSTM-CRF : The Bi-LSTM-CRF model (Huang et al., 2015) with a 100dimensional hidden layer, a dropout layer with probability 0.5, a batch size of 32, and an initial learning rate of 0.01 with a decay rate of 0.05.
The model is optimised with stochastic gradient descent and we stop if the accuracy does not improve in 10 epochs.
• CogNN : Our proposed model.
We use GloVe trained on HomeName to get a 100dimensional word embedding.
For CogNN, we use the same hyperparameters and training parameters as those in Bi-LSTM-CRF.
The optimal hyperparameters are obtained with a standard grid search on the developing dataset.
More Details of the preprocessing, word embedding training, model implementation, and example output are given in the supplemental material.
Fusion strategies We also study the impact of using the detailed name form annotation scheme in different ways (cf.
Figure 4):
• No fusion : Training an independent model that learns to label the input sequence with the BIE, FML, or FI label types but not a combination of any two types of the labels.
• Early fusion : Training an independent model that learns to label the input sequence with a cartesian product of the BIE, FML, and FI label types, e.g., to label ‘ John Doe ’ with Begin First Full End Last Full .
• Late fusion : Training sub-models each focusing on one label type and merging all the predicted labels afterwards to yield the ﬁnal prediction by using every span of tokens with name label as a name (so we do not report the token level performance).
• In-network fusion : Training two submodels each focusing on one label type and sharing the learning signals in the intermediate levels of the sub-models.
Our CogNN model use this strategy.
Results We report the token level performance, which reﬂects the model capability to recognise each person name token.
We also report the name level performance, which reﬂects the model capability to recognise a whole person name without missing any token.
As Table 1 shows, overall, the neural models perform much better than the non-neural models, especially on the name level.
When examining the performance of Bi-LSTM-CRF, we ﬁnd that early fusion is much worse than no fusion.
This is expected as early fusion of different name form types leads to too many classes to be predicted.
Even for a two-token name, it may have (3 × 3 × 2) 2 =possible name form combinations.
Late fusion offers better performance, which indicates that the separately trained networks on different annotations have their own focuses.
However, it does not take advantage of the correlations between name form types and is not as good as our in-network fusion strategy.
Our proposed model CogNN outperform the baseline models.
CogNN outperforms the best baseline results by up to 5.64% and 5.35% in terms of F1-score in token level and name level, respectively.
The reason is that our model can capture the underlying relationships among different name forms when training and gain higher prediction [Table]
conﬁdence. Also, CogNN yields best results when the models are jointly trained with input annotations BIE and FI.
This is consistent with the observation on the no-fusion models where a model trained with either BIE or FI annotations outperforms a model trained with FML annotations [Table]
5. Applicability on Newswire Articles
We further show the applicability of our CogNN model and detailed name form annotation scheme on traditional newswire texts.
Dataset We use the CoNLL-2003 dataset which contains 1,393 annotated English newswire articles that focus on four types of named entities: person (PER), location (LOC), organisation (ORG) and miscellaneous entity (MISC).
This dataset does not come with detailed name form annotations.
We add annotations using the same method described in Section 3.
Annotations We compare the following combinations of annotations:
• PER : Using only PER labels.
• FI : Using only FI labels.
• FML : Using only FML labels.
• CoNLL : Using all original labels in CoNLL2003.
• FI
+ CoNLL : Replacing PER by FI labels in CoNLL-2003.
• FML + CoNLL : Replacing PER by FML labels in CoNLL-2003.
Since the detailed name form labels are necessary for training CogNN
[Table]
we use the following four pairs of input annotations for CogNN: [PER,
FI], [PER, FML], [CoNLL, FI + CoNLL]
[Table]
and [Table] [CoNLL, FML + CoNLL].
Models and Hyperparameters
We compare
CRF , Bi-LSTM-CRF , and CogNN .
We use the same model hyperparameters and training parameters as described in Section 5.1.
We initialise word embeddings with GloVe pretrained 100dimensional embeddings.
Unknown words are randomly initialised.
All the above models are trained, developed, and tested on the training, developing, and testing datasets in CoNLL-2003.
Results From Table2, we see that neural models perform better than the non-neural model, which is consistent with the results in Section 5.1.
When providing extra ORG, LOC, and MISC annotations apart from PER to CRF and Bi-LSTMCRF, the recall increases while the precision decreases.
This indicates that the extra annotations help recognise more named entity tokens but may also misguide the model.
In comparison, CogNN is less impacted.
When providing extra FI or FML annotations apart from PER to CRF and Bi-LSTM-CRF, the performance of both models does not improve while that of CogNN improves.
Our improvements mainly lie in the precision, which indicates that CogNN can well distinguish person name tokens from others.
These results also indicate that only applying the detailed name form annotations on newswire data for the existing models is not enough.
Our CogNN model is essential to make use of the extra detailed name form information.
Although, the advantage of CogNN on formal
English newswire articles is smaller than that on the HomePub dataset (Section 5.1).
The main reason is that the name forms in newswire articles are less ﬂexible compared with those in academic homepages, which impinges the importance of adding extra name form information.
Conclusion
We studied the person name recognition problem in academic homepages.
We propose a new name annotation scheme which provides more detailed information for various forms of names.
We constructed a new dataset HomeName .
To take advantages of the detailed name form information, we proposed the CogNN model that makes use of two sub-networks to learn whether a token is part of a name and its forms, respectively.
Through coattention, the two sub-networks help each other to boost the overall name recognition accuracy.
We performed an experimental study to evaluate model effectiveness and applicability.
The results showed that CogNN outperforms state-of-the-art
NER models by up to 5.64% and 5.35% in terms of F1-score in token and name levels on academic homepages, while achieving comparable performance on a traditional NER benchmark dataset CoNLL-2013.
We will release HomeName and the annotation tools for public use upon paper publication.
For future work, we plan to investigate the CogNN model on other tasks that exhibit dependencies between each other, such as POS tagging and Chunking.
We also plan to investigate using three subnetworks in CogNN, one for each of the BIE,
FML and FI dimensions, with a dual decomposition in the fusion steps.
Late fusion would be another interesting direction to explore.
A Supplemental Material
In this supplementary section, we provide additional details for the Dataset (Section 3) and the
Experiments (Section 5).
A. HomeName Dataset
File Structure Our dataset is constructed based on the HomePub dataset 3 , which contains academic homepages with publication strings manually labelled.
It contains 2,500 subfolders and each subfolder contains three ﬁles for a webpage:
• An HTML ﬁle containing the page source.
• A TXT ﬁle containing the visible text of the webpage, which is rendered by python’s Selenium 4 package.
• A JSON ﬁle containing publication annotations.
We edit the JSON ﬁles to add name annotations using the following format:
Annotation Tool
Annotation of homepages is time-consuming, especially when a homepage contains many names in complex forms.
We developed a semi-automatic tool 5 to assist the annotation, which has ﬁve main functionalities:
• Group label :
This functionality helps annotate a group of names of the same form.
For example, ‘Doe J’ and ‘Joon-gi L’ have the same forms and can be annotated at once.
• Index : This functionality helps ﬁnd all positions of a given name string in the TXT ﬁle.
• Mask :
This functionality helps annotators to proofread the text and ﬁnd unlabelled names.
It replaces all the names already annotated with a special token ‘ANNOTATED’ .
• Validate :
This functionality runs a simple automated quality check of the annotations.
It checks: (1) whether the position indices of the names annotated in the JSON ﬁle are consistent with the names appeared in the TXT ﬁle; and (2) whether each annotated name comes with the name form under the threedimensional annotation schemes.
• Compare :
This functionality locates disagreement between two annotators’ labels on the same homepage.
It identiﬁes the list of names with inter-annotator disagreement.
Annotators There are 6 annotators to annotate the dataset.
The annotators are postgraduate students who have taken machine learning subjects.
We provide a one-hour training to each annotator.
We provide the annotators with an annotation scheme and two example pages that are already annotated.
We ask each annotator to annotate six pages.
We examine the results and provide guidance on how to improve the annotation quality.
We highlighted the following at training:
• Any named entities such as places, buildings, organizations, prizes, honored titles or books, which are named after a person, should not be annotated as a person’s name.
• Words connected with a hyphen or an apostrophe should not be split into multiple tokens.
For example, both ‘Joon-gi’ and ‘O’Keeffe’ both have only one token.
• Nobiliary particles 6 , e.g., ‘van’ , ‘zu’ and ‘de’ , should be annotated as last names.
Each academic homepage is annotated by two annotators.
We ask all the annotators to annotate using our annotation tool and also note down any pages with uncertain name labels in the comment ﬁeld.
After their annotations, we summarise the disagreement between annotators.
We make a decision on the disagreement and also check the un- certain pages and names.
We send feedback when they annotate every 230 homepages.
Annotation Analysis
• Conﬁdence: Only 3.64% of all the homepages contain annotations that are uncertain as ﬂagged by the annotators, while 78.08% of these pages are actually correctly labelled.
This indicates that the annotators have high conﬁdence in their annotations.
• Inter-annotator Agreement: We compute the inter-annotator agreement on name strings and name forms using Cohens Kappa measurement.
The annotators have higher agreement on name strings ( κ = 0 . 63 ) and lower agreement on detailed name forms ( κ = 0 . 41 ).
The disagreement is mainly in homepages with a long string of consecutive name tokens such that different annotators may disagree on which tokens to form a name.
The annotators may also disagree on whether a name token is a ﬁrst name, middle name, or last name.
This is difﬁcult especially when the context is unclear.
• Time: On average, it takes 16 minutes to annotate an academic homepage with our tool.
Dataset Analysis
In total, the HomeName dataset contains 2,087 academic homepages from 286 institutes, i.e., 7.29 pages per institute (standard deviation
A total of 34,880 names are annotated and 70,864 name position indices are recorded.
On average, a name appears twice in an academic homepage.
Most names begin with last names (64.73%) while the rest mostly begin with ﬁrst names.
Only 13 names start with middle names.
Most names contain at least one initial (66.57%).
The two most frequent name forms are Begin
Last Full End First Initial and
Begin First Full End
Last Full .
Table 3 summarises the annotation results and the dataset.
A. Experiment Details
Preprocessing We focus on English webpages and ﬁrst convert any text in Unicode to ASCII using Unidecode 7 .
We then split the text into sentences using the sentence tokenizer in NLTK.
The sentences are further tokenized on whitespace and punctuations except for hyphens and apostrophes.
Every punctuation is considered as a single token to retain the structural information.
Word Embedding
We train 100 dimensional
Summary of Annotation
Conﬁdence Uncertain pages 3.64% Acc. on uncert.
names 78.08%
Inter-annotator agreement( κ )
Names 0 .
Names forms 0 .
Time 16 min
Summary
of Dataset
Total Homepages 2,
Total Institutes
Average Institutes 7.
STD.
Institutes 7.
Total Names Indexes 70,
Total Names 34,
Contain Initial 23, Begin with Last Name 22, Begin with Middle Name Begin with First Name 12, word embeddings using GloVe 8 on HomeName , with a window size of 15, minimun vocabulary count of 5, full passes through cooccurrence matrix of 15, and an initial learning rate of 0.05.
Implementation
• CRF: We use the Java implementation provided by the Stanford NLP group 9 with default parameter settings.
The software provides a generic implementation of linear chain CRF model.
• Bi-LSTM-CRF: We implemente Bi-LSTMCRF using Theano 10 and Lasagne 11 .
The word embeddings are fed into a Bi-LSTM layer as input.
Dropout is applied to the output of Bi-LSTM layer to avoid overﬁtting.
The output is further fed into a linear chain CRF layer to predict the tokens labels.
• CogNN: Our Co-guided Neural Network is also implemented on Theano and Lasagne following the description in Section 4.
Dropout is applied on the Bi-LSTM layers.
We use a standard grid search to ﬁnd the best hyperparameter values.
We choose the initial learning rate among [0 . 001 , 0 . 01 , 0 . 1] , the decay rate among [0 . 05 , 0 . 1] , the dimen- edu/software/ sion of hidden layer among [50 , 100 , 200] , the dropout rate among [0 . 2 , 0 . 5]
A. Example Output
Figure 6 shows a sample output of different models.
In the ﬁgure, tokens in italics are the ground truth, while tokens in bold are those predicted as names.
We see that all the baseline models contain wrong predictions while the proposed CogNN model successfully recognise all the names.
Proceedings of the National Academy of Sciences of the United States of America
Kime C , Sakaki-Yumoto M , Goodrich L ,
Hayashi Y , Sami S , Derynck R , Asahi M , Panning B , Yamanaka S , Tomoda K
Activators and repressors : A balancing act for Xinactivation .
CogNN ([BIE, FI])
Proceedings of the National Academy of Sciences of the United States of America
Kime C , Sakaki-Yumoto M , Goodrich L ,
Hayashi Y , Sami S , Derynck R , Asahi M , Panning B , Yamanaka S , Tomoda K
Activators and repressors : A balancing act for Xinactivation .
Bi-LSTM-CRF (BIE)
Proceedings of the National Academy of Sciences of the United States of America
Kime C , Sakaki-Yumoto M , Goodrich L ,
Hayashi Y , Sami S , Derynck R , Asahi M , Panning B , Yamanaka S , Tomoda K
Activators and repressors : A balancing act for Xinactivation .
CRF (BIE)
Proceedings of the National Academy of Sciences of the United States of America
Kime C , Sakaki-Yumoto M , Goodrich L ,
Hayashi Y , Sami S , Derynck R , Asahi M , Panning B , Yamanaka S , Tomoda K
Activators and repressors : A balancing act for Xinactivation .
[Figure]