Selecting Key Views for Zero-Shot Entity Linking
Authors: Xuhui Sui
Ying Zhang
Kehui Song
Baohang Zhou
Xiaojie Yuan
Wensheng Zhang
Venue: EMNLP 2023 Findings (2023)
Keywords: Zero-shot entity linking
Multi-view

Abstract: Entity linking, which aligns mentions in the text to entities in knowledge bases, is essential for many natural language processing tasks. Considering the real-world scenarios, recent research hotspot of entity linking has focused on the zero-shot setting, where mentions need to link to unseen entities and only the description of each entity is provided. This task challenges the language understanding ability of models to capture the coherence evidence between the mention context and entity description. However, entity descriptions often contain rich information from multiple views, and a mention with context only relates to a small part of the information. Other irrelevant information will introduce noise, which interferes with models to make the right judgments. Furthermore, the existence of these information also makes it difficult to synthesize key information. To solve these problems, we select key views from descriptions and propose a KVZEL framework for zero-shot entity linking. Specifically, our KVZEL first adopts unsupervised clustering to form sub views. Then, it employs a mention-aware key views selection module to iteratively accumulate mention-focused views. This puts emphasis on capturing mention-related information and allows long-range key information integration. Finally, we aggregate key views to make the final decision. Experimental results show the effectiveness of our KVZEL and it achieves the new state-of-the-art on the zero-shot entity linking dataset.

Selecting Key Views for Zero-Shot Entity Linking
Xuhui Sui', Ying Zhang', Kehui Song',
Baohang Zhou', Xiaojie Yuan', Wensheng Zhang" College of Computer Science, VCIP, TMCC, TBI Center, Nankai University, China Institute of Automation, Chinese Academy of Sciences, China {suixuhui, songkehui, zhoubaohang nankai edu. cn (yingzhang, (uannj)eeankee edu. cn, zhangwensheegghhomaii.cccc com
Abstract
James has won four NIIA clamponsiips, four NBA MVP awardss four NBA Finals PVPanssds three All-Star MVP awards.
the movie Space New Legacy which starting
Entity linking, which aligns mentions in the text to entities in knowledge bases, is essen- tial many natural language processing tasks. Considering the real-world scenarios, recent re- search hotspot of entity linking has focused on the zero-shot setting, where mentions need to link to unseen entities and only the description each entity is provided. This task challenges the language understanding ability of models to capture the coherence evidence between the mention context and entity description. How- ever, entity descriptions often contain rich in- formation from multiple views, and a mention with context only relates to a small part of the information. Other irrelevant information will introduce noise, which interferes with models make the right judgments. Furthermore, the existence of these information also makes it dif- ficult to synthesize key information. solve these problems, select key views from de- scriptions and propose a KVZEL framework for zero-shot entity linking. Specifically, our KVZEL first adopts unsupervised clustering to form sub views. Then, employs a mention- aware key views selection module iteratively accumulate mention-focused views. This puts emphasis on capturing mention relattd infor- mation and allows long-range key information integration. Finally, we aggregate key views to make the final decision. Experimental results show the effectiveness of our KVZEL and it achieves the new state-of-the-art on the zero- shot entity linking dataset.
father,
LeSron, Jumes
LaBron Reymone James St. is Alnericnn professional besketball National Hoskethall Association (NBA)...
View
1
James mamted high swoethoat Savamak James, They theee chilitien: two sont-Bronng and Bryce and deighter Zhut....
james and his bustness parnerr Mavenck Carter owa production company SpringHill Foteraimment....
View
He his 19 ESPY
Sotumtay Night
View 4
stared sports Space Jan:. Nev Logacy (2021)...
An example of entity description with many views.
Mention with context
Mention with content
Sub
Descripioon
Description
Sub views
Mention with contect
Mantion conkett
(b) Other methods and our proposed framework.
Figure 1: The motivation illustration of our proposed framework. Usually, an entity can be linked by different mentions from multiple views. Our proposed frame- work comes from this observation and selects key views from entity descriptions better align mentions with divergent context.
such as semantic search (Blanco al., 2015), con- tent analysis (Huang et al., 2018) and question an- swering (Welbl et al., 2018; et al., 2020). Tradi- tional EL approaches are built on the assumption that the train and test set share the same entity dis- tributions, which means that linked entities in the test set are seen during training. However, in many real-worle scenarios, the process of labeling data is labor-intensive and error-prone. More importantly, for some specialized domains such as medical and legal domains, the training data is sensitive or pro- prietary. Thus, it not easy to obtain labeled data for these domains. Therefore, EL models need to
1 Introduction
Entity linking (EL) is the task of assigning ambigu- ous mentions in textual input to their corresponding entities in given real-world knowledge base. EL, fundamental task in the information extraction area, plays important role in many downstream natural language processing (NLP) applications,
"Corresponding author.

---

have the capability of generalizing new entities of new domains.
to group the sentences, and those in each cluster compose sub view. When judging whether a mention text matches the entity, humans tend to read the whole description and only focus on the view that is most relevant to the mention. Consid- cring that this mention may be related to multiple views of entity description, after finding that a sin- gle view is insufficient to make the judgment, hu- mans will use the previously relevant views clues and move on to find the next relevant view. With the inspiration of the human's 's thinking process, our KVZEL designs a mention-awar key views selce- tion module to iteratively accumulate the most rele- vant views. These views are then aggregated into a new text, which serves as the dention-focuseee and highly-condensed version of original entity descrip- tion. Finally, the text replaces the description and is utilized by our entity linking module to make the final decision.
To address this, the zero-shot entity linking task (Logeswaran et al., 2019) has been proposed, where mentions need to be linked to unseen entities of new domains. Moreover, considering that there is no external knowledge (e.g. frequency statistics and structured data) in many specialized domains, the textual description for each entity is the only infor- mation provided in this task. Based on the above, we can see that the entity description, as the only identification information of entities, is crucial for this task. Some previous works (Yao et al., 2020; Tang et al., 2021) have realized its importance and constructed models to circumvent the length limi- tation problem of BERT-based models to capture richer entity description information.
However, to the best of our knowledge, all previ- ous methods utilize the same entire entity descrip- tion to match different mentions with divergent context. This seems struggling to manage entities with rich multiple views information. We argue that only part of description information related mentions and other irrelevant information will introduce noise and make models reduce emphasis on the corresponding details. An example shown in Figure la, the entity description of "LeBron James" contains multiple views information, such as his NBA journey, his family, etc. For the first mention, it only relates to "View 1" in the entity description. Other views are useless in determining whether the "James" should link the entity "Le- Bron James". : For the second mention, it related to "View 2" and "View 4". However, the descrip- tion of "LeBron James" mainly concentrates on his professional journey, which will introduce noise to interfere with entity linking models matching "his father" and "LeBron James". Furthermore, "View 2" and "View 4" may exist in different parts of the description, which makes it not trivial to synthesize the key information of the two views.
To summarize, our major contributions are shown as follows:
To the best of our knowledge, this work is the first to select key views from entity descrip- tions to link different mentions with divergent context. This is very meaningful for entities with rich multiple views information.
We propose a novel framework KVZEL for zero-shot entity linking, which imitates the human's 's thinking process to iteratively accu- mulate mention-related views. These views are aggregated to measure the matching score between mentions and entities.
Experimental results on zero-shot entity link- ing dataset (Logeswaran et al., 2019) show that our KVZEL achieves new state-of-the-an performance. Further analysis demonstrates the effectiveness of our framework.
2 Related Work
2.1 Entity Linking
To address these problems, in this paper, we try to select key views from entity descriptions for different mentions with divergent context, which is shown in Figure lb. This can also help entity linking models solve the problem of length limi- tation in BERT-based models, since the selected views are only part of entity description. We pro- pose a novel Key Views selection framework for Zero-shot Entity Linking in this paper, KVZEL in short. KVZEL first adopts unsupervised clustering
Entity linking, as a bridge between real-world text and standard knowledge base, has been widely ex- plored in recent years. Many entity linking meth- ods (Sun et al., 2015; Yamada al., 2016; Le and Titov, 2018; Yang al., 2019; Tedeschi al., 2021; Barba al., 2022) have achieved great success in both academic research and industrial applications. While these results are impressive, the models in these methods are learned under the setting where

---

Mention-Aware Key Views Selection
Clues
0.05
Clues
V.
[SEP]
[CLS] me
[SEP]
W
0.78
A Argmax
Clues
V
SEP
0.10
[CLS] me
[SEP] t
BERT
Clues
SSEP]
Z
Match
[CLS] me
[SEP]
X
FFNN
Clues
|SEP]
No Match
Description
Clustering
U
S1
[CLS] me [SEP] V. V22 [SEP]
BERT
BERT
[CLS] me [SEP] Vo [SEP]
St
Scores
FFNN
Sub View Clustering
Entity Linking
Figure 2: The overall architecture of our KVZEL framework, which includes three modules: a sub view clustering module, mention-aware key views selection module and entity linking module.
linked entities in the test set are available during training. Furthermore, with the expansion of knowl- edge in knowledge bases, many state-of-the methods (Ganea and Hofmann, 2017; Kundu et al.. 2018; Raiman and Raiman, 2018; Orr et al., 2021) I) have made full of external knowledge. How- ever, in many real-world scenarios, labeled data and external knowledge are not easily available in many specialized domains. Based on empirical evidence, the performance of models deteriorates substantially when dealing with new entities of new domains without external knowledge. This moti- vates many researchers to study zero-shot entity linking (Logeswaran al., 2019).
length. Yao et al. (2020) solves the problem by ini- tializing larger position embeddings to expand the effective reading length of BERT. Tang et al. (2021) proposes bidirectional multi-paragraph reading model which segments long description into short paragraphs make use of more description infor- mation. However, the consideration of paragraphs that are irrelevant to the mention may introduce noise in the final representation and limit the long- range dependencies between relevant information. This motivates us to select key views from entity descriptions for different mentions with divergent context address these problems.
Methodology
2.2 Zero-Shot Entity Linking
Figure 2 shows the overall architecture of our pro- posed KVZEL framework. We first use the sub view clustering module to generate sub views from descriptions. Then design mention-awee key views selection module to select relevant sub views for different mentions with divergent context. Fi- nally, our entity linking module makes the final decision based on the selected views. In this sec- tion, we introduce these three modules in detail.
shoo entity linking is the task where mentions must be linked to unseen entities and only the de- scription of each entity is available. It consists of two phases: candidate generation (Wu et al., 2020; Sun et al., 2022; Sui et al., 2022) and candidate ranking (Yao et al., 2020; Tang al., 2021). Fol- lowing previous works (Logeswaran et al., 2019; Yao et al., 2020; Tang et al., 2021), we only fo- cus on the candidate ranking phase and use the traditional Information Retrieval (IR) approach of BM25 (Robertson et al., 1994) in the candidate generation phase. Logeswaran et al. (2019) uses cross-encoder architecture that concatenates men- tion context and entity description and feeds it into BERT (Devlin al., 2019) produce the match- ing score by performing cross-attention. However, BERT-based models may lose important descrip- tion information due to the limitation of their input
3.1 Sub View Clustering
The goal of our sub view clustering module is to cluster sentence chunks describing the same view together,
to select key views later. For each given entity description, we first segment it into sentence chunks S [$1 each individual sentence in the description is taken as a chunk. However, the understanding of short sentences often not easy and needs the help
.... sz.) Si. Intuitively,

---

context, A simple solution is to segment the entire description into chunks of fixed length. However, this destroys the syntactic relationship and may lead to clustering errors. Thus, we fuse these two ways and set a fixed capacity c for each chunk. Each chunk loads as many sentences as possible. After loading sentences, the sentence sequences in chunks are padded to with zero values.
[ENT] the special token to separate entity title and entity description. The clues are concatenated with the n views separately and special separator token [SEP] is added at the end. We feed them into a BERT encoder (fBERT,) to extract features. It pro- duces matrix representation X = [z] 22. ..... [nl] to represent the clues-view pairs, where E Ied is the output of the last hidden layer corresponding to the position of the [CLS] token for the i-th view and d is the dimension of hidden states of BERT.
We feed the sentence chunks S into BERT (De- vlin al., 2019) in the form of [CLS] 8; [SEP] extract features, which are the vectors in the last hidden layer corresponding to the position of [CLS] tokens. Then, we adopt mean-shift (Cheng, 1995) clustering approach to group sen- tence chunks. Compared to other traditional unsu- pervised clustering approaches (such as k-means (Lloyd, 1982), spectral clustering (Ng al., 2001), etc), mean-shift does not need specify the num- ber of clusters, which is more suitable for our ap- plication scenario. Those sentence chunks in each cluster compose a sub view:
3.2.2 Module Training
Since the process of labeling data is labor-intensive and error-prone, we do not have any labels about key views. Therefore, we introduce matching task to enable module learn relevance scores between mentions and views. We argue that views that are more relevant the mention can contribute more to determining whether the mention and the candidate entity are matched.
Specifically, we use a trainable weight parameter Wa E RIxd as one of the inputs and utilize the SoftMax function to calculate the attention scores of the matching result with regard to the views:
= Mean-Shift(BERT((S BBERT,))
3.2
Mention-Aware Key Views Selection
The goal of our mention-aware key views selection module is to iteratively accumulate mention-related views for the final entity linking process. This is similar to human reading through the whole de- scription view by view to measure the relevance of the mention and cach view. Then they will focus on the view that is most relevant to the mention. When moving on to find the next relevant view, they will still remember previously relevant views and utilize them clues to help this process.
G = SoftMax (WoX (W.X))
where G RR Ixn, Then, we utilize the weighted aggregation with the attention scores to obtain the mention-entity representation: = GX,, where Z E Rd. Finally, we get the binary prediction ý by feeding the representation into feed-forward neural network FFNN; = FFNN Or₂). The prediction measures whether the mention and the candidate entity matched.
We employ cross-entropy as our loss objective, which calculated as follows:
3.2.1 Feature Extraction
For given mention m. suppose there are P can- didate entities of m and we have n sub views V = [Vi V22,.. ...,. Vn] of the p-th candidate entity ep- In the initial stage, our clues only consist of the mention m and the entity title:
Cseiection = (ylogg + (1 yy)oog(1 -)) (2)
where takes the value 1 if the mention and the candidate entity are matched.
3.2.3 Key Views Selection
[CLS] me [SEP] L
(1)
During the matching task training, the vector G learns to measure the degree to which each view contributes to determining whether mentions and entities are matched. In the inference stage, we use it as the relevance scores between clues and all views of the candidate entity and utilize an Argmax function to obtain the most key view:
where me and are the inputs of the mention m and the entity title. Following Wu et al. (2020), me and t are constructed as: me ol [Ms] mention [Me] er and 1 litle [ENT], where mention, cl, cr, title are the word-piece tokens of the mention, context before and after the mention and the entity title respectively. [Ms] and [Me] are special tokens to tag the mention, and
Argmax
(3)

---

Algorithm 1:
The key views selection pro- of our KVZEL.
Domains
Length
Entities
Mentions
Training
Input: The mention with context m, candidate entity Cp. candidate view list V of Ep. number of iterations Q.
American Football
665.06
31929
3898
Doctor Who
264.14
40281
8334
Fallout
229.68
16992
3286
Output: The selected view list V.
Final Fantasy
497.35
14044
6041
Military
870.55
104520
13063
Construct initial clues by Eq. 1.
Pro Wrestling
639.53
10133
1392
2 for each iteration = to Q do Calculate Leeeectin by Eq. 2 and train our key views selection module.
StarWars
379.76
87056
11224
World of Warcraft
242.16
27677
1437
Validation
Calculate the most key view V by Eq. in the inference stage.
Coronation Street
264.55
17809
1464
Muppets
161.50
21344
2028
Icc Hockey
282.62
28684
2233
Add V to existing clues and the selected view list V.
Elder Scrolls
269.03
21712
4275
Test
Remove Vi from the candidate view list V.
Forgotten Realms
257.29
15603
1200
Lego
223.86
10076
1199
end return
Star Trek
393.21
34430
4227
YuGiOh
643.68
10031
3374
Table I: Overall statisties of the 7eshel dataset, which
consists of the average length documents, the number of entities and labeled mentions.
where A is the view marker. Then, we add this view existing clues. And we will
Va (e.g. V/2 not consider the view V₁ in the next iteration. We summarize the key views selection process of our KVZEL in Algorithm 1.
entity is calculated using a SoftMax function:
exp(8p)
8 = FFNN(UP;::; 0F)) 8p
3.3
Entity Linking
2j=1 exp(3,)
The goal of our entity linking module is to aggre- gate key views into a new text des to make the final decision. Considering that the first view sentences (view Vo) are able to generally describe the entity, we first add the first view Vo to dés to enhance the overall understanding of the entity. Then sort the selected key views maintain the relative ordering the original description to preserve the syntactic dependencies. The key views are then added to des one by one. The new text des serves as the mention m focused highly-condensed ver- sion of the entity description of ep. Finally, des replaces the original entity description and is read by entity linking module.
where P is the number of candidate entities of the mention We employ cross-entropy loss objective, which is calculated as follows:
C Linking
Σ p=1
yp log Sp + +(1 +11 - yp) log sps
where Up E (0, 1} and takes the value 1 if the candidate entity 6p is the gold entity of the mention otherwise it takes the value 0.
Experiments
4.1 Dataset
Following Wu et al. (2020) and similar to Eq. 1, the input text of the mention m and the candidate entity ep is constructed as: [CLS] me [SEP] t des [SEP]. Then the input text is entered into BERT encoder (θBERT,) to produce the vector represen- tation Up, which is the output of the last hidden layer corresponding the position of the [CLS] token. Finally, the representation Up input into a ferw-forward neural network FFNN and the score &p between the mention m and its p-th candidate
Following recent zero-shot entity linking works (Logeswaran et al., 2019; Wu et al., 2020; Yao et al., 2020; Tang et al., 2021), we evaluate our frame- work KVZEL under the Zeshel dataset, ! which is constructed by Logeswaran et al. (2019). Overall statistics of the dataset are shown in Table The dataset was built based on the documents in Wikia. ? Wikia consists of encyclopedias, each specialized
'tttps://gi com/ com//aajnugen/zeshee Phttps://www wikia.com com

---

Methods
Forgotten Realms
Lego
Star Trek
YuGiOh
Macro Acc.
Micro Acc.
85.60
76.90
75,80
67.22
76.38
74.21
Prepeat
77.58
Uni-MPR
85.55
77.42
78.23
68.29
77.62
75.78
Bi- Si-MPR
89.09
77.18
79.20
69.98
78,61
76.70
KVZEL w/o iteration
87.10
76.90
78.42
71.36
78.45
77.35
KVZEL w/o Vo
88,30
76.08
78.96
73.41
79.19
78.24
KVZEI. (Ours)
88.80
77.93
80.65
73.50
80.22
79.30
Table 2: Results on the test set of Zeshel dataset. Macro Acc. represents the average accuracy of these four test domains. Miero Acc. represents the weighted average accuracy of these four domains. In the results, the highest values in bold and the results unavailable are left blank. All scores averaged rums using different random seeds, and our results over all baselines are statistically significant with < 0.05 with the t-test.
in a particular subject such as sports and film series. Each particular subject can be considered as a do- main. Zeshel consists of specialized domains, domains for training, 4 for validation, and for test. Each domain has its individual entity dictionary, allowing the performance evaluation entire un- seen entities of new domains. The training set has 49,275 labeled mentions while the validation and test sets both have 10,000 unseen mentions.
the cross-encoder (Logeswaran et al., 2019; Wu et al., 2020), which feeds the concatenation of men- tion context and entity description to BERT and out- puts the [CLS] token's 's embeddings to produce the relevance scores. Erepeat (Yao et al., 2020) extends the effective reading length of BERT by repeating the position embedding to solve the long rongerange modeling problem in entity descriptions. Uni-MPR (Tang et al., 2021) segments long description into short paragraphs to make use of more description information. Bi-MPR (Tang al., 2021) is based on Uni-MPR, which segments mention context into short paragraphs to incorporate more mention infor- mation. Note that we do not consider any domain- adaptive techniques in our KVZEL and comparison methods, which improves the final performance but outsides work scope.
4.2 Implementation Details
For a fair comparison, we use the BERT-basc- uncased (Devlin et al., 2019) as our encoder and use the accuracy as our evaluation metric. For the sub view clustering module, the capacity c is set to 64. For the mention-aware key views selection module, following Tang et al. (2021), set the maximum sequence length of the mention context be 256. We train this module epoch per iter- ation using a batch size of and a learning rate of 2e-5. The number of iterations Q is set to 3, which can get the best results. We use the AdamW (Loshchilov and Hutter, 2017) optimizer to opti- mize our module. For the entity linking module, also following Tang et al. (2021), the max length for the mention context and entity description are both set to 256. We train this module in 5 epochs. All parameters are also optimized by Adam W, with learning rate 2c-5, and 10% warmup steps. Exper- iments were conducted on two NVIDIA GeForce RTX A6000 GPUs with 48 GB memory.
4.4 Overall Performance
The comparison results on the Zeshel datasct are shown in Table 2. We can observe that our KVZEL outperforms all baseline methods on average, il- lustrating the effectiveness of our KVZEL and the superiority of selecting key views for different men- tions with divergent context. It's worth noting that all baselines perform worse on the YuGiOh do- main. We conjecture that this is because the entities in this domain has more views since we find that the entities with longer descriptions contain more views. This consistent with our claim that pre- vious methods seem struggling to manage entities with rich multiple views information. We observe that KVZEL makes significant improvement on the domains (e.g. Star Trek and YuGiOh) whose descriptions have rich multiple views information. Our K VZZE avoids the noise brought by irrelevant view information and allows long-range key infor- mation integration, which is very effective for these
4.3 Comparison Methods
For the quantitative evaluation of our KVZEL, we use the following state-of-the-art methods (Lo- geswaran et al., 2019; Wu al., 2020; Yao al., 2020; Tang et al., 2021) for comparison. All of these methods including our KVZEL are based on

---

90
Length
Number
Proportion
88
(0, 200)
2752
40.40%
Ave.
Miero Star Trek-
[200, 500)
2025
29.73%
84-
Forgotten Realms-
Lego
YuGiOb
[500, 1000)
954
14.00%
82-
1000, 2000)
554
8.13%
50-
20000, +00)
527
7.74%
76-
Table 3: The statistics of entities with different descrip- tion lengths. We partition the entities by the tokens number in entity descriptions.
74
70-
2
5
Number of Views
90
Cross-enonder
Figure 3: The effect the number of selected views. Dashed lines indicate the accuracy on the four test do- mains and solid lines indicate the average accuracy.
KVZEL
80
Accuricy
70
domains. general, our KVZEL outperforms the baseline methods for 1.61% % and 2.60% on Macro Acc. and Micro Acc. on the test set of Zeshel dataset, respectively.
50
200)
[200, 500)
[500, 1000) (1000.2000)
5 Analysis
Length of Entity Descroptions
5.1
Ablation Study
Figure 4: The effect of cross-encoder and our KVZEL on entities with different description lengths. We calcu- late metrics based the partition in Table 3.
To better understand our KVZEL framework, investigate the contribution of some components through ablation studies, and the results are also shown in Table 2. Considering that our entity link- ing module is based on cross-encoder, the compari- son results to cross-encoder could also be regarded the ablation study to justify the advantage of KVZEL. To evaluate the contribution of iter- atively accumulating key views, we train the key views selection module only once and select the same number of key views for the entity linking module. We find that this leads to significant performance drop, which demonstrates the effec- tiveness of utilizing the previously selected views clues to move on to find the next key view. We then do not add the first view Vo to des and only consider the selected views in our entity linking module. It can be observed that KVZEL w/o Vo performs worse than KVZEL. This indicates that the adding of Vo helps our entity linking module understand entities better, since we observe the first view sentences of most entities are able to generally describe the entity.
can find that the required number of selected views changes with different domains. For domains with rich views in entity descriptions (c.g. YuGiOh), the integration of more information from multiple views becomes particularly important. While for domains whose views are not as rich (e.g. Forgot- ten Realms, Lego), single or two views are suffi- cient to help our entity linking module to correctly link to gold entities. In general, we can observe that our KVZEL with 3 selected views performs best average. If the number of selected views is too small, there will be enough information to help our entity linking module make the correct decision. Conversely, if this number is too large, this will introduce noise from irrelevant views and cause the performance to drop.
5.3 Effect of Description Length
Through our observation, we find that the enti- ties with longer descriptions contain more views. To further show the effectiveness of our proposed KVZEL framework, evaluate the effect of cross- encoder and our KVZEL on entities with different description lengths. The statistics entities on the test set of Zeshel dataset are shown in Table and
5.2
Effect of Number of Selected Views
We also explore the effect of different numbers of selected views on our KVZEL performance. The experimental results are shown in Figure 3. We

---

Mention with context
Description input baselines
Deseription input KVZFI.
Lucifer reuses the cat single-stud mould introduced for the Friends theme. It fea- tures Lucifer stunding up with bent back, clevated hend looking struight ahead and long tail lying flat curled...
The first selected view: The cal is a single-stud mould, which features it standing up with bent back, clevated head looking straight ahead und long tail lying flat curled around the left side...
The Cat is Friends animal figure intro- duced 2013. The black one, released in the first series of Friends Animals, be- came available even in December 201
The first selected view: Akiza chose to leave the academy due to it's eli- tist ways, which made Sherry feel be- trayed and harbor a hatred for Akiza... The second selected view: After the Duel, Sherry and Akiza become friends again after Sherry questions Akiza on where she learned to drive a Duel Run- ner with her eyes closed...
Having just defeated Sherry, Akiza walks over to her. Sherry protests that Akiza ran away from Academy. Ak- iza admits that she ran away, and ac- knowledges that Duel Academy Sane- Iuary may loster elite... She usks Akiza where she learned the trick of riding her Duel Runner with her eyes shut...
Sherry LeBlanc is a Psychic Duelist with an ability referred to as Hand Scan, in which she can view her upponent's 's cur- rent hand. Design. Appearance, Sherry has long, waist length, blond hair with bungs that curl away from her lace a horizontal direction and has emerald color eyes...
Table 4: Qualitative analysis of test samples in Zeshel dataset. We analyze the gold entity description input of baselines and our KVZEL. The mentions in text are written in bold.
the experimental results are shown in Figure 4. We can find that the cross-encoder (without selecting key views) obtains passable performance on enti- ties with few views but fails to manage those multi- view entities the length of descriptions increases. For example, the accuracy of cross-encoder is over 80% for entities with 0-500 description tokens but about 60% for entities with over 1000 description tokens. This is the limitation for zero-shot entity linking, Our KVZEL avoids the noise brought by irrelevant view information and allows long-range key information integration by iteratively accumu- late key views. This helps our framework manage the entities with rich multiple views information. Our KVZEL significantly improves the accuracy to 71.66% (+10.29%) and 66.98% (+12.33%) for entities with 1000-2000 tokens and over 2000 to- kens, respectively. Furthermore, considering its ability to avoid noise brought by irrelevant views information, our KVZEL also slightly improves the performance on entities with few views, which achieves the accuracy of 80.67% (+0.26%) and 83.21% (+2.12%) for entities with 0-200 tokens and 200-500 tokens respectively.
of "Catherine Cat" and "Sherry LeBlanc" like base- lines. This indicates that baseline methods can only capture the rough meaning of a description from a general view, while our KVZEL is able to dive into the descriptions and capture more fine- ine-grained mantic information from key views. Furthermore, for the second example, the mention with context is related to multiple views. A key observation is that our KVZEL can precisely select key views to aggregate to make the final decision, which indi- cates that our KVZEL has the ability of long-range key information integration.
Conclusion
In this paper, we explore an issue that was over- looked when only textual descriptions are avail- able in zero-shot entity linking scenarios, that is, only part of description information is related to mentions and other irrelevant information will in- troduce noise and make models reduce emphasis on the corresponding details. To overcome this is- sue, propose a KVZEL framework to select key views from entity descriptions for different men- tions with divergent context. Specifically, it first adopts unsupervised clustering to form sub views. Then, it imitates the human's thinking process to it- eratively accumulate mention-related views, which helps our entity linking module have the ability of long-range information integration. The selected views are aggregated into new text, which places the original description as the input of our entity linking module make the final decision. Experimental results demonstrate that the proposed KVZEL framework achieves new state-of-the-ard performance on the zero-shot entity linking dataset.
5.4 Case Study
To conduct a qualitative analysis, Table 4 shows two examples from the test set of Zeshel dataset. While the baseline methods tend to input the en- tire entity descriptions, our KVZEL only inputs the mention-focused views of entity descriptions. This indeed helps our framework avoid noise brought by irrelevant information and enhances emphasis on the corresponding details. For these two exam- plcs, our KVZEL can point to the gold entities "Cat (Friends)" and "Sherry LeBlanc (manga)" instead

---

Limitations
2018. Entity-based query recommendation for long- tail queries. ACM Trans. Knowl. Discov. Data. 12(6):44:1-64:44
Although our KVZEL achieves new state-of-the-an for zero-shot entity linking, it also poses several limitations follows: 1) In this work, for simplic- ity, we only use the traditional unsupervised cluster- ing approach meen--hiff to form sub views. How- ever, other state-of-the-arr clustering approaches also can be used and may work better. We leave the selection of a more effective cluster approach to future work. 2) In this work, we only focus on the zero-shot entity linking setting, which is low-resource scenario and only provides the textual description information for each entity. This task challenges the ability of entity linking models to understand the language consistency between men- tion contexts and entity descriptions. Our KVZEL improves this ability by selecting key views from entity descriptions. Therefore, when our KVZEL is extended to other entity linking settings which may have much external knowledge (e.g. frequeney statisties and structure data, etc.) and not focus on the language understanding ability so much, the improvement of performance may be insignificant.
Gourab Kundu, Avirup Sil. Radu Florian, and Wael Hamza. 2018. Neural cross-lingual coreference res- olution and its application to entity linking. In ACL. pages 400-
Phong Le and Ivan Titov. 2018. Improving entity link- ing by modeling latent relations between mentions. In ACI, pages 1595-1604.
Belinda Z. Li, Sewon Min, Srinivasan Iyer, Yashar
Mehdad, and Wen-tau Yih, 2020. Efficient one-pass end-to-end entity linking for questions. In EMNLP, pages 6433-6441.
Stuart P. Lloyd. 1982. Least squares quantization in PCM. IEEE Trans. Inf. Theory, 28(2):129
Lajanugen Logeswaran, Ming- Wei Chang, Kenton Lee,
Kristina Toutanova, Jacob Devlin, and Honglak Lee. 2019. Zero-shot entity linking by reading entity de- scriptions. In ACL, pages 3449-3460.
Nlya Loshchilov and Frank Hutter. 2017. Fixing weight decay regularization in adam. CoRR,
abs/1711.05101.
Andrew Y. Ng, Michael Jordan, and Yair Weiss, 2001. On spectral clustering: Analysis and an algorithm. In NeurlPS, pages 849-856.
Acknowledgments
This research is supported by the National Natu- ral Science Foundation of China (No. 62272250, 62002178), the Natural Science Foundation of Tianjin, China (No. 22ICIQIC0010).
Laurel J. Orr, Megan Leszczynski. Neel Guha, Sen Wu, Simran Arora, Xiao Ling, and Christopher Ré. 2021. Bootleg: Chasing the tail with sulf-supervised named entity disambiguation. In CIDR.
Jonathan Raiman and Olivier Raiman. 2018. Deeptype: Multilingual entity linking by neural type system evolution. In AAAÍ, pages 5406-5413.
References
Edoardo Barba, Luigi Procopio, and Roberto Navigli. 2022. Extend: Extractive entity disambiguation. In ACL, pages 2478-2488
Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. Okapi at TREC-3. In TREC, pages 109-126.
Xuhui Sui, Ying Zhang, Kehui Song, Baohang Zhou, Guoqing Zhao, Xin Wei, and Xiaojie Yuan. 2022. Im- proving zero-shot entity linking candidate generation with ultra-fin fine entity type information. In COLING. pages 2429-2437.
Roi Blanco, Giuseppe Ottaviano, and Edgar Meij. 2015, Fast and apficceefficeent entity linking for queries. In WSDM, pages 179-188.
Yizong Cheng, 1995. Mean shift, mode seeking, and clustering. IEEE Trans. Pattern Anal. Mach. Intell., 17(8):700-799.
Kai Sun, Richong Zhang, Samuel Mensah, Yongyi Mao. and Xudong Liu. 2022. A transformational biencoder with in-domain negative sampling for zero-shot entity linking. In Findings of ACL, pages 1449-144 1458.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In NAACI, pages 4171-4186.
Yaming Sun, Lei Lin, Duyu Tang, Nan Yang, Zhenzhou Ji, and Xiaolong Wang. 2015. Modeling mention, context and entity with neural networks for entity disambiguation. In IJCAI, pages 1333-1339.
Octavian-Eugen Ganea and Thomas Hofmann. 2017. Deep joint entity disambiguation with local neural attention. In EMNLP, pages 2619-2629.
Hongyin Tang, Xingwu Sun, Beihong Jin, and Fuzheng Zhang. 2021. A bidirectional malti-par read- ing model for zero-shot entity linking. In AAAI. pages 13889-13897.
Zhipeng Huang. Bogdan Cautis, Reynold Cheng, Yudian Zheng, Nikos Mamoulis, and Jing Yan.

---

Simone Tedeschi, Simone Conia, Francesco Cecconi. and Roberto Navigli. 2021. Named entity recognition for entity linking: What works and what's s next. In Findings of EMNLP, pages 2584-2596.
Johannes Welbl. Pontus Stenetorp. and Sebastian Riedel. 2018. Constructing datasets for multi-hop reading comprehension across documents. Trans. Assoc. Comput. Linguistics, 6:287-302.
Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. Scalable zero- shot entity linking with dense entity retrieval. In EMNLP, pages 6397-6407.
Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, and Yoshiyasu Takefuji. 2016. Joint learning of the em- bedding of words and entities for named entity dis- ambiguation. CoNLL, pages 250-259.
Xiyuan Yang, Xiaotao Gu, Sheng Lin, Siliang Tang, Yueting Zhuang, Fei Wu, Zhigang Chen, Guoping Hu, and Xiang Ren. 2019. Learning dynamic context augmentation for global entity linking. EMNLP, pages 27 1-28..
Zonghai Yao. Liangliang Cao, and Huapu Pan. 2020. Zero-shot entity linking with efficient long range quence modeling. Findings of EMNLP, pages 2517-2522.


Reviews:
[{"review_id": "9RMnpXBcoe", "reviewer": "Reviewer_7XKJ", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and sincerely appreciate your positive feedback. We would like to address the specific questions below.\n\n1.Single approach to capture sub-view information\n\nIn practice, we have explored many other solutions. However, due to the spatial limitations, we only present the most effective solutions in the manuscript. In our sub view clustering module, we tried different solutions to form sentence chunks. In our mention-aware key views selection module, we explored several heuristic methods. In our entity linking module, we explored multiple solutions at both the feature level and sentence level to aggregate key views. We will include a detailed description of these solutions in the appendix of our revised version to provide more information.\n\n2.Missing references about entity aspect linking\n\nOur proposed method differs from entity aspect linking. In entity aspect linking setting, each entity has multiple predefined aspects. The goal of entity aspect linking is to link a mention to one of these aspects to obtain more fine-grained information. In contrast, our method aims to select mention-related views of each entity and aggregate them to determine whether the entity matches the mention without irrelevant information noise. We will describe the difference in the related work section of our revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- (+) A novel approach to include aspect entity in a transformer-based zero-shot EL framework.\n- (+) Thorough evaluation and competitive results.", "reasons_to_reject": "- (-) Single approach to capture aspect / sub-view information (since this is the heart of the paper, it would have been more informative to explore more/different solutions).", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents a methodology to include entity aspect (here called sub view) information for zero-shot entity linking.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Previous work introducing entity aspect linking:\n\n- https://madoc.bib.uni-mannheim.de/44381/1/entity-aspect-linking.pdf \n- https://www.dcs.gla.ac.uk/~shubham/publications/pdf/cikm2022.pdf"}, "extra_text": {}}, {"review_id": "8pxCDbxPzM", "reviewer": "Reviewer_EoeX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and would like to address the specific questions below.\n\n1.Key view selection method connects to and differs from previous works.\n\nThanks for your suggestion. While we acknowledge that clustering sentences and selecting the most relevant ones are not new, we believe our approach is unique in the context of entity linking. To the best of our knowledge, this is the first paper to select mention-related information from entity description in this task. Additionally, our method differs from selecting the most relevant sentence methods by introducing the novel idea of forming sub views and selecting relevant views. Each view contains sentences that describe the same aspect. Utilizing all sentences in the selected views can help our model understand these aspects better to capture more fine-grained semantic information. We will further elaborate on this in the revised version of this paper.\n\n2.Hard coding of V0\n\nConsidering that V0 generally describes the entity, we manually add it to key views to make the final entity linking decision. As you described, it works may because the corpus consists of encyclopedias. However, for other corpora where V0 is not a summative description, there is no necessity to manually add V0 to key views. In such cases, the proposed key views selection method is able to figure out key views automatically.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The challenge is real and important.\n- The method is overall reasonable with good experimental support.", "reasons_to_reject": "- Clustering sentences based on semantics or topics is not new. Selecting the most relevant sentences from a long document is also not new. I recommend the authors review relevant work and discuss how their key view selection method connects to and differs from previous work.\n- The hard coding of V0 into the entity linking module looks unnatural. It works because the corpus consists of encyclopedias. Such hard coding can be questionable on other corpora. I would expect a good key view selection method should be able to figure out V0 automatically if V0 is indeed very useful.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper argues that, when using entity descriptions for entity linking, breaking down long descriptions into multiple key views is better than taking the entire description into a cross-encoder. The authors propose a key view selection algorithm, and show it brings up entity linking performance on a dataset with multiple domains.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "CJpBKU7GD4", "reviewer": "Reviewer_zpya", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful comments and appreciate your positive feedback. We would like to address the specific questions below.\n\n1.Extend to decoder-only models\n\nOur approach can easily be extended to decoder-only models by replacing BERT with GPT. Representations can be obtained by getting the last position output of GPT. However, for a fair comparison, following all previous works, we use the BERT-base-uncased to obtain representations in this paper. Furthermore, GPT uses the decoder-only unidirectional Transformer model, which may limit its ability to capture global information and is mainly used for text generation tasks. On the other hand, BERT employs a bidirectional Transformer model, which allows it to capture contextual information from both left and right contexts. This enables BERT to better understand the semantic and obtain representations. Thus, we only consider using the BERT-base-uncased to obtain representations in this paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Strengths: \n1. The paper is well written and the proposed approach is interesting. \n2. The propose method performs better than the baseline approaches on Zeshel dataset across multiple domains. The ablation results show that the first selected view plays a vital role on the performance of the model. \n", "reasons_to_reject": "Weaknesses:\n1. It is not clear how to extend this approach to decoder-only models which have become more prominent recently.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper focuses on the problem of zero-shot entity linking where only entity description has been provided. The authors argue that only part of the provided description is related to the mention and rest of the part introduce noise and divert the attention of the model from the actual answer. For this, they propose KVZEL, Key View selection framework for Zero-shot Entity Linking. This involves breaking an entity description into multiple sentence chunks which are embedded using an BERT model and are clustered into multiple views using mean-shift clustering approach. For a given mention, a set of views is selected which is then utilized intead of the entity description. This helps to fit the relevant part of description within BERT model max sequence length.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper under review presents a methodology to include entity aspect information for zero-shot entity linking. The proposed approach breaks down long entity descriptions into multiple key views. 

In terms of soundness, overall, the paper is regarded as having a good level of support for its major claims. All three reviewers recognize that the paper contributes novel work to the topic of entity linking. The study has been praised as being well-structured, and having a relevant and important challenge at its core.

Despite the acknowledged strengths of the paper, the reviewers collectively pointed out certain weaknesses. Notably, one reviewer noted that the manuscript could benefit from reviewing relevant previous work in the area of selecting relevant sentences and clustering them semantically. Lastly, a concern was raised about how this approach would extend to autoregressive entity linking.
Decision: Accept-Findings