A dynamic programming algorithm for span-based nested named-entity recognition in O(n2)
Year: 2023

Abstract:
Span-based nested named-entity recognition (NER) has a cubic-time complexity using a variant of the CYK algorithm. We show that by adding a supplementary structural constraint on the search space, nested NER has a quadratictime complexity, that is the same asymptotic complexity than the non-nested case. The proposed algorithm covers a large part of three standard English benchmarks and delivers comparable experimental results.

heading: 1 Introduction
text: Named entity recognition (NER) is a fundamental problem in information retrieval that aims to identify mentions of entities and their associated types in natural language documents. As such, the problem can be reduced to the identification and classification of segments of texts. In particular, we focus on mentions that have the following properties:
1. continuous, i.e. a mention corresponds to a contiguous sequence of words;
2. potentially nested, i.e. one mention can be inside another, but they can never partially overlap.
Four examples are shown in Figure 1. In a span-based setting, recognition for nested NER has a cubic-time complexity (Finkel and Manning, 2009; Fu et al., 2021) using variants of the Cocke-Younger-Kasami (CYK) algorithm (Kasami, 1965; Younger, 1967; Cocke, 1970). If we restrict the search space to non-nested mentions, then recognition can be realized in quadratic time using a semi-Markov model (Sarawagi and Cohen, 2004). An open question is whether it is possible to design algorithms with better time-complexity/search space trade-offs.
In this paper, we propose a novel span-based nested NER algorithm with a quadratic-time complexity, that is with the same time complexity as
the semi-Markov algorithm for the non-nested case. Our approach is based on the observation that many mentions only contain at most one nested mention of length strictly greater than one. As such, we follow a trend in the syntactic parsing literature that studies search-spaces that allow the development of more efficient parsing algorithms, both for dependency and constituency structures (Pitler et al., 2012, 2013; Satta and Kuhlmann, 2013; GómezRodríguez et al., 2010; Corro, 2020), inter alia.
Our main contributions can be summarized as follows:
• We present the semi-Markov and CYK-like models for non-nested and nested NER, respectively — although we do not claim that these approaches for NER are new, our presentation of the CYK-like algorithm differs from previous work as it is tailored to the NER problem and guarantees uniqueness of derivations;
• We introduce a novel search space for nested NER that has no significant loss in coverage compared to the standard one, see Table 5;
• We propose a novel quadratic-time recognition algorithm for the aforementioned search space;
• We experiment our quadratic-time algorithm on three English datasets (ACE-2004, ACE2005 and GENIA) and show that it obtains comparable results to the cubic-time algorithm.

heading: 2 Related work
text: Span-based methods: Semi-Markov models have been first proposed in the generative modeling framework for time-serie analysis and word segmentation (Janssen and Limnios, 1999; Ge, 2002).
10712
Sarawagi and Cohen (2004) introduced a discriminative variant for NER. Arora et al. (2019) extended this approach with a task-tailored structured SVM loss (Tsochantaridis et al., 2004). Inference algorithms for semi-Markov models have a O(n2) time complexity, where n is the length of the input sentence. Unfortunately, semi-Markov models can only recognize non-nested mentions. Finkel and Manning (2009) proposed a representation of nested mentions (together with part-of-speech tags) as a phrase structure, enabling the use of the CYK algorithm for MAP inference. Influenced by recent work in the syntactic parsing literature on spanbased model, i.e. models without an explicit grammar (Hall et al., 2014; Stern et al., 2017), Fu et al. (2021) proposed to rely on these span-based phrase structure parsers for nested NER. As structures considered in NER are not stricto sensu complete phrase-structures, they use a latent span model. Inference in this model has a O(n3) time complexity. Lou et al. (2022) extended this approach to lexicalized structures (i.e. where each mention has an explicitly identified head), leading to a O(n4) time complexity for inference due to the richer structure.
Tagging-based methods: NER can be reduced to a sentence tagging problem using BIO and BILOU schemes (Ratinov and Roth, 2009) to bypass the quadratic-time complexity of semi-Markov models. MAP Inference (resp. marginal inference) is then a linear-time problem using the Viterbi algorithm (resp. forward-backward algorithm).1 However, this approach cannot incorporate span features neither be used for nested entities. Alex et al. (2007) and Ju et al. (2018) proposed to rely on several tagging layers to predict nested entities. Shibuya and Hovy (2020) introduced an extension
1It is quadratic in the number of tags, but we assume the input of the algorithm is the sentence only.
of the Viterbi algorithm that allows to rely on BIO tagging for nested NER by considering second-best paths. To leverage the influence of outer entities, Wang et al. (2021) rely on different potential functions for inner entities. Note that algorithms for the second-best paths method have a O(n2) time complexity, that is similar to the span-based alogithm we propose.
Hypergraph-based methods: Lu and Roth (2015) proposed an hypergraph-based method for nested NER. Although this approach is appealing for its O(n) (approximate) inference algorithms, it suffers from two major issues: (1) the marginal inference algorithm overestimate the partition function; (2) the representation is ambiguous, that is a single path in the hypergraph may represent different analysis of the same sentence. Muis and Lu (2017) proposed a different hypergraph withO(n2) inference algorithms that solves issue (1) but still exhibits issue (2). Katiyar and Cardie (2018) extended hypergraph methods to rely on neural network scoring. Wang and Lu (2018) proposed a novel hypergraph method that fixes issues (1) and (2) but their approach does not forbid partially overlapping mentions.
Unstructured methods: Several authors proposed to predict the presence of a mention on each span independently, sometimes with specialized neural architectures (Xu et al., 2017; Sohrab and Miwa, 2018; Zheng et al., 2019; Xia et al., 2019; Wang et al., 2020; Tan et al., 2020; Zaratiana et al., 2022), inter alia. Note that these approaches classify O(n2) spans of text independently, hence the time-complexity is similar to the approach proposed in this paper but they cannot guarantee wellformedness of the prediction.

heading: 3 Nested named-entity recognition
text: In this section, we introduce the nested NER problem and the vocabulary we use through the paper.

heading: 3.1 Notations and vocabulary
text: Let s = s1...sn be a sentence of n words. Without loss of generality, we assume that all sentences are of the same size. We use interstice (or fencepost) notation to refer to spans of s, i.e. si:j = si+1...sj if 0 ≤ i < j ≤ n, the empty sequence if 0 ≤ i = j ≤ n and undefined otherwise. We denote M the set of possible mentions in a sentence and T the set of mention types, e.g. T = {PER, ORG, GPE, ...}. Without loss of generality, we assume that T ∩ {→,↔, 7→,← [} = ∅. A mention is denoted ⟨t, i, j⟩ ∈M s.t. t ∈ T, 0 ≤ i < j ≤ n, where i (resp. j) is called the left border (resp. right border). An analysis of sentence s is denoted y ∈ {0, 1}M where ym = 1 (resp. ym = 0) indicates that mention m ∈ M is included in the analysis (resp. is not included). For example, the analysis of sentence 1 in Figure 1 is represented by a vector y where y⟨PER,0,1⟩ = 1, y⟨PER,5,8⟩ = 1 and all other elements are equal to zero. A mention ⟨t, i, j⟩ is said to be inside another mention ⟨t′, i′, j′⟩ iff i′ < i < j ≤ j′ or i′ ≤ i < j < j′.
Let y be the analysis of a sentence. We call first level mentions all mentions in y that are not inside another mention of the analysis. We call nested mentions all mentions that are not first level mentions. For example, the first level mentions of the analysis of sentence 2 in Figure 1 are ⟨PER, 0, 1⟩ “this” and ⟨PER, 2, 8⟩ “your second one he has missed”. We call children of mention m ∈ M the set C ⊆ M of mentions that are inside m but not inside another mention that is inside m. Conversely, m is said to be the parent of each mention in C. For example, in sentence 2 in Figure 1, the mention ⟨PER, 2, 8⟩ “your second one he has missed” has two children, ⟨PER, 2, 3⟩ “your” and ⟨PER, 5, 6⟩ “he”. In sentence 4 in Figure 1, ⟨GEP, 5, 6⟩ “Germany” is a child of ⟨GEP, 4, 6⟩ “Bonn, Germany” but it is not a child of ⟨PER, 2, 6⟩“Fabian from Bonn, Germany”. The left neighborhood (resp. right neighborhood) of a nested mention is the span between the left border of its parent and its left border (resp. between its right border and the right border of its parent). For example, in sentence 2 in Figure 1, mention ⟨PER, 5, 6⟩ “he” has left
neighborhood s2:5 “your second one” and right neighborhood s6,8 “has missed”.
The set of possible analyses is denoted Y . We will consider three different definitions of Y :
1. the set of analyses where no disjoint mention spans overlap, corresponding to non-nested NER;
2. the set of analyses where one mention span can be inside another one but cannot partially overlap, corresponding to nested NER;
3. the set 2 with additional constraint that a mention must contain at most one child with a span length strictly greater to one.

heading: 3.2 Inference problems
text: The weight of an analysis y ∈ Y is defined as the sum of included mention weights. Let w ∈ RM be a vector of mention weights. The probability of an analysis is defined via the Boltzmann or “softmax” distributions:
p(y|w) = exp(w ⊤y)
Z(w) ,
where Z(w) = ∑
y′∈Y exp(w ⊤y′) is the partition
function. Note that, in general, the set Y is of exponential size but Z(w) can nonetheless be efficiently computed via dynamic programming.
The training problem aims to minimize a loss function over the training data. We focus on the negative log-likelihood loss function defined as:
ℓ(w,y) = −w⊤y + logZ(w).
Note that this loss function is convex in w. This differentiates us from previous work that had to rely on non-convex losses (Fu et al., 2021; Lou et al., 2022). Moreover, note that the loss function used by Fu et al. (2021) and Lou et al. (2022) requires to compute the log-partition twice, one time with “normal” weights and one time with masked weights. The difference lays in the fact that we will use algorithms that are tailored for the considered search space Y whereas Fu et al. (2021) and Lou et al. (2022) introduced latent variables in order to be able to rely on algorithms designed for a different problem, namely syntactic constituency parsing. Note that the partial derivatives of logZ(w) are the marginal distributions of mentions (Wainwright et al., 2008). Hence, we will refer to computing logZ(w) and its derivatives as marginal inference,
a required step for gradient based optimization at training time.
At test time, we aim to compute the highest scoring structure given weights w:
y∗ = argmax y∈Y p(y|w) = argmax y∈Y w⊤y
We call this problem MAP inference. For many problems in natural language processing, marginal inference and MAP inference can be computed via dynamic programming over different semirings (Goodman, 1999) or dynamic programming with smoothed max operators (Mensch and Blondel, 2018). However, we need to ensure the uniqueness of derivations property so that a single analysis y ∈ Y has exactly one possible derivation under the algorithm. Otherwise, the same analysis would be counted several times when computing the partition function, leading to an overestimation of its value.

heading: 4 Related algorithms
text: In this section, we present semi-Markov and CYK-like algorithms for non-nested and nested NER, respectively. Our presentation is based on the weighted logic programming formalism, also known as parsing-as-deduction (Pereira and Warren, 1983). We refer the reader to Kallmeyer (2010, Chapter 3) for an introduction to this formalism. The space and time complexities can be directly inferred by counting the maximum number of free variables in items and deduction rules, respectively. To the best of our knowledge, the presentation of the CYK-like algorithm is novel as previous work relied on the “actual” CYK algorithm (Finkel and Manning, 2009) or its variant for span-based syntactic parsing (Lou et al., 2022; Fu et al., 2021).

heading: 4.1 Non-nested named-entity recognition
text: The semi-Markov algorithm recognizes a sentence from left to right. Items are of the following forms:
• [t, i, j] s.t. t ∈ T and 0 ≤ i < j ≤ n: represent the mention ⟨t, i, j⟩;
• [→, i] s.t. 0 ≤ i ≤ n: represent a partial analysis of the sentence covering words s0:i.
Axioms are items of the form [→, 0] and [t, i, j]. The first axiom form represents an empty partial analysis and the second set of axioms represent all possible mentions in the sentence. We assign weight w⟨t,i,j⟩ to axiom [t, i, j], for all t ∈ T, i, j ∈
N s.t. 0 ≤ i < j ≤ n. The goal of the algorithm is the item [→, n].
Deduction rules are defined as follows: [→, i] [t, i, j] (a) [→, j] [→, i− 1] (b)
[→, i] Rule (a) appends a mention spanning words si:j to a partial analysis, whereas rule (b) advances one position by assuming word si:i+1 is not covered by a mention.
A trace example of the algorithm is given in Table 1. Soundness, completeness and uniqueness of derivations can be directly induced from the deduction system. The time and space complexities are both O(n2|T |).

heading: 4.2 Nested named-entity recognition
text: We present a CYK-like algorithm for nested named entity recognition. Contrary to algorithms proposed by Finkel and Manning (2009) and Fu et al. (2021), inter alia, our algorithm directly recognizes the nested mentions and does not require any “trick” to take into account non-binary structures, words that are not covered by any mention or the fact that a word in a mention may not be covered by any of its children. As such, we present an algorithm that is tailored for NER instead of the usual “hijacking” of constituency parsing algorithms. This particular presentation of the algorithm will allow us to simplify the presentation of our novel contribution in Section 5.
Items are of the following forms:
• [t, i, j] as defined previously;
• [→, i] as defined previously;
• [7→, i, j] with 0 ≤ i < j ≤ n: represent the partial analysis of a mention and its nested structure starting at position i.
• [↔, i, j] with 0 ≤ i < j ≤ n: represent the full analysis of a mention spanning si:j , including its internal structure (i.e. full analysis of its children).
Axioms and goals are the same as the ones of the semi-Markov algorithm presented in Section 4.1, with supplementary set of items of form [7→, i, i] that are used to start recognizing the internal structure of a mention starting at position i.
The algorithm consists of two steps. First, the internal structure of mentions are constructed in a bottom-up fashion. Second, first level mentions (and their internal structures) are recognized in a
similar fashion to the semi-Markov model. The deduction rules for bottom-up construction are defined as follows:
[ 7→, i, k] [↔, k, j] (c) i < k [7→, i, j] [7→, i, j − 1] (d) [7→, i, j]
[↔, i, k] [↔, k, j] (e) [7→, i, j] [↔, i, j − 1] (f) [ 7→, i, j]
[7→, i, j] [t, i, j] (g) i < j
[↔, i, j] Rule (c) concatenates an analyzed mention to a partial analysis of another mention — note that the constraint forbids that right antecedent shares its left border with its parent. Rule (d) advances of one position in the partial structure, assuming the analyzed mention starting at i does not have a child mention covering sj−1:j . Rules (e) and (f) are used to recognize the internal structure of a mention that has a child sharing the same left border. Although the latter two deduction rules may seem far-fetched, they cannot be simplified without breaking the uniqueness of derivations property or breaking the prohibition of self loop construction of↔ items. Finally, rule (g) finishes the analysis of a mention and its internal structure.
Note that this construction is highly similar to the dotted rule construction in the Earley algorithm (Earley, 1970). Moreover, contrary to Stern et al. (2017), we do not introduce null labels for implicit binarization. The benefit of our approach is that there is no spurious ambiguity in the algorithm, i.e. we guaranty uniqueness of derivations. Therefore, we can use the same deduction rules to compute the log-partition function of the negative log-likelihood loss. This is not the case of the approach of Stern et al. (2017), which forces them to rely on a structured hinge loss.
Deduction rules for the second step are defined as follows:
[→, i] [↔, i, j] (h) [→, j] [→, i− 1] (i) [→, i]
They have similar interpretation to the rules of the semi-Markov model where we replaced mentions by possibly nested structures.
A trace example of the algorithm is given in Table 2. Although the algorithm is more involved than usual presentations, our approach directly maps a derivation to nested mentions and guarantee uniqueness of derivations. The space and time complexities are O(n2|T |) and O(n3|T |), respectively.
5 O(n2) nested named-entity recognition In this section, we describe our novel algorithm for quadratic-time nested named entity recognition. Our algorithm limits its search space to mentions that contain at most one child of length strictly greater to one.
Items are of the following forms:
• [t, i, j] as defined previously;
• [→, i] as defined previously;
• [7→, i, j] as defined previously;
• [↔, i, j] as defined previously;
• [← [, i, j] with 0 ≤ i < j ≤ 0: represents a partial analysis of a mention and its internal structure, where its content will be recognized by appending content on the left instead of the right.
Axioms and goals are the same than the one of the CYK-like algorithm presented in Section 4.2 — importantly, there is no extra axiom for items of the form [← [, i, j].
For the moment, assume we restrict nested mentions that have a length strictly greater to the ones that share their left boundaries with their parent. We can re-use rules (d), (f), (g), (h) and (i) together with the following two deduction rules:
[7→, i, j − 1] [↔, j − 1, j] (j) i < j − 1
[7→, i, j] [↔, i, j − 1] [↔, j − 1, j]
(k) [7→, i, j]
More precisely, we removed the two rules inducing a cubic-time complexity in the CYK-like algorithm and replaced them with quadratic-time rules. This transformation is possible because our search space forces the rightmost antecedents of these two rules to cover a single word, hence we do not need to introduce an extra free variable. However, in this form, the algorithm only allows the child mention of length strictly greater to one to share its left boundary with its parent.
We now extend the algorithm to the full targeted search space. The intuition is as follows: for a given mention, if it has a child mention of length strictly greater than one that does not share its left border with its parent, we first start recognizing this child mention and its left neighborhood and then move to right neighborhood using previously
defined rules. We start the recognition of the left neighborhood using the two following rules:
[↔, i, i+ 1] [↔, i+ 1, j] (l) i+ 2 < j
[←[, i, j]
[↔, i+ 1, j] (m) i+ 2 < j
[←[, i, j] where the constraints ensure antecedents [↔, i + 1, j] are non-unary (otherwise we will break the uniqueness of derivations constraint). Rule (l) (resp. (m)) recognizes the case where span si:i+1 contains (resp. does not contain) a mention. The following rules are analogous to rules (d) and (j) but for visiting the left neighborhood instead of the right one:
[←[, i+ 1, j] (n) [←[, i, j] [↔, i, i+ 1] [←[, i+ 1, j] (o) [←[, i, j] Finally, once the left neighborhood has been recognized, we move to the right one using the following rule:
[←[, i, j] (p)
[ 7→, i, j] Using the aforementioned rules, our algorithm has time and space complexities of O(n2|T |). We illustrate the difference with the CYK-like algorithm with a trace example in Table 2: in this specific example, the two analyses differ only by the application of a single rule. Table 3 contains a trace example where all nested mentions have a size one, so the parent mention is visited from left to right. Table 4 contains a trace example where we need to construct one internal structure by visiting the left neighborhood of the non-unary child mention from right to left.
Soundness and completeness can be proved by observing that, for a given mention, any children composition can be parsed with deduction rules as long as there is at most one child with a span strictly greater to one. Moreover, these are the only children composition that can be recognized. Finally, uniqueness of derivations can be proved as there is a single construction order of the internal structure of a mention.
Infinite recursion. An important property of our algorithm is that it does not bound the number of allowed recursively nested mentions. For example, consider the phrase “[Chair of [the Committee of [Ministers of [the Council of [Europe]]]]]”. Not only can this nested mention structure be recognized by our algorithm, but any supplementary “of” precision would also be recognized.
Possible extension. Note that we could extend the algorithm so that we allow each mention to have at most one child of length strictly greater to a predefined constant c, and other children should have a length less or equal to c. However, as fixing c = 1 results in a good cover of datasets we use, we do not consider this extension in this work.

heading: 6 Experimental results
text: Data. We evaluate our algorithms on the ACE2004 (Doddington et al., 2004), ACE-2005 (Walker et al., 2006) and GENIA (Kim et al., 2003) datasets. We split and pre-process the data using the tools distributed by Shibuya and Hovy (2020).
Data coverage. As our parsing algorithm considers a restricted search space, an important question is whether it has a good coverage of NER datasets. Table 5 shows the maximum recall we can achieve with the algorithms presented in this paper. Note that no algorithm achieve a coverage of 100% as there is a small set of mentions with exactly the same span2 and mentions that overlap partially. We observe that the loss of coverage for our quadratic-
2This can be easily fixed by collpasing theses mentions, a standard trick used in the constituency parsing literature, see (Stern et al., 2017)
time algorithm is negligible compared to the cubictime algorithm for all datasets.
Timing. We implemented the three algorithms in C++ and compare their running time for MAP inference in Figure 2. The proposed algorithm is way faster than the CYK-like. If we would parse only sentences of 300 words and we only consider the time spend in the decoding algorithm (i.e. ignoring the forward pass in the neural network), the CYK-like algorithm couldn’t even decode 50 sentences in a second whereas our algorithm could decode more than 1500 sentences on an Intel Core i5 (2.4 GHz) processor. As such, we hope that our algorithm will allow future work to consider NER on longer spans of text.
Neural architecture and hyperparameters. Our neural network is composed of a finetuned BERT model3 (Devlin et al., 2019) followed by 3 bidirectional LSTM layers (Hochreiter and Schmidhuber, 1997) with a hidden size of 400. When the BERT tokenizer splits a word, we use the output embedding of the the first token. Mention weights (i.e. values in vector w) are computed using two biaffine layers (Dozat and Manning, 2017), one labeled and one unlabeled, with independent left and right projections of dimension 500 and RELU activation functions.
3bert-base-uncased as distributed at https: //huggingface.co/bert-base-uncased
We use a negative log-liklihood loss (i.e. CRF loss) with 0.1-label smoothing (Szegedy et al., 2016). The learning rate is 1 × 10−5 for BERT parameters and 1× 10−3 for other parameters. We use an exponential decay scheduler for learning rates (decay rate of 0.75 every 5000 steps). We apply dropout with probability of 0.1 at the output of BERT, LSTM layers and projection layers. We keep the parameters that obtains the best F1-measure on development data after 20 epochs.
Results. We report experimental results in Table 6. Note that our goal is not to establish a novel SOTA for the task but to assess whether our quadratic-time algorithm is well-suited for the nested NER problem, therefore we only compare our models with recent work using the same datasplit and comparable neural architectures (i.e. BERTbased and without lexicalization). Any method that modifies the cubic-time parser to improve results can be similarly introduced in our parser. Our implementation of the CYK-like cubic-time parser obtains results close to comparable work in the literature. Importantly, we observe that, with the proposed quadratic-time algorithm, F1-measure results are (almost) the same on GENIA and the the degradation is negligible on ACE-2004 and ACE2005 (the F1-measure decreases by less than 0.5).

heading: 7 Conclusion
text: In this work, we proposed a novel quadratic-time parsing algorithm for nested NER, an asymptotic improvement of one order of magnitude over previously proposed span-based algorithms. We showed that the novel search-space has a good coverage of English datasets for nested NER. Despite having the same time-complexity than semi-Markov models, our approach achieves comparable experimental results to the cubic-time CYK-like algorithm.
As such, we hope that our algorithm will be used as a drop-in fast replacement for future work in nested NER, where the cubic-time algorithm has often been qualified of slow. Future work could consider the extension to lexicalized mentions.
Limitations
An obvious limitation of our work is the considered search space. Although we showed that it is well suited for the data used in practice by the NLP community, this may not hold in more general settings.
Moreover, we only experiment in English. We suspect that similar results would hold for morphologically-rich languages as we expect, in the latter case, that constituents are shorter (i.e. morphologically-rich languages heavily rely on morphological inflection, so we expect more mentions spanning a single word), see (Haspelmath and Sims, 2013, Section 1.2 and Table 1.1). However, this is not guaranteed and future work needs to explore the multilingual setting.
Finally, in this work we do not consider discontinuous mentions, which is an important setting in real world scenario.

heading: Acknowledgments
text: We thank François Yvon, Songlin Yang and the anonymous reviewers for their comments and suggestions. This work benefited from computations done on the Saclay-IA platform and on the HPC resources of IDRIS under the allocation 2022- AD011013727 made by GENCI.
I apologize to the reviewers for not adding the supplementary experimental results: being the sole author of this article, I unfortunately did not find the time to do so.

heading: A For every submission:
text: A1. Did you describe the limitations of your work?
Left blank.
A2. Did you discuss any potential risks of your work? Left blank.
A3. Do the abstract and introduction summarize the paper’s main claims? Left blank.
A4. Have you used AI writing assistants when working on this paper? Left blank.

heading: B Did you use or create scientific artifacts?
text: Left blank.
B1. Did you cite the creators of artifacts you used? Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.

heading: C Did you run computational experiments?
text: Left blank.
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.

heading: D Did you use human annotators (e.g., crowdworkers) or research with human participants?
text: Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: As such, we hope that our algorithm will be used as a drop-in fast replacement for future work in nested NER, where the cubic-time algorithm has often been qualified of slow. Future work could consider the extension to lexicalized mentions. Limitations
An obvious limitation of our work is the considered search space. Although we showed that it is well suited for the data used in practice by the NLP community, this may not hold in more general settings. Moreover, we only experiment in English. We suspect that similar results would hold for morphologically-rich languages as we expect, in the latter case, that constituents are shorter (i. e. morphologically-rich languages heavily rely on morphological inflection, so we expect more mentions spanning a single word), see (Haspelmath and Sims, 2013, Section 1. 2 and Table 1. 1). However, this is not guaranteed and future work needs to explore the multilingual setting. Finally, in this work we do not consider discontinuous mentions, which is an important setting in real world scenario