Don’t Parse, Choose Spans! Continuous and Discontinuous Constituency Parsing via Autoregressive Span Selection
Year: 2023

Abstract:
We present a simple and unified approach for both continuous and discontinuous constituency parsing via autoregressive span selection. Constituency parsing aims to produce a set of non-crossing spans so that they can form a constituency parse tree. We sort gold spans in a predefined order and train a pointer network to autoregressively select spans by that order. To deal with a discontinuous span, we consecutively select its subspans from left to right, label all but the last subspans with a special discontinuous label, and label the last subspan with the whole discontinuous span’s label. We use a simple heuristic to output valid trees from selected spans so that our approach is able to predict all possible continuous and discontinuous constituency trees without sacrificing data coverage and without the need to use expensive chart-based parsing algorithms. Extensive experiments show that our model achieves stateof-the-art or competitive performance on all benchmarks of continuous and discontinuous constituency parsing .1

heading: 1 Introduction
text: Constituency parsing is a fundamental task in natural language processing, having many applications in downstream tasks such as language modeling (Sartran et al., 2022) and machine translation (Akoury et al., 2019). Continuous constituency parsing is not capable of capturing discontinuous language phenomena, such as wh-movements and extraposition, which are deemed unavoidable in syntactic analysis (McCawley, 1982; Bunt et al., 1987; Müller, 2004). Discontinuous constituency parsing enables modeling these phenomena by allowing each constituent node governing nonadjacent strings in the yields, resulting in discontinuous parse trees as exemplified in Fig. 1(b). In this work
∗Corresponding author 1Code is available at https://github.com/
sustcsonglin/disco-pointer.
we study both continuous and discontinuous constituency parsing under a unified framework.
Both continuous and discontinuous parsing can be framed as span prediction problems. In continuous parsing, each span corresponds to a single interval (of the observed sentence), while in discontinuous parsing a discontinuous span could correspond to multiple nonadjacent intervals, the number of which we refer to as fan-out. Fig. 1 shows the span representations of both types of parse trees. Based on span representations, span-based methods decompose the score of a parse tree into scores of spans, scoring all possible spans and finding the
8420
best parse with dynamic programming algorithms. They have achieved high accuracy in both continuous parsing (Stern et al., 2017; Kitaev and Klein, 2018; Zhang et al., 2020b) and discontinuous parsing (Corro, 2020; Stanojević and Steedman, 2020). However, they have several issues: • Their strong conditional independence as-
sumption is problematic. Span-based methods score each span independently for global training and decoding, hoping that powerful neural encoders could capture sufficient contextual information for parsing. However, Cui et al. (2022) show that independent local span scoring is suboptimal and they propose auxiliary training objectives to inject non-local features into neural span-based continuous parsing. In discontinuous parsing, span-based parsers impose even stronger conditional independence assumptions when scoring discontinuous spans to alleviate high space and time complexities,2 which exacerbates the issue of insufficient modeling of non-local contextual information. • Data coverage of discontinuous parsing is low. Current span-based discontinuous parsers can only deal with discontinuous spans of fan-out of at most two due to high parsing time complexity, thereby having limited data coverage. For instance, the most expressive variant of LCFRS-2 (Stanojević and Steedman, 2020) can only cover up to 87% discontinuous spans in the NEGRA treebank, greatly upper-bounding the prediction performance on discontinuous constituents. • Global decoding for discontinuous parsing is time consuming. Continuous parsing needs the cubic-time CKY algorithm for global decoding, which is not problematic when efficient parallel implementations (Zhang et al., 2020b; Rush, 2020) are available. However, discontinuous parsing to LCFRS-2 needs O(n6) time, which is computationally prohibitive. While Corro (2020) and Stanojević and Steedman (2020) propose to restrict the search space to speed up parsing,3
2Scoring all fan-out-2 spans needs O(n4) time and space, which is computationally expensive (mainly in space complexity to store scoring tensors). To solve this issue, Corro (2020) and Stanojević and Steedman (2020) decompose the scoring of discontinuous spans to the scoring of multiple continuous spans (e.g., left and right subspans and the gap in between), making stronger conditional independence assumptions than the continuous parsing case.
3For example, Corro (2020) proposes a cubic-time discontinuous parser that can only predict the simplest discontinuous structures, i.e., a discontinuous parent span can only have two continuous subspans. Though their parser has a high overall
that further damages data coverage.
In this work, we present a simple yet effective approach to address all aforementioned problems via autogressive span selection. 4 To solve the first issue, we sort gold spans by a predefined order and train a pointer network (Vinyals et al., 2015) to autoregressively choose spans by that order. As such, the independence assumption of span-based parsing is weakened due to the existence of the autoregressive neural decoder, which captures span correlations in a similar way to Nguyen et al. (2021) and Yang and Tu (2022) (but see section 4 for how our method differs from theirs). To solve the second issue, we devise a simple strategy of decoding discontinuous spans that consecutively selects all continuous subspans of a discontinuous span from left to right and labels all but the last subspans with a special discontinuous label and the last subspan with the label of the entire discontinuous span. As such, we can connect an arbitrary number of subspans to obtain a discontinuous span, so we can decode discontinuous spans of arbitrary fanout and thus arbitrary discontinuous constituency parse trees, enjoying perfect data coverage. For the third issue, we find a frustratingly simple and fairly fast strategy that works quite well. Concretely, we use greedy decoding to output spans in an unconstrained manner and use a heuristic to resolve possible inconsistency (i.e., two spans crossing each other) afterwards by simply discarding conflicting spans. Since O(n) spans are needed to decode, our method needs linear steps for parsing, thus is efficient. In continuous parsing, this simple strategy combined with our autoregressive neural decoder works even better than global CKY decoding in our experiments. In discontinuous parsing, it removes the need to use expensive chart-based algorithms for decoding, greatly reducing the computational complexity.
We conduct extensive experiments on benchmarks, achieving state-of-the-art performance on PTB and competitive performance on CTB for continuous parsing; and state-of-the-art performance on three benchmarks: Tiger, NeGra, and DPTB for discontinuous parsing.
parsing accuracy, the discontinuous F1 is much lower than other comparable methods due to the restriction in search space.
4Span-based methods can be viewed as non-autoregressive span selection with independence assumptions, in analogy to autoregressive vs. non-autoregressive machine translation.

heading: 2 Approach

heading: 2.1 Constituency parsing as span selection
text: We formally frame (dis)continuous constituency parsing as a span selection problem. A (dis)continuous constituency parse tree t comprises a set of nodes and for each node s we have yield(s) = {s1, ..., sl}which is the set of sorted token indices in the yield of s in t with s1 < · · · < sl. The bidirectional conversion between spans and trees are shown as follows.
Tree to spans. It is somewhat trivial to derive the span representation of a tree node s based on yield(s) by merging consecutive token indices into continuous intervals represented by left and right boundary indices. We do not allow two resulting intervals sharing any boundaries for eliminating potential ambiguities. If a single interval is obtained, then it is a continuous span; if multiple intervals are obtained, then it is a discontinuous span. Fig. 1 shows examples of span representations converted from trees.
Spans to tree. We say a set of spans S is consistent (to form a valid tree) iff ∀s, t ∈ S, yield(s) ∩ yield(t) = ∅ or yield(s) ⊂ yield(t) or yield(t) ⊂ yield(s).5 We can build a tree from any consistent S as follows. For each s with | yield(s)| < n, we define Ps ⊂ S as a set of spans in which s is properly contained. We say t is parent node of s if t = arg min
t′∈Ps len(t′). We can thus
determine the parent node of each span and thereby construct the whole tree. We find it convenient to reconstruct the tree if all spans are sorted in postorder tree traversal order , i.e., spans of smaller end position (i.e., maxyield(s)) come first, and for tie breaking, spans of smaller width come first. As such, we only need to sequentially scan the sorted spans to build the tree from the bottom up: for a scanned span we build a node for it by looking up prior decoded nodes to find all its children nodes. Finally, we add a <TOP> node spanning the whole sentence to connect all unconnected nodes for obtaining the final parse tree.
It also gives us a simple heuristic to build a tree from a set of inconsistent spans: sort spans by postorder, scan sorted spans and if a span crosses to any prior decoded spans, simply discard it, otherwise build a node for it likewise the aforementioned procedure. We adopt this strategy in the
5Here we slightly abuse the notation yield.
post-processing stage to build trees.
Handling discontinuous spans. We use a simple strategy to deal with discontinuous spans. For a given discontinuous span of fan-out f {(l1, r1), ..., (lf , rf )}, we label all but the last subspans (i.e., {(l1, r1), ..., (lf−1, rf−1)}) with a special discontinuous label <dis> and the last subspan (i.e., (lf , rf )) with the label of the entire discontinuous span. For example, a discontinuous span ((0, 2), (4, 7), (10, 14)),NP) would be transformed to (0, 2,<dis>), (4, 7,<dis>), (10, 14,NP). As such, we can reduce discontinuous parsing to continuous parsing, except that we need to guarantee all continuous subspans of a discontinuous span occur consecutively in the (ordered) decoded span list (and also with correctly predicted labels <dis>), so that we can connect subspans to recover the parent discontinuous span. We achieve it using an autoregressive decoder as described later. The key advantage here is that we can handle discontinuous spans of arbitrary fan-out easily.
Example. We take the discontinuous parse tree from Fig. 1(b) for example to show how we convert it to a list of sorted spans. First, we sort all spans in post-order, obtaining {(0, 2,NP), (5, 7,NP), (4, 7,PP), ((0, 2), (4, 7),NP), (7, 8,NP), ((0, 2), (3, 8),VP) (0, 8,VP)}. Then we use the introduced simple trick to handle discontinuous spans, resulting in {(0, 2,NP), (5, 7,NP), (4, 7,PP), (0, 2,<dis>), (4, 7,NP), (7, 8,NP), (0, 2,<dis>), (3, 8,VP), (0, 8,VP)}.

heading: 2.2 Neural architecture
text: Input. Given a sentence w = w1, · · · , wn, we add <bos> at w0 and <eos> at wn+1, and sort the gold spans (for discontinuous parsing we first do the transformation descried earlier) by post-order to get S = {(li, ri, yi)}i where li, ri, yi are the left boundary index, right boundary index, and the label index of the i-th span, respectively. Since we adopt an autoregressive selection strategy, we need to inform the model when to stop. We achieve it by appending a special indicator span (0, 0,<end>) to S, and denote the resulting size of S as m.
Encoder. We tokenize the input sentence and feed it into pre-trained language models such as BERT (Devlin et al., 2019) and XLNet (Yang et al.,
LSTM
<start>
(0,2) NP
LSTM
(0,2,NP)
(5,7) NP
LSTM
(4,7) PP
LSTM
(0,2) <dis>
LSTM
(0,2,<dis>)
(4,7) NP
LSTM
(7,8) NP
LSTM
(7,8,NP)
(0,2) <dis>
LSTM
(3,8) VP
LSTM
(3,8,VP)
(0,8) VP
LSTM
(0,8,VP)
(0,0) <end>
(5,7,NP) (4,7,PP) (4,7,NP) (0,2,<dis>)
Figure 2: The depiction of the decoder architecture and how it outputs spans of the discontinuous parse tree from Fig. 1(b).
2019) to obtain word-level 6 contextualized embedding x = x1, · · · , xn, then feed x into a multi-layer bidirectional LSTM (BiLSTM):
. . . , ( −→ fi , ←− fi ), · · · = BiLSTM([. . . , xi, . . . ])
We use the fencepost representation (Cross and Huang, 2016; Stern et al., 2017) to represent the i-th boundary sitting between xi and xi+1:
bi = [ −→ fi ; ←−− fi+1]
and obtain span embedding ei,j for (i, j) using the concatenation of the LSTM-minus feature (Wang and Chang, 2016; Cross and Huang, 2016) and max-pooling over representations of all words in the span:
ei,j = [bj − bi; max-pooling(xi+1, ..., xj)] Decoder. For the decoder we use a unidirectional LSTM network,
d1 = LSTM(d0, e start) dt = LSTM(dt−1, [MLP(elt−1,rt−1);Eyt−1 ]), t ≥ 2 where dt is the hidden state of the LSTM decoder at time step t; d0, estart are randomly initialized trainable vectors; E is the label embedding matrix. Fig. 2 demonstrates the working mechanism of the autoregressive decoder for outputting spans of the discontinuous parse tree from Fig. 1(b).
For each step, the decoder hidden state is used as a query to select the target spans and then the decoder hidden state and the selected span are used together to predict the label of the selected span. For span selection, we use deep triaffine attention (Zhang et al., 2020a) to estimate the score sti,j of selecting the span (i, j) at time step t,
b l/r i = MLP l/r(bi) d′t = MLP decoder(dt) sti,j = TriAff(b l i, b r j , d ′ t),
6For each word we take its last subtoken’s representation from the last output layer of BERT or XLNet
For span labeling, we compute the label score gt ∈ RL as,
Ht = MLPlabel([dt; elt−1,rt−1 ]) gt = HtET
where L is the size of label set, E is the label embedding matrix. We remark that it is crucial to incorporate decoder state embedding into label prediction in discontinuous parsing. A span can be both a standalone continuous span and a continuous subspan of a discontinuous parent span (e.g, (0, 2) and (4, 7) in Fig. 1(b)) simultaneously. If we do not use the decoder state embedding to provide side information, it would be hard to distinguish such difference, thereby confusing the model.

heading: 2.3 Training and decoding
text: We decompose the training loss L as the span selection loss and the span labeling loss,
L = Lselect + Llabeling
Lselect = − m∑
t=1
log exp{stlt,rt}∑
0≤i<j≤n or i=j=0
exp{sti,j}
Llabeling = − m∑
t=1 log exp{gtyt}∑L j=1 exp{gtj}
For inference, we autoregressively decode spans with no constraint until the special span (0, 0) is selected, and use the simple heuristic (introduced in Sect. 2.1) to build the final parse.

heading: 3 Experiments

heading: 3.1 Setting
text: Data. For continuous parsing, we conduct experiments on Penn Treebank (PTB, Marcus et al., 1993) 3.0 with the standard splits of section 02-21 for training, section 22 for development and section 23 for testing; and Chinese Treebank (CTB,
Xue et al., 2005) 5.1 with the same split from (Zhang et al., 2020b). For multilingual experiments, we follow Cui et al. (2022) and select six languages from SPMRL 2013-2014 shared task (Seddah et al., 2013), including three rich-resource languages: French (fr), German (de), and Korean (ko), and three low-resource languages: Hungarian (hu), Basque (eu), and Polish (pl). For discontinuous parsing, we conduct experiments on discontinuous version of the English PTB (DPTB, Evang and Kallmeyer, 2011) using the standard split; German Negra treebank (Skut et al., 1997) using the split from Dubey and Keller (2003); and German Tiger treebank (Brants et al., 2001) using the split from SPRML 2014 shared task (Seddah et al., 2014).
Evaluation. We use the script evalb7 and discodop8 to evaluate continuous and discontinuous parsing respectively, reporting labeled F1 measures for continuous parsing; labeled overall F1 meansures and labeled discontinuous F1 measures (DF1) for discontinuous parsing. All reported values are averaged over three independent runs with different random seeds.
Implementation details. We refer readers to Appendix A for details.

heading: 3.2 Continuous parsing results
text: Main result. We re-implement the neural TreeCRF parser (Zhang et al., 2020b) — a strong span-based parser — as our baseline, using the same neural architectures (for scoring) and training settings for a fair comparison.
Table 1 shows the results on PTB test set. Our model outperforms the TreeCRF baseline and all other compared models. Interestingly, our model performs similarly to (Yang and Tu, 2022) (both their and our models have similar encoder-decoder neural architectures), suggesting that their bespoke decoding algorithm is not very useful in improving performance. We posit the improvement of their model mainly stems from the autoregressive decoder (sec. 3.4). Table 2 show the results on CTB test set and we can see our model outperforms the TreeCRF baseline and obtains competitive performance compared to other models.
Multilingual evaluation. We additionally report the results on multilingual SPMRL datasets fol-
7https://nlp.cs.nyu.edu/evalb 8https://github.com/andreasvc/
disco-dop
lowing the settings from Cui et al. (2022). For rich-resourced languages our model clearly improves the performance, while for low-resourced languages our approach performs similarly to Cui et al. (2022). We posit that a learnable neural decoder is data hungry, and thus when the amount of training data is limited our model might not be advantageous.

heading: 3.3 Discontinuous parsing results
text: Table 4 shows the main results of discontinuous parsing.9 Our model outperforms the recent parser Sun et al. (2022), obtaining state-of-the-art performances. Notably, our model’s discontinuous F1 measures are significantly higher than all other approaches. This is surprising since our model is conceptually much more simpler, indicating that there is no need to design complex models for discontinuous parsing and warranting rethinking of this research field.
9The running speed is measured based on a single NVIDIA A40 GPU and each batch contains five thousands tokens

heading: 3.4 Ablation study
text: We conduct ablation studies on PTB (with BERTlarge) and show the results on Table 5.
The influence of the decoder. As previously mentioned, span-based parsing amounts to nonautoregreesive span selection with independent assumptions. To show the importance of incorporating an autoregressive neural decoder, we train a span-based parser with local span binary classification loss (Teng and Zhang, 2018) and perform greedy decoding as used in our approach. We find it hurts the performance significantly, verifying the importance of neural decoder when adopting greedy decoding. We further use the CKY algorithm for decoding with the implicit binarization (Stern et al., 2017; Kitaev and Klein, 2018) strategy10 instead of using greedy decoding and observe quite marginal improvement, which aligns with the experimental results of (Fig. 1, Zhang et al., 2019) in dependency parsing, questioning the utility of global decoding, while we indeed observe that global TreeCRF training (Zhang et al., 2020b) improves the performance compared to local training (but still underperforms our model). We believe this is because TreeCRF modeling weakens the strong independence assumption by introducing a global factor connecting all possible spans to enforce hard tree constraint (Smith and Eisner, 2008). However we are not aware of any greedy (or local) decoding strategies for globally-trained TreeCRF parser, thus cannot investigate the influ-
10It is also possible to use the algorithm from (Section 4.2, Corro, 2022) for global non-binary parsing
ence of global decoding here.
The effect of span selection order. It is also possible to use other (span) sorting orders e.g. the pre-order tree traversal order as used in (Nguyen et al., 2021). We find using post-order indeed performs better than pre-order, which might explain the performance gap between (Nguyen et al., 2021) and (Yang and Tu, 2022) on PTB. We propose that this phenomenon can be attributed to the order in which spans are generated. In general, shorter spans are more easily predictable. Post-order generation prioritizes the generation of child spans before parent spans, whereas pre-order generation generates parent spans before child spans. As a result, pre-order generation is more prone to the issue of error propagation.

heading: 4 Related work
text: Continuous parsing. Continuous parsing is wellstudied in the literature. Span-based parsing is the most popular paradigm and achieves great success. The main issue of span-based parsing stems from the unreasonable conditional independence assumptions imposed in local span scoring. To resolve this issue, one line of work aims to enhance span representation learning. Mrini et al. (2020) propose a Label Attention Layer to bring label information into encoding; Tian et al. (2020) incorporate n-gram features and enhance self-attention mechanism to learn better span representation; and Cui et al. (2022) introduce auxiliary tasks to help the model learn non-local patterns. Another line of work focuses on improving output modeling (Zhang et al., 2019) by wearkening independence assumptions. Zhang et al. (2020b) propose a spanbased TreeCRF parser, which can be viewed as introducing a global (hard tree constraint) factor (Smith and Eisner, 2008) to connect all spans and thereby reduce independence assumptions. Xin et al. (2021) weaken the independence assumption by taking into accounts the correlations of sibling spans under a common parent span, designing an O(n4) dynamic programming algorithm to incorporate the interaction scores between sibling spans into global decoding. Our model weakens the independence assumption by using a neural autoregressive decoder so that spans are decoded conditioned on all previously decoded spans.
Discontinuous parsing. Discontinuous parsing is a much more complex problem than continuous
parsing due to the much larger search space of discontinuous parse trees. Reducing discontinuous
parsing to simpler problems such as nonprojective dependency parsing (Fernández-González and Martins, 2015; Corro et al., 2017; Fernández-González and Gómez-Rodríguez, 2020) and continuous constituency parsing (Versley, 2016; FernándezGonzález and Gómez-Rodríguez, 2021; Sun et al., 2022) is a viable way to decrease the complexity of discontinuous parsing. This work is closely related to the line of work of reduction to continuous constituency parsing. Boyd (2007) and Versley
(2016) convert the original discontinuous treebanks into continuous ones, using special nonterminal nodes to encode discontinuous constituents similar to pseudo-projective parsing (Nivre and Nilsson, 2005). After treebank transformation, any continuous constituency parsers can be used for training and decoding, and post-processing is needed to recover discontinuous trees. In another line of work that does not involve treebank transformation, researchers reorder words in the sentence so that a discontinuous tree becomes continuous, and then they train off-the-shelf constituency parsers on the resulting reordered sentences. This strategy is referred to as reorder-then-parse. Research in this line mainly focuses on the design of the reordering module. Fernández-González and GómezRodríguez (2021) use a pointer network to do the reordering, while Sun et al. (2022) use a dependency graph parsing strategy to improve the reordering accuracy, which they show is crucial to the final parsing performance. In this work, we view a discontinuous span as a combination of multiple continuous subspans and use a simple labeling mechanism together with the autoregressive decoder to connect all (nonadjacent) subspans. Our approach is conceptually simpler than previous models, not involving any treebank transformation or reordering and eliminating the need for off-the-shelf continuous constituency parsers.
Constituency parsing with pointer nets. Our model has a very similar neural architecture compared to prior pointer network-based constituency parsers: Nguyen et al. (2021) and Yang and Tu (2022). The major difference is that our model directly points to spans using deep triaffine attentions, while their models point to span boundaries using deep biaffine attentions so that spans are implicitly built. Therefore, their models are highly coupled with bespoke decoding algorithms: Nguyen et al. (2021) decode spans with pre-order tree traversal by splitting each input parent span into two subspans based on the selected splitting point (i.e., boundary); Yang and Tu (2022) decode spans with post-order tree traversal, and since every two consecutively visited spans share a boundary, their method selects the unshared boundary for each new span. We believe bespoke decoding algorithms are not the keys for obtaining high parsing accuracy and show a much simpler greedy decoding strategy instead. Moreover, both of their models are hard to
be extended to discontinuous parsing. 11 This is because their bespoke decoding algorithms leverage locality properties of continuous parse trees, which does not hold for discontinuous parse trees due to the cross dependencies of discontinuous spans. In contrast, our model is very flexible in decoding spans and can be easily extended to discontinuous parsing.
Seq2seq constituency parsing. Our work is also closely connected to the recent work of (FernándezGonzález and Gómez-Rodríguez, 2023), who propose a unified seq2seq method for both continuous and discontinuous parsing like ours. The main difference is that they linearize (discontinuous) constituency parse trees to transition action sequences (e.g., SHIFT, REDUCE). As such, they rely on transition systems to convert a decoded action sequence to a parse tree. In contrast, we directly linearize trees to span sequences and leverage pointer networks to decode a span for each step .

heading: 5 Conclusion
text: In this work, we have presented a simple yet effective method for both continuous and discontinuous constituency parsing. We showed that an autoregressive decoder is more desirable than global CKY decoding in span-based continuous parsing, and with a simple labeling mechanism, we obtained state-of-the-art performance in discontinuous parsing.
Limitations
Though autoregressive span selection effectively weakens the conditional independence assumptions imposed by current span-based parsing methods, this strategy imposes another arguably unreasonable inductive bias of forcing a predefined span selection order. We find using post order performs fairly well but this does not necessarily means it is the best order for span selection, and this work might leave other potentially better order unexplored. Future works might consider using set prediction methods (Sui et al., 2020; Tan et al., 2021)
11For instance, if we want to extend the model of Nguyen et al. (2021) to discontinuous parsing (assuming binarization), the number of splitting points is nondeterministic which can be difficult to handle: a continuous parent span could be split into two continuous subspans with one splitting point, or one continuous subspan and one discontinuous subspan with two splitting points, or two discontinuous subspans with at least three splitting points (depending on the fan-out of subspans). The cases of discontinuous parent spans are even more complicated.
to eliminate such implausible inductive bias (of forcing orders) while still benefiting from explicit span correlation modeling.
Another issue is regarding time complexity. Though our model needs only linear steps (in sentence length) for parsing, each step takes O(n2) time to select a single span over O(n2) total spans, making the overall time complexity cubic. We remark that for each step the O(n2) operation is parallelizable and—with full GPU parallelization— fairly fast in practice, but it would still be problematic when the sentence is extremely long.

heading: Acknowledgments
text: This work was supported by the National Natural Science Foundation of China (61976139).

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: after conclusion selection, page 9
7 A2. Did you discuss any potential risks of your work? we do not see any ethical implications or risks of our work

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: abstract and section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? section 3

heading: 3 B1. Did you cite the creators of artifacts you used?
text: section 3.1
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Not applicable. Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
7 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
7 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.
C 3 Did you run computational experiments? section 3

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? section 3.2 shows the discontinuous parsing speed
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? section3 and appendix A

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? section 3

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? section 3.1
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: Future works might consider using set prediction methods (Sui et al. , 2020; Tan et al. , 2021)
11For instance, if we want to extend the model of Nguyen et al. (2021) to discontinuous parsing (assuming binarization), the number of splitting points is nondeterministic which can be difficult to handle: a continuous parent span could be split into two continuous subspans with one splitting point, or one continuous subspan and one discontinuous subspan with two splitting points, or two discontinuous subspans with at least three splitting points (depending on the fan-out of subspans). The cases of discontinuous parent spans are even more complicated. to eliminate such implausible inductive bias (of forcing orders) while still benefiting from explicit span correlation modeling. Another issue is regarding time complexity. Though our model needs only linear steps (in sentence length) for parsing, each step takes O(n2) time to select a single span over O(n2) total spans, making the overall time complexity cubic. We remark that for each step the O(n2) operation is parallelizable and—with full GPU parallelization— fairly fast in practice, but it would still be problematic when the sentence is extremely long