Struct-XLM: A Structure Discovery Multilingual Language Model for Enhancing Cross-lingual Transfer through Reinforcement Learning
Authors: Linjuan Wu
Weiming Lu
Venue: EMNLP 2023 Main (2023)
Keywords: Cross-lingual Representaion Alignment
Cross-lingual Transfer Learning
Reinforcement Learning

Abstract: Cross-lingual transfer learning heavily relies on well-aligned cross-lingual representations. The syntactic structure is recognized as beneficial for cross-lingual transfer, but limited researches utilize it for aligning representation in multilingual pre-trained language models (PLMs). Additionally, existing methods require syntactic labels that are difficult to obtain and of poor quality for low-resource languages. To address this gap, we propose Struct-XLM, a novel multilingual language model that leverages reinforcement learning (RL) to autonomously discover universal syntactic structures for improving the cross-lingual representation alignment of PLM. Struct-XLM integrates a policy network (PNet) and a translation ranking task. The PNet is designed to discover structural information and integrate it into the last layer of the PLM through the structural multi-head attention module to obtain structural representation. The translation ranking task obtains a delayed reward based on the structural representation to optimize the PNet while improving the alignment of cross-lingual representation. Experiments show the effectiveness of the proposed approach for enhancing cross-lingual transfer of multilingual PLM on the XTREME benchmark.

Struct-XLM: A Structure Discovery Multilingual Language Model for Enhancing Cross-lingual Transfer through Reinforcement Learning
Linjuan Wu- and Weiming Lu
Cooleege of Computer Science and Technology, Zhejiang University Ahiiaaaa-Zhejiang University Joint Research Institute of Frontier Technologies (wulinjuan525, luwm)@zju. edu. cn
Abstract
ing. Previous researches extend the pre-training tasks by introducing novel pre-training objectives at different levels of granularity, such as token-level (Luo et al., 2021; Zhang et al., 2023), word-level (Huang et al., 2019; Wei et al., 2021), and sentence- level (Huang et al., 2019; Wei et al., 2021; Chi et al., 2021; Ouyang et al., 2021). These objec- tives provide explicit guidance for the model to learn aligned information across languages. Some models (Luo et al., 2021: Ouyang et al., 2021) proposed cross-lingual attention mechanisms to aligned cross-lingual representation. However, there is still limited research on aligning cross- lingual representations with syntactic structure- level information.
Cross-lingual transfer learning heavily relics on well-aligned cross-lingual representations. The syntactic structure is recognized as beneficial for cross-lingual transfer, limited researches utilize it for aligning representation in multi- lingual pre-trained language models (PLMs). Additionally, existing methods require syntac- tic labels that difficult obtain and of poor quality for low-resource languages. To address this gap, we propose Struct-XI.M, novel mul- tilingual language model that leverages rein- forcement learning (RI.) to autonomously dis- universal syntactic structures for improv- ing the cross- looss-l representation alignment of PLM. Struct-XL.M integrates policy net- work (PNet) and a translation ranking task. The PNet designed to discover structural infor- mation and integrate into the last layer of the PL.M through the structural multi-head atten- tion module to obtain structural representation. The translation ranking task obtains delayed reward based on the structural representation to optimize the PNet while improving the align- ment of cross- lingual representation. Experi- ments show the effectiveness of the proposed approach for enhancing cross- loss-linguul transfer of multilingual PLM on the XTREME bench- mark!.
The performance of cross-lingual transfer is also influenced by structural differences among lan- guages (Ahmad et al., 2021; Wu et al., 2022a). And the incorporation of structural information. particularly syntax, has proven beneficial. Syntax- augmented mBERT (Ahmad et al., 2021) proposed a Graph Attention Network (GAN) to learn uni- versal dependency tree labels and integrate them into the self-attention mechanism of mBERT for enhancing cross-lingus transfer. The utilization of syntactic structures also demonstrates improve- ments in various cross-lingual tasks, such cross- lingual structured sentiment analysis (Zhang et al., 2022), cross-lingual semantic role labeling (Fei et al., 2020), and cross-lingual word sense disam- biguation (Zhu, 2020). However, existing methods heavily rely on dependency treebanks or syntactic tagging tools to obtain syntactic labels, which are often limited in quantity quality, particularly for low-resource languages.
1 Introduction
Cross-lingual representation is a crucial compo- nent in the development of multilingual natural lan- guage processing (NLP) systems (Hu et al., 2020). Various multilingual Pre-trained Language Models (PLMs) like XLM-R (Conneau al., 2020), XY- LENT (Patra et al., 2022), and VECO2.0 (Zhang al., 2023) have demonstrated their effectiveness cross-lingual transfer learning.
In this paper, we propose Struct-XLM, mul- tilingual language model with structure discovery based on multilingual PLM, that aims improve the alignment of cross-lingual representations of PLM by automatically learning universal structural information, such as constituent syntactic structure.
The alignment of cross-lingual representation is vital for effective cross-lingual transfer learn-
'Corresponding authors. 'The code is available at https://giithun com/ wwlinjunn/truct-XUu

---

As shown in Figure 1, employ reinforcement learning (RL) to discover these syntactic structures without explicit structure annotations. Following Zhang et al. (2018), we formulate the structure discovery as a sequential decision problem opti- mized through policy gradient and implemented by a masked self-attention mechanism. Addition- ally, we introduce structural multi-head attention mechanism, as shown in Figure 2, to effectively integrate the acquired structural information into the last layer of the multilingual pre-trained LM. Our model follows a Siamese framework and is trained using a policy network (PNet) and trans- lation ranking task, in which the latter provides delayed rewards for RL while enhancing cross- lingual alignment. During the fine-tuning phase for downstream tasks, the parameters of the PNet are kept frozen.
Subsequent works designed more pre-training ob- jectives guide representation alignment with par- allel corpora at different granularities. For instance, Unicoder (Huang et al., 2019) integrates cross- lingual word discovery (CWD) and cross-lingual paraphrase classification (CPC) to learn the under- lying word alignments between two languages and provide an explicit focus the alignment of sen- tences. InfoXLM (Chi al., 2021) adds sentence- level contrastive learning loss for bilingual pairs maximize the mutual information between trans- lation sentence pairs. HICTL (Wei et al., 2021) proposed sentence- and word-level contrastive learning to distinguish parallel sentences and re- lated words. VECO2.0 (Zhang et al., 2023) adds token-level contrastive loss for synonym alignment with the thesaurus dictionary. In addition using parallel corpora, ERNIE-M (Ouyang et al., 2021) integrates Back-translation into the pre-training pro- cess to generate pseudo- pseudo-parallel parallel pairs for monolin- gual corpora. Moreover, ERNIE-M (Ouyang et al., 2021) and VECO (Luo et al., 2021) proposed a cross-attention module that builds interdependence between languages in model inner.
To summarize, our main contributions are as follows.
We propose Struct-XLM, novel multilingual language model that autonomously discovers universal structural information with RL to en- hance cross- lingual alignment without explicit structure annotations.
Existing work has enhanced the alignment of cross-lingual representations at multiple granular- ities (token-level, word-level, and sentence-level), Considering that differences in syntactic structure between languages also affect alignment, we focused on introducing syntactic structural level information into cross-lingual representations enhance alignment. Since syntactic tags are chal- lenging to obtain, especially for low-resource lan- guages, we introduce reinforcement learning meth- ods to learn multilingual structural information without explicit structure annotations and guide the model to learn cross-lingual aligned representa- tions machine translation ranking tasks.
We introduce a structural multi-head atten- tion mechanism that integrates the learned structural information into the last layer of the multilingual PLM for improving cross-lingual transfer ability.
Experiments on the 7 tasks of XTREME, our model shows superior average performance compared to the baseline PLM by 4.1 points and is competitive with the InfoXLMM model using only 1/5000 of the amount of training data.
2 Related Work
2.2 Structure-augmentted Cross-lingual transfer
2.1 Cross-lingual Representation Alignment
Multilingual pre-trained models such as mBERT (Pires et al., 2019), XLM (Conneau and Lample, 2019), and XLM-R (Conneau et al., 2020) have ex- hibited promising abilities of cross-lingual transfer, even in the absence of explicit encouragement of learning aligned representations across languages. These models are typically pre-trained with mul- tilingual masked language modeling (MMLM) (Pires et al., 2019) and translation masked language modeling (TLM) (Conneau and Lample, 2019).
The use of universal dependency parse structures has been found to be beneficial for cross-lingual transfer in cross-lingual NLP tasks (Ahmad et al., 2021; Fei et al., 2020; Zhang et al., 2022; Zhu, 2020; Xu et al., 2022). Many approaches have uti- lized these structures by fusing them with input sequences or incorporating them into self-attention mechanisms enhance the learning of language syntax. Most of the work is bask-oriented(Zhang et al., 2022; Wu et al., 2020; Zhu, 2020; Wu et al.,

---

Delayed Reward
Add & Norm
Structural represemation
Feed Forward
Mean
Add Additive Margin
R
Add
Sigmoid
Action
Structural Multi-
L
are
Head Attention
are
Linear
Maksed
State Sie
1 Layer
Attention
Pretrained LM
Lu
Policy Network
Structural Encoder
T
T
Add Additive Margin
Target Text
Mean
Source Text
Senuctural representation
a
Policy Network
Structural Encoder
Translation Ranking Task
The framework Struct-XLM. The policy network (PNet) samples an action at each state. The structural encoder offers state representation to PNet and outputs the final representation to calculate translation ranking loss, which provides a delayed reward to PNet.
Figure I:
2022b; Xu al., 2022; Wu al., 2023), but Syntax- augmented mBERT (Ahmad et al., 2021) enables structural learning without the inference phase by learning structural encoders and incorporating structural coding into self-attention mechanisms. So it can be generalized to arbitrary cross-lingual understanding tasks. However, these approaches require syntactically labeled data, which can be difficult to obtain in high quality across multiple languages.
adopt a Siamese framework to learn representations from parallel sentence pairs (X, Y), with shared parameters across components. The structural en- coder is initialized with multilingual PLM and provides state representation S to PNet. PNet uti- lizes stochastic policy sample actions at each state, generating action vector a =aaa.... a2..."" for the sentence x x tokens. The action vector can represent the struc- ture of the sentence. As depicted Figure 2, we convert the action vector into an action matrix and feed it into the scaled dot-product self-attention module to construct a structural multi-head atten- tion module in the last layer of PLM. The Struc- tural Encoder then outputs multilingual structural representation for the Translation Ranking Task module calculate the ranking loss and provides delayed reward to PNet. The reinforcement learn- ing process is naturally handled using the policy gradient method (Sutton et al., 1999).
comprising
To address this challenge. we follow Zhang et al. (2018) to utilize reinforcement learning to learn the universal structure for enhancing cross- lingual transfer without explicit structure annota- tions. Zhang et al. (2018) proposed an RL method learn monolingual sentence representation by discovering structure for text classification. We extended their method to improve the alignment of cross-lingual representation.
3 Method
These three components are interleaved together: the state representation of PNet is derived from the Structural Encoder, the calculation of ranking loss relies on the final structured representation, and PNet receives rewards from the Translation Ranking Task module to guide policy learning.
3.1
Overview
In this paper, we propose a framework that inte- grates structural information to enhance the cross- lingual transfer ability of multilingual PLM by im- proving the alignment of cross-lingual representa- tions. The framework of the Struct-XLM model, shown in Figure I, consists three key compo- nents: the Policy Network (PNet), Structural En- coder, and Translation Ranking Task module. We
3.2 Structural Encoder
Our structural encoder initialized by multilingual PLM. For learning structural representation, we propose a structural multi-heas attention module

---

in the last layer of PLM, and one of the structural self-attention flowing is shown in Figure 2. The original attention probability matrix denoted as:
MatMul
0
0
0
0
0
Multiply
QKT Vd
E = softmax
(I)
Softmax
0
where the dot-product is scaled by - dg denotes the dimension of query matrix Q Y key matrix K. In our structural multi-head attention module. the E is not only determined by Q and K but also Action Matrix A generated from the actor vector, which guides each word only pay attention to the words within the same constituent.
Scale
0
0
Action metrix
MatMul
f
0 O
Action vector
Q
K
V
Input 'Tii them competitive edge for the interim period.'
sentence consists of constituents, such as noun phrase or verb phrase. The action vector from PNet provides information about the boundaries of con- stituent structure in sentences ("O' and '1' denote the word is inside or is end boundary of con- stituent), where split the sentence into several con- stituents. In Figure 2, for example, the phrase 'a competitive edge' is identified as constituent e by the sub action vector {0, 0, 1}. This action vector is then converted into a sub-action matrix:
Figure The framework structural self-attention.
3.3 Translation Ranking Task
For enhancing cross-lingual representation, we in- troduce the translation ranking task with in-batch negative sampling (Yang et al., 2019a; Feng et al., 2022):
log
0
1
(, (
A
(2)
0
m
(4)
where A represents ith word pay attention to the jth word in this constituent. There is special case when only one word is divided into a single constituent, its sub-action matrix is an identity ma- trix. The final Action Matrix A is obtained by splicing all the sub-action matrices by the diagonal and filling with 0 the remaining positions. The detail of algorithm can be see in Appendix
The embedding space similarity of sentence pairs and y given by d(x, y) mean (heannhh)mea( m is additive margin around positive pairs, which improves the separation between translations and non- non-translations.
Considering that there are n-way parallel pairs in data, we introduce in-batch labels E IR N to improve the translation ranking loss. 1;j represents yj and x; translation pair and ly 0 represents Yj and 2; non-translation pair. The N batch size. The new loss can be defined as:
By combining the Action Matrix A with the soft- max attention mechanism, we calculate the struc- tural attention probability E as follows:
QKT
(assss) m
E A softmax(
(3)
C
log
N
((: (:) + Σ
where O represents glement-wiss multiplication. In the case of multi-head attention, different heads share the same action matrix A and obtain the structural representation h with the dimension of dmmode xg. The structural representation is then transformed into a sentence embedding using mean-pood pooling. Finally, the translation ranking loss is calculated based on this sentence embedding.
(5)
The loss aims rank the true translations of m;; over all other alternatives in the same batch. L is asymmetric and depends on whether the softmax is applied to the source or target sentences. To achieve bidirectional symmetry. the final loss can

---

Refer to Zhang et al. (2018), our training process consists of three steps. Firstly, we pre-train the be obtained by summing the source-to-target loss, C, and the target-to-sen Srgettto-source loss, L':
ranking loss and reward. The reward is defined
Σ
Lirans = L¹
(6) R = log
j=1,1-
Σ
ed(2.5))
m
n=11=00
3.4 Policy Network
(9) where the 1j is the label indicates the yj whether translation æ. This reward considered de- layed reward since it is obtained after constructing the entire representation.
The policy network includes masked self- attention module, linear layer, and sigmoid layer. The masked self-attention limited each state only to notice the previous state and predict the action in the sequence. We briefly introduce state, action and policy, reward, and objective function follows: State Each state vector S, with a dimension of doodde is each token representation from the Structural Encoder without the input of the ac- tion vector. The complete state representation S (81, ; ST) E I Txxmodd encoded into an action embedding space using a masked self- attention layer and a linear layer.
Objective Function Following Zhang et al. (2018), optimize the parameters of PNet using REIN- FORCE algorithm (Sutton et al., 1999) and policy gradient methods, aiming maximize the expected reward as shown below.
((0) Σ
II Te(a()) (10)
8181 SSaT
and the gradient with likelihood ratio trick is de- fined as follows:
Action and Policy We adopt binary actions (/u- side, End} to discover constituent structure, in- dicating that a word is inside or at the end of constituent. PNet adopts stochastic policy (a,]s;; θ), which represents the probability of selecting action a state The policy is defined as follows:
T
VeJ(H)-
Σ
R1. ve (og))(
(11)
3.5 Training Process
(a)(: o SA(SS) b)
(7) Structural Encoder using the Translation Ranking Task. Then, we pre-train the PNet while keeping the parameters of the other two models fixed. Fi- nally, we jointly train all three components. To (8) alleviate the challenges of training RL from scratch and reduce variance, we also adopt warm-start approach in the first step. Specifically, we collect a corpus of 13.7k sentence pairs from the UD 2.9 Treebank (Zeman et al., 2021), which provides syn- tax tree labels. These labels are only used as heuristic signal to split sentences into constituents solely for the warm start in the first step.
QKT
SA(S) softmax(
dmmadel Q = 5.s, K S VWS
where represents the sigmoid function, + [(WQ, Wh WK), W,b) denotes the parameters of PNet. W WK and WV are the weights for the query matrix Q, key matrix K, and value ma- trix V of masked self-attention layer SA(.), with dimension dmodel. The W and b are the weight and bias of the linear layer. During training, the action is sampled according the probability in Eq. 7. During the test, the action with the maxi- mal probability (i.e., af ar argmaz,a (ar) se; Q)) is chosen obtain the action vector prediction. Reward After sampling all the actions from PNet, the structural representation of a sentence x de- termined by our Structural Encoder. This repre- sentation is then used to calculate the translation
Experiments
4.1 Implementation Details
We used the XLMM-Rarge model as base pre- trained LM to initialize the Structural Encoder, and the whole Struct-XLM model has 564M parame- ters (559M for fine-tuning). To train our model, we collected an 8MB parallel corpus, including 13.7k sentence pairs (English is the source language), and more detail in Appendix A. In order to facilitate

---

Pair
Structured prediction
Question answering
Datasets
XNLI
PAWS-X
POS
NER
XQuAD
MLQA
TyDiQA
AVG
#Langs
15
7
33
40
11
7
9
Metrics
Acc.
Acc.
FI
FI
EM
FI
EM
FI
EM
Model
Xarg-Rtarge (Hu et al., 2020)
79.2
86.4
73.8
65.4
76.6
60.8
71.6
53.2
65.1
45.0
67.7
InfoXTM((CC) et al., 2021)
81.4
73.6
55.2
ERNIE-M(OUyapg al., 2021)
81.9
89.5
73.7
55.3
XY-LENT(Patra ul., 2022)
80.5
89.7
76.8
71.3
53.2
67.1
51.5
VECCO2.0ZZhang et al., 2023)
80.3
88.5
75.4
67.2
78.9
63.7
72.7
54.2
71.1
54.7
70.7
FRNIF-M
80.1
87.9
76.7
65.9
78.2
62.7
72.8
54.6
70.3
52.7
70.2
InfoXLM
82.1
89.3
75.6
79.3
63.8
73.5
72.2
54.6
71.3
Struct-XLM
81.2
90.1
77.6
68.6
79.5
64.1
73.2
54.8
73.1
$6.0
71.8
Table I: Results of tasks XTREME benchmark. The detailed results for each language are in Appendix D. denotes the results from our re-implement.
the smooth update of the policy gradient during training, we introduced a suppression factor that is multiplied to Eq. 11. This factor is set to 0.5. Dur- ing the training process, we employed the Adam optimization algorithm to update the model param- eters. The initial learning rate was set 6e-6 for the first step, which ran for 5 epochs. The second step ran for 2 epochs with a learning rate of 3e-6, and the last step utilized epochs with a learning rate of 3e-6. The batch size is 5 for all steps.
We do not compare our model with Syntax- augmented mBERT due to its small parameter size.
4.3 Evaluation
In our evaluation, we perform experiments on seven multilingual tasks from the XTREME (Hu et al., 2020) benchmark. These tasks cover a wide range of languages. For sentence-pair classification, we evaluate the Cross-lingusl Natrual Language Inference dataset (XNLI) (Conneau al., 2018), and Cross-lingual Paraphrase Adversaries from Word Scrambling dataset (PAWS-X) (Yang et al., 2019b). For structured prediction, POS tagging from the Universal Dependencies v2.5 treebanks (Nvire et al., 2020) and NER from Wikiann (Pan et al., 2017). For cross-lingual question answer- ing: XQuAD (Artetxe et al., 2020), MLQA (Lewis et al., 2020), and the gold passage version of the TyDiQA dataset (TyDiQA-GoldP) (Clark al., 2020). Here, for a cross- lross-lingual setting, all tasks provide English training data and dev/test set in all involved languages. We consider the zero-shot cross- goossllnngull setting, i.e. only using the English data for fine-tuning, The hyper-parameters setting for fine-tuning are shown in Appendix C.
4.2 Baselines
Except for the base model XLM-Rtarge> we com- pare the cross-lingual performance of our proposed model against 4 strong cross-lingual models:
Info-XLM focuses on maximizing mutual in- formation between translation pairs. We com- pare Struct-XLM with its large size version (558M parameter), which continues training Xare on 42GB parallel corpora.
ERNIE-M incorporates cross-attention masked language modeling on both parallel and monolingual corpora. The parameter size of the large version model and the training data is the same as Info-XLM.
XY-LENT leverages novel sampling strate- gies with X-Y bitexts to learn the cross-lingual alignment representation. It has twice big- ger vocabulary size than XLM-Rtarge but with smaller parameter size (477M for the base version) by training on ELeCTRA-style tasks.
4.4 Results
In Table 1, we present the results of Struct-XLM and baselines on the seven tasks of XTREME. The results demonstrate that Struct-XIM outperforms XLM-R Rtarge and latest model VECO2.0 (Zhang et al., 2023) on all tasks, with an average improve- ment of 4.1% and 1.1% respectively. Compared to the best strong baseline InfoXLM (Chi et al., 2021), we have a competitive average performance and we only use 8MB of training data, which is
VECO2.0 bridges the representations of syn- onym pairs embedded in the bilingual corpus based on VECO. It has 559M parameters and is trained with 2.5TB monolingual data and 4TB parallel pairs.

---

Pair sentence
Structured prediction
Question nnswering MLQA(F)
Dulasets
XNLI
PAWS-X
POS
NER
XQUADIPI)
XQUADIEM)
MLQMEM)
TYDIQMFII
TYDIQAIEM)
AVG
2300
14.5
Info et 202()
8.9
12.7
19.1
ERNIE-MODryang 2021)
7.9
7.5
12.5
18,9
XY-LENT(Pun 2022)
VECOZ.11
9.2
8.6
21.4
18.1
10.2
16.2
13.3
20.11
2.2
6.8
12.6
ERNIF-M!
888
9.3
200
184
10.1
15.0
12.8
19.0
4.0
7.0
12.4
12.7
Strac-XLM
97
6.9
19.0
15.8
9.5
15.6
13.1
19.6
2.9
9.6
12.2
Table 2: The transfer gap of different models XTREME tasks, where the lower the score the better transferability.
Pair sentence
Structured prediction
Question answering
Datasets
XNLI
PAWS-X
POS
NER
XQuAD
MLQA
TyDiQA
AVG
Metries
Acc.
Acc.
F1
EM
EM
FI
EM
XLM- (Hu al., 2020)
79.2
86.4
73.8
65.4
76.6
60.8
71.6
53.2
65.1
45.0
67.7
Struct-XLM
81.2
90.1
77.6
68.6
79.5
64.1
73.2
54.8
73.1
56.0
71.8
action
80.4
89.4
76.1
66.7
78.9
63.5
54.2
71.6
54.1
70.7
w/o warm-start
79.7
89.0
76.3
65.6
78.6
62.8
72.0
54.0
71.8
53.7
70.4
only warm-start
79.7
88.5
75.0
64.5
78.1
62.4
71.7
53.8
70.8
52.8
69.7
w/o PNet
79.4
88.3
74.9
62.9
77.5
61.8
71.3
53.3
70.9
52.8
69.3
Table 3: Ablation results on each model. The detailed results for each language are in Appendix D.
1/5000 parallel data used by InfoXLM. Struct- XLM also surpasses XY-LENT on 8/10 metrics, with an average advancement of 2.5%.
Model
de
fr
zh
avg
XLM-R XLM-Riarge
Rtagge
67.5
66.5
56.7
66.0
VECO2.0
90.5
86.1
88.1
80.2
86.2
In sentence-pair classification tasks, Struct-XLM performs slightly better than the best model XY- LENT on the PAWS-X by 0.4%. However, on the XNLI, it is inferior to Info-XLMM by 0.9%. This may be because the translation ranking task in Struct-XLM focuses more on detecting meaning equivalence rather than fine-grained distinctions of meaning overlap. For structured prediction tasks, Struct-XLM outperforms all baselines and has the lowest transfer gap shown in Table 2, which can be attributed the structure discovery. In question answering, Struct-XLM, is inferior to the best EM score by 0.5% on MLQA. However, it achieves the best performance on the XQuAD and TyDiQA datasets. surpasses InfoXLM by 0.9% and 1.4% in EM and Fl metrics on TyDiQA, respectively. Overall, Struct-XLM delivers impressive perfor- mance with only an SMB parallel corpus. The improvement can be attributed the multilin- gual structured representation alignment achieved through RL and the translation ranking task. Fur- ther investigation and analysis will be conducted in the ablation study and analysis experiments.
XLM-R
82.9
72.5
66.1
75.3
large
InfoXLM
92.7
89.1
92.2
81.6
88.9
Struct-XLM
93.5
90.4
92.2
84.4
90.1
Table 4: BUCC scores for each language. the results from re- ie-mmmmmentttion.
denotes
between the English test set and the average perfor- mance across other languages. It is observed that the transfer gap of Struct- is relatively high for certain metrics, such as accuracy in XNLI and PAWS-X, X, and EM in TyDiQA This is because the model's performance in English has significantly improved. However, Struct-XLM demonstrates lower average transfer gap than baselines, indicat- ing improved cross-lingual transferability.
5.2 Ablation study
To explore the impact PNet and RL, we fine-tune the Structural Encoder Part (i.e. w/o action) or the model only trained by the first step (i.e. only warm- start). To discuss the significance of the warm-start step, we also evaluate the ablation model without it (w/o warm-start). Moreover, train language model that only has Struetural Encoder and Trans- lation Ranking Task module (i.e. w/o PNet). The results are shown in Table 3, and the results of Struct-XLM in 7 tasks always be the best. Specif- ically, the Struct-XLM only trained on the first
5 Analysis
5.1
Transfer Gap
Table presents the transfer gap of Struct-XL.M and baselines on the XTREME benchmark. The transfer gap refers to the difference in performance

---

HOXLM
consistently achieves high retrieval Fl scores even in the last few layers. Although the last layer of Struct-XLM without the action component (i.e., w/o action) exhibits a lower FI score, still ben- efits from enhancing the aligned knowledge of other layers to achieve excellent performance on XTREME.
Structural probe (Hewitt and Manning, 2019; Chi al., 2020) demonstrates that the middle lay- ers of PLMs capture richer syntactic information. Struct-XIL obtain significant improvements the middle layers, indicating that it has learned structural information and provides better-aligne representations compared to XLM-R targe-
Figure 3: Evaluation results (FI score) of different lay- ers on BUCC cross-lingual sentence retrieval.
step (i.e. only warm-start) surpasses the language model without PNet (i.e. w/o PNet) with an av- erage advancement of 0.4%, which indicates the significance of the simple structural inspire infor- mation to improve the cross-lingual transfer abil- ity. The evaluation of the model without action (i.e. w/o action) also obtains greater improve- ment than baseline XLM-RRarge by 3.0% on the average. This suggests that RL training greatly enhances the cross-lingual transfer ability of the multilingual PLM, even without the subsequent use of the PNet module in fine-tuning. The model w/o warm-start doesn't surpass the Struct-XLM model, confirming the effectiveness of our warm- start procedure. From the results, we can observe that all components and training settings contribute to overall performance improvement.
Model
Pen
Flas
Put
Ryg
Flas
Struct-XLM
34,49
66.84
45.50
35.67
57.58
43.14
w/o warm-star
30.57
62.13
40.98
31.63
56.48
39.67
Table 5: The precision(P), recall(R), and Fl scores of the constituent boundary prediction.
5.4 Analyze of Discovered Structure
Quantitatively Analyze Struct-XLLM aim to quire structural information that facilitates cross- lingual alignment, we employ retrieval tasks in Sec- tion 5.3 to quantitatively analyze whether the infor- mation obtained through RL contributes to cross- lingual alignment. Furthermore, we try to conduct quantitative analysis of the discovered structure by evaluating the constituents' boundary predicted by the action vector. The labeled test set is from the warm-start step, including English (en) and tar- get languages (tgt) two-way. where the recall of the boundaries predicted by Struct-XIM 66.84% and 57.58% in English and target language, respec- tively, which suggests that Struct-XLM is able to correctly predict the constituent boundaries in the labeled test set to some extent. Perhaps Struct- XLM tends to predict more constituents, it gets low precision (P). We also report the results of the model w/o warm-start prove the influence of the warm-starr step for structural discovery.
5.3 Cross-Lingual Representations
To assess the alignment of cross-lingual sentence representations obtained from our Struct-XLM, we introduce the BUCC sentence retrieval task. Ta- ble 4 presents the F1 scores of four languages on BUCC, where the representations of our model and XLM-Riarge and InfoXLM are extracted from the middle layer (13th layer) following the choice in Hu et al. (2020). The evaluation results demon- strate that Struct-XLM produces better-aligned cross-lingual sentence representations compared XLM- Laagee and InfoXLM, achieving an aver- age performance improvement of 14.8% and 1.2%. respectively.
Qualitatively Analyze Table 6 presents some inter- esting structures discovered by Struct-XLM. Some constituents can form sentence with complete meaning, such as " 4 (Passengers ride the gondola), while others may consist of just single word. In these examples, the structures discovered by Struct-XILM exhibit more flexibility in terms of length and amount of constituents com- pared to predefined structures. In the case of En-
Furthermore, we analyze the average results of all layers of the models BUCC, as depicted in Figure 3. For XLM-Riarge and InfoXLM, we ob- serve a performance drop in the last few layers, which is expected as these layers are encouraged to focus on toven-level embedding due to the pre- trained objectives. On the other hand, Struct-XLMM

---

Lang.
Semeetree
Sentence
Predetined
The fessibility stuty] estimates would take passengers about four minutes so cross the Potomac River on the gondola.
from XLMM
The feasibility study estimates that it would take passengers about four minutes to cross the Potomac River on the gondela.
Predefined from Sinct-XLMM
matkustajikaa kuluis noin nelja minuuttia Potomar River joen ylitimiseen kontissillaa Tocuteeliacusttekkik iinkkkkkkikese sceea arvicidaan, clt milusstjjlla kulaisi
Togquetteeustrnkikkkee arricciane,
noin netai minuotiaa Potorac
River -joon ylitmmissenn konthisillla
Prodefined
Der Machharkiitsttuic zufolge
könnten die Passagirre den Potomac River mit der Gondel in ungefähr 4 Minuten lierqqurees. Der Macharkketssstudie zufolge kionnten Passagiere den Potomac River mit der Gondel in ungefiinr Minuten therqueren.
from Sinct-XLLM
Predelined
O estudo de viabolidade
estima que passageiros iriam levar quatro minutos para atravessar Rio Potomac em a göndola. estima que passagetios iriom levar quatro minutos pam atrovessur Rin Potornac em goodola.
from Sinct-XLMM
estado de viabilidade
Prodefined
Na podatawic wyliczen podatawio wyliczen
stacuje sip. 2s przokroczenie tacki Potomac gondola zajokoby pasažerrm okolo caterech minut.
from Stract XLM
stacuje sip, to przekroczenie rocki
Potomac gondela zujekoby passkerom okolo
caterech minut.
Predefined
RR:-- # WHRR 3.8 12// 9+81
SInadtXX.M
W1: 98 MEAEE (ERR HERES (5/0
Table 6: The comparison of the predefined structures and those discovered by Struct-XLM in multiple languages.
glish, being a source language aligned with multi- ple languages, the number of constituents obtained tends to be smaller. However, Struct-XL.M some- times struggles to identify the correct boundaries between phrases, seen in the example where the term "Potomac River" in Finnish (fi) is split into two parts. This task is extremely challenging for any model without explicit structure annotations.
Limitations
While Struct-XLM presents advancements in im- proving cross-lingual representation alignment, it also has some limitations. We outline these limita- tions in this section.
Training Data
The strong baseline models we compare against may benefit from large amounts of parallel data for training, which contributes to their superior performance. Since policy gradient RL learning re- quires multiple sampling learning for each sample, our model takes about 30 hours to complete three stages of training on an NVIDIA RTX A6000 using 8MB of training data. In our case, due to compu- tational constraints, a limited amount of parallel data (8M) for train- ing. Increasing the amount of parallel data without labels in the last two steps could potentially im- prove the performance of our model and provide a better-aligned cross-lingual representation.
6 Conclusion
In conclusion, we introduced Struct-XLM, a novel approach to enhance cross-lingual transfer learning in multilingual language models. By leveraging reinforcement learning without explicit structure annotations, the Struct-XLM discovers structures and improves the alignment of cross longull repre- sentations. Specifically, we employ policy gradi- ent RL and a translation ranking task to discover constituent syntactic structures and enhance cross- lingual transfer. Experimental results on seven XTREME benchmark tasks demonstrated the ef- fectiveness of StrMet-LLM, outperforming baseline PLM models with an average improvement of 4. points and being competitive with the best strong baseline with small training data. Our analysis also revealed that Struct-XLM improves performance on the sentence retrieval task BUCC, particularly in the middle layers of the pre- pre-trained model, indi- cating the acquisition of valuable structural infor- mation for better-aligned representations.
were only able to utilize
Learning Structure from Middle Layer
As mentioned, the structural probe (Chi et al., 2020) has shown that the middle layer of multilingual PLMs captures rich syntactic information. How- ever, it is worth considering that syntax and seman- tics are intertwined, and the primary goal of cross- lingual alignment is semantic alignment. There- fore. we explore whether structures learned from the final layer, which captures more semantic infor- mation, can enhance alignment. structures dis- covered from the middle layer representations can further improve alignment, it would provide addi- tional evidence for the effectiveness of our method,
Acknowledgement
This work is supported by the Fundamental Re- search Funds for the Central Universities (No. 226- 2023-00060), the Key Research and Development Program of Zhejiang Province (No. 2021C01013). National Key Research and Development Project of China (No. 2018AAA0101900, and MOE En- gineering Research Center of Digital Library.
Siamese Framework vs. Single Tower Version
The Siamese framework used in Struct-XLM. which employs separate encoders for parallel sen-

---

tence pairs, is effective in capturing cross-lingual alignment (Guo et al., 2018; Yang et al., 2019at 9a; Feng et al., 2022). However, its token-level align- ment may not be sufficient. An alternative ap- proach could involve concatenating the parallel sen- tence pairs into single input, allowing for more comprehensive learning from the parallel corpus and potentially encouraging the policy network to capture more universal structures through attention mechanisms.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke 7ettle- moyer, and Veselin Stoyanov. 2020. Unsupervised tross-lingual representation learning at scale. In Pro- ceedings of the 58th Annual Meeting of the Associa- tion for Computational Linguistics, ACI. 2020, pages 144008451.
Alexis Conneau and Guillaume Lample. 2019. Cross- lingual language model pretraining. In Proceedings of the 2019 Annual Conference on Neural Informa- tion Processing Systems, NeurlPS 2019, pages 7057- 7067.
Overall, these limitations provide opportunities for future research to further enhance the perfor- mance and capabilities of cross-lingual representa- tion learning models like Struct-XLM.
Alexis Conneau, Ruty Rinott, Guillaume Lample, Ad- ina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: evaluating cross- lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natu- ral Language Processing, Brussels, Belgium, Octo- ber 31 November 4, 2018, pages
References
Wasi Uddin Ahmad, Haoran Li, Kai-Wei Chang, and Yashar Mchdad. 2021. Syntax-augmented multilin- gual BERT for cross-linge lingual transfer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. ACL/JJNLPP 2021. (Volume 1: Long Papers), Viriual Event, August 1-6, 2021, pages 4538-4554.
Hao Fei, Meishan Zhang, Fei Li, and Donghong Ji. 2020. Cross-lingual semantic role labeling with model transfer. IEEE ACM Trans. Audio Speech Lang. Process. 28:2427-2437.
Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Ari- vazhagan, and Wei Wang. 2022. -aggust BERT sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Compu- national Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Treland, May 22-27, 2022, pages 878-
Lars Ahrenberg. 2007. Lines: An english-swedisg par- allel treebank. In Proceedings of the 16th Nordic Conference of Computational Linguistics, NODAL- IDA 2007, Tartu, Estonia, May 2007, pages 270-273.
891.
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. On the cross-lingual transferability of mono- lingual representations. In Proceedings of the 58th Annual Meeting of the Association for Computa- tional Linguistics, ACL 2020, Online, July 5-10, 2020, pages 4623-4637.
Mandy Guo, Qinlan Shen, Yinfei Yang, Heming Ge, Daniel Cer, Gustavo Hernández Abrego, Keith Stevens, Noah Constant, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Effective parallel corpus mining using bilingual sentence embeddings. In Proceedings of the Third Conference on Machine Translation: Research Papers, WMT 2018, Belgium, Brussels, October 31 November 1, 2018, pages 165- 176.
Ethan A. Chi, John Hewitt, and Christopher D. Man- ning. 2020. Finding universal grammatical relations in multilingual BERT. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pages 5564-5577.
John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word represen- tations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, NAACL-HLT 2019, pages 4129-4133
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal. Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou. 2021. Infoxim: An information- thhooreticc framework for cross-lingans lingull language model pre-training. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, NAACL-HLL 2021, On- line, June 6-11, 2021, pages 73888
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Gra- ham Neubig, Orhan Firat, and Melvin Johnson. 2020. XTREME: A massively multilingual multi- task benchmark for evaluating cross-lingual gener- alisation. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, pages 4411 4421..
Jonathan H. Clark, Jennimaria Palomaki, Vitaly Niko- laev, Eunsol Choi, Dan Garrette, Michael Collins. and Tom Kwiatkowski. 2020. Tydi QA: A bench- mark for information-sceking question answering in typologically diverse languages. Trans. Assoc. Com- put. Linguistics, 8:454-470.
Haoyang Huang. Yaobo Liang, Nan Duan, Ming Gong. Linjun Shou, Daxin Jiang, and Ming Zhou. 2019.

---

Unicoder: A universal language encoder by pre- training with multiple cross lingual tasks. In Pro- ceedings of the 2019 Conference on Empirical Meth- ods in Natural Language Processing and the 9th In- ternational Joint Conference on Natural Language Processing, EMNLP-IICNEI 2019, Hong Kong, China, November 3-7, 2019, pages 2485-2494.
Xiangpeng Wei, Rongxiang Weng. Yue Hu, Luxi Xing. Heng Yu, and Weihua Luo. 2021. On learning uni- versal representations across languages. In 9th In- ternational Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021.
Linjuan Wu, Shaojuan Wu, Xiaowang Zhang, Deyi Xiong. Shizhan Chen, Zhiqiang Zhuang, and Zhiy- ong Feng. 2022a. Learning disentangled semantic representations for zero-shot cross- mosss-lnguul transfer in multilingual machine reading comprchension. In Proceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), ACI.2022, Dublin, Ireland, May 22-27, 2022. pages 991-1000
Patrick S. H. Lewis, Barlas Oguz, Ruty Rinott, Sebas- tian Riedel, and Holger Schwenk. 2020. MLQA: evaluating cross-lingual extractive question answer- ing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7315-7330.
Fuli Luo, Wei Wang. Jiahao Liu, Yijia Liu, Bin Bi, Song- fang Huang, Fei Huang, and Luo Si. 2021. VECÓ: variable and flexible cross-lingual pre-training for language understanding and generation. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1tth International Joint Conference on Natural Language Processing, ACL/JCCLP 2021, (Volume 1: Long Pupers), Virtual Event, August 1-6, 2021, pages 3980-3994.
Yiquan Wu, Kun Kuang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Jun Xiao, Yueting Zhuang, Luo Si, and Fei Wu. 2020. De-biased court's 's view generation with causality. Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 763-780.
Yiquan Wu, Yifei Liu, Weiming Lu, Yating Zhang. Jun Feng, Changlong Sun, Fei Wu, and Kun Kuang. 2022b. Towards interactivity and interpretability: A rationale-basee legal judgment prediction framework. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 4787-4799
J. Nvire, M. Abrams, et al. 2020. Universal dependen- cies 2.2.
Xuan Ouyang, Shuohuan Wang, Chao Pang. Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2021. ERNIF-M: enhanced multilingual representation by aligning cross- loogull semantics with monolingual corpora. In Proceedings of the 2021 Conference on Empirical Methods in Natural Longuage Processing, EMNLP 2021, Virtual Event Punta Cana, Domini- can Republic, 7-11 November, 2021, pages 27-38.
Yiquan Wu, Weiming Lu, Yating Zhang, Adam Jatowt, Jun Feng, Changlong Sun, Fei Wu, and Kun Kuang. 2023. Focus- aware response generation in inquiry conversation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12585- 12599.
Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Noth- man, Kevin Knight, and Heng Ji. 2017. Cross- lingual name tagging and linking for 282 languages. In Pro- ceedings of the 55th Annual Meeting of the Asso- ciation for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 August 4, Volume 1: Long Papers. pages 1946-1958.
Liyan Xu, Xuchao Zhang, Bo Zong. Yanchi Liu, Wei Cheng, Jingchao Ni. Haifeng Chen, Liang Zhao, and Jinho D. Choi. 2022. Zero-shot cross- looss-liggual ma- chine reading comprehension via inter-sentence de- pendency graph. In -iithh AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Four Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 March 1, 2022, pages 11538-11546.
Barun Patra, Saksham Singhal, Shaohan Huang, Zewen Chi, Li Dong, Furu Wei, Vishrav Chaudhary, and Xia Song. 2022. Beyond english-centric bitexts for better multilingual language representation learning. CoRR, abs/2210.1442
Yinfei Yang, Gustavo Hernández Ábrego. Steve Yuan, Mandy Guo, Qinlan Shen, Daniel Cer, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil, 2019a. Im- proving multilingual sentence embedding using bi- directional dual encoder with additive margin soft- max. Proceedings of the Twenty- Eightt Interna- tional Joint Conference on Artificial Intelligence, IJ- CAI 2019, Macao, China, August 10-16, 2019, pages 3370-5378
Telmo Pires, Eva Schlinger, and Dan Garrette. 2019, How multilingual is multilingual bert? In Proceed- ings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, pages 4996- 5001.
Richard S. Sutton, David A. McAllester, Satinder Singh, and Yishay Mansour. 1999 Policy gradient methods for reinforcement learning with function approxima- tion. In Advances in Neural Information Processing Systems 12, INIPS Conference, Denver, Colorado, USA, November 29- December 4. 1999], pages 1057- 1063.
Yinfei Yang. Yuan Zhang, Chris Tar, and Jason Baldridge. 2019b. PAWS-X: A cross-lingual ad- versarial dataset for paraphrase identification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the

---

9th International Joint Conference on Natural Lan- guage Processing, EMNLP-IJCNE 2019, Hong Kong, China, November 3-7, 2019, pages 3685-3690.
A Statistic of Training Data
The training data for Struct-XLM is sourced from various treebanks in UD 2.9 (Zeman et al., 2021). In particular, we select 20 languages (including English) parallel universal dependencies treebanks from PUD? Atis", LinES (Ahrenberg, 2007), and PurTUT treebanks. English is the source language. 'he statisties on the number of sentences available for each language are shown in the table 7.
Daniel 7eman, Joakim Nivre, Mitchell Abrams, et al. 2021. Universal dependencies 2.9. LINDAT/CLARIAH-CZ digital library at the Insti- tute of Formal and Applied Linguistics (ÚFAL), Fac- ulty of Mathematics and Physics, Charles University.
Qi Zhang, Jie Zhou, Qin Chen, Qingchun Bai, Jun Xiao, and Liang He. 2022. knowledge-enhane -enannee-eanneeddd adver- sarial model for cross-lingual structured sentiment analysis. In International Joint Conference Neu- ral Networks, IJCNN 2022, Padua, Italy, July 18-23, 2022, pages 1-8. IEEE.
B Algorithm
The algorithm is to convert the action vector to the action matrix, as mentioned in Section 3.2.
Tianyang Zhang, Minlie Huang, and Li Zhao. 2018. Learning structured representation for text classifi- cation via reinforcement learning. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applica- tions of Artificial Inteligence (IAAI-18), and the 8th AAAI Symposium Educational Advances in Artifi- cial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 6053-6060.
C Hyper-parameters for Fine-Tuning
In Table 8, we report the hyper-parameters for fine- tuning Struct-XLM on the XTREME seven tasks.
Algorithm 1 Convert the action vector to the action
Zhenru Zhang. Chuangi Tan, Songfang Huang, and Fei Huang. 2023. VECO 2.0: Cross-lingual language model pre-training with multi- granularaaul contrastive learning. CoRR, abs/2304.08205.
matrix
Input: action vector a, length of sentence I.
Output: action matrix A
1: number of constituents p ==
2: the list of constituents C, in which ith constituents include start index zturt and end index and
Xingran Zhu. 2020. Cross-lingual word sense disam-
biguation using mbert embeddings with syntactic de- pendencies. CoRR, abs/2012.05300.
3: Cp
4: for j
do
5: a end
then
6:
=j --1
Appendices
7:
p+ =
8:
if j < L then
9:
c Attrrt
10:
end 11: end if
Lang-
Train
Dev
Test
English(en)
12038
882
800
12: end for
13: e
L-1 14: for in do
Swedish(sv)
1634
406
348
15: if etatart == and then
Italian(it)
1528
76
109
16:
sub-action matrix A.;
[1]
French(fr)
951
17:
else
Turkish(tr)
686
40
18:
Ar,
where 1.1 Rex" and
German(de)
502
20
20
and Co gaart + 1 the length of constituent G.
Chinese(zh)
502
20
20
19:
end iisentence-wise sub-action matrix As,
Japanese(ja)
502
20
20
Finnish(fi)
(0' A., J.here R (=x(1--<"111
E Rexccturt and Or E
Arabic(ar)
502
20
20
Spanish(cs)
502
20
20
20: end for
Indonesian(id)
502
20
20
21: return
Korean(ko)
502
Ae
Thai(th)
Agg
Czechícs)
502
20
20
A
Russian(ru)
502
20
20
Åcp
Hindi(hi)
502
20
20
Polish(pl)
502
Portuguesc(pt))
502
20
Icclandic(is)
211
20
20
2 hhtts:://gtthb wttpp://gituuu..oo/ooo////////ea htwp://htwt..oooooo//wnnneneoooo dataset/
Table The statistics of training data in each language.

---

D Results for each task and language
We show the detailed results for all tasks and lan- guages in Tables 9 (XNLI), 10 (PAWS-X), 11 (POS), 12 (NER), 13 (XQuAD), 14 (MLQA), 15 (IyDiQA-GoldP). denotes the results from our re-implement.
Based on the experimental results in Table 9-15, even languages without warm-start training data can show improvements. For instance, in Table 11, Basque(eu), Tagalog(t),, and Yoruba(yo) do not have training corpora, yet they exhibit signifi- cant improvements compared to the best baseline result, with increases of 1.2%, 1.9%, and 19.2%, re- spectively. Moreover, in Table 12, Persian(fa) and Malay(ms) without warm-start training corpora im- prove by 3.5% and 2.7% than the best baseline result.

---

XNLI
PAWS-X
NER
POS
XQUAD/MIQQ
TYDiQA
Batch size
32
[16,32]
[16,32]
(8,16,32)
[16,32]
(16,32)
Learning rate
[6,8]e-6
(8.9.10,20)e-6
(6.... -9ee-
[6,... -9ee-
16,8,10,20)|-6
(9,10,20)c-6
Warmup Epochs
(12500,5000) steps
(5%,10%)
(5%.10%%,19)
(5%.10%)
10%
10%
10
(5.10)
(5,10)
[5,10)
(2.3,4)
[5,10,20]
Table 8:
used for fine-tuning on the downstream tasks.
Lang.
en
ar
bg
de
el
fr
SW
th
tr
ur
vi
zh
AVG
XLM-Ritrg: (lu 2020)
88.7
77.2
83
82.5
80.8
83.7
82.2
75.6
79.1
71.2
77.4
78
71.7
79.3
78.2
79.2
InfoXLMiChi al. 2021)
89.7
84.5
85.5
84.1
83.4
84.2
81.3
80.9
80.4
80.8
78.9
80.9
77.9
74.8
73.7
81.4
InfoXIM
90.1
80.2
84.5
86.33
83.4
85.8
85.7
78.9
81.1
75.6
79.0
81.1
74.8
82.7
82.7
82.1
ORNIE-M(Ouyang et al., 202
89.3
81.2
84.5
84.4
83.7
85.7
85.1
78.6
82.0
76.2
79.2
81.2
75.4
81.9
80.5
81.9
ERNIB-M X
88.3
79.1
83.1
82.8
82.4
84.0
82.7
77.2
80.4
74.2
76.8
78.4
72.4
80.6
78.8
80.1
XY-LENT(Purra al., 2022)
87.7
79.7
83.0
83.7
75.5
77.9
79.3
80.3
VBC02.0 Sihha.,yZhang al., 2023)
88.9
79.1
83.4
82.7
83.2
76.7
80.7
71.9
77.8
79.5
72.8
80.6
79.9
80.3
Struct-XLM
84.3
w/o action
78.7
83,2
84.3
82.2
83.4
79.3
78.2
79.4
80.9
80.4
w/o 88.9
78.5
82.9
82.4
81.4
84.0
82.9
75.9
79,8
71.2
77.4
77.8
73.0
79.3
79.9
79.7
only warm-sttart
88.9
78.5
83.2
82.8
81.8
84.4
82.7
76.0
79.6
71.6
77.4
78.4
72.1
79.6
78.9
79.7
w/o PNet
88.8
78.1
82,9
82.8
81.6
83.9
82.7
75.7
79.0
714
77.0
77.7
71.4
78.8
78.7
79.4
Table 9: XNLI accuracy scores for each language.
Lang.
ko
zh
AVG
XLr al., 2020)
94.7
89.7
90.1
90.4
78.7
79.0
82.3
86.4
InfoXLM
95.1
85.4
86.6
89.3
ERNIF.M!
95.9
91.0
92.1
91.4
80.0
82.1
82.7
87.9
ERNIE M(Onyag et 2021)
96.0
91.9
91.4
92.2
83.9
84.5
86.9
89.5
LENT((aa al., 2022)
95.5
92.3
92.5
93.2
84.0
83.7
86.7
89.7
VECO2 OZZhng et al., 2023)
95.8
91.6
92.0
81.6
84.5
Strnet-XL MMnet-XI.MM
91.7
92.4
85.0
wlo action
96.0
91.3
92.1
84.6
85.0
85.6
89.4
w/o warm start
95.9
92.2
91.7
92.5
82.4
82.8
85.5
89.0
only warm start
95.6
91.2
91.8
91.9
82.4
81.6
84.7
88.5
weo
91.6
92.1
82.2
83.8
88.3
Table 10: PAWS-X accuracy scores for each language.
Lang
en
af
ar
bg
de
e5
ct
cu
fa
fi
fr
he
hi
hu
id
XLM-RRe-y.HHo 2020)
96.1
89.8
67.5
88.1
88.5
86.3
88.3
86.5
72.5
70.6
85.8
87.2
68.3
76.4
82.6
72.4
89.4
VECO2.0<Zhang al., 2023)
96.2
89.4
70.0
88.6
89.7
86.6
89.0
87.2
75.1
71.4
86.1
87.7
70.2
74.7
84.2
72.8
89.8
InfoXI.M
96.0
88.6
69.2
88.1
88.9
87.3
88.4
85.7
74.5
71.6
85.8
87.
71.9
69.3
83.1
72.7
88.3
ERNIF-M
85.5
72.2
72.3
89.7
71.6
69.9
72.9
w/o action
96.1
88.9
69.3
89.0
89.2
89.9
87.4
76.1
71.2
86.2
89.4
65.5
73.4
83.0
73.3
w/o warm-start
96.0
89.7
69.6
88.9
88.9
87.6
89.2
86.9
72.7
71.0
86.1
88.3
70.4
77.7
83.4
72.9
89.7
only warm-start
95.8
88.5
67.5
88.3
89.1
87.1
88.6
86.7
74.2
70.5
85.3
87.9
66.8
72.9
82.3
72.9
89.9
PNet
95.7
88.4
85.3
73.3
72.9
ja
mnr
ru
ur
yo
XLM (Hu et al., 2020)
15.9
78.1
53.9
80.8
189.5
87.6
89.5
65.2
86.6
47.2
92.2
76.3
70.3
56.8
24.6
25.7
73.8
VECO2.0ZZaagg al., 2023)
36.2
78.3
53.4
84.7
189.8
85.8
89.8
64,9
84.5
50.9
93.3
76.8
67
58.8
23.2
40.7
75.4
InfoLMM
78.0
889.3
92.9
58.7
ERNIF-M!
54.7
79.0
53.8
84.0
89..4
88.6
89.7
61.9
86.4
58.1
92.1
77.5
72.6
59.2
24.2
63.7
76.7
Struct XLM
39.2
80..7
52.7
85.5
990.1
911.2
90.6
67.8
86.7
57.1
95.2
77.2
73.2
60.2
45.9
52.3
77.6
w/o action
34.1
79.7
51.8
84.2
889.3
85.5
90.2
66.2
87.0
53.2
93.1
76.5
70.4
59.5
40.7
39.7
76.1
wlo warm- start
38.3
79.2
53.4
83.6
89.7
88.7
90.1
64.1
86,9
54.4
94.0
76.7
73.0
60.3
26.1
51.4
76.3
only w/o PNet
30.2
78.7
84.5
89.2
89.7
94.1
36.6
77.8
50.5
83.1
88.9
87.9
88.8
62.8
82.1
50.4
95.1
75.9
68.5
58.4
34.9
43.2
74.9
Table 11: POS results (Accuracy) for each language

---

Lang
bn
de
In
tr
hcc
bi
ha
i4
11
jv
XLM--Ra-y-lRR (Hu et al. 2020)
84.7
78.9
53.0
81.4
75.5
78.8
79.5
79.5
79.1
00.9
61.9
79.2
50.5
56.8
73.0
79.8
53
81.3
23.2
62.5
0EZ022002hang 2023)
84.8
51.8
81.3
90.7
75.8
66.3
81.1
M0.7
56.2
73.1
82.1
66.1
InfoXLM
$/.6
$1.0
57.3
83.6
77.1
81.1
$1.5
81.6
78.9
68.5
53.5
$1.5
82.3
56.9
72.9
$1.7
54.4
82.7
28.7
66.1
ERNIE-M!
5544
79.4
77.2
Stmic-XI.M N.M
84.0
60.7
59.1
83.9
79.6
80.8
81.2
814
79.2
67.4
69.8
81.8
81.1
58.6
72.6
81.1
55.5
82.1
29.6
67.2
wio action
84.0
54,4
82.5
78.8
79.4
78.6
64.3
66.4
81.0
50.3
55.3
81.4
63.8
w/o waatt-sartt
82.8
78.1
59.1
79.3
73.8
77.9
77.8
73.5
79.6
61.9
62.6
79.9
78.8
54.1
70.8
79.2
52.6
81.7
20.1
62.8
only wart- start
82.6
80.0
47.2
80.9
75.4
78.6
77.3
77.6
79.2
67.3
61.1
80.7
79.6
50.9
69.4
80.2
51.9
81.5
19.2
65.3
w/o PNer
81.6
50.1
79.6
75.3
77.7
77.0
75.9
76.4
59.6
54.3
78.7
79.4
66.9
78.7
54.1
80.8
61.4
ka
1kk
kn
mn
mn
m1
vi
2/h
XLM HHu et al., 2020)
21.6
56.2
60
67.8
68.1
57.1
54.3
54
81.9
09.1
70.5
59.5
55.8
1.3
73.2
76.
56.4
79.4
33.6
33.1
VECO2 OrZaang al., 2023)
72.8
54.3
78.0
InfoXI ERNIE-M
70.5
57.3
60.1
64.6
67.5
71.6
58.6
85.2
83.1
74.6
71.7
59.7
55.4
2.1
75.22
K1.9
71.9
77.4
50.0
38.3
69.5
55.4
56.7
67.9
70.1
83.9
81.6
71.2
70.7
54.3
76.0
76.8
30.4
SructtXL.M
72.0
56.7
59.7
68.2
68.9
74.3
58.1
84.7
83.0
72.6
70.5
00.6
56.8
3.0
75.9
$1.5
70.7
78.2
58.3
32,9
who action
70.1
54.4
60.6
65.1
65.4
69.4
57.6
83.8
82.6
71.1
69.9
60.8
56.0
2.7
73.2
80.9
69.1
78.7
39.0
34,4
w/o statt
69.4
51.2
57.8
68.2
70.2
66.0
58.3
82.0
79.5
65.7
70.0
58.3
56.4
1.4
75.0
79.0
66.4
75.8
52.5
34.4
only warm-start
61.1
52.7
53.6
62.9
64.4
69,4
50.2
83.7
82.0
68.4
69.4
55.6
51.6
2.1
72.1
N0.7
62.2
75.8
42.4
26.7
w/o PNct
60.5
52.7
68.7
83.5
66.2
68.9
50.5
70.8
79.7
75.5
22.6
Table 12: NER results (FI Score) for each language
Lang:
XIM-Rarg-(HHe (H 2020)
86. 56.7775.7
68.6499.
80.48334
79.8661.7
82.0063.9
76.7/59.7 7
80, 1/64.3
74 74.28228
75.9/59
70 179.0
59.3/50.0
76 69608
XY-LENT(Puru 2022)
N7.2776.0
72.9/560
500664.5
79.663.5
81.2/63.1
75.3/593.
77.2/61.5
70.9199.5
74.058.7
T7. 77.4299.2
69/0661.0
76.862.1
Ohhang et 2023)
68:2/88.4
74,2/56.
81.0163.7
81.5H636
$3.1/6155
76.3/60.3
$0.1/61.4
76.0666.1 6
76.68088
50.5160.8
70.1/90.9
78. 516.77
Infa/I
88.77883
77.1/59.5
81.164.0
81.5663.
K38665.5
77.7/6077
77,7808
80 60.11111
66.2/57.1
ERNIE-M M"
87.4766.3
73.6/5.9
80.56334
80.5/62.4
62.3/63.2
77.8/614
80.2/64.6
74.6663.0
76.7650.
90.76116
66.4157.2
78.2/22.7
StucI-XLM
88.2776.2
77.4:59.8
81.26399
31.06332
83,3/65.1
77.7860.2
81.4/64.7
77.2/6.11
76.86612
S1.022.2
69.2/59.2
79.5164.1
weo action
67.6/6677
76.3/59.0
80.863..8
80.5662.9
83.1/61.2
76.7660.5
76.265.55
76.28002
69.0199.2
78 73.9663.5
who worm-tart
87.976.6
75.9/58.4
40.463.2
10.1661.
81. 9/6224
76 76.8/903
80.2/63.3
76.5/64.2
76. 36.3660.3
80 80.5661.
68.6/997
78 8-6.82
only wio PNct
87.1/75.8
75.2/58.2
80./63.44
80.36610
82.062.88
76.3/59.8
80.6/64.1
74.3062.9
76,0/9..7
79-9661.1
66.4/57.2
78.1/62.4
87.1/76.3
75.4058.0
80.0662.1
80.09628
$1,2/224
75.0558.
79:4882.8
73/463.0
75.0/59.0
80/16113
66.1/51.3
77.5161.8
Table 13: XQuAD results (F1 EM) for each language.
Lang
en
ar
AVG
XLM MLM-Rinny.lll et 2020)
83.5/70.6
66.6/7711
70.1/54.9
74.1/56.6
70.6/53.1
74.0/52.9
62.1/37.0
71.6/53.2
InfoXLM
84.7/71.7
67.5:47.3
71.7/56.9
74.9/56.9
72.2/54.0
74.8/53.
68.7/45.3
73.5/55.1
MiCo et 2021)
84.5/71.6
67464766
71.2/56.2
75.1/57.5
72.5/54,2
76.2/54.1
69.2/454
73.6/55.2
ERNIE-M
83.77708
64.5
70.5/55.7
74.0/5.11
70.246.7
7/.6
ERNIE MrOuyang al., 2021)
84.4/115
67.4/47.2
70.8/55.9
74.8/56.6
72.6:54.7
75.0
71.1/47.
73.7/55.3
PEa al., 2022)
83.1/70.3
63.9/439
68.9/54.0
73.355..1
69.0/51.7
72.0/520
68.0445.2
71.3/53.2
VECOZ (KZlang et 2023)
84,1/71,4
74.3/56.3
70.3/54.9
66,5146.5
71.5/53.7
74.2/53.1
67,9/4377
72.7/54.2
Struct XLM
84.4/71.6
66.8/46.4
71.4/55.9
74.9/57.6
71.9/53.7
74.7/53.6
73.2/54.8 68.1
wfo action
84.1/70.8
66.7/66.3
70.7/55.3
74.3/56.5
71.5/53.4
74.3/82.8
66.4/44.3
72.6/542
w/o wart-sstrrt
83.9/70.7
66 145.5
71 71.0/55.3
74.3/56.2
71 71.953.77
73.9/52.8
62 62.9/43.6
72.0/54.0
wart-start
83.7/70.4
66.044.11
70.4/55.2
74.2/56.0
74.1/526
62.5/44.4
71.7/53.8
w/o PNet
84.0/70.8
66.044..6
70.4/56.3
74.1/56.2
70.2/52.6
73 8/52.6
60,7/38.9
71.3/33.3
Table 14: MLQA results (F1 / EM) for each language.
Lang.
bn
fi
id
ko
AVG
XLM-RRay. (Hi et al., 2020)
71.5/6688
67.6/40.4
64.0/47.8
70.5/53.2
77.46619
31.9/10.9
67.0/42.1
14.81
70.1443.6
65.1/45.0
XY-LENT(Para 2022)
73.4/99.1
71.6/54.1
63.7/11.3
66.5/52.3
77.0663.4
57.2/43.5
68.0/49.0
67.3/1.11
59.439.33
67.1/51.
VECO2 et 2023)
73.0/60.7
73.7/55.8
63.8/16.9
72.8/66.9
79.6/66.7
60.5/47.8
67.7/39.9
73.260.1
75.6/7..1
71.1/4477
InfeXLM
76.4633.9
75.6/57.0
67.1/52.2
71.3/55.1
79.9/63.4
61.0/19.3
69.2/1..
72.5/54.5
77.2/5..5
72.2/54.6
ERNIE M'
73.8/58.9
73.4/53.4
65.7/50.4
72.2/55.0
77.4/60.7
60.2/4..1
66.7/4177
69.4/53.3
73.5/53.7
70.3/52.7
StructXLLL
75.7/64.5
75.3/56.0
69.2/53.1
75.0/58.2
62.8/50.0
69.1543.5
72.6/55.3
76.7/56.1
73.1/56.0
wlo action
73.5/60.9
74.9/53.7
66.3/47.8
73:7/57.0
80.8/66.5
61.9/50.0
67. 1/401
72.8/57.7
73.8/33.5
71.6/44.1
wamm-start
74.4662.5
73.0/51.5
70.1/52.2
73.8/56.3
61.4/47.8
66.6-90.1
72.8/54.5
74.3/55.0
71,8/53.7
only wamm-sstart
72.5/59.1
73.9/54.2
65.3/50.4
72.7/56.1 1
80.0/64.2
60.5/47.5
65 65.9/38.8
72 72.6/55.5
73.7449.5
70.8/52.8
72.9/58.9
73.6/55.1
68.9/51.3
73.5/57.0
79.9/61.6
61.0'18.6
65.6/37.3
72.4/55.3
70.5/46.5
70.9/52.8
Table 15: TyDiQA-GoldP results (F1 / /MM) for each language.


Reviews:
[{"review_id": "9Qw1aYUeDp", "reviewer": "Reviewer_eQtC", "summary": "", "questions": "(A) What is the statistical significance score of Struct-XLM in comparison to info-XLM?\n(B) How good were the extracted structures?\n(C) What is the rationale behind mapping the action vector to the action matrix? Have ablations been conducted when directly using the action vector does not yield improvements?\n(D) Additional ablation is needed: What are the consequences if the warm-start step is omitted? Is there no benefit from the multilingual representation?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for acknowledging our contributions to cross-lingual alignment and low-resource language research with the idea of introducing syntactic structure. We appreciate your feedback and the time you've taken to evaluate our work. Based on your comments, we would like to make a response and hope for your further comments.\n\n- **R1: About the motivation statement for each step of our method.**\n\nIn this version, we emphasized explaining the methodology but didn't describe deeply the underlying motivations in section 3. We appreciate your feedback as a reader for pointing this out, and we will enhance it in the new version.\n\nIn lines 245-249, we explained how the action vector can represent the structure of the sentence, which provides information about the constituent structure. In the new manuscript, we'll add a syntactic parse tree example for better understanding. This parse tree can be projected to the sequence level to identify constituents, as shown in Figure 2: 'This gives them $I$ a competitive edge $|$ for the interim period $|$.' It segments the sentence into three constituents, excluding punctuation. Thus, the action vector predicts sentence constituent boundaries. In Section 3.3, we'll explain our choice of the translation ranking task, known to enhance multilingual representation. As for PNet's motivation, we discussed it in lines 83-87 and will emphasize it in Section 3.4.\n\n- **QC: About the mapping from the action vector to the action matrix.**\n\nThe action vector represents sentence constituent boundaries, but each word needs a sub-action vector to focus on other words in the same component. Hence, we'll convert the action vector to the action matrix. They essentially convey the same information, eliminating the need for ablation.\n\nWe acknowledge that our unclear previous explanation in lines 245-249 may cause confusion. We will revise our description. Additionally, we can provide a rough algorithm for computing the action matrix from the action vector. The final version will be added to the new manuscript.\n\n---\n\nAlgorithm: Convert the action vector to the action matrix\n\n---\n\nInput: action vector $a$, length of sentence $L$\n\nOutput: action matrix $A$\n\n1. Initialize action matrix $A$ as a zero matrix with dimension $L$.\n2. Iterate over the vector $a$ to get the start and end index of each constituent, save in a list $C=[[c_1^{start}, c_1^{end}]...[c_i^{start}, c_i^{end}]...[c_n^{start}, c_n^{end}]]$, where $c_i$ represents a constituent.\n3. Iterate over the list $C$:\n\nIf $c_i^{start}=c_i^{end}$, $c_i$ consists of single word, and A[i][i]=1,\n\nElse sub-action matrix $A^{c_i}$ is a matrix with a value of 1 except that the diagonal is 0, and the dimension is the length of constituents $c_i$. The starting position of $A^{c_i}$ in $A$ is $(c_i^{start}, c_i^{start})$\n\n4. Output the action matrix $A$\n\n---\n\n- **R3QB: About quantitative analysis of syntax discovery.**\n\nAs we aim to acquire structural information that facilitates cross-lingual alignment, we employed retrieval tasks in Section 5.3 to quantitatively analyze whether the information obtained through RL contributes to cross-lingual alignment.\n\nIn response to your suggestion, we try to conduct a quantitative analysis of the discovered structure by evaluating the constituents' boundary predicted by the action vector. The labeled test set is from the warm-start step, including English (en) and target languages (tgt) two-way. The results are shown in Table-a below, the boundary predicted by Struct-XLM can be 66.84% and 57.58% recall in English and target languages, respectively, which means Struct-XLM can correctly predict $>55\\%$ constituents’ boundary in labeled test sets in target languages. Struct-XLM tends to predict more constituents (an average of 11-12 constituents in the target language sentence and the average in the test set is 6-7), as we mentioned in Lines 558-561. This may be the reason for the low precision (P). We also report the results of the model w/o warm-start to prove the influence of the warm-start step for structural discovery.\n\nTable a. The precision(P), recall(R), and F1 scores of the constituent boundary prediction.\n\n| **Models**     | **P**$_{en}$ | **R**$_{en}$ | F1$_{en}$ | **P**$_{tgt}$ | **R**$_{tgt}$ | F1$_{tgt}$ |\n| -------------- | -------------- | -------------- | ----------- | --------------- | --------------- | ------------ |\n| Struct-XLM     | **34.49**      | **66.84**      | **45.50**   | **35.67**       | **57.58**       | **43.14**    |\n| w/o warm-start | 30.57          | 62.13          | 40.98       | 31.63           | 56.48           | 39.67        |\n\n- **R2QA: About the statistical significance score of Struct-XLM in comparison to info-XLM.**\n\nWe conducted significance score calculations on the dataset where our model achieved the best performance in Table 1, as well as the average of seven tasks in the XTREME benchmark (Task$_{AVG}$), as shown in Table-b below. In the average of the XTREME benchmark, the statistical significance score of Struct-XLM in comparison to info-XLM is 0.22. **As you mentioned, we should be competitive but not superior with the InfoXLM model, achieving similar results using only 1/5000 training data of InfoXLM.** My analysis in lines 444-449 is inaccurate in this regard. We will make the necessary adjustments to the discourse of the relevant experimental analysis in the new version.\n\nTable b. The statistical significance score.\n\n| **Models**                  | **PAWS-X** | **POS** | **NER** | **XQuAD(F1)** | **XQuAD(EM)** | **TyDiQA(F1)** | **TyDiQA(EM)** | **Task**$_{AVG}$ |\n| --------------------------- | ---------- | ------- | ------- | ------------- | ------------- | -------------- | -------------- | ------------------ |\n| XLM-Rlarge(Hu et al., 2020) | <0.01      | <0.01   | <0.01   | <0.05         | <0.05         | <0.05          | <0.05          | <0.01              |\n| XY-LENT(Patra et al., 2022) | 0.40       | \\--     | \\--     | <0.01         | <0.05         | <0.01          | <0.05          | <0.05              |\n| VECO2.0(Zhang et al., 2023) | <0.05      | <0.05   | <0.05   | 0.09          | 0.32          | <0.05          | 0.26           | <0.01              |\n| ERNIE-M$^\\dagger$         | <0.05      | 0.47    | <0.01   | <0.05         | <0.05         | <0.01          | <0.01          | <0.01              |\n| InfoXLM$^\\dagger$         | 0.19       | 0.08    | 0.37    | 0.36          | 0.35          | 0.15           | 0.10           | 0.22               |\n\n- **QD: About additional ablation.**\n\nThank you for bringing this to our attention. We unintentionally missed evaluating the ablation model without the warm-start step (w/o warm-start). We've now conducted this experiment and added the results in the table below. According to the results, the model w/o warm-start performs close to the model w/o an action vector to inference and outperforms other ablation models and the PLM baseline, demonstrating the effectiveness of our approach. However, it doesn't surpass our final Struct-XLM model, confirming the effectiveness of our warm-start procedure. We will include these new results and analyses in the next version.\n\nTable c. Ablation results on each model. The results except the model w/o warm-start is new, others are from Table 3.\n\n| **Models**      | **XNLI** | **PAWS-X** | **POS**  | **NER**  | **XQuAD(F1)** | **XQuAD(EM)** | **MLQA(F1)** | **MLQA(EM)** | **TyDiQA(F1)** | **TyDiQA(EM)** | **AVG**  |\n| --------------- | -------- | ---------- | -------- | -------- | ------------- | ------------- | ------------ | ------------ | -------------- | -------------- | -------- |\n| XLM-R           | 79.2     | 86.4       | 73.8     | 65.4     | 76.6          | 60.8          | 71.6         | 53.2         | 65.1           | 45.0           | 67.7     |\n| Struct-XLM      | **81.2** | **89.8**   | **76.8** | **68.0** | **79.4**      | **63.9**      | **73.0**     | **54.6**     | **72.9**       | **55.4**       | **71.5** |\n| w/o action      | 80.4     | 89.4       | 76.1     | 66.7     | 78.9          | 63.5          | 72.6         | 54.2         | 71.6           | 54.1           | 70.7     |\n| w/o warm-start  | 79.7     | 89.0       | 76.3     | 65.6     | 78.6          | 62.8          | 72.0         | 54.0         | 71.8           | 53.7           | 70.4     |\n| only warm-start | 79.7     | 88.5       | 75.0     | 64.5     | 78.1          | 62.4          | 71.7         | 53.8         | 70.8           | 52.8           | 69.7     |\n| w/o PNet        | 79.4     | 88.3       | 74.9     | 62.9     | 77.5          | 61.8          | 71.3         | 53.3         | 70.9           | 52.8           | 69.3     |\n\n- **R4: About the limitation from parallel data for warm-start training.**\n\nBased on the experimental results in Table 8-14, even languages without warm-start training data can show improvements. For instance, in Table 10, eu, tl, and yo do not have training corpora, yet they exhibit significant improvements compared to the best baseline result, with increases of 1.2%, 1.9%, and 16.0%, respectively. Moreover, in Table 11, fa and ms without warm-start training corpora improve by 3.5% and 2.7% than the best baseline result. However, for more low-resource languages (such as languages not included in the PLM pre-training set), we have no data to prove whether our method is effective. We will supplement the relevant experimental analysis and limitation analysis in the new version.\n\nYour comments have been instrumental in improving the quality of our paper, and we are thankful for your expertise.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) The paper presents a novel contribution by automatically discovering syntactic structures. This holds the potential to bring benefits to numerous low-resource languages that encounter limitations in labeled datasets.\n(2) Integrating the syntactic structure to enhance cross-lingual alignment and expand language coverage is intuitive. On a theoretical level, the paper is supported by relevant and well-incorporated literature references. I do not have any concerns in this regard.\n(3) The paper is well written. The mode performance is compared with state-of-the-art baselines.", "reasons_to_reject": "(1) The paper In its current form, the methodology section of the paper presents a series of steps conducted to perform an experiment and attain results. However, it falls short in conveying the underlying intuition or motivations driving each of these steps, which is crucial for establishing a connection with the reader. For instance, in line 214, the authors assert that \"The action vector can represent the structure of the sentence,\" but this assertion lacks elaboration in sections 3.2 where the underlying motivation should be articulated. A similar observation applies to sections 3.3 and 3.4.\n(2) I view this model as an alternative competitive approach with info-XLM, rather than superior. Observing Table-1, 2, and 4, it becomes apparent that the performance improvement is not notably substantial and might not possess statistical significance. The authors should provide the statistical significance scores to ensure a more robust conclusion. Given the results, the paper's claims could be adjusted from asserting superiority to positioning the model as competitive. The paper in it’s current form lacks this type of analysis.\n(3) The paper lacks a detailed analysis of the discovered structures. While there are efforts made in section 5.4, they are not very convincing. Since automated structure extraction is the main claim of the paper (ultimately contributing to improved cross-lingual alignment and transfer), I believe it should be the primary focus. A potential direction could involve establishing quantitative similarities or correlations between the discovered structures and human-extracted structures using a small dataset.\n(4) Another limitations of the work is the utilization of a 13.7k parallel corpus for warm-start training. This essentially restricts the model's applicability to a smaller set of low-resource languages. There should be an analysis or experimentation conducted to overcome this limitation.", "questions_to_authors": "(A) What is the statistical significance score of Struct-XLM in comparison to info-XLM?\n(B) How good were the extracted structures?\n(C) What is the rationale behind mapping the action vector to the action matrix? Have ablations been conducted when directly using the action vector does not yield improvements?\n(D) Additional ablation is needed: What are the consequences if the warm-start step is omitted? Is there no benefit from the multilingual representation?", "paper_topic_and_main_contributions": "The paper introduces a novel multilingual pre-trained language model (mPLM) named Struct-XLM. This model aims to enhance cross-lingual alignment and subsequent cross-lingual transfer by incorporating structural information into the existing mPLM, specifically XLM. This structural information is automatically extracted through a reinforcement learning (RL) algorithm.\n\nThe architectural design of the model comprises three key components: the Policy Network (PNet), the Structural Encoder, and the Translation Ranking Task module. The Structural Encoder (mPLM) serves as an environment, furnishing state representations utilized by PNet to generate action vectors corresponding to sentence structures.  This action vector is used with a multi-head attention module in the last layer of mPLM, and Translation Ranking Task to provide a delayed reward to PNet. Empirical evaluations are conducted on the XTREME benchmark across seven tasks, and the paper claims improved performance across these tasks. The literature review section is thoroughly written and appreciated, and relevant works are cited. The Paper is compared with several baseline models including, Info-XLM, ERNIE-M, XY-LENT and VECO2.0.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "0LPvG4vXmX", "reviewer": "Reviewer_Jy7e", "summary": "", "questions": "Is it possible to run the training process without warm-start or how much does a reasonable training process depend on a reasonable initialization? Would it be possible to quantify the changes that RL discovered compared to given structures in annotated data?\nIs it necessary to have warm-start data in all languages that are included?\nHow important is batch-size optimization for the in-batch labeling for the contrastive training objective?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to begin by expressing our sincere gratitude to you for your time and expertise in evaluating our paper. In your review, you raised several important points that we would like to address.\n\n- **Q1: About the ablation without warm-start step**\n\nWe deeply apologize for the absence of an ablation experiment without the warm-start step in our initial submission. We have conducted this experiment and reported the results in the table below. According to the results, the model without the warm-start step performs close to the model w/o action vector to inference and outperforms other ablation models and the PLM baseline, demonstrating the effectiveness of our approach. However, it doesn't surpass our final Struct-XLM model, confirming the effectiveness of our warm-start procedure. We will include these new results and analyses in the next version.\n\nTable a. Ablation results on each model. \n\n| **Models**      | **XNLI** | **PAWS-X** | **POS**  | **NER**  | **XQuAD(F1)** | **XQuAD(EM)** | **MLQA(F1)** | **MLQA(EM)** | **TyDiQA(F1)** | **TyDiQA(EM)** | **AVG**  |\n| --------------- | -------- | ---------- | -------- | -------- | ------------- | ------------- | ------------ | ------------ | -------------- | -------------- | -------- |\n| XLM-R           | 79.2     | 86.4       | 73.8     | 65.4     | 76.6          | 60.8          | 71.6         | 53.2         | 65.1           | 45.0           | 67.7     |\n| Struct-XLM      | **81.2** | **89.8**   | **76.8** | **68.0** | **79.4**      | **63.9**      | **73.0**     | **54.6**     | **72.9**       | **55.4**       | **71.5** |\n| w/o action      | 80.4     | 89.4       | 76.1     | 66.7     | 78.9          | 63.5          | 72.6         | 54.2         | 71.6           | 54.1           | 70.7     |\n| w/o warm-start  | 79.7     | 89.0       | 76.3     | 65.6     | 78.6          | 62.8          | 72.0         | 54.0         | 71.8           | 53.7           | 70.4     |\n| only warm-start | 79.7     | 88.5       | 75.0     | 64.5     | 78.1          | 62.4          | 71.7         | 53.8         | 70.8           | 52.8           | 69.7     |\n| w/o PNet        | 79.4     | 88.3       | 74.9     | 62.9     | 77.5          | 61.8          | 71.3         | 53.3         | 70.9           | 52.8           | 69.3     |\n\n- **Q2: About quantitative analysis of the changes that RL discovered compared to given structures in annotated data**\n\nAs our structural discovery is unsupervised, we indirectly confirm that RL can discover structures better suited for cross-lingual alignment via the sentence retrieval task in Section 5.3. In response to your suggestion, we quantitatively assessed the constituent boundary (i.e. number 1 in the action vector) predicted by PNet. The labeled test set is from the warm-start step, including English (en) and target languages (tgt) two-way. The results are shown in Table-b below, the boundary predicted by Struct-XLM can be 66.84% and 57.58% recall in English and target languages, respectively, which means Struct-XLM can correctly predict $>55\\%$ constituents’ boundary in labeled test sets in target languages. Struct-XLM tends to predict more constituents (an average of 11-12 constituents in the target language sentence and the average in the test set is 6-7), as we mentioned in Lines 558-561. This may be the reason for the low precision (P). We also report the results of the model w/o warm-start to prove the influence of the warm-start step for structural discovery.\n\nTable b. The precision(P), recall(R), and F1 scores of the constituent boundary predict\n\n| **Models**     | **P**$_{en}$ | **R**$_{en}$ | F1$_{en}$ | **P**$_{tgt}$ | **R**$_{tgt}$ | F1$_{tgt}$ |\n| -------------- | -------------- | -------------- | ----------- | --------------- | --------------- | ------------ |\n| Struct-XLM     | **34.49**      | **66.84**      | **45.50**   | **35.67**       | **57.58**       | **43.14**    |\n| w/o warm-start | 30.57          | 62.13          | 40.98       | 31.63           | 56.48           | 39.67        |\n\n- **Q3: About the necessary of warm-start data in all languages**\n\nIt's not necessary for all languages to have warm-start data. For instance, in Table 10, eu, tl, and yo do not have training corpora, yet they exhibit significant improvements in POS tasks compared to the best baseline result, with increases of 1.2%, 1.9%, and 16.0%, respectively. Moreover, in Table 11, fa and ms without warm-start training corpora improve by 3.5% and 2.7% than the best baseline result. We will incorporate analysis in the new version regarding the performance improvements in low-resource languages that lack warm-starting data.\n\n- **Q4: About the importance of batch-size setting**\n\nOur goal is to bring parallel sentences closer and non-parallel sentences farther apart. However, since our training dataset only consists of parallel sentence pairs, we adopted a batch sampling approach to construct comparable data containing non-parallel sentences. The setting of batch size is unlimited, and we think that a larger batch size should be able to better distinguish parallel data from non-parallel data. However, due to the particularity of RL training, we recommend setting the value smaller, and we use 5.\n\nThank you again for acknowledging the writing and methodology of our work. We look forward to further discussions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is very well written and the approach is novel and effective. The idea of using reinforcement learning to discover structure based on translation ranking loss as a reward is interesting and seems to work well to align source and target language representations.", "reasons_to_reject": "I don't see immediate reasons to reject the paper.", "questions_to_authors": "Is it possible to run the training process without warm-start or how much does a reasonable training process depend on a reasonable initialization? Would it be possible to quantify the changes that RL discovered compared to given structures in annotated data?\nIs it necessary to have warm-start data in all languages that are included?\nHow important is batch-size optimization for the in-batch labeling for the contrastive training objective?", "paper_topic_and_main_contributions": "The paper introduces a novel multilingual language model that discovers structure and aligns representations across languages better than competing models. The approach incorporates reinforcement learning with binary actions to discover constituent structure. In the experimental section, the authors show that the model is very competitive and improves cross-lingual transfer even though it is trained on rather small datasets.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Some spaces missing between text and citation references."}, "extra_text": {}}, {"review_id": "BwejgiLhJ2", "reviewer": "Reviewer_xL1S", "summary": "", "questions": "1. Why apply the action matrix masking after the softmax in eq.3? doesn't that entail that weights E no longer sum to 1? Intuitively, I think this entails that you potentially have unbalanced attention outputs with very different norms?\n2. Line 304, is \"previous state\" a typo, and did you mean all \"previous states\"? or are you only attending a single step?\n3.  Lines 414-415: why not compare with a model with fewer parameters? you could verify that you observe the expected increase in performances in such a case.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our sincere appreciation for your time and effort in reviewing our paper. Your expertise and thoughtful feedback have been invaluable in improving the quality of our work.\n\n- **R1: About the performance of the proposed approach in low-resource scenarios.**\n\nThank you very much for acknowledging the data efficiency of our approach and for your valuable feedback. We will include an analysis of the improvements in low-resource language performance in the new version, which was omitted in the previous version due to space constraints. For instance, in Table 10, eu, tl, and yo do not have training corpora, yet they exhibit significant improvements in POS tasks compared to the best baseline result, with increases of 1.2%, 1.9%, and 16.0%, respectively. Moreover, in Table 11, fa and ms without warm-start training corpora improve by 3.5% and 2.7% than the best baseline result.\n\n- **R2: About quantitative analysis of syntax discovery.**\n\nI'm sorry that we accidentally used the absolute expression \"learns universal syntactic structures\". The expression \"learning universal structural information\" in line 79 is more in line with our original intention. More specifically, the structure information we learn is a coarse-grained constituent boundary information, namely action vector. It predicts constituent boundaries, thereby segmenting sentences into constituents. We will revise our representation in the new version and add evaluation results of constituent boundary prediction to quantify our learned structure. The labeled test data comes from the warm-start step.\n\nThe results are shown in Table-a below, the boundary predicted by Struct-XLM can be 66.84% and 57.58% recall in English and target languages, respectively, which means Struct-XLM can correctly predict $>55\\%$ constituents’ boundary in labeled test sets in target languages. Struct-XLM tends to predict more constituents (an average of 11-12 constituents in the target language sentence and the average in the test set is 6-7), as we mentioned in Lines 558-561. This may be the reason for the low precision (P). We also report the results of the model w/o warm-start to prove the influence of the warm-start step for structural discovery.\n\nTable a. The precision(P), recall(R), and F1 scores of the constituent boundary predict\n\n| **Models**     | **P**$_{en}$ | **R**$_{en}$ | F1$_{en}$ | **P**$_{tgt}$ | **R**$_{tgt}$ | F1$_{tgt}$ |\n| -------------- | -------------- | -------------- | ----------- | --------------- | --------------- | ------------ |\n| Struct-XLM     | **34.49**      | **66.84**      | **45.50**   | **35.67**       | **57.58**       | **43.14**    |\n| w/o warm-start | 30.57          | 62.13          | 40.98       | 31.63           | 56.48           | 39.67        |\n\n- **R3: About the statistical significance score of Struct-XLM in comparison to info-XLM.**\n\nWe conducted significance score calculations on the dataset where our model achieved the best performance in Table 1, as well as the average of seven tasks in the XTREME benchmark (Task$_{AVG}$), as shown in the Table below. When compared to the InfoXLM model, we did not observe significant differences. However, our model achieved similar results using only 1/5000 training data of InfoXLM. My explanation in lines 444-449 is inaccurate in this regard, we should be competitive with but not superior to the InfoXLM model. We will make the necessary adjustments to the relevant experimental analysis in the new version.\n\nTable b. The statistical significance score.\n\n| **Models**                  | **PAWS-X** | **POS** | **NER** | **XQuAD(F1)** | **XQuAD(EM)** | **TyDiQA(F1)** | **TyDiQA(EM)** | **Task**$_{AVG}$ |\n| --------------------------- | ---------- | ------- | ------- | ------------- | ------------- | -------------- | -------------- | ------------------ |\n| XLM-Rlarge(Hu et al., 2020) | <0.01      | <0.01   | <0.01   | <0.05         | <0.05         | <0.05          | <0.05          | <0.01              |\n| XY-LENT(Patra et al., 2022) | 0.40       | \\--     | \\--     | <0.01         | <0.05         | <0.01          | <0.05          | <0.05              |\n| VECO2.0(Zhang et al., 2023) | <0.05      | <0.05   | <0.05   | 0.09          | 0.32          | <0.05          | 0.26           | <0.01              |\n| ERNIE-M$^\\dagger$         | <0.05      | 0.47    | <0.01   | <0.05         | <0.05         | <0.01          | <0.01          | <0.01              |\n| InfoXLM$^\\dagger$         | 0.19       | 0.08    | 0.37    | 0.36          | 0.35          | 0.15           | 0.10           | 0.22               |\n\nSince this benchmark is publicly available, and each dataset has official, standardized splits, further experiments are required to investigate whether overfitting may occur due to data partitioning. We believe this is an interesting direction, as the choice of data in a multilingual setting can indeed impact cross-lingual performance.\n\n- **R4: About limitations.**\n\nWe attempted to analyze our limitations in terms of data, method, resource constraints, and time constraints. Moreover, we proposed some actions to improve. We appreciate the guidance from the reviewer, and We will carefully consider distinguishing the sections on limitations and future work more clearly, as per your suggestion.\n\n- **Q1: About the design of action matrix masking after the SoftMax in eq.3.**\n\nIn the source code of the Transformer, there is a dropout operation applied to the attention matrix, which causes the attention matrix not to sum to 1. Given this, we directly use the action matrix after the SoftMax operation, like a dropout operation. We have also observed that after adding the action matrix, applying SoftMax normalization again may not be suitable because the self-attention weights are relatively small, and the parts with action matrix values of 0 would be assigned relatively large weights. We attempted to modify the 0 values in the action matrix to a very small number like 1e-9, but this led to underflow issues when multiplied by 1e-9 due to the very small weights in the padding position. Consequently, we opted for the computation method illustrated in Figure 2.\n\n- **Q2: About \"previous state\" in Line 304.**\n\nAs described in Lines 83-85, we formulate the structure discovery as a sequential decision problem, so we only attend to a single step. It's possible that our previous explanation lacked detail and caused confusion. In addition to this clarification, we will revise the manuscript based on the modification suggestions you provided later on. We greatly appreciate your corrections.\n\n- **Q3: About the question of why it is not compared with the Syntax-MBERT model.**\n\nComparing our model to the Syntax-MBERT model may not be entirely fair due to the differences in the number of model parameters. In fact, the results reported in the Syntax-MBERT paper were even lower than our XLM-R baseline. Considering the distinctions between our approach and Syntax-MBERT, we have not replicated the Syntax-MBERT model based on XLM-R or implement our model based on mBERT for comparison. Two key differences are that they explicitly use syntactic labels, and they utilize syntactic annotation not only for pre-training syntactic representations but also for fine-tuning each task.\n\nOf course, as you mentioned, if surpassing the Syntax-MBERT model based on XLM-R would provide stronger evidence, we have initiated efforts to replicate it.\n\n- **About the suggestion of presentation improvements.**\n\nFor the conversion of the action vector to an action matrix, a more formal description is indeed needed. we provide a rough algorithm for computing the action matrix from the action vector. The final version will be added to the new manuscript.\n\n---\n\nAlgorithm: Convert the action vector to the action matrix\n\n---\n\nInput: action vector $a$, length of sentence $L$\n\nOutput: action matrix $A$\n\n1. Initialize action matrix $A$ as a zero matrix with dimension $L$.\n2. Iterate over the vector $a$ to get the start and end index of each constituent, save in a list $C=[[c_1^{start}, c_1^{end}]...[c_i^{start}, c_i^{end}]...[c_n^{start}, c_n^{end}]]$, where $c_i$ represents a constituent.\n3. Iterate over the list $C$:\n\nIf $c_i^{start}=c_i^{end}$, $c_i$ consists of single word, and A[i][i]=1,\n\nElse sub-action matrix $A^{c_i}$ is a matrix with a value of 1 except that the diagonal is 0, and the dimension is the length of constituents $c_i$. The starting position of $A^{c_i}$ in $A$ is $(c_i^{start}, c_i^{start})$\n\n4. Output the action matrix $A$\n\n---\n\nThank you again for your helpful comments and suggestions on our work. Looking forward to further communication.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed retraining approach is data-efficient,  requiring only a 8MB corpus (the authors contrast this with the 40GB+ requirements of concurrent methods).\n- As far as I can tell, there seems to be significant novelty in the proposed approach, which successfully combines RL approaches, a translation ranking task and a Siamese network to  improve the cross-lingual capabilities of LLMs\n- The authors conduct substantive experiments to verify their claims.", "reasons_to_reject": "- In my opinion, the main advantage of the proposed approach over presented competitors is its low data requirements. This is however somewhat irrelevant, given that the approach is based on LLMs which are already data intensive. To a certain extent, a focus on verifying the viability of the proposed approach in low-resource scenarios would have been more coherent.\n- Some claims are contradicted or not substantiated by the results provided by the authors: while the abstract and introduction suggest that the model \"learns universal syntactic structures\", the qualitative analysis reported in section 5.4 does not seem to match with expert syntactic annotations. If the authors intend to make this claim, then supplementary experiments would be required (e.g., measure the learned constituency structure against existing constituency benchmarks). \n- Despite the numerous experiments conducted by the authors, the low margin of improvements would have call for statistical significance testing: as the paper currently stands, I believe there is a possibility that the reported results correspond to overfitting to the standard test splits of the cross-lingual benchmarks proposed here [(Gorman & Bedrick, 2019)](https://aclanthology.org/P19-1267/).\n- The limitations section reads more as a series of possible future works rather than a thorough self-critique.", "questions_to_authors": "1. Why apply the action matrix masking after the softmax in eq.3? doesn't that entail that weights E no longer sum to 1? Intuitively, I think this entails that you potentially have unbalanced attention outputs with very different norms?\n2. Line 304, is \"previous state\" a typo, and did you mean all \"previous states\"? or are you only attending a single step?\n3.  Lines 414-415: why not compare with a model with fewer parameters? you could verify that you observe the expected increase in performances in such a case.", "paper_topic_and_main_contributions": "This paper introduces a re-training RL procedure for multilingual LLMs, aimed at fostering their understanding of syntactic structures. \nThis procedure hinges on determining constituency-like structures to solve a cross-lingual translation ranking task.\nOverall, the paper proposes a data-efficient way of re-training data-hungry LLMs..", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- you use T as a scalar exponent in a few instances (lines 212 qand 213, in particular); Please use lowercase nonbold for scalar indices, and especially avoid T exponents as they can easily be mixed up with transpose operations.\n- consider providing explicit formulas for computing the action matrix from the action vector. You are essentially computing $\\mathbf{1} - \\mathrm{Id}$ for every pseudo-constituent (with $\\mathbf{1}$ a square matrix of ones); combining these in a block-matrix notation might allow you to provide a more formal overview of the computations you propose.\n- please check the manuscript for typos and ungrammatical / confusing sentences. Some examples:\n  + line 421: \"Natrual\" instead of \"Natural\"\n  + lines 149--151: \" [...] proposed a cross-attention module that builds interdependence between languages in model inner\" ; I'm uncertain what you mean by \"model inner\"\n  + line 356 \"Refer to Zhang et al (2018), our training process consists of three steps:\"  -> you probably mean \"Similar to Zhang et al. ...\"\n  + line 549: \"Qualitatively Analyze\": prefer nominal titles, e.g. \"Qualitative Analysis\""}}]

Author rebuttal:

Meta-review: The paper introduces a new multilingual LLM, Struct-XLM - which understand the underlying structure and aligns the representations across languages. The approach uses RL mechanism to discover the structural information using a translation ranking task. 

The paper proposes a novel mechanism to discover structural information and uses it bring alignment across langauges. This approach is quite useful for low resource languages. 
The paper is motivated well with detailed explanation and experimentation to verify their claims

Authors should update the draft with the discussion and additional details shared during the rebuttal phase.
Decision: Accept-Main