SlowGuess/ACL_2020_OCR huggingface dataset:
As described in Section 5.1, Sasano and Okumura (2016) reported that type-A examples prefer the ACC-DAT order and type-B examples prefer the DAT-ACC order. We used the same examples as Sasano and Okumura (2016) used. We analyzed the difference in the trend of argument order between type-A and type-B examples in each verb. Table 8 shows the verbs, which show a significant change in the argument order between type-A and type-B examples  $(p < 0.05$  in a two-proportion z-test). In the experiment using CLM, 31 verbs show the trend that type-A examples more prefer the ACC-DAT order to type-B, and 17 verbs show contrary trends. In the experiment using SLM, 38 verbs show the trend that type-A examples more prefer the ACC-DAT order to type-B, and 11 verbs show contrary trends. These results show that the number of verbs, where the ACC-DAT order is preferred by type-A examples rather than type-B, is significantly larger  $(p < 0.05$  with a two-sided sign test). This experimental design follows Sasano and Okumura (2016).
B.4 Word order and co-occurrence of verb and arguments
We evaluate the claim that an argument frequently co-occurring with the verb tends to be placed near the verb. We examine the relationship between each example's word order trend and  $\Delta$ NPMI.  $\Delta$ NPMI is calculated as follows:
Fairseq model	architecture
adaptive softmax cut off	transformer_lm
50,000, 140,000
Optimizer	algorithm	Nesterov accelerated gradient (nag)
learning rates	1e-5
momentum	0.99
weight decay	0
clip norm	0.1
Learning rate scheduler	type	cosine
warmup updates	16,000
warmup init learning rate	1e-7
max learning rate	0.1
min learning rate	1e-9
t mult (factor to grow the length of each period)	2
learning rate period updates	270,000
learning rate shrink	0.75
Training	batch size	4608 tokens
epochs	3
Table 5: Hyperparameters of the LMs.
Model	ACC-DAT is preferred	DAT-ACC is preferred
Verb	\( R_{ACC-DAT}^{v} \)	S&O	Verb	\( R_{ACC-DAT}^{v} \)	S&O
CLM	“例える” (compare)	0.993	0.945	“表する” (to table)	0.001	0.013
“換算する” (converted)	0.992	0.935	“澄ます” (put onairs)	0.000	0.017
“押し出す” (extruded)	0.979	0.923	“煮やす” (cook inside)	0.000	0.019
“見立てる” (mitateru)	0.994	0.919	“鳴る” (close the eyes)	0.001	0.021
“変換” (conversion)	0.975	0.898	“竦ゆる” (shrug)	0.002	0.022
SLM	“例える” (compare)	0.993	0.926	“喰する” (kissuru)	0.003	0.018
“押し出す” (extruded)	0.979	0.914	“表する” (to table)	0.001	0.018
“監禁” (confinement)	0.885	0.912	“澄ます” (put onairs)	0.000	0.021
“役立てる” (help)	0.933	0.904	“拔かす” (leave out)	0.002	0.022
“掃す” (attributable)	0.838	0.903	“踏み入股る” (step into)	0.002	0.025
Table 6: The verbs with the top five and the worst five  $R_{\mathrm{ACC-DAT}}^v$  in each LM. The "S&O" columns show the ACC-DAT rate reported in Sasano and Okumura (2016).
$$
\begin{array}{l} \Delta \mathrm {N P M I} = \mathrm {N P M I} \left(n _ {\mathrm {D A T}}, v\right) \\ - \operatorname {N P M I} \left(n _ {\mathrm {A C C}}, v\right), \\ \end{array}
$$
where  $\mathrm{NPMI}(n_c,v) = \frac{\mathrm{PMI}(n_c,v)}{-\log(p(n_c,v))}$ ,
$$
\mathrm {P M I} (n _ {c}, v) = \log \frac {p (n _ {c} , v)}{p (n _ {c}) p (v)},
$$
where,  $v$  is a verb and  $n_c$  ( $c \in \mathrm{DAT}$ , ACC) is its argument.
C Data used in Section 5.2, Section 6, and Appendix F
First, we randomly collected 50M sentences from 3B web pages. Note that there is no overlap between the collected sentences and the training data of LMs. Next, we obtained the sentences that satisfy the following criteria:
- There is a verb (placed at the end of the sentence) with more than two arguments (accompanying the case particle ga, o, ni, or de), where dependency distance between the verb and arguments is one.
Each argument (with its descendant) has fewer than 11 morphemes in the argument.
In each example, the verb (satisfying the above condition), its arguments, and the descendants of the arguments are extracted. Example sentences are created by concatenating the verb, its argument, and the descendants of the arguments with preserving their order in the original sentences.
In the experiments in Section 5.2, we analyzed the word order trend of the TIM and LOC constituents. We regard the constituent (argument and its descendants) satisfying the following condition as the TIM constituent:
- Accompanying the postpositional case particle "に" (DAT).
Show-type	Pass-type
Verb	CLM	SLM	S&O	Verb	CLM	SLM	S&O	Verb	CLM	SLM	S&O
“知らする”(notify)	.718	.754	.522	“尻す”(put back)	.366	.395	.771	“漏らす”(leak)	.152	.207	.332
“預けり”(deposit)	.426	.391	.399	“止る”(lodge)	.638	.704	.748	“浮かご”(float)	.387	.406	.255
“見せる”(show)	.353	.429	.301	“包む”(wrap)	.316	.356	.603	“向しぃ”的(direct)	.291	.319	.251
“被せる”(cover)	.240	.224	.256	“伝わる”(inform)	.419	.460	.522	“残す”(leave)	.323	.318	.238
“教える”(teach)	.297	.293	.235	“乗せる”(place on)	.556	.498	.496	“埋うの”(bury)	.405	.430	.223
“授しお”的(give)	.101	.084	.186	“届た”的(deliver)	.364	.419	.491	“混せる”(blend)	.336	.276	.200
“浴びせる”(shower)	.113	.121	.177	“並た”的(range)	.423	.485	.481	“当て”的(hit)	.287	.320	.185
“貸す”(lend)	.253	.213	.118	“ふつた”的(knock)	.333	.344	.436	“掛た”的(hang)	.285	.288	.108
“着せる”(dress)	.115	.109	.113	“付た”的(attach)	.326	.329	.368	“重ね”的(pile)	.226	.263	.084
-	-	-	-	“渡す”的(pass)	.349	.336	.362	“建て”的(build)	.117	.099	.069
-	-	-	-	“落とす”的 drop)	.379	.397	.351	-	-	-	-
Macro Avg.	.291	.291	.305	Macro Avg.	.347	.364	.361
Table 7: Overlap of the results of LMs and that of Sasano and Okumura (2016) on the relationship of the ACC-DAT rate and verb types. Each score corresponding to a verb denotes its DAT-ACC rate. The "S&O" columns show the ACC-DAT rate reported in Sasano and Okumura (2016). There is no significant difference between the distributions of the DAT-ACC rate in two verb types.
- Containing time category morphemes<sup>20</sup>.
We regard the constituent (argument and its descendants) satisfying the following condition as the LOC constituent:
- Accompanying the postpositional case particle “ $\mathcal{C}$ ”.
- Containing location category morphemes $^{20}$ .
81k examples were created. The averaged number of characters in a sentence was 45.1 characters. The number of occurrences of each case is shown in Table 9. The scrambling process conducted in the experiments (Sections 5.2 and 6) is the same as described in Section 4.
D Details on Section 5.3 (adverb)
Table 10 shows the correlation between the result of LMs and that of Koizumi and Tamaoka (2006). The column "Canonical" shows the position, which is significantly preferred over the other positions. "A," "S," "O," and "V" denote "adverb," "subject," "object," and "verb," respectively. The sequence of the alphabets corresponds to their order; for example, "ASOV" indicates the order: adverb  $<$  subject  $<$  object  $<$  verb. Following Koizumi and Tamaoka (2006), we examined the three candidate positions of the adverb: "ASOV," "SAOV," and "SOAV." The score  $r$  denotes the Pearson correlation coefficient of the preferred ranks of each adverb position to that reported in Koizumi and Tamaoka (2006).
E Details on Section 6.2 (topicalization)
We topicalized a specific constituent by moving the constituent to the beginning of the sentence and
Figure 4: Correlation between the ACC-DAT rate and the rate that the ACC argument is more likely to be topicalized than DAT for each verb. Each plot corresponds to the result of each verb.
adding the adverbial particle “は” (TOP). Strictly speaking, conjunctions are preferentially placed at the beginning of the sentence rather than topicalized constituents. The examples we used do not include the conjunctions at the beginning of the sentence. The adverbial particle was added according to the rules shown in Table 12.
Claim (i): Table 11 shows the  $t_{a|b}$  for each pair of the case  $a$  (row) and  $b$  (column). The results show that the more anterior the case  $a$  is and the more posterior the case  $b$  is in the canonical word order, the larger the  $t_{a|b}$  is.
Claim (ii): Figure 4 shows that the more a verb prefers the ACC-DAT order, the more ACC case tends to be topicalized. The X-axis denotes the ACC-DAT rate of the verb, and the Y-axis denotes the trend that ACC is more likely to be topicalized than DAT.
$^{20}$ :identified by JUMAN
Model	Verbs whose type-A examples prefer the ACC-DAT or-der	Verbs whose type-B examples prefer the ACC-DAT or-der
CLM	“預る”(deposit), “置く”(place), “持續”(have), “入る”(put in), “納う”(pay), “郵送”(mail), “供給”(supply), “出す”(put out), “運む”(transport), “流す”(shed), “掛る”(hang), “飾る”(decorate), “広ける”(spread), “移す”(transfer), “殘す”(leave), “配送”(deliver), “送る”(send), “投掷る”(throw), “送付”(send), “返却”(return), “届ける”(send), “戾)”(return), “着ける”(wear), “上ごる”(increase), “落と)”(drop), “載ります”(publish), “変更”(change), “納入”(deliver), “卸す”(unload), “揭載”(publish), “通)”(get X through)	“配布”(distribute), “渡す”(pass), “トレーナット”(present), “合わせる”(match), “見せる”(show), “提供”(offer), “与える”(give), “当てる”(hit), “回す”(turn), “追加”(add), “貸す”(lend), “展示”(exhibit), “据える”(lay), “依賴”(request), “插入”(insert), “缠)”(collect), “請求”(claim)
SLM	“預る”(deposit), “置く”(place), “賴む”(ask), “入る”(put in), “納う”(pay), “郵送”(mail), “出す”(put out), “運む”(transport), “流す”(shed), “掛ける”(hang), “広ける”(spread), “移す”(transfer), “残)”(leave), “リクエスト”(request), “配送”(deliver), “送る”(send), “投掷る”(throw), “送付”(send), “求)”(ask), “提出”(submit), “届ける”(deliver), “要求”(request), “戾”(return), “寄付”(donate), “寄贈”(donation), “着ける”(wear), “乗せる”(place), “上ごる”(increase), “落と”(drop), “貼る”(stick), “分ける”(divide), “ばらまく”(scatter), “はねる”(fit), “支払う”(pay), “配達”(deliver), “卸す”(unload), “缠)”(collect), “通)”(get X through)	“トレーナット”(present), “持續”(have), “合わせる”(match), “見せる”(show), “向/tagる”(point), “提供”(offer), “装備”(equip), “追加”(add), “展示”(exhibit), “据える”(lay), “採用”(adopt)
Table 8: The verbs which show a significant change in the argument order trend depending on the semantic role of its dative argument. The scores denote the DAT-ACC rate. Type-A corresponds to the examples with an inanimate goal dative argument. Type-B corresponds to the examples with an animate processor dative argument. The number of type-A verbs is significantly larger than that of type-B verbs.
Case	#occurrence
TIM	11,780
LOC	15,544
NOM	55,230
DAT	56,243
ACC	57,823
Table 9: The number of occurrence for each case in the data used in Section 5.2, Section 6, and Appendix F
F Additional analysis: adverbial particles and their effect for word order
The adverbial particles We can add supplementary information with adverbial particles. The adverbial particle "は" (TOP) is the typical one. In Example (12), the adverbial particle "も" (also), instead of "た" (ACC), implies that there is another thing the teacher gave to the student ("a teacher gave not only φ but also a book to a student").
(12)生徒に 本圭も あけた.
student-DAT also book-ACC gave.
Experiments A constituent accompanying the adverbial particle “は” (TOP) is moved to the beginning of the sentence (Noda, 1996). However, it
is not clear whether other adverbial particles also have the above property. In this section, we evaluate the following claim: a different adverbial particle shows different degrees of the effects for the word order.
For each example  $s \in S$  collected from Japanese Wikipedia, we replaced the postpositional particle with a specific adverbial particle, following the rules in Table 12. We used four typical adverbial particles: “は” (TOP), “ご” (emphasis), “も” (also), and “だ)” (only). Two variants of word order, Non-moved, and Moved were created for each example. Example (13) is an example focusing on the ACC case with the particle “も” (also).
(13) Original: 本を あた.
Non-moved: 生徒に 本姿も あた。
Moved: 本壺も 生徒に 本壺 あけた.
also book-ACC student-DAT book-ACC gave.
We compared the generation probabilities between the Non-moved and Moved orders. We calculated the rate that the Moved order is preferred in each combination of the case types and the adverbial particles.
Model	MODAL	TIME	MANNER	RESULTIVE
Canonical	r	Canonical	r	Canonical	r	Canonical	r
CLM	ASOV	1.	ASOV, SAOV	1.	SAOV, SOAV	0.5	SAOV, SOAV	1.
SLM	ASOV	1.	SAOV	0.5	SAOV, SOAV	1.	SOAV	0.5
Koizumi(2016)	ASOV	-	ASOV, SAOV	-	SAOV, SOAV	-	SAOV, SOAV	-
Table 10: Overlap of the preference of the adverb position of LMs and that of Koizumi and Tamaoka (2006). The column "Canonical" shows the adverb position, which is significantly preferred over the other positions. The score  $r$  denotes the Pearson correlation coefficient of the preferred rank of three possible adverb positions obtained from LMs to that of Koizumi and Tamaoka (2006).
TIM	PLC	NOM	DAT	NOM
TIM	-	.490	.329	.720	.698
PLC	.510	-	.484	.748	.742
NOM	.671	.516	-	.804	.852
DAT	.280	.252	.196	-	.536
NOM	.302	.258	.148	.464	-
(a) CLM
TIM	PLC	NOM	DAT	NOM
TIM	-	.538	.402	.676	.711
PLC	.462	-	.553	.757	.749
NOM	.598	.447	-	.774	.834
DAT	.324	.243	.226	-	.552
NOM	.289	.251	.166	.448	-
Results The results are shown in Table 13. When using "は" (TOP) as a postpositional particle, the Moved order is preferred to Non-moved, which is consistent with the well-known characteristics of topicalization described in Section 6. In addition, the degree of preference between Moved and Non-moved differs depending on the adverbial particles. Furthermore, the results indicate that the anterior case in the canonical word order is likely to move to the beginning of the sentence by the effect of the adverbial particle.
Additional experiments and results We analyzed the trend of double object order when a specific case accompanies an adverbial particle. Figure 5 shows the result when the ACC argument accompanies an adverbial particle, and Figure 6 shows the result when the DAT argument accompanies an adverbial particle. The left parts of these figures show the result of CLM, and the right part of these figures shows the result of SLM. The X-axis denotes the ACC-DAT/DAT-ACC rate of the verb when both of the arguments do not accom
(b) SLM
Table 11: The scores denote  $t_{a|b}$ . The row corresponds to the case  $a$ , the column corresponds to  $b$ . Higher  $t_{a|b}$  suggests the trend that the case  $a$  is more likely to be topicalized than the case  $b$ .
Original case particle	After the adverbial particle “は” (TOP) is added
が (TOP)	はは
に (TIM, DAT)	には
を (ACC)	きは
て (LOC)	ては
Table 12: Rules of deleting the original case particle when the adverbial particle “は” (TOP) is added. This rule is also applied when adding the other adverbial particles (Appendix F).
pany an adverbial particle. The Y-axis denotes the ACC-DAT/DAT-ACC rate when a specific case accompanies an adverbial particle. The results show that the case accompanying an adverbial particle is likely to be placed near the beginning of the sentence. In addition, the degree of the above trend depends on the adverbial particles. These results suggest that some adverbial particles have a effect for word order.
Model	Toritate particle	TIM	LOC	NOM	DAT	ACC	Avg.
CLM	“は”(TOP)	.715	.777	.675	.624	.623	.683
“这也”(emphasis)	.492	.423	.521	.313	.486	.447
“も”(also)	.560	.557	.458	.343	.271	.438
“んだけ”(only)	.385	.340	.312	.227	.184	.331
Avg.	.538	.525	.544	.377	.391	-
SLM	“は”(TOP)	.667	.751	.635	.565	.580	.640
“这也”(emphasis)	.567	.596	.574	.398	.462	.519
“も”(also)	.511	.531	.457	.292	.259	.410
“んだけ”(only)	.334	.309	.285	.172	.126	.303
Avg.	.520	.547	.560	.357	.357	-
Table 13: The scores denote that the Moved order is preferred over the Non-moved order when the corresponding case (column) accompanies the corresponding particle (row). The trend is different depending on the case and particle.
(a) CLM
(b) SLM
Figure 5: Change of the ACC-DAT order when the ACC argument accompanies an adverbial particle. These results indicate that the ACC argument with an adverbial particle  $(\mathrm{ACC}^{\mathrm{adv}})$  is more likely to be placed before the DAT argument. In addition, this trend differs for each particle.
(a) CLM
(b) SLM
Figure 6: Change of the DAT-ACC order when the DAT argument accompanies an adverbial particle. These results indicate that the DAT argument with an adverbial particle  $(\mathrm{DAT}^{\mathrm{adv}})$  is more likely to be placed before the ACC argument. In addition, this trend differs for each particle.