We examine a methodology using neural language models ( LMs ) for analyzing the word order of language . This LM-based method has the potential to overcome the difficulties existing methods face , such as the propagation of preprocessor errors in count-based methods . In this study , we explore whether the LMbased method is valid for analyzing the word order . As a case study , this study focuses on Japanese due to its complex and flexible word order . To validate the LM-based method , we test ( i ) parallels between LMs and human word order preference , and ( ii ) consistency of the results obtained using the LM-based method with previous linguistic studies . Through our experiments , we tentatively conclude that LMs display sufficient word order knowledge for usage as an analysis tool . Finally , using the LMbased method , we demonstrate the relationship between the canonical word order and topicalization , which had yet to be analyzed by largescale experiments . Model Verbs whose type-A examples prefer the ACC-DAT order Verbs whose type-B examples prefer the ACC-DAT order CLM " 預ける " ( deposit ) , " 置く " ( place ) , " 持つ " ( have ) , " 入 れる " ( put in ) , " 納める " ( pay ) , " 郵送 " ( mail ) , " 供 給 " ( supply ) , " 出す " ( put out ) , " 運ぶ " ( transport ) , " 流 す " ( shed ) , " 掛ける " ( hang ) , " 飾る " ( decorate ) , " 広 げる " ( spread ) , " 移す " ( transfer ) , " 残す " ( leave ) , " 配 送 " ( deliver ) , " 送る " ( send ) , " 投げる " ( throw ) , " 送 付 " ( send ) , " 返却 " ( return ) , " 届ける " ( send ) , " 戻す " ( return ) , " 着ける " ( wear ) , " 上げる " ( increase ) , " 落と す " ( drop ) , " 載せる " ( publish ) , " 変更 " ( change ) , " 納 入 " ( deliver ) , " 卸す " ( unload ) , " 掲載 " ( publish ) , " 通 す " ( get X through ) " 配 布 " ( distribute ) , " 渡 す " ( pass ) , " プ レ ゼ ン ト " ( present ) , " 合わせる " ( match ) , " 見せる " ( show ) , " 提 供 " ( offer ) , " 与える " ( give ) , " 当てる " ( hit ) , " 回す " ( turn ) , " 追加 " ( add ) , " 貸す " ( lend ) , " 展示 " ( exhibit ) , " 据える " ( lay ) , " 依頼 " ( request ) , " 挿入 " ( insert ) , " 纏 める " ( collect ) , " 請求 " ( claim ) SLM " 預ける " ( deposit ) , " 置く " ( place ) , " 頼む " ( ask ) , " 入 れる " ( put in ) , " 納める " ( pay ) , " 郵送 " ( mail ) , " 出 す " ( put out ) , " 運ぶ " ( transport ) , " 流す " ( shed ) , " 掛け る " ( hang ) , " 広げる " ( spread ) , " 移す " ( transfer ) , " 残 す " ( leave ) , " リクエスト " ( request ) , " 配送 " ( deliver ) , " 送る " ( send ) , " 投げる " ( throw ) , " 送付 " ( send ) , " 求 める " ( ask ) , " 提出 " ( submit ) , " 届ける " ( deliver ) , " 要 求 " ( request ) , " 戻す " ( return ) , " 寄付 " ( donate ) , " 寄贈 " ( donation ) , " 着ける " ( wear ) , " 乗せる " ( place ) , " 上げ る " ( increase ) , " 落とす " ( drop ) , " 貼る " ( stick ) , " 分け る " ( divide ) , " ばらまく " ( scatter ) , " はめる " ( fit ) , " 支 払う " ( pay ) , " 配達 " ( deliver ) , " 卸す " ( unload ) , " 纏め る " ( collect ) , " 通す " ( get X through ) " プレゼント " ( present ) , " 持つ " ( have ) , " 合わせる " ( match ) , " 見せる " ( show ) , " 向ける " ( point ) , " 提供 " ( offer ) , " 装備 " ( equip ) , " 追加 " ( add ) , " 展示 " ( exhibit ) , " 据える " ( lay ) , " 採用 " ( adopt ) Speakers sometimes have a range of options for word order in conveying a similar meaning . A typical case in English is dative alternation : ( 1 ) a. A teacher gave a student a book . b. A teacher gave a book to a student . Even for such a particular alternation , several studies ( Bresnan et al . , 2007 ; Hovav and Levin , 2008 ; Colleman , 2009 ) investigated the factors determining this word order and found that the choice is not random . For analyzing such linguistic phenomena , linguists repeat the cycle of constructing hypotheses and testing their validity , usually through psychological experiments or count-based methods . However , these approaches sometimes face difficulties , such as scalability issues in psychological experiments and the propagation of preprocessor errors in count-based methods . Compared to the typical approaches for evaluating linguistic hypotheses , approaches using LMs have potential advantages ( Section 3.2 ) . In this study , we examine the methodology of using LMs for analyzing word order ( Figure 1 ) . To validate the LM-based method , we first examine if there is a parallel between canonical word order and generation probability of LMs for each word order . Futrell and Levy ( 2019 ) reported that English LMs have human-like word order preferences , which can be one piece of evidence for validating the LM-based method . However , it is not clear whether the above assumption is valid even in languages with more flexible word order . In this study , we specifically focus on the Japanese language due to its complex and flexible word order . There are many claims on the canonical word order of Japanese , and it has attracted considerable attention from linguists and natural language processing ( NLP ) researchers for decades ( Hoji , 1985 ; Saeki , 1998 ; Miyamoto , 2002 ; Matsuoka , 2003 ; Koizumi and Tamaoka , 2004 ; Nakamoto et al . , 2006 ; Shigenaga , 2014 ; Sasano and Okumura , 2016 ; Orita , 2017 ; Asahara et al . , 2018 ) . We investigated the validity of using Japanese LMs for canonical word order analysis by conducting two sets of experiments : ( i ) comparing word order preference in LMs to that in Japanese speakers ( Section 4 ) , and ( ii ) checking the consistency Topic Time Location Subject ( Adverb ) Indirect object Direct object Verb Notation TOP TIM LOC NOM -DAT ACC -Typical particle " は " ( wa ) " に " ( ni ) " で " ( de ) " が " ( ga ) -"に " ( ni ) " を " ( o ) -Related section 6 5.2 5.2 5.2 5.3 5.1 5.1 5.1 Table 1 : Overview of the typical cases in Japanese , their typical particles , and the sections where the corresponding case is analyzed . The well-known canonical word order of Japanese is listed from left to right . between the preference of LMs with previous linguistic studies ( Section 5 ) . From our experiments , we tentatively conclude that LMs display sufficient word order knowledge for usage as an analysis tool , and further explore potential applications . Finally , we analyzed the relationship between topicalization and word order of Japanese by taking advantage of the LM-based method ( Section 6 ) . In summary , we : • Discuss and validate the use of LMs as a tool for word order analysis as well as investigate the sensitivity of LMs against different word orders in non-European language ( Section 3 ) ; • Find encouraging parallels between the results obtained with the LM-based method and those with the previously established method on various hypotheses of canonical word order of Japanese ( Sections 4 and 5 ) ; and • Showcase the advantages of an LM-based method through analyzing linguistic phenomena that is difficult to explore with the previous data-driven methods ( Section 6 ) . We have proposed to use LMs as a tool for analyzing word order in Japanese . Our experimental results support the validity of using Japanese LMs for canonical word order analysis , which has the potential to broaden the possibilities of linguistic research . From an engineering view , this study supports the use of LMs for scoring Japanese word order automatically . From the viewpoint of the linguistic field , we provide additional empirical evidence to various word order hypotheses as well as demonstrate the validity of the LM-based method . We plan to further explore the capability of LMs on other linguistic phenomena related to word order , such as " given new ordering " ( Nakagawa , 2016 ; Asahara et al . , 2018 ) . Since LMs are language-agnostic , analyzing word order in another language with the LM-based method would also be an interesting direction to investigate . Furthermore , we would like to extend a comparison between machine and human language processing beyond the perspective of word order .

Challenge: Linguistical approaches to analyze word order phenomena have scalability and preprocessor error propagation problems, and the use of language models is limited in English.
Approach: They validate language models as a tool to study word order in Japanese by examining the relationship between canonical word order and generation probability.
Outcome: They show that language models have sufficient word order knowledge in Japanese to be used as a tool for linguists.