Annotating Modality Expressions and Event Factuality for a Japanese Chess
Commentary Corpus
Suguru Matsuyoshi1 , Hirotaka Kameko2 , Yugo Murawaki3 , Shinsuke Mori4
1

Graduate School of Informatics and Engineering, The University of Electro-Communications
2
Graduate School of Engineering, The University of Tokyo
3
Graduate School of Informatics, Kyoto University
4
Academic Center for Computing and Media Studies, Kyoto University
1
Chofugaoka, Chofu, Tokyo, Japan 2 Hongo, Bunkyo-ku, Tokyo, Japan
3,4
Yoshida-honmachi, Sakyo-ku, Kyoto, Japan1
2
matuyosi@uec.ac.jp, kameko@logos.t.u-tokyo.ac.jp, 3 murawaki@i.kyoto-u.ac.jp, 4 forest@i.kyoto-u.ac.jp
Abstract
In recent years, there has been a surge of interest in the natural language processing related to the real world, such as symbol grounding,language generation, and nonlinguistic data search by natural language queries. We argue that shogi (Japanese chess) commentaries,
which are accompanied by game states, are an interesting testbed for these tasks. A commentator refers not only to the current board
state but to past and future moves, and yet such references can be grounded in the game tree, possibly with the help of modern game-tree
search algorithms. For this reason, we previously collected shogi commentaries together with board states and have been developing agame commentary generator. In this paper, we augment the corpus with manual annotation of modality expressions and event factuality.
The annotated corpus includes 1,622 modality expressions, 5,014 event class tags and 3,092 factuality tags. It can be used to train
a computer to identify words and phrases that signal factuality and to determine events with the said factuality, paving the way forgrounding possible and counterfactual states.
Keywords: game commentary, modality, factuality, symbol grounding

1. Introduction
The field of natural language processing (NLP) has experienced a resurgence of interest in the symbol grounding
problem (Harnad, 1990). An increasing number of datasets
available align natural language expressions with real world
objects in the form of images and videos (Hashimoto et al.,2014; Mori et al., 2014b; Ferraro et al., 2015), and systems
built on top of such datasets typically perform image/videoto-text generation (Ushiku et al., 2011; Yang et al., 2011).
These systems are, however, more akin to great apes thanto humans in the sense that “[t]heir lives are lived entirelyin the present” (Donald, 1991). While there is no evidence that nonhuman animals communicate past episodes
and planned future events, human language is abundantwith them (Szagun, 1978). It is even suggested that thefaculty of language has a close connection to the ability toimage the future (Suddendorf and Corballis, 1997).
Here we argue that commentaries for an extensive-form
game (e.g., chess) are an ideal testbed for developing
truly intelligent systems. Specifically, we focus on shogi(Japanese chess). Unlike typical image captions, humancomments on shogi games are full of references to past andfuture moves as we will see in Subsection 2.2. Yet, thanks
to the well-definedness of the world, many of such references can be grounded in a game tree. Although ambiguities inherent in natural language remain a challenging
problem, modern game-tree search algorithms (Tsuruoka
et al., 2002) help distinguish realistic states from unrealistic ones. For this reason, we have collected shogi commentaries together with the corresponding board states andhave been developing a game commentary generator (Moriet al., 2016; Kameko et al., 2015).

Figure 1: Starting setup of shogi (left: normal depiction,right: chess-like depiction).
A human commentator usually expresses to shogi fans thedegree of confidence on the future move he is describing,which reflects his evaluation of the game-tree. Since confidence is expressed through a wide variety of words andphrases, which we call modality expressions, identifying
such expressions and binding the factual statuses to events
are necessary steps toward automatic generation of humanlike commentaries.
To this end, we annotate a shogi corpus for a commentator’s confidence in this work. Specifically, we adopt a
three-layer annotation scheme: modality expression, eventclass and factuality. The first layer specifies constituents ofa sentence construction conveying a certain level of confidence. The second layer extracts event mentions that can bemapped into a game tree. The last layer denotes confidence
level of each event mention.

2. Game and Commentary
2.1.

Shogi

Shogi is a two-player board game similar to chess as illus-

Tag
Tu
Po
Pi
Ps
Mc
Mn
Me
St
Ca
Ev
Ee
Re
Ph
Pa
Pq
Hu
Ti
Ac
Ap
Ao
Ot

Meaning
Turn
Position
Piece
Piece specifier
Move complimentMove nameMove evaluation
StrategyCastleEvaluation: entire
Evaluation: part
RegionPhase
Piece attribute
Piece quantity
Human
Time
Player action
Piece action
Other action
Other notion

Table 1: Shogi-specific named entity tags.

trated in Fig 1. The goal of each player is to capture theopponent’s king. Each player moves one piece alternately.
The biggest difference from chess is that a player can dropa captured piece back onto the board instead of making a
move. For detailed explanation of shogi, please refer to(Leggett, 2009).

2.2. Shogi Commentary
Professional players and writers give commentaries of professional matches for shogi fans. Mori et al. (2016) collected shogi commentaries with board states described in
Shogi Forsyth-Edwards notation (SFEN). The corpus consists of 6,523 matches including 744,327 sentences and11,083,669 words. Mori et al. (2016) first segmented ninematches, or 2,041 sentences, automatically with a text analyzer KyTea1 and then corrected word boundaries manually. Finally they manually added to the sentences shogispecific named entities (NEs) tags defined in Table 1. We
call the annotated corpus the Shogi Game Commentary
Corpus (SGC corpus).
In the SGC corpus, we observed that the following contents were mainly expressed with recollection of the pastand imagination of the future:• Reason behind moves.
この桂馬打ちは飛車取りを狙ったものだ (This dropof knight aims at capturing the opponent’s rook.)
角の仇ですね (The move must be a revenge for hisbishop.)• Prediction of the next moves and strategies.
美濃囲いが有効だ (Black will build Mino castle formaking it better.)1

http://www.phontron.com/kytea/

ゴキゲン中飛車との予測は外れた (I expected thatwhite used cheerful central rook strategy, but he didnot.)• Mentioning future directions.
△１四香とすれば決戦 (White’s Lx1d would shift thephase to the end game.)

3. Japanese Modality Expressions
We augment the SGC corpus with annotations of modality
expressions and event factuality. Table 2 shows a sample
text annotated according to our annotation scheme. In thissection, we describe the first layer of modality expressions.
Modality expressions are words and multi-word expressions which impose a proposition in a sentence on propositional modality and event modality (Palmer, 2001). Hereare some examples of Japanese modality expressions (EVdenotes an event mention and ME denotes a modality expression):
EX 1 後手は歩を成り捨てEV たME (White sacrificeEV edME the pawn in the oppenent’s zone.)
EX 2 こ の 試 合 で は 居 飛 車 を採用EV
す るかもしれないME (White mayME useEV rook
stay strategy in the game.)
EX 3 おそらくME △ １ 四 香 が良好EV
white’s Lx1d will be goodEV .)(ProbablyME

EX 4 ▲８五桂の跳ねEV 出しを防いME だ (The move
preventedME black’s Nx8e attackEV .)EX 5 ここで△１四歩と受けEV ればME 先手はつらい(IfME white choosesEV Px1d, black will have a hard
time.)In EX 1, “た (PAST)” is an auxiliary verb for past tense andindicates that the event is a fact. In EX 2, “かもしれない(may)” is a multi-word expression functioning as an auxiliary verb and expressing the low possibility of the event. In
EX 3, “おそらく (probably)” is an adverb which suggeststhat the event is probable. “防い (prevent)” is a verb used
as a modality expression while in EX 4 it is used a mainverb and can be seen as an event mention. This counterfactive verb makes it explicit that the event “跳ね (attack)”
did not happen. In EX 5, “ば (if)” is a conjunctive particle
that leads to a conditional construction. Factuality of event
mentions in the hypothetical construction is absolutely uncertain.
In the first layer of our annotation scheme, we mark up
modality expressions as described above.

3.1. POS
There have been several studies of detecting Japanese
modality expressions (Suzuki et al., 2012; Izumi et al.,2013) (Kamioka et al., 2015). However, they focus on auxiliary verbs and functional multi-word expressions. Modal
adverbs and conjunctive particles are largely out of scope ofthese studies. By contrast, we target all types of modality
expressions for mark-up regardless of their parts-of-speech.

LayerString

先手 は 美濃 囲い が崩れ て い る の で 、 飛車 交換 は 後手 の 得 に な り そう だ 。
black TOP Mino castle NOM break have PP becauserook change TOP white ’s good to be
be likely to
POS
N
P
N
N
P
V
P V Sf P Aux Pnc NNP
N P N P V Sf Adj Aux Pnc
NE
Tu-B O Ca-B Ca-I O
Ao-B O O O O O O Mn-B Mn-I O Tu-B O Ee-B O Ao-B OOO OModalityOOOOO MEn-B O O O O O OOOOO O O O O O MEa-B O O
Event class
EVe
EVe
EVf
EVi
EVe
Factuality
FNc
FPc
FPr

Table 2: Five annotation layers for a commentary string with a well-formatted board state.

3.2. Tags
We define the following two groups of tags: factuality andtime.
3.2.1. Factuality GroupThe following five tags concern possibility and polarity.
MEy suggests the target event is factual.

銀 の捕獲EV に成功MEy (Black succeededMEy in
capturingEV the silver general.)MEa suggests the target event is possibly factual.

先手はこのあと居飛車に組むEV 可能性が高いMEa(Black is likely toMEa useEV static rook strategy.)
ME0 suspends mentioning possibility of the target event.

その間に先手玉に迫るEV 手段があるかどうかME0(Is thereME0 a way to attackEV black’s king before thecastle is completed?)
MEm suggests the target event is possibly counterfactual.

後手の飛車もあまりMEm 利いEV ていない (White’srook hasEV littleMEm effect on it.)MEn suggests the target event is counterfactual.

△ ９ 四 歩 と はしEV なかっMEn た (He did notMEn
takeEV Px9d.)

3.2.2. Time GroupWe introduce the following three tags from the viewpointof timeline regardless of polarity.
MEp suggests the target event was in the past.

ここで銀交換EV に応じたMEp (They changeEV -edMEp
each other’s silver general.)

4.

Event Classes and Factuality

In this section, we describe the second and third layers of
our annotation scheme. In the second layer, we classify
event mentions into event classes. The objective of the classification is to distinguish from the others event mentions
that can be mapped into a game tree. In the third layer, wemark up the factual statuses of events.

4.1. Event
A sentence conveying information contains not only propositions, but also modality, polarity and the writer’s attitude. A proposition is expressed as an event mention ina sentence. Following TimeML (Sauri et al., 2006), we
consider events a cover term for situations that happen or
occur. Propositions describing states or circumstances arealso considered as events.

4.2. Event Classes
As mentioned in the FactBank annotation guidelines (Sauri,2008), it is not preferable to target all event mentions in atext for factuality mark-up. This is because a writer’s attitude such as wish, command, permission, question andhypothesis is expressed in a text. An event mention surrounded by such an attitude expression is not suitable forfactuality mark-up because the factuality of the event isabsolutely uncertain. We believe that these events should
be separated from events that are free from a writer’s attitude. For this reason, we classify event mentions into several event classes from the viewpoint of attitude in advanceof factuality mark-up.
Automatic extraction of event mentions in Japanese text iscomplicated by the existence of grammaticalized verbs andadjectives. Japanese text includes a non-negligible numberof these tokens. We mark up grammaticalized tokens inadvance of factuality mark-up.
We define the following two groups of class tags: attitude
and grammaticalization.

MEf suggests the target event will or will not happen in the

future.
先手は将来MEf 的に右辺に玉を囲うEV ことになる(White will eventuallyMEf besiegeEV the opponent’s
king at the right side of the board.)
MEh suggests the target event is hypothesized or has a con-

4.2.1. Attitude GroupThe following five classes of tags reflect a writer’s attitude.
EVa With wish, request, command or obligation.

多くのファンに楽しEVa んでもらいたい (I wish manyfans enjoyEVa this game.)dition.
ここで△１四歩と受けEV ればMEh 先手はつらい(IfMEh white choosesEV Px1d, black will have a hard
time.)

EVq With interrogative.

先手は桂を取るEVq か (Will black attackEVq white’sknight?)EVi Imaginary event, i.e., in conditional construction or not

instantiated.
ここで△１四歩と受けEVi れば先手はつらいEVi(If white choosesEVi Px1d, black will haveEVi a hard
time.)
EVp With granting a permission.

サ イ ン 会 だ け の参加EVp も 可 能 で す (You canparticipateEVp only in an autograph session.)
EVs Simile or metaphor.

３四の銀をあざ笑EVs うかのように玉を進行させる(The king is proceeding as if it ridiculedEVs the silver
general of Sx3d.)
4.2.2. Grammaticalization GroupWe introduce the following three classes of tags from theviewpoint of lexical/grammatical usage.
EVe Event mention whose core is a lexical predicate withsome arguments.
後手は歩を成り捨てEVe た (White sacrificeEVe -ed thepawn in the oppenent’s zone.)
EVc Constituent of a named entity or a verb phrase whoserole is a modifier. While it is not a normal event mention, it can be interpreted as an event mention if required in some tasks such as recognizing textual entailment and information extraction.
遊びEVc 駒を竜にぶつけた (He made an idleEVc piece
invade a path of the opponent’s promoted rook.)
EVf Grammaticalized function word or a constituent of afunctional multi-word expression.
封じ手についEVf て検討が進んでいる (Examinationof [= について (lit. be attachedEVf to)] the next movehas been progressing.)

4.3. Factuality Tags
In the third layer of our annotation scheme, we mark up a
factuality tag for an event mention whose class is EVe in
the previous phase.
As factuality tags, we use factuality values proposed in
FactBank (Sauri, 2008). However, the value Uu is the sole
exception. It is not adopted in our scheme because a predicate which would have Uu has been annotated with an EVq
or EVi tag in the previous phase. Consequently, we use thefollowing six tags for factuality mark-up. (P stands for the
propositon mentioned in a target event. The corresponding
values in FactBank are shown in brackets):
FPc Certainly the case that P. <CT+>

後手は歩を成り捨てFPc た (White sacrificeFPc -ed thepawn in the oppenent’s zone.)FPr Probably the case that P. <PR+>

おそらく△１四香が良好FPr (Probably white’s Lx1d
will be goodFPr .)

Tag
MEy-B
MEa-B
ME0-B
MEm-B
MEn-B
MEp-B
MEf-B
MEh-B

Total
EVa
EVq
EVi
EVp
EVs
EVe
EVc
EVf

Total
FPc
FPr
FPs
FNc
FNr
FNs

Total

Freq.
49
224
158
21
269
692
59
150
1,622
39
111
707
7
4
3,092
293
761
5,014
2,646
233
35
140
34
4
3,092

Ratio
3%
14%
10%
1%
16%
43%
4%
9%
100%
1%
2%
14%
0%
0%
62%
6%
15%
100%
86%
7%
1%
5%
1%
0%
100%

Table 3: Frequency and ratio of each tag of the three-layer
annotation.

FPs Possibly the case that P. <PS+>

この試合では居飛車を採用FPs するかもしれない(White may useFPs rook stay strategy in the game.)
FNc Certainly not the case that P. <CT−>

角交換FNc には応じなかった (He was not tempted tochangeFNc each other’s bishop.)
FNr Probably not the case that P. <PR−>

穴熊に組むFNr つもりはないだろう (I don’t think
white will useFNr anaguma castle.)FNs Possibly not the case that P. <PS−>△９四歩は指しFNs づらいかもしれません (Maybe
it is hard to takeFNs Px9d.)

5.

Annotated Corpus

We explain our annotation process and show some statistics
on our annotated corpus. We also report preliminary experiments on modality expression detection and event factuality
analysis using it.

5.1. Annotation Process
In annotation process, we annotated the SGC corpus withmodality expressions, event classes and factuality in this order. Because a modality expression can be made of morethan one word, we adopt the IOB2 tagging style (Sang and
Meulder, 2003) for our modality expression layer. The

Tag
MEy
MEa
ME0
MEm
MEn
MEp
MEf
MEh

Freq.
21
103
105
7
140
370
32
66

Precision
0.00
0.63
0.63
1.00
0.75
0.91
0.92
0.69

Recall
0.00
0.68
0.60
0.14
0.71
0.92
0.34
0.47

F-measure
0.00
0.65
0.62
0.25
0.73
0.92
0.50
0.56

Table 4: Results of modality expression detection.
Tag
FPc
FPr
FPs
FNc
FNr
FNs

Freq.
1,349
116
21
67
16
3

Precision
0.64
0.34
0.00
0.50
0.00
0.00

Recall
0.63
0.14
0.00
0.15
0.00
0.00

F-measure
0.64
0.20
0.00
0.23
0.00
0.00

Table 5: Results of event factuality analysis.

features. The second module searches for the best tag sequence by using conditional random fields referring to thetag-confidence pairs.
Tables 4 and 5 show the results of modality expression detection and event factuality analysis, respectively. In thetables, “Freq.” indicates the number of a target tag in thetest data. From Table 4, we can see that the F-measures ofthe modality expression tags whose frequencies are morethan 100 are high although the training data is small in size.
We expect that increasing the size of the corpus can leadto higher performance of modality expression detection forshogi commentaries. In the event factuality analysis task,
we obtained the F-measure of 0.64 for the FPc (factual) tag
while the F-measures for the other tags are low, as shownin Table 5. This suggests that event factuality analysis forshogi commentaries is a difficult task, although increasing
the size of the corpus might be effective. Given the obvious
dependency of event factuality analysis on modality expressions, we expect to improve performance of event factuality
analysis by incorporating the results of modality expression
detection as features.

5.4. Availability
annotation was performed by a single annotator. To prepare concise guidelines for annotating our tags for otherannotators and examine inter-annotator agreement are future work.

We plan to distribute our corpus, except textdata,4 in
our website http://www.ar.media.kyoto-u.ac.
jp/data/game/home-e.html. For detailed explanations, readers may visit it.

5.2. Some Statistics
Table 3 shows the frequency and ratio of each tag in our
three-layer annotation. Our corpus contains 1,622 modality expressions and 3,092 lexical event mentions (EVe).
Among them 2,646 are labeled with FPc (factual), while
140 are labeled with FNc (counterfactual). The remaining
are judged as being in hedge contexts.

5.3. Experiments
To give a glimpse of how difficult the tasks of modality expression detection and event factuality analysis are, we ranon the corpus a simple tagger with a basic set of features.
As described in Subsection 2.2., the SGC corpus consistsof nine matches. We used the latest match2 among them as
test data and the others as training data in experiments. The
numbers of the sentences in the training and the test data
are 968 and 1,072, respectively.
For both of modality expression detection and event factuality analysis, we adopt sequential labeling of tags including the O (“not the case”) tag. In the experiments, weused PWNER3 (Sasada et al., 2015) as a tool for sequential labeling. PWNER is a named entity recognizer, whichis composed of two modules. The first module enumerates
IOB2 tags with confidences for each word independently
from the tags for other words. It uses character strings andcharacter types (Chinese character, hiragana, Arabic numeral, and so on) of words surrounding the target word as

6. Application
The most important applications of our corpus are text analysis such as modality expression detection and factuality
analysis as employed in Subsection 5.3. Below we discuss
several future research directions.

6.1. Factuality Analysis
Every sentence in our corpus has the corresponding board
state (the state history if necessary) and NE labels. They
can assist in event factuality analysis. Automatic simultaneous tagging of NE and factuality might work well.

6.2. Automatic Commentary Generation
With our corpus, we can improve automatic commentary
generation (Kaneko, 2012; Kameko et al., 2015). The previous work proposed a two-step approach where identification of characteristic words for the given game state is followed by language model-based generation. With consideration for event factuality in addition to characteristic words,
the generator is expected to choose appropriate modality
expressions. We can try generation using automaticallygenerated templates (Reiter, 1995; Mori et al., 2014a) or
deep learning with NEs in place of dialog acts (Wen et al.,2015).
4

2

This is a title match called “Meijinsen,” which causes longer
text than a usual match.
3
http://www.ar.media.kyoto-u.ac.jp/tool/
PWNER/home.html

The game records and the commentary sentences are distributed in the website: http://www.meijinsen.jp (inJapanese) for a fee. We provide a helper script to download
the records and the text at https://github.com/hkmk/
shogi-comment-tools.

6.3. Game State Retrieval
We can also create a system for game state search by natural language queries. A previous study proposed search forgame states by piece positions (Ganguly et al., 2014). NE
recognition enables a user to search by strategy names andmove evaluation (Ushiku et al., 2017). Factuality analysis
allows for fine grained retrieval of game states, for example, the one in which the given strategy was judged good by
a commentator, and the one where the strategy was mentioned as an option but was not adopted by the player.

6.4. Symbol Grounding
One of the most interesting research directions is symbol
grounding. While symbol grounding of a content word to
the world is a straight concept, grounding of a modality expression such as “must” and “may” to images, videos, manyother forms of media is an open question. An applicationof modal logic (Kripke, 1963) to shogi game tree as a setof possible worlds can be a solution to grounding of somemodality expressions.
We believe that there are many other novel applications
including bilingual lexicon acquisition for function words
and modality expressions based on symbol grounding(Kiela et al., 2015).

7.

Conclusion

In this paper, we described our annotation scheme ofmodality expressions, event classes, and factuality. We annotated the text of a shogi commentary corpus with thesetags. The annotated corpus includes 1,622 modality expressions, 5,014 event class tags and 3,092 factuality tags.
As illustrated in the example sentences at Subsection 2.2.,
human language, which is different from great ape communication, is abundant with recollection of the past andimagination of the future. Our work leads to the tasks ofautomatically identifying modality expressions and binding
the factual statuses to events in a text. They are necessary
steps toward automatic analysis and generation of humancommentaries.
The most interesting characteristics of our corpus is thatevery commentary is associated with a game state (realworld). This will enable NLP and AI researchers to tackle
various new problems such as commentary generation, intelligent game state search, and symbol grounding.

8. Bibliographical References
Donald, M. (1991). Origins of the Modern Mind: Three
Stages in the Evolution of Culture and Cognition. Harvard University Press.
Ferraro, F., Mostafazadeh, N., Huang, T.-H., Vanderwende,L., Devlin, J., Galley, M., and Mitchell, M. (2015). A
survey of current datasets for vision and language research. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages
207–213.
Ganguly, D., Leveling, J., and Jones, G. J. (2014). Retrieval of similar chess positions. In Proceedings of the37th annual international ACM SIGIR conference, pages
687–696. ACM.

Harnad, S. (1990). The symbol grounding problem. Physica D, 42:335–346.
Hashimoto, A., Sasada, T., Yamakata, Y., Mori, S., and Minoh, M. (2014). KUSK dataset: Toward a direct understanding of recipe text and human cooking activity.
In Proceedings of the SixthInternational Workshop on
Cooking and Eating Activities.
Izumi, T., Imamura, K., Asami, T., Saito, K., Kikui, G., andSato, S. (2013). Normalizing complex functional expressions in Japanese predicates: Linguistically-directed
rule-based paraphrasing and its application. ACM Transactions on Asian Language Information Processing,12(3):1–20.
Kameko, H., Mori, S., and Tsuruoka, Y. (2015). Learninga game commentary generator with grounded move expressions. In Proceedings of the 2015 IEEE Conference
on Computational Intelligence and Games.
Kaneko, T. (2012). Real time commentary system forshogi. In First Workshop on Games and NLP.
Kiela, D., Vulić, I., and Clark, S. (2015). Visual bilingual
lexicon induction with transferred convnet features. In
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 148–158.
Kripke, S. A. (1963). Semantical considerations on modal
logic. Acta Philosophica Fennica, 16:83–94.
Leggett, T. (2009). Japanese chess : the game of shogi.
Tuttle Publishing.
Mori, S., Maeta, H., Sasada, T., Yoshino, K., Hashimoto,A., Funatomi, T., and Yamakata, Y. (2014a). Flowgraph2text: Automatic sentence skeleton compilation forprocedural text generation. In Proceedings of the theEighth International Conference on Natural Language
Generation, pages 118–122.
Mori, S., Maeta, H., Yamakata, Y., and Sasada, T. (2014b).
Flow graph corpus from recipe texts. In Proceedings of
the Nineth International Conference on Language Resources and Evaluation, pages 2370–2377.
Mori, S., Richardson, J., Ushiku, A., Sasada, T., Kameko,
H., and Tsuruoka, Y. (2016). A japanese chess commentary corpus. In Proceedings of the Tenth International
Conference on Language Resources and Evaluation.
Palmer, F. (2001). Mood and Modality Second edition.
Cambridge University Press.
Reiter, E. (1995). Nlg vs. templates. In Proceedings ofthe the Fifth European Workshop on Natural Language
Generation, pages 147–151.
Sang, E. F. T. K. and Meulder, F. D. (2003). Introduction
to the conll-2003 shared task: Language-independent
named entity recognition. In Proceedings of the Seventh
Conference on Computational Natural Language Learning, pages 142–147.
Sasada, T., Mori, S., Kawahara, T., and Yamakata, Y.
(2015). Named entity recognizer trainable from partiallyannotated data. In Proceedings of the Eleventh International Conference Pacific Association for Computational
Linguistics.
Suddendorf, T. and Corballis, M. C. (1997). Mental time
travel and the evolution of the human mind. Genetic, So-cial, and General Psychology Monographs, 123(2):133–
167.
Suzuki, T., Abe, Y., Toyota, I., Utsuro, T., Matsuyoshi, S., and Tsuchiya, M. (2012). Detecting
Japanese compound functional expressions using canonical/derivational relation. In International Conference on
Language Resources and Evaluation.
Szagun, G. (1978). On the frequency of use of tenses in english and german children’s spontaneous speech. Child
Development, 49(3):898–901.
Tsuruoka, Y., Yokoyama, D., and Chikayama, T. (2002).
Game-tree search algorithm based on realization probability. ICGA Journal, 25(3):145–152.
Ushiku, Y., Harada, T., and Kuniyoshi, Y. (2011). Automatic sentence generation from images. In Proceedings of the 19th Annual ACM International Conference
on Multimedia, pages 1533–1536.
Ushiku, A., Mori, S., Kameko, H., and Tsuruoka, Y.
(2017). Game state retrieval with keyword queries. In
annual international ACM SIGIR conference.
Wen, T.-H., Gasic, M., Mrksic, N., Su, P.-H., Vandyke, D.,
and Young, S. (2015). Semantically conditioned lstmbased natural language generation for spoken dialoguesystems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages
207–213.
Yang, Y., Teo, C. L., III, H. D., and Aloimonos, Y. (2011).
Corpus-guided sentence generation of natural images. In
Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing.

9. Language Resource References
Kamioka, Y., Narita, K., Mizuno, J., Kanno, M., and Inui,K. (2015). Semantic annotation of japanese functional
expressions and its impact on factuality analysis. In Proceedings of The 9th Linguistic Annotation Workshop,
pages 52–61.
Sauri, R., Littman, J., Knippen, B., Gaizauskas,R., Setzer, A., , and Pustejovsky, J., (2006).
TimeML Annotation Guidelines Version 1.2.1.
https://catalog.ldc.upenn.edu/docs/
LDC2006T08/timeml_annguide_1.2.1.pdf.
Sauri, R., (2008). FactBank 1.0 Annotation Guidelines.
https://catalog.ldc.upenn.edu/docs/
LDC2009T23/annotationGuidelines.pdf.