A Study on Encoding Natural Language into Distributed Representations 自然言語の分散表現へのエンコードに関する研究

Modeling the meaning of a text is a crucial technique for NLP applications such as relation extraction, textual entailment recognition, paraphrase detection, and so on. Thus, the goal of this thesis is to compute the meanings of words and phrases in order to comprehend the meaning of a text. To compute the meanings of words, the most basic way is constructing a co-occurrence matrix between words. After the construction, we can regard each row of the matrix as semantic vectors of words and compute the similarity of meanings of words by computing the similarity of semantic vectors based on the distributional hypothesis [Harris, 1954]. In addition to the above distributional representation, recent researches proposed various ways to learn distributed representations, which are dense real-valued vectors encoding syntactic and semantic information, and demonstrated distributed representations are useful to capture the meanings of words. To expand those meaning representations of words into arbitrary-length phrases, the most naive approach is to regard a phrase as a single unit (word) and to acquire distributional/distributed representations of words as usual. However, this approach might suffer from the datasparseness problem. To adress the datasparseness problem, we should compute the meaning of a phrase from constituent words. In this thesis, we propose two neural network based methods: modified Recursive Neural Network and Gated Additive Composition. Modified Recursive Neural Network can handle a transition of a polarity by each word such as increase and decrease. Gated Additive Composition can take account of an importance of a word. For example, content words such as verbs are more important than function words such as prepositions in computing the meaning of a phrase. The latter model is more general than the former model because the latter model can also handle an important dimension for representing polarities. We have few datasets to evaluate computing the meaning of phrases. Thus, in this thesis, we construct a new evaluation dataset that contains phrase pairs (specifically, relational pattern pairs) with similarity rating annotations. Using the constructed dataset, we show that the proposed method (specifically, Gated Additive Composition) is superior to othe neural network based methods. In addition to the above evaluation, we evaluate several methods on various datasets to explore the efficient way to compose distributed representations of arbitrary-length phrases from distributed representations of constituent words. Moreover, we indicate that the efficiency of distributed representations of phrases by applying the distributed representations to an application task. We also propose the method that encodes syntactic and semantic information such as part-of-speech (POS) tags. Through experiments, we show that the proposed method outperforms the method that uses only distributed representations of words. In other words, this thesis also indicates an effect of distributed representations of syntactic and semantic information whom traditional NLP parses.