This paper describes an open-source implementation of the so-called continuous space language model and its application to statistical machine translation. The underlying idea of this approach is to attack the data sparseness problem by performing the language model probability estimation in a continuous space. The projection of the words and the probability estimation are both performed by a multi-layer neural network. This paper describes the theoretical background of the approach, efficient algorithms to handle the computational complexity, and gives implementation details and reports experimental results on a variety of tasks.
CSLM REF is an open source toolkit implementing a continuous space language model which has a similar architecture to our own.
Continuous-Space Language Models for Statistical Machine Translation
The Prague Bulletin of Mathematical Linguistics
Computer Science