Shallow Semantically-Informed PBSMT and HPBSMT
Tsuyoshi Okita
Qun Liu
Josef van Genabith
Dublin City University
Glasnevin, Dublin 9, Ireland{tokita,qliu,josef}@computing.dcu.ie
AbstractThis
paper
describes
shallowsemantically-informed
HierarchicalPhrase-based SMT (HPBSMT) andPhrase-Based SMT (PBSMT) systems
developed at Dublin City University
for participation in the translation task
between EN-ES and ES-EN at the Workshop on Statistical Machine Translation(WMT 13). The system uses PBSMTand HPBSMT decoders with multipleLMs, but will run only one decoding
path decided before starting translation.
Therefore the paper does not present a
multi-engine system combination. We
investigate three types of shallow semantics: (i) Quality Estimation (QE) score,(ii) genre ID, and (iii) context ID derived
from context-dependent language models.
Our results show that the improvement is
0.8 points absolute (BLEU) for EN-ES
and 0.7 points for ES-EN compared tothe standard PBSMT system (single bestsystem). It is important to note that wedeveloped this method when the standard(confusion network-based) system combination is ineffective such as in the casewhen the input is only two.

1

Introduction

This paper describes shallow semanticallyinformed Hierarchical Phrase-based SMT(HPBSMT) and Phrase-Based SMT (PBSMT)
systems developed at Dublin City University
for participation in the translation task between
EN-ES and ES-EN at WMT 13. Our objectives
are to incorporate several shallow semantics into
SMT systems. The first semantics is the QE score
for a given input sentence which can be used to
select the decoding path either of HPBSMT or

PBSMT. Although we call this a QE score, thisscore is not quite a standard one which does nothave access to translation output information. The
second semantics is genre ID which is intended tocapture domain adaptation. The third semantics
is context ID: this context ID is used to adjust thecontext for the local words. Context ID is used ina continuous-space LM (Schwenk, 2007), but is
implicit since the context does not appear in theconstruction of a continuous-space LM. Note that
our usage of the term semantics refers to meaning
constructed by a sentence or words. The QE
score works as a sentence level switch to select
HPBSMT or PBSMT, based on the semantics
of a sentence. The genre ID gives an indicationthat the sentence is to be translated by genre IDsensitive MT systems, again based on semantics
on a sentence level. The context-dependent LM
can be interpreted as supplying the local context
to a word, capturing semantics on a word level.
The architecture presented in this paper is substantially different from multi-engine system combination. Although the system has multiple paths,
only one path is chosen at decoding when processing unseen data. Note that standard multi-engine
system combination using these three semantics
has been presented before (Okita et al., 2012b;
Okita et al., 2012a; Okita, 2012). This paper also
compares the two approaches.
The remainder of this paper is organized as follows. Section 2 describes the motivation for ourapproach. In Section 3, we describe our proposedsystems, while in Section 4 we describe the experimental results. We conclude in Section 5.

2 Motivation
Model Difference of PBSMT and HPBSMTOur motivation is identical with a system combination strategy which would obtain a better translation if we can access more than two translations.
Even though we are limited in the type of MT sys-

tems, i.e. SMT systems, we can access at least
two systems, i.e. PBSMT and HPBSMT systems.
The merit that accrues from accessing these twotranslation is shown in Figure 1. In this example between EN-ES, the skirts of the distribution
shows that around 20% of the examples obtain thesame BLEU score, 37% are better under PBSMT,
and 42% under HPBSMT. Moreover, around 10%of sentences show difference of 10 BLEU points.
Even a selection of outputs would improve the results. Unfortunately, some pitfall of system combination (Rosti et al., 2007) impact on the process
when the number of available translation is only
two. If there are only two inputs, (1) the mismatchof word order and word selection would yield abad combination since system combination relies
on monolingual word alignment (or TER-basedalignment) which seeks identical words, and (2)Minimum Bayes Risk (MBR) decoding, which isa first step, will not work effectively since it relies on voting. (In fact, only selecting one of thetranslation outputs is even effective: this method
is called system combination as well (Specia et al.,2010).) Hence, although the aim is similar, we do
not use a system combination strategy, but we develop a semantically-informed SMT system.

Figure 1: Figure shows the difference of sentencebased performance between PBSMT and HPBSMT systems.
Relation of Complexity of Source Sentence andPerformance of HPBSMT and PBSMT It isinteresting to note that PBSMT tends to be better than HPBSMT for European language pairs
as the recent WMT workshop shows, while HPBSMT shows often better performance for distantlanguage pairs such as EN-JP (Okita et al., 2010b)

and EN-ZH in other workshops.
Under the assumption that we use the same
training corpus for training PBSMT and HPBSMT
systems, our hypothesis is that we may be able
to predict the quality of translation. Note that although this is the analogy of quality estimation,
the setting is slightly different in that in test phase,
we will not be given a translation output, but only
a source sentence. Our aim is to predict whether
HPBSMT obtains better translation output than
PBSMT or not. Hence, our aim does not requirethat the quality prediction here is very accuratecompared to the standard quality estimation task.
We use a feature set consisting of various characteristics of input sentences.

3 Our Methods: Shallow Semantics
Our system accommodates PBSMT and HPBSMT
with multiple of LMs. A decoder which handles
shallow semantic information is shown in Table
3.1.
3.1 QE Score
Quality estimation aims to predict the quality oftranslation outputs for unseen data (e.g. by building a regressor or a classifier) without access toreferences: the inputs are translation outputs andsource sentences in a test phase, while in a training
phase the corresponding BLEU or HTER scores
are used. In this subsection, we try to build a regressor with the similar settings but without supplying the translation outputs. That is, we supply
only the input sentences. (Since our method is nota quality estimation for a given translation output,
quality estimation may not be an entirely appropriate term. However, we borrow this term for thispaper.) If we can build such a regressor for PBSMT and HPBSMT systems, we would be able
to select a better translation output without actually translating them for a given input sentence.
Note that we translate the training set by PBSMTand HPBSMT in a training phase only to supply
their BLEU scores to a regressor (since a regressor is a supervised learning method). Then, we
use these regressors for a given unseen source sentence (which has no translation output attached) topredict their BLEU scores for PBSMT and HPBSMT.
Our motivation came from the comparison ofa sequential learning system and a parser-based
system. The typical decoder of the former is a

Input: Foreign sent f=f1 ,...,f1f , language model,translation model, rule table.
Output: English translation e
ceScore = predictQEScore(fi )if (ceScore == HPBSMTBetter)for span length l=1 to 1f do
for start=0..1f -1 dogenreID = predictGenreID(fi )end = start + 1
forall seq s of entries and words in span[start,end] doforall rules r do
if rule r applies to chart seq s thencreate new chart entry cwith LM(genreID)
add chart entry c to chart
return e from best chart entry in span [0,1f ]else:
genreID = predictGenreID(fi )place empty hypothesis into stack 0
for all stacks 0...n-1 dofor all hypotheses in stack do
for all translation options do
if applicable thencreate new hyp with LM(ID)place in stackrecombine with existing hyp if
possibleprune stack if too big
return e
predictQEScore()predictGenreID()predictContextID(wordi , wordi−1 )Table 1: Decoding algorithm: the main algorithmof PBSMT and HPBSMT are from (Koehn, 2010).
The modification is related to predictQEScore(),predictGenreID(), and predictContextID().
ample, the (context-free) grammar-based system
has the capability of handling various difficul-

ties caused by inserted clauses, coordination, long
Multiword Expressions, and parentheses, while
the sequential learning system does not (This issince this is what the aim of the context-freegrammar-based system is.) These difficulties are
manifest in input sentences.
1.0

true BLEU difference of PBSMT and HPBSMT
predicted BLEU difference of PBSMT and HPBSMT

0.5
difference of BLEU points

Viterbi decoder while that of the latter is a CockeYounger-Kasami (CYK) decoder (Younger, 1967).
The capability of these two systems provides
an intuition about the difference of PBSMT and
HPBSMT: the CYK decoder-based system hassome capability to handle syntactic constructions
while the Viterbi decoder-based system has only
the capability of learning a sequence. For ex-

0.0

−0.5

−1.0
0

50

100

150
sample ID

200

250

300

Figure 2: A blue line shows the true BLEU difference between PBSMT and HPBSMT (y-axis)
where x-axis is the sample IDs reordered in descending order (blue), while green dots show the
BLEU absolute difference (y-axis) of the typicalsamples where x-axis is shared with the above.
This example is sampled 300 points from newstest2013 (ES-EN). Even if the regressor does notachieve a good performance, the bottom line of theoverall performance is already really high in thistricky problem. Roughly, even if we plot randomly
we could achieve around 80 - 90% of correctness.
Around 50% of samples (middle of the curve) do
not care (since the true performance of PBSMTand HPBSMT are even), there is a slope in the left
side of the curve where random plot around thiscurve would achieve 15 - 20% among 25% of correctness (the performance of PBSMT is superior),
and there is another slope in the right side of thecurve where random plot would achieve again 15- 20% among 25% (the performance of HPBSMT
is superior). In this case, accuracy is 86%.
If we assume that this is one major difference
between these two systems, the complexity of the
input sentence will correlate with the difference oftranslation quality of these two systems. In thissubsection, we assume that this is one major difference of these two systems and that the complexity of the input sentence will correlate with the difference of translation quality of these two systems.
Based on these assumptions, we build a regressor

for each system for a given input sentence where ina training phase we supply the BLEU score measured using the training set. One remark is that the
BLEU score which we predict is only meaningful in a relative manner since we actually generatea translation output in preparation phase (there isa dependency to the mean of BLEU score in the
training set). Nevertheless, this is still meaningful
as a relative value if we want to talk about theirdifference, which is what we want in our settings
to predict which system, either PBSMT or HPBSMT, will generate a better output.
The main features used for training the regressor are as follows: (1) number of / length of inserted clause / coordination / multiword expressions, (2) number of long phrases (connection by‘of’; ordering of words), (3) number of OOVwords (which let it lower the prediction quality),(4) number of / length of parenthesis, etc. We obtained these features using parser (de Marneffe etal., 2006) and multiword extractor (Okita et al.,2010a).
3.2 Genre ID
Genre IDs allow us to apply domain adaptation
technique according to the genre ID of the testset.
Among various methods of domain adaptation, weinvestigate unsupervised clustering rather than already specified genres.
We used (unsupervised) classification via Latent Dirichlet Allocation (LDA) (Blei et al., 2003)
to obtain genre ID. LDA represents topics asmultinomial distributions over the W uniqueword-types in the corpus and represents documents as a mixture of topics.
Let C be the number of unique labels in the
corpus. Each label c is represented by a W dimensional multinomial distribution φc over the
vocabulary. For document d, we observe both thewords in the document w(d) as well as the document labels c(d) . Given the distribution over topics θd , the generation of words in the document is
captured by the following generative model.
1. For each label c ∈ {1, . . . C}, sample a distribution over word-types φc ∼ Dirichlet(·|β)
2. For each document d ∈ {1, . . . , D}(a) Sample a distribution over its observedlabels θd ∼ Dirichlet(·|α)(b) For each word i ∈ {1, . . . , NdW }i. Sample a label
Multinomial(θd )

(d)

∼

(d)

∼
la-

zi

ii. Sample a word wi
Multinomial(φc ) from the(d)bel c = ziUsing topic modeling (or LDA) as describedabove, we perform the in-domain data partitioning
as follows, building LMs for each class, and running a decoding process for the development set,
which will obtain the best weights for cluster i.
1. Fix the number of clusters C, we explore values from small to big.1
2. Do unsupervised document classification (orLDA) on the source side of the training, development and test sets.
3. Separate each class of training sets and build
LM for each cluster i (1 ≤ i ≤ C).
4. Separate each class of development set (keepthe original index and new index in the allocated separated dataset).
5. (Using the same class of development set):
Run the decoder on each class to obtain the
n-best lists, run a MERT process to obtain thebest weights based on the n-best lists, (Repeat
the decoding / MERT process several iterations. Then, we obtain the best weights for aparticular class.)
For the test phase,
1. Separate each class of the test set (keep theoriginal index and new index in the allocatedseparated dataset).
2. Suppose the test sentence belongs to cluster
i, run the decoder of cluster i.
3. Repeat the previous step until all the test sentences are decoded.
3.3 Context ID
Context ID semantics is used through the reranking of the n-best list in a MERT process(Schwenk, 2007; Schwenk et al., 2012; Le et al.,2012). 2-layer ngram-HMM LM is a two layer
version of the 1-layer ngram-HMM LM (Blunsom and Cohn, 2011) which is a nonparametric1
Currently, we do not have a definite recommendation on
this. It needs to be studied more deeply.

Bayesian method using hierarchical Pitman-Yor
prior. In the 2-layer LM, the hidden sequence ofthe first layer becomes the input to the higher layerof inputs. Note that such an architecture comesfrom the Restricted Boltzmann Machine (Smolensky, 1986) accumulating in multiple layers in order to build deep belief networks (Taylor and Hinton, 2009). Although a 2-layer ngram-HMM LM
is inferior in its performance compared with othertwo LMs, the runtime cost is cheaper than these.
ht denotes the hidden word for the first layer, h̄tdenotes the hidden word for the second layer, wi
denotes the word in output layer. The generativemodel for this is shown below.
(1)
ht |h̄t ∼ F (φ̄st )wt |ht ∼ F (φst )(2)

wi |w1:i−1 ∼ PY(di , θi , Gi )(3)

where α is a concentration parameter, θ is astrength parameter, and Gi is a base measure.
Note that these terms belong to the hierarchicalPitman-Yor language model (Teh, 2006). We useda blocked inference for inference. The performance of 2-layer LM is shown in Table 3.

4

Experimental Settings

We used Moses (Koehn et al., 2007) for PBSMTand HPBSMT systems in our experiments. The
GIZA++ implementation (Och and Ney, 2003) of
IBM Model 4 is used as the baseline for wordalignment: Model 4 is incrementally trained by
performing 5 iterations of Model 1, 5 iterationsof HMM, 3 iterations of Model 3, and 3 iterations of Model 4. For phrase extraction thegrow-diag-final heuristics described in (Koehn etal., 2003) is used to derive the refined alignment
from bidirectional alignments. We then perform
MERT process (Och, 2003) which optimizes the
BLEU metric, while a 5-gram language model isderived with Kneser-Ney smoothing (Kneser and
Ney, 1995) trained with SRILM (Stolcke, 2002).
For the HPBSMT system, the chart-based decoderof Moses (Koehn et al., 2007) is used. Most of theprocedures are identical with the PBSMT systems
except the rule extraction process (Chiang, 2005).
The procedures to handle three kinds of semantics are implemented using the already mentioned algorithm. We use libSVM (Chang and Lin,2011), and Mallet (McCallum, 2002) for Latent
Dirichlet Allocation (LDA) (Blei et al., 2003).
For the corpus, we used all the resources provided for the translation task at WMT13 for lan-discrete representationNP

projection layer

continuous−space language
model [Schwenk, 2007]

hidden layerN
output layer
probability estimationneural network

2nd hidden layer
2nd RBM

1st hidden layer
1st RBMoutput layer
ngram language model

2−layer ngram−HMM language model

output layer
ngram language model

2−layer conditional RBM language model

Figure 3: Figure shows the three kinds of contextdependent LM. The upper-side shows continuousspace language model (Schwenk, 2007). The
lower-left shows ours, i.e. the 2-layer ngramHMM LM. The lower-right shows the 2-layer conditional Restricted Boltzmann Machine LM (Taylor and Hinton, 2009).
guage model, that is parallel corpora (Europarl
V7 (Koehn, 2005), Common Crawl corpus, UN
corpus, and News Commentary) and monolingual
corpora (Europarl V7, News Commentary, andNews Crawl from 2007 to 2012).
Experimental results are shown in Table 2.
The left-most column (sem-inform) shows our results. The sem-inform made a improvement of 0.8
BLEU points absolute compared to the PBSMT
results in EN-ES, while the standard system combination lost 0.1 BLEU points absolute comparedto the single worst. For ES-EN, the sem-inform
made an improvement of 0.7 BLEU points absolute compared to the PBSMT results. These improvements over both of PBSMT and HPBSMT
are statistically significant by a paired bootstrap
test (Koehn, 2004).

5 Conclusion
This paper describes shallow semanticallyinformed HPBSMT and PBSMT systems developed at Dublin City University for participation in
the translation task at the Workshop on Statistical
Machine Translation (WMT 13). Our system has

EN-ES
BLEU
BLEU(11b)
BLEU-cased
BLEU-cased(11b)
NIST
Meteor
WER
PER
ES-EN
BLEU
BLEU(11b)
BLEU-cased
BLEU-cased(11b)
NIST
MeteorWER
PER

sem-inform
30.3
30.3
29.0
29.0
7.91
0.580
53.7
41.3
sem-inform
31.1
31.1
29.7
29.7
7.87
0.615
54.8
41.3

PBSMT
29.5
29.5
28.4
28.4
7.74
0.579
55.4
42.4
PBSMT
30.4
30.4
29.1
29.1
7.79
0.612
55.4
41.8

HPBSMT
28.2
28.2
27.1
27.1
7.35
0.577
59.3
46.0
HPBSMT
23.1∗
23.1∗
22.3∗
22.3∗
6.67∗
0.533∗
62.5∗
48.3∗

syscomb
28.1
28.1
27.0
27.0
7.35
0.577
59.2
45.8
syscomb
28.8
28.8
27.9
27.9
7.40
0.612
59.3
45.8

aug-syscomb
28.5
28.5
27.5
27.5
7.36
0.578
58.9
45.5
aug-syscomb
29.9
29.9
28.8
28.8
7.71
0.613
56.1
41.9

Table 2: Table shows the score where “sem-inform” shows our system. Underlined figure shows theofficial score. “syscomb” denotes the confusion-network-based system combination using BLEU, while“aug-syscomb” uses three shallow semantics described in QE score (Okita et al., 2012a), genre ID (Okita
et al., 2012b), and context ID (Okita, 2012). Note that the inputs for syscomb and aug-syscomb are theoutput of HPBSMT and PBSMT. HPBSMT from ES to EN has marked with ∗ , which indicates that thisis trained only with Europarl V7.

EN
newstest12
newstest11
newstest10
newstest09

2-layer ngramHMM LM
130.4
146.2
156.4
176.3

SRILM
140.3
157.1
166.8
187.1

Table 3: Table shows the perplexity of contextdependent language models, which is 2-layer
ngram HMM LM, and that of SRILM (Stolcke,
2002) in terms of newstest09 to 12.
PBSMT and HPBSMT decoders with multipleLMs, but our system will execute only one path,
which is different from multi-engine system
combination. We consider investigate three typesof shallow semantic information: (i) a Quality
Estimate (QE) score, (ii) genre ID, and (iii) a
context ID through context-dependent language
models. Our experimental results show that the
improvement is 0.8 points absolute (BLEU) for
EN-ES and 0.7 points for ES-EN compared tothe standard PBSMT system (single best system).
We developed this method when the standard(confusion network-based) system combination isineffective such as in the case when the input isonly two.
A further avenue would be the investigation ofother semantics such as linguistic semantics, including co-reference resolution or anaphora resolution, hyper-graph decoding, and text understanding. Some of which are investigated in the context
of textual entailment task (Okita, 2013b) and wewould like to extend this to SMT task. Anotherinvestigation would be the integration of genre ID
into the context-dependent LM. The preliminary
work shows that such integration would decrease
the overall perplexity (Okita, 2013a).

Acknowledgments
We thank Antonio Toral and Santiago Cortés
Vaı́lo for providing parts of their processing
data. This research is supported by the Science
Foundation Ireland (Grant 07/CE/I1142) as partof the Centre for Next Generation Localisation(http://www.cngl.ie) at Dublin City University.

References
David Blei, Andrew Y. Ng, and Michael I. Jordan.
2003. Latent dirichlet allocation. Journal of Machine Learning Research, 3:9931022.
Phil Blunsom and Trevor Cohn. 2011. A hierarchicalpitman-yor process hmm for unsupervised part ofspeech induction. In Proceedings of Annual Meeting of the Association for Computational Linguistics(ACL11), pages 865–874.
Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM
Transactions on Intelligent Systems and Technology,
2:27:1–27:27.
David Chiang. 2005. A hierarchical phrase-basedmodel for statistical machine translation. In Proceedings of the 43th Annual Meeting of the Association for Computational Linguistics (ACL-05), pages
263–270.
Marie-Catherine de Marneffe, Bill MacCartney, andChristopher D. Manning. 2006. Generating typed
dependency parses from phrase structure parses. In
Proceedings of the Sixth International Conferenceon Language Resources and Evaluation (LREC2006), pages 449–454.
Reinhard Kneser and Hermann Ney. 1995. Improved backing-off for n-gram language modeling.
In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing,pages 181–184.
Philipp Koehn, Franz Och, and Daniel Marcu. 2003.
Statistical phrase-based translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computationa Linguistics (HLT / NAACL
2003), pages 115–124.
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris
Callison-Burch, Marcello Federico, Nicola Bertoldi,
Brooke Cowan, Wade Shen, Christine Moran,
Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra
Constantin, and Evan Herbst. 2007. Moses: Open
source toolkit for Statistical Machine Translation. In
Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion
Volume Proceedings of the Demo and Poster Sessions, pages 177–180.
Philipp Koehn. 2004. Statistical significance tests formachine translation evaluation. In Proceedings ofConference on Empirical Methods in Natural Language Processing (EMNLP 2004), pages 388–395.
Philipp Koehn. 2005. Europarl: A parallel corpus forstatistical machine translation. In Proceedings of theMachine Translation Summit, pages 79–86.
Philipp Koehn. 2010. Statistical machine translation.
Cambridge University Press.

Hai-Son Le, Thomas Lavergne, Alexandre Allauzen,
Marianna Apidianaki, Li Gong, Aurelien Max,
Artem Sokolov, Guillaume Wisniewski, and Francois Yvon. 2012. Limsi at wmt12. In Proceedings of the Seventh Workshop on Statistical MachineTranslation, pages 330–337.
Andrew Kachites McCallum.
2002.
Mallet: A machine learning for language toolkit.
http://mallet.cs.umass.edu.
Franz Och and Hermann Ney. 2003. A systematiccomparison of various statistical alignment models.
Computational Linguistics, 29(1):19–51.
Franz Och. 2003. Minimum Error Rate Training in
Statistical Machine Translation. In Proceedings ofthe 41st Annual Meeting of the Association for Computational Linguistics, pages 160–167.
Tsuyoshi Okita, Alfredo Maldonado Guerra, Yvette
Graham, and Andy Way. 2010a. Multi-Word Expression sensitive word alignment. In Proceedings of the Fourth International Workshop On Cross
Lingual Information Access (CLIA2010, collocatedwith COLING2010), Beijing, China., pages 1–8.
Tsuyoshi Okita, Jie Jiang, Rejwanul Haque, Hala AlMaghout, Jinhua Du, Sudip Kumar Naskar, andAndy Way. 2010b. MaTrEx: the DCU MT System
for NTCIR-8. In Proceedings of the MII Test Collection for IR Systems-8 Meeting (NTCIR-8), pages
377–383.
Tsuyoshi Okita, Raphaël Rubino, and Josef van Genabith.
2012a.
Sentence-level quality estimation for mt system combination. In Proceedingsof ML4HMT Workshop (collocated with COLING
2012), pages 55–64.
Tsuyoshi Okita, Antonio Toral, and Josef van Genabith. 2012b. Topic modeling-based domain adaptation for system combination. In Proceedingsof ML4HMT Workshop (collocated with COLING
2012), pages 45–54.
Tsuyoshi Okita. 2012. Neural Probabilistic Language
Model for System Combination. In Proceedingsof ML4HMT Workshop (collocated with COLING
2012), pages 65–76.
Tsuyoshi Okita. 2013a. Joint space neural probabilistic language model for statistical machine translation. Technical Report at arXiv, 1301(3614).
Tsuyoshi Okita. 2013b. Local graph matching withactive learning for recognizing inference in text at
ntcir-10. NTCIR 10 Conference, pages 499–506.
Antti-Veikko I. Rosti, Spyros Matsoukas, and Richard
Schwartz. 2007. Improved word-level system combination for machine translation. In Proceedings ofthe 45th Annual Meeting of the Association for Computational Linguistics, pages 312–319.

Holger Schwenk, Anthony Rousseau, and Mohammed
Attik. 2012. Large, pruned or continuous space language models on a gpu for statistical machine translation. In NAACL-HLT workshop on the Future of
Language Modeling for HLT, pages 11–19.
Holger Schwenk. 2007. Continuous space language
models. Computer Speech and Language, 21:492–
518.
Paul Smolensky. 1986. Chapter 6: Information processing in dynamical systems: Foundations of harmony theory. In Rumelhart, David E.; McLelland, James L. Parallel Distributed Processing:
Explorations in the Microstructure of Cognition,
1:194281.
Lucia Specia, D. Raj, and Marco Turchi. 2010. Machine translation evaluation versus quality estimation. Machine Translation, Springer, 24(1):39–50.
Andreas Stolcke. 2002. SRILM – An extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language Processing, pages 901–904.
Graham Taylor and Geoffrey Hinton. 2009. Factoredconditional restricted boltzmann machines for modeling motion style. In Proceedings of the 26th International Conference on Machine Learning (ICML),
pages 1025–1032.
Yee Whye Teh. 2006. A hierarchical bayesian language model based on pitman-yor processes. In
Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics (ACL-06),
Prague, Czech Republic, pages 985–992.
Daniel H. Younger. 1967. Recognition and parsing ofcontext-free languages in time n3. Information andControl, 10(2):189208.