Question: In the work of Andrzejewski et al., what did they incorporate knowledge by?
Evidence: Topic modeling approaches are unsupervised statistical algorithms that usually considers each document as a "bag of words". There were several attempts to enrich word-based topic models (=unigram topic models) with additional prior knowledge or multiword expressions.
Andrzejewski et al. BIBREF5 incorporated knowledge by Must-Link and Cannot-Link primitives represented by a Dirichlet Forest prior. These primitives were then used in BIBREF6 , where similar words are encouraged to have similar topic distributions. However, all such methods incorporate knowledge in a hard and topic-independent way, which is a simplification since two words that are similar in one topic are not necessarily of equal importance for another topic.
Xie et al. BIBREF7 proposed a Markov Random Field regularized LDA model (MRF-LDA), which utilizes the external knowledge to improve the coherence of topic modeling. Within a document, if two words are labeled as similar according to the external knowledge, their latent topic nodes are connected by an undirected edge and a binary potential function is defined to encourage them to share the same topic label. Distributional similarity of words is calculated beforehand on a large text corpus.
In BIBREF8 , the authors gather so-called lexical relation sets (LR-sets) for word senses described in WordNet. The LR-sets include synonyms, antonyms and adjective-attribute related words. To adapt LR-sets to a specific domain corpus and to remove inappropriate lexical relations, the correlation matrix for word pairs in each LR-set is calculated. This matrix at the first step is used for filtrating inappropriate senses, then it is used to modify the initial LDA topic model according to the generalized Polya urn model described in BIBREF9 . The generalized Polya urn model boosts probabilities of related words in word-topic distributions.
Gao and Wen BIBREF10 presented Semantic Similarity-Enhanced Topic Model that accounts for corpus-specific word co-occurrence and word semantic similarity calculated on WordNet paths between corresponding synsets using the generalized Polya urn model. They apply their topic model for categorizing short texts.
All above-mentioned approaches on adding knowledge to topic models are limited to single words. Approaches using ngrams in topic models can be subdivided into two groups. The first group of methods tries to create a unified probabilistic model accounting unigrams and phrases. Bigram-based approaches include the Bigram Topic Model BIBREF11 and LDA Collocation Model BIBREF12 . In BIBREF13 the Topical N-Gram Model was proposed to allow the generation of ngrams based on the context. However, all these models are enough complex and hard to compute on real datasets.
The second group of methods is based on preliminary extraction of ngrams and their further use in topics generation. Initial studies of this approach used only bigrams BIBREF14 , BIBREF15 . Nokel and Loukachevitch BIBREF16 proposed the LDA-SIM algorithm, which integrates top-ranked ngrams and terms of information-retrieval thesauri into topic models (thesaurus relations were not utilized). They create similarity sets of expressions having the same word components and sum up frequencies of similarity set members if they co-occur in the same text.
In this paper we describe the approach to integrate whole manual thesauri into topic models together with multiword expressions.
Andrzejewski et al. [6] incorporated knowledge by Must-Link and Cannot-Link primitives represented by a Dirichlet Forest prior. These primitives were then used in [7], where similar words are encouraged to have similar topic distributions.
Answer: By Must-Link and Cannot-Link primitives represented by a Dirichlet Forest prior.