Venue: ACL 2020

# Bayesian Hierarchical Words Representation Learning

Oren Barkan1*, Idan Rejwan12*, Avi Caciularu13* and Noam Koenigstein14

###### Abstract

This paper presents the Bayesian hierarchical Words Representation (BHWR) learning algorithm. BHWR facilitates Variational Bayes word representation learning combined with semantic taxonomy modeling via hierarchical priors. By propagating relevant information between related words, BHWR utilizes the taxonomy to improve the quality of such representations. Evaluation of several linguistic datasets demonstrates the advantages of BHWR over suitable alternatives that facilitate Bayesian modeling with or without semantic priors. Finally, we further show that BHWR produces better representations for rare words.

+
Footnote †: Equal contribution.

_Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 3871–3877

July 5 - 10, 2020. ©2020 Association for Computational Linguistics

## 1 Introduction

In the last decade, a plethora of methods were proposed for learning vector representations for words Mikolov et al. (2013); Pennington et al. (2014); Barkan (2017), sentences Lin et al (2017); Barkan et al. (2020a), items Barkan et al. (2016); Barkan et al. (2019); Barkan et al. (2020b); Barkan et al. (2020c), and medical concepts Luo et al. (2019). In the domain of natural language understanding, neural word embedding models are designed to learn distributed word representations as vectors in a latent space. In this space, arithmetic operations between the word vectors encode semantic and syntactic information. Specifically, the seminal works by Mikolov et al. (2013); Pennington et al. (2014); Bojanowski et al. (2017), exhibited state-of-the-art performance on various linguistic tasks Finkelstein et al. (2001); Luong et al. (2013); Rogers et al. (2018).

The major focus of most previous models was on optimizing the utilization of co-occurrence relations for learning representations, e.g., learning the probability of word \(x\) to appear in the vicinity of word \(y\) . Yet, often, additional side information can be leveraged for learning finer embeddings. In this work, we focus on incorporating word semantic taxonomy, which is particularly useful for learning representations of rare words and for learning word representations from a small-size corpus.

To this end, we introduce the Bayesian Hierarchical Words Representation (BHWR) learning algorithm. BHWR presents two complementary properties: Bayesian modeling of word representations aside with hierarchical priors that naturally support semantic taxonomy. BHWR is based on a Variational Bayes (VB) optimization that enables the mapping of words into probability densities in the latent space.

A key advantage of BHWR is the utilization of word taxonomy for the propagation of relevant information between related words. For example, consider the words 'anode' and 'cathode'. Both words have a common relationship to the word

'electrode' which appears hierarchically above them in the taxonomy knowledge base. Assume the word 'cathode' frequently appears in the corpus, while the words 'anode' and 'electrode' do not appear in the corpus or occur very infrequently. A model that relies solely on co-occurrence relations will fail to infer the semantic proximity between 'cathode' and 'anode'. However, a model that utilizes word taxonomy will learn a representation for the parent word 'electrode' based on its child 'cathode'. Moreover, the parent word 'electrode' will serve as an informative prior for 'anode', and the representation of 'anode' will fall back to its prior 'electrode'. Finally, if more occurrences of 'anode' will be added to the training dataset, its representation can smoothly transition away from its prior position in accordance with the co-occurrences patterns in the data.

Besides the semantic information added to the word representations via the hierarchical prior, the Bayesian modeling by itself helps deal with the problem of rare words. These words suffer from insufficient statistics, and their respective embeddings are quite sensitive to noise. This problem becomes acute in the case of point estimate solutions that do not model uncertainty. In contrast, Bayesian solutions learn the entire posterior density and hence are more robust to overfitting Bishop (2006).

We train BHWR on a small annotated corpus Miller et al. (1993) and evaluate its overall performance as well as the improvement on rare words. Our findings show that BHWR outperforms other non-contextualized word embedding methods that facilitate either Bayesian modeling or semantic taxonomy.

## 2 Related Work

Incorporating lexical-semantic information in learning word embeddings has been suggested in the past. In Faruqui et al. (2015), a post-processing technique was introduced in order to refine pre-trained word representations using relational information from semantic lexicons. In Li et al. (2016), hierarchical taxonomy was utilized for improving document categorization and concept clustering. Recently, linguistic knowledge bases were utilized for enhancing _contextualized_ word embeddings Huang et al. (2019); Levine et al. (2019).

The abovementioned works finetune pre-trained representations, while injecting external contextual information for words (some w.r.t. a specific task). Unlike these works, BHWR facilitates Bayesian learning of non-contextualized word embeddings, in combination with hierarchical taxonomy information which is not task-specific. Hence, a direct comparison between BHWR and these works is unfitting.

More relevant to our work is the Bayesian Skip-Gram (BSG) model from Barkan et al. (2017). However, BSG does not allow the use of external information such as word taxonomy. Hence, in our experiments, we compared BHWR to BSG and further apply the method from Faruqui et al. (2015) to enhance BSG with word taxonomy information.

## 3 Bayesian Hierarchical Words Representation

In this section, we describe the model and derive a VB solution that is finally translated to the Bayesian Hierarchical Words Representation (BHWR) learning algorithm.

### Model

Let \(W=\{w_{i}\}^{NW}_{i=1}\) and \(\mathcal{I}=\{i\}^{Nw}_{i=1}\) , be a vocabulary and a corresponding index set, respectively. We define \(\pi\colon\mathcal{I}\to\mathbb{P}(\mathcal{I})\) and \(\omega\colon\mathcal{I}\to\mathbb{P}(\mathcal{I})\) (\(\mathbb{P}(\mathcal{I})\) is the power set of \(\mathcal{I}\)) s.t. \(\pi_{i}\triangleq\pi[i]\) and \(\omega_{i}\triangleq\omega[i]\) are the sets of parents and children word indices for the word index \(i\) , respectively. This forms a hierarchical structure (network) in which a word can appear either as a leaf or as an internal node (parent).

Let \(u_{i},v_{i},h^{u}_{i},h^{v}_{i}\in\mathbb{R}^{k}\) be the context and target leaf and parent representations of the word \(w_{i}\)

Figure 1: A graphical model of BHWR.

 

respectively. For example, if \(n\in\pi[i]\), then we use \(h_{n}^{u}\) as the parent node for both \(u_{i}\) and \(h_{i}^{u_{i}}\). In addition, we define \(s_{i}^{u}\triangleq|\tau_{i}|^{-1}\sum_{n\in\pi_{t}}h_{n}^{u}\) if \(\tau_{i}\neq\emptyset\), otherwise \(s_{i}^{u}\triangleq\vec{0}\) , where \(s_{i}^{v}\) is defined in the same manner.

Let \(T=(w_{t_{i}})_{i=1}^{L}\) be a text corpus and let \(c_{max}\in\mathbb{N}\) be the context window parameter. We iterate over \(T\) and for each word \(w_{t_{i}}\), we sample a random window size \(c\in\{1,...,c_{max}\}\) to form a multiset of positive examples \(I_{p}^{i}=\{(t_{i},j)|j\in\{t_{l-c},...,t_{i+c}\}\backslash\{t_{i}\}\}\) and a corresponding multiscale of negative examples \(I_{N}^{i}=\big{\{}(t_{i},n_{j})\big{\}}_{j=1}^{|I_{p}^{i}|}\), where \(n_{j}\) is sampled according to the unigram distribution raised to the power of \(0.75\). Then, we define \(I_{P}\triangleq U_{i=1}^{L}I_{p}^{i}\) and \(I_{N}\triangleq U_{i=1}^{L}I_{N}^{i}\) as the positive and negative sets and further define \(I_{D}\triangleq I_{P}\cup I_{N}\).

Let \(d:J\times J\rightarrow\{1,-1\}\) with \(d_{ij}=1\) if \((i,j)\in I_{P}\) and \(-1\) otherwise, and let \(\mathcal{D}=\{d_{ij}|(i,j)\in I_{D}\}\). We model the likelihood of \(d_{ij}\) given the model parameters as \(p\big{(}d_{ij}|u_{i},v_{j}\big{)}=\sigma(d_{ij}u_{i}^{T}v_{j})\), where \(\sigma(a)\triangleq\frac{1}{1+e^{-a}}\). We further assume normal hierarchical priors as follows:

\[p(H^{u}|\tau_{h^{u}})=\prod_{i\in J}\mathcal{N}(h_{i}^{u};s_{i}^ {u},\tau_{h^{u}}^{-1}I),\] \[p(U|H^{u},\tau_{u})=\prod_{i\in J}\mathcal{N}(u_{i};s_{i}^{u}, \tau_{u}^{-1}I),\]

where \(\tau_{h^{u}}\) and \(\tau_{u}\) are the precision hyperparameters. In the same manner, we assume normal hierarchical priors \(p(V|H^{v},\tau_{v})\) and \(p(H^{v}|\tau_{h^{v}})\). Then, the joint density of \(\mathcal{D}\) and the model parameters \(\theta=\{U,V,H^{u},H^{v}\}\) given the precision hyperparameters \(\mathcal{T}=\{\tau_{u},\tau_{v},\tau_{h^{u}},\tau_{h^{v}}\}\) is given by

\[p(\mathcal{D},\theta|\,\mathcal{T})=p(\mathcal{D}|\theta)p(\theta|\, \mathcal{T}), \tag{1}\]

with

\[p(\mathcal{D}|\theta)=p(\mathcal{D}|U,V)=\prod_{(i,j)\in I_{D}}\sigma\big{(} d_{ij}u_{i}^{T}v_{j}\big{)},\]

and

\[p(\theta|\,\mathcal{T})=\\ p(U|H^{u},\tau_{u})p(H^{u}|\tau_{h^{u}})p(V|H^{v},\tau_{v})p(H^{v}| \tau_{h^{v}}).\]

Figure 1 presents a graphical model of BHWR.

Our goal is to compute posterior predictive distribution for an arbitrary \(d_{ij}^{*}\) given \(\mathcal{D}\) (which is not necessarily in \(\mathcal{D}\)). The probability of the words \(w_{i}\) and \(w_{j}\) to co-occur is given by

\[p\big{(}d_{ij}^{*}=1\big{|}\mathcal{D},\mathcal{T}\big{)}=\int\sigma(u_{i}^{ T}v_{j})p(\theta|\mathcal{D},\mathcal{T})d\theta. \tag{2}\]

### Posterior Approximation

Since the posterior \(p(\theta|\mathcal{D},\mathcal{T})\) in Eq. (2) is intractable, we turn to VB approximation Bishop (2006) of \(p(\theta|\mathcal{D},\mathcal{T})\) via a fully factorized distribution

\[q(\theta)=q(U)q(V)q(H^{u})q(H^{v})\]

\[\prod_{i\in J}q(u_{i})\prod_{i\in J}q(v_{i})\prod_{i\in J}q(h_{i}^{ u})\prod_{i\in J}q(h_{i}^{v}).\]

The posterior approximation \(q(\theta)\) is obtained via the minimization of the KL divergence from the true posterior, namely the minimization of \(D_{KL}(q(\theta)||p(\theta|\mathcal{D},\mathcal{T}))\) , which is equivalent Bishop (2006) to the maximization of (negative) variational free energy

\[\mathcal{L}(q)\triangleq\int q(\theta)\log\frac{p(\theta|\mathcal{D},\mathcal{ T})}{q(\theta)}\,d\theta.\]

\(\mathcal{L}(q)\) is maximized via an iterative procedure that is guaranteed to converge to a local optima (as the optimization is non-convex): At each iteration, we update each parameter \(z\in\theta\), in turn, according to the following update rule:

\[q^{*}(z)=\exp\bigl{(}\mathbb{E}_{q(\theta\setminus z)}[\log p(\theta,\mathcal{ D}|\mathcal{T})]+c\bigr{)}. \tag{3}\]

However, a straightforward application of Eq. (3) will run useless, as the term \(p(\theta,\mathcal{D}|\mathcal{T})\) includes the likelihood \(p(\mathcal{D}|\theta)\), which consists of sigmoid functions that are not conjugate to the normal prior \(p(\theta|\,\mathcal{T})\) from Eq. (1). Therefore, by introducing an additional variational parameter \(\xi_{ij}\), we can utilize the logistic bound from Jaakkola and Jordan (1996)

for lower bounding the log likelihood \(p(\mathcal{D}|\theta)\) with a squared exponential function as follows:

\[\begin{array}{ll}&\log p(\mathcal{D}|\theta)\geq\log p_{\xi}(\mathcal{D}| \theta)=\\ &\sum_{(i,j)\in I_{D}}\frac{d_{ij}(u_{i}^{T}v_{j}+b_{j})}{2}-\lambda\bigl{(} \xi_{ij}\bigr{)}\bigl{(}u_{i}^{T}v_{j}v_{j}^{T}u_{i}-\\ &&\xi_{ij}\bigr{)}+\log\sigma\bigl{(}\xi_{ij}\bigr{)}\end{array}\]

with \(\lambda(a)\triangleq\frac{1}{2a}\Bigl{(}\sigma(a)-\frac{1}{2}\Bigr{)}\).

Moreover, this bound is tight for

\[\xi_{ij}=\sqrt{\sum_{m=1}^{k}(\sigma_{u_{ik}}^{2}+\mu_{u_{ik}}^{2})(\sigma_{ v_{jk}}^{2}+\mu_{v_{jk}}^{2})}~{}. \tag{4}\]

\(p_{\xi}(\mathcal{D}|\theta)\) enables a conjugate relation with \(p(\theta|\,\mathcal{T})\) that results in normal density estimators \(q^{*}(z)\), \(z\in\theta\). Hence, for each \(z\in\theta\), we update the precision \(P_{z}\) and mean \(\mu_{z}\) (the sufficient statistics), following the update rule from Eq. (3) 

Specifically, for \(q(h_{i}^{u})\) and \(q(u_{i})\), the parameters updates are

\[P_{h_{i}^{u}}=\left(\tau_{h^{u}}+\sum_{m\in o_{i}}\frac{\tau_{u}+ \tau_{h^{u}}u}{|\pi_{m}|^{2}}\right)I, \tag{5}\] \[\mu_{h_{i}^{u}}=P_{h_{i}^{u}}^{-1}\Big{[}\Sigma_{m\in o_{i}}\Big{(} \frac{1}{|\pi_{m}|}\big{(}\tau_{u}\mu_{u_{m}}+\] \[\qquad\qquad\quad\tau_{h^{u}}\mu_{hh}^{u}\big{)}-\frac{\tau_{ u}+\tau_{h^{u}}}{|\pi_{m}|^{2}}\Sigma_{n\in\pi_{m}(\{i\}}\mu_{h^{u}}\big{)} \Big{]},\]

and

\[P_{u_{i}}=\tau_{u}l+2\sum_{j\in I_{ui_{i}}}\lambda(\xi_{ij})\mathbb{ E}_{q(\theta\setminus u_{i})}\big{[}v_{j}v_{j}^{T}\big{]}, \tag{6}\] \[\mu_{u_{i}}=\tfrac{1}{2}P_{u_{i}}^{-1}\Big{[}\sum_{j\in I_{ui_{i} }}d_{ij}\mu_{v_{j}}+\frac{\tau_{u}}{|\pi_{i}|}\sum_{m\in\pi_{i}}\mu_{hu}^{u} \Big{]},\]

respectively, where \(I_{ui}=\{j|(i,j)\in I_{D}\}\) and \(\mathbb{E}_{q(\theta\setminus u_{i})}\big{[}v_{j}v_{j}^{T}\big{]}=P_{u_{i}}^{- 1}+\mu_{v_{j}}\mu_{v_{j}}^{T}.\) The parameter updates for \(q(h_{i}^{v})\) and \(q(v_{i})\) are symmetric in the Eqs. (5) and (6), respectively.

### The BHWR Algorithm

The BHWR algorithm can be summarized as follows:

1. For each \(z\in\theta\) , sample \(\mu_{z}\sim N(0,I)\) and initialize \(P_{g}=I\).
2. Update \(\xi_{ij}\) using Eq. (4), update \(q(h_{i}^{u})\) and \(q(u_{i})\) using Eqs. (5) and (6), and update \(q(h_{i}^{v})\) and \(q(v_{i})\) using the symmetric versions of Eq. (5) and Eq. (6), respectively.
3. Repeat step 2 until convergence.

### Posterior Predictive Approximation

Finally, we approximate the integral from Eq. (2) by replacing the posterior with its factorized approximation

\[\int\sigma\big{(}u_{i}^{T}v_{j}\big{)}p(\theta|D, \mathcal{T})d\theta\approx\int\sigma\big{(}u_{i}^{T}v_{j}\big{)} q(\theta)d\theta\] \[\approx\int\sigma(x)N(x;\mu_{x},\sigma_{x}^{2})dx\] \[\approx\sigma\left(\mu_{x}/\sqrt{1+\pi\sigma_{x}^{2}/8}\right), \tag{7}\]

where \(x=u_{i}^{T}v_{j}\) and its density is approximated using normal density (using \(x\)'s first two moments under \(q\)). The final transition follows the logistic Gaussian integral approximation suggested by MacKay (1992).

In practice, the similarity score for a pair of words \(w_{t}\) and \(w_{j}\) was based on two different versions of Eq. (7): The first by assigning \(x=u_{i}^{T}u_{j}\) and the second with \(x=v_{t}^{T}v_{j}\). Then, the average of these two scores is taken as the final similarity score. Our experiments revealed that this technique yields better results.

## 4 Experimental Setup and Results

The experimentations in this section are focused on word similarity. Next, we present the training corpus, evaluated models, evaluation tasks, and the results.

### Training Corpus

We use SemCor Miller et al. (1993), which contains 37,176 annotated sentences with 820,411 words and a vocabulary size of 11,766 words. Each word's parent is taken to be its WordNet Miller et al. (1990) hypernym, e.g., for the words 'anode' and 'cathode', the parent word is 'electrode'.

### Models and Configurations

We compare Bayesian Hierarchical Words Representation **BHWR**) with the Skip-Gram with negative sampling **SG**) model from Mikolov et al. (2015) model, and the Bayesian Skip-Gram **BSG**) model from Barkan (2017). For each model, we consider two versions: The first uses the word representations produced by the model as is. In the second version, we further refine the learned word representations by applying the post-processing step from Farauqui et al. (2015). This enables the incorporation of word taxonomy information also to the SG and BSG methods. Overall, we consider six different model configurations; the post-processing versions of the modeled are marked with a '-P' suffix.

All models were trained till convergence. We used subsampling parameter Mikolov et al. (2013) of \(10^{-4}\) and a negative to positive ratio of \(1\). The precision hyperparameters were set to \(\tau_{u}=\tau_{v}=0.1\) and to \(\tau_{h^{u}}=\tau_{h^{v}}=0.001\). The embedding dimension was set to \(k=50\).

### Evaluation Tasks

The word similarity evaluation includes several different datasets: WordSim-353 **WS**Finkelstein et al. (2001), Stanford's Contextual Word Similarities **SCWS**Huang et al. (2012), Rare Words **RW**Luong et al. (2013), **MEN**Bruni et al. (2014) and SimLex-999 **SL**Hill et al. (2015). Note that these datasets are annotated by humans' similarities of words.

For BHWR and BSG, scoring a pair of words is done by using the posterior predictive approximation (Section 3.4). For SG, we compute \(\frac{1}{2}\Big{(}C\big{(}u_{i},u_{j}\big{)}+C\big{(}v_{i},v_{j}\big{)}\Big{)}\), where \(C\) is the cosine 

similarity function (recall SG is based on a point estimate solution).

Finally, for each combination of dataset and method, we report the Spearman rank correlation in terms of percentage.

### Results

Table 1 presents the results for all combinations of models and datasets. In the last column, we report for each model the average score across all datasets. The table is partitioned into two sections that present the regular and the post-processed versions of the models. For each dataset and section, the best and second-best scores are boldfaced and underlined, respectively. Next, we turn to discuss the main trends presented in Tab. 1.

First, we consider the regular model versions (first three rows). BHWR significantly outperforms BSG and SG across all datasets, and BSG comes second with a noticeable difference. This demonstrates the merit of the Bayesian treatment (BSG > SG) and the modeling of word taxonomy (BHWR > BSG).

Next, we turn to examine the post-processed versions (last three rows). We observe a significant boost to the results of all the models, which serves as an independent evaluation and reinforcement to the effectiveness of the post-processing method from Faruqui et al. (2015). BHWR-P again surpasses the other models by a large margin, while BSG-P and SG-P are on par. An interesting observation is that the post-processing method is found to be instrumental not only for SG and BSG but also for BHWR that utilizes word taxonomy inherently (BHWR < BHWR-P). This can be explained by the fact that the method of Faruqui et al. (2015) uses additional lexical information such as synonyms, which are not incorporated in BHWR. Yet, BHWR alone (without post-processing) still outperforms both BSG-P and SG-P. This result demonstrates the advantage of BHWR that facilitates learning of co-occurrences relations together word taxonomy, simultaneously.

Note that the results in Tab.1 are suboptimal when compared to Pennington et al. (2014): This is clearly related to the small corpus size used in this work. In the future, we plan to conduct an evaluation on larger corpora that are not necessarily annotated.

Finally, In order to demonstrate the strength of BHWR for words with only a few occurrences in the corpus, we further compare the models' performance on rare words. Table 2 shows the results on the word similarity tasks for words that occurred in the corpus five times or less. We observe that the gaps between BHWR and the other models become even more significant, either with or without the utilization of the post-processing from Faruqui et al. (2015).

## 5 Conclusion and Future Work

We presented BHWR - A word representation learning model, facilitating Bayesian learning of co-occurrences relations together with word taxonomy via hierarchical priors. When trained on a small corpus, BHWR exhibits a significant performance gain over other word embedding methods across various word similarity datasets. Importantly, a remarkable improvement is obtained for rare words. Moreover, BHWR outperforms all other baselines even when the latter are enhanced with the post-processing taxonomy refinement procedure from Faruqui et al. (2015). Finally, when combining BHWR with the post-processing from Faruqui et al. (2015), further improvement is observed.

In the future, we plan to extend the applicability of the presented model to other linguistics tasks as well as recommendations and medical inference tasks.

\begin{table}
\begin{tabular}{c|c c c c c|c} \hline
**Model** & **MEN** & **RW** & **SCWS** & **SL** & **WS** & **AVG** \\ \hline BHWR & **42.7** & **28.2** & **43.2** & **15.6** & **38.2** & **33.6** \\ BG & 35.0 & 27.6 & 39.6 & 13.2 & 28.6 & 28.8 \\ SG & 38.3 & 23.0 & 36.2 & 13.6 & 27.0 & 27.6 \\ \hline BHWR-P & **49.9** & **29.2** & **47.6** & **18.5** & **33.6** & **36.8** \\ BSG-P & 40.6 & 28.2 & 41.9 & 14.6 & 31.5 & 31.4 \\ SG-P & 41.2 & 27.7 & 41.0 & 14.7 & 31.2 & 31.2 \\ \hline \end{tabular}
\end{table}
Table 1: Word similarity evaluation.

\begin{table}
\begin{tabular}{c|c c c c c|c} \hline
**Model** & **MEN** & **RW** & **SCWS** & **SL** & **WS** & **AVG** \\ \hline BHWR & **37.7** & **25.3** & **31.7** & **9.9** & **42.9** & **29.5** \\ BSG & 27.9 & 24.9 & 27.4 & 9.4 & 34.4 & 24.8 \\ SG & 28.2 & 24.2 & 23.1 & 9.2 & 22.5 & 21.4 \\ \hline BHWR-P & **46.4** & **29.2** & **35.9** & **12.8** & **42.8** & **33.4** \\ BSG-P & 37.2 & 25.6 & 30.3 & 11.0 & 32.0 & 27.2 \\ SG-P & 37.0 & 25.4 & 30.6 & 11.2 & 30.8 & 27.0 \\ \hline \end{tabular}
\end{table}
Table 2: Word similarity evaluation for rare words.

 



## References

* [1]O. Barkan and N. Koenigstein (2016)Item2vec: neural item embedding for collaborative filtering. In IEEE Machine Learning for Signal Processing (MLSP), pp. 1-6. Cited by: SS3.4.
* O. Barkan (2017)Bayesian neural word embedding. In Proceedings of AAAI Conference on Artificial Intelligence, pp. 3135-3143. Cited by: SS1.
* O. Barkan, N. Koenigstein, E. Yogev and O. Katz (2019)CB2CF: a neural multiview content-to-collaborative filtering model for completely cold item recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems, pp. 228-236. Cited by: SS1.
* O. Barkan, A. Caciularu, O. Katz, and N. Koenigstein (2020)Attentive tem2vec: Neural Attentive User Representations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* O. Barkan, O. Katz, and N. Koenigstein (2020)Causal inference in context: the concept revisited. In Proceedings of the 10th international conference on World Wide Web, pp. 406-414. Cited by: SS1.
* O. Barkan, N. Koenigstein, E. Yogev and O. Katz (2019)CB2CF: a neural multiview content-to-collaborative filtering model for completely cold item recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems, pp. 228-236. Cited by: SS1.
* O. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* O. Barkan, N. Koenigstein, E. Yogev, and O. Katz (2019)CB2CF: a neural multiview content-to-collaborative filtering model for completely cold item recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems, Pages 228-236. Cited by: SS1.
* O. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendation. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* O. Barkan, N. Koenigstein, E. Yogev, and O. Katz (2019)CB2CF: a neural multiview content-to-collaborative filtering model for completely cold item recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems, Pages 228-236. Cited by: SS1.
* O. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. Barkan, O. Katz, and N. Koenigstein (2020)Recursive user representation for completely cold item recommendations. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Cited by: SS1.
* D. J. MacKay (1992)A variational approach to bayesian logistic regression models and their extensions. In Sixth International Workshop on Artificial Intelligence and Statistics, Vol. 82, pp. 4. Cited by: SS1.
* Y. Levine, B. Lenz, Or Dagan, D. Padnos, O. Sharir, S. Shalev-Shwartz, A. Shashua, and Y. Shoham (2019)SenseBERT: driving some sense into bert. ArXivabs/1908.05646. Cited by: SS1.
* Y. Li, R. Zheng, T. Tian, Z. Hu, R. Iyer, and K. P. Sycara (2016)Joint embedding of hierarchical categories and entities for concept categorization and dataless classification. In Proceedings COLING, pp. 2678-2688. Cited by: SS1.
* D. Luo, H. Xu, and C. Lawrence (2019)Interpretable ICD Code Embeddings with Self-and Mutual-Attention Mechanisms. arXiv preprint arXiv:1906.05492. Cited by: SS1.
* Th. Luong, R. Socher, and C. Manning (2013)Better word representations with recursive neural networks for morphology. In Proceedings of the Conference on Computational Natural Language Learning (CoNLL), Cited by: SS1.



David J. C. MacKay. 1992. The evidence framework applied to classification networks. In _Neural Computation_, 4(5):720-736.
* Mikolov et al. [2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In _Advances in Neural Information Processing Systems 26_, pages 3111-3119. Curran Associates, Inc.
* Miller et al. [1990] George A Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine J Miller. 1990. Introduction to wordnet: An on-line lexical database. In _International journal of lexicography_, 3(4):235-244.
* Miller et al. [1993] George A Miller, Claudia Leacock, Randee Tengi, and Ross T Bunker. 1993. A semantic concordance. In _Proc. of the Workshop on Human Language Technology_, pages 303-308. Association for Computational Linguistics.
* Pennington et al. [2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: global vectors for word representation. In _Empirical Methods in Natural Language Processing (EMNLP)_.
* Rogers et al. [2018] Anna Rogers, Shashwath Hosur Ananthakrishna, and Anna Rumshisky. 2018. What's in your embedding, and how it predicts task performance. In _Proceedings of the International Conference on Computational Linguistics (ACL)_.