TASK DEFINITION: In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
PROBLEM: Three different datasets have been used to train our models: the Toronto book corpus, Wikipedia sentences and tweets.  Our Sent2Vec models also on average outperform or are at par with the C-PHRASE model, despite significantly lagging behind on the STS 2014 WordNet and News subtasks. This observation can be attributed to the fact that a big chunk of the data that the C-PHRASE model is trained on comes from English Wikipedia, helping it to perform well on datasets involving definition and news items.  
 Question: Do they report results only on English data?

SOLUTION: Yes

PROBLEM: The proposed Multimodal Differential Network (MDN) consists of a representation module and a joint mixture module. We use a triplet network BIBREF41 , BIBREF42 in our representation module. The triplet network consists of three sub-parts: target, supporting, and contrasting networks. All three networks share the same parameters. The Mixture module brings the image and caption embeddings to a joint feature embedding space. 
 Question: How do the authors define a differential network?

SOLUTION: The proposed Multimodal Differential Network (MDN) consists of a representation module and a joint mixture module.

PROBLEM: The goal of an language model is to assign meaningful probabilities to a sequence of words. Given a set of tokens $\mathbf {X}=(x_1,....,x_T)$, where $T$ is the length of a sequence, our task is to estimate the joint conditional probability $P(\mathbf {X})$ which is

were $(x_{1}, \ldots , x_{i-1})$ is the context. An Intrinsic evaluation of the performance of Language Models is perplexity (PPL) which is defined as the inverse probability of the set of the tokens and taking the $T^{th}$ root were $T$ is the number of tokens We propose an approximation of the joint probability as,

This type of approximations has been previously explored with Bi-directional RNN LM's BIBREF9 but not for deep transformer models. We therefore, define a pseudo-perplexity score from the above approximated joint probability. 
 Question: How is pseudo-perplexity defined?

SOLUTION:
Answer with content missing: (formulas in selection): Pseudo-perplexity is perplexity where conditional joint probability is approximated.