In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

Ex Input:
Given the fact that the research on offensive language detection has to a large extent been focused on the English language, we set out to explore the design of models that can successfully be used for both English and Danish.

Ex Output:
What is the challenge for other language except English


Ex Input:
More specifically, we observe the impact of: (i) pre-trained word embeddings BIBREF11, BIBREF12, recurrent BIBREF13 and transformer-based sentence encoders BIBREF14 as question representation strategies; (ii) distinct convolutional neural networks used for visual feature extraction BIBREF15, BIBREF16, BIBREF17; and (iii) standard fusion strategies, as well as the importance of two main attention mechanisms BIBREF18, BIBREF19.

Ex Output:
What type of experiments are performed?


Ex Input:
Contextualized word embeddings, sentence embeddings, such as deep contextualized word representations BIBREF20 , BERT BIBREF22 , encode the complex characteristics and meanings of words in various context by jointly training a bidirectional language model.  The second method is to use sentence embeddings, BERT. It is used to a generate single 786-dimensional sentence embedding from 10k-dimensional one-hot vector or distribution over previous words and then merge into a single context vector with two different merging methods. Using this approach, we can obtain a more dense, informative, fixed-length vectors to encode conversational-context information, $e^k_{context}$ to be used in next $k$ -th utterance prediction. We use contextual gating mechanism in our decoder network to combine the conversational-context embeddings with speech and word embeddings effectively. Our gating is contextual in the sense that multiple embeddings compute a gate value that is dependent on the context of multiple utterances that occur in a conversation.  Let $e_w = e_w(y_{u-1})$ be our previous word embedding for a word $y_{u-1}$ , and let $e_s = e_s(x^k_{1:T})$ be a speech embedding for the acoustic features of current $k$ -th utterance $x^k_{1:T}$ and $e_c = e_c(s_{k-1-n:k-1})$ be our conversational-context embedding for $n$ -number of preceding utterances ${s_{k-1-n:k-1}}$ . Then using a gating mechanism:

$$g = \sigma (e_c, e_w, e_s)$$ (Eq. 15)

where $\sigma $ is a 1 hidden layer DNN with $\texttt {sigmoid}$ activation, the gated embedding $e$ is calcuated as

$$e = g \odot (e_c, e_w, e_s) \\ h = \text{LSTM}(e)$$ (Eq. 16)

and fed into the LSTM decoder hidden layer.  The output of the decoder $h$ is then combined with conversational-context embedding $e_c$ again with a gating mechanism,

$$g = \sigma (e_C, h) \\ \hat{h} = g \odot (e_c, h)$$ (Eq. 17)

Then the next hidden layer takes these gated activations, $\hat{h}$ , and so on.

Ex Output:

How are sentence embeddings incorporated into the speech recognition system?