In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

We map each relation type $R(x,y)$ to at least one parametrized natural-language question $q_x$ whose answer is $y$ .
How is the input triple translated to a slot-filling task?

In our setup, our starting point is a base model, trained on NLI data. Rather than employing automated adversarial methods, here the model's “adversary” is a human annotator. Given a context (also often called a “premise” in NLI), and a desired target label, we ask the human writer to provide a hypothesis that fools the model into misclassifying the label. One can think of the writer as a “white hat” hacker, trying to identify vulnerabilities in the system. For each human-generated example that is misclassified, we also ask the writer to provide a reason why they believe it was misclassified.
Do they use active learning to create their dataset?

Let all word vectors be normalized and $W$ be the word matrix. Inspired by BIBREF21 , where vector space models are used for evaluating topic coherence, we suggest to estimate the interpretability of $k$ th component as $ \operatorname{interp}_k W = \sum _{i,j=1}^N W_{i,k} W_{j,k} \left(W_i \cdot W_j \right). $ 

How do they evaluate interpretability in this paper?