In this task, you will be presented with a context from an academic paper and you have to write an answerable question based on the context. Your questions can be extractive, abstractive, or yes-no questions.

Q: Our novelties include:

Using self-play learning for the neural response ranker (described in detail below).

Optimizing neural models for specific metrics (e.g. diversity, coherence) in our ensemble setup.

Training a separate dialog model for each user, personalizing our socialbot and making it more consistent.

Using a response classification predictor and a response classifier to predict and control aspects of responses such as sentiment, topic, offensiveness, diversity etc.

Using a model predictor to predict the best responding model, before the response candidates are generated, reducing computational expenses.

Using our entropy-based filtering technique to filter all dialog datasets, obtaining higher quality training data BIBREF3.

Building big, pre-trained, hierarchical BERT and GPT dialog models BIBREF6, BIBREF7, BIBREF8.

Constantly monitoring the user input through our automatic metrics, ensuring that the user stays engaged.

A: What is novel in author's approach?
****
Q: The improvement when the difficult subset with expert annotations is mixed with the remaining crowd annotation is 3.5 F1 score, much larger than when a random set of expert annotations are added.

A: How much higher quality is the resulting annotated data?
****
Q: GE-FL reduces the heavy load of instance annotation and performs well when we provide prior knowledge with no bias. In our experiments, we observe that comparable numbers of labeled features for each class have to be supplied. We randomly select $t \in [1, 20]$ features from the feature pool for one class, and only one feature for the other. Our methods are also evaluated on datasets with different unbalanced class distributions. We manually construct several movie datasets with class distributions of 1:2, 1:3, 1:4 by randomly removing 50%, 67%, 75% positive documents. Incorporating KL divergence is robust enough to control unbalance both in the dataset and in labeled features while the other three methods are not so competitive.

A:
How do they define robustness of a model?
****