In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

[Q]: To test the difficulty of our dataset, we checked the majority class label and the accuracies of five state-of-the-art NLI models adopting different approaches: BiMPM (Bilateral Multi-Perspective Matching Model; BIBREF31 , BIBREF31 ), ESIM (Enhanced Sequential Inference Model; BIBREF32 , BIBREF32 ), Decomposable Attention Model BIBREF33 , KIM (Knowledge-based Inference Model; BIBREF34 , BIBREF34 ), and BERT (Bidirectional Encoder Representations from Transformers model; BIBREF35 , BIBREF35 ). Regarding BERT, we checked the performance of a model pretrained on Wikipedia and BookCorpus for language modeling and trained with SNLI and MultiNLI.
Question: What NLI models do they analyze?
[A]: BiMPM ESIM Decomposable Attention Model KIM BERT

[Q]: Our novelties include:

Using self-play learning for the neural response ranker (described in detail below).

Optimizing neural models for specific metrics (e.g. diversity, coherence) in our ensemble setup.

Training a separate dialog model for each user, personalizing our socialbot and making it more consistent.

Using a response classification predictor and a response classifier to predict and control aspects of responses such as sentiment, topic, offensiveness, diversity etc.

Using a model predictor to predict the best responding model, before the response candidates are generated, reducing computational expenses.

Using our entropy-based filtering technique to filter all dialog datasets, obtaining higher quality training data BIBREF3.

Building big, pre-trained, hierarchical BERT and GPT dialog models BIBREF6, BIBREF7, BIBREF8.

Constantly monitoring the user input through our automatic metrics, ensuring that the user stays engaged.
Question: What is novel in author's approach?
[A]: They use self-play learning , optimize the model for specific metrics, train separate models per user, use model and response classification predictors, and filter the dataset to obtain higher quality training data.

[Q]: To tackle this issue, we present a new evaluation dataset that covers a wide range of monotonicity reasoning that was created by crowdsourcing and collected from linguistics publications (Section "Dataset" ).
Question: Do they release MED?
[A]:
Yes