In this task, you will be presented with a context from an academic paper and a question separated with a 
. You have to answer the question based on the context.
--------
Question: We use OpenNMT BIBREF24 as the implementation of the NMT system for all experiments BIBREF5 . PBMT-R is a phrase-based method with a reranking post-processing step BIBREF18 . Hybrid performs sentence splitting and deletion operations based on discourse representation structures, and then simplifies sentences with PBMT-R BIBREF25 . SBMT-SARI BIBREF19 is syntax-based translation model using PPDB paraphrase database BIBREF26 and modifies tuning function (using SARI). Dress is an encoder-decoder model coupled with a deep reinforcement learning framework, and the parameters are chosen according to the original paper BIBREF20 . 
 Question: what state of the art methods did they compare with?

Answer: OpenNMT PBMT-R Hybrid SBMT-SARI Dress


Question: We also looked at where there was complete agreement by all annotators that a triple extraction was incorrect. In total there were 138 of these triples originating from 76 unique sentences.  
 Question: What is the most common error type?

Answer: all annotators that a triple extraction was incorrect


Question: Our novelties include:

Using self-play learning for the neural response ranker (described in detail below).

Optimizing neural models for specific metrics (e.g. diversity, coherence) in our ensemble setup.

Training a separate dialog model for each user, personalizing our socialbot and making it more consistent.

Using a response classification predictor and a response classifier to predict and control aspects of responses such as sentiment, topic, offensiveness, diversity etc.

Using a model predictor to predict the best responding model, before the response candidates are generated, reducing computational expenses.

Using our entropy-based filtering technique to filter all dialog datasets, obtaining higher quality training data BIBREF3.

Building big, pre-trained, hierarchical BERT and GPT dialog models BIBREF6, BIBREF7, BIBREF8.

Constantly monitoring the user input through our automatic metrics, ensuring that the user stays engaged. 
 Question: What is novel in author's approach?

Answer:
They use self-play learning , optimize the model for specific metrics, train separate models per user, use model  and response classification predictors, and filter the dataset to obtain higher quality training data.