Question: What five dialogue attributes were analyzed?
Evidence: Curriculum Plausibility ::: Conversational Attributes ::: Specificity
A notorious problem for neural dialogue generation model is that the model is prone to generate generic responses. The most unspecific responses are easy to learn, but are short and meaningless, while the most specific responses, consisting of too many rare words, are too difficult to learn, especially at the initial learning stage. Following BIBREF11, we measure the specificity of the response in terms of each word $w$ using Normalized Inverse Document Frequency (NIDF, ranging from 0 to 1):
Curriculum Plausibility ::: Conversational Attributes ::: Repetitiveness
Repetitive responses are easy to generate in current auto-regressive response decoding, where response generation loops frequently, whereas diverse and informative responses are much more complicated for neural dialogue generation. We measure the repetitiveness of a response $r$ as:
Curriculum Plausibility ::: Conversational Attributes ::: Continuity
A coherent response not only responds to the given query, but also triggers the next utterance. An interactive conversation is carried out for multiple rounds and a response in the current turn also acts as the query in the next turn. As such, we introduce the continuity metric, which is similar to the query-relatedness metric, to assess the continuity of a response $r$ with respect to the subsequent utterance $u$, by measuring the cosine similarities between them.
Curriculum Plausibility ::: Conversational Attributes ::: Model Confidence
Despite the heuristic dialogue attributes, we further introduce the model confidence as an attribute, which distinguishes the easy-learnt samples from the under-learnt samples in terms of the model learning ability. A pretrained neural dialogue generation model assigns a relatively higher confidence probability for the easy-learnt samples than the under-learnt samples. Inspired by BIBREF16, BIBREF17, we employ the negative loss value of a dialogue sample under the pretrained model as the model confidence measure, indicating whether a sampled response is easy to be generated. Here we choose the attention-based sequence-to-sequence architecture with a cross-entropy objective as the underlying dialogue model.
Curriculum Plausibility ::: Conversational Attributes ::: Query-relatedness
A conversation is considered to be coherent if the response correlates well with the given query. For example, given a query “I like to paint”, the response “What kind of things do you paint?” is more relevant and easier to learn than another loosely-coupled response “Do you have any pets?”. Following previous work BIBREF14, we measure the query-relatedness using the cosine similarities between the query and its corresponding response in the embedding space: $\textit {cos\_sim}(\textit {sent\_emb}(c), \textit {sent\_emb}(r))$, where $c$ is the query and $r$ is the response. The sentence embedding is computed by taking the average word embedding weighted by the smooth inverse frequency $\textit {sent\_emb}(e)=\frac{1}{|e|}\sum _{w\in {}e}\frac{0.001}{0.001 + p(w)}emb(w)$ of words BIBREF15, where $emb(w)$ and $p(w)$ are the embedding and the probability of word $w$ respectively.
Curriculum Plausibility ::: Conversational Attributes ::: Specificity
A notorious problem for neural dialogue generation model is that the model is prone to generate generic responses.
Curriculum Plausibility ::: Conversational Attributes ::: Repetitiveness
Repetitive responses are easy to generate in current auto-regressive response decoding, where response generation loops frequently, whereas diverse and informative responses are much more complicated for neural dialogue generation.
Curriculum Plausibility ::: Conversational Attributes ::: Continuity
A coherent response not only responds to the given query, but also triggers the next utterance.
Curriculum Plausibility ::: Conversational Attributes ::: Model Confidence
Despite the heuristic dialogue attributes, we further introduce the model confidence as an attribute, which distinguishes the easy-learnt samples from the under-learnt samples in terms of the model learning ability.
Curriculum Plausibility ::: Conversational Attributes ::: Query-relatedness
A conversation is considered to be coherent if the response correlates well with the given query.
Answer: The answers are shown as follows:
* Model Confidence
* Continuity
* Query-relatedness
* Repetitiveness
* Specificity