Is My NLP Model Working?
The Answer is Harder Than You Think
Graham Neubig

A New Paradigm in NLP:
Text Generation through LMs+Prompting
Q: What is the capital of Pennsylvania?
A:

Harrisburg

It’s for real this time. After months of legal drama, bad memes and will-they-or-won’t-they-chaos to put yourfavorite rom-com to shame, Elon Musk has closed his $44 billion acquisition of Twitter. Musk sealed the deal
Thursday night, taking Twitter private and ousting a handful of top executives — CEO Parag Agrawal included —in the process.

TL;DR:Elon Musk has bought Twitter.

How Good is Resulting Text?
How do the muscles in your arm work?
The muscles in your arm work by contracting and pulling on the bones inyour arm. This movement is possible because the muscles are attachedto the bones by tendons.
Why do we sleep? Why do we dream?
We sleep to give our bodies and brains a chance to rest and repair
themselves. Dreams may help us process and store memories, or they
may be a way for our brains to work through problems or anxiety.
From GPT-3 versus Google Search by David E. Weekly

https://twitter.com/dweekly/status/1580676295444217857

How Bad is Generated Text? - Factuality
Q: What are the largest states in the US by population and surface area?

A:Alaska and Texas

Q: What is the net worth of the CEO of Twitter?
Jack Dorsey is the CEO of Twitter.
A:

He is worth $2.5 billion.

How Bad is Generated Text? - Coherence / Plausibility
This is a story about a programmer on her first day of work.
She was given a task to write a program that would print the numbers
from 1 to 100.
But for multiples of three print “Fizz” instead of the number and for themultiples of five print “Buzz”.
For numbers which are multiples of both three and five print “FizzBuzz”.
She was given a week to complete the task.
She was given a week to complete the task.
She was given a week to complete the task.

Evaluating generated text is
now about as hard asgenerating it.

The NLP Development Pipeline
Training Data

SystemIdeas

Testing Data

Evaluation Result

How Hard is Evaluation?
Classification → easy, measure exact matchTranslation → more difficult, many different good translations withthe same semantics
Dialog → even more difficult, many different good answers withdifferent semantics
Quiz: What about question answering? Summarization?

The Gold-standard?: Manual Evaluation??

0.8
Hypothesis 1
Source??

0.5
Hypothesis 2

An Alternative: Automatic Evaluation??

0.8
Hypothesis 1
SourceReference??

0.5
Hypothesis 2

The Old Reliables: BLEU/ROUGE Score
Reference: I am giving a talk at a data science conference
Hyp 1: I am giving a talk at a conference about data sciencelots of overlap → high score
Hyp 2: This talk is about recent advances in medical imaging
little overlap → low score

Why is Evaluation Hard?
Reference: I am giving a talk at a data science conference
Hyp 1: I am giving a talk at a political science conference
lots of overlap but bad output
Hyp 2: My lecture will be given to the meeting on data analytics
little overlap but good output(particularly difficult for open-ended problems)Embedding-based Evaluation

BERTScore

Zhang, Tianyi, et al. "Bertscore: Evaluating text generation with bert." arXiv preprint arXiv:1904.09675 (2019).

https://github.com/Tiiiger/bert_score

Learning to Evaluate
0.5
SourceUpdate0.4
Difference

Reference??

Hypothesis 1

0.1
Rei, Ricardo, et al. "COMET: A neural framework for MTevaluation." arXiv preprint arXiv:2009.09025 (2020).

https://unbabel.github.io/COMET/Learning to Evaluate w/ Pseudo-data

UniEvalZhong, Ming, et al. "Towards a Unified Multi-Dimensional Evaluator for Text Generation." (2022).

https://github.com/maszhongming/UniEval

Generative Text Evaluation
Use the probability of a generative model to evaluate text??

P(??

)
SourceHypothesisP(Hypothesis??

P(HypothesisSource??

)

Reference)

P(Hypothesis)
Reference

Yuan, Weizhe, Graham Neubig, and Pengfei Liu. "Bartscore: Evaluating generated text as text generation."
Advances in Neural Information Processing Systems 34 (2021): 27263-27277.

https://github.com/neulab/BARTScore

Generative Pre-training,Discriminative Fine-tuning

T5ScoreQin, Yiwei, et al. "T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics." (2022).

https://github.com/qinyiwei/T5Score

How Do We Evaluate Evaluation?

??

??

Human

Automatic0.80.7

0.50.1

Correlation
Pearson = 0.23
Kendall = 0.33??

0.10.50.60.4??

Meta-Evaluation Results

So many evaluation metrics!
What to do next?
● Multi-metric evaluation● Metric-aware training/inference● Fine-grained analysis

Multi-dimensional evaluation oftext-generation tasks.

Current Text Generation Evaluation Standard

https://arxiv.org/abs/2203.16804

Why are we stuck?
● Running evaluation is slow, requires software install,GPU● There’s always other things to do!

Critique: A Simple Evaluation API for TextSummaries

Summary Quality

Translations

Dialogs

Translation Quality

Fluency

Question Answers

Toxicity

Factual Consistency

https://docs.inspiredco.ai/critique/API-based Usage
import os
from inspiredco.critique import Critiqueclient = Critique(api_key=os.environ["INSPIREDCO_API_KEY"])
dataset = [{"target": "This is a really nice test sentence."},{"target": "This sentence not so good."},]results = client.evaluate(
metric="uni_eval",config={"task": "summarization", "evaluation_aspect": "fluency"},dataset=dataset,)for datapoint, result in zip(dataset, results["examples"]):print(f"Text: {datapoint['target']}, Fluency: {result['value']}")●Several lines of code, trivial installation, no GPUs required
https://docs.inspiredco.ai/critique/Online Interface

https://dashboard.inspiredco.ai/

Metric-aware
Training/InferenceTo the next step!
Training Data

SystemIdeas

Testing Data

Evaluation Result

Metric-aware Training
Standard MLE Loss

Ranking Loss Basedon Metrics
Liu, Yixin, et al. "BRIO: Bringing order to abstractive summarization." (2022).

https://arxiv.org/abs/2203.16804Metric-aware Reranking●●●

Sample a bunch of outputs and rerank according to metrics
Reference-free metrics, just rerank according to metric
Reference-using metrics, use minimum Bayes risk

My name is Bob.
This is true.
This isn’t true.
This is not true.

Error Matrix(e.g. 1-metric)

Probability

Bayes Risk

0.0 0.9 0.9 0.9
0.9 0.0 0.5 0.5
0.9 0.5 0.0 0.1
0.9 0.5 0.1 0.0

0.2
0.19
0.18
0.17

☆

0.486
0.355
0.292 ☆
0.293

=Fernandes, Patrick, et al. "Quality-aware decoding for neural machine translation." (2022).

https://arxiv.org/abs/2205.00978

Metric-aware Prompt Optimization●Prompting methods are hard to train can benefit from systematic analysis

Prompt Gym

Evaluate different models, different
prompts, different metrics

https://github.com/inspired-cognition/prompt-gym/

Fine-grained Analysis andUnderstanding of NLP Models

To the next step!
Training Data

SystemIdeas

Testing Data

Evaluation Result

NLP Debugging: Understanding the Flaws in Our Systems●We have a number, but where do we go next?

●Fine-grained aggregate analysis“Your model is under-performing on short sentences.”●Case studies“Caution, potentially incorrect sentence:”
Source:Voda byla skvělá.
Reference: The water was great.
Hypothesis: The water was.

A Case Study: Russian-English Translation

Overall performance: Similar by lexical metrics,
but green system better in COMET.

Example-based Aggregate Analysis

Green system better at short sentences:-> Green system might be better at resolving
cross-sentence ambiguity.

Token-based Aggregate Analysis

Green system better at short words, blue systembetter at long words.
-> Green system needs work on technical terms?

Looking Through Examples#Source

Reference

Hyp1

Hyp2

335

Также в зонеThe ban on photography Also in the exclusionотчуждения снят запрет in the exclusion zone has zone, a ban onна съемку.
also been lifted.
shooting was lifted.

A ban on filming has
also been lifted in theexclusion zone.

358Кто же мог оказаться
лучше Гуфа?

Who could be better than Who could be better
Guf?
than Guf?

Who could havebeen better thanGoof?

364

У него пушечныеудары.

His strikes are likecannon blows.

He has cannon strikes. He has cannonballs.

https://explainaboard.inspiredco.ai

What’s next?

Still Challenges!
●Evaluation: “arms race” of evaluation, generation, and human standard

Generation●Evaluation

Automating Fine-grained Analysis: how to discover interesting behaviors
automatically?

“Your model is under-performing on sentenceswith numerals greater than 5000.”●

Few-shot Evaluation/Improvement for New Tasks