NLP Lecture 14 @ CMU — A Watch👓 & Read Treat🍨
By: Jeevesh Juneja
Tags: Optimization, Machine Learning, Sampling, Structured Prediction, NLP

Search Based Structured Prediction — by Graham Neubig

In our last few lecture notes(11 , 12 & 13) we have been talking all about Structured Prediction, i.e., predicting a label for input sequence, where the space of all possible labels is too big, but has some inherent structure. We were trying to utilize this structure all through our previous endeavours, to efficiently navigate the big output space.

Up until now, in each of our globally normalized model, we have been trying to use DP(which utilizes the structure of output space) to calculate the partition function. But, we have had to construct a new type of Dynamic Program, each time the structure of the output space changes. Moreover, these programs have time complexity which is O(n²V) at least. Here n is the size of the output and V is the number of options available at each of the n steps required to make the output of size n. A natural question arises,

Is there any better way to deal with partition functions ?

Exercise 14.1: Where else have we encountered problems with partition functions? What did we do then? We are going to use similar techniques now. [[Hint : Deep Learning Book Ch 18 & Ch 19 ]]

Said precisely, we are going to approximate the partition function.

In the first method, we approximate the sum of scores of all possible outputs, by choosing k samples according to the probability distribution which can be obtained using the local scores(e.g., transition and emission scores)**. In the second method, we choose k most likely samples to approximate our sum.

From the second approach, we observe that we don’t need to normalize our model, if we only want to get top few samples. But there may be other places, where we might need actual probabilities. For example, when two models are concatenated, and the second one needs to know how sure is the first one in its prediction.

Since max probabilities correspond to max scores, we don’t need normalization,

Exercise 14.2: The structure of CNNs allows them to learn extrapolations(from discrete points into the continuous space)that are consistent with the human vision(i.e., visually similar images lie close by in the continuous space). Does the input and output of, say POS tagging task, for a very large number of sentences, provide enough signal, to say, a transformer, to allow it to extrapolate discrete points into probability distributions over entire space, in a way consistent with human sense of POS tags over sentences. (i.e, can the model be expected to assign 0.5 probabilities to the two possible POS tag sequences for “I saw a girl with a telescope”) ? What extra signal & in what form can we provide to the model to improve performance ?

Generally speaking, it is observed that NLP models have good predictions, but not so good calibration, to the real world.

Using Scores Intelligently for Learning

Why don’t we directly maximize scores of the correct output using gradient ascent ?

Because, when trying to maximize the score of the correct sample, we may end up moving in a direction that reduces its probability.

But, what if we can make sure that doesn’t happen ? We need to add terms to the loss function that make sure that the probabilities of incorrect outputs don’t get increased. That is exactly what the Structured Perceptron Algorithm does.

Moving in a direction that maximizes score of the correct output, while minimizing the score of the predicted(max. score) sequence. It is like a see-saw : maximizing one, minimizing another. Notice that the domain of argmax has been reduced.

The above formulation can be written as a model having the following loss function :

This loss function will give similar effect even if the predicted output is approximately the one with max. score. Also, it sits nicely with the argmax over the reduced domain used in the previous slide.

To improve the performance of the above method, instead of just making the highest scoring prediction sit on the other side of the see-saw, we can collect all the predictions having score higher than the correct one, and make sure that we increase the score of the correct output, while reducing the score of all the output sequences having score higher than the correct output. The loss function then becomes :

This would have same problems as partition function as it enumerates all possible outputs.

What people usually choose to do, in case they want to avoid partition function, is to train using teacher forcing for pre-training and then use more complicated training algorithms, like above, later on. That is, local optimization, followed by global one, to include global structure and reduce exposure bias***.

Exercise 14.3: What would teacher forcing look like in case of a model like CRFs from Lecture 11 ? [[Hint : Think, outputs at each time-step]]

Notice, how we have to search for the maximum scoring sequence, even during optimization. That is why this lecture is called “search based structured prediction” .

Optimizing Decision Boundary

In the above regime, the training on one sample will stop as soon as it becomes the one with the highest score. But that might not be optimal boundary so we can use a hinge loss, to increase the margin. That is, continue to increase the score of correct sample till it becomes larger than the second highest sample, by a margin of m.

This hinge loss can be used in a per time-step fashion too :

And we can set different margins for different (correct, incorrect) decision pairs, corresponding to how far we want to go from each incorrect output :

We can also extend the cost-augmented feature to the problem when y’s are sequences, in the following way :

We do 1-BLEU, because we want the margin to be greater when the two sequences are very dissimilar, i.e., when BLEU is low.

Exercise 14.4 : Assume our task is of POS tagging & our training data consists of labelled sentences. We train a encoder-decoder architecture, with teacher forcing and MLE loss. We know, from before, that given

enough capacity for the model, infinite data, large enough batch size, long enough training time. soft-max loss function

our model will be able to learn(or memorize) every little detail, about every sentence. If batch size is infinite and the model sees that of the 2 possible POS tagging for “I saw a girl with a telescope” , one occurs 3 times and the other occurs 1 time in the batch; then the POS tag sequences will get assigned probabilities 3/4 and 1/4 respectively.

Will the same probabilities be assigned, if we use step-wise hinge loss instead of soft-max? What about the case when we use cost-augmented hinge loss? If we relax assumption 2 and 3, which of the three losses (cost-augmented HL, hinge loss, and MLE) is expected to give highest calibration? Highest accuracy?

Exercise 14.5 : We know, from before, that the function space represented by CNNs, is the same as that represented by NNs. This means, given assumptions (1–4) of the previous exercise, we can “overcome” the inductive bias of CNNs and make them represent any function that a simple NN can. Similarly, hinge loss & cost-augmented hinge loss come with their own inductive bias. Can we “overcome” these bias if assume (1–4) of previous exercise? Will the answer be different for different learning rates ? If the inductive bias introduced by the hinge/cost-augmented hinge loss exactly match the corresponding bias in the data, will the hinge/cost-augmented hinge approach be strictly better, in terms of accuracy, than soft-max ? What about in terms of calibration ?

Things to think upon :

Try to formalize how the accuracy of different tasks changes as the inductive bias of the model, approaches(smoothly) that in the real data. Carry out similar analysis for sequence based hinge/cost-augmented hinge loss.

Simple Remedies For Exposure Bias

Scheduled Sampling : During teacher forcing, instead of sending the correct output of the previous time steps, we sometimes sample from the distribution predicted by the model at the previous time step, and use that for teacher forcing. How frequently, we sample from the predicted distribution can be slowly annealed from 0 to some constant.

This approach will not work nicely, when there are two possible output sequences for a single input sentence, and we sample from the probability distribution according to one, and end up using that sample to predict the other sequence.

For example, in case of NMT, some sentence in some other language can be translated into both “Dogs are barking” and “The dogs are barking”. So, if we sample “Dogs” at first time step and if the given target sentence is “The dogs are barking”, we will be using “Dogs” to predict “dogs” at the next time step; and hence will be maximizing the probability of “Dogs dogs are barking”.

Exercise 14.6:

Will this approach work fine in case of POS tagging? Even when there are two different POS sequences corresponding to the same sentence? Why? [[Hint : Think about what kind of error might occur in POS tagging]] What about tasks where the predictions at next step are restricted by the teacher-forcing input of previous step ? For example, in program generation, when you have predicted “func” in previous step, you must predict its arguments at next step. How will you adopt the above approach to that case ? [[Hint : Choose what you are going to predict at each time step from the based on the actual previous outputs in the target sequence. See here.]]

Drop Out Inputs : Input a zero vector instead of the input from the previous time step. The model relies more on context as compared to previous input. How frequently dropout occurs, is decided by a hyper-parameter.

Corrupt Training Data : We replace the correct target sentence with a corrupted one. The corrupted sentence is sampled according to the following distribution :

where,

and the function r can be the edit-distance/ hamming distance between the sequences y and y* or, 1-BLEU . The temperature factor, tau, decides how sharp the distribution will be be. This is like label smoothing.

The last method, works unexpectedly better(empirically), than the previous two.

Using Beam Search For Optimization : We use beam search at training time, and increase the probability of the correct output over the one with the max. score by some margin, at each time step.

Explanation : The higher the word, the higher is its score in the beam. Yellow boxes are correct sequence. Grey ones are the with max. score. Arrows indicate which previous word’s beam, is the current word derived from. As soon as correct word is run-off from the beam, we drop all words in the beam and expand the correct word instead, e.g.”run” above.