Paper Explained- Underspecification Present in Modern Machine Learning
By: Nakshatra Singh
Tags: MIT, Naturallanguageprocessing, Google, Epidemiology, Machine Learning

Underspecification in a simple epidemiological model. The image is taken from the paper.

Introduction & Overview

The authors believe that underspecification is one of the key reasons for the poor behavior of Machine Learning models when deployed in real-world domains. Think of it like this, you have a big training set, you train your model on the training set, then you test it on the testing set, and usually, they come from the same distribution. However, there is a caveat here, when you deploy your trained model to production in the real-world, the distribution of the data is very different and the model might not as well perform well. So, the underspecification problem the authors identify is when all the models from your training procedure work equally well on the test set, however, they perform very differently in the real world scenario. There might be one of the models (from different random seed values) which performs well even in the real world but the other models don’t perform as well. This confirms that the train-test split pipeline is underspecified, the train-test split simply doesn't capture the importance of real-world data. There are two ways to solve this problem, either you test all the models coming out of the pipeline on the real-world data distribution, and the one that works well you deploy that model or the other way is that you add specification during the model training pipeline phase making sure all the real-world features are trained on using that pipeline.

Underspecification in an Epidemiological Model

Specifically, the authors consider the simple Susceptible-Infected-Recovered (SIR) model that is often used as the basis of epidemic models in infectious disease epidemiology. This model is specified in terms of the rates at which the number of susceptible (S), infected (I), and recovered (R) individuals in a population of size N, change over time:

Image is taken from the paper.

In this model, the parameter β represents the transmission rate of the disease from the infected to susceptible populations, and the parameter D represents the average duration that an infected individual remains infectious. Let’s say hypothetically, that a pandemic is about to begin, and some data points are given to us. We need to predict the trajectory of the pandemic with those data points so what we are going to do is you want to fit β and D parameters to the data points. There is actually a unique solution however because of the exponential rise of the trajectory, the solution is numerically not well specified. Importantly, during the early stages of a pandemic, when observed data points are small, the parameters of the model are underspecified by this training task. This is because, at this stage, the number of susceptible is approximately constant at the total population size (N), and the number of infections grows approximately exponentially at rate β −1/. We can see that the approximate exponential rate we are dealing with in this model is β−1/D. If we derive some number with this expression, there are many settings/infinite number of pairs which equates the L.H.S and R.H.S of the equation. There are many different predictors (the pairs we talked about earlier, to be more concise) to choose from, the one which performs the best on real-world data is deployed. The training setup is not different for different predictors, the only thing here is that the variance of predictors is so large w.r.t real-world data that some predictors perform well and the others perform poorly. Different initialization distributions (For example, Gamma or Normal) result in different distributions of predicted trajectories. The authors say this is a classic example of underspecification.

Underspecification in a Random Feature Model

Underspecification is also a natural consequence of overparameterization, which is a key property of many modern neural networks. When there are more parameters than data points, the learning problem is ingrained to be underspecified. The authors implement a simple two-layer network where the first layer is completely random, it’s sort of like a linear model. You assume some kind of distribution on the data, some distribution on the weight matrix entries (W), then you train the theta parameter (θ) and make some statements about what happens with that model. F𝓌(X) = θᵀσ(Wx).

Think of it like this, the weight matrices you get out of your training procedure can be drastically different and the solution to each of them can be also different. But we can construct an adversarial dataset that projects itself in such a way that it specifies the model in a better way.

Underspecification in Medical Genomics

This is a medical genomic example where you can see there is training data, evaluation data that comes from the same distribution, and then evaluation data that comes outside of the distribution. So, this is more of a domain shift example. You can see if you train on the training data and then you evaluate on the training data you get a normalized mean squared error, lower is better. The red dot is a specific heuristic representative that performs just a bit better. If you go to the evaluation set on the same data you can see the range of performance is quite similar to the training set but is still acceptable. But, when you go the sets out-of-distribution the range of performance is just very-very big. Further, the authors show a plot to compare how well a model does in the evaluation set in distribution vs evaluation set out-of-distribution and you see the correlation is fairly weak, you would want to expect a straight line.

Underspecification in linear models in medical genomics. (Left) Performance of a PRS model using genetic features in the British training set, the British evaluation set, and the “nonBritish” evaluation set, as measured by the normalized mean squared error (MSE divided by the true variance, lower is better). Each dot represents a PRS predictor. Image is taken from the paper.

Underspecification in BERT Models

In NLP, the authors test their demonstrations with BERT. This is interesting because not only they test different seeds for fine-tuning BERT but they also test different seeds for pre-training BERT. So, in these language models, you have a pre-training phase and a fine-tuning phase, and both of them have random seeds. The authors show that the random seed of pre-training will actually play a big role in how these models perform in the stress test. In the semantic textual similarity (STS) task, a predictor takes in two sentences as input and scores their similarity using a certain metric. Predictors for the STS task is obtained by fine-tuning BERT on the U.S. Bureau of Labor Statistics (BLS) dataset. The predictor achieves constant accuracy, measured in terms of correlation ranging from 0.87 to 0.90. In the pronoun resolution (PR) task, the input is a sentence with a pronoun that could refer to one of two possible antecedents, and the predictor must determine which of the antecedents is the correct one. Predictors of the PR task is obtained by fine-tuning BERT on the OntoNotes dataset. The predictor achieves constant accuracy, ranging from 0.960 to 0.965.

Reliance on gendered correlations is affected by random initialization. (Left) The gap in similarity for female and male template sentences is correlated with the gender statistics of the occupation, shown in two randomly-initialized fine-tunes. (Right) Pretraining initialization significantly affects the distribution of gender biases encoded at the fine-tuning stage. Image is taken from the paper.

These plots show three important patterns:

There is a large spread in correlation with BLS statistics: on the STS task, correlations range from 0.3 to 0.7; on the pronoun resolution task, the range is 0.26 to 0.51. There is a weak relationship between test accuracy performance and gendered correlation. [{STS task: Confidence Interval = (0.00, 0.39), PR task: Confidence Interval= (−0.13, 0.29)}]. This indicates that learning accurate predictors does not require learning strong correlations. Third, the encoding of spurious correlations is sensitive to the random seed at pretraining, and not just fine-tuning.

Conclusion & Comments

I fully agree with what the paper proposes. Testing your model in a deployed environment may not solve the problem of underspecification but it seems to me that is not an upright solution. As we saw in the epidemiology demonstration that simply isn't possible. Not everyone can train a language model from scratch specifically for a task. So we do need pre-trained checkpoints. A practical solution to this problem could be to check all the model checkpoints to see which one is good or bad on the particular dimension they care about. What I would say is, it’s not always possible to add the specification you want and keep the model performance the same. Adding constraints won’t make the model reach the same performance. All in all, it is a paper you can agree with, the sentiment, the analysis, the problem, and the examples they show are real. Especially for a company like Google, this is fairly important as they build huge S.O.T.A models.

If you enjoyed this article and gained insightful knowledge, consider buying me a coffee ☕️ by clicking here :)

References

Underspecification Presents Challenges for Credibility in Modern Machine Learning, MIT- October 28, 2020.

If you liked this post, please make sure to clap 👏. 💬 Connect? Let’s get social: http://myurls.co/nakshatrasinghh.