To continuously improve quality and reflect changes in data, machine learning-based services have to regularly re-train and update their core models. In the setting of language models, we show that a comparative analysis of model snapshots before and after an update can reveal a surprising amount of detailed information about the changes in the data used for training before and after the update . We discuss the privacy implications of our findings, propose mitigation strategies and evaluate their effect.
To continuously improve quality and reflect changes in data, machine learning applications have to regularly retrain and update their core models. We show that a differential analysis of language model snapshots before and after an update can reveal a surprising amount of detailed information about changes in the training data. We propose two new metrics---differential score and differential rank---for analyzing the leakage due to updates of natural language models. We perform leakage analysis using these metrics across models trained on several different datasets using different methods and configurations . We discuss the privacy implications of our findings, propose mitigation strategies and evaluate their effect.
R
learning-based services
learning applications
R
re-train
retrain
R
In the setting of language models, we
We
R
comparative analysis of
differential analysis of language
D
the
R
data used for training before and after the update
training data. We propose two new metrics---differential score and differential rank---for analyzing the leakage due to updates of natural language models. We perform leakage analysis using these metrics across models trained on several different datasets using different methods and configurations