Paper:
We design a new algorithm for batch active learning with deep neural network models. Our algorithm, Batch Active learning by Diverse Gradient Embeddings (BADGE), samples groups of points that are disparate and high-magnitude when represented in a hallucinated gradient space, a strategy designed to incorporate both predictive uncertainty and sample diversity into every selected batch. Crucially, BADGE trades off between diversity and uncertainty without requiring any hand-tuned hyperparameters. While other approaches sometimes succeed for particular batch sizes or architectures, BADGE consistently performs as well or better, making it a useful option for real world active learning problems.

Review:
This paper examines the problem of warm-starting the training of neural networks. In particular, a generalization gap arises when the network is trained on the full training set from the start versus being warm-started, where the network is initially (partially) trained on a subset of the training set, then switched to the full training set. This problem is practical, as it is often preferable to train online while data is collected to make up-to-date predictions for tasks (such as in online advertising or recommendation systems), but it has been found that retraining is necessary in order to obtain optimal performance. The paper also mentions active learning, domain shift, and transfer learning as two other relevant and important problems.
The paper attempts to investigate this phenomena from a few different avenues: (1) simple experiments segmenting the training set into two different subsets, then training to completion or partial training on the first subset before switching to training on the full set; (2) looking at the gradients of warm-started models; (3) adding regularization; (4) warm-starting all layers, then training only last layer; (5) perturbing the warm-started parameters. 
Strengths:
I believe very strongly in the practical impact of the problems presented in this paper. These indeed are challenging problems that are relevant to industry that have not been given sufficient attention in the academic literature. I appreciate the initial experimentation on this subject, and the clear demonstration of the problem through simple experiments. The paper is also well-written.
Weaknesses:
Some questions I had include:
- Why is the Pearson correlation between parameters of the neural network a good way of measuring the correlation to their initialization?
- Why is it surprising that the magnitude of the gradients of the "new" data is higher than at a random initialization?
- Why does this phenomena occur even though the data is sampled from the same distribution? 
- Does this work have any relationship with work on generalization such as:
[1] Recht, Benjamin, et al. "Do CIFAR-10 classifiers generalize to CIFAR-10?." arXiv preprint arXiv:1806.00451 (2018).
[2] Recht, Benjamin, et al. "Do ImageNet Classifiers Generalize to ImageNet?." arXiv preprint arXiv:1902.10811 (2019).
etc.
Although I like the topic of this paper, the investigation seems too preliminary at this point. There is no clear hypothesis towards answering the problems proposed in the paper. There is also no analysis, which places the burden on the numerical experiments to demonstrate something interesting, and the experiments seem sparse and small-scale. For these reasons, I am inclined to reject this paper at this time, but I strongly encourage further exploration into the topic. 
Some potential questions or directions could include:
1. What if only a single epoch of training is used on 50% of the data? Does the gap appear in that setting? I ask because one would expect that a single epoch of training on 50% of the data, then training on new data would be equivalent to training on the full dataset from the start.
2. How does this gap change with respect to the (relative) amount of new data introduced into the problem? For example, if one were to only add a single datapoint to the training set, would one still observe this behavior? Could one potentially add data more incrementally (rather than half of the training set) and potentially mitigate this drastic change in the problem?
3. There are optimization algorithms specifically designed for stochastic optimization (with a fixed distribution) versus for online optimization (online gradient, Adagrad). Is the online optimization framework perhaps more "realistic" than the stochastic optimization framework in these streaming/warm-starting settings?