Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work studies the how the accuracy of a fixed point changes when multiple models with increasing sizes are used for predictions which it defines as the learning profile of a point. They show that there can be points which are positively correlated with the average accuracy of models and there are certain points which are negatively correlated. They also observe that there are differences between pretained models and randomly initialized models. In particular, the number of points with non-monotone accuracy is much lower for pretained models  as compared to randomly initialized models.
strength_and_weaknesses: The idea of studying the accuracy of fixed points with respect to changing models is interesting. There exists points which have a negative correlation with average accuracy is quite interesting. The observation about the difference between pretrained models and randomly initialized models is also quite surprising and would be interesting to look into. 

However, I feel that some of the previous works have not been properly cited. [1] already had this observation that a few points are misclassified by larger models but correctly classified by smaller models and hence, this observation is not really new. The authors cite this paper but it would be good to state the differences more clearly.

[1] Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level
clarity,_quality,_novelty_and_reproducibility: The paper is well written and clear and there are some novel observations.
summary_of_the_review: The extensive study of point wise profiles seems interesting and the observation about the difference between pretrained and randomly initialized models is interesting.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies a new performance measuring method that measures the performance of a collection of models when evaluated on a single input point. The authors compare the models' average performance on the test distribution and their pointwise performance on an individual point. The empirical results show that for some individual test points, the average performance has a weak and even negative correlation to the performance on the individual point. Then the authors construct a dataset called CIFAR-10-NEG out of CIFAR-10 so that accuracy on CIFAR-10-NEG is negatively correlated with the accuracy on CIFAR-10 test.
strength_and_weaknesses: Strength: the paper is novel and the demonstration is pretty clear. The authors provide a lot of figures that are very helpful to understand the paper.

Weakness: 

1. The explanation of why such a negative correlation phenomenon appears is not provided. I suspect that the existence of this phenomenon may depend on the specific models and dataset. A possible explanation might be that the models do not have enough many parameters and thus do not have the capability to fit all samples in the dataset. In other words, the models have to sacrifice the performance on a small portion of data to get a better overall average performance. If so, then I am afraid this phenomenon is less interesting.

2. It may be hard to identify the different types of points mentioned on page 6 beforehand since multiple models need to be trained and evaluated on each sample. This process may take a very long time, yet is not helpful to improve the average performance.
clarity,_quality,_novelty_and_reproducibility: This paper is novel and clearly written. The overall quality of this paper is high.
summary_of_the_review: I like this paper in general, due to the novelty of the topic and the new phenomenon. However, I do have some concerns about whether such a negative correlation phenomenon is common in general or only a corner case for specific models and datasets.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In their manuscript entitled, "Deconstructing distributions: a pointwise framework of learning", the authors present an approach to identifying a certain class of outliers in training sets: those points for which improvements to the global model accuracy correspond to degradation in individual accuracy. 

strength_and_weaknesses: Perhaps the greatest strength of this paper is the comprehensiveness of its investigation in terms of considering a good selection of the relevant literature and presenting a decent slice of empirical examples and theoretical considerations.  At face value the system presented for characterising the outlying data points and/or understanding the model families considered seemed novel and interesting to me; however, on further considering I found myself wondering whether the opposite was true.  I give the following criticisms from the perspective of a statistician who works with large datasets and complex models, not as a practitioner of the particular deep learning models for (e.g.) image classification to which this manuscript is addressed. 

That the 'fitting' (generally considered) of certain points in a dataset degrades as the overall performance improves is in the most general sense a trivial observation.  Even in a well-specified model there will be expected that some data points are from the tails of the distribution and as we learn to represent the distribution overall these tail points naturally become 'less expected' under the overall distribution.   More relevant is the miss-specified cases where we might think of these as contaminating data points for instance, e.g. long tailed or shot noise not expected under a white noise assumption; in the literature on 'robustness' and 'robust regression' we have many ways to classify these so-called 'leverage' points and quantify their influence on the model fit (e.g. Rousseeuw & Hubert, 2011).  Similarly, in the Bayesian literature we have many visualisations and posterior predictive checking methods to do the same and stimulate model development (e.g. Gabry et al., 2017). 

Admittedly, in these examples above, we are primarily considering these points with regard to the best-performing model (e.g. in a Bayesian sense, at the empirical Bayes hyper-parameter estimate), rather than in terms of a sequence including a long tail of poorly performing models.  To this end I suppose the crux of assessing this paper must come down to convincing the reviewer/reader that there is additional value in constructing these curves of training behaviour as a function of global model accuracy rather than just looking at which points are hard to fit for the best trained models (ie., best overall accuracy).  Here I would be interested in the authors' reply and the other reviewers' thoughts. 

clarity,_quality,_novelty_and_reproducibility: The clarity is overall high, quality is high, novelty is uncertain, reproducibility should be high.

summary_of_the_review: An extensive and capable investigation, which might be highly interesting although I reserve judgement for now!
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This is an empirical paper that proposes a new way to examine the predictions of machine learning models. In particular, it proposes to compare how well a variety of models predict a single, fixed test point. The authors note that models have a few surprising features with respect to this measure: in commonly used datasets, a large number of datapoints can have prediction quality negatively correlated with overall accuracy on the model's training set, and that pretraining often removes this negative correlation. The authors also note that various simplified theoretical models of machine learning training do not capture this behavior.
strength_and_weaknesses: **Edit 10-29-2022 after reading other reviews.** I think reviewer diDC brings up a really good point that I think the authors should really address -- what about the behavior observed in this paper is dissimilar from more basic situations? E.g. take the fitting of the MLE of the mean of a Gaussian with gradient descent (I know you can just take the mean and skip gradient descent, but the point is to get a series of better and better "models"). If you randomly initialize at one of the datapoints, it a reasonable fraction of datapoints will be negatively correlated with overall "model" performance (basically, everything that lies further away from the mean than the initialization). Is there more complex behavior going on that this toy example can't capture? If so, do the authors know if this more complex behavior will / will not show up in basic regression models (linear / logistic regression)?


-------------------

This paper is a little different from many machine learning papers in that it does not prove any new theory, it does not provide state of the art empirical results, and it does not answer any open questions in the field. What it does provide is some new interesting / mysterious empirical observations about the behavior of machine learning models that (to my knowledge) were not previously well known; I especially appreciated the fact that the authors have found some new ways in which toy theoretical models for explaining machine learning do not match actual machine learning models. From that perspective, I think this is a solid paper that could lead to thought provoking discussions at the conference. 

I had a few scattered issues with the current draft of the paper:

- The paper only constructs families of models with increasing training accuracy by using more training time. The authors argue that other ways of constructing families of models (e.g. more complex models / more training data) might be expected to perform similarly. But there aren't experiments in the paper to demonstrate this, and I think this limitation should be emphasized more in the paper.
- I think the definitions of easy / hard / compatible / non-monotone points should go before the discussion of those different types of points. I also felt the enumerated discussion of these points was a little unnecessary, and could be cut to move more important things into the main text.
- Model similarity uses $L_1$ distance, whereas the definition of easy / hard / compatible uses $L_2$. Why the difference?
- Right under Fig 7, there is a reference to Figure 7(b). Should this be 7(c)?
- Why was 0.1 chosen as a threshold for nmono$(P_z)$?

A few comments on Appendix C
- I thought the results in here were pretty interesting. It might be worth adding a paragraph or two in the main text to summarize these results  (currently references to them are a little scattered and unclear).
- I don't think $H$ was ever defined as the entropy.
- "... obtained by sampling $z_1, \dots, z_n$ and letting $p_n(y \mid z_1, \dots, z_n)...$ I think you mean to condition on $x$ as well here?
- Is Lemma C.2 meant to hold for all $x$? This should be stated explicitly.
- "we assume that the posterior correctly models the world, that is..." This is not always true (I would say it is almost never true). It could be true asymptotically as $n \to \infty$ if the model's likelihood is correctly specified. I think this could use some more discussion.
- "Hence $p_{n+1} = p_z$ with probability $\alpha_z$." I don't think there's really a sense in which a distribution equals another distribution with a certain probability; the distributions are equal or they are not. This sentence doesn't seem necessary for the rest of the proof in any case.
clarity,_quality,_novelty_and_reproducibility:  I think the paper is very well-written and easy to understand (except for a few issues in Appendix C raised above). As far as I know, the results in the paper are novel.
summary_of_the_review: Overall, I think this paper demonstrates a few interesting empirical results, and would be a good addition to the community.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: In this paper, the authors take a rather untraditional perspective and approach:  to study a fix data point's profile (assuming there are multiple models with increasing sizes are trained and used for predictions), which is the the relationship between models' average performance on the test distribution and the models' performance on this individual point. Interestingly, the authors show that there can be points which are positively correlated or negatively corrected with the average accuracy of models. Comparing between pretrained models and randomly initialized models, the authors observe in experiments that the number of points with non-monotone accuracy is much lower for pretained models as compared to randomly initialized models.

The committee appreciates the clarity, organization, and quality of the writing/demonstration. The perspective is rather novel. The investigation conducted in the paper is comprehensive and the empirical phenomena presented are interesting and novel. 

However, there are concerns about missing explanations or insights on explaining the observed phenomena, as well as concerns about lack of discussion on how to efficiently apply this new perspective into model developments to improve prediction. 

justification_for_why_not_higher_score: However, there are concerns about missing explanations or insights on explaining the observed phenomena, as well as concerns about lack of discussion on how to efficiently apply this new perspective into model developments to improve prediction. 
justification_for_why_not_lower_score: The committee appreciates the clarity, organization, and quality of the writing/demonstration. The perspective is rather novel. The investigation conducted in the paper is comprehensive and the empirical phenomena presented are interesting and novel.
Decision: Accept: poster