Data Checklist: On Unit-Testing Datasets with Usable Information

Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-23 00:54:28

Abstract: Model checklists (Ribeiro et al., 2020) have emerged as a useful tool for understanding the behavior of LLMs, analogous to unit-testing in software engineering. However, despite datasets being a key determinant of model behavior, evaluating datasets -- e.g., for the existence of annotation artifacts -- is largely done ad hoc, once a problem in model behavior has already been found downstream.
In this work, we take a more principled approach to unit-testing datasets by proposing a taxonomy based on the $\mathcal{V}$-information literature. We call a collection of such unit tests a data checklist.
Using the checklist, not only are we able to recover known artifacts in well-known datasets such as SNLI, but we also discover previously unknown artifacts in preference datasets for LLM alignment.
Data checklists further enable a new kind of data filtering, which we use to improve the efficacy and data efficiency of preference alignment.

Main paper: 
  
 Appendix:  


Decision: This paper proposes a “data checklist” for unit-testing datasets to identify artifacts and ensure the data aligns with model behavior, using a taxonomy based on V-information literature. 

Strengths:
1. Novelty: Reviewers all noted that there has been surprisingly few papers looking specifically at dataset quality, and they appreciated the concept of data checklists for datasets (Reviewer W11a, Vgca).
2. The research artifacts (frameworks and tools) could be beneficial (Reviewer W11a).
3. The research is well conducted, including mathematically well-formulated propositions (Reviewer Vgca) and sufficient verification of known dataset artifacts (Reviewer FFSv).

Weaknesses:
1. The paper used too many acronyms without providing enough background contexts on the datasets and technical methods, which make it a bit difficult to read (Reviewer W11a)
2. There is a lack of comparison with prior work, though reviewers did acknowledge that there tend to be less related work in the field (Reviewer W11a, Vgca, z3Dx)
3. Needs better explanation and definition of terms like word complexity and more context for the tests (Reviewer W11a, Reviewer Vgca).

In their rebuttal, the authors added clarifications to the notations used in the paper, and clarified how they compare to prior work (e.g., discovering dataset artifacts). Their final ratings are all above the acceptance ratio. That said, some reviewers still strongly encouraged the authors to release the code and revise the writing. 

I believe the paper presents a significant contribution to the field with its innovative approach to dataset evaluation, and the presentation issues can be fixed after acceptance.

Reviews: review: paper_summary: paper_summary: This paper proposes the concept of "data checklists", which are taxonomies of unit tests that formalize measures of usable information in a dataset. Although there are similar checklists for testing models, this appears to be the first attempt to develop checklists for the data itself, demonstrating the originality of the idea. The experiments carried out with various popular datasets for tasks such as toxic language detection and natural language inference show the utility of the approach. While the discussion of the methodology seems clear, the paper overall is not particularly accessible to readers lacking in-depth knowledge of popular datasets, which detracts from its value.
strength_weakness: strength_weakness: * The paper assumes a lot of knowledge on the part of the reader about datasets and training techniques. Every dataset is initially referred to as an acronym with no explanation of the content or purpose of the dataset until much later in the paper, if ever (e.g., SNLI, HH-harmless, SHP). (The one exception is DWMW17.) Similarly, many training techniques are never expanded or explained (e.g, RLHF, DPO). I know there are lots of people who are already familiar with all the datasets and all the training techniques, but a person from outside the LLM space who are interested in using the library to learn more about the relevance particular dataset for a new task might really struggle to get out the information they need from the paper.
* The discussion of prior work (which I think should be moved to the front of the paper) talks a lot about model checklists, with only brief references to prior work on finding dataset artifacts, the focus of the paper. I feel like the prior work that is directly related could be highlighted more to provide more context for the work.
* While the results show that the approach is doing something, there is no comparison to any other approach. I know this is a new idea, but is there any prior work that does something similar that could be considered?
questions: questions: On page 6, you mention that word complexity might impact preference, but you never define word complexity. I recommend explaining exactly what is meant, perhaps with examples of words with low and low complexity.
paper_summary: paper_summary: This paper proposes a data checklist. This is a taxonomy of unity tests based on the V-information. This is an interesting topic and worth studying.
Some parts are hard to follow due to the use of several abbreviations.
SNLI, RLHF, and NLU are not defined in this paper.
It is not clear to me how the 10 tests were defined. The paper just presents their formula without further description of the rationale behind each one.
The section Related Work should be relocated at the beginning of the text.
strength_weakness: strength_weakness: -	I think the paper needs more focus. It presents too many different “contributions” in the text. Perhaps it could be extended and published as a journal paper.
questions: questions: For me, the paper misses a better understanding of each test domain. I mean, which range of values is expected for each test?
paper_summary: paper_summary: The paper adopts a V-information-based checklist to evaluate datasets. The authors argue that structured dataset audits should precede model behavior analysis. The proposed taxonomy starts with dataset viability, and then the other dataset feature-based measures take place, e.g., applicability, the selected feature is helpful for the downstream task. The authors frame the checklist as a tool to judge model efficacy and data efficiency. In particular, they show that one can filter some dataset instances using the appropriate V-information-based measure from the checklist without performance degradation on models. The experiments verify some undesired (the ratio of overlap between premise and hypothesis in the case of entailment classification) and desired (insufficiency of the profane words) artifacts in SNLI and hate speech detection datasets, respectively. They further support their claims with some V-information-related attributes (word complexity and response length) in text-to-text datasets, such as preference alignment.
strength_weakness: strength_weakness: The authors claim that they provide a library for easy creation of checklists for text generation as well as classification. However, they do not give the library link for the practical evaluation.
questions: questions: The title does not reflect the content precisely. "An Axiomatic Basis for Dataset Evaluation" would be an alternative.
paper_summary: paper_summary: This paper presents a novel framework for evaluating the quality of datasets used to train languages models using an information-theoretic model. The authors describe this framework and how it can measure applicability, non-exclusivity, necessity and insufficiency of certain features and show application of this to NLI, hate speech and human preferences. They apply this framework to data filtering and show some improvements in task performance on smaller filtered datasets.
strength_weakness: strength_weakness: There are not really any baselines in this work, although I also am not sure what to compare this work too as well, as there are (surprisingly) few papers looking at the quality of data going into LMs. This recent [NeurIPS paper](https://proceedings.neurips.cc/paper_files/paper/2023/file/6b9aa8f418bde2840d5f4ab7a02f663b-Paper-Conference.pdf) may be quite relevant.
The first set of results in Section 4 are really interesting, but the conclusions are based on measuring relative to a value $\espilon$ that has been chosen arbitrarily. It seems strong to make claims such as in Table 2. I am also not sure of what the connection is between the "V-info" in Table 2 and the pass criterion. 
The second set of results (Sec 5) are also very exciting, but are only on two datasets and don't seem to have significance analysis or comparisons to baselines.

## Minor
The references section has not been composed with the same care as the rest of the paper. There are captialisation errors, incomplete citations and references to arXiv for papers that have been published at peer-reviewed venues.
questions: questions: How was $\espilon$ selected and what effect would changing it to 0.1 or 0.001 have?
Can you explain the V-Info scores in Table 2

Comments: title: Acknowledgement
Acknowledgment
thank you
comment: Thank you for the comments. I hope the paper canbe improved with all these new information.
Thanks for the explanations. Great work.
Dear Authors,

Thank you for your response. I couldn't reach the supplementary material due to a technical issue. Providing an anonymized repository would be a good alternative. I already have a high score.

Best,
FFSv
Thank you for your response, authors!

* The page limitation is not really an excuse for not providing enough information for people to understand the paper. Part of writing papers is figuring out how to work within the page limit, which I hope you will do. :)

* I would move the related work to the beginning of the paper. It is difficult to understand the strength and the context of the contribution.

* Ah yes, complexity is definitely not what I thought it was! I think it would be helpful to rename it and explicitly define it in the paper.