R ep M atch: Quantifying Cross-Instance Similarities in Representation Space (2024)

Abstract: Advances in dataset analysis techniques have enabled more sophisticated approaches to analyzing and characterizing training data instances, often categorizing data based on attributes such as “difficulty”. In this work, we introduce RepMatch, a novel method that characterizes data through the lens of similarity.RepMatch quantifies the similarity between subsets of training instances by comparing the knowledge encoded in models trained on them, overcoming the limitations of existing analysis methods that focus solely on individual instances and are restricted to within-dataset analysis.Our framework allows for a broader evaluation, enabling similarity comparisons across arbitrary subsets of instances, supporting both dataset-to-dataset and instance-to-dataset analyses. We validate the effectiveness of RepMatch across multiple NLP tasks, datasets, and models. Through extensive experimentation, we demonstrate that RepMatch can effectively compare datasets, identify more representative subsets of a dataset (that lead to better performance than randomly selected subsets of equivalent size), and uncover heuristics underlying the construction of some challenge datasets.

## introduction contemporary machine learning models are deeply influenced by the datasets on which they are trained. the characteristics of a dataset, encompassing the quality and diversity of its instances, are critical in shaping a model's learning effectiveness and its capability to generalize. recent advancements in the field have led to the development of methodologies that facilitate the analysis and categorization of data instances based on specific attributes, notably "difficulty" @xcite @xcite , as well as other attributes, such as noisiness, atypicality, prototypicality, and distributional outliers @xcite . these methodologies often involve ranking or categorizing training instances based on specific attributes to identify types that may require specialized processing or treatment. one possible application is to detect mislabeled or noisy examples; removing these from the training data can lead to more effective training @xcite . additionally, these analysis methods are instrumental in studying and uncovering dataset artifacts @xcite . despite their contributions, existing methods are often limited by their focus on individual instances without the capacity to evaluate subsets of data as a whole. moreover, they are generally confined to analyses within a single dataset, lacking the ability to compare across different datasets or perform comprehensive cross-dataset evaluations. in response to these limitations, we introduce a novel technique, called repmatch, that offers a refined lens for the analysis: quantifying the similarities between subsets of training instances from the perspective of the models trained on them. specifically, we measure the similarity between two subsets, s ⊆ d and s ′ ⊆ d ′ of the training datasets d and d ′ (where d and d ′ could be the same dataset), by comparing the models trained exclusively on each subset. the subsets are deemed similar if the representation space learned by the model trained on s closely aligns with that learned by the model trained on s ′ . this reformulation overcomes previous limitations by enabling the analysis and evaluation of similarities among arbitrary subsets of instances-from individual examples to entire datasets-across varied sources. specifically, rep-match facilitates: comparing two models-particularly modern, heavily parameterized ones with expansive weight matrices-presents a significant challenge. to address this, we constrain the set of trainable parameters, i.e., the updates in the representation space, by leveraging low-rank adaptation @xcite . lora efficiently captures changes in a weight matrix through a low-rank matrix, primarily to expedite the fine-tuning process. by encapsulating all significant training-induced changes within a low-rank matrix, we can quantify the similarity between two models through a direct comparison of the corresponding changes in their low-rank representation spaces. we have validated the efficacy of repmatch through a series of experiments across various nlp tasks, datasets, and models. the results demonstrate that lora matrices exhibit significant similarities among similar tasks, underscoring the reliability of repmatch. additionally, for each model, we identified a small yet representative subset within each dataset; models trained on these subsets consistently outperform those trained on comparably sized random subsets. in a definitive demonstration of cross-dataset utility, repmatch successfully uncovers heuristics used in the automatic construction of the hans challenge dataset. ## related work quantifying the similarity between two distinct datasets is a well-researched topic. the theoretical concept of data similarity is linked to the traditional kullback-leibler divergence @xcite , a non-symmetric measure that quantifies the dissimilarity between two probability distributions. empirical measures like the maximal mean discrepancy @xcite have also been employed; mmd compares the means of samples drawn from two distributions in a highdimensional feature space. building on the theoretical concepts, researchers have sought practical methods to estimate task difficulty and dataset similarity. @xcite utilized an information-theoretic approach to estimate task difficulty, demonstrating a strong correlation between their introduced hardness measure and empirical difficulty on transferability. a similar measure was proposed by @xcite to quantify similarity between datasets. moving towards more empirical settings, @xcite presented a method to predict interdataset similarity using a set of pre-trained autoencoders. their approach involves inputting unknown data samples into these auto-encoders and evaluating the differences between the reconstructed outputs and the original inputs. while effective, this method requires additional computational resources and may be sensitive to the randomness inherent in the training environment. our method addresses these challenges by requiring no additional computation beyond regular fine-tuning with lora and remains robust to training randomness. unlike previous approaches, rep-match does not impose constraints on the size of the subsets it compares, which allows it to be categorized under data selection research. this flexibility contrasts with methods like that of @xcite , who used training dynamics to divide a dataset into subsets of easy-to-learn, hardto-learn, and ambiguous instances. their method has limitations in analyzing individual instances or performing cross-dataset analysis. the metric presented by @xcite can be used to quantify the complexity of individual instances relative to a specific distribution, which is useful for comparing datasets or subsets within a single dataset. however, unlike repmatch, this technique can not be used for comparisons of instances or segments across different datasets. in the realm of data selection, a stream of prior research has aimed to find subsets of training examples that achieve performance close to training on the full dataset by using gradient information @xcite @xcite . recently, @xcite proposed a method to estimate the influence function of a training data point to identify influential data in an instruction-tuning setting. while these methods focus on optimizing training efficiency, they may not directly address the comparison of dataset similarities. as similar if a model trained on s 1 (denoted as m s 1 ) exhibits a representation space akin to that of a model trained on s 2 (m s 2 ). during standard fine-tuning, alterations to a specific weight matrix w i (a specific weight matrix in layer i) are captured by ∆w i , also known as the adaptation matrix. after fine-tuning, the updated model weights are then represented as ŵi = w i + ∆w i . these adaptation matrices are responsible for extracting task-specific features from the input and incorporating them into the pre-trained weight matrices. since the pre-trained weights w i remain constant across both models, comparing the representation spaces of m s 1 and m s 2 effectively boils down to analyzing the differences in ∆w i . the challenge in comparing models arises from the substantial size and high dimensionality of the weight matrices, particularly in modern language models. to manage this complexity, we propose using the lora method to encapsulate the ∆w i matrices in a low-rank format. in the following sections, we will provide a brief introduction to the lora method and explain how it facilitates our comparison of adaptation matrices between models. ## analysis possibilities using repmatch the repmatch similarity metric is unconstrained by the size or origin of the subsets, thus facilitating its application in a multitude of scenarios. for instance, it enables comparisons between individual instances and an entire dataset, or between subsets from distinct datasets. in this section, we demonstrate the reliability of repmatch for analysis in two of the possible scenarios: dataset-level and instance-level. to establish this, it is necessary to show that repmatch is robust against the stochastic nature of the training environment. here, the specific focus is on the training seed, verifying the extent to which repmatch is affected by alterations in the initial point of the training. in this section, we first demonstrate that rep-match between two models trained on the same dataset with different random seeds is orders of magnitude higher than that of a model with partially random weights. then, we show that the rep-match between two models trained on a specific instance with different random seeds is higher than the average repmatch between random instances of the same dataset. for both these experiments, we used the sst-2 sentiment analysis dataset @xcite . ## experiments in this section we show that datasets related to a specific task exhibit significantly higher repmatch scores than datasets associated with unrelated tasks (section 5.2). we also carry out an experiment in which we isolate a small subset of representative instances with the highest repmatch scores (to the whole dataset). notably, a model trained on this subset consistently outperforms one trained on a randomly selected subset of the same size (section 5.3). finally, in section 5.3.1, we show that repmatch can be used across various datasets to identify heuristic patterns. ## the effect of rank as explained in section 3.1, the lora method constrains the weight update matrices, ∆w r i s, to have a low rank of r. in section 5.3, the rank was set to 1. here, the instance-level experiments were repeated with a rank of 4 instead. since computing repmatch with higher ranks requires significantly more computational time, a random subset of 25,000 instances was used for each dataset. in the case of sst-5, this is unsurprising, as the dataset itself is inherently imbalanced. when selecting the 100 most representative samples with rank 4, the majority belonged to the dominant label class, in contrast to sampling with rank 1 which was more balanced. these results require further scrutiny since the experiments were conducted on a small portion of the entire dataset. we leave further analysis to future work. ## conclusion in this study, we approached the problem of dataset analysis from a unique perspective. we proposed a method to identify similarities between subsets of training instances by examining the similarities within the representation space of models trained on them. we overcame the challenges of complexity and heavy parameters of language models by utilizing the lora method to constrain changes in the representation space. although we employed lora, alternative parameter-efficient fine-tuning methods @xcite that limit weight updates might be beneficial in this setting. our findings suggest that repmatch can be employed to compare tasks and datasets, conduct instance-level analysis to discover heuristics in a dataset, and perform subset analysis to identify a smaller subset that achieves reasonable performance and outperforms a randomly selected subset of the same size. the experiments demonstrated that the proposed method can be utilized in a variety of situations and is not limited by the size of the subset or its domain. repmatch proves useful for comparing tasks and datasets, conducting instance-level analysis to uncover dataset heuristics, and identifying high-performing subsets. we hope the technique opens new avenues for analysis of datasets and models, from the new viewpoint of knowledge captured by a model from a training instance.