Text Characterization Toolkit ( TCT ) (2022)

Abstract: We present a tool, Text Characterization Toolkit (TCT), that researchers can use to study characteristics of large datasets. Furthermore, such properties can lead to understanding the influence of such attributes on models’ behaviour. Traditionally, in most NLP research, models are usually evaluated by reporting single-number performance scores on a number of readily available benchmarks, without much deeper analysis. Here, we argue that – especially given the well-known fact that benchmarks often contain biases, artefacts, and spurious correlations – deeper results analysis should become the de-facto standard when presenting new models or benchmarks. TCT aims at filling this gap by facilitating such deeper analysis for datasets at scale, where datasets can be for training/development/evaluation. TCT includes both an easy-to-use tool, as well as off-the-shelf scripts that can be used for specific analyses. We also present use-cases from several different domains. TCT is used to predict difficult examples for given well-known trained models; TCT is also used to identify (potentially harmful) biases present in a dataset.

## introduction nlp technology has progressed tremendously over the recent decades with significant advances in algorithms and modeling. yet, by comparison, our understanding lags behind significantly for datasets (including all datasets types in the model life cycle: training, validation, evaluation) that contribute to model performance. this is mostly due to the lack of frameworks, methods, and tools to draw insights into datasets, especially at scale. most nlp models, to date, are evaluated using a relatively small number of readily available evaluation benchmarks, that are often created automatically, or via crowd-sourcing (e.g. @xcite @xcite . it is well-known that most popular (evaluation) datasets are rife with biases, dataset artefacts and spurious correlations, and are prone to be solved with shortcuts @xcite . presenting models with adversarial examples for which those biases or correlations do not hold, often results in stark performance drops (e.g. linzen, 2020; @xcite @xcite @xcite . at best, using datasets with such known issues might result in overestimation of a models' capability on the task in question, which may not be reflective of how well they can execute this task in more realistic scenarios. more worrying, however, is that training or finetuning on datasets that contain biases and artefacts may result in models implementing undesired, biased behaviour (e.g. @xcite . additionally, datasets are usually treated as homogeneous collections of text, performance for which is captured in a single number -even though there is often a substantial difference between the difficulty/complexity of different examples in a dataset (e.g. @xcite . research papers rarely report thorough analysis of performance broken down by characteristics of the data set examples ignoring underlying patterns performance numbers may reflect. the problem is exacerbated by the pervasiveness of benchmarks coupled with a leaderboard competitive culture, where what counts most is system rank. in part, this may be due to the fact that deeper analysis of results -especially when a number of different datasets is involved -is complex and timeconsuming, and there are no standard frameworks or protocols that practitioners can resort to. the problem is even more pervasive, where we curate datasets for development and evaluation. how we curate, create, select data plays a critical role in understanding our models. many nlp models (even beyond text) require up/down sampling of specific types of data. these processes should rely on principled characterization of data for any given model. to this end, we believe that the existence of a standard toolkit that provides an easy to use set of tools and metrics allowing researchers to analyze and systematically characterize datasets, at scale, involved in the model life cycle, while gaining insights into the relationship between model performance and data properties could become more common place. in this paper, we introduce the text characterization toolkit foot_0 (tct), which aims to enable researchers to gain a detailed understanding of the datasets and models they create -with minimal effort. tct is inspired by the coh-metrix toolkit @xcite , a collection of over 100 diverse text characteristics intended for use for text analysis in various applications. tct offers these capabilities at scale by design. while tct can process a dataset of 20000 paragraphs in less than a minute on a macbook pro laptop, the very same library, for instance, can also be used as part of a distributed pyspark pipeline to compute text characteristics for a full snapshot of common crawl (3.1b web pages) in a matter of hours. while text characteristics in tct are currently implemented for the english language only, the framework is designed to scale to any new language via configuration of new resource files and spacy backends. in this paper we present: 1. a repository of text metrics that can help reveal (latent) patterns in datasets coupled with model performance on these datasets; 2. a set of off-the-shelf analysis tools that researchers can use in a simple notebook to study properties of the dataset and the influence of those properties on model behaviour; 3. a framework that enables the community to share, reuse and standardize metrics and analyses methods/tools; 4. use cases that demonstrate the efficacy of tct in practice covering language model prompting, translation, and bias detection. with these contributions, we aspire to help the nlp community in particular and the ai community at large improve how we assess nlp models, and get closer to a scenario where providing detailed results' analyses becomes the standard for nlp research. equally important, we believe that tct could be an effective tool for data selection for both training and evaluation, in particular at scale. ## [tct] text characterization toolkit tct consists of two main components: ## example use cases in order to demonstrate the ability of tct to produce meaningful and actionable insights, we provide 3 examples of its use on real world data. for each one of these use cases, a thorough description of the experimental setup and results is included in appendix b and reference notebooks are provided in the examples directory of the tct repository. predicting accuracy of opt baselines open pre-trained transformer language models (opt, @xcite ) are a sequence of publicly released llms that span model scales from 125m to 175b parameters. while scaling thees models lead to significant gains on numerous benchmarks, they still occasionally produce results that would seem trivially wrong to any human. with the help of tct, we attempted to better understand the robustness of these models quantitatively. we use the multi-variable logistic regression analysis tool to fit a model that predicts the accuracy of the 6.7b parameter opt model on the hellaswag common-sense inference task @xcite based on simple characteristics such as mean word length and concreteness. using this model we can identify subsets of the test data with model accuracy as low as 40% and as high as 90% -shown on figure 3 . ## related work there exist several tools that offer similar functionality to tct in some specific respects. data-lab @xcite is a tool for detailed data analysis that, among other things, allows users to inspect datasets through the lens of a few text characteristics such as text length, lexical diversity and gender-related features. the know your data foot_1 tool allows for inspection of image data, it surfaces spurious correlations, biases and imbalances in datasets. however, both tools do not connect model behavior to properties of datasets. @xcite predicts overall hardness of classification datasets based on label statistics and a few text characteristics such as readability and lexical diversity. explainaboard @xcite focuses on model performance analysis and provides a model performance breakdown by simple attributes of data points such as text length, providing a functionality most similar to our work. our toolkit distinguishes itself by including a much wider variety of text characteristics. as a concrete example, explainaboard does not provide features similar to wrdprp2 (personal pronoun incidence), wrdpolc (mean word polysemy) or dessc (number of sentences per text), each playing an important role in our chosen use cases analyses described in §3. tct also features a multivariable analysis tool that can identify larger variations in model performance compared to simpler, single-variable analyses. by packaging our toolkit as a simple python library used in notebooks -in contrast to the previously described feature-rich systems -we also intend to minimize the effort needed to both use it as well as contribute to it (eg. crowd sourcing more functionality). the coh-metrix tool @xcite collected the most diverse set of text characteristics to our knowledge, designed for various use cases in linguistics and pedagogy. however, the tool, developed in 2004, is slow as it is designed to process a single document at a time, relatively difficult to access, and the underlying word databases are outdated. our toolkit aims to make a subset of the coh-metrix metrics easily accessible to the nlp community, while simultaneousy addressing the scale impediment noted in the original coh-metrix tool. ## future work as illustrated in §2 we envision tct to be a framework and an associated tool that allows for community contributions, crowdsourcing even more functionality and use cases. future work involves usage of the tool: firstly, we encourage creators of new datasets to use tct as a data annotation tool, to extract a wide range of dataset statistics in a straightforward manner, and report about them in academic publications for transparency. such statistics could be included in datasheets and data cards @xcite , and they can aid in outlier detection during data (pre-)processing and cleaning. we also prompt dataset creators to perform statistical analyses capturing which features are pre-dictive of the gold targets before further training computational models, to ensure one is aware about potential shortcut learning opportunities due to biases in the dataset. naturally, not all correlations are bad or avoidable -e.g. consider sentences containing the word 'fantastic' that are likely to have a positive label in sentiment analysis -but others are good to be aware of when working with a dataset -e.g. consider a natural language inference task where all sentences with the label 'entailed' have an atypical average word length. such analyses could be included in a 'cautions' section with a dataset's release. a third type of usage would be by owners of new models, that, on the one hand, use tct to measure whether some dataset characteristics are predictive of success and failure by their model, and, on the other hand, provide performance on subclasses of samples. one may already know that model performance is lower for longer sentences, but what about performance on different readability classes, classes with varying amounts of causal connectives or different ratings for syntactic complexity (e.g. synle)? tct will help answer those questions. understanding how the model performance fluctuates for different data subsets provides further understanding in model robustness, and can, in turn, improve datasets' quality if model owners report back on biases identified in datasets.