SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Aniruddha Sen
University of Massachusetts
Amherst, MA 01003
Christine Task
Knexus Research
Oxon Hill, MD 20745
christine.task@knexusresearch.com
Dhruv Kapur
Ann Arbor, MI 48109
Gary Howarth
Boulder, CO 80305
gary.howarth@nist.gov
Karan Bhagat
The Collaborative Research Cycle (CRC) is a National Institute of Standards and Technology (NIST) benchmarking program intended to strengthen understanding of tabular data deidentification technologies. Deidentification algorithms are vulnerable to the same bias and privacy issues that impact other data analytics and machine learning applications, and it can even amplify those issues by contaminating downstream applications. This paper summarizes four CRC contributions: theoretical work on the relationship between diverse populations and challenges for equitable deidentification; public benchmark data focused on diverse populations and challenging features; a comprehensive open source suite of evaluation metrology for deidentified datasets; and an archive of more than 450 deidentified data samples from a broad range of techniques. The initial set of evaluation results demonstrate the value of the CRC tools for investigations in this field.
Deidentification algorithms take records linked to individuals and attempt to produce data that is dissociated from individuals but remains useful for analysis. Effective deidentification permits organizations to safely share useful information from potentially sensitive data. Such data can be used to train machine learning algorithms; expose fraud, waste, and abuse; improve health outcomes; and much more. Many approaches are available to deidentify data. Some approaches, such as statistical disclosure control, redact or suppress information that is deemed particularly identifying. Other approaches, such as synthetic data algorithms leverage generative models to reproduce sensitive data distributions using new, synthetic records. Differential privacy strengthens deidentification with a rigorous mathematical definition of privacy, producing synthetic data with quantifiable bounds on the influence of any real individual. While deidentification release algorithms may (but not necessarily) improve privacy, they can also distort data by introducing artifacts and bias. Identifying and resolving these issues is important, but it is not trivial to do.
Intuitively, individuals with many unusual values in the data are easier to identify than those with relatively common demographic characteristics. Consider data with two subpopulations of equal size: group A, who are internally homogeneous and have highly-correlated features; and group B, who are internally heterogeneous with relatively independent features. We can infer that group
B will have more uniquely identifiable individuals. Heavy handed deidentification will eliminate or perturb group B to a greater extent than A, at the likely expense of reducing or meaningfully altering the representation of group B. We use subpopulation dispersal to refer to the sparsification of subpopulations, such as that caused by increased feature independence. We believe work on benchmarks containing subpopulation dispersal is imperative to overcoming these risks. Further, we advocate for evaluation metrics that explore the effects of subpopulation dispersal and other challenges induced by real-world data.
In this work we introduce the Collaborative Research Cycle (CRC) a benchmarking effort to compare deidentification methods hosted by the National Institute of Standards and Technology (NIST). The CRC includes target data, evaluation metrics, and a repository containing community-created de-identified data and their evaluation results. Our benchmark dataset contains examples of challenging, real-world conditions, such as subpopulation dispersal (Section 3.1). Together this program supports unprecedented rigorous exploration of deidentification algorithm behavior.
There are several existing synthetic data evaluation libraries, such as SDMetrics and YData (see Table 2 for a larger selection). These tools are focused on evaluating deidentified data with arbitrary schema and providing application-specific feedback. Our effort distinguishes itself by providing specific benchmarking data and a venue to contribute directly comparable samples of deidentification algorithm outputs and their evaluation results. We believe that shared benchmark data alongside common evaluation metrics promote understanding and exploration of a problem by providing common resources, vocabulary, and analytic framework. Thus, we have created the CRC as an arena to test, compare, and discuss varying approaches on equal footing.
We begin with a formal analysis of subpopulation dispersal. This concept underlies one source of tension between diversity, equity, and privacy that impacts all deidentification algorithms operating in real-world conditions. Specifically we show that subpopulations with greater feature independence leads to smaller cell counts in tabular data. We believe benchmark tabular data for deidentification technologies must present subpopulation dispersal to provide a realistically challenging target.
Next, we introduce a benchmark dataset of real demographic data, the NIST Diverse Communities Data Excerpts [1] (the Excerpts). The data are curated from the American Community Survey from the U.S. Census Bureau with 24 features and three geographic samples. We introduce a suite of benchmark software, the SDNist Deidentified Data Report Tool [2], which provides metrology and visualization functions comparing groundtruth and deidentified data. We use SDNist to demonstrate the presence of subpopulation dispersal within the Excerpts.
Finally, we introduce the CRC Data and Metrics Archive, a repository of crowd-sourced, deidentified instances of the Excerpts, all of which are benchmarked with SDNist. Members of industry, academia, and government have contributed more than 450 fully-evaluated entries to the repository, spanning many different deidentification approaches including differentially private techniques [3], cell suppression based  $k$ -anonymity[4], GAN-based synthetic data, and many others. We use SDNist and the archive to illustrate the impact of subpopulation dispersal on deidentification performance.
The CRC is a comprehensive effort to equip algorithm developers, data owners, and theoreticians the tools to evaluate and compare tabular deidentification technologies. All of the data (ground truth and deidentified instances) and evaluation software are open-sourced to allow transparent analysis. Further, the archived data has extensive, standardized metadata to facilitate parsing and comparing approaches, such as by feature subset, algorithm type, etc.
2 Distributional Diversity and Subpopulation Dispersal
We use the term diverse populations to refer to populations containing two or more subpopulations which differ from each other in terms of their feature correlations. By feature correlation we mean the predictive power of a feature value on other feature values for a given record. Intuitively, weaker feature correlations are more challenging for deidentification algorithms to model accurately. Here, we provide a robust formal analysis of these dynamics which helped inspire the Diverse Community Excerpts benchmark data.
Consider data represented in a histogram, with bins counting the number of occurrences of each record type (formally defined below). Individuals in bins with small counts are more uniquely identifiable—there are fewer people like them. Many privacy approaches focus on protecting these
Figure 1: Boxes represent partition of observations by an increasing number of categorical features. Blue group features are highly correlated and orange group features are more independent. The orange group is, therefore, dispersed into small-count bins (in grey). The percentage label shows the relative representation of each group if small count bins were suppressed to protect privacy.
individuals. Traditional statistical disclosure control techniques like  $k$ -anonymity operate by perturbing or redacting records in small bins. Randomization approaches that rely on additive noise, such as differential privacy or subsampling, have much larger relative impact on these small bins. Non-differentially private synthetic data generators may have difficulty modeling sparser areas of the distribution. Thus, deidentified data fidelity is more challenging for individuals in small bins. However these individuals are not necessarily unimportant outliers. Depending on feature independence it is possible for a large subpopulation to become dispersed into small bins.
In diverse data the same schema may disperse one subpopulation and preserve the other, causing the first to potentially be erased by deidentification while the maintaining the second (see Figure 1). Therefore, it is vital to use diverse data to study algorithm behavior. We first introduce two relevant information theoretic tools and our notation. We then formally define dispersal ratio and derive bounds on the relationship between diversity and dispersal.
Definition 2.1 (Entropy). The entropy of a discrete random variable  $X$  is defined as:
$$
\mathrm {H} (X) = - \sum_ {x \in \mathcal {X}} p (x) \log p (x)
$$
where  $\mathcal{X}$  denotes the range of  $X$  [5]. For this paper, we will be considering the empirical definition of entropy or observed entropy from the probability distribution corresponding to a histogram.
Definition 2.2 (Uncertainty Coefficient). The uncertainty coefficient  $U(X|F)$  is an information theoretic metric for quantifying the entropy ( $H$ ) correlation between two random variables  $X$  and  $F$  (in our case an existing feature set  $F$  and a new added feature  $X$ ) [6]. It is a normalized form of mutual information. Note that  $0 \leq U(X|F) \leq 1$ . It is defined as:
$$
U (X | F) = \frac {H (X) - H (X | F)}{H (X)}
$$
An uncertainty coefficient of 1 implies that the random variable  $X$  can be completely predicted by knowing the value of  $F$ , and an uncertainty coefficient of 0 implies that the random variable  $X$  is independent of  $F$ . Thus,  $U(X|F)$  inversely correlates with the independence of  $X$  for some  $F$ . Note that this metric is asymmetric and may not be equal if the variables are exchanged, i.e.  $U(X|F) \neq U(F|X)$ . This metric can be used only for categorical data, which makes it useful for a table-based partition schema where the data is distributed into cells with categorical feature labels.
Consider a population  $P$  of individuals  $i$  distributed in a table-based partitioned schema  $S$ . The proportion of the population  $P$ , with a given assignment of feature values corresponding to a bin in the schema  $S$ , is calculated as  $p(\text{bin}_{S(i)}) = \frac{|\{i| \text{bin}_{S(i)}\}|}{|P|}$ . Let the multivariate distribution of the initial feature set be described by  $F$  in the schema  $S$ , and the univariate distribution of the new feature be described by  $X$  in the schema  $(S + X)$ . The observed entropy  $H(X)$  is dependent on the distribution of  $X$ , which remains the same over this operation for any added feature. We assume the size of the population is much larger than the number of bins, i.e.,  $|P| >> |Range(F + X)|$ .
Definition 2.3. (Dispersal Ratio) Let the dispersal ratio for a population  $P$  with the addition of feature  $X$  be defined as:
$$
D i s p e r s e (S, X, P) = | b i n _ {(S + X) (P)} | / | b i n _ {S (P)} |
$$
where  $\text{bin}_{S(P)}$  is defined as the set of all bins in the histogram corresponding to  $P$  distributed in the schema  $S$ .
Real-world tabular survey data is often comprised primarily of non-ordinal, categorical features (multiple choice answers) that fall neatly into histogram bins. For a sub-population of a fixed size, definition 2.3 ratio captures the increase in the number of histogram bins for the population when a new feature is introduced. Thus, it captures the resulting reduction of the average count per bin and increases to the number of small-count bins. The lemma below provides some basic intuition behind the choice. The main result of this section is given in Theorem 2.3.
Lemma 2.1. An uncertainty coefficient of 1 is equivalent to a dispersal ratio of 1.
$$
U (X | F) = 1 \iff D i s p e r s e (S, X, P) = 1
$$
Lemma 2.2. An uncertainty coefficient of 0 leads to the maximum dispersal ratio.
$$
U (X | F) = 0 \Longrightarrow D i s p e r s e (S, X, P) = | R a n g e (X) |
$$
The above lemmas provide good intuition for our main argument in this section. Let's call these the trivial bounds for dispersal ratio on adding a feature  $X$  to the schema. Moreover, we are also interested in quantifying how different patterns of feature correlations, represented by the uncertainty coefficient, impact the dispersal ratio for values within these bounds. The following theorem allows us to establish bounds on dispersal ratio as a function of the uncertainty coefficient or independence.
Theorem 2.3. Dispersal Ratio is bounded from above and below as function of the independence of the added feature as follows
$$
\frac {| P | \cdot f (u)}{\log (| P |) | R a n g e (F) |} \geq D i s p e r s e (S, X, P) \geq \frac {2 ^ {f (u)}}{| R a n g e (F) |}
$$
where  $f(u) \coloneqq (1 - u)H(X) + H(F)$  with  $u = U(X|F)$ .
Proof. The following is a short sketch of the proof. The complete proof is detailed in Appendix C.2.
Let some arbitrary  $u = U(X|F)$ . Applying the well-known upper bound on entropy [5],
$$
H (X, F) \leq \log_ {2} (| R a n g e (X, F) |) \tag {1}
$$
provides a lower bound on dispersal ratio. Similarly, the observed entropy can be lower bounded by observing that each record in  $S$  discretely contributes to the histogram. This provides an upper bound on the dispersal ratio.
Theorem 2.3 shows that, for some fixed value of entropy of the added feature, the non-trivial upper and lower bounds for the dispersal ratio decrease as the uncertainty coefficient increases, and vice-versa, according to the described behaviour of  $f(u)$ .
Now, we want to compare the effect of adding a new feature  $X_{1}$  or  $X_{2}$  to the schema. Let us assume that  $X_{1}$  is more "independent" than  $X_{2}$  of the distribution of  $P$  in  $S$  (note that this is equivalent to considering a single feature  $X$  and two diverse subpopulations  $P1, P2$  with differing relationships to  $X$ ). We can then say that the uncertainty coefficient  $u_{1} = U(X_{1}|F)$  is lesser for  $X_{1}$  as compared to  $u_{2}$  corresponding to  $X_{2}$ . We can use our results from the above theorem to make this comparison as follows in Theorem 2.4. We define a couple of terms first for ease of notation.
Let the non-trivial lower bound for the dispersal ratio  $(>1)$  on adding feature  $X$  be denoted as
$$
L B \left(\text {D i s p e r s e} (S, X, P)\right) = \frac {2 ^ {(1 - u) H (X) + H (F)}}{| R a n g e (F) |} \tag {2}
$$
Let the non-trivial upper bound for the dispersal ratio  $(< |Range(X)|)$  on adding feature  $X$  be denoted
$$
U B (D i s p e r s e (S, X, P)) = \frac {| P | \cdot (1 - u) H (X) + H (F)}{\log (| P |) | R a n g e (F) |} \tag {3}
$$
Theorem 2.4. Consider two features  $X_{1}$  and  $X_{2}$ , identical in terms of entropy, that can be added to the schema. If  $X_{1}$  has higher independence than  $X_{2}$  with respect to  $F$ , it is equivalent to  $X_{1}$  having a higher LB and higher UB for the dispersal ratio.
$$
U \left(X _ {1} | F\right) \leq U \left(X _ {2} | F\right) \Longleftrightarrow L B (D i s p e r s e (S, X _ {1}, P) \geq L B (D i s p e r s e (S, X _ {2}, P)
$$
$$
U \left(X _ {1} | F\right) \leq U \left(X _ {2} | F\right) \iff U B (D i s p e r s e (S, X _ {1}, P) \geq U B (D i s p e r s e (S, X _ {2}, P)
$$
Proof. This follows from Theorem 2.3. The complete proof is detailed in Appendix C.2.  $\square$
3 Introducing the Diverse Community Excerpts
The Excerpts are in the public domain and were designed to explore algorithm behavior on realistic data with diverse subpopulations (see Figure 2). We selected these data to be tractable, in light of a recurring problem identified in the NIST Differential Privacy Synthetic Data Challenge [7], NIST Differential Privacy Temporal Map Challenge, and the UNECE High-Level Working Group for the Modernisation of Statistics (HLG-MOS) Synthetic Data Test Drive [8], where the target data was too large or complex to identify, diagnose and address shortcomings in the deidentification data. This is a serious problem; consumers of deidentified government data cannot afford to overlook even subtle introductions of bias, artifacts, or privacy leaks. But data properties like subpopulation dispersal can induce defects that are not visible in aggregate utility metrics used by most privacy researchers and data competitions. And deeper exploration is intractable when considering hundreds of features and millions of records. Addressing these issues requires tools designed to make them accessible.
The Excerpts consist of a curated geography and feature set derived from the significantly larger 2019 American Community Survey (ACS) Public Use Microdata Sample (PUMS) [9], a product of the U.S. Census Bureau. The Census Bureau applies privacy measures to the data[10], and no independent privacy risks of this subset of the data. The Excerpts serve as benchmark data for two currently active, open source projects at the National Institute of Standards and Technology (NIST)-SDNist Deidentified Data Report Tool and the 2023 Collaborative Research Cycle (CRC).
The Excerpts' feature set was developed with input from U.S. Census Bureau experts in adaptive sampling design (see [11]). To identify a small set of communities with challenging, diverse distributions, the Excerpts leverage previous work on geographical differences in CART-modeled synthetic data (see [12, Appendix B]). The open source SDNist library which accompanies the excerpts was developed with input from HLG-MOS participants and synthetic data contractors working with the U.S. Census. In this section we provide an overview of the Excerpts; in the next section we demonstrate their ability to identify and diagnose problems in deidentification algorithms.
3.1 Data Overview
Feature Selection The original ACS schema contains over four hundred features, which poses difficulties for diagnosing shortcomings in deidentification algorithms. The Excerpts use a small but representative selection of 24 features, covering major census categories: demographic, household and family, geographic, financial, work and education, disability, and survey weights (discussed in the 'Challenges' below). Several Excerpts' features were not in the original ACS features and were designed to provide easy access to certain information. Population DENSITY allows models to distinguish rural and urban geographies, INDP CAT aggregates industry codes into categories, PINCP DECILE aggregates incomes into percentile bins relative to the record's state, and EDU simplifies school
Figure 2: Effect of feature addition to dispersal rate of the Excerpts data (National sample).
ing to focus on milestone grades and degrees. The Excerpts' repository has detailed information including several recommended feature subsets for use in diverse subpopulation analyses.
Recommended Feature Subsets To support algorithms that require smaller schema and explore the impact of different types and combinations of features, we provide recommended subsets of the 24 features (see Appendix D). Algorithm performance may differ significantly between feature subsets (see Appendix E.4).
Subpopulation Dispersal Figure 2 shows subgroup dispersal by race in the Excerpts data; as new features are added that are more independent for one group and more tightly correlated for other groups, the independent group becomes dispersed across the features space. The experiments in Section 5.1 use the CRC archive to demonstrate the impact of this on algorithm performance for the largest four race + sex subgroups. However, we note there's nothing 'magic' about this choice of demographic subgroups, other population partitions exhibit similar behavior (e.g., partitions by
sex + disability, employment sector). A rigorous multidimensional understanding of the relationship between variation in patterns/strength of feature correlations, variations in distribution density, and pitfalls for deidentification is the future research this benchmarking program is designed to support.
Three Excerpts Data Sets Data sets of differing sizes, reflecting different types of communities, induce different algorithm behavior. To prevent overfitting research conclusions to a particular type of community, the Excerpts contains three target data sets. The "Massachusetts" data contains 7634 records drawn from five Public Use Microdata Areas (PUMAs) of demographically homogeneous communities from the North Shore to the west of the greater Boston area; the "Texas" data with 9276 records is drawn from six racially diverse PUMAs of communities surrounding Dallas-Fort Worth, Texas area. The "National" data with 27254 records is drawn from 20 diverse PUMAs across the United States; it is an expansion of a challenging PUMA set used during the design of the 2020 Decennial Census Disclosure Avoidance System algorithm [12]. The experiments in this paper use the National Excerpts data.
Metadata and Documentation: The Diverse Community Excerpts include JSON data dictionaries with complete feature definitions, a github README with detailed usage guidance, and illustrative "postcard" introductions to the real-world communities in the data.
Challenges: The Excerpts have been designed to be representative of real-world survey data conditions. In addition to diversity, these include other challenging properties: logical constraints between features (e.g.,  $\mathrm{AGE} = 6$  places a constraint on MARITAL STATUS) that can be difficult for synthetic data generators (see Figure 6). Additional modeling challenges include heterogeneous feature types and cardinalities (e.g., DEYE has two categorical values while INDUSTRY has 271, INCOME is an integer while POVERTY INCOME RATIO is continuous). Uneven feature granularity can amplify problems with unequal subpopulation dispersal (ACS 2019 RAC1P uses one code for Asian, but four detailed codes for Native American). We also include the sampling weights that survey data users need to simulate a full population; these lose their original meaning after deidentification changes the data sample, and this is a largely unsolved problem.
Limitations: Although ACS data consumers generally assume the data remains representative of the real population, the ACS PUMS data has had basic statistical disclosure control deidentification applied (as noted above), which may impact its distribution. Additionally, there are shortcomings in the Diverse Community Excerpts that we plan to address in future versions: the Excerpts do not currently include the ACS Household IDs (which would support joining individuals in the same households for social network synthesis), Individual IDs (relevant for reidentification research), or a clear training/testing partition (important for differential privacy research).
4 Introducing the SDNist Deidentified Data Report Tool
The SDNist Deidentified Data Report Tool is a Python library that is undergoing active development. SDNist evaluates fidelity, utility, and privacy of a given set of deidentified instances of the Excerpts and generates human- and machine-readable summary quality reports enumerated and illustrated for each utility and privacy metric. It is comprised largely of fidelity metrics drawn from data stakeholders through the HLG-MOS Synthetic Data Test Drive, and aims to provide a comprehensive view of algorithm behavior. Examples of complete reports, including detailed metric definitions, are linked in the Appendix E. A metric overview is below; usage is demonstrated in Section 5.
Univariate and Correlation Metrics The SDNist report covers univariate feature distributions as well as pairwise feature correlations with Pearsons and Kendall Tau Coefficients.