Tab-Cleaner: Weakly Supervised Tabular Data Cleaning via Pre-training for E -commerce Catalog (2023)

Abstract: Product catalogs, conceptually in the form of text-rich tables, are self-reported by individual retailers and thus inevitably contain noisy facts. Verifying such textual attributes in product catalogs is essential to improve their reliability. However, popular methods for processing free-text content, such as pre-trained language models, are not particularly effective on structured tabular data since they are typically trained on free-form natural language texts. In this paper, we present Tab-Cleaner, a model designed to handle error detection over text-rich tabular data following a pre-training / fine-tuning paradigm. We train Tab-Cleaner on a real-world Amazon Product Catalog table w.r.t millions of products and show improvements over state-of-the-art methods by 16\% on PR AUC over attribute applicability classification task and by 11\% on PR AUC over attribute value validation task.

## introduction product catalogs are widely used by e-commerce websites to organize product information @xcite . they can be conceptualized as wide tables where each row corresponds to a product and each column corresponds to an attribute ( due to product catalogs' wide tabular format and rich textual content, attribute cleaning poses a number of challenges as we outline below. c1: product catalogs are structured tables with unstructured textual values. errors in product catalogs are indicated by column-wise, rowwise, and table-wise inconsistencies ( c2: attributes in product catalogs are strongly correlated with each other. for example, in c3: product catalogs are extraordinarily wide. considering that some attributes of product catalogs may be text-heavy (e.g., product titles and descriptions are often very long.), a superlong sequence will arise from concatenating all textual attributes of a specific product. existing table representation models @xcite restrict input sequence length to a certain budget (e.g., 512) by truncation, which will inevitably cause information loss. to address the above challenges, we present tab-cleaner, a transformer model with a hierarchicalattention mechanism and trained with the pretraining / finetuning paradigm to facilitate data cleaning over text-rich tabular data for ecommerce catalog. our proposed model is generic. it applies not only to the product domain but also excels in other domains which involve text-rich tabular data. in summary, this paper makes the following contributions. product title product category flavor ingredient color size r1 brand a tortilla chips spicy queso, 6 -2 oz bags † chips-and-crisps spicy queso ground corn, chipotle pepper powder, paprika extract, spices -6 -2 oz bags r2 brand b bean chips spicy queso † , high protein and fiber, gluten free, vegan snack, 5.5 ounce (pack of 6) chips-and-crisps cheddar navy beans, cayenne pepper, paprika extract, dehydrated spices -5.5 ounce (pack of 6) r3 brand c organic honey, blossom, 17.6 ounce † honey blossom 100% pure raw honey straight from the hive -17.6 ounce r4 brand d bpa free no spill sippy cup, orange (9 ounce) † baby-drinkware orange -orange 9 ounce r5 brand f women's spa studio green tea eye pads 2 pack-each contains 5 treatment (total 10 treatments) † green-teas green tea aloe, camellia sinensis leaf extract, panax ginseng root extract green total 10 treatments † we mask the brand of the products to avoid revealing sensitive information. ## problem definition given a product catalog table t , each row corresponds to a product (p i ) and each column corresponds to an attribute (a j ). cell t ij is the value of attribute j of product i containing a list of tokens. attributes in product catalog data can be broadly divided into two classes: we formally define the problem of data cleaning over the product catalog table as follows: given: a product catalog table t , identify: incorrect cells about feature attributes {t ij } p @xmath0,a @xmath1, we follow the pre-training/finetuning paradigm to alleviate the need for large-scale labeled data for data cleaning. tab-cleaner is first pre-trained on an unlabeled product catalog corpus with tabular structure-aware pre-training objectives carefully designed to capture the tabular structure. then, tab-cleaner fine-tunes the model using manually curated labeled data. ## 175 we demonstrate that tab-cleaner is effective in detecting both inapplicable attributes and incorrect attribute values in experimental studies. ## experiments in this section, we evaluate tab-cleaner over two different data cleaning downstream tasks on realworld amazon datasets. ## conclusion we have proposed tab-cleaner, a transformerbased model designed specifically for data-cleaning tasks on text-rich tabular catalog data. it provides a versatile solution for data cleaning tasks by efficiently handling the unique challenges posed by text-rich tabular catalog data. to enhance the efficiency of training and reduce memory consumption, we have introduced a novel hierarchical attention mechanism. this mechanism enables a sparse attention pattern, allowing for the effective processing of long sequences. we train tab-cleaner on a real-world amazon product catalog w.r.t millions of products and show that we can improve over sota methods greatly. cls col tok 0 tok1 col tok'0 tok'1 col tok''0 tok''1 ... ... ... tok2 tok3 tok'2 + + + + + + + + + + + + + token embeddings pos0 pos0 pos1 pos2 pos0 pos1 pos2 pos0 pos1 pos2 ... ... ... pos3 pos4 pos3 + + + + + + + + + + + + position embeddings col0 col0 col0 col1 col1 col1 col2 col2 col2 ... ... ... col0 col0 col1 + + + + + + + + + + + + h h col h c col h c ... ... ... c c c column embeddings header/cell embeddings cls + cls + col e [cls] e [col] e0 e1 e' [col] e'0 e'1 e'' [col] e''0 e''1 ... ... e2 e3 e'2 tab-cleaner ... o[cls] o[col] o1 o2 o' [col] o' 1 o'2 o'' [col] o'' 1 o''2 ... ... o3 o4 o'3 ... product representation attribute representation [cls] [col] product title: brand a tortilla chips ... [col] flavor: spicy queso ... [col] ingredient: ground corn ... r 1 context attribute 1 feature attribute 1 feature attribute 2 figure 5: example input of tab-cleaner. tab-cleaner flattens each row in the input table into a sequence of tokens. the token embeddings are combined with additional embeddings to capture tabular structure.