DCoM: A Deep Column Mapper for Semantic Data Type Detection

Detection of semantic data types is a very crucial task in data science for automated data cleaning , schema matching , data discovery , semantic data type normalization and sensitive data identification . Existing methods include regular expression-based or dictionary lookup-based methods that are not robust to dirty as well unseen data and are limited to a very less number of semantic data types to predict . Existing Machine Learning methods extract a large number of engineered features from data and build logistic regression , random forest or feedforward neural network for this purpose . In this paper , we introduce DCoM , a collection of multi-input NLP-based deep neural networks to detect semantic data types where instead of extracting a large number of features from the data , we feed the raw values of columns ( or instances ) to the model as texts . We train DCoM on 686,765 data columns extracted from the VizNet corpus with 78 different semantic data types . DCoM outperforms other contemporary results with a quite significant margin on the same dataset achieving a support-weighted F1 score of 0.925 . 1 INTRODUCTION . Robust data preprocessing pipeline enables organizations to make data-driven business decisions . Most unstructured and semi-structured data after initial pre-processing is available in tabular format for further processing . These tabular datasets have to go through a process of data curation and quality check before consumption . Companies apply standard data quality checks/rules on businesscritical columns to access and maintain the quality of data . Additionally , some of these datasets may contain sensitive information e.g . Protected Health Information ( PHI ) , Personally Identifiable Information ( PII ) , etc and need to be masked/de-identified i.e . identifying all columns which are Social Security Numbers , First Names , telephone numbers , etc . The first step in this data curation and quality check process is column level semantic tagging/mapping . There have been many attempts by Ramnandan et al . ( 2015 ) , Limaye et al . ( 2010 ) , Puranik ( 2012 ) , Hulsebos et al . ( 2019 ) , and Zhang et al . ( 2019 ) to automate this process . Traditionally , Semantic tagging is usually done using handcrafted rule-based systems e.g . tools from Google ( 2019 ) , Microsoft ( 2019 ) , etc . In some cases , column descriptions are also available . However , as the data grows exponentially , the cost associated with maintaining a rule-based system also increases . The semantic data type tagging is still largely manual where data stewards manually scan through databases and map columns of interest . Most ML-based approaches e.g . Zhang et al . ( 2019 ) , Hulsebos et al . ( 2019 ) and Pham et al . ( 2016 ) use corresponding metadata , atomic datatypes , column description along with handcrafted features as input data for training the model . We herein , propose DCoM-Deep Column Mapper ; a generic collection of Deep learning based Semantic mappers to map the columns to semantic data types . These models take the raw values of the columns ( or instances ) as inputs considering those as texts and build NLP-based deep learning architectures to predict the semantic type of the columns ( or instances ) . We also extracted 19 engineered features and used those as auxiliary features to DCoM models . We refrained from extracting a large number of features because we wanted to make the DCoM models learn those features along with some interesting high-level features on their own , for better semantic data type detection . As we are exploring NLP in this problem , we make the leverage of using advanced NLP-based deep learning layers or models such as Bi-LSTM , BERT , etc . 2 RELATED WORK . Microsoft ( 2019 ) Power BI and Google ( 2019 ) Data Studio use some regular expression based patterns for dictionary looks ups of column headers and values to detect a limited number of semantic data types . Venetis et al . ( 2011 ) build a database of value-type mappings , then assign semantic types using a maximum likelihood estimator based on column values . Syed et al . ( 2010 ) use column values and headers to build a Wikitology query to map columns to semantic classes . Ramnandan et al . ( 2015 ) use heuristics to first separate numerical and textual types , then describe those types using the Kolmogorov-Smirnov ( K-S ) test and Term Frequency-Inverse Document Frequency ( TF-IDF ) , respectively . Pham et al . ( 2016 ) use slightly more features , including the Mann-Whitney test for numerical data and Jaccard similarity for textual data , to train logistic regression and random forest models . Goel et al . ( 2012 ) use conditional random fields to predict the semantic type of each value within a column , then combine these predictions into a prediction for the whole column . Limaye et al . ( 2010 ) use probabilistic graphical models to annotate values with entities , columns with types , and column pairs with relationships . Puranik ( 2012 ) proposes a “ specialist approach ” combining the predictions of regular expressions , dictionaries , and machine learning models . More recently , Yan & He ( 2018 ) introduced a system that , given a search keyword and set of positive examples , synthesizes type detection logic from open source GitHub repositories . Hulsebos et al . ( 2019 ) built a multi-input deep neural network model for detecting 78 semantic data types , extracting 686 , 765 data columns from VizNet corpus from Hu et al . ( 2019 ) . They extracted a total of 1,588 features for each column to train the model , thus resulting in a support-weighted F1 score of 0.89 , exceeding that of machine learning baselines , dictionary and regular expression benchmarks , and the consensus of crowdsourced annotations . Zhang et al . ( 2019 ) introduced SATO , a hybrid machine learning model to automatically detect the semantic types of columns in tables , exploiting the signals from the context as well as the column values . It combines a deep learning model trained on a large-scale table corpus with topic modelling and structured prediction to achieve support-weighted and macro average F1 scores of 0.925 and 0.735 , respectively , exceeding the state-of-the-art performance by a significant margin . Recently Deng et al . ( 2020 ) used a structure-aware Transformer encoder to capture semantics and knowledge in large-scale data . Wang et al . ( 2021 ) proposed Table Convolution Network and Suhara et al . ( 2021 ) proposed a multi-task learning approach ( called Doduo ) which outperform the existing results in semantic data type detection . Iida et al . ( 2021 ) devised a simple pre-training objective ( corrupt cell detection ) that learns exclusively from tabular data and reaches the state-of-the-art on a suite of table-based prediction tasks . While in this paper we do not experiment with the context information of the column values , our work is mostly aligned to Hulsebos et al . ( 2019 ) . Keeping that in mind , we are planning to research and implement the context of the column values in tables in our DCoM models for our future works . The paper is organized as follows : Section 3 describes the data used for DCoM models . In Section 4 we present the details of data preparation and architecture for DCoM models . Section 5 contains the training , evaluation and inference procedures of DCoM models while Section 6 discussed about the results of extensive experiments . In Section 7 , we talk about some known limitations from the data as well as application point of view and finally , in Section 8 , we present the concluding remarks and some future directions . 3 DATA . We have used the dataset prepared by Hulsebos et al . ( 2019 ) and compared our model performance with them considering their results to be baselines . This dataset contains 686,765 instances with 78 unique classes ( or semantic data types ) . It is divided into 60/20/20 training/validation/testing splits . The instance count for classes varies from 584 ( affiliate class ) to 9088 ( type class ) . The count distribution for classes is shown in Figure 2 of Appendix A . To provide a more clear picture of the data a sample of the dataset is also shown in Table 7 of Appendix A . 4 PROPOSED METHOD . Prior approaches ( Venetis et al. , 2011 ; Limaye et al. , 2010 ) to semantic type detection trained models , such as logistic regression , decision trees , feedforward neural network ( Hulsebos et al. , 2019 ) , extracting various features from the data . We , on the other hand , treated the data like natural language ( text ) and used the data itself as the input to the model . We used a very small number of hand-engineered features unlike Hulsebos et al . ( 2019 ) . Therefore , our DCoM models have two inputs , the values of the instances as text or natural language and hand-engineered features . We present two types of DCoM models based on the way we feed the text input to the model . 4.1 DCOM WITH SINGLE SEQUENCE INPUT . In this subsection , we discuss how the values of each instance can be fed to the model as a single sequence input to the DCoM . Considering the scenario , we can not simply pass the inputs separating values of each input with any separator token . This gives the model wrong information about the sequence of the data resulting in faulty training and poor performance on the unseen data . For example , if we consider the last example from Table 7 and create the input , Deletes the property < SEP > Lets you edit the value of the property < SEP > Script execution will be stopped for the model , the model gets to learn that the value Lets you edit the value of the property has a relative position between Deletes the property and Script execution will be stopped , which is quite wrong because the values in an instance do not have any relative position between them . To mitigate the problem we introduce permutations from mathematics . With permutation , we order r items from the set of ni items . Here ni items are all the values of an instance i , and r ∈ [ 1 , n ] . Doing so , the above instance can be broken down into multiple subsets . A sample of the subset is shown in Table 1 . If we feed all the new instances of the subset to the model , it does not learn any relative positional information of Lets you edit the value of the property with respect to other values in that instance unlike earlier . Therefore , for the value Deletes the property , the model only learns the relative positions of the tokens e.g . Deletes , the , property , etc in a value , but not the relative position of the values in an instance . This helps the model getting the actual information present in the data for accurate prediction on the output . This permutation method also helps in augmenting new instances which help in training the data-hungry deep learning models with enriched data . It is not feasible to generate all the possible permutations before the training because of the huge number of subsets . Instead , during training , we sample r between 1 and n for each of the instances and generate one permutation for each instance . More than one instance can be generated , but training the model for multiple epochs will result same for both cases .

Summary: The goal of the work is to associate semantic data types to table attributes. This task is of importance to data cleaning, schema matching,  etc. The authors claim that existing methods use regular expressions or dictionary lookups, and proposed a learning approach to the problem. They feed raw values (strings) to neural classifiers, and train their models on 686,765 data columns extracted from the VizNet corpus with 78 different semantic data types. The textual values processed by LSTM/Transformer/BERT layers, where in addition the authors use 19 tatistical features from Hulsebos et al. (2019).  Overall, they achieve F1 score of 0.925.  Analysis showed that high performance is obtained for classes that contain a finite set of valid values, e.g. grades or industry.  On the other hand, numerical values or in general values that appear in multiple classes, are more challenging to classify.