DCoM: A Deep Column Mapper for Semantic Data Type Detection

Detection of semantic data types is a very crucial task in data science for automated data cleaning , schema matching , data discovery , semantic data type normalization and sensitive data identification . Existing methods include regular expression-based or dictionary lookup-based methods that are not robust to dirty as well unseen data and are limited to a very less number of semantic data types to predict . Existing Machine Learning methods extract a large number of engineered features from data and build logistic regression , random forest or feedforward neural network for this purpose . In this paper , we introduce DCoM , a collection of multi-input NLP-based deep neural networks to detect semantic data types where instead of extracting a large number of features from the data , we feed the raw values of columns ( or instances ) to the model as texts . We train DCoM on 686,765 data columns extracted from the VizNet corpus with 78 different semantic data types . DCoM outperforms other contemporary results with a quite significant margin on the same dataset achieving a support-weighted F1 score of 0.925 . 1 INTRODUCTION . Robust data preprocessing pipeline enables organizations to make data-driven business decisions . Most unstructured and semi-structured data after initial pre-processing is available in tabular format for further processing . These tabular datasets have to go through a process of data curation and quality check before consumption . Companies apply standard data quality checks/rules on businesscritical columns to access and maintain the quality of data . Additionally , some of these datasets may contain sensitive information e.g . Protected Health Information ( PHI ) , Personally Identifiable Information ( PII ) , etc and need to be masked/de-identified i.e . identifying all columns which are Social Security Numbers , First Names , telephone numbers , etc . The first step in this data curation and quality check process is column level semantic tagging/mapping . There have been many attempts by Ramnandan et al . ( 2015 ) , Limaye et al . ( 2010 ) , Puranik ( 2012 ) , Hulsebos et al . ( 2019 ) , and Zhang et al . ( 2019 ) to automate this process . Traditionally , Semantic tagging is usually done using handcrafted rule-based systems e.g . tools from Google ( 2019 ) , Microsoft ( 2019 ) , etc . In some cases , column descriptions are also available . However , as the data grows exponentially , the cost associated with maintaining a rule-based system also increases . The semantic data type tagging is still largely manual where data stewards manually scan through databases and map columns of interest . Most ML-based approaches e.g . Zhang et al . ( 2019 ) , Hulsebos et al . ( 2019 ) and Pham et al . ( 2016 ) use corresponding metadata , atomic datatypes , column description along with handcrafted features as input data for training the model . We herein , propose DCoM-Deep Column Mapper ; a generic collection of Deep learning based Semantic mappers to map the columns to semantic data types . These models take the raw values of the columns ( or instances ) as inputs considering those as texts and build NLP-based deep learning architectures to predict the semantic type of the columns ( or instances ) . We also extracted 19 engineered features and used those as auxiliary features to DCoM models . We refrained from extracting a large number of features because we wanted to make the DCoM models learn those features along with some interesting high-level features on their own , for better semantic data type detection . As we are exploring NLP in this problem , we make the leverage of using advanced NLP-based deep learning layers or models such as Bi-LSTM , BERT , etc . 2 RELATED WORK . Microsoft ( 2019 ) Power BI and Google ( 2019 ) Data Studio use some regular expression based patterns for dictionary looks ups of column headers and values to detect a limited number of semantic data types . Venetis et al . ( 2011 ) build a database of value-type mappings , then assign semantic types using a maximum likelihood estimator based on column values . Syed et al . ( 2010 ) use column values and headers to build a Wikitology query to map columns to semantic classes . Ramnandan et al . ( 2015 ) use heuristics to first separate numerical and textual types , then describe those types using the Kolmogorov-Smirnov ( K-S ) test and Term Frequency-Inverse Document Frequency ( TF-IDF ) , respectively . Pham et al . ( 2016 ) use slightly more features , including the Mann-Whitney test for numerical data and Jaccard similarity for textual data , to train logistic regression and random forest models . Goel et al . ( 2012 ) use conditional random fields to predict the semantic type of each value within a column , then combine these predictions into a prediction for the whole column . Limaye et al . ( 2010 ) use probabilistic graphical models to annotate values with entities , columns with types , and column pairs with relationships . Puranik ( 2012 ) proposes a “ specialist approach ” combining the predictions of regular expressions , dictionaries , and machine learning models . More recently , Yan & He ( 2018 ) introduced a system that , given a search keyword and set of positive examples , synthesizes type detection logic from open source GitHub repositories . Hulsebos et al . ( 2019 ) built a multi-input deep neural network model for detecting 78 semantic data types , extracting 686 , 765 data columns from VizNet corpus from Hu et al . ( 2019 ) . They extracted a total of 1,588 features for each column to train the model , thus resulting in a support-weighted F1 score of 0.89 , exceeding that of machine learning baselines , dictionary and regular expression benchmarks , and the consensus of crowdsourced annotations . Zhang et al . ( 2019 ) introduced SATO , a hybrid machine learning model to automatically detect the semantic types of columns in tables , exploiting the signals from the context as well as the column values . It combines a deep learning model trained on a large-scale table corpus with topic modelling and structured prediction to achieve support-weighted and macro average F1 scores of 0.925 and 0.735 , respectively , exceeding the state-of-the-art performance by a significant margin . Recently Deng et al . ( 2020 ) used a structure-aware Transformer encoder to capture semantics and knowledge in large-scale data . Wang et al . ( 2021 ) proposed Table Convolution Network and Suhara et al . ( 2021 ) proposed a multi-task learning approach ( called Doduo ) which outperform the existing results in semantic data type detection . Iida et al . ( 2021 ) devised a simple pre-training objective ( corrupt cell detection ) that learns exclusively from tabular data and reaches the state-of-the-art on a suite of table-based prediction tasks . While in this paper we do not experiment with the context information of the column values , our work is mostly aligned to Hulsebos et al . ( 2019 ) . Keeping that in mind , we are planning to research and implement the context of the column values in tables in our DCoM models for our future works . The paper is organized as follows : Section 3 describes the data used for DCoM models . In Section 4 we present the details of data preparation and architecture for DCoM models . Section 5 contains the training , evaluation and inference procedures of DCoM models while Section 6 discussed about the results of extensive experiments . In Section 7 , we talk about some known limitations from the data as well as application point of view and finally , in Section 8 , we present the concluding remarks and some future directions . 3 DATA . We have used the dataset prepared by Hulsebos et al . ( 2019 ) and compared our model performance with them considering their results to be baselines . This dataset contains 686,765 instances with 78 unique classes ( or semantic data types ) . It is divided into 60/20/20 training/validation/testing splits . The instance count for classes varies from 584 ( affiliate class ) to 9088 ( type class ) . The count distribution for classes is shown in Figure 2 of Appendix A . To provide a more clear picture of the data a sample of the dataset is also shown in Table 7 of Appendix A . 4 PROPOSED METHOD . Prior approaches ( Venetis et al. , 2011 ; Limaye et al. , 2010 ) to semantic type detection trained models , such as logistic regression , decision trees , feedforward neural network ( Hulsebos et al. , 2019 ) , extracting various features from the data . We , on the other hand , treated the data like natural language ( text ) and used the data itself as the input to the model . We used a very small number of hand-engineered features unlike Hulsebos et al . ( 2019 ) . Therefore , our DCoM models have two inputs , the values of the instances as text or natural language and hand-engineered features . We present two types of DCoM models based on the way we feed the text input to the model . 4.1 DCOM WITH SINGLE SEQUENCE INPUT . In this subsection , we discuss how the values of each instance can be fed to the model as a single sequence input to the DCoM . Considering the scenario , we can not simply pass the inputs separating values of each input with any separator token . This gives the model wrong information about the sequence of the data resulting in faulty training and poor performance on the unseen data . For example , if we consider the last example from Table 7 and create the input , Deletes the property < SEP > Lets you edit the value of the property < SEP > Script execution will be stopped for the model , the model gets to learn that the value Lets you edit the value of the property has a relative position between Deletes the property and Script execution will be stopped , which is quite wrong because the values in an instance do not have any relative position between them . To mitigate the problem we introduce permutations from mathematics . With permutation , we order r items from the set of ni items . Here ni items are all the values of an instance i , and r ∈ [ 1 , n ] . Doing so , the above instance can be broken down into multiple subsets . A sample of the subset is shown in Table 1 . If we feed all the new instances of the subset to the model , it does not learn any relative positional information of Lets you edit the value of the property with respect to other values in that instance unlike earlier . Therefore , for the value Deletes the property , the model only learns the relative positions of the tokens e.g . Deletes , the , property , etc in a value , but not the relative position of the values in an instance . This helps the model getting the actual information present in the data for accurate prediction on the output . This permutation method also helps in augmenting new instances which help in training the data-hungry deep learning models with enriched data . It is not feasible to generate all the possible permutations before the training because of the huge number of subsets . Instead , during training , we sample r between 1 and n for each of the instances and generate one permutation for each instance . More than one instance can be generated , but training the model for multiple epochs will result same for both cases .

Summary: This paper develops a semantic type detection model DCoM that takes column values as text in addition to auxiliary features extracted by the exiting semantic type detection method Sherlock. The paper proposes two versions: (1) Single, which takes a single sequence, and (2) Multi, which takes multiple sequences using permutation. Experimental results on the VizNet corpus show that DCoM models with DistilBERT/ELECTRA outperform Sherlock and other non-Deep Learning models.