Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The goal of the work is to associate semantic data types to table attributes. This task is of importance to data cleaning, schema matching,  etc. The authors claim that existing methods use regular expressions or dictionary lookups, and proposed a learning approach to the problem.
They feed raw values (strings) to neural classifiers, and train their models on 686,765 data columns extracted from the VizNet corpus with 78 different semantic data types. The textual values processed by LSTM/Transformer/BERT layers, where in addition the authors use 19 tatistical features from Hulsebos et al. (2019).  Overall, they achieve F1 score of 0.925.

Analysis showed that high performance is obtained for classes that contain a finite set of valid values, e.g. grades or industry. 
On the other hand, numerical values or in general values that appear in multiple classes, are more challenging to classify. 

main_review: Pros:
- The task is interesting/important
- Good performance is reported

Cons:
- While the the context information of the column values is of interest, modeling this information is deferred to future work.
- Classifying the semantic type based only on the string value is too simplistic. One of the main challenges of this task in practice is to resolve ambiguities, where the authors do not address this aspect altogether. 
- Assuming that the classifier/s may just memorize values per category, the authors should check/report the overlap between train and test.
- Otherwise, it would be interesting to train and test on different datasets, to assess learning generalization.
- I'm not sure that the dataset contains enough diverse examples for learning 70+ categories.

Some typos:
- many attempts by Microsoft (2019): cannot cite like this.. (repeats several times)
- We refrained ourselves extracting --> refrained from
- The output of the same is aggregated --> ??
- Figure 2 uses a too small font.

summary_of_the_review: The approach is overly simplistic, as it does not account for ambiguities. Methodologically, I'm not convinced that the model generalizes, as there may be significant overlap between the train and test examples.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper develops a semantic type detection model DCoM that takes column values as text in addition to auxiliary features extracted by the exiting semantic type detection method Sherlock. The paper proposes two versions: (1) Single, which takes a single sequence, and (2) Multi, which takes multiple sequences using permutation. Experimental results on the VizNet corpus show that DCoM models with DistilBERT/ELECTRA outperform Sherlock and other non-Deep Learning models. 
main_review: ### Strength

- (S1) The paper combines learned representation and manually crafted features for further improvements on the semantic detection task.


### Weaknesses

- (W1) The novelty and technical significance are limited.
- (W2) Missing important references and comparison against the state-of-the-art methods.
- (W3) DCoM-Multi does not show improvements over DCoM-Single.

### Major comments

(W1)

Using raw table values for semantic type detection is a commonly used approach and several models have been already developed [1-4]. The models in [1-3] also incorporate table structures (i.e., row/column) into the model, which is not considered in DCoM.

- [1] Xiang Deng, Huan Sun, Alyssa Lees, You Wu, Cong Yu, TURL: Table Understanding through Representation Learning, VLDB 2021 (https://arxiv.org/abs/2006.14806)
- [2] Hiroshi Iida, Dung Thai, Varun Manjunatha, Mohit Iyyer, TABBIE: Pretrained Representations of Tabular Data, NAACL 2021 (https://arxiv.org/abs/2105.02584)
- [3] Daheng Wang, Prashant Shiralkar, Colin Lockard, Binxuan Huang, Xin Luna Dong, Meng Jiang, TCN: Table Convolutional Network for Web Table Interpretation, WebConf 2021. (https://arxiv.org/abs/2102.09460)
- [4] Yoshihiko Suhara, Jinfeng Li, Yuliang Li, Dan Zhang, Çağatay Demiralp, Chen Chen, Wang-Chiew Tan, Annotating Columns with Pre-trained Language Models, arXiv April 2021 (https://arxiv.org/abs/2104.01785)

Furthermore, all the models and Sato [Zhang et al. (2019)] are multi-column models that consider all columns in the same table to detect the semantic types in a joint manner. As discussed in [Zhang et al. (2019)], the table context is essential to semantic type detection. 


(W2)

As commented above, the paper should cite and compare with [1-4] in addition to Sato, which is cited but not compared in the paper. 


(W3) 

According to Table 3, DCoM-Multi does not show improvements over DCoM-Single. 


### Minor comments

- Why did the author(s) use the 19 features out of 27 Sherlock features?
- Do the feature importance scores for different models (e.g., DistilBERT) follow the same or a different trend against the feature importance scores reported in Table 6?
- Typo: we -> We (p.5 2nd paragraph)

summary_of_the_review: The paper has critical issues with respect to novelty and technical significance. The paper should cite the missing papers and compare with those models (and Sato.) Thus, I wouldn’t recommend publishing the paper in its current form.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the problem of semantic type detection that takes a column of texts then predicts a header name for the column. Existing methods mostly adopt regular expressions or dictionary-lookup or feature engineering. The paper introduces a new method, called DCoM that is different from existing methods. DCoM adopts deep neural networks to process raw text tokens directly instead of relying on feature engineering. The paper conducts extensive experiments and shows that DCoM outperforms a number of contemporary methods on VizNet dataset, which has 78 types. 
main_review: Main Review

The paper studies a very practical problem called semantic type detection. The problem is fundamental to tabular data that consists of columns and headers. Given the widespread adoption of tabular data in database systems, the paper has the potential to benefit extensive applications. The paper presents a new method DCoM for the problem. DCoM avoids the feature engineering step, which is common in existing methods. Also, DCoM achieves satisfactory performance that hits a support-weighted F1 as high as 0.925. However, the paper can be more convincing if several aspects can be taken into consideration. Details are as the following:  

1.	Pretrained models should be included as baselines. The proposed DCoM adopts a pre-trained model in the design. For example, DCoM-Single-DistillBERT leverages the pre-trained model DistillBERT that comes from HuggingFace Transformers library. The library contains other pre-trained models, including BERT, RoBERTa, and ALBERT. The interface of using these models is the same as DistillBERT. It will be interesting to see whether different pre-trained model leads to different performance of DCoM, and more importantly, whether these models standalone can solve the semantic type detection problem adequately. The paper will be more convincing if it can include the support-weighted F1 scores of one or more of these pre-trained models as baselines. For experiments, the semantic type detection problem can be naturally formulated as sequence prediction, which is the strength of pre-trained models.

2.	Ablation study is favorable. DCoM presents impressive performance when compared with existing methods. However, it remains unclear why DCoM can achieve superior performance. Is it because DCoM uses higher-dimensional word vectors or DCoM uses larger neural network? The paper can be more insightful if it conducts some ablation study to find out why DCoM is superior.


3.	Real applications can be discussed further. Semantic type detection seems a fundamental problem in tabular data, and an immediate application is header prediction. However, header prediction may not be common because the headers and columns in a table are usually serialized at the same time. Semantic type detection can be more useful if it can affect many downstream tasks (e.g. column matching and schema matching). Some discussions on applications will be helpful to make DCoM adopted extensively in the industry.

summary_of_the_review: Summary of the review
The paper studies a novel problem called semantic type detection, which has potential applications over tabular data. However, the paper should include pre-trained models as baselines to show the proposed DCoM outperforms the state-of-the-art methods by a significant margin.
review_title: null

Meta-review: The paper studies semantic type detection.
 The problem is of practical significance to  i  tabular data.
 However, in its current form, there are concerns about  the scope of novelty and technical significance.
Decision: Reject