Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a self-supervised learning method for tabular data. Specifically, it adopts a pipeline similar to DINO – it matches the categorical distribution produced from an EMA teacher network and a student network but the classifier is built with a queue of past embeddings instead of a learned linear layer. The paper shows empirical results and ablation studies on multiple datasets and demonstrates the effectiveness of the proposed method on tabular data.
strength_and_weaknesses: Strength
1. The motivation of the proposed method is clear: to build a SSL method similar to the existing semi-supervised methods for tabular data.
2. All the building pieces (with slight differences) exist in SSL for image data and therefore the proposed method is simple and straightforward.

Weakness
1. The novelty is limited to empirical application to tabular data.
2. The paper does not seem to have compared against many similar methods it mentioned (SwAV for instance). As the proposed method is  not tailored for tabular data from a pure methodology perspective, it is unclear if the proposed method makes more sense than just applying DINO, SwAV, etc. to the tabular datasets used in the experiments.
clarity,_quality,_novelty_and_reproducibility: The paper is easy to follow. The technical novelty is limited - all building pieces (with slight differences) already exist in other literature and it is similar to FixMatch.
summary_of_the_review: The idea of the proposed SSL method is straightforward and seems to be effective. However, it shares too many similarities with SSL methods for images without any pieces which are special to tabular data. It is therefore unclear if the performance gain is unique for the specific design in this paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes Q-match, a self-supervised method for tabular data. Similar to Fixmatch and DINO, the proposed method matches the feature distribution induced by a teach network and a student network, with a feature queue similar to moco. Results on several tabular data benchmarks indicate that the proposed method outperform previous self-supervised methods.
strength_and_weaknesses:  Strengths:   
1.	The paper is well-written and easy to follow. The algorithm is described clearly. For each part of the algorithm, the authors provide the motivation and mention the counterpart in vision methods.    
2.	The proposed method is simple and has good performance. The method does not include many tricks, but a rather original implementation of the principled self-supervised learning method. It is encouraging to see it works well on tabular data, across various datasets and settings.    
Weaknesses:    
1.	Despite the good performance, I’m also curious to see the contribution of each part of the model. How does loss function affect the performance? What if you use the VCIReg loss? How does the queue affect the performance? If you remove the queue and use large batchsize following SimCLR, or simply use Simsiam, what will happen? It would be better if the authors include them as an ablation study.

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written. I believe the method can be reproduced based on the information provided in the paper.
summary_of_the_review: Self-supervised learning has been a very successful technique across vision and NLP. This paper extends the common self-supervised techniques to tabular data and shows its effectiveness. Although the authors could have provided more justifications of their method, I tend to accept this paper.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper adapts the Fixmatch method [Sohn et al., 2020] from semi-supervised to self-supervised learning, and significantly improves the results compared with the benchmarks in experiments.
strength_and_weaknesses: Strength:

1. The paper is relatively well-written and easy to follow.
2.  The paper conducts extensive experiments to validate the proposed method empirically.

Weaknesses.

1. The work has limited novelty on top of Fixmatch [Sohn et al., 2020].
clarity,_quality,_novelty_and_reproducibility: Providing the pseudo-code (Algorithm 1) is insufficient to reproduce many experimental results in the article. 
summary_of_the_review: The contribution of this article is insufficient to be accepted to the conference. However,  I believe the paper is an excellent workshop paper.


###########

I thank the authors for their response. However, I still think the contribution of the article is relatively small so keep my score unchanged.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes to use a queue of embeddings of samples from the unlabeled dataset to improve the performance of self-supervised learning. It presents that the proposed method have achieved success on the tabular datasets.
strength_and_weaknesses: Strength:

This paper has conducted some interesting experiments.

Weaknesses:

1. The technical novelty of this paper is very limited. The proposed method is just a combination of some existing methods. 

2. The motivation of the proposed method is unclear. This paper has not explained why the proposed method can achieve proper performance. 

3. The writing of this paper requires significantly improvement. There are many typos and grammar mistakes. For example, in the **method** subsection of Chapter 2, the word "student" in the sentence "We pass $x_{i,s}$ Through the student model to produce the student embedding $z_{i,s}$" should be "teacher", In the **Semi-supervised Learning** subsection of Charpter 4, the word "teach" in the second last sentence may be "teacher". 
clarity,_quality,_novelty_and_reproducibility: This paper lacks of originality.
summary_of_the_review: The technical novelty of this paper is very limited.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes using FixMatch-like distribution matching technique but for fully-unsupervised (self-supervised) learning of Tabular data. The basic model is similar to previously known non-contrastive architecture with Student and Teacher (exponential moving average of student weights). However the loss is modified to be like FixMatch. The loss matches softmax distribution of the projected student and teacher embeddings on Q directions in the embedding space. These Q directions are cached teacher embeddings from previous batches stored in a FIFO queue. The queue idea is slightly novel and simple and reviewers identified is not specific to Tabular data. The experimental results are promising but a bit underwhelming since most of the performance numbers are close to next-best baseline's numbers when incorporating the standard errors. Authors also did a series of interesting ablation studies and did additional experiments as requested by reviewers. It would be good if authors can try to extensively replicate these comparisons in non-tabular domains and provide some explanation for why this method works. Unfortunately code was not provided for reviewers to check reproducibility.
justification_for_why_not_higher_score: Experimental results doesn't show consistent improvements over baseline. Code is not provided.
justification_for_why_not_lower_score: The idea is simple and promising, and not just in tabular domain. Paper is well written.
Decision: Reject