Title: Correlation-based Self-Supervision for Few-shot Tabular Learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Tabular Data, Self-supervision, Variable Correlation, Few-shot Learning

Abstract: Despite its paramount importance in many real-world applications, e.g., few-shot learning, self-supervision for tabular data remains challenging. The inherently heterogeneous nature of tabular data substantially impedes the generation of pseudo-labels with high fidelity. The commonly utilized random-based selection methodology largely ignores the complex relationships among features and might induce substantial noise into the self-supervision process, especially when the selected features have little relations with others. To address this issue, this paper proposes a simple yet effective solution: utilizing the correlation among original features to qualitatively evaluate the possible quality of pseudo-labels. Accordingly, a correlation-based randomness, rather than pre-defined uniform randomness, is proposed to select features for pseudo-label generation according to their overall correlations towards others. We employ our design in VIME, SCARF, STUNT and SAINT. The experimental results in various few-shot classification tasks reveal significant performance improvements across 50 OpenML datasets, compared to the original design. Code and datasets are available at the supplemental file.