  Corporate credit rating reflects the level of corporate credit and plays a
crucial role in modern financial risk control. But real-world credit rating
data usually shows long-tail distributions, which means heavy class imbalanced
problem challenging the corporate credit rating system greatly. To tackle that,
inspried by the recent advances of pre-train techniques in self-supervised
representation learning, we propose a novel framework named Contrastive
Pre-training for Corporate Credit Rating (CP4CCR), which utilizes the
self-surpervision for getting over class imbalance. Specifically, we propose
to, in the first phase, exert constrastive self-superivised pre-training
without label information, which want to learn a better class-agnostic
initialization. During this phase, two self-supervised task are developed
within CP4CCR: (i) Feature Masking (FM) and (ii) Feature Swapping(FS). In the
second phase, we can train any standard corporate redit rating model
initialized by the pre-trained network. Extensive experiments conducted on the
Chinese public-listed corporate rating dataset, prove that CP4CCR can improve
the performance of standard corporate credit rating models, especially for
class with few samples.
