Edit model card

KF-DeBERTa-base-cross-STS

pre-trained model: KF-DeBERTa-base-cross-NLI (https://huggingface.co/deliciouscat/kf-deberta-base-cross-nli)

trained data:

  • klue/sts: 1epoch
  • dkoterwa/kor-sts: 2epoch

label scaling: 0~5 -> -1->1

bi-encoder STS 학습을 위한 dataset augmentation을 상정하고 훈련하였습니다. (https://arxiv.org/abs/2010.08240)

cosine similarity로 학습할 수 있도록 scaling 된 output이 추론됩니다.

Downloads last month
18
Safetensors
Model size
186M params
Tensor type
F32
·
Inference Examples
Inference API (serverless) does not yet support transformers models for this pipeline type.

Datasets used to train deliciouscat/kf-deberta-base-cross-sts